Object Detection API
Object Detection API Analysis Results
Descriptive Statistics
| Metric | Clarifai | Google Vision |
| Raw Accuracy | 81.04% | 94.00% |
| Mean Confidence (TP) | 0.9059 | 0.4155 |
| Median Confidence (TP) | 0.9304 | 0.5959 |
| Mean Confidence (FP) | 0.7844 | 0.0291 |
| Median Confidence (FP) | 0.9056 | 0.0000 |
| False Confidence (>0.8, FP) | 85.71% | 0.0% |
Figure 1. Mean and Median Confidence Scores by Model
Correlation Between Confidence and Accuracy
| Correlation | Clarifai(r) | Google(r) |
| Confidence vs. Accuracy (TP) | -0.003 | 0.887 |
| Confidence vs. Accuracy (FP) | -0.381 | -0.145 |
Clarifai’s confidence scores are not strongly predictive of label correctness, suggesting potential overconfidence or semantic drift. Google, by contrast, demonstrates a robust positive correlation between confidence and accuracy — though it rarely returns confident predictions above 0.9.
Accuracy by Confidence Threshold
| Confidence Bucket | Clarifai (Pos) | Clarifai (Neg) | Google (Pos) | Google (Neg) |
| > 0.9 | 80.57% | 78.21% | N/A | N/A |
| 0.5–0.9 | 76.25% | 74.64% | 98.12% | 33.34% |
| < 0.5 | 85.0% | 97.5% | 11.11% | 67.55% |
Google Vision rarely returns high-confidence predictions above 0.9, and when it does, the number is too small to support a statistically meaningful accuracy estimate in that confidence band. For this reason, Google’s >0.9 entries are marked as “N/A” in the grouped accuracy analysis. In contrast, most of Google’s predictions cluster between 0.5 and 0.9, where it performs with very high precision. Clarifai, on the other hand, generates labels across the full confidence spectrum including frequent predictions above 0.9, but with a substantially higher false confidence rate in that range, highlighting its tendency to over-commit semantically.
Statistical Significance
Mann-Whitney U test
Comparision: Clarifai confidence scores correct labels (TP) vs incorrect labels (FP)
p-value = 0.0249
Stastically significant – Clarifai assigns meaningfully different confidence scores depending on whether it was correct or not. Despite its overconfidence overall, the model’s confidence is still sensitive to correctness.
Paired t- test
Comparision: Confidence scores on correct predictions (TP) by Clarifai vs Google
p-value = 7.79 × 10⁻¹⁴
Stastically highly significant – Clarifai is consistently and substantially more confident than Google when both models are correct – confirming the models differ not just in what they predict, but how confidently they do so.
Interpretation
Google Vision API excels in precision and calibration, but is notably conservative in its label generation. While its raw accuracy is high (94.00%), the model exhibits low average confidence even when correct (mean TP score = 0.4155), and it avoids making confident incorrect predictions altogether (false confidence = 0.0%). This results in cautious outputs that under-detect naturalistic or symbolically ambiguous features — a reflection of its training bias toward structured, commercial image domains like product photography, signage, and well-lit indoor scenes.
Clarifai, by contrast, provides broader semantic coverage and high average confidence when correct (TP score = 0.9059), but this comes with a tradeoff: 85.71% of its false positives are scored above 0.8, indicating a tendency toward overconfidence. This reflects Clarifai’s architecture, which is optimised for flexibility and generalised content tagging across diverse media, including art, lifestyle, and conceptual domains. The model returns a much wider array of labels, often including symbolically resonant features such as: trees, sky, grass, sunset, water and symbolic concepts like freedom, outdoors, connection.
These qualities are not just incidental but deeply relevant to the aims of this research, which centres on evolutionary and ecological conditions for human thriving, themes that require interpretive, not just literal, recognition.
Conclusion
For tasks that prioritise calibrated confidence, precision, and the detection of concrete, well-defined objects (e.g., in urban, commercial, or technical settings), Google Vision is the stronger model. However, for research involving naturalistic imagery, symbolic content, and conceptual meaning, as is the case in this Capstone project; Clarifai provides greater semantic richness, even if that richness comes at the cost of over-labeling and reduced precision.
The distinct training priorities of each model explain these differences: Google is tuned for reliability in structured domains, while Clarifai is trained for breadth and contextual nuance. For conceptual image analysis rooted in developmental psychology, environmental symbolism, and visual culture, Clarifai is the more aligned tool, provided its outputs are filtered or interpreted through confidence thresholds or human calibration.
Threshold Analysis for Clarifai Confidence Scores
The threshold analysis graph plots four key performance metrics—precision, recall, F1 score, and Youden’s J statistic—across a range of Clarifai confidence thresholds from 0.01 to 1.0. This visualisation reveals how the model’s behaviour shifts as the cutoff for label inclusion increases, illustrating the tradeoff between including more data and maintaining label reliability. At lower thresholds, recall is high but false positives increase, lowering precision. As the threshold rises, precision improves while recall declines.
The point at which these metrics optimally balance occurs around threshold = 0.87, where Clarifai achieves over 91% precision, 95.5% recall, and a near-maximum F1 score of 0.933. This threshold also yields the highest Youden’s J value, indicating the best separation between correct and incorrect predictions, and the lowest cost under an error-weighting model where false positives are considered twice as damaging as false negatives. Given the symbolic and affective focus of this project, where false positives risk semantic distortion, this threshold was selected to ensure trustworthy label output without significant loss of meaningful content.