Select Page

Object Detection API

Object Detection Analysis

Manual Calibration Design

Manual calibration was undertaken to ensure that automated object detection models were not only technically accurate but also semantically aligned with the theoretical and conceptual aims of this project. This process is essential in computer vision research, particularly when working with images that contain symbolic, ecological, or affective meaning which may be overlooked or mislabelled by pre-trained AI models (Everingham et al., 2010; Russakovsky et al., 2015).

A total of 50 images were selected across the project’s core themes. Each image was manually annotated to establish a ground truth set of expected concepts. These included both literal and symbolic indicators relevant to the project’s theoretical framework (e.g., “tree,” “gesture,” “freedom,” “sunset”). Images were then submitted to both APIs. All returned labels included a model-generated confidence score ∈ [0, 1.0].

Label outputs were classified as:

  • True Positive (TP): expected label present in model output
  • False Negative (FN): expected label not returned
  • False Positive (FP): label returned but not present in ground truth

Quantitative Metrics

For each model, I computed:

  • Raw object-level accuracy: TP / (TP + FN)
  • Mean and median confidence scores, disaggregated by TP vs. FP
  • Pearson correlation (r)between confidence score and label correctness
  • Stratified accuracy by confidence bucket: Low (<0.5), Medium (0.5–0.9), High (>0.9)
  • False confidence rate: FP rate for predictions with confidence > 0.8

Additionally, statistical significance was assessed using:

  • Mann-Whitney Utest (Clarifai positive vs. negative confidence)
  • Paired t-test(Clarifai vs. Google for correct labels)
  • Wilcoxon signed-ranktest (Clarifai vs. Google raw accuracy)

All scores were parsed as floats and aggregated first by image, then across the full sample.

Interpretive Value

This manual calibration process enabled the identification of systematic biases and model tendencies such as overconfidence in Clarifai and conservatism in Google Vision ensuring that subsequent image analysis was both methodologically rigorous and conceptually valid.

Notably, Clarifai was selected as the primary tool for further analysis because it consistently detected a wider range of objects and more subtle, contextually relevant features within naturalistic and symbolic images. However, this broad semantic coverage came with a risk of false positives, often due to high confidence in incorrect labels.

To address this, the calibration process established a defensible confidence threshold (Clarifai: 0.87) for reliable label inclusion. By applying this threshold, I was able to harness Clarifai’s greater sensitivity and breadth capturing more of the visual and conceptual diversity relevant to the project’s focus on human developmental and evolutionary needs while still ensuring that only the most trustworthy predictions were included in the final analysis.

In summary, manual calibration provided an essential quality check, grounding automated analysis in human interpretation, aligning object detection outputs with the project’s conceptual framework, and enabling the effective use of Clarifai’s strengths within a controlled and robust methodological structure.

←→