Grounding requires evidence-sensitive task tests beyond fluent captions

Construct matched images and captions where both images contain similar objects but the correct relation differs. The captions can use the same words in a changed order, preventing a bag-of-objects strategy from solving every comparison. Winoground is an original example of this evaluation idea. Check annotation validity and whether the detail survives preprocessing; a cropped-away object makes the test ambiguous rather than revealing a reasoning failure. Preserve the scoring direction and aggregation unit so a matched group is not counted as several independent successes without its joint requirement.

For two images and their intended captions, test both directions. Holding each image fixed, its correct caption should score above the alternative. Holding each caption fixed, its correct image should score above the alternative. A model can satisfy all row comparisons while failing a column comparison because one image receives generally larger scores. Report the component results and the all-required group criterion. These quantities are not interchangeable with generative fluency, answer length, or a pooled score over unrelated easy examples.

A controlled media change can test whether an answer depends on the evidence required by the task. Swap the relevant relation, remove a clearly annotated region, or replace an audio event while keeping the instruction fixed and labels valid. Include irrelevant-region or unrelated-content changes as controls. Cropping and occlusion can introduce artifacts or remove context, so an output change alone does not prove the intended semantic mechanism. Verify that the intervention actually changes the target fact and that the resulting input remains interpretable.

Attribution maps, attention weights, and gradients can suggest where to investigate, but a highlighted area alone does not establish that the model used that area in the required causal or semantic way. Combine diagnostics with task-specific perturbations, alternative explanations, and repeated examples. Distinguish a causal intervention on the model input from a causal claim about the real-world scene. A score difference under an edited image measures behavior under that edit; it does not by itself certify human-like reasoning or identify every feature responsible for an answer.

Before attributing a failure to grounding, verify image ownership, feature visibility, preprocessing, and prompt serialization. Then evaluate relation types, small details, OCR, temporal order, and missing-evidence cases by cohort. Include text-only and shuffled-media baselines when they are meaningful, with the same scoring and output rules, to expose language priors or leakage. Report uncertainty at the group or asset level rather than pretending related captions and crops are independent. A strong result on one curated contrast set should be described with its dataset and task scope.

The vector lab demonstrates a fully specified numerical intervention, not learned grounding. Reset, set both angle 0° and magnitude 1, predict x-component 1, then inspect or Play. Reset and set both angle 180° and magnitude 1 for x-component−1. The reference direction is fixed and the output change follows cosine by construction. There are no images, captions, latent features, attribution maps, or reasoning task. The activity illustrates holding one parameter fixed while changing another; actual grounding requires the validated media contrasts and behavioral controls described here.