Visual AI still stumbles when it must learn from examples alone

On July 2, 2026, six researchers from LMU, the Munich Center for Machine Learning, and Apple submitted the VICIS preprint to arXiv. It tests whether models can infer a rule shared by images without a written instruction. The example sounds simple: show wooden objects, then ask for a dog that preserves the animal while applying “wood.” The challenge is separating the evidence that defines the instruction from noise.

Apple’s page for the work expands VICIS as Visual Concept Inference from Sets: infer a concept shared by a small set and apply it to a query image. Its lasting contribution is a test for any claim of “learning from examples”: identify the hidden rule, prevent copying, and introduce ambiguity.

What VICIS actually asks a model to do

VICIS gives a model a context set whose images share an unlabeled concept, plus a query image. The output is not the concept’s name but new images. They must respect the dimension defined by the set and preserve the relevant instantiation from the query.

That condition prevents an easy test. If the context contains several dog breeds and the query shows a bulldog, generating any dog would show that the system summarized the set, not what it transferred from the query. The formal design therefore includes a target image: query and target share the same specific concept instantiation while other features may vary.

Three roles should be written down separately before evaluating a tool:

1. Rule: the dimension shared by the examples, such as material, shape, or a level in a category.
2. Instantiation: the query’s value within that dimension, such as bulldog within dog.
3. Freedom: features the output may vary without becoming wrong, such as background, pose, or lighting.

Without those definitions, an attractive image can look correct even when the model followed an accidental cue. The useful question is not “does it look similar?” but “did it preserve exactly what it had to preserve and change only what it was allowed to change?”

Images alone at test time does not mean unsupervised learning

The proposed system divides the task among three modules. A set learner processes the examples jointly and calculates directions representing the concept space. It then projects the query representation onto that space to isolate the relevant instantiation. Finally, a diffusion generator produces an image conditioned on that projection. The paper describes this separation as concept discovery, instantiation extraction, and generation.

No text or label is supplied at inference time. Training, however, still needs a structure that can produce coherent sets, queries, and targets. The authors begin with synthetic data in which they control shape, size, position, and color. For real images, they use hierarchies derived from WordNet and ImageNet.

That detail prevents an inflated reading of “learns just by looking.” WordNet organizes word senses into synonym sets and semantic relations; ImageNet was built on that structure to associate images with synsets. VICIS uses this scaffold to construct episodes such as animal → mammal → dog → bulldog. The model does not see those names when answering a query, but the laboratory used them to decide which examples belong together and what counts as the correct answer.

The distinction transfers to other claims: text-free input describes the evaluation interface; label-free training describes the source of the learning signal. They are not the same assertion.

One accuracy score cannot expose both shortcuts

The authors evaluate the hierarchical task with two measures. Accuracy uses a classifier to determine whether a generation falls in the correct branch. Diversity is intended to reveal whether the model merely copies the query instead of producing different but valid examples.

In the main table of version 2, the proposed model records 46.34% mean accuracy per concept and 0.81 diversity. ILLUME+ 3B records 37.15% and 0.57; BAGEL 7B-MoT, 26.05% and 0.46; and a Visual Prompting model trained for the task, 26.02% and 0.70. These are author-reported results on the animal subtree of their benchmark, not a general ranking of visual models.

The same table supplies an important counterweight. When accuracy is averaged over possible query instantiations, ILLUME+ reaches 55.00% and the proposed system 54.46%. Saying simply that one “beats” the other would erase the fact that the measures answer different questions.

The most revealing controls are deliberately bad. Copying the query image achieves near-perfect accuracy but only 0.47 diversity. Generating random images reaches 0.77 diversity but near-zero accuracy. One score rewards one of these shortcuts. The pattern applies beyond VICIS: a generative evaluation needs to measure fidelity to the constraint and permitted variation at the same time.

More examples help little; distractors teach a great deal

A strong test does not keep the context clean. It varies the number of examples and inserts cases that do not fit to find out whether the model is really using the set.

According to the paper’s robustness experiments, per-concept accuracy moves from 44.74% with two images to 45.26% with three, 46.34% with five, and 46.56% with seven. Most of the improvement runs out early. Seven examples do not guarantee much deeper understanding than five within this test.

The noise experiment is more demanding. Replacing one of five images with a random image reduces accuracy from 46.34% to 37.62%; with two distractors, it falls to 26.01%. Diversity does not fall: it remains at 0.812 and then rises to 0.906. That does not mean the system improves. It means the outputs become more varied while the correct rule is lost.

To test a tool, build matched sets: one clean; one with an ambiguous example; one with an obvious distractor; and one in which two different rules are plausible. A reliable system should degrade visibly, lower its confidence, or request clarification. If it answers every set with the same confidence, the interface is hiding the uncertainty that the demonstration should reveal.

Generalizing to sketches is not understanding every concept

The authors also change the visual domain. They test queries and contexts made from sketches, as well as ImageNet-21k classes not seen in training. Performance declines relative to photographs from the original domain, although the proposed model retains an advantage over Visual Prompting in the reported configurations.

This is useful because ImageNet-Sketch was created to test transfer beyond the photographic domain: it preserves ImageNet categories while replacing many texture and color cues with drawings. It still remains inside the taxonomy. A good sketch result shows tolerance to that representational shift, not an ability to infer every style, cultural intention, or property that is difficult to fit into WordNet.

The study also includes a small evaluation of closed models. Its appendix reports 100 images per model, manual categorization by experts, and a prompt that supplies the hierarchy. That protocol is evidence of a failure mode, but it does not have the same scope as the main hierarchical table. Nor should general systems and a model trained specifically for VICIS be described as if they received the same preparation.

The architecture is a substantial research prototype: the paper reports 765 million trainable parameters and 24 hours of training on 16 H100 GPUs. The result shows that the capability can be isolated and improved. It does not show that it is already reliable in every image generator.

A five-test record for the next demonstration

When a product claims to learn from visual references, five questions turn an impression into evidence:

1. What rule must it infer? Write it down before viewing the output, along with other plausible rules shared by the examples.
2. What must it preserve from the query? Change the background, pose, or style to see whether it keeps the relevant instantiation rather than the whole image.
3. Which shortcuts score well? Try copying the query and generating random diversity. If either earns a good score, a metric is missing.
4. What happens under disagreement? Add distractors and ambiguous sets. Record whether the system detects conflict, asks a question, or invents certainty.
5. Where does the evidence stop? Separate training and test data, change the visual domain, and disclose the taxonomy, classifier, and human review used to judge outputs.

VICIS does not prove that a machine sees as a person does. It offers something more useful than that phrase: a way to separate rule, instantiation, and variation, then penalize both copying and uncontrolled creativity. A reader who keeps these five questions can evaluate the next visual demonstration even after the model name, vendor, and benchmark have changed.