A museum model may recognize a porcelain vase while missing where, when, and by whom it was made. The Appear2Meaning benchmark tests that deeper attribution task by asking vision-language models to recover structured museum metadata from an image alone. Across nine models, full-record exact-match accuracy ranged from 0.5% to 2.9%.

Five fields from one image

Most heritage captioning datasets, including earlier heritage datasets such as Artpedia, ArtCap, and EmoArt, reward descriptions of an object’s appearance or emotional tone. Appear2Meaning evaluates the controlled fields used in collection records: title, cultural tradition, historical period, geographic origin, and creator, which may be an artist, maker, or workshop.

The benchmark treats those fields as latent attributes inferred from visual evidence. Models may generate captions as an intermediate step, but evaluation scores only the structured outputs. This design measures whether a system can connect visible features to historical and cultural knowledge when no catalog metadata is supplied.

How 750 objects were selected

Researchers drew the dataset from the Getty and Metropolitan Museum of Art collections. Rule-based retrieval produced candidate objects, followed by two stages of human verification. The team sampled 50 artifacts for each included culture-and-object-type combination, producing 750 verified records.