Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval
TLDR
Proposes PrediCIR, using a world model to predict missing target visual content for zero-shot composed image retrieval, achieving state-of-the-art results.
Reasoning
Strengths: novel application of a world model for prediction in ZS-CIR, strong generalization across six tasks, and significant performance improvements. Weaknesses: limited architectural details and reliance on existing image-caption pairs may constrain applicability.
Read-first score
Read-first score 44.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 8.
Field roles
Rank sensitivity
Stability: volatile; rank range: 376.
Keyword Scores
Deep Analysis
Innovations
- Prediction-based mapping network PrediCIR that adaptively predicts missing target visual content in reference images before mapping
- World view generation module that constructs a source view by omitting visual content from a target view, using an action derived from image-caption pairs
- Target content prediction module that trains a world model as a predictor to adaptively predict missing visual information guided by user intention in latent space
Methodology
PrediCIR comprises two modules: a world view generation module that creates a source view by omitting certain visual content from a target view, paired with an action representing manipulation intent from image-caption pairs; and a target content prediction module that trains a world model to predict the missing visual information in latent space guided by user intention. The model maps an image with predicted relevant information to a pseudo-word token without extra supervision.
Key Results
The model achieves consistent performance boosts ranging from 1.73% to 4.45% over the best methods on six zero-shot composed image retrieval tasks, establishing new state-of-the-art results.