USS: Unified Spatial-Semantic Prompts for Embodied Visual Tracking with Latent Dynamics Learning

arXiv 2026 52.1 method, application

TLDR

USS introduces unified spatial-semantic prompts and a latent world model for embodied visual tracking, improving target grounding and temporal robustness.

Reasoning

The paper presents a novel paradigm shift from text-only to spatial-semantic prompts, supported by real-robot experiments showing higher success rates. However, the latent world model is a secondary component, and the paper focuses on tracking rather than general world modeling.

Read-first score

Read-first score 52.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 17.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

95.3

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.95329477

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment,result

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,github

Topical relevance 29%

24.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 359.

Keyword Scores

world model

world dynamics prediction

generative world model

video world model

world simulator

interactive world model

model-based reinforcement learning world model

Deep Analysis

Innovations

Paradigm shift from text-only target indication to unified spatial-semantic prompting for embodied visual tracking
USS framework supporting multiple prompt types (text, point, bounding box, mask) within a single architecture
Incorporation of a latent world model for predicting future representations via self-supervised alignment to improve temporal robustness

Methodology

USS is an end-to-end embodied tracking framework that encodes heterogeneous prompts (text, point, bounding box, mask) with modality-specific encoders, fuses prompt tokens with visual features through hybrid attention, and decodes compact prompt-conditioned representations into egocentric waypoints. To enhance temporal robustness, it integrates a latent world model that predicts future representations via self-supervised alignment.

Key Results

Real-robot experiments show that explicit spatial target cues yield higher success rates than text-only prompts, especially with similar distractors and longer-horizon tracking. In simulation, USS achieves state-of-the-art performance among non-MLLM-based methods and competitive results against recent MLLM-based approaches with faster inference speed.

Limitations

Text-only prompts are still supported but may lead to ambiguous target grounding in cluttered scenes with multiple semantically similar objects
The method requires an initial spatial cue (point, bounding box, or mask) for optimal performance, which may not always be available
The latent world model's self-supervised alignment may not generalize to highly unpredictable or novel dynamics
Simulation benchmark comparisons are limited to non-MLLM methods and a subset of MLLM approaches; not all state-of-the-art methods are covered

Resources

Paper