Inference-time Policy Steering via Vision and Touch

arXiv 2026 47.6 method

TLDR

ViTaL uses vision and touch for inference-time policy steering via bi-level optimization and a visuo-tactile latent world model, improving contact-rich manipulation success by 51%.

Reasoning

The paper presents a novel multimodal steering framework with strong real-world results, but its focus on contact-rich tasks may limit generality. The bi-level optimization and tactile-guided diffusion are well-motivated, though the abstract lacks detailed ablation or comparison to other world model approaches.

Read-first score

Read-first score 47.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 28.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Citation impact 18%

91.8

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.91787084

Topical relevance 29%

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=none

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=github

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridge

Rank sensitivity

Stability: volatile; rank range: 390.

Keyword Scores

world model

world dynamics prediction

world simulator

generative world model

model-based reinforcement learning world model

interactive world model

video world model

Deep Analysis

Innovations

ViTaL: a visuo-tactile inference-time steering framework that formulates multimodal guidance as a bi-level optimization problem
High-level visual sampling-and-verification for long-horizon mode selection
Low-level tactile-guided diffusion editing for short-horizon local contact refinement
Learning a visuo-tactile latent world model with semantically aligned visual and tactile verifiers, including a novel text-conditioned tactile reward

Methodology

ViTaL employs a bi-level optimization approach: at the high level, visual sampling-and-verification selects long-horizon behaviors; at the low level, tactile-guided diffusion editing refines action sequences over a shorter horizon to satisfy local contact requirements. It learns a visuo-tactile latent world model and uses semantically aligned visual and tactile verifiers, including a text-conditioned tactile reward that scores predicted tactile futures in latent space. The framework is evaluated on three real-world contact-rich manipulation tasks.

Key Results

ViTaL improves overall success by 51% over the base policy, outperforms unimodal steering by at least 33%, and exceeds naive multimodal fusion by at least 20% across three contact-rich manipulation tasks.

Resources

Paper