Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving

arXiv 2026 66 method, application

TLDR

CoPhy combines VLM knowledge distillation and an auto-regressive BEV world model with dual-reward GRPO for safer autonomous driving.

Reasoning

Strengths: novel integration of cognitive and physical components, interpretable world model, state-of-the-art results on NAVSIM benchmarks. Weaknesses: reliance on VLM distillation may have limitations, world model is BEV-based not full video, evaluation limited to simulation benchmarks.

Read-first score

Read-first score 66, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 59.

Recency 6%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 29%

84.3

Uses existing LLM keyword relevance scores normalized to 0-100. world model,world simulator,generative world model,interactive world model,video world model,world dynamics prediction,model-based reinforcement learning world model

Citation impact 18%

82.4

Uses OpenAlex-shaped citation metadata as a bibliometric attention signal, separate from paper quality. citation_normalized_percentile=0.82424512

Methodology quality 18%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,experiment,metric,result

Reproducibility 18%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Citation velocity 12%

Citation velocity estimates citations per publication-year to reduce old-paper bias. velocity=0.00

Field roles

FoundationFrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 410.

Keyword Scores

world model

model-based reinforcement learning world model

world simulator

interactive world model

world dynamics prediction

generative world model

video world model

Deep Analysis

Innovations

Distilling VLM knowledge into the BEV encoder and discarding the VLM entirely, retaining cognitive ability at zero inference cost while releasing the cognitive channel as a pluggable interface for optional human language commands.
Building an auto-regressive BEV world model that explicitly predicts future semantic maps conditioned on candidate actions, serving as an interpretable physical sandbox from which safety metrics are directly derived.
Optimizing the driving policy via GRPO with a novel dual-reward mechanism: a physical reward derived from BEV rollouts enforcing hard safety constraints, and a cognitive reward from a language-aligned scorer ensuring intent compliance.

Methodology

CoPhy is a cognitive-physical reinforcement learning framework for autonomous driving. It first distills VLM knowledge into the BEV encoder, then discards the VLM to achieve zero-cost cognitive ability. It builds an auto-regressive BEV world model to predict future semantic maps conditioned on candidate actions, providing an interpretable physical sandbox. The driving policy is optimized via GRPO with a dual-reward mechanism combining physical safety rewards from world model rollouts and cognitive intent compliance rewards from a language-aligned scorer.

Key Results

CoPhy achieves state-of-the-art results on NAVSIM v1 and v2 benchmarks, and enables safer driving via cognitively informed scene compliance and flexible intent control through user-defined language instructions.

Resources

Paper