VL-SAFE: Vision-Language Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Driving
TLDR
Proposes VLM-SAFE, an offline safe RL framework integrating vision-language models with world models for semantic safety-aware autonomous driving.
Reasoning
Strengths include novel integration of VLM for semantic safety signals with world model for imagination, addressing sample efficiency and safety. Weaknesses: abstract lacks details on experimental validation and real-world performance; reliance on offline data may limit generalization.
Read-first score
Read-first score 62.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 44.
Field roles
Rank sensitivity
Stability: volatile; rank range: 136.
Keyword Scores
Deep Analysis
Innovations
- Integration of a vision-language model (VLM) to provide semantic safety signals grounded in scene understanding for autonomous driving RL
- Use of a learned world model to imagine future trajectories from observed context, enabling offline reasoning without real-world interaction
- VLM-based safety evaluation of imagined futures to explicitly couple future anticipation with semantic risk assessment
- Unified closed-loop framework (observe-imagine-evaluate-act) for safer and more efficient offline policy learning
Methodology
VL-SAFE is an offline safe RL framework that follows a human cognitive loop of observe-imagine-evaluate-act. It uses offline driving data, a VLM to provide semantic safety signals, a world model to predict future trajectories, and VLM-based safety evaluation of those imagined futures to guide actor-critic policy optimization.
Key Results
In simulation experiments, VL-SAFE achieves improved safety, stronger robustness under traffic-density shift, and a better safety-performance trade-off compared to representative baselines.