Epistemic Uncertainty for Test-Time Discovery
TLDR
UG-TTT uses ensemble of low-rank adapters to measure epistemic uncertainty, guiding exploration in automated scientific discovery to overcome reward plateaus.
Reasoning
The paper introduces a novel method (UG-TTT) that leverages epistemic uncertainty via ensemble disagreement to improve exploration in automated scientific discovery, addressing a key limitation of standard RL. Strengths include a clear problem formulation and empirical validation on four benchmarks. Weaknesses include limited scope of benchmarks and lack of detailed task descriptions in the abstract.
Read-first score
Read-first score 58.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 65.
Field roles
Rank sensitivity
Stability: volatile; rank range: 26.
Keyword Scores
Deep Analysis
Innovations
- Epistemic uncertainty measured via mutual information between ensemble predictions and weight hypotheses as an exploration bonus in policy gradient
- Ensemble of low-rank adapters over a frozen base model to capture per-token disagreement
- Nuclear norm regularizer to keep adapters distinct and preserve exploration signal
Methodology
UG-TTT freezes a base LLM and trains a small ensemble of low-rank adapters. Per-token epistemic uncertainty is quantified as the mutual information between the ensemble's predictions and the adapter weight hypotheses, isolating disagreement from insufficient coverage. This uncertainty is added as an exploration bonus to the policy gradient, guiding the policy toward low-coverage regions. A nuclear norm regularizer maintains adapter diversity.
Key Results
On four scientific discovery benchmarks, UG-TTT increased maximum reward on three tasks and maintained higher solution diversity; ablation showed the nuclear norm regularizer is essential for sustaining exploration.
Limitations
- Did not increase maximum reward on one of the four benchmarks