SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
TLDR
Introduces SAEScientist-Bench, a benchmark for evaluating AI agents' autonomous interpretability research using Sparse Autoencoders; agents show promise but lag experts.
Reasoning
The paper presents a novel benchmark with concrete tasks, expert baselines, and multiple agent configurations, providing empirical evidence of autonomous discovery capabilities. However, it also reveals limitations such as frequent misinterpretation of experimental measurements, and the focus is narrow to SAE interpretability rather than general scientific research.
Read-first score
Read-first score 70.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 74.
Field roles
Rank sensitivity
Stability: volatile; rank range: 19.
Keyword Scores
Deep Analysis
Innovations
- Introduces SAEScientist-Bench, a benchmark for evaluating whether AI agents can conduct autonomous mechanistic interpretability research using SAEs.
- Positions post-hoc monitoring and auditing with mechanistic interpretability as a missing pillar for reliable recursive self-improvement.
- Defines an evaluation protocol in which agents design contrastive probes and search a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT, judged against Neuronpedia-anchored expert reference features on activation rank, contrastive selectivity, and causal steering.
- Provides empirical evidence that frontier agents show genuine discovery capability but remain below expert performance, especially in causal generation steering.
Methodology
Given a target concept, AI agents design contrastive probes and navigate a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT to discover the optimal feature. Performance is evaluated against curated expert reference features anchored on Neuronpedia using activation rank, concept selectivity on contrastive texts, and causal steering. The study compares 10 agent configurations across 20 tasks.
Key Results
Frontier agents demonstrate genuine discovery capabilities and lead different evaluation dimensions, but remain well behind the expert baseline, approaching expert levels on separating target concepts from contrastive controls while lagging substantially in causal generation steering. Further analysis shows agents can design contrasts to rule out spurious candidates but frequently misinterpret experimental measurements.
Limitations
- Agents remain well behind the expert baseline overall.
- Agents lag substantially in causal generation steering.
- Agents frequently misinterpret experimental measurements.