Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture
TLDR
A functional architecture enforces statistical rigor in AI-driven discovery by controlling false discovery rate via a monadic DSL and formal verification.
Reasoning
Strengths: novel integration of formal methods (Lean4, SPARK/Ada) to ensure statistical rigor, with simulation and adversarial evaluation. Weaknesses: lacks real-world empirical validation; case studies may be synthetic, limiting generalizability.
Read-first score
Read-first score 55.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 68.
Field roles
Rank sensitivity
Stability: volatile; rank range: 68.
Keyword Scores
Deep Analysis
Innovations
- Research monad: a Haskell embedded DSL that enforces error budget updates for every hypothesis test, making it impossible to test without accounting for multiplicity.
- Declarative scaffold with OS-level sandbox that physically isolates validation data from the environment where LLM-generated code runs.
- Machine-checked formalization of LORD++ online FDR control in Lean 4, including derivation of the error budget and proofs of marginal and full FDR control.
- Verification of the budget's wealth invariant over IEEE 754 floating-point arithmetic using SPARK/Ada, closing the gap from theorem to implementation.
- First verified chain from theorem to floating-point implementation for an online FDR procedure.
Methodology
The architecture combines a Haskell EDSL (Research monad) that forces hypothesis tests to consume an error budget, and a sandboxed execution environment that withholds validation data from LLM-generated code. The design is grounded in a Lean 4 formalization of LORD++ online FDR control, with proofs of FDR control, and the wealth invariant is verified over IEEE 754 arithmetic in SPARK/Ada. Evaluation uses simulation, end-to-end case studies, and adversarial testing.
Key Results
Simulation shows the architecture holds false discovery rate near 1% against a 5% target, while a naive approach reaches 41%. End-to-end case studies demonstrate that valid tests avoid false discoveries but still detect real effects, and adversarial evaluation confirms generated code cannot access held-out data even when given its exact path.