Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks

创新点

• Introduces Memento 3, enabling frozen LLM agents to continually learn explicit world models through external memory.
• Uses a natural-language rulebook as persistent semantic memory that stores revisable environment-dynamics hypotheses while leaving unknown aspects underspecified.
• Compiles the rulebook into executable code for prediction and planning.
• Defines a continual loop of observation, reflection, rule revision, compilation, and verification that uses prediction errors to refine the rulebook and code.
• Accepts updated code only when an LLM judges it faithful to the rulebook and cell-exact replay reproduces observed transitions.
• Proposes a model-based route to recursive self-improvement with a frozen underlying LLM, where verified world-model updates guide subsequent interaction and learning.
• Includes a population extension that maintains multiple world models in parallel, shares interaction evidence, and uses predictions to guide exploration.

方法

Memento 3 equips frozen LLM agents with an external natural-language rulebook as persistent semantic memory, which is compiled into executable code for prediction and planning. The agent iterates through observation, reflection, rule revision, compilation, and verification, accepting code updates only when the LLM judges them faithful to the rulebook and cell-exact replay matches observed transitions. Evaluation uses ARC-AGI-3 public games and an Atari Pong case study, reporting Relative Human Action Efficiency and action counts.

关键结果

On ARC-AGI-3, the single-model agent clears every level of all 25 public games, achieves a mean Relative Human Action Efficiency of 100.0, and uses 44% of the human action count. In the Atari Pong case study, a learned feedback controller wins 21:0 in each of three evaluated episodes with different openings without further LLM calls.