LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithms
TLDR
LLM evolutionary algorithm automatically generates Bayesian optimization code, outperforming baselines on benchmark functions.
Reasoning
The paper presents a novel method using LLMs with evolution strategies to generate BO algorithms, showing strong empirical results on standard benchmarks. However, it lacks real-world validation and focuses narrowly on BO algorithm generation, limiting generalizability.
Read-first score
Read-first score 58.4, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.
Field roles
Rank sensitivity
Stability: volatile; rank range: 92.
Keyword Scores
Deep Analysis
Innovations
- Using LLMs to automatically generate full Bayesian optimization algorithm code (initial design, surrogate model, acquisition function) rather than just components or non-BO algorithms.
- An evolutionary framework (LLaMEA-BO) that iteratively refines LLM-generated BO algorithms through selection, combination, and mutation via controlled prompt variations.
- Demonstrating that LLM-generated BO algorithms can outperform state-of-the-art baselines without any fine-tuning of the LLM.
Methodology
The framework uses an evolution strategy where an LLM generates multiple candidate BO algorithms in Python, preserving key components. Candidates are evaluated on the BBOB test suite, and top performers are selected, combined, and mutated via prompt variations for iterative refinement.
Key Results
LLM-generated algorithms outperform state-of-the-art BO baselines on 19 out of 24 BBOB functions in dimension 5 and generalize well to higher dimensions and different tasks (Bayesmark).