IdeaAMBIG:基准测试研究想法规范中影响实现的关键缺口
TLDR
IdeaAMBIG:用660个证据实例评测研究规范能否被编码代理直接实现。
评分理由
The paper presents a novel benchmark combining real-world and synthetic gaps, evaluating LLMs on codification-readiness, defect localization, and clarification generation. Strengths include evidence-grounded construction and multi-model evaluation; weaknesses are not fully visible from the abstract, though the low defect localization rates suggest this capability remains a key bottleneck.
Read-first 评分解释
综合优先阅读分 34.6,由主题、引用、图谱、方法、可复现性和近期性等信号加权得到。 原始总分保留为 18。
研究版图角色
前沿论文
排序敏感性
稳定性:volatile;排名波动范围:21。