Text-to-CAD Evaluation with CADTests
TLDR
Introduces CADTestBench, a test-based benchmark for evaluating Text-to-CAD models using executable software tests.
Reasoning
The paper addresses a critical gap in Text-to-CAD evaluation with a novel automated testing approach. Its strengths include a practical benchmark and demonstrated utility for guiding generation, though the abstract lacks details on real-world deployment or limitations of the tests.
Read-first score
Read-first score 59.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 63.
Field roles
Rank sensitivity
Stability: volatile; rank range: 20.
Keyword Scores
Deep Analysis
Innovations
- New evaluation perspective for Text-to-CAD based on automated testing
- CADTestBench: the first test-based benchmark for Text-to-CAD
- CADTests: executable software tests that verify geometric and topological requirements of generated CAD models
- Using CADTests to guide CAD model generation, yielding simple baselines that surpass current methods
Methodology
The paper introduces CADTestBench, a benchmark that uses CADTests (executable software tests) to evaluate Text-to-CAD models by checking whether generated CAD models satisfy the geometric and topological constraints from input prompts. They benchmark recent methods and also employ CADTests to guide generation, creating simple baselines that outperform existing approaches.
Key Results
Comprehensive benchmarking of recent Text-to-CAD methods is conducted; simple baselines guided by CADTests surpass the performance of current methods.