Foundation Models for Automatic CAD Generation
TLDR
Empirical study of foundation models for text-to-CAD generation, introducing LLMForge with iterative refinement and visual feedback, evaluated on 97 mechanical part designs.
Reasoning
The paper provides a thorough empirical evaluation with multiple models and a curated benchmark, demonstrating strong results. However, the scope is limited to four geometry families and mechanical parts, and the abstract does not detail limitations or comparisons to prior work.
Read-first score
Read-first score 57.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 75.
Field roles
Rank sensitivity
Stability: volatile; rank range: 9.
Keyword Scores
Deep Analysis
Innovations
- LLMForge: a multi-model text-to-CAD framework with JSON-schema validation, analytic feature scoring, mesh synthesis, and multi-round iterative refinement
- Two critique regimes: IterTracer (analytic visual metrics via ray-tracing) and IterVision (VLM semantic critic with chain-of-thought visual reasoning)
- Unified evaluation pipeline and curated benchmark of 97 engineering design problems across four geometry families
- Empirical comparison of seven foundation models revealing that compact instruction-tuned models can match larger systems
Methodology
LLMForge integrates LLMs for text-to-CAD generation with JSON-schema validation, analytic feature scoring, and mesh synthesis, using iterative refinement under two critique types: IterTracer with Phong-shaded ray-tracing and analytic metrics, and IterVision with a VLM (Qwen2.5-VL-72B) performing chain-of-thought visual reasoning. The framework is evaluated on a benchmark of 97 mechanical design problems spanning plates, boxes, flanged cylinders, and L-brackets, using seven foundation models.
Key Results
Under IterTracer, the top four models achieve a tight cluster of mean scores (0.885–0.890) and 98.97% mesh success, showing compact models can rival larger ones. IterVision attains 100% watertight mesh generation for the leading model but exposes difficulties with rotationally symmetric parts like cylinders, where visual and semantic scoring diverge.
Limitations
- Systematic difficulty with rotationally symmetric geometries (e.g., cylinders) where visual and semantic scoring diverge
- Benchmark limited to four canonical geometry families of mechanical parts
- VLM-based critic may not fully align with analytic metrics on certain shapes