Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve

arXiv 2020 41.7 method

TLDR

Mask2CAD jointly detects objects in images and retrieves the most similar CAD model and pose using a joint embedding space.

Reasoning

The paper's strength lies in leveraging large-scale 3D datasets for CAD-based object representation from real images, with experiments on Pix3D and ScanNet. However, it is limited to retrieval from existing CAD models, not generative or parametric modeling, and does not address novel shape synthesis.

Read-first score

Read-first score 41.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 43.

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,dataset,experiment

Recency 8%

42.4

Uses a gentle age decay so recent papers surface without erasing older foundations. 2020

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=dataset

Topical relevance 42%

26.9

Uses existing LLM keyword relevance scores normalized to 0-100. AI for CAD,computer-aided design,neural CAD,generative CAD,parametric CAD,B-Rep,boundary representation,constructive solid geometry,CSG,sketch extrusion,CAD generation,CAD reconstruction,text-to-CAD,image-to-CAD,point cloud to CAD,CAD program

Field roles

Methodology anchor

Rank sensitivity

Stability: volatile; rank range: 92.

Keyword Scores

image-to-CAD

computer-aided design

AI for CAD

CAD reconstruction

neural CAD

generative CAD

CAD generation

parametric CAD

B-Rep

boundary representation

constructive solid geometry

CSG

sketch extrusion

text-to-CAD

point cloud to CAD

CAD program

Deep Analysis

Innovations

Joint detection of objects in real-world images and retrieval of the most similar CAD model with pose optimization
Construction of a joint embedding space between image regions and 3D CAD models for retrieval
CAD-based representation of objects ensuring valid, efficient shape representation for applications
New image-to-3D baseline on ScanNet with larger shape diversity, real-world occlusions, and challenging views

Methodology

Mask2CAD detects objects in RGB images and for each detected object, retrieves the most similar CAD model and its pose by learning a joint embedding space between image regions and 3D CAD models, leveraging large-scale 3D model datasets.

Key Results

Experiments on Pix3D show advantage over state of the art; a new baseline on ScanNet is proposed for future research.

Resources

Paper