Menu

Zelili AI

ShapeR

From Meta (Facebook AI Research)

Robust Conditional 3D Shape Generation from Casual Captures – Metric Reconstruction Using Multimodal Inputs
Tool Release Date

16 Jan 2026

Tool Users
N/A
0.0

Pricing Model

Starting Price

$0/Month

Visit ShapeR
👁 99

About This AI

ShapeR is a research project from Meta Reality Labs (Facebook AI Research) that introduces a novel approach to metric 3D shape generation from casual image sequences.

It preprocesses input data to extract per-object multimodal information including sparse SLAM points, images, camera poses, and text captions.

The model generates high-quality, metric-accurate 3D meshes conditioned on these inputs using a rectified flow transformer and VecSet latents.

ShapeR operates object-centrically, enabling scene-wide reconstruction by applying it to each detected object.

It handles real-world challenges like occlusions, clutter, variable viewpoints, and resolutions, outperforming prior methods in in-the-wild scenarios.

The project includes a new evaluation dataset with 178 objects across 7 diverse scenes, featuring paired multi-view images, SLAM points, and complete 3D annotations.

Inference supports configurable modes: quality (16 views, 50 steps), balance (recommended, 16 views, 25 steps), and speed (4 views, 10 steps).

Released on January 16, 2026, with pretrained models on Hugging Face, full code on GitHub, and a research paper on arXiv (2601.11514).

Primarily for non-commercial research use under CC-BY-NC license, it advances conditional 3D reconstruction for applications in AR/VR, robotics, and content creation.

Key Features

  1. Metric 3D shape generation: Produces scale-accurate meshes from casual captures using SLAM points and images
  2. Multimodal conditioning: Leverages sparse point clouds, multi-view images, poses, and text captions for robust reconstruction
  3. Object-centric processing: Reconstructs individual objects then composes into full scenes
  4. Configurable inference modes: Quality, balance (recommended), and speed options for different trade-offs
  5. Post-processing tools: Mesh cleanup, simplification, transformation to world coordinates, and visualization
  6. In-the-wild evaluation dataset: New benchmark with 178 objects in 7 scenes for realistic testing
  7. Pretrained models: Available on Hugging Face for immediate inference
  8. Full open-source code: GitHub repo with preprocessing, inference, and data exploration notebooks

Price Plans

  1. Free ($0): Fully open-source code, pretrained models, dataset, and documentation available on GitHub and Hugging Face under CC-BY-NC license for non-commercial research use
  2. Commercial (Not Available): License prohibits commercial use; contact Meta for potential licensing inquiries

Pros

  1. Strong in-the-wild performance: Handles occlusions, clutter, and casual captures better than prior methods
  2. Metric accuracy: Generates scale-consistent 3D shapes suitable for AR/VR and robotics
  3. Comprehensive release: Includes code, pretrained models, paper, dataset, and demo resources
  4. Research-grade quality: Advances conditional 3D generation with rectified flow transformer
  5. Easy experimentation: Balance mode offers good speed/quality trade-off for testing

Cons

  1. Non-commercial license: CC-BY-NC restricts use to research; no commercial applications allowed
  2. Requires preprocessing: Needs SLAM points, poses, and detection (e.g., Aria pipeline) before inference
  3. Compute-intensive: Quality mode (50 steps) can be slow on standard hardware
  4. Setup complexity: Involves Conda, CUDA, and specific package installation per INSTALL.md
  5. Research-focused: Not a user-friendly app; aimed at developers and researchers
  6. Limited adoption metrics: Very recent release with no widespread user numbers reported

Use Cases

  1. AR/VR content creation: Generate accurate 3D object models from phone captures
  2. Robotics simulation: Create metric 3D assets for training and testing in virtual environments
  3. Research in 3D reconstruction: Benchmarking and extending conditional shape generation methods
  4. Scene understanding: Reconstruct real-world objects with scale for computer vision studies
  5. Dataset augmentation: Use generated shapes to expand training data for downstream tasks

Target Audience

  1. Computer vision researchers: Studying 3D reconstruction and generative models
  2. AR/VR developers: Needing metric-accurate 3D assets from casual inputs
  3. Robotics engineers: Requiring realistic object models for simulation
  4. Academic teams: Using the evaluation dataset and code for papers and experiments
  5. AI enthusiasts: Experimenting with open-source 3D generation models

How To Use

  1. Clone repo: Git clone https://github.com/facebookresearch/ShapeR
  2. Install environment: Follow INSTALL.md to set up Conda, CUDA, and packages
  3. Download model: Get pretrained weights from Hugging Face (facebook/ShapeR)
  4. Prepare input: Use preprocessed pickle file with SLAM points, images, poses, captions
  5. Run inference: python infer_shape.py --input_pkl sample.pkl --config balance --output_dir output
  6. Customize: Add flags like --do_transform_to_world or --simplify_mesh for post-processing
  7. Visualize: Check saved meshes and visualizations in output directory

How we rated ShapeR

  • Performance: 4.5/5
  • Accuracy: 4.7/5
  • Features: 4.4/5
  • Cost-Efficiency: 5.0/5
  • Ease of Use: 3.8/5
  • Customization: 4.6/5
  • Data Privacy: 5.0/5
  • Support: 4.0/5
  • Integration: 4.3/5
  • Overall Score: 4.5/5

ShapeR integration with other tools

  1. Hugging Face: Pretrained models and evaluation dataset hosted for easy download and inference
  2. GitHub: Full source code, installation guide, inference scripts, and data exploration notebooks
  3. Python Ecosystem: Built with PyTorch and compatible with standard 3D libraries for visualization and processing
  4. Research Pipelines: Integrates with SLAM systems like Aria MPS for preprocessing input data
  5. Local Compute: Runs on GPU hardware with no cloud dependency for core reconstruction

Best prompts optimised for ShapeR

  1. N/A - ShapeR is not a prompt-based generative tool like text-to-image/video; it reconstructs 3D shapes from preprocessed multimodal input data (image sequences, SLAM points, poses, captions) rather than natural language prompts.
  2. N/A - The model uses fixed conditioning on input data files; no user-provided text prompts are required for shape generation.
  3. N/A - Focus is on conditional reconstruction from casual captures, not free-form prompting.
ShapeR advances metric 3D shape reconstruction from casual captures with strong real-world robustness and multimodal conditioning. Fully open-source for research, it excels in accuracy and scene understanding. Setup and non-commercial license limit broader use, but it’s a valuable resource for AR/VR and robotics researchers seeking high-quality conditional generation.

FAQs

  • What is ShapeR?

    ShapeR is a research project from Meta Reality Labs for robust conditional 3D shape generation and metric reconstruction from casual image sequences using multimodal inputs like SLAM points and captions.

  • When was ShapeR released?

    The code, models, and paper were publicly released on January 16, 2026, with arXiv publication 2601.11514.

  • Is ShapeR free to use?

    Yes, the code and pretrained models are freely available on GitHub and Hugging Face under CC-BY-NC license for non-commercial research use.

  • What license does ShapeR use?

    Primarily CC-BY-NC (Creative Commons Attribution-NonCommercial) for research purposes; check NOTICE file for component-specific terms.

  • What inputs does ShapeR require?

    Preprocessed pickle files containing per-object SLAM point clouds, multi-view images, camera poses, text captions, and optionally ground truth meshes.

  • Does ShapeR support commercial use?

    No, the CC-BY-NC license prohibits commercial applications; suitable only for non-commercial research and academic purposes.

  • Where can I find the ShapeR paper?

    The full paper is available on arXiv (2601.11514) and as PDF on the project page: https://facebookresearch.github.io/ShapeR.

  • Is there a demo or evaluation dataset for ShapeR?

    Yes, pretrained models on Hugging Face, a new evaluation dataset with 178 objects, and code for inference/visualization are provided.

Newly Added Tools​

CodeRabbit

$0/Month

Code Genius

$0/Month

AskCodi

$20/Month

PearAI

$0/Month
ShapeR Alternatives

GLM-OCR

$0/Month

Lummi AI

$10/Month

Bing Image Creator

$0/Month

Latest AI News

ShapeR Reviews

0.0
0.0 out of 5 stars (based on 0 reviews)
Excellent0%
Very good0%
Average0%
Poor0%
Terrible0%

There are no reviews yet. Be the first one to write one.