Menu

Zelili AI

UniSH

From Academic Research (Hong Kong University of Science and Technology)

Unified Feed-Forward Framework for Joint 3D Scene and Human Reconstruction from Monocular Video
Tool Release Date

3 Jan 2026

Tool Users
N/A
0.0

Pricing Model

Starting Price

$0/Month

Visit UniSH
👁 95

About This AI

UniSH is a cutting-edge research project and AI model that unifies scene and human reconstruction in a single feed-forward pass.

It takes monocular video as input to jointly recover high-fidelity metric-scale 3D scene geometry, human point clouds, camera parameters, and coherent SMPL bodies.

The framework bridges strong priors from scene reconstruction and human motion recovery (HMR), using a Reconstruction Branch for per-frame camera extrinsics/confidence/pointmaps and a Human Body Branch for global SMPL shape and per-frame pose parameters.

Features are fused via AlignNet to predict global scene scale and SMPL translations for alignment.

It employs robust distillation from an expert depth model for refined human surface details and a two-stage supervision scheme (coarse on synthetic data, fine-tune on real data optimizing geometric correspondence).

UniSH achieves state-of-the-art performance on human-centric scene reconstruction and highly competitive results on global human motion estimation.

Capable of handling challenging dynamic scenes with strong spatial-temporal coherence in one forward pass.

Released as open research in January 2026 with project page, arXiv paper, and code likely available on GitHub.

Primarily for computer vision researchers, 3D reconstruction developers, AR/VR applications, robotics, animation, and motion analysis.

No commercial pricing or hosted service; fully academic/open-source oriented.

Key Features

  1. Joint 3D scene and human reconstruction: Recovers scene geometry, human point clouds, cameras, and SMPL bodies together
  2. Feed-forward single-pass inference: No iterative optimization; processes video in one forward network pass
  3. Monocular video input: Works with ordinary single-camera footage without depth sensors
  4. Metric-scale output: Produces coherent, real-world sized reconstructions
  5. Reconstruction Branch: Predicts per-frame camera extrinsics, confidence maps, and pointmaps
  6. Human Body Branch: Estimates global SMPL shape and per-frame pose parameters
  7. AlignNet fusion: Aligns scene and human via global scale and per-frame translations
  8. Robust human detail distillation: Refines surface from expert depth model for high-fidelity humans
  9. Two-stage supervision: Coarse synthetic pretraining followed by real-data fine-tuning on geometric correspondence
  10. Strong spatial-temporal coherence: Handles dynamic scenes with consistent geometry and motion

Price Plans

  1. Free ($0): Fully open academic/research project with paper, project page, and likely code/weights on GitHub; no cost for use, implementation, or experimentation

Pros

  1. State-of-the-art human-centric reconstruction: Leads benchmarks for unified scene-human tasks
  2. Competitive global motion estimation: Strong results without complex post-processing
  3. Efficient single-pass design: Faster inference than iterative or multi-stage methods
  4. High-fidelity details: Excellent human surface refinement and scene accuracy
  5. Handles challenging dynamics: Robust to motion, occlusion, and complex interactions
  6. Open research impact: Advances toward real-time 3D understanding from video
  7. Metric-scale coherence: Unified alignment prevents scale drift between scene and humans

Cons

  1. Research-oriented only: No hosted demo or easy-to-use app; requires local implementation
  2. Hardware demands: Large model likely needs strong GPU for practical inference
  3. No real-time yet: Feed-forward but not optimized for live video in release
  4. Limited accessibility: Academic code/weights may require setup and expertise
  5. No commercial support: Pure research; no enterprise features or API
  6. Monocular limitations: Performance may degrade on very fast motion or extreme views
  7. Early-stage release: January 2026 arXiv; community adoption still emerging

Use Cases

  1. AR/VR content creation: Generate 3D scene-human models from video for immersive experiences
  2. Robotics and embodied AI: Reconstruct environments with humans for navigation/training
  3. Motion capture and animation: Accurate human pose/shape from monocular video
  4. Autonomous systems simulation: Build realistic dynamic scenes with people
  5. Film and VFX pre-production: Quick 3D reconstruction from footage for digital doubles
  6. Computer vision research: Benchmarking or extending unified reconstruction methods
  7. Sports analysis: Track and reconstruct athlete movements in real scenes

Target Audience

  1. Computer vision researchers: Advancing 3D reconstruction and human motion
  2. AR/VR developers: Needing fast scene-human modeling from video
  3. Robotics engineers: Simulating human-included environments
  4. Animation and VFX artists: Creating digital humans and scenes from real footage
  5. Academic institutions: Students and professors in CV/ML labs
  6. AI enthusiasts in 3D vision: Experimenting with open-source reconstruction models

How To Use

  1. Visit project page: Go to murphylmf.github.io/UniSH for paper, demos, and code links
  2. Read arXiv paper: Download arXiv:2601.01222 for architecture and method details
  3. Clone repository: If code released, git clone the GitHub repo (check issues for HF release)
  4. Install dependencies: Set up PyTorch, dependencies per repo requirements
  5. Download model: Get pretrained weights if available on Hugging Face or project page
  6. Run inference: Input monocular video; model outputs scene geometry, SMPL params, point clouds
  7. Visualize results: Use provided scripts or tools like MeshLab/Blender for 3D viewing

How we rated UniSH

  • Performance: 4.8/5
  • Accuracy: 4.9/5
  • Features: 4.6/5
  • Cost-Efficiency: 5.0/5
  • Ease of Use: 3.8/5
  • Customization: 4.7/5
  • Data Privacy: 5.0/5
  • Support: 4.0/5
  • Integration: 4.4/5
  • Overall Score: 4.6/5

UniSH integration with other tools

  1. PyTorch Ecosystem: Built with PyTorch for easy extension and training
  2. Hugging Face (Potential): Model weights likely hosted for inference pipelines
  3. 3D Visualization Tools: Outputs compatible with MeshLab, Blender, Unity/Unreal for rendering
  4. Computer Vision Libraries: Integrates with Open3D, PyTorch3D for post-processing
  5. Research Frameworks: Compatible with SMPL/SMPL-X body models and standard CV datasets

Best prompts optimised for UniSH

  1. N/A - UniSH is a feed-forward computer vision reconstruction model that takes monocular video as input, not text prompts. It processes raw video frames directly without user text descriptions.
  2. N/A - This is a research model for 3D scene/human reconstruction from video; no text-to-3D or prompt-based generation feature.
  3. N/A - Usage involves feeding video input to the network; no manual prompting required or supported.
UniSH represents a major research advance in unified 3D scene and human reconstruction from monocular video, achieving SOTA human-centric results with efficient single-pass inference. Fully open and free, it’s ideal for CV researchers and developers in AR/VR/robotics. Technical setup limits accessibility for non-experts, but its metric-scale coherence and detail refinement make it a powerful tool for next-gen 3D understanding.

FAQs

  • What is UniSH?

    UniSH is a research AI model and framework for joint metric-scale 3D reconstruction of scenes and humans from monocular video in a single feed-forward pass.

  • Who created UniSH?

    UniSH was developed by researchers at Hong Kong University of Science and Technology (Murphy Li and team), released on arXiv in January 2026.

  • Is UniSH free to use?

    Yes, it is an open academic research project with paper and likely code/weights freely available; no commercial pricing.

  • What does UniSH reconstruct?

    It jointly recovers high-fidelity scene geometry, human point clouds, camera parameters, and coherent metric-scale SMPL bodies from video.

  • What input does UniSH take?

    Monocular video (single camera footage) as input, no need for depth sensors or multi-view setup.

  • Is UniSH real-time capable?

    The feed-forward design enables efficient inference, though real-time performance depends on hardware and optimization.

  • Where can I find UniSH code?

    Check the project page murphylmf.github.io/UniSH/ or arXiv 2601.01222 for code release; GitHub likely hosts implementation.

  • What are UniSH’s main applications?

    Suited for AR/VR, robotics simulation, motion capture, VFX, autonomous systems, and computer vision research.

Newly Added Tools​

CodeRabbit

$0/Month

Code Genius

$0/Month

AskCodi

$20/Month

PearAI

$0/Month
UniSH Alternatives

CodeRabbit

$0/Month

Code Genius

$0/Month

AskCodi

$20/Month

Latest AI News

UniSH Reviews

0.0
0.0 out of 5 stars (based on 0 reviews)
Excellent0%
Very good0%
Average0%
Poor0%
Terrible0%

There are no reviews yet. Be the first one to write one.