Community Projects

ManiSkill alone only provides the framework for working with robot learning and simulation. We are excited to see all the different algorithms, benchmarks, and tools the community has built on ManiSkill to make new advancements at the frontier of robotics. We hand-picked a number of notable projects that are well made and recommend those interested in diving deep into different areas of robotics/embodied AI to check out these projects.

Sim-to-Real Transfer

• Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics - Fast (< 10 minutes) sim-to-real transfer of skills for the low-cost SO101 robot arm.

• HandelBot: Real-World Piano Playing via Fast Adaptation of Dexterous Robot Policies - Sim-to-real transfer of piano skills.

• DexDrummer: In-Hand, Contact-Rich, and Long-Horizon Dexterous Robot Drumming - Sim-to-real transfer of drumming skills.

Simulation Evaluations for Real-World policies

• GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulation - Gaussian splatting added to custom ManiSkill environments for evaluation of real-world robotics foundation models and sim-to-real transfer.

• Evaluating Real-World Robot Manipulation Policies in Simulation - One of the first widely used evaluation benchmarks for robotics foundation models.

• What Can RL Bring to VLA Generalization? An Empirical Study - Systematic study of RL finetuning for VLA foundation models.

• RoboTwin - A scalable simulation data generator and benchmark targeting bimanual manipulation, testing robotics foundation models and imitation learning algorithms.

• Molmospaces - A large scale benchmark of tasks for robotics evaluation in apartment scenes.

Benchmarks/Datasets

(note: some benchmarks may overlap with real world policy evaluation projects)

Robotics Benchmarks/Datasets

• MIKASA-Robo-VLA - A memory-intensive robotic manipulation benchmark for Vision-Language-Action research.

• RoboFactory: Exploring Embodied Agent Collaboration with Compositional Constraints - Benchmark on multi-agent manipulation as well as data collection framework.

• Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation - Large dataset of dextrous manipulation for grasping and articulation in simulation generated via generative models.

Other

• VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents

• Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation

Algorithms/Models

• TD-MPC2 / Newt - Training massively multitask world models for continuous control problems.

• FlashSAC - Improved version of the SAC RL algorithm to adapt to the high throughput regime of GPU parallelized simulators.

• A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation - Affordance based hiearchical model for robot manipulation trained on a variety of data sources including simulation data.

• RLinf - RL infrastructure for robotics foundation models.

Teleoperation

• ImVR: Immersive VR Teleoperation System for General Purpose Manipulation - Local connection/ALVR based teleoperation.

• Maniskill Quest Teleop - WebXR based teleoperation.