mani_skill.envs =============== .. py:module:: mani_skill.envs Submodules ---------- .. toctree:: :maxdepth: 1 /api/mani_skill/envs/minimal_template/index /api/mani_skill/envs/sapien_env/index /api/mani_skill/envs/scene/index /api/mani_skill/envs/scenes/index /api/mani_skill/envs/sim2real_env/index /api/mani_skill/envs/tasks/index /api/mani_skill/envs/template/index /api/mani_skill/envs/utils/index Classes ------- .. autoapisummary:: mani_skill.envs.AntRun mani_skill.envs.AntWalk mani_skill.envs.AssemblingKitsEnv mani_skill.envs.CartpoleBalanceEnv mani_skill.envs.CartpoleSwingUpEnv mani_skill.envs.DrawSVGEnv mani_skill.envs.DrawTriangleEnv mani_skill.envs.EmptyEnv mani_skill.envs.FMBAssembly1Env mani_skill.envs.HopperHopEnv mani_skill.envs.HopperStandEnv mani_skill.envs.HumanoidPlaceAppleInBowl mani_skill.envs.HumanoidRun mani_skill.envs.HumanoidStand mani_skill.envs.HumanoidStandEnv mani_skill.envs.HumanoidWalk mani_skill.envs.InsertFlowerEnv mani_skill.envs.LiftPegUprightEnv mani_skill.envs.OpenCabinetDoorEnv mani_skill.envs.OpenCabinetDrawerEnv mani_skill.envs.PegInsertionSideEnv mani_skill.envs.PickClutterYCBEnv mani_skill.envs.PickCubeEnv mani_skill.envs.PickSingleYCBEnv mani_skill.envs.PlaceSphereEnv mani_skill.envs.PlugChargerEnv mani_skill.envs.PokeCubeEnv mani_skill.envs.PullCubeEnv mani_skill.envs.PullCubeToolEnv mani_skill.envs.PushCubeEnv mani_skill.envs.PushTEnv mani_skill.envs.PutCarrotOnPlateInScene mani_skill.envs.PutEggplantInBasketScene mani_skill.envs.PutSpoonOnTableClothInScene mani_skill.envs.QuadrupedReachEnv mani_skill.envs.QuadrupedSpinEnv mani_skill.envs.RoboCasaKitchenEnv mani_skill.envs.RollBallEnv mani_skill.envs.RotateCubeEnv mani_skill.envs.RotateSingleObjectInHand mani_skill.envs.RotateValveEnv mani_skill.envs.SO100GraspCubeEnv mani_skill.envs.SceneManipulationEnv mani_skill.envs.StackCubeEnv mani_skill.envs.StackGreenCubeOnYellowCubeBakedTexInScene mani_skill.envs.StackPyramidEnv mani_skill.envs.TableTopFreeDrawEnv mani_skill.envs.TransportBoxEnv mani_skill.envs.TurnFaucetEnv mani_skill.envs.TwoRobotPickCube mani_skill.envs.TwoRobotStackCube Functions --------- .. autoapisummary:: mani_skill.envs.register_env Package Contents ---------------- .. py:class:: AntRun(*args, **kwargs) Bases: :py:obj:`AntEnv` **Task Description:** Ant moves in x direction at 4 m/s **Randomizations:** - Ant qpos and qvel have added noise from uniform distribution [-1e-2, 1e-2] **Success Conditions:** - No specific success conditions. .. py:class:: AntWalk(*args, **kwargs) Bases: :py:obj:`AntEnv` **Task Description:** Ant moves in x direction at 0.5 m/s **Randomizations:** - Ant qpos and qvel have added noise from uniform distribution [-1e-2, 1e-2] **Success Conditions:** - No specific success conditions. .. py:class:: AssemblingKitsEnv(asset_root=f'{ASSET_DIR}/tasks/assembling_kits', robot_uids='panda_wristcam', num_envs=1, reconfiguration_freq=None, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** The robot must pick up one of the misplaced shapes on the board/kit and insert it into the correct empty slot. **Randomizations:** - the kit geometry is randomized, with different already inserted shapes and different holes affording insertion of specific shapes. (during reconfiguration) - the misplaced shape's geometry is sampled from one of 20 different shapes. (during reconfiguration) - the misplaced shape is randomly spawned anywhere on top of the board with a random z-axis rotation **Success Conditions:** - the misplaced shape is inserted completely into the correct slot .. py:method:: _check_in_slot(obj, height_eps=0.003) .. py:method:: _check_pos_diff(pos_eps=0.02) .. py:method:: _check_rot_diff(rot_eps=np.deg2rad(4)) .. py:method:: _get_kit_builder_and_goals(kit_id) .. py:method:: _get_object_builder(object_id, static = False, color_id = 0) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: _parse_json(path) Parse kit JSON information and return the goal positions and rotations .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['sparse', 'none'] .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda_wristcam'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _episode_json .. py:attribute:: _episodes .. py:attribute:: _kit_dir .. py:attribute:: _models_dir .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/AssemblingKits-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.PandaWristCam] .. py:attribute:: asset_root .. py:attribute:: color .. py:attribute:: episode_idx :value: None .. py:attribute:: object_scale .. py:attribute:: symmetry .. py:class:: CartpoleBalanceEnv(*args, **kwargs) Bases: :py:obj:`CartpoleEnv` **Task Description:** Use the Cartpole robot to balance a pole on a cart. **Randomizations:** - Pole direction is randomized around the vertical axis. the range is [-0.05, 0.05] radians. **Fail Conditions:** - Pole is lower than the horizontal plane .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/MS-CartpoleBalance-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:class:: CartpoleSwingUpEnv(*args, **kwargs) Bases: :py:obj:`CartpoleEnv` **Task Description:** Use the Cartpole robot to swing up a pole on a cart. **Randomizations:** - Pole direction is randomized around the whole circle. the range is [-pi, pi] radians. **Success Conditions:** - No specific success conditions. The task is considered successful if the pole is upright for the whole episode. .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:attribute:: SUPPORTED_REWARD_MODES :value: ('normalized_dense', 'dense', 'none') .. py:class:: DrawSVGEnv(*args, svg=None, robot_uids='panda_stick', **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Instantiates a table with a white canvas on it and a svg path specified with an outline. A robot with a stick is to draw the triangle with a red line. **Randomizations:** - the goal svg's position on the xy-plane is randomized - the goal svg's z-rotation is randomized in range [0, 2 $\pi$] **Success Conditions:** - the drawn points by the robot are within a euclidean distance of 0.05m with points on the goal svg .. py:method:: _after_control_step() Code that runs after each action has been taken. On GPU simulation this is called right before observations are fetched from the GPU buffers. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: success_check() .. py:attribute:: BRUSH_COLORS :value: [[0.8, 0.2, 0.2, 1]] The colors of the brushes. If there is more than one color, each parallel environment will have a randomly sampled color. .. py:attribute:: BRUSH_RADIUS :value: 0.01 The brushes radius .. py:attribute:: CANVAS_THICKNESS :value: 0.02 How thick the canvas on the table is .. py:attribute:: DOT_THICKNESS :value: 0.003 thickness of the paint drawn on to the canvas .. py:attribute:: MAX_DOTS :value: 1000 The total "ink" available to use and draw with before you need to call env.reset. NOTE that on GPU simulation it is not recommended to have a very high value for this as it can slow down rendering when too many objects are being rendered in many scenes. .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['sparse'] .. py:attribute:: SUPPORTED_ROBOTS :type: [panda_stick] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: THRESHOLD :value: 0.1 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/DrawSVG-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.robots.panda.panda_stick.PandaStick .. py:class:: DrawTriangleEnv(*args, robot_uids='panda_stick', **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Instantiates a table with a white canvas on it and a goal triangle with an outline. A robot with a stick is to draw the triangle with a red line. **Randomizations:** - the goal triangle's position on the xy-plane is randomized - the goal triangle's z-rotation is randomized in range [0, 2 $\pi$] **Success Conditions:** - the drawn points by the robot are within a euclidean distance of 0.05m with points on the goal triangle .. py:method:: _after_control_step() Code that runs after each action has been taken. On GPU simulation this is called right before observations are fetched from the GPU buffers. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: generate_triangle_with_points(n, vertices) .. py:method:: success_check() .. py:attribute:: BRUSH_COLORS :value: [[0.8, 0.2, 0.2, 1]] The colors of the brushes. If there is more than one color, each parallel environment will have a randomly sampled color. .. py:attribute:: BRUSH_RADIUS :value: 0.01 The brushes radius .. py:attribute:: CANVAS_THICKNESS :value: 0.02 How thick the canvas on the table is .. py:attribute:: DOT_THICKNESS :value: 0.003 thickness of the paint drawn on to the canvas .. py:attribute:: MAX_DOTS :value: 300 The total "ink" available to use and draw with before you need to call env.reset. NOTE that on GPU simulation it is not recommended to have a very high value for this as it can slow down rendering when too many objects are being rendered in many scenes. .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['sparse'] .. py:attribute:: SUPPORTED_ROBOTS :type: [panda_stick] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: THRESHOLD :value: 0.025 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/DrawTriangle-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.robots.panda.panda_stick.PandaStick .. py:class:: EmptyEnv(*args, robot_uids='panda', **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['none'] This is just a dummy environment for showcasing robots in a empty scene .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:class:: FMBAssembly1Env(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Task Description ---------------- This task is a simulation version of one of the Multi-Object Multi-Stage Manipulation Tasks (Assembly1) from [Functional Manipulation Benchmark (Luo et. al)](https://functional-manipulation-benchmark.github.io/index.html). The goal here is to assemble parts together with the help of a reorientation fixture onto a red board. Randomizations -------------- - TODO Success Conditions ------------------ - All objects are in the target locations. See Assembly 1 in https://functional-manipulation-benchmark.github.io/files/index.html Visualization: TODO: ADD LINK HERE .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['sparse', 'none'] .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: mani_skill.agents.robots.panda.panda.Panda .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: HopperHopEnv(*args, **kwargs) Bases: :py:obj:`HopperEnv` **Task Description:** Hopper robot stays upright and moves in positive x direction with hopping motion **Randomizations:** - Hopper robot is randomly rotated [-pi, pi] radians about y axis. - Hopper qpos are uniformly sampled within their allowed ranges **Success Conditions:** - No specific success conditions. The task is considered successful if the hopper hops for the whole episode. .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:class:: HopperStandEnv(*args, **kwargs) Bases: :py:obj:`HopperEnv` **Task Description:** Hopper robot stands upright **Randomizations:** - Hopper robot is randomly rotated [-pi, pi] radians about y axis. - Hopper qpos are uniformly sampled within their allowed ranges **Success Conditions:** - No specific success conditions. .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:class:: HumanoidPlaceAppleInBowl(*args, robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`HumanoidPickPlaceEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _grasp_release_reward() a dense reward that rewards the agent for opening their hand .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['normalized_dense', 'dense', 'sparse', 'none'] sets up a basic scene with a apple to pick up and place on a dish .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:class:: HumanoidRun(*args, robot_uids='humanoid', **kwargs) Bases: :py:obj:`HumanoidEnvStandard` **Task Description:** Humanoid moves in x direction at running pace **Randomizations:** - Humanoid qpos and qvel have added noise from uniform distribution [-1e-2, 1e-2] **Fail Conditions:** - Humanoid robot torso link leaves z range [0.7, 1.0] .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:attribute:: agent :type: Union[mani_skill.agents.robots.humanoid.Humanoid] .. py:class:: HumanoidStand(*args, robot_uids='humanoid', **kwargs) Bases: :py:obj:`HumanoidEnvStandard` **Task Description:** Humanoid robot stands upright **Randomizations:** - Humanoid robot is randomly rotated [-pi, pi] radians about z axis. - Humanoid qpos and qvel have added noise from uniform distribution [-1e-2, 1e-2] **Fail Conditions:** - Humanoid robot torso link leaves z range [0.7, 1.0] .. py:method:: _get_obs_state_dict(info) Get (ground-truth) state-based observations. .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:attribute:: agent :type: Union[mani_skill.agents.robots.humanoid.Humanoid] .. py:class:: HumanoidStandEnv(*args, robot_uids='unitree_h1_simplified', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_sparse_reward(obs, action, info) Computes the sparse reward. By default this function tries to use the success/fail information in returned by the evaluate function and gives +1 if success, -1 if fail, 0 otherwise. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['sparse', 'none'] .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: HumanoidWalk(*args, robot_uids='humanoid', **kwargs) Bases: :py:obj:`HumanoidEnvStandard` **Task Description:** Humanoid moves in x direction at walking pace **Randomizations:** - Humanoid qpos and qvel have added noise from uniform distribution [-1e-2, 1e-2] **Fail Conditions:** - Humanoid robot torso link leaves z range [0.7, 1.0] .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:attribute:: agent :type: Union[mani_skill.agents.robots.humanoid.Humanoid] .. py:class:: InsertFlowerEnv(*args, robot_uids='floating_ability_hand_right', robot_init_qpos_noise=0.02, num_envs=1, reconfiguration_freq=None, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _after_reconfigure(options) Add code here that should run immediately after self._reconfigure is called. The torch RNG context is still active so RNG is still seeded here by self._episode_seed. This is useful if you need to run something that only happens after reconfiguration but need the GPU initialized so that you can check e.g. collisons, poses etc. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_actors(env_idx) .. py:method:: _initialize_agent(env_idx) .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate(**kwargs) Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: _clearance :value: 0.003 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: Union[mani_skill.agents.robots.FloatingAbilityHandRight] .. py:attribute:: asset_path :value: 'Instance of pathlib.Path/tasks/oakink-v2/align_ds' .. py:attribute:: flower_spawn_half_size :value: 0.05 .. py:attribute:: hand_init_height :value: 0.25 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: target_area .. py:class:: LiftPegUprightEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A simple task where the objective is to move a peg laying on the table to any upright position on the table **Randomizations:** - the peg's xy position is randomized on top of a table in the region [0.1, 0.1] x [-0.1, -0.1]. It is placed flat along it's length on the table **Success Conditions:** - the absolute value of the peg's y euler angle is within 0.08 of $\pi$/2 and the z position of the peg is within 0.005 of its half-length (0.12). .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/LiftPegUpright-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: peg_half_length :value: 0.12 .. py:attribute:: peg_half_width :value: 0.025 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: OpenCabinetDoorEnv(*args, robot_uids='fetch', robot_init_qpos_noise=0.02, reconfiguration_freq=None, num_envs=1, **kwargs) Bases: :py:obj:`OpenCabinetDrawerEnv` **Task Description:** Use the Fetch mobile manipulation robot to move towards a target cabinet and open the target drawer out. **Randomizations:** - Robot is randomly initialized 1.6 to 1.8 meters away from the cabinet and positioned to face it - Robot's base orientation is randomized by -9 to 9 degrees - The cabinet selected to manipulate is randomly sampled from all PartnetMobility cabinets that have drawers - The drawer to open is randomly sampled from all drawers available to open **Success Conditions:** - The drawer is open at least 90% of the way, and the angular/linear velocities of the drawer link are small **Goal Specification:** - 3D goal position centered at the center of mass of the handle mesh on the drawer to open (also visualized in human renders with a sphere). .. py:attribute:: TRAIN_JSON .. py:attribute:: handle_types :value: ['revolute', 'revolute_unwrapped'] .. py:class:: OpenCabinetDrawerEnv(*args, robot_uids='fetch', robot_init_qpos_noise=0.02, reconfiguration_freq=None, num_envs=1, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Use the Fetch mobile manipulation robot to move towards a target cabinet and open the target drawer out. **Randomizations:** - Robot is randomly initialized 1.6 to 1.8 meters away from the cabinet and positioned to face it - Robot's base orientation is randomized by -9 to 9 degrees - The cabinet selected to manipulate is randomly sampled from all PartnetMobility cabinets that have drawers - The drawer to open is randomly sampled from all drawers available to open **Success Conditions:** - The drawer is open at least 90% of the way, and the angular/linear velocities of the drawer link are small **Goal Specification:** - 3D goal position centered at the center of mass of the handle mesh on the drawer to open (also visualized in human renders with a sphere). .. py:method:: _after_control_step() Code that runs after each action has been taken. On GPU simulation this is called right before observations are fetched from the GPU buffers. .. py:method:: _after_reconfigure(options) Add code here that should run immediately after self._reconfigure is called. The torch RNG context is still active so RNG is still seeded here by self._episode_seed. This is useful if you need to run something that only happens after reconfiguration but need the GPU initialized so that you can check e.g. collisons, poses etc. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_cabinets(joint_types) .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: handle_link_positions(env_idx = None) .. py:attribute:: SUPPORTED_ROBOTS :value: ['fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: TRAIN_JSON .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/OpenCabinetDrawer-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Fetch] .. py:attribute:: all_model_ids .. py:attribute:: handle_types :value: ['prismatic'] .. py:attribute:: min_open_frac :value: 0.75 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: PegInsertionSideEnv(*args, robot_uids='panda_wristcam', num_envs=1, reconfiguration_freq=None, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Pick up a orange-white peg and insert the orange end into the box with a hole in it. **Randomizations:** - Peg half length is randomized between 0.085 and 0.125 meters. Box half length is the same value. (during reconfiguration) - Peg radius/half-width is randomized between 0.015 and 0.025 meters. Box hole's radius is same value + 0.003m of clearance. (during reconfiguration) - Peg is laid flat on table and has it's xy position and z-axis rotation randomized - Box is laid flat on table and has it's xy position and z-axis rotation randomized **Success Conditions:** - The white end of the peg is within 0.015m of the center of the box (inserted mid way). .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: has_peg_inserted() .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda_wristcam'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: _clearance :value: 0.003 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PegInsertionSide-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.panda.PandaWristCam] .. py:property:: box_hole_pose .. py:property:: goal_pose .. py:property:: peg_head_pos .. py:property:: peg_head_pose .. py:class:: PickClutterYCBEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, num_envs=1, reconfiguration_freq=None, episode_json = None, **kwargs) Bases: :py:obj:`PickClutterEnv` Base environment picking items out of clutter type of tasks. Flexibly supports using different configurations and object datasets .. py:method:: _load_model(model_id) .. py:attribute:: DEFAULT_EPISODE_JSON :value: 'Instance of pathlib.Path/tasks/pick_clutter/ycb_train_5k.json.gz' .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PickClutterYCB-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:class:: PickCubeEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch', 'xarm6_robotiq', 'so100', 'widowxai'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PickCube-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch, mani_skill.agents.robots.XArm6Robotiq, mani_skill.agents.robots.SO100, mani_skill.agents.robots.WidowXAI] .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: cube_spawn_center :value: (0, 0) .. py:attribute:: cube_spawn_half_size :value: 0.05 .. py:attribute:: goal_thresh :value: 0.025 .. py:attribute:: human_cam_eye_pos :value: [0.6, 0.7, 0.6] .. py:attribute:: human_cam_target_pos :value: [0.0, 0.0, 0.35] .. py:attribute:: max_goal_height :value: 0.3 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: sensor_cam_eye_pos :value: [0.3, 0, 0.6] .. py:attribute:: sensor_cam_target_pos .. py:class:: PickSingleYCBEnv(*args, robot_uids='panda_wristcam', robot_init_qpos_noise=0.02, num_envs=1, reconfiguration_freq=None, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Pick up a random object sampled from the [YCB dataset](https://www.ycbbenchmarks.com/) and move it to a random goal position **Randomizations:** - the object's xy position is randomized on top of a table in the region [0.1, 0.1] x [-0.1, -0.1]. It is placed flat on the table - the object's z-axis rotation is randomized - the object geometry is randomized by randomly sampling any YCB object. (during reconfiguration) **Success Conditions:** - the object position is within goal_thresh (default 0.025) euclidean distance of the goal position - the robot is static (q velocity < 0.2) **Goal Specification:** - 3D goal position (also visualized in human renders) **Additional Notes** - On GPU simulation, in order to collect data from every possible object in the YCB database we recommend using at least 128 parallel environments or more, otherwise you will need to reconfigure in order to sample new objects. .. py:method:: _after_reconfigure(options) Add code here that should run immediately after self._reconfigure is called. The torch RNG context is still active so RNG is still seeded here by self._episode_seed. This is useful if you need to run something that only happens after reconfiguration but need the GPU initialized so that you can check e.g. collisons, poses etc. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'panda_wristcam', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PickSingleYCB-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.panda.panda.Panda, mani_skill.agents.robots.panda.panda_wristcam.PandaWristCam, mani_skill.agents.robots.fetch.fetch.Fetch] .. py:attribute:: all_model_ids .. py:attribute:: goal_thresh :value: 0.025 .. py:attribute:: model_id :value: None .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: PlaceSphereEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Place the sphere into the shallow bin. **Randomizations:** - The position of the bin and the sphere are randomized: The bin is initialized in [0, 0.1] x [-0.1, 0.1], and the sphere is initialized in [-0.1, -0.05] x [-0.1, 0.1] **Success Conditions:** - The sphere is placed on the top of the bin. The robot remains static and the gripper is not closed at the end state. .. py:method:: _build_bin(radius) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PlaceSphere-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: block_half_size .. py:attribute:: edge_block_half_size .. py:attribute:: inner_side_half_len :value: 0.02 .. py:attribute:: radius :value: 0.02 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: short_side_half_size :value: 0.0025 .. py:class:: PlugChargerEnv(*args, robot_uids='panda_wristcam', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** The robot must pick up one of the misplaced shapes on the board/kit and insert it into the correct empty slot. **Randomizations:** - The charger position is randomized on the XY plane on top of the table. The rotation is also randomized - The receptacle position is randomized on the XY plane and the rotation is also randomized. Note that the human render camera has its pose fixed relative to the receptacle. **Success Conditions:** - The charger is inserted into the receptacle .. py:method:: _build_charger(peg_size, base_size, gap) .. py:method:: _build_receptacle(peg_size, receptacle_size, gap) .. py:method:: _compute_distance() .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['none', 'sparse'] .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda_wristcam'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: _base_size :value: [0.02, 0.015, 0.012] .. py:attribute:: _clearance :value: 0.0005 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _peg_gap :value: 0.007 .. py:attribute:: _peg_size :value: [0.008, 0.00075, 0.0032] .. py:attribute:: _receptacle_size :value: [0.01, 0.05, 0.05] .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PlugCharger-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.PandaWristCam] .. py:property:: charger_base_pose .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: PokeCubeEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A simple task where the objective is to poke a red cube with a peg and push it to a target goal position. **Randomizations:** - the peg's xy position is randomized on top of a table in the region [0.1, 0.1] x [-0.1, -0.1]. It is placed flat along it's length on the table - the cube's x-coordinate is fixed to peg's x-coordinate + peg half-length (0.12) + 0.1 and y-coordinate is randomized in range [-0.1, 0.1]. It is placed flat on the table - the cube's z-axis rotation is randomized in range [-$\pi$/ 6, $\pi$ / 6] - the target goal region is marked by a red/white circular target. The position of the target is fixed to be the cube xy position + [0.05 + goal_radius, 0] **Success Conditions:** - the cube's xy position is within goal_radius (default 0.05) of the target's xy position by euclidean distance - the robot is static .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PokeCube-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: goal_radius :value: 0.05 .. py:attribute:: peg_half_length :value: 0.12 .. py:attribute:: peg_half_width :value: 0.025 .. py:property:: peg_head_pos .. py:property:: peg_head_pose .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: PullCubeEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A simple task where the objective is to pull a cube onto a target. **Randomizations:** - the cube's xy position is randomized on top of a table in the region [0.1, 0.1] x [-0.1, -0.1]. - the target goal region is marked by a red and white target. The position of the target is fixed to be the cube's xy position - [0.1 + goal_radius, 0] **Success Conditions:** - the cube's xy position is within goal_radius (default 0.1) of the target's xy position by euclidean distance. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PullCube-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: goal_radius :value: 0.1 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: PullCubeToolEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description** Given an L-shaped tool that is within the reach of the robot, leverage the tool to pull a cube that is out of it's reach **Randomizations** - The cube's position (x,y) is randomized on top of a table in the region "". It is placed flat on the table - The target goal region is the region on top of the table marked by "" **Success Conditions** - The cube's xy position is within the goal region of the arm's base (marked by reachability) .. py:method:: _build_l_shaped_tool(handle_length, hook_length, width, height) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Normalizes the dense reward by the maximum possible reward (success bonus) .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ('normalized_dense', 'dense', 'sparse', 'none') .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PullCubeTool-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: arm_reach :value: 0.35 .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: cube_size :value: 0.02 .. py:attribute:: goal_radius :value: 0.3 .. py:attribute:: handle_length :value: 0.2 .. py:attribute:: height :value: 0.05 .. py:attribute:: hook_length :value: 0.05 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: width :value: 0.05 .. py:class:: PushCubeEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A simple task where the objective is to push and move a cube to a goal region in front of it **Randomizations:** - the cube's xy position is randomized on top of a table in the region [0.1, 0.1] x [-0.1, -0.1]. It is placed flat on the table - the target goal region is marked by a red/white circular target. The position of the target is fixed to be the cube xy position + [0.1 + goal_radius, 0] **Success Conditions:** - the cube's xy position is within goal_radius (default 0.1) of the target's xy position by euclidean distance and the cube is still on the table. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PushCube-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: goal_radius :value: 0.1 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: PushTEnv(*args, robot_uids='panda_stick', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A simulated version of the real-world push-T task from Diffusion Policy: https://diffusion-policy.cs.columbia.edu/ In this task, the robot needs to: 1. Precisely push the T-shaped block into the target region, and 2. Move the end-effector to the end-zone which terminates the episode. [2 Not required for PushT-easy-v1] **Randomizations:** - 3D T block initial position on table [-1,1] x [-1,2] + T Goal initial position - 3D T block initial z rotation [0,2pi] **Success Conditions:** - The T block covers 90% of the 2D goal T's area .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: pseudo_render_intersection() 'pseudo render' algo for calculating the intersection made custom 'psuedo renderer' to compute intersection area all computation in parallel on cuda, zero explicit loops views blocks in 2d in the goal tee frame to see overlap .. py:method:: quat_to_z_euler(quats) .. py:method:: quat_to_zrot(quats) .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda_stick'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: T_dynamic_friction :value: 3 .. py:attribute:: T_mass :value: 0.8 .. py:attribute:: T_static_friction :value: 3 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/PushT-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.robots.PandaStick .. py:attribute:: ee_starting_pos2D .. py:attribute:: ee_starting_pos3D .. py:attribute:: goal_offset .. py:attribute:: goal_z_rot .. py:attribute:: intersection_thresh :value: 0.9 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: tee_spawnbox_xlength :value: 0.2 .. py:attribute:: tee_spawnbox_xoffset :value: -0.1 .. py:attribute:: tee_spawnbox_ylength :value: 0.3 .. py:attribute:: tee_spawnbox_yoffset :value: -0.1 .. py:class:: PutCarrotOnPlateInScene(**kwargs) Bases: :py:obj:`mani_skill.envs.tasks.digital_twins.bridge_dataset_eval.base_env.BaseBridgeEnv` Base Digital Twin environment for digital twins of the BridgeData v2 .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: get_language_instruction(**kwargs) .. py:attribute:: objects_excluded_from_greenscreening :value: ['bridge_carrot_generated_modified', 'bridge_plate_objaverse_larger'] object ids that should not be greenscreened .. py:attribute:: scene_setting :value: 'flat_table' .. py:class:: PutEggplantInBasketScene(**kwargs) Bases: :py:obj:`mani_skill.envs.tasks.digital_twins.bridge_dataset_eval.base_env.BaseBridgeEnv` Base Digital Twin environment for digital twins of the BridgeData v2 .. py:method:: _load_lighting(options) Loads lighting into the scene. Called by `self._reconfigure`. If not overriden will set some simple default lighting .. py:method:: evaluate(*args, **kwargs) Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: get_language_instruction(**kwargs) .. py:attribute:: objects_excluded_from_greenscreening :value: ['eggplant'] object ids that should not be greenscreened .. py:attribute:: scene_setting :value: 'sink' .. py:class:: PutSpoonOnTableClothInScene(**kwargs) Bases: :py:obj:`mani_skill.envs.tasks.digital_twins.bridge_dataset_eval.base_env.BaseBridgeEnv` Base Digital Twin environment for digital twins of the BridgeData v2 .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: get_language_instruction(**kwargs) .. py:attribute:: objects_excluded_from_greenscreening :value: ['table_cloth_generated_shorter', 'bridge_spoon_generated_modified'] object ids that should not be greenscreened .. py:class:: QuadrupedReachEnv(*args, robot_uids='anymal-c', **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _compute_undesired_contacts(threshold=1.0) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['anymal_c', 'unitree_go2_simplified_locomotion'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: _UNDESIRED_CONTACT_LINK_NAMES :type: list[str] :value: None .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: mani_skill.agents.robots.anymal.anymal_c.ANYmalC .. py:attribute:: default_qpos :type: torch.Tensor .. py:class:: QuadrupedSpinEnv(*args, robot_uids='anymal-c', **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _compute_undesired_contacts(threshold=1.0) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['anymal_c', 'unitree_go2_simplified_locomotion'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: _UNDESIRED_CONTACT_LINK_NAMES :type: list[str] :value: None .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: mani_skill.agents.robots.anymal.anymal_c.ANYmalC .. py:attribute:: default_qpos :type: torch.Tensor .. py:class:: RoboCasaKitchenEnv(*args, robot_uids='fetch', env_configuration='default', controller_configs=None, gripper_types='default', base_types='default', initialization_noise='default', use_camera_obs=True, use_object_obs=True, reward_scale=1.0, reward_shaping=False, placement_initializer=None, has_renderer=False, has_offscreen_renderer=True, render_camera='robot0_agentview_center', render_collision_mesh=False, render_visual_mesh=True, render_gpu_device_id=-1, control_freq=20, horizon=1000, ignore_done=False, camera_names='agentview', camera_heights=256, camera_widths=256, camera_depths=False, renderer='mujoco', renderer_config=None, init_robot_base_pos=None, seed=None, layout_and_style_ids=None, layout_ids=None, style_ids=None, scene_split=None, generative_textures=None, obj_registries=('objaverse', ), obj_instance_split=None, use_distractors=False, translucent_robot=False, randomize_cameras=False, fixtures_only=False, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: _setup_kitchen_references() setup fixtures (and their references). this function is called within load_model function for kitchens .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: register_fixture_ref(ref_name, fn_kwargs) Registers a fixture reference for later use. Initializes the fixture if it has not been initialized yet. :param ref_name: name of the reference :type ref_name: str :param fn_kwargs: keyword arguments to pass to get_fixture :type fn_kwargs: dict :returns: fixture object :rtype: Fixture .. py:method:: sample_object(groups, exclude_groups=None, graspable=None, microwavable=None, washable=None, cookable=None, freezable=None, split=None, obj_registries=None, max_size=(None, None, None), object_scale=None, rng=None) Sample a kitchen object from the specified groups and within max_size bounds. :param groups: groups to sample from or the exact xml path of the object to spawn :type groups: list or str :param exclude_groups: groups to exclude :type exclude_groups: str or list :param graspable: whether the sampled object must be graspable :type graspable: bool :param washable: whether the sampled object must be washable :type washable: bool :param microwavable: whether the sampled object must be microwavable :type microwavable: bool :param cookable: whether whether the sampled object must be cookable :type cookable: bool :param freezable: whether whether the sampled object must be freezable :type freezable: bool :param split: split to sample from. Split "A" specifies all but the last 3 object instances (or the first half - whichever is larger), "B" specifies the rest, and None specifies all. :type split: str :param obj_registries: registries to sample from :type obj_registries: tuple :param max_size: max size of the object. If the sampled object is not within bounds of max size, function will resample :type max_size: tuple :param object_scale: scale of the object. If set will multiply the scale of the sampled object by this value :type object_scale: float :returns: kwargs to apply to the MJCF model for the sampled object dict: info about the sampled object - the path of the mjcf, groups which the object's category belongs to, the category of the object the sampling split the object came from, and the groups the object was sampled from :rtype: dict .. py:attribute:: EXCLUDE_LAYOUTS :value: [] .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['none'] Initialized a Base Kitchen environment. :param robots: Specification for specific robot arm(s) to be instantiated within this env (e.g: "Sawyer" would generate one arm; ["Panda", "Panda", "Sawyer"] would generate three robot arms) :param env_configuration: Specifies how to position the robot(s) within the environment. Default is "default", which should be interpreted accordingly by any subclasses. :type env_configuration: str :param controller_configs: If set, contains relevant controller parameters for creating a custom controller. Else, uses the default controller for this specific task. Should either be single dict if same controller is to be used for all robots or else it should be a list of the same length as "robots" param :type controller_configs: str or list of dict :param base_types: type of base, used to instantiate base models from base factory. Default is "default", which is the default base associated with the robot(s) the 'robots' specification. None results in no base, and any other (valid) model overrides the default base. Should either be single str if same base type is to be used for all robots or else it should be a list of the same length as "robots" param :type base_types: None or str or list of str :param gripper_types: type of gripper, used to instantiate gripper models from gripper factory. Default is "default", which is the default grippers(s) associated with the robot(s) the 'robots' specification. None removes the gripper, and any other (valid) model overrides the default gripper. Should either be single str if same gripper type is to be used for all robots or else it should be a list of the same length as "robots" param :type gripper_types: None or str or list of str :param initialization_noise: dict containing the initialization noise parameters. The expected keys and corresponding value types are specified below: :`'magnitude'`: The scale factor of uni-variate random noise applied to each of a robot's given initial joint positions. Setting this value to `None` or 0.0 results in no noise being applied. If "gaussian" type of noise is applied then this magnitude scales the standard deviation applied, If "uniform" type of noise is applied then this magnitude sets the bounds of the sampling range :`'type'`: Type of noise to apply. Can either specify "gaussian" or "uniform" Should either be single dict if same noise value is to be used for all robots or else it should be a list of the same length as "robots" param :Note: Specifying "default" will automatically use the default noise settings. Specifying None will automatically create the required dict with "magnitude" set to 0.0. :type initialization_noise: dict or list of dict :param use_camera_obs: if True, every observation includes rendered image(s) :type use_camera_obs: bool :param placement_initializer: if provided, will be used to place objects on every reset, else a UniformRandomSampler is used by default. :type placement_initializer: ObjectPositionSampler :param has_renderer: If true, render the simulation state in a viewer instead of headless mode. :type has_renderer: bool :param has_offscreen_renderer: True if using off-screen rendering :type has_offscreen_renderer: bool :param render_camera: Name of camera to render if `has_renderer` is True. Setting this value to 'None' will result in the default angle being applied, which is useful as it can be dragged / panned by the user using the mouse :type render_camera: str :param render_collision_mesh: True if rendering collision meshes in camera. False otherwise. :type render_collision_mesh: bool :param render_visual_mesh: True if rendering visual meshes in camera. False otherwise. :type render_visual_mesh: bool :param render_gpu_device_id: corresponds to the GPU device id to use for offscreen rendering. Defaults to -1, in which case the device will be inferred from environment variables (GPUS or CUDA_VISIBLE_DEVICES). :type render_gpu_device_id: int :param control_freq: how many control signals to receive in every second. This sets the abase of simulation time that passes between every action input. :type control_freq: float :param horizon: Every episode lasts for exactly @horizon timesteps. :type horizon: int :param ignore_done: True if never terminating the environment (ignore @horizon). :type ignore_done: bool :param camera_names: name of camera to be rendered. Should either be single str if same name is to be used for all cameras' rendering or else it should be a list of cameras to render. :Note: At least one camera must be specified if @use_camera_obs is True. :Note: To render all robots' cameras of a certain type (e.g.: "robotview" or "eye_in_hand"), use the convention "all-{name}" (e.g.: "all-robotview") to automatically render all camera images from each robot's camera list). :type camera_names: str or list of str :param camera_heights: height of camera frame. Should either be single int if same height is to be used for all cameras' frames or else it should be a list of the same length as "camera names" param. :type camera_heights: int or list of int :param camera_widths: width of camera frame. Should either be single int if same width is to be used for all cameras' frames or else it should be a list of the same length as "camera names" param. :type camera_widths: int or list of int :param camera_depths: True if rendering RGB-D, and RGB otherwise. Should either be single bool if same depth setting is to be used for all cameras or else it should be a list of the same length as "camera names" param. :type camera_depths: bool or list of bool :param renderer: Specifies which renderer to use. :type renderer: str :param renderer_config: dictionary for the renderer configurations :type renderer_config: dict :param init_robot_base_pos: name of the fixture to place the near. If None, will randomly select a fixture. :type init_robot_base_pos: str :param seed: environment seed. Default is None, where environment is unseeded, ie. random :type seed: int :param layout_and_style_ids: list of layout and style ids to use for the kitchen. :type layout_and_style_ids: list of list of int :param layout_ids: layout id(s) to use for the kitchen. -1 and None specify all layouts -2 specifies layouts not involving islands/wall stacks, -3 specifies layouts involving islands/wall stacks, -4 specifies layouts with dining areas. :type layout_ids: (list of) LayoutType or int :param style_ids: style id(s) to use for the kitchen. -1 and None specify all styles. :type style_ids: (list of) StyleType or int :param generative_textures: if set to "100p", will use AI generated textures :type generative_textures: str :param obj_registries: tuple containing the object registries to use for sampling objects. can contain "objaverse" and/or "aigen" to sample objects from objaverse, AI generated, or both. :type obj_registries: tuple of str :param obj_instance_split: string for specifying a custom set of object instances to use. "A" specifies all but the last 3 object instances (or the first half - whichever is larger), "B" specifies the rest, and None specifies all. :type obj_instance_split: str :param use_distractors: if True, will add distractor objects to the scene :type use_distractors: bool :param translucent_robot: if True, will make the robot appear translucent during rendering :type translucent_robot: bool :param randomize_cameras: if True, will add gaussian noise to the position and rotation of the wrist and agentview cameras :type randomize_cameras: bool .. py:attribute:: SUPPORTED_ROBOTS :value: ['fetch', 'none'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:property:: _default_viewer_camera_config .. py:attribute:: _ep_meta .. py:attribute:: fixtures_only :value: False .. py:attribute:: generative_textures :value: None .. py:attribute:: init_robot_base_pos :value: None .. py:attribute:: layout_and_style_ids .. py:attribute:: obj_instance_split :value: None .. py:attribute:: obj_registries :value: ('objaverse',) .. py:attribute:: placement_initializer :value: None .. py:attribute:: randomize_cameras :value: False .. py:attribute:: reward_scale :value: 1.0 .. py:attribute:: reward_shaping :value: False .. py:attribute:: translucent_robot :value: False .. py:attribute:: use_distractors :value: False .. py:attribute:: use_object_obs :value: True .. py:class:: RollBallEnv(*args, robot_uids='panda', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A simple task where the objective is to push and roll a ball to a goal region at the other end of the table **Randomizations:** - The ball's xy position is randomized on top of a table in the region [0.2, 0.5] x [-0.4, 0.7]. It is placed flat on the table - The target goal region is marked by a red/white circular target. The position of the target is randomized on top of a table in the region [-0.4, -0.7] x [0.2, -0.9] **Success Conditions:** - The ball's xy position is within goal_radius (default 0.1) of the target's xy position by euclidean distance. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/RollBall-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.robots.Panda .. py:attribute:: ball_radius :type: float :value: 0.035 .. py:attribute:: goal_radius :type: float :value: 0.1 .. py:attribute:: reached_status :type: torch.Tensor .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: RotateCubeEnv(*args, robot_uids='trifingerpro', robot_init_qpos_noise=0.02, difficulty_level = 4, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Modified from https://github.com/NVIDIA-Omniverse/IsaacGymEnvs/blob/main/isaacgymenvs/tasks/trifinger.py https://github.com/NVIDIA-Omniverse/IsaacGymEnvs/blob/main/isaacgymenvs/cfg/task/Trifinger.yaml .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_actors(env_idx) .. py:method:: _initialize_agent(env_idx) .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: _sample_object_goal_poses(env_idx, difficulty) Sample goal poses for the cube and sets them into the desired goal pose buffer. :param instances: A tensor constraining indices of environment instances to reset. :param difficulty: Difficulty level. The higher, the more difficult is the goal. Possible levels are: - 0: Random goal position on the table, no orientation. - 1: Random goal position on the table, including yaw orientation. - 2: Fixed goal position in the air with x,y = 0. No orientation. - 3: Random goal position in the air, no orientation. - 4: Random goal pose in the air, including orientation. .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: ARENA_RADIUS :value: 0.195 .. py:attribute:: SUPPORTED_ROBOTS :value: ['trifingerpro'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: agent :type: mani_skill.agents.robots.TriFingerPro .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: difficulty_level :value: 4 .. py:attribute:: goal_radius :value: 0.02 .. py:attribute:: max_com_distance_to_center .. py:attribute:: max_height :value: 0.1 .. py:attribute:: max_len :value: 0.065 .. py:attribute:: min_height :value: 0.0325 .. py:attribute:: radius_3d .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: sim_config the final sim config after merging user overrides with the environment default .. py:attribute:: size :value: 0.065 .. py:class:: RotateSingleObjectInHand(*args, robot_init_qpos_noise=0.02, obj_init_pos_noise=0.02, difficulty_level = -1, num_envs=1, reconfiguration_freq=None, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _after_reconfigure(options) Add code here that should run immediately after self._reconfigure is called. The torch RNG context is still active so RNG is still seeded here by self._episode_seed. This is useful if you need to run something that only happens after reconfiguration but need the GPU initialized so that you can check e.g. collisons, poses etc. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_actors(env_idx) .. py:method:: _initialize_agent(env_idx) .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate(**kwargs) Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: _clearance :value: 0.003 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: Union[mani_skill.agents.robots.AllegroHandRightTouch] .. py:attribute:: difficulty_level :value: -1 .. py:attribute:: hand_init_height :value: 0.25 .. py:attribute:: obj_heights :type: torch.Tensor .. py:attribute:: obj_init_pos_noise :value: 0.02 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: RotateValveEnv(*args, robot_init_qpos_noise=0.02, valve_init_pos_noise=0.02, difficulty_level = -1, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_actors(env_idx) .. py:method:: _initialize_agent(env_idx) .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_articulations() .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate(**kwargs) Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: _clearance :value: 0.003 .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: agent :type: Union[mani_skill.agents.robots.DClaw] .. py:attribute:: capsule_offset :value: 0.01 .. py:attribute:: difficulty_level :value: -1 .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: valve_init_pos_noise :value: 0.02 .. py:class:: SO100GraspCubeEnv(*args, robot_uids='so100', control_mode='pd_joint_target_delta_pos', greenscreen_overlay_path=None, domain_randomization_config = SO100GraspCubeDomainRandomizationConfig(), domain_randomization=True, base_camera_settings=dict(fov=52 * np.pi / 180, pos=[0.5, 0.3, 0.35], target=[0.3, 0.0, 0.1]), spawn_box_pos=[0.3, 0.05], spawn_box_half_size=0.2 / 2, **kwargs) Bases: :py:obj:`mani_skill.envs.tasks.digital_twins.base_env.BaseDigitalTwinEnv` **Task Description:** A simple task where the objective is to grasp a cube with the SO100 arm and bring it up to a target rest pose. **Randomizations:** - the cube's xy position is randomized on top of a table in a region of size [0.2, 0.2] x [-0.2, -0.2]. It is placed flat on the table - the cube's z-axis rotation is randomized to a random angle **Success Conditions:** - the cube is lifted, grasped, and the robot returns to a rest pose above the surface of the table .. py:method:: _before_control_step() Code that runs before each action has been taken via env.step(action). On GPU simulation this is called before observations are fetched from the GPU buffers. .. py:method:: _get_obs_agent() Get observations about the agent's state. By default it is proprioceptive observations which include qpos and qvel. Controller state is also included although most default controllers do not have any state. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_lighting(options) Loads lighting into the scene. Called by `self._reconfigure`. If not overriden will set some simple default lighting .. py:method:: _load_scene(options) Load assets for a digital twin scene in .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: sample_camera_poses(n) .. py:attribute:: SUPPORTED_OBS_MODES :value: ['none', 'state', 'state_dict', 'rgb+segmentation'] The string observation modes the environment supports. Note that "none" and "any_texture" are special keys. none indicates no observation data is generated. "any_texture" indicates that any combination of image textures generated by cameras are supported e.g. rgb+depth, normal+segmentation, albedo+rgb+depth etc. For a full list of supported textures see .. py:attribute:: SUPPORTED_ROBOTS :value: ['so100'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: mani_skill.agents.robots.so100.so_100.SO100 .. py:attribute:: base_camera_settings what the camera fov, position and target are when domain randomization is off. DR is centered around these settings .. py:attribute:: domain_randomization :value: True whether randomization is turned on or off. .. py:attribute:: domain_randomization_config domain randomization config .. py:attribute:: spawn_box_half_size :value: 0.1 .. py:attribute:: spawn_box_pos :value: [0.3, 0.05] .. py:class:: SceneManipulationEnv(*args, robot_uids='fetch', scene_builder_cls = 'ReplicaCAD', build_config_idxs=None, init_config_idxs=None, num_envs=1, reconfiguration_freq=None, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` A base environment for simulating manipulation tasks in more complex scenes. Creating this base environment is only useful for explorations/visualization, there are no success/failure metrics or rewards. :param robot_uids: Which robot to place into the scene. Default is "fetch" :param fixed_scene: When True, will never reconfigure the environment during resets unless you run env.reset(seed=seed, options=dict(reconfigure=True)) and explicitly reconfigure. If False, will reconfigure every reset. :param scene_builder_cls: Scene builder class to build a scene with. Default is ReplicaCAD. Furthermore, any of the AI2THOR SceneBuilders are supported in this environment. :param build_config_idxs: which build configs (static builds) to sample. Your scene_builder_cls may or may not require these. :type build_config_idxs: optional :param init_config_idxs: which init configs (additional init options) to sample. Your scene_builder_cls may or may not require these. :type init_config_idxs: optional .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_lighting(options) Loads lighting into the scene. Called by `self._reconfigure`. If not overriden will set some simple default lighting .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: reset(seed=None, options=None) Reset the ManiSkill environment with given seed(s) and options. Typically seed is either None (for unseeded reset) or an int (seeded reset). For GPU parallelized environments you can also pass a list of seeds for each parallel environment to seed each one separately. If options["env_idx"] is given, will only reset the selected parallel environments. If options["reconfigure"] is True, will call self._reconfigure() which deletes the entire physx scene and reconstructs everything. Users building custom tasks generally do not need to override this function. If options["reset_to_env_states"] is given, we expect there to be options["reset_to_env_states"]["env_states"] and optionally options["reset_to_env_states"]["obs"], both with batch size equal to the number of environments being reset. "env_states" can be a dictionary or flat tensor and we skip calling the environment's _initialize_episode function which generates the initial state on a normal reset. If "obs" is given we skip calling the environment's get_obs function which can save some compute/time. Returns the observations and an info dictionary. The info dictionary is of type .. highlight:: python .. code-block:: python { "reconfigure": bool # (True if the env reconfigured. False otherwise) } Note that ManiSkill always holds two RNG states, a main RNG, and an episode RNG. The main RNG is used purely to sample an episode seed which helps with reproducibility of episodes and is for internal use only. The episode RNG is used by the environment/task itself to e.g. randomize object positions, randomize assets etc. Episode RNG is accessible by using `self._batched_episode_rng` which is numpy based and `torch.rand` which can be used to generate random data on the GPU directly and is seeded. Note that it is recommended to use `self._batched_episode_rng` if you need to ensure during reconfiguration the same objects are loaded. Reproducibility and seeding when there is GPU and CPU simulation can be tricky and we recommend reading the documentation for more recommendations and details on RNG https://maniskill.readthedocs.io/en/latest/user_guide/concepts/rng.html Upon environment creation via gym.make, the main RNG is set with fixed seeds of 2022 to 2022 + num_envs - 1 (seed is just 2022 if there is only one environment) During each reset call, if seed is None, main RNG is unchanged and an episode seed is sampled from the main RNG to create the episode RNG. If seed is not None, main RNG is set to that seed and the episode seed is also set to that seed. This design means the main RNG determines the episode RNG deterministically. .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: build_config_idxs :value: None .. py:attribute:: init_config_idxs :value: None .. py:attribute:: scene_builder :type: mani_skill.utils.scene_builder.SceneBuilder .. py:class:: StackCubeEnv(*args, robot_uids='panda_wristcam', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** The goal is to pick up a red cube and stack it on top of a green cube and let go of the cube without it falling **Randomizations:** - both cubes have their z-axis rotation randomized - both cubes have their xy positions on top of the table scene randomized. The positions are sampled such that the cubes do not collide with each other **Success Conditions:** - the red cube is on top of the green cube (to within half of the cube size) - the red cube is static - the red cube is not being grasped by the robot (robot must let go of the cube) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda_wristcam', 'panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/StackCube-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: StackGreenCubeOnYellowCubeBakedTexInScene(**kwargs) Bases: :py:obj:`mani_skill.envs.tasks.digital_twins.bridge_dataset_eval.base_env.BaseBridgeEnv` Base Digital Twin environment for digital twins of the BridgeData v2 .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:method:: get_language_instruction(**kwargs) .. py:attribute:: MODEL_JSON :value: 'info_bridge_custom_baked_tex_v0.json' .. py:attribute:: objects_excluded_from_greenscreening :value: ['baked_green_cube_3cm', 'baked_yellow_cube_3cm'] object ids that should not be greenscreened .. py:class:: StackPyramidEnv(*args, robot_uids='panda_wristcam', robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** - The goal is to pick up a red cube, place it next to the green cube, and stack the blue cube on top of the red and green cube without it falling off. **Randomizations:** - all cubes have their z-axis rotation randomized - all cubes have their xy positions on top of the table scene randomized. The positions are sampled such that the cubes do not collide with each other **Success Conditions:** - the blue cube is static - the blue cube is on top of both the red and green cube (to within half of the cube size) - none of the red, green, blue cubes are grasped by the robot (robot must let go of the cubes) _sample_video_link = "https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/StackPyramid-v1_rt.mp4" .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['none', 'sparse'] .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda_wristcam', 'panda', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: TableTopFreeDrawEnv(*args, robot_uids='panda_stick', **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** Instantiates a table with a white canvas on it and a robot with a stick that draws red lines. This environment is primarily for a reference / for others to copy to make their own drawing tasks. **Randomizations:** None **Success Conditions:** None **Goal Specification:** None .. py:method:: _after_control_step() Code that runs after each action has been taken. On GPU simulation this is called right before observations are fetched from the GPU buffers. .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: BRUSH_COLORS :value: [[0.8, 0.2, 0.2, 1]] The colors of the brushes. If there is more than one color, each parallel environment will have a randomly sampled color. .. py:attribute:: BRUSH_RADIUS :value: 0.01 The brushes radius .. py:attribute:: CANVAS_THICKNESS :value: 0.02 How thick the canvas on the table is .. py:attribute:: DOT_THICKNESS :value: 0.003 thickness of the paint drawn on to the canvas .. py:attribute:: MAX_DOTS :value: 1010 The total "ink" available to use and draw with before you need to call env.reset. NOTE that on GPU simulation it is not recommended to have a very high value for this as it can slow down rendering when too many objects are being rendered in many scenes. .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['none'] .. py:attribute:: SUPPORTED_ROBOTS :type: [panda_stick] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/TableTopFreeDraw-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.robots.panda.panda_stick.PandaStick .. py:class:: TransportBoxEnv(*args, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A G1 humanoid robot must find a box on a table and transport it to the other table and place it there. **Randomizations:** - the box's xy position is randomized in the region [-0.05, -0.05] x [0.2, 0.05] - the box's z-axis rotation is randomized to a random angle in [0, np.pi/6] **Success Conditions:** - the box is resting on top of the other table .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: ['unitree_g1_simplified_upper_body_with_head_camera'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/UnitreeG1TransportBox-v... a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.robots.unitree_g1.g1_upper_body.UnitreeG1UpperBodyWithHeadCamera .. py:property:: box_left_grasp_point .. py:property:: box_right_grasp_point .. py:attribute:: init_robot_pose .. py:attribute:: init_robot_qpos .. py:class:: TurnFaucetEnv(*args, robot_uids='panda_wristcam', robot_init_qpos_noise=0.02, reconfiguration_freq=None, num_envs=1, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` Superclass for ManiSkill environments. :param num_envs: number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs. :param obs_mode: observation mode to be used. Must be one of ("state", "state_dict", "none", "sensor_data", "rgb", "depth", "segmentation", "rgbd", "rgb+depth", "rgb+depth+segmentation", "rgb+segmentation", "depth+segmentation", "pointcloud") The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera) :param reward_mode: reward mode to use. Must be one of ("normalized_dense", "dense", "sparse", "none"). With "none" the reward returned is always 0 :param control_mode: control mode of the agent. "*" represents all registered controllers, and the action space will be a dict. :param render_mode: render mode registered in @SUPPORTED_RENDER_MODES. :param shader_dir: shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders. Previously the options are "default", "rt", "rt-fast". "rt" means ray-tracing which results in more photorealistic renders but is slow, "rt-fast" is a lower quality but faster version of "rt". :type shader_dir: Optional[str] :param enable_shadow: whether to enable shadow for lights. Defaults to False. :type enable_shadow: bool :param sensor_configs: configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see :doc:`the sensors documentation `. :type sensor_configs: dict :param human_render_camera_configs: configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs. :type human_render_camera_configs: dict :param viewer_camera_configs: configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs. :type viewer_camera_configs: dict :param robot_uids: list of robots to instantiate and control in the environment. :type robot_uids: Union[str, BaseAgent, list[Union[str, BaseAgent]]] :param sim_config: Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. ``sim_config=dict(scene_config=dict(solver_iterations=25))``. If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the ``self._default_sim_config`` attribute that generally should not be completely overriden. :type sim_config: Union[SimConfig, dict] :param reconfiguration_freq: How frequently to call reconfigure when environment is reset via `self.reset(...)` Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again. :type reconfiguration_freq: int :param sim_backend: By default this is "auto". If sim_backend is "auto", then if ``num_envs == 1``, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be "physx_cpu" or "physx_cuda" to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass "physx_cuda:n" where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is "physx_cpu", num_envs can only be equal to 1. :type sim_backend: str :param render_backend: By default this is "gpu". If render_backend is "gpu" or it's alias "sapien_cuda", then we auto select a GPU to render with. It can be "sapien_cuda:n" where n is the ID of the GPU to render with. If this is "cpu" or "sapien_cpu", then we try to render on the CPU. If this is "none" or None, then we disable rendering. Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don't need to do rendering you can simply disable it by setting render_backend to "none" or None. :type render_backend: str :param parallel_in_single_scene: By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering. :type parallel_in_single_scene: bool :param enhanced_determinism: By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead. :type enhanced_determinism: bool .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_REWARD_MODES :value: ['sparse', 'none'] .. py:attribute:: SUPPORTED_ROBOTS :value: ['panda', 'panda_wristcam', 'fetch'] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:attribute:: TRAIN_JSON .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: agent :type: Union[mani_skill.agents.robots.Panda, mani_skill.agents.robots.Fetch] .. py:attribute:: all_model_ids .. py:property:: current_angle .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:attribute:: train_info .. py:class:: TwoRobotPickCube(*args, robot_uids=('panda_wristcam', 'panda_wristcam'), robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** The goal is to pick up a red cube and lift it to a goal location. There are two robots in this task and the goal location is out of reach of the left robot while the cube is out of reach of the right robot, thus the two robots must work together to move the cube to the goal. **Randomizations:** - cube has its z-axis rotation randomized - cube has its xy positions on top of the table scene randomized such that it is in within reach of the left robot but not the right. - the target goal position (marked by a green sphere) of the cube is randomized such that it is within reach of the right robot but not the left. **Success Conditions:** - red cube is at the goal location .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: [('panda_wristcam', 'panda_wristcam')] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/refs/heads/main/figures/environment_demos/TwoRobotPick... a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.multi_agent.MultiAgent[Tuple[mani_skill.agents.robots.panda.Panda, mani_skill.agents.robots.panda.Panda]] .. py:attribute:: cube_half_size :value: 0.02 .. py:attribute:: goal_thresh :value: 0.025 .. py:property:: left_agent :type: mani_skill.agents.robots.panda.Panda .. py:property:: right_agent :type: mani_skill.agents.robots.panda.Panda .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:class:: TwoRobotStackCube(*args, robot_uids=('panda_wristcam', 'panda_wristcam'), robot_init_qpos_noise=0.02, **kwargs) Bases: :py:obj:`mani_skill.envs.sapien_env.BaseEnv` **Task Description:** A collaborative task where two robot arms need to work together to stack two cubes. One robot must pick up the green cube and place it on the target region, while the other robot picks up the blue cube and stacks it on top of the green cube. The cubes are initially positioned such that each robot can only reach one cube - the green cube is near the right robot and the blue cube is near the left robot. This requires coordination between the robots to complete the stacking task. **Randomizations:** - Both cubes have random rotations around their z-axis - The xy positions of both cubes on the table are randomized, while ensuring: - The cubes do not collide with each other - The green cube remains reachable by the right robot - The blue cube remains reachable by the left robot - The goal region is placed along the midline between the robots (y=0), with randomized x position **Success Conditions:** - The blue cube is stacked on top of the green cube (within half a cube size) - The green cube is placed on the red/white target region - Both cubes are released by the robots (not being grasped) .. py:method:: _get_obs_extra(info) Get task-relevant extra observations. Usually defined on a task by task basis .. py:method:: _initialize_episode(env_idx, options) Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions .. py:method:: _load_agent(options) loads the agent/controllable articulations into the environment. The default function provides a convenient way to setup the agent/robot by a robot_uid (stored in self.robot_uids) without requiring the user to have to write the robot building and controller code themselves. For more advanced use-cases you can override this function to have more control over the agent/robot building process. :param options: The options for the environment. :type options: dict :param initial_agent_poses: The initial poses of the agent/robot. Providing these poses and ensuring they are picked such that they do not collide with objects if spawned there is highly recommended to ensure more stable simulation (the agent pose can be changed later during episode initialization). :type initial_agent_poses: Optional[Union[sapien.Pose, Pose]] :param build_separate: Whether to build the agent/robot separately. If True, the agent/robot will be built separately for each parallel environment and then merged together to be accessible under one view/object. This is useful for randomizing physical and visual properties of the agent/robot which is only permitted for articulations built separately in each environment. :type build_separate: bool .. py:method:: _load_scene(options) Loads all objects like actors and articulations into the scene. Called by `self._reconfigure`. Given options argument is the same options dictionary passed to the self.reset function .. py:method:: compute_dense_reward(obs, action, info) Compute the dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: compute_normalized_dense_reward(obs, action, info) Compute the normalized dense reward. :param obs: The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.) :type obs: Any :param action: The most recent action. :type action: torch.Tensor :param info: The info dictionary. :type info: dict .. py:method:: evaluate() Evaluate whether the environment is currently in a success state by returning a dictionary with a "success" key or a failure state via a "fail" key This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards. By default if not overriden this function returns an empty dictionary .. py:attribute:: SUPPORTED_ROBOTS :value: [('panda_wristcam', 'panda_wristcam')] Override this to enforce which robots or tuples of robots together are supported in the task. During env creation, setting robot_uids auto loads all desired robots into the scene, but not all tasks are designed to support some robot setups .. py:property:: _default_human_render_camera_configs Add default cameras for rendering when using render_mode='rgb_array'. These can be overriden by the user at env creation time .. py:property:: _default_sensor_configs Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time .. py:property:: _default_sim_config .. py:attribute:: _sample_video_link :value: 'https://github.com/haosulab/ManiSkill/raw/main/figures/environment_demos/TwoRobotStackCube-v1_rt.mp4' a link to a sample video of the task. This is mostly used for automatic documentation generation .. py:attribute:: agent :type: mani_skill.agents.multi_agent.MultiAgent[Tuple[mani_skill.agents.robots.panda.Panda, mani_skill.agents.robots.panda.Panda]] .. py:attribute:: goal_radius :value: 0.06 .. py:property:: left_agent :type: mani_skill.agents.robots.panda.Panda .. py:property:: right_agent :type: mani_skill.agents.robots.panda.Panda .. py:attribute:: robot_init_qpos_noise :value: 0.02 .. py:function:: register_env(uid, max_episode_steps=None, override=False, asset_download_ids = [], **kwargs) A decorator to register ManiSkill environments. :param uid: unique id of the environment. :type uid: str :param max_episode_steps: maximum number of steps in an episode. :type max_episode_steps: int :param asset_download_ids: asset download ids the environment depends on. When environments are created this list is checked to see if the user has all assets downloaded and if not, prompt the user if they wish to download them. :type asset_download_ids: list[str] :param override: whether to override the environment if it is already registered. :type override: bool .. rubric:: Notes - `max_episode_steps` is processed differently from other keyword arguments in gym. `gym.make` wraps the env with `gym.wrappers.TimeLimit` to limit the maximum number of steps. - `gym.EnvSpec` uses kwargs instead of **kwargs!