mani_skill.envs.tasks.dexterity

Submodules

• mani_skill.envs.tasks.dexterity.insert_flower
• mani_skill.envs.tasks.dexterity.rotate_single_object_in_hand
• mani_skill.envs.tasks.dexterity.rotate_valve

Classes

InsertFlowerEnv	Superclass for ManiSkill environments.
RotateSingleObjectInHand	Superclass for ManiSkill environments.
RotateValveEnv	Superclass for ManiSkill environments.

Package Contents

class mani_skill.envs.tasks.dexterity.InsertFlowerEnv(*args, robot_uids='floating_ability_hand_right', robot_init_qpos_noise=0.02, num_envs=1, reconfiguration_freq=None, **kwargs)

Bases: mani_skill.envs.sapien_env.BaseEnv

Superclass for ManiSkill environments.

Parameters:

• num_envs – number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs.

• obs_mode – observation mode to be used. Must be one of (“state”, “state_dict”, “none”, “sensor_data”, “rgb”, “depth”, “segmentation”, “rgbd”, “rgb+depth”, “rgb+depth+segmentation”, “rgb+segmentation”, “depth+segmentation”, “pointcloud”) The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera)

• reward_mode – reward mode to use. Must be one of (“normalized_dense”, “dense”, “sparse”, “none”). With “none” the reward returned is always 0

• control_mode – control mode of the agent. “*” represents all registered controllers, and the action space will be a dict.

• render_mode – render mode registered in @SUPPORTED_RENDER_MODES.

• shader_dir (Optional[str]) –

  shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders.

  Previously the options are “default”, “rt”, “rt-fast”. “rt” means ray-tracing which results in more photorealistic renders but is slow, “rt-fast” is a lower quality but faster version of “rt”.

• enable_shadow (bool) – whether to enable shadow for lights. Defaults to False.

• sensor_configs (dict) – configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see the sensors documentation.

• human_render_camera_configs (dict) – configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs.

• viewer_camera_configs (dict) – configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs.

• robot_uids (Union[str, BaseAgent, list[Union[str, BaseAgent]]]) – list of robots to instantiate and control in the environment.

• sim_config (Union[SimConfig, dict]) – Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. sim_config=dict(scene_config=dict(solver_iterations=25)). If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the self._default_sim_config attribute that generally should not be completely overriden.

• reconfiguration_freq (int) – How frequently to call reconfigure when environment is reset via self.reset(…) Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again.

• sim_backend (str) – By default this is “auto”. If sim_backend is “auto”, then if num_envs == 1, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be “physx_cpu” or “physx_cuda” to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass “physx_cuda:n” where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is “physx_cpu”, num_envs can only be equal to 1.

• render_backend (str) –

  By default this is “gpu”. If render_backend is “gpu” or it’s alias “sapien_cuda”, then we auto select a GPU to render with. It can be “sapien_cuda:n” where n is the ID of the GPU to render with. If this is “cpu” or “sapien_cpu”, then we try to render on the CPU. If this is “none” or None, then we disable rendering.

  Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don’t need to do rendering you can simply disable it by setting render_backend to “none” or None.

• parallel_in_single_scene (bool) – By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering.

• enhanced_determinism (bool) – By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead.

_after_reconfigure(options)

Add code here that should run immediately after self._reconfigure is called. The torch RNG context is still active so RNG is still seeded here by self._episode_seed. This is useful if you need to run something that only happens after reconfiguration but need the GPU initialized so that you can check e.g. collisons, poses etc.

Parameters:

options (dict) –

_get_obs_extra(info)

Get task-relevant extra observations. Usually defined on a task by task basis

Parameters:

info (dict) –

_initialize_actors(env_idx)

Parameters:

env_idx (torch.Tensor) –

_initialize_agent(env_idx)

Parameters:

env_idx (torch.Tensor) –

_initialize_episode(env_idx, options)

Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions

Parameters:

• env_idx (torch.Tensor) –

• options (dict) –

_load_scene(options)

Loads all objects like actors and articulations into the scene. Called by self._reconfigure. Given options argument is the same options dictionary passed to the self.reset function

Parameters:

options (dict) –

compute_dense_reward(obs, action, info)

Compute the dense reward.

Parameters:

• obs (Any) – The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.)

• action (torch.Tensor) – The most recent action.

• info (dict) – The info dictionary.

Return type:

float

compute_normalized_dense_reward(obs, action, info)

Compute the normalized dense reward.

Parameters:

• obs (Any) – The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.)

• action (torch.Tensor) – The most recent action.

• info (dict) – The info dictionary.

evaluate(**kwargs)

Evaluate whether the environment is currently in a success state by returning a dictionary with a “success” key or a failure state via a “fail” key

This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards.

By default if not overriden this function returns an empty dictionary

Return type:

dict

_clearance = 0.003

property _default_human_render_camera_configs

Add default cameras for rendering when using render_mode=’rgb_array’. These can be overriden by the user at env creation time

property _default_sensor_configs

Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time

property _default_sim_config

agent: mani_skill.agents.robots.FloatingAbilityHandRight

asset_path = 'Instance of pathlib.Path/tasks/oakink-v2/align_ds'

flower_spawn_half_size = 0.05

hand_init_height = 0.25

robot_init_qpos_noise = 0.02

target_area

class mani_skill.envs.tasks.dexterity.RotateSingleObjectInHand(*args, robot_init_qpos_noise=0.02, obj_init_pos_noise=0.02, difficulty_level=-1, num_envs=1, reconfiguration_freq=None, **kwargs)

Bases: mani_skill.envs.sapien_env.BaseEnv

Superclass for ManiSkill environments.

Parameters:

• num_envs – number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs.

• obs_mode – observation mode to be used. Must be one of (“state”, “state_dict”, “none”, “sensor_data”, “rgb”, “depth”, “segmentation”, “rgbd”, “rgb+depth”, “rgb+depth+segmentation”, “rgb+segmentation”, “depth+segmentation”, “pointcloud”) The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera)

• reward_mode – reward mode to use. Must be one of (“normalized_dense”, “dense”, “sparse”, “none”). With “none” the reward returned is always 0

• control_mode – control mode of the agent. “*” represents all registered controllers, and the action space will be a dict.

• render_mode – render mode registered in @SUPPORTED_RENDER_MODES.

• shader_dir (Optional[str]) –

  shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders.

  Previously the options are “default”, “rt”, “rt-fast”. “rt” means ray-tracing which results in more photorealistic renders but is slow, “rt-fast” is a lower quality but faster version of “rt”.

• enable_shadow (bool) – whether to enable shadow for lights. Defaults to False.

• sensor_configs (dict) – configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see the sensors documentation.

• human_render_camera_configs (dict) – configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs.

• viewer_camera_configs (dict) – configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs.

• robot_uids (Union[str, BaseAgent, list[Union[str, BaseAgent]]]) – list of robots to instantiate and control in the environment.

• sim_config (Union[SimConfig, dict]) – Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. sim_config=dict(scene_config=dict(solver_iterations=25)). If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the self._default_sim_config attribute that generally should not be completely overriden.

• reconfiguration_freq (int) – How frequently to call reconfigure when environment is reset via self.reset(…) Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again.

• sim_backend (str) – By default this is “auto”. If sim_backend is “auto”, then if num_envs == 1, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be “physx_cpu” or “physx_cuda” to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass “physx_cuda:n” where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is “physx_cpu”, num_envs can only be equal to 1.

• render_backend (str) –

  By default this is “gpu”. If render_backend is “gpu” or it’s alias “sapien_cuda”, then we auto select a GPU to render with. It can be “sapien_cuda:n” where n is the ID of the GPU to render with. If this is “cpu” or “sapien_cpu”, then we try to render on the CPU. If this is “none” or None, then we disable rendering.

  Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don’t need to do rendering you can simply disable it by setting render_backend to “none” or None.

• parallel_in_single_scene (bool) – By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering.

• enhanced_determinism (bool) – By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead.

• difficulty_level (int) –

_after_reconfigure(options)

Add code here that should run immediately after self._reconfigure is called. The torch RNG context is still active so RNG is still seeded here by self._episode_seed. This is useful if you need to run something that only happens after reconfiguration but need the GPU initialized so that you can check e.g. collisons, poses etc.

Parameters:

options (dict) –

_get_obs_extra(info)

Get task-relevant extra observations. Usually defined on a task by task basis

Parameters:

info (dict) –

_initialize_actors(env_idx)

Parameters:

env_idx (torch.Tensor) –

_initialize_agent(env_idx)

Parameters:

env_idx (torch.Tensor) –

_initialize_episode(env_idx, options)

Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions

Parameters:

• env_idx (torch.Tensor) –

• options (dict) –

_load_scene(options)

Loads all objects like actors and articulations into the scene. Called by self._reconfigure. Given options argument is the same options dictionary passed to the self.reset function

Parameters:

options (dict) –

compute_dense_reward(obs, action, info)

Compute the dense reward.

Parameters:

• obs (Any) – The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.)

• action (torch.Tensor) – The most recent action.

• info (dict) – The info dictionary.

compute_normalized_dense_reward(obs, action, info)

Compute the normalized dense reward.

Parameters:

• obs (Any) – The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.)

• action (torch.Tensor) – The most recent action.

• info (dict) – The info dictionary.

evaluate(**kwargs)

Evaluate whether the environment is currently in a success state by returning a dictionary with a “success” key or a failure state via a “fail” key

This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards.

By default if not overriden this function returns an empty dictionary

Return type:

dict

_clearance = 0.003

property _default_human_render_camera_configs

Add default cameras for rendering when using render_mode=’rgb_array’. These can be overriden by the user at env creation time

property _default_sensor_configs

Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time

property _default_sim_config

agent: mani_skill.agents.robots.AllegroHandRightTouch

difficulty_level = -1

hand_init_height = 0.25

obj_heights: torch.Tensor

obj_init_pos_noise = 0.02

robot_init_qpos_noise = 0.02

class mani_skill.envs.tasks.dexterity.RotateValveEnv(*args, robot_init_qpos_noise=0.02, valve_init_pos_noise=0.02, difficulty_level=-1, **kwargs)

Bases: mani_skill.envs.sapien_env.BaseEnv

Superclass for ManiSkill environments.

Parameters:

• num_envs – number of parallel environments to run. By default this is 1, which means a CPU simulation is used. If greater than 1, then we initialize the GPU simulation setup. Note that not all environments are faster when simulated on the GPU due to limitations of GPU simulations. For example, environments with many moving objects are better simulated by parallelizing across CPUs.

• obs_mode – observation mode to be used. Must be one of (“state”, “state_dict”, “none”, “sensor_data”, “rgb”, “depth”, “segmentation”, “rgbd”, “rgb+depth”, “rgb+depth+segmentation”, “rgb+segmentation”, “depth+segmentation”, “pointcloud”) The obs_mode is mostly for convenience to automatically optimize/setup all sensors/cameras for the given observation mode to render the correct data and try to ignore unnecessary rendering. For the most advanced use cases (e.g. you have 1 RGB only camera and 1 depth only camera)

• reward_mode – reward mode to use. Must be one of (“normalized_dense”, “dense”, “sparse”, “none”). With “none” the reward returned is always 0

• control_mode – control mode of the agent. “*” represents all registered controllers, and the action space will be a dict.

• render_mode – render mode registered in @SUPPORTED_RENDER_MODES.

• shader_dir (Optional[str]) –

  shader directory. Defaults to None. Setting this will override the shader used for all cameras in the environment. This is legacy behavior kept for backwards compatibility. The proper way to change the shaders used for cameras is to either change the environment code or pass in sensor_configs/human_render_camera_configs with the desired shaders.

  Previously the options are “default”, “rt”, “rt-fast”. “rt” means ray-tracing which results in more photorealistic renders but is slow, “rt-fast” is a lower quality but faster version of “rt”.

• enable_shadow (bool) – whether to enable shadow for lights. Defaults to False.

• sensor_configs (dict) – configurations of sensors to override any environment defaults. If the key is one of sensor names (e.g. a camera), the config value will be applied to the corresponding sensor. Otherwise, the value will be applied to all sensors (but overridden by sensor-specific values). For possible configurations see the documentation see the sensors documentation.

• human_render_camera_configs (dict) – configurations of human rendering cameras to override any environment defaults. Similar usage as @sensor_configs.

• viewer_camera_configs (dict) – configurations of the viewer camera in the GUI to override any environment defaults. Similar usage as @sensor_configs.

• robot_uids (Union[str, BaseAgent, list[Union[str, BaseAgent]]]) – list of robots to instantiate and control in the environment.

• sim_config (Union[SimConfig, dict]) – Configurations for simulation if used that override the environment defaults. If given a dictionary, it can just override specific attributes e.g. sim_config=dict(scene_config=dict(solver_iterations=25)). If passing in a SimConfig object, while typed, will override every attribute including the task defaults. Some environments define their own recommended default sim configurations via the self._default_sim_config attribute that generally should not be completely overriden.

• reconfiguration_freq (int) – How frequently to call reconfigure when environment is reset via self.reset(…) Generally for most users who are not building tasks this does not need to be changed. The default is 0, which means the environment reconfigures upon creation, and never again.

• sim_backend (str) – By default this is “auto”. If sim_backend is “auto”, then if num_envs == 1, we use the PhysX CPU sim backend, otherwise we use the PhysX GPU sim backend and automatically pick a GPU to use. Can also be “physx_cpu” or “physx_cuda” to force usage of a particular sim backend. To select a particular GPU to run the simulation on, you can pass “physx_cuda:n” where n is the ID of the GPU, similar to the way PyTorch selects GPUs. Note that if this is “physx_cpu”, num_envs can only be equal to 1.

• render_backend (str) –

  By default this is “gpu”. If render_backend is “gpu” or it’s alias “sapien_cuda”, then we auto select a GPU to render with. It can be “sapien_cuda:n” where n is the ID of the GPU to render with. If this is “cpu” or “sapien_cpu”, then we try to render on the CPU. If this is “none” or None, then we disable rendering.

  Note that some environments may require rendering functionalities to work. Moreover it is sometimes difficult to determine before running an environment if your machine can render or not. If you encounter some issue with rendering you can first try to double check your NVIDIA drivers / Vulkan drivers are setup correctly. If you don’t need to do rendering you can simply disable it by setting render_backend to “none” or None.

• parallel_in_single_scene (bool) – By default this is False. If True, rendered images and the GUI will show all objects in one view. This is only really useful for generating cool videos showing all environments at once but it is not recommended otherwise as it slows down simulation and rendering.

• enhanced_determinism (bool) – By default this is False and env resets will reset the episode RNG only when a seed / seed list is given. If True, the environment will reset the episode RNG upon each reset regardless of whether a seed is provided. Generally enhanced_determinisim is not needed and users are recommended to pass seeds into the env reset function instead.

• difficulty_level (int) –

_get_obs_extra(info)

Get task-relevant extra observations. Usually defined on a task by task basis

Parameters:

info (dict) –

_initialize_actors(env_idx)

Parameters:

env_idx (torch.Tensor) –

_initialize_agent(env_idx)

Parameters:

env_idx (torch.Tensor) –

_initialize_episode(env_idx, options)

Initialize the episode, e.g., poses of actors and articulations, as well as task relevant data like randomizing goal positions

Parameters:

• env_idx (torch.Tensor) –

• options (dict) –

_load_articulations()

_load_scene(options)

Loads all objects like actors and articulations into the scene. Called by self._reconfigure. Given options argument is the same options dictionary passed to the self.reset function

Parameters:

options (dict) –

compute_dense_reward(obs, action, info)

Compute the dense reward.

Parameters:

• obs (Any) – The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.)

• action (torch.Tensor) – The most recent action.

• info (dict) – The info dictionary.

compute_normalized_dense_reward(obs, action, info)

Compute the normalized dense reward.

Parameters:

• obs (Any) – The observation data. By default the observation data will be in its most raw form, a dictionary (no flattening, wrappers etc.)

• action (torch.Tensor) – The most recent action.

• info (dict) – The info dictionary.

evaluate(**kwargs)

Evaluate whether the environment is currently in a success state by returning a dictionary with a “success” key or a failure state via a “fail” key

This function may also return additional data that has been computed (e.g. is the robot grasping some object) that may be reused when generating observations and rewards.

By default if not overriden this function returns an empty dictionary

Return type:

dict

_clearance = 0.003

property _default_human_render_camera_configs

Add default cameras for rendering when using render_mode=’rgb_array’. These can be overriden by the user at env creation time

property _default_sensor_configs

Add default (non-agent) sensors to the environment by returning sensor configurations. These can be overriden by the user at env creation time

agent: mani_skill.agents.robots.DClaw

capsule_offset = 0.01

difficulty_level = -1

robot_init_qpos_noise = 0.02

valve_init_pos_noise = 0.02