mani_skill.envs.utils.observations#

Submodules#

Attributes#

ALL_VISUAL_TEXTURES

set of all standard textures that can come from cameras

Classes#

BaseSensor

Base class for all sensors

BaseSensorConfig

Camera

Implementation of the Camera sensor which uses the sapien Camera.

CameraObsTextures

ObservationModeStruct

A dataclass describing what observation data is being requested by the user

Functions#

parse_obs_mode_to_struct(obs_mode)

Given user supplied observation mode, return a struct with the relevant textures that are to be captured

sensor_data_to_pointcloud(observation, sensors)

convert all camera data in sensor to pointcloud data

Package Contents#

class mani_skill.envs.utils.observations.BaseSensor(config)[source]#

Base class for all sensors

Parameters:

config (BaseSensorConfig) –

capture()[source]#

Captures sensor data and prepares it for it to be then retrieved via get_obs for observations and get_image for a visualizable image.

Some sensors like rgbd cameras need to take a picture just once after each call to scene.update_render. Generally this should also be a non-blocking function if possible.

Return type:

None

abstract get_images()[source]#

This returns the data of the sensor visualized as an image (rgb array of shape (B, H, W, 3)). This should not be used for generating agent observations. For example lidar data can be visualized as an image but should not be in a image format (H, W, 3) when being used by an agent.

Return type:

torch.Tensor

abstract get_obs(**kwargs)[source]#

Retrieves captured sensor data as an observation for use by an agent.

abstract get_params()[source]#

Get parameters for this sensor. Should return a dictionary with keys mapping to torch.Tensor values

Return type:

dict

setup()[source]#

Setup this sensor given the current scene. This is called during environment/scene reconfiguration.

Return type:

None

config#
property uid#
class mani_skill.envs.utils.observations.BaseSensorConfig[source]#
uid: str#
class mani_skill.envs.utils.observations.Camera(camera_config, scene, articulation=None)[source]#

Bases: mani_skill.sensors.base_sensor.BaseSensor

Implementation of the Camera sensor which uses the sapien Camera.

Parameters:
capture()[source]#

Captures sensor data and prepares it for it to be then retrieved via get_obs for observations and get_image for a visualizable image.

Some sensors like rgbd cameras need to take a picture just once after each call to scene.update_render. Generally this should also be a non-blocking function if possible.

get_images(obs)[source]#

This returns the data of the sensor visualized as an image (rgb array of shape (B, H, W, 3)). This should not be used for generating agent observations. For example lidar data can be visualized as an image but should not be in a image format (H, W, 3) when being used by an agent.

Return type:

torch._tensor.Tensor

get_obs(rgb=True, depth=True, position=True, segmentation=True, normal=False, albedo=False, apply_texture_transforms=True)[source]#

Retrieves captured sensor data as an observation for use by an agent.

Parameters:

  • rgb (bool) –

  • depth (bool) –

  • position (bool) –

  • segmentation (bool) –

  • normal (bool) –

  • albedo (bool) –

  • apply_texture_transforms (bool) –

get_params()[source]#

Get parameters for this sensor. Should return a dictionary with keys mapping to torch.Tensor values

config: CameraConfig#
class mani_skill.envs.utils.observations.CameraObsTextures[source]#
albedo: bool[source]#
depth: bool[source]#
normal: bool[source]#
position: bool[source]#
rgb: bool[source]#
segmentation: bool[source]#
class mani_skill.envs.utils.observations.ObservationModeStruct[source]#

A dataclass describing what observation data is being requested by the user

state: bool[source]#

whether to include flattened state data which generally means including privileged information such as object poses

state_dict: bool[source]#

whether to include state data which generally means including privileged information such as object poses

property use_state[source]#

whether or not the environment should return ground truth/privileged information such as object poses

visual: CameraObsTextures[source]#

textures to capture from cameras

mani_skill.envs.utils.observations.parse_obs_mode_to_struct(obs_mode)[source]#

Given user supplied observation mode, return a struct with the relevant textures that are to be captured

Parameters:

obs_mode (str) –

Return type:

ObservationModeStruct

mani_skill.envs.utils.observations.sensor_data_to_pointcloud(observation, sensors)[source]#

convert all camera data in sensor to pointcloud data

Parameters:
mani_skill.envs.utils.observations.ALL_VISUAL_TEXTURES = ['rgb', 'depth', 'segmentation', 'position', 'normal', 'albedo'][source]#

set of all standard textures that can come from cameras