robometric_frame.safety

Safety and Robustness metrics for robotics policy evaluation.

This module provides metrics for evaluating the safety and robustness of robotics policies, including collision detection, obstacle proximity, and risk assessment.

All safety metrics use user-defined distance functions to evaluate trajectories against environment constraints.

class robometric_frame.safety.CollisionRate(distance_fn, collision_threshold=0.0, **kwargs)[source]

Compute Collision Rate for robotics policy safety evaluation.

Collision Rate is calculated as:

\[\text{CR} = \frac{N_{\text{collisions}}}{T_{\text{steps}}}\]

where \(N_{\text{collisions}}\) is the total number of collision occurrences and \(T_{\text{steps}}\) is the total number of trajectory steps.

This metric uses a user-defined distance function to compute distances to obstacles, then applies a threshold to detect collisions. A collision is detected when the distance is less than or equal to the threshold.

Parameters:

distance_fn (Callable[[Tensor, Any], Tensor]) –
User-defined function that computes distances to obstacles. Signature: distance_fn(trajectory: Tensor, environment: Any) -> Tensor - trajectory: Shape (…, L, D) where L is trajectory length, D is spatial dims - environment: User-defined environment representation (optional) - Returns: Tensor of shape (…, L) with distances to nearest obstacle

at each trajectory point (positive values)
collision_threshold (float) – Distance threshold for collision detection. Distances less than or equal to this value are considered collisions. Default: 0.0
**kwargs (Any) – Additional keyword arguments passed to the base Metric class.

Example

>>> from robometric_frame.safety import CollisionRate
>>> import torch
>>> # Define a distance function
>>> def simple_distance_fn(trajectory, environment=None):
...     # Distance to walls at ±5
...     x_coords = trajectory[..., 0]
...     dist_to_walls = torch.minimum(
...         torch.abs(x_coords - 5),
...         torch.abs(x_coords + 5)
...     )
...     return dist_to_walls
>>> metric = CollisionRate(distance_fn=simple_distance_fn, collision_threshold=0.5)
>>> # Trajectory with some points close to walls
>>> trajectory = torch.tensor([[0.0, 0.0], [3.0, 0.0], [4.8, 0.0], [1.0, 0.0]])
>>> metric.update(trajectory)
>>> result = metric.compute()
>>> result['collision_rate'].item()  # One point at distance 0.2 <= 0.5
0.25

Example (with environment):

>>> # Define distance function with environment obstacles
>>> def obstacle_distance_fn(trajectory, environment):
...     # environment is a dict with obstacle positions and radii
...     min_distances = torch.full(trajectory.shape[:-1], float('inf'))
...     for obs_pos, obs_radius in zip(environment['positions'], environment['radii']):
...         # Compute distance to obstacle surface
...         distances = torch.norm(trajectory - obs_pos, dim=-1) - obs_radius
...         min_distances = torch.minimum(min_distances, distances)
...     return torch.clamp(min_distances, min=0.0)  # Ensure non-negative
>>> environment = {
...     'positions': [torch.tensor([2.0, 2.0]), torch.tensor([5.0, 5.0])],
...     'radii': [0.5, 0.5]
... }
>>> metric = CollisionRate(distance_fn=obstacle_distance_fn)
>>> trajectory = torch.tensor([[0.0, 0.0], [2.0, 2.0], [3.0, 3.0], [4.0, 4.0]])
>>> metric.update(trajectory, environment=environment)
>>> result = metric.compute()

Example (batched):

>>> # Batch of trajectories
>>> metric = CollisionRate(distance_fn=simple_distance_fn, collision_threshold=0.5)
>>> trajectories = torch.tensor([
...     [[0.0, 0.0], [1.0, 0.0], [2.0, 0.0]],
...     [[4.7, 0.0], [4.9, 0.0], [5.0, 0.0]]
... ])
>>> metric.update(trajectories)
>>> result = metric.compute()

full_state_update: bool = False

is_differentiable: bool = False

higher_is_better: bool = False

total_collisions: Tensor

total_steps: Tensor

__init__(distance_fn, collision_threshold=0.0, **kwargs)[source]

Initialize the CollisionRate metric.

update(trajectory, environment=None)[source]

Update metric state with trajectory and collision information.

Parameters:

trajectory (Tensor) –
Trajectory tensor of shape (…, L, D) where: - … represents any number of batch dimensions (can be empty) - L is the number of trajectory points - D is the spatial dimensionality (typically 2 or 3)

Examples of valid shapes: - (L, D): Single trajectory - (B, L, D): Batch of B trajectories - (B, T, L, D): Batch with time/episode dimension
environment (Optional[Any]) – Optional environment representation passed to distance_fn. Can be any type (dict, object, tensor, etc.) that the user’s distance function expects.

Raises:

ValueError – If trajectory has invalid shape or distances are negative.
RuntimeError – If distance_fn returns invalid shape or type.

Return type:

None

Example

>>> metric = CollisionRate(distance_fn=my_distance_fn)
>>> trajectory = torch.randn(10, 2)  # 10 points in 2D
>>> metric.update(trajectory)
>>> # With environment
>>> metric.update(trajectory, environment={'obstacles': [...]})

compute()[source]

Compute collision rate statistics.

Returns:

‘collision_rate’: Ratio of collision steps to total steps
’total_collisions’: Total number of collision occurrences
’total_steps’: Total number of trajectory steps
’collision_percentage’: Collision rate as percentage (0-100)

Return type:

Dictionary containing

Raises:

RuntimeError – If no trajectories have been recorded.

Example

>>> metric = CollisionRate(distance_fn=my_distance_fn)
>>> metric.update(trajectory)
>>> result = metric.compute()
>>> print(f"Collision rate: {result['collision_rate'].item():.2%}")
>>> print(f"Total collisions: {result['total_collisions'].item()}")

class robometric_frame.safety.ObstacleProximity(distance_fn, **kwargs)[source]

Compute Obstacle Proximity for robotics policy safety evaluation.

Obstacle Proximity is calculated as:

\[\text{OP} = \text{mean}\left(\min_t d_t^{\text{robot} \rightarrow \text{obstacle}}\right)\]

where \(d_t\) is the distance from the robot to the nearest obstacle at time \(t\). For each trajectory, we find the minimum distance, then compute the mean of all these minimum distances across all trajectories.

This metric uses a user-defined distance function that computes the distance from trajectory points to obstacles in the environment. The design allows users to implement custom distance calculations based on their specific robot geometry and environment representation.

Parameters:

distance_fn (Callable[[Tensor, Any], Tensor]) –
User-defined function that computes distances to obstacles. Signature: distance_fn(trajectory: Tensor, environment: Any) -> Tensor - trajectory: Shape (…, L, D) where L is trajectory length, D is spatial dims - environment: User-defined environment representation (optional) - Returns: Tensor of shape (…, L) with distances to nearest obstacle

at each trajectory point (positive values)
**kwargs (Any) – Additional keyword arguments passed to the base Metric class.

Example

>>> from robometric_frame.safety import ObstacleProximity
>>> import torch
>>> # Define a simple distance function
>>> def simple_distance_fn(trajectory, environment=None):
...     # Distance to walls at ±10
...     x_coords = trajectory[..., 0]
...     dist_to_walls = torch.minimum(
...         torch.abs(x_coords - 10),
...         torch.abs(x_coords + 10)
...     )
...     return dist_to_walls
>>> metric = ObstacleProximity(distance_fn=simple_distance_fn)
>>> # Single trajectory: min distance is 2.0
>>> trajectory = torch.tensor([[0.0, 0.0], [5.0, 0.0], [8.0, 0.0]])
>>> metric.update(trajectory)
>>> result = metric.compute()
>>> result['mean_min_distance'].item()  # Mean of [2.0] = 2.0
2.0

Example (with environment):

>>> # Define distance function with environment obstacles
>>> def obstacle_distance_fn(trajectory, environment):
...     # environment contains obstacle positions
...     min_distances = torch.full(trajectory.shape[:-1], float('inf'))
...     for obs_pos in environment['positions']:
...         # Compute distance to this obstacle for all points
...         distances = torch.norm(trajectory - obs_pos, dim=-1)
...         min_distances = torch.minimum(min_distances, distances)
...     return min_distances
>>> environment = {
...     'positions': [torch.tensor([5.0, 5.0]), torch.tensor([10.0, 10.0])]
... }
>>> metric = ObstacleProximity(distance_fn=obstacle_distance_fn)
>>> trajectory = torch.tensor([[0.0, 0.0], [3.0, 3.0], [4.0, 4.0]])
>>> metric.update(trajectory, environment=environment)
>>> result = metric.compute()

Example (batched):

>>> # Batch of trajectories
>>> metric = ObstacleProximity(distance_fn=simple_distance_fn)
>>> # Trajectory 1: distances [10, 9, 8] -> min = 8
>>> # Trajectory 2: distances [5, 4, 3] -> min = 3
>>> trajectories = torch.tensor([
...     [[0.0, 0.0], [1.0, 0.0], [2.0, 0.0]],
...     [[5.0, 0.0], [6.0, 0.0], [7.0, 0.0]]
... ])
>>> metric.update(trajectories)
>>> result = metric.compute()
>>> result['mean_min_distance'].item()  # Mean of [8, 3] = 5.5
5.5

full_state_update: bool = False

is_differentiable: bool = False

higher_is_better: bool = True

sum_min_distances: Tensor

num_trajectories: Tensor

__init__(distance_fn, **kwargs)[source]

Initialize the ObstacleProximity metric.

update(trajectory, environment=None)[source]

Update metric state with trajectory and distance information.

Parameters:

trajectory (Tensor) –
Trajectory tensor of shape (…, L, D) where: - … represents any number of batch dimensions (can be empty) - L is the number of trajectory points - D is the spatial dimensionality (typically 2 or 3)

Examples of valid shapes: - (L, D): Single trajectory - (B, L, D): Batch of B trajectories - (B, T, L, D): Batch with time/episode dimension

For each trajectory, the minimum distance across L points is computed, then these minimums are accumulated to compute the mean.
environment (Optional[Any]) – Optional environment representation passed to distance_fn. Can be any type (dict, object, tensor, etc.) that the user’s distance function expects.

Raises:

ValueError – If trajectory has invalid shape or distances are negative.
RuntimeError – If distance_fn returns invalid shape or type.

Return type:

None

Example

>>> metric = ObstacleProximity(distance_fn=my_distance_fn)
>>> trajectory = torch.randn(10, 2)  # 10 points in 2D
>>> metric.update(trajectory)
>>> # With environment
>>> metric.update(trajectory, environment={'obstacles': [...]})

compute()[source]

Compute obstacle proximity statistics.

Returns:

‘mean_min_distance’: Mean of minimum distances across all trajectories
’sum_min_distances’: Sum of all minimum distances
’num_trajectories’: Number of trajectories evaluated

Return type:

Dictionary containing

Raises:

RuntimeError – If no trajectories have been recorded.

Example

>>> metric = ObstacleProximity(distance_fn=my_distance_fn)
>>> metric.update(trajectory)
>>> result = metric.compute()
>>> print(f"Average minimum distance: {result['mean_min_distance'].item():.2f}m")

class robometric_frame.safety.RiskFactor(distance_fn, epsilon=1e-06, **kwargs)[source]

Compute Risk Factor for robotics policy safety evaluation.

Risk Factor is calculated as:

\[\text{RF} = \frac{1}{T} \sum_{t=1}^{T} \frac{1}{d_t}\]

where \(d_t\) is the distance from the robot to the nearest obstacle at time \(t\), and \(T\) is the total number of trajectory steps. This metric provides a comprehensive safety assessment by penalizing trajectories that stay close to obstacles, with risk increasing as the robot approaches obstacles.

This metric uses a user-defined distance function that computes the distance from trajectory points to obstacles in the environment. The reciprocal of these distances creates a risk measure where smaller distances result in higher risk values.

Parameters:

distance_fn (Callable[[Tensor, Any], Tensor]) –
User-defined function that computes distances to obstacles. Signature: distance_fn(trajectory: Tensor, environment: Any) -> Tensor - trajectory: Shape (…, L, D) where L is trajectory length, D is spatial dims - environment: User-defined environment representation (optional) - Returns: Tensor of shape (…, L) with distances to nearest obstacle

at each trajectory point (positive values)
epsilon (float) – Small value added to distances to avoid division by zero. Default: 1e-6
**kwargs (Any) – Additional keyword arguments passed to the base Metric class.

Example

>>> from robometric_frame.safety import RiskFactor
>>> import torch
>>> # Define a simple distance function
>>> def simple_distance_fn(trajectory, environment=None):
...     # Distance to a wall at x=10
...     x_coords = trajectory[..., 0]
...     return torch.abs(10 - x_coords)
>>> metric = RiskFactor(distance_fn=simple_distance_fn)
>>> # Trajectory getting closer to wall: distances [5, 2, 1]
>>> trajectory = torch.tensor([[5.0, 0.0], [8.0, 0.0], [9.0, 0.0]])
>>> metric.update(trajectory)
>>> result = metric.compute()
>>> # RF = mean([1/5, 1/2, 1/1]) = mean([0.2, 0.5, 1.0]) = 0.567
>>> result['risk_factor'].item()
0.5666...

Example (with environment):

>>> # Define distance function with environment obstacles
>>> def obstacle_distance_fn(trajectory, environment):
...     # environment contains obstacle positions
...     min_distances = torch.full(trajectory.shape[:-1], float('inf'))
...     for obs_pos in environment['positions']:
...         distances = torch.norm(trajectory - obs_pos, dim=-1)
...         min_distances = torch.minimum(min_distances, distances)
...     return min_distances
>>> environment = {
...     'positions': [torch.tensor([5.0, 5.0])]
... }
>>> metric = RiskFactor(distance_fn=obstacle_distance_fn)
>>> trajectory = torch.tensor([[0.0, 0.0], [3.0, 3.0], [4.5, 4.5]])
>>> metric.update(trajectory, environment=environment)
>>> result = metric.compute()

Example (batched):

>>> # Batch of trajectories
>>> metric = RiskFactor(distance_fn=simple_distance_fn)
>>> # Trajectory 1: distances [5, 4, 3]
>>> # Trajectory 2: distances [2, 1, 0.5]
>>> trajectories = torch.tensor([
...     [[5.0, 0.0], [6.0, 0.0], [7.0, 0.0]],
...     [[8.0, 0.0], [9.0, 0.0], [9.5, 0.0]]
... ])
>>> metric.update(trajectories)
>>> result = metric.compute()
>>> # Higher risk for trajectory 2 (closer to wall)

full_state_update: bool = False

is_differentiable: bool = False

higher_is_better: bool = False

total_risk: Tensor

total_steps: Tensor

__init__(distance_fn, epsilon=1e-06, **kwargs)[source]

Initialize the RiskFactor metric.

update(trajectory, environment=None)[source]

Update metric state with trajectory and distance information.

Parameters:

trajectory (Tensor) –
Trajectory tensor of shape (…, L, D) where: - … represents any number of batch dimensions (can be empty) - L is the number of trajectory points - D is the spatial dimensionality (typically 2 or 3)

Examples of valid shapes: - (L, D): Single trajectory - (B, L, D): Batch of B trajectories - (B, T, L, D): Batch with time/episode dimension
environment (Optional[Any]) – Optional environment representation passed to distance_fn. Can be any type (dict, object, tensor, etc.) that the user’s distance function expects.

Raises:

ValueError – If trajectory has invalid shape or distances are negative.
RuntimeError – If distance_fn returns invalid shape or type.

Return type:

None

Example

>>> metric = RiskFactor(distance_fn=my_distance_fn)
>>> trajectory = torch.randn(10, 2)  # 10 points in 2D
>>> metric.update(trajectory)
>>> # With environment
>>> metric.update(trajectory, environment={'obstacles': [...]})

compute()[source]

Compute risk factor statistics.

Returns:

‘risk_factor’: Average risk across all trajectory points
’total_risk’: Total accumulated risk
’total_steps’: Total number of trajectory points

Return type:

Dictionary containing

Raises:

RuntimeError – If no trajectories have been recorded.

Example

>>> metric = RiskFactor(distance_fn=my_distance_fn)
>>> metric.update(trajectory)
>>> result = metric.compute()
>>> print(f"Risk factor: {result['risk_factor'].item():.4f}")
>>> # Lower values indicate safer trajectories

Modules

`base`	Base class for robotics policy safety metrics.
`collision_rate`	Collision Rate metric for robotics policy safety evaluation.
`obstacle_proximity`	Obstacle Proximity metric for robotics policy safety evaluation.
`risk_factor`	Risk Factor metric for robotics policy safety evaluation.