Extension

Extensions are FastSim's pluggable functionality layer. They hook into the simulation lifecycle at specific points without modifying the core.

How Extensions Work

Each extension implements Extension.enable(sim) to register callbacks during initialization. Extensions can hook into step_callback (each frame), terminate_callback (on shutdown), or run in a controller_thread.

Built-in Extensions

data_collect — Observation Collection

Registers observers to collect data from the scene (RGB images, depth maps, joint states, end-effector poses). Provides collect_observations() (main thread) and request_observations() (thread-safe).

record — Recording and Export

Depends on data_collect. Asynchronously writes each frame's observations to the filesystem, then post-processes on termination to generate HDF5, video, and other artifacts. Supports parallel ffmpeg + nvenc video encoding and S3 cloud storage backend.

replay — Trajectory Replay

Loads trajectories from record.h5 and drives entities along recorded trajectories in simulation.

server — REST API

Starts a Flask HTTP server exposing all @apiclass controller methods as REST endpoints.

benchmark — Episode Evaluation

Runs a policy in episodes on a separate control thread, evaluating goal conditions and reporting success rates.

teleop — Real-time Teleoperation

Supports real-time robot control from multiple input devices:

Control modes: ee_absolute, ee_delta, joint_position.

pipeline — Batch Job Execution

Sequentially executes multiple configuration files (jobs) in a single simulator session. Supports stop_on_failure and result aggregation.

snapshot — Simulation State Snapshot

Saves and restores complete simulation state (entity poses, joint states, planner attachments, task metadata) as .npz files. Enables task breakpoint resume.

task_executor — Automatic Task Execution

Runs configured tasks in a controller thread. Supports interactive debugging on action failure and snapshot restore before execution.

domain_randomization — Domain Randomization

Randomizes scene properties per episode: object poses, lighting, textures, and physics parameters. Used to improve policy generalization.

ros_bridge — ROS 2 Bridge

Bidirectional real-time communication between FastSim and ROS 2 — publishes sensor data as ROS topics and receives control commands.

Interactive Debugger

When task_executor.debug: true, an interactive REPL activates on action failure with commands: state, inspect, save, restore, eval, retry, skip, continue, quit.