diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index f09017991f..05460711ac 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -36,26 +36,32 @@ repos:
- id: check-yaml
- id: check-toml
- id: end-of-file-fixer
+ exclude: ^(outputs/|examples/hil_serl_simulation_training/outputs/)
- id: trailing-whitespace
+ exclude: ^(outputs/|examples/hil_serl_simulation_training/outputs/)
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.4
hooks:
- id: ruff-format
+ exclude: ^(outputs/|examples/hil_serl_simulation_training/outputs/)
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
+ exclude: ^(outputs/|examples/hil_serl_simulation_training/outputs/)
- repo: https://github.com/adhtruong/mirrors-typos
rev: v1.34.0
hooks:
- id: typos
args: [--force-exclude]
+ exclude: ^(outputs/|examples/hil_serl_simulation_training/outputs/)
- repo: https://github.com/asottile/pyupgrade
rev: v3.20.0
hooks:
- id: pyupgrade
args: [--py310-plus]
+ exclude: ^(outputs/|examples/hil_serl_simulation_training/outputs/)
##### Markdown Quality #####
- repo: https://github.com/rbubley/mirrors-prettier
diff --git a/examples/grid_hil_serl/README.md b/examples/grid_hil_serl/README.md
new file mode 100644
index 0000000000..60eac62bff
--- /dev/null
+++ b/examples/grid_hil_serl/README.md
@@ -0,0 +1,152 @@
+# Grid HIL SERL Environment
+
+This example demonstrates a **simplified HIL-SERL setup** for computer vision-based grid position prediction. Instead of complex robotic manipulation, the algorithm learns to predict which of the 64 grid cells contains a red cube based on camera images, with human feedback during training.
+
+## Overview
+
+The environment consists of:
+- An 8x8 grid world with high-definition visual rendering
+- A red cube that randomly spawns at grid cell centers
+- Top-left origin coordinate system (0,0) = top-left corner
+- Automatic high-definition image capture (1920x1080)
+
+## Files
+
+- `grid_scene.xml` - Mujoco scene definition with 8x8 grid
+- `grid_cube_randomizer.py` - Main script for randomizing cube positions
+- `README.md` - This documentation
+
+## Usage
+
+### 1. Test the Environment
+```bash
+cd examples/grid_hil_serl
+python grid_cube_randomizer.py
+```
+
+### 2. Record Demonstrations
+```bash
+# Record training data automatically
+python record_grid_demo.py --episodes 50 --steps 10
+
+# Or use LeRobot's recording script (standard dataset format)
+python -m lerobot.scripts.rl.gym_manipulator --config_path record_grid_position_lerobot.json
+```
+
+### 3. Train HIL-SERL Policy
+```bash
+# Terminal 1: Start learner
+python -m lerobot.scripts.rl.learner --config_path train_grid_position.json
+
+# Terminal 2: Start actor (with human feedback)
+python -m lerobot.scripts.rl.actor --config_path train_grid_position.json
+```
+
+### Command Line Options
+```bash
+# Environment testing
+python grid_cube_randomizer.py --interval 2.0 --no-save
+
+# Recording options
+python record_grid_demo.py --episodes 100 --steps 5 --output ./my_recordings
+```
+
+## Features
+
+### Grid System
+- **8x8 grid**: 64 total cells
+- **Coordinate system**: (0,0) = top-left, (7,7) = bottom-right
+- **Cell centers**: Cube spawns at precise grid cell centers
+- **High-definition**: 32x32 texture with 256x256 resolution
+
+### Cube Positioning
+- **Random placement**: Uniform random distribution across all 64 cells
+- **Precise positioning**: Cube lands exactly at grid cell centers
+- **Physics compliant**: Proper velocity reset for instant teleportation
+- **Visual feedback**: Clear console output of cell coordinates
+
+### Image Capture
+- **HD resolution**: 1920x1080 (Full HD)
+- **Automatic saving**: Images saved after each cube repositioning
+- **Professional quality**: Suitable for datasets and documentation
+- **Top-down view**: Camera positioned for complete grid visibility
+
+## Coordinate System
+
+```
+(0,0) → (-3.5, 3.5) (7,0) → (3.5, 3.5)
+ ↘ ↙
+(0,7) → (-3.5, -3.5) (7,7) → (3.5, -3.5)
+```
+
+## HIL-SERL Workflow
+
+This simplified setup demonstrates the core HIL-SERL concept with minimal complexity:
+
+### Training Phase (Offline)
+1. **Automatic Data Collection**: Environment randomly places cube in different grid positions
+2. **Supervised Learning**: Algorithm learns to predict grid position from images
+3. **Ground Truth Labels**: Exact grid coordinates provided for each image
+
+### Human-in-the-Loop Phase (Online)
+1. **Algorithm Prediction**: Model predicts cube position from camera images
+2. **Human Feedback**: Human indicates if prediction is correct/incorrect
+3. **Iterative Learning**: Model improves based on human guidance
+
+### Key Simplifications
+- **No Robot Control**: Focus purely on computer vision prediction
+- **Discrete Predictions**: 64 possible outputs (one per grid cell)
+- **Perfect Ground Truth**: Exact position labels available
+- **Visual Task Only**: No complex motor control or physics
+
+## Integration with LeRobot
+
+The environment integrates with LeRobot's HIL-SERL framework through:
+
+1. **Custom Gym Environment**: `GridPositionPrediction-v0` registered with gymnasium
+2. **LeRobot-Compatible Interface**: Proper observation/action space formatting
+3. **Config Files**: `record_grid_position.json` and `train_grid_position.json`
+4. **Dataset Collection**: Automated recording of image-position pairs
+
+## Technical Details
+
+- **Physics**: Mujoco physics engine with proper joint control
+- **Rendering**: Offscreen rendering with PIL for image saving
+- **Randomization**: NumPy-based random number generation
+- **Threading**: Proper event handling for viewer controls
+
+## Example Output
+
+```
+Loading scene: grid_scene.xml
+
+==================================================
+8x8 Grid Cube Randomizer
+==================================================
+This scene shows an 8x8 grid with a randomly positioned cube.
+Cube position randomizes every 3.0 seconds.
+
+Controls:
+ R: Manually randomize cube position
+ S: Save current camera view to img.jpg
+ Space: Pause/unpause
+ Esc: Exit
+ Camera: Mouse controls for rotation/zoom
+==================================================
+Spawning cube at grid cell (3, 5) -> position (-0.5, -1.5)
+Camera view saved to: img.jpg
+Spawning cube at grid cell (1, 2) -> position (-2.5, 1.5)
+Camera view saved to: img.jpg
+```
+
+## Dependencies
+
+- mujoco
+- numpy
+- PIL (Pillow)
+- gymnasium (optional, for integration)
+
+## Related Examples
+
+- `hil_serl_simulation_training/` - Full HIL-SERL training examples
+- `lekiwi/` - Real robot integration examples
diff --git a/examples/grid_hil_serl/grid_cube_randomizer.py b/examples/grid_hil_serl/grid_cube_randomizer.py
new file mode 100644
index 0000000000..45ee111f19
--- /dev/null
+++ b/examples/grid_hil_serl/grid_cube_randomizer.py
@@ -0,0 +1,161 @@
+#!/usr/bin/env python
+
+"""
+Random Grid Cube Spawner
+
+This script loads the 8x8 grid scene and randomly positions a cube
+in one of the 64 grid cells. The cube spawns at integer coordinates
+within the grid boundaries.
+"""
+
+import numpy as np
+import mujoco
+import mujoco.viewer
+import argparse
+import time
+from PIL import Image
+
+
+def save_camera_view(model, data, filename="img.jpg"):
+ """
+ Save the current camera view to a JPEG image file.
+
+ Args:
+ model: Mujoco model
+ data: Mujoco data
+ filename: Output filename (default: img.jpg)
+ """
+ try:
+ # Create a high-definition renderer for the current camera
+ renderer = mujoco.Renderer(model, height=1080, width=1920)
+
+ # Update the scene and render
+ renderer.update_scene(data, camera="grid_camera")
+ img = renderer.render()
+
+ if img is not None:
+ # Convert to PIL Image and save
+ image = Image.fromarray(img)
+ image.save(filename)
+ print(f"Camera view saved to: {filename}")
+ else:
+ print("Warning: Could not capture camera view")
+
+ # Clean up renderer (if close method exists)
+ if hasattr(renderer, 'close'):
+ renderer.close()
+
+ except Exception as e:
+ print(f"Error saving image: {e}")
+
+
+def randomize_cube_position(model, data, grid_size=8):
+ """
+ Randomly position the cube in one of the grid cells.
+
+ Args:
+ model: Mujoco model
+ data: Mujoco data
+ grid_size: Size of the grid (8x8)
+ """
+ # For 8x8 grid: generate random cell indices from 0-7 for both x and y
+ # This gives us coordinates for each of the 64 grid cells
+ x_cell = np.random.randint(0, 8) # 0 to 7 inclusive
+ y_cell = np.random.randint(0, 8) # 0 to 7 inclusive
+
+ # Convert cell indices to center positions (offset by 0.5 from grid lines)
+ # X: left(0) = -3.5, right(7) = 3.5
+ x_pos = (x_cell - grid_size // 2) + 0.5
+ # Y: top(0) = 3.5, bottom(7) = -3.5 (flipped coordinate system)
+ y_pos = (grid_size // 2 - y_cell) - 0.5
+
+ print(f"Spawning cube at grid cell ({x_cell}, {y_cell}) -> position ({x_pos}, {y_pos})")
+
+ # Set the cube position and velocity (free joint has 6 DOF: 3 pos + 3 vel)
+ cube_joint_id = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "cube_joint")
+
+ # Set position (x, y, z) - keep rotation as identity (0, 0, 0)
+ data.qpos[model.jnt_qposadr[cube_joint_id]:model.jnt_qposadr[cube_joint_id] + 6] = [x_pos, y_pos, 0.5, 0, 0, 0]
+
+ # Reset velocity to zero (linear and angular velocities)
+ data.qvel[model.jnt_dofadr[cube_joint_id]:model.jnt_dofadr[cube_joint_id] + 6] = [0, 0, 0, 0, 0, 0]
+
+ return x_pos, y_pos
+
+
+def run_grid_viewer(xml_path, randomize_interval=2.0, auto_save=True):
+ """
+ Run the grid viewer with random cube positioning.
+
+ Args:
+ xml_path: Path to the XML scene file
+ randomize_interval: How often to randomize cube position (seconds)
+ auto_save: Whether to automatically save camera view after each repositioning
+ """
+ print(f"Loading scene: {xml_path}")
+ model = mujoco.MjModel.from_xml_path(xml_path)
+ data = mujoco.MjData(model)
+
+ print("\n" + "="*50)
+ print("8x8 Grid Cube Randomizer")
+ print("="*50)
+ print("This scene shows an 8x8 grid with a randomly positioned cube.")
+ print(f"Cube position randomizes every {randomize_interval} seconds.")
+ print()
+ print("Controls:")
+ print(" R: Manually randomize cube position")
+ print(" S: Save current camera view to img.jpg")
+ print(" Space: Pause/unpause")
+ print(" Esc: Exit")
+ print(" Camera: Mouse controls for rotation/zoom")
+ print("="*50)
+
+ last_randomize_time = 0
+
+ with mujoco.viewer.launch_passive(model, data) as viewer:
+ # Initial randomization
+ x, y = randomize_cube_position(model, data)
+ mujoco.mj_forward(model, data)
+
+ while viewer.is_running():
+ current_time = time.time()
+
+ # Auto-randomize every few seconds
+ if current_time - last_randomize_time > randomize_interval:
+ x, y = randomize_cube_position(model, data)
+ mujoco.mj_forward(model, data)
+ # Force viewer to update the scene
+ viewer.sync()
+ # Save the current camera view if auto_save is enabled
+ if auto_save:
+ save_camera_view(model, data, "img.jpg")
+ last_randomize_time = current_time
+
+ # Small delay to prevent excessive CPU usage
+ time.sleep(0.01)
+
+ print("\nViewer closed.")
+
+
+def main():
+ parser = argparse.ArgumentParser(description="8x8 Grid Cube Randomizer")
+ parser.add_argument("--xml", type=str, default="grid_scene.xml",
+ help="Path to XML scene file")
+ parser.add_argument("--interval", type=float, default=3.0,
+ help="Randomization interval in seconds")
+ parser.add_argument("--no-save", action="store_true",
+ help="Disable automatic saving of camera views")
+
+ args = parser.parse_args()
+
+ try:
+ run_grid_viewer(args.xml, args.interval, not args.no_save)
+ except FileNotFoundError:
+ print(f"Error: Could not find XML file '{args.xml}'")
+ print("Make sure the XML file exists in the current directory.")
+ except Exception as e:
+ print(f"Error: {e}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/grid_hil_serl/grid_position_env.py b/examples/grid_hil_serl/grid_position_env.py
new file mode 100644
index 0000000000..c7c1a081a9
--- /dev/null
+++ b/examples/grid_hil_serl/grid_position_env.py
@@ -0,0 +1,276 @@
+#!/usr/bin/env python
+
+"""
+Grid Position Prediction Environment for HIL-SERL
+
+This environment provides image observations and grid position labels for training
+a position prediction policy using HIL-SERL. The task is simplified to predict
+which of the 64 grid cells contains the red cube.
+"""
+
+import numpy as np
+import gymnasium as gym
+from gymnasium import spaces
+import mujoco
+import mujoco.viewer
+from pathlib import Path
+from PIL import Image
+import time
+
+# Register the environment with gymnasium
+gym.register(
+ id="GridPositionPrediction-v0",
+ entry_point="examples.grid_hil_serl.grid_position_env:GridPositionLeRobotEnv",
+ kwargs={"xml_path": "grid_scene.xml"}
+)
+
+
+class GridPositionEnv(gym.Env):
+ """
+ Simplified environment for grid position prediction.
+
+ Observations: High-definition RGB images (1920x1080)
+ Actions: None (prediction-only task)
+ Reward: Binary feedback from human (correct/incorrect prediction)
+ """
+
+ def __init__(self, xml_path="grid_scene.xml", render_mode="rgb_array", show_viewer: bool = True):
+ super().__init__()
+
+ # Load Mujoco model
+ self.xml_path = Path(__file__).parent / xml_path
+ if not self.xml_path.exists():
+ raise FileNotFoundError(f"XML file not found: {self.xml_path}")
+
+ self.model = mujoco.MjModel.from_xml_path(str(self.xml_path))
+ self.data = mujoco.MjData(self.model)
+
+ # Setup renderer for observations
+ self.renderer = mujoco.Renderer(self.model, height=1080, width=1920)
+ # Optional interactive viewer for visualization
+ self.viewer = None
+ self._show_viewer = bool(show_viewer)
+ if self._show_viewer:
+ try:
+ self.viewer = mujoco.viewer.launch_passive(self.model, self.data)
+ except Exception:
+ self.viewer = None
+
+ # Observation space: High-definition RGB images
+ self.observation_space = spaces.Box(
+ low=0,
+ high=255,
+ shape=(1080, 1920, 3),
+ dtype=np.uint8
+ )
+
+ # Action space: Discrete grid positions (0-63 for 8x8 grid)
+ # We'll use a dummy action space since this is prediction-only
+ self.action_space = spaces.Discrete(1) # Placeholder
+
+ # Environment state
+ self.current_position = None
+ self.grid_size = 8
+
+ # Initialize to random position
+ self._randomize_cube_position()
+
+ def _randomize_cube_position(self):
+ """Randomize cube position and return grid coordinates."""
+ # Generate random cell indices (0-7 for 8x8 grid)
+ x_cell = np.random.randint(0, self.grid_size)
+ y_cell = np.random.randint(0, self.grid_size)
+
+ # Convert to physical positions (top-left origin)
+ x_pos = (x_cell - self.grid_size // 2) + 0.5
+ y_pos = (self.grid_size // 2 - y_cell) - 0.5
+
+ # Set cube position
+ cube_joint_id = mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_JOINT, "cube_joint")
+ self.data.qpos[mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_JOINT, "cube_joint"):mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_JOINT, "cube_joint") + 6] = [x_pos, y_pos, 0.5, 0, 0, 0]
+ self.data.qvel[mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_JOINT, "cube_joint"):mujoco.mj_name2id(self.model, mujoco.mjtObj.mjOBJ_JOINT, "cube_joint") + 6] = [0, 0, 0, 0, 0, 0]
+
+ mujoco.mj_forward(self.model, self.data)
+ if self.viewer is not None:
+ try:
+ self.viewer.sync()
+ except Exception:
+ pass
+
+ # Store current grid position (0-63)
+ self.current_position = y_cell * self.grid_size + x_cell
+
+ return self.current_position
+
+ def reset(self, seed=None, options=None):
+ """Reset environment to random cube position."""
+ super().reset(seed=seed)
+
+ # Randomize cube position
+ position = self._randomize_cube_position()
+
+ # Get observation
+ obs = self._get_observation()
+
+ return obs, {"grid_position": position}
+
+ def step(self, action):
+ """
+ Step function for HIL-SERL compatibility.
+
+ In this simplified version, actions are not used for control,
+ but for receiving human feedback on predictions.
+ """
+ # For now, just randomize position on each step
+ # In real HIL-SERL, this would be triggered by the learning algorithm
+ position = self._randomize_cube_position()
+ obs = self._get_observation()
+
+ # Placeholder reward before wrapper computes it (will be recomputed in wrapper)
+ reward = 0.0
+ terminated = False
+ truncated = False
+
+ info = {
+ "grid_position": position,
+ "x_cell": position % self.grid_size,
+ "y_cell": position // self.grid_size
+ }
+
+ return obs, reward, terminated, truncated, info
+
+ def _get_observation(self):
+ """Get current camera observation."""
+ self.renderer.update_scene(self.data, camera="grid_camera")
+ img = self.renderer.render()
+ return img
+
+ def render(self):
+ """Render current observation."""
+ return self._get_observation()
+
+ def close(self):
+ """Clean up resources."""
+ # Renderer doesn't have a close method in this version
+ try:
+ if self.viewer is not None:
+ self.viewer.close()
+ except Exception:
+ pass
+
+ def get_current_position(self):
+ """Get current grid position (0-63)."""
+ return self.current_position
+
+ def validate_prediction(self, predicted_position):
+ """
+ Validate a predicted position against current cube position.
+
+ Args:
+ predicted_position: Integer 0-63 representing predicted grid cell
+
+ Returns:
+ bool: True if prediction is correct
+ """
+ return predicted_position == self.current_position
+
+
+# LeRobot-compatible wrapper
+class GridPositionLeRobotEnv:
+ """
+ LeRobot-compatible wrapper for the grid position environment.
+ """
+
+ def __init__(self, xml_path="grid_scene.xml"):
+ self.env = GridPositionEnv(xml_path, show_viewer=True)
+
+ # LeRobot observation space format - use 'observation.image' as expected by SAC
+ self.observation_space = {
+ "observation.image": spaces.Box(
+ low=0, high=255, shape=(1080, 1920, 3), dtype=np.uint8
+ )
+ }
+
+ # Action is predicted grid cell as 2-int vector [x, y] in [0,7]
+ self.action_space = spaces.Box(low=0.0, high=7.0, shape=(2,), dtype=np.float32)
+
+ def reset(self, seed=None, options=None):
+ obs, info = self.env.reset(seed=seed, options=options)
+
+ # Convert to LeRobot format
+ lerobot_obs = {
+ "observation.image": obs
+ }
+
+ return lerobot_obs, info
+
+ def step(self, action):
+ # Compute reward based on predicted [x, y] vs current ground-truth before advancing env
+ gt_position = int(self.env.get_current_position())
+ gt_x = gt_position % self.env.grid_size
+ gt_y = gt_position // self.env.grid_size
+
+ # Clamp and round action to valid cell indices
+ action = np.asarray(action, dtype=np.float32)
+ pred_x = int(np.clip(np.rint(action[0]), 0, self.env.grid_size - 1))
+ pred_y = int(np.clip(np.rint(action[1]), 0, self.env.grid_size - 1))
+
+ # Dense reward shaping based on L1 grid error (normalized to [0,1])
+ max_l1 = 2 * (self.env.grid_size - 1)
+ l1_error = abs(pred_x - gt_x) + abs(pred_y - gt_y)
+ reward = 1.0 - (l1_error / max_l1)
+
+ # Advance environment to next random position
+ obs, _, terminated, truncated, info = self.env.step(0)
+
+ # Convert to LeRobot format
+ lerobot_obs = {
+ "observation.image": obs
+ }
+
+ # Enrich info with prediction details
+ info = {
+ **info,
+ "gt_x": gt_x,
+ "gt_y": gt_y,
+ "pred_x": pred_x,
+ "pred_y": pred_y,
+ "correct": pred_x == gt_x and pred_y == gt_y,
+ "error.l1": float(l1_error),
+ }
+
+ return lerobot_obs, reward, terminated, truncated, info
+
+ def render(self):
+ return self.env.render()
+
+ def close(self):
+ return self.env.close()
+
+
+if __name__ == "__main__":
+ # Test the environment
+ env = GridPositionEnv()
+
+ print("Testing Grid Position Environment")
+ print("=" * 40)
+
+ obs, info = env.reset()
+ print(f"Initial position: {info['grid_position']}")
+ print(f"Observation shape: {obs.shape}")
+
+ # Test a few steps
+ for i in range(5):
+ obs, reward, terminated, truncated, info = env.step(0)
+ pos = info['grid_position']
+ x_cell = info['x_cell']
+ y_cell = info['y_cell']
+ print(f"Step {i+1}: Grid cell ({x_cell}, {y_cell}) = Position {pos}")
+
+ # Save sample image
+ img = Image.fromarray(obs)
+ img.save(f"sample_position_{pos}.jpg")
+ print(f" Saved image: sample_position_{pos}.jpg")
+
+ env.close()
+ print("Environment test completed!")
diff --git a/examples/grid_hil_serl/grid_scene.xml b/examples/grid_hil_serl/grid_scene.xml
new file mode 100644
index 0000000000..8340aa74d0
--- /dev/null
+++ b/examples/grid_hil_serl/grid_scene.xml
@@ -0,0 +1,37 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/examples/grid_hil_serl/img.jpg b/examples/grid_hil_serl/img.jpg
new file mode 100644
index 0000000000..2a794ea883
Binary files /dev/null and b/examples/grid_hil_serl/img.jpg differ
diff --git a/examples/grid_hil_serl/record_grid_demo.py b/examples/grid_hil_serl/record_grid_demo.py
new file mode 100644
index 0000000000..ed8153b742
--- /dev/null
+++ b/examples/grid_hil_serl/record_grid_demo.py
@@ -0,0 +1,188 @@
+#!/usr/bin/env python
+
+"""
+Grid Position Prediction Recording Script
+
+This script records demonstrations for the grid position prediction task.
+It automatically randomizes cube positions and records image observations
+with corresponding grid position labels.
+"""
+
+import argparse
+import json
+import numpy as np
+from pathlib import Path
+from PIL import Image
+
+import sys
+
+# Ensure this script can import the local 'lerobot' package whether installed or not.
+# We add both the repository root and the 'src' directory (common layout: src/lerobot/...)
+_THIS_FILE = Path(__file__).resolve()
+_REPO_ROOT = _THIS_FILE.parents[2]
+_SRC_DIR = _REPO_ROOT / "src"
+for _p in (str(_REPO_ROOT), str(_SRC_DIR)):
+ if _p not in sys.path:
+ sys.path.insert(0, _p)
+
+from grid_position_env import GridPositionLeRobotEnv
+from lerobot.datasets.lerobot_dataset import LeRobotDataset
+
+
+def record_demonstrations_from_config(config_path: str):
+ """
+ Record demonstrations using a LeRobot-style JSON config.
+
+ Expects a config with keys similar to gym_manipulator:
+ - env.task (we use GridPositionPrediction-v0)
+ - env.fps (optional, default 10)
+ - dataset.repo_id, dataset.root, dataset.num_episodes_to_record, dataset.push_to_hub, dataset.task
+ """
+ with open(config_path, "r") as f:
+ cfg = json.load(f)
+
+ # Defaults and required fields
+ env_cfg = cfg.get("env", {})
+ dataset_cfg = cfg.get("dataset", {})
+
+ task_name = env_cfg.get("task", "GridPositionPrediction-v0")
+ fps = int(env_cfg.get("fps", 10))
+
+ repo_id = dataset_cfg.get("repo_id", "username/grid-position-prediction")
+ root = dataset_cfg.get("root", "./recording_grid_position_lerobot")
+ num_episodes = int(dataset_cfg.get("num_episodes_to_record", 10))
+ steps_per_episode = int(dataset_cfg.get("steps_per_episode", 5))
+ push_to_hub = bool(dataset_cfg.get("push_to_hub", False))
+ task_string = dataset_cfg.get("task", "grid_position_prediction")
+
+ root_path = Path(root)
+ if root_path.exists():
+ base = root_path
+ counter = 1
+ while True:
+ candidate = base.parent / f"{base.name}_{counter:03d}"
+ if not candidate.exists():
+ root_path = candidate
+ break
+ counter += 1
+
+ print(f"Recording {num_episodes} episodes to {root_path}")
+ print("=" * 50)
+
+ # Create environment
+ env = GridPositionLeRobotEnv()
+
+ # Target image size for training (keep aspect ratio via center-square crop, then resize)
+ target_h, target_w = 128, 128
+
+ # Helper: center-square crop and resize to (128,128)
+ def _center_crop_resize(image_hwc: np.ndarray, out_h: int = 128, out_w: int = 128) -> np.ndarray:
+ h, w, _ = image_hwc.shape
+ side = min(h, w)
+ top = (h - side) // 2
+ left = (w - side) // 2
+ crop = image_hwc[top : top + side, left : left + side]
+ img = Image.fromarray(crop)
+ img = img.resize((out_w, out_h), resample=Image.BILINEAR)
+ return np.asarray(img, dtype=np.uint8)
+
+ # Prime env and process one frame to validate pipeline
+ obs, info = env.reset()
+ image_hwc = obs["observation.image"] # H, W, C uint8
+ image_hwc = _center_crop_resize(image_hwc, target_h, target_w)
+
+ # Define features for LeRobot dataset
+ # Observation: input image only; Action: target grid position as [x, y] integers
+ # Include reward/done to be compatible with RL replay buffer expectations
+ features = {
+ "observation.image": {"dtype": "image", "shape": (3, target_h, target_w), "names": None},
+ "action": {"dtype": "int64", "shape": (2,), "names": None},
+ "next.reward": {"dtype": "float32", "shape": (1,), "names": None},
+ "next.done": {"dtype": "bool", "shape": (1,), "names": None},
+ }
+
+ # Initialize LeRobot dataset (images saved as files, rows in parquet)
+ dataset = LeRobotDataset.create(
+ repo_id=repo_id,
+ fps=fps,
+ features=features,
+ root=root_path,
+ use_videos=False,
+ )
+
+ total_frames = 0
+
+ for ep in range(num_episodes):
+ print(f"\nRecording episode {ep + 1}/{num_episodes}")
+
+ # Ensure a fresh episode
+ if dataset.episode_buffer is not None:
+ dataset.clear_episode_buffer()
+
+ # Reset env for each episode
+ obs, info = env.reset()
+ current_position = int(info["grid_position"]) # 0..63
+
+ for step in range(steps_per_episode):
+ # Save current image to img.jpg (for quick visual check)
+ image_array = obs["observation.image"] # HWC uint8
+ image_array = _center_crop_resize(image_array, target_h, target_w)
+ Image.fromarray(image_array).save("img.jpg")
+
+ # Prepare frame for dataset: action stores the ground-truth position
+ action_xy = np.array([
+ current_position % 8,
+ current_position // 8,
+ ], dtype=np.int64)
+ # Offline pretraining: use zero reward; mark done on last step of episode
+ is_last = (step == steps_per_episode - 1)
+ frame = {
+ "observation.image": image_array, # LeRobot will write image file
+ "action": action_xy,
+ "next.reward": np.array([0.0], dtype=np.float32),
+ "next.done": np.array([is_last], dtype=bool),
+ }
+
+ dataset.add_frame(frame=frame, task=task_string)
+ total_frames += 1
+
+ print(
+ f" Step {step + 1}: Position {current_position} "
+ f"(action=[{current_position % 8}, {current_position // 8}])"
+ )
+
+ # Advance environment using ground-truth action to satisfy env API
+ obs, reward, terminated, truncated, info = env.step(action_xy)
+ current_position = int(info["grid_position"]) # next label
+
+ # Save episode to parquet + meta
+ dataset.save_episode()
+ print(f" Saved episode {ep:06d} to dataset")
+
+ # Optionally push to hub
+ if push_to_hub:
+ print("\nPushing dataset to Hugging Face Hub...")
+ dataset.push_to_hub()
+
+ env.close()
+
+ print(f"\nRecording complete! Total frames: {total_frames}")
+ print(f"Dataset saved to: {root_path}")
+
+
+def main():
+ parser = argparse.ArgumentParser(description="Record Grid Position Prediction Demonstrations (LeRobot format)")
+ parser.add_argument("--config_path", type=str, required=True, help="Path to LeRobot-style recording JSON config")
+
+ args = parser.parse_args()
+
+ try:
+ record_demonstrations_from_config(args.config_path)
+ except Exception as e:
+ print(f"Error: {e}")
+ import traceback
+ traceback.print_exc()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/grid_hil_serl/record_grid_position_lerobot.json b/examples/grid_hil_serl/record_grid_position_lerobot.json
new file mode 100644
index 0000000000..1ae86bfc53
--- /dev/null
+++ b/examples/grid_hil_serl/record_grid_position_lerobot.json
@@ -0,0 +1,18 @@
+{
+ "env": {
+ "type": "gym_manipulator",
+ "name": "grid_position",
+ "task": "GridPositionPrediction-v0"
+ },
+ "dataset": {
+ "repo_id": "spirosperos/grid-position-prediction",
+ "root": "./recording_grid_position_lerobot",
+ "task": "grid_position_prediction",
+ "num_episodes_to_record": 10,
+ "replay_episode": null,
+ "push_to_hub": true
+ },
+ "mode": "record"
+}
+
+
diff --git a/examples/grid_hil_serl/train_grid_position.json b/examples/grid_hil_serl/train_grid_position.json
new file mode 100644
index 0000000000..a3d238853d
--- /dev/null
+++ b/examples/grid_hil_serl/train_grid_position.json
@@ -0,0 +1,108 @@
+{
+ "output_dir": null,
+ "job_name": "grid_position",
+ "resume": false,
+ "seed": 1000,
+ "num_workers": 4,
+ "batch_size": 256,
+ "steps": 100000,
+ "log_freq": 50,
+ "save_checkpoint": true,
+ "save_freq": 10,
+ "wandb": {
+ "enable": false,
+ "project": "grid_hil_serl",
+ "disable_artifact": true
+ },
+ "dataset": {
+ "repo_id": "spirosperos/grid-position-prediction",
+ "root": "/home/nikola/lerobot_spes/recording_grid_position_lerobot_004",
+ "revision": null,
+ "use_imagenet_stats": false
+ },
+ "policy": {
+ "type": "sac",
+ "n_obs_steps": 1,
+ "repo_id": "spirosperos/grid-position-policy",
+ "normalization_mapping": {
+ "VISUAL": "MEAN_STD",
+ "ACTION": "MIN_MAX"
+ },
+ "input_features": {
+ "observation.image": {
+ "type": "VISUAL",
+ "shape": [3, 128, 128]
+ }
+ },
+ "output_features": {
+ "action": {
+ "type": "ACTION",
+ "shape": [2]
+ }
+ },
+ "device": "cuda",
+ "storage_device": "cpu",
+ "use_amp": false,
+ "use_torch_compile": false,
+ "shared_encoder": true,
+ "freeze_vision_encoder": false,
+ "vision_encoder_name": "helper2424/resnet10",
+ "image_encoder_hidden_dim": 32,
+ "image_embedding_pooling_dim": 8,
+ "latent_dim": 64,
+ "actor_network_kwargs": { "hidden_dims": [256, 256], "activate_final": true },
+ "critic_network_kwargs": { "hidden_dims": [256, 256], "activate_final": true, "final_activation": null },
+ "num_critics": 2,
+ "actor_lr": 0.0003,
+ "critic_lr": 0.0003,
+ "temperature_lr": 0.0003,
+ "temperature_init": 0.01,
+ "discount": 0.97,
+ "critic_target_update_weight": 0.005,
+ "policy_update_freq": 1,
+ "utd_ratio": 2,
+ "grad_clip_norm": 10.0,
+ "offline_buffer_capacity": 1000,
+ "online_buffer_capacity": 10000,
+ "online_steps": 1000000,
+ "online_step_before_learning": 100,
+ "async_prefetch": false,
+ "actor_learner_config": {
+ "learner_host": "127.0.0.1",
+ "learner_port": 50053,
+ "policy_parameters_push_frequency": 1,
+ "queue_get_timeout": 2
+ },
+ "push_to_hub": true,
+ "license": null,
+ "private": null,
+ "tags": null
+ },
+ "env": {
+ "type": "gym_manipulator",
+ "task": "GridPositionPrediction-v0",
+ "mode": "auto_supervision",
+ "fps": 10,
+ "device": "cuda",
+ "features": {
+ "observation.image": {
+ "type": "VISUAL",
+ "shape": [3, 128, 128]
+ },
+ "action": {
+ "type": "ACTION",
+ "shape": [2]
+ }
+ },
+ "features_map": {
+ "observation.image": "observation.image",
+ "action": "action"
+ },
+ "wrapper": {
+ "resize_size": [128, 128],
+ "display_cameras": false
+ }
+ }
+}
+
+
diff --git a/examples/hil_serl_simulation_training/hi_rl_test_gamepad.json b/examples/hil_serl_simulation_training/hi_rl_test_gamepad.json
new file mode 100644
index 0000000000..4b040a9df2
--- /dev/null
+++ b/examples/hil_serl_simulation_training/hi_rl_test_gamepad.json
@@ -0,0 +1,92 @@
+{
+ "type": "hil",
+ "wrapper": {
+ "gripper_penalty": -0.02,
+ "display_cameras": false,
+ "add_joint_velocity_to_observation": true,
+ "add_ee_pose_to_observation": true,
+ "crop_params_dict": {
+ "observation.images.front": [
+ 0,
+ 0,
+ 128,
+ 128
+ ],
+ "observation.images.wrist": [
+ 0,
+ 0,
+ 128,
+ 128
+ ]
+ },
+ "resize_size": [
+ 128,
+ 128
+ ],
+ "control_time_s": 40.0,
+ "use_gripper": true,
+ "fixed_reset_joint_positions": [
+ 0.0,
+ 0.195,
+ 0.0,
+ -2.43,
+ 0.0,
+ 2.62,
+ 0.785
+ ],
+ "reset_time_s": 2.0,
+ "control_mode": "gamepad"
+ },
+ "use_viewer": true,
+ "use_gamepad": true,
+ "name": "franka_sim",
+ "mode": "record",
+ "repo_id": "username/panda-pick-gamepad",
+ "dataset_root": "./recordings_static_gamepad",
+ "task": "PandaPickCubeGamepad-v0",
+ "num_episodes": 30,
+ "episode": 0,
+ "pretrained_policy_name_or_path": null,
+ "device": "cuda",
+ "push_to_hub": true,
+ "fps": 10,
+ "random_block_position": true,
+ "features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ },
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 4
+ ]
+ }
+ },
+ "features_map": {
+ "observation.images.front": "observation.images.front",
+ "observation.images.wrist": "observation.images.wrist",
+ "observation.state": "observation.state",
+ "action": "action"
+ },
+ "reward_classifier_pretrained_path": null,
+ "number_of_steps_after_success": 0
+}
diff --git a/examples/hil_serl_simulation_training/hil_serl_simulation_training_guide_README.md b/examples/hil_serl_simulation_training/hil_serl_simulation_training_guide_README.md
new file mode 100644
index 0000000000..1543b7b51e
--- /dev/null
+++ b/examples/hil_serl_simulation_training/hil_serl_simulation_training_guide_README.md
@@ -0,0 +1,98 @@
+# LeRobot Training Guide: Panda Cube Picking
+
+A concise guide for training a robot policy to pick cubes using LeRobot's HIL-SERL framework.
+
+## Installation
+
+Install LeRobot with HIL-SERL support as shown in the [official documentation](https://huggingface.co/docs/lerobot/hilserl):
+
+```bash
+pip install -e ".[hilserl]"
+```
+
+## Recording Demonstrations
+
+Proceed with recording sample episodes for pretraining the reinforcement learning algorithm:
+
+```bash
+python -m lerobot.scripts.rl.gym_manipulator --config_path examples/hil_serl_simulation_training/hi_rl_test_gamepad.json
+```
+
+**Important Notes:**
+
+- **Recommended**: Collect 30-40 episodes including some failed picking attempts
+- **Control**: Use a gamepad controller for better end-effector control (configuration already set up)
+- **Dataset Storage**:
+ - `dataset_root` points to local folder where dataset is saved
+ - `repo_id` points to HuggingFace account for dataset upload
+ - **Prerequisite**: Install and authorize HuggingFace CLI with token to push datasets and replace the username in the `hi_rl_test_gamepad.json` and `train_gym_hil_env_gamepad.json`
+
+## Training the Policy
+
+Start both the learner and actor using these commands:
+
+```bash
+# Terminal 1: Start learner
+python -m lerobot.scripts.rl.learner --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json
+
+# Terminal 2: Start actor
+python -m lerobot.scripts.rl.actor --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json
+```
+
+The learner initializes with parameters from the offline training dataset.
+
+## Human Intervention Strategy
+
+**Important**: Allow the policy a few episodes to test picking on its own before intervening.
+
+**Best practices** for human intervention (see [official guide](https://huggingface.co/docs/lerobot/hilserl#guide-to-human-interventions)):
+
+1. **Guide the end effector back to the object** if it's drifting away from the cube
+2. **Guide the end effector above the cube** and let the policy try to pick it and raise it
+3. **Completely pick the object** if needed
+
+**Progression Strategy:**
+
+- As training progresses, interventions should become less aggressive
+- Once the policy manages to pick the cube on its own, start intervening only to guide the end effector to the approximate area where the object is found
+
+## Training Results
+
+The fully trained policy result can be seen in this video:
+
+
+
+## Resuming Training from Checkpoint
+
+Checkpoints are automatically saved based on the `save_freq` parameter in `train_gym_hil_env_gamepad.json` and are stored in the `outputs` folder.
+
+To resume training, both actor and learner need to be called with the policy path:
+
+```bash
+# Terminal 1: Resume learner
+python -m lerobot.scripts.rl.learner --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json --policy.path=examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model
+
+# Terminal 2: Resume actor
+python -m lerobot.scripts.rl.actor --config_path examples/hil_serl_simulation_training/train_gym_hil_env_gamepad.json --policy.path=examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model
+```
+
+Where `policy.path` is the full path to the locally saved checkpoints.
+
+## Monitoring Training
+
+Set the Weights & Biases flag to `true` in `train_gym_hil_env_gamepad.json` to follow live training results.
+
+The trained intervention rate should approximately look like this after a while:
+
+
+
+## Configuration Files
+
+- `hi_rl_test_gamepad.json`: Recording configuration
+- `train_gym_hil_env_gamepad.json`: Training configuration
+
+## Key Parameters
+
+- **Save Frequency**: Controlled by `save_freq` in training config
+- **Checkpoint Location**: `outputs/train/YYYY-MM-DD/HH-MM-SS_jobname/checkpoints/`
+- **Policy Path**: Points to `pretrained_model/` directory within checkpoints
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/config.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/config.json
new file mode 100644
index 0000000000..b1806c8fd9
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/config.json
@@ -0,0 +1,185 @@
+{
+ "type": "sac",
+ "n_obs_steps": 1,
+ "normalization_mapping": {
+ "VISUAL": "MEAN_STD",
+ "STATE": "MIN_MAX",
+ "ENV": "MIN_MAX",
+ "ACTION": "MIN_MAX"
+ },
+ "input_features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ }
+ },
+ "output_features": {
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 3
+ ]
+ }
+ },
+ "device": "cuda",
+ "use_amp": false,
+ "push_to_hub": true,
+ "repo_id": "spirosperos/panda-pick-gamepad-policy",
+ "private": null,
+ "tags": null,
+ "license": null,
+ "dataset_stats": {
+ "observation.images.front": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.images.wrist": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.state": {
+ "min": [
+ -0.6897139549255371,
+ -1.1421763896942139,
+ -0.5745007991790771,
+ -2.97829008102417,
+ -0.2710767090320587,
+ 1.3246592283248901,
+ -0.04057434946298599,
+ -0.21261805295944214,
+ -0.4548068344593048,
+ -0.6540042757987976,
+ -0.3644964098930359,
+ -1.1057522296905518,
+ -0.40768879652023315,
+ -0.2220114767551422,
+ 0.0,
+ 0.19176171720027924,
+ -0.3013063669204712,
+ 0.00362197193317115
+ ],
+ "max": [
+ 0.5107022523880005,
+ 0.5516204237937927,
+ 0.5620884299278259,
+ -1.3330878019332886,
+ 0.32758936285972595,
+ 3.119610548019409,
+ 1.8364211320877075,
+ 0.25358933210372925,
+ 0.36316126585006714,
+ 0.14765967428684235,
+ 0.49947625398635864,
+ 0.144814133644104,
+ 0.2820609211921692,
+ 0.7382049560546875,
+ 255.0,
+ 0.6012658476829529,
+ 0.3005995750427246,
+ 0.5004003643989563
+ ]
+ }
+ },
+ "storage_device": "cpu",
+ "vision_encoder_name": "helper2424/resnet10",
+ "freeze_vision_encoder": true,
+ "image_encoder_hidden_dim": 32,
+ "shared_encoder": true,
+ "num_discrete_actions": 3,
+ "image_embedding_pooling_dim": 8,
+ "online_steps": 1000000,
+ "online_env_seed": 10000,
+ "online_buffer_capacity": 100000,
+ "offline_buffer_capacity": 100000,
+ "async_prefetch": false,
+ "online_step_before_learning": 100,
+ "policy_update_freq": 1,
+ "discount": 0.97,
+ "temperature_init": 0.01,
+ "num_critics": 2,
+ "num_subsample_critics": null,
+ "critic_lr": 0.0003,
+ "actor_lr": 0.0003,
+ "temperature_lr": 0.0003,
+ "critic_target_update_weight": 0.005,
+ "utd_ratio": 2,
+ "state_encoder_hidden_dim": 256,
+ "latent_dim": 64,
+ "target_entropy": null,
+ "use_backup_entropy": true,
+ "grad_clip_norm": 10.0,
+ "critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true
+ },
+ "policy_kwargs": {
+ "use_tanh_squash": true,
+ "std_min": 1e-05,
+ "std_max": 5.0,
+ "init_final": 0.05
+ },
+ "discrete_critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_learner_config": {
+ "learner_host": "127.0.0.1",
+ "learner_port": 50051,
+ "policy_parameters_push_frequency": 1,
+ "queue_get_timeout": 2.0
+ },
+ "concurrency": {
+ "actor": "threads",
+ "learner": "threads"
+ },
+ "use_torch_compile": true
+}
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/model.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/model.safetensors
new file mode 100644
index 0000000000..93e1dd0c7f
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a122fcb6fd8ec8d483352e47d007f445505dfebe360724c7835d23700970841
+size 25603220
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/train_config.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/train_config.json
new file mode 100644
index 0000000000..8463c4e8c5
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/pretrained_model/train_config.json
@@ -0,0 +1,402 @@
+{
+ "dataset": {
+ "repo_id": "spirosperos/panda-pick-gamepad",
+ "root": null,
+ "episodes": null,
+ "image_transforms": {
+ "enable": false,
+ "max_num_transforms": 3,
+ "random_order": false,
+ "tfs": {
+ "brightness": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "brightness": [
+ 0.8,
+ 1.2
+ ]
+ }
+ },
+ "contrast": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "contrast": [
+ 0.8,
+ 1.2
+ ]
+ }
+ },
+ "saturation": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "saturation": [
+ 0.5,
+ 1.5
+ ]
+ }
+ },
+ "hue": {
+ "weight": 1.0,
+ "type": "ColorJitter",
+ "kwargs": {
+ "hue": [
+ -0.05,
+ 0.05
+ ]
+ }
+ },
+ "sharpness": {
+ "weight": 1.0,
+ "type": "SharpnessJitter",
+ "kwargs": {
+ "sharpness": [
+ 0.5,
+ 1.5
+ ]
+ }
+ }
+ }
+ },
+ "revision": null,
+ "use_imagenet_stats": false,
+ "video_backend": "pyav"
+ },
+ "env": {
+ "type": "hil",
+ "task": "PandaPickCubeGamepad-v0",
+ "fps": 10,
+ "features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ },
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 3
+ ]
+ }
+ },
+ "features_map": {
+ "observation.images.front": "observation.images.front",
+ "observation.images.wrist": "observation.images.wrist",
+ "observation.state": "observation.state",
+ "action": "action"
+ },
+ "name": "franka_sim",
+ "use_viewer": true,
+ "gripper_penalty": 0.0,
+ "use_gamepad": true,
+ "state_dim": 18,
+ "action_dim": 4,
+ "episode_length": 100,
+ "random_block_position": true,
+ "video_record": {
+ "enabled": false,
+ "record_dir": "videos",
+ "trajectory_name": "trajectory"
+ },
+ "reward_classifier_pretrained_path": null,
+ "robot_config": null,
+ "teleop_config": null,
+ "wrapper": {
+ "control_mode": "gamepad",
+ "display_cameras": false,
+ "add_joint_velocity_to_observation": true,
+ "add_current_to_observation": false,
+ "add_ee_pose_to_observation": true,
+ "crop_params_dict": {
+ "observation.images.front": [
+ 0,
+ 0,
+ 128,
+ 128
+ ],
+ "observation.images.wrist": [
+ 0,
+ 0,
+ 128,
+ 128
+ ]
+ },
+ "resize_size": [
+ 128,
+ 128
+ ],
+ "control_time_s": 30.0,
+ "fixed_reset_joint_positions": [
+ 0.0,
+ 0.195,
+ 0.0,
+ -2.43,
+ 0.0,
+ 2.62,
+ 0.785
+ ],
+ "reset_time_s": 2.0,
+ "use_gripper": true,
+ "gripper_quantization_threshold": 0.8,
+ "gripper_penalty": -0.02,
+ "gripper_penalty_in_reward": false
+ },
+ "mode": null,
+ "repo_id": "spirosperos/panda-pick-gamepad",
+ "dataset_root": null,
+ "num_episodes": 30,
+ "episode": 0,
+ "device": "cuda",
+ "push_to_hub": true,
+ "pretrained_policy_name_or_path": null,
+ "number_of_steps_after_success": 0
+ },
+ "policy": {
+ "type": "sac",
+ "n_obs_steps": 1,
+ "normalization_mapping": {
+ "VISUAL": "MEAN_STD",
+ "STATE": "MIN_MAX",
+ "ENV": "MIN_MAX",
+ "ACTION": "MIN_MAX"
+ },
+ "input_features": {
+ "observation.images.front": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.images.wrist": {
+ "type": "VISUAL",
+ "shape": [
+ 3,
+ 128,
+ 128
+ ]
+ },
+ "observation.state": {
+ "type": "STATE",
+ "shape": [
+ 18
+ ]
+ }
+ },
+ "output_features": {
+ "action": {
+ "type": "ACTION",
+ "shape": [
+ 3
+ ]
+ }
+ },
+ "device": "cuda",
+ "use_amp": false,
+ "push_to_hub": true,
+ "repo_id": "spirosperos/panda-pick-gamepad-policy",
+ "private": null,
+ "tags": null,
+ "license": null,
+ "dataset_stats": {
+ "observation.images.front": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.images.wrist": {
+ "mean": [
+ 0.485,
+ 0.456,
+ 0.406
+ ],
+ "std": [
+ 0.229,
+ 0.224,
+ 0.225
+ ]
+ },
+ "observation.state": {
+ "min": [
+ -0.6897139549255371,
+ -1.1421763896942139,
+ -0.5745007991790771,
+ -2.97829008102417,
+ -0.2710767090320587,
+ 1.3246592283248901,
+ -0.04057434946298599,
+ -0.21261805295944214,
+ -0.4548068344593048,
+ -0.6540042757987976,
+ -0.3644964098930359,
+ -1.1057522296905518,
+ -0.40768879652023315,
+ -0.2220114767551422,
+ 0.0,
+ 0.19176171720027924,
+ -0.3013063669204712,
+ 0.00362197193317115
+ ],
+ "max": [
+ 0.5107022523880005,
+ 0.5516204237937927,
+ 0.5620884299278259,
+ -1.3330878019332886,
+ 0.32758936285972595,
+ 3.119610548019409,
+ 1.8364211320877075,
+ 0.25358933210372925,
+ 0.36316126585006714,
+ 0.14765967428684235,
+ 0.49947625398635864,
+ 0.144814133644104,
+ 0.2820609211921692,
+ 0.7382049560546875,
+ 255.0,
+ 0.6012658476829529,
+ 0.3005995750427246,
+ 0.5004003643989563
+ ]
+ }
+ },
+ "storage_device": "cpu",
+ "vision_encoder_name": "helper2424/resnet10",
+ "freeze_vision_encoder": true,
+ "image_encoder_hidden_dim": 32,
+ "shared_encoder": true,
+ "num_discrete_actions": 3,
+ "image_embedding_pooling_dim": 8,
+ "online_steps": 1000000,
+ "online_env_seed": 10000,
+ "online_buffer_capacity": 100000,
+ "offline_buffer_capacity": 100000,
+ "async_prefetch": false,
+ "online_step_before_learning": 100,
+ "policy_update_freq": 1,
+ "discount": 0.97,
+ "temperature_init": 0.01,
+ "num_critics": 2,
+ "num_subsample_critics": null,
+ "critic_lr": 0.0003,
+ "actor_lr": 0.0003,
+ "temperature_lr": 0.0003,
+ "critic_target_update_weight": 0.005,
+ "utd_ratio": 2,
+ "state_encoder_hidden_dim": 256,
+ "latent_dim": 64,
+ "target_entropy": null,
+ "use_backup_entropy": true,
+ "grad_clip_norm": 10.0,
+ "critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true
+ },
+ "policy_kwargs": {
+ "use_tanh_squash": true,
+ "std_min": 1e-05,
+ "std_max": 5.0,
+ "init_final": 0.05
+ },
+ "discrete_critic_network_kwargs": {
+ "hidden_dims": [
+ 256,
+ 256
+ ],
+ "activate_final": true,
+ "final_activation": null
+ },
+ "actor_learner_config": {
+ "learner_host": "127.0.0.1",
+ "learner_port": 50051,
+ "policy_parameters_push_frequency": 1,
+ "queue_get_timeout": 2.0
+ },
+ "concurrency": {
+ "actor": "threads",
+ "learner": "threads"
+ },
+ "use_torch_compile": true
+ },
+ "output_dir": "outputs/train/2025-08-05/13-45-02_default",
+ "job_name": "default",
+ "resume": false,
+ "seed": 1000,
+ "num_workers": 4,
+ "batch_size": 256,
+ "steps": 100000,
+ "eval_freq": 20000,
+ "log_freq": 1,
+ "save_checkpoint": true,
+ "save_freq": 2000,
+ "use_policy_training_preset": true,
+ "optimizer": {
+ "type": "multi_adam",
+ "lr": 0.001,
+ "weight_decay": 0.0,
+ "grad_clip_norm": 10.0,
+ "optimizer_groups": {
+ "actor": {
+ "lr": 0.0003
+ },
+ "critic": {
+ "lr": 0.0003
+ },
+ "temperature": {
+ "lr": 0.0003
+ }
+ }
+ },
+ "scheduler": null,
+ "eval": {
+ "n_episodes": 50,
+ "batch_size": 50,
+ "use_async_envs": false
+ },
+ "wandb": {
+ "enable": true,
+ "disable_artifact": true,
+ "project": "franka_sim",
+ "entity": null,
+ "notes": null,
+ "run_id": "vcej2ove",
+ "mode": null
+ }
+}
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_param_groups.json
new file mode 100644
index 0000000000..1862ca0432
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_param_groups.json
@@ -0,0 +1,32 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0,
+ 1,
+ 2,
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_state.safetensors
new file mode 100644
index 0000000000..6da5c8f157
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/actor/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2a6161778634fb671d504d2a1ddaab6debe545620c3773c48ddd06400f29e35
+size 945016
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_param_groups.json
new file mode 100644
index 0000000000..fa6e015086
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_param_groups.json
@@ -0,0 +1,90 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0,
+ 1,
+ 2,
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11,
+ 12,
+ 13,
+ 14,
+ 15,
+ 16,
+ 17,
+ 18,
+ 19,
+ 20,
+ 21,
+ 22,
+ 23,
+ 24,
+ 25,
+ 26,
+ 27,
+ 28,
+ 29,
+ 30,
+ 31,
+ 32,
+ 33,
+ 34,
+ 35,
+ 36,
+ 37,
+ 38,
+ 39,
+ 40,
+ 41,
+ 42,
+ 43,
+ 44,
+ 45,
+ 46,
+ 47,
+ 48,
+ 49,
+ 50,
+ 51,
+ 52,
+ 53,
+ 54,
+ 55,
+ 56,
+ 57,
+ 58,
+ 59,
+ 60,
+ 61,
+ 62,
+ 63,
+ 64,
+ 65,
+ 66,
+ 67,
+ 68,
+ 69
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_state.safetensors
new file mode 100644
index 0000000000..39cadf5ca6
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/critic/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdefe9b85637cd0c1d897fea73bf42e8f5d186cd52e22ab1bd48fa53f4657494
+size 7141144
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_param_groups.json
new file mode 100644
index 0000000000..8403db78c1
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_param_groups.json
@@ -0,0 +1,80 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0,
+ 1,
+ 2,
+ 3,
+ 4,
+ 5,
+ 6,
+ 7,
+ 8,
+ 9,
+ 10,
+ 11,
+ 12,
+ 13,
+ 14,
+ 15,
+ 16,
+ 17,
+ 18,
+ 19,
+ 20,
+ 21,
+ 22,
+ 23,
+ 24,
+ 25,
+ 26,
+ 27,
+ 28,
+ 29,
+ 30,
+ 31,
+ 32,
+ 33,
+ 34,
+ 35,
+ 36,
+ 37,
+ 38,
+ 39,
+ 40,
+ 41,
+ 42,
+ 43,
+ 44,
+ 45,
+ 46,
+ 47,
+ 48,
+ 49,
+ 50,
+ 51,
+ 52,
+ 53,
+ 54,
+ 55,
+ 56,
+ 57,
+ 58,
+ 59
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_state.safetensors
new file mode 100644
index 0000000000..4df296bd39
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/discrete_critic/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1cf9c8277e7f2656cd7be71ea3f13d12a8c7373fe68dce54e6bb1e61d3ae4043
+size 6198632
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/rng_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/rng_state.safetensors
new file mode 100644
index 0000000000..7f769dc138
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/rng_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed8d082a4c8ba4930ef223fc726e932ae8e7835a56bc95b97b787fde40b5a0ea
+size 15708
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_param_groups.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_param_groups.json
new file mode 100644
index 0000000000..b93fdca550
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_param_groups.json
@@ -0,0 +1,21 @@
+[
+ {
+ "lr": 0.0003,
+ "betas": [
+ 0.9,
+ 0.999
+ ],
+ "eps": 1e-08,
+ "weight_decay": 0,
+ "amsgrad": false,
+ "maximize": false,
+ "foreach": null,
+ "capturable": false,
+ "differentiable": false,
+ "fused": null,
+ "decoupled_weight_decay": false,
+ "params": [
+ 0
+ ]
+ }
+]
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_state.safetensors b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_state.safetensors
new file mode 100644
index 0000000000..b46bc7a046
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/temperature/optimizer_state.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d007eeb289842c38b424534d18ab905a11ec701421192cced69284545f4d94fe
+size 228
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_state.pt b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_state.pt
new file mode 100644
index 0000000000..37d5b0e963
Binary files /dev/null and b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_state.pt differ
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_step.json b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_step.json
new file mode 100644
index 0000000000..39090bbb98
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/0002000/training_state/training_step.json
@@ -0,0 +1,3 @@
+{
+ "step": 2000
+}
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/last b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/last
new file mode 120000
index 0000000000..ec0e1b9d54
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/checkpoints/last
@@ -0,0 +1 @@
+0002000
\ No newline at end of file
diff --git a/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log
new file mode 100644
index 0000000000..2ecb985a98
--- /dev/null
+++ b/examples/hil_serl_simulation_training/outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log
@@ -0,0 +1,362385 @@
+INFO 2025-08-05 13:45:02 /learner.py:154 Learner logging initialized, writing to outputs/train/2025-08-05/13-45-02_default/logs/learner_default.log
+INFO 2025-08-05 13:45:02 /learner.py:155 {'batch_size': 256,
+ 'dataset': {'episodes': None,
+ 'image_transforms': {'enable': False,
+ 'max_num_transforms': 3,
+ 'random_order': False,
+ 'tfs': {'brightness': {'kwargs': {'brightness': [0.8,
+ 1.2]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'contrast': {'kwargs': {'contrast': [0.8,
+ 1.2]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'hue': {'kwargs': {'hue': [-0.05,
+ 0.05]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'saturation': {'kwargs': {'saturation': [0.5,
+ 1.5]},
+ 'type': 'ColorJitter',
+ 'weight': 1.0},
+ 'sharpness': {'kwargs': {'sharpness': [0.5,
+ 1.5]},
+ 'type': 'SharpnessJitter',
+ 'weight': 1.0}}},
+ 'repo_id': 'spirosperos/panda-pick-gamepad',
+ 'revision': None,
+ 'root': None,
+ 'use_imagenet_stats': False,
+ 'video_backend': 'pyav'},
+ 'env': {'action_dim': 4,
+ 'dataset_root': None,
+ 'device': 'cuda',
+ 'episode': 0,
+ 'episode_length': 100,
+ 'features': {'action': {'shape': [3],
+ 'type': },
+ 'observation.images.front': {'shape': [3, 128, 128],
+ 'type': },
+ 'observation.images.wrist': {'shape': [3, 128, 128],
+ 'type': },
+ 'observation.state': {'shape': [18],
+ 'type': }},
+ 'features_map': {'action': 'action',
+ 'observation.images.front': 'observation.images.front',
+ 'observation.images.wrist': 'observation.images.wrist',
+ 'observation.state': 'observation.state'},
+ 'fps': 10,
+ 'gripper_penalty': 0.0,
+ 'mode': None,
+ 'name': 'franka_sim',
+ 'num_episodes': 30,
+ 'number_of_steps_after_success': 0,
+ 'pretrained_policy_name_or_path': None,
+ 'push_to_hub': True,
+ 'random_block_position': True,
+ 'repo_id': 'spirosperos/panda-pick-gamepad',
+ 'reward_classifier_pretrained_path': None,
+ 'robot_config': None,
+ 'state_dim': 18,
+ 'task': 'PandaPickCubeGamepad-v0',
+ 'teleop_config': None,
+ 'type': 'hil',
+ 'use_gamepad': True,
+ 'use_viewer': True,
+ 'video_record': {'enabled': False,
+ 'record_dir': 'videos',
+ 'trajectory_name': 'trajectory'},
+ 'wrapper': {'add_current_to_observation': False,
+ 'add_ee_pose_to_observation': True,
+ 'add_joint_velocity_to_observation': True,
+ 'control_mode': 'gamepad',
+ 'control_time_s': 30.0,
+ 'crop_params_dict': {'observation.images.front': [0,
+ 0,
+ 128,
+ 128],
+ 'observation.images.wrist': [0,
+ 0,
+ 128,
+ 128]},
+ 'display_cameras': False,
+ 'fixed_reset_joint_positions': [0.0,
+ 0.195,
+ 0.0,
+ -2.43,
+ 0.0,
+ 2.62,
+ 0.785],
+ 'gripper_penalty': -0.02,
+ 'gripper_penalty_in_reward': False,
+ 'gripper_quantization_threshold': 0.8,
+ 'reset_time_s': 2.0,
+ 'resize_size': [128, 128],
+ 'use_gripper': True}},
+ 'eval': {'batch_size': 50, 'n_episodes': 50, 'use_async_envs': False},
+ 'eval_freq': 20000,
+ 'job_name': 'default',
+ 'log_freq': 1,
+ 'num_workers': 4,
+ 'optimizer': {'grad_clip_norm': 10.0,
+ 'lr': 0.001,
+ 'optimizer_groups': {'actor': {'lr': 0.0003},
+ 'critic': {'lr': 0.0003},
+ 'temperature': {'lr': 0.0003}},
+ 'type': 'multi_adam',
+ 'weight_decay': 0.0},
+ 'output_dir': 'outputs/train/2025-08-05/13-45-02_default',
+ 'policy': {'actor_learner_config': {'learner_host': '127.0.0.1',
+ 'learner_port': 50051,
+ 'policy_parameters_push_frequency': 1,
+ 'queue_get_timeout': 2.0},
+ 'actor_lr': 0.0003,
+ 'actor_network_kwargs': {'activate_final': True,
+ 'hidden_dims': [256, 256]},
+ 'async_prefetch': False,
+ 'concurrency': {'actor': 'threads', 'learner': 'threads'},
+ 'critic_lr': 0.0003,
+ 'critic_network_kwargs': {'activate_final': True,
+ 'final_activation': None,
+ 'hidden_dims': [256, 256]},
+ 'critic_target_update_weight': 0.005,
+ 'dataset_stats': {'observation.images.front': {'mean': [0.485,
+ 0.456,
+ 0.406],
+ 'std': [0.229,
+ 0.224,
+ 0.225]},
+ 'observation.images.wrist': {'mean': [0.485,
+ 0.456,
+ 0.406],
+ 'std': [0.229,
+ 0.224,
+ 0.225]},
+ 'observation.state': {'max': [0.5107022523880005,
+ 0.5516204237937927,
+ 0.5620884299278259,
+ -1.3330878019332886,
+ 0.32758936285972595,
+ 3.119610548019409,
+ 1.8364211320877075,
+ 0.25358933210372925,
+ 0.36316126585006714,
+ 0.14765967428684235,
+ 0.49947625398635864,
+ 0.144814133644104,
+ 0.2820609211921692,
+ 0.7382049560546875,
+ 255.0,
+ 0.6012658476829529,
+ 0.3005995750427246,
+ 0.5004003643989563],
+ 'min': [-0.6897139549255371,
+ -1.1421763896942139,
+ -0.5745007991790771,
+ -2.97829008102417,
+ -0.2710767090320587,
+ 1.3246592283248901,
+ -0.04057434946298599,
+ -0.21261805295944214,
+ -0.4548068344593048,
+ -0.6540042757987976,
+ -0.3644964098930359,
+ -1.1057522296905518,
+ -0.40768879652023315,
+ -0.2220114767551422,
+ 0.0,
+ 0.19176171720027924,
+ -0.3013063669204712,
+ 0.00362197193317115]}},
+ 'device': 'cuda',
+ 'discount': 0.97,
+ 'discrete_critic_network_kwargs': {'activate_final': True,
+ 'final_activation': None,
+ 'hidden_dims': [256, 256]},
+ 'freeze_vision_encoder': True,
+ 'grad_clip_norm': 10.0,
+ 'image_embedding_pooling_dim': 8,
+ 'image_encoder_hidden_dim': 32,
+ 'input_features': {'observation.images.front': {'shape': [3,
+ 128,
+ 128],
+ 'type': },
+ 'observation.images.wrist': {'shape': [3,
+ 128,
+ 128],
+ 'type': },
+ 'observation.state': {'shape': [18],
+ 'type': }},
+ 'latent_dim': 64,
+ 'license': None,
+ 'n_obs_steps': 1,
+ 'normalization_mapping': {'ACTION': ,
+ 'ENV': ,
+ 'STATE': ,
+ 'VISUAL': },
+ 'num_critics': 2,
+ 'num_discrete_actions': 3,
+ 'num_subsample_critics': None,
+ 'offline_buffer_capacity': 100000,
+ 'online_buffer_capacity': 100000,
+ 'online_env_seed': 10000,
+ 'online_step_before_learning': 100,
+ 'online_steps': 1000000,
+ 'output_features': {'action': {'shape': [3],
+ 'type': }},
+ 'policy_kwargs': {'init_final': 0.05,
+ 'std_max': 5.0,
+ 'std_min': 1e-05,
+ 'use_tanh_squash': True},
+ 'policy_update_freq': 1,
+ 'private': None,
+ 'push_to_hub': True,
+ 'repo_id': 'spirosperos/panda-pick-gamepad-policy',
+ 'shared_encoder': True,
+ 'state_encoder_hidden_dim': 256,
+ 'storage_device': 'cpu',
+ 'tags': None,
+ 'target_entropy': None,
+ 'temperature_init': 0.01,
+ 'temperature_lr': 0.0003,
+ 'type': 'sac',
+ 'use_amp': False,
+ 'use_backup_entropy': True,
+ 'use_torch_compile': True,
+ 'utd_ratio': 2,
+ 'vision_encoder_name': 'helper2424/resnet10'},
+ 'resume': False,
+ 'save_checkpoint': True,
+ 'save_freq': 2000,
+ 'scheduler': None,
+ 'seed': 1000,
+ 'steps': 100000,
+ 'use_policy_training_preset': True,
+ 'wandb': {'disable_artifact': True,
+ 'enable': True,
+ 'entity': None,
+ 'mode': None,
+ 'notes': None,
+ 'project': 'franka_sim',
+ 'run_id': None}}
+DEBUG 2025-08-05 13:45:02 git/cmd.py:1270 Popen(['git', 'version'], cwd=/home/nikola/lerobot, stdin=None, shell=False, universal_newlines=False)
+DEBUG 2025-08-05 13:45:02 git/cmd.py:1270 Popen(['git', 'version'], cwd=/home/nikola/lerobot, stdin=None, shell=False, universal_newlines=False)
+DEBUG 2025-08-05 13:45:02 git/util.py:494 sys.platform='linux', git_executable='git'
+DEBUG 2025-08-05 13:45:02 git/cmd.py:1270 Popen(['git', 'rev-parse', '--show-toplevel'], cwd=/home/nikola/lerobot, stdin=None, shell=False, universal_newlines=False)
+DEBUG 2025-08-05 13:45:02 git/util.py:494 sys.platform='linux', git_executable='git'
+DEBUG 2025-08-05 13:45:02 git/cmd.py:1270 Popen(['git', 'cat-file', '--batch-check'], cwd=/home/nikola/lerobot, stdin=, shell=False, universal_newlines=False)
+INFO 2025-08-05 13:45:03 db_utils.py:103 Track this run --> [1m[33mhttps://wandb.ai/spes-robotics/franka_sim/runs/vcej2ove[0m
+INFO 2025-08-05 13:45:03 /learner.py:313 Initializing policy
+WARNING 2025-08-05 13:45:03 /factory.py:153 You are instantiating a policy from scratch and its features are parsed from an environment rather than a dataset. Normalization modules inside the policy will have infinite values by default without stats from a dataset.
+INFO 2025-08-05 13:45:03 /learner.py:692 [LEARNER] gRPC server started
+DEBUG 2025-08-05 13:45:03 ionpool.py:1049 Starting new HTTPS connection (1): huggingface.co:443
+DEBUG 2025-08-05 13:45:04 tionpool.py:544 https://huggingface.co:443 "HEAD /helper2424/resnet10/resolve/main/config.json HTTP/1.1" 307 0
+DEBUG 2025-08-05 13:45:04 tionpool.py:544 https://huggingface.co:443 "HEAD /api/resolve-cache/models/helper2424/resnet10/e16ff8c2c9fe2130c0e3eabfd142f131e59be97f/config.json HTTP/1.1" 200 0
+DEBUG 2025-08-05 13:45:04 tionpool.py:544 https://huggingface.co:443 "HEAD /helper2424/resnet10/resolve/main/configuration_resnet.py HTTP/1.1" 307 0
+DEBUG 2025-08-05 13:45:04 tionpool.py:544 https://huggingface.co:443 "HEAD /api/resolve-cache/models/helper2424/resnet10/e16ff8c2c9fe2130c0e3eabfd142f131e59be97f/configuration_resnet.py HTTP/1.1" 200 0
+DEBUG 2025-08-05 13:45:04 tionpool.py:544 https://huggingface.co:443 "HEAD /helper2424/resnet10/resolve/main/modeling_resnet.py HTTP/1.1" 307 0
+DEBUG 2025-08-05 13:45:04 tionpool.py:544 https://huggingface.co:443 "HEAD /api/resolve-cache/models/helper2424/resnet10/e16ff8c2c9fe2130c0e3eabfd142f131e59be97f/modeling_resnet.py HTTP/1.1" 200 0
+INFO 2025-08-05 13:45:05 /learner.py:323 ================================================================================
+INFO 2025-08-05 13:45:05 /learner.py:324 POLICY CONFIGURATION DEBUG INFO:
+INFO 2025-08-05 13:45:05 /learner.py:325 ================================================================================
+INFO 2025-08-05 13:45:05 /learner.py:326 Policy type: SACPolicy
+INFO 2025-08-05 13:45:05 /learner.py:327 Input features: {'observation.images.front': PolicyFeature(type=, shape=(3, 128, 128)), 'observation.images.wrist': PolicyFeature(type=, shape=(3, 128, 128)), 'observation.state': PolicyFeature(type=, shape=(18,))}
+INFO 2025-08-05 13:45:05 /learner.py:328 Output features: {'action': PolicyFeature(type=