To add a new policy to the backend, create a new file phosphobot/am/your_policy.py. You must implement two main components:
- Config Validators: Define dataclasses inheriting from
HuggingFaceModelValidator, HuggingFaceAugmentedValidator, and LeRobotSpawnConfig to validate model configurations and spawn settings. - Policy Class: Create a class inheriting from
LeRobot. This class must implement class methods to return your specific validator and spawn config classes, and an _prepare_model_inputs method to handle model-specific preprocessing (e.g., adding prompts for SmolVLA or detection instructions for ACT).
Example implementation structure:
class YourPolicyHuggingFaceModelValidator(HuggingFaceModelValidator):
type: Literal["model_type"]
class YourPolicyHuggingFaceAugmentedValidator(HuggingFaceAugmentedValidator):
type: Literal["model_type"]
class YourPolicySpawnConfig(LeRobotSpawnConfig):
hf_model_config: YourPolicyHuggingFaceAugmentedValidator # type: ignore[assignment]
class YourPolicy(LeRobot):
@classmethod
def _get_model_validator_class(cls) -> type:
return YourPolicyHuggingFaceModelValidator
@classmethod
def _get_augmented_validator_class(cls) -> type:
return YourPolicyHuggingFaceAugmentedValidator
@classmethod
def _get_spawn_config_class(cls) -> type:
return YourPolicySpawnConfig
def _prepare_model_inputs(self, config, state, image_inputs) -> Dict[str, np.ndarray | str]:
inputs: Dict[str, np.ndarray | str] = {
config.input_features.state_key: state,
**image_inputs,
}
# Add model-specific input processing here (e.g., prompts or custom preprocessing)
return inputs
