Hi everyone!
Sharing another companion tool for Monty: tbp.teleop. It lets you watch a training or inference experiment run live, and if you want, take over and drive the sensor around the object yourself.
tbp.plot is for visualizing the logs after a run. This one is for watching the run happen live and controlling the agent actions. It is attached as an experiment step hook. Two hooks are provided as a Hydra hooks config group:
monitor: just watch and set the experiment speed.interactive: watch and teleoperate, choosing the agent’s action at each step.
Add one to your experiment’s defaults (/hooks: [monitor|interactive]), or pick it at run time as a Hydra override:
python run.py -cd src/tbp/teleop/conf experiment=example +hooks=interactive
# or +hooks=monitor
In monitor mode there’s a speed slider. In interactive mode you get a little D-pad around the patch instead (WASD or arrow keys for the four headings, space to jump when the model offers one, delete to end the episode), plus a slider to control the action step size.
Give it a try and let me know what you think. Ideas for improvements and PRs are very welcome!