The big three (logs, metrics, and traces) are insufficient for the Monty use case. We also need a fourth, what I’ve been calling “snapshots”. I wasn’t able to find a good industry standard name for these, they depend on context: buffers, frames, snapshots, scrapes, etc. It’s the things that would allow you to see the live visualization. Which reminds me, have you run an experiment with experiment.config.show_sensor_output=true?
python run.py experiment=base_config_10distinctobj_dist_agent experiment.config.show_sensor_output=true
What I’m calling “snapshot” is the the things that go into “Camera image”, “Sensor depth image”, and “MLH”. They’re not logs, metrics, or traces.
And no, I’m not saying we should base64 encode those in a structured log 
. What I am saying is that in the telemetry pipeline, references to those snapshots should be available for a handler to do a live visualization of the kind the live plotter is doing.