Interested in contributing: Add More Wandb Logging for Unsupervised Learning

Hi @haoyucn, welcome and thank you for the help!

I don’t think this warrants an RFC. You can probably just do this via a direct pull request.

Regarding what metrics to add, @vclay pointed out that a good place to start would be adding the metrics that we report in our unsupervised learning benchmarks: Benchmark Experiments

From the coding perspective, I think this will require reading additional properties from data["BASIC"] and or data["DETAILED"] inside the WandB handlers in wandb_handlers.py, but I haven’t looked at it in detail, so perhaps more complexity will surface as you work on it.

One consideration I would keep in mind is to see if these can be added without adding any, or as little as possible, new methods elsewhere in Monty. The reason being is that I hope we can move away from this logging path next quarter (26Q3) toward the approach described in 333 Telemetry collection is configurable down to the Python module level.

Let us know if you have further questions.

Cheers!

4 Likes