Track reward and other parameters value during each step

[anaskn]

Thank you for doing a great job on the Omnisafe framework! I'm using Omnisafe with a customized environment I created based on "omnisafe/tests/simple_env.py". I'm wondering if there's an easy way to track the reward throughout each episode to see how the learning is progressing. Additionally, I'd like to track other parameter values my custom environment generates during each step.

[Gaiejj]

Thank you for your affirmation of OmniSafe. In fact, OmniSafe utilizes the EpochLogger from omnisafe/common/logger to record data for each epoch. If you wish to display episode-wise data, you simply need to set the steps_per_epoch parameter to the number of steps per episode. Afterwards, you can view the data (as you mentioned, reward) for each episode in both the terminal and the progress.csv file.

Currently, OmniSafe does not support recording data for each individual step. This is because in the reinforcement learning paradigm, training often involves a large number of steps, reaching up to 1e6 steps or more, and recording data for each step becomes extremely challenging. If you could provide a detailed example regarding "tracking other parameter values my custom environment generates during each step," we can offer better guidance on how to adjust the logger to record such data.