You can monitor many metrics for each of your pipelines and blocks. Soon, you’ll be able to monitor aggregate metrics across all pipelines and blocks.
Here are the following metrics that are monitored for each pipeline and block:
- Pipeline runs completed successfully
- Pipeline runs failed
- Block runs completed successfully
- Block runs failed
- How long each block ran for
- CPU usage for each block run (coming soon)
- Memory usage for each block run (coming soon)
- Data validation success for each block (coming soon)
- Data validation failure for each block (coming soon)