HPC stats scripts
Utilities for summarizing CPU, GPU, memory, and efficiency for SLURM and PBS scheduled jobs.
Scientific problem
Requested HPC resources can differ substantially from actual usage, while scheduler accounting output is cumbersome to inspect across many jobs.
Motivation
The scripts were developed to make resource utilization easy to review when profiling and sizing computational workloads.
My role
- Developed the monitoring and reporting utilities.
Methodology
- Parse SLURM and PBS accounting information.
- Summarize CPU, GPU, memory, runtime, and resource-efficiency metrics.
Results
- Provides compact utilization summaries for profiling HPC jobs and adjusting future resource requests.
Software
- Nvidia-ml-py3
- NumPY
- Matplotlib
- Python
- Bash
- SLURM
- PBS
- GPU