When you’re writing CUDA applications, one of the most important things you need to focus on to write great code is data transfer performance. This applies to both single-GPU and multi-GPU systems alike. One of the tools you can use to understand the memory characteristics of your GPU system is NVIDIA NVbandwidth.
In this blog post, we’ll explore what NVbandwidth is, how it works, its key features, and how you can use it to test and evaluate your own NVIDIA GPU systems. This post is intended for CUDA developers, system architects, and ML infrastructure engineers who need to measure and validate GPU interconnect performance.
What is NVbandwidth?
NVbandwidth is a CUDA-based tool that measures bandwidth and latency for various memory copy patterns across different links using either copy engine (CE) or kernel copy methods. It reports the current measured bandwidth on your system, providing valuable insights into the performance characteristics of your GPU setup. While modern GPUs boast impressive compute capabilities, their performance is frequently limited by how quickly data can be moved between different devices:
- CPU memory to GPU memory
- GPU memory to CPU memory
- GPU memory to GPU memory
Understanding these performance characteristics helps developers:
- Evaluate system performance
- Measure memory access latency
- Measure bandwidth in single and multi-node GPU deployments
- Understand the performance implications of different memory transfer patterns
- Diagnose bandwidth bottlenecks in CUDA applications
- Optimize memory transfer patterns for specific workloads
- Compare bandwidth and latency across multiple GPUs in a system
- Performance monitoring and validation
Previous
