Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Computing and CUDA Architecture

• Architectural distinctions between CPUs and GPUs
• NVIDIA GPU streaming multiprocessor model explained
• Overview of the CUDA programming model
• Heterogeneous computing concepts and the host-device paradigm

Establishing the CUDA Development Environment

• Installation procedures for CUDA Toolkit 13.x
• NVCC compiler functionality and build workflows
• Environment verification through device queries
• IDE integration and essential development tools

Writing and Launching CUDA Kernels

• Syntax and qualifiers for kernel functions
• Launch configuration mechanics and execution processes
• Vector addition examples and fundamental data-parallel patterns
• CUDA error checking macros implementation

CUDA Thread Hierarchy and Execution Model

• Organization of grids, blocks, and threads
• Thread indexing and global ID calculation methods
• Warp execution dynamics and the SIMT model
• Occupancy levels and resource utilization metrics

GPU Memory Architecture and Management

• Memory types: global, shared, constant, and registers
• Allocating and deallocating device memory
• Host-to-device and device-to-host data transfers
• Leveraging shared memory for intra-block collaboration

Unified Memory and Data Migration

• Unified memory model and managed allocations
• Page migration mechanisms and on-demand paging
• Asynchronous prefetching using cudaMemPrefetchAsync
• Memory advice hints for optimizing access patterns

System-Wide Profiling with Nsight Systems

• Analyzing timelines within Nsight Systems
• Identifying CPU-GPU synchronization points
• Visualizing kernel execution and memory transfer events
• Interpreting system-level performance data

Kernel Optimization with Nsight Compute

• Interactive kernel profiling using Nsight Compute
• Analyzing memory throughput and bandwidth
• Evaluating compute utilization and warp state statistics
• Guided analysis techniques and optimization rules

Concurrent Streams and Asynchronous Operations

• Understanding CUDA streams and the default stream behavior
• Overlapping kernel execution with data transfers
• Stream synchronization mechanisms and CUDA events
• Multi-stream pipeline design patterns

Error Handling and Debugging Tools

• CUDA API error codes and recovery strategies
• Using compute-sanitizer for memory access validation
• Kernel debugging with cuda-gdb
• Assertions and synchronous error detection methods

Profile-Driven Optimization Workflow

• Iterative profiling methodology adoption
• Bottleneck identification and prioritization strategies
• Performance regression testing procedures
• Documentation of optimization decisions

End-to-End Accelerated Application Project

• Designing a comprehensive GPU-accelerated solution
• Integrating profiling throughout the development lifecycle
• Performance benchmarking and reporting practices
• Production deployment considerations