Speaker
Description
Containers have become a robust alternative to traditional environment modules for managing AI workloads on High-Performance Computing (HPC) systems. Optimized solutions, such as the NVIDIA vLLM container from the NVIDIA GPU Cloud (NGC) catalog, offer a streamlined approach by providing pre-configured environments including CUDA, PyTorch, and the NVIDIA Collective Communications Library (NCCL) for optimized multi-node communication.
While these images often work seamlessly out of the box, researchers frequently encounter scenarios where additional software dependencies must be integrated into a read-only container image. Modifying large base images for minor software additions can be computationally expensive and difficult to manage. This presentation discusses the use of overlays as an efficient solution to this challenge. We will demonstrate how overlays allow users to persist changes and add custom packages to existing containers without the need for full image rebuilds, thereby maintaining environment portability while providing the flexibility required for specialized research workflows.