GPU Fleet FinOps: Scheduling, Right-Sizing, and Cost Governance for DGX, MIG, and Preemptible Capacity

Abstract

As organizations increasingly scale Generative AI (GenAI) and Large Language Model (LLM) workloads, GPU-accelerated computing has become the dominant line item in cloud expenditure ¹,.² This paper presents a "GPU Fleet FinOps" blueprint, a unified operating model for the financial and operational optimization of large-scale GPU fleets, including NVIDIA DGX systems, MIG-partitioned GPUs, and preemptible (spot) capacity. We identify critical, unaddressed challenges: chronic low utilization of premium hardware ³, ⁴, ⁵, "capacity island" fragmentation from Multi-Instance GPU (MIG) ⁶, ⁷, ⁸, ⁹, the high failure rate of workloads on preemptible instances ¹⁰, ¹¹, ¹², and a lack of financial accountability. We propose an integrated solution built on three pillars: a FinOps-aware GPU scheduling layer ¹³, policy-driven right-sizing with quota management ¹⁴, ¹⁵, and robust, interruption-aware job design ¹⁶,.¹⁷ This framework connects low-level scheduling and governance decisions to business-centric unit economics, such as "cost per training run" and "cost per 1k inferences" ¹⁸, ¹⁹, providing a practical architecture for aligning high-performance GPU investments with measurable business value