GPUs & TPUs
Exam guide§2.1
Accelerators for ML and compute-heavy workloads. The exam mostly tests which one and the attachment/availability caveats.
When GPU vs. TPU
- GPU - general-purpose acceleration: ML training/inference, video transcoding, HPC, rendering, CUDA workloads. Flexible across frameworks.
- TPU - Google's custom ASIC, purpose-built for large-scale TensorFlow/JAX matrix math. Best for very large training jobs on supported frameworks.
CUDA, PyTorch flexibility, mixed workloads, rendering→GPU
Video transcoding, HPC, small/varied models→GPU
Huge TensorFlow/JAX training at scale→TPU
- Google introduced the TPU in 2016; it is a custom ASIC (domain-specific hardware, vs general-purpose CPUs/GPUs).
- TPUs are generally faster and more energy-efficient than current GPUs/CPUs for ML - they tailor the architecture to ML math like matrix multiplication.
- Best fit: models that train for long durations and large models with large effective batch sizes.
Attaching GPUs to VMs
- Add GPUs to Compute Engine VMs on supported machine types (N1 with attached GPUs, or accelerator-optimized A2/A3/G2).
- Install the NVIDIA driver (startup script or DL image).
- GPU VMs must set on-host-maintenance = TERMINATE (they cannot live migrate).
- Also available on GKE node pools and (some) Cloud Run for serverless inference.
A VM with a GPU cannot use live migration - its availability policy must be TERMINATE (with automatic restart). Combined with GPU quota + regional availability, this is the common "why won't my GPU VM start" trap.
- GPUs/TPUs require explicit quota in the region/zone before you can create them.
- They exist only in specific zones - not every region has every accelerator.
- Spot GPUs are cheaper but reclaimable; commit to reserved capacity for guaranteed availability.