Trading Compute for Memory: Using torch.utils.checkpoint in PyTorch
Stop hitting CUDA out-of-memory errors. Learn how to use torch.utils.checkpoint to trade a bit of compute time for significant GPU memory savings in deep PyTorch models.
Loading…
ReadMeFeed / Community knowledge
#gpu-memory
Real questions. Useful conversations. Find the people who know your stack.
pytorchResource exhausted OOM - GPU BFC allocator
GPU Memory Management in TensorFlow TensorFlow utilizes a Best-Fit with Coalescing (BFC) allocator to manage GPU memory. By default, the runtime preallocates a large fraction of the visible GPU memory to reduce fragmentation and improve performance during tensor operations. This preallocation strategy can lead to resource exhaustion errors when the BFC alloc
Gradient Accumulation Memory Limits and Batch Normalization Parity
Gradient Accumulation Behavior PyTorch supports simulating larger batch sizes by summing gradients over multiple forward and backward passes before executing an optimizer step. This technique is intended to keep memory consumption proportional to the micro-batch size rather than the effective total batch size. While the .grad attribute accumulates values acr
WebGPU Pipeline Layout and Storage Buffer Alignment Constraints
WebGPU utilizes a pipeline layout system to decouple resource definitions from specific pipeline states, allowing for more efficient switching between pipelines that share the same bind group layouts. When utilizing storage buffers for read-write access in compute shaders, the API enforces strict alignment requirements. While minUniformBufferOffsetAlignment
webgpuBufferGeometry attribute usage and GPU memory overhead
Optimizing GPU memory in Three.js requires a balance between vertex precision and resource allocation. When utilizing BufferGeometry , developers can define custom attributes using typed arrays to manage how vertex data is stored and transmitted to the GPU. The setUsage() method allows for hinting whether an attribute is static or dynamic, which informs the
threejsVK_ERROR_OUT_OF_DEVICE_MEMORY during memory heap allocation
An application utilizing the Vulkan API manages memory by requesting specific heap types based on the VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT flag. While the allocation logic functions correctly on high-capacity development hardware, it fails on production devices with smaller VRAM footprints or different memory architectures. The current implementation relies o
vulkan