Architecting a Vulkan Sub-allocation System to Avoid Allocation Limits

The Problem: The vkAllocateMemory Bottleneck

Vulkan drivers impose a hard limit on the total number of active memory allocations, defined by maxMemoryAllocationCount in the VkPhysicalDeviceMemoryProperties structure. For many GPUs, this limit is surprisingly low (often 4,096). If an application creates a separate vkAllocateMemory call for every buffer and image, it will quickly hit this ceiling, resulting in VK_ERROR_OUT_OF_DEVICE_MEMORY even when gigabytes of physical VRAM remain available.

The solution is a sub-allocation strategy: requesting a few large blocks of VkDeviceMemory from the driver and manually carving them into smaller slices for individual resources.

The Smallest Suitable Design

A minimal sub-allocator requires a manager that tracks large memory blocks (chunks) and the offsets within those blocks currently assigned to resources. The architecture consists of three primary components:

• The Block Manager: Handles the actual vkAllocateMemory calls. It requests blocks of a fixed large size (e.g., 128MB or 256MB) to minimize driver overhead.
• The Offset Tracker: A data structure (such as a linked list of free blocks or a buddy allocator) that manages the internal address space of each VkDeviceMemory block.
• The Resource Mapping: A lookup table that associates a VkBuffer or VkImage with its parent VkDeviceMemory handle and its specific VkDeviceSize offset.

Implementation Example: Resource Binding

When creating a buffer, the application must first query the memory requirements to ensure the sub-allocated slice is compatible with the hardware.

// 1. Create the buffer object
vkCreateBuffer(device, &bufferInfo, nullptr, &buffer);

// 2. Query requirements (Alignment and Memory Type)
VkMemoryRequirements memReqs;
vkGetBufferMemoryRequirements(device, buffer, &memReqs);

// 3. Request a slice from the sub-allocator
// The allocator must ensure the returned offset is a multiple of memReqs.alignment
Allocation slice = memoryManager.allocate(memReqs.size, memReqs.alignment, memReqs.memoryTypeBits);

// 4. Bind the buffer to the specific offset within the large block
vkBindBufferMemory(device, buffer, slice.deviceMemory, slice.offset);

Trust and Data Boundaries

The primary trust boundary exists between the Application Resource Request and the GPU Hardware Constraints. The sub-allocator must act as a strict validator for the following:

• Alignment: Every offset must satisfy the alignment field of VkMemoryRequirements. Failure to do so results in undefined behavior or hardware faults.
• Non-Coherent Atom Size: For memory that is HOST_VISIBLE but not HOST_COHERENT, the allocator must align offsets to the nonCoherentAtomSize to prevent data corruption during vkFlushMappedMemoryRanges.
• Memory Type Indices: The allocator must only provide memory from a heap that matches the memoryTypeBits bitmask provided by the driver for that specific resource.

Operational Checks and Diagnostics

Because memory is managed manually, the driver cannot report per-resource leaks. The following checks are required for stability:

Metric	Check Method	Failure Indicator
Allocation Count	Track total vkAllocateMemory calls.	Approaching maxMemoryAllocationCount.
Fragmentation	Compare total free space vs. largest contiguous block.	Unable to allocate a large resource despite having enough total free bytes.
Heap Utilization	Compare allocated bytes against VkPhysicalDeviceMemoryProperties.	VK_ERROR_OUT_OF_DEVICE_MEMORY.

Failure Modes

• Heap Exhaustion: When vkAllocateMemory fails, the system must either attempt to defragment existing blocks (by moving resources, which requires updating descriptors) or return a fatal error to the application.
• Device Loss: If a VK_ERROR_DEVICE_LOST occurs during an asynchronous transfer to a sub-allocated region, all associated VkDeviceMemory blocks must be treated as invalid.
• BAR Window Overflow: Over-allocating HOST_VISIBLE memory can exhaust the PCIe Base Address Register (BAR) window on some hardware, even if the GPU has plenty of VRAM. This requires a separate pool for host-visible memory with a smaller total cap.

Conditions for Design Change

This sub-allocation architecture is suitable for standard static and dynamic resources. However, the design must be replaced or augmented if:

1. Sparse Binding is Required: If the application uses VK_BUFFER_CREATE_SPARSE_BINDING_BIT, the manual offset management is replaced by the Vulkan Sparse Binding API, which allows virtual memory to be mapped to physical pages non-contiguously.
2. Massive Single Resources: If a single resource (e.g., a 4K texture array) exceeds the size of a manageable block, the allocator must support "dedicated allocations" where vkAllocateMemory is called specifically for that one resource.

Verification and Rollback

To verify the implementation, run the application with Vulkan Validation Layers enabled. The layers will trigger an error if vkBindBufferMemory is called with an offset that violates alignment requirements.

Rollback: Since this system manages state, any failure during a block allocation should trigger a cleanup of all previously allocated VkDeviceMemory handles in reverse order of creation to prevent driver-level memory leaks.