Vulkan Memory Allocation Limits and VMA Sub-allocation Strategy

Direct answer

Separate lifetime classes into distinct VMA pools and size blocks from device limits and observed allocation sizes. Verify compliance by querying maxMemoryAllocationCount at startup and tracking the peak number of VkDeviceMemory objects created by VMA per pool on each target GPU.

Confirmed facts

• VkPhysicalDeviceLimits::maxMemoryAllocationCount is a device limit returned by vkGetPhysicalDeviceProperties. The total number of simultaneous VkDeviceMemory objects must stay below it.
• VMA supports custom pools via VmaPoolCreateInfo and VmaAllocatorCreateInfo. Allocations from a pool are sub-allocated from larger VkDeviceMemory blocks created by VMA.
• Alignment and granularity constraints come from VkPhysicalDeviceLimits: bufferImageGranularity, minMemoryMapAlignment, and nonCoherentAtomSize. Block sizes should be multiples of these values.

Likely explanation

Mixing high-frequency short-lived buffers with long-term static images in one pool causes external fragmentation. Short-lived frees create holes that cannot be reused efficiently by large, long-lived images, and image alignment requirements can split blocks. Separating lifetimes lets each pool coalesce and reuse memory within its own churn pattern.

Pool configuration for mixed workloads

1. Create two pools in the same allocator:
   • Transient buffer pool: for high-frequency short-lived buffers. Use smaller initial block size, e.g., 4–16 MiB, and allow VMA to grow the pool. Prefer VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT or default buddy for fast reuse.
   • Static image pool: for long-term static images. Use larger blocks, e.g., 64–256 MiB, sized to typical image footprints. Keep this pool stable to preserve alignment.
2. Size blocks from observed distribution and device limits. Choose block sizes that are multiples of bufferImageGranularity and minMemoryMapAlignment to avoid internal waste.
3. Assign allocation flags consistently: transient buffers often use VMA_MEMORY_USAGE_CPU_TO_GPU or GPU_ONLY with host visible, static images use VMA_MEMORY_USAGE_GPU_ONLY with device local.
4. Disable defragmentation for the static pool unless testing shows benefit; enable opportunistic defragmentation only for the transient pool.

Verifying maxMemoryAllocationCount across vendors

1. At startup query limits: vkGetPhysicalDeviceProperties → limits.maxMemoryAllocationCount. Log per vendor.
2. Enable VMA statistics and record peak VkDeviceMemory count per pool during a representative peak workload. VMA stats expose number of allocations per pool.
3. Sum peaks across all pools and compare to the queried limit with margin, e.g., keep ≤ 80% of limit.
4. Repeat on each target GPU vendor and driver version. Adjust block size up to reduce allocation count, or reduce pool count, if peak approaches the limit.

Assumption: you are using VMA with explicit pools, not default per-allocation. If you are using dedicated allocations for images, the recommendation changes.

One diagnostic that changes the recommendation: the typical allocation size distribution and peak concurrent allocation count per class on your target GPUs. Without that, block sizing remains an estimate.