Vulkan Memory Choice: Device-Local vs Host-Visible Buffer Updates

Decision and constraints

Vulkan requires explicit memory management. When you create a buffer or image, you must select a VkMemoryType from the physical device and bind it with vkBindBufferMemory. Each memory type exposes VkMemoryPropertyFlags that determine whether the CPU can map it and whether the GPU can access it efficiently.

The decision depends on three constraints:

• Update frequency: per-frame dynamic data versus static assets loaded once.
• Hardware class: discrete GPUs use separate VRAM across PCIe; integrated GPUs share system RAM.
• Alignment rules: host-visible memory must respect nonCoherentAtomSize and buffer offset alignments.

Supported memory options

Memory class	Typical flags	CPU access	GPU bandwidth	Best for
Device-local	VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT	Usually not host visible	Highest, local VRAM bandwidth	Static or rarely updated resources
Host-visible	VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT, often with HOST_COHERENT_BIT	Direct map via vkMapMemory	Lower, PCIe or shared bus	Frequent small CPU writes, readback
Host-cached	HOST_VISIBLE + HOST_CACHED	Direct map with CPU cache	Variable, possible GPU stalls	Streaming data with CPU prefetch

On integrated architectures, a single heap can report both DEVICE_LOCAL_BIT and HOST_VISIBLE_BIT. In that case, an explicit copy to device-local memory is often unnecessary.

Trade-offs for high-performance updates

Device-local memory gives the GPU the fastest reads because data lives in VRAM. The cost is that the CPU usually cannot write to it directly. You must use a staging buffer: a temporary host-visible buffer, then vkCmdCopyBuffer to move data into the device-local destination.

Host-visible memory avoids the copy and simplifies the code path. The penalty is that GPU reads travel over PCIe on discrete systems, which becomes a bottleneck for large vertex or storage buffers. If the memory lacks HOST_COHERENT_BIT, you must also call vkFlushMappedMemoryRanges after CPU writes.

Allocating many small blocks directly increases fragmentation risk. Sub-allocating from larger blocks reduces allocator pressure and improves reuse.

Concrete implementation: select a memory type

Query the physical device once after vkEnumeratePhysicalDevices. The following host-side helper picks a type that satisfies required flags, then prefers additional flags such as device-locality.

VkPhysicalDeviceMemoryProperties memProps;
vkGetPhysicalDeviceMemoryProperties(physicalDevice, &memProps);

uint32_t selectMemoryType(
    VkPhysicalDevice physicalDevice,
    uint32_t typeFilter,
    VkMemoryPropertyFlags required,
    VkMemoryPropertyFlags preferred)
{
    VkPhysicalDeviceMemoryProperties props;
    vkGetPhysicalDeviceMemoryProperties(physicalDevice, &props);

    for (uint32_t i = 0; i < props.memoryTypeCount; ++i) {
        if ((typeFilter & (1u << i)) == 0) continue;
        VkMemoryPropertyFlags flags = props.memoryTypes[i].properties;
        if ((flags & required) == required) {
            if ((flags & preferred) == preferred) {
                return i;
            }
        }
    }

    for (uint32_t i = 0; i < props.memoryTypeCount; ++i) {
        if ((typeFilter & (1u << i)) == 0) continue;
        if ((props.memoryTypes[i].properties & required) == required) {
            return i;
        }
    }
    return UINT32_MAX;
}

For a per-frame uniform buffer on a discrete GPU, use HOST_VISIBLE_BIT | HOST_COHERENT_BIT as required flags. For a large vertex buffer filled through staging, require DEVICE_LOCAL_BIT and allow host-visible as a preferred fallback for integrated hardware.

Staging buffer pattern

1. Create a host-visible staging buffer with VK_BUFFER_USAGE_TRANSFER_SRC_BIT.
2. Map the staging memory, write CPU data, then unmap.
3. Create the destination buffer with VK_BUFFER_USAGE_TRANSFER_DST_BIT plus the final usage, such as VK_BUFFER_USAGE_VERTEX_BUFFER_BIT.
4. Bind the destination to device-local memory.
5. Record vkCmdCopyBuffer from staging to destination and submit on a transfer queue.
6. Synchronize before using the destination buffer in rendering.

Validation and practical checks

• Run with Vulkan Validation Layers to catch offset alignment violations involving minStorageBufferOffsetAlignment and nonCoherentAtomSize.
• Inspect memoryHeaps and memoryTypes on the target hardware to confirm whether any heap is both device-local and host-visible.
• Measure frame-time delta between direct host-mapped writes and staging-buffer copies on the same hardware to quantify the PCIe cost.
• Always check that selectMemoryType does not return UINT32_MAX before calling vkAllocateMemory.

Limitations

The staging copy adds one GPU submission and synchronization point. For very small updates, that overhead can dominate the cost. On integrated GPUs, the copy may be handled efficiently by the driver or skipped entirely, making host-visible device-local memory the simpler path.