Managing Multi-GPU Workloads with Vulkan Device Groups

The Multi-GPU Coordination Problem

When developing high-fidelity graphics or compute-heavy applications, a single GPU often becomes the bottleneck. Traditional multi-GPU setups required managing multiple VkDevice handles—each with its own memory heap and command queue—creating massive overhead in resource synchronization and memory duplication.

Vulkan Device Groups (introduced in Vulkan 1.1) solve this by allowing you to treat a set of physical GPUs as a single logical device. Instead of managing separate devices, you manage one VkDevice that can dispatch work across a group of physical hardware units, simplifying the API surface for split-frame rendering and distributed compute.

How Device Groups Abstract Hardware

A Device Group is a collection of VkPhysicalDevice handles that the driver identifies as capable of working together. When creating a logical device, you specify a VkDeviceGroupDeviceCreateInfo structure to bind these physical devices into one group.

Once bound, the application uses device masks—bitmasks where each bit represents a GPU in the group—to control where commands execute and where resources reside. For example, a mask of 0x1 targets the first GPU, while 0x3 targets both the first and second GPUs simultaneously.

Resource Replication and Sharing

One of the most critical decisions in a device group setup is how to handle memory. You cannot assume every GPU has access to the same VRAM. Vulkan provides VkDeviceGroupResourceCreateFlags to define the lifecycle of a resource:

• Replicated: The driver automatically creates a copy of the resource on every GPU in the group. This is ideal for read-only textures or constant buffers used by all GPUs.
• Shared/Unique: The resource exists on a specific GPU. If another GPU needs it, the application must explicitly handle the transfer or use a peer-to-peer memory access path if supported by the hardware.

Worked Example: Enumerating and Submitting to a Group

To use device groups, first verify that the hardware supports them during instance and device enumeration. The following host-side code checks for multi-GPU groups. It requires a valid VkInstance and should run before logical device creation.

// Host-side enumeration; requires a valid VkInstance handle
uint32_t groupCount = 0;
vkEnumeratePhysicalDeviceGroups(instance, &groupCount, nullptr);

if (groupCount == 0) {
    // No device groups exposed by this driver
    return;
}

std::vector<VkPhysicalDeviceGroupProperties> groups(groupCount);
vkEnumeratePhysicalDeviceGroups(instance, &groupCount, groups.data());

for (const auto& group : groups) {
    if (group.physicalDeviceCount > 1) {
        // Multi-GPU group detected; inspect group.physicalDevices
    }
}

When submitting work, replace the standard VkSubmitInfo with VkDeviceGroupSubmitInfo. This lets you specify a deviceMask for each queue submission. In a split-frame rendering scenario, you would submit two different command buffers—one with a mask of 0x1 for the top half of the screen and one with 0x2 for the bottom half.

Trade-offs and Hardware Limitations

Device groups are powerful, but they are not a "magic button" for performance. Three primary limitations deserve attention:

Limitation	Impact	Mitigation
Heterogeneous Hardware	GPUs in a group may have different VRAM capacities or feature sets.	Query VkPhysicalDeviceProperties for each member; target the lowest common denominator.
Inter-GPU Latency	Moving data between GPUs over PCIe is significantly slower than local VRAM access.	Use replicated resources for read-heavy data to avoid runtime transfers.
Driver Variance	While mandatory in Vulkan 1.1, some mobile or legacy drivers may expose limited or buggy implementations.	Enable Vulkan Validation Layers during development to catch mask mismatches.

Verifying the Setup

To confirm your device group is functioning correctly, do not rely on visual output alone, because a failure on one GPU can be masked by another. Use these checks:

1. Validation Layers: Run with VK_LAYER_KHRONOS_validation. The layers report misuse of device masks or missing resource replication as validation errors.
2. Timing Queries: Use vkCmdWriteTimestamp on each GPU separately. If one GPU is significantly slower, you may have a load-balancing issue in split-frame logic.
3. Memory Tracking: Use a tool such as RenderDoc or a vendor profiler to verify that replicated resources occupy space on all targeted physical devices.