Choosing a Vulkan Pipeline Cache Strategy for Startup Performance

Decision and constraints

When an application creates many graphics or compute pipelines at startup or during runtime, each vkCreateGraphicsPipelines or vkCreateComputePipelines call can trigger shader compilation, which adds noticeable latency. The goal is to reduce this overhead while staying within memory and disk budgets and handling possible driver or hardware changes.

Supported options

Option	Description	Pros	Cons
None	No VkPipelineCache object; pipelines are built from scratch each time.	Zero memory overhead; simplest code.	Highest CPU time; repeated shader compilation on every run.
In‑memory cache	A VkPipelineCache created with VK_PIPELINE_CACHE_CREATE_EXTERNALLY_SYNCED_BIT (if accessed from multiple threads) that lives only in process memory.	Fast reuse; no I/O; easy to share across threads.	Lost on process exit; memory grows with number of pipelines.
Persistent on‑disk cache	Cache data is serialized to a file after creation (vkGetPipelineCacheData) and reloaded on subsequent runs (vkCreatePipelineCache with pInitialData).	Survives across runs; reduces startup shader compile time significantly.	Requires disk I/O; cache can become stale when driver/GPU changes; needs invalidation logic.

Trade‑offs

The core trade‑off is between latency and resource usage:

• None avoids any memory or disk cost but suffers the highest pipeline creation latency.
• In‑memory gives immediate reuse with virtually no added latency, scaling linearly with RAM consumption proportional to pipeline complexity.
• Persistent adds a small disk read/write cost at start‑up and shutdown, but can cut pipeline creation time by tens of percent on subsequent launches, at the expense of needing cache‑validation when the underlying driver or hardware changes.

If the application runs short‑lived processes or has tight RAM budgets, the in‑memory cache may be sufficient. For long‑running apps or those launched frequently (e.g., games, CAD tools), a persistent cache usually yields the best user‑experience.

Concrete implementation

The following C++‑like snippet shows how to create, use, persist, and reload a Vulkan pipeline cache. Error handling is omitted for brevity; production code should check every Vulkan return value.

// 1. Create cache (externally synchronized if used from multiple threads)
VkPipelineCacheCreateInfo cacheInfo{};
cacheInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_CACHE_CREATE_INFO;
cacheInfo.flags = VK_PIPELINE_CACHE_CREATE_EXTERNALLY_SYNCED_BIT; // remove if single‑threaded
VkPipelineCache pipelineCache;
vkCreatePipelineCache(device, &cacheInfo, nullptr, &pipelineCache);

// 2. Use cache when creating pipelines
VkGraphicsPipelineCreateInfo pipeInfo{};
// ... fill pipeInfo ...
pipeInfo.pipelineCache = pipelineCache; // <-- important
VkPipeline graphicsPipeline;
vkCreateGraphicsPipelines(device, pipelineCache, 1, &pipeInfo, nullptr, &graphicsPipeline);

// 3. After all pipelines are created, retrieve cache data
size_t dataSize = 0;
vkGetPipelineCacheData(device, pipelineCache, &dataSize, nullptr);
std::vector cacheData(dataSize);
vkGetPipelineCacheData(device, pipelineCache, &dataSize, cacheData.data());

// 4. Persist to disk (example: binary file)
std::ofstream out("pipeline_cache.bin", std::ios::binary);
out.write(reinterpret_cast(cacheData.data()), dataSize);
out.close();

// 5. On subsequent runs, load the file and prime the cache
std::ifstream in("pipeline_cache.bin", std::ios::binary | std::ios::ate);
size_t fileSize = in.tellg();
in.seekg(0, std::ios::beg);
std::vector loadedData(fileSize);
in.read(reinterpret_cast(loadedData.data()), fileSize);
in.close();

VkPipelineCacheCreateInfo loadInfo{};
loadInfo.sType = VK_STRUCTURE_TYPE_PIPELINE_CACHE_CREATE_INFO;
loadInfo.pInitialData = loadedData.data();
loadInfo.initialDataSize = fileSize;
vkCreatePipelineCache(device, &loadInfo, nullptr, &pipelineCache);

// 6. Use the cache as shown in step 2

// 7. Cleanup when no longer needed
vkDestroyPipelineCache(device, pipelineCache, nullptr);

Validation and practical checks

• Measure latency: Record the time before and after the batch of vkCreate*Pipeline calls (e.g., using std::chrono::high_resolution_clock). Compare a run with no cache, an in‑memory cache, and a persistent cache. A successful persistent cache should show a measurable reduction (often >20 %) on the second launch.
• Check cache data size: After pipeline creation, vkGetPipelineCacheData must return a non‑zero dataSize. Zero indicates the driver stored nothing, which may happen if the application creates very few pipelines or if the cache is disabled.
• Verify correct reuse: With the validation layer VK_EXT_debug_utils enabled, look for messages such as "Pipeline cache hit" or the absence of shader compiler output after the first run.
• Detect stale cache: If the driver version, GPU, or relevant VkPhysicalDeviceFeatures change, the cache may return VK_ERROR_INVALID_PIPELINE_CACHE_EXT on vkCreatePipelineCache. In that case, delete the cache file and recreate it.

Limitations

Persistent caches are tied to the specific Vulkan driver, device, and hardware configuration. They are not portable across different GPU vendors or even different driver versions from the same vendor. The cache file can grow large if many complex pipelines are created; applications may need to impose a maximum size and discard old entries or rebuild the cache periodically.

Quick verification checklist

1. Run the application once with the cache enabled and note the pipeline creation time.
2. Close the application, verify that a non‑zero pipeline_cache.bin file exists on disk.
3. Run the application a second time and confirm that pipeline creation time is lower and that no shader‑compiler warnings appear in the validation output.
4. If you update the graphics driver, delete the cache file and repeat the test to ensure the application falls back to rebuilding the cache.