Local Memory Tiling vs. Global Memory Coalescing for High-Bandwidth Kernels

When designing data-intensive kernels in OpenCL, developers must decide between leveraging __local memory for scratchpad tiling or relying on optimized global memory access patterns. Local memory offers significantly lower latency and higher bandwidth but is strictly limited in capacity per compute unit.

A primary constraint is the impact on hardware occupancy. Allocating a large portion of the available local memory (queried via CL_DEVICE_LOCAL_MEM_SIZE) can restrict the number of active work-groups that can reside on a compute unit simultaneously, potentially offsetting the performance gains from reduced global memory transactions.

Conversely, relying solely on global memory requires strict coalesced access to maintain efficiency, yet it avoids the overhead of explicit data staging and synchronization barriers.

• Under what specific data-to-compute ratios does the cost of staging data from global to local memory outweigh the latency benefits of SRAM?
• How does the reduction in active work-groups due to local memory allocation typically impact overall throughput compared to the latency of non-coalesced global memory access?