Local Memory Tiling vs. Global Memory Coalescing for High-Bandwidth Kernels

There is no portable data-to-compute ratio that makes __local tiling a win. The deciding variable is reuse: how many useful __local reads each staged global load produces. If that factor is close to 1, a coalesced global-only kernel is the better default. If the same global data is read many times—stencil halos, matrix tiles, shared look-up tables—tiling usually pays off, provided the tile still leaves enough work-groups resident per compute unit and the device local memory is dedicated SRAM rather than global-backed.

Confirmed by the OpenCL specification

• __local memory is per-work-group scratchpad. The per-compute-unit budget is CL_DEVICE_LOCAL_MEM_SIZE; the static plus dynamic allocation for one kernel is reported by CL_KERNEL_LOCAL_MEM_SIZE. Large allocations can reduce the number of concurrently resident work-groups or cause launch failure.
• The standard does not mandate dedicated SRAM. Query CL_DEVICE_LOCAL_MEM_TYPE; if it reports CL_GLOBAL, local reads may not be faster than global reads, and tiling adds copy overhead without latency benefit.
• barrier(CLK_LOCAL_MEM_FENCE) orders work-items only inside one work-group. It does not share data across work-groups.
• There is no portable coalescing or occupancy metric in OpenCL. Behavior depends on device, driver, and work-group geometry.

Likely explanation for the trade-off

On typical GPUs, a non-coalesced global access pattern turns one work-group-wide memory instruction into many device transactions, saturating the memory system. A coalesced pattern issues far fewer transactions. Tiling helps by converting repeated scattered reads into one staged, coalesced global transfer followed by low-latency __local reuse. However, every byte moved into __local costs an extra global read plus a write, and a barrier. If reuse is low, those extra transactions outweigh the saved latency. If reuse is high but the tile size forces only one work-group per compute unit, the device may lose the parallelism needed to hide memory latency, so throughput can fall even though individual __local reads are faster.

Steps for this decision

1. Estimate the reuse factor: count useful __local reads per staged global load. Values near 1 favor global-only; values much greater than 1 favor tiling.
2. Query the device before sizing tiles. In OpenCL 1.1+ you can inspect the properties that matter:

cl_ulong local_size, cacheline;
cl_device_local_mem_type lmem_type;
size_t pref_mult;
clGetDeviceInfo(dev, CL_DEVICE_LOCAL_MEM_SIZE, sizeof(local_size), &local_size, NULL);
clGetDeviceInfo(dev, CL_DEVICE_LOCAL_MEM_TYPE, sizeof(lmem_type), &lmem_type, NULL);
clGetDeviceInfo(dev, CL_DEVICE_GLOBAL_MEM_CACHELINE_SIZE, sizeof(cacheline), &cacheline, NULL);
clGetKernelWorkGroupInfo(kernel, dev, CL_KERNEL_PREFERRED_WORK_GROUP_SIZE_MULTIPLE,
                         sizeof(pref_mult), &pref_mult, NULL);

1. Choose work-group sizes as multiples of CL_KERNEL_PREFERRED_WORK_GROUP_SIZE_MULTIPLE, keep global accesses contiguous within a work-group, and align buffers to CL_DEVICE_GLOBAL_MEM_CACHELINE_SIZE and the device base-address alignment.
2. Build two kernels with identical math: one coalesced global version and one __local-tiled version. Vary tile size and work-group size, then time with OpenCL events and clFinish on the target device.

Verification and limits

Because OpenCL does not define a portable coalescing or occupancy metric, treat any ratio as device-specific. Vendor profiling counters can compare global transactions, local traffic, and achieved occupancy; without them, compare runtime while scaling work-groups per compute unit. Do not infer a fixed bytes-per-FLOP threshold—caches, memory controllers, and warp/wavefront width differ across devices.

One detail changes the recommendation: if you need a device-specific verdict, report the measured reuse factor per staged tile and whether CL_DEVICE_LOCAL_MEM_TYPE is dedicated SRAM or global-backed. Those two values determine whether tiling is likely to win.