Queueable Chaining vs Batch Chunking for High-Volume Apex Processing
0 reputation · 02 Aug 2020, 17:43 UTC
0 reputation · 02 Aug 2020, 17:43 UTC
When a scheduled integration must process a data set that exceeds the SOQL row governor limit but remains within the documented batch ceiling, the choice between Queueable Apex with self-chaining and Batch Apex with Database.executeBatch() becomes a design trade-off rather than a simple volume check.
Queueable Apex supports complex object types and job chaining, but chaining is limited to one child job per parent in synchronous contexts, and deep chains can obscure transaction flow during debugging. Batch Apex provides explicit chunking through the start, execute, and finish lifecycle and exposes job state via AsyncApexJob, yet queuing too many batches simultaneously can trigger Concurrent Request limits.
The unresolved decision is whether the operational visibility of AsyncApexJob and the structured chunk boundaries of Batch Apex outweigh the flexibility of Queueable execute methods for a nightly job that needs conditional logic between chunks. Is the debugging cost of Queueable chaining acceptable when each transaction requires distinct parameter handling, or does the risk of Concurrent Request limits make a single Batch job the safer default?
34775 reputation · 02 Aug 2020, 18:06 UTC
Assuming a recent API version where Queueable supports non-primitive types, the choice reduces to one question: can the per-chunk parameters be derived from the chunk scope or stored as batch state? If yes, use a single Batch Apex job. If each transaction genuinely requires distinct parameter handling that depends on prior chunk results, Queueable chaining is the appropriate choice despite the added debugging overhead.
Batch Apex divides the workload into explicit chunks via the start, execute, and finish lifecycle. Each execute transaction receives fresh governor limits, and the entire job is tracked under one AsyncApexJob record, giving you a single place to monitor status and retries.
Queueable Apex supports complex data types and allows one job to enqueue another. Each chained job runs in its own transaction, but the logs appear as isolated entries without an automatic parent-child link, which makes tracing a multi-step integration harder.
When a nightly integration needs conditional logic between chunks—such as skipping a segment or altering behavior based on a prior result—Queueable chaining is more flexible because each job can accept distinct constructor arguments. Batch Apex applies identical logic to every chunk unless you carry state forward, and instance variables do not persist between execute invocations.
A single Batch job occupies one asynchronous job slot regardless of how many chunks it processes. A Queueable chain occupies the async queue for each link; if the design branches or multiple chains run concurrently, queue pressure can rise and delay other asynchronous work. Deep chains also risk approaching stack depth limits and obscure error propagation. The exact impact depends on org limits and current load.
QueryLocator offset or a chunk index. If this works, implement a stateless execute and avoid Database.Stateful unless absolutely necessary.execute so you can manually stitch the transaction logs.AsyncApexJob. For Batch, verify that governor limits reset per chunk by logging Limits.getLimitRows() inside execute. For Queueable, query ParentJobId to confirm chain depth.Use these scoped checks in the Developer Console or anonymous Apex:
// Inside Batch execute
System.debug('SOQL rows limit this chunk: ' + Limits.getLimitRows());
// Inspect Queueable chain depth
SELECT Id, ParentJobId, Status
FROM AsyncApexJob
WHERE JobType = 'Queueable'
ORDER BY CreatedDate DESC
LIMIT 10;
// Check current async queue pressure
SELECT Id, JobType, Status
FROM AsyncApexJob
WHERE Status IN ('Queued', 'Processing')
LIMIT 20;
Do the distinct per-chunk parameters depend on the outcome of the immediately preceding chunk, or can they be precomputed from the chunk index? If the latter, a single Batch job is the safer default; if the former, Queueable chaining is justified.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.