ActiveRecord Eager Loading vs. Query Builder for Large Dataset Retrieval
0 reputation · 16 Apr 2023, 15:44 UTC
0 reputation · 16 Apr 2023, 15:44 UTC
When designing a read-heavy reporting module in Yii 2, there is a trade-off between using ActiveRecord with eager loading and using the Query Builder for raw data retrieval.
ActiveRecord simplifies development through object-oriented abstraction and the with() method to prevent N+1 query issues. However, instantiating a large number of model objects increases memory consumption. Conversely, Query Builder returns raw arrays, which reduces overhead but bypasses model-level business logic and event triggers.
For a scenario requiring the retrieval of several thousand records for a read-only view, the primary constraint is balancing PHP memory limits against the need for maintainable code.
29275 reputation · 17 Apr 2023, 02:30 UTC
For a read-only view over several thousand rows, use asArray() (or Query Builder) for the bulk retrieval and reserve ActiveRecord for the small subset of rows that actually need model logic. There is no universal record-count threshold where ActiveRecord "breaks" — the crossover depends on row width, loaded relations, and your memory_limit, so it must be measured, not assumed. What matters more than the ActiveRecord-versus-Query-Builder choice is whether you buffer all rows at once; batching or streaming keeps peak memory bounded regardless of approach.
Confirmed behavior: with() collapses related queries into a bounded number of SQL statements, eliminating N+1 — but it does not reduce the number of model objects created. Every returned row still becomes a full ActiveRecord instance carrying attribute storage, relation state, and behavior overhead. asArray() and createCommand()->queryAll() skip hydration entirely and return plain arrays, which lowers per-row memory. The trade-off is real: array results bypass model events, validation, accessors, and behaviors, which can silently change output if your models transform attributes (e.g., date formatting, decrypted fields, computed getters).
Likely, but verify in your environment: the claim that arrays use dramatically less memory per row. The actual gap varies with column count, relation depth, and attached behaviors. PHP's own memory allocator, output buffering, and the PDO driver's result buffering all influence peak usage, so numbers from someone else's benchmark won't transfer to your schema.
memory_limit — a reasonable working rule is to keep peak usage well under half the limit so concurrent requests don't compound it.// Bulk read-only retrieval as arrays, streamed in chunks
$query = (new \yii\db\Query())
->from('order')
->where(['status' => 'closed']);
foreach ($query->batch(500) as $rows) {
foreach ($rows as $row) {
// render row; peak memory stays ~500 rows
}
}
If you want to keep the ActiveRecord query API (scopes, relation definitions) without hydration cost, Order::find()->with('customer')->asArray()->batch(500) gives you both. Hydrate individual models only where business logic is needed:
$order = Order::findOne($id); // events, behaviors, validation intact
memory_get_usage(true) / memory_get_peak_usage(true) in a CLI script that mirrors production row width and size.db log target in debug to confirm with() actually prevents N+1 (and that you're not lazy-loading inside the loop by accident).asArray(), and raw Query Builder. Decide from observed headroom, not a threshold from a blog post.Do any of the models in this report override afterFind(), accessors, or attach behaviors that transform attribute values? If yes, switching to arrays will silently change the output, and the safer path is batched ActiveRecord or explicitly replicating those transformations in the query — so that answer determines whether the hybrid or the pure-array approach is correct here.
Use comments to ask for clarification. Post a solution as an answer.
29,275 reputation · 16 Apr 2023, 22:50 UTC
ActiveRecord eager loading with with() or includes() can double memory consumption when handling thousands of rows, because each model instance adds overhead. A rule of thumb: if each model uses ~2 KB, >1 000 records may exceed typical PHP memory limits. Using Query Builder with pluck() and find_each(batch_size: 500) streams raw arrays, keeping memory low while preserving query control. Verify by enabling ActiveRecord logging and measuring GC statistics for both approaches.