PHP Generators For Memory-Efficient Data Streams

The problem with loading everything into memory

When PHP scripts need to process large datasets—multi‑gigabyte log files, CSV exports, or API pagination—the instinct is often to read the whole thing into an array with file() or json_decode(file_get_contents()). That approach loads every record into memory at once, which can trigger out‑of‑memory errors or degrade performance on production servers.

Generators give you lazy iteration

PHP generators, introduced in PHP 5.5, let you yield values one at a time. Each time the generator is advanced via foreach, it resumes execution right after the last yield, producing the next value on demand. The Generator class implements the Iterator interface, so it works seamlessly with foreach and any function that expects an iterator, without needing external libraries or configuration.

Worked example: streaming a log file

Here's a generator that reads a file line by line, yielding each record so the caller can process it immediately:

$handle = fopen('/var/log/app.log', 'r'); if ($handle) { while (($line = fgets($handle)) !== false) { yield $line; } fclose($handle); }

Iterate it with foreach as usual:

foreach (streamLogLines('/var/log/app.log') as $line) { // process each line without ever holding the whole file in memory error_log(trim($line)); }

Limitations to keep in mind

• Single‑use state: A generator can be iterated only once. After the foreach finishes, the generator is exhausted. To process the data again, create a new generator instance—there is no rewind() that resets the same object.
• Exception propagation: If an exception is thrown inside a generator and not caught, it propagates to the caller and terminates the generator’s scope. Use try/catch inside the generator or Generator::throw() to manage controlled error handling.

Closing thought

For any data‑heavy PHP task—log analysis, CSV imports, infinite sequences—start with a generator. Pair it with foreach, keep the iteration path single‑pass, and remember to instantiate a fresh generator when you need to re‑process the same source. When error handling is required, wrap the yield loop in a try/catch and consider Generator::close() to release any open resources such as file handles.