MATLAB Tall Arrays: Processing Columnar Data That Exceeds RAM

When a dataset is too large to load with readtable or readall, the practical answer in MATLAB is usually a tall array: wrap a datastore in tall(ds), write ordinary-looking numeric code, and call gather once at the end. MATLAB defers execution, streams the data through in chunks, and combines partial results — so a mean over 500 CSV files uses roughly the memory of one chunk, not the whole dataset.

Why the naive approach fails first

The typical failure mode is a folder of sensor logs, simulation outputs, or transaction exports that individually load fine but collectively exceed RAM. readall(ds) on a datastore over that folder either errors or pushes the machine into swap, at which point everything slows to a crawl. The two supported ways out are an explicit chunk loop over the datastore, or a tall array. Tall arrays are the better default for reductions and statistics; manual chunking wins when you need stateful control (more on that trade-off below).

How the lazy execution graph works

A tall array does not hold data. Operations on it — arithmetic, filtering, mean, grouping — are recorded as a graph of deferred operations. Nothing touches disk until you call gather. At that point MATLAB reads the datastore block by block, applies your operations to each block, and combines the partial results. For a mean, that means accumulating per-block sums and counts, then dividing at the end. This is why chunked reductions are exact, not approximations.

A worked configuration: streaming mean over many CSVs

Run this in the MATLAB desktop or a script; no special permissions are needed beyond read access to the data folder. Assumes a release from the last several years (R2016b introduced tall arrays; behavior here is long-standing).

% Point a datastore at every CSV in the folder
ds = tabularTextDatastore('C:\data\sensor_logs\*.csv');

% Wrap it in a tall array — nothing is read yet
t = tall(ds);

% Build the deferred computation
m = mean(t.SensorA, 'omitnan');

% Execute: streams chunks, combines partial sums/counts
result = gather(m)

result is a normal in-memory scalar. The placeholder to adjust is the file pattern and the variable name (SensorA must match a column header in your CSVs). The 'omitnan' flag matters: real-world logs almost always contain gaps, and the default behavior propagates NaN into the final answer.

Verifying the result before trusting it at scale

Before pointing this at a production dataset, check it against the in-memory path on a small subset:

% Small-scale cross-check on one file
tbl = readtable('C:\data\sensor_logs\run001.csv');
expected = mean(tbl.SensorA, 'omitnan');
ds1 = tabularTextDatastore('C:\data\sensor_logs\run001.csv');
actual = gather(mean(tall(ds1).SensorA, 'omitnan'));
assert(abs(expected - actual) < 1e-12)

Two more useful checks: confirm readall(ds) on the full folder actually fails or swaps on your machine (that validates the premise), and confirm the function you plan to use lists tall-array support in its documentation's extended-capabilities section for your installed release — support varies by function and version.

What ports easily, and what does not

Common reductions — sum, mean, std, min, max — and grouped statistics via splitapply-style patterns work directly, so much existing numeric code ports with minimal edits. Filtering rows by condition (t(t.SensorA > 100, :)) also fits the chunked model naturally, since each block can be filtered independently.

The hard cases are operations that need global ordering or random access:

• Sorting an entire tall column is expensive or impractical, because chunks arrive unordered and a global sort needs all data at once.
• Positional indexing like t(1000000, :) fights the model — row numbers are not meaningful until data is gathered.
• Unsupported functions error late. Some calls fail only at gather time, after you have built the whole pipeline. Test end-to-end on a small dataset early, not after deployment.

The most common performance mistake: gathering twice

Every gather call re-executes the entire lazy pipeline from the datastore. This code reads all the files twice:

% Bad: two full passes over the data
m = gather(mean(t.SensorA));
s = gather(std(t.SensorA));

Either gather both in one call ([m, s] = gather(mean(t.SensorA), std(t.SensorA)), which shares one pass) or gather an intermediate tall result once and compute the rest in memory. You can observe the cost directly: time one gather, then time two consecutive gathers of the same expression — the second is not cached.

Tall arrays versus a manual chunk loop

The real engineering decision is usually between these two, not between tall arrays and bigger RAM:

Criterion	Tall array	Manual datastore loop
Code shape	Looks like normal MATLAB	Explicit while/read loop, you manage state
Best for	Reductions, statistics, filtering, grouped summaries	Stateful algorithms: custom recursive filters, running window logic
Control over chunk boundaries	None (MATLAB chooses)	Full
Parallel scaling	Automatic with Parallel Computing Toolbox	You implement it (parfor, parfeval)

If your algorithm carries state between rows — an IIR filter, a custom event detector with memory — a tall array fights you, because chunks are processed independently and possibly out of order. Write the manual loop. If you are computing statistics, histograms, or filtered extracts, the tall array is less code and fewer bugs.

Scaling beyond one machine's disk

With Parallel Computing Toolbox, a parallel pool (parpool) lets tall-array chunks be processed across workers, and datastores can point at cloud storage such as S3 or Azure Blob for datasets larger than local disk. The code above barely changes — the datastore location and the pool configuration do the work. Exact cloud-storage support and performance characteristics are version-sensitive, so confirm against your release's documentation before committing a cluster design.

Limits to keep in mind

Tall arrays are a streaming engine, not a database. They do not index your data, they re-read from the source on every gather, and they only help when your computation decomposes into per-chunk work with combinable partial results. If you find yourself gathering the same filtered subset repeatedly, the right move is to gather it once into an ordinary table and work in memory from there — the tall array has already done its job.