When to Use MATLAB Tall Arrays for Out‑of‑Memory Data

Problem: your data won’t fit in RAM

You have a collection of CSV files that together exceed the memory available on your machine, but the analysis you need to run—such as computing a mean, a grouped summary, or fitting a simple model—produces a result that is small enough to hold in memory. Rewriting the code to work with low‑level file I/O or to chunk the data manually is error‑prone and hides the expressive syntax you already use for in‑memory tables.

Thesis: tall arrays let you keep familiar MATLAB syntax while deferring computation until you actually need the result

A tall array is a reference to data stored in a datastore. MATLAB builds an execution graph of the operations you write (e.g., mean, groupsummary) and only reads the underlying files when you call gather. This lazy evaluation lets you chain many transformations without incurring repeated I/O, and the same code works for both small in‑memory arrays and tall arrays—provided the functions you use support the tall datatype.

Worked example: compute a grouped mean from CSV files

Assume you have a folder data/ containing many CSV files, each with columns ID, Category, and Value. The goal is to find the average Value per Category.

1. Create a datastore that treats the files as a single tabular source.

   % Run in MATLAB (R2023b or later)
   fs = filedatastore('data/*.csv', 'ReadFcn', @readtable);
   ds = tabularTextDatastore('data/*.csv', ...
       'Format', '%s%s%f', ...
       'TreatAsEmpty', 'N/A', ...
       'MissingValue', NaN);

2. Wrap the datastore in a tall array.

   tt = tall(ds);   % tt is a tall table

3. Write the analysis using standard table syntax.

   % Group by Category and compute mean of Value
   result = groupsummary(tt, 'Category', 'mean', 'Value');
   % result is still a tall array; no data has been read yet

4. Trigger the computation and bring the small result into memory.

   gatheredResult = gather(result);   % <-- actual file read happens here
   % gatheredResult is a regular in‑memory table

If you have the Parallel Computing Toolbox, you can start a local pool beforehand and the same gather will distribute the reads across workers:

parpool;   % optional, requires PCT
% the code above stays unchanged

Trade‑offs and limitations

• Function support: Not every MATLAB function works with tall arrays. Unsupported calls throw an error only when gather is invoked, which can be surprising mid‑script. Always check the “Extended Capabilities” section of a function’s documentation for tall‑array support.
• Access pattern: Tall arrays allow only sequential, block‑wise access. Random row indexing (e.g., tt(1000,:)) is prohibited; algorithms that need shuffling or random access must be redesigned or performed on a sampled subset.
• File format and partitioning: Performance hinges on how the data is split across files. A few large, well‑chunked files (e.g., Parquet or binary) usually read faster than many tiny CSVs because each file open incurs overhead.
• Result size: Tall arrays are efficient only when the final output after gather fits comfortably in memory. If you need the entire dataset as an array (e.g., for a deep‑learning training loop that requires random batches), tall arrays are not the right tool.

When to choose tall arrays versus alternatives

Use tall arrays when:

• The data is columnar and stored in files you can point to with a datastore.
• Your analysis consists of supported reductions, groupings, or model‑fitting functions that produce a compact summary.
• You prefer to keep the high‑level MATLAB syntax you already know.

Consider other approaches when:

• The required output is itself large (e.g., you need to retain every row after a complex transformation). In that case, a straight datastore with readall in chunks or a custom loop may be clearer.
• Your algorithm needs random access or row‑wise shuffling; you might need to sample, index, or use a different out‑of‑memory framework (e.g., SQL databases or Spark).
• You have access to a cluster and the computation is embarrassingly parallel across rows; a parfor over a datastore might give you more control.

Actionable checklist

1. Verify that the functions you plan to use list “Tall Arrays” under Extended Capabilities.
2. Benchmark a small subset: compute the same result with gather on a tall array and with ordinary in‑memory code on a subset; ensure the numbers match.
3. If you have the Parallel Computing Toolbox, time the workflow with and without a parpool to see the speed‑up on your actual file layout.
4. Monitor disk I/O (e.g., via MATLAB’s profiler or OS tools) to confirm that the number of file reads matches the number of passes implied by your operation chain.

By following these steps you can decide quickly whether tall arrays will let you keep your familiar MATLAB workflow while working with data that exceeds RAM.