Architecting Fault‑Tolerant Asynchronous Computation in MATLAB with parfeval

Requirements

When you need to process very large matrices or arrays—tens of gigabytes of data—serial MATLAB code will quickly hit memory or time limits. The Parallel Computing Toolbox (PCT) offers parfeval for asynchronous, read‑only function evaluation on a cluster of workers. The core requirements for this approach are:

• MATLAB R2020a or newer (parfeval was introduced in 2020a).
• All workers must run the same MATLAB release and have the required toolboxes installed.
• Data passed to workers is copied, not shared; therefore inputs must be immutable or deep‑copied to avoid accidental mutation.
• Workers must have sufficient memory to hold the data slices you intend to process.
• The application must tolerate occasional worker failures and provide a deterministic result.

Minimal Design

The smallest viable architecture uses a persistent parpool, submits tasks via parfeval, and collects results with fetchOutputs. A clean‑up hook guarantees that the pool closes even if the script aborts.

% Define a helper that computes the sum of a matrix slice
function s = sliceSum(A)
    s = sum(A(:));
end

% --- Main script ---
% 1. Create or reuse a parpool
if isempty(gcp('nocreate'))
    pool = parpool('local', 4, 'IdleTimeout', 30); % 4 workers, 30‑s idle timeout
else
    pool = gcp();
end

% 2. Prepare large input data
A = rand(10000, 10000); % 100M elements ≈ 800 MB

% 3. Split data into chunks (here 4 chunks for 4 workers)
chunkSize = ceil(size(A,1)/pool.NumWorkers);
chunks = mat2cell(A, repmat(chunkSize,1,pool.NumWorkers), size(A,2));

% 4. Submit asynchronous tasks
futures = parallel.FevalFuture.empty(1,0);
for k = 1:pool.NumWorkers
    futures(k) = parfeval(@sliceSum, 1, chunks{k});
end

% 5. Retrieve results
results = zeros(1, pool.NumWorkers);
for k = 1:pool.NumWorkers
    [~, idx, out] = fetchOutputs(futures(k));
    results(idx) = out{1};
end

% 6. Aggregate final result
totalSum = sum(results);

% 7. Clean up
cleanup = onCleanup(@() delete(pool));

Key points:

• The function sliceSum is pure—no side effects.
• Each worker receives a distinct slice; no shared memory is used.
• All futures are collected in a loop; fetchOutputs blocks until the task finishes.

Trust and Data Boundaries

Because PCT copies input arguments to workers, you can rely on data isolation. However, large inputs may exceed the worker’s memory. If you anticipate exceeding the per‑worker limit, chunk the data further or use MATLAB’s distributed arrays:

D = distributed(A); % automatically partitions across workers
f = parfeval(@(X) sum(X(:)), 1, D); % D is passed as a distributed array
[~,~,out] = fetchOutputs(f);

Always verify that the worker memory footprint is below the limit by inspecting parpool('status') or using memory within a worker.

Operational Checks

Monitoring the health of the asynchronous pipeline is essential. The following checks should be performed periodically:

• Future status: f.Status returns 'finished', 'running', or 'failed'. Polling is inexpensive and can be done in a loop or via parfevalWait.
• Worker health: parpool('status') displays each worker’s Idle or Busy state. A worker marked Failed indicates a crash.
• Timeouts: The Timeout option of parfeval can abort a task that runs too long. Set it to a value that reflects your SLA.

Example of polling:

while any([futures.Status] ~= 'finished')
    pause(1); % wait 1 second
    disp(parpool('status'));
end

Failure Modes and Mitigations

Typical failure scenarios include:

• Worker crash or network drop: future.State becomes 'failed'. Mitigation: implement retry logic or checkpoint intermediate results to disk.
• Data size exceeding worker memory: The task throws an out‑of‑memory error. Mitigation: pre‑check size, use distributed arrays, or split data into smaller chunks.
• License mismatch: A worker lacking the required toolbox fails immediately. Mitigation: validate toolbox presence before submitting.

Retry logic example:

maxRetries = 3;
for k = 1:pool.NumWorkers
    attempts = 0;
    while attempts < maxRetries
        try
            [~,~,out] = fetchOutputs(futures(k));
            results(k) = out{1};
            break; % success
        catch ME
            attempts = attempts + 1;
            warning('Worker %d failed (%s). Retry %d/%d', k, ME.message, attempts, maxRetries);
            if attempts == maxRetries
                error('Worker %d failed after %d retries', k, maxRetries);
            end
        end
    end
end

When to Re‑Design

Consider a different architecture if:

• The number of workers required exceeds the local machine’s capacity; a true cluster or HPC scheduler may be needed.
• Tasks are highly interdependent, requiring shared state; then a message‑passing or distributed array approach is preferable.
• Latency constraints demand real‑time response; asynchronous execution may introduce unacceptable delays.

Practical Verification

To confirm correctness and performance:

1. Create a test matrix of size 10,000×10,000 and compute its sum serially: serialSum = sum(A(:));
2. Run the asynchronous script above and compare: assert(abs(totalSum - serialSum) < 1e-10);
3. Inspect each future’s Status and State to ensure they report 'finished' and 'succeeded'.
4. Configure IdleTimeout=30, submit a long task (e.g., a 60‑second sleep), wait, then verify that workers shut down after idle period using parpool('status').

Limitations

• Data is copied to each worker; for very large inputs this may become a bottleneck.
• Shared memory is not available; any global state must be passed explicitly.
• Fault tolerance is limited to retrying failed futures; there is no built‑in checkpointing for long‑running tasks.
• The approach assumes a homogeneous environment; heterogeneous clusters may produce inconsistent results.

Despite these constraints, parfeval provides a lightweight, fault‑tolerant way to scale MATLAB computations asynchronously while keeping data integrity.