Dyalog Component Files vs JSON for APL State Persistence

When Dyalog state requires perfect APL fidelity—like complex nested arrays—component files preserve rank and structure where JSON serialization risks data loss. Use component files internally; use JSON externally.

Losing Structure in Serialization

If your Dyalog application relies on complex APL data—like nested arrays, specific ranks, or internal references—and you serialize this state to JSON, you risk losing crucial structural fidelity during the conversion round-trip. The useful takeaway is this: use Dyalog component files (e.g., `.dcf`) for internal, durable APL state, and reserve JSON only for external data exchange.

Dyalog's Native Persistence Advantage

Dyalog provides system functions like ⎕FCREATE, ⎕FAPPEND, and ⎕FREAD to persist APL arrays directly into binary component files. Unlike JSON, which is a text-based, schema-less format, these component files natively preserve the array's rank, shape, datatype, and internal nesting without requiring complex serialization/deserialization logic.

Choosing the Right Persistence Tool

The engineering decision hinges on the consumer of the data. For checkpoints, internal caches, or append-only logs strictly used by Dyalog processes, component files are superior because they maintain exact APL semantics. JSON, conversely, is the standard for interoperability with non-APL services or human configuration editing.

Worked Example: Preserving Nested Structure

To verify fidelity, we can create a component file containing a complex APL structure. Assume a Dyalog session is active (requires standard Dyalog user permissions).

Configuration Steps

Run ⎕FCREATE '/tmp/test.dcf'. (Creates the file.)
Run ⎕FAPPEND . (Appends the complex array structure.)
Run ⎕FREAD 1. (Reads the first record back into memory.)

Verification Check

The resulting array from ⎕FREAD 1 must exactly match the input , including rank and datatype, confirming no structural loss occurred.

Trade-offs and Limitations

The primary limitation is vendor lock-in: component files are opaque binary formats and cannot be easily read by Python or JavaScript. Furthermore, Dyalog component files do not inherently provide full ACID transaction semantics; concurrent writes from multiple Dyalog processes require external locking mechanisms.

Actionable Step

Before committing state, run ⎕VERSION within your Dyalog session to confirm the availability and expected behavior of the ⎕F* functions for your specific Dyalog release. Check the Dyalog Language Reference for your version to confirm the exact API.

When APL Fidelity Matters: Component Files vs. JSON for Dyalog State