Separate Integer and String Key Tables vs Mixed Table for Large Lua Datasets

Short Answer

Splitting integer and string keys into separate tables can reduce memory and speed up array iteration when your integer keys are sparse, but it is not a universal win. If your integer keys are already contiguous from 1, a single mixed table already stores them in the array part, and splitting mainly adds a second table object and extra GC tracking overhead.

What Is Confirmed vs. What Is Likely

Confirmed behavior in standard Lua: every table has an array part and a hash part. The array part stores positive integer keys starting at 1 that are reasonably dense. All other keys—including sparse integers, zero, negative integers, and strings—live in the hash part. A for i = 1, #t loop over a table whose integer keys are contiguous is generally faster than iterating a mixed table with next() or pairs(), because the interpreter walks the array part directly.

Likely, but workload-dependent: separating a sparse integer sequence from string keys can lower memory because the hash part no longer stores integer key/value pairs that would otherwise be interleaved with string keys. It can also reduce rehash pressure during growth. However, the second table adds a fixed header cost, and the garbage collector must now track two objects instead of one.

GC Pause Times

Do not assume splitting shortens GC pauses. Lua versions differ significantly: Lua 5.1 uses a non-incremental stop-the-world collector, while 5.3 and 5.4 default to incremental marking. Under incremental GC, more objects can mean more work spread across cycles, not necessarily shorter individual pauses. Under stop-the-world GC, two smaller tables may sweep faster than one large table, but the extra object header can offset the gain. The net effect depends on your Lua version and collector settings.

How to Verify for Your Dataset

1. Measure baseline memory with collectgarbage("count") after building the mixed table, then build the split tables and measure again.
2. Benchmark iteration: time a numeric for loop over the dedicated array table versus a pairs() walk over the mixed table using os.clock().
3. Check key density: if #mixedTable is close to the number of integer keys, the array part is already doing its job and splitting is unlikely to help.

One Diagnostic That Changes the Recommendation

Are your integer keys contiguous starting at 1, or are they sparse? If they are contiguous, keep one table—the array part already gives you O(1) integer access and fast iteration. If they are sparse, split them so the hash part does not carry redundant integer entries.