What is the idiomatic way to combine bounded slices and buffered reads for paginating a large dataset in Zig?

Short answer

There is no single library type in Zig's standard library that unifies paging over slices and streams, because the two cases genuinely differ: a slice page can be a non-owning view, while a stream page must point into a buffer you own. The conventional pattern is a small custom iterator struct with a next() method that returns ?[]T (or ?[]const T) of at most N items, with two constructors — one taking a slice, one taking a reader plus a caller-supplied buffer. Bounds clamping lives inside next(), and a custom iterator is preferred over materializing pages into an ArrayList because it avoids per-page allocation.

One correction to the premise

Two details in the question don't match the actual std API (0.13/0.14 era):

• std.mem.slice is not a function for creating bounded views. Bounded views are made with slicing syntax: data[start..end]. Clamping is done with @min(start + page_size, data.len).
• ArrayList(T).items is a field (the backing slice), not an items() method or an iterator. Iterators in std are structs with a next() method returning an optional.

Slice-backed pages

This case is trivial and allocation-free. Keep an offset; each call returns a sub-slice clamped to the remaining length:

fn nextSlice(self: *Self) ?[]const T {
    if (self.offset >= self.data.len) return null;
    const end = @min(self.offset + self.page_size, self.data.len);
    defer self.offset = end;
    return self.data[self.offset..end];
}

The final page is simply shorter. Returning null at exhaustion is the idiomatic Zig iterator convention (matching std.mem.split, std.mem.tokenize, etc.).

Reader-backed pages

For a stream, the page must point into a buffer the iterator owns (or borrows from the caller). The key facts about the reader contract:

• Zig readers signal end-of-stream by returning 0 from read(), not via an error.EOF.
• Wrap a file with file.reader(), then std.io.bufferedReader(...) for chunked reads; std.io.AnyReader is obtained via .any() if you need type erasure.

The loop is: if the buffered region holds fewer than a page's worth of bytes, move the unconsumed tail to the front of the buffer (std.mem.copyForwards) and read again; stop refilling at EOF (a 0 return) or when the buffer is full. Then return a slice over up to N items and advance the consume position.

A caveat on generic element types

For a byte stream this is straightforward (T = u8). For arbitrary T, casting a byte buffer to []T raises alignment and endianness questions that a plain cast does not solve. The honest recommendation: either restrict the streaming variant to bytes and let the caller decode items, or define the stream page as raw bytes and deserialize per item. Presenting a byte-buffer cast as a generic []T page is unsound for most types.

Where bounds validation lives

Inside the iterator. next() never returns more than page_size items and returns a shorter final page; the caller loops until null. Putting the clamp at the call site duplicates logic and invites off-by-one errors.

Mutation between page requests

The iterator holds no snapshot. If the backing array is mutated or the file is appended to between calls, you can see duplicated, skipped, or stale items — and a slice page becomes invalid if the source array is freed or resized. If concurrent mutation is possible, synchronize externally (a lock, or re-check a version/length before each page). This is the one diagnostic detail that changes the design: if your source is immutable during a query, the simple design above is sufficient; if not, you need snapshot semantics that this pattern does not provide.

Buffer size and throughput

Buffer size does not affect correctness, only syscall frequency. Very small buffers (e.g. 256 bytes) increase read-call overhead on large files; 4–64 KiB is the usual range, and std.io.bufferedReader's default is in that neighborhood. Measure with your actual file sizes before tuning.

Verification

Test with zig test on 0.13/0.14: page an array of 1000 items at size 50 and assert the concatenation equals the original; write a temp file of known bytes, page it through a buffered reader, and compare byte-for-byte. Cover the edge cases: empty source, page size larger than the total, and a partial final page. Use std.testing.allocator to confirm zero allocations per page.