Dovecot 2.3+ mailbox_list_index: concurrent write serialization bottleneck

Context

Since Dovecot 2.3 the mailbox_list_index feature is enabled by default, maintaining a per-user index (dovecot.list.index plus .log) that accelerates LIST and LSUB commands. The index is updated under an exclusive fcntl/flock lock for every mailbox write operation (EXPUNGE, MOVE, COPY, etc.).

Constraint

The design chooses strict consistency over throughput: there is no lock-free or optimistic concurrency path for the list index. Under concurrent writers the lock serializes updates, and the default mailbox_list_index_very_dirty_syncs=yes delays fsync, causing latency spikes that grow with the number of simultaneous writers and the size of the index. Filesystem choice (ext4, XFS, NFS) and locking implementation further affect the profile.

Trade-offs

Disabling the index for hot mailboxes (mailbox_list_index_include_inbox=no) or increasing the dirty-sync interval reduces contention but degrades LIST performance or risks index corruption on unclean shutdown. Neither workaround eliminates the fundamental serialization bottleneck for heavily shared mailboxes.

What configuration or deployment strategies have been validated to keep LIST latency predictable while preserving durability for mailboxes that experience sustained concurrent write loads? Are there filesystem-level tuning parameters (mount options, lock manager settings) that materially reduce the contention window without sacrificing crash safety? Does Dovecot’s roadmap include an alternative index architecture (e.g., per-mailbox sharding or lock-free reads) that would change this trade-off in a future major release?