When does Dovecot lazy expunge actually free quota and shrink the index under concurrent IMAP load?

Question

I'm evaluating whether to enable lazy_expunge = yes on a Dovecot 2.3.x server that hosts busy mailboxes in mdbox format. The documented behavior is clear in principle: expunged messages stay in storage and in the index until a later expunge pass, which avoids immediate disk I/O on the hot path.What I cannot pin down is the practical timing. If deleted messages remain indexed, quota should not reflect the freed space right away, and the index should keep growing until doveadm expunge runs. But the documentation does not state whether mailbox close, a periodic plugin trigger, or only an explicit doveadm expunge -A actually processes the deferred removals, and how that interacts with many simultaneous IMAP sessions on the same mailbox.I plan to verify with doveadm status -v counters and imaptest load, but before designing the test I want to understand the intended semantics.Specifically: which events trigger processing of lazy-expunged messages in 2.3.x, and is it purely manual? Does concurrent access from multiple sessions delay or duplicate that processing? And is there a supported way to bound how much deferred-deletion storage accumulates before quota appears to recover?

Wasi · Answer

Answer
With lazy_expunge = yes Dovecot does not free quota or shrink the index immediately when a message is expunged via IMAP. The message is moved to the lazy_expunge namespace (usually a hidden .EXPUNGED mailbox) and its storage and index entries remain until an explicit expunge operation runs against that namespace.
Confirmed facts (from Dovecot 2.3.x behavior)

IMAP EXPUNGE, CLOSE or UID EXPUNGE only marks messages as expunged; quota usage and index size stay unchanged.
Actual removal occurs only when something runs doveadm expunge (or equivalent) on the lazy_expunge mailbox; there is no automatic trigger on mailbox close or via a plugin timer in 2.3.x.
Quota reflects the freed space only after the deferred expunge pass physically deletes the message files; until then the bytes still count toward quota.
The main mailbox index retains entries for expunged messages until the deferred pass removes them, so index growth continues between cleanups.
Concurrent IMAP sessions do not cause duplicate processing; they only add normal lock contention when a cleanup pass runs.
There is no built-in cap on the lazy-expunge backlog size; administrators must schedule periodic doveadm expunge with age or size criteria to bound accumulation.

Steps to verify in your environment

Enable lazy_expunge = yes and note the configured lazy_expunge namespace (default .EXPUNGED).
Run doveadm status -v or doveadm quota get -u user to record baseline quota and index size.
Expunge a set of messages via IMAP (or doveadm expunge -u user mailbox).
Check that quota and index size have not changed and that the messages appear in the lazy_expunge namespace.
Run doveadm expunge -u user mailbox .EXPUNGED savedbefore 1d (adjust namespace and age as needed).
After the command finishes, run the status/quota commands again; quota should drop and the dovecot.index file should be smaller.
Optionally, repeat under imaptest load to see lock contention in the logs but no duplicate cleanup.

Missing diagnostic detail
If your quota is enforced by the filesystem rather than Dovecot's quota plugin, lazy-expunged bytes will always count toward the limit until the files are physically deleted. Confirm whether you rely on Dovecot's quota plugin or external filesystem quota to interpret the numbers correctly.

When does Dovecot lazy expunge actually free quota and shrink the index under concurrent IMAP load?

1 answer

Answer

Confirmed facts (from Dovecot 2.3.x behavior)

Steps to verify in your environment

Missing diagnostic detail

Your perspective belongs here.

0 question comments