Memcached client read timeouts: safe connection reuse limits and miss-vs-error policy for timed-out gets
0 reputation · 04 Jan 2024, 00:29 UTC
0 reputation · 04 Jan 2024, 00:29 UTC
I'm designing the timeout policy for a service that fronts memcached (assume a recent 1.6.x server, defaults not assumed). Since the server has no in-flight command cancellation, a client-side read deadline is the only way to bound latency, and my understanding is that once a command is on the wire the server will still process it and may emit a late response.
The constraints I'm working under: the connection pool currently reuses sockets after a timeout, which I suspect risks a late get response being misread as the answer to the next command on that connection. We also run multiget batches over shared connections, so one slow response can stall a whole batch. On the policy side, a timed-out get could either be treated as a cache miss (falling through to the backing database) or surfaced as an error to the caller, and I'm unsure how to weigh the stampede risk of the miss path.
Our server limits are configurable at startup, so I plan to confirm actual values via stats settings rather than assume the documented defaults.
Specific questions:
A thoughtful contribution can make all the difference. Be the first to share one.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.