Question
Text-protocol and binary-protocol clients: value serialization interoperability in memcached
Dev River
0 reputation · 25 Dec 2020, 07:59 UTC
136.9K views1
Loading…
0 reputation · 25 Dec 2020, 07:59 UTC
When a memcached deployment incorporates both text-protocol and binary-protocol clients, the interoperability of stored value serialization becomes a design concern. The text protocol treats values as null-terminated strings, whereas the binary protocol (supported since v1.5) encodes length-prefixed byte sequences. A key set via one protocol and retrieved via the other may produce unexpected byte representation, length mismatches, or protocol-specific encoding artifacts. This question addresses how the server normalizes or preserves value format across protocol boundaries, and whether client implementations assume protocol-agnostic storage.
Version-specific behavior adds further uncertainty. Memcached releases prior to 1.5 lack binary protocol support entirely, while later versions may apply implicit encoding conversions during cross-protocol operations. The interaction between CAS tokens, which are protocol-dependent, and mixed-protocol value access compounds the ambiguity. Additionally, deployment configurations such as slab allocation and eviction policies may treat stored bytes differently depending on perceived type.
What representation does a binary-protocol GET return for a key originally SET by a text-protocol client, and vice versa? Does the server preserve the exact byte sequence, apply null-termination, or reinterpret length prefixes? How do CAS operations behave when token generation and comparison span protocol boundaries?
35775 reputation · 25 Dec 2020, 09:42 UTC
Memcached preserves the exact byte sequence stored, regardless of which protocol wrote it. A binary-protocol GET on a key set via text protocol returns the original bytes including the trailing null terminator. A text-protocol GET on a key set via binary protocol returns the raw bytes without adding or stripping null terminators. The server does not reinterpret length prefixes or apply implicit encoding conversions.
The text protocol stores values as null-terminated strings. When a text client sends set key 0 0 5\r\nhello\r\n, the server stores the six bytes h e l l o \0. A binary GET returns a length-prefixed response containing those exact six bytes. The binary client receives the null terminator as part of the value payload.
The binary protocol stores length-prefixed byte sequences with no required terminator. When a binary client sets a 5-byte value hello, the server stores exactly those five bytes. A text GET returns VALUE key 0 5\r\nhello\r\nEND\r\n — the five bytes followed by the protocol's own \r\n delimiter, not a null terminator. The text client must not assume a trailing \0 exists.
CAS tokens are protocol-specific opaque values. A token obtained via binary GETS/GETK is only valid for subsequent binary CAS operations. A token from text gets works only with text cas. The server compares tokens within the same protocol context; cross-protocol CAS attempts fail with EXISTS or protocol errors. Token generation uses the same internal item version counter, but the wire-format encoding differs.
memcachedtelnet localhost 11211 (text) to set testkey 0 0 5 then hello.memcached-tool or libmemcached) to get testkey and hex-dump the value payload.68 65 6c 6c 6f 00 ("hello\0").\0 appears before \r\n.What memcached version is deployed? Pre-1.5 releases lack binary protocol entirely; versions 1.5+ may have protocol-specific bug fixes affecting edge cases (e.g., zero-length values, max-item-size limits). The exact version determines whether the above behavior is guaranteed or if a known regression applies.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.