ERR Operation against a key holding the wrong kind of value during Redis schema migration

Trade‑offs: Manual Backup/Restore vs. Idempotent Migration Scripts

Manual backup/restore (BGSAVE/RDB or redis-cli --rdb) gives a full point‑in‑time snapshot. It is simple to reason about — restore replaces the entire dataset — but it requires downtime or a read‑replica promotion for large datasets, and the restore is all‑or‑nothing. Testing restore time on staging with production‑like data volume is essential because fork() memory pressure and disk I/O can cause latency spikes.

Idempotent migration scripts written in Lua (EVAL/EVALSHA) or as Redis Functions (Redis 7+) run atomically on the server. They can check a key’s type with TYPE, conditionally migrate, and clean up on failure. The trade‑off is development effort: each key pattern needs a tailored script, and scripts must be chunked to stay under the lua-time-limit (default 5 s). Lua scripts are atomic — if the script errors, none of its commands apply — but they do not automatically roll back commands that ran before the script in a MULTI/EXEC block.

Detecting and Handling “Wrong Kind of Value” Errors

The error surfaces at command execution time (e.g., calling HSET on a string key). Applications should catch the Redis reply error, log the key name, expected type, and actual type, then trigger a compensating action. A safe pattern:

1. Use SCAN (never KEYS) to enumerate candidate keys.
2. For each key, run a Lua script that:
   • Checks TYPE.
   • Renames the original key to a backup suffix.
   • Writes the new data structure.
   • Verifies the new structure.
   • Deletes the backup on success; on failure, renames the backup back.
3. Run in small batches with application‑level coordination (e.g., a migration queue) to limit replication lag.

Atomic Migration with Redis Functions (Redis 7+)

Redis Functions allow stored server‑side functions that replicate via AOF/replication. They can encode multi‑step migrations with explicit compensation logic, but there is no automatic rollback — compensation must be coded. Deploy function changes atomically with FUNCTION LOAD followed by FUNCTION FLUSH and reload. Modules like RediSearch or RedisJSON add data types but do not change the atomicity constraints; the same compensation pattern applies.

Practical Verification Steps

• Test the migration script on a copy of production data (redis-cli --rdb + redis-server --loadmodule) before running live.
• Inject “wrong kind of value” errors deliberately (e.g., SET key val then HSET key field val) to verify error handling and compensation path.
• Measure BGSAVE/RDB restore time on staging with production data size to validate downtime window.
• Verify Lua/Function atomicity: run a script that does TYPE, RENAME, HSET, then error — confirm no partial writes remain.
• Monitor replication lag during migration on replica sets; large key rewrites can cause lag.

One Missing Diagnostic Detail

What is the approximate number of keys and total data size you need to migrate? This determines whether a full RDB restore is feasible or if a batched Lua/Function approach is required.