Akka Cluster Serialization Binding Incompatibility with Akka Persistence Journal During Rolling Upgrade

Short answer

A serializer binding mismatch does not directly eject a cluster member. It becomes an exit risk when a node cannot deserialize cluster gossip, sharding messages, or persistent actor commands from a peer. The resulting failure can make the node appear unreachable to the failure detector or crash the actor system, triggering exit. If a node exits while a persistent actor is replaying its journal, recovery stops and the actor cannot read journal state until the node rejoins with a compatible serializer.

Likely explanation versus confirmed behavior

Likely, not confirmed: member exits during a mixed-version rollout correlate with deserialization failures on legacy or upgraded nodes, and CoordinatedShutdown may terminate the actor system before replay completes.

Confirmed behavior: Akka Persistence does not automatically migrate journal or snapshot formats between versions. Removing a serializer binding for a class still present in journaled events makes those events unreadable. Cluster sharding passivation only affects in-memory entity passivation and shard rebalancing; it does not convert or clear journal state, so a reactivated entity replays the same incompatible journal and fails without manual migration.

Steps for this case

1. Identify every persistent event and snapshot class used by your actors.
2. Retain legacy serializer bindings on upgraded nodes for those classes until all journal entries and snapshots are migrated or rewritten.
3. Map event and snapshot classes to explicit serializer IDs in akka.actor.serialization-bindings; avoid default Java serialization for persistent state.
4. Test recovery in a mixed-version cluster with one legacy and one upgraded node before extending the rollout.
5. Review akka.coordinated-shutdown so shutdown does not run while persistent actors are recovering; timing is version- and plugin-specific.

Sharding passivation and schema changes

Automating entity passivation cannot bypass a journal schema change. Passivation removes the in-memory entity and may rebalance the shard, but activation replays the existing journal. Without manual state migration or re-initialization, reactivated entities fail on the same incompatible events.

Assumptions and missing detail

This assumes standard Akka Cluster with Akka Persistence, no custom downing provider, and default failure-detection settings. Exact CoordinatedShutdown behavior across versions requires testing your specific setup.

Which Akka version and journal plugin are you using? Schema compatibility and serialization behavior are plugin- and version-specific, and that detail determines whether binding retention alone is sufficient or whether journal migration is required.