Rancher Management Server and Cluster Agent: Connection State Reconciliation

State Synchronization After Network Partitions

Rancher relies on the cattle-cluster-agent to maintain a bidirectional communication channel with the management server. This integration ensures that configuration updates are pushed to downstream clusters and state changes are reported back to the manager.

When a network partition occurs between the agent and the management server, the TCP connection is severed. Upon restoration of connectivity, the agent must re-establish the session and synchronize any missed state changes. There is uncertainty regarding the specific reconciliation loop timing and the priority of updates during this recovery phase, which can lead to temporary state drift between the management plane and the downstream cluster.

Given the long-polling or WebSocket-based mechanism used for these updates, what is the expected behavior for reconciling pending state changes immediately following a reconnection? Does the agent prioritize the most recent desired state from the manager or the accumulated local state changes from the downstream cluster?