argocd-application-controller OOMKilled despite sharding — how to size shards correctly

Our Argo CD instance manages several thousand resources across multiple clusters, and the argocd-application-controller pod keeps getting OOMKilled as its memory climbs steadily over hours. From the documentation and metrics, this looks like the informer/watch cache growing with the number of live managed objects rather than a classic leak, but the restarts are disruptive enough that we need to act on it.

We enabled controller sharding to spread the load, assuming a consistent-hash split of applications across replicas would cap per-replica memory. Memory per replica did drop, but we have no principled way to pick the shard count. Too few shards and each replica still OOMs; too many and we worry about wasted overhead and how leader election behaves across replicas during restarts.

We can see workqueue depth and informer counts in the controller's Prometheus metrics, and we have heap profiles from the pprof endpoint, but nothing tells us the memory-per-object ceiling we should plan around.

Questions

• Is there a documented heuristic or formula for choosing the number of controller shards based on managed resource count or cluster count?
• How should we interpret heap profile output to confirm the informer cache is the dominant consumer before adding more shards?
• Does increasing shard count change leader-election or reconciliation behavior in ways we should account for during rolling restarts?