How can I accurately diagnose CouchDB performance bottlenecks using measurable metrics?

I aim to pinpoint the primary contributors to latency and reduced throughput in my CouchDB cluster by relying on quantifiable measurements rather than anecdotal observations. The environment is production‑facing, so any data collection must add minimal overhead and avoid interfering with ongoing replication or query workloads.

Which specific metrics (such as request latency, view compaction time, disk I/O wait, or replication lag) provide the most reliable signals of bottlenecks? How can I isolate the overhead introduced by replication from that caused by query processing? What sampling interval or collection frequency yields stable data without noticeably affecting system performance?