Answer
Use the Performance Efficiency pillar measurement cycle: define normal, collect at all layers, compare to baseline, remediate, and re-measure. This makes diagnosis repeatable and objective rather than reactive.
1. Establish a repeatable baseline
Define KPIs tied to business outcomes and capture them under steady-state load. For EC2-based apps typical KPIs are p95/p99 latency, requests per second throughput, error rate %, and resource utilization.
- Resource baseline: CPUUtilization, memory pressure, EBS Burst Balance, Network In/Out, and I/O wait. Enhanced Monitoring gives 1-second granularity for micro-burst detection.
- Application baseline: request latency, throughput, and error rate from the app or load balancer.
Record thresholds in the AWS Well-Architected Tool as review artifacts so baselines persist across reviews.
2. Collect data at all layers
The Framework recommends measuring infrastructure, application, and network together.
- Infrastructure: CloudWatch metrics for EC2, EBS, and Auto Scaling Groups. Enable detailed monitoring for standard metrics and Enhanced Monitoring where sub-minute visibility is needed.
- Application: AWS X-Ray for distributed tracing to map latency to specific services, methods, and backend calls.
- Network: VPC Flow Logs and CloudWatch Network metrics to spot drops or throttling.
Assumption: you have CloudWatch enabled on the instances and X-Ray SDK instrumented in the app. If not, first enable collection before analysis.
3. Analyze deviations
Compare current dashboards to baseline. Confirmed analysis patterns:
- Saturation: high CPU with flat throughput suggests compute bound. Check CPU credit balance on burstable types.
- I/O bound: EBS Burst Balance at zero or high Read/Write latency indicates throttling.
- Latency hotspot: X-Ray service map shows increased segment duration for a specific service or DB query.
Likely explanation for unexplained latency: external dependencies or micro-bursting not visible in 5-minute averages. Verify with high-resolution metrics and synthetic checks.
4. Remediate and validate
Remediation follows measurement.
- Scale: add capacity via Auto Scaling Groups or resize instance type based on sustained utilization.
- Optimize: right-size, use Graviton for price-performance, add caching with ElastiCache, or move batch work off critical path.
Validate by re-running the same load profile and confirming KPIs return to baseline. Document changes in the Well-Architected Tool for repeatability.