IBM Equal Access Accessibility Checker needs-review results: how should CI pipelines treat non-violation findings?

Direct Answer

needs-review and manual-check results should be treated as warnings, not errors, in CI pipelines. They must be documented with rationale and tracked as backlog items, but should not block releases when properly justified.

Why This Approach

The IBM Equal Access Accessibility Checker can only automate a subset of WCAG success criteria. When it encounters custom widgets, shadow DOM, iframes, or canvas-based UI, it flags uncertainty rather than compliance. A zero-violation scan proves absence of detected issues, not presence of accessibility conformance.

Implementation Steps

1. Configure CI to treat needs-review as warnings - Use exit codes or flags to prevent blocking merges
2. Require documentation in each PR - Add a comment explaining why the finding is acceptable
3. Track all needs-review items - Create tickets linked to the PR for follow-up audit
4. Prioritize by user impact - Critical workflows (login, checkout, forms) need immediate manual verification; secondary features can wait for scheduled audits

Determining Urgency

Apply this decision tree:

• Block release: Any needs-review on primary user flows (authentication, payment, core navigation)
• Require manual verification before merge: Secondary flows that affect >5% of users
• Accept with documentation: Edge cases with minimal user impact

Verification Protocol

To maintain policy alignment as rule sets evolve:

1. Quarterly manual audits - Validate flagged components with screen readers and keyboard navigation
2. Version testing - Before upgrading the checker, run it against known test cases to confirm classification behavior
3. Documentation review - Every 6 months, review all accepted needs-review items to ensure they're still accurate

Missing Diagnostic Detail

One detail needed for complete guidance: What is your team's definition of "critical user flow"? This determines which needs-review findings require immediate manual verification versus documentation-only acceptance.