Diagnosing APL Numeric Comparison Failures Caused by Comparison Tolerance (⎕CT)

Problem

Two numeric values that appear identical on the screen evaluate to 0 with the = comparison operator, or ≡ returns 0 for nested arrays, even though they are not mathematically equal.

Diagnostic Table

• Symptom: a = b returns 0 despite identical display
• Symptom: ⌊x differs from expected integer
• Symptom: ≡ returns 0 for nested arrays

Ordered Checks

1. Query the current tolerance: ⎕CT. Note the value; it defaults to about 1E‑14 in many implementations.
2. Display the values with high precision: ⎕PP←17 then a and b. Hidden binary digits become visible.
3. Temporarily set tolerance to zero: ⎕CT←0. Re‑run the failing expression; if it now evaluates to 1 (true) the issue was tolerance.
4. If decimal comparison is intended, use ⎕DCT (decimal context) or convert numbers to integer/decimal types before comparing.
5. Isolate the problematic array or expression with ⎕DR or inspect its type to ensure both operands are numeric and not character or complex.

Fixes

• Set ⎕CT←0 locally around the exact scalar comparison, then restore the original value after testing.
• Round the values before comparison, e.g., ⌈a⌉ = ⌈b⌉, to eliminate tiny binary differences.
• Prefer ≡ for exact array matches; it bypasses tolerance.
• Use integer arithmetic or decimal types where available to avoid floating‑point representation errors.

Escalation Criteria

If the comparison still fails with ⎕CT←0, suspect a data‑type mismatch (character vs numeric), complex numbers, or vendor‑specific primitive behavior. Capture ⎕CT, ⎕PP, ⎕DR, and a minimal failing expression before contacting vendor support.

Verification Steps

1. Enter ⎕CT in the session to read the current tolerance.
2. Run the failing expression, then set ⎕CT←0 and re‑run it to confirm the result flips to the expected equality.
3. Print a suspect value with high ⎕PP (e.g., ⎕PP←17) to reveal hidden binary digits.
4. Test in a fresh workspace or isolated namespace so tolerance changes do not affect unrelated code, and record ⎕CT, ⎕PP, and ⎕DR before and after.