REXX 6.0 Unicode build vs. legacy 8-bit semantics: migrating a low-traffic text script

Migration Requirements for REXX 6.0 Unicode

To migrate a legacy REXX script to the UTF-8 build, you must transition from byte-oriented logic to character-aware logic. In the UTF-8 build, strings are stored as variable-width sequences, meaning a single character may occupy multiple bytes. This breaks any logic that assumes 1 byte equals 1 character.

1. String Operations Requiring Revision

The following operations must be reviewed because they typically operate on bytes rather than Unicode code points:

• LENGTH(): Returns the number of bytes. For UTF-8 strings containing international characters, this will exceed the actual character count.
• SUBSTR(): Uses byte offsets. Attempting to slice a string at a byte index that falls in the middle of a multi-byte character will result in corrupted text.
• POS(): Returns the byte position of a substring, not the character index.
• Comparisons: While basic equality usually works for identical UTF-8 sequences, any logic relying on specific byte values (e.g., checking for a specific extended ASCII character in code page 1252) will fail.

2. Runtime Detection of Unicode Support

A single script can remain portable across both build types by detecting the interpreter's capabilities at startup. Use the following approach to determine if the Unicode build is active:

/* Detection Logic */
if RXFUNCQUERY('UNICODE') then
  do
    say "Unicode build detected"
    unicode_enabled = 1
  end
else
  do
    say "Legacy 8-bit build detected"
    unicode_enabled = 0
  end

If RXFUNCQUERY is unavailable in your specific distribution, examine the output of PARSE VERSION for the 'U' flag or the string "Unicode".

3. Consistent Migration Approach

For low-traffic workloads, the most stable path is to implement wrapper functions. This allows you to remove external conversion tools while maintaining compatibility with legacy builds.

1. Define Character-Aware Wrappers: Create functions like LENGTH_CHAR(). If unicode_enabled is false, it simply calls LENGTH(). If true, it uses the Unicode-aware logic provided by the REXX 6.0 build.
2. Standardize I/O: Ensure all file reads and writes are handled by the UTF-8 build's native I/O, which removes the need for external conversion libraries.
3. Validation: Test the script using a dataset containing multi-byte characters (e.g., accented Latin or CJK) to ensure SUBSTR and LENGTH calculations are accurate.

Assumptions and Uncertainty

This guidance assumes the use of a standard REXX 6.0 distribution where UTF-8 is an optional build flag. Because different distributions may implement Unicode-aware variants of built-in functions differently, you must verify if your specific build provides LENGTH as a character count or if it remains byte-oriented. If the latter is true, you must implement a manual UTF-8 code-point counter.

Missing Diagnostic: To refine this recommendation, please provide the output of SAY VERSION() from your target environment to confirm the exact build flags enabled.