Parsing Delimited Strings in Rexx Using CALL PARSE

The Problem: Splitting Strings into Variables

When processing configuration files, CSV data, or command-line arguments in Rexx, you often need to split a single string into multiple distinct variables based on a specific character. While manual looping with POS and SUBSTR functions works, it is verbose and error-prone.

The CALL PARSE instruction is the most efficient way to tokenize a string. It allows you to map a delimited string directly into a list of variables or a stem (an associative array), reducing dozens of lines of manual string manipulation to a single statement.

How CALL PARSE Works

The CALL PARSE instruction follows this general syntax:

CALL PARSE token-list string [delimiter] [options]

The token-list can be a comma-separated list of variable names or a stem variable (e.g., myArray.). The string is the target text to be split, and the delimiter is the character or string that triggers the split.

Practical Example: Processing a CSV Record

Consider a scenario where you have a string representing a user record: "John Doe,Engineering,555-0102,New York". You want to extract these into a stem for easy indexing.

/* Run this on a standard Rexx interpreter (e.g., Regina or ooRexx) */
record = "John Doe,Engineering,555-0102,New York"

/* Split the record using a comma as the delimiter */
/* The dot after 'user' indicates a stem variable */
CALL PARSE user. record ','

/* Verification: Print the results */
DO i = 1 TO 4
  SAY "Field " i ": " user.i
END

Expected Result:

• user.1 = "John Doe"
• user.2 = "Engineering"
• user.3 = "555-0102"
• user.4 = "New York"

Handling Edge Cases and Options

Standard parsing can lead to unexpected results when dealing with empty fields or specific data types. Rexx provides options to modify this behavior:

Option	Effect	Use Case
T	Treats delimiters as literal characters	When the delimiter is a special character that might be interpreted by the parser.
N	Numeric conversion	Ensures tokens are treated as numbers for mathematical operations immediately after parsing.
W	Word extraction	Useful when delimiters are variable lengths of whitespace.

The "Empty Token" Pitfall

By default, if two delimiters appear consecutively (e.g., "Value1,,Value3"), some Rexx implementations may skip the empty token. If your data relies on positional columns where an empty string is a valid value, you must verify your interpreter's behavior. To ensure empty fields are preserved, ensure you are using a stem variable and check the resulting index count.

Limitations and Performance

• Memory Overhead: Using CALL PARSE on extremely large strings (several megabytes) can lead to performance degradation because Rexx must allocate memory for every token created in the stem. For massive files, process the file line-by-line rather than loading the entire file into a single string.
• Quoted Delimiters: If your delimiter appears inside a quoted string (e.g., "John "The Boss" Doe,Engineering"), CALL PARSE may split the string inside the quotes unless the input is pre-processed or the specific interpreter's quoting rules are applied.
• State Modification: CALL PARSE overwrites the variables in the token-list. If you are reusing a stem variable, clear it first to avoid leftover data from previous operations.

Verification Checklist

To verify your CALL PARSE implementation is working correctly, perform these three checks:

1. Boundary Test: Pass a string with no delimiters. The entire string should be assigned to the first variable/stem index.
2. Empty String Test: Pass an empty string. Ensure the program does not crash and handles the resulting empty variables gracefully.
3. Trailing Delimiter Test: Pass a string ending with a delimiter (e.g., "A,B,C,"). Verify if the interpreter creates a final empty token or ignores the trailing delimiter.