Empty FS field splitting behavior limits in POSIX awk versus gawk

In gawk, setting the Field Separator (FS) to an empty string (""") causes awk to split records into individual characters. However, this behavior is not strictly defined by the POSIX standard, leading to significant inconsistencies between gawk and other awk implementations.

POSIX vs. Gawk Behavior

The POSIX standard for awk specifies that if FS is not defined, it defaults to a space-like character sequence. When FS is explicitly set, the standard does not explicitly define the behavior for an empty string.

• Gawk (GNU Awk): Treats FS = """ as an instruction to split the record by every character. NF becomes the number of characters in the line, and $1, $2, etc., represent each character.
• POSIX-compliant Awk (e.g., mawk, BWK awk): Most implementations either treat FS = """ as the default separator (whitespace) or may throw an error. They do not natively support character-level splitting via an empty string.

Consecutive Delimiters

The treatment of consecutive delimiters differs entirely based on whether FS is empty or a defined multi-character string:

• When FS is empty (gawk only): Every character is a delimiter. No "empty fields" are created because every single position (including spaces) is treated as a field.
• When FS is a specific string (e.g., FS = ","): Consecutive delimiters are treated as separate field boundaries. If two commas appear back-to-back, awk identifies an empty field between them, incrementing the field count accordingly.

Portable Strategy for Character-Parsing

To ensure consistent field counts and portability across POSIX awk and gawk, do not rely on FS = """ for character-level processing. Instead, use the split() function with a manual regular expression.

The following pattern is portable for character-level splitting:

# Portable way to split a line into characters
{
    n = split($0, chars, ".")
    for (i = 1; i <= n; i++) {
        print chars[i]
    }
}

Note: Using a dot (.) as the separator in split() effectively targets every character, mimicking the gawk FS = """ behavior across environments.

Diagnostic Requirement

Does your log data contain non-printable characters or null bytes that might prevent standard split() from functioning correctly?