Failed to parse date field [@timestamp]: unable to parse date when timezone offset present but no timezone configured

Answer

By default, Elastic Beats (Filebeat, Metricbeat, etc.) do **not** treat an embedded timezone offset as the event’s timezone. If the input configuration does not specify a timezone, Beats fall back to UTC and ignore any offset characters in the timestamp string.

Confirmed facts

• Beats parse timestamps using Go’s time.Parse layout. The layout string supplied in the timestamp field (or the default RFC3339/Nano) must contain a zone pattern (Z, Z07:00, -07:00, etc.) for the offset to be recognized.
• If the layout lacks a zone pattern, any offset present in the log line is treated as literal text and causes a parsing error, which Beats logs as “failed to parse date field”.
• When a parsing error occurs, Beats either drops the line or stores the raw string in a fallback field, depending on the ignore_older and skip_on_error settings, but the event’s @timestamp is set to the Beat’s internal clock (UTC) unless a timezone option is explicitly set.

Steps to enable offset interpretation

1. Identify the timestamp format used by your logs, including the offset (e.g., 2006-01-02T15:04:05.000-07:00).
2. In the Beat’s input configuration, set the timestamp field to a layout that contains a zone pattern, for example:

filebeat.inputs:
- type: log
  paths: ["/var/log/app/*.log"]
  timestamp:
    format: "2006-01-02T15:04:05.000-07:00"

1. If you prefer to keep the existing format and let Beats apply a timezone, add a timezone option under the input:

filebeat.inputs:
- type: log
  paths: ["/var/log/app/*.log"]
  timezone: "America/New_York"

Fallback when offset missing or malformed

• If the timestamp string does not contain a valid offset, Beats uses the value of the timezone setting (if set) or defaults to UTC.
• When the offset is malformed (e.g., +25:00), the parse fails and Beats falls back to the same rule: configured timezone or UTC.

Opt‑out without breaking existing pipelines

• Leave the timezone option unset and keep the timestamp format without a zone pattern; Beats will continue to treat timestamps as UTC.
• Alternatively, explicitly set timezone: "" (empty string) to disable any automatic zone inference while preserving the existing layout.

These steps ensure consistent behavior across Filebeat and Metricbeat, avoid timestamp drift across DST changes, and maintain compatibility with configurations that rely on UTC fallback.