Preserving Original Local Time vs UTC-Normalized Storage in Ecto Schemas
0 reputation · 11 Jul 2023, 12:53 UTC
0 reputation · 11 Jul 2023, 12:53 UTC
When modeling timestamps that must retain a user's original wall-clock time and time zone (for example, a scheduling feature where "2024-03-10 02:30" in America/New_York is distinct from the same wall-clock in Europe/London), two documented Ecto patterns emerge. The built-in :utc_datetime_usec type maps to PostgreSQL timestamptz, which normalizes all input to UTC on write and discards the source offset. The alternative is :naive_datetime_usec (timestamp without time zone) paired with a separate varchar or int column holding the IANA zone name or offset, preserving the exact local representation.
Both approaches work with Ecto 3.10+ and PostgreSQL, but they impose different constraints: timestamptz simplifies range queries and ordering in UTC while requiring application-level zone tracking; the naive-plus-zone column keeps original semantics intact but complicates SQL-level filtering, sorting, and migrations. Custom composite types (e.g., Timex.Ecto.DateTimeWithTimezone) embed zone data in a single column yet remain third-party and adapter-specific.
Which strategy do you adopt when the product requirement is to reconstruct the user's original local time exactly as entered, while still supporting efficient UTC-based analytics queries?
35775 reputation · 11 Jul 2023, 23:25 UTC
Adopt a hybrid pattern: store the user's exact wall-clock time in a naive_datetime_usec column, store the IANA time zone in a separate varchar column, and add a generated timestamptz column for UTC-based analytics. This reconstructs the original local time exactly while letting PostgreSQL index and sort the derived UTC value.
:utc_datetime_usec maps to PostgreSQL timestamptz, which normalizes input to UTC and discards the source offset.:naive_datetime_usec maps to PostgreSQL timestamp without time zone, preserving the entered wall-clock value.timestamptz column for efficient range and ordering queries.The application reads local_time and time_zone together to reproduce the user's original input. Analytics queries filter and sort directly on utc_time, avoiding per-row application conversion.
def change do
alter table(:schedules) do
add :local_time, :naive_datetime_usec
add :time_zone, :string
end
execute """
ALTER TABLE schedules
ADD COLUMN utc_time timestamptz
GENERATED ALWAYS AS (local_time AT TIME ZONE time_zone) STORED;
"""
create index(:schedules, [:utc_time])
end
schema "schedules" do
field :local_time, :naive_datetime_usec
field :time_zone, :string
field :utc_time, :utc_datetime_usec, read_after_writes: true
timestamps()
end
utc_time back from the database for analytics queries.psql -c '\d schedules' and confirm utc_time is timestamp with time zone GENERATED ALWAYS AS ... STORED.~N[2024-03-10 02:30:00] with 'America/New_York' and confirm local_time round-trips unchanged while utc_time shows the normalized instant.EXPLAIN on a WHERE utc_time >= ... query to confirm the index is used.This assumes PostgreSQL 12 or newer for stored generated columns. It also assumes you store IANA zone names rather than raw offsets, because offsets alone do not represent historical DST rules. Ambiguous wall-clock times during fall-back transitions still require application-level disambiguation.
One missing diagnostic detail changes this recommendation: is your production PostgreSQL version 12 or newer? If not, stored generated columns are unavailable and you should maintain the UTC column in the application or via a trigger instead.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.