Skip to content

fix(arrow/scalar): align timestamp timezone parsing - #1109

Open
fallintoplace wants to merge 10 commits into
apache:mainfrom
fallintoplace:fix/scalar-parse-timestamp-timezone
Open

fix(arrow/scalar): align timestamp timezone parsing#1109
fallintoplace wants to merge 10 commits into
apache:mainfrom
fallintoplace:fix/scalar-parse-timestamp-timezone

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

MakeScalarParam, ParseScalar, timestamp builders, JSON decoding, and typed CSV parsing should use the same timestamp timezone rules. This also adds support for Arrow fixed-offset timezone strings.

What changes are included in this PR?

  • Validate timestamp timezone metadata through TimestampType.GetZone.
  • Accept UTC metadata case-insensitively, including forms such as Utc.
  • Require the input to contain an offset exactly when the timestamp type has a non-empty timezone.
  • Keep timezone-naive values offset-free.
  • Preserve the instant represented by an explicit input offset.
  • Keep numeric timestamp values as raw epoch values after validating the type timezone.
  • Validate CSV timestamp metadata before reading rows, including null-only columns.
  • Keep timezone offsets in CSV output for timezone-aware timestamps so CSV writer and reader round trips work.
  • Preserve an explicitly supplied empty custom timestamp layout.

Are these changes tested?

  • go test ./arrow ./arrow/scalar ./arrow/array ./arrow/compute -count=1
  • go test ./arrow/csv -run Timestamp -count=1
  • go test -race ./arrow ./arrow/scalar ./arrow/array ./arrow/compute ./arrow/csv -run Timestamp -count=1

Are there any user-facing changes?

Timestamp string parsing now rejects mismatched timezone presence:

  • timezone-less type plus no offset: accepted
  • timezone-less type plus an offset: rejected
  • timezone-aware type plus an offset: accepted
  • timezone-aware type without an offset: rejected

Explicit offsets still determine the represented instant, and mixed-case UTC metadata remains accepted. CSV output now includes the offset for timezone-aware timestamps.

@fallintoplace fallintoplace changed the title fix(arrow/scalar): honor timestamp timezones in ParseScalar fix(arrow/scalar): honor timezones in ParseScalar Aug 5, 2026
@fallintoplace fallintoplace changed the title fix(arrow/scalar): honor timezones in ParseScalar fix(arrow/scalar): parse zoned timestamps consistently Aug 6, 2026
@fallintoplace
fallintoplace marked this pull request as draft August 7, 2026 10:16
@fallintoplace fallintoplace changed the title fix(arrow/scalar): parse zoned timestamps consistently fix(arrow/scalar): align timestamp timezone parsing Aug 7, 2026
@fallintoplace
fallintoplace marked this pull request as ready for review August 12, 2026 17:57
@fallintoplace
fallintoplace marked this pull request as draft August 12, 2026 17:58
@fallintoplace
fallintoplace marked this pull request as ready for review August 12, 2026 18:33
@zeroshade

Copy link
Copy Markdown
Member

@fallintoplace need to fix the failing tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants