Skip to content

[Feature] Image-datetime format-aware serialization and deserialization#15

Merged
cjackett merged 7 commits intomainfrom
dev
Apr 7, 2026
Merged

[Feature] Image-datetime format-aware serialization and deserialization#15
cjackett merged 7 commits intomainfrom
dev

Conversation

@cjackett
Copy link
Copy Markdown
Collaborator

@cjackett cjackett commented Apr 7, 2026

Summary

This PR merges the dev branch into main to release v1.4.0. It makes image-datetime serialization and deserialization aware of the image-datetime-format field, so that datetime strings are written using the format declared in the iFDO file rather than Pydantic's default ISO 8601 output. The default format is %Y-%m-%d %H:%M:%S.%f with a space separator, as specified by the iFDO v2.1.0 schema. Loaded datetimes are parsed using the declared format and stored as UTC-aware objects. The format is inherited from the image-set-header to image-set-items, and from the first frame to subsequent frames for video items. A backwards-compatibility fallback ensures existing files with mismatched datetime strings continue to load without error.

Problem

The image-datetime field was being serialized by Pydantic's default datetime handler, producing ISO 8601 strings with a T separator (e.g. 2019-03-04T08:37:24Z) regardless of the declared image-datetime-format. This violated the iFDO v2.1.0 spec, which defines the default format as %Y-%m-%d %H:%M:%S.%f. Additionally, the image-datetime-format field was not used during deserialization, meaning datetime strings were not validated or parsed according to their declared format.

Solution

A new ifdo/_datetime/ module handles format-aware parsing and serialization. A model_validator pre-processes raw dict data on load, converting datetime strings to UTC-aware datetime objects using the declared format with a lenient fallback for mismatched strings. A model_serializer on iFDO propagates the correct format to each ImageData instance before serialization, and a field_serializer on image_datetime writes the string using that format. The image_datetime_format field has been moved from ImageCaptureFields to ImageCoreFields to reflect its role in both capture and core metadata.

Design

The format inheritance logic mirrors the iFDO spec: the image-set-header format is the default for all items, overridden per-item by a locally declared image-datetime-format, with video frames inheriting from the first frame. The serializer uses a private _image_datetime_format attribute set on a deepcopy of the iFDO object to avoid mutating state during serialization. The backwards-compatibility fallback in _check_image_item silently passes on ValueError, leaving non-matching strings for Pydantic's own ISO 8601 handling.

Impact

Datetime strings in newly written iFDO files will now use the spec-defined space-separated format, improving compliance. Existing files that contain ISO 8601 datetime strings load cleanly via the fallback. Consumers that parse image-datetime strings directly rather than using the Python library may see a format change in output files.

Testing

Tests for the new _datetime module cover format-matching (asserting parsed datetime objects), format fallback (asserting strings are left unchanged), format inheritance for images and videos, and round-trip load/save behaviour. All 8 tests pass.

Breaking Changes

The serialized format of image-datetime changes from ISO 8601 (e.g. 2019-03-04T08:37:24Z) to the spec-defined default (e.g. 2019-03-04 08:37:24.000000) for files that do not declare a custom image-datetime-format. Consumers that parse datetime strings directly from iFDO files should update accordingly.

Added Files

  • ifdo/_datetime/__init__.py: Public exports for the datetime module
  • ifdo/_datetime/_check_datetime.py: Format-aware datetime parsing with backwards-compatible fallback
  • ifdo/_datetime/_format.py: Default datetime format constant
  • ifdo/_datetime/_serialize_datetime.py: Format propagation logic for serialization
  • tests/test_check_datetime.py: Tests for datetime parsing and fallback behaviour
  • tests/test_serialize_datetime.py: Tests for datetime serialization

Modified Files

  • ifdo/models/ifdo.py: Added model_validator and model_serializer for format-aware datetime handling
  • ifdo/models/ifdo_core.py: Moved image_datetime_format here from ImageCaptureFields; added field_serializer for image_datetime
  • ifdo/models/ifdo_capture.py: Removed image_datetime_format (moved to ImageCoreFields)
  • pyproject.toml: Bumped version to 1.4.0
  • config/mypy.ini: Added pydantic mypy plugin
  • tests/test_load.py: Updated datetime assertions to expect UTC-aware objects
  • tests/test_save.py: Updated for new serialization format

GermanHydrogen and others added 7 commits July 1, 2025 10:11
Previously, `_check_image_item` raised a ValueError when the stored `image-datetime` string did not match `image-datetime-format`, breaking existing iFDO files where these fields were inconsistent — a common case when files were produced by tools that set `image-datetime-format` to describe the intended serialization format while Pydantic serialized the datetime as ISO 8601 regardless. The validator now catches parse failures and leaves the string unchanged, allowing Pydantic's own ISO 8601 handling to take over; tests are updated to assert this fallback behaviour.
Renamed the misspelled `check_datatime_format` ("data" + "time") to `check_datetime_format` ("date" + "time") across all call sites and exports, and corrected the companion validator method `_validate_image_datatime` to `_validate_image_datetime`.
Datetime format-aware serialization and deserialization constitutes new behaviour rather than a pure bug fix, as datetimes are now always loaded as UTC-aware objects and written in the spec-defined space-separated format instead of ISO 8601, warranting a minor version bump.
Hi @GermanHydrogen,

Thanks for this PR. I've added a couple of commits on top: one to make the datetime format parsing lenient so that existing iFDO files with mismatched datetime strings continue to load without error, and one to fix a typo in the exported function name (`check_datatime_format` -> `check_datetime_format`). I've also bumped the version to 1.4.0 given the behaviour change. I'll look to package this up and get it into the Marimba v1.1.0 release.
@cjackett cjackett merged commit 5b101bd into main Apr 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants