Skip to content

feat: add serialization support for Set, Map, Date, RegExp, and BigInt via ISerializer contract#1136

Merged
Belphemur merged 6 commits into
developfrom
copilot/add-support-for-sets-and-maps
Mar 10, 2026
Merged

feat: add serialization support for Set, Map, Date, RegExp, and BigInt via ISerializer contract#1136
Belphemur merged 6 commits into
developfrom
copilot/add-support-for-sets-and-maps

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 10, 2026

Set, Map, RegExp, and BigInt don't survive JSON.stringify — Sets/Maps become {}, BigInt throws. Dates worked via a regex heuristic that could false-positive on ISO-looking strings. This adds a __type/__value envelope pattern with a pluggable ISerializer interface.

Changes

  • ISerializer contract (src/adapter/data/ISerializer.ts) — type, serialize(), deserialize(), test() methods for custom type support
  • Built-in serializers (src/adapter/data/Serializers.ts) — DateSerializer, SetSerializer, MapSerializer, RegExpSerializer, BigIntSerializer with defaultSerializers exported as a frozen (immutable) array
  • JsonAdapter rewrite — replaced regex-based date reviver with serializer-driven replacer/reviver. Uses this[key] in the replacer to access raw values before .toJSON() runs (needed for Date). The legacy parseDates parameter has been removed — Date handling is now managed entirely through the serializer system. The reviver uses a Map<string, ISerializer> keyed by type for O(1) deserialization lookup
  • __type escaping — user data that naturally contains __type properties is automatically escaped during serialization and unescaped during deserialization, preventing false type matching
  • Config.addSerializer() — new method on Config to easily register custom serializers without manually constructing adapters
  • ExportsISerializer, all serializer classes, and defaultSerializers exported from package entry point
  • README updated — documents built-in type serialization, serialized format table, and custom serializer usage via both Config.addSerializer() and defaultSerializers spread pattern

Serialized format

{ "__type": "Set", "__value": [1, 2, 3] }
{ "__type": "Map", "__value": [["key", "value"]] }
{ "__type": "Date", "__value": "2023-01-01T00:00:00.000Z" }
{ "__type": "RegExp", "__value": { "source": "^test$", "flags": "gi" } }
{ "__type": "BigInt", "__value": "9007199254740993" }

Consumer extensibility

import { JsonDB, Config, ISerializer } from 'node-json-db';

const urlSerializer: ISerializer = {
  type: "URL",
  serialize: (value: URL) => value.href,
  deserialize: (value: string) => new URL(value),
  test: (value: any) => value instanceof URL,
};

const config = new Config('mydb');
config.addSerializer(urlSerializer);
const db = new JsonDB(config);

Or with full control via defaultSerializers:

import { JsonAdapter, FileAdapter, defaultSerializers } from 'node-json-db';

const adapter = new JsonAdapter(
  new FileAdapter('mydb.json', false),
  false,
  [...defaultSerializers, mySerializer]
);

Breaking changes

  • Date serialization now uses { "__type": "Date", "__value": "..." } instead of bare ISO strings. Existing databases with dates stored as plain ISO strings will read them back as strings, not Date objects.
  • The parseDates parameter has been removed from both Config and JsonAdapter constructors. Date handling is now controlled by the presence of DateSerializer in the serializers list.
Original prompt

This section details on the original issue you should resolve

<issue_title>Is there official support/no support for Sets?</issue_title>
<issue_description>I swear I read before something about Sets not working, but I'm striking out finding it again. Is it supposed to handle Sets, or is that out of scope currently? It looks like it saves them as empty objects.</issue_description>

<agent_instructions>Let implement support for maps, set, date, etc and define contract that could be used for custom serialization. We should use a __type as differentiator and update the existing jsonAdapter</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@Belphemur Currently, it doesn't support Set, Map etc ...

For that we'd need to write/update the custom json serializer and deserializer to support those type</comment_new>


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 2 commits March 10, 2026 11:40
…t via __type pattern

- Add ISerializer contract interface for custom type serialization
- Add built-in serializers: DateSerializer, SetSerializer, MapSerializer, RegExpSerializer, BigIntSerializer
- Update JsonAdapter to use serializer-based replacer/reviver (replaces legacy regex date parsing)
- Export all serializer types for consumer extensibility
- Add comprehensive tests for all built-in types, nested types, and custom serializers

Co-authored-by: Belphemur <[email protected]>
Copilot AI changed the title [WIP] Add support for Sets and Maps in JSON serialization feat: add serialization support for Set, Map, Date, RegExp, and BigInt via ISerializer contract Mar 10, 2026
Copy link
Copy Markdown
Contributor

@stefnotch stefnotch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this break the roundtripping guarantee? As in, if I have something that is valid to serialize, then the following should always hold true: deserialize(serialize(x)) == x

But if I pass in an object that already has a __type and __value, then this seems to be broken? At least I don't see any place in the code where those things get escaped.

As in, I think if I do deserialize(serialize({ "__type": "RegExp", "__value": { "source": "hello\\s+world", "flags": "gi" } }) then I'd suddenly get a regular expression back.

Correctly implementing it requires escaping https://stackoverflow.com/questions/29085197/how-do-you-json-stringify-an-es6-map/73155667#73155667

@Belphemur
Copy link
Copy Markdown
Owner

/gemini review

@Belphemur Belphemur marked this pull request as ready for review March 10, 2026 11:57
Copilot AI review requested due to automatic review settings March 10, 2026 11:57
@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.66%. Comparing base (6632572) to head (ff62c96).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1136      +/-   ##
===========================================
+ Coverage    99.60%   99.66%   +0.05%     
===========================================
  Files           12       13       +1     
  Lines         1531     1770     +239     
  Branches       226      362     +136     
===========================================
+ Hits          1525     1764     +239     
  Misses           6        6              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds serialization support for JavaScript types that don't survive JSON.stringify natively: Set, Map, RegExp, and BigInt. It also improves the existing Date serialization by replacing the previous regex-heuristic approach with a pluggable ISerializer interface and a __type/__value envelope pattern. This is a breaking change for existing databases that stored Date values as bare ISO strings.

Changes:

  • Adds ISerializer interface and five built-in serializers (DateSerializer, SetSerializer, MapSerializer, RegExpSerializer, BigIntSerializer) with defaultSerializers array
  • Rewrites JsonAdapter's replacer/reviver to use the new serializer chain instead of a regex-based date reviver
  • Exports new types and classes from the package entry point (src/JsonDB.ts)

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/adapter/data/ISerializer.ts New interface defining the serializer contract (type, serialize, deserialize, test)
src/adapter/data/Serializers.ts Five concrete serializer implementations + defaultSerializers export
src/adapter/data/JsonAdapter.ts Rewritten replacer/reviver using the serializer chain; new serializers constructor parameter
src/JsonDB.ts Re-exports ISerializer, serializer classes, and defaultSerializers for consumers
test/adapter/adapters.test.ts Integration tests for new serializer behavior in JsonAdapter
test/adapter/Serializers.test.ts Unit tests for each serializer class and defaultSerializers

Comment thread test/adapter/adapters.test.ts Outdated
Comment thread src/adapter/data/Serializers.ts Outdated
Comment thread src/JsonDB.ts
Comment thread src/adapter/data/JsonAdapter.ts Outdated
@Belphemur
Copy link
Copy Markdown
Owner

@copilot apply changes based on the comments in this thread
Remove the parseDate config too, we don't need it anymore, we'll use the new serializer.

Add a AddCustomSerializer method to the Config to simplify adding new serializer for user of the lib. Update the readme too.

Also check this comment and see if we can fix the issue
#1136 (review)

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 10, 2026

@copilot apply changes based on the comments in this thread
Remove the parseDate config too, we don...

All changes applied in c33e9aa:

  • Removed parseDates from both JsonAdapter and Config constructors entirely
  • Added Config.addSerializer() method with JSDoc and usage example
  • Froze defaultSerializers with Object.freeze() to prevent accidental mutation
  • Fixed require() → ES import in test file
  • Added __type escaping (inspired by the SO answer) — user data that naturally contains __type properties is now escaped during serialization and unescaped during deserialization, preventing false type matching
  • Updated README with type serialization table, usage examples, and custom serializer documentation via Config.addSerializer() and defaultSerializers spread pattern

@Belphemur
Copy link
Copy Markdown
Owner

@all-contributors add @stefnotch for code

@allcontributors
Copy link
Copy Markdown
Contributor

@Belphemur

I've put up a pull request to add @stefnotch! 🎉

@Belphemur
Copy link
Copy Markdown
Owner

@stefnotch thank you for the review and the great answer on stack overflow.
I've added your version of the code with the escaping and added you in the contributors :)

@Belphemur Belphemur requested a review from stefnotch March 10, 2026 12:46
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a robust and extensible serialization mechanism for Set, Map, Date, RegExp, and BigInt types, effectively addressing the issue of these types not being correctly handled by JSON.stringify. The implementation utilizes a __type/__value envelope pattern with a pluggable ISerializer interface, allowing for custom type serialization. The JsonAdapter has been refactored to integrate this new system, and the Config class now provides a convenient addSerializer() method for registering custom serializers. Comprehensive tests have been added to validate the new functionality, covering various scenarios including edge cases and the __type escaping mechanism. The README.md has been updated to reflect these significant changes, providing clear documentation and examples for both built-in and custom serializers. Overall, this is a well-implemented and thoroughly tested feature.

Comment thread README.md
Repository owner deleted a comment from allcontributors Bot Mar 10, 2026
@Belphemur
Copy link
Copy Markdown
Owner

@copilot inside the JsonAdapter, let's use a Map of Serializer that is based on the type key for faster deserialization.

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 10, 2026

@copilot inside the JsonAdapter, let's use a Map of Serializer that is based on the type key for faster deserialization.

Done in bb0318c. The constructor now builds a ReadonlyMap<string, ISerializer> keyed by type from the serializers array, and the reviver uses serializerMap.get(value.__type) for O(1) lookup instead of a linear scan.

The writeAsync replacer still uses the array with test(rawValue) since serializer selection there requires runtime type-checking (instanceof/typeof) — there's no key to look up until a match is found.

@Belphemur Belphemur merged commit ac65a44 into develop Mar 10, 2026
10 checks passed
@Belphemur Belphemur deleted the copilot/add-support-for-sets-and-maps branch March 10, 2026 13:18
github-actions Bot pushed a commit that referenced this pull request Mar 10, 2026
# [2.6.0](v2.5.0...v2.6.0) (2026-03-10)

### Features

* add serialization support for Set, Map, Date, RegExp, and BigInt via ISerializer contract ([#1136](#1136)) ([ac65a44](ac65a44))
@github-actions
Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 2.6.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Is there official support/no support for Sets?

4 participants