Skip to content

CLAUDE.md

Latest commit

 

History

History
361 lines (270 loc) · 16.6 KB

CLAUDE.md

File metadata and controls

361 lines (270 loc) · 16.6 KB
Raw

TagDataTranslation Library - Claude Code Context

Project Overview

TagDataTranslation is a C# library implementing GS1's Tag Data Translation (TDT) specification for encoding and decoding EPC (Electronic Product Code) identifiers used in RFID tags.

Supported Standards

  • TDT 2.2 - Full support for all standard EPC schemes
  • TDS 2.3 - Support for '+' and '++' schemes with hostname encoding
  • Standards in markdown are in the parent Mimasu repo: docs/standards/md/gs1/tdt/ and docs/standards/md/gs1/tds/

Tech Stack

  • .NET 8.0 / 9.0 / 10.0 (multi-targeting)
  • xUnit for testing
  • JSON-based scheme definitions (in src/TagDataTranslation/Schemes2/ folder)

Key Components

TDTEngine.cs

The main translation engine. Key methods:

  • Translate(input, inputFormat, parameterList) - Main translation method
  • ProcessInput() - Parses input and extracts fields
  • ProcessOutput() - Formats output to requested level

Scheme Files (src/TagDataTranslation/Schemes2/*.json)

JSON definitions for each EPC scheme containing:

  • Level definitions (BINARY, BARE_IDENTIFIER, GS1_DIGITAL_LINK, etc.)
  • Field patterns and extraction rules
  • Encoding/decoding rules

src/TagDataTranslation/Encoding/ folder

Helper classes for specific encoding methods:

  • HostnameEncoder.cs - TDS 2.3 hostname encoding with optimizations

TDS 2.3 Implementation Details

'+' Schemes (Plus schemes)

Single-plus schemes like SGTIN+, SSCC+, etc. that:

  • Use variable-length serial encoding
  • Support GS1 Digital Link URIs with id.gs1.org hostname
  • Do NOT encode custom hostnames

'++' Schemes (Plus-plus schemes)

Note: The '++' scheme JSON files are custom implementations created by Claude, not official GS1 scheme definitions. They are based on the TDS 2.3 specification but the JSON schema files themselves are not from GS1.

Double-plus schemes like SGTIN++, SSCC++, etc. that:

  • Include all features of '+' schemes
  • Additionally encode a custom hostname in binary
  • Support branded GS1 Digital Link URIs (e.g., https://coca-cola.com/01/...)

Hostname Encoding (Section 14.5.16)

Two methods supported:

  1. Code 40 (indicator bit 0): For uppercase-only hostnames

    • 16 bits per 3 characters, r = 1600i1 + 40i2 + i3 + 1
    • Character set per TDS table 14-8: PAD=0, A-Z=1-26, '-'=27, '.'=28, ':'=29, 0-9=30-39 (shared table EncodedAICodec.UrnCode40Table)

  2. 7-bit ASCII with optimizations (indicator bit 1): For mixed-case hostnames

    • Uses optimization tables for common TLDs and subdomains
    • .com, .org, .net etc. encoded as single 7-bit sequence
    • id., www., qr. encoded as single 7-bit sequence
    • Country TLDs encoded as 14-bit sequences

Important: The hostname length field indicates number of 7-bit sequences, NOT number of output characters.

Variable-Length Alphanumeric Encoding (Section 14.5.6)

Format: 3-bit encoding indicator + 5-bit length + variable data

Indicator Method Bits per char
0 Numeric ~3.32 bits/digit
1 Upper hex 4 bits
2 Lower hex 4 bits
3 Base64 URI-safe 6 bits
4 7-bit ASCII 7 bits
5 URN Code 40 ~5.33 bits

'++' Scheme BINARY Pattern

All '++' schemes require trailing ([01]*) in their BINARY pattern to capture variable-length serial and hostname data after the fixed fields.

Example for SGTIN++:

"pattern": "^11111101([01])([01]{3})([01]{56})([01]*)"

'++' Scheme Structure

Most '++' schemes follow this structure:

  • Header (8 bits) - unique per scheme
  • DataToggle (1 bit) - +AIDC data indicator
  • Filter (3 bits)
  • Fixed fields (scheme-specific, BCD encoded)
  • Serial (variable-length alphanumeric)
  • Hostname (1-bit encoding + 6-bit length + data)

Validation Errors

Besides the TDT 2.2 exception codes, the engine throws:

  • TDTInvalidCheckDigit — a GS1 key in the input has a wrong check digit (Gs1CheckDigitValidator)
  • TDTInvalidSchemeDefinition — invalid character-set regex, unknown rule function or encoding in a scheme file
  • TDTUndefinedField — a grammar field has no value (never silently omitted), or dataToggle=1 without +AIDC data

Known Issues

TDS 2.3 Standard Errata

See docs/TDS-2.3-Errata.md for documented errors in the TDS 2.3 specification, including:

  • SGTIN++/DSGTIN++ hostname errors in E.3
  • SSCC++/ITIP++ header errors in E.3

Note: Errors in '++' scheme JSON files are not "errata" since the JSON files are custom implementations, not from GS1. Only errors in the official TDS 2.3 specification document should be documented in the errata file.

Testing Requirements

Tests are in a private submodule. After cloning, initialize:

git submodule update --init --recursive

All tests must pass. Tests are never allowed to fail. Before committing any changes, ensure all tests pass by running:

dotnet test test/TagDataTranslation.Tests/TagDataTranslation.Tests.csproj

Test Coverage

Use the coverage script to generate an HTML coverage report:

./scripts/coverage.sh

This runs all tests with coverlet, generates a Cobertura XML file, and produces an HTML report at coveragereport/index.html (opens automatically on macOS).

Coverage-driven test improvement process:

  1. Run ./scripts/coverage.sh to generate the report
  2. Read the Cobertura XML (test/TagDataTranslation.Tests/TestResults/*/coverage.cobertura.xml) to identify uncovered lines/branches per class
  3. Prioritize gaps by impact: internal classes with InternalsVisibleTo access can be tested directly; otherwise test through TDTEngine.Translate()
  4. Write tests targeting the uncovered lines, run the suite, re-run coverage to verify improvement

Key coverage notes:

  • InternalsVisibleTo("TagDataTranslation.Tests") is set in AssemblyInfo.cs — internal classes like EncodedAICodec, VariableLengthFieldCodec, PlusPlusFieldConverter can be tested directly
  • Generated JSON serializer code (TdtJsonContext, TableJsonContext) will always have partial coverage — this is expected
  • Table lookup classes (TableB, TableK, TableE) are exercised indirectly through scheme translations; their query methods may show low coverage if only used by specific scheme paths
  • coveragereport/ and TestResults/ are gitignored

Build Commands

# Build all targets
dotnet build src/TagDataTranslation/TagDataTranslation.csproj

# Run tests
dotnet test test/TagDataTranslation.Tests/TagDataTranslation.Tests.csproj

# Run specific test categories
dotnet test --filter "FullyQualifiedName~TDS23"
dotnet test --filter "FullyQualifiedName~TDT22"

# Run benchmarks
dotnet run -c Release --project test/TagDataTranslation.Benchmarks

# Build npm WASM package
cd npm && npm run build

# Run npm smoke test
cd npm && node test/smoke.js

npm Package (@mimasu/tdt)

Architecture

The npm package wraps the .NET library via WebAssembly. The build pipeline:

  1. dotnet publish compiles the WASM project (sdk/wasm/) targeting browser-wasm
  2. npm/scripts/build.js copies the _framework/ output to npm/dist/wasm/
  3. npm/dist/index.js loads the .NET WASM runtime and exposes createEngine()

Key Files

Path Description
sdk/wasm/TagDataTranslation.Wasm.csproj WASM project (net10.0, browser-wasm)
sdk/wasm/JsInterop.cs JSExport methods callable from JavaScript
sdk/wasm/Program.cs Minimal entry point required by runtime
sdk/wasm/main.js WASM module entry point
npm/package.json npm package metadata
npm/dist/index.js CJS wrapper with createEngine()
npm/dist/index.mjs ESM re-export
npm/dist/index.d.ts TypeScript type definitions
npm/scripts/build.js Build script (WASM compile + license copy)
npm/test/smoke.js Smoke test for encode/decode/tryTranslate
examples/NodeApp/ Example Node.js app using the published package

.NET 10 WASM Gotchas

  • SDK: Use Microsoft.NET.Sdk (not Microsoft.NET.Sdk.BlazorWebAssembly) for library-style WASM
  • AllowUnsafeBlocks: Required — the JSExport source generator emits unsafe code in .NET 10
  • JsonSerializerIsReflectionEnabledByDefault: Must be true — trimmed WASM disables reflection-based JSON by default, but TDTEngine uses System.Text.Json with reflection to load scheme files
  • TrimmerRootAssembly: Must include TagDataTranslation — without this, the IL trimmer strips model constructors, causing DeserializeNoConstructor errors at runtime
  • Entry point: .NET 10 requires a Program.cs with Main (even for library-style WASM)
  • Output path: .NET 10 outputs to AppBundle/_framework/ (not Blazor's wwwroot/_framework/)
  • getAssemblyExports: Returns a Promise in .NET 10 — must await it
  • Initialization order: Call dotnet.create(), then getAssemblyExports(), then runMain()

Publishing to npm

Pushing a v* tag publishes the package via .github/workflows/release-npm.yml using npm trusted publishing (GitHub OIDC, no NPM_TOKEN). The trusted publisher is configured on npmjs.com for @mimasu/tdt: repository dannyhaak/TagDataTranslation, workflow release-npm.yml.

To publish an existing tag again (for example after a failed run), run the workflow manually:

gh workflow run release-npm.yml -R dannyhaak/TagDataTranslation -f tag=v4.0.0

A re-run of an old run would use the workflow file from that tag, so use workflow_dispatch instead.

Local build only (no publish):

cd npm && npm run build

The build script auto-copies LICENSING.md from the repo root to npm/LICENSE.md (gitignored) so the license ships with every publish.

Release Checklist

  • Always update CHANGELOG.md before tagging a new release. Follow Keep a Changelog format with Added/Changed/Fixed sections.

Package Details

  • Scope: @mimasu (public)
  • License: BSL-1.1
  • Size: ~2.6 MB compressed, ~15.7 MB unpacked (includes .NET WASM runtime)
  • Engine requirement: Node.js >= 18.0.0

File Locations

Path Description
src/TagDataTranslation/ Main library
src/TagDataTranslation/Schemes2/ JSON scheme definitions
src/TagDataTranslation/Encoding/ Encoding helper classes
src/TagDataTranslation/Tables/ Lookup tables (Table F, K, E, B)
test/TagDataTranslation.Tests/ Unit tests
docs/ Errata and plans
docs/TDS-2.3-Errata.md Known errors in TDS 2.3 specification
docs/Scheme-Conversion-Errata.md Errors found in XML to JSON conversion

Adding New Schemes

  1. Create JSON scheme file in src/TagDataTranslation/Schemes2/
  2. Define all levels (BINARY, BARE_IDENTIFIER, etc.)
  3. Add BINARY pattern with appropriate capture groups
  4. For '++' schemes, add variableLengthField and hostnameField definitions
  5. Add tests in appropriate test file

Performance

Caching Architecture

The Translate() hot path uses several caches to avoid repeated work:

  • Regex cache: ConcurrentDictionary<string, Regex> in TDTEngine — compiled regex patterns shared across all engine instances
  • Character set regex cache: ConcurrentDictionary<string, Regex?> in RuleExecutor — caches ValidateCharacterset patterns (null = invalid pattern, reported as TDTInvalidSchemeDefinition)
  • Grammar token cache: ConcurrentDictionary<string, GrammarToken[]> in TDTEngine — parsed grammar strings cached as token arrays
  • Pre-sorted fields/rules: Option.Field sorted by Seq at load time; Level.ExtractRules/FormatRules pre-split and sorted at load time
  • BinaryConverter lookup tables: Static arrays for hex↔binary conversion (no per-character Convert calls)
  • Static grammar regex: Single compiled Regex instance for grammar parsing

All caches are static and thread-safe. They grow monotonically (no eviction) which is fine because the set of patterns/grammars is bounded by the scheme definitions.

Benchmarks

Run with dotnet run -c Release --project test/TagDataTranslation.Benchmarks. Results on Apple M1 Pro, .NET 8.0:

Benchmark Mean Allocated
SGTIN-96 encode 7.82 us 9.9 KB
SGTIN-96 decode 7.65 us 9.2 KB
SGTIN++ encode 24.31 us 75.3 KB
SGTIN++ decode 5.02 us 7.8 KB
HexToBinary (96-bit) 99 ns 480 B
BinaryToHex (96-bit) 54 ns 192 B
Failure (random hex) 17.82 us 504 B
Failure (random binary) 18.27 us 784 B

Performance Guidelines

  • Do NOT create new Regex(pattern) in the hot path — use GetCachedRegex(pattern) in TDTEngine or the RuleExecutor charset cache
  • Do NOT use .OrderBy().ToList() on fields/rules — they are pre-sorted at load time
  • Pre-size StringBuilder when output length is predictable (e.g., hex.Length * 4 for HexToBinary)

Debugging Tips

  • Use TryTranslateDetails() for detailed translation information
  • Binary patterns must match exactly - check bit counts
  • For '++' schemes, hostname length is in sequences, not characters
  • GS1 standards in markdown are in the parent Mimasu repo: docs/standards/md/gs1/

Parameter Keys (parameterList argument)

Six keys can be passed in the parameterList argument to Translate():

Key Source Description
filter TDT 2.2 Packaging level. Range is scheme-dependent: 0-7 (3-bit, most), 0-15 (4-bit, USDOD), 0-63 (6-bit, ADI)
gs1companyprefixlength TDT 2.2 GS1 Company Prefix length (6-12)
tagLength TDT 2.2 Target tag length in bits (e.g., 96, 198)
dataToggle TDT 2.2 AIDC data indicator (0/1) for '+' and '++' schemes
uriStem TDT 2.2 Custom URI stem for GS1_DIGITAL_LINK output. Default: https://id.gs1.org. Code: TDTEngine.cs ~line 800

uriStem vs hostname

Both refer to a domain name but come from different GS1 standards and serve different purposes:

  • uriStem (TDT 2.2): formatting parameter — controls the URL prefix in GS1_DIGITAL_LINK output. Not stored in tag binary.
  • hostname (TDS 2.3): identity data — encoded into '++' scheme binary via Section 14.5.16 hostname encoding. Lives on the physical tag.

hostname is NOT a parameterList key. It's a field in the '++' scheme BARE_IDENTIFIER level (e.g., gtin=...;serial=...;hostname=coca-cola.com). When decoding '++' binary, hostname is extracted and used to construct GS1_DIGITAL_LINK output.

Output Formats

The engine supports 7 output formats plus legacy aliases. Key notes:

  • ELEMENT_STRING is in the TDT 2.2 enum and the code's LevelType, but ZERO scheme JSON files define it — using it throws TDTLevelNotFound
  • GS1_AI_JSON replaced ELEMENT_STRING in TDT 2.1 as "a less ambiguous alternative"
  • TEI is only used by ADI-var (aerospace/defence)
  • GS1_AI_JSON output is built from the scheme grammar like any other level; values are JSON-escaped and +AIDC data is added as extra members

Important Implementation Notes

'+' Scheme JSON Files

The '+' scheme JSON files (SGTIN+.json, SSCC+.json, etc.) are from the GS1 standard. Only change them to fix a verified error, and record it in src/TagDataTranslation/Schemes/ERRATA.md. They support GS1 Digital Link URIs with ANY hostname, not just id.gs1.org.

'++' Scheme JSON Files

The '++' scheme JSON files are custom implementations and CAN be modified as needed to match the TDS 2.3 specification.

Scheme Selection Ambiguity

Several schemes can match the same non-binary input. The engine sorts candidates by:

  1. For a Digital Link on id.gs1.org and for GS1_AI_JSON input (no custom hostname), '+' schemes before '++' schemes
  2. Longest prefixMatch (so a custom-host Digital Link selects the '++' scheme, prefix https://)

CPI-var and CPI+ both match GS1_AI_JSON with tagLength=var; this is an open question for GS1 (TDT 2.3 review comment 5.1).

Field Name Consistency

Field names MUST match across all levels of a scheme:

  • BAD: itipBinary in BINARY level, itip in BARE_IDENTIFIER level (no conversion rule)
  • GOOD: itip in both BINARY and BARE_IDENTIFIER levels

'++' Scheme Specific Notes

DSGTIN++: Requires multiple options for different date types (like DSGTIN+):

  • Option 0: prodDate (date type indicator 0000)
  • Option 4: expDate (date type indicator 0100)
  • etc.

GRAI++: GS1_DIGITAL_LINK should capture 14-digit grai field (not 13 digits + hardcoded 0):

  • Pattern: \\/8003\\/([0-9]{14})...
  • Grammar: '/8003/' grai urlEscapedSerial

GDTI++: BARE_IDENTIFIER should use ;serial= separator:

  • Pattern: ^gdti=([0-9]{13});serial=...
  • Grammar: 'gdti=' gdti ';serial=' serial ';hostname=' hostname

ITIP++: Use combined itip field (18 digits = gtin + piece + total):

  • BARE_IDENTIFIER: itip=095211411234540102;serial=rif981;hostname=...
  • GS1_DIGITAL_LINK: /8006/095211411234540102/21/rif981

License

BSL 1.1 - See LICENSING.md