Skip to content

Add regen-determinism check to CI for entity-registry.yaml #758

Description

@rafaelscosta

Context

During investigation of #754 (which turned out to be a false alarm — see refutation comment there), discovered that PR #752 committed an entity-registry.yaml that did not match the output of running populate-entity-registry.js against the same source tree. The drift was 9 entries with reduced `dependencies` arrays.

Root cause was operator regen against a partially-edited working tree. Algorithm itself is deterministic.

A pre-merge CI check could catch this category of drift mechanically.

Proposal

Add a CI step (in validate-yaml.yml or a new validate-ids-registry.yml) that runs on every PR touching:

  • .aiox-core/development/agents/**
  • .aiox-core/development/tasks/**
  • .aiox-core/development/workflows/**
  • .aiox-core/development/templates/**
  • .aiox-core/development/checklists/**
  • .aiox-core/development/scripts/populate-entity-registry.js
  • .aiox-core/data/entity-registry.yaml

Steps:

  1. Run `node .aiox-core/development/scripts/populate-entity-registry.js`
  2. Compare against committed registry via `git diff --exit-code .aiox-core/data/entity-registry.yaml`
  3. Fail the check if any non-timestamp lines differ

Implementation sketch

```yaml

  • name: Validate IDS registry determinism
    run: |
    cp .aiox-core/data/entity-registry.yaml /tmp/registry-committed.yaml
    node .aiox-core/development/scripts/populate-entity-registry.js

    Strip timestamps before diffing (lastUpdated, lastVerified always change)

    python3 -c "
    import yaml
    def strip_ts(d):
    if isinstance(d, dict):
    for k in list(d.keys()):
    if k in ('lastUpdated', 'lastVerified', 'checksum'):
    del d[k]
    else:
    strip_ts(d[k])
    elif isinstance(d, list):
    for x in d:
    strip_ts(x)
    return d
    committed = strip_ts(yaml.safe_load(open('/tmp/registry-committed.yaml')))
    regenned = strip_ts(yaml.safe_load(open('.aiox-core/data/entity-registry.yaml')))
    if committed != regenned:
    import sys, json
    print('Registry drift detected. Re-run populate-entity-registry.js locally and commit the result.')
    sys.exit(1)
    print('Registry is in sync with sources.')
    "
    ```

Acceptance criteria

  • CI step added that fails when committed registry diverges from regen output (modulo timestamps + checksums)
  • Error message points to local remediation command
  • Documented in `docs/guides/release-procedure.md` as the new gate
  • Tested by intentionally desyncing in a draft PR and confirming the gate blocks

Cost / benefit

~15min implementation, ~5s per PR runtime overhead, catches the category of drift that produced #754 (false alarm) and would catch real drift before merge.

Source

Investigation in `outputs/audit/2026-05-18-issue-754-refutation.md` (local, gitignored). #754 closure comment has full evidence.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions