[FEATURE]: Add initial conformance tests to the a2ui core SDK

[jacobsimionato]

The agent SDK currently has platform agnostic conformance tests.

We should add similar conformance tests to the core framework.

Let's start by adding a suite for the data model, e.g. conforming that updateing the data model at different paths etc should work correctly for all edge cases.

This can replace the existing unit tests for the data model in many cases.

1. Research how the framework currently works for the agent SDK
2. Plan how to translate that to the core SDK, including where it should be described in blueprints and readmes etc so that coding agents can discover and manage it. perhaps there should be a skill explaining how our conformance tests work overall. We may want to do some changes to repo structure, e.g. add a /conformance folder with the conformance test definitions and YAML sources of truth
3. Define some tests cases for data model in the core SDK
4. Hook up the conformance tests to the Python and TS data model implementations for now

Please come up with detailed plan for all of this before doing the implementation.

[jacobsimionato]

Implementation Plan: Conformance Tests for A2UI Core Framework (DataModel)

1. Research & Architectural Alignment

   • The Agent SDK currently uses language-agnostic YAML conformance suites under agent_sdks/conformance/suites/, validated by a JSON schema (conformance_schema.json) and executed by language-specific test harnesses (such as test_conformance.py in a2ui_agent).
   • For the Client/Core Framework SDKs (a2ui_core in Python, @a2ui/web_core in TypeScript/Node, and future SDKs), we will introduce a top-level /conformance directory as the central home for Core Framework conformance test definitions and YAML sources of truth.

2. Repository Structure & Documentation

   • Add /conformance/README.md: Explaining the overall A2UI conformance testing strategy, folder structure, YAML schema, and how Client/Core SDK implementations run test harnesses against these suites.
   • Add /conformance/conformance_schema.json: JSON Schema for validating Core Framework YAML test suites (including action validation for set, get, subscribe, unsubscribe, verify_subscription, etc.).
   • Update AGENTS.md and root README.md to document /conformance/ so AI coding agents and contributors can discover and manage conformance suites.
   • Create .agents/skills/a2ui-conformance-testing/SKILL.md: A dedicated AI agent skill explaining how our conformance tests work overall, how to create/maintain YAML test cases, and how to hook up new language SDKs.
   • Reference a2ui-conformance-testing in .agents/skills/a2ui-implement-new-sdks-for-client-language/SKILL.md (in Phase 3: Core Model Layer -> Verification).

3. DataModel Conformance Test Suite (/conformance/suites/data_model.yaml)

   • Define a comprehensive suite of test cases covering all edge cases for DataModel:
     • Basic Get/Set: Root and nested paths, new path insertion.
     • Auto-vivification & Intermediate Creation: Automatic dict/list creation when setting nested paths.
     • Removing Properties: Removing keys when setting to null or undefined.
     • Array / List Handling: Numeric indices, appending, sparse arrays, and nested structures.
     • Subscriptions & Reactivity: Exact path match, ancestor match (container semantics), descendant match, root match, multiple subscribers, and unsubscribe behavior.
     • RFC 6901 JSON Pointer Escaping: ~1 (/), ~0 (~), and combined escape sequences.
     • Security & Error Handling: Prototype pollution protection (__proto__, constructor, prototype), leading-zero array indices, and non-numeric array indexing.

4. Hooking Up to Python and TypeScript DataModel Implementations

   • Python (a2ui_core): Add agent_sdks/python/a2ui_core/tests/test_data_model_conformance.py, implementing a pytest harness that loads conformance/suites/data_model.yaml, executes the steps against a2ui.core.state.DataModel, and asserts conformance.
   • TypeScript (@a2ui/web_core): Add renderers/web_core/src/v0_9/state/data-model-conformance.test.ts, implementing a Node test runner harness that loads conformance/suites/data_model.yaml, executes the steps against DataModel, and asserts conformance.
   • Add conformance/tests/test_conformance_yaml.py to ensure all Core YAML files validate against conformance/conformance_schema.json.

[jacobsimionato]

This sounds great - please go ahead!

[kimgranlund]

Great to see this moving — the DataModel-layer YAML suite in the approved plan looks like the right foundation.

One note for later scoping: over in agent-ui (a TypeScript A2UI renderer/validator implementation) we maintain a small implementation-agnostic conformance pack at a different layer — wire-payload/catalog validation (parse, schema, version, catalog-membership, ID-graph, pointer verdicts over createSurface/updateComponents/updateDataModel streams): 16 payload-in/verdict-out fixtures as pure JSON, runnable against any validator (packages/agent-ui/a2ui/conformance/). Three fixtures are this repo's own basic-catalog examples, byte-verbatim.

Not proposing it for this issue — the DataModel plan underway is its own thing and we wouldn't want to fork it. Flagging it as reference material, and as a possible sibling suite once the YAML convention here lands; happy to port our pack to that convention in a separate issue if there's interest.