Skip to content

RFC 038: Evidence exchange bridges #72

@dannymeijer

Description

@dannymeijer

Use this issue to track InQL RFC 038, proposed in PR #60 at docs/rfcs/038_evidence_exchange_bridges.md.

Area

  • Specification (RFCs)
  • Documentation

Summary

RFC 038 defines evidence exchange bridges between InQL's internal evidence model and external or adjacent formats. Bridges may project InQL evidence into downstream formats and may ingest external artifacts such as transformation manifests, source catalogs, schema catalogs, run results, and orchestration metadata as evidence inputs.

Motivation

InQL evidence should be useful outside InQL, and external project artifacts should be usable as evidence inputs when their source, scope, version, confidence, and lossiness are explicit. Without a bridge contract, each integration would reconstruct lineage, quality, metadata, and run evidence independently, causing semantic drift.

Proposal sketch

  • Define inbound and outbound evidence exchange bridges.
  • Preserve semantic target references and evidence versions where possible.
  • Preserve external artifact identity, source location, artifact version, confidence, and diagnostics on inbound records.
  • Allow external artifacts to seed metadata, lineage hints, quality observations, run observations, and target mappings without becoming authoritative InQL semantics.
  • Support transformation-framework artifacts such as manifests, catalogs, run results, source definitions, model metadata, tests, tags, exposures, and documentation scaffolds.
  • Cover representative artifact families such as dbt manifests and run results, Glue Data Catalog or Hive Metastore snapshots, Airflow or MWAA DAG metadata, Dagster assets, Prefect deployment metadata, OpenLineage events, DataHub or OpenMetadata catalog records, and Great Expectations-style quality results.
  • Require explicit loss reporting for fields or semantic distinctions that a target format cannot represent.

Alternatives considered

The draft rejects adopting one external lineage model internally, leaving all exchange to downstream systems, requiring hosted ingestion, and treating transformation project artifacts as authoritative semantics.

Impact / compatibility

This is additive. Existing evidence artifacts can remain local while future work adds explicit exchange paths. Bridge mappings and imported artifact schemas must be versioned or fingerprinted where possible.

Implementation notes (optional)

Review PR #60 for the draft. This work should keep external artifacts as evidence inputs or projections, not internal truth.

Checklist

  • I checked for an existing RFC or issue covering this.
  • I can describe how this impacts existing code and how to migrate (if needed).

Metadata

Metadata

Assignees

No one assigned

    Labels

    RFCRFC design and planningdocumentationImprovements or additions to documentationspecificationdocs/rfcs/ normative RFCs

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions