Skip to content

architecture.md

Latest commit

 

History

History
121 lines (94 loc) · 6.2 KB

architecture.md

File metadata and controls

121 lines (94 loc) · 6.2 KB
Raw

Architecture

relay is the BASE challenge plane for verifiable web scraping: task generation and sealing, work-unit coordination, L0–L5 proof verification, multi-factor scoring, and weight export for the BASE master.

Trust model: cryptographically-anchored trust-but-audit. Residual risks (for example TEE.fail on self-hosted hardware) are documented in THREAT_MODEL.md. This document describes the product architecture, not a deployment diary.

End-to-end flow

flowchart TB
  Gen[Task generator] --> Policy[Open-web policy gate]
  Policy --> Seal[Task seal]
  Seal --> WU[work_units surface]
  WU --> Assign[Assign to enclave workers]
  Assign --> Scrape[Complete scrape + ScrapeProof]
  Scrape --> Bridge[Result / fold intake]
  Bridge --> Verify[L0–L5 verify]
  Verify --> Score[Multi-factor scorer]
  Score --> Weights[get_weights]
  Master[BASE master / validators] --> WU
  Master --> Weights
Loading

Miners do not see plaintext confidential targets on the host-visible work-unit surface. Targets live in sealed ciphertext; workers with attested measurements (and where configured, key release) open material only inside the enclave.

Layers L0–L5

Verification is layered. L0–L2 form the authenticity spine used as a hard reward gate. L3–L5 contribute secondary factors and integrity outcomes once authenticity is viable.

flowchart LR
  P[ScrapeProof] --> L0[L0 sanity]
  L0 --> L1[L1 multi-TEE RATS]
  L1 --> L2[L2 cert + binding]
  L2 --> L3[L3 geo]
  L3 --> L4[L4 completeness / quorum]
  L4 --> L5[L5 anti-bot / integrity]
  L5 --> Gate{Authenticity gate}
  Gate -->|L1+L2 open| Credit[Eligible for weighted credit]
  Gate -->|closed| Zero[Zero economic credit]
Loading
Layer What it checks Scoring impact
L0 Schema sanity, field presence, nonce freshness / replay hygiene Reject short-circuits further layers; reward gate remains closed
L1 Multi-TEE remote attestation (RATS), measurement against allowlist, TCB posture Must pass (or admitted degraded path) for authenticity openness
L2 TLS certificate chain to target host + report-data binding to result hashes Completes authenticity open when combined with L1; forgery-class bind failure can trigger integrity slash
L3 Geo session: landmarks, RTT triangulation, IP/content corroboration composition Soft confidence and spoof resistance; not a substitute for L1+L2
L4 Completeness of the assigned scrape, assignment binding, quorum and canary behavior Completeness weight and cross-scrape consistency
L5 Anti-bot signals, integrity, attribution (including proxy-class awareness where set) Integrity and penalty path; fake-failure and cloak disputes route through audit rather than blind auto-slash

Implementation lives primarily under src/relay/verify/ (l0_sanity, l1_attest, l2_certificate / l2_binding, geo session hooks, l4_*, l5_*) composed by the verify pipeline and exposed at POST /internal/v1/verify.

Geo (L3)

L3 estimates whether the scrape relative to landmark RTT and related signals is consistent with the claimed zone, not absolute location proof.

Typical ingredients:

  • Landmark probes and physics-bounded latency upper limits
  • Triangulation / composition across landmarks
  • Optional IP geo and content corroboration factors
  • Standing and penalty hooks for geo spoof scenarios

Operator and probe surface: /internal/v1/geo/* and related standing reads. Geo confidence is a secondary factor; closed authenticity still yields zero reward credit.

Scoring surface

After verify outcomes are available, multi-factor scoring produces hotkey standing and the weight map served to BASE.

flowchart LR
  Auth[Authenticity L1+L2 gate] --> MF[Multi-factor composition]
  Comp[L4 completeness / quorum] --> MF
  Geo[L3 geo confidence] --> MF
  AB[L5 anti-bot / integrity] --> MF
  Pen[Penalties and audit] --> MF
  MF --> GW[GET /internal/v1/get_weights]
Loading
Surface Path family Role
Authenticity gate Verify + scoring authenticity Hard open/closed on L1+L2; closed means no economic credit
Completeness / quorum / canary L4 + scoring Reward fullness and replica consistency
Geo confidence L3 + scoring Secondary geographic consistency
Anti-bot / integrity L5 + penalties Integrity slash and rolling standing
TEE.fail tiering Confidentiality + scoring Managed-cloud vs self-hosted residual handling
Witness proxy Optional degraded tier Disabled by default; when enabled, scored strictly below TEE (WITNESS_PROXY.md)
Weights export GET /internal/v1/get_weights Hotkey → raw weight map for BASE master

Scoring and verify share honesty constraints: no absolute authenticity claim, no “trustless” framing of TEE quotes alone. Witness-only submissions are rejected with a closed reward gate unless the witness-proxy tier is explicitly enabled.

Policy and confidentiality

Concern Product behavior
Open-web policy Pre-seal gate (scheme/host, canonicalize, deny lists, SSRF, robots/ToS, PII and abuse screens). See OPEN_WEB_POLICY.md
Content confidentiality Seal / unseal for tasks and results; host-visible surfaces avoid plaintext confidential targets
Assignment boundary Higher-reward / sensitive tiers route with attestation and hosted-TEE residual posture in mind
Retention / PII Operator policy in RETENTION_AND_PII.md

Runtime shape

Item Detail
Framework FastAPI ASGI (relay.app:app)
Storage SQLite via SQLAlchemy async + aiosqlite
Tooling uv project, ruff, pytest with coverage gate in CI
Default challenge port 21080
Optional canary 21090
Optional mock-master 21110

Public health and capability surface includes /health and /version (capabilities advertise get_weights and work_units where enabled). Internal routes sit under /internal/v1 with bearer token and challenge slug.

Related docs