Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,4 +48,8 @@ pr.md

# MiMoCode
.mimocode/
.mimocode
.mimocode

# Python
__pycache__/
*.pyc
45 changes: 44 additions & 1 deletion docs/ADMIN_RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,31 @@ stellar contract invoke --id "$CONTRACT_ID" --source-account admin --network "$N
- **Partial success:** unknown / already-verified entries are counted as
`failed` and skipped; the batch does **not** abort. Inspect the returned
`BatchSummary` — a `success_rate < 100` is informational, not an error.
- For large lists use `scripts/bulk_verify.sh` (handles paging + RPC pacing).
- For large lists use `scripts/bulk_verify.sh` (see below).

#### `scripts/bulk_verify.sh` — paged `batch_verify` adapter (Issue #286)

```bash
CONTRACT_ID="$CONTRACT_ID" SOURCE=admin NETWORK="$NETWORK" \
./scripts/bulk_verify.sh --file usernames.txt \
[--dry-run] [--batch-size 25] [--pace-ms 500] \
[--audit-log verify-audit.log] [--continue-on-error] [--no-batch]
```

- Reads usernames one per line (blank lines and `#` comments ignored) and
calls `batch_verify` in pages of `--batch-size` (default and ceiling 25 =
`MAX_WRITE_BATCH`).
- **Fallback:** if the deployed contract has no `batch_verify`, the script
detects the CLI error on the first call, prints a note, and finishes the
run with per-username `verify`. Force this with `--no-batch`.
- **Idempotency:** a per-batch `successful < total` is reported as `PARTIAL`
(already-verified / unknown usernames), not a hard failure — it matches the
contract's `BatchSummary` semantics. In fallback mode an `AlreadyVerified`
error is likewise treated as success.
- `--dry-run` submits no transaction and logs the intended batches.
- `--audit-log` appends one JSON line per batch/username:
`{"timestamp","scope","target","network","result","detail"}`.
- Exit code is non-zero if any username ended unverified.

#### `revoke_verification` — withdraw verification

Expand Down Expand Up @@ -440,6 +464,25 @@ CONTRACT_ID="$CONTRACT_ID" SOURCE=admin NETWORK="$NETWORK" ./scripts/export_regi
`registry-export-<network>.json`. `SOURCE` must sign as the admin. Take an
export **before** any bulk verify/remove or dashboard migration.

#### `scripts/payout_allowlist.sh` — verified-only payout CSV (Issue #285)

```bash
CONTRACT_ID="$CONTRACT_ID" NETWORK="$NETWORK" \
./scripts/payout_allowlist.sh [--output allowlist.csv] \
[--include-unverified] [--include-bots]
```

- Pages the **unauthenticated** `get_public_paginated` read — no admin key;
`SOURCE` can be any funded identity (defaults to `default`).
- Writes `payout-allowlist-<network>.csv` with columns
`github_username,payout_address,stellar_address,verified,registered_at`
(`payout_address` falls back to `stellar_address` when unset).
- **Verified-only by default:** unverified registrations and records flagged
`is_bot` are excluded so a squatter or CI account is never paid. Opt back in
with `--include-unverified` / `--include-bots`.
- Read-only; pagination completeness is enforced (a stalled cursor aborts).
Payment submission is out of scope.

---

## Storage TTL Maintenance (Keeper)
Expand Down
42 changes: 42 additions & 0 deletions docs/DASHBOARD_SYNC.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,48 @@ asserts the local `verified` flag is `true` exactly once — i.e. the second
apply is detected as a duplicate by `(ledger_sequence, tx_hash)` and produces
no state change, no duplicate row, and no double count in any aggregate.

### Stable event ID (Issue #283)

The composite key above is fine for a relational store but awkward to pass
around as one value. Consumers that want a single opaque id per event should
derive it deterministically from the delivery envelope:

```
event_id = "{network_id}:{contract_id}:{ledger_sequence}:{tx_hash}:{event_index}"
```

- `network_id` — lower-hex SHA-256 of the network passphrase (same value as
`domain.network_id`, see [EVENT_INDEXING.md](EVENT_INDEXING.md#event-domain-separation-issue-226)).
- `contract_id` — the emitting contract's `C...` address (`domain.contract_id`).
- `ledger_sequence` — ledger the event was emitted in.
- `tx_hash` — hex transaction hash that emitted it.
- `event_index` — zero-based position of this event within that transaction's
event list. Required because one `batch_verify` / `batch_remove` transaction
emits many events with the same topic in the same ledger.

**Algorithm for a consumer:**

1. On each delivery, compute `event_id`.
2. If `event_id` is already in the applied-set, drop the delivery — it is a
reconnect replay, a catch-up re-read, or a worker retry. Do nothing else.
3. Otherwise apply the event (a last-write-wins field/record update, never an
increment — see the table above), then record `event_id` in the applied-set.
4. A full re-sync of the entire stream is therefore a no-op once every id has
been seen.

**Uniqueness scope: one contract instance on one network.** `network_id` and
`contract_id` are baked into the id, so it never collides across a redeploy or
another network — an upgrade re-emitting history, or a testnet stream leaking
into a public-network table, both produce ids that do not match anything in the
target store. Consumers that prefer a fixed-width id may hash the string
(e.g. SHA-256) — the hash inherits the same uniqueness scope.

A language-neutral fixture of replayed deliveries and the expected end state
lives at
[`tests/testdata/event_replay_fixture.json`](../tests/testdata/event_replay_fixture.json);
`tests/event_replay.rs` (`cargo test event`) applies it through a reference
consumer and asserts the stream is replay-safe.

See [ABI.md — Events](ABI.md#events) for the full topic/payload reference per
event type.

Expand Down
26 changes: 26 additions & 0 deletions docs/EVENT_INDEXING.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,29 @@ topic assignments, and no event was removed or renamed. Consumers that ignore
unknown fields need no change to keep working — they simply do not get the
deduplication benefit. Consumers that parse events positionally must be updated,
because `domain` is appended to each payload.

---

## Stable event ID (Issue #283)

For a single opaque id per event, derive it from the delivery envelope:

```
event_id = "{network_id}:{contract_id}:{ledger_sequence}:{tx_hash}:{event_index}"
```

`event_index` is the zero-based position of the event within its transaction —
required because one `batch_verify` / `batch_remove` transaction emits many
events sharing a topic in one ledger. Uniqueness scope is one contract instance
on one network. Full algorithm, the replay fixture
(`tests/testdata/event_replay_fixture.json`), and the reference consumer test
(`tests/event_replay.rs`, `cargo test event`) are in
[DASHBOARD_SYNC.md](DASHBOARD_SYNC.md#stable-event-id-issue-283).

## GraphQL subgraph schema (Issue #284)

[`docs/subgraph/`](subgraph/) has a Graph-Protocol `schema.graphql` for
`RegisteredEvent`, `VerifiedEvent`, and `RemovedEvent` plus a derived
`Contributor` aggregate, the event → entity mapping, a handler sketch, an
example query, and how to run a schema check and a local event stream without a
hosted indexer.
121 changes: 121 additions & 0 deletions docs/subgraph/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# TrustBridge Registry Subgraph (Issue #284)

The dashboard currently polls RPC directly. This directory is a **schema +
mapping spec** so Wave UIs can query contributor history from a subgraph instead
of writing an indexer from scratch. Operating a hosted indexer is out of scope
for this repo — [`scripts/event_indexer.sh`](../../scripts/event_indexer.sh)
remains the runnable local reference.

- [`schema.graphql`](schema.graphql) — entities for `RegisteredEvent`,
`VerifiedEvent`, `RemovedEvent`, plus a derived `Contributor` aggregate.

## Event → entity mapping

Field names and types below are copied from the on-chain `#[contractevent]`
definitions in [`src/events.rs`](../../src/events.rs). Do not add payload fields
the contract does not emit.

| Contract event | Topic symbol | Payload fields (from `src/events.rs`) | Entity |
|---|---|---|---|
| `RegisteredEvent` | `registered_event` | `github_username` (topic), `stellar_address`, `timestamp`, `sponsor: Option<Address>` | `RegisteredEvent` |
| `VerifiedEvent` | `verified_event` | `github_username` (topic), `stellar_address`, `timestamp`, `domain` | `VerifiedEvent` |
| `RemovedEvent` | `removed_event` | `github_username` (topic), `stellar_address`, `timestamp`, `domain` | `RemovedEvent` |

Notes and gotchas:

- **`RegisteredEvent` has no `domain` field.** Only `VerifiedEvent` and
`RemovedEvent` carry `EventDomain` (Issue #226). The schema reflects this —
`RegisteredEvent.domain` does not exist. Take `contractId` / `networkId` for a
registration from the deployment the subgraph is pointed at.
- **`EventDomain`** = `{ contract_id: Address, network_id: BytesN<32>,
contract_version: (u32,u32,u32), domain_version: u32 }`. Mapped as an embedded
type, not a queryable entity.
- **`reason_code`** belongs to `VerificationRevokedEvent` and `PausedEvent`, not
to any of the three events modelled here. It is intentionally absent.
- **Batch removes**: `batch_remove` emits one `RemovedEvent` per removed
contributor, all in one transaction. The entity `id` includes the per-tx
`eventIndex` so they do not collide.
- **Entity `id`** is the stable event id from
[`DASHBOARD_SYNC.md`](../DASHBOARD_SYNC.md#stable-event-id-issue-283):
`{networkId}:{contractId}:{ledgerSequence}:{txHash}:{eventIndex}`. Using it as
the primary key makes ingestion replay-idempotent for free.
- **`Contributor` aggregate** is last-write-wins keyed on `ledgerSequence`:
a `RegisteredEvent` sets `stellarAddress` and clears `verified`; a
`VerifiedEvent` sets `verified = true`; a `RemovedEvent` sets `removed = true`
and `stellarAddress = null`. A later `RegisteredEvent` clears `removed`.
- Treat the subgraph as a change-notification cache. After any gap, reconcile
against `get_public_paginated` on-chain — see `DASHBOARD_SYNC.md`.

## Mapping handler sketch

```ts
export function handleRegistered(ev: RegisteredEvent): void {
let e = new RegisteredEventEntity(eventId(ev)); // networkId:contractId:ledger:tx:index
e.githubUsername = ev.params.github_username;
e.stellarAddress = ev.params.stellar_address;
e.timestamp = ev.params.timestamp;
e.sponsor = ev.params.sponsor; // may be null
e.ledgerSequence = ev.ledger;
e.txHash = ev.transaction.hash;
e.contributor = ev.params.github_username;
e.save();
touchContributor(ev.params.github_username, ev.ledger, ev.params.timestamp, {
stellarAddress: ev.params.stellar_address, verified: false, removed: false,
});
}
```

`handleVerified` / `handleRemoved` are the same shape, reading `domain.*` from
the payload and updating the `Contributor` `verified` / `removed` flags.

## Running locally

No hosted service is required to develop against this schema:

```bash
# 1. schema check — the schema is plain GraphQL SDL
npx graphql-schema-linter docs/subgraph/schema.graphql
# or, with the Graph tooling:
npx --yes @graphprotocol/graph-cli@latest codegen --skip-migrations \
--output-dir /tmp/tb-subgraph docs/subgraph/schema.graphql

# 2. produce a local event stream to map against
CONTRACT_ID=C... ONESHOT=1 ./scripts/event_indexer.sh
# -> ./.indexer/events-<network>.jsonl (one raw event per line)

# 3. a full local Graph Node stack (Postgres + IPFS + graph-node) via
# docker-compose is the standard path once a manifest exists; that manifest
# is deployment-specific (contract address, start block) and is not checked
# in here.
```

## Example query

```graphql
{
# every currently-verified, not-removed contributor
contributors(where: { verified: true, removed: false }, orderBy: lastEventAt, orderDirection: desc) {
githubUsername
stellarAddress
firstRegisteredAt
lastEventAt
}

# full history for one username
registeredEvents(where: { githubUsername: "octocat" }, orderBy: ledgerSequence) {
id
stellarAddress
sponsor
timestamp
}
verifiedEvents(where: { githubUsername: "octocat" }, orderBy: ledgerSequence) {
id
timestamp
domain { contractId networkId contractVersion }
}
removedEvents(where: { githubUsername: "octocat" }, orderBy: ledgerSequence) {
id
timestamp
}
}
```
104 changes: 104 additions & 0 deletions docs/subgraph/schema.graphql
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
"""
TrustBridge registry subgraph schema (Issue #284).

Covers the three events a dashboard needs to reconstruct contributor history
without a bespoke indexer: RegisteredEvent, VerifiedEvent, RemovedEvent.
Field names and types mirror the on-chain `#[contractevent]` definitions in
`src/events.rs` exactly — see docs/subgraph/README.md for the event -> entity
mapping. No payload field here is invented; anything the contract does not emit
is not modelled.

This schema is written in the Graph Protocol subgraph dialect (entity
directives, `Bytes`, `BigInt`). It is a schema + mapping spec only; operating a
hosted indexer is out of scope for this repo.
"""

"""
Deployment provenance carried by VerifiedEvent and RemovedEvent (`EventDomain`,
Issue #226). RegisteredEvent does NOT carry this field — see README.
Embedded, not a top-level entity: it has no identity of its own.
"""
type EventDomain {
"Emitting contract instance (C... address)."
contractId: Bytes!
"SHA-256 of the network passphrase."
networkId: Bytes!
"Contract version at emit time, as [major, minor, patch]."
contractVersion: [Int!]!
"Schema version of the domain envelope itself. Currently 1."
domainVersion: Int!
}

"""
One `RegisteredEvent`: a GitHub username registered or re-registered to a
Stellar address. Topic: `registered_event`. `github_username` is the event
topic. `sponsor` is present only when a third party paid for the registration.
"""
type RegisteredEvent @entity(immutable: true) {
"Stable event id: {networkId}:{contractId}:{ledgerSequence}:{txHash}:{eventIndex} (Issue #283)."
id: ID!
githubUsername: String!
stellarAddress: Bytes!
timestamp: BigInt!
sponsor: Bytes
ledgerSequence: BigInt!
txHash: Bytes!
contributor: Contributor!
}

"""
One `VerifiedEvent`: an admin or Verifier marked a contributor verified.
Topic: `verified_event`. `github_username` is the event topic.
"""
type VerifiedEvent @entity(immutable: true) {
"Stable event id (Issue #283)."
id: ID!
githubUsername: String!
stellarAddress: Bytes!
timestamp: BigInt!
domain: EventDomain!
ledgerSequence: BigInt!
txHash: Bytes!
contributor: Contributor!
}

"""
One `RemovedEvent`: a registration removed by the registrant or an admin.
Topic: `removed_event`. `github_username` is the event topic. A large
`batch_remove` emits one `RemovedEvent` per successfully removed contributor,
all in the same transaction — `eventIndex` in the `id` keeps them distinct.
"""
type RemovedEvent @entity(immutable: true) {
"Stable event id (Issue #283)."
id: ID!
githubUsername: String!
stellarAddress: Bytes!
timestamp: BigInt!
domain: EventDomain!
ledgerSequence: BigInt!
txHash: Bytes!
contributor: Contributor!
}

"""
Derived, mutable aggregate: current best-known state of one GitHub username on
one contract instance. Rebuilt by the mappings from the event stream; the
contract's own storage remains the source of truth for reconciliation.
"""
type Contributor @entity {
"The GitHub username (lower-cased, as stored on-chain)."
id: ID!
githubUsername: String!
"Address from the most recent RegisteredEvent, or null once removed."
stellarAddress: Bytes
"True after a VerifiedEvent, false after a RemovedEvent or before first verify."
verified: Boolean!
"False until a RemovedEvent with no later RegisteredEvent."
removed: Boolean!
firstRegisteredAt: BigInt
lastEventAt: BigInt!
lastLedgerSequence: BigInt!
registrations: [RegisteredEvent!]! @derivedFrom(field: "contributor")
verifications: [VerifiedEvent!]! @derivedFrom(field: "contributor")
removals: [RemovedEvent!]! @derivedFrom(field: "contributor")
}
Loading