Promote to main: close the stamp ownership hole

.env.example

@@ -163,6 +163,12 @@ POOL_ADMIN_ADDRESSES=
 # STAMP_POOL_IMMEDIATE_REPLENISH=true         # Purchase replacement immediately on release (default: true)
 # STAMP_POOL_STATE_FILE=data/pool_state.json  # State file for pool persistence across restarts
 # STAMP_OWNERSHIP_FILE=data/stamp_owners.json # Stamp ownership tracking for x402 enforcement
+
+# May any caller use a batch that is not in the ownership registry?
+# false (default) fails closed. Set true only to recover from a lost
+# STAMP_OWNERSHIP_FILE, which would otherwise lock owners out of batches
+# they paid for — re-register, then set it back.
+STAMP_OWNERSHIP_ALLOW_UNTRACKED=false
 #
 # Note: Pool will auto-purchase stamps when reserve is low.
 # Ensure Bee node wallet has sufficient xBZZ and xDAI.

CLAUDE.md

@@ -160,6 +160,9 @@ CORS (browser access):
 - `estimatedReadyAt`: ISO 8601 timestamp when stamp should be usable (null for external stamps)
 - `propagationStatus`: `"ready"` / `"propagating"` / `"unknown"` (null if undetermined)
 
+**Stamp ownership enforcement** (`app/services/stamp_ownership.py`, when `X402_ENABLED`):
+Every batch a caller can obtain is registered to them — pool acquire, direct purchase, and for-owner all call `register_stamp`. Batches the pool buys for its own inventory are registered as `POOL_OWNER` (`"pool"`) at purchase and on sync, and `check_access` **refuses** them: a caller receives one by acquiring it, which re-registers it to them. A batch absent from the registry is also refused; `STAMP_OWNERSHIP_ALLOW_UNTRACKED=true` restores the old permissive default and exists solely to recover from a lost registry file. Before #312 the pool's inventory was untracked and the untracked default was *allow*, so anyone could store data on batches the gateway had paid for — one production batch reached 50% utilisation without ever being acquired.
+
 **Access mode field** (included in all stamp responses):
 - `accessMode`: `"owned"` (exclusive to a wallet via x402), `"shared"` (free tier), or `null` (not tracked)
 
@@ -555,6 +558,69 @@ feature branches → dev → main
 - NEVER use `crtahlin/swarm_connect` - that is the upstream fork, not the main repo
 - Use `git remote -v` to verify remotes if unsure
 
+## Configuration: where it lives, and when a change actually takes effect
+
+Every one of these has bitten us. The pattern is always the same — a change is
+made, something reports success, and nothing happens.
+
+| what you change | reaches the running system when | overwritten by a deploy? |
+|---|---|---|
+| GitHub environment variable (`vars.*`) | next deploy writes it into the env file | n/a — this is the source of truth |
+| `/opt/swarm_connect.env`, `/opt/swarm_connect_dev.env` | **container is RECREATED** — not on restart | **yes**, rewritten from GitHub vars |
+| `/opt/swarm_connect_host.env` | next deploy appends it to `.env` | no — deploy only ever appends it |
+| `monitoring/alloy/config.alloy` | Alloy container is recreated | yes, but see below |
+| `deploy/Caddyfile` | **never automatically** — copy to `/etc/caddy/` and `systemctl reload caddy` | no |
+| `monitoring/alerting/*.json`, dashboards | **never automatically** — run `scripts/apply_grafana.py` | no |
+
+### `docker restart` does not reload `env_file`
+
+Docker reads `env_file` when it **creates** a container, not when it starts one.
+So editing `/opt/swarm_connect.env` and running `docker restart` changes nothing,
+and the container reports healthy while running the old configuration.
+
+```bash
+# WRONG — silently keeps the old values
+docker restart swarm_connect-provenance_gateway-1
+
+# RIGHT
+cd /opt/swarm_connect && docker compose up -d --force-recreate --no-deps provenance_gateway
+```
+
+This cost a real incident: the stamp pool target was lowered, the container was
+restarted, and it kept the old target of five while the pool held two — reporting
+`low_reserve_warning: true` and preparing to buy three replacements, the exact
+opposite of the change being made.
+
+### Changing a setting properly
+
+Both, in this order, or the change is temporary:
+
+1. `gh variable set NAME --repo datafund/swarm_connect --env production --body VALUE`
+   — so the next deploy keeps it.
+2. Edit `/opt/swarm_connect.env` and **force-recreate** the container — so it
+   applies now rather than at the next deploy.
+
+Doing only (2) means the next deploy reverts it. Doing only (1) means nothing
+changes until someone deploys.
+
+### A bind-mounted FILE pins its inode
+
+`config.alloy` is mounted as a single file. A file bind mount pins the inode, and
+`git pull` **replaces** the file rather than editing it, so the container goes on
+reading the old one indefinitely. Compose does not help: the service definition is
+unchanged, so `up -d` correctly does nothing.
+
+Changes to it were silently dead from 2026-08-25 until #301, which force-recreates
+Alloy on every deploy. If you add another single-file mount, it will need the same.
+
+### Not everything in the repo is deployed
+
+`deploy/Caddyfile` and `monitoring/alerting/alert-rules.json` are version
+controlled for review and history. Nothing applies them. They can drift from what
+is running, and have. Diff before assuming, and never overwrite the live
+`/etc/caddy/Caddyfile` from the repo without diffing first — doing that once
+reintroduced `tls internal` and broke TLS for two minutes.
+
 ## Deployment Workflow
 
 ### Auto-Deployment Triggers

TEST_STRATEGY.md

@@ -285,3 +285,19 @@ usable stamps.
 Reachability is not a gate. Whether production is up says nothing about whether this
 run intended to touch it. And opting in to live tests is not the same decision as
 opting in to spending, which is why the purchase has its own switch.
+
+## Stamp ownership: test the lock AND the door
+
+`tests/test_stamp_ownership.py` covers enforcement — untracked batches refused,
+pool-owned batches refused to paid, free-tier and anonymous callers alike, and
+the `STAMP_OWNERSHIP_ALLOW_UNTRACKED` escape hatch not reaching pool inventory.
+
+`tests/test_stamp_pool.py::TestPoolInventoryIsRegisteredAsOwned` covers the other
+half: that the pool actually registers what it buys, that a registration failure
+does not lose a batch already paid for, and that sync adopts inventory bought
+before the change.
+
+Both halves are needed, and the second is the one easy to omit. Enforcement tests
+construct registry state by hand, so they pass whether or not any code registers
+anything. That asymmetry is how #312 stayed open: the check was fine, nothing
+was claiming ownership of the pool's own batches, and every test agreed.

app/api/models/stamp.py

@@ -66,9 +66,14 @@ class StampDetails(BaseModel):
     )
 
     # --- Access Control Fields ---
-    accessMode: Optional[Literal["owned", "shared"]] = Field(
+    accessMode: Optional[Literal["owned", "shared", "pool"]] = Field(
         None,
-        description="Access mode: 'owned' (exclusive to a wallet via x402 payment), 'shared' (free tier, anyone can use), null (not in ownership registry)."
+        description=(
+            "Access mode: 'owned' (exclusive to a wallet via x402 payment), "
+            "'shared' (free tier, anyone can use), 'pool' (gateway inventory — "
+            "not usable directly; acquire it via POST /api/v1/pool/acquire, which "
+            "transfers ownership to you), null (not in ownership registry)."
+        )
     )
 
     # --- Calculated Fields ---

app/core/config.py

@@ -85,6 +85,13 @@ class Settings(BaseSettings):
 
     # Stamp ownership: file path for persisting stamp ownership records
     STAMP_OWNERSHIP_FILE: str = "data/stamp_owners.json"
+    # When a batch is absent from the ownership registry, may any caller use it?
+    #
+    # False (the default) fails closed. True restores the pre-#312 behaviour and
+    # exists for one situation: the registry file is lost, every batch becomes
+    # untracked at once, and legitimate owners would be locked out of batches
+    # they paid for. Turn it on to recover, re-register, and turn it off again.
+    STAMP_OWNERSHIP_ALLOW_UNTRACKED: bool = False
 
     # === Debug Proxy (read-only Bee diagnostics, signature-gated) ===
     # Comma-separated 0x addresses allowed to read Bee diagnostics via

app/services/stamp_ownership.py

@@ -19,6 +19,12 @@
 logger = logging.getLogger(__name__)
 
 
+# Owner value for batches the gateway bought for its own pool and has not handed
+# out. Not a wallet address, and deliberately not a valid one, so it can never
+# collide with a real owner.
+POOL_OWNER = "pool"
+
+
 class StampOwnershipManager:
     """
     Tracks stamp ownership and enforces access control.
@@ -116,9 +122,34 @@ def check_access(
         with self._lock:
             entry = self._registry.get(batch_id)
 
-        # Stamp not in registry -> allowed (backward compatibility)
+        # Stamp not in registry.
+        #
+        # This used to return allowed, for backward compatibility with batches
+        # predating the registry. The set that default actually covered was not
+        # legacy callers: every path by which a caller obtains a batch —
+        # /pool/acquire, POST /stamps/, /stamps/for-owner — registers it. What it
+        # covered was the POOL'S OWN INVENTORY, bought and funded by the gateway
+        # and not yet handed to anyone, which is precisely what no caller should
+        # be writing to. A production pool batch reached 50% utilisation without
+        # ever being acquired (#312).
+        #
+        # Pool inventory is now registered at purchase time as POOL_OWNER, so the
+        # untracked set is empty in normal operation and denying it costs nothing.
+        #
+        # STAMP_OWNERSHIP_ALLOW_UNTRACKED restores the old behaviour. It exists
+        # for one specific situation: STAMP_OWNERSHIP_FILE is lost or reset, every
+        # batch becomes untracked at once, and legitimate owners would otherwise
+        # be locked out of batches they paid for. Failing closed is right by
+        # default; this is the deliberate escape hatch, not a shrug.
         if entry is None:
-            return True, "stamp not tracked, backward compatibility"
+            if settings.STAMP_OWNERSHIP_ALLOW_UNTRACKED:
+                return True, "stamp not tracked, permissive mode enabled"
+            return False, "stamp is not registered to any owner"
+
+        # Gateway-owned pool inventory. Nobody may write to it directly — a
+        # caller receives one by acquiring it, which re-registers it to them.
+        if entry["owner"] == POOL_OWNER:
+            return False, "stamp belongs to the gateway pool; acquire it first"
 
         # Shared stamps -> always allowed
         if entry["owner"] == "shared":

app/services/stamp_pool.py

@@ -411,7 +411,55 @@ def add_stamp_to_pool(self, batch_id: str, depth: int, amount: int, ttl: int, la
             self._pool[batch_id] = stamp
             logger.info(f"Added stamp {batch_id[:16]}... to pool (depth={depth})")
             self._save_state()
-            return stamp
+
+        # Register as gateway-owned, outside the lock — the ownership manager
+        # takes its own.
+        #
+        # Ownership used to be recorded only when a caller ACQUIRED a batch, so
+        # everything sitting in the pool was untracked, and check_access let any
+        # caller write to an untracked batch. A production pool batch reached 50%
+        # utilisation without ever being acquired: capacity the gateway had paid
+        # for, consumed by someone who never asked for it (#312).
+        #
+        # Registering at purchase closes the window entirely — the batch is owned
+        # from the moment it exists. Acquiring it re-registers it to the caller.
+        try:
+            from app.services.stamp_ownership import POOL_OWNER, stamp_ownership_manager
+            stamp_ownership_manager.register_stamp(
+                batch_id=batch_id,
+                owner=POOL_OWNER,
+                mode="pool",
+                source="pool_purchase",
+            )
+        except Exception as e:
+            # Never fail a purchase over bookkeeping — the batch exists and was
+            # paid for. Log loudly: until it is registered it is writable by
+            # anyone, which is the whole point of this change.
+            logger.error(
+                f"Failed to register pool ownership for {batch_id[:16]}...: {e}. "
+                "The batch is unprotected until registered."
+            )
+
+        return stamp
+
+    def _register_pool_ownership(self, batch_ids: Set[str]) -> None:
+        """Record pool-held batches as gateway-owned, so nobody may write to them."""
+        if not batch_ids:
+            return
+        try:
+            from app.services.stamp_ownership import POOL_OWNER, stamp_ownership_manager
+            for batch_id in batch_ids:
+                stamp_ownership_manager.register_stamp(
+                    batch_id=batch_id,
+                    owner=POOL_OWNER,
+                    mode="pool",
+                    source="pool_sync",
+                )
+        except Exception as e:
+            logger.error(
+                f"Failed to register pool ownership for {len(batch_ids)} batch(es): {e}. "
+                "They are unprotected until registered."
+            )
 
     async def sync_from_bee_node(self) -> int:
         """
@@ -504,6 +552,19 @@ async def sync_from_bee_node(self) -> int:
             if valid_ids != known_ids:
                 self._save_state(extra_ids=unreadable_ids)
 
+            # Adopt everything in the pool as gateway-owned.
+            #
+            # Registering at purchase (add_stamp_to_pool) only protects batches
+            # bought from now on. Batches already held predate that and would
+            # stay untracked — and therefore writable by anyone — for as long as
+            # they live, which is exactly the state that let a production batch
+            # reach 50% utilisation unasked (#312).
+            #
+            # Idempotent, and does not disturb a batch already owned by someone:
+            # only AVAILABLE batches are in the pool, and one acquired by a caller
+            # was removed from it at release.
+            self._register_pool_ownership(valid_ids)
+
             self._last_sync_ok = True
             return synced_count
 

app/services/swarm_api.py

@@ -9,7 +9,7 @@
 
 from app.core.config import settings
 from app.services.http_client import get_client
-from app.services.stamp_ownership import stamp_ownership_manager
+from app.services.stamp_ownership import POOL_OWNER, stamp_ownership_manager
 from app.services.metrics import bee_api_errors_total
 
 logger = logging.getLogger(__name__)
@@ -588,7 +588,18 @@ async def get_all_stamps_processed() -> List[Dict[str, Any]]:
             # Determine access mode from ownership registry
             ownership_info = stamp_ownership_manager.get_stamp_info(batch_id)
             if ownership_info:
-                access_mode = "owned" if ownership_info.get("mode") == "paid" else "shared"
+                owner = ownership_info.get("owner")
+                if owner == POOL_OWNER:
+                    # Gateway inventory. Reported distinctly because the
+                    # alternatives both lie: "shared" tells a client it may use
+                    # the batch, which check_access refuses, and null means
+                    # "unknown to the registry", which it is not. A caller gets
+                    # one of these by acquiring it, which re-registers it to them.
+                    access_mode = "pool"
+                elif ownership_info.get("mode") == "paid":
+                    access_mode = "owned"
+                else:
+                    access_mode = "shared"
             else:
                 access_mode = None