hyperpb decode engine: GMO/Encounter/DiskEncounter on arena-based lazy parsing

CLAUDE.md

@@ -89,6 +89,52 @@ Level 30+ is required for most methods to ensure data quality.
 
 Processing is gated by `ScanParameters` — a set of boolean flags resolved from config scan rules based on the request's `scan_context` and geographic location. This allows different scan areas to process different entity types.
 
+### Proto decode engines
+
+Golbat supports two protobuf decoders selected per method via `[proto_engine]`
+config. See `docs/adding-a-proto.md` for the full pipeline (accessor
+generation → engine registration → decode wiring → shadow coverage → tests)
+to follow when adding a new method or proto.
+
+**Standard decoder** (`"std"`): protobuf-go, the original stable decoder.
+
+**Hyperpb decoder** (`"hyperpb"`): High-performance arena-allocated decoder via `buf.build/go/hyperpb`, wrapped by generated `pogoshim` accessors. Generated by `cmd/pogoshimgen` via `scripts/genshim.sh` after `vbase.pb.go` updates. Arena-allocated message access is faster (cache-friendly layout, reduced allocations) but requires careful lifetime management:
+- Shim accessors are generated as thin wrappers. They never escape `decodeWithArena()` — no storage in caches, channels, or goroutines; data is copied on demand.
+- Payload bytes are owned by the arena and freed when `decodeWithArena()` returns. Getters that need persistence copy into independently-allocated structures (strings/bytes are cloned out of the arena; repeated strings via `StringList`).
+- The `pogoshim` generated file is several hundred KB and treated as generated code (spot-check only on review). `pogoshim/manual.go` hand-writes the one construct the generator doesn't emit — `map<K,V>` fields — following the generator's own conventions; see that file's header before extending it.
+
+**Coverage**: as of Wave 3, every client-proto method Golbat decodes runs
+through the proto engine — GMO, encounters, fort details, gym info, quests,
+map forts, routes, incidents, invasions, battle state, contests, size-
+leaderboard entries, station details, tappables, event RSVPs, and social
+(proxy request/response, friend details, player search). The one exception is
+the push-gateway lobby-count path (`decode_push_gateway.go`): it extracts a
+handful of scalars and nothing else, so it stays on plain `proto.Unmarshal`
+unconditionally — there's no `*pogo.X`/`pogoshim.X` retention hazard to design
+around, and std is already as cheap as that low-volume message type needs
+(see the doc comment on `decodePushGateway`).
+
+**Engine selection** (`[proto_engine]` config) resolves per method in this order (see `engineFor` in `protoengine.go`):
+1. A legacy explicit key (`gmo`, `encounter`, `disk_encounter`), if non-empty — wins outright, preserving pre-Wave-3 config compatibility.
+2. `[proto_engine.overrides].<method>`, if present and non-empty — one entry per method key (see `protoengine.go`'s `engMethod*` constants and `engineSpecs` table for the full list).
+3. `default` (config default: `"hyperpb"`).
+
+An unrecognized value (anything other than `"std"`, `"hyperpb"`, or `""`) logs a one-time `[PROTO_ENGINE]` warning at startup and falls back to running on `"std"`.
+
+- **Rollback**: set the method to `"std"` via its legacy key (gmo/encounter/disk_encounter) or an `[proto_engine.overrides]` entry, then restart — no rebuild needed.
+
+**Shadow verification** (config `shadow_sample_rate`, default 0.01):
+- Sampled dual-decode: both engines process a subset of packets, compare decoded field digests.
+- Mismatch → `golbat_proto_shadow_total{result="mismatch"}` incremented, `[PROTO_SHADOW]` log at ERROR.
+- Set to 0 to disable; useful for validating engine parity after updates.
+
+**Go compiler PGO** (`default.pgo`, build-time):
+- `default.pgo`, when present, is automatically applied by Go (>= 1.21) when building the main package via the default `-pgo=auto` build flag. It is captured via `make pgo-capture` (see below) and should be committed to the repo once a production profile exists — not yet present on this branch.
+
+**Hyperpb runtime PGO** (config `proto_engine.pgo`, default `true`):
+- Runtime warmup (`proto_engine.pgo`, DEFAULT OFF): when enabled, the first 256 packets per method (or 10 minutes, whichever comes first) record a live-traffic profile and the parser tables are recompiled from it. Off by default because hyperpb v0.1.3's Recompile duplicates repeated-string elements (see the TestHyperpbRecompileRepeatedStringDuplication canary — it fails when an upstream upgrade fixes this, which is the signal to re-enable).
+- Refresh periodically: `GOLBAT_URL=https://host:9001 GOLBAT_SECRET=... make pgo-capture` on a production instance captures a profile and commits it.
+
 ## Entity Model
 
 ### Struct Pattern

Makefile

@@ -12,4 +12,29 @@ FLAGS = go_json
 golbat: FORCE
 	go build -tags $(FLAGS) golbat
 
+# --- Profile-guided optimization --------------------------------------------
+# Go (>= 1.21) automatically applies ./default.pgo when building the golbat
+# main package — locally and in the Docker build (which copies the repo). So
+# committing default.pgo makes every build profile-guided; no flag needed.
+# Refresh it occasionally (game updates shift hot paths), then commit:
+#
+#   GOLBAT_URL=https://host:9001 GOLBAT_SECRET=... make pgo-capture
+#
+GOLBAT_URL ?= http://127.0.0.1:9001
+GOLBAT_SECRET ?=
+PGO_SECONDS ?= 120
+
+pgo-capture: FORCE  ## capture a CPU profile from a running golbat into default.pgo
+	curl -fsS $(if $(GOLBAT_SECRET),-H "X-Golbat-Secret: $(GOLBAT_SECRET)") \
+	  "$(GOLBAT_URL)/debug/pprof/profile?seconds=$(PGO_SECONDS)" -o default.pgo.tmp
+	mv default.pgo.tmp default.pgo
+	@echo "captured default.pgo ($$(du -h default.pgo | cut -f1)) — commit it to apply to all builds"
+
+pgo-status: FORCE  ## report whether builds will be profile-guided
+	@if [ -f default.pgo ]; then \
+	  echo "default.pgo present ($$(du -h default.pgo | cut -f1)) — builds are profile-guided"; \
+	else \
+	  echo "no default.pgo — builds are NOT profile-guided (run make pgo-capture against prod)"; \
+	fi
+
 FORCE: ;