fix(liveuamap): restore resilient optional incident enrichment - #518
Merged
Conversation
BigBodyCobain
marked this pull request as ready for review
August 18, 2026 23:45
BigBodyCobain
force-pushed
the
fix/liveuamap-resilience-516-517
branch
from
August 19, 2026 05:50
b850a7b to
9924079
Compare
Owner
Author
|
Cleaned this PR back to the known-good LiveUAMap-only head The 10 later commits that introduced the unrelated GitHub has re-triggered CI and Docker Publish on the restored head; I am treating those fresh runs as the final merge gate. |
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rehabilitates LiveUAMap as optional enrichment for Global Incidents without making the paid API a requirement or degrading the GDELT baseline.
Fixes #516.
Fixes #517.
UX contract
#516 — Docker Playwright packaging
Root cause was the browser-cache ownership/location boundary, not a missing Python Playwright dependency: browser installation runs as root, while the container executes as
backenduserwith/appas HOME.PLAYWRIGHT_BROWSERS_PATH=/ms-playwrightbefore browser installation.USER backenduserthat the runtime user resolves an executable Chromium and that the matchingchromium_headless_shell-*binary exists and is executable.#517 — upstream payload drift
The old parser assumed
json.loads(ovens)was alwayslist[dict]and called.get()on every item. The reported current payload reaches the parser as strings, so that assumption crashes every region.New
liveuamap_parser.pyseparates representation decoding from provider/network logic and defensively supports bounded forms including:data,events,markers, etc.)Malformed/scalar entries are skipped, point coordinates are validated/range-checked, parser depth and candidate counts are bounded, and drift logs describe only payload shape rather than dumping raw data.
Provider resilience
LIVEUAMAP_API_URLis configured.Docker/ops
docker-compose.ymlnow actually forwards the documented browser override and the optional API provider settings:SHADOWBROKER_ENABLE_LIVEUAMAP_SCRAPERLIVEUAMAP_API_URLLIVEUAMAP_API_KEYLIVEUAMAP_API_AUTH_HEADERLIVEUAMAP_API_AUTH_SCHEMELIVEUAMAP_API_TIMEOUT_SAdded
docs/LIVEUAMAP.mdcovering provider order, failure semantics, Docker behavior, and the non-blocking UX contract.Tests / review
Focused coverage now gates CI for:
/api/liveuamap/*privileged proxy/CSRF boundaryA pre-PR diff review also caught and removed a superseded regression-test draft that referenced an earlier helper shape, preventing a latent full-suite failure.
Final CI status
Current head
9924079f5331a13f887d4f16d527758418154c48is green:The successful backend multi-arch Docker builds exercise the runtime-user Playwright packaging assertion added for #516.
Upstream limitation
I could not capture the reporter's exact current
ovenspayload from LiveUAMap because automated access from the review environment is currently answered with HTTP 403, and #517 has no payload attachment/comment yet. The fix therefore does not pretend one guessed schema is canonical; it normalizes the reported string case plus several bounded historical/common JSON shapes and fails safely on unknown shapes.Non-goals
Ready for review.