Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/validate-public-mirror.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Validate Public Mirror

on:
push:
branches:
- main
pull_request:
workflow_dispatch:

jobs:
validate-public-mirror:
runs-on: ubuntu-latest

steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Run cross-platform public mirror validator
shell: pwsh
run: |
python scripts/validate-public-mirror.py *> public-mirror-validation.log
$exitCode = $LASTEXITCODE
Get-Content public-mirror-validation.log
if ($exitCode -ne 0) {
Write-Error "Public mirror validation failed with exit code $exitCode"
exit $exitCode
}
- name: Upload public mirror validation log
if: always()
uses: actions/upload-artifact@v4
with:
name: public-mirror-validation-log
path: public-mirror-validation.log
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Clear Lake Watch is a late-prototype / early-MVP public dashboard for organizing

It is not official public-health guidance, an official advisory, a validated forecast, or a deployed sensor network.

This public mirror is a static reviewed snapshot generated on May 5, 2026, and it is not live lake conditions.

**Live public mirror:** [coreytshaffer.github.io/clear-lake-watch](https://coreytshaffer.github.io/clear-lake-watch/)

![Clear Lake Watch dashboard preview](assets/clear-lake-watch-preview.png)
Expand All @@ -19,6 +21,8 @@ It is not official public-health guidance, an official advisory, a validated for
- Private/local records, reviewer notes, raw field records, and trusted-review paths are excluded from the public mirror.
- Dashboard is not official guidance, a health advisory, a validated forecast, or a deployed sensor network.

See [`docs/epistemic_safety_case.md`](docs/epistemic_safety_case.md) for the project’s reviewer-facing explanation of freshness, provenance, warning-preservation, and fail-closed publication boundaries.

## Reviewer Start Here

For a 5-minute review:
Expand Down
40 changes: 35 additions & 5 deletions app.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ const mapReviewStatusElement = document.querySelector("#map-review-status");
const siteReviewGridElement = document.querySelector("#site-review-grid");
const productGridElement = document.querySelector("#product-grid");
const sourceStatusGridElement = document.querySelector("#source-status-grid");
const sourceFreshnessLegendElement = document.querySelector("#source-freshness-legend");
const sourceOutputGridElement = document.querySelector("#source-output-grid");
const manifestNotesElement = document.querySelector("#manifest-notes");
const weatherContextGridElement = document.querySelector("#weather-context-grid");
Expand Down Expand Up @@ -1436,11 +1437,17 @@ const renderDataProducts = (products = []) => {
};

const renderSourceStatus = (manifestData) => {
if (!sourceStatusGridElement && !sourceOutputGridElement && !manifestNotesElement) {
if (
!sourceStatusGridElement &&
!sourceFreshnessLegendElement &&
!sourceOutputGridElement &&
!manifestNotesElement
) {
return;
}

sourceStatusGridElement?.replaceChildren();
sourceFreshnessLegendElement?.replaceChildren();
sourceOutputGridElement?.replaceChildren();
manifestNotesElement?.replaceChildren();

Expand All @@ -1453,6 +1460,29 @@ const renderSourceStatus = (manifestData) => {
return;
}

const freshnessLegend = Array.isArray(manifestData.freshnessLegend)
? manifestData.freshnessLegend
: [];

freshnessLegend.forEach((entry) => {
const item = document.createElement("article");
item.className = "source-freshness-item";

const term = document.createElement("h4");
term.textContent = entry.term ?? "Freshness field";

const field = document.createElement("p");
field.className = "source-status-meta";
field.textContent = entry.field ? `Manifest field: ${entry.field}` : "Manifest field: not specified";

const definition = document.createElement("p");
definition.className = "source-status-note";
definition.textContent = entry.definition ?? "Freshness meaning is not defined in the manifest.";

item.append(term, field, definition);
sourceFreshnessLegendElement?.append(item);
});

manifestData.sources.forEach((source) => {
const card = document.createElement("article");
const status = source.status ?? "unknown";
Expand Down Expand Up @@ -1487,7 +1517,7 @@ const renderSourceStatus = (manifestData) => {
freshness.textContent =
source.resourceAgeDays === null || source.resourceAgeDays === undefined
? source.note
: `Resource file age: ${source.resourceAgeDays} days. ${source.note}`;
: `Resource freshness: source file was ${source.resourceAgeDays} days old at refresh. Observation freshness is listed separately above. ${source.note}`;

card.append(title, statusBadge, meta, date, freshness);
sourceStatusGridElement?.append(card);
Expand Down Expand Up @@ -1590,15 +1620,15 @@ const renderSnapshotStatusStrip = (manifestData) => {
value: fhabsReports?.latestObservationDate
? formatDate(fhabsReports.latestObservationDate)
: "Unavailable",
note: "Report dates reflect what is present in the public FHABS source file.",
note: "Observation freshness from FHABS report records; this is separate from source-file freshness.",
kind: "reported",
},
{
label: "Latest FHABS lab-linked sample",
value: fhabsResults?.latestObservationDate
? formatDate(fhabsResults.latestObservationDate)
: "Unavailable",
note: "Lab-linked result records may lag the public report stream.",
note: "Observation freshness from lab-linked records; source-resource age is tracked separately.",
kind: "reported",
},
{
Expand Down Expand Up @@ -1643,7 +1673,7 @@ const renderFreshnessBadge = (generatedAt, { unavailable = false } = {}) => {
note.className = "freshness-note";
note.textContent = unavailable
? "The public data bundle could not be loaded, so snapshot values are not being shown."
: "Source observation dates may be older than the dashboard refresh time.";
: "Observation freshness can be older than dashboard refresh time; resource freshness only says whether the source file was recently accessible.";

freshnessRowElement.append(badge, note);
};
Expand Down
204 changes: 115 additions & 89 deletions data/manifest.json
Original file line number Diff line number Diff line change
@@ -1,90 +1,116 @@
{
"generatedAt": "2026-05-05T01:44:50.0255703-07:00",
"schemaVersion": 1,
"dashboard": "Clear Lake Watch",
"status": "ok",
"sourceFreshnessMaxAgeDays": 14,
"sources": [
{
"id": "usgs-lake-level",
"label": "USGS Lakeport lake level",
"source": "USGS",
"status": "ok",
"rowCount": 29,
"latestObservationDate": "2026-05-03",
"station": "11450000",
"parameter": "00065",
"note": "USGS daily values used for the Lakeport lake-level card and hydrology series."
},
{
"id": "usgs-cole-creek-discharge",
"label": "USGS Cole Creek discharge",
"source": "USGS",
"status": "ok",
"rowCount": 29,
"latestObservationDate": "2026-05-03",
"station": "11449820",
"parameter": "00060",
"note": "USGS daily values used for tributary-flow context and hydrology series."
},
{
"id": "fhabs-bloom-reports",
"label": "FHABS bloom reports",
"source": "California Water Boards FHABS",
"status": "ok",
"rowCount": 3429,
"clearLakeRowCount": 230,
"latestObservationDate": "2025-09-07",
"resourceUrl": "https://data.ca.gov/dataset/ab672540-aecd-42f1-9b05-9aad326f97ec/resource/c6a36b91-ad38-4611-8750-87ee99e497dd/download/bloom-report_2026-05-04.csv",
"resourceDate": "2026-05-04",
"resourceAgeDays": 1,
"note": "Public report records filtered to Clear Lake and normalized into reports, map markers, arm summaries, and report-pattern analytics."
},
{
"id": "fhabs-results",
"label": "FHABS lab results",
"source": "California Water Boards FHABS",
"status": "ok",
"rowCount": 2080,
"clearLakeRowCount": 169,
"latestObservationDate": "2024-01-11",
"resourceUrl": "https://data.ca.gov/dataset/ab672540-aecd-42f1-9b05-9aad326f97ec/resource/9d4e1df4-0cd6-4165-9e63-effcafd9dccc/download/hab-results_2026-05-04.csv",
"resourceDate": "2026-05-04",
"resourceAgeDays": 1,
"note": "Public lab-linked result records filtered to Clear Lake and normalized into observation-shaped records."
}
],
"outputs": [
{
"file": "data/live.json",
"recordCount": 13,
"description": "Dashboard live snapshot cards, map markers, analytics bundle, and data product index."
},
{
"file": "data/reports.json",
"recordCount": 230,
"description": "Normalized FHABS Clear Lake report records."
},
{
"file": "data/observations.json",
"recordCount": 62,
"description": "Normalized USGS and FHABS observation records."
},
{
"file": "data/sites-normalized.json",
"recordCount": 11,
"description": "Normalized stable site registry export."
},
{
"file": "data/analytics.json",
"recordCount": 20,
"description": "Precomputed reporting-pattern and observation-coverage analytics."
}
],
"notes": [
"Observation dates may be older than the dashboard generation time.",
"Lakeport lake-level values are shown as feet Rumsey, with approximate elevation calculated using Zero Rumsey = 1318.256 ft above mean sea level.",
"FHABS report counts represent reporting activity, not direct bloom intensity.",
"This manifest describes the generated public-data snapshot and is not official public-health guidance."
]
}
"generatedAt": "2026-05-05T01:44:50.0255703-07:00",
"schemaVersion": 1,
"dashboard": "Clear Lake Watch",
"status": "ok",
"sourceFreshnessMaxAgeDays": 14,
"sources": [
{
"id": "usgs-lake-level",
"label": "USGS Lakeport lake level",
"source": "USGS",
"status": "ok",
"rowCount": 29,
"latestObservationDate": "2026-05-03",
"station": "11450000",
"parameter": "00065",
"note": "USGS daily values used for the Lakeport lake-level card and hydrology series."
},
{
"id": "usgs-cole-creek-discharge",
"label": "USGS Cole Creek discharge",
"source": "USGS",
"status": "ok",
"rowCount": 29,
"latestObservationDate": "2026-05-03",
"station": "11449820",
"parameter": "00060",
"note": "USGS daily values used for tributary-flow context and hydrology series."
},
{
"id": "fhabs-bloom-reports",
"label": "FHABS bloom reports",
"source": "California Water Boards FHABS",
"status": "ok",
"rowCount": 3429,
"clearLakeRowCount": 230,
"latestObservationDate": "2025-09-07",
"resourceUrl": "https://data.ca.gov/dataset/ab672540-aecd-42f1-9b05-9aad326f97ec/resource/c6a36b91-ad38-4611-8750-87ee99e497dd/download/bloom-report_2026-05-04.csv",
"resourceDate": "2026-05-04",
"resourceAgeDays": 1,
"note": "Public report records filtered to Clear Lake and normalized into reports, map markers, arm summaries, and report-pattern analytics."
},
{
"id": "fhabs-results",
"label": "FHABS lab results",
"source": "California Water Boards FHABS",
"status": "ok",
"rowCount": 2080,
"clearLakeRowCount": 169,
"latestObservationDate": "2024-01-11",
"resourceUrl": "https://data.ca.gov/dataset/ab672540-aecd-42f1-9b05-9aad326f97ec/resource/9d4e1df4-0cd6-4165-9e63-effcafd9dccc/download/hab-results_2026-05-04.csv",
"resourceDate": "2026-05-04",
"resourceAgeDays": 1,
"note": "Public lab-linked result records filtered to Clear Lake and normalized into observation-shaped records."
}
],
"outputs": [
{
"file": "data/live.json",
"recordCount": 13,
"description": "Dashboard live snapshot cards, map markers, analytics bundle, and data product index."
},
{
"file": "data/reports.json",
"recordCount": 230,
"description": "Normalized FHABS Clear Lake report records."
},
{
"file": "data/observations.json",
"recordCount": 62,
"description": "Normalized USGS and FHABS observation records."
},
{
"file": "data/sites-normalized.json",
"recordCount": 11,
"description": "Normalized stable site registry export."
},
{
"file": "data/analytics.json",
"recordCount": 20,
"description": "Precomputed reporting-pattern and observation-coverage analytics."
}
],
"notes": [
"Observation dates may be older than the dashboard generation time.",
"Resource freshness and observation freshness are separate checks.",
"Lakeport lake-level values are shown as feet Rumsey, with approximate elevation calculated using Zero Rumsey = 1318.256 ft above mean sea level.",
"FHABS report counts represent reporting activity, not direct bloom intensity.",
"This manifest describes the generated public-data snapshot and is not official public-health guidance."
],
"fieldDefinitions": {
"generatedAt": "Dashboard snapshot generation time. This describes when the public JSON files were built, not when lake conditions were observed.",
"sourceFreshnessMaxAgeDays": "Review threshold used by validation to warn when source observations are older than the dashboard freshness window.",
"latestObservationDate": "Observation freshness field. This is the newest environmental observation date available for a source within the public snapshot.",
"resourceDate": "Resource freshness field for downloadable source files. This is the date embedded in the source file URL or release, not an environmental observation date.",
"resourceAgeDays": "Resource freshness field for downloadable source files. This is how old the source file was when the dashboard snapshot was generated.",
"status": "Machine-readable source load status for the snapshot refresh pass."
},
"freshnessLegend": [
{
"term": "Dashboard snapshot freshness",
"field": "generatedAt",
"definition": "When Clear Lake Watch generated these public JSON files."
},
{
"term": "Observation freshness",
"field": "latestObservationDate",
"definition": "How recent the newest environmental record is for a source."
},
{
"term": "Resource freshness",
"field": "resourceDate / resourceAgeDays",
"definition": "How recent the downloaded source file is, separate from the age of the observations inside it."
}
]
}
21 changes: 21 additions & 0 deletions docs/architecture_sketch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Architecture Sketch

This sketch is intentionally lightweight. It shows the trust boundary for future sensor data without implying a deployed pipeline.

```mermaid
flowchart LR
A["Field Sensor"] --> B["Local Gateway"]
B --> C["MQTT Topic"]
C --> D["Intake Validator"]
D -->|invalid or broken| E["Quarantine / Dead Letter"]
D -->|passes deterministic checks| F["Review Queue"]
F -->|approved for internal use| G["Internal Store"]
G -->|reviewed public-safe export only| H["Clear Lake Watch Public Layer"]
```

## Boundary Notes

- The local gateway and MQTT layer are internal transport surfaces.
- The intake validator is where fail-closed behavior starts.
- Quarantine and dead-letter paths preserve auditability without turning bad records into public claims.
- Clear Lake Watch should read only reviewed public-safe exports, not raw sensor traffic.
Loading
Loading