Skip to content

Add architecture and contributor documentation - #758

Merged
glenflorendo merged 1 commit into
hackforla:mainfrom
gregpawin:docs/architecture-documentation
Sep 15, 2026
Merged

glenflorendo merged 1 commit into
hackforla:mainfrom
gregpawin:docs/architecture-documentation

Conversation

@gregpawin

Copy link
Copy Markdown
Member

Description

Adds eight topic documents under docs/, linked from a new "Detailed documentation" section in docs/README.md.

The repository spans a Next.js app, an Express API, two Python data pipelines, and three databases, and there was no
single place explaining how those relate or which parts are actually wired together. New contributors had to infer it
from source.

Document Covers
01-overview.md Mission and high-level architecture
02-monorepo-structure.md Workspace layout, tooling, local dev workflow
03-web-application.md Next.js map app, state, Socrata integration
04-backend-api.md Express/MongoDB API and its contract
05-data-pipeline.md Polars + SQLite/PostGIS ingestion and schemas
06-legacy-data-science.md Older ETL and the normalized PostGIS schema
07-data-sources-and-schemas.md Dataset IDs, column dictionary, reference files
08-roadmap-and-open-questions.md Known gaps and open decisions

The docs distinguish what works from what is aspirational — that the web app does not currently call the Express API, for
instance — so the diagrams are not misread as the intended end state. Anywhere behavior was unclear from source, it is
recorded as an open question in 08 rather than guessed at.

The change to docs/README.md is purely additive: it appends a section and touches nothing upstream already had.

Related Issues

Refs #695

This PR has no dedicated issue and does not implement any of #695's acceptance criteria — it documents the pipeline
work referenced there, which is why it is linked at all. Since the PR template requires an approved issue, tell me
whether you would like me to open a documentation issue for this, or whether you would rather fold these docs into the
wiki instead of the repo. Happy to close this if documentation lives elsewhere by convention.

Testing

Not applicable — documentation only, no executable code.

Verified every relative link resolves. Three of them (05-data-pipeline.md and docs/README.md pointing into
data-science/beta_pipeline/) depend on the pipeline PR, so this should merge after that one or those links will
404. Confirmed the mermaid diagrams render on GitHub.

Checklist

  • I have followed all conventions outlined in our documentation.
  • I have fully tested my changes and confirmed that all new and existing tests pass.
  • I have written meaningful commit messages for all changes.
  • I have linted and formatted my changes to follow the code style of this repository.
  • I have updated our documentation, accordingly.
  • I have checked currently opened pull requests to ensure that there are no pending pull request for the same
    changes or issues.
  • I have confirmed that this pull request fully meets the acceptance criteria for all related issues listed above.

Last box unchecked: this is documentation and does not satisfy #695's criteria, which are about ingestion behavior.

The repository spans a Next.js app, an Express API, two Python data
pipelines, and three databases, with no single place explaining how they
relate or which parts are actually wired together. New contributors had
to infer that from source.

Add eight topic documents under docs/, linked from a new "Detailed
documentation" section in docs/README.md:

- 01-overview: mission and high-level architecture
- 02-monorepo-structure: workspace layout, tooling, local dev
- 03-web-application: Next.js map app, state, Socrata integration
- 04-backend-api: Express/MongoDB API and its contract
- 05-data-pipeline: Polars + SQLite/PostGIS ingestion and schemas
- 06-legacy-data-science: older ETL and normalized PostGIS schema
- 07-data-sources-and-schemas: dataset IDs and column dictionary
- 08-roadmap-and-open-questions: known gaps and open decisions

The docs are explicit about what is aspirational versus working - the
web app does not currently call the Express API, for instance - so the
diagrams are not mistaken for the intended end state.

Note: 05-data-pipeline.md and docs/README.md link to
data-science/beta_pipeline/, so this should merge after the pipeline
work or those three links will 404.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown

@gregpawin, this Pull Request is not linked to a valid issue. Please provide a valid linked issue in "Related Issues" above, using the format of "Resolves #" + issue number.

@glenflorendo
glenflorendo merged commit 03aee0f into hackforla:main Sep 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants