Skip to content

Conversation

@jrepp
Copy link
Owner

@jrepp jrepp commented Nov 20, 2025

Why

This branch consolidates multiple infrastructure improvements including protobuf restructuring to resolve import cycles, comprehensive test coverage improvements across patterns, and Phase 3 auth integration for Consumer/Producer patterns.

How

  • Moved protobuf options to prism.common package to resolve import cycles
  • Added ReportLifecycleEvent RPC for proxy-pattern lifecycle events
  • Implemented CreateNamespace RPC handler with pattern selection
  • Fixed Docker build and Go workspace configuration issues
  • Integrated tests with proxy instead of direct backend access

What Changed

  • Proto: Restructured imports, added lifecycle events RPC (control_plane.proto)
  • Patterns: Added auth integration for Consumer/Producer patterns
  • Testing: Improved test coverage (patterns/producer from 57.8% to 75.9%)
  • Infrastructure: Fixed Go workspace configuration, Docker build improvements
  • Documentation: Added MEMO-092 (pkg/config refactoring plan), fixed validation errors

43 commits consolidating pattern infrastructure work.

Co-Authored-By: Claude [email protected]

jrepp and others added 4 commits October 22, 2025 20:59
User request: "create a new branch to test the taskfile testing infrastructure, if possible we should compose the taskfile from some sub task files - the integration testing should probably live in it's own taskfile in the ./testing directory - I want to have a task that will ensure the docker-compose is up and run through each integration test that runs against this set of services"

Created modular testing infrastructure with:
- testing/Taskfile.yml with comprehensive test management
- Infrastructure lifecycle tasks (infra-up, infra-down, infra-status)
- Integration test tasks with auto-managed infrastructure
- Acceptance test tasks for all patterns
- Backend-specific test tasks
- MCP integration test tasks
- Comprehensive test suites (test-all, test-all-with-mcp)
- Quick smoke test for development

Updated root Taskfile.yml to include testing/Taskfile.yml via includes
All tasks namespaced as test-infra:* for clarity

Benefits:
- Separation of concerns (build vs testing infrastructure)
- Easier maintenance with focused test Taskfile
- Automatic infrastructure management with defer cleanup
- Supports both individual and comprehensive test execution
- Clear documentation in testing/README.md

Co-Authored-By: Claude <[email protected]>
User request: "let's simplify the tasks so it's just test: for the namespace"

Changed namespace from test-infra to test for cleaner, more intuitive usage:
- Updated root Taskfile.yml includes section
- Updated all documentation in testing/README.md
- All testing tasks now accessible as task test:*

Examples:
- task test:infra-up (was test-infra:infra-up)
- task test:test-all (was test-infra:test-all)
- task test:help (was test-infra:help)

Benefits:
- Shorter, cleaner namespace
- More intuitive for users
- Consistent with common task naming patterns

Co-Authored-By: Claude <[email protected]>
User request: "do unit tests fit into this new task namespace? yes"

Moved ALL test tasks from root Taskfile.yml to testing/Taskfile.yml under
unified test: namespace with clear hierarchy:

Unit Tests (Fast, No Dependencies):
- unit-all, unit-proxy, unit-core, unit-prismctl, unit-operator
- unit-drivers (memstore, redis, nats, kafka, postgres)

Integration Tests (Requires Infrastructure):
- test-integration-all, integration-rust, integration-k8s
- test-integration-lifecycle, test-integration-dataplane, etc.

Acceptance Tests:
- test-acceptance-all (docker-compose based)
- acceptance-testcontainers (testcontainers based)
- acceptance-parallel

Parallel Execution:
- parallel, parallel-fast, parallel-fail-fast

Comprehensive Suites:
- all (unit + integration + acceptance)
- all-with-mcp

Changes:
- Added all unit tests to testing/Taskfile.yml
- Added testcontainers-based acceptance tests
- Added Rust and K8s integration tests
- Added parallel execution tasks
- Removed all 250+ lines of test tasks from root Taskfile.yml
- Updated comprehensive suites to include unit tests
- Updated testing/README.md with new structure and hierarchy
- Updated help task with complete test organization

Total: 55 test tasks now consolidated under test: namespace

Benefits:
- Single namespace for ALL tests (unit, integration, acceptance)
- Clear hierarchy: Unit -> Integration -> Acceptance -> All
- Fast iteration: Unit tests require no infrastructure
- Easy discovery: task --list | grep test:
- Consistent naming: Predictable patterns throughout
- Root Taskfile focused on builds, not tests

Co-Authored-By: Claude <[email protected]>
User request: "the information priority is that we want to have a very very high level mermaid diagram with a single line of text that encapsulates the power of the prism data layer after the diagram we want to describe 3 key features with short value based differentiators, then a quickstart command example with the local stack and a grpcurl output from key/val then the badges and the rest of the readme but we should keep the readme very focused on time to value by the mermaid diagram we should include links to the documentation main sections, user guide, intro, architectures, rfc - the information priority should naturally lead a reader to discover more depth and value"

User request: "use the key documents and adrs and a competitive analysis to understand the key differentiators"

User request: "calling out the netflix inspiration is also good I think to help people understand the focus"

Changes:
- Add Netflix inspiration tagline prominently at top
- Simplify architecture diagram with clickable docs links
- Replace generic features with 3 data-driven differentiators:
  * 10-100x performance (16x P50 latency, 10x throughput, 25x less memory)
  * Self-service infrastructure (days → minutes provisioning)
  * Real backends testing (Docker-based, sub-second feedback)
- Add quickstart with task commands and grpcurl KeyValue example
- Move badges after quickstart (value-first approach)
- Add competitive comparison table vs Netflix Data Gateway
- Highlight: "Netflix proved architecture at scale. Prism makes it accessible"

All numbers sourced from ADR-001 (Rust performance), ADR-002 (client config),
and ADR-004 (local testing). Information hierarchy optimized for time-to-value.

Co-Authored-By: Claude <[email protected]>
Copilot AI review requested due to automatic review settings November 20, 2025 22:11
@mergify mergify bot added documentation Improvements or additions to documentation infrastructure size/xs labels Nov 20, 2025
@mergify
Copy link
Contributor

mergify bot commented Nov 20, 2025

This PR has merge conflicts with the base branch. Please resolve them.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR restructures the testing infrastructure by consolidating all test-related tasks into a dedicated test: namespace, improving organization and discoverability. The changes include creating a new testing directory with comprehensive documentation, moving all test tasks from the root Taskfile to a dedicated testing Taskfile, and enhancing the README with a more compelling introduction that better positions Prism's value proposition.

Key Changes:

  • Created testing/Taskfile.yml with organized test hierarchy (unit, integration, acceptance)
  • Moved 30+ test tasks from root Taskfile to dedicated test namespace
  • Added comprehensive testing documentation in testing/README.md
  • Enhanced main README with stronger positioning and quickstart example

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.

File Description
testing/Taskfile.yml New comprehensive test infrastructure with 50+ tasks organized by test type
testing/README.md Complete testing guide with examples, architecture, and troubleshooting
Taskfile.yml Removed test tasks, added test namespace include, added migration note
README.md Enhanced introduction, reorganized content, added quickstart example

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mergify
Copy link
Contributor

mergify bot commented Dec 4, 2025

This PR has been inactive for 14 days. Please update it or close it if it's no longer needed.

@mergify mergify bot added stale and removed stale labels Dec 4, 2025
@mergify
Copy link
Contributor

mergify bot commented Dec 18, 2025

This PR has been inactive for 14 days. Please update it or close it if it's no longer needed.

@mergify mergify bot added stale and removed stale labels Dec 18, 2025
@mergify
Copy link
Contributor

mergify bot commented Jan 1, 2026

This PR has been inactive for 14 days. Please update it or close it if it's no longer needed.

@mergify mergify bot added stale and removed stale labels Jan 1, 2026
User request: "land PR 203"

Co-Authored-By: Claude <[email protected]>
@mergify mergify bot removed the has-conflicts label Jan 5, 2026
@jrepp
Copy link
Owner Author

jrepp commented Jan 5, 2026

@Mergifyio queue

@mergify
Copy link
Contributor

mergify bot commented Jan 5, 2026

queue

🟠 Waiting for conditions to match

Details
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • #approved-reviews-by>=1
      • check-success=PR Status Check
      • any of: [🛡 GitHub branch protection]
        • check-neutral = PR Status Check
        • check-skipped = PR Status Check
        • check-success = PR Status Check
      • any of: [🛡 GitHub repository ruleset rule main]
        • check-neutral = PR Status Check
        • check-skipped = PR Status Check
        • check-success = PR Status Check
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub repository ruleset rule main]
      • -draft
      • base=main
      • label!=do-not-merge
      • label!=work-in-progress
  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed

@mergify
Copy link
Contributor

mergify bot commented Jan 5, 2026

Merge Queue Status

🟠 Waiting for queue conditions

Required conditions to enter a queue
  • any of [🔀 queue conditions]:
    • all of [📌 queue conditions of queue default]:
      • #approved-reviews-by>=1
      • check-success=PR Status Check
      • any of [🛡 GitHub branch protection]:
        • check-neutral = PR Status Check
        • check-skipped = PR Status Check
        • check-success = PR Status Check
      • any of [🛡 GitHub repository ruleset rule main]:
        • check-neutral = PR Status Check
        • check-skipped = PR Status Check
        • check-success = PR Status Check
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub repository ruleset rule main]
      • -draft
      • base=main
      • label!=do-not-merge
      • label!=work-in-progress
  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of [📌 queue -> configuration change requirements]:
    • -mergify-configuration-changed
    • check-success = Configuration changed

@mergify
Copy link
Contributor

mergify bot commented Jan 5, 2026

The PR Status Check has failed. Please review the CI logs and fix any issues.

Common issues:

  • Test failures
  • Linting errors
  • Documentation validation failures

You can run checks locally:

task test-parallel-fast  # Run tests
task lint-parallel       # Run linters
uv run tooling/validate_docs.py  # Validate docs

@mergify
Copy link
Contributor

mergify bot commented Jan 5, 2026

🧪 CI Insights

Here's what we observed from your CI run for f5b6331.

❌ Job Failures

Pipeline Job Health on main Retries 🔍 CI Insights 📄 Logs
PR Status Check PR Status Check Unknown 0 View View

@jrepp jrepp changed the title Jrepp - Readme Cleanup Add pattern common infrastructure and improve test coverage Jan 6, 2026
@jrepp jrepp added the minor Small improvements or changes label Jan 6, 2026
@mergify
Copy link
Contributor

mergify bot commented Jan 6, 2026

The PR Status Check has failed. Please review the CI logs and fix any issues.

Common issues:

  • Test failures
  • Linting errors
  • Documentation validation failures

You can run checks locally:

task test-parallel-fast  # Run tests
task lint-parallel       # Run linters
uv run tooling/validate_docs.py  # Validate docs

@jrepp
Copy link
Owner Author

jrepp commented Jan 6, 2026

@Mergifyio queue

@mergify
Copy link
Contributor

mergify bot commented Jan 6, 2026

queue

☑️ Command queue ignored because it is already running from a previous command.

@mergify
Copy link
Contributor

mergify bot commented Jan 14, 2026

The PR Status Check has failed. Please review the CI logs and fix any issues.

Common issues:

  • Test failures
  • Linting errors
  • Documentation validation failures

You can run checks locally:

task test-parallel-fast  # Run tests
task lint-parallel       # Run linters
uv run tooling/validate_docs.py  # Validate docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation infrastructure minor Small improvements or changes size/xs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants