diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 175cd9d..6d9dd74 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -25,6 +25,14 @@ "skills": [ "./skills/gh-enrich-pr-description" ] + }, + { + "name": "nats", + "description": "NATS messaging skills for designing subject hierarchies with naming conventions and wildcard patterns", + "source": "./", + "skills": [ + "./skills/nats-design-subject" + ] } ] } diff --git a/README.md b/README.md index 1bce9d3..e968179 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,8 @@ claude plugin install diataxis@trogonstack | Plugin | Description | |--------|-------------| | `diataxis` | Documentation skills following the Diataxis framework — generate READMEs and reorganize docs into tutorials, how-to guides, reference, and explanation sections | +| `gh` | GitHub workflow skills for enriching PR descriptions, automating code review context, and streamlining collaboration through the gh CLI | +| `nats` | NATS messaging skills for designing subject hierarchies, naming conventions, and wildcard patterns for scalable pub/sub, request/reply, and streaming architectures | ## Contributing diff --git a/skills/nats-design-subject/SKILL.md b/skills/nats-design-subject/SKILL.md new file mode 100644 index 0000000..bff1e31 --- /dev/null +++ b/skills/nats-design-subject/SKILL.md @@ -0,0 +1,325 @@ +--- +name: nats-design-subject +description: >- + Design NATS subject hierarchies for messaging patterns (pub/sub, request/reply, + streaming). Apply naming conventions, segmentation strategies, and wildcard + patterns to create scalable subject architectures. Use when designing NATS + messaging systems, planning multi-tenant communication, or auditing existing + subject hierarchies. Do not use for: (1) NATS server configuration or cluster + setup, (2) client library implementation or connection code, (3) debugging + connectivity or performance issues, (4) choosing between NATS and other + messaging systems. +allowed-tools: AskUserQuestion, Write, Read, Shell +--- + +# Design NATS Subject Hierarchy + +Design a subject architecture that subscribers can efficiently navigate using wildcards, with proper segment ordering, tenant isolation, and growth path. + +## Interview Phase + +**Skip interview if ALL of these are already specified:** +- Messaging patterns (pub/sub, request/reply, streaming) +- Multi-tenancy needs (single/multi-tenant, scale requirements) +- Security requirements (authorization, tenant isolation) +- Persistence needs (JetStream vs core NATS) + +**Always interview if**: Migrating existing subjects (needs anti-pattern audit first) + +### Questions + +1. **Scope** — "Is this greenfield design or migrating existing subjects?" + - Impact: Migration needs anti-pattern audit first (see [references/anti-patterns.md](references/anti-patterns.md)) + +2. **Multi-Tenancy & Scale** — "Do you need: (A) Single tenant, (B) Multi-tenant with isolation, (C) Massive scale with regions/shards?" + - Impact: Determines whether tenant prefix is needed and segmentation depth + +3. **Messaging Patterns** — "Which patterns do you use? (A) Pub/Sub only, (B) Request/Reply, (C) Streaming/JetStream, (D) All/mix?" + - Impact: JetStream needs stream-aware subject design; request/reply has its own conventions + +4. **Security** — "Do you need subject-based authorization or tenant isolation?" + - Impact: Determines whether tenant/role prefixes are needed and permission boundaries + +5. **Persistence** — "Do you need JetStream persistence or core NATS only?" + - Impact: Determines stream/consumer subject design and retention considerations + +--- + +## When to Use + +- Designing a new NATS messaging system +- Planning multi-tenant subject isolation +- Organizing device telemetry or event streams +- Setting up request/reply patterns across microservices +- Defining event subject structure for event-sourced systems +- Building agentic AI platforms with inter-agent messaging + +## When NOT to Use + +- Configuring NATS server or cluster settings (infrastructure, not subject design) +- Writing NATS client code or connection logic (implementation, not architecture) +- Choosing between NATS and other messaging systems (technology evaluation) +- Debugging existing NATS connectivity or performance issues + +--- + +## Workflow + +### 1. Identify Domain Boundaries + +List all business domains involved (orders, payments, inventory, etc.). Each domain becomes a top-level subject segment. + +### 2. Choose a Pattern + +Match the user's scenario to a pattern: + +| Use Case | Pattern | Example | +|----------|---------|---------| +| **Simple Domain** | `{domain}.{action}.{scope}` | `orders.created.us-west` | +| **Multi-Region** | `{domain}.{action}.{region}.{id}` | `devices.telemetry.us-east.sensor-456` | +| **Multi-Tenant SaaS** | `{tenant}.{domain}.{action}.{id}` | `acme-corp.analytics.processed.report-123` | +| **Multi-Tenant AI** | `agents.{action}.{tenant}.{agent-id}.{task-id}` | `agents.task-assigned.tenant-abc.agent-xyz.task-123` | +| **Request/Reply** | `{service}.request` / `{service}.reply` | `orders.request` / `orders.reply` | +| **Event Sourcing** | `{aggregate}.{action}.v{version}.{id}` | `orders.order.created.v1.order-123` | + +For full pattern details with subscriber paths and scaling guidance, read [references/patterns.md](references/patterns.md). + +### 3. Order Segments Strategically + +Apply these rules when ordering segments left-to-right: + +- **Broad to specific**: Domain → Action → Scope → Identifier +- **Low-cardinality left, high-cardinality right**: Regions (few values) before IDs (millions of values) +- **Never put IDs or UUIDs before actions** + +``` +✓ GOOD: orders.created.us-west.order-123 + ↑ ↑ + low-card high-card + +✗ BAD: orders.order-123.us-west.created + ↑ + high-card early (kills wildcard filtering) +``` + +Why this matters: NATS wildcard matching scans left-to-right. High-cardinality values on the left force subscribers into inefficient `orders.*.us-west.created` patterns that must match thousands of IDs. + +For common ordering mistakes and migration strategies, read [references/anti-patterns.md](references/anti-patterns.md). + +### 4. Plan Subscriber Paths + +For each domain, document how subscribers will filter: + +``` +orders.> → All order events +orders.created.> → All order creation events +orders.created.us-west.> → Orders created in US West +orders.created.us-west.order-123 → Specific order +``` + +Design subjects for subscribers, not publishers. Subscribers determine how you organize — a good hierarchy lets them efficiently filter with wildcards. + +### 5. Design Security Model (if multi-tenant) + +If the user needs tenant isolation or role-based access: + +- Tenant ID as first segment enables subject-based authorization (`acme-corp.>`) +- Separate admin subjects from user operations (`_admin.>` for platform ops) +- Apply least-privilege permissions per service + +For authorization patterns and tenant isolation examples, read [references/security.md](references/security.md). + +### 6. Design JetStream Streams (if persistence needed) + +If the user needs JetStream: + +- One stream per domain (or per tenant for isolation) +- Consumer filters for fine-grained routing +- Retention policies per domain (financial: years, telemetry: days) +- Keep to 4-6 subject segments — use consumer filters instead of deeper hierarchies + +For stream design, consumer patterns, and migration from core NATS, read [references/jetstream.md](references/jetstream.md). + +### 7. Validate and Write Output + +Present the design using this template: + +```markdown +# NATS Subject Architecture: [System Name] + +## Domain Overview +[Describe the domains and their interactions] + +## Subject Hierarchy + +### Domain: [Name] +- `domain.action.{scope}.{id}` +- `domain.action.{scope}.{id}` + +Subscriber paths: +- `domain.>` — All events +- `domain.action.>` — Specific action + +[Repeat for each domain] + +## Multi-Tenancy Model +[How tenant isolation works via subjects, if applicable] + +## Security Model +[Authorization rules per role/service, if applicable] + +## JetStream Streams +[Stream definitions and consumer filters, if applicable] + +## Quality Validation +[Run checklist below] +``` + +### Example Output + +```markdown +# NATS Subject Architecture: IoT Smart Building Platform + +## Domain Overview + +Smart building system with 10,000+ sensors across multiple regions sending temperature, humidity, and occupancy data. Needs real-time monitoring, regional aggregation, and alerting. + +## Subject Hierarchy + +### Domain: Devices +- `devices.telemetry.{region}.{device-id}.{metric}` +- `devices.telemetry.us-west.sensor-456.temperature` +- `devices.telemetry.us-west.sensor-456.humidity` +- `devices.telemetry.eu-central.sensor-789.occupancy` + +Subscriber paths: +- `devices.telemetry.>` — All telemetry (global monitoring) +- `devices.telemetry.us-west.>` — Regional dashboard (US West) +- `devices.telemetry.>.>.temperature` — All temperature readings + +### Domain: Alerts +- `alerts.triggered.{severity}.{region}.{device-id}` +- `alerts.triggered.critical.us-west.sensor-456` + +Subscriber paths: +- `alerts.triggered.critical.>` — Critical alerts only +- `alerts.triggered.>.us-west.>` — Regional alert dashboard + +## Multi-Tenancy Model + +Not applicable (single organization) + +## Security Model + +- Building operators: `devices.telemetry.>`, `alerts.>` (subscribe only) +- Alert service: `alerts.>` (publish + subscribe) +- Admin: `>` (full access) + +## JetStream Streams + +Stream: `telemetry-us-west` + Subjects: `devices.telemetry.us-west.>` + Retention: 24h (high volume) + +Stream: `alerts` + Subjects: `alerts.>` + Retention: 30d + +## Quality Validation + +✓ All segments follow broad-to-specific order +✓ Device IDs at rightmost position +✓ Naming consistent (lowercase, hyphens) +✓ Regional filtering efficient +✓ 4 segments (within 4-6 limit) +``` + +For complete real-world examples across microservices, IoT, SaaS, event sourcing, and agentic AI platforms, read [references/use-cases.md](references/use-cases.md). + +--- + +## Quick Start (Simple Cases) + +If you're designing a simple single-domain system without multi-tenancy: + +1. **Use Pattern 1** (Simple Domain): `{domain}.{action}.{id}` +2. **Skip references** — follow the workflow above +3. **Example**: `orders.created.order-123`, `payments.authorized.payment-456` + +For multi-region, multi-tenant, or event-sourcing needs, continue with full workflow and read references as needed. + +--- + +## Reference Navigation + +The skill includes 5 detailed reference documents — read them as needed during workflow steps: + +- **[patterns.md](references/patterns.md)**: Read when choosing initial pattern (step 2) +- **[anti-patterns.md](references/anti-patterns.md)**: Read when migrating or auditing existing subjects +- **[security.md](references/security.md)**: Read when multi-tenancy/authorization needed (step 5) +- **[jetstream.md](references/jetstream.md)**: Read when persistence needed (step 6) +- **[use-cases.md](references/use-cases.md)**: Read for complete worked examples per domain + +Don't read all references upfront — use them progressively as the workflow requires. + +--- + +## Naming Rules + +- All lowercase with hyphens: `orders.created` ✓ +- Never underscores: `orders_created` ✗ +- Never mixed case: `Orders.Created` ✗ +- Keep to 4-6 segments maximum + +--- + +## Quality Checklist + +- [ ] All segments follow broad-to-specific order (`{domain}.{action}.{scope}.{id}`) +- [ ] No UUID/ID fields appear before action/scope segments +- [ ] Naming consistent (all lowercase, hyphens, no underscores) +- [ ] Documented subscriber wildcard paths for each domain +- [ ] No subjects deeper than 6 segments +- [ ] Multi-tenancy isolation is clear (tenant ID positioning documented) +- [ ] Security subjects defined for admin/monitoring access +- [ ] No conflicting patterns (e.g., `orders.created.123` vs `orders.123.created`) +- [ ] High-cardinality decision documented (why ID placement chosen) + +## Testing Your Design + +Validate your subject hierarchy before deployment: + +```bash +# Start local NATS server +nats-server -D + +# Test subscriber wildcards +nats sub "orders.>" # Should match all order subjects +nats sub "orders.created.>" # Should match only creations +nats sub "orders.created.us-west.>" # Should match region-specific + +# Test publishing +nats pub "orders.created.us-west.order-123" "test message" + +# Verify JetStream streams (if applicable) +nats stream info orders-stream +nats consumer info orders-stream order-consumer +``` + +For existing deployments, audit current subjects: + +```bash +# List active subjects (requires monitoring enabled) +nats server report jetstream + +# Check subject permissions +nats server check connection --account +``` + +## Reference Documentation + +- **[Patterns](references/patterns.md)**: 6 hierarchy patterns with subscriber paths and scaling guidance +- **[Anti-Patterns](references/anti-patterns.md)**: 8 common mistakes with detection, fixes, and migration strategies +- **[Security & Multi-Tenancy](references/security.md)**: Authorization patterns and tenant isolation +- **[JetStream Design](references/jetstream.md)**: Stream filters, consumer subjects, and retention policies +- **[Use Cases](references/use-cases.md)**: Complete examples for microservices, IoT, SaaS, event sourcing, agentic AI diff --git a/skills/nats-design-subject/references/anti-patterns.md b/skills/nats-design-subject/references/anti-patterns.md new file mode 100644 index 0000000..a82520b --- /dev/null +++ b/skills/nats-design-subject/references/anti-patterns.md @@ -0,0 +1,519 @@ +# NATS Subject Anti-Patterns + +This reference documents 8 common mistakes in subject design with detection, fixes, and migration strategies. + +**Related references**: For correct patterns see [patterns.md](patterns.md). For security-specific mistakes see [security.md](security.md). For JetStream-specific design see [jetstream.md](jetstream.md). + +## Anti-Pattern 1: High-Cardinality Left ⚠️ CRITICAL + +**What it looks like**: +``` +✗ BAD: +orders.order-123.created.us-west +orders.order-456.created.us-west +orders.order-789.created.us-west + +✓ GOOD: +orders.created.us-west.order-123 +orders.created.us-west.order-456 +orders.created.us-west.order-789 +``` + +**Why it's bad**: +- Kills hierarchical filtering +- Subscriber must use wildcard: `orders.*.created.us-west.>` (matches thousands) +- Cannot efficiently say "all orders created in US West" +- High memory usage for wildcard subscriptions + +**Detection**: +```nats +# If you're writing subscribers like this, you have the problem: +subscribers: orders.*.created.> # Wildcard in middle = high-cardinality left +``` + +**Fix**: +1. Restructure subjects: Move ID to rightmost position +2. Update publishers to use new subject format +3. Update subscribers accordingly + +**Migration Strategy**: +``` +Phase 1 (week 1-2): + - Start new subjects: orders.created.us-west.order-123 (new ID position) + - Keep publishing to both old and new subjects + - Old subscribers still work: orders.order-123.> + - New subscribers use: orders.created.us-west.> + +Phase 2 (week 3-4): + - Update all subscribers to new subjects + - Verify both old and new working + - Monitor for gaps + +Phase 3 (week 5): + - Stop publishing to old subjects + - Decommission old subscriptions + - Celebrate! +``` + +**Real Example (E-Commerce)**: +``` +BEFORE (❌ high-cardinality left): +orders.12345678.created (customer ID on left) +orders.12345678.shipped (must filter by customer first) +orders.12345679.created (10 million customers = 10 million prefixes) + +AFTER (✓ good): +orders.created.customer-12345678 +orders.shipped.customer-87654321 +Subscriber: orders.created.customer-12345678.> +Subscriber: orders.shipped.> (all shipments efficiently) +``` + +--- + +## Anti-Pattern 2: Inconsistent Segment Ordering + +**What it looks like**: +``` +✗ BAD (mixed ordering): +orders.created.us-west.order-123 (action before region) +orders.us-west.created.order-456 (region before action) +payments.authorized.payment-789 (no region) +payments.us-west.authorized.payment-101 (different order) + +✓ GOOD (consistent): +orders.created.us-west.order-123 (always: domain.action.region.id) +orders.shipped.us-west.order-456 +payments.authorized.us-west.payment-789 +payments.failed.us-west.payment-101 +``` + +**Why it's bad**: +- Subscribers get confused with inconsistent patterns +- Different teams invent different orderings +- Impossible to write efficient wildcard filters +- Maintenance nightmare as system grows + +**Detection**: +```bash +# Check your subjects for inconsistency +grep -o '[^.]*\.[^.]*\.[^.]*\.[^.]*' subjects.txt | sort | uniq +# If outputs vary wildly, you have the problem +``` + +**Fix**: +1. Define canonical ordering for your domains +2. Document in architecture decision record +3. Use linter/validator in CI/CD + +**Validation Checklist**: +``` +For all subjects: +- [ ] Layer 1: Always domain? +- [ ] Layer 2: Always action/event? +- [ ] Layer 3: Always scope (region/tenant)? +- [ ] Layer 4+: Always identifier? +``` + +**Real Example**: +``` +INCONSISTENT ORDERING (❌): +users.profile.updated.us-west.user-123 (L2=profile) +users.updated.us-west.user-123 (L2=action, skip profile) +admins.us-west.updated.admin-456 (L2=region) + +Frustration: Subscriber needs: +users.*.updated.> # Some have profile in L2 +users.>.updated.> # Others don't + +CONSISTENT ORDERING (✓): +users.profile.updated.us-west.user-123 (always: domain.context.action.region.id) +users.admin.updated.us-west.user-456 (always same order) + +Subscriber: +users.profile.updated.us-west.> +users.admin.updated.us-west.> +users.>.updated.us-west.> # All same ordering! +``` + +--- + +## Anti-Pattern 3: Over-Segmentation (Too Many Layers) + +**What it looks like**: +``` +✗ BAD (9 segments): +orders.created.v1.us-west.us-west-2a.customer-123.order-456.line-item-789.warehouse-101 + +✓ GOOD (4-5 segments): +orders.created.us-west.order-456 + (other details in message headers/payload) +``` + +**Why it's bad**: +- Subjects get unwieldy and error-prone +- Subscribers have to navigate deep hierarchies +- Makes JetStream consumer filters complex +- Maintenance burden grows exponentially + +**Detection**: +```bash +# Count segments +echo "orders.created.v1.us-west.customer.order-456.line-item.warehouse" | tr '.' '\n' | wc -l +# If > 6, you likely have over-segmentation +``` + +**When Each Segment is Justified**: +- L1 Domain: Always needed (orders, payments) +- L2 Action: Nearly always (created, updated, cancelled) +- L3 Scope: Usually (region, tenant, environment) +- L4 ID: Usually (order-id, customer-id) +- L5+ Only if: High-volume filtering needs (e.g., warehouse-specific subscribers) + +**Fix**: +1. Move low-cardinality data to message headers/properties +2. Use JetStream consumer filters for fine-grained routing +3. Keep subjects to 4-5 segments max + +**Better Approach Using JetStream**: +``` +Subjects (simple): +orders.created.us-west.order-456 + +Message headers/properties (detailed): +headers: + X-LineItem: 789 + X-Warehouse: 101 + X-API-Version: 1 + +JetStream Consumer Filter: + Subject filter: orders.created.us-west.> + (Consumer can also filter by headers if needed) +``` + +--- + +## Anti-Pattern 4: Under-Segmentation (Not Enough Layers) + +**What it looks like**: +``` +✗ BAD (too simple): +orders.order-123 +payments.payment-456 +users.user-789 + +Subscriber: orders.> (gets ALL order events, cannot filter by action) + +✓ GOOD (proper segmentation): +orders.created.order-123 +orders.shipped.order-456 +payments.authorized.payment-789 +``` + +**Why it's bad**: +- Subscribers receive too many unwanted messages +- Cannot filter by event type or region +- Inefficient use of bandwidth and memory +- Hard to add complexity later + +**Detection**: +```bash +# If subjects only have 2 segments, check if subscribers would benefit from filtering +grep ">\." subscriptions.txt | wc -l +# If high count, you need more segmentation +``` + +**Fix**: +1. Add action/event type as L2 +2. Add scope (region, tenant) as L3 if relevant +3. Follow the quick decision matrix from SKILL.md + +**Real Example**: +``` +UNDER-SEGMENTED (❌): +orders.order-123 +orders.order-456 +orders.order-789 + +// Subscriber forced to handle all order events: +await nc.subscribe("orders.>", (msg) => { + // Must parse message to determine if it's created, shipped, cancelled, etc. + const eventType = msg.data.type; + if (eventType === 'created') { ... } + if (eventType === 'shipped') { ... } + if (eventType === 'cancelled') { ... } +}); + +PROPER SEGMENTATION (✓): +orders.created.order-123 +orders.shipped.order-456 +orders.cancelled.order-789 + +// Subscribers get exactly what they need: +await nc.subscribe("orders.created.>", (msg) => { + // Only created events, no parsing needed + handleOrderCreated(msg.data); +}); +``` + +--- + +## Anti-Pattern 5: Ambiguous Segment Names + +**What it looks like**: +``` +✗ BAD (ambiguous): +orders.process.us-west.order-123 (process = what? created? shipped?) +events.handle.us-west.data-456 (handle = what type?) +system.data.us-west.item-789 (data = what? storage? transmission?) + +✓ GOOD (clear): +orders.created.us-west.order-123 +events.processed.us-west.data-456 +system.configuration.us-west.item-789 +``` + +**Why it's bad**: +- New developers misunderstand the architecture +- Subjects proliferate (people guess at names) +- Hard to build tooling and dashboards +- Error-prone configuration + +**Detection**: +```bash +# Audit segment names for clarity +grep -o '\.[^.]*\.' subjects.txt | sort | uniq +# Look for vague names: process, handle, data, event, etc. +``` + +**Fix**: +1. Use specific event names: `created`, `updated`, `deleted`, `shipped`, not `process` +2. Use specific scopes: `us-west`, `eu-central`, `tenant-abc` +3. Avoid generic terms: `data`, `event`, `message` + +**Naming Guide**: +``` +Clear Actions: +✓ created, updated, deleted, cancelled, shipped, arrived, paid, failed +✗ process, handle, changed, occurred + +Clear Scopes: +✓ us-west, eu-central, tenant-abc, department-sales +✗ region, scope, context, place + +Clear Types: +✓ orders, payments, users, devices +✗ things, objects, entities, stuff +``` + +--- + +## Anti-Pattern 6: Version Number Misplacement + +**What it looks like**: +``` +✗ BAD (version in wrong position): +v1.orders.created.us-west.order-123 (version on left) +orders.v1.created.us-west.order-123 (version after domain) + +✓ GOOD (version in right position): +orders.created.v1.us-west.order-123 (version after action) +orders.created.us-west.v1.order-123 (version after scope) +``` + +**Why it's bad**: +- Version on left fragments subscriptions by version +- Makes it hard to upgrade versions gradually +- Breaks the broad-to-specific ordering principle + +**When to Use Version Numbers**: +- Evolving event schemas +- Deprecating old event formats +- Running multiple API versions in parallel + +**Fix**: +1. Version numbers belong after action/scope, before identifiers +2. Use version in consumer filters, not subject structure when possible + +**Migration with Versioning**: +``` +Current: orders.created.v1.us-west.order-123 +Target: orders.created.v2.us-west.order-123 + +Phase 1: Dual publish + - Publish both v1 and v2 subjects simultaneously + - Old subscribers still work: v1 + - New subscribers use: v2 + +Phase 2: Read from both + - Services read from both: orders.created.v1.> and orders.created.v2.> + - Process both formats + +Phase 3: Switch to v2 + - All publishers send v2 + - Old subscribers deprecate + +Phase 4: Clean up + - Stop publishing v1 + - Remove v1 subscribers +``` + +--- + +## Anti-Pattern 7: Mixed Casing and Separators + +**What it looks like**: +``` +✗ BAD (inconsistent casing): +orders.OrderCreated.us-west.order-123 (PascalCase in segment) +orders.created.US_WEST.order_123 (UPPER_CASE with underscore) +Orders.created.us-west.Order-123 (mixed everywhere) + +✓ GOOD (consistent lowercase + hyphens): +orders.created.us-west.order-123 +payments.authorized.us-east.payment-456 +users.registered.us-central.user-789 +``` + +**Why it's bad**: +- Typos and case mismatches cause silent failures +- Harder to read documentation +- Tooling and monitoring gets confused + +**Detection**: +```bash +# Check for uppercase in subjects +grep '[A-Z]' subjects.txt +# Check for underscores in separators +grep '_' subjects.txt +``` + +**Standard**: +- All lowercase +- Use hyphens, never underscores +- Never use CamelCase or snake_case in segments + +**Real Example**: +``` +INCONSISTENT (❌): +Orders.created.US_WEST.order_123 # Typo city +orders.Created.us_west.order-123 # Inconsistent casing +ORDERS.CREATED.US-WEST.ORDER-123 # All caps (bad for terminal logs) + +CONSISTENT (✓): +orders.created.us-west.order-123 # Lowercase, hyphenated, clear +``` + +--- + +## Anti-Pattern 8: No Wildcard Subscription Planning + +**What it looks like**: +``` +✗ BAD (ad-hoc subjects, no planning): +orders.created.order-123 +orders.updated.order-456 +orders.cancelled.order-789 +orders.shipped.order-101 +orders.delivered.order-202 + +Subscribers guess at patterns: + Subscribe: orders.> (too broad, gets all) + Subscribe: orders.created.> (specific, works) + Subscribe: orders.*.> (oops, high-cardinality!) + +✓ GOOD (planned subscribers): +// Subject design: +orders.created.us-west.order-123 +orders.created.eu-central.order-456 + +// Subscribers documented: +Dashboard: orders.> (all regional order events) +Region monitor: orders.>.us-west.> (all orders in region) +Created events only: orders.created.> (all creations, all regions) +``` + +**Why it's bad**: +- Subscribers are ad-hoc and inefficient +- Can't optimize subject structure +- Performance problems discovered late +- Onboarding new subscribers is guesswork + +**Fix**: +1. Design subjects with subscriber paths in mind +2. Document all intended subscriber patterns +3. Test subscriber filters for efficiency + +**Planning Worksheet**: +``` +For each domain, document: +1. Subject format: {domain}.{action}.{scope}.{id} +2. Subscriber patterns: + - All events: domain.> + - Domain-specific: domain.action.> + - Scope-specific: domain.>.scope.> + - Specific resource: domain.>.scope.resource-id + +3. Validate each pattern works and is efficient +``` + +**Real Example**: +``` +UNPLANNED (❌): +devices.sensor-456.reading.temperature +devices.reading.sensor-456.temperature +devices.temperature.sensor-456.reading +(Subscribers confused, mix of patterns) + +PLANNED (✓): +Device hierarchy: devices.telemetry.{region}.{device-id}.{metric} +devices.telemetry.us-west.sensor-456.temperature +devices.telemetry.us-west.sensor-456.humidity + +Planned subscribers: +- All telemetry: devices.telemetry.> +- Regional: devices.telemetry.us-west.> +- Specific device: devices.telemetry.us-west.sensor-456.> +- Specific metric: devices.telemetry.>.>.temperature +``` + +--- + +## Migration Checklist + +When fixing anti-patterns, follow this checklist: + +- [ ] **Audit Current Subjects**: List all current subject patterns +- [ ] **Identify Problems**: Which anti-patterns are present? +- [ ] **Design Target Architecture**: New subject format(s) +- [ ] **Create Migration Plan**: Phased approach (dual-publish, parallel, etc.) +- [ ] **Test in Dev**: Verify new subjects work with existing code +- [ ] **Update Publishers**: Modify to publish to new subjects (keep old if in Phase 1) +- [ ] **Update Subscribers**: Update to subscribe to new subjects +- [ ] **Monitor Migration**: Track old vs new usage +- [ ] **Clean Up**: Remove old subjects once migration complete +- [ ] **Document**: Update architecture docs with new patterns + +--- + +## Anti-Pattern Prevention + +**In Code Review, Check For**: +``` +✓ All subjects start with domain +✓ Action/event type is always L2 (or L3 for multi-tenant) +✓ No IDs or UUIDs before action/scope +✓ Consistent casing (lowercase) and separators (hyphens) +✓ Max 5-6 segments per subject +✓ Subscriber patterns documented +``` + +**In Design Phase, Validate**: +``` +✓ Can I efficiently subscribe to "all events in domain"? +✓ Can I efficiently subscribe to "specific action only"? +✓ Can I efficiently subscribe to "specific scope only"? +✓ Do my subscriber patterns avoid wildcards in the middle? +✓ Is my subject format documented for new developers? +``` + diff --git a/skills/nats-design-subject/references/jetstream.md b/skills/nats-design-subject/references/jetstream.md new file mode 100644 index 0000000..8697d6e --- /dev/null +++ b/skills/nats-design-subject/references/jetstream.md @@ -0,0 +1,597 @@ +# NATS JetStream Subject Design + +This reference covers subject patterns specific to NATS JetStream (streams, consumers, persistence, and event distribution). + +**Related references**: For core subject hierarchy patterns see [patterns.md](patterns.md). For stream-level security and tenant isolation see [security.md](security.md). For common design mistakes see [anti-patterns.md](anti-patterns.md). + +## JetStream Subject Basics + +### Streams vs Publishers + +JetStream **streams** define which subjects to persist: + +```nats +stream: { + name: "orders-stream" + subjects: ["orders.>"] # Stream captures subjects matching this filter +} +``` + +This doesn't stop publishers from publishing elsewhere—streams define what JetStream captures. + +### Consumer Filtering + +JetStream **consumers** can further filter subjects: + +```nats +stream: "orders-stream" +subjects: ["orders.>"] # Stream captures all orders + +consumer: { + name: "order-created-consumer" + filter_subject: "orders.created.>" # Consumer only sees creations +} +``` + +**Key Insight**: Design subjects considering both stream capture and consumer filtering. + +--- + +## Pattern 1: Simple Domain Stream + +**Use when**: Single domain, simple event types, single consumer. + +**Stream Definition**: +```nats +stream: { + name: "orders" + subjects: ["orders.>"] # Captures all order subjects + max_age: 7d + storage: file +} +``` + +**Subjects**: +``` +orders.created.us-west.order-123 +orders.shipped.us-west.order-456 +orders.cancelled.us-east.order-789 +``` + +**Consumers**: +```nats +# Consumer 1: All order events +consumer: { + name: "all-orders" + filter_subject: "orders.>" # All order events + deliver_policy: new # Start with new messages +} + +# Consumer 2: Creations only +consumer: { + name: "order-creations" + filter_subject: "orders.created.>" # Creations only +} + +# Consumer 3: Regional (US West) +consumer: { + name: "us-west-orders" + filter_subject: "orders.>.us-west.>" # All order actions in US West +} +``` + +--- + +## Pattern 2: Multi-Domain Streams (One per Domain) + +**Use when**: Clear domain boundaries, separate scaling needs, independent retention. + +**Stream Definitions**: +```nats +stream: { + name: "orders-domain" + subjects: ["orders.>"] + max_age: 30d # Keep order history long +} + +stream: { + name: "payments-domain" + subjects: ["payments.>"] + max_age: 7y # Keep payment history for 7 years +} + +stream: { + name: "inventory-domain" + subjects: ["inventory.>"] + max_age: 1d # Short retention (high volume) +} +``` + +**Advantages**: +- Each domain scales independently +- Retention policies per domain +- Clear separation of concerns +- Easy to troubleshoot per domain + +**Consumer Patterns**: +```nats +# Cross-domain: Process orders and their payments +consumer: { + name: "order-with-payments" + filter_subject: "orders.created.>" # Subscribe to orders stream + # Then separately subscribe to payments-domain stream +} +``` + +--- + +## Pattern 3: Multi-Tenant Streams + +**Use when**: Multi-tenant SaaS, separate streams per tenant, complete isolation. + +**Stream Naming**: +```nats +stream: { + name: "tenant-acme-corp" # One stream per tenant + subjects: ["acme-corp.>"] +} + +stream: { + name: "tenant-startup-inc" + subjects: ["startup-inc.>"] +} + +stream: { + name: "tenant-big-enterprise" + subjects: ["big-enterprise.>"] +} +``` + +**Subjects**: +``` +# Tenant A +acme-corp.orders.created.us-west.order-123 +acme-corp.payments.authorized.us-west.payment-456 + +# Tenant B +startup-inc.orders.created.eu-central.order-789 +startup-inc.payments.authorized.eu-central.payment-101 + +# Tenant C +big-enterprise.orders.created.us-east.order-abc +big-enterprise.payments.authorized.us-east.payment-def +``` + +**Consumers (Tenant-Scoped)**: +```nats +# Tenant A - Orders only +stream: "tenant-acme-corp" +consumer: { + name: "acme-orders-consumer" + filter_subject: "acme-corp.orders.>" +} + +# Tenant B - Payments only +stream: "tenant-startup-inc" +consumer: { + name: "startup-payments-consumer" + filter_subject: "startup-inc.payments.>" +} +``` + +**Advantages**: +- Complete tenant isolation +- Different retention per tenant +- Different redundancy per tenant +- Easy to add/remove tenants +- Per-tenant backups + +**Scaling Considerations**: +- With 100 tenants = 100 streams (scalable with modern hardware) +- With 1000 tenants = consider sharding or hierarchical streams + +--- + +## Pattern 4: Event-Sourced Aggregate Streams + +**Use when**: Event-sourcing, CQRS, domain-driven design. + +**Subject Format**: +``` +{aggregate-type}.{action}.v{version}.{aggregate-id} +``` + +**Stream Definition**: +```nats +stream: { + name: "order-events" + subjects: [ + "orders.order.created.>", # All order creation events + "orders.order.updated.>", # All order updates + "orders.order.shipped.>", # All order shipments + "orders.order.cancelled.>" # All cancellations + ] + max_age: never # Audit trail, never expire + storage: file +} +``` + +**Subjects**: +``` +orders.order.created.v1.order-123 (1st event) +orders.order.updated.v1.order-123 (2nd event) +orders.order.shipped.v1.order-123 (3rd event) + +# Version upgrade scenario +orders.order.created.v2.order-456 (v2 event) +orders.order.updated.v2.order-456 +``` + +**Consumers**: +```nats +# Consumer 1: Rebuild aggregate from events +consumer: { + name: "order-rebuilder" + filter_subject: "orders.order.>" # All order events + deliver_policy: all # From the beginning + flow_control: enable # Handle backpressure + max_deliver: 3 # Retry failed messages +} + +# Consumer 2: Track completions +consumer: { + name: "order-completions" + filter_subject: "orders.order.shipped.>" # Shipped events only +} + +# Consumer 3: Track errors +consumer: { + name: "order-errors" + filter_subject: "orders.order.failed.>" # Failed events only +} +``` + +**Event Sourcing Benefits**: +- Full audit trail of state changes +- Can rebuild any aggregate at any point in time +- Replay events for debugging +- Events are the source of truth + +--- + +## Pattern 5: Stream Mirror (Replication) + +**Use when**: Multi-region, disaster recovery, high availability. + +**Primary Stream (Region 1)**: +```nats +stream: { + name: "orders-primary" + subjects: ["orders.>"] + storage: file +} +``` + +**Mirror Stream (Region 2)**: +```nats +stream: { + name: "orders-replica" + mirror: { + name: "orders-primary" # Mirror from primary + domain: "us-west" # Primary is in US West + } +} +``` + +**Subject Behavior**: +``` +Publisher publishes to: orders.created.us-west.order-123 + ↓ +Primary stream captures: orders.> + ↓ +Mirror stream replicates: orders.> + ↓ +Consumers in Region 2 read from mirror + +Failover: + If primary fails → consumers switch to mirror + Mirror stream same subject naming, seamless failover +``` + +--- + +## Pattern 6: Subject Transforms (Republish Pattern) + +**Use when**: Transform events, fan-out to derived streams, normalizations. + +**Source Subjects**: +``` +orders.created.us-west.order-123 +orders.created.eu-central.order-456 +``` + +**Transform to Derived Stream**: +```nats +stream: { + name: "orders-raw" + subjects: ["orders.>"] +} + +stream: { + name: "orders-by-region" # Derived stream + sources: [{ + name: "orders-raw" + }] +} + +consumer: { + name: "transform-by-region" + filter_subject: "orders.created.>" # Source + deliver_subject: "orders.region-{region}.created.>" # Transform target +} +``` + +**Transform Flow**: +``` +Input: orders.created.us-west.order-123 + ↓ (Consumer with subject transform) +Output: orders.region-us-west.created.order-123 + +Input: orders.created.eu-central.order-456 + ↓ +Output: orders.region-eu-central.created.order-456 +``` + +**Use Cases**: +- Normalize subject hierarchies +- Fan-out to multiple derived streams +- Create read models +- Prepare data for analytics + +--- + +## Pattern 7: Multi-Tenant Stream Isolation + +**Scenario**: SaaS with multi-tenant streams where subjects are tenant-prefixed. + +**Stream Design**: +```nats +# Option A: One stream per tenant (recommended) +stream: { + name: "tenant-acme-corp" + subjects: ["acme-corp.>"] # All acme events +} + +stream: { + name: "tenant-startup-inc" + subjects: ["startup-inc.>"] +} + +# Option B: Single stream, multiple tenants +stream: { + name: "all-tenants" + subjects: [ + "acme-corp.>", + "startup-inc.>", + "big-enterprise.>" + ] +} +``` + +**Consumer (Option A - Recommended)**: +```nats +stream: "tenant-acme-corp" +consumer: { + name: "acme-orders" + filter_subject: "acme-corp.orders.>" +} +``` + +**Consumer (Option B - Less Recommended)**: +```nats +stream: "all-tenants" +consumer: { + name: "acme-orders" + filter_subject: "acme-corp.orders.>" # Filter by tenant prefix +} +``` + +**Recommendation**: Option A (one stream per tenant) is better: +- Easier scaling +- Tenant isolation +- Different retention per tenant +- Better disaster recovery + +--- + +## Retention Policies by Domain + +Different domains need different retention: + +```nats +# Financial (long retention) +stream: { + name: "payments-stream" + subjects: ["payments.>"] + max_age: 7y # 7 year audit requirement +} + +# Orders (medium retention) +stream: { + name: "orders-stream" + subjects: ["orders.>"] + max_age: 90d # 90 days, then archived +} + +# Metrics/Telemetry (short retention) +stream: { + name: "telemetry-stream" + subjects: ["devices.telemetry.>"] + max_age: 24h # High volume, 1 day only + max_bytes: 100gb # Also limit by size +} + +# Audit (never expire) +stream: { + name: "audit-stream" + subjects: ["_audit.>"] + max_age: never # Never delete audit logs +} +``` + +--- + +## High-Volume Subject Design (IoT) + +**Challenge**: Device telemetry generates 1M+ events/sec with high cardinality device IDs. + +**Subject Design**: +``` +devices.telemetry.{region}.{device-id}.{metric} +devices.telemetry.us-west.sensor-456.temperature +devices.telemetry.us-west.sensor-789.humidity +``` + +**Stream Strategy**: +```nats +# One stream per region (separate load) +stream: { + name: "telemetry-us-west" + subjects: ["devices.telemetry.us-west.>"] + max_bytes: 500gb # Limit stream size + discard_policy: new # Drop oldest if full +} + +stream: { + name: "telemetry-eu-central" + subjects: ["devices.telemetry.eu-central.>"] + max_bytes: 500gb +} +``` + +**Consumer Strategy**: +```nats +# Consumer 1: Temperature metrics only +consumer: { + name: "temp-aggregator" + filter_subject: "devices.telemetry.us-west.>.temperature" +} + +# Consumer 2: Specific device +consumer: { + name: "sensor-456-monitor" + filter_subject: "devices.telemetry.us-west.sensor-456.>" +} + +# Consumer 3: All metrics (aggregation) +consumer: { + name: "all-metrics-stream" + filter_subject: "devices.telemetry.us-west.>" +} +``` + +**Scaling Tactics**: +- One stream per region (not global) +- Consumer per metric type +- Aggressive retention (1-7 days) +- Size limits to force data aging + +--- + +## Consumer Ordering Considerations + +**Subject Order Matters for Message Ordering**: + +```nats +Stream subjects: "orders.>" + +Message 1: orders.created.us-west.order-1 +Message 2: orders.shipped.us-east.order-2 +Message 3: orders.created.us-east.order-3 + +Consumer reads in order: 1, 2, 3 +(Subjects are different, so ordering by receipt time) +``` + +**For Aggregate Ordering Use Aggregate ID**: +```nats +Subjects with aggregate ID: orders.{action}.{order-id} + +orders.created.order-1 +orders.updated.order-1 ← Same order-id, maintains ordering +orders.shipped.order-1 + +Consumer: filter_subject: "orders.>order-1" +Reads in insertion order: created → updated → shipped +``` + +--- + +## Message Headers and Metadata + +**Don't Put Everything in Subject—Use Headers**: + +```nats +Subject: orders.created.us-west.order-123 +Headers: + X-Correlation-Id: abc-123-def + X-User-Id: user-456 + X-Tenant-Id: acme-corp + X-Timestamp: 2026-01-24T12:34:56Z +Payload: { orderId: "order-123", total: 99.99, ... } +``` + +**Benefits**: +- Subject stays readable (not bloated) +- Headers for filtering/metadata +- Payload for business data +- Supports header-based filtering in consumers (future) + +--- + +## JetStream Configuration Checklist + +When designing JetStream subjects: + +- [ ] **Stream Subjects Defined**: Clear subject filters for each stream? +- [ ] **Consumer Filters**: Each consumer has appropriate filter_subject? +- [ ] **Retention Policy**: max_age set appropriately per domain? +- [ ] **Storage Strategy**: File vs memory decided per workload? +- [ ] **Ordering Semantics**: Aggregate ID in subject for ordered consumers? +- [ ] **Headers Used**: Heavy data in headers, not subject? +- [ ] **Mirroring**: Cross-region streams planned? +- [ ] **Scaling Plan**: Stream per region/tenant/domain as needed? +- [ ] **Backpressure**: flow_control enabled for slow consumers? +- [ ] **Monitoring**: Subjects planned for stream/consumer metrics? + +--- + +## Migration: Core NATS → JetStream + +**Scenario**: Existing pub/sub, now want persistence. + +**Step 1: Existing Core NATS**: +```nats +Publisher: orders.created.us-west.order-123 +Subscriber: orders.created.> +``` + +**Step 2: Add JetStream (Dual System)**: +```nats +# Keep existing subjects (unchanged) +Publisher: orders.created.us-west.order-123 +Subscriber 1 (Core): orders.created.> (memory only) +Subscriber 2 (JS): orders.created.> (persisted) +``` + +**Step 3: Switch to JetStream**: +```nats +# Publishers unchanged +Publisher: orders.created.us-west.order-123 + +# Subscribers migrate to JetStream +Consumer: filter_subject: "orders.created.>" +``` + +**No subject changes needed** if your subjects are well-designed! + diff --git a/skills/nats-design-subject/references/patterns.md b/skills/nats-design-subject/references/patterns.md new file mode 100644 index 0000000..837e6bd --- /dev/null +++ b/skills/nats-design-subject/references/patterns.md @@ -0,0 +1,471 @@ +# NATS Subject Hierarchy Patterns + +This reference covers 6 proven subject hierarchy patterns and 5 segmentation strategies for different use cases. + +**Related references**: For authorization on these patterns see [security.md](security.md). For JetStream stream design see [jetstream.md](jetstream.md). For common mistakes see [anti-patterns.md](anti-patterns.md). + +## Pattern 1: Simple Domain Pattern (3 Segments) + +**Use when**: Single region, straightforward domains, learning NATS. + +``` +{domain}.{action}.{id} + +Examples: +- orders.created.order-123 +- orders.shipped.order-456 +- inventory.reserved.item-789 +- payments.authorized.payment-101 +``` + +**Subscriber Paths**: +```nats +orders.> # All order events +orders.created.> # All order creations +inventory.> # All inventory events +``` + +**Best For**: E-commerce backend, simple microservices, IoT with single location. + +**Scaling**: Works for 10k-100k events/sec with moderate subscriber count. Add region/tenant at layer 3 when scaling. + +--- + +## Pattern 2: Multi-Region Pattern (4-5 Segments) + +**Use when**: Geographically distributed services, compliance/data residency, regional failover. + +``` +{domain}.{action}.{region}.{id} + +Examples: +- orders.created.us-west.order-123 +- orders.created.eu-central.order-456 +- devices.telemetry.ap-south.device-789 +- payments.processed.us-east.payment-101 +``` + +**Subscriber Paths**: +```nats +orders.> # All orders (any region) +orders.created.> # All order creations (any region) +orders.created.us-west.> # Order creations in US West +orders.>.us-west.> # All order actions in US West (less efficient) +``` + +**Best For**: Global e-commerce, distributed IoT networks, compliance-required data residency. + +**Patterns**: +- Region as L3: Efficient regional filtering +- Region as L4: Less efficient, use L3 for regional subscriptions + +--- + +## Pattern 3: Multi-Tenant Pattern (4-6 Segments) + +**Use when**: SaaS platform, multi-tenant app, shared infrastructure. + +``` +{tenant}.{domain}.{action}.{id} + +Examples: +- acme-corp.orders.created.order-123 +- acme-corp.orders.updated.order-456 +- startup-inc.analytics.processed.report-789 +- startup-inc.users.registered.user-101 +``` + +**Subscriber Paths**: +```nats +acme-corp.> # All events for tenant (admin dashboard) +acme-corp.orders.> # All order events for tenant +acme-corp.orders.created.> # All order creations for tenant +``` + +**Authorization** (with NATS auth): +``` +User from acme-corp: + Publish: acme-corp.> + Subscribe: acme-corp.> + +User from startup-inc: + Publish: startup-inc.> + Subscribe: startup-inc.> + +Admin user: + Publish: > + Subscribe: > +``` + +**Best For**: Multi-tenant SaaS, white-label platforms, shared NATS clusters. + +**Variation with Region** (5 segments): +``` +{tenant}.{domain}.{action}.{region}.{id} + +acme-corp.orders.created.us-west.order-123 +``` + +--- + +## Pattern 4: Request/Reply Pattern + +**Use when**: Synchronous microservices, command-response, RPC-like communication. + +``` +Request Subject: {service}.request.{request-type} +Reply Subject: {service}.reply.{correlation-id} + +Examples: +orders.request.get-order +orders.request.calculate-total +payments.request.authorize + +Reply inbox: _INBOX.{auto-generated-id} +``` + +**Implementation**: +``` +Requester publishes to: orders.request.get-order +With reply-to: _INBOX.abc123 + +Responder subscribes to: orders.request.> + Reads reply-to header + Publishes response to: _INBOX.abc123 +``` + +**Built-in Support**: NATS provides automatic reply-to handling for request/reply. + +**Best For**: Synchronous service calls, command-query, request-response patterns. + +--- + +## Pattern 5: Event Sourcing Pattern + +**Use when**: CQRS systems, domain-driven design, event-sourced aggregates. + +``` +{aggregate-type}.{action}.v{version}.{aggregate-id} + +Examples: +- orders.order.created.v1.order-123 +- orders.order.updated.v1.order-123 +- accounts.account.debited.v1.account-456 +- accounts.account.credited.v1.account-456 +``` + +**Rationale**: +- Aggregate type (orders) groups related events +- Action (created, updated) describes what happened +- Version for API evolution +- ID for traceability + +**Subscriber Paths**: +```nats +orders.> # All order aggregate events +orders.order.> # All events for order aggregate +orders.order.created.> # All order creation events +orders.order.created.v1.> # All v1 order creations +``` + +**JetStream Stream Subject**: +``` +Stream name: orders-domain +Subject filter: orders.> + +Consumers subscribe with filters: +Filter: orders.order.created.v1.> (only order creations) +Filter: orders.order.> (all order events) +``` + +**Best For**: Event-sourced systems, domain-driven design, audit trails. + +--- + +## Pattern 6: Temporal/Versioned Pattern + +**Use when**: Versioned APIs, time-series data, deprecation management. + +``` +{domain}.{action}.v{version}.{resource-id} +OR +{domain}.{action}.{timestamp}.{id} + +Examples: +- orders.created.v2.order-123 (API version) +- devices.telemetry.2026-01-24.sensor-456 (time-based) +- prices.updated.v3.product-789 (schema version) +``` + +**Version Management**: +``` +Version 1 (deprecated): orders.created.v1.order-123 +Version 2 (current): orders.created.v2.order-123 +Version 3 (beta): orders.created.v3.order-123 + +Subscriber paths: +orders.created.v2.> # Subscribe to v2 only +orders.created.> # Subscribe to all versions (less safe) +``` + +**Time-Series Variant**: +``` +devices.telemetry.2026-01-24.sensor-456 +devices.telemetry.2026-01-23.sensor-456 + +Subscriber: devices.telemetry.2026-01-24.> (current day only) +Subscriber: devices.telemetry.>.> (all days) +``` + +**Best For**: API evolution, device telemetry, multi-version deployments. + +--- + +## Segmentation Strategies + +### Strategy 1: Domain-Based (DDD Bounded Contexts) + +Organize by business domains from DDD: + +``` +orders.{action}.{id} +payments.{action}.{id} +inventory.{action}.{id} +shipping.{action}.{id} +``` + +**When**: Microservices with clear domain boundaries. + +**Subscribers**: +- `orders.>` - Order service +- `payments.>` - Payment service +- `inventory.>` - Inventory service + +--- + +### Strategy 2: Tenant-Based (Multi-Tenancy) + +Organize by tenant with domains nested: + +``` +{tenant}.{domain}.{action}.{id} +``` + +**When**: SaaS platforms, multi-tenant apps. + +**Subscribers**: +- `acme-corp.>` - All events for ACME Corp +- `acme-corp.orders.>` - ACME Corp orders +- `startup-inc.orders.>` - StartUp Inc orders + +**Authorization**: Tenant prefix enables subject-based auth. + +--- + +### Strategy 3: Regional (Geo-Distribution) + +Organize by geographic region: + +``` +{domain}.{action}.{region}.{id} +``` + +**When**: Global systems, data residency requirements, compliance. + +**Subscribers**: +- `orders.>.us-west.>` - Orders in US West +- `orders.>.eu-central.>` - Orders in EU Central +- `orders.>` - All orders globally + +**Use Case**: GDPR compliance (EU data stays in EU). + +--- + +### Strategy 4: Temporal (Time-Series) + +Organize by time for IoT and monitoring: + +``` +{device-type}.{metric}.{region}.{device-id}.{timestamp} +OR +{device-type}.{metric}.{date}.{device-id} +``` + +**When**: IoT telemetry, device monitoring, high-velocity streams. + +**Examples**: +``` +sensors.temperature.us-west.sensor-456 +sensors.temperature.us-west.sensor-789 +sensors.humidity.us-west.sensor-456 +``` + +**Subscribers**: +- `sensors.temperature.us-west.>` - All temperature in US West +- `sensors.>.us-west.sensor-456` - All metrics from specific sensor +- `sensors.>` - All sensor data + +**JetStream Streams**: +``` +Stream: sensor-data +Subject: sensors.> + +Consumers filter by: +- sensors.temperature.> (temperature only) +- sensors.humidity.> (humidity only) +- sensors.>.us-west.> (region only) +``` + +--- + +### Strategy 5: Versioning (Evolution) + +Organize by API/schema version: + +``` +{domain}.{action}.v{version}.{id} +``` + +**When**: APIs evolving, schema changes, gradual rollout. + +**Examples**: +``` +orders.created.v1.order-123 (old schema) +orders.created.v2.order-123 (new schema) +users.updated.v1.user-456 (v1 update) +users.updated.v2.user-456 (v2 update) +``` + +**Migration**: +1. v1 service runs old version +2. v2 service runs new version +3. New service publishes to v2 subjects +4. Old service publishes to v1 subjects +5. Subscribers filter by version they understand +6. Once v1 fully deprecated, remove subjects + +--- + +## Real-World Example Combinations + +### E-Commerce Microservices + +Combines: Domain-based + Multi-region + +``` +orders.created.us-west.order-123 +orders.created.eu-central.order-456 +payments.authorized.us-west.payment-789 +inventory.reserved.us-west.item-101 +shipping.dispatched.eu-central.order-456 +``` + +**Subscribers**: +``` +Order Service: orders.> +Payment Service: payments.> +Inventory Service: inventory.> +Shipping Service: shipping.> +Regional Dashboard (US West): >.us-west.> +Regional Dashboard (EU): >.eu-central.> +``` + +--- + +### Multi-Tenant SaaS Platform + +Combines: Tenant-based + Domain-based + Regional + +``` +acme-corp.orders.created.us-west.order-123 +acme-corp.analytics.processed.report-456 +startup-inc.orders.created.eu-central.order-789 +startup-inc.analytics.processed.report-101 +``` + +**Subscribers**: +``` +ACME Corp Admin Dashboard: acme-corp.> +StartUp Inc Admin: startup-inc.> +Orders Service: >.orders.> +Analytics Service: >.analytics.> +Regional Monitor (US West): >.>.us-west.> +``` + +--- + +### IoT Device Telemetry + +Combines: Temporal + Domain-based + Regional + +``` +devices.telemetry.us-east.sensor-456.temperature +devices.telemetry.us-east.sensor-456.humidity +devices.telemetry.ap-south.device-789.voltage +devices.telemetry.ap-south.device-789.current +``` + +**Subscribers**: +``` +Temperature Monitor: devices.telemetry.>.>.temperature +US East Aggregator: devices.telemetry.us-east.> +Device 456 Dashboard: devices.telemetry.>.sensor-456.> +All Telemetry: devices.telemetry.> +``` + +**JetStream Stream**: +``` +Stream: iot-telemetry +Subject: devices.telemetry.> + +Consumer: temperature-only +Filter: devices.telemetry.>.>.temperature + +Consumer: us-east-only +Filter: devices.telemetry.us-east.> + +Consumer: sensor-456 +Filter: devices.telemetry.>.sensor-456.> +``` + +--- + +### Multi-Tenant Agentic AI Platform + +Combines: Tenant-based + Agent-centric + Task-based + +``` +agents.task-assigned.tenant-abc.agent-xyz.task-123 +agents.task-completed.tenant-abc.agent-xyz.task-123 +agents.capabilities.tenant-abc.llm-agent +agents.collaborate.tenant-abc.session-456.agent-xyz +platform.monitoring.all-tenants.agent-health +platform.monitoring.tenant-abc.agent-metrics +``` + +**Subscribers**: +``` +AI Agent (xyz) for tenant-abc: agents.>.tenant-abc.agent-xyz.> +All agents in tenant-abc: agents.>.tenant-abc.> +Platform Monitor: platform.monitoring.> +LLM agent capability discovery: agents.capabilities.tenant-abc.llm-agent +``` + +**Security via Subjects**: +- Tenant A's agents: `agents.>.tenant-a.>` (cannot see tenant-b) +- Tenant B's agents: `agents.>.tenant-b.>` (cannot see tenant-a) +- Admin monitor: `platform.monitoring.>` + +--- + +## Comparison Matrix + +| Pattern | Best For | Complexity | Scalability | Auth Support | +|---------|----------|-----------|-------------|--------------| +| Simple | Learning, single domain | Low | 100k events/sec | Good | +| Multi-Region | Global systems | Medium | 1M+ events/sec | Good | +| Multi-Tenant | SaaS, shared infra | Medium-High | 1M+ events/sec | Excellent | +| Request/Reply | Sync services | Low | 10k req/sec | Good | +| Event Sourcing | CQRS, event-sourced | Medium | 100k events/sec | Good | +| Temporal | IoT, time-series | High | 10M+ events/sec | Good | + diff --git a/skills/nats-design-subject/references/security.md b/skills/nats-design-subject/references/security.md new file mode 100644 index 0000000..467b650 --- /dev/null +++ b/skills/nats-design-subject/references/security.md @@ -0,0 +1,684 @@ +# NATS Subject-Based Security and Multi-Tenancy + +This reference covers authorization patterns, tenant isolation, and security best practices using NATS subject hierarchies. + +**Related references**: For subject hierarchy patterns see [patterns.md](patterns.md). For JetStream stream-level isolation see [jetstream.md](jetstream.md). For common security mistakes see [anti-patterns.md](anti-patterns.md). + +## Subject-Based Authorization Basics + +NATS authorization works by allowing/denying subjects via permissions. Your subject hierarchy directly enables or blocks access. + +### Permission Types + +```nats +permissions: + publish: + allow: ["orders.>"] # Can publish to any order subject + deny: ["orders.cancelled"] # Cannot publish to cancellations + + subscribe: + allow: ["orders.>"] # Can subscribe to any order subject + deny: ["orders.*.admin"] # Cannot subscribe to admin actions +``` + +### Wildcard Rules + +- `>` matches any number of segments (multi-level) +- `*` matches exactly one segment +- Applied left-to-right + +```nats +Subject: orders.created.us-west.order-123 + +Matches: +✓ orders.> (any order) +✓ orders.created.> (any created order) +✓ orders.created.us-west.> (any created in region) +✓ orders.*.us-west.> (any action in region) +✗ orders.*.> (order action, but only one level) +``` + +--- + +## Pattern 1: Tenant Isolation with Prefix + +**Subject Format**: +``` +{tenant}.{domain}.{action}.{scope}.{id} +``` + +**Examples**: +``` +acme-corp.orders.created.us-west.order-123 +acme-corp.orders.shipped.us-west.order-456 +startup-inc.orders.created.eu-central.order-789 +startup-inc.payments.authorized.eu-central.payment-101 +``` + +**Authorization Configuration**: + +```nats +# ACME Corp user +user acme-admin { + username: "admin@acme-corp.com" + permissions { + publish { + allow: ["acme-corp.>"] # Can only publish to own tenant + } + subscribe { + allow: ["acme-corp.>"] # Can only subscribe to own tenant + } + } +} + +# StartUp Inc user +user startup-admin { + username: "admin@startup-inc.com" + permissions { + publish { + allow: ["startup-inc.>"] # Can only publish to own tenant + } + subscribe { + allow: ["startup-inc.>"] # Can only subscribe to own tenant + } + } +} + +# Platform admin (multi-tenant) +user platform-admin { + username: "platform-admin" + permissions { + publish { + allow: [">"] # Can publish anywhere + } + subscribe { + allow: [">"] # Can subscribe anywhere + } + } +} +``` + +**Effectiveness**: ✓ Excellent. Complete tenant isolation with prefix-based auth. + +**Pros**: +- Simple to understand +- Direct authorization mapping +- Built on NATS native permissions +- Easy to add/remove tenants + +**Cons**: +- Admin operations need separate subjects (see Pattern 3) +- Cross-tenant analytics harder (see Pattern 3) + +--- + +## Pattern 2: Role-Based Access with Tiers + +**Subject Format**: +``` +{tenant}.{role}.{domain}.{action}.{scope}.{id} +``` + +**Examples**: +``` +acme-corp.admin.orders.created.us-west.order-123 +acme-corp.user.orders.status.us-west.order-456 +acme-corp.service.orders.shipped.us-west.order-789 +``` + +**Authorization**: +```nats +# Admin user (full access within tenant) +user acme-admin { + permissions { + publish { + allow: ["acme-corp.admin.>", "acme-corp.user.>"] + } + subscribe { + allow: ["acme-corp.admin.>", "acme-corp.user.>"] + } + } +} + +# Regular user (limited to user tier) +user acme-user { + permissions { + publish { + allow: ["acme-corp.user.orders.>"] # Only user-level order operations + } + subscribe { + allow: ["acme-corp.user.orders.>"] + } + } +} + +# Internal service (full access) +user acme-service { + permissions { + publish { + allow: ["acme-corp.service.>"] + } + subscribe { + allow: ["acme-corp.>"] # Subscribe to all tiers + } + } +} +``` + +**Effectiveness**: ✓ Good. Role separation at subject level. + +**Pros**: +- Fine-grained role control +- Internal services separate from user-facing +- Admin vs user operations clearly separated + +**Cons**: +- More complex subject hierarchy (6-7 segments) +- Harder for subscribers to navigate (see Anti-Pattern 3 in anti-patterns.md) + +--- + +## Pattern 3: Separate Admin Subjects for Multi-Tenancy + +**Subject Format**: +``` +# Tenant-scoped (normal operations) +{tenant}.{domain}.{action}.{scope}.{id} + +# Admin/monitoring (platform-wide, separate root) +_admin.{operation}.{tenant}.{resource}.{details} +platform.monitoring.{tenant}.{metric} +``` + +**Examples**: +``` +# User-facing operations +acme-corp.orders.created.us-west.order-123 +startup-inc.orders.shipped.eu-central.order-456 + +# Admin operations (monitoring, platform ops) +_admin.audit.acme-corp.orders.created.order-123 +_admin.audit.startup-inc.orders.shipped.order-456 + +platform.monitoring.acme-corp.event-count +platform.monitoring.startup-inc.event-latency +``` + +**Authorization**: +```nats +# Tenant user (normal operations only) +user acme-user { + permissions { + publish { + allow: ["acme-corp.>"] + } + subscribe { + allow: ["acme-corp.>"] + } + } +} + +# Platform admin (admin and monitoring only) +user platform-admin { + permissions { + publish { + allow: ["_admin.>", "platform.monitoring.>"] + } + subscribe { + allow: ["_admin.>", "platform.monitoring.>"] + } + } +} + +# System service (publishes audit logs and metrics) +user system-service { + permissions { + publish { + allow: ["_admin.>", "platform.monitoring.>"] + } + subscribe { + allow: [] + } + } +} +``` + +**Effectiveness**: ✓ Excellent. Clean separation of concerns. + +**Pros**: +- Tenant subjects remain simple (4-5 segments) +- Admin operations completely separate +- Platform observability isolated + +**Cons**: +- Two parallel subject hierarchies to maintain +- Audit trails in separate subjects from operations + +--- + +## Pattern 4: Cross-Tenant Analytics with Admin Aggregation + +**Scenario**: Multi-tenant SaaS needs analytics across all tenants, but tenants can only see their own data. + +**Subject Format**: +``` +# Tenant-specific operations +{tenant}.{domain}.{action}.{scope}.{id} + +# Analytics aggregation (admin-only) +analytics.all-tenants.{metric}.{dimension} +analytics.{tenant}.{metric}.{dimension} +``` + +**Examples**: +``` +# User operations (tenant-isolated) +acme-corp.orders.created.us-west.order-123 +startup-inc.orders.created.eu-central.order-789 + +# Analytics (aggregated) +analytics.all-tenants.total-orders.created +analytics.all-tenants.avg-latency.orders +analytics.acme-corp.total-orders.created +analytics.startup-inc.total-orders.created + +analytics.all-tenants.revenue.by-region +analytics.all-tenants.top-customers.by-spend +``` + +**Architecture**: +``` +┌─────────────────────────────┐ +│ Tenant Events │ +├─────────────────────────────┤ +│ acme-corp.orders.created.> │ +│ startup-inc.orders.created >│ +└────────────┬────────────────┘ + │ (read) + ┌─────┴──────┐ + │ Aggregator │ (service account) + └─────┬──────┘ + │ (write) + ┌─────┴────────────────────┐ + │ Analytics Subjects │ + ├──────────────────────────┤ + │ analytics.all-tenants.> │ + │ analytics.{tenant}.> │ + └──────────────────────────┘ +``` + +**Authorization**: +```nats +# Aggregator service (reads tenant data, writes analytics) +user aggregator-service { + permissions { + publish { + allow: ["analytics.>"] # Write to analytics + } + subscribe { + allow: ["acme-corp.orders.>", + "startup-inc.orders.>", + "other-tenant.orders.>"] # Read all tenant order events + } + } +} + +# Tenant user (can only see own analytics) +user acme-analyst { + permissions { + publish { + deny: ["*"] # Cannot publish + } + subscribe { + allow: ["analytics.acme-corp.>"] # Only own tenant analytics + } + } +} + +# Platform admin (sees all analytics) +user platform-admin { + permissions { + publish { + deny: ["*"] + } + subscribe { + allow: ["analytics.>"] # All analytics including cross-tenant + } + } +} +``` + +**Effectiveness**: ✓ Excellent for analytics in multi-tenant systems. + +**Pros**: +- Tenants see their analytics only +- Admin sees cross-tenant analytics +- Separates operational events from analytics +- Scales well (aggregator can be clustered) + +**Cons**: +- Requires aggregator service +- Analytics is eventually consistent +- Add operational complexity + +--- + +## Pattern 5: AI Agent Sandbox Isolation (Multi-Tenant) + +**Scenario**: Multi-tenant platform with AI agents. Each tenant's agents must be isolated, but platform can monitor all. + +**Subject Format**: +``` +agents.{action}.{tenant}.{agent-id}.{details} +agents.capabilities.{tenant}.{agent-type} +agents.collaborate.{tenant}.{session}.{agent-id} +platform.monitoring.all-tenants.agent-health +platform.monitoring.{tenant}.agent-metrics +``` + +**Examples**: +``` +# Agent operations (tenant-isolated) +agents.task-assigned.tenant-acme.agent-llm-1.task-abc +agents.task-completed.tenant-acme.agent-llm-1.task-abc +agents.task-assigned.tenant-startup.agent-code-1.task-xyz +agents.task-completed.tenant-startup.agent-code-1.task-xyz + +# Capability discovery (tenant-scoped) +agents.capabilities.tenant-acme.llm +agents.capabilities.tenant-acme.code +agents.capabilities.tenant-startup.llm + +# Agent collaboration (within tenant) +agents.collaborate.tenant-acme.session-123.agent-llm-1 +agents.collaborate.tenant-acme.session-123.agent-code-1 + +# Platform monitoring (admin only) +platform.monitoring.all-tenants.agent-health +platform.monitoring.all-tenants.task-success-rate +platform.monitoring.tenant-acme.agent-metrics +platform.monitoring.tenant-acme.error-rate +``` + +**Authorization**: +```nats +# Agent in Tenant A (complete isolation) +user agent-tenant-acme { + permissions { + publish { + allow: ["agents.task-assigned.tenant-acme.>", + "agents.task-completed.tenant-acme.>", + "agents.collaborate.tenant-acme.>"] + } + subscribe { + allow: ["agents.task-assigned.tenant-acme.>", + "agents.task-completed.tenant-acme.>", + "agents.capabilities.tenant-acme.>", + "agents.collaborate.tenant-acme.>"] + } + } +} + +# Agent in Tenant B (isolated from Tenant A) +user agent-tenant-startup { + permissions { + publish { + allow: ["agents.task-assigned.tenant-startup.>", + "agents.task-completed.tenant-startup.>", + "agents.collaborate.tenant-startup.>"] + } + subscribe { + allow: ["agents.task-assigned.tenant-startup.>", + "agents.task-completed.tenant-startup.>", + "agents.capabilities.tenant-startup.>", + "agents.collaborate.tenant-startup.>"] + } + } +} + +# Platform monitoring (admin) +user platform-monitor { + permissions { + publish { + allow: ["platform.monitoring.>"] + } + subscribe { + allow: ["platform.monitoring.>"] + } + } +} + +# System orchestrator (manages agents) +user system-orchestrator { + permissions { + publish { + allow: ["agents.>", "platform.monitoring.>"] + } + subscribe { + allow: ["agents.>", "platform.monitoring.>"] + } + } +} +``` + +**Security Model**: +``` +Tenant A Agents: +✓ Can publish/subscribe to: agents.*.tenant-acme.> +✗ Cannot see: agents.*.tenant-startup.> +✗ Cannot see: platform.monitoring.> + +Tenant B Agents: +✓ Can publish/subscribe to: agents.*.tenant-startup.> +✗ Cannot see: agents.*.tenant-acme.> +✗ Cannot see: platform.monitoring.> + +Platform Admin: +✓ Can see all agent activity +✓ Can see metrics and health +✓ Cannot directly control agents (separate admin service) +``` + +**Effectiveness**: ✓ Excellent for agentic AI platforms. + +**Pros**: +- Complete tenant isolation at subject level +- Platform visibility without tenant access +- Inter-agent collaboration within tenant +- Scales to many agents and tenants + +**Cons**: +- More complex subject hierarchy +- Requires careful permission management +- Audit trail needs separate subjects + +--- + +## Pattern 6: Environment Separation (Dev/Staging/Prod) + +**Scenario**: Single NATS cluster serves dev, staging, and production. Complete isolation needed. + +**Subject Format**: +``` +{environment}.{tenant}.{domain}.{action}.{scope}.{id} +``` + +**Examples**: +``` +dev.acme-corp.orders.created.us-west.order-123 +staging.acme-corp.orders.created.us-west.order-456 +prod.acme-corp.orders.created.us-west.order-789 + +dev.startup-inc.orders.created.eu-central.order-abc +staging.startup-inc.orders.created.eu-central.order-def +prod.startup-inc.orders.created.eu-central.order-ghi +``` + +**Authorization**: +```nats +# Dev team (dev environment only) +user dev-team { + permissions { + publish { + allow: ["dev.>"] + } + subscribe { + allow: ["dev.>"] + } + } +} + +# Staging team (staging only) +user staging-team { + permissions { + publish { + allow: ["staging.>"] + } + subscribe { + allow: ["staging.>"] + } + } +} + +# Production team (prod only) +user prod-team { + permissions { + publish { + allow: ["prod.>"] + } + subscribe { + allow: ["prod.>"] + } + } +} + +# CI/CD (can promote between environments) +user ci-cd { + permissions { + publish { + allow: ["dev.>", "staging.>", "prod.>"] + } + subscribe { + allow: ["dev.>", "staging.>", "prod.>"] + } + } +} +``` + +**Effectiveness**: ✓ Good for environment isolation. + +**Pros**: +- Single cluster, fully isolated environments +- Subject-based auth prevents cross-environment leakage +- Easy to promote from dev → staging → prod + +**Cons**: +- Adds subject layer (more segments) +- Complex permissions for multi-role users + +--- + +## Least Privilege Principle + +### Example: Order Processing Service + +**Over-Permissive** (❌): +```nats +user order-service { + permissions { + publish { allow: [">"] } # Can publish ANYTHING + subscribe { allow: [">"] } # Can subscribe to ANYTHING + } +} +``` + +**Least Privilege** (✓): +```nats +user order-service { + permissions { + publish { + allow: [ + "orders.created.>", # Create orders + "orders.updated.>", # Update orders + "orders.cancelled.>", # Cancel orders + "inventory.reserved.>" # Call inventory service + ] + } + subscribe { + allow: [ + "inventory.reserved.>", # Listen for reservations + "payments.authorized.>" # Listen for payment confirmations + ] + } + } +} +``` + +**Benefits**: +- Service cannot accidentally publish to wrong domain +- Service cannot snoop on unrelated topics +- Compromised service has limited blast radius +- Clear documentation of service dependencies + +--- + +## Audit Trail Pattern + +**Subject Format**: +``` +_audit.{operation}.{user}.{timestamp}.{resource} +_audit.{domain}.{action}.{timestamp}.{details} +``` + +**Examples**: +``` +_audit.publish.user@acme-corp.2026-01-24T12:34:56Z.orders.created +_audit.subscribe.admin@platform.2026-01-24T12:34:57Z.tenant-acme.orders +_audit.orders.created.2026-01-24T12:34:58Z.order-123 +_audit.orders.deleted.2026-01-24T12:34:59Z.order-456 +``` + +**Authorization**: +```nats +# Audit service (writes audit logs) +user audit-service { + permissions { + publish { + allow: ["_audit.>"] + } + subscribe { + deny: ["*"] + } + } +} + +# Security team (reads audit logs only) +user security-team { + permissions { + publish { + deny: ["*"] + } + subscribe { + allow: ["_audit.>"] + } + } +} +``` + +--- + +## Security Checklist + +When designing multi-tenant subject architecture: + +- [ ] **Tenant Isolation**: Tenant prefix prevents cross-tenant access? +- [ ] **Least Privilege**: Each user/service has minimum permissions? +- [ ] **Admin Subjects**: Admin operations separate from user operations? +- [ ] **Audit Trail**: All sensitive operations logged to audit subjects? +- [ ] **Environment Separation**: Dev/staging/prod isolated by subject? +- [ ] **Role-Based**: Roles clearly separated (user/service/admin)? +- [ ] **Denial Rules**: Explicit deny rules for sensitive subjects? +- [ ] **Monitoring**: Platform metrics isolated from tenant operations? +- [ ] **Credential Rotation**: Plan for credential management? +- [ ] **Documentation**: Permission matrix documented for audit? + diff --git a/skills/nats-design-subject/references/use-cases.md b/skills/nats-design-subject/references/use-cases.md new file mode 100644 index 0000000..5ede505 --- /dev/null +++ b/skills/nats-design-subject/references/use-cases.md @@ -0,0 +1,414 @@ +# NATS Subject Architecture: Domain-Specific Use Cases + +Complete worked examples showing how subject patterns, security, and JetStream come together for 5 real-world domains. Each use case focuses on the end-to-end flow — for pattern details see [patterns.md](patterns.md), for authorization see [security.md](security.md), for stream design see [jetstream.md](jetstream.md). + +## Index + +1. [Microservices (E-Commerce)](#use-case-1-microservices-e-commerce) - Multi-service pub/sub + request/reply patterns +2. [IoT / Device Telemetry](#use-case-2-iot--device-telemetry) - High-volume sensor data with regional aggregation +3. [Multi-Tenant SaaS](#use-case-3-multi-tenant-saas) - Tenant isolation + authorization patterns +4. [Event Sourcing / CQRS](#use-case-4-event-sourcing--cqrs) - Aggregate streams + projection patterns +5. [Agentic AI Platform](#use-case-5-agentic-ai-platform) - Agent communication + task routing + +--- + +## Use Case 1: Microservices (E-Commerce) + +### Scenario + +E-commerce platform with microservices: Orders, Payments, Inventory, Shipping. Services communicate via NATS pub/sub and request/reply. + +### Subject Architecture + +``` +{service}.{action}.{region}.{resource-id} + +- orders.created.us-west.order-123 +- orders.cancelled.us-west.order-456 +- payments.authorized.us-west.payment-789 +- payments.failed.us-west.payment-101 +- inventory.reserved.us-west.item-456 +- shipping.dispatched.us-east.shipment-123 +``` + +### Request/Reply for Synchronous Calls + +``` +Order Service → Inventory Service: "Do I have this item?" + +Request Subject: inventory.request.check-availability +Reply-To: _INBOX.order-service-abc-123 + +Inventory Service Response: + Publishes to: _INBOX.order-service-abc-123 + With result: {available: true, quantity: 5} +``` + +### Cross-Service Subscriber Map + +``` +Order Service subscribes to: +- payments.authorized.> (payment confirmations) +- inventory.reserved.> (fulfillment ready) +- shipping.dispatched.> (tracking updates) + +Inventory Service subscribes to: +- orders.created.> (new orders to reserve) +- orders.cancelled.> (release reservations) +- inventory.request.> (availability queries) + +Shipping Service subscribes to: +- orders.paid.> (ready to ship) + +Order Dashboard subscribes to: +- orders.> (all order events, any region) +- orders.>.us-west.> (regional dashboard) +``` + +### Multi-Region Deployment + +``` +Regional Dashboard: orders.created.us-west.> +Global Dashboard: orders.created.> +``` + +### JetStream Retention by Domain + +```nats +stream: "order-events" subjects: ["orders.>"] max_age: 90d +stream: "payment-events" subjects: ["payments.>"] max_age: 7y # Compliance +stream: "inventory-events" subjects: ["inventory.>"] max_age: 30d +stream: "shipping-events" subjects: ["shipping.>"] max_age: 1y +``` + +For full stream configuration see [jetstream.md](jetstream.md) Pattern 2 (Multi-Domain Streams). + +--- + +## Use Case 2: IoT / Device Telemetry + +### Scenario + +Smart building system: 10,000 sensors across regions sending temperature, humidity, and occupancy. Needs real-time monitoring, aggregation, and alerting. + +### Subject Architecture + +``` +devices.{metric}.{region}.{device-id} + +- devices.temperature.us-west.sensor-456 +- devices.humidity.us-west.sensor-456 +- devices.occupancy.us-west.floor-1-zone-a +``` + +### Message Content + +```json +Subject: devices.temperature.us-west.sensor-456 +Headers: + X-Timestamp: 2026-01-24T12:34:56Z + X-Device-Type: temperature-sensor + X-Battery: 87% +Payload: { + temperature: 22.5, + unit: "celsius", + accuracy: 0.1 +} +``` + +### Subscriber Paths + +``` +Real-Time Dashboard: +- devices.temperature.us-west.> (all temps in US West) +- devices.humidity.us-west.> (all humidity in US West) + +Temperature Alerting: +- devices.temperature.> (all temperatures, any region) + +Regional Aggregator: +- devices.>.us-west.> (all metrics in region) + +Specific Sensor Debug: +- devices.>.us-west.sensor-456 (all metrics from one sensor) +``` + +### Aggregation Workflow + +``` +Raw telemetry (10,000 sensors every 30 sec) + ↓ +Consumer: raw-telemetry (filter: devices.temperature.us-west.>) + ↓ (process: calculate avg, min, max per floor) + ↓ +Publish to: devices.aggregated.hourly.us-west.floor-1 + ↓ +Consumer: aggregation-stream (filter: devices.aggregated.>) + ↓ +Store in time-series DB or separate stream +``` + +### Alerting + +``` +Temperature out of range: +- Subscribe: devices.temperature.> +- Alert if: value > 28 or value < 18 +- Publish: devices.alerts.temperature-high.us-west.sensor-456 + +Offline device: +- Subscribe: devices.temperature.> +- Alert if: no message in 60 seconds +- Publish: devices.alerts.offline.us-west.sensor-456 +``` + +### High-Volume Strategy + +With 10,000 devices sending every 30 seconds = ~1,000 messages/sec total. + +- One stream per region (not global) +- Consumer per metric type +- Aggressive retention: 7 days raw, 1 year aggregated +- Size limits to force data aging + +For stream configuration see [jetstream.md](jetstream.md) High-Volume Subject Design section. + +--- + +## Use Case 3: Multi-Tenant SaaS Platform + +### Scenario + +Analytics SaaS serving 100 customers. Each tenant has events, queries, reports. Strict isolation required. + +### Subject Architecture + +``` +{tenant}.{domain}.{action}.{scope}.{id} + +Tenant A (ACME Corp): +- acme-corp.events.ingested.2026-01-24.event-123 +- acme-corp.queries.created.2026-01-24.query-456 +- acme-corp.reports.generated.2026-01-24.report-789 + +Tenant B (StartUp Inc): +- startup-inc.events.ingested.2026-01-24.event-101 +- startup-inc.queries.created.2026-01-24.query-202 +``` + +### Subscriber Paths + +``` +Tenant-scoped: +ACME Dashboard: acme-corp.> +ACME Query Engine: acme-corp.queries.created.> + +Cross-tenant (admin only): +Platform Monitor: analytics.> +Usage Tracker: analytics.usage.daily.> +``` + +### Analytics Aggregation Flow + +This is the unique challenge for multi-tenant SaaS — how to aggregate across tenants while maintaining isolation: + +``` +Individual tenant events: +acme-corp.events.ingested.2026-01-24.event-123 +startup-inc.events.ingested.2026-01-24.event-456 + + ↓ Aggregator service (reads all tenants, writes analytics) + +Aggregated analytics (admin only): +analytics.usage.acme-corp.events-per-day +analytics.usage.all-tenants.total-events-per-day +analytics.cost.all-tenants.monthly-revenue +``` + +### GDPR Compliance via Region Filtering + +``` +{tenant}.{domain}.{action}.{region}.{id} + +EU customers: startup-inc.*.*.eu-central.> +US customers: startup-inc.*.*.us-*.> +``` + +For authorization configuration see [security.md](security.md) Pattern 1 (Tenant Isolation) and Pattern 4 (Cross-Tenant Analytics). For stream-per-tenant setup see [jetstream.md](jetstream.md) Pattern 3. + +--- + +## Use Case 4: Event-Sourced Systems (CQRS) + +### Scenario + +Order processing domain using event sourcing. All state changes are immutable events. Separate read models (projections) from command model (aggregates). + +### Subject Architecture + +``` +{aggregate}.{action}.v{version}.{aggregate-id} + +Orders Aggregate: +- orders.order.created.v1.order-123 (event 1) +- orders.order.confirmed.v1.order-123 (event 2) +- orders.order.paid.v1.order-123 (event 3) +- orders.order.shipped.v1.order-123 (event 4) + +Accounts Aggregate: +- accounts.account.opened.v1.account-456 +- accounts.account.credited.v1.account-456 +- accounts.account.debited.v1.account-456 +``` + +### Event Versioning + +``` +Old version (deprecated): orders.order.created.v1.order-1 +New version (current): orders.order.created.v2.order-3 + +Transition strategy: +1. Services read both v1 and v2 +2. New publishers send v2 only +3. Migrate all consumers to v2 +4. Retire v1 after full migration +``` + +### CQRS Flow + +``` +User Command: "Create Order" + ↓ +Command Handler: CreateOrderCommand + ↓ +Aggregate: OrderAggregate.createOrder() + ↓ +Publish to: orders.order.created.v1.order-123 + ↓ +┌─────────────────────────────────────────┐ +│ Multiple Consumers (Projections) │ +├─────────────────────────────────────────┤ +│ OrderStatusProjection │ +│ ↓ updates OrderStatus read model │ +│ │ +│ CustomerOrdersProjection │ +│ ↓ updates CustomerOrders read model │ +│ │ +│ RevenueProjection (waits for order.paid)│ +│ ↓ later updates Revenue read model │ +└─────────────────────────────────────────┘ + ↓ +Read Models ready for queries +``` + +### Projection Subscriber Patterns + +``` +Current Order Status: orders.order.> → OrderStatusReadModel +Orders by Customer: orders.order.> → CustomerOrdersReadModel +Revenue by Region: orders.order.paid.> → RevenueReadModel +``` + +For JetStream event store configuration and consumer patterns see [jetstream.md](jetstream.md) Pattern 4 (Event-Sourced Aggregate Streams). + +--- + +## Use Case 5: Agentic AI Platforms (Multi-Tenant) + +### Scenario + +Multi-tenant platform where AI agents autonomously process tasks. Each tenant has isolation, agents collaborate within tenant, platform monitors all. + +### Subject Architecture + +``` +agents.{action}.{tenant}.{agent-id}.{task-id} +agents.capabilities.{tenant}.{agent-type} +agents.collaborate.{tenant}.{session-id}.{agent-id} +platform.monitoring.{tenant}.{metric} +``` + +### Multi-Agent Workflow + +This is the unique value of the agentic AI pattern — orchestrated multi-step task processing: + +``` +User Request (Tenant ACME): "Implement OAuth2 in our API" + +Orchestrator publishes: +agents.task-assigned.tenant-acme.agent-planning-1.task-001 + → Planning Agent analyzes requirements + +Planning Agent collaborates: +agents.collaborate.tenant-acme.session-001.agent-planning-1 + → Requests code agents for implementation + +Orchestrator fans out: +agents.task-assigned.tenant-acme.agent-code-1.task-002 +agents.task-assigned.tenant-acme.agent-code-2.task-003 + → Code agents implement + test in parallel + +Code agents report: +agents.task-completed.tenant-acme.agent-code-1.task-002 +agents.task-completed.tenant-acme.agent-code-2.task-003 + +Orchestrator routes to review: +agents.task-assigned.tenant-acme.agent-review-1.task-004 + → Review agent checks quality → done +``` + +### Subscriber Paths + +``` +Agent Task Queue: +Agent LLM-1: agents.task-assigned.tenant-acme.agent-llm-1.> + +Inter-Agent Collaboration: +All agents in session: agents.collaborate.tenant-acme.session-001.> + +Orchestrator Tracking: +Completions: agents.task-completed.tenant-acme.> +Failures: agents.task-failed.tenant-acme.> + +Capability Discovery: +All agents: agents.capabilities.tenant-acme.> + +Platform Monitoring: +Admin: platform.monitoring.> +``` + +### Message Example + +```json +Subject: agents.task-assigned.tenant-acme.agent-llm-1.task-abc-123 +Headers: + X-Priority: high + X-Deadline: 2026-01-24T14:00:00Z + X-Correlation-Id: request-456 +Payload: { + taskId: "task-abc-123", + type: "code-review", + context: { + repository: "oauth2-api", + pullRequestId: "pr-789" + }, + requirements: [ + "Review security best practices", + "Verify test coverage" + ] +} +``` + +For tenant isolation authorization see [security.md](security.md) Pattern 5 (AI Agent Sandbox Isolation). For JetStream stream setup see [jetstream.md](jetstream.md) Pattern 3. + +--- + +## Comparison Matrix + +| Use Case | Subject Depth | Domain Count | Subscriber Count | Throughput | Retention | +|----------|---|---|---|---|---| +| Microservices | 4 | 3-10 | 50-100 | 10k-100k/sec | 30d-7y | +| IoT | 4 | 1 (many dimensions) | 20-50 | 1M+/sec | 1-7 days | +| Multi-Tenant SaaS | 5 | 3-8 | 50-200 | 100k-1M/sec | 30-90d | +| Event Sourcing | 4 | 5-20 | 100+ | 10k-100k/sec | forever | +| Agentic AI | 5 | 2-3 | 100-1000+ | 10k-100k/sec | 30-365d |