feat(messaging): add a messaging category with SQS and Pub/Sub - #157
Conversation
|
| Filename | Overview |
|---|---|
| packages/api/src/adapter-aws/AwsSqsAdapter.ts | SQS messaging adapter with name-based ids, attribute normalization, FIFO create, and range validation. |
| packages/api/src/adapter-gcp/GcpPubSubAdapter.ts | Pub/Sub topics adapter using PUT create and short topic ids. |
| packages/api/src/cloudProxy.ts | Registers AwsSqsAdapter and GcpPubSubAdapter on the shared registry. |
| packages/api/src/cloud-spi/types.ts | Widens CloudResource.type to KnownResourceType plus open string brand. |
| packages/api/src/cloud-spi/messagingSchema.ts | AWS/GCP messaging ServiceSchema fields, columns, and CRUD capabilities. |
Reviews (4): Last reviewed commit: "chore(messaging): merge main into messag..." | Re-trigger Greptile
There was a problem hiding this comment.
Thanks Saif, this is a strong one. Using the queue name as the stable id and resolving it per call with GetQueueUrl matches what the AWS console does, and catching the FifoQueue attribute gap (which neither the stub nor Floci core would have surfaced) shows real care for the actual AWS contract. Both lockfiles are regenerated, tests are colocated and thorough on both adapters, and the catalog row follows the new AGENTS.md pattern exactly.
One coordination note before anything else: SQS and the messaging category are the most contested spot in the queue right now. Three earlier open PRs add SQS under a queue category (#137, #142, #148), and #144 and #146 also introduce a messaging category. I am coordinating the taxonomy (queue vs messaging) and how these land, so please treat that as an open question rather than settled by whichever PR merges first. Nothing for you to do yet, but a rebase or renaming may be requested once that call is made.
Two small things on the code itself, neither blocking:
- The new doc comment on
CloudResource.typegivessqs-queueas an example, but the adapter emitsqueueandfifo-queue. Could you align the example so the convention reads unambiguously? visibilityTimeoutandmessageRetentionPeriodare free-text and passed straight through, so a non-numeric value only fails at the SQS call. A quick numeric check increate()with aValidationErrorwould give the form a friendlier error. Fine as a follow-up if you prefer.
The Pub/Sub PUT-on-resource-path detail and the degrade-one-row behavior on a failed describe are both exactly right. Nice work.
The sidebar carried a route-less "Queue" placeholder that could never light up, while Floci core had three seeded SQS queues the whole time. This replaces it with a real category serving both AWS SQS and GCP Pub/Sub. One category rather than separate queue and topic ones: the delivery semantics differ, but a console renders both the same way — a named endpoint with a depth and a few settings — so splitting them would double the nav for nothing. Two contract details worth naming: - SQS identifies a queue by URL, but a URL embeds the endpoint and changes if the runtime is re-pointed, which makes it a poor resource id. The adapter uses the queue name and resolves it via GetQueueUrl per call, as the AWS console does. Timestamps are converted from SQS epoch seconds to the ISO strings the contract expects. - Pub/Sub creates with PUT on the resource path, not POST to the collection. A queue whose GetQueueAttributes fails still lists, with the detail omitted — a describe failure should degrade one row, not blank the table. This is also the first service to need `CloudResource.type` beyond its original eleven literals. The union is now open with a documented convention and a `KnownResourceType` alias: nothing dispatches exhaustively on it, so a closed union meant editing two packages per adapter for no safety. `CloudServiceType` stays closed, because that one is route-addressable. Verified against the live stack: the three seeded queues list with their real ARNs, depths and visibility timeouts, and create/inspect/delete round-trips on both providers.
The adapter accepted a .fifo name and reported the resulting queue as `fifo-queue`, but never sent `Attributes.FifoQueue=true` on create. Real SQS rejects that request outright. It went unnoticed because neither layer of verification could see it: the unit test used a stub that accepts any input, and Floci core infers FIFO from the name suffix, so creating one locally succeeded. The gap would only have appeared against AWS. Adds tests asserting the attribute is sent for a .fifo name and omitted otherwise.
baa8e11 to
95a9e0d
Compare
|
Rebased onto This branch's open README regenerated from Gate green after the rebase: lint, type-check, 463 tests, build. |
visibilityTimeout and messageRetentionPeriod were free text passed straight through, so a bad value only failed at CreateQueue. Also corrects the CloudResource.type doc example, which said sqs-queue where the adapter emits queue and fifo-queue.
caff3da to
3084c30
Compare
|
@hectorvent Both taken, thank you. 1. Doc example corrected. The 2. Numeric validation added, with SQS's own bounds rather than a generic "must be a number": Encoding the provider's range rather than just the type follows the same principle as the On the taxonomy coordination: understood, and no objection to whatever the maintainer decides. Flagging one input in case it is useful: this branch now carries the open Gate green: lint, type-check, 465 tests, build. (I pushed a version with a type error in the test file first and amended it within the minute — the checks you see on the final SHA are the real ones.) |
|
🎉 This PR is included in version 0.4.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
# [0.4.0](floci-io/floci-ui@0.3.0...0.4.0) (2026-09-01) ### Bug Fixes * **ec2:** include catalog AMIs in launch selector ([floci-io#191](floci-io#191)) ([b72135d](floci-io@b72135d)) ### Features * **aws:** add CloudFormation adapter to Cloud Explorer ([floci-io#184](floci-io#184)) ([f3d6105](floci-io@f3d6105)), closes [floci-io#81](floci-io#81) [floci-io#75](floci-io#75) [floci-io#81](floci-io#81) [floci-io#75](floci-io#75) [floci-io#81](floci-io#81) * **aws:** add EventBridge explorer ([floci-io#146](floci-io#146)) ([42944e9](floci-io@42944e9)), closes [floci-io#85](floci-io#85) * **aws:** add IAM to Cloud Explorer ([floci-io#145](floci-io#145)) ([fc50d19](floci-io@fc50d19)), closes [floci-io#79](floci-io#79) * **aws:** add Secrets Manager resource adapter ([floci-io#193](floci-io#193)) ([4811866](floci-io@4811866)) * **azure:** add databases and split Cosmos NoSQL ([floci-io#149](floci-io#149)) ([2e704f4](floci-io@2e704f4)), closes [floci-io#92](floci-io#92) [floci-io#67](floci-io#67) [floci-io/floci-az#138](floci-io/floci-az#138) [floci-io#143](floci-io#143) * **azure:** add Service Bus explorer ([floci-io#144](floci-io#144)) ([a9f0d06](floci-io@a9f0d06)), closes [floci-io#89](floci-io#89) * **eks:** Manage nodegroups and Fargate profiles via Cloud Proxy ([floci-io#194](floci-io#194)) ([3091cc9](floci-io@3091cc9)), closes [floci-io#106](floci-io#106) * **loadbalancing:** add an AWS Elastic Load Balancing adapter ([floci-io#168](floci-io#168)) ([10d4298](floci-io@10d4298)), closes [floci-io#162](floci-io#162) [floci-io#162](floci-io#162) [floci-io#156](floci-io#156) * **messaging:** add a messaging category with SQS and Pub/Sub ([floci-io#157](floci-io#157)) ([dfa6d1c](floci-io@dfa6d1c)), closes [floci-io#155](floci-io#155) [floci-io#155](floci-io#155) [floci-io#156](floci-io#156) [floci-io#156](floci-io#156) * **secretsmanager:** add JSON key-value editor for secret values ([floci-io#195](floci-io#195)) ([8e88961](floci-io@8e88961)), closes [floci-io#151](floci-io#151) * **ses:** Add AWS SES mailbox to Cloud Explorer ([floci-io#196](floci-io#196)) ([6389a56](floci-io@6389a56)), closes [floci-io#130](floci-io#130)
… Manager (#156) Narrowed: this is now **secrets only**, off `main`. Cloud SQL/GKE are #155 and messaging is #157 — all three independent. Both runtimes serve secret metadata, but the console had only an AWS-specific page and nothing for GCP. | Cloud | Was | Now | |---|---|---| | AWS Secrets Manager | dedicated page outside Cloud Explorer | SPI adapter; page retained for value reveal | | GCP Secret Manager | not present | list, create, inspect, delete | ## No secret value reaches a resource object Neither adapter reads a value and no schema returns one. A value on `CloudResource.metadata` would flow into the inspector, the client-side query cache and the request telemetry — reveal has to be an explicit, uncached action. Both test files assert this rather than trusting the implementation: ```ts // AWS: GetSecretValue is never sent, and no value-shaped key survives expect(sent.every((c) => c.constructor.name !== 'GetSecretValueCommand')).toBe(true) for (const key of Object.keys(resource.metadata)) expect(key.toLowerCase()).not.toContain('value') // GCP: reading a payload needs versions/latest:access — never requested expect(calls.every((c) => !c.url.includes(':access'))).toBe(true) ``` ## On the create-payload review comment Half right, and the useful half is now fixed. The premise isn't: AWS marks both `SecretString` and `SecretBinary` **optional** on `CreateSecret`, and a valueless create succeeds against the runtime too — ``` $ curl -X POST .../aws/services/secrets/resources -d '{"secretName":"floci-noval"}' create: 201 → exists: floci-noval ``` — so create wasn't failing and the matrix wasn't wrong. But a secret you can't put a value in is close to useless, and the schema offered no way to supply one. Create now accepts an optional `secretValue` and forwards it once. That direction is safe: input never lands on a resource, and there's a test asserting the value does not appear in the returned object. ## `routeByCloud` New catalog field, and the part I'd most like a second opinion on. AWS keeps its dedicated page because that's still the only way to read a value; GCP goes through the generic explorer. A single `route` per catalog entry can't express that, and pointing GCP at an AWS-only page would be worse. So the entry carries `routeByCloud: {aws: '/secretsmanager'}`, and the override disappears when the page migrates. I considered migrating the AWS page here instead, but that drops value reveal — a real regression — until the row-action mechanism exists. This felt like the honest intermediate rather than a silent capability loss. ## Small correctness note AWS delete uses `ForceDeleteWithoutRecovery`. Without it the secret enters a 30-day recovery window and keeps appearing in the list after the user deleted it, which reads as a broken delete. ## Overlap to be aware of `CloudResource.type` is widened identically here and on #157, so whichever lands second is a no-op for that hunk. ## Verification `pnpm lint`, `type-check`, `test`, `build` pass. Live round-trip on both providers, with routing resolving per cloud: ``` aws Secrets Manager available route=/secretsmanager gcp Secret Manager available route=secrets azure Secrets Manager coming_soon route=secrets ``` ## Type of change - [ ] Bug fix (`fix:`) - [x] New feature / service UI (`feat:`) - [ ] Breaking change (`feat!:` or `fix!:`) - [ ] Docs / chore ## Area - [ ] Frontend (`packages/frontend`) - [x] API / Cloud Proxy (`packages/api`) - [x] Cloud Explorer adapter / schema - [ ] Build / CI / Docker ## Checklist - [x] `pnpm lint`, `pnpm type-check`, `pnpm test`, and `pnpm build` pass locally - [x] New or updated tests added where it makes sense (`bun test` in `packages/api`) - [x] No fake/mock data added - [x] Commit messages / PR title follow [Conventional Commits](https://www.conventionalcommits.org/) --------- Co-authored-by: fredpena <f.ant.pena@gmail.com>
Split out of #155 so each service lands on its own. Independent of #155 and #156 — this branch is off
main.The sidebar carried a route-less "Queue" placeholder that could never light up, while Floci core has had three seeded SQS queues the whole time:
One category for both SQS and Pub/Sub rather than separate queue/topic ones: the delivery semantics differ, but a console renders both the same way — a named endpoint with a depth and a few settings — so splitting them would double the nav for nothing.
Contract details worth a look
GetQueueUrlper call, as the AWS console does. Epoch-second timestamps are converted to the ISO strings the contract expects.Attributes.FifoQueue=trueis set for a.fifoname. Worth calling out because neither layer of verification caught its earlier absence: the unit stub accepted any input, and Floci core infers FIFO from the suffix so it worked locally too. It would only have failed against AWS. Caught in review on feat(gcp): add Cloud SQL and GKE adapters #155 and fixed here.PUTon the resource path, notPOSTto the collection.GetQueueAttributesfails still lists, with detail omitted — a describe failure should degrade one row, not blank the table.One SPI change
CloudResource.typewas a closed union of eleven literals;queue,fifo-queueandtopicare the first additions since. It is now open with aKnownResourceTypealias documenting the convention — nothing dispatches exhaustively on it, so a closed union meant editing two packages per adapter for no safety.CloudServiceTypestays closed, because that one is route-addressable and a typo there should 404.The identical widening also appears on #156, so whichever lands second is a no-op for that hunk.
Adds
@aws-sdk/client-sqs(bothbun.lockandpnpm-lock.yamlupdated — the API installs withbun --frozen-lockfile, which I learned the hard way).Verification
pnpm lint,type-check,test,buildpass. Live round-trip on both providers:README service matrix regenerated from the registry.
Type of change
fix:)feat:)feat!:orfix!:)Area
packages/frontend)packages/api)Checklist
pnpm lint,pnpm type-check,pnpm test, andpnpm buildpass locallybun testinpackages/api)