Scoping a first-class AWS Bedrock provider with native SigV4 auth (follow-up to #50)

[cru-Luis-Rodriguez]

Following up on #50, where you said:

we don't yet have a dedicated built-in Bedrock preset (with native SigV4 auth, region selection, etc.) — if that's something you'd find valuable and are interested in contributing, we'd welcome a PR adding Bedrock as a first-class provider in internal/llm/providers.go. Happy to help scope it out in an issue if you'd like to discuss the approach first.

Taking you up on the scoping offer. I have this working and tested against a live Bedrock account, and there is one decision I would rather have your view on before opening a PR: the AWS SDK dependency.

The gap the existing paths leave

To be clear about what is already possible, since #50 covered it: the LiteLLM preset and the OpenAI-compatible endpoint both work. The case neither covers is an organisation whose Bedrock access is governed entirely through IAM and SSO.

• LiteLLM proxy means a process to run in front of every workstation and CI runner. Workable, but it is infrastructure to operate for what is otherwise a self-contained CLI.
• Custom provider against the OpenAI-compatible endpoint needs a Bedrock API key — a long-lived static credential to generate, distribute and rotate. That is exactly what organisations move to SSO to stop doing. Ours are short-lived credentials from aws sso login, scoped by an IAM permission set; there is no key to put in a config file.

Native SigV4 removes both. ocr then authenticates the same way every other AWS tool on the machine already does, and access is granted or revoked by editing an IAM policy.

Shape of the change

Bedrock serves the same Messages API, so very little is new. bedrock.WithConfig from anthropic-sdk-go handles the signing, the model-into-path rewrite, and anthropic_version injection — and it returns an option.RequestOption, which is exactly the type NewAnthropicClient already assembles a slice of. So AnthropicClient is reused as-is behind a new protocol constant, following the extension contract protocol.go documents for itself.

• protocol.go — one constant plus its NormalizeProtocol / ValidateProtocol entries
• providers.go — a bedrock preset, and an AmbientAuth flag marking providers that authenticate from the environment rather than an api_key
• resolver.go — two checks taught about ambient auth (below)
• client.go — NewAnthropicBedrockClient, essentially the existing constructor minus the auth headers
• tests, including one asserting api_key is still required for every non-ambient provider

The resolver changes are the part most likely to want your opinion. tryProviderConfig requires a non-empty api_key, and ResolveEndpointWithModelOverride requires both URL and Token before an endpoint counts as complete. Bedrock has none of the three — the region supplies the host, SigV4 supplies the credentials — so a correct config fell through every strategy and surfaced no valid LLM endpoint configured, i.e. the error for having configured nothing at all. I gated both on AmbientAuth rather than special-casing a provider name, but I am not attached to that shape.

Region and profile resolve from the standard AWS chain, with optional aws_region / aws_profile in the provider entry so a run is reproducible without exporting environment variables first.

The dependency question

It adds aws-sdk-go-v2/config as the only direct dependency, and 13 transitive modules with it (credentials, service/sso, service/ssooidc, service/sts, feature/ec2/imds, internal/ini, smithy-go, and a few smaller ones). That is real weight for a provider most users will not enable, and I would understand hesitation.

Options, happy to follow whichever you prefer:

1. Direct dependency — simplest; one go.mod entry and it works out of the box.
2. Build tag — //go:build bedrock, keeping the SDK out of default builds, at the cost of a second build configuration and a release-artefact decision.
3. Separate module or plugin — cleanest dependency story, most work; I would want your guidance on placement given extensions/ and plugins/.

I lean toward (1) for usability but will defer to whatever fits your release story. If the answer is (2) or (3), knowing now saves a rewrite.

One thing that may belong in the SDK instead

Worth surfacing because it affects scope. bedrock.WithConfig prefers bearer-token auth whenever cfg.BearerAuthTokenProvider is non-nil, and config.LoadDefaultConfig populates that from the AWS SSO token cache. So any SSO-authenticated caller silently sends its SSO OIDC access token instead of signing, and Bedrock answers:

403 Forbidden {"Message":"Invalid API Key format: Must start with pre-defined prefix"}

Diagnosis is slower than it should be because the SDK reports the pre-middleware URL (/v1/messages) in the error, which makes the path rewriting look broken when it is working correctly.

My patch clears that provider unless AWS_BEARER_TOKEN_BEDROCK was set deliberately. It works, but it is arguably a workaround for an anthropic-sdk-go bug. If you would rather see it fixed there first, that is fair and would shrink the ocr-side patch. I also noticed bedrock/bedrockmantle.go resolves auth by an explicit documented precedence, which reads as the more considered design of the two — possibly relevant if you have a view on which path a built-in preset should take.

Happy to open the PR as soon as you have a preference on the dependency, or to link the branch first if seeing the code would help you decide.

[stay-foolish-forever]

@cru-Luis-Rodriguez

Luis — thank you for the exceptionally thorough write-up. The gap analysis (LiteLLM proxy / static key vs. IAM+SSO) is exactly the right framing, the proposed shape maps cleanly onto protocol.go / providers.go / resolver.go / client.go, and surfacing the bedrock.WithConfig SSO bearer-token pitfall up front saves everyone a painful debugging session. This is a model scoping issue.

Dependency: going with option (1)

We'd like to go with option 1 (direct dependency). The usability cost of a build tag or a split module — a second build configuration, a release-artefact decision, and "install the plugin" friction for the one provider an SSO shop actually needs — outweighs the dependency weight for us. We can revisit if the aws-sdk-go-v2 footprint ever becomes a real problem, but the SDK is well-maintained and the modules you listed are the standard, expected set.

What we'd like nailed down before merge

The resolver/client changes are clear. The part we have less visibility on is how AmbientAuth ripples into the config file format and the config-related CLI interactions. ocr has an interactive Provider wizard (ocr config provider), a model picker (ocr config model), and a non-interactive ocr config set/unset — all of which currently assume an api_key. Before we commit to a shape, could you expand on:

1. Provider entry shape. With AmbientAuth, is api_key now forbidden, merely optional, or still accepted as an override? And where do aws_region / aws_profile live — new fields on providerEntryConfig (so they're JSON-serializable in providers.bedrock), or read purely from the AWS env chain? A sample config.json snippet for a working Bedrock entry would make this concrete.
2. Interactive wizard (ocr config provider). Today applyOfficialProviderConfig hard-gates on a non-empty api_key (or a set EnvVar). For an ambient provider the wizard presumably needs to skip the API-key step and instead collect region/profile. How do you envision that branch — a new wizard path keyed off AmbientAuth, and does it still run the connection test at the end?
3. Non-interactive config (ocr config set/unset). Do aws_region / aws_profile need to be settable via ocr config set provider.bedrock.aws_region …? That path goes through setProviderValue, so we'd want to know whether new keys need explicit allow-listing there.
4. Model list semantics. Bedrock model identifiers are region-scoped and inference-profile IDs are long (<base>_inference_profile... style). How should the bedrock preset's Models list read, and does ocr config model validation against that list still make sense, or does ambient/Bedrock need a free-form model entry (like custom providers)?
5. Connection test (ocr llm test) messaging. When aws sso login hasn't been run or the IAM role is missing, what does the user see? Worth agreeing on a helpful error here so it doesn't read like a generic 403.

You don't need to answer all five in the issue — flagging them is enough so we both know the surface area.

SSO bearer-token bug

Happy to take the "clear BearerAuthTokenProvider unless AWS_BEARER_TOKEN_BEDROCK is set" workaround in the ocr patch for now, and decide in the PR whether to also push a fix upstream to anthropic-sdk-go. Good catch either way.

Next step

Go ahead and open the PR (option 1, direct dependency) whenever it's ready — ideally with the sample config.json from (1) in the PR description. We'll work through the config/CLI interaction details above directly in the PR, which is a better venue than this issue for line-by-line review.

Thanks again — looking forward to this landing.

[cru-Luis-Rodriguez]

@stay-foolish-forever The PR is up at #705 — option 1, direct dependency, with the sample Bedrock config.json in the description as you asked. Happy to work the five config/CLI questions there rather than here.

One housekeeping ask: the five comments below your reply appear to be spam — generic emoji one-liners about RAG chunking, annotation specs, and cache invalidation, nothing to do with Bedrock auth. If you read them the same way, could you hide them as off-topic?