research: what Kubernetes support would actually cost, and the marker decision everything waits on

[lex00]

Why this exists

Kubernetes support keeps being discussed as "choudoufu is AWS-only, so everything would have to be built". That is not what the tree says, and the gap is both smaller and differently shaped than that framing suggests. This issue records what is already there, what actually breaks, what a port would drop rather than reimplement, and the one decision everything else waits on.

Every claim below cites a file, a line or an issue. Anything unverified is marked so.

What already landed

#326 closed as fixed on 2026-08-21, not deferred. internal/live/identity/table_generated.go carries four ratified rows with NonAWSProvider: true: kubernetes_config_map, kubernetes_cluster_role_binding, kubernetes_namespace, kubernetes_storage_class. internal/live/identity/table.go documents the flag and names #326 as the first crossing of that line. internal/live/identity/kubernetesconfigmap_test.go pins the behaviour by value: ClassConcrete, ImportID == "default/my-config", and a refusal rather than a guessed "default" when metadata.namespace is absent.

So identity resolution and planning work for those four types today. What does not work is narrower, and the tree already says so in two places: all four appear in live/LIMITATIONS.md's untaggable residue, the set that is neither taggable nor parent-readable, and live/MARKERS.md lists them among the admitted types absent from the marker-governable grant table.

Property	Status for the four Kubernetes types
Resolve and plan	works
Carry a marker	no; markers.TagSurface finds no tags attribute
Reachable by the estate sweep	no
Governable by an ownership condition	no

The concrete consequence: delete a kubernetes_config_map block from source and the live ConfigMap is orphaned with no run that will ever propose removing it. That is a sweep gap, not an identity gap.

#326's closing comment also corrects its own body, and the correction matters for scoping: the blocker was not a marker carrier and not a discovery transport, it was that live/import-grammar.json declares a single AWS provider and there is no non-AWS import-grammar source. Its NAMESPACE/NAME grammar was read from provider docs rather than measured, and that remains unverified.

The decision everything waits on

Does the marker carry the resource address, or only the estate?

live/MARKERS.md explains why the address is there today: tofu-address answers which config block owns a resource, and the marker admission path rests on that value matching an address that exists in configuration. That path is ClassNeedsDiscovery, where the identity is server-assigned and recoverable only by marker discovery. It is an AWS property. AWS hands back opaque ids and generated names, so there is no natural key to join a live object to a configuration line, and the tag has to carry one.

Kubernetes returns the natural key. Group, version, kind, namespace and name, with the name authored in the configuration we already parse. kubernetes_config_map resolves ClassConcrete precisely because both components are operator-supplied. So the address does not need to be on the object; re-binding can go through the name.

This framing came from the chant session that owns packages/k8s-client, and the measurements support it strongly.

Measured, against internal/live/check/testdata/identity-golden.txt (1108 instances, 707 unique addresses). Escaping was reproduced in Python from markers.EscapeAddress rather than executed, so treat the counts as reproduced-not-run.

Quantity	Value
Escaped addresses over the 256-char AWS cap today	17
Of those 17, from the adversarial overlong-address fixture	17
Escaped ≤ 63 and a legal Kubernetes label value	333 (47%)
Escaped ≤ 63 but an illegal label value	342 (48%)
Escaped over 63	32
Unique addresses containing : after escaping	369

Two readings of that table. First, the continuation scheme (tofu-address-2 through tofu-address-4, MaxContinuations = 4) is headroom that has never bound on a real estate: every address needing it today is from a deliberately adversarial lint fixture. Second, at 63 characters it binds immediately and on ordinary shapes, and length is the smaller problem: 342 addresses are short enough and still illegal, mostly because of :, which markers.escapeAddressWith writes to open an instance key. markerkey's introducer + is also illegal in a label value. internal/live/markerkey does not carry over; its design target is the AWS tag charset, and markers.AddressMatches already reconciles three grammars.

Kubernetes limits, verified against k8s.io/apimachinery@v0.35.2: label value max 63 characters, grammar (([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?, annotations uncapped individually but 262144 bytes per object in total and not selectable server-side at all.

Options, in the order I would consider them.

1. Estate-only marker. A label carrying tofu-estate and nothing else; re-bind through group, kind, namespace and name. Estate names already validate as [a-z][a-z0-9-]{0,127}, so names up to 63 fit and longer ones become a new refusal. Cost: an object no longer says which config line owns it, so a reader holding only the object cannot answer that, and live-mv on Kubernetes becomes a no-op rather than a governed write.
2. Hashed label plus annotation. A digest for selection, the full address in an annotation for legibility. Cost: the selectable side stops being self-describing, which is most of the pitch, and a hash collision becomes a new class of mis-bind against the safety rule.
3. Port the escaping. A fourth grammar plus continuations at 63 characters, which on the measurement above means three or four continuation labels for ordinary module-nested addresses. MARKERS.md already documents that a split address breaks StringEquals conditions on the first chunk. Not recommended; the measurement is the argument.

Recommendation: option 1, with generateName refused rather than papered over.

What the port drops rather than reimplements

Two of the marker vocabulary's four mechanisms fall away, and both for the same reason.

Continuations, because an estate-only marker is short by construction.

tofu-slot. MARKERS.md writes a slot only when the configuration does not settle which live resource is which, and internal/live/slots/doc.go says such instances are interchangeable by construction. On Kubernetes they cannot be: the API server enforces uniqueness per resource, namespace and name, so a count-expanded set must carry distinct names or the second apply fails. The fungible-set case does not exist.

What genuinely breaks

The one-call sweep, and claim 14 with it. internal/live/discovery/tagging.go's sweepViaTagging replaces a per-type loop with a single GetResources call, filtered server-side on the marker, covering the whole admission table at once. Kubernetes has no cross-kind label-filtered list. A sweep there is discovery, then one list per kind per namespace: C + (K - C) x N calls for K kinds and N namespaces. Claim 14's headline survives in weakened form, because a label-selected list still returns only the estate's objects and does not grow with the cluster, but "one call" does not survive, and the claim would have to be restated per substrate. Its BREAK=1 control has no Kubernetes analogue, since both arms there are already kind-enumerated.

One consolation, and it is real: /apis enumerates every kind the cluster serves, CRDs included. tagging.go's ProblemUnsweepableOwnedType failure mode, where an admitted type outside the generated table is owned, orphaned and unreachable by any future run, cannot occur when the universe is asked rather than tabulated.

Claim 13's boundary is advisory. The claim rests on IAM conditions on aws:ResourceTag and aws:RequestTag, reaching 682 of 1027 admitted types. Kubernetes RBAC cannot express it: PolicyRule has exactly Verbs, APIGroups, Resources, ResourceNames and NonResourceURLs, with no selector, condition or attribute predicate. ResourceNames is a static allowlist of names.

The in-tree equivalent is ValidatingAdmissionPolicy, GA in admissionregistration/v1, which has what the job needs: CEL with object, oldObject, request and authorizer, and an ObjectSelector that is a label selector. oldObject is what fences a write by the label an object already carries, which is the aws:ResourceTag semantic. Two structural differences must be stated in the headline rather than a caveat:

• Admission covers CREATE, UPDATE, DELETE, CONNECT. There is no read or list admission, so a Kubernetes fence is write-only where an IAM condition can fence a describe.
• A policy is installed cluster-wide by a cluster admin. MARKERS.md makes AWS handover two IAM changes and no tag writes; the Kubernetes equivalent is a change to a shared cluster object with a different blast radius.

So: on AWS the marker is an authorization primitive out of the box. On Kubernetes the label is advisory until someone installs and maintains an admission policy. Kyverno and Gatekeeper are the out-of-tree alternatives; their exact capability surface is unverified here.

Deletion, which is where I would refuse rather than port. internal/live/policy/verb.go sets DefaultVerb[UndeclaredTagged] = Delete, on by default and needing no scope block because the marker is its own scope. Four hazards, in descending order of how much they should worry us:

1. A controller copies template labels. A Deployment's pod-template labels reach its ReplicaSets and Pods; a StatefulSet's volumeClaimTemplate labels reach its PVCs. If the estate marker lands in a pod template, every generated Pod and ReplicaSet carries it, is undeclared by construction, and becomes a delete candidate. HANDOFF.md calls a wrong marker silent, adopting or displacing a real object; a controller-propagated marker is a wrong marker nobody wrote. Whether chant is currently exposed to this is unverified; the Kubernetes property is not in doubt.
2. An owner still exists. Deleting a controller-created object triggers a recreate, indistinguishable in a plan from convergence failing.
3. A finalizer blocks. The delete returns success, sets deletionTimestamp, and the object stays. The next plan finds it still there and still marked, and the estate never converges. chant skips objects already terminating, which prevents a delete loop without making the object go away.
4. Propagation policy is unset, so the server default applies. Deleting a marked Namespace takes everything inside it, marked or not.

The state of the art next door has not settled this either: chant #1179 is open, and its owned-only prune has only ever run against a fake cluster.

What Kubernetes makes better

An issue that lists only breakage reads as a reason not to do this, and that would be wrong. Four things the model gets stronger on:

Field-level ownership. metadata.managedFields records which manager last wrote each path. MARKERS.md's "Protecting the markers" worries that anyone with tagging permissions can strip a marker with nothing recording who; on Kubernetes the API server names the manager that removed a label. policy/verb.go describes untag as "source owns it now" with no way to see whether source is winning; server-side apply makes that per-field and observable.

Server-side apply as a conflict primitive. A 409 names the competing manager and the contested paths. choudoufu's nearest equivalent is ProblemCollision, two live objects claiming one address; Kubernetes offers the finer version, two writers claiming one field, refused by the server, with force an explicit opt-in.

Server-side dry run. The API server validates, defaults and runs admission without persisting. AWS has nothing equivalent, and plan_approval already commits to being stricter than stock, so a server-validated plan is strictly stronger evidence than a locally computed one.

Runtime type discovery, as above.

The identity exceptions

Most Kubernetes objects land in the easiest class. Two do not, and both should be explicit limits rather than things the marker tries to solve.

generateName. The server mints the name, so the natural key is unknowable before the create. That is exactly ClassNeedsDiscovery, exactly the case that needs an address in the marker, and the one shape that would drag the whole address-in-label machinery back in. It appears zero times in chant's lexicon and client, so it is unhandled next door rather than solved. Whether hashicorp/kubernetes exposes metadata.generate_name at all is unverified and is one schema read; if it does not, the exception may be moot for the Terraform surface. Recommendation either way: refuse, consistent with how a missing namespace is already refused rather than guessed.

Controller-created objects. ReplicaSets, Pods, Jobs from CronJobs, PVCs from volumeClaimTemplates, EndpointSlices. The author chose neither the name nor the object. What they need is not identity but exclusion, most cheaply a non-empty metadata.ownerReferences test applied before anything reaches a delete quadrant.

Two more found while researching, not previously on any list:

• API version is not part of the key. Uniqueness is per group, resource, namespace and name; the version is a representation. An address change that only moves api_version is not a move, and MARKERS.md's rename rule has no vocabulary for that.
• Server-created singletons. The default ServiceAccount and kube-root-ca.crt exist in every namespace with no owner reference in some versions. Safe under today's Keep default, and a trap for anyone setting undeclared_untagged = "delete" with a namespace scope.

Proposed order

1. Correct the record on corpus-eks-basic: kubernetes_config_map has no marker carrier - a non-AWS provider, not a schema gap #326 wherever the "AWS-only, nothing exists" framing appears. The identity half landed; the residual gap is the sweep and LIMITATIONS.md already names it.
2. Decide the marker shape. Recommendation: estate-only, re-bind through the natural key, refuse generateName. Everything else waits on this.
3. Restate claim 14 per substrate. The one-call sweep is a property of the Resource Groups Tagging API and no engineering recovers it.
4. Restate claim 13 with the admission-control dependency in the headline.
5. Refuse undeclared_tagged = "delete" on Kubernetes until controller-created objects can be excluded by owner reference.
6. File the four wins in "What Kubernetes makes better" as their own units.

Credit

The reframe from "how do we fit an address into a label" to "is the name a sufficient join key" came from the chant session that owns packages/k8s-client, along with the ownership-label correction, the deletion warnings, and the controller-copies-labels hazard. chant's own answers are in packages/core/src/ownership.ts and lexicons/k8s/src/deep-observe.ts.

Refs #326, #243, and chant #1179.