add Kubernetes AI Conformance working group #8515

fedebongio · 2025-07-09T17:53:13Z

Which issue(s) this PR fixes:

This PR is a follow up on the email sent 2 weeks ago on WG creation, see https://groups.google.com/a/kubernetes.io/g/dev/c/u6I_mCRC4lE

cc @janetkuo

sigs.yaml

cblecker · 2025-07-09T18:16:06Z

/hold
for review

fedebongio · 2025-07-09T18:31:47Z

cc @mfahlandt

OWNERS_ALIASES

nimbinatus · 2025-07-09T19:58:57Z

sigs.yaml

+    efficiently run AI/ML workloads. \n"
+  charter_link: charter.md
+  stakeholder_sigs:
+  - Architecture


I believe this should list the same SIGs that are listed as stakeholders in your charter.

IMHO this should also have SIG testing given that a suite of tests is in scope.

cc @aojea @pohly

As a TL I am +1 to sponsor the WG.

Okay, so does everybody agree to list only: SIG Architecture and SIG Testing for now? Please let me know so I can make the change and get this merged asap to start meeting

+1 as SIG Testing TL.

+1 as SIG Arch Chair

janetkuo · 2025-07-09T21:13:13Z

cc SIG Arch chairs @derekwaynecarr @dims @johnbelamaric for approval

dims · 2025-07-09T22:54:53Z

@janetkuo let me paste @derekwaynecarr 's question from doc here

Derek Carr
Derek Carr
5:25 PM Jul 7
I apologize for being out of office during the meeting, but it looks like this proposal has evolved from an earlier form I had seen for networking conformance/benchmarking.

I want to understand more, but my general concern is that the document linked is larger in scope and its not clear how a single vendor may or may not be able to assert conformance depending on what it prereqs in its current form (particularly if it requires multiple vendors to reach a solution).  For example, the availability of a production DRA driver does not exist at this time from any GPU vendor (let alone a Kubernetes distribution).

Isn't this inverting the process?  Should a WG be formed to explore what should or should not be done (including if conformance is needed) and then allow vendors (hardware and software) to participate to appropriately determine if the requirements as written could be fulfilled?

franciscojavierarceo

I'd like to advocate for an explicit nomination process for the leads of AI Conformance.

wg-ai-conformance/charter.md

mrunalp · 2025-07-10T18:32:30Z

wg-ai-conformance/charter.md

+
+#### Code, Binaries and Services
+
+- The primary artifact will be the Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.


In which repository will this be maintained? Who will be the maintainers of the conformance specification?

This is an open question from the SIG arch meeting.

IIRC there was loose agreement to follow up in the WG to discuss options for how best to implement and host this and present them to the relevant groups (SIG Arch, SIG Testing, CNCF, ....).

In think this should be clarified, it is undetermined if there will be binary artifacts in the kubernetes organization. They may be in the CNCF, or the workgroup might determine that there's an alternate approach.

wg-ai-conformance/charter.md

andreyvelich

Thank you for this @fedebongio!

wg-ai-conformance/charter.md

Co-authored-by: Benjamin Elder <[email protected]>

Co-authored-by: Janet Kuo <[email protected]>

Fixing 80 char line correctly

fedebongio · 2025-07-11T19:28:35Z

/retest

fedebongio · 2025-07-14T16:25:26Z

@janetkuo let me paste @derekwaynecarr 's question from doc here

Derek Carr
Derek Carr
5:25 PM Jul 7
I apologize for being out of office during the meeting, but it looks like this proposal has evolved from an earlier form I had seen for networking conformance/benchmarking.

I want to understand more, but my general concern is that the document linked is larger in scope and its not clear how a single vendor may or may not be able to assert conformance depending on what it prereqs in its current form (particularly if it requires multiple vendors to reach a solution).  For example, the availability of a production DRA driver does not exist at this time from any GPU vendor (let alone a Kubernetes distribution).

Isn't this inverting the process?  Should a WG be formed to explore what should or should not be done (including if conformance is needed) and then allow vendors (hardware and software) to participate to appropriately determine if the requirements as written could be fulfilled?

Pointing here for consistency, I've replied in the doc: https://docs.google.com/document/d/1BlmHq5uPyBUDlppYqAAzslVbAO8hilgjqZUTaNXUhKM/edit?disco=AAABjeq6AqY

johnbelamaric · 2025-07-14T19:22:32Z

I am +1 for this with my SIG Arch hat.

andreyvelich · 2025-07-15T14:17:39Z

wg-ai-conformance/charter.md

+
+#### Code, Binaries and Services
+
+- The primary artifact will be the Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.


@fedebongio I guess, we should change this, right ?

Suggested change

- The primary artifact will be the Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.

- The primary artifact will be the CNCF Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.

We can only name things Kubernetes, for it to be CNCF, it would have to be blessed by CNCF bodies like TOC and TAGs

(that can happen later)

I think, conformance naming is done by the GB eventually based on CNCF Charter, sections 5(b)(iv), 5(d)(viii)

but agree with @dims this can happen and be changed later

sig-list.md

wg-ai-conformance/README.md

Fixing the time format (thank you!) Co-authored-by: Rita Zhang <[email protected]>

…nizing Chairs

k8s-ci-robot · 2025-07-17T07:06:44Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fedebongio, franciscojavierarceo, janetkuo, mfahlandt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mfahlandt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mfahlandt · 2025-07-17T07:10:11Z

I think we are good to go to move it to review for steering?

@kubernetes/steering-committee
/assign @kubernetes/steering-committee

aojea · 2025-07-18T16:27:51Z

wg-ai-conformance/charter.md

+- The primary artifact will be the (working title for now) "CNCF Kubernetes AI Conformance" specification and a suite of tests to demonstrate conformance.
+
+#### Cross-cutting and Externally Facing Processes
+
+- The Working Group will consider its primary problem-solving objective complete upon the successful definition and initial adoption of a stable (working title for now) "CNCF Kubernetes AI Conformance" specification.
+- Once the foundational conformance is established and widely recognized, the ongoing maintenance and evolution of the conformance would be evaluated, and could ideally transition to an existing or newly formed Special Interest Group (SIG) with a long-term charter, at which point the Working Group would dissolve.


I have certain doubts with this paragraph

The WG definition in https://github.com/kubernetes/community/blob/master/committee-steering/governance/wg-governance.md#is-it-a-working-group-yes-if
indicates clearly that a WG

It does not own any code

A suite of tests is code, and there is also references to

and could ideally transition to an existing or newly formed Special Interest Group (SIG) with a long-term charter

So, if this code is in kubernetes it has to be owned by a SIG before, as "transition" implies it was owned by someone before, and a WG can not be the owner as indicated before, we need to clarify this.

The reference to "newly formed Special Interest Group (SIG)" is tendentious and irrelevant to the work and scope of this WG, I do not think it should be here.

aojea · 2025-07-18T18:39:05Z

wg-ai-conformance/charter.md

+#### Cross-cutting and Externally Facing Processes
+
+- The Working Group will consider its primary problem-solving objective complete upon the successful definition and initial adoption of a stable (working title for now) "CNCF Kubernetes AI Conformance" specification.
+- Once the foundational conformance is established and widely recognized, the ongoing maintenance and evolution of the conformance would be evaluated, and could ideally transition to an existing or newly formed Special Interest Group (SIG) with a long-term charter, at which point the Working Group would dissolve.


Suggested change

- Once the foundational conformance is established and widely recognized, the ongoing maintenance and evolution of the conformance would be evaluated, and could ideally transition to an existing or newly formed Special Interest Group (SIG) with a long-term charter, at which point the Working Group would dissolve.

- Once the foundational conformance is established and widely recognized, the ongoing maintenance and evolution of the conformance would be evaluated, and could ideally transition to a Special Interest Group (SIG) with a long-term charter, at which point the Working Group would dissolve.

fedebongio added 2 commits July 9, 2025 10:43

feat: add Kubernetes AI Conformance working group

3ed496b

feat: add Kubernetes AI Conformance working group

e35c899

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. committee/steering Denotes an issue or PR intended to be handled by the steering committee. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. labels Jul 9, 2025

k8s-ci-robot requested a review from derekwaynecarr July 9, 2025 17:53

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 9, 2025

k8s-ci-robot requested a review from dims July 9, 2025 17:53

janetkuo approved these changes Jul 9, 2025

View reviewed changes

sigs.yaml Outdated Show resolved Hide resolved

sigs.yaml Outdated Show resolved Hide resolved

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 9, 2025

franciscojavierarceo reviewed Jul 9, 2025

View reviewed changes

OWNERS_ALIASES Show resolved Hide resolved

nimbinatus reviewed Jul 9, 2025

View reviewed changes

franciscojavierarceo suggested changes Jul 10, 2025

View reviewed changes

mfahlandt requested changes Jul 10, 2025

View reviewed changes

wg-ai-conformance/charter.md Show resolved Hide resolved

franciscojavierarceo approved these changes Jul 10, 2025

View reviewed changes

mrunalp reviewed Jul 10, 2025

View reviewed changes

BenTheElder reviewed Jul 10, 2025

View reviewed changes

wg-ai-conformance/charter.md Outdated Show resolved Hide resolved

andreyvelich reviewed Jul 10, 2025

View reviewed changes

wg-ai-conformance/charter.md Outdated Show resolved Hide resolved

fedebongio and others added 3 commits July 10, 2025 16:55

Update wg-ai-conformance/charter.md

00be35b

Co-authored-by: Benjamin Elder <[email protected]>

Update sigs.yaml

5c82784

Co-authored-by: Janet Kuo <[email protected]>

Update sigs.yaml

65f7742

Co-authored-by: Janet Kuo <[email protected]>

pacoxu mentioned this pull request Jul 11, 2025

CNCF/Kubernetes launched AI Conformance pacoxu/AI-Infra#6

Open

fedebongio added 2 commits July 11, 2025 11:05

Update sigs.yaml

e051457

Fixing 80 char line correctly

Update sigs.yaml

de92136

regenerating after small changes and recommitting

c509fa4

fedebongio added 2 commits July 14, 2025 19:56

Removing long list of SIGs

485c874

Adding CNCF to the Charterwq

dc732ff

andreyvelich reviewed Jul 15, 2025

View reviewed changes

ritazh reviewed Jul 15, 2025

View reviewed changes

sig-list.md Outdated Show resolved Hide resolved

ritazh reviewed Jul 15, 2025

View reviewed changes

wg-ai-conformance/README.md Show resolved Hide resolved

fedebongio and others added 4 commits July 15, 2025 09:35

Update wg-ai-conformance/README.md

de0879b

Fixing the time format (thank you!) Co-authored-by: Rita Zhang <[email protected]>

Several minor fixes

bd87281

merging locally

1337cdf

generating one more time

68cf397

k8s-ci-robot added the sig/testing Categorizes an issue or PR as relevant to SIG Testing. label Jul 15, 2025

fedebongio requested a review from mfahlandt July 15, 2025 17:08

Adding representatives from Microsoft and Redhat to the group of Orga…

f9bd39f

…nizing Chairs

mfahlandt approved these changes Jul 17, 2025

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 17, 2025

aojea reviewed Jul 18, 2025

View reviewed changes

mfahlandt mentioned this pull request Jul 18, 2025

REQUEST: New membership for mattfarina kubernetes/org#5710

Open

11 tasks


		#### Code, Binaries and Services

		- The primary artifact will be the Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.

	- The primary artifact will be the Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.
	- The primary artifact will be the CNCF Kubernetes AI Conformance specification and a suite of tests to demonstrate conformance.

	- Once the foundational conformance is established and widely recognized, the ongoing maintenance and evolution of the conformance would be evaluated, and could ideally transition to an existing or newly formed Special Interest Group (SIG) with a long-term charter, at which point the Working Group would dissolve.
	- Once the foundational conformance is established and widely recognized, the ongoing maintenance and evolution of the conformance would be evaluated, and could ideally transition to a Special Interest Group (SIG) with a long-term charter, at which point the Working Group would dissolve.

add Kubernetes AI Conformance working group #8515

Are you sure you want to change the base?

add Kubernetes AI Conformance working group #8515

Conversation

fedebongio commented Jul 9, 2025

Uh oh!

Uh oh!

Uh oh!

cblecker commented Jul 9, 2025

Uh oh!

fedebongio commented Jul 9, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BenTheElder Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janetkuo commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dims commented Jul 9, 2025

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andreyvelich left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fedebongio commented Jul 11, 2025

Uh oh!

fedebongio commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

johnbelamaric commented Jul 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mfahlandt Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

k8s-ci-robot commented Jul 17, 2025

Uh oh!

mfahlandt commented Jul 17, 2025

Uh oh!

aojea Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

BenTheElder Jul 10, 2025 •

edited

Loading

janetkuo commented Jul 9, 2025 •

edited

Loading

fedebongio commented Jul 14, 2025 •

edited

Loading

mfahlandt Jul 15, 2025 •

edited

Loading

aojea Jul 18, 2025 •

edited

Loading