Platform Schema Representation Alternatives?

[giulioroggero]

I'm was wondering if the current schema representation of Platforms are too much related to Infrastructure and DevOps missing the opportunity to extend the self-service approach for Data, ML and other technical capabilities useful for product teams to develop applications in a seamless way.

The current picture in the whitepaper

is a good starting point but is still related to infra and devops.

///
Added Comment to this issue - April 15th 2024 - The link to original diagrams https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.p
///

What to you think about the following representations as an high level overview?

Or a more technical one like this one

Or this one?

Are just some ideas, I'd happy to elaborate more the idea.

[earth2marsh]

I'm interested in this topic! My area of expertise is around API platforms.

[giulioroggero]

I'm interested in this topic! My area of expertise is around API platforms.

@earth2marsh what do you think about the proposals?

[stephanedicesare]

I am interested in contributing as well. I have two comments:

• I like how Humanitec show resources as a standalone category. This makes clear that there are platform-level (abstracted) resources, which have a different interface than the non-abstracted ones.
• In my representation, I have a "common layer", which represents general principles / best practices, and pushes requirements to the other layers/planes. For example, regarding Observability, a self-service dashboard would be under resources, logging guidelines under the common layer, and the actual observability tooling under infrastructure.
  I'll be in touch when I have more time!

[earth2marsh]

@earth2marsh what do you think about the proposals?

While I like the attempt to find alternatives, my concern is that we haven't found the right framing. I'm not confident that that will happen in isolation—I suspect there are a number of different conversations that need to happen before we find a pattern that brings convergence.

I'm happy to ideate in a small or 1:1 brainstorming session if that's useful. If so, hit me up on the CNCF Slack?

[giulioroggero]

@earth2marsh what do you think about the proposals?

While I like the attempt to find alternatives, my concern is that we haven't found the right framing. I'm not confident that that will happen in isolation—I suspect there are a number of different conversations that need to happen before we find a pattern that brings convergence.

I'm happy to ideate in a small or 1:1 brainstorming session if that's useful. If so, hit me up on the CNCF Slack?

Good idea. We can organize a small brainstorming.

[stephanedicesare]

I like the view in the third diagram best, because it shows the different kind of users and what their entry points are.
I would propose the following changes:

• have DevOps platform and Infrastructure platform at the same level,
• have an overarching box for topics such as security, monitoring, identity, secrets: these topics apply to all boxes in the diagram – but they might have different implementations. For example, secrets might be managed differently in the infrastructure than they are in the applications,
• introduce a resources box at the same level than data fabric. This would include "curated" resources, which make some of the concerns transparent. For example, a database at the resource level could be cloud-agnostic and include company-specific tagging.

@giulioroggero If you send me the original diagram, I am happy to propose an updated version.

[giulioroggero]

@stephanedicesare here is the public link to the slide. https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.p

[abangser]

This is fantastic work team! I think it would be great to get eyes on it in the working group calls and possibly raise a time for a more deep dive as @earth2marsh suggested. Our next working group call is April 23rd and I know we have some time set aside for another topic, but could likely get 10-15 minutes on this if it makes sense to you?

I think we have an opportunity here to really make this about App Delivery (our parent TAG) and incorporate the platform concept as well as the app orchestration and infrastructure orchestration which are growing communities within the TAG and becoming WGs in their own rights (cc/ #588 and #589)

[RobertKelly]

@stephanedicesare here is the public link to the slide. https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.p

Looking great, BTW. Excited to see where it goes. Like the original, it will get a lot of milage helping paint a great picture.

I also made a comment and added a slide to show my thoughts on making team/innersource more front and center. I'm sure there is a better way, but I took a crack at it.

[giulioroggero]

@abangser I can participate the 23rd at 4pm UTC.

[giulioroggero]

I added to the file another representation that Highlight Product and Platform Teams better

https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.g2cc3e9189d1_0_0

[earth2marsh]

I wonder if it would help to describe specifically what it is we're hoping to communicate in a new graphic? The existing diagram (copied in this issue--the first image) is described in the whitepaper as illustrating:

the relationships between products, platforms, and capability providers.

I believe @giulioroggero 's initial initial criticism was that the diagram in the whitepaper is too specific to infra/devops and misses out on other technical capabilities like Data or ML. Perhaps this could be two complementary diagrams:

1. An overview of how platforms support an organization's software development life-cycl. This would be along the lines of what @krumware has proposed as another post: platform of platforms
2. An example of one specific kind of platform, such as an app delivery platform layer atop infrastructure tooling.

Once we have those, we could update the whitepaper to replace the current graphic and add language that introduces the platform-of-platforms concepts (which could get another name, too). Regardless, both of these topics might benefit from a brainstorming session.

[krumware]

Here are some other variations I use for planning (I grey out users/providers that are not yet served, and remove those that are not relevant for that platform):

[giulioroggero]

@krumware in the variation I see still DevOps and Infrastructure (may be it's my bias). What I suggest is to highlight Data/API/Event products that are reusable by other teams and on top a way to orchestrate that product to build applications.

It's like a three layer approach:

• infra + devops --> output is a set of self service tools that allow you to create API/Events/Data Products
• catalogs/marketplace --> output is the set of items crafted in the layer before and available to be composed by application builder
• data/api/event orchestrator/composer --> output is a set of self service tools that allow you to create application reusing API/Events/Data Products

The whole platform (or platform of platforms) is the valuable asset for a company to build application faster, reusing resources (all kind of resources) and guaranteeing quality because more time the resource is used more is becoming stable and tested

What do you think about?

[krumware]

Mine are just variations for the starting point, that aren't intended to be more detailed. (I agree that the language needs updating in the original, I just kept it because it is easier for the C level to compare against the published materials) Your additions are definitely relevant and helpful for showing more comprehensive design.

Worth noting, a marketplace, API catalog (for custom applications), data fabric, etc, can be unnecessary for many companies, so I'd take caution in including those out of the gate as they could be interpreted as mandatory as a starting point. The variations in this thread are great for demonstrating more comprehensive implementations.

I'd be curious to step through topology at different phases of maturity and see what that looks like as well.

[bryanrossUK]

I'm really interested in this topic, and would love to see the architectural representation take into account some of the non-technical aspects of Platform Engineering, such as team topologies and product mindset (#282). As a consumer, I'm not sure I would want to necessarily see project or vendor logos included - I think that perhaps belongs in lower level material that specifically talk about implementation.

[giulioroggero]

After the last meeting (2024-04-23 - https://docs.google.com/document/d/1_smeS9-j-SuHJi0VXjx4g9xiD2-tgqhnlwf5oSMDQgg/edit) I tried to merge all suggestions in one schema. Here is the result (slide 17 https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.g2cee77fe96d_3_1448).

I put together:

• the concept of Platform of Platforms (proposed by @krumware and discussed here [Community] Platforms of Platforms and methods of defining interaction layers between software products/platforms #542) naming it Platform Engineering ++
• the collaboration aspect highlighted by @RobertKelly
• the original schema representation keeping the concepts of Platform Interfaces and Platform Capabilities
• the suggestion of @stephanedicesare about the representation of DevOps and Infrastructure
• and other feedbacks from all participants in that conversation

The result is this one

The Platform is Composed by several Platforms:

• infrastructure: provides all capabilities to run workloads, store data and move information between workloads
• devops: provides the complete lifecycle of a resources (infrastructure, data, code, api, event, ml and more); the standardization of a resource lifecycle
• data: provides tools for data management with the pattern of Data Fabric and allowing a Data Mesh approach
• ML (or AI): the lifecycle management and tools for AI, ML, LLM
• composable: tools to compose resources ( (infrastructure, data, code, api, event, ml and more) to create Products and Applications
• marketplace: the internal place to reuse resources (infrastructure, applications, data/api products) and compose them; more time a resource is reused in different context inside a company more it becomes stable and valuable because is more tested. Composed resources can become also new items of the marketplace
• collaborations: people are working at different platform levels, product teams are working with platforms, all need collaboration tools and support

On top there are Product Teams that build Digital Products and Applications using in a self-service way the Platform of Platform thanks to Platform Interfaces.

Bottom are listed the Providers of Resources.

In details each Platform can provide different capabilities like the following ones

What do you think about? Feedbacks, proposals, questions are welcomed. You can edit the diagrams using that link copying it and improving https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.g2cee77fe96d_3_1540

[MichaelBK223]

Thank you for bringing your insights to this Working Group. To facilitate
discussion on how your work could be applied to the Platform Working Group's Platform Capability Guide, I created a mapping from your work to that Guide. My thought was that the Platform of Platforms perspective provides a way to deliver platform as a product. If I have mistaken the intent of your most appreciated work, I apologize. Best regards, Michael
Platform Working Group draft v3.pdf

[giulioroggero]

Hi @MichaelBK223, it's a great mapping! I try to clarify better my proposal to favorite the conversation and brainstorming (if the concepts that I'll describe below have been already discussed I'm sorry, please help me to navigate in the existing discussions).

I suggest to split the concept of Platform and Builders:

• Platform of Platforms is an ensemble of Platforms with a fluent DevX. In Platform Engineering ++ (Platform of Platforms ) all people involved in providing/curating/consuming capabilities are both users and providers of others. Is a place for collaboration. A platform of Technologies --> a platform of People. Thanks to that you can build your own Digital Platforms/Products/Applications for end users: a Business Platforms where end users (customers) are buying products, sharing business services, collaborate (like Airbnb, Uber, Facebook and all native digital platforms that are the target of not digital native industries that are evolving in the long digital transformation journey)
• Platform Builders is the set of tools that allow you to Compose your Platform as a Product using the Capabilities above of Platform of Platforms.

This is my opinionated way to describe the current terminology, please help me to better define the following:

• Business Digital Platform: the set of applications provided by a company and used by customers to satisfy their needs (some examples: Airbnb, Uber, Facebook, Home Baking Application, Transportation Application)
• Platform Foundation: a set of Composable Capabilities, organized in a Platform of Platforms, that are the basis for building the Internal Developer Platform for Company Teams (IT Ops, SRE, Cloud Engineers, Software Dev, Data Eng, ML Eng, API Managers, Business Analysts and all people involved in building and operating a Business Digital Platform)
• Platform Builder: the Composer that, starting from the Platform Foundation Capabilities, allow to build and compose your Internal Developer Platform with the aim of providing a self-service and governed DevX to Product Teams that are in charge to build the Business Digital Platform
• Internal Developer Platform: an instance of the Platform Foundation with the Capabilities needed for crafting and operation the Business Digital Platforms

Platform Engineering++ is the software engineering practices that focus on the above topics and journey and includes (not substitute) all good Software Engineering habits: eXtreme Programming (and in particular collective code owenership --> inner source/knowhow), Agile, DevOps, Clean Code, Cloud Native and more.
I proposed to add "++" because the current meaning of Platform Engineering is focusing on the first two capabilities of Platform of Platforms, Infrastructure and DevOps, but there are more elements to keep in mind to provide a seamless DevX for creating and operating a Business Digital Platform.

In the other way I'm agree that everything is a resources (eg: a bucket storage, an API definition, a micro frontend, a docker image ...) and the core Platform is managing the lifecycle of a resources keeping in mind the relationships between resources. But I'm worry that keeping this simple view as a basis in long term will provide more confusion because the resource type (infrastructure, devops, api ...) needs different competencies and skills for providing its "X as a product" for other people. The risk of not highlighting all aspects of a Platform of Platforms is that Platform would become a barrier of collaboration because is not see as an initiative of all people but "just" and IT Ops/DevOps initiative.

We have today the opportunity to create a platform that is inclusive by design for all people involved in software/digital, not only tech people.

My two cents :-)

[MichaelBK223]

Your insights are greatly appreciated. Thank you for doing so much for the platform initiatives. Having seen times when the barrier to 'X' was 'X', itself, I completely agree with your concern about such a limit being in place before 'X' has a chance to change things for the better.

[loujaybee]

In coming across this, I do agree the original diagram is very infra/devops related (but, I really like it, and have found it useful). However, I do also agree an ML or data oriented platform diagram could be very helpful. I'd suggest: to rename the issue and re-scope on that tighter topic.

[wimnat]

@giulioroggero love your diagram. I proposed some changes as an additional slide in the doc with my reasoning in the speaker comments.

However, it's probably better to continue that conversation here so posting this message as well.

[giulioroggero]

@loujaybee thanks for the suggestion. What name do you suggest for that issue?

[giulioroggero]

Thanks @wimnat for the slide 18 (https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.g2ddc2112a99_0_0).

What do you think about add this discussion and proposal in a PR for the Platforms White Paper? As suggested we can improve the original schema keeping it simple and add more "flavored" schemas for Data, ML, Composability and some end to end possibile use cases.

That question is for all participants to that conversation.

[MichaelBK223]

The contributions supporting this proposed evolution provide great confidence that the group can put us on a path not only to platform as a product (as already included) but to platform as code. In a platform of platforms model, each of the 'platforms' could have a future representation 'as code'. Organizations could integrate the 'as code platforms' that they needed. One other thought I had was that relative to infrastructure as a vertical was that maybe the infrastructure and security platforms would bracket the shape and the flavored schemas might be 'service platforms' to the overall structure. As always, thank you for asking for our inputs and congratulations on moving this conversation forward so eloquently.

[wimnat]

What do you think about add this discussion and proposal in a PR for the Platforms White Paper?

I guess i'd be keen to learn from the leaders of the group what they consider as the next iteration of the paper? The diagram and direction of "Platform of Platforms" I feel is significant enough to warrant a paper v1.1 or even v2. If so, probably best to define the scope of what that would look like before opening up a PR.

Is there a Platforms WG roadmap anywhere?

[abangser]

I guess i'd be keen to learn from the leaders of the group what they consider as the next iteration of the paper? The diagram and direction of "Platform of Platforms" I feel is significant enough to warrant a paper v1.1 or even v2. If so, probably best to define the scope of what that would look like before opening up a PR.

Thanks for raising this, and I would agree. Given the depth of discussion that went into the paper and the global nature of this group, we need to take into consideration the cost/value for any edits as it also requires translations etc.

I do not believe we have a timing right now for a second version, but we have had a number of questions, ideas, and opportunities that we would pull in when we do one.

Our next call is May 28 and it may be worth trying to collect the top concerns with the paper as a whole before then to try and evaluate if this is the right time to tackle a V2. If there is an appetite for this, it is best suited as a separate issue as we can then point to this as one addition. The project is likely 9-12 months of effort to review, plan, draft and publish so we will need the right level of needed edits and community engagement to make it realistic. (cc/ @roberthstrand, @krumware, @joshgav ).

[roberthstrand]

I do not believe we have a timing right now for a second version, but we have had a number of questions, ideas, and opportunities that we would pull in when we do one.

I agree. We have an ongoing project that we should focus on, with the platform as a product white paper. I have mentioned before that I think the next in line to do for the working group is to focus on developing a second version of the white paper, and while we might have discussions on what to bring into the next version, I think it is best that we wait until we feel that the ongoing work with the new white paper is moving along nicely.

Our next call is May 28 and it may be worth trying to collect the top concerns with the paper as a whole before then to try and evaluate if this is the right time to tackle a V2. If there is an appetite for this, it is best suited as a separate issue as we can then point to this as one addition.

We discussed this a bit in the previous meeting, but landing a sort of time frame for when we could kick off work on a v2 could be useful. I know that I would like to ensure I have the proper time to help with the v2.

[roberthstrand]

For the proposed change on the illustration of a platform, I think we could try and make an illustration that shows the overall structure with interface and capabilities, users and external providers. That as a concept makes a lot of sense to me, and I want that to continue to exist.

There are some great suggestions here, but I personally would like to see a HLD for these concepts in one illustration, then add complexity based on the type of platform one is trying to illustrate. It might just be me, but I think that would make it easier for more people to first understand the concept of a platform without crowding the picture.

Essentially, what I am suggesting is that we rewrite the "Capabilities of platforms" to be less opinionated on the types of capabilities, then add a section that deep dives a bit into these based on some of the platform archetypes that is the most common.

[giulioroggero]

Essentially, what I am suggesting is that we rewrite the "Capabilities of platforms" to be less opinionated on the types of capabilities, then add a section that deep dives a bit into these based on some of the platform archetypes that is the most common.

It sounds good!

[abangser]

One option is also to write a blog explaining the concerns with the current version and ideas on improvements. This is a lower cost (and lower barrier) way to start exploring alignment within and outside our group with updates before a full revisit.

Not sure if that's interesting?

[giulioroggero]

One option is also to write a blog explaining the concerns with the current version and ideas on improvements. This is a lower cost (and lower barrier) way to start exploring alignment within and outside our group with updates before a full revisit.

It's a good idea, I can try to write a first draft

[giulioroggero]

Starting from the proposal to simplify we can start from a really simple diagram like this one?

https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.g2dfd9102897_0_45

I promise that this is my last comment :-). I'll write a blog post to describe all

[MichaelBK223]

How would the Platform Engineering Maturity Model be used in relation to this? I can see how we could apply such a design to the platform architecture, but am less sure about how we could use it to support the Workgorup's Maturity Model. Thanks to all for the work that will be so helpful to so many.

…

On Fri, May 24, 2024 at 4:20 AM Giulio Roggero ***@***.***> wrote: Starting from the proposal to simplify we can start from a really simple diagram like this one? Screenshot.2024-05-24.at.10.17.48.png (view on web) <https://github.com/cncf/tag-app-delivery/assets/642404/06639ec1-e502-401b-b9dc-6bd21ab79240> https://docs.google.com/presentation/d/1BgiaEXVhBc4TtVownr19xgYuUQcEMNzxxH-w_uygMQU/edit#slide=id.g2dfd9102897_0_45 I promise that this is my last post :-). I'll write a blog post to describe all — Reply to this email directly, view it on GitHub <#586 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO5UJIXMVT2HSGVPY52XSJ3ZD3Z5DAVCNFSM6AAAAABE5H6ZT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRYHA4DONZTGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- *Michael Kestigian* *Technical Business Consultant* ***@***.*** ***@***.***>* *cell: 860-214-6197*

[giulioroggero]

As shared on Slack and during the last WG Platforms call I wrote a draft for a blog post that I'd like to post on CNCF TAG App Delivery blog - https://tag-app-delivery.cncf.io/blog/.

The 4th June 2024 at 5pm CEST has been scheduled a call to discuss the topic https://calendar.google.com/calendar/event?action=TEMPLATE&tmeid=NmI0NW40OTk5bHYzYWY1OTNmN2QyY2I3ODAgY185NjVkNzAzYWMyYzM3M2E4ZTkwZmNlMjhmNTNiYmY5OWRhZjhhMjcyNTc4MTA0NmVkZGE4MDUwMjgyYTMwZTQ5QGc&tmsrc=c_965d703ac2c373a8e90fce28f53bbf99daf8a2725781046edda8050282a30e49%40group.calendar.google.com

Here is the form for the blog post, after the review on google drive I will create a PR on the website

Contribution Description

In this blog post, I'll explore Platform Engineering, covering its diverse interpretations and implementations across organizations. While current Platform Engineering practices effectively address many aspects of simplifying people's lives, there are other areas that require attention. Data, ML, API, Security, Privacy, and others are crucial for a better Software Product Lifecycle. It is essential to consider whether these teams can benefit from the existing Platform Engineering practices. Without an holistic approach we run the risk of creating new barriers between Platform Engineers and Developers, similar to the barriers that existed in the past between IT Operations and Developers.

Related Working Group (WG)

Platforms

Contribution type

Blog

Why TAG App Delivery?

Because enterprises are looking to modernize their systems and the way of work. New paradigms are emerging and it's important learn from the past and avoid the mistake to create next barriers for collaboration.

Related projects/technologies

This blogpost is also related to the following content:

• the CNCF Platform White Paper (https://github.com/cncf/tag-app-delivery/blob/main/platforms-whitepaper/v1/index.md)
• the Platform Glossary (https://github.com/cncf/tag-app-delivery/blob/main/platforms-wg/glossary/_index.md)
• the Platform Engineering Maturity Model (https://github.com/cncf/tag-app-delivery/blob/main/platforms-maturity-model/v1/index.md)
• the Platform of Platform initiative ([Community] Platforms of Platforms and methods of defining interaction layers between software products/platforms #542).

Affiliation disclosure

This is an original post, vendor neutral, never published elsewhere. The aim of the post is to start a brainstorming about the topic of Platform Engineering and the future of this discipline.

Additional collaborators

The post is still ongoing, at the moment reviewers are: @loujaybee, @techmaharaj but feel free to add comments to the document

Draft

https://docs.google.com/document/d/1i3UgXDhJbDDMVNbb4jDcRE444uHYnqtH5HB-LajyDl4/edit

[giulioroggero]

PR opened #697

[giulioroggero]

The blog post about this topic is public! https://tag-app-delivery.cncf.io/blog/proposal-platform-engineering-/

I close the issue, thanks to everybody has contributed to the discussion, it has been really appreciated!