Move documentation to be user journey focused, not OTel concepts focused

[markmandel]

Hi all,

To set some context, I work on (amongst other things) Agones, which has used OpenCensus in Go for a number of years, with the primary use case being Metrics (we don't really do trace right now). We have the understanding that we'll need to eventually move to OTel if we want to have a long term supported observability library.

First I want to preface this by recognising all the awesome work you do for OTel, so apolgies up front for being critical, but please know it's with the intent of improvement.

A while back (April) I looked at how to do the migration from Open Census, and I got really stuck on the onboarding process (especially compared to what I was previously used to with Open Census documentation). I ended up chatting with a few people at Kubecon EU about it, and rightly so - they suggested I provide some feedback here! So here I am.

The TL;DR I found is that I found that Open Census's documentation is focused around what I want to do (i.e. do I want to export metrics? Do I want to log something, do I want to implement tracing?), whereas it seems to me like OTel is focused around how does OTel work (what are the concepts, what is a collector, what is an exporter?), which for onboarding creates quite a large barrier to entry -- as I need to understand OTel before I can do anything, rather than learning how to do something, and using that as the vehicle to learn the underlying concepts.

I wrote this friction log back in April, so hopefully it's still relevant (looking at the docs, still seems to be the case), but this was the struggle I was finding, and eventually gave up on.

Use case: I want to create metrics in Go (language doesn't actually matter, but it's what i know best), and export it to Google Cloud Monitoring (Also doesn't quite matter here either), and I've never used OTel before.

• I go to https://opentelemetry.io/ and click on Docs (https://opentelemetry.io/docs/)
• I read that page, and look for metrics, I see nothing in the navbar, but I do see "metrics" on the docs landing page at the top ("...such as traces, metrics, logs"), I'll click "metrics" then!
• I get a section on "Reliability & Metrics" - cool. I know what metrics are, but I need to do a thing, and this isn't explaining that yet. Okay, I see another link in that section on "metrics", let's see where that goes.
• That link goes to exactly where I am. Why is it a link? Well that's frustrating. Okay, now I'm on my own to work out my own adventure.
• Next stop, I see on the navbar there's a "getting started" section. Awesome! I love Quickstarts, that will get me going on my journey!
• I click "Getting Started and I get two options: "Dev" and "Ops". To be honest, I've no idea what this necessarily means, but I'm mostly sure I'm "dev", so let's go there.
• "This is the getting-started page for you if: You develop software." oh nice, I am in the right place! Awesome.
• I'm just trying to get something done, so I'm going to skip the "What is OpenTelemetry?, How can I instrument dependencies without touching their code?, How can I instrument my application manually?" - also it gives me a bunch of terms I'm not really ready to understand yet (exporters? propogators? resources? I've no idea what that is).
• "Next, you can deep dive into the documentations for the language you are using:" -- awesome - this is what I want. So I'm going to click "Go". I click that, nice, I see the status, etc. Let's go into "Getting Started".
• "Welcome to the OpenTelemetry for Go getting started guide! This guide will walk you through the basic steps in installing, instrumenting with, configuring, and exporting data from OpenTelemetry" - love it, that's what I want to do.
• Simple Fibonacci demo, fantastic.
• Got to Trace Instrumentation. Uh oh, not what I want to do. Skipping past it, scrolling down...
• I get into exporters, resources... but nothing on metrics. Uh oh. (If I dig through Java, same thing, I can check other languages but I expect the same).
• Okay, now I'm on my own again, and I don't know where to go. When in doubt -- search? (https://opentelemetry.io/search/?q=Metrics)
• Searching "Metrics" gets me to the top result of: https://opentelemetry.io/docs/concepts/signals/metrics/ - which.... is useful, but doesn't let me get anything done.
• Let's try searching again, and just clicking everything that comes up, because I'm out of ideas at this point.
• I end up at https://opentelemetry.io/docs/reference/specification/metrics/semantic_conventions/ and https://opentelemetry.io/docs/reference/specification/metrics/api/ and https://opentelemetry.io/docs/reference/specification/metrics/sdk/ seems to sort of help, but maybe I'll come back to this, since I see some concepts possibly explained?
• Okay, last idea! Let's see if there are examples I can look at! Let's go to https://github.com/open-telemetry
• I find https://github.com/open-telemetry, nice.. let's see if there is a Go repo...
• https://github.com/open-telemetry/opentelemetry-go good stuff, I see there is an example repository, we drop in there...
• Okay, so the examples are based on.... what product they integrate with, but not the end user case. prometheus is usually used for metrics, so let's see what's in there! (https://github.com/open-telemetry/opentelemetry-go/blob/main/example/prometheus/main.go)
• Oh hey! I got lucky! I see metrics!
• This gives me a starting point, but now I have to go search through the Otel docs and I assume the Go API reference for this code to work out what each bit is, and how it all works together, but now I'm tired, and I don't know if I want to. I'm tempted to just reach for platform specific tools because it's an easier onboarding path, and even if the docs aren't great there, it's probably a wider community, so I can find more resources.

To reiterate my hight level point: I found that the docs and onboarding process for Otel are written from the perspective of someone who knows what OTel is. It didn't seem to me to be written from the perspective of someone who has a user journey they are going on. This is where OC's docs where excellent -- they were user first, and that made them great for getting things done.

My strong suggestion would be to restructure the docs, into something that is close to what Open Census has:

• Quckstarts, broken down by use case (metric, trace, logging), per language
• A Tracing top level section
• A Metrics top level section
• A logging top level section
• A section on installation and operation
  etc etc.

Happy to discuss more, but wanted to pass the feedback on in hope that it's useful, and we can get more people (myself included!) onto OTel!

[svrnm]

Hey @markmandel, thank you very much for sharing your journey with our docs. While it hurts reading some of what you wrote, you are spot on and we are well aware of most of the issues you called out. Let me try to address some of them here, but before going into that let me put a few things upfront:

1. We had some discussions on "journey focused" documentation already, that's why you find the "Devs" and "Ops" Getting Started, it's a work in progress change that's mainly stuck because of the second point
2. The docs across languages are fairly inconsistent right now. That has some historic reasons of very big differences in stability across the spec and across those languages. With metrics & logs being stable now, we are at a tipping point and started to fix that.
3. While we discussed some major reworks of the docs in the past a few times, we always came to the conclusion that we will not do a big project redoing everything that's planned to take 3 months and takes 3 years, instead we do incremental change, this is SLOW but at the end will let to something
4. Finally, we (the docs team) are fairly limited in size, so we are eagerly hoping for people to help us out to improve our documentation (yes that's an implicit invite to help :-) although I guess you have your own hand full of work for Agones and Quilkin, etc)

Again, I appreciate your feedback, I just wanted to fill in the context, now on some of your individual points:

A while back (April) I looked at how to do the migration from Open Census

The migration guides (living in the spec) have been created in April and we announced sunsetting OpenCensus in May, so you might find some more help here: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/compatibility/opencensus.md

I go to https://opentelemetry.io/ and click on Docs (https://opentelemetry.io/docs/)

It looks like we need to make it more obvious that going through the "Getting Started" (for Devs & Ops) is the way to go if you want to take action

I get a section on "Reliability & Metrics" - cool. I know what metrics are, but I need to do a thing, and this isn't explaining that yet. Okay, I see another link in that section on "metrics", let's see where that goes.

Oh... that shouldn't happen.

I click "Getting Started and I get two options: "Dev" and "Ops". To be honest, I've no idea what this necessarily means, but I'm mostly sure I'm "dev", so let's go there.

We went back and forth with that and still didn't hit the nail apparently 😆 ... Looks like we need to rework that.

also it gives me a bunch of terms I'm not really ready to understand yet (exporters? propogators? resources? I've no idea what that is).

Looks like we need some concept pages for these as well.

From there on, let me comment on the Go docs: Metrics for go are in the process of becoming stable (see here for more details https://github.com/orgs/open-telemetry/projects/34 and check in with the go community). As outlined before this is a common pattern across the documentation ( I don't call it "problem" or "issue" on purpose, since it's just a matter of fact that OpenTelemetry is graduating and that means that some things are just not there yet)

See our docs also on that point: https://opentelemetry.io/docs/instrumentation/go/manual/#creating-metrics

To reiterate my hight level point: I found that the docs and onboarding process for Otel are written from the perspective of someone who knows what OTel is. It didn't seem to me to be written from the perspective of someone who has a user journey they are going on.

I do not disagree with that, only can point out the reasons as I have done above

This is where OC's docs where excellent -- they were user first, and that made them great for getting things done.
My strong suggestion would be to restructure the docs, into something that is close to what Open Census has:

For sure a place we can learn from (or even copy over some existing content)

Quckstarts, broken down by use case (metric, trace, logging), per language

The "Getting Started" / Quickstart is focused around automatic instrumentation right now (for those languages that have one), where you get all of that in one shot (if available). For languages which do not have automatic instrumentation (which go is an instance of, although there is some auto instrumentation in the works), we have no consistent idea yet, but it will be probably one guide having all signals (traces, logs, metrics) done together.

A Tracing top level section
A Metrics top level section
A logging top level section

Looking at the OpenCensus pages, this very closely looks like our concept pages on those topics (https://opentelemetry.io/docs/concepts/signals/) (minus SOME code examples in the OC docs, minus a bunch of docs that are just not there yet, e.g. for logs)

A section on installation and operation
etc etc.

Happy to discuss more, but wanted to pass the feedback on in hope that it's useful, and we can get more people (myself included!) onto OTel!

Your feedback is very useful. I reiterate that because I am very sure that some of my feedback on your feedback sounds defensive and not appreciative, so I apologies for that.

[cartermp]

Yeah, I think there's two distinct problems here:

1. Metrics for Go are still very new. They suffered a lot of stability issues for over a year and so we intentionally didn't document anything because stuff was breaking all the time. It was just an absolute slog to implement for the Go SIG and they got it done, but there's docs gaps now.
2. We don't have a good quickstart-like experience at the top level. There's sorta that on a per-language level, for some languages, but you also need to know to get in there.

I think (1) is much more directly solvable.

I think (2) could be tricker from an IA standpoint. Since we need to consider operator workflows there's no good single quickstart unless we divide it up by Developer/Operations. And even then, that stratification can be challenging because someone in an operations role may indeed also have to add instrumentation for a language.

[markmandel]

Thanks for the measured response and commentary!

Your feedback is very useful. I reiterate that because I am very sure that some of my feedback on your feedback sounds defensive and not appreciative, so I apologies for that.

Not at all! I think we're all on the same page here trying to get to a good place 👍🏻

While we discussed some major reworks of the docs in the past a few times, we always came to the conclusion that we will not do a big project redoing everything that's planned to take 3 months and takes 3 years, instead we do incremental change, this is SLOW but at the end will let to something

This totally makes sense, and I figured it was what was going to happen 👍🏻

Metrics for Go are still very new. They suffered a lot of stability issues for over a year and so we intentionally didn't document anything because stuff was breaking all the time. It was just an absolute slog to implement for the Go SIG and they got it done, but there's docs gaps now.

I totally get the impetus (and I've definitely done this myself! 😄 ) - what I feel would make things less generally confusing/frustrating (at least in the short term) as a new user is to be super clear on what is documented and isn't (particularly for each language) - ideally at a high level, to make choosing which navigation path to go down.

I hate to keep using OC here 😄 but if I look at the Quickstarts for different languages:

I can immediately see that C# doesn't have a quickstart for Metrics. While it may be ideal to have it long term, it short circuits any attempt to go into a Quickstart assuming that what I need might be there, since I know it's not.

This could be done in the OTel docs as a structure of the top level nav of the Quickstarts (where I think you might end up eventually anyway?) a change to the titles ("Getting Started with Tracing"), and/or maybe even a header on each quickstart stating which area is being covered.

So TL;DR: my recommendation -- just be super clear about what isn't documented as much as what is.

We don't have a good quickstart-like experience at the top level. There's sorta that on a per-language level, for some languages, but you also need to know to get in there.
I think (2) could be tricker from an IA standpoint. Since we need to consider operator workflows there's no good single quickstart unless we divide it up by Developer/Operations

This is where I would come back to structuring by "what do I want to do?" - as an end user, I've no idea if I'm dev or ops -- but I do know what I want to do. Is it create a trace? Deploy some kind of collector / draw pretty graphs traces etc" - that's far easier for me to navigate than anything else. And to agree with your point, the lines between dev and ops blur all the time 😄 it's too hard to know which side someone thinks they are on at any given moment.

[svrnm]

This could be done in the OTel docs as a structure of the top level nav of the Quickstarts (where I think you might end up eventually anyway?) a change to the titles ("Getting Started with Tracing"), and/or maybe even a header on each quickstart stating which area is being covered.

I don't think we can easily split out the "Getting Started" by Signal: for some languages you get all of them through the Getting Started in one shot (see Java for example). I also struggle with this because it would add yet another level into the documentation, bloating everything even more.

I see your point that we need to do a better job on showing what is available in which language, we can think about a few ways on improving here, although this is a temporary issue as we hope to have traces, metrics & logs with all major languages done eventually: there is no reason for a language to not implement them all.

This is where I would come back to structuring by "what do I want to do?" - as an end user, I've no idea if I'm dev or ops -- but I do know what I want to do. Is it create a trace? Deploy some kind of collector / draw pretty graphs traces etc" - that's far easier for me to navigate than anything else. And to agree with your point, the lines between dev and ops blur all the time 😄 it's too hard to know which side someone thinks they are on at any given moment.

We are already giving you a set of things you want to do:

• Automatic Instrumentation (all the signals for you for free)
• Manual Instrumentation (Create a Trace, create a Metric, create a log)
• Collector Documentation

Have a look at the Java Docs, which are ones of the most complete ones here:

https://opentelemetry.io/docs/instrumentation/java/manual/

The right site navigation bar gives you some of what you are looking for.

Yes, we could improve things by splitting out sections into pages, but I am currently reluctant to do that, since it adds a LOT more complexity to the docs while things are so much in a flux right now.