Skip to content

Use new classic histogram conversion algorithm #1919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dricross wants to merge 35 commits into
base: main
Choose a base branch
from
Open

Use new classic histogram conversion algorithm #1919

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
4e59bea
Generate histogram test case inputs
dricross Sep 8, 2025
1ca64bb
Add expected values for each test case
dricross Sep 8, 2025
3dec2ed
Add expected min. Add visualization utility
dricross Sep 9, 2025
8a0a875
fixup 0/1 bucket test cases. add unbounded histogram case
dricross Sep 9, 2025
e21d8f1
Add more test cases, validation on more percentiles
dricross Sep 11, 2025
ff63673
Bring in exponential mapping function
dricross Sep 19, 2025
a361fbb
Update exponential algorithm. add accuracy tests
dricross Sep 23, 2025
41f6b42
New algorithm passes all unit tests. Added comparison tool
dricross Sep 26, 2025
0d02f75
documentation and testing updates
dricross Sep 29, 2025
afbf52b
some optimization for exponential mapping cw
dricross Sep 29, 2025
c2b07aa
replace math.Pow
dricross Sep 29, 2025
2143b12
spread remainder linearly
dricross Sep 29, 2025
40bbb2d
test lognormal/webiull for all algorithms
dricross Sep 29, 2025
3bb1cd7
Add cloudwatch tester program
dricross Sep 29, 2025
f7e9cde
Remove intermediate entries array
dricross Sep 29, 2025
056030e
adjust overflow safe average calc
dricross Sep 29, 2025
98e68db
Remove unnecessary logic
dricross Oct 1, 2025
da38150
Remove logarithm and replace thresholds
dricross Oct 1, 2025
b3272dd
Add some comments
dricross Oct 1, 2025
d3aa684
Update test data
dricross Oct 1, 2025
b7e95f2
change density detection. fix maximum value count
dricross Oct 6, 2025
39270c1
use histograms in otel
dricross Oct 13, 2025
fd72632
integrate with new conversion algorithm
dricross Oct 15, 2025
abac8cd
Drop invalid histograms
dricross Oct 16, 2025
65351b7
remove test data
dricross Oct 24, 2025
7744dce
Move histogram tests to contrib repo
dricross Oct 24, 2025
5d1e164
Moved code to contrib repo
dricross Oct 24, 2025
10e5b33
make fmt
dricross Oct 24, 2025
2d2f7b2
Update pkg/aws contrib reference
dricross Oct 24, 2025
838e05f
fixup unit tests
dricross Oct 27, 2025
f58d56d
Update contrib repo
dricross Oct 27, 2025
160c59e
Skip new conversion method for adapter receiver metrics
dricross Oct 28, 2025
48c0e45
drop adapter receiver attribute from otel dimensions
dricross Oct 28, 2025
8fdb780
Share dimension expansion logic. Remove unused var
dricross Dec 4, 2025
3c86421
Rebase
dricross Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 41 additions & 25 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ replace (
)

replace (
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/aws => github.com/amazon-contributing/opentelemetry-collector-contrib/pkg/aws v0.0.0-20251027190804-3b43f3b3b3ef
// For clear resource attributes after copy functionality https://github.com/amazon-contributing/opentelemetry-collector-contrib/pull/148
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/resourcetotelemetry => github.com/amazon-contributing/opentelemetry-collector-contrib/pkg/resourcetotelemetry v0.0.0-20251120035228-2492d12f5d81
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza => github.com/amazon-contributing/opentelemetry-collector-contrib/pkg/stanza v0.0.0-20251120035228-2492d12f5d81
Expand Down Expand Up @@ -99,12 +100,26 @@ replace github.com/openshift/api v3.9.0+incompatible => github.com/openshift/api
// forces version bump to support log group classes
replace github.com/aws/aws-sdk-go => github.com/aws/aws-sdk-go v1.48.6

// We are pinned to the following OTel versions:
// - OTel: v1.35.0
// - OTel Collector (experimental/stable): v0.124.1 / v1.30.0
replace (
go.opentelemetry.io/collector/featuregate => go.opentelemetry.io/collector/featuregate v1.30.0
go.opentelemetry.io/collector/service => go.opentelemetry.io/collector/service v0.124.0
go.opentelemetry.io/otel => go.opentelemetry.io/otel v1.35.0
go.opentelemetry.io/otel/metric => go.opentelemetry.io/otel/metric v1.35.0
go.opentelemetry.io/otel/sdk => go.opentelemetry.io/otel/sdk v1.35.0
go.opentelemetry.io/otel/sdk/metric => go.opentelemetry.io/otel/sdk/metric v1.35.0
go.opentelemetry.io/otel/trace => go.opentelemetry.io/otel/trace v1.35.0
)

require (
github.com/BurntSushi/toml v1.3.2
github.com/Jeffail/gabs v1.4.0
github.com/amazon-contributing/opentelemetry-collector-contrib/extension/awsmiddleware v0.124.0
github.com/aws/aws-sdk-go v1.55.7
github.com/aws/aws-sdk-go-v2 v1.36.3 // indirect
github.com/aws/aws-sdk-go-v2/config v1.29.13 // indirect
github.com/aws/aws-sdk-go-v2/service/cloudwatch v1.30.2
github.com/bigkevmcd/go-configparser v0.0.0-20200217161103-d137835d2579
github.com/deckarep/golang-set/v2 v2.3.1
Expand Down Expand Up @@ -138,6 +153,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/extension/sigv4authextension v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/extension/storage/filestorage v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/aws v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/resourcetotelemetry v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza v0.124.1
github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.124.1
Expand Down Expand Up @@ -176,7 +192,7 @@ require (
github.com/shirou/gopsutil v3.21.11+incompatible
github.com/shirou/gopsutil/v3 v3.24.5
github.com/shirou/gopsutil/v4 v4.25.3
github.com/stretchr/testify v1.10.0
github.com/stretchr/testify v1.11.1
github.com/xeipuuv/gojsonschema v1.2.0
go.opentelemetry.io/collector/client v1.30.0
go.opentelemetry.io/collector/component v1.30.0
Expand Down Expand Up @@ -223,10 +239,10 @@ require (
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.27.0
golang.org/x/exp v0.0.0-20250106191152-7588d65b2ba8
golang.org/x/net v0.39.0
golang.org/x/sync v0.13.0
golang.org/x/sys v0.32.0
golang.org/x/text v0.24.0
golang.org/x/net v0.46.0
golang.org/x/sync v0.17.0
golang.org/x/sys v0.37.0
golang.org/x/text v0.30.0
gopkg.in/fsnotify.v1 v1.4.7
gopkg.in/natefinch/lumberjack.v2 v2.0.0
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
Expand All @@ -241,7 +257,7 @@ require (
require (
cloud.google.com/go/auth v0.15.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.7 // indirect
cloud.google.com/go/compute/metadata v0.6.0 // indirect
cloud.google.com/go/compute/metadata v0.7.0 // indirect
collectd.org v0.4.0 // indirect
github.com/Azure/azure-sdk-for-go v67.1.0+incompatible // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.17.0 // indirect
Expand All @@ -253,7 +269,7 @@ require (
github.com/Azure/go-autorest/autorest/adal v0.9.23 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.3.3 // indirect
github.com/Code-Hex/go-generics-cache v1.5.1 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.27.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.29.0 // indirect
github.com/IBM/sarama v1.45.1 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Microsoft/hcsshim v0.12.9 // indirect
Expand All @@ -272,7 +288,6 @@ require (
github.com/armon/go-metrics v0.4.1 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aws/aws-msk-iam-sasl-signer-go v1.0.1 // indirect
github.com/aws/aws-sdk-go-v2/config v1.29.14 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.67 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.30 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.34 // indirect
Expand All @@ -298,7 +313,7 @@ require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cloudwego/base64x v0.1.4 // indirect
github.com/cloudwego/iasm v0.2.0 // indirect
github.com/cncf/xds/go v0.0.0-20250121191232-2f005788dc42 // indirect
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
github.com/containerd/cgroups/v3 v3.0.3 // indirect
github.com/containerd/containerd/api v1.8.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
Expand Down Expand Up @@ -336,7 +351,7 @@ require (
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/go-logfmt/logfmt v0.6.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
Expand Down Expand Up @@ -438,7 +453,7 @@ require (
github.com/moby/sys/mountinfo v0.7.2 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/mostynb/go-grpc-compression v1.2.3 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f // indirect
Expand Down Expand Up @@ -571,7 +586,7 @@ require (
go.opentelemetry.io/contrib/otelconf v0.15.0 // indirect
go.opentelemetry.io/contrib/propagators/b3 v1.35.0 // indirect
go.opentelemetry.io/contrib/zpages v0.60.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc v0.11.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp v0.11.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0 // indirect
Expand All @@ -584,27 +599,28 @@ require (
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.35.0 // indirect
go.opentelemetry.io/otel/log v0.11.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/sdk v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/sdk v1.37.0 // indirect
go.opentelemetry.io/otel/sdk/log v0.11.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
go.uber.org/zap/exp v0.3.0 // indirect
golang.org/x/arch v0.8.0 // indirect
golang.org/x/crypto v0.37.0 // indirect
golang.org/x/mod v0.24.0 // indirect
golang.org/x/oauth2 v0.28.0 // indirect
golang.org/x/term v0.31.0 // indirect
golang.org/x/crypto v0.43.0 // indirect
golang.org/x/mod v0.28.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/time v0.11.0 // indirect
golang.org/x/tools v0.32.0 // indirect
golang.org/x/tools v0.37.0 // indirect
golang.org/x/tools/godoc v0.1.0-deprecated // indirect
gonum.org/v1/gonum v0.16.0 // indirect
google.golang.org/api v0.226.0 // indirect
google.golang.org/genproto v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250218202821-56aae31c358a // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250428153025-10db94c68c34 // indirect
google.golang.org/grpc v1.72.0 // indirect
google.golang.org/protobuf v1.36.6 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250804133106-a7a43d27e69b // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20251007200510-49b9836ed3ff // indirect
google.golang.org/grpc v1.76.0 // indirect
google.golang.org/protobuf v1.36.10 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
Expand Down
Loading
Loading