prometheus aggregation gateway

This service accepts Prometheus metrics, validetes, aggregates, and exposes them ready for scraping in /metrics endpoint¹².

Features

• consume batch by JSON http body (aggregation gateway)
• consume one by URL Path or Query (hit counter)
• no dependencies
• 400LOC
• 100% coverage
• defensive validation
• histograms³ with fixed buckets
• histograms with t-digest⁴

Bring it to your own http server

func main() {
	var config pag.PromAggGatewayServerConfig
	configBytes, _ := os.ReadFile(os.Getenv("PAG_CONFIG_PATH"))
	if err := yaml.Unmarshal(configBytes, &config); err != nil {
		log.Fatal(err)
	}

	s := pag.NewPromAggGatewayServer(config)

	http.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK) })
	http.HandleFunc("GET /hit", s.ConsumeMetricFromURLQuery)
	http.HandleFunc("POST /metrics", s.ConsumeMetrics)
	http.HandleFunc("GET /metrics", s.GetMetrics)

	log.Fatal(http.ListenAndServe(":8080", nil))
}

ADR

• 2025-04-04 URL Query consumes only single metric for simple API (histograms break down into multiple metric names, hence not accepted in URL Query)
• 2024-12-16 not providing max, min aggregations, instead encouraging using histograms
• 2024-12-03 not provide avg based on total sum(x) / count(x), to reduce complexity in configuration, keep code flexible and small, and this can be done downstream anyways
• 2024-12-03 using _count in label naming to match open telemetry histograms
• 2024-12-03 using units in metric name, to keep closer to Prometheus and reduce complexity of API
• 2024-12-02 not using Prometheus Pushgateway⁵, because it does not aggregate metrics
• 2024-12-02 not zapier⁶ prom aggregation gateway, because: too many 3rd party dependencies (e.g. gin, cobra); no defensive validation;

Footnotes

1. https://prometheus.io/docs/practices/naming/ ↩

2. https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md ↩

3. https://prometheus.io/docs/practices/histograms/ ↩

4. https://github.com/ndx-technologies/tdigest ↩

5. https://github.com/prometheus/pushgateway ↩

6. https://github.com/zapier/prom-aggregation-gateway ↩