Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 66 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,6 @@ jobs:
- name: Export OpenAPI spec
run: python scripts/export_openapi.py

#- name: Set up Helm
# uses: azure/setup-helm@v4

#- name: Lint Helm chart
# run: helm lint ./helm/graphql-meter --set secret.jwtSecret=ci-chart-secret-1234567890

test:
runs-on: ubuntu-latest
needs: quality
Expand Down Expand Up @@ -121,21 +115,80 @@ jobs:
python-version: "3.12"
cache: pip

- name: Install dependencies (for lock generation)
- name: Install dependencies and generate lockfile
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"

- name: Generate pip freeze lock
run: pip freeze > requirements.lock
pip freeze > requirements.txt

- name: Run Trivy vulnerability scan (filesystem)
uses: aquasecurity/trivy-action@v0.35.0
with:
scan-type: fs
scan-ref: .
format: table
exit-code: "1"
severity: CRITICAL,HIGH
format: json
output: trivy-results.json
exit-code: "0" # Always return 0 here, let Python handle the failure logic
severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL # Tell Trivy to report ALL severities
vuln-type: library
ignore-unfixed: true
scanners: vuln
env:
TRIVY_DETECTION_PRIORITY: comprehensive

- name: Parse Results and Enforce Policy
run: |
python - <<'PY'
import json, sys
from collections import Counter

with open("trivy-results.json", encoding="utf-8") as f:
results = json.load(f).get("Results") or []

scanned = [i for i in results if i.get("Target") not in ("", "-") and i.get("Type") not in ("", "-")]
if not scanned:
print("ERROR: Trivy scanned zero supported targets.")
sys.exit(1)

severity_counts = Counter({'UNKNOWN': 0, 'LOW': 0, 'MEDIUM': 0, 'HIGH': 0, 'CRITICAL': 0})
high_crit_count = 0

print("========================================")
print(" DETAILED VULNERABILITY LOG ")
print("========================================")

for i in results:
target = i.get("Target", "Unknown Target")
vulns = i.get("Vulnerabilities") or []

if vulns:
print(f"\nTarget: {target}")
for v in vulns:
pkg = v.get("PkgName", "Unknown")
sev = v.get("Severity", "UNKNOWN")
vid = v.get("VulnerabilityID", "UNKNOWN")
title = v.get("Title", "No title provided")

# Increment counters
severity_counts[sev] += 1
if sev in {"HIGH", "CRITICAL"}:
high_crit_count += 1

print(f" [{sev.ljust(8)}] {pkg} ({vid}): {title}")

print("\n========================================")
print(" TOTAL DETECTION SUMMARY ")
print("========================================")
for sev in ['CRITICAL', 'HIGH', 'MEDIUM', 'LOW', 'UNKNOWN']:
print(f"{sev.ljust(10)}: {severity_counts[sev]}")

print("----------------------------------------")
print(f"Total Vulns : {sum(severity_counts.values())}")
print("========================================\n")

if high_crit_count > 0:
print(f"❌ ERROR: Pipeline failing due to {high_crit_count} HIGH/CRITICAL vulnerabilities.")
sys.exit(1)

print(f"✅ Success: No HIGH or CRITICAL vulnerabilities found. (Total scanned targets: {len(scanned)})")
PY
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -217,3 +217,7 @@ backend/data/
frontend/vendor/
learnings.md
.github/*.md
.python-version
notes/*
alembic/
alembic*
117 changes: 76 additions & 41 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,84 +1,124 @@
<p align="center">
<img src="assets/graphql-meter-readme.svg" alt="GraphQL Meter" width="480" />
</p>

# GraphQL Meter

**Schema-driven GraphQL performance testing — zero infrastructure, one container.**
**Schema-driven GraphQL performance testing, GraphQL load testing, and GraphQL client workflows in one container.**

[![CI](https://github.com/vanditsramblings/graphql-meter/actions/workflows/ci.yml/badge.svg)](https://github.com/vanditsramblings/graphql-meter/actions/workflows/ci.yml)
[![Release](https://img.shields.io/github/v/release/vanditsramblings/graphql-meter?color=green)](https://github.com/vanditsramblings/graphql-meter/releases)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)
[![Docker](https://img.shields.io/badge/docker-ghcr.io-0db7ed)](https://github.com/vanditsramblings/graphql-meter/pkgs/container/graphql-meter)

[Quick Start](#getting-started) · [Features](#features) · [Installation](#installation) · [Configuration](#configuration) · [Architecture](#architecture)
[Quick Start](#quick-start) · [Features](#features) · [Installation](#installation) · [Configuration](#configuration) · [Architecture](#architecture)

---

## What is GraphQL Meter?
## GraphQL Performance Testing

GraphQL Meter turns a GraphQL schema into runnable performance tests, live dashboards, and repeatable comparisons. It is built for teams that want GraphQL performance testing without writing custom scripts or stitching together separate tools.

GraphQL Meter is a self-hosted platform that transforms your GraphQL schema into fully configured performance tests. Paste a schema, select operations, set traffic distribution, and start load testing -- all from a single web interface. No YAML files, no test scripts to write, no external infrastructure to manage.
It ships as a **single container** with Locust, k6, a built-in GraphQL client, and run history for regression analysis.

| At a glance | Details |
|:---|:---|
| Primary use | GraphQL performance testing and GraphQL load testing |
| Secondary use | GraphQL API testing, schema discovery, and client verification |
| Delivery | Single container, no build step, no external orchestration |
| Engines | Locust and k6, isolated from the FastAPI process |
| Outputs | Live metrics, run comparisons, and trend views |

It ships as a **single container** with everything included: two load-testing engines (Locust and k6), a real-time dashboard, run history, trend analysis, and a built-in GraphQL client for pre-test verification.
## Why We Built It

**The problem it solves:** Load testing GraphQL APIs typically requires writing custom scripts, managing test data, configuring authentication, and stitching together multiple tools. GraphQL Meter eliminates this setup cost by auto-discovering operations from your schema and generating everything needed to run, monitor, and compare tests.
GraphQL teams usually need to solve the same problems before they can test at all:

<!-- TODO: Add hero screenshot of the dashboard -->
<!-- ![Dashboard](docs/screenshots/dashboard.png) -->
- Write custom load scripts for every API shape.
- Manually manage test data and request variables.
- Configure authentication, TLS, and client certificates.
- Switch between multiple tools for schema discovery, execution, and comparison.

GraphQL Meter removes that setup cost by auto-discovering operations from your schema and generating everything needed to run, monitor, and compare tests.

![Dashboard](assets/00-dashboard.png)

---

## Features

| Capability | What it does |
|:---|:---|
| **Schema-Driven Tests** | Paste schema → auto-discover ops → generate typed test data |
| **Dual Engine** | Locust (Python) + k6 (Go), subprocess-isolated, switchable per run |
| **Live Monitoring** | p50/p90/p95/p99 charts updating every 2 seconds |
| **Run Comparison** | Side-by-side delta with green/red regression highlights |
| **Environments** | TLS/mTLS, client certs, 6 auth provider types with Fernet encryption |
| **Built-in GQL Client** | Verify queries before running load tests |
| **Runtime Config** | Adjust limits, toggle engines, enable debug — no restart needed |
| Capability | What it covers | Why it matters |
|:---|:---|:---|
| **Schema-driven testing** | Discover queries and mutations from a GraphQL schema | Faster test setup with less manual wiring |
| **Dual load engines** | Locust and k6, switchable per run | Compare engine behavior on the same workload |
| **Live monitoring** | Throughput, error rate, and p50/p90/p95/p99 latency | See regressions while a test is running |
| **Run comparison** | Side-by-side delta analysis across runs | Spot performance drift quickly |
| **Environment profiles** | TLS/mTLS, certs, custom headers, and auth providers | Test realistic GraphQL API environments safely |
| **Built-in GraphQL client** | Query execution before load testing | Validate schema, auth, and responses first |
| **Runtime configuration** | Toggle limits and engines without restart | Adjust behavior from the UI when needed |

### Schema-Driven Test Generation
Paste a GraphQL schema and GraphQL Meter will parse it (via AST with regex fallback), discover all queries and mutations, and generate type-aware test variables with smart defaults. No hand-authored test scripts required.

<!-- TODO: Screenshot of schema parsing step -->
<!-- ![Schema Parsing](docs/screenshots/schema-parse.png) -->
Paste a GraphQL schema and GraphQL Meter will parse it, discover operations, and generate type-aware test variables with smart defaults. No hand-authored test scripts required.

![Schema Parsing](assets/04-schema-parsing.png)

The wizard automatically discovers operations and allows you to configure traffic distribution:

![Operations](assets/05-operations.png)

### Dual Engine Support
Choose between **Locust** (Python, greenlet-based) and **k6** (Go binary, scenarios-based) per test run. Both engines are subprocess-isolated from the main server -- they never share the FastAPI process. Compare results across engines to validate findings.

Choose between **Locust** and **k6** per test run. Both engines are subprocess-isolated from the main server, so they never share the FastAPI process. Compare results across engines to validate findings.

### Real-Time Monitoring
Watch throughput, response times (p50/p90/p95/p99), error rates, and per-operation breakdowns update live every 2 seconds. Interactive SVG charts with multi-series support render directly in the browser with no chart library dependency.

<!-- TODO: Screenshot of live test monitoring -->
<!-- ![Live Monitoring](docs/screenshots/live-test.png) -->
Watch throughput, response times, error rates, and per-operation breakdowns update live every 2 seconds. Interactive SVG charts render directly in the browser with no chart library dependency.

![Live Monitoring](assets/07-1-livemonitoring.png)

![Live Monitoring](assets/07-2-livemonitoring.png)

### Test Configuration Wizard
A 3-step wizard guides test setup: define global parameters, select operations with TPS percentage distribution (must sum to 100%), and review before starting. Saved configurations are reusable across runs.

A 3-step wizard guides test setup: define global parameters, select operations with TPS percentage distribution, and review before starting. Saved configurations are reusable across runs.

### Run Comparison and Trend Analysis
Compare any two runs side-by-side with delta highlighting (green = improved, red = regressed). View latency and throughput trends over the last N runs for any test configuration to catch regressions early.

<!-- TODO: Screenshot of comparison view -->
<!-- ![Compare](docs/screenshots/compare.png) -->
Compare any two runs side-by-side with delta highlighting. View per-operation deltas and overall metrics:

![Compare Runs](assets/09-comparison-view-data.png)

View latency and throughput trends over the last N runs for any test configuration to catch regressions early:

![Performance Trends](assets/10-trends-analysis.png)

### Environment Profiles
Define multiple target environments with distinct base URLs, TLS/mTLS settings, client certificates (PEM, PFX, cert+key), custom headers, and linked authentication providers. Switch between dev, staging, and production targets without reconfiguring tests.

Define multiple target environments with distinct base URLs, TLS/mTLS settings, client certificates, custom headers, and linked authentication providers. Switch between dev, staging, and production targets without reconfiguring tests:

![Environment Profiles](assets/11-environment-profiles.png)

### Encrypted Authentication
Six auth provider types: Bearer Token, Basic Auth, API Key, OAuth2 Client Credentials, OAuth2 Password, and Custom JWT. All secrets encrypted at rest with Fernet AES. Thread-safe token caching with automatic refresh for OAuth2 flows.

Six auth provider types: Bearer Token, Basic Auth, API Key, OAuth2 Client Credentials, OAuth2 Password, and Custom JWT. All secrets are encrypted at rest with Fernet. Token caching is thread-safe and automatically refreshes OAuth2 flows.

### Built-In GraphQL Client
Verify queries against your target API before running load tests. Split-pane editor with variables/headers panels, environment and auth provider resolution, saved requests, and import from test configurations.

Verify queries against your target API before running load tests. The split-pane editor supports variables, headers, environment resolution, auth provider resolution, saved requests, and import from test configurations:

![GraphQL Client](assets/12-graphql-client.png)

### Runtime Configuration
Adjust concurrency limits, enable/disable engines, toggle debug mode, and tune polling intervals from the Settings page without restarting the server. All changes take effect immediately for the current session.

### Dark Professional UI
Grafana/k6-inspired dark theme with CSS custom properties. No build step -- Preact + HTM served as vendored ES modules. Every page handles loading, empty, and error states.
Adjust concurrency limits, enable or disable engines, toggle debug mode, and tune polling intervals from the Settings page without restarting the server. Changes take effect immediately for the current session:

![Runtime Configuration](assets/13-runtime-configuration.png)


---

## Getting Started
## Quick Start

The fastest path to a running instance:

Expand Down Expand Up @@ -106,7 +146,6 @@ docker run -p 8899:8899 ghcr.io/vanditsramblings/graphql-meter:latest

# Run with custom configuration
docker run -p 8899:8899 \
-e JWT_SECRET=your-secret-key-here \
-e MAX_CONCURRENT_RUNS=5 \
-e ENABLE_K6=true \
-e ENABLE_LOCUST=true \
Expand Down Expand Up @@ -177,7 +216,6 @@ helm install graphql-meter ./helm/graphql-meter

# Install with custom values
helm install graphql-meter ./helm/graphql-meter \
--set secret.jwtSecret=my-production-secret \
--set persistence.size=5Gi \
--set resources.limits.memory=2Gi

Expand Down Expand Up @@ -260,8 +298,6 @@ All settings are controlled via environment variables or a `.env` file. Copy `.e

| Variable | Default | Description |
|:---|:---|:---|
| `JWT_SECRET` | (change me) | Secret key for JWT HS256 signing. **Must be changed in production.** |
| `JWT_EXPIRY_HOURS` | `24` | Token expiration time |
| `ENCRYPTION_KEY` | (auto) | Fernet key for encrypting auth provider secrets. Auto-derived from `JWT_SECRET` if empty. |

### Load Testing
Expand Down Expand Up @@ -301,14 +337,13 @@ All settings are controlled via environment variables or a `.env` file. Copy `.e

**Environment variables** (highest priority):
```bash
export JWT_SECRET=my-production-secret
export MAX_CONCURRENT_RUNS=5
graphql-meter
```

**Docker environment**:
```bash
docker run -e JWT_SECRET=my-secret -e ENABLE_K6=false -p 8899:8899 ghcr.io/vanditsramblings/graphql-meter
docker run -e ENABLE_K6=false -p 8899:8899 ghcr.io/vanditsramblings/graphql-meter
```

**.env file** (loaded automatically from working directory):
Expand Down
Binary file added assets/00-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/01-login.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/02-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/03-test-configs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/04-schema-parsing.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/05-operations.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/06-review-and-run.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/07-1-livemonitoring.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/07-2-livemonitoring.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/08-comparison-view.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/08-test-history.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/09-comparison-view-data.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/10-trends-analysis.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/11-environment-profiles.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/12-graphql-client.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/13-runtime-configuration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 28 additions & 0 deletions assets/graphql-meter-icon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
29 changes: 29 additions & 0 deletions assets/graphql-meter-readme.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion frontend/components/Sidebar.js
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ export function Sidebar({ currentPath }) {
return html`
<aside class="sidebar">
<div class="sidebar-logo">
<div class="logo-icon"><${Icon} name="gauge" size=${20} /></div>
<div class="logo-icon"><img src="/favicon.svg" alt="GraphQL Meter" width="28" height="28" /></div>
<h1>GraphQL Meter</h1>
</div>

Expand Down
Loading
Loading