Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions backend/docs/BACKEND_CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,24 @@ it('starts empty', async () => {

> **Note:** `resetDb` requires a running PostgreSQL instance (see `make -C backend db-up`).
> Unit tests that mock the Prisma client do not need it.

## N+1 query guard (issue #1243)

Every list/pagination endpoint must have an N+1 guard. Use the
`assertConstantQueryCount` helper from `src/common/testing/queryCounter.ts`:

```ts
import { assertConstantQueryCount } from '../../src/common/testing/queryCounter.js';

it('fires a constant number of queries for any page size', async () => {
await assertConstantQueryCount(async (pageSize) => {
return myService.listItems({ page: 1, limit: pageSize });
});
});
```

The helper runs your function twice — once with page size 1, once with 50 — and
fails if the number of database queries changes. Add your test to
`tests/nPlusOne.test.ts` or alongside your service in a `*.test.ts` file.

See [`docs/N_PLUS_ONE_DETECTION.md`](./N_PLUS_ONE_DETECTION.md) for full documentation.
117 changes: 117 additions & 0 deletions backend/docs/N_PLUS_ONE_DETECTION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# N+1 Query Detection (Issue #1243)

This document describes the N+1 query detection system added to the Stellar-Tipz backend test suite.

## What is an N+1 Query?

An N+1 query problem occurs when an endpoint fetches a list of N items and then
fires an additional database query _for each item_ to hydrate associated data.
For example, fetching 50 profiles and then running 50 separate `findUnique` calls
for tip stats — 51 total queries — instead of a single batched `groupBy` — 1 query.

At scale, N+1 patterns cause:
- Exponential database load as page sizes grow.
- Increased API latency for list endpoints.
- Database connection pool exhaustion.

## Detection Helpers

All helpers live in `src/common/testing/queryCounter.ts` and are re-exported from
`tests/helpers/queryCounter.ts` for integration tests.

### `countQueries<T>(action)`

Runs an async action in a query-counting context and returns:

```ts
{ result: T, count: number, queries: CapturedQuery[] }
```

**Example**:
```ts
import { countQueries } from '../../src/common/testing/queryCounter.js';

const { count, queries } = await countQueries(() =>
profilesService.listProfiles(1, 20)
);
// Inspect which queries ran:
console.log(queries.map(q => `${q.model}.${q.operation}`));
```

### `assertConstantQueryCount<T>(runner, sizes?)`

The primary N+1 regression guard. Runs `runner(smallPageSize)` and
`runner(largePageSize)` and asserts the query count is identical.

```ts
import { assertConstantQueryCount } from '../../src/common/testing/queryCounter.js';

it('fires constant queries regardless of page size', async () => {
await assertConstantQueryCount(async (pageSize) => {
return analyticsService.getTopTippers(1, pageSize);
});
});
```

Default sizes are `[1, 50]`. Override for endpoints with smaller max limits:

```ts
await assertConstantQueryCount(runner, [1, 10]);
```

## How It Works

The helper uses Node.js `AsyncLocalStorage` to create a per-call context that is
transparently propagated through all async continuations — including `await` chains,
`Promise.all`, and `setTimeout`. A Prisma `$use` middleware registered on the
singleton client intercments the counter for every database operation in the active context.

```
assertConstantQueryCount(runner)
└── countQueries(runner(small)) ← ALS context #1
└── queryCounterMiddleware() ← increments ctx #1
└── countQueries(runner(large)) ← ALS context #2
└── queryCounterMiddleware() ← increments ctx #2
└── assert(count1 === count2)
```

Since the `AsyncLocalStorage` context is isolated per `storage.run()` call,
concurrent `countQueries` calls never interfere with each other.

## Registered Patterns (issue #1243 initial pass)

| Location | Pattern fixed |
|---|---|
| `profiles.service.ts` `listProfiles` | `users.map(u => getTipStats(u.id))` → single `tip.groupBy({ toAddress: { in: addresses } })` |
| `analytics.service.ts` `getTopTippers` | `grouped.map(row => user.findUnique(...))` → single `user.findMany({ stellarAddress: { in: addresses } })` |
| `analytics.service.ts` `getCreatorAnalytics` topTippers | `sortedTippers.map(([addr]) => user.findUnique(...))` → single `user.findMany({ stellarAddress: { in: tipperAddresses } })` |

## Adding Coverage for New Endpoints

Whenever you add or modify a list endpoint:

1. Add a `describe` block to `tests/nPlusOne.test.ts`.
2. Use `assertConstantQueryCount` with realistic sizes.
3. If the endpoint calls services that rely on the database, mock the Prisma
client or use the real one with a test database (see `vitest.setup.ts`).

```ts
describe('newModule.listItems — no N+1', () => {
it('fires a constant number of queries for any page size', async () => {
await assertConstantQueryCount(async (pageSize) => {
return newModuleService.listItems({ page: 1, limit: pageSize });
});
});
});
```

## Running the Tests

```bash
# Unit tests only (no database required)
cd backend
npm test -- --testPathPattern="queryCounter|nPlusOne"

# Full test suite
npm test
```
163 changes: 163 additions & 0 deletions backend/src/common/testing/queryCounter.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
/**
* N+1 query detection unit tests (issue #1243).
*/

import { describe, it, expect } from 'vitest';
import {
queryCounterMiddleware,
countQueries,
assertConstantQueryCount,
} from './queryCounter.js';
import type { Prisma } from '@prisma/client';

// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------

/** Creates a minimal Prisma-middleware `params` object. */
function makeParams(model = 'Tip', action = 'findMany'): Prisma.MiddlewareParams {
return {
model: model as Prisma.ModelName,
action: action as Prisma.PrismaAction,
args: {},
dataPath: [],
runInTransaction: false,
};
}

/** Runs the middleware in a simulated no-op way that calls next immediately. */
async function runMiddleware(params: Prisma.MiddlewareParams): Promise<void> {
await queryCounterMiddleware(params, async (p) => p);
}

// ---------------------------------------------------------------------------
// queryCounterMiddleware
// ---------------------------------------------------------------------------

describe('queryCounterMiddleware', () => {
it('passes through to next and returns its result', async () => {
const expected = { id: '1' };
const result = await queryCounterMiddleware(makeParams(), async () => expected);
expect(result).toBe(expected);
});

it('does not increment count when no countQueries context is active', async () => {
// No storage.run() wrapping this call — should be a no-op
await expect(runMiddleware(makeParams())).resolves.not.toThrow();
});
});

// ---------------------------------------------------------------------------
// countQueries
// ---------------------------------------------------------------------------

describe('countQueries', () => {
it('returns count 0 when no middleware queries are fired', async () => {
const { result, count, queries } = await countQueries(async () => 42);
expect(result).toBe(42);
expect(count).toBe(0);
expect(queries).toHaveLength(0);
});

it('counts queries fired via queryCounterMiddleware inside the action', async () => {
const { count, queries } = await countQueries(async () => {
await runMiddleware(makeParams('User', 'findUnique'));
await runMiddleware(makeParams('Tip', 'findMany'));
});

expect(count).toBe(2);
expect(queries[0]).toMatchObject({ model: 'User', operation: 'findUnique' });
expect(queries[1]).toMatchObject({ model: 'Tip', operation: 'findMany' });
});

it('isolates counts between concurrent countQueries calls', async () => {
const [a, b] = await Promise.all([
countQueries(async () => {
await runMiddleware(makeParams('User', 'findMany'));
return 'a';
}),
countQueries(async () => {
await runMiddleware(makeParams('Tip', 'count'));
await runMiddleware(makeParams('Tip', 'aggregate'));
return 'b';
}),
]);

expect(a.count).toBe(1);
expect(b.count).toBe(2);
expect(a.result).toBe('a');
expect(b.result).toBe('b');
});

it('exposes durationMs for each captured query', async () => {
const { queries } = await countQueries(async () => {
await runMiddleware(makeParams('Goal', 'create'));
});

expect(queries[0].durationMs).toBeGreaterThanOrEqual(0);
});
});

// ---------------------------------------------------------------------------
// assertConstantQueryCount
// ---------------------------------------------------------------------------

describe('assertConstantQueryCount', () => {
it('passes when query count is the same for both page sizes', async () => {
// Simulate a well-batched function: always 2 queries regardless of page size
await expect(
assertConstantQueryCount(async (pageSize) => {
await runMiddleware(makeParams('Tip', 'groupBy')); // query 1: aggregate
await runMiddleware(makeParams('User', 'findMany')); // query 2: batch hydrate
return Array.from({ length: pageSize }, (_, i) => i);
}),
).resolves.not.toThrow();
});

it('throws when query count grows with result-set size (N+1)', async () => {
// Simulate a broken function: 1 extra query per result item
await expect(
assertConstantQueryCount(
async (pageSize) => {
await runMiddleware(makeParams('Tip', 'findMany')); // base query
for (let i = 0; i < pageSize; i++) {
await runMiddleware(makeParams('User', 'findUnique')); // N+1 for each item
}
},
[1, 10] as [number, number], // use smaller sizes to keep the test fast
),
).rejects.toThrow(/N\+1 query detected/);
});

it('includes a helpful diagnostic message on failure', async () => {
let errorMessage = '';
try {
await assertConstantQueryCount(
async (pageSize) => {
for (let i = 0; i < pageSize; i++) {
await runMiddleware(makeParams('Tip', 'findUnique'));
}
},
[1, 3] as [number, number],
);
} catch (e) {
errorMessage = (e as Error).message;
}

expect(errorMessage).toContain('page_size=1');
expect(errorMessage).toContain('page_size=3');
expect(errorMessage).toContain('Tip.findUnique');
expect(errorMessage).toContain('batch');
});

it('accepts custom size pairs', async () => {
await expect(
assertConstantQueryCount(
async (_pageSize) => {
await runMiddleware(makeParams('User', 'count'));
},
[5, 25] as [number, number],
),
).resolves.not.toThrow();
});
});
Loading