Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 36 additions & 11 deletions src/output/sbom-dependency-edges.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { loadNpmLockGraph } from "../parsers/npm-lock-graph.js";
import { loadPnpmLockGraph } from "../parsers/pnpm-lock-graph.js";
import type { PackageRef, ScanInput } from "../types.js";

/**
Expand All @@ -24,29 +25,53 @@ function addEdge(edges: DependencyEdge[], seen: Set<string>, child: string, pare
}

/**
* Resolves dependency edges from the npm lock graph.
* Resolves dependency edges from the lockfile's dependency graph.
*
* The graph keeps a complete child-to-parents map (`parentsFor`) that is not
* subject to the five-path cap applied to `PackageRef.paths`. Reconstructing
* edges from those truncated paths loses roughly a quarter of the routes on a
* large tree, and a package whose surviving routes all traverse filtered-out
* packages ends up with no parent at all.
*
* Returns an empty list for anything other than an npm lockfile, so pnpm, Yarn
* and Bun fall back to the caller's path-derived behaviour rather than silently
* losing every edge.
* Returns an empty list for lockfiles with no graph implementation yet (Yarn
* and Bun), so those fall back to the caller's path-derived behaviour rather
* than silently losing every edge.
*/
export function resolveDependencyEdges(scanInput: ScanInput, allPackages: PackageRef[]): DependencyEdge[] {
if (scanInput.source !== "package-lock" || !scanInput.filePath) return [];
/**
* The slice of a lockfile graph the edge builder needs. npm and pnpm build
* their graphs very differently, but both can answer these three questions,
* which is all it takes to reconstruct the dependency relationships.
*/
type EdgeGraph = {
nodeIdsFor(name: string, version: string | null): readonly string[];
parentsFor(nodeId: string): readonly string[];
getNode(nodeId: string): Readonly<{ name: string; version: string | null }> | null;
};

let graph: ReturnType<typeof loadNpmLockGraph>;
/**
* Returns the lock graph for lockfiles that have one, or null for everything
* else so the caller falls back to path-derived edges rather than emitting an
* SBOM with no dependency relationships at all.
*/
function loadEdgeGraph(scanInput: ScanInput): EdgeGraph | null {
if (!scanInput.filePath) return null;
try {
graph = loadNpmLockGraph(scanInput.filePath, { includePaths: false });
if (scanInput.source === "package-lock") {
return loadNpmLockGraph(scanInput.filePath, { includePaths: false });
}
if (scanInput.source === "pnpm-lock") {
return loadPnpmLockGraph(scanInput.filePath);
}
} catch {
// A lockfile we cannot read is not worth failing an SBOM over. The caller
// falls back to path-derived edges.
return [];
// A lockfile we cannot read is not worth failing an SBOM over.
return null;
}
return null;
}

export function resolveDependencyEdges(scanInput: ScanInput, allPackages: PackageRef[]): DependencyEdge[] {
const graph = loadEdgeGraph(scanInput);
if (!graph) return [];

// Only packages present in the document can be referenced. `--prod-only` and
// similar filters shrink this set without shrinking the graph, and an edge to
Expand Down
120 changes: 120 additions & 0 deletions src/parsers/pnpm-lock-graph.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
import {
readAndParsePnpmLock,
parsePnpmPackageKey,
parsePnpmPackageKeyV9,
normalizePnpmDepRef,
normalizePnpmDepRefV9,
} from "./pnpm-lock.js";

/**
* A node in the pnpm dependency graph, identified by its lockfile key.
*
* pnpm encodes peer dependencies into the key (`vite@5.0.0(react@19.0.0)`), so
* several raw keys can describe the same `name@version`. Keys are cleaned of
* that suffix, which is why `nodeIdsFor` returns a list rather than one id.
*/
export type PnpmGraphNode = { name: string; version: string };

export type PnpmLockGraph = {
nodeIdsFor(name: string, version: string | null): readonly string[];
parentsFor(nodeId: string): readonly string[];
getNode(nodeId: string): Readonly<PnpmGraphNode> | null;
};

const EMPTY: readonly string[] = Object.freeze([]);

type KeyParser = (key: string) => { key: string; name: string; version: string } | null;
type RefNormalizer = (depName: string, depRef: unknown) => string | null;

function packageKey(name: string, version: string | null): string {
return `${name}@${version ?? ""}`;
}

/**
* Reads the child keys declared by one lockfile entry. pnpm splits runtime
* dependencies across `dependencies` and `optionalDependencies`; both are real
* edges in the installed tree, so both count.
*/
function childKeysOf(meta: any, normalize: RefNormalizer): string[] {
const children = new Set<string>();
for (const depMap of [meta?.dependencies, meta?.optionalDependencies]) {
if (!depMap || typeof depMap !== "object") continue;
for (const [depName, depRef] of Object.entries<any>(depMap)) {
const resolved = normalize(String(depName), depRef);
if (resolved) children.add(resolved);
}
}
return [...children];
}

/**
* Builds the complete child-to-parents map for a pnpm lockfile.
*
* The parser already walks this same structure, but `collectPnpmPaths` caps it
* at five paths of at most ten segments per package. Reconstructing edges from
* those truncated paths loses routes on any sizeable tree, and a package whose
* surviving routes all traverse filtered-out packages ends up with no parent at
* all. This keeps every edge.
*
* Returns an empty graph for a lockfile it cannot read, so the caller falls
* back to path-derived behaviour rather than emitting an SBOM with no edges.
*/
export function loadPnpmLockGraph(filePath: string): PnpmLockGraph {
const parsed = readAndParsePnpmLock(filePath);

// pnpm v9 moved the dependency graph out of `packages` and into `snapshots`;
// `packages` there carries only resolution metadata and declares no edges.
const isV9 = !!parsed?.snapshots;
const section = isV9 ? parsed.snapshots : parsed?.packages;
const parseKey: KeyParser = isV9 ? parsePnpmPackageKeyV9 : parsePnpmPackageKey;
const normalize: RefNormalizer = isV9 ? normalizePnpmDepRefV9 : normalizePnpmDepRef;

const nodesById = new Map<string, PnpmGraphNode>();
const nodeIdsByPackageKey = new Map<string, string[]>();
const parentsByChild = new Map<string, Set<string>>();

const register = (rawKey: string): string | null => {
const ref = parseKey(String(rawKey));
if (!ref) return null;
if (!nodesById.has(ref.key)) {
nodesById.set(ref.key, { name: ref.name, version: ref.version });
const pkgKey = packageKey(ref.name, ref.version);
const ids = nodeIdsByPackageKey.get(pkgKey);
if (ids) ids.push(ref.key);
else nodeIdsByPackageKey.set(pkgKey, [ref.key]);
}
return ref.key;
};

for (const [rawKey, meta] of Object.entries<any>(section ?? {})) {
const parentId = register(rawKey);
if (!parentId) continue;

for (const rawChild of childKeysOf(meta, normalize)) {
const childId = register(rawChild);
// A dependency that resolves to nothing in this lockfile, a workspace
// link or an unresolved optional, is not a node and cannot carry an edge.
if (!childId || childId === parentId) continue;
const parents = parentsByChild.get(childId);
if (parents) parents.add(parentId);
else parentsByChild.set(childId, new Set([parentId]));
}
}

const frozenParents = new Map<string, readonly string[]>();
for (const [child, parents] of parentsByChild) {
frozenParents.set(child, Object.freeze([...parents]));
}

return {
nodeIdsFor(name: string, version: string | null): readonly string[] {
return nodeIdsByPackageKey.get(packageKey(name, version)) ?? EMPTY;
},
parentsFor(nodeId: string): readonly string[] {
return frozenParents.get(nodeId) ?? EMPTY;
},
getNode(nodeId: string): Readonly<PnpmGraphNode> | null {
return nodesById.get(nodeId) ?? null;
},
};
}
10 changes: 5 additions & 5 deletions src/parsers/pnpm-lock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ function hasProjectImporterSections(document: Record<string, unknown>): boolean
// the lockfile, so a re-scan in the same process always sees the fresh contents.
const parsedLockCache = new Map<string, unknown>();

function readAndParsePnpmLock(filePath: string): any {
export function readAndParsePnpmLock(filePath: string): any {
const stat = fs.statSync(filePath);
const cacheKey = `${filePath}::${stat.mtimeMs}::${stat.size}`;
if (parsedLockCache.has(cacheKey)) return parsedLockCache.get(cacheKey);
Expand Down Expand Up @@ -307,15 +307,15 @@ function pathsEqual(left: string[], right: string[]): boolean {
return left.every((value, index) => value === right[index]);
}

function parsePnpmPackageKey(key: string): { key: string; name: string; version: string } | null {
export function parsePnpmPackageKey(key: string): { key: string; name: string; version: string } | null {
const cleaned = key.replace(/^\//, "").split("(")[0];
const match = cleaned.match(/^(@?[^/]+(?:\/[^/]+)?)\/([^/]+)$/);
if (!match) return null;
const [, name, version] = match;
return { key, name, version };
}

function parsePnpmPackageKeyV9(key: string): { key: string; name: string; version: string } | null {
export function parsePnpmPackageKeyV9(key: string): { key: string; name: string; version: string } | null {
// strip leading slash (pnpm v9 can emit /@babel/core@7.20.0) then peer-dep suffix (pkg@1.0.0(peer@2.0.0))
const cleaned = key.replace(/^\//, "").split("(")[0];
const idx = cleaned.lastIndexOf("@");
Expand All @@ -326,7 +326,7 @@ function parsePnpmPackageKeyV9(key: string): { key: string; name: string; versio
return { key: cleaned, name, version };
}

function normalizePnpmDepRef(depName: string, depRef: unknown): string | null {
export function normalizePnpmDepRef(depName: string, depRef: unknown): string | null {
if (typeof depRef === "string") {
const cleaned = depRef.replace(/^link:/, "").replace(/^workspace:/, "").split("(")[0];
if (!cleaned || cleaned.startsWith(".") || cleaned.startsWith("..")) return null;
Expand All @@ -343,7 +343,7 @@ function normalizePnpmDepRef(depName: string, depRef: unknown): string | null {
return null;
}

function normalizePnpmDepRefV9(depName: string, depRef: unknown): string | null {
export function normalizePnpmDepRefV9(depName: string, depRef: unknown): string | null {
if (typeof depRef === "string") {
const cleaned = depRef.replace(/^link:/, "").replace(/^workspace:/, "").replace(/^\//, "").split("(")[0];
if (!cleaned || cleaned.startsWith(".") || cleaned.startsWith("..")) return null;
Expand Down
46 changes: 46 additions & 0 deletions tests/fixtures/lockfile-pnpm-v9-graph/pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 38 additions & 0 deletions tests/output/sbom-dependency-edges.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -147,3 +147,41 @@ describe("resolveDependencyEdges", () => {
expect(resolveDependencyEdges(scanInput("/nonexistent/package-lock.json"), [pkg("x", "1.0.0")])).toEqual([]);
});
});

describe("resolveDependencyEdges - pnpm", () => {
const PNPM = "tests/fixtures/lockfile-pnpm-v9-graph/pnpm-lock.yaml";

const pkg = (name: string, version: string): PackageRef =>
({ name, version, ecosystem: "npm", paths: [] }) as PackageRef;

const allPackages = [
pkg("express", "4.17.1"),
pkg("vite", "5.0.0"),
pkg("body-parser", "1.19.0"),
pkg("postcss", "8.4.0"),
pkg("ms", "2.0.0"),
];

const edgesFor = () =>
resolveDependencyEdges({ source: "pnpm-lock", filePath: PNPM } as ScanInput, allPackages);

it("resolves edges from the pnpm lock graph instead of returning nothing", () => {
expect(edgesFor().length).toBeGreaterThan(0);
});

it("keeps every parent of a package reached through more than one route", () => {
const parentsOfMs = edgesFor()
.filter(e => e.child === "ms@2.0.0")
.map(e => e.parent)
.sort();
expect(parentsOfMs).toEqual([
"body-parser@1.19.0",
"express@4.17.1",
"vite@5.0.0",
]);
});

it("anchors a root-level package to the project rather than orphaning it", () => {
expect(edgesFor()).toContainEqual({ child: "express@4.17.1", parent: null });
});
});
34 changes: 34 additions & 0 deletions tests/parsers/pnpm-lock-graph.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { loadPnpmLockGraph } from "../../src/parsers/pnpm-lock-graph.js";

const V9 = "tests/fixtures/lockfile-pnpm-v9-graph/pnpm-lock.yaml";

describe("loadPnpmLockGraph", () => {
it("resolves every parent of a package reached through more than one route", () => {
const graph = loadPnpmLockGraph(V9);

const nodeIds = graph.nodeIdsFor("ms", "2.0.0");
expect(nodeIds.length).toBeGreaterThan(0);

const parents = nodeIds
.flatMap(id => [...graph.parentsFor(id)])
.map(id => graph.getNode(id))
.filter((n): n is { name: string; version: string } => n !== null)
.map(n => `${n.name}@${n.version}`)
.sort();

// ms is depended on by express, vite and body-parser. Path-derived edges
// lose routes; the lock graph must report all three.
expect([...new Set(parents)]).toEqual([
"body-parser@1.19.0",
"express@4.17.1",
"vite@5.0.0",
]);
});

it("reports no parents for a package only the root importer depends on", () => {
const graph = loadPnpmLockGraph(V9);
const nodeIds = graph.nodeIdsFor("express", "4.17.1");
const parents = nodeIds.flatMap(id => [...graph.parentsFor(id)]);
expect(parents).toEqual([]);
});
});
6 changes: 3 additions & 3 deletions website/docs/spdx.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@ Dependency relationships are derived from the resolved dependency paths, so a tr

**How complete the graph is depends on your package manager.**

For **npm** projects, edges come directly from the resolved lockfile graph, so the graph is complete. Measured on a large monorepo (`@lit-internal/monorepo`, 2060 packages), every package has a parent edge except the root project itself, which correctly has none.
For **npm** and **pnpm** projects, edges come directly from the resolved lockfile graph, so the graph is complete. Measured on `@lit-internal/monorepo` (npm, 2060 packages) and on two large pnpm monorepos, every package has a parent edge except the root project itself, which correctly has none.

For **pnpm, Yarn and Bun**, edges are derived from recorded dependency paths instead, and the scanner keeps at most five paths per package as a deliberate bound on large trees. A package reachable by more than five routes will therefore have some edges missing, and one whose five recorded routes all run through packages excluded from the scan can end up without a parent edge. The package list is always complete; only the edges between packages can be partial.
For **Yarn** and **Bun**, edges are derived from recorded dependency paths instead, and the scanner keeps at most five paths per package as a deliberate bound on large trees. A package reachable by more than five routes will therefore have some edges missing, and one whose five recorded routes all run through packages excluded from the scan can end up without a parent edge. The package list is always complete; only the edges between packages can be partial.

If you need an exhaustive dependency graph on a non-npm project, that is a known limitation rather than a bug.
If you need an exhaustive dependency graph on a Yarn or Bun project, that is a known limitation rather than a bug, tracked in [#1109](https://github.com/OWASP/cve-lite-cli/issues/1109) and [#1108](https://github.com/OWASP/cve-lite-cli/issues/1108).

### Why licenses are sometimes NOASSERTION

Expand Down