Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,10 @@ node_modules/
dist/
*.tgz
.DS_Store

.nuxt
.output
docs/.data
docs/.nitro
docs/.cache
docs/.wrangler
6 changes: 5 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ pnpm fmt # oxlint --fix + oxfmt
pnpm test # vitest watch mode
pnpm test:run # vitest single run (CI-style)
pnpm release # test + build + changelogen --release --push
pnpm docs # Docus site + lookup explorer on :3000 (after pnpm build)
pnpm docs:build # Cloudflare Workers build of the docs
```

### Run a single test
Expand All @@ -40,6 +42,7 @@ src/
test/
├── unit/ # deterministic, mocked
└── e2e/ # live HTTP against real registries
docs/ # Docus site: guide, registry pages, live lookup explorer on Workers (see docs/AGENTS.md)
```

### Dependency direction
Expand Down Expand Up @@ -87,7 +90,7 @@ test/
- Plugin-based via abstract `Registry` subclasses registered with `register()` and resolved by `create()`. No hardcoded switch logic.
- Importing `@agntn/registries` registers the built-in adapters as an intentional side effect.
- Each adapter normalizes upstream payloads into core types before returning.
- No adapter-to-adapter imports.
- No imports between adapters.

### Tests

Expand Down Expand Up @@ -148,3 +151,4 @@ test/
- Bulk fetch helpers skip failed packages instead of failing all — this is intentional.
- Cache is optional decorator, never mandatory in core flows.
- e2e smoke tests are network-sensitive — failures may be transient.
- `sideEffects` in `package.json` must cover `dist/_chunks/*.mjs`: obuild emits the adapter `register()` calls into a chunk, and a bundler that trusts `sideEffects` drops it otherwise (guarded by `test/unit/side-effects.test.ts`).
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@ There is no embeddable TypeScript library that normalizes across registries. The

@agntn/registries fills that gap. One `fetchPackage` call, same response shape, regardless of whether the package lives on npm or Packagist. Uses [PURL (ECMA-427)](https://github.com/package-url/purl-spec) for addressing, so `pkg:npm/lodash` and `pkg:cargo/serde` resolve through the same code path. Storage-backed caching with a lockfile keeps things fast on repeated lookups.

Docs and the live lookup explorer: [registries.agntn.dev](https://registries.agntn.dev). The source lives in [`docs/`](./docs); run `pnpm docs` after `pnpm build` for a local copy.

## Features

- 🔍 **Single API, six registries** - npm, PyPI, crates.io, RubyGems, Packagist, Arch Linux (official + AUR)
- 📦 **PURL-native** - [ECMA-427](https://github.com/package-url/purl-spec) identifiers as first-class input
- 🏷️ **Normalized data model** - same `Package`, `Version`, `Dependency`, `Maintainer` types everywhere
- 💾 **Storage-backed cache + lockfile** - unstorage-native, sha256 integrity checks, configurable TTL
- 💾 **Cache on unstorage with a lockfile** - sha256 integrity checks, configurable TTL
- ⌨️ **CLI included** - `registries info npm/lodash`, `registries versions cargo/serde`, `registries deps pypi/flask@3.1.1`
- 🔁 **Retry + backoff** - exponential backoff with jitter, rate limiter interface
- 🪶 **ESM-only, zero CJS** - built with [obuild](https://github.com/unjs/obuild)
Expand Down Expand Up @@ -78,7 +80,7 @@ Add `--json` for machine-readable output, `--no-cache` to skip the cache.

### AI SDK tool

`@agntn/registries/ai` exports a ready-made tool for AI SDK apps:
`@agntn/registries/ai` exports a tool for AI SDK apps that needs no wiring:

```ts
import { generateText } from "ai";
Expand Down Expand Up @@ -265,7 +267,7 @@ interface Package {
homepage: string;
documentation: string; // docs URL (docs.rs, readthedocs, rubydoc, etc.)
repository: string;
licenses: string; // SPDX-normalized
licenses: string; // normalized to SPDX
keywords: string[];
namespace: string; // e.g. "@vue" for npm scoped packages
latestVersion: string;
Expand Down
14 changes: 14 additions & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
.output
.data
.nuxt
.nitro
.cache
dist
node_modules
logs
*.log
.DS_Store
.env
.env.*
!.env.example
.wrangler
64 changes: 64 additions & 0 deletions docs/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# docs/

Docus site for `@agntn/registries`. Markdown lives in `content/`. The lookup explorer is a Vue page in the Nuxt app backed by Nitro routes over the library, not a script.

## Layout

```
docs/
├── nuxt.config.ts # extends: ['docus'], cloudflare_module preset (Workers)
├── app/app.config.ts # title, github, theme
├── app/app.css # theme tokens (light + .dark), shared `registries-*` classes
├── app/components/ # Docus overrides: AppHeaderLogo, AppHeaderCTA (nav), AppFooterLeft, DocsAsideLeftBody
├── app/components/content/ # MDC components (`::landing-home`, `::registry-facts`, `::lookup-explorer`)
├── app/components/OgImage/ # Docs.takumi and Landing.takumi override the Docus OG templates
├── app/assets/fonts.css # @font-face for the TTFs served from public/fonts (site and OG images)
├── app/composables/ # useLandingLookup (one clock for every live panel), useSubNavigation
├── app/utils/ # ecosystems table, formatting, recorded landing samples
├── app/pages/lookup.vue # explorer, own route outside the docs layout
├── server/api/ # package, versions, dependencies, maintainers, ecosystems over the library
├── server/utils/query.ts # parameter caps, cache, rate limit, error mapping
├── content/index.md # landing
├── content/1.guide/ # getting started, purl, lookups, cache, cli, agents, custom, explorer
└── content/2.registries/ # one page per ecosystem
```

## Commands

```bash
pnpm install # from docs/, after pnpm build in the repo root
pnpm dev # http://localhost:3000
pnpm build # Cloudflare Workers output in .output/, content routes prerendered
pnpm deploy # build, then wrangler deploy to registries.agntn.dev
pnpm generate # static output only; the /api routes need the worker
```

Deployment: Nitro preset `cloudflare_module`. Nuxt Content needs a D1 binding named `DB` and the response cache a KV binding named `CACHE`; `wrangler.jsonc` carries both and the `NUXT_SITE_URL` var, Nitro merges it into the generated `.output/server/wrangler.json`. Create them once with `wrangler d1 create agntn-registries` and `wrangler kv namespace create CACHE` and put the ids in `wrangler.jsonc`; the ids there are placeholders until then.

The site imports `@agntn/registries` from `file:..`. Build the parent package first.

Resolution traps, both caused by the repo root being a pnpm workspace:

- `pnpm-workspace.yaml` sets `shamefullyHoist: true`. Without it `docs/node_modules` holds only direct dependencies, Node walks up to the root `node_modules`, and the server bundle can get a second copy of Vue.
- `nuxt.config.ts` pins `workspaceDir` to `docs/` and disables telemetry, which would otherwise be resolved from the root.

## Live data

- `server/api/*.get.ts` resolve the PURL with `createFromPURL` and call `fetchPackage`, `fetchVersions`, `fetchDependencies` and `fetchMaintainers` on the adapter, with a `Client` of one retry and a twenty second timeout. The page shows what a script would get.
- Every route goes through `cachedAnswer` in `server/utils/query.ts`: exact parameters as the key, the library's `DEFAULT_TTL` for the data type, nothing for a thrown failure. Do not bypass it: the registries behind it are public services. A cache miss also counts against `RATE_LIMIT` (30 new queries a minute per address, 429 past it); cache hits are free.
- Library errors are mapped in `toHttpError`: `NotFoundError` 404, `InvalidPURLError` and `UnknownEcosystemError` 400, `RateLimitError` 429, `HTTPError` 502.
- `app/utils/landing-fixtures.ts` holds answers recorded through the library so the landing paints before the worker answers. Regenerate it with a script over `dist/index.mjs` (`createFromPURL` plus the four lookups for the six example PURLs); never edit the recorded values by hand.
- In production the cache lives in the KV binding `CACHE` (`$production.nitro.storage.cache`); locally it is in memory.
- The explorer applies its deep link through a `watch(route.query)` that fires once: a prerendered page hydrates with an empty query and Nuxt restores the address after mount.

## OG images

- `app/components/OgImage/Docs.takumi.vue` and `Landing.takumi.vue` override the Docus templates of the same name and are rendered by Takumi at build time. Takumi has no CSS variables, so the theme colours from `app.css` are repeated there as literals.
- nuxt-og-image does not see the faces `@nuxt/fonts` generates on this Nuxt version, but it parses `@font-face` rules from the files in `css`. That is why `app/assets/fonts.css` declares the five TTFs in `public/fonts` and `fonts.families` uses the `local` provider: the site and the OG images share the same files.
- The landing OG file is named from the SEO description. Nitro refuses to write a prerender path containing `..`, so a description ending in a period (`out..png`) is silently skipped and the landing ships with a dead `og:image`. Keep the description in `content/index.md` free of a trailing period.

## Constraints

- Registry metadata is untrusted data. Render descriptions, keywords and maintainer fields as text; never `v-html`.
- Ecosystem names, icons, class names and example PURLs live once in `app/utils/registries.ts`. The sidebar, the landing grid, the explorer and `::registry-facts` read from it.
- Keep the docs API shapes (`PackageAnswer` and friends) in the route files; the explorer mirrors them as local interfaces.
94 changes: 94 additions & 0 deletions docs/app/app.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
export default defineAppConfig({
docus: {
colorMode: "dark",
},
seo: {
title: "@agntn/registries",
description:
"One TypeScript interface over npm, PyPI, crates.io, RubyGems, Packagist and Arch Linux: PURL in, normalized package metadata out.",
},
header: {
title: "@agntn/registries",
},
github: {
url: "https://github.com/agntn/registries",
branch: "main",
rootDir: "docs",
},
socials: {
github: "https://github.com/agntn/registries",
npm: "https://www.npmjs.com/package/@agntn/registries",
},
ui: {
colors: {
primary: "violet",
neutral: "slate",
},
button: {
slots: {
base: "h-9 rounded-lg px-3.5 text-sm leading-none font-medium cursor-pointer transition-colors",
},
compoundVariants: [
{
color: "primary",
variant: "solid",
class: "registries-primary-fill ring-0",
},
{
color: "neutral",
variant: "outline",
class: "registries-neutral-outline ring-0",
},
],
},
pageHeader: {
slots: {
root: "py-8 border-b border-muted",
headline: "registries-eyebrow mb-3",
title: "text-3xl sm:text-4xl font-medium tracking-tight text-highlighted",
description: "text-base leading-7 text-muted",
},
},
contentSurround: {
slots: {
link: "rounded-xl registries-frame border-0 bg-default hover:bg-muted",
linkLeadingIcon: "text-muted",
},
},
prose: {
callout: {
slots: {
base: "rounded-xl px-4 py-3.5",
},
},
card: {
slots: {
base: "rounded-xl registries-frame border-0 p-5 bg-default hover:bg-muted",
icon: "size-5 mb-3 text-muted transition-colors group-hover:text-primary",
title: "text-sm font-medium",
description: "text-sm text-muted",
},
},
cardGroup: {
base: "grid grid-cols-1 sm:grid-cols-2 gap-3 my-5 *:my-0",
},
table: {
slots: {
root: "rounded-xl registries-frame",
},
},
pre: {
slots: {
header: "border-default bg-default",
base: "border-default bg-muted",
},
},
},
pageHero: {
slots: {
title: "font-medium tracking-tight",
description: "text-base leading-7 sm:text-lg",
},
},
},
});
Loading
Loading