Repository rules to fetch third-party npm packages
Load these with,
load("@aspect_rules_js//npm:npm_import.bzl", "npm_translate_lock", "npm_import")
These use Bazel's downloader to fetch the packages. You can use this to redirect all fetches through a store like Artifactory.
See https://blog.aspect.dev/configuring-bazels-downloader for more info about how it works and how to configure it.
npm_translate_lock
is the primary user-facing API.
It uses the lockfile format from pnpm because it gives us reliable
semantics for how to dynamically lay out node_modules
trees on disk in bazel-out.
To create pnpm-lock.yaml
, consider using pnpm import
to preserve the versions pinned by your existing package-lock.json
or yarn.lock
file.
If you don't have an existing lock file, you can run npx pnpm install --lockfile-only
.
Advanced users may want to directly fetch a package from npm rather than start from a lockfile.
npm_import
does this.
npm_translate_lock_rule(name, custom_postinstalls, dev, lifecycle_hooks_envs, lifecycle_hooks_exclude, lifecycle_hooks_execution_requirements, no_optional, npm_package_lock, package_json, patch_args, patches, pnpm_lock, prod, public_hoist_packages, repo_mapping, run_lifecycle_hooks, verify_node_modules_ignored, warn_on_unqualified_tarball_url, yarn_lock)
Repository rule to generate npm_import rules from pnpm lock file.
The pnpm lockfile format includes all the information needed to define npm_import rules, including the integrity hash, as calculated by the package manager.
For more details see, https://github.com/pnpm/pnpm/blob/main/packages/lockfile-types/src/index.ts.
Instead of manually declaring the npm_imports
, this helper generates an external repository
containing a helper starlark module repositories.bzl
, which supplies a loadable macro
npm_repositories
. This macro creates an npm_import
for each package.
The generated repository also contains BUILD files declaring targets for the packages
listed as dependencies
or devDependencies
in package.json
, so you can declare
dependencies on those packages without having to repeat version information.
Bazel will only fetch the packages which are required for the requested targets to be analyzed. Thus it is performant to convert a very large pnpm-lock.yaml file without concern for users needing to fetch many unnecessary packages.
Setup
In WORKSPACE
, call the repository rule pointing to your pnpm-lock.yaml file:
load("@aspect_rules_js//npm:npm_import.bzl", "npm_translate_lock")
# Read the pnpm-lock.yaml file to automate creation of remaining npm_import rules
npm_translate_lock(
# Creates a new repository named "@npm_deps"
name = "npm_deps",
pnpm_lock = "//:pnpm-lock.yaml",
# Recommended attribute that also checks the .bazelignore file
verify_node_modules_ignored = "//:.bazelignore",
)
Next, there are two choices, either load from the generated repo or check in the generated file. The tradeoffs are similar to this rules_python thread.
- Immediately load from the generated
repositories.bzl
file inWORKSPACE
. This is similar to thepip_parse
rule in rules_python for example. It has the advantage of also creating aliases for simpler dependencies that don't require spelling out the version of the packages. However it causes Bazel to eagerly evaluate thenpm_translate_lock
rule for every build, even if the user didn't ask for anything JavaScript-related.
# Following our example above, we named this "npm_deps"
load("@npm_deps//:repositories.bzl", "npm_repositories")
npm_repositories()
- Check in the
repositories.bzl
file to version control, and load that instead. This makes it easier to ship a ruleset that has its own npm dependencies, as users don't have to install those dependencies. It also avoids eager-evaluation ofnpm_translate_lock
for builds that don't need it. This is similar to theupdate-repos
approach from bazel-gazelle.
In a BUILD file, use a rule like write_source_files to copy the generated file to the repo and test that it stays updated:
write_source_files(
name = "update_repos",
files = {
"repositories.bzl": "@npm_deps//:repositories.bzl",
},
)
Then in WORKSPACE
, load from that checked-in copy or instruct your users to do so.
ATTRIBUTES
Name | Description | Type | Mandatory | Default |
---|---|---|---|---|
name | A unique name for this repository. | Name | required | |
custom_postinstalls | A map of package names or package names with their version (e.g., "my-package" or "[email protected]") to a custom postinstall script to apply to the downloaded npm package after its lifecycle scripts runs. If the version is left out of the package name, the script will run on every version of the npm package. If a custom postinstall scripts exists for a package as well as for a specific version, the script for the versioned package will be appended with && to the non-versioned package script. |
Dictionary: String -> String | optional | {} |
dev | If true, only install devDependencies | Boolean | optional | False |
lifecycle_hooks_envs | Environment variables applied to the preinstall, install and postinstall lifecycle hooks on npm packages. The environment variables can be defined per package by package name or globally using "". Variables are declared as key/value pairs of the form "key=value". For example: lifecycle_hooks_envs: { "": ["GLOBAL_KEY1=value1", "GLOBAL_KEY2=value2"], "@foo/bar": ["PREBULT_BINARY=http://downloadurl"], } | Dictionary: String -> List of strings | optional | {} |
lifecycle_hooks_exclude | A list of package names or package names with their version (e.g., "my-package" or "[email protected]") to not run lifecycle hooks on | List of strings | optional | [] |
lifecycle_hooks_execution_requirements | Execution requirements applied to the preinstall, install and postinstall lifecycle hooks on npm packages. The execution requirements can be defined per package by package name or globally using "". For example: lifecycle_hooks_execution_requirements: { "": ["requires-network"], "@foo/bar": ["no-sandbox"], } | Dictionary: String -> List of strings | optional | {} |
no_optional | If true, optionalDependencies are not installed | Boolean | optional | False |
npm_package_lock | The package-lock.json file written by npm install .When set, the package_json attribute must be set as well. Exactly one of [pnpm_lock, npm_package_lock, yarn_lock] should be set. |
Label | optional | None |
package_json | The package.json file. From this file and the corresponding package-lock.json/yarn.lock file (specified with the npm_package_lock/yarn_lock attributes), a pnpm-lock.yaml file will be generated using pnpm import .Note that any changes to the package.json file will invalidate the npm_translate_lock repository rule, causing it to re-run on the next invocation of Bazel. Mandatory when using npm_package_lock or yarn_lock, otherwise must be unset. |
Label | optional | None |
patch_args | A map of package names or package names with their version (e.g., "my-package" or "[email protected]") to a label list arguments to pass to the patch tool. Defaults to -p0, but -p1 will usually be needed for patches generated by git. If patch args exists for a package as well as a package version, then the version-specific args will be appended to the args for the package. | Dictionary: String -> List of strings | optional | {} |
patches | A map of package names or package names with their version (e.g., "my-package" or "[email protected]") to a label list of patches to apply to the downloaded npm package. Paths in the patch file must start with extract_tmp/package where package is the top-level folder in the archive on npm. If the version is left out of the package name, the patch will be applied to every version of the npm package. |
Dictionary: String -> List of strings | optional | {} |
pnpm_lock | The pnpm-lock.yaml file. Exactly one of [pnpm_lock, npm_package_lock, yarn_lock] should be set. |
Label | optional | None |
prod | If true, only install dependencies | Boolean | optional | False |
public_hoist_packages | A map of package names or package names with their version (e.g., "my-package" or "[email protected]") to a list of Bazel packages in which to hoist the package to the top-level of the node_modules tree when linking. This is similar to setting https://pnpm.io/npmrc#public-hoist-pattern in an .npmrc file outside of Bazel, however, wild-cards are not yet supported and npm_translate_lock will fail if there are multiple versions of a package that are to be hoisted. |
Dictionary: String -> List of strings | optional | {} |
repo_mapping | A dictionary from local repository name to global repository name. This allows controls over workspace dependency resolution for dependencies of this repository.<p>For example, an entry "@foo": "@bar" declares that, for any time this repository depends on @foo (such as a dependency on @foo//some:target , it should actually resolve that dependency within globally-declared @bar (@bar//some:target ). |
Dictionary: String -> String | required | |
run_lifecycle_hooks | If true, runs preinstall, install and postinstall lifecycle hooks on npm packages if they exist | Boolean | optional | True |
verify_node_modules_ignored | node_modules folders in the source tree should be ignored by Bazel. This points to a .bazelignore file to verify that all nested node_modules directories pnpm will create are listed.See bazelbuild/bazel#8106 |
Label | optional | None |
warn_on_unqualified_tarball_url | - | Boolean | optional | True |
yarn_lock | The yarn.lock file written by yarn install .When set, the package_json attribute must be set as well. Exactly one of [pnpm_lock, npm_package_lock, yarn_lock] should be set. |
Label | optional | None |
npm_import(name, package, version, deps, transitive_closure, root_package, link_workspace, link_packages, run_lifecycle_hooks, lifecycle_hooks_execution_requirements, lifecycle_hooks_env, integrity, url, patch_args, patches, custom_postinstall)
Import a single npm package into Bazel.
Normally you'd want to use npm_translate_lock
to import all your packages at once.
It generates npm_import
rules.
You can create these manually if you want to have exact control.
Bazel will only fetch the given package from an external registry if the package is required for the user-requested targets to be build/tested.
This is a repository rule, which should be called from your WORKSPACE
file
or some .bzl
file loaded from it. For example, with this code in WORKSPACE
:
npm_import(
name = "npm__at_types_node_15.12.2",
package = "@types/node",
version = "15.12.2",
integrity = "sha512-zjQ69G564OCIWIOHSXyQEEDpdpGl+G348RAKY0XXy9Z5kU9Vzv1GMNnkar/ZJ8dzXB3COzD9Mo9NtRZ4xfgUww==",
)
This is similar to Bazel rules in other ecosystems named "_import" like
apple_bundle_import
,scala_import
,java_import
, andpy_import
.go_repository
is also a model for this rule.
The name of this repository should contain the version number, so that multiple versions of the same package don't collide. (Note that the npm ecosystem always supports multiple versions of a library depending on where it is required, unlike other languages like Go or Python.)
To consume the downloaded package in rules, it must be "linked" into the link package in the
package's BUILD.bazel
file:
load("@npm__at_types_node__15.12.2__links//:defs.bzl", npm_link_types_node = "npm_link_imported_package")
npm_link_types_node(name = "node_modules")
This links @types/node
into the node_modules
of this package with the target name :node_modules/@types/node
.
A :node_modules/@types/node/dir
filegroup target is also created that provides the the directory artifact of the npm package.
This target can be used to create entry points for binary target or to access files within the npm package.
NB: You can choose any target name for the link target but we recommend using the node_modules/@scope/name
and
node_modules/name
convention for readability.
When using npm_translate_lock
, you can link all the npm dependencies in the lock file for a package:
load("@npm//:defs.bzl", "npm_link_all_packages")
npm_link_all_packages(name = "node_modules")
This creates :node_modules/name
and :node_modules/@scope/name
targets for all direct npm dependencies in the package.
It also creates :node_modules/name/dir
and :node_modules/@scope/name/dir
filegroup targets that provide the the directory artifacts of their npm packages.
These target can be used to create entry points for binary target or to access files within the npm package.
If you have a mix of npm_link_all_packages
and npm_link_imported_package
functions to call you can pass the
npm_link_imported_package
link functions to the imported_links
attribute of npm_link_all_packages
to link
them all in one call. For example,
load("@npm//:defs.bzl", "npm_link_all_packages")
load("@npm__at_types_node__15.12.2__links//:defs.bzl", npm_link_types_node = "npm_link_imported_package")
npm_link_all_packages(
name = "node_modules",
imported_links = [
npm_link_types_node,
]
)
This has the added benefit of adding the imported_links
to the convienence :node_modules
target which
includes all direct dependencies in that package.
NB: You can pass an name to npm_link_all_packages and this will change the targets generated to "{name}/@scope/name" and "{name}/name". We recommend using "node_modules" as the convention for readability.
To change the proxy URL we use to fetch, configure the Bazel downloader:
-
Make a file containing a rewrite rule like
rewrite (registry.nodejs.org)/(.*) artifactory.build.internal.net/artifactory/$1/$2
-
To understand the rewrites, see UrlRewriterConfig in Bazel sources.
-
Point bazel to the config with a line in .bazelrc like common --experimental_downloader_config=.bazel_downloader_config
PARAMETERS
npm_translate_lock(name, npm_package_lock, yarn_lock, pnpm_version, kwargs)
Wrapper macro around npm_translate_lock_rule
This macro creates a "pnpm" repository. rules_js currently only uses this repository when npm_package_lock or yarn_lock are used rather than pnpm_lock. Set pnpm_version to None to inhibit this repository creation.
The user can create a "pnpm" repository before calling this in order to override.
PARAMETERS
Name | Description | Default Value |
---|---|---|
name | - |
none |
npm_package_lock | - |
None |
yarn_lock | - |
None |
pnpm_version | - |
"7.9.1" |
kwargs | - |
none |