Skip to content

Commit 0829109

Browse files
EMaherEMaher
committed
updating articles for air-gapped environments
1 parent a4f5edb commit 0829109

7 files changed

Lines changed: 912 additions & 478 deletions

‎docs/README.md‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
| [Getting Started](getting-started.md) | Install and run your first extract → publish cycle in 10 minutes |
1010
| [Command Reference](commands/) | Detailed docs for [extract](commands/extract.md), [publish](commands/publish.md), [init](commands/init.md) |
1111
| [CI/CD Integration](ci-cd/) | Set up [GitHub Actions](ci-cd/github-actions.md) or [Azure DevOps](ci-cd/azure-devops.md) pipelines |
12-
| [Walkthroughs](walkthrough/) | Step-by-step guides: [Air-gapped GitHub Actions](walkthrough/air-gapped-github-actions.md), [Air-gapped Azure DevOps](walkthrough/air-gapped-azure-devops.md) |
12+
| [Walkthroughs](walkthrough/) | Step-by-step guides: [Air-gapped GitHub Actions](walkthrough/air-gapped-github-actions.md) (local registry or offline tarball), [Air-gapped Azure DevOps](walkthrough/air-gapped-azure-devops.md) (local registry or offline tarball) |
1313

1414
## How It Works
1515

@@ -80,8 +80,12 @@ docs/
8080
│ ├── overview.md — System design overview
8181
│ └── design-principles.md — Architecture principles
8282
├── walkthrough/
83-
│ ├── air-gapped-github-actions.md — Air-gapped setup with GitHub Actions
84-
│ └── air-gapped-azure-devops.md — Air-gapped setup with Azure DevOps
83+
│ ├── air-gapped-github-actions.md — Air-gapped GitHub Actions (chooser)
84+
│ ├── air-gapped-github-actions-local-registry.md — GitHub Actions via internal npm registry (GHES Packages)
85+
│ ├── air-gapped-github-actions-offline-tarball.md — GitHub Actions via offline tarball + npm cache
86+
│ ├── air-gapped-azure-devops.md — Air-gapped Azure DevOps (chooser)
87+
│ ├── air-gapped-azure-devops-local-registry.md — Azure DevOps via Azure Artifacts feed
88+
│ └── air-gapped-azure-devops-offline-tarball.md — Azure DevOps via offline tarball + npm cache
8589
└── troubleshooting/
8690
├── common-errors.md — Error messages and solutions
8791
├── debugging-guide.md — Debugging with --log-level

‎docs/walkthrough/air-gapped-azure-devops-local-registry.md‎

Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
# Air-Gapped Setup: Azure DevOps — Local npm Registry
2+
3+
Deploy APIM configuration using apiops-cli on [self-hosted Azure Pipelines agents](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops#self-hosted-agents) with **no internet access** at runtime. This walkthrough uses an Azure Artifacts npm feed as the package source so agents never reach the public npm registry.
4+
5+
> Looking for the alternative that doesn't require a registry? See [Offline Tarball walkthrough](air-gapped-azure-devops-offline-tarball.md).
6+
7+
---
8+
9+
## When to Use This Guide
10+
11+
- Self-hosted agents in a private network with no outbound internet
12+
- You can host (or already have) an Azure Artifacts npm feed reachable from the agent
13+
- Corporate networks that block access to the public npm registry
14+
15+
---
16+
17+
## Architecture Overview
18+
19+
```mermaid
20+
flowchart LR
21+
subgraph Connected Zone
22+
A[npm install from public registry] --> B[Azure Artifacts feed]
23+
end
24+
subgraph Air-Gapped Network
25+
B -->|Controlled sync window| C[Local feed replica]
26+
C --> D[Self-hosted Agent]
27+
D --> E[npm ci]
28+
E --> F[apiops extract / publish]
29+
end
30+
```
31+
32+
---
33+
34+
## Prerequisites
35+
36+
| Requirement | Details |
37+
|-------------|---------|
38+
| **Connected workstation** | A machine with internet access to seed the feed |
39+
| **Node.js 22.x** | Installed on both the workstation and the agent (includes npm) |
40+
| **[Self-hosted Azure Pipelines agent](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops#self-hosted-agents)** | Registered in your agent pool, running in the air-gapped network |
41+
| **Azure connectivity from agent** | The agent must reach your APIM instance's ARM endpoint (network-level, not npm) |
42+
| **Local npm registry** | An [Azure Artifacts npm feed](https://learn.microsoft.com/en-us/azure/devops/artifacts/npm/npmrc?view=azure-devops) accessible from the air-gapped network |
43+
44+
> **On-premises Azure DevOps:** If you run [Azure DevOps Server](https://learn.microsoft.com/en-us/azure/devops/server/install/get-started?view=azure-devops-2022) (formerly TFS), the same approach applies — Azure Artifacts is included in the server installation.
45+
46+
---
47+
48+
## Step 1 — Configure the Azure Artifacts npm Feed
49+
50+
Set up an Azure Artifacts npm feed that serves packages to your air-gapped agents without requiring internet access at install time.
51+
52+
1. **[Create a new feed](https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops#create-a-feed)** in your Azure DevOps organization.
53+
2. **[Configure an upstream source](https://learn.microsoft.com/en-us/azure/devops/artifacts/how-to/set-up-upstream-sources?view=azure-devops)** pointing to `https://registry.npmjs.org`. The upstream is only used during controlled sync windows; once `@peterhauge/apiops-cli` and its dependencies are cached, the feed serves them locally.
54+
3. **[Populate the feed](https://learn.microsoft.com/en-us/azure/devops/artifacts/npm/npmrc?view=azure-devops)** from a connected workstation by running `npm install @peterhauge/apiops-cli` against the feed registry URL. This pulls the package and its transitive dependencies into the feed cache.
55+
4. **[Add a project `.npmrc`](https://learn.microsoft.com/en-us/azure/devops/artifacts/npm/npmrc?view=azure-devops)** that points `registry=` at your feed URL and sets `always-auth=true`. Commit this file so pipelines and developers resolve against the local feed.
56+
57+
> **Tip:** Use the **Connect to feed** button in the Azure Artifacts UI to get the exact registry URL and a ready-to-copy `.npmrc` snippet for your feed.
58+
59+
---
60+
61+
## Step 2 — Initialize the Repository
62+
63+
```bash
64+
apiops init \
65+
--ci azure-devops \
66+
--environments dev,prod \
67+
--non-interactive
68+
```
69+
70+
This generates:
71+
72+
| File | Purpose |
73+
|------|---------|
74+
| `package.json` | Declares the CLI as a dependency |
75+
| `pipelines/run-extractor.yaml` | Extract pipeline |
76+
| `pipelines/run-publisher.yaml` | Publish pipeline |
77+
| `configuration.*.yaml` | Override templates |
78+
79+
---
80+
81+
## Step 3 — Generate the Lock File
82+
83+
```bash
84+
npm install
85+
```
86+
87+
This creates `package-lock.json`. Commit it — the lock file is **required** for `npm ci` to work.
88+
89+
---
90+
91+
## Step 4 — Configure the Self-Hosted Agent
92+
93+
Install and register the agent in the air-gapped network per the [self-hosted agent documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/linux-agent?view=azure-devops):
94+
95+
Ensure:
96+
97+
1. **Node.js 22.x** is installed and on `PATH`
98+
2. **Network access to the Azure Artifacts feed** — the agent can resolve packages from the local feed
99+
3. **Network access to Azure ARM** — the agent must reach `management.azure.com` (or [sovereign cloud equivalent](https://learn.microsoft.com/en-us/azure/developer/identity/national-cloud))
100+
4. **Network access to Azure DevOps** — the agent must reach your Azure DevOps org for job dispatch
101+
5. **Git** is installed (required by the `checkout` step)
102+
103+
> **Agent pool:** Add your air-gapped agents to a [dedicated agent pool](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/pools-queues?view=azure-devops) (e.g., `air-gapped-pool`) so pipelines target them explicitly.
104+
105+
---
106+
107+
## Step 5 — Modify Pipelines for Air-Gapped Operation
108+
109+
The generated pipelines (`pipelines/run-extractor.yaml` and `pipelines/run-publisher.yaml`) need minimal edits:
110+
111+
| Edit | What to Change |
112+
|------|----------------|
113+
| **Agent pool** | Replace `pool: vmImage: ubuntu-latest` with `pool: name: air-gapped-pool` ([pool YAML schema](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/pool?view=azure-pipelines)) |
114+
| **Remove NodeTool task** | Delete the `NodeTool@0` step (Node.js is pre-installed on the agent) |
115+
| **Add feed auth** | Insert [`npmAuthenticate@0`](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/npm-authenticate-v0?view=azure-pipelines) before the `npm ci` step (see below) |
116+
117+
### Feed Authentication
118+
119+
Add this task before any `npm ci` step in both pipelines:
120+
121+
```yaml
122+
- task: npmAuthenticate@0
123+
inputs:
124+
workingFile: .npmrc
125+
```
126+
127+
`npmAuthenticate@0` reads the committed `.npmrc` and injects credentials for the Azure Artifacts feed using the pipeline's built-in build service identity — no service connection needed for feeds in the same organization. The `npm ci` step then works as-is.
128+
129+
> **Azure authentication:** The `AzureCLI@2` task handles Azure authentication separately, via the service connection. The service connection injects tokens for `DefaultAzureCredential`.
130+
131+
---
132+
133+
## Step 6 — Configure Variable Groups and Service Connections
134+
135+
Follow the standard [Azure DevOps integration guide](../ci-cd/azure-devops.md#variable-groups-configuration) to set up:
136+
137+
1. **Variable group `apim-common`** — variables shared across all environments (e.g., subscription ID, tenant ID, common tags)
138+
2. **Variable groups `apim-dev`, `apim-prod`** — environment-specific variables (e.g., APIM instance name, resource group) that override values from `apim-common`
139+
3. **Service connections** — Azure Resource Manager connections scoped to your APIM instances; they can be referenced from either variable group
140+
141+
These are configured in Azure DevOps and are injected at pipeline runtime.
142+
143+
---
144+
145+
## Step 7 — Commit and Validate
146+
147+
```bash
148+
git add .
149+
git commit -m "feat: air-gapped apiops setup with local registry"
150+
git push
151+
```
152+
153+
Trigger the extract pipeline manually from **Pipelines → Run pipeline** and verify:
154+
155+
1. `npm ci` resolves all packages from the local Azure Artifacts feed (no calls to npmjs.org)
156+
2. `apiops extract` authenticates via the service connection and runs successfully
157+
158+
> **✅ Setup complete.** Your air-gapped apiops pipelines are now operational. The remaining sections cover ongoing maintenance and reference material — read them as needed.
159+
160+
---
161+
162+
## Upgrading the CLI Version
163+
164+
Sync the feed during a connectivity window to pull the new version, then update `package.json` and regenerate `package-lock.json`. Commit both.
165+
166+
---
167+
168+
## Troubleshooting
169+
170+
| Problem | Cause | Fix |
171+
|---------|-------|-----|
172+
| `npm ci` fails with `E404` | Package not in local feed | Sync the feed during a connectivity window |
173+
| `npm ci` fails with "lockfile mismatch" | `package-lock.json` out of sync with `package.json` | Re-run `npm install` on connected workstation, commit updated lock file |
174+
| `npx apiops` not found | `npm ci` didn't complete or `.bin` not in PATH | Verify `node_modules/.bin/apiops` exists after install |
175+
| Azure auth fails | Agent can't reach Entra ID or ARM endpoint | Verify network allows traffic to `login.microsoftonline.com` and `management.azure.com` (or sovereign equivalents) |
176+
| `AzureCLI@2` service connection error | Service connection not linked or misconfigured | Verify variable group is linked to pipeline and connection name matches |
177+
| Agent not picking up jobs | Pool name mismatch or agent offline | Confirm pool name in YAML matches the registered agent pool |
178+
| `npmAuthenticate@0` fails | Feed permissions or `.npmrc` path wrong | Ensure the build service identity has Reader access to the feed |
179+
180+
---
181+
182+
## Further Reading
183+
184+
- [Offline Tarball walkthrough](air-gapped-azure-devops-offline-tarball.md) — alternative for environments without a registry
185+
- [apiops init reference](../commands/init.md)
186+
- [Azure DevOps integration](../ci-cd/azure-devops.md) — standard (connected) setup
187+
- [Authentication guide](../guides/authentication.md) — service principal and managed identity options
188+
- [Azure Artifacts npm feeds](https://learn.microsoft.com/en-us/azure/devops/artifacts/npm/npmrc?view=azure-devops) — official feed setup docs
189+
- [Self-hosted agents](https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops#self-hosted-agents) — agent installation and configuration
190+
- [Azure DevOps Server](https://learn.microsoft.com/en-us/azure/devops/server/install/get-started?view=azure-devops-2022) — on-premises installation
191+
- [National cloud endpoints](https://learn.microsoft.com/en-us/azure/developer/identity/national-cloud) — sovereign cloud identity configuration
192+
- [Entra ID authentication endpoints](https://learn.microsoft.com/en-us/azure/developer/identity/national-cloud#azure-ad-authentication-endpoints) — per-cloud token acquisition endpoints

0 commit comments

Comments
 (0)