Skip to content
Draft
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions vcpkg/TOC.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
href: consume/install-locally-modified-package.md
- name: Install a dependency from a Git-based registry
href: consume/git-registries.md
- name: Authenticate to private Git registries
- name: Authenticate to private Git repositories
href: users/authentication.md
- name: Authenticate to private Git registries in your GitHub Actions workflows
href: consume/gha-authentication.md
Expand All @@ -67,11 +67,14 @@
items:
- name: Set up a vcpkg asset cache
href: consume/asset-caching.md
- name: Use a custom script as an asset caching storage provider
- name: Use a custom script as an asset caching source
# TODO: Modernize article: examples/asset-caching-source-nuget.md
href: examples/asset-caching-source-nuget.md
- name: Set up caching using Azure Storage and Azure DevOps
href: consume/caching-workload-identity-federation.md
- name: Authenticate to third-party services
displayName: third-party, authentication, GitHub, Azure DevOps, NuGet, GitHub Packages, GitHub Actions Cache, Azure Blob Storage
href: consume/third-party-authentication.md
- name: Produce packages
expanded: true
items:
Expand Down
35 changes: 20 additions & 15 deletions vcpkg/concepts/asset-caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,24 @@ description: Use asset caching with vcpkg to mirror your asset's downloads to im
author: vicroms
ms.author: viromer
ms.topic: concept-article
ms.date: 01/10/2024
ms.date: 10/03/2024
---
# Asset caching with vcpkg

[!INCLUDE [experimental](../../includes/experimental.md)]

Asset caching is a feature that allows vcpkg to use alternative download
endpoints to acquire a package's assets. When asset caching is enabled vcpkg
attempts to download assets (such as source code and build tools) from a
configured mirror location. If the asset is not available in the mirror location, vcpkg will download it from its original source and then store a copy in the mirror for future use.
A fundamental operation of vcpkg is downloading files from the Internet, things like: source code archives, tool executables,
source code patches, etc. We call anything that is downloaded from the network an **asset**.
Comment thread
vicroms marked this conversation as resolved.
Outdated

Asset caching can help with these common development scenarios:
Asset caching allows you to configure alternative download locations (caches) to acquire assets. When enabled, vcpkg first
attempst to acquire assets from any configured cache location, and falls back to the original download if no cached asset
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

attempts

is found. Optionally, vcpkg stores the asset in the cache for future use.

* Improving reliability of continuous integration runs.
* Mirroring download assets in trusted locations for air-gapped environments.
* Maintaining continuity of business if third party sources are changed or become unavailable.
This feature helps mitigate these common development concerns:

* Reducing dependency from external networks. Possibly, also reducing download times.
* Mirroring assets in trusted locations for air-gapped environments.
* Maintaining continuity of business if assets hosted by third-parties are changed or unavailable.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Reducing dependency from external networks. Possibly, also reducing download times.
* Mirroring assets in trusted locations for air-gapped environments.
* Maintaining continuity of business if assets hosted by third-parties are changed or unavailable.
* Reducing download times and bandwidth costs.
* Mirroring assets in trusted locations for air-gapped or offline environments.
* Maintaining continuity of business if assets hosted by third-parties are changed or unavailable.


## Configuration

Expand All @@ -28,15 +30,18 @@ Asset caching is configured via:
* The `X_VCPKG_ASSET_SOURCES` environment variable, or
* The `--x-asset-sources` command-line option.

In both cases, the expected value is a semicolon-delimited list of sources. Each
source has a specific syntax depending on its storage backend (see the [asset
caching reference documentation](../users/assetcaching.md) to learn how to
configure asset caching sources).
The expected value is a semicolon-delimited list of asset caching sources. Read the [asset caching reference
documentation](../users/assetcaching.md) to learn the configuration syntax.

Example using Azure Blob Storage as an asset cache:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Example using Azure Blob Storage as an asset cache:
### Example using Azure Blob Storage:


```PowerShell
vcpkg install zlib --x-asset-sources="x-azurl,mystorageaccount.blob.core.windows.net,${env:SAS_TOKEN},readwrite"
```

## Next step

Here are some tasks to try next:

* [Set up an asset cache](../consume/asset-caching.md)
* [Use a custom script as an asset caching source for
NuGet](../examples/asset-caching-source-nuget.md)
* [Use a custom script as an asset caching source](../examples/asset-caching-source-nuget.md)
34 changes: 21 additions & 13 deletions vcpkg/consume/asset-caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Tutorial: Set up a vcpkg asset cache"
description: Learn to set up a local asset cache to mirror download assets.
author: vicroms
ms.author: viromer
ms.date: 09/02/2024
ms.date: 10/03/2025
ms.topic: tutorial
#CustomerIntent: As a beginner vcpkg user, I want to set up a local asset cache to mirror downloaded artifacts
zone_pivot_group_filename: zone-pivot-groups.json
Expand All @@ -12,10 +12,12 @@ zone_pivot_groups: shell-selections
# Tutorial: Set up a vcpkg asset cache

All asset caches are configured through the `X_VCPKG_ASSET_SOURCES` environment variable. The value
of `X_VCPKG_ASSET_SOURCES` follows a specific [configuration
syntax](../users/assetcaching.md).
of `X_VCPKG_ASSET_SOURCES` follows a specific [configuration syntax](../users/assetcaching.md).

In this tutorial you'll learn how to:
In this tutorial, you'll create a local asset cache location to store downloaded artifacts.

you want to use instead,
Comment thread
vicroms marked this conversation as resolved.
Outdated
replace the URL and provide a SAS token as the second parameter in the configuration string.

> [!div class="checklist"]
> * [Create an asset cache directory](#1---create-an-asset-cache-directory)
Expand All @@ -28,15 +30,21 @@ In this tutorial you'll learn how to:

## 1 - Create an asset cache directory

You can use the Azure Blob Storage backend "[`x-azurl`](../users/assetcaching.md#x-azurl)", to store
assets. Alternatively, you can use filesystem directories to store
your download assets, including network locations. To do so, you need to use the
`file://` protocol in your asset cache URL.
vcpkg offers two types of asset cache providers:

* `x-azurl`: which is intended to work with Azure Blob Storage containers, but works just as well with other service providers
and even local or network filesystems.
* `x-script`: which lets you add your own business logic to handle asset caching throug a customizable script or executable.

This tutorial uses, `x-azurl` as the provider to store assets in a local directory in your filesystem using the URLS with
the `file://` protocol.

> [!NOTE]
> You can use these same steps to create an asset cache with Azure Blob Storage. Replace any URL in the tutorial with the
> URL for your storage and provide a SAS Token for authentication.
>
> See the [instructions for authenticating to an Azure Blob Storage](third-party-authentication.md#asset-cache-azure-blob-storage).

In this tutorial, you'll create a local asset cache location to store downloaded
artifacts. If you have an Azure DevOps Blob Storage you want to use instead,
replace the URL and provide a SAS token as the second parameter in the
configuration string.

1. Create a directory to serve as an asset cache location (substitute with any locations of your choosing):

Expand Down Expand Up @@ -82,7 +90,7 @@ set "X_VCPKG_ASSET_SOURCES=clear;x-azurl,file://D:/vcpkg/asset-cache,,readwrite"
This `X_VCPKG_ASSET_SOURCES` configuration adds the following source strings:

* `clear`, disables any previously configured asset cache
* `x-azurl,file://C:/vcpkg/asset-cache,,readwrite`, sets a filesystem asset cache, located in
* `x-azurl,file://D:/vcpkg/asset-cache,,readwrite`, sets a filesystem asset cache, located in
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* `x-azurl,file://D:/vcpkg/asset-cache,,readwrite`, sets a filesystem asset cache, located in
* `x-azurl,file:///D:/vcpkg/asset-cache,,readwrite`, sets a filesystem asset cache, located in

`D:\vcpkg\asset-cache`, with read-write permissions.
::: zone-end

Expand Down
230 changes: 230 additions & 0 deletions vcpkg/consume/third-party-authentication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
---
title: "Authenticate to third-party services"
description: Explain authentication methods for third-party services consumed by vcpkg
author: vicroms
ms.author: viromer
ms.date: 03/03/2025
ms.topic: tutorial
zone_pivot_group_filename: zone-pivot-groups.json
zone_pivot_groups: shell-selections
---
# Authenticate to third-party services
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider breaking this into multiple separate articles, because it seems like this article is serving disjoint audiences.


This article explains how to engage vcpkg features that require you to authenticate to third-party sevices.

## <a name="asset-cache-azure-blob-storage"></a> Asset cache provider: Azure Blob Storage

The [`x-azurl`](../users/assetcaching.md#x-azurl) expects an Azure Blob Storage URL endpoint and a Shared Access Signature
Token (SAS Token) to authenticate when uploading and retrieving asset artifacts.

Read the Azure documentation to [learn more about SAS Tokens](/rest/api/storageservices/delegate-access-with-shared-access-signature).

### Prerequisites

* An active [Azure account](https://azure.microsoft.com/free/cognitive-services/). If you don't have one, you can [create
a free account](https://azure.microsoft.com/free/).
* An [Azure Blob Storage account](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM).
* A [container](/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container) to store and organize the files
within your storage account.

### 1 - Create a Shared Access Signature Token in the Azure portal

1 - Open the [Azure portal](https://portal.azure.com/#home) and navigate to your container.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renders as a paragraph, not as a list.

1 - From the portal home click on Storage Accounts,
2 - Select the account for your asset cache and expand it,
3 - on the right side menu, find your container under Data Storage > Containers and click it to open its own page.
2 - Create an Account key SAS Token for your container.
1 - In the container page, click on Settings > Shared access tokens
2 - Select Account Key as the signing method
3 - Configure the token permissions, ensure that the token has read and write permissions enabled.
4 - Set the expiration for your token, by default, SAS Tokens have a 30-days lifetime, but you can increase it.
5 - Click on Generate SAS token and URL
6 - Copy the Blob SAS token and SAS URL and store them somewhere secure. This information is only displayed once when
creating the token and cannot be viewed again after you leave this page.

### 2 - Configure your asset cache to use your Azure Blob Storage container and SAS token

To use your Azure Blob Container as an asset caching provider, you need to add a configuration string to the [`X_VCPKG_ASSET_SOURCES`
environment variable](../users/config-environment.md#x_vcpkg_asset_sources).

The configuration string follows the format [`x-azurl,<storage URL>,<SAS token>[,<permissions>]`](../users/assetcaching.md#x-azurl).
Where `<storage URL>` is your Azure Storage Account URL, which you an obtain from the SAS URL; `<SAS token>` is the token
generated in the previous step; and `<permissions>` can be one of `read`, `write`, or `readwrite`.

```powershell
set STORAGE_SAS_TOKEN="sp=rw&s...%3D"
set X_VCPKG_ASSET_SOURCES="x-azurl,mystorageaccount.blob.core.windows.net,${STORAGE_SAS_TOKEN},readwrite"
```

Read the [asset caching documentation](../concepts/asset-caching.md) to learn more about configuring asset caching sources.

## Binary cache provider: Azure Artifacts + Universal Packages

**TBD preamble**

### Prerequisites

* [Azure CLI](/cli/azure/install-azure-cli).
* An [Azure DevOps project](/azure/devops/organizations/projects/create-project).
* An [Azure Artifacts feed](/azure/devops/artifacts/start-using-azure-artifacts#create-a-new-feed)

### 1 - Authenticate using Azure CLI

> [!IMPORTANT]
> The Azure CLI executable needs to be part of your `PATH` variable so that vcpkg can find it and use it.

Read the instructions to [authenticate using the Azure command-line tool](/cli/azure/authenticate-azure-cli) using the
method that better suits your use case.

Example: Interactive authentication

```PowerShell
az login
```

vcpkg uses the credentials stored by Azure CLI to access your Azure Artifacts feed.

### 2 - Configure your binary cache to use your Azure Artifacts Universal Packages feed

Add a configuration string to the [`VCPKG_BINARY_SOURCES`](../users/config-environment.md#vcpkg_binary_sources) environment
variable.

The configuration string follows the format [`x-az-universal,<organization>,<project>,<feed>[,<permissions>]`](../reference/binarycaching.md#x-azuniversal).
Where `<organization>` and `<project>` refer to your Azure DevOps project, you can leave the `<project>` argument empty to
use an organization level feed; `<feed>` is your Azure Artiracts feed's name. The `<permissions>` argument is optional,
and can be one of `read`, `write`, or `readwrite`.

```powershell
set VCPKG_BINARY_SOURCES="x-az-universal,vcpkg,demo,example-cache,readwrite"
```

## Binary cache provider: GitHub Actions cache

For Continuous Integration (CI) workflows running in GitHub Actions, using the GitHub Actions cache is a convenient way
to enable binary caching without requiring third-party storage.

### Prerequisites

- A project running a GitHub Actions workflow

### 1 - Export the GitHub Actions Cache environment variable

vcpkg needs the `ACTIONS_CACHE_URL` and `ACTIONS_RUNTIME_TOKEN` to be exported as environment variables in your workflow.

Copy the following lines before invoking vcpkg in your workflow file:

```YAML
- name: Export GitHub Actions cache environment variables
uses: actions/github-script@v7
with:
script: |
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
```

### 2 - Configure your binary cache to use GitHub Actions Cache

Set the [`VCPKG_BINARY_SOURCES`](../users/config-environment.md#vcpkg_binary_sources) environment variable in your workflow
as follows:

```YAML
env:
VCPKG_BINARY_SOURCES: "clear;x-gha,readwrite"
```

The [`x-gha`](../reference/binarycaching.md#github-actions-cache) provider only takes the optional `<permissions>` argument,
its value can be one of`read`, `write`,or `readwrite`.

## Binary cache provider: NuGet

NuGet package feeds are supported by many cloud-storage providers, like [Azure Artifacts](<https://www.visualstudio.com/docs/package/nuget/publish>)
and [GitHub Package registry](<https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-nuget-registry>).
They also provide access control capabilities, which make them suitable for limiting access within your organization or
work group.

### Prerequisites

- A NuGet packages feed

### 1 - Add `nuget.exe` to PATH

vcpkg acquires a copy of the `nuget.exe` executable to use for binary caching operations.

Run the `vcpkg fetch nuget` command to obtain the location of the NuGet executable and add it to path.

Example of `vcpkg fetch nuget` output:

```powershell
vcpkg fetch nuget
C:\dev\vcpkg\downloads\tools\nuget-6.10.0-windows\nuget.exe
```

Modify your `PATH` environment variable to add the directory containing `nuget.exe`.

Example in PowerShell:

```powershell
$env:PATH="C:\dev\vcpkg\downloads\tools\nuget-6.10.0-windows;$env:PATH"
```

### <a name="nuget-add-source"></a> 2 - Add your private feed as a NuGet packages source

> ![NOTE]
> Be aware that encrypted passwords are only supported on Windows. Moreover, they can only be decrypted on the same machine
> and by the same user who originally encrypted them.

Add your feed as a NuGet source and include its authentication credentials.

```PowerShell
nuget.exe sources add -Name <feed-name> -Source <feed-url> -UserName <username> -Password <password>
```

Set `<username>` and `<password>` to your authentication credentials and `<feed-url>` to point at your feed's URL.

> ![WARNING]
> Storing passwords in clear text is strongly discouraged. For more information on managing credentials securely, refer
> to the [security best practies for consuming packages from private feeds](/nuget/consume-packages/consuming-packages-authenticated-feed#security-best-practices-for-managing-credentials).

On non-Windows you may need to add the `-StorePasswordInClearText` option to the command-line.

## Binary cache provider: Nuget + Azure Artifacts

### Create a feed

1. Sign in to your Azure DevOps organization and navigate to your project.
2. Open the "Artifacts" tab.
3. Click the "+ Create Feed" button.
4. Configure your feed's name, visibility and scope. We recommend unchecking the "Include packages from common public sources" for vcpkg binary caching feeds.
5. Click the "Create" button.

:::image type="content" source="../resources/create-nuget-feed.png" alt-text="Screenshot of the Create new feed window":::

### Connect to a feed

1. Sign in to your Azure DevOps organization and navigate to your project.
2. Open the "Artifacts" tab
3. Select your feed from the dropwdown menu.
4. Select "Connect to Feed", and then select Visual Studio from the left pane.
5. Copy the Name and Source values from Machine setup

:::image type="content" source="../resources/connect-to-nuget-feed.png" alt-text="Screenshot of the Machine setup to connect with Visual Studio":::

### Add your feed as a NuGet source

#### Method 1: Create a Personal Access Token to authenticate

1. Sign in to your Azure DevOps organization
2. From your home page, click the Settings icon and then Personal Access Tokens
:::image type="content" source="../resources/ado-options-pat.png" alt-text="Screenshot of the User Settings menu with the Personal Access Tokens option highlighted":::
3. Click the "+New Token" button
4. Set your token's name, expiration date, and scopes. In the Packages scope, make sure to give the token readwrite access. Then click "Create".
:::image type="content" source="../resources/ado-create-pat.png" alt-text="Screenshot of the Create a new personal access tokens window":::
5. Copy the token and store it in a secure location. NOTE: The token cannot be displayed again after you close this window.
:::image type="content" source="../resources/ado-copy-pat.png" alt-text="Screenshot of the window showing the newly created personal access token":::
6. Follow the steps to [add a private feed as a NuGet packages source](#nuget-add-source)
7. Use your Personal Access Token as the password to authenticate to your NuGet feed.

### Set an API Key for your feed

Azure Artifacts requires that packages include an API Key. However, the key can be any arbitrary string.

4 changes: 2 additions & 2 deletions vcpkg/examples/asset-caching-source-nuget.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: How to create a x-script Asset Caching source for NuGet
title: Use a custom script as an asset caching source
description: Learn about how to create a x-script Asset Caching source for NuGet
ms.date: 01/10/2024
ms.topic: tutorial
---

# Creating a x-script Asset Caching source for NuGet
# Use a custom script as an asset caching source

[!INCLUDE [experimental](../../includes/experimental.md)]

Expand Down
Binary file added vcpkg/resources/ado-copy-pat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added vcpkg/resources/ado-create-pat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added vcpkg/resources/ado-options-pat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added vcpkg/resources/connect-to-nuget-feed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added vcpkg/resources/create-nuget-feed.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading