-
Notifications
You must be signed in to change notification settings - Fork 95
Add article: Authenticate to third-party services #420
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 4 commits
7f3048b
de89955
29e357c
95dfd1a
02f467d
6f546d2
cba0986
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -4,22 +4,24 @@ description: Use asset caching with vcpkg to mirror your asset's downloads to im | |||||||||||||
| author: vicroms | ||||||||||||||
| ms.author: viromer | ||||||||||||||
| ms.topic: concept-article | ||||||||||||||
| ms.date: 01/10/2024 | ||||||||||||||
| ms.date: 10/03/2024 | ||||||||||||||
| --- | ||||||||||||||
| # Asset caching with vcpkg | ||||||||||||||
|
|
||||||||||||||
| [!INCLUDE [experimental](../../includes/experimental.md)] | ||||||||||||||
|
|
||||||||||||||
| Asset caching is a feature that allows vcpkg to use alternative download | ||||||||||||||
| endpoints to acquire a package's assets. When asset caching is enabled vcpkg | ||||||||||||||
| attempts to download assets (such as source code and build tools) from a | ||||||||||||||
| configured mirror location. If the asset is not available in the mirror location, vcpkg will download it from its original source and then store a copy in the mirror for future use. | ||||||||||||||
| A fundamental operation of vcpkg is downloading files from the Internet, things like: source code archives, tool executables, | ||||||||||||||
| source code patches, etc. We call anything that is downloaded from the network an **asset**. | ||||||||||||||
|
|
||||||||||||||
| Asset caching can help with these common development scenarios: | ||||||||||||||
| Asset caching allows you to configure alternative download locations (caches) to acquire assets. When enabled, vcpkg first | ||||||||||||||
| attempst to acquire assets from any configured cache location, and falls back to the original download if no cached asset | ||||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. attempts |
||||||||||||||
| is found. Optionally, vcpkg stores the asset in the cache for future use. | ||||||||||||||
|
|
||||||||||||||
| * Improving reliability of continuous integration runs. | ||||||||||||||
| * Mirroring download assets in trusted locations for air-gapped environments. | ||||||||||||||
| * Maintaining continuity of business if third party sources are changed or become unavailable. | ||||||||||||||
| This feature helps mitigate these common development concerns: | ||||||||||||||
|
|
||||||||||||||
| * Reducing dependency from external networks. Possibly, also reducing download times. | ||||||||||||||
| * Mirroring assets in trusted locations for air-gapped environments. | ||||||||||||||
| * Maintaining continuity of business if assets hosted by third-parties are changed or unavailable. | ||||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
|
|
||||||||||||||
| ## Configuration | ||||||||||||||
|
|
||||||||||||||
|
|
@@ -28,15 +30,18 @@ Asset caching is configured via: | |||||||||||||
| * The `X_VCPKG_ASSET_SOURCES` environment variable, or | ||||||||||||||
| * The `--x-asset-sources` command-line option. | ||||||||||||||
|
|
||||||||||||||
| In both cases, the expected value is a semicolon-delimited list of sources. Each | ||||||||||||||
| source has a specific syntax depending on its storage backend (see the [asset | ||||||||||||||
| caching reference documentation](../users/assetcaching.md) to learn how to | ||||||||||||||
| configure asset caching sources). | ||||||||||||||
| The expected value is a semicolon-delimited list of asset caching sources. Read the [asset caching reference | ||||||||||||||
| documentation](../users/assetcaching.md) to learn the configuration syntax. | ||||||||||||||
|
|
||||||||||||||
| Example using Azure Blob Storage as an asset cache: | ||||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
|
|
||||||||||||||
| ```PowerShell | ||||||||||||||
| vcpkg install zlib --x-asset-sources="x-azurl,mystorageaccount.blob.core.windows.net,${env:SAS_TOKEN},readwrite" | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| ## Next step | ||||||||||||||
|
|
||||||||||||||
| Here are some tasks to try next: | ||||||||||||||
|
|
||||||||||||||
| * [Set up an asset cache](../consume/asset-caching.md) | ||||||||||||||
| * [Use a custom script as an asset caching source for | ||||||||||||||
| NuGet](../examples/asset-caching-source-nuget.md) | ||||||||||||||
| * [Use a custom script as an asset caching source](../examples/asset-caching-source-nuget.md) | ||||||||||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -3,7 +3,7 @@ title: "Tutorial: Set up a vcpkg asset cache" | |||||
| description: Learn to set up a local asset cache to mirror download assets. | ||||||
| author: vicroms | ||||||
| ms.author: viromer | ||||||
| ms.date: 09/02/2024 | ||||||
| ms.date: 10/03/2025 | ||||||
| ms.topic: tutorial | ||||||
| #CustomerIntent: As a beginner vcpkg user, I want to set up a local asset cache to mirror downloaded artifacts | ||||||
| zone_pivot_group_filename: zone-pivot-groups.json | ||||||
|
|
@@ -12,10 +12,12 @@ zone_pivot_groups: shell-selections | |||||
| # Tutorial: Set up a vcpkg asset cache | ||||||
|
|
||||||
| All asset caches are configured through the `X_VCPKG_ASSET_SOURCES` environment variable. The value | ||||||
| of `X_VCPKG_ASSET_SOURCES` follows a specific [configuration | ||||||
| syntax](../users/assetcaching.md). | ||||||
| of `X_VCPKG_ASSET_SOURCES` follows a specific [configuration syntax](../users/assetcaching.md). | ||||||
|
|
||||||
| In this tutorial you'll learn how to: | ||||||
| In this tutorial, you'll create a local asset cache location to store downloaded artifacts. | ||||||
|
|
||||||
| you want to use instead, | ||||||
|
vicroms marked this conversation as resolved.
Outdated
|
||||||
| replace the URL and provide a SAS token as the second parameter in the configuration string. | ||||||
|
|
||||||
| > [!div class="checklist"] | ||||||
| > * [Create an asset cache directory](#1---create-an-asset-cache-directory) | ||||||
|
|
@@ -28,15 +30,21 @@ In this tutorial you'll learn how to: | |||||
|
|
||||||
| ## 1 - Create an asset cache directory | ||||||
|
|
||||||
| You can use the Azure Blob Storage backend "[`x-azurl`](../users/assetcaching.md#x-azurl)", to store | ||||||
| assets. Alternatively, you can use filesystem directories to store | ||||||
| your download assets, including network locations. To do so, you need to use the | ||||||
| `file://` protocol in your asset cache URL. | ||||||
| vcpkg offers two types of asset cache providers: | ||||||
|
|
||||||
| * `x-azurl`: which is intended to work with Azure Blob Storage containers, but works just as well with other service providers | ||||||
| and even local or network filesystems. | ||||||
| * `x-script`: which lets you add your own business logic to handle asset caching throug a customizable script or executable. | ||||||
|
|
||||||
| This tutorial uses, `x-azurl` as the provider to store assets in a local directory in your filesystem using the URLS with | ||||||
| the `file://` protocol. | ||||||
|
|
||||||
| > [!NOTE] | ||||||
| > You can use these same steps to create an asset cache with Azure Blob Storage. Replace any URL in the tutorial with the | ||||||
| > URL for your storage and provide a SAS Token for authentication. | ||||||
| > | ||||||
| > See the [instructions for authenticating to an Azure Blob Storage](third-party-authentication.md#asset-cache-azure-blob-storage). | ||||||
|
|
||||||
| In this tutorial, you'll create a local asset cache location to store downloaded | ||||||
| artifacts. If you have an Azure DevOps Blob Storage you want to use instead, | ||||||
| replace the URL and provide a SAS token as the second parameter in the | ||||||
| configuration string. | ||||||
|
|
||||||
| 1. Create a directory to serve as an asset cache location (substitute with any locations of your choosing): | ||||||
|
|
||||||
|
|
@@ -82,7 +90,7 @@ set "X_VCPKG_ASSET_SOURCES=clear;x-azurl,file://D:/vcpkg/asset-cache,,readwrite" | |||||
| This `X_VCPKG_ASSET_SOURCES` configuration adds the following source strings: | ||||||
|
|
||||||
| * `clear`, disables any previously configured asset cache | ||||||
| * `x-azurl,file://C:/vcpkg/asset-cache,,readwrite`, sets a filesystem asset cache, located in | ||||||
| * `x-azurl,file://D:/vcpkg/asset-cache,,readwrite`, sets a filesystem asset cache, located in | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| `D:\vcpkg\asset-cache`, with read-write permissions. | ||||||
| ::: zone-end | ||||||
|
|
||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,230 @@ | ||
| --- | ||
| title: "Authenticate to third-party services" | ||
| description: Explain authentication methods for third-party services consumed by vcpkg | ||
| author: vicroms | ||
| ms.author: viromer | ||
| ms.date: 03/03/2025 | ||
| ms.topic: tutorial | ||
| zone_pivot_group_filename: zone-pivot-groups.json | ||
| zone_pivot_groups: shell-selections | ||
| --- | ||
| # Authenticate to third-party services | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider breaking this into multiple separate articles, because it seems like this article is serving disjoint audiences. |
||
|
|
||
| This article explains how to engage vcpkg features that require you to authenticate to third-party sevices. | ||
|
|
||
| ## <a name="asset-cache-azure-blob-storage"></a> Asset cache provider: Azure Blob Storage | ||
|
|
||
| The [`x-azurl`](../users/assetcaching.md#x-azurl) expects an Azure Blob Storage URL endpoint and a Shared Access Signature | ||
| Token (SAS Token) to authenticate when uploading and retrieving asset artifacts. | ||
|
|
||
| Read the Azure documentation to [learn more about SAS Tokens](/rest/api/storageservices/delegate-access-with-shared-access-signature). | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| * An active [Azure account](https://azure.microsoft.com/free/cognitive-services/). If you don't have one, you can [create | ||
| a free account](https://azure.microsoft.com/free/). | ||
| * An [Azure Blob Storage account](https://portal.azure.com/#create/Microsoft.StorageAccount-ARM). | ||
| * A [container](/azure/storage/blobs/storage-quickstart-blobs-portal#create-a-container) to store and organize the files | ||
| within your storage account. | ||
|
|
||
| ### 1 - Create a Shared Access Signature Token in the Azure portal | ||
|
|
||
| 1 - Open the [Azure portal](https://portal.azure.com/#home) and navigate to your container. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Renders as a paragraph, not as a list. |
||
| 1 - From the portal home click on Storage Accounts, | ||
| 2 - Select the account for your asset cache and expand it, | ||
| 3 - on the right side menu, find your container under Data Storage > Containers and click it to open its own page. | ||
| 2 - Create an Account key SAS Token for your container. | ||
| 1 - In the container page, click on Settings > Shared access tokens | ||
| 2 - Select Account Key as the signing method | ||
| 3 - Configure the token permissions, ensure that the token has read and write permissions enabled. | ||
| 4 - Set the expiration for your token, by default, SAS Tokens have a 30-days lifetime, but you can increase it. | ||
| 5 - Click on Generate SAS token and URL | ||
| 6 - Copy the Blob SAS token and SAS URL and store them somewhere secure. This information is only displayed once when | ||
| creating the token and cannot be viewed again after you leave this page. | ||
|
|
||
| ### 2 - Configure your asset cache to use your Azure Blob Storage container and SAS token | ||
|
|
||
| To use your Azure Blob Container as an asset caching provider, you need to add a configuration string to the [`X_VCPKG_ASSET_SOURCES` | ||
| environment variable](../users/config-environment.md#x_vcpkg_asset_sources). | ||
|
|
||
| The configuration string follows the format [`x-azurl,<storage URL>,<SAS token>[,<permissions>]`](../users/assetcaching.md#x-azurl). | ||
| Where `<storage URL>` is your Azure Storage Account URL, which you an obtain from the SAS URL; `<SAS token>` is the token | ||
| generated in the previous step; and `<permissions>` can be one of `read`, `write`, or `readwrite`. | ||
|
|
||
| ```powershell | ||
| set STORAGE_SAS_TOKEN="sp=rw&s...%3D" | ||
| set X_VCPKG_ASSET_SOURCES="x-azurl,mystorageaccount.blob.core.windows.net,${STORAGE_SAS_TOKEN},readwrite" | ||
| ``` | ||
|
|
||
| Read the [asset caching documentation](../concepts/asset-caching.md) to learn more about configuring asset caching sources. | ||
|
|
||
| ## Binary cache provider: Azure Artifacts + Universal Packages | ||
|
|
||
| **TBD preamble** | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| * [Azure CLI](/cli/azure/install-azure-cli). | ||
| * An [Azure DevOps project](/azure/devops/organizations/projects/create-project). | ||
| * An [Azure Artifacts feed](/azure/devops/artifacts/start-using-azure-artifacts#create-a-new-feed) | ||
|
|
||
| ### 1 - Authenticate using Azure CLI | ||
|
|
||
| > [!IMPORTANT] | ||
| > The Azure CLI executable needs to be part of your `PATH` variable so that vcpkg can find it and use it. | ||
|
|
||
| Read the instructions to [authenticate using the Azure command-line tool](/cli/azure/authenticate-azure-cli) using the | ||
| method that better suits your use case. | ||
|
|
||
| Example: Interactive authentication | ||
|
|
||
| ```PowerShell | ||
| az login | ||
| ``` | ||
|
|
||
| vcpkg uses the credentials stored by Azure CLI to access your Azure Artifacts feed. | ||
|
|
||
| ### 2 - Configure your binary cache to use your Azure Artifacts Universal Packages feed | ||
|
|
||
| Add a configuration string to the [`VCPKG_BINARY_SOURCES`](../users/config-environment.md#vcpkg_binary_sources) environment | ||
| variable. | ||
|
|
||
| The configuration string follows the format [`x-az-universal,<organization>,<project>,<feed>[,<permissions>]`](../reference/binarycaching.md#x-azuniversal). | ||
| Where `<organization>` and `<project>` refer to your Azure DevOps project, you can leave the `<project>` argument empty to | ||
| use an organization level feed; `<feed>` is your Azure Artiracts feed's name. The `<permissions>` argument is optional, | ||
| and can be one of `read`, `write`, or `readwrite`. | ||
|
|
||
| ```powershell | ||
| set VCPKG_BINARY_SOURCES="x-az-universal,vcpkg,demo,example-cache,readwrite" | ||
| ``` | ||
|
|
||
| ## Binary cache provider: GitHub Actions cache | ||
|
|
||
| For Continuous Integration (CI) workflows running in GitHub Actions, using the GitHub Actions cache is a convenient way | ||
| to enable binary caching without requiring third-party storage. | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - A project running a GitHub Actions workflow | ||
|
|
||
| ### 1 - Export the GitHub Actions Cache environment variable | ||
|
|
||
| vcpkg needs the `ACTIONS_CACHE_URL` and `ACTIONS_RUNTIME_TOKEN` to be exported as environment variables in your workflow. | ||
|
|
||
| Copy the following lines before invoking vcpkg in your workflow file: | ||
|
|
||
| ```YAML | ||
| - name: Export GitHub Actions cache environment variables | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || ''); | ||
| core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || ''); | ||
| ``` | ||
|
|
||
| ### 2 - Configure your binary cache to use GitHub Actions Cache | ||
|
|
||
| Set the [`VCPKG_BINARY_SOURCES`](../users/config-environment.md#vcpkg_binary_sources) environment variable in your workflow | ||
| as follows: | ||
|
|
||
| ```YAML | ||
| env: | ||
| VCPKG_BINARY_SOURCES: "clear;x-gha,readwrite" | ||
| ``` | ||
|
|
||
| The [`x-gha`](../reference/binarycaching.md#github-actions-cache) provider only takes the optional `<permissions>` argument, | ||
| its value can be one of`read`, `write`,or `readwrite`. | ||
|
|
||
| ## Binary cache provider: NuGet | ||
|
|
||
| NuGet package feeds are supported by many cloud-storage providers, like [Azure Artifacts](<https://www.visualstudio.com/docs/package/nuget/publish>) | ||
| and [GitHub Package registry](<https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-nuget-registry>). | ||
| They also provide access control capabilities, which make them suitable for limiting access within your organization or | ||
| work group. | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - A NuGet packages feed | ||
|
|
||
| ### 1 - Add `nuget.exe` to PATH | ||
|
|
||
| vcpkg acquires a copy of the `nuget.exe` executable to use for binary caching operations. | ||
|
|
||
| Run the `vcpkg fetch nuget` command to obtain the location of the NuGet executable and add it to path. | ||
|
|
||
| Example of `vcpkg fetch nuget` output: | ||
|
|
||
| ```powershell | ||
| vcpkg fetch nuget | ||
| C:\dev\vcpkg\downloads\tools\nuget-6.10.0-windows\nuget.exe | ||
| ``` | ||
|
|
||
| Modify your `PATH` environment variable to add the directory containing `nuget.exe`. | ||
|
|
||
| Example in PowerShell: | ||
|
|
||
| ```powershell | ||
| $env:PATH="C:\dev\vcpkg\downloads\tools\nuget-6.10.0-windows;$env:PATH" | ||
| ``` | ||
|
|
||
| ### <a name="nuget-add-source"></a> 2 - Add your private feed as a NuGet packages source | ||
|
|
||
| > ![NOTE] | ||
| > Be aware that encrypted passwords are only supported on Windows. Moreover, they can only be decrypted on the same machine | ||
| > and by the same user who originally encrypted them. | ||
|
|
||
| Add your feed as a NuGet source and include its authentication credentials. | ||
|
|
||
| ```PowerShell | ||
| nuget.exe sources add -Name <feed-name> -Source <feed-url> -UserName <username> -Password <password> | ||
| ``` | ||
|
|
||
| Set `<username>` and `<password>` to your authentication credentials and `<feed-url>` to point at your feed's URL. | ||
|
|
||
| > ![WARNING] | ||
| > Storing passwords in clear text is strongly discouraged. For more information on managing credentials securely, refer | ||
| > to the [security best practies for consuming packages from private feeds](/nuget/consume-packages/consuming-packages-authenticated-feed#security-best-practices-for-managing-credentials). | ||
|
|
||
| On non-Windows you may need to add the `-StorePasswordInClearText` option to the command-line. | ||
|
|
||
| ## Binary cache provider: Nuget + Azure Artifacts | ||
|
|
||
| ### Create a feed | ||
|
|
||
| 1. Sign in to your Azure DevOps organization and navigate to your project. | ||
| 2. Open the "Artifacts" tab. | ||
| 3. Click the "+ Create Feed" button. | ||
| 4. Configure your feed's name, visibility and scope. We recommend unchecking the "Include packages from common public sources" for vcpkg binary caching feeds. | ||
| 5. Click the "Create" button. | ||
|
|
||
| :::image type="content" source="../resources/create-nuget-feed.png" alt-text="Screenshot of the Create new feed window"::: | ||
|
|
||
| ### Connect to a feed | ||
|
|
||
| 1. Sign in to your Azure DevOps organization and navigate to your project. | ||
| 2. Open the "Artifacts" tab | ||
| 3. Select your feed from the dropwdown menu. | ||
| 4. Select "Connect to Feed", and then select Visual Studio from the left pane. | ||
| 5. Copy the Name and Source values from Machine setup | ||
|
|
||
| :::image type="content" source="../resources/connect-to-nuget-feed.png" alt-text="Screenshot of the Machine setup to connect with Visual Studio"::: | ||
|
|
||
| ### Add your feed as a NuGet source | ||
|
|
||
| #### Method 1: Create a Personal Access Token to authenticate | ||
|
|
||
| 1. Sign in to your Azure DevOps organization | ||
| 2. From your home page, click the Settings icon and then Personal Access Tokens | ||
| :::image type="content" source="../resources/ado-options-pat.png" alt-text="Screenshot of the User Settings menu with the Personal Access Tokens option highlighted"::: | ||
| 3. Click the "+New Token" button | ||
| 4. Set your token's name, expiration date, and scopes. In the Packages scope, make sure to give the token readwrite access. Then click "Create". | ||
| :::image type="content" source="../resources/ado-create-pat.png" alt-text="Screenshot of the Create a new personal access tokens window"::: | ||
| 5. Copy the token and store it in a secure location. NOTE: The token cannot be displayed again after you close this window. | ||
| :::image type="content" source="../resources/ado-copy-pat.png" alt-text="Screenshot of the window showing the newly created personal access token"::: | ||
| 6. Follow the steps to [add a private feed as a NuGet packages source](#nuget-add-source) | ||
| 7. Use your Personal Access Token as the password to authenticate to your NuGet feed. | ||
|
|
||
| ### Set an API Key for your feed | ||
|
|
||
| Azure Artifacts requires that packages include an API Key. However, the key can be any arbitrary string. | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.