AOSM CLI - Fixing a zip-slip security bug for code that was using tar.extractall() on nfd build command (#9234)

* AOSM CLI - Fixing a zip-slip security bug for code that was using tar.extractall() on `nfd build` command

* fixing tests

* fixing tests

* review updates

---------

Co-authored-by: Daniel Steven <[email protected]>

Author: cliffparsons

src/aosm/HISTORY.rst

@@ -2,9 +2,20 @@
 
 Release History
 ===============
+
+2.0.0b3
+++++++++
+* Fixing a zip-slip security bug for code that was using tar.extractall() on `nfd build` command.
+* This version requires a minimum of 2.70.0 Azure core CLI. See install instructions: https://github.com/MicrosoftDocs/azure-docs-cli/blob/main/docs-ref-conceptual/Latest-version/install-azure-cli.md
+
 2.0.0b2
 ++++++++
-* Remove msrestazure dependency
+* Added sns generate-config, build and deploy commands.
+* Added a check to make sure resource type used in ARM template are in allowed list.
+* Fixed multi NF RETs issue in nsdvs.
+* Fixed: Better exception messages for Azure permissions problems.
+* Fixed: skip-steps help text to reflect current implementation.
+* Added finetuning of parameter exposure with yaml comments for CNF
 
 2.0.0b1
 ++++++++

src/aosm/README.md

@@ -1,6 +1,6 @@
 # Microsoft Azure CLI 'aosm' Extension
 
-This package is for the 'aosm' extension to support Azure Operator Service Manager 
+This package is for the 'aosm' extension to support Azure Operator Service Manager
 functions.
 i.e. `az aosm`
 
@@ -23,16 +23,18 @@ These commands help with the publishing of Network Function Definition and Netwo
 Service Design resources.
 
 ## Overview of function
+
 A generic workflow of using the tool would be:
+
 - Find the pre-requisite items you require for your use-case
 - Run a `generate-config` command to output an example JSON config file for subsequent commands
 - Fill in the config file
 - Run a `build` command to output one or more bicep templates for your Network Function Definition or Network Service Design
 - Review the output of the build command, edit the output as necessary for your requirements
 - Run a `publish` command to:
-    * Create all pre-requisite resources such as Resource Group, Publisher, Artifact Stores, Groups
-    * Deploy those bicep templates
-    * Upload artifacts to the artifact stores
+  - Create all pre-requisite resources such as Resource Group, Publisher, Artifact Stores, Groups
+  - Deploy those bicep templates
+  - Upload artifacts to the artifact stores
 
 ### Pre-requisites
 
@@ -45,14 +47,16 @@ image that would be used for the VNF Virtual Machine.
 #### CNFs
 
 For CNFs you must have these packages installed on the machine you are running the CLI from:
--  `helm` package installed . Instructions on how to do this can be found [here](https://helm.sh/docs/intro/install/).
--  `docker` installed only in some circumstances, those being if the source image is in your local docker repository, or you do not have subscription-wide permissions required to push charts and images. See the remainder of this section for further details. Docker provides packages that easily configure docker on [Windows](https://docs.docker.com/docker-for-windows/), or [Linux](https://docs.docker.com/engine/install/#supported-platforms) systems.
+
+- `helm` package installed . Instructions on how to do this can be found [here](https://helm.sh/docs/intro/install/).
+- `docker` installed only in some circumstances, those being if the source image is in your local docker repository, or you do not have subscription-wide permissions required to push charts and images. See the remainder of this section for further details. Docker provides packages that easily configure docker on [Windows](https://docs.docker.com/docker-for-windows/), or [Linux](https://docs.docker.com/engine/install/#supported-platforms) systems.
 
 For CNFs, you must provide:
-* Helm packages with an associated schema. These files must be on your disk and will be referenced in the `cnf-input.jsonc` config file. 
-* A reference to an existing Azure Container Registry which contains the images for your CNF. Currently, only one ACR and namespace is supported per CNF. The images to be copied from this ACR are populated automatically based on the helm package schema. You must have Reader/AcrPull permissions on this ACR. To use this, fill in `source_registry` and optionally `source_registry_namespace` in the cnf-input.jsonc file.
-* Optionally, you can provide a file (on disk) path_to_mappings which is a copy of values.yaml with your chosen values replaced by deployment parameters, thus exposing them as parameters to the CNF.
-* When filling in the cnf-input.jsonc file, you must list helm packages in the order they are to be deployed. For example, if A must be deployed before B, your cnf-input.jsonc should look something like this:
+
+- Helm packages with an associated schema. These files must be on your disk and will be referenced in the `cnf-input.jsonc` config file.
+- A reference to an existing Azure Container Registry which contains the images for your CNF. Currently, only one ACR and namespace is supported per CNF. The images to be copied from this ACR are populated automatically based on the helm package schema. You must have Reader/AcrPull permissions on this ACR. To use this, fill in `source_registry` and optionally `source_registry_namespace` in the cnf-input.jsonc file.
+- Optionally, you can provide a file (on disk) path_to_mappings which is a copy of values.yaml with your chosen values replaced by deployment parameters, thus exposing them as parameters to the CNF.
+- When filling in the cnf-input.jsonc file, you must list helm packages in the order they are to be deployed. For example, if A must be deployed before B, your cnf-input.jsonc should look something like this:
 
         "helm_packages": [
             {
@@ -73,19 +77,21 @@ For CNFs, you must provide:
             },
 
 ##### Permissions for publishing CNFs
+
 If sourcing the CNF images from an existing ACR, you need to have `Reader`/`AcrPull` permissions
-from this ACR, and ideally, `Contributor` role + `AcrPush` role (or a custom role that allows the `importImage` action and `AcrPush`) over the whole subscription in order to be able to import to the new Artifact store. If you have these, you 
+from this ACR, and ideally, `Contributor` role + `AcrPush` role (or a custom role that allows the `importImage` action and `AcrPush`) over the whole subscription in order to be able to import to the new Artifact store. If you have these, you
 do not need docker to be installed locally, and the image copy is very quick.
 
 If you do not have the subscription-wide permissions then you can run the `az aosm nfd publish` command using the `--no-subscription-permissions` flag to pull the image to your local machine and then push it to the Artifact Store using manifest credentials scoped only to the store. This requires docker to be installed locally.
 
 #### NSDs
-For NSDs, you will need to have a Resource Group with a deployed Publisher, Artifact Store, Network Function Definition and Network Function Definition Version. You can use the `az aosm nfd` commands to create all of these resources.
 
+For NSDs, you will need to have a Resource Group with a deployed Publisher, Artifact Store, Network Function Definition and Network Function Definition Version. You can use the `az aosm nfd` commands to create all of these resources.
 
 ### Command examples
 
 #### Before you start
+
 `az login` to login to the Azure CLI.
 `az account set --subscription <subscription>` to choose the subscription you will work on.
 
@@ -104,7 +110,7 @@ Create an example config file for building a definition
 
 `az aosm nfd generate-config`
 
-This will output a file called `cnf-input.jsonc` which must be filled in. 
+This will output a file called `cnf-input.jsonc` which must be filled in.
 Once the config file has been filled in the following commands can be run.
 
 Build an nfd definition locally
@@ -115,7 +121,6 @@ Publish a pre-built definition
 
 `az aosm nfd publish --build-output-folder cnf-cli-output`
 
-
 #### NSDs
 
 Get help on command arguments
@@ -129,7 +134,7 @@ Create an example config file for building a definition
 
 `az aosm nsd generate-config`
 
-This will output a file called `nsd-input.jsonc` which must be filled in. 
+This will output a file called `nsd-input.jsonc` which must be filled in.
 Once the config file has been filled in the following commands can be run.
 
 Build an nsd locally
@@ -140,12 +145,11 @@ Publish a pre-built design
 
 `az aosm nsd publish --build-output-folder nsd-cli-output`
 
-
 ## Bug Reporting
 
 It would be much appreciated if you could report these so that we're aware of them!
 
-Please see [Logging](#logging) for how to view and collect logs. 
+Please see [Logging](#logging) for how to view and collect logs.
 
 Please describe what you are doing and if possible provide the input and output files.
 
@@ -157,20 +161,25 @@ CLI issues should be tagged and triaged as UX bugs.
 ## Logging
 
 The CLI uses the standard Azure CLI logging mechanism. To enable logging to the console, you can use the following flags depending on the desired level of logging:
+
 - `--verbose` - This flag changes the logging level to Info and above.
 - `--debug` - This flag changes the logging level to Debug and above.
 - `--only-show-errors` - This flag changes the logging level to Error only, suppressing Warning.
 
 It is also possible to enable logging to file by running the following command:
+
 ```
 az config set logging.enable_log_file=true
 ```
-This will create a log file in the `~/.azure/logs` directory. 
+
+This will create a log file in the `~/.azure/logs` directory.
 
 **Note:** The above command will enable logging for all Azure CLI commands until the logging is disabled again by the user. Not disabling file logging could slow down the performance of the CLI. To disable file logging, run the following command:
+
 ```
 az config set logging.enable_log_file=false
 ```
 
 ## Development
+
 Information about setting up and maintaining a development environment for this extension can be found [here](https://eng.ms/docs/strategic-missions-and-technologies/strategic-missions-and-technologies-organization/azure-for-operators/aiops/aiops-orchestration/aosm-product-docs/processes/cli_contributing).

src/aosm/azext_aosm/_help.py

@@ -121,3 +121,26 @@
       - name: Publish a Network Service Design.
         text: az aosm nsd publish --build-output-folder my-nsd-output-folder
 """
+
+helps[
+    "aosm sns"
+] = """
+    type: group
+    short-summary: Manage AOSM Site Network Services.
+    long-summary: |
+      A Site Network Service (SNS) is a collection of network functions along with platform that come together to offer a service..
+"""
+
+helps[
+    "aosm sns generate-config"
+] = """
+    type: command
+    short-summary: Generate configuration file for building an AOSM SNS.
+    long-summary: |
+      Generates a configuration file that you can use to build an AOSM Site Network Service (SNS). The configuration file is a JSONC file that contains the required parameters for building the SNS. You must complete the configuration file with your specific values before building the SNS.
+    examples:
+      - name: Generate a configuration file for a Site Network Service.
+        text: az aosm sns generate-config
+      - name: Generate a configuration file for a Site Network Service and write to a specific file.
+        text: az aosm sns generate-config --output-file my-sns-input-config.jsonc
+"""

src/aosm/azext_aosm/_params.py

@@ -28,11 +28,9 @@ def load_arguments(self: AzCommandsLoader, _):
     nf_skip_steps = get_enum_type(
         [BICEP_PUBLISH, ARTIFACT_UPLOAD, IMAGE_UPLOAD, HELM_TEMPLATE]
     )
-    ns_skip_steps = get_enum_type([BICEP_PUBLISH, ARTIFACT_UPLOAD])
 
     # Set the argument context so these options are only available when this specific command
     # is called.
-
     with self.argument_context("aosm nfd") as c:
         c.argument(
             "definition_type",
@@ -77,11 +75,10 @@ def load_arguments(self: AzCommandsLoader, _):
             "skip",
             arg_type=nf_skip_steps,
             help=(
-                "Optional skip steps. 'bicep-publish' will skip deploying the bicep "
-                "template; 'artifact-upload' will skip uploading any artifacts; "
-                "'image-upload' will skip uploading the VHD image (for VNFs) or the "
-                "container images (for CNFs); 'helm-template' will skip templating the "
-                "helm charts (for CNFs)."
+                "Optional skip step. Providing the string 'helm-template' will skip "
+                "templating the helm charts (for CNFs), but is very likely to result in "
+                "a broken deployment as image versions will not be parsed. Intended only for "
+                "temporarily unblocking during development."
             ),
         )
         c.argument(
@@ -118,7 +115,9 @@ def load_arguments(self: AzCommandsLoader, _):
                 " the required parameters for building the NSD."
             ),
         )
-        c.argument("skip", arg_type=ns_skip_steps, help="Optional skip steps")
+        # NSD skip steps are not currently implemented. This is a record of how to add them.
+        # You also need to add the `skip` parameter in the relevant function in custom.py.
+        # c.argument("skip", arg_type=ns_skip_steps, help="Optional skip steps")
         c.argument(
             "clean",
             arg_type=get_three_state_flag(),
@@ -145,3 +144,41 @@ def load_arguments(self: AzCommandsLoader, _):
                 "Requires Docker to be installed locally."
             ),
         )
+
+    with self.argument_context("aosm sns") as c:
+        c.argument(
+            "output_file",
+            options_list=["--output-file"],
+            help="The name of the output file to write the generated config text to.",
+            required=False,
+        )
+        c.argument(
+            "config_file",
+            options_list=["--config-file", "-f"],
+            type=file_type,
+            completer=FilesCompleter(allowednames="*.jsonc"),
+            help=(
+                "The path to the configuration file. This is a JSONC file that contains"
+                " the required parameters for building the NSD."
+            ),
+        )
+        c.argument(
+            "build_output_folder",
+            options_list=["--build-output-folder", "-b"],
+            type=file_type,
+            completer=FilesCompleter(allowednames="*.json"),
+            help="Path to the folder to deploy, created by the build command.",
+        )
+        c.argument(
+            "no_subscription_permissions",
+            options_list=["--no-subscription-permissions", "-u"],
+            completer=FilesCompleter(allowednames="*.json"),
+            help=(
+                "Pass this flag if you do not have permission to import to the "
+                "Publisher subscription (Contributor role + AcrPush role, or a "
+                "custom role that allows the importImage action and AcrPush over the "
+                "whole subscription). This means that the image artifacts will be "
+                "pulled to your local machine and then pushed to the Artifact Store. "
+                "Requires Docker to be installed locally."
+            ),
+        )

src/aosm/azext_aosm/_validators.py

@@ -9,7 +9,7 @@ def example_name_or_id_validator(cmd, namespace):
     # See:
     # https://github.com/Azure/azure-cli/blob/dev/doc/authoring_command_modules/authoring_commands.md#supporting-name-or-id-parameters
     from azure.cli.core.commands.client_factory import get_subscription_id
-    from azure.mgmt.core.tools import is_valid_resource_id, resource_id
+    from msrestazure.tools import is_valid_resource_id, resource_id  # pylint: disable=import-error
 
     if namespace.storage_account:
         if not is_valid_resource_id(namespace.RESOURCE):

src/aosm/azext_aosm/azext_metadata.json

@@ -1,4 +1,4 @@
 {
     "azext.isPreview": true,
-    "azext.minCliCoreVersion": "2.54.0"
+    "azext.minCliCoreVersion": "2.70.0"
 }