Skip to content

Generate Python Mgmt SDK from Typespec

Jump to bottom
Yuchao Yan edited this page Dec 12, 2024 · 9 revisions

Getting Started - Generating Management SDK with MPG and Typespec

Prerequisites

Setting Up Your Basic Environment

  • Python 3.8 or newer is required

    • Download for Windows
    • For Linux:
      • Install Python 3 with sudo apt install python3
      • Install Python 3 Pip with sudo apt install python3-pip
      • If needed, install Python 3.{?}-venv with sudo apt install python3.{?}-venv

  • Node.js 18.3 LTS or newer is required

Preparing Your Repo

Identifying tspconfig.yaml

Find the tspconfig.yaml file for your package in the Rest repo. Ensure there's a configuration for the Python SDK similar to the one shown below and in this example

parameters:  
  "service-dir":  
    default: "sdk/SERVICE_DIRECTORY_NAME"  
options:  
  "@azure-tools/typespec-python":  
    package-dir: "azure-mgmt-PACKAGE_NAME"  
    package-name: "{package-dir}"  
    generate-test: true  
    generate-sample: true  
    flavor: "azure"

Note: Please replace SERVICE_DIRECTORY_NAME/PACKAGE_NAME with the actual values as needed.

Installing Necessary Dependencies

  1. Install typespec-client-generator-cli globally:
npm install -g @azure-tools/typespec-client-generator-cli
  1. Create Python virtual environments and activate it:
PS C:\dev\azure-sdk-for-python> python -m venv .venv  
PS C:\dev\azure-sdk-for-python> .\.venv\Scripts\Activate.ps1 (Windows)  
   /C/dev/azure-sdk-for-python> source .venv/bin/activate (Linux)
  1. Install Python dependencies:
(.venv) PS C:\dev\azure-sdk-for-python> python .\scripts\dev_setup.py -p azure-core  
(.venv) PS C:\dev\azure-sdk-for-python> pip install tox
(.venv) PS C:\dev\azure-sdk-for-python> pip install setuptools

Generate SDK

  1. Create a local json file named generatedInput.json outside SDK repo with content similar to that shown below
   {
     "dryRun": false,
     "specFolder": "LOCAL_AZURE-REST-API-SPECS_REPO_ROOT", // e.g. "C:/dev/azure-rest-api-specs"
     "headSha": "SHA_OF_AZURE-REST-API-SPECS_REPO", // use ' git rev-parse HEAD ' on the local rest repo root 
     "repoHttpsUrl": "https://github.com/Azure/azure-rest-api-specs",
     "relatedTypeSpecProjectFolder": [
       "specification/SERVICE_DIRECTORY_NAME/PACKAGE_DIRECTORY_NAME/" // e.g specification/contosowidgetmanager/Contoso.Management
     ]
   }
  1. Run command to generate SDK
(.venv) PS C:\dev\azure-sdk-for-python> python -m packaging_tools.sdk_generator ..\generatedInput.json ..\temp.json
(.venv) PS C:\dev\azure-sdk-for-python> python -m packaging_tools.sdk_package ..\temp.json ..\generatedOutput.json
  1. View information about the generated SDK in generateOutput.json

Necessary info about python SDK

Before update the generated SDK code, it is better to know some key concepts about Python SDK. If you already know, please skip this part.

Project service name and package name

Two key pieces of information for your project are the service_name and package_name.

The service_name is the short name for the Azure service. The service_name should match across all the SDK language repos and should be name of the directory in the specification folder of the Rest repo that contains the REST API definition file. An example is Service Bus, whose API definitions are in the specification/servicebus folder of the Rest repo, and uses the service_name "servicebus". Not every service follows this convention, but it should be the standard unless there are strong reasons to deviate.

In Python, a project's package name is the name used to publish the package in PyPI. For management plane libraries, the package_name could be just azure-mgmt-{service_name}. An example is "azure-mgmt-servicebus".

Project folder structure

Before we start, we probably should get to know the project folder for SDK repo.

Normally, the folder structure would be something like:

  • sdk/{service_name}/{package_name}: the PROJECT_ROOT folder
  • /tests: folder of manual test files
  • /generated_tests: folder of auto-generated test files
  • /samples: folder of manual sample files
  • /generated_samples: folder of auto-generated sample files
  • azure-mgmt-{module_name}: package name. Usually, package name is same with part of ${PROJECT_ROOT} folder. After release, you can find it in pypi. For example: you can find azure-mgmt-webpubsub in pypi.
  • there are also some other files (like setup.py, README.md, etc.) which are necessary for a complete package.

More details on the structure of Azure SDK repos is available in the Azure SDK common repo.

What to do after generating the SDK code

The generated code is not enough to release at once and you need to update it for better usage experience. Please follow the steps to check the code:

How to write README.md

README.md is very useful doc for customers to learn more about your service, which appeals to users to use your service and SDK. After generation, please write more info in README.md and you could refer to webpubsub

How to write tests

You should write tests that ensure all APIs fulfil their contract and algorithms work as specified. This is a requirement of the Azure SDK Guidelines.

See the Management Plane Codegen Quick Start for Test for information on how to write and run tests for the Python SDK.

How to write sample

Samples can make it quicker and easier for customers to use the package. It could be simple which just shows how to use the package; Or it could be a complex scenario example for customers to create their own service quickly.

You should add samples in the samples directory under ${PROJECT_ROOT}. Each sample should have its own folder and contain a Readme.md(example) which introduces simply how to run the sample.

If you want to run sample with dev mode, just run pip install -e .(detail doc) under ${PROJECT_ROOT} instead of pip install azure.myservice.

How to create package

If you want to create a private package, go to ${PROJECT_ROOT} folder and run:

pip install wheel
python setup.py bdist_wheel

Create/Update the ci.yml

Now, if everything looks good to you, you can submit a PR in azure-sdk-for-python repo with all the changes you made above. Before you do that, you need to add/update the ci.yml file. Depends on whether there's already one in sdk/{servicename} folder.

If there's no such file then you can add the following template.

# DO NOT EDIT THIS FILE
# This file is generated automatically and any changes will be lost.

trigger:
  branches:
    include:
    - main
    - hotfix/*
    - release/*
    - restapi*
  paths:
    include:
    - sdk/{service_name}/

pr:
  branches:
    include:
    - main
    - feature/*
    - hotfix/*
    - release/*
    - restapi*
  paths:
    include:
    - sdk/{service_name}/

extends:
  template: ../../eng/pipelines/templates/stages/archetype-sdk-client.yml
  parameters:
    ServiceDirectory: {service_name}
    Artifacts:
    - name: { package_name }
      safeName: { safeName }

Please change the {service_name}, {package_name} and {safeName} into yours.

If there's already a ci.yml file in your project path. then the only thing you need to do is to add the package_name and safeName of yours into that ci.yml.

Usually, safeName is derived from package name by removing all -. For example: webpubwub.

Configuring CODEOWNERS

If a new module group is being added, which requires adding a new sub-folder directory under sdk, you'll want to add a new entry to CODEOWNERS. This allows GitHub to automatically assign the specified users as reviewers to all PRs opened against the new module group.

Release

Here is the Release Checklist you should know before release. Once ready, make the PR to https://github.com/Azure/azure-sdk-for-python. After the PR is merged, it is time to release package.

Make PR

DPG can only help you generate SDK code, there is something you need to update manually:

CHANGELOG.md

CHANGELOG can help customers know the change of new version quicky, so you need to update the it according to the change of this new version. It is also necessary to update release date like 1.0.0b1 (2022-02-02)(rough time is fine and no need to be very accurate).

Version Number

You shall update the version number according to package version rule.

Fix CI for PR

You may meet the CI failures after submitting the PR, so please refer to How-to-fix-CI to fix it.

Clone this wiki locally