Dataverse Configuration Migration Tool

This repository contains a custom .NET CLI tool designed to export and import configuration data into Microsoft Dataverse environments. It streamlines the migration of configuration data, supports schema validation, and offers extensibility for advanced scenarios.

Download latest release

Get latest version of the tool built on this release

Note

• Interactive Login is now available with the CLI tool. The use of service principal is no more mandatory but still recommanded for automation scenarios.
• To automate the use of the tool, let's say within a pipeline (e.g Github Action,Azure Devops Pipeline),appsettings.Production.json will need to be setup manually with your azure service principal credentials.
• Quick Guide to create an azure service principal

Why ❓

Configuration Migration Tool and the PowerPlatform Cli Tool (pac data import verb) seem to have it's limitations when automating in ci/cd. Also, these two only works on a windows machine.
This new tool enables you to:

• customize/extend the tool to your needs
• use in CD pipeline easily
• runs on windows and linux

⭐Features⭐

🆕 ✔️ Interactive Login is now available by using --il parameter. Service Principal is no more required.🆕
🆕 ✔️ Data export now supports a new parameter --AllowEmptyFields or --enable-empty-fields to export empty fields. Useful to clear values on target environments 🆕
✔️ Import configuration data into Dataverse
✔️ Export configuration data from Dataverse
✔️ Schema validation and rule-based checks
✔️ Support for special characters and self-hierarchy imports
✔️ Extensible for custom validation and reporting
✔️ Many to Many Relationships supported
✔️ Owner,Team And Business Unit field is now supported. If by Unique Id is not found in target env, it matches by name otherwise it ignores the field. Caching is used to avoid unnecessary calls to dataverse.
✔️ Supports schemas and data files generated from :

• Configuration Migration Tool for XrmToolBox
• PowerPlatform CLI Export
• Configuration Migration tool

❌Field types not supported:

• MultiSelectOptionsets
• Image
• File

Upcoming features 🔜

🔜 Supports for multiselect optionset

Important

• Data import/export features for XrmToolBox is no longer planned.
  • Since the cli tool is made with .Net Core and XTB Plugins is in .Net Framework, It's really hard to keep a sharable codebase and retricts the usage of some modern libraries.

Getting Started

Prerequisites

• .NET 8.0 SDK
• Access to a Microsoft Dataverse environment

🛠️Building the Tool🛠️

1. Clone the repository:

   git clone https://github.com/dotnetprog/dataverse-configuration-migration-tool.git
   cd dataverse-configuration-migration-tool/src/Dataverse.ConfigurationMigrationTool/Dataverse.ConfigurationMigrationTool.Console

2. Build the project:

   dotnet build

Usage

Before running the tool, set your dataverse variables securely using dotnet user-secrets:

For Service principal

cd src/Dataverse.ConfigurationMigrationTool/Dataverse.ConfigurationMigrationTool.Console
dotnet user-secrets set "Dataverse:ClientId" "<your-client-id>"
dotnet user-secrets set "Dataverse:ClientSecret" "<your-client-secret>"
dotnet user-secrets set "Dataverse:Url" "<your-env-url>"

For Interactive Login

cd src/Dataverse.ConfigurationMigrationTool/Dataverse.ConfigurationMigrationTool.Console
dotnet user-secrets set "Dataverse:Url" "<your-env-url>"

Run the CLI tool with the required arguments (no need to pass clientId or clientSecret on the command line):

example

dotnet run --environment DOTNET_ENVIRONMENT=Development --project Dataverse.ConfigurationMigrationTool.Console -- import --data "path/to/data.xml" --schema "path/to/schema.xml"

💻 Command Line Arguments 💻

Verb: import

• --data : Path to the data xml file, you can use export-data command or the microsoft tool (see last section).(Absolute or relative path)
• --schema : Path to the schema XML file (Absolute or relative path)
• 🆕--il or -il : Optional flag to use interactive login (user will be prompted to log in) instead of service principal. Useful for local testing/execution.

Note

using interactive login to import data will request user to login twice as it instantiates two different connections to dataverse for performance purposes.

Example

using dotnet run from the solution folder (.sln)

dotnet run --environment DOTNET_ENVIRONMENT=Development --project Dataverse.ConfigurationMigrationTool.Console -- import --data "C:\temp\data.xml" --schema "C:\temp\data_schema.xml" --il

using directly the tool executable

cd path/to/tool__executable_folder
Dataverse.ConfigurationMigrationTool.Console.exe import --data "C:\temp\data.xml" --schema "C:\temp\data_schema.xml" --il

Verb: export-data

• --schema : Path to the schema XML file (Absolute or relative path)
• --output : output file path to save the exported data. This file can be used for the import command. (Absolute or relative path)
• 🆕--AllowEmptyFields or --enable-empty-fields : Optional flag to include fields with empty values in the export. Useful for clearing values in the target environment.
• 🆕--il or -il : Optional flag to use interactive login (user will be prompted to log in) instead of service principal. Useful for local testing/execution.

Example

using dotnet run from the solution folder (.sln)

dotnet run --environment DOTNET_ENVIRONMENT=Development --project Dataverse.ConfigurationMigrationTool.Console -- export-data --schema "C:\temp\data_schema.xml" --output "C:\temp\exported_data.xml" --enable-empty-fields --il

using directly the tool executable

cd path/to/tool__executable_folder
Dataverse.ConfigurationMigrationTool.Console.exe export-data --schema "C:\temp\data_schema.xml" --output "C:\temp\exported_data.xml" --enable-empty-fields --il

Tip

• To use service principal credentials, the interactive login flag --il or -il must be omitted.
• If you use the executable directly, you must edit the appsettings.Production.json file with your service principal credentials
• If you use dotnet run, you can set the secrets using dotnet user-secrets as shown above or edit the appsettings.Development.json file.
• you can also set environment variables Dataverse__ClientId, Dataverse__ClientSecret and Dataverse__Url to override the settings in the json files.
• you can also use these commandline arguments --Dataverse:ClientId, --Dataverse:ClientSecret and --Dataverse:Url to override the settings in the json files.

🤝 Contributing 🤝

Contributions are welcome! To get started:

1. Fork the repository and create a new branch for your feature or bugfix.
2. Make your changes and add tests as appropriate.
3. Ensure all tests pass:

   dotnet test ../Console.Tests/Dataverse.ConfigurationMigrationTool.Console.Tests.csproj

4. Submit a pull request with a clear description of your changes.

Please review open issues and the todo list below for ideas on what to contribute.

🚀Continuous Delivery🚀

This repository includes example GitHub Actions workflows for building, testing, and deploying the tool:

• main-pipeline.yml: Handles build, test, artifact publishing, and triggers deployment workflows.
• cd-pipeline.yml: Example deployment workflow for importing configuration data into a Dataverse environment using published artifacts and environment secrets.

You can use these workflows as a starting point for your own CI/CD automation. See the workflow files for details on environment variables, secrets, and deployment steps.

</> Generating Schema and Data Files </>

🆕 You can now create your schema definition file from XrmToolBox using Configuration Migration Tool for XrmToolBox🆕

To use the import or export-data command, you need a schema file and/or a data file. These can be generated from your Dataverse environment using the official Configuration Migration tool. For detailed instructions, refer to the Microsoft documentation:

• Create a schema and export configuration data

This guide explains how to:

• Create a schema file that defines the data to export
• Export configuration data from your environment

Note

the data_schema.xml and data.xml will be inside the exported zip. You need to extract those files and use them to import command as described above.