feat(build): consume package.xml manifest for ROS

[remimimimimi]

Overview

Allow providing a --path option that you can point to a package to be built.

Currently supported file formats are: recipe.yaml (rattler-build backend), package.xml ( ros backend ) and pixi.toml itself.

In this case, from your current/or upper workspace channels will be taken, and our DiscoveryBackend will try to find your package from --path.

note: package.xml should be explicitly set in --path.

How to test it?

You can do pixi run install-debug-as pixi-xml

and then in the root of https://github.com/ruben-arts/ros_workspace

run pixi-xml build --package-manifest src/navigator/package.xml

[ruben-arts]

Generally looks good, I've not tested it yet, but I have a few changes I think would make this even better.

[ruben-arts]

I would approve with my changes added

[nichmor]

The error messages:

[Hofer-Julian]

@nichmor we should add a test for this at https://github.com/prefix-dev/pixi-build-testsuite

[Hofer-Julian]

@nichmor @remimimimimi @ruben-arts I don't really get why you introduce --package-manifest here. I would have expected that you just use --manifest-path here, and mean a package manifest instead of a workspace manifest. At the moment, --manifest-path is an accepted option by pixi build, but I don't understand how this could be used in a sensible way.

[ruben-arts]

@Hofer-Julian because that then points to the workspace. As it always needs a workspace manifest

[Hofer-Julian]

@Hofer-Julian because that then points to the workspace. As it always needs a workspace manifest

Sure, but the way that workspace manifests are detected is well-defined.
You first check in the package manifest directory, and then you go a few directories up until you find a workspace manifest.

There's no need for something like this:

pixi build --manifest-path /path/to/workspace-manifest \
           --package-manifest /path/to/package-manifest

As a user, this would make the most sense to me, since I am already used to this flag:

pixi build --manifest-path /path/to/package-manifest

[nichmor]

@Hofer-Julian because that then points to the workspace. As it always needs a workspace manifest

Sure, but the way that workspace manifests are detected is well-defined. You first check in the package manifest directory, and then you go a few directories up until you find a workspace manifest.

There's no need for something like this:

pixi build --manifest-path /path/to/workspace-manifest \
           --package-manifest /path/to/package-manifest

As a user, this would make the most sense to me, since I am already used to this flag:

pixi build --manifest-path /path/to/package-manifest

--manifest-path is treated like: hey, this package.xml is your workspace manifest. In our case, package-xml is a shortcut for the package section in some pixi.toml - we still need to have a workspace manifest from which we will get channels for ros backend.

If we allow manifest-path to point to a package.xml, we will not have a workspace to anchor to ( even if we can create one on the fly), and we don't have any channels to read from.

[Hofer-Julian]

@nichmor and I discussed this. Tbh, I still don't really understand why we'd support specifying a different workspace manifest than the one that belongs to a package manifest, but I appreciate that I might be missing use cases.

What is really important to me though that we provide good defaults.

So if I am at a directory that contains a workspace manifest and I do the following

pixi build --package-manifest ../project/recipe/recipe.yaml

It should find the workspace manifest of ../project not the one in my current working directory.

You can then override this by specifying --manifest-path:

pixi build --package-manifest ../project/recipe/recipe.yaml --manifest-path .

We should also add a couple of tests to test the behaviour

[ruben-arts]

Sorry for all the confusion that I created with my initial proposal, @Hofer-Julian explained his point well and we discussed the following changes.

Let's remove --manifest-path from pixi build as there doesn't seem to be a request to define a workspace manifest path yet.

We should change --package-manifest into --path as the "package manifest path" which aligns with pixi global install and the TOML version of a dependency. Allowing us to extend this in the future with --git and --url.

So it would look like this with the current supported design:

pixi build --path src/pixi.toml
pixi build --path src/pyproject.toml
pixi build --path src/recipe.yaml
# or for any of the above
pixi build --path src/

# For package.xml we only support direct definition
# Similar to the TOML definition:
# package = { path = "src/package.xml" }
pixi build --path src/package.xml

We don't want to extend this list of directly supported package manifests, but instead add a --backend flag
So in the future it should become this for any package of any type:

pixi build --path cargo.toml --backend pixi-build-rust
pixi build --path pyproject.toml --backend pixi-build-python
pixi build --path package.xml  --backend pixi-build-ros

[nichmor]

temporary blocked by prefix-dev/pixi-build-backends#451

[nichmor]

made the changes - and added tests for multiple scenarios ( implicit /explicit recipe.yaml, and implicit/explicit package.xml)

[ruben-arts]

It doesn't work when I do:

autoware_launch/autoware_launch on  main [!] 
❯ ls
CHANGELOG.rst  CMakeLists.txt  README.md  autoware_launch.drawio.svg  config  launch  package.xml  rviz  test

autoware_launch/autoware_launch on  main [!] 
➜ pixi build --path package.xml
Error:   × unable to canonicalize ''
  ╰─▶ No such file or directory (os error 2)

it does work with ./package.xml

[nichmor]

It doesn't work when I do:

autoware_launch/autoware_launch on  main [!] 
❯ ls
CHANGELOG.rst  CMakeLists.txt  README.md  autoware_launch.drawio.svg  config  launch  package.xml  rviz  test

autoware_launch/autoware_launch on  main [!] 
➜ pixi build --path package.xml
Error:   × unable to canonicalize ''
  ╰─▶ No such file or directory (os error 2)

it does work with ./package.xml

tested and fixed the problem with referencing the same package.xml

Also added new test cases here: prefix-dev/pixi-build-testsuite#94

[nichmor]

temporary until we merge the new test-cases

[Hofer-Julian]

Our current workflow is that we only verify that things work in unison by adding a .env.ci file at the testsuite repo. You can revert this change

[ruben-arts]

It works for me now! Thanks