Add support for referenced Git repositories

[mateusrodrigues]

This idea came up when thinking about the debian/eng/test-runner and debian/tests/regular-tests directories and how they essentially duplicate files already available and maintained in separate Git repositories.

So, in order to avoid also having these files in the debian/ source tree, we can make it so Flamenco understands external references to Git repositories in a structured and standardized file.

For example, inside the debian/eng/test-runner directory, we can have a file called flamenco-git.{source_package}.{release} (exact name TBD) that essentially states that that directory should contain the files present in x repository, at y commit or z tag, when building the dist folder of source_package targeting release.

The structure of that file can be something similar to:

repo=https://github.com/canonical/dotnet-test-runner
tag=v1.1.0
commit=eafbc12

Where tag xor commit are present. Exact structure also TBD.

In general, this solves for test-runner and regular-tests, but also allows us to make other components of the packaging external references, such as versionlib, and better separates the concerns of the source-build repo from the maintenance of these other libraries and tools.

As these are very preliminary thoughts, I'm open to discuss if and how we should make this even more general purpose.

[mateusrodrigues]

Thinking more deeply on this, I believe that external reference files could be written in such a way as to allow for more types of references other than Git (as long the tool supports them).

I think the idea is to start by supporting Git, but we could also reference tarballs, for example.

The file could be then called something like flamenco.external.{source_package}.{release}. The content would, then, contain what kind of linkage this is, e.g.

type=git
repo=https://github.com/canonical/dotnet-test-runner
tag=v1.1.0

This could also be a JSON file, in which case we would make use of .NET's own JSON (de)serializer to parse it.

As we were discussing earlier, we could also create a file that describes the orig tarball location, which could be called something like flamenco.orig.{source_package} and points to where we get the orig tarball from. I intentionally removed {release}, as the orig tarball has to be the same across releases for that same package version. Although I feel that this orig tarball discussion should be tracked by another issue 🤔