Now using Travis stages to test, upload artifacts, and publish

[jverbus]

Using Travis stages in this way ensures that all build versions are successful before we publish any artifacts. This is in contrast to the previous version where it was possible, but unlikely, for a new version to be published for the baseline dependencies (Spark 2.3.0 and Scala 2.11.8), even if another build using different Spark/Scala versions failed.

New build pipeline using Travis stages

The first "test" stage runs all build variations in parallel to ensure they succeed.

The second "upload" stage is a kludge to work around a limitation of Shipkit: mockito/shipkit#858. This second stage uploads all artifacts using the current project version defined in version.properties. This only runs if the code has been merged into the master branch.

The third "release" stage runs the Shipkit ciPerformRelease command. This (sadly, but inconsequently) re-uploads the baseline (Spark 2.3.0 and Scala 2.11.8) artifact, but it also pushes another commit to the GitHub repo adding version documentation, bumps the version in the version.properties file, and creates a git tag with the version number. This only runs if the code has been merged into the master branch.

Testing

I did some testing to verify the new stages definitions work. Here is a successful run when on a non-master branch:

https://travis-ci.org/github/linkedin/isolation-forest/builds/691611286

One can see only the "test" stage jobs run.

Here is a run (intentionally killed) that simulates the stages when on master:

https://travis-ci.org/github/linkedin/isolation-forest/builds/691610972

As you can see, the test and upload stages have three jobs as expected, and the release stage has one job.

We won't be able to actually validate with a proper release until it is merged to master.