diff --git a/documentation/smg/10_build_release_process.md b/documentation/smg/10_build_release_process.md index 73f254d480..4cf3903a42 100644 --- a/documentation/smg/10_build_release_process.md +++ b/documentation/smg/10_build_release_process.md @@ -1,4 +1,20 @@ -# Build/Release pipeline +# Build/release process + +This document covers the release process for releasing Horace to users. +Users can obtain versioned releases from these github pages via the Releases +section on the right sidebar of the Code page of this repository. Releases +are also propagated to the STFC/ISIS IDAaaS computing service. This document +describes how to create such releases and make the available. + +In the process of creating such releases, the build pipelines on Jenkins +are used to compile mex files and test the build, in addition to identifying +the releases and propagating them. This process is similar to the build +process used to validate pull requests when merging them to the master branch. + +Production of Releases is detailed in the Releases section below. + + +## Build/Release pipeline Build and test pipelines have been created in ANVIL for the Horace and Herbert source. @@ -6,7 +22,7 @@ source. All users should be using a build that has been built, tested and packaged using the build pipelines. -## Master +### Master The `master` branch should always be 'releasable' with all tests passing. Builds are run on all target platforms to create `.zip`/`.tar.gz` release @@ -24,7 +40,7 @@ platform. This is equivalent to the artifact created by the legacy MATLAB create release process. -## Features +### Features Feature development and bug fixes should be performed on branches and reviewed using a GitHub Pull Request (PR). @@ -43,19 +59,115 @@ platforms, and the code has been reviewed, the PR can be merged. -## Releases +### Releases + +There are 4 levels of release: + - major - numbered with a single integer n to give Vn, e.g. V4. + - minor - numbered with an additional integer m to give Vn.m e,g. V4.1. + - patch - numbered with a further additional integer o to give Vn.m.o` +e.g V4.1.3. + - bugfix - numbered with the patch release number on which it is based, +plus the sha of the originating commit. + +All releases actually use all three integers to +designate them; however major releases always have m=o=0 and minor releases +always have o=0. + +For the purposes of generating releases, the mechanics of major and minor +release production are identical as given below. Patch releases may also use +this mechanism, but it is not recommended and the preferred mechanism is given +below in the following section. + +Major and minor releases will always incorporate every change to the master +branch since the last non-patch release, and for this reason their creation +should always be well-tested and -documented to tell users what has changed +in them. In particular, as IDAaaS users will always pick up the latest +IDAaaS release rather than consciously choosing to download it over the +previous release, so they should not be surprised by changes. Patch releases +generated by the alternative release process will only incorporate changes +that fix the headlined bugs and will not include other changes that have entered +the master branch. + +#### Major and minor releases + +These should always be planned in advance so that users, particularly those +on IDAaaS, are aware of what is different from the previous version, as all +changes on the master branch will be exposed to them. + +Creation of a release starts with identification of a commit on the master +branch as suitable for constructing a release. Based on this commit, +the Jenkins release branches (one for each build platform) are run to generate +release artifacts which the user can download from GitHub. + +Before the release process is started, the release number has to be manually +determined from previous release numbers. For example, if the previous release +was V4.4, next release will be eithr V5.0 (major) or V4.5 (minor). + +This release number should then be inserted into the file `Horace/VERSION` to +replace the previous version number. It will be used by the Jenkins release +pipelines to generate the correct release tags. It should be committed to +the master branch in the normal way via a pull requested. Ideally this PR +should only include release artifacts (i.e. the release number and the release +notes) but in the past it has been changed combined with other developments, although +in those cases a release has not been generated. + +A release notes file should also be created in `Horace\documentation` named +with the release number and suffixed `.md`. See previous examples for what this +should contain. The contents of this should also be copied for insertion into +the Deploy build as described below - apparently this is not directly copied +from the codebase. + +With all this in place, the Jenkins release pipelines can be run. It appears +sufficient to run these on each platform (linux: currently Rocky8 at the time +of writing, and windows: currently VM-Win10) for the latest Matlab version supported +in the Jenkins pipelines (currently 2022a). See below for how this appears in +IDAaaS. + +Taking the `Release-Rocky8-2022a` pipeline as an example, clicking on that link +on the Jenkins Horace page gets you to the pipeline page. In the left sidebar +click the Build With Parameters link. On the build page, fill in the following +fields: + - branch: set to `master` + - release_type: set to `release` + - agent: use preset value (here `rocky8`) + - leave remaining fields blank (they will default) + - tick box to build documents + - leave box to run benchmarks unticked +and click the `build` button. The pipeline will do the build including running +the tests. Make sure these have passed. Fix any bugs. + +Repeat this for the `VM-Win10` pipeline for your Matlab version. + +In order to deploy the code built with these pipelines, you will need 6 pieces +of information. Two of these are the names of the pipelines (`Release-Rocky8-2022a +and Release-VM-Win10); two are the build numbers for each platform (seen top left +of the stage view of the pipeline pages); one is the version number from Horace/VERSION(as noted above); +one is the commit sha for the commit from which the release was made. This can +be obtained from the last line of the Last Succesful Artefacts section of the +pipeline page for either of the build platforms; click on the View link at +the right of the sha line (both builds will have used the same commit. + +Now run the relevant Deploy pipeline. There is only one of these. Go into the +pipeline page and from the left sidebar, click the Build With Parameters link. +On the build page, fill in the following fields: + - tag_sha: sha number noted above + - version number: as noted above + - release_job_ids: On line 1 copy `Release-Rocky8-2022a`, comma, build number +(the number of the build on the first line of Stage View), semi-colon; and +on line 2 repeat for the `Release-windows-VM-Win10` pipeline. + - release body: this should be a copy of the release notes. + - draft: (check-box) see below + - prerelease: (checkbox) see below +It is thought that if either of the checkboxes are ticked, then it is +possible to repeat the release/deploy process; the release tag is not +finalized. The Jenkins build artifacts are are not accessible outside of STFC. End users will be directed to GitHub to access packaged releases. -Release pipelines have been created on ANVIL, -these are to be used to create release candidates. +The deploy pipeline will: -Deploy pipelines have also been created to push release candidates to GitHub. - -This will: - -- tag the Git revision with the release number (`vm.n.o`) +- tag the Git revision with the release number (`Vm.n.o`) - copy the built artifact to the corresponding GitHub release The release tag will be an anchor for any subsequently needed release branch @@ -67,7 +179,7 @@ This will follow [semantic versioning](https://semver.org/) and is used in the build process to set the version number in the MATLAB and compiled C++ components. -## Deployment on ISIS computational services machines +### Deployment on ISIS computational services machines To provide rapid response to user requests, bug-fixes and all advantages of constant deployment process, @@ -96,27 +208,32 @@ the release process, the code, exposed through the links above is checked out and switched to the recently released, so users can user the released version of the code. -## Patch releases - -The experience shows that the main reasons for the bugs, -identified by users is small changes in user configuration, -specific for a user, and correct but #unexpected users operations with the code. -This is why, the bugfix process in ISIS normally starts from user sharing -screen with the member of the support team and the demonstration of the issue -to a member of the support team. - -As soon as the issue is confirmed and is obvious that the bug fixing needs -changes in code base, -the member of the support team should switch user to the code tree, -where the changes would not affect other users. -The symbolic links to `Horace_bugfix` and `Herbert_bugfix` pointing to separate -clones of git repository are provided for this purpose. -At release, they are pointing to the same location as regular Horace/Herbert -branches. - -To switch user to these branches, one issues -`herbert_on(/usr/local/mprogs/Herbert_bugfix/herbert_core)` and -`horace_on(/usr/local/mprogs/Horace_bugfix/horace_core)` commands. +#### Patch releases + +Patch-fix releases are made because a bug has been identified in the +latest major or minor release. To avoid exposing the users to recent +changes on the master branch, a patch release is made to add only fixes +for the bug to the latest major/minor release. + +Before creating a branch to fix the bug, a patch branch is created if need be, +If the last release was a major or minor release, then the patch branch is +branched from the commit for that release on the master branch. This is to +give a code branch without any of the changes which have been subsequently +made on the master branch. If the last release was a patch release then the +patch branch will already exist. In either case the changes required to fix +the bug will be merged onto the patch branch, creating the release. + +To fix the bug, a normal bugfix PR branch is also made from the last release. +Fixes will be made on this bugfix branch and the branch will be merged with +master using a PR as normal. The branch should NOT be deleted. Instead it +will be merged locally with the patch release branch, and the patch release +branch should then be pushed to github. + +Once the fix is in the patch branch, a release should then be created from +the merge commit in the same way as for major or minor releases. The release +build pipelines should build the release and the deploy pipeline should deploy +it to the github release page. After the release is succesful the bugfix +branch may be deleted but the patch release branch should be kept. If parallel execution is necessary for bug-fixing, the supporter also needs to do similar changes in worker_4tests script @@ -169,7 +286,7 @@ Disadvantages: - slower; have to wait for full build and test pipeline to execute -## Hot-fix Releases +### Hot-fix Releases If a particular fix is required quickly on a target system, the full build and test process can be bypassed so that facility users do not lose critical