add to release instructions

documentation/smg/10_build_release_process.md

@@ -1,12 +1,28 @@
-# Build/Release pipeline
+# Build/release process
+
+This document covers the release process for releasing Horace to users.
+Users can obtain versioned releases from these github pages via the Releases
+section on the right sidebar of the Code page of this repository. Releases
+are also propagated to the STFC/ISIS IDAaaS computing service. This document
+describes how to create such releases and make the available. 
+
+In the process of creating such releases, the build pipelines on Jenkins
+are used to compile mex files and test the build, in addition to identifying
+the releases and propagating them. This process is similar to the build
+process used to validate pull requests when merging them to the master branch.
+
+Production of Releases is detailed in the Releases section below.
+
+
+## Build/Release pipeline
 
 Build and test pipelines have been created in ANVIL for the Horace and Herbert
 source.
 
 All users should be using a build that has been built, tested and packaged
 using the build pipelines.
 
-## Master
+### Master
 
 The `master` branch should always be 'releasable' with all tests passing.
 Builds are run on all target platforms to create `.zip`/`.tar.gz` release
@@ -24,7 +40,7 @@ platform.
 This is equivalent to the artifact created by the legacy MATLAB create release
 process.
 
-## Features
+### Features
 
 Feature development and bug fixes should be performed on branches and reviewed
 using a GitHub Pull Request (PR).
@@ -43,19 +59,115 @@ platforms, and the code has been reviewed, the PR can be merged.
 
 <img src="./images/10_developmentworkflow.png">
 
-## Releases
+### Releases
+
+There are 4 levels of release:
+   - major - numbered with a single integer n to give Vn, e.g. V4.
+   - minor - numbered with an additional integer m to give Vn.m e,g. V4.1.
+   - patch - numbered with a further additional integer o to give Vn.m.o`
+e.g V4.1.3.
+   - bugfix - numbered with the patch release number on which it is based,
+plus the sha of the originating commit.
+
+All releases actually use all three integers to
+designate them; however major releases always have m=o=0 and minor releases
+always have o=0.
+
+For the purposes of generating releases, the mechanics of major and minor 
+release production are identical as given below. Patch releases may also use
+this mechanism, but it is not recommended and the preferred mechanism is given 
+below in the following section.
+
+Major and minor releases will always incorporate every change to the master
+branch since the last non-patch release, and for this reason their creation
+should always be well-tested and -documented to tell users what has changed 
+in them. In particular, as IDAaaS users will always pick up the latest
+IDAaaS release rather than consciously choosing to download it over the
+previous release, so they should not be surprised by changes. Patch releases
+generated by the alternative release process will only incorporate changes
+that fix the headlined bugs and will not include other changes that have entered
+the master branch. 
+
+#### Major and minor releases
+
+These should always be planned in advance so that users, particularly those
+on IDAaaS, are aware of what is different from the previous version, as all
+changes on the master branch will be exposed to them.
+
+Creation of a release starts with identification of a commit on the master
+branch as suitable for constructing a release. Based on this commit,
+the Jenkins release branches (one for each build platform) are run to generate
+release artifacts which the user can download from GitHub.
+
+Before the release process is started, the release number has to be manually
+determined from previous release numbers. For example, if the previous release
+was V4.4, next release will be eithr V5.0 (major) or V4.5 (minor).  
+
+This release number should then be inserted into the file `Horace/VERSION` to
+replace the previous version number. It will be used by the Jenkins release
+pipelines to generate the correct release tags. It should be committed to
+the master branch in the normal way via a pull requested. Ideally this PR
+should only include release artifacts (i.e. the release number and the release
+notes) but in the past it has been changed combined with other developments, although
+in those cases a release has not been generated.
+
+A release notes file should also be created in `Horace\documentation` named
+with the release number and suffixed `.md`. See previous examples for what this
+should contain. The contents of this should also be copied for insertion into
+the Deploy build as described below - apparently this is not directly copied
+from the codebase.
+
+With all this in place, the Jenkins release pipelines can be run. It appears
+sufficient to run these on each platform (linux: currently Rocky8 at the time
+of writing, and windows: currently VM-Win10) for the latest Matlab version supported
+in the Jenkins pipelines (currently 2022a). See below for how this appears in 
+IDAaaS.
+
+Taking the `Release-Rocky8-2022a` pipeline as an example, clicking on that link
+on the Jenkins Horace page gets you to the pipeline page. In the left sidebar
+click the Build With Parameters link. On the build page, fill in the following
+fields:
+   - branch: set to `master`
+   - release_type: set to `release`
+   - agent: use preset value (here `rocky8`)
+   - leave remaining fields blank (they will default)
+   - tick box to build documents
+   - leave box to run benchmarks unticked
+and click the `build` button. The pipeline will do the build including running
+the tests. Make sure these have passed. Fix any bugs.
+
+Repeat this for the `VM-Win10` pipeline for your Matlab version.
+
+In order to deploy the code built with these pipelines, you will need 6 pieces
+of information. Two of these are the names of the pipelines (`Release-Rocky8-2022a
+and Release-VM-Win10); two are the build numbers for each platform (seen top left
+of the stage view of the pipeline pages); one is the version number from Horace/VERSION(as noted above);
+one is the commit sha for the commit from which the release was made. This can
+be obtained from the last line of the Last Succesful Artefacts section of the
+pipeline page for either of the build platforms; click on the View link at
+the right of the sha line (both builds will have used the same commit.
+
+Now run the relevant Deploy pipeline. There is only one of these. Go into the 
+pipeline page and from the left sidebar, click the Build With Parameters link. 
+On the build page, fill in the following fields:
+   - tag_sha: sha number noted above
+   - version number: as noted above
+   - release_job_ids: On line 1 copy `Release-Rocky8-2022a`, comma, build number 
+(the number of the build on the first line of Stage View), semi-colon; and
+on line 2 repeat for the `Release-windows-VM-Win10` pipeline.
+   - release body: this should be a copy of the release notes.
+   - draft: (check-box) see below
+   - prerelease: (checkbox) see below
+It is thought that if either of the checkboxes are ticked, then it is
+possible to repeat the release/deploy process; the release tag is not
+finalized.
 
 The Jenkins build artifacts are are not accessible outside of STFC.
 End users will be directed to GitHub to access packaged releases.
 
-Release pipelines have been created on ANVIL,
-these are to be used to create release candidates.
+The deploy pipeline will:
 
-Deploy pipelines have also been created to push release candidates to GitHub.
-
-This will:
-
-- tag the Git revision with the release number (`vm.n.o`)
+- tag the Git revision with the release number (`Vm.n.o`)
 - copy the built artifact to the corresponding GitHub release
 
 The release tag will be an anchor for any subsequently needed release branch
@@ -67,7 +179,7 @@ This will follow [semantic versioning](https://semver.org/)
 and is used in the build process to set the version number in the MATLAB
 and compiled C++ components.
 
-## Deployment on ISIS computational services machines
+### Deployment on ISIS computational services machines
 
 To provide rapid response to user requests,
 bug-fixes and all advantages of constant deployment process,
@@ -96,27 +208,32 @@ the release  process,
 the code, exposed through the links above is checked out and switched to the
 recently released, so users can user the released version of the code.
 
-## Patch releases
-
-The experience shows that the main reasons for the bugs,
-identified by users is small changes in user configuration,
-specific for a user, and correct but #unexpected users operations with the code.
-This is why, the bugfix process in ISIS normally starts from user sharing
-screen with the member of the support team and the demonstration of the issue
-to a member of the support team.
-
-As soon as the issue is confirmed and is obvious that the bug fixing needs
-changes in code base,
-the member of the support team should switch user to the code tree,
-where the changes would not affect other users.
-The symbolic links to `Horace_bugfix` and `Herbert_bugfix` pointing to separate
-clones of git repository are provided for this purpose.
-At release, they are pointing to the same location as regular Horace/Herbert
-branches.
-
-To switch user to these branches, one issues
-`herbert_on(/usr/local/mprogs/Herbert_bugfix/herbert_core)` and
-`horace_on(/usr/local/mprogs/Horace_bugfix/horace_core)` commands.
+#### Patch releases
+
+Patch-fix releases are made because a bug has been identified in the 
+latest major or minor release. To avoid exposing the users to recent
+changes on the master branch, a patch release is made to add only fixes
+for the bug to the latest major/minor release.
+
+Before creating a branch to fix the bug, a patch branch is created if need be,
+If the last release was a major or minor release, then the patch branch is 
+branched from the commit for that release on the master branch. This is to
+give a code branch without any of the changes which have been subsequently
+made on the master branch. If the last release was a patch release then the
+patch branch will already exist. In either case the changes required to fix
+the bug will be merged onto the patch branch, creating the release.
+
+To fix the bug, a normal bugfix PR branch is also made from the last release. 
+Fixes will be made on this bugfix branch and the branch will be merged with
+master using a PR as normal. The branch should NOT be deleted. Instead it
+will be merged locally with the patch release branch, and the patch release
+branch should then be pushed to github. 
+
+Once the fix is in the patch branch, a release should then be created from 
+the merge commit in the same way as for major or minor releases. The release
+build pipelines should build the release and the deploy pipeline should deploy
+it to the github release page. After the release is succesful the bugfix
+branch may be deleted but the patch release branch should be kept.
 
 If parallel execution is necessary for bug-fixing,
 the supporter also needs to do similar changes in worker_4tests script
@@ -169,7 +286,7 @@ Disadvantages:
 
 - slower; have to wait for full build and test pipeline to execute
 
-## Hot-fix Releases
+### Hot-fix Releases
 
 If a particular fix is required quickly on a target system, the full build and
 test process can be bypassed so that facility users do not lose critical