Skip to content

Commit

Permalink
eclipse-n4jsGH-1503: As a user I want the N4JS IDE and headless build…
Browse files Browse the repository at this point in the history 
…er to report its version (part 2) (eclipse-n4js#1516)

* use a better default for the language version

* temporary: disable publishing of n4js-libs in publish-n4js-libs.sh

* move file 'version.json' from n4js-libs to root folder

* rename products + copy them to global target folder in repo root folder

* support for debug publishing in eclipse-publish.jenkinsfile

* disallow use of dist-tag 'latest' when not on branch master

* include version number in files names of n4jsc.jar and products

* temporary: configure a dist-tag in version.json

* assert headless builder is reporting the correct version

* clean up names of folders inside product .zip files

* update documentation

* further update documentation

* revert temporary changes used only for testing the publish builds
  • Loading branch information
@mor-n4
mor-n4 authored Oct 17, 2019
1 parent 3919814 commit e784c75
Show file tree
Hide file tree
Showing 12 changed files with 266 additions and 48 deletions.
40 changes: 36 additions & 4 deletions builds/org.eclipse.n4js.product.build/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,15 +111,47 @@ Contributors:
<rootFolders>
<!-- for MacOSX, make sure you use a folder ending with .app , see bug
463670 -->
<macosx>N4jside.app</macosx>
<linux>n4jside</linux>
<win32>n4jside</win32>
<macosx>n4jsIDE.app</macosx>
<linux>n4jsIDE</linux>
<win32>n4jsIDE</win32>
</rootFolders>
<archiveFileName>n4jside</archiveFileName>
<archiveFileName>n4js-ide</archiveFileName>
</product>
</products>
</configuration>
</plugin>
<!-- Copy products to folder target/products below the repository root folder and rename them to have more succinct names. -->
<plugin>
<groupId>com.coderplus.maven.plugins</groupId>
<artifactId>copy-rename-maven-plugin</artifactId>
<version>${codehaus-copy-rename-maven-plugin.version}</version>
<executions>
<execution>
<id>copy-file</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<overWrite>true</overWrite>
<fileSets>
<fileSet>
<sourceFile>${project.build.directory}/products/n4js-ide-win32.win32.x86_64.zip</sourceFile>
<destinationFile>${project.basedir}/../../target/products/n4js-ide_win.zip</destinationFile>
</fileSet>
<fileSet>
<sourceFile>${project.build.directory}/products/n4js-ide-macosx.cocoa.x86_64.zip</sourceFile>
<destinationFile>${project.basedir}/../../target/products/n4js-ide_macos.zip</destinationFile>
</fileSet>
<fileSet>
<sourceFile>${project.build.directory}/products/n4js-ide-linux.gtk.x86_64.zip</sourceFile>
<destinationFile>${project.basedir}/../../target/products/n4js-ide_linux-gtk.zip</destinationFile>
</fileSet>
</fileSets>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Expand Down
107 changes: 105 additions & 2 deletions docs/org.eclipse.n4js.design/chapters/03_releng/releng.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,106 @@ Contributors:

= Release Engineering

[[sec:overview_of_published_artifacts]]
== Overview of Published Artifacts

The main published artifacts are:

* the headless builder, i.e. n4jsc.jar,
* the IDE products for Windows, macOS, and Linux,
* the N4JS libraries (or "n4js-libs", for short), including the N4JS runtime libraries and
the 'mangelhaft' test framework (i.e. the projects in folder `n4js-libs/packages`).

The former two are published to `download.eclipse.org` (see https://download.eclipse.org/n4js/) and
the latter is published to the official public npm registry at https://www.npmjs.com/.

Other published artifacts include various documentation and the N4JS web site.


[[sec:versioning]]
== Versioning

=== Example

The main published artifacts (see <<sec:overview_of_published_artifacts>>) are versioned
as follows:

[cols="10,6,1"]
|===
| Language Version (applies to n4jsc.jar as well) | Version of n4js-libs | Note

| n4js-ide_macos_0.14.20.v20190912-0200.dmg | 0.14.20 |
| n4js-ide_macos_0.14.20.v20190913-0200.dmg | | 1.
| n4js-ide_macos_0.14.20.v20190914-0200.dmg | |
| n4js-ide_macos_0.14.21.v20190915-0200.dmg | 0.14.21 | 2.
| n4js-ide_macos_0.14.21.v20190916-0200.dmg | |
| n4js-ide_macos_0.15.0.next_v20190916-1422.dmg | 0.15.0-next.20190916.1422 | 3.
| n4js-ide_macos_0.14.21.v20190917-0200.dmg | |
| n4js-ide_macos_0.15.0.next_v20190917-1138.dmg | 0.15.0-next.20190917.1138 |
| n4js-ide_macos_0.14.21.v20190918-0200.dmg | |
| n4js-ide_macos_0.15.0.v20190919-0200.dmg | 0.15.0 | 4.
| n4js-ide_macos_0.15.0.v20190920-0200.dmg | |
|===

Notes:

1. New IDE published for unchanged version 0.14.20 of n4js-libs.
2. When publishing a new version of n4js-libs, the major/minor/patch segment of the IDE is increased accordingly.
3. Experimental publishing of breaking changes in n4js-libs with a pre-release tag and npm dist-tag 'next'.
The tag is also prepended to the last segment of the IDE version.
NOTE: since the version scheme of maven/OSGi does not support something like semver's pre-release tags,
these IDE versions must not be published to any public registries!
4. Release of the breaking changes from release channel 'next' to release channel 'latest'.


=== Configuring Versioning

The developer can conveniently control versioning via file `version.json`:

* When introducing non-breaking changes in the n4js-libs, nothing has to be changed in `version.json`;
version increments will be computed automatically by incrementing the patch segment.
* When introducing breaking changes in the n4js-libs, edit file `version.json` to request an
increment of the major/minor segment.
* In any case, if publishing from a topic branch is desired for testing purposes before merging
to master, simply configure a "tag" in file `version.json` and run the publishing build from
that branch. This will publish the n4js-libs under an npm dist-tag and publish the other
artifacts to separate folders (and won't change the LATEST folders).
NOTE: don't forget to set property "tag" back to an empty string in `version.json` before merging to master.

In case of breaking changes only in the Java / Xtend code of the core language (IDE or headless builder)
we do not yet increment any versions except updating time stamps, so file `version.json` does not have
to be changed in this case.


=== Details of Version Computation

The versioning of all the main published artifacts (see <<sec:overview_of_published_artifacts>>)
is controlled by shell script `compute-version.sh`. This script will compute the correct, incremented
version of the currently running build and write all version-related information to a newly
created file `version-info.json` in the root folder of the repository. In addition, it will update
the place-holder version in file `plugins/org.eclipse.n4js/res/language-version.properties` to make
the version number available to the Java code at runtime. Actual publishing will then be performed
by script `publish-n4js-libs.sh` (for the n4js-libs) and by plain file copying in the Jenkinsfile
`eclipse-publish.jenkinsfile` of the publishing build (for the `n4jsc.jar` and the IDE products).

The logic for finding the correct, incremented version number for a currently running build
applied by script `compute-version.sh` is as follows:

1. As a basis for the current build's version number, use the major, minor, and patch segments
of the newest version of the n4js-libs available on `npmjs.org` for the dist-tag `latest`.
2. Check whether a new version of the n4js-libs has to be published to `npmjs.org` by the current
build. This is the case of the current commit ID of top-level folder `n4js-libs` of the local
git working copy differs from the commit ID of the newest version of the n4js-libs available
on `npmjs.org` for the dist-tag `latest`.
3. If a new version of n4js-libs has to be published, increment the major, minor, or patch segment(s)
depending on the configuration in `version.json`.
Otherwise, don't increment these segments.
4. Now, versions used for publishing are:
a. For publishing the n4js-libs, use the major, minor, patch segment resulting from step 3.
b. For publsihing the headless builder and the IDE products, use the major, minor, patch segment
resulting from step 3 and append a time stamp segment.


== Nightly build on Eclipse infrastructure

The N4JS IDE, headless n4jsc.jar, and the N4JS update site is being built on the Eclipse Common Build
Expand Down Expand Up @@ -83,8 +183,11 @@ execute-hlc-integration-tests:: run integration tests using the headless jar (re
Available system properties:

[horizontal]
noTests:: suppress execution of all tests
startAndKeepVerdaccio:: enforce starting and suppress stopping of the test verdaccio (see <<sec:test-verdaccio>>)
noTests:: if defined (i.e. "-DnoTests"), suppress execution of all tests
languageVersion:: an optional version string to show in the Eclipse About dialog of IDE products (should be defined by publishing builds)
startAndKeepVerdaccio:: if defined (i.e. "-DstartAndKeepVerdaccio"), enforce starting and suppress stopping of the test verdaccio (see <<sec:test-verdaccio>>)
fatUpdateSite:: if set to true (i.e. "-DfatUpdateSite=true"), the generated update site will include all third-party dependencies
excludeJRE:: if defined (i.e. "-DexcludeJRE"), the Java runtime environment will not be bundled with the IDE products



Expand Down
17 changes: 17 additions & 0 deletions docs/org.eclipse.n4js.design/chapters/04_tipsTricks/tipsTricks.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,23 @@ This chapter is only about coding; add information on things like Eclipse setup
chapters. Similarly, this chapter is intended to provide just a quick overview, check-list and reminder; add detailed
information and diagrams to one of the succeeding chapters.


== Versioning

The rule of thumb for configuring the version increments through file `version.json` is as follows:

* when introducing non-breaking changes in the n4js-libs, nothing has to be changed in `version.json`;
(version increments will be computed automatically by incrementing the patch segment).
* when introducing breaking changes in the n4js-libs, edit file `version.json` to request an
increment of the major/minor segment.
* in case of breaking changes ONLY in the core language (IDE or headless builder), we do not yet
increment any versions (except updating time stamps which is done automatically), so file
`version.json` does not have to be changed in this case.

For details, including how to configure a pre-release tag / dist-tag for publishing for testing
purposes, see <<sec:versioning>>.


== Naming

* The internal handling of N4JS project names is non-trivial (due to the support for npm scopes), see
Expand Down
71 changes: 52 additions & 19 deletions jenkins/eclipse-publish.jenkinsfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ def MAVEN_OPTIONS = '-U -V -B -e ' +
'-Dmaven.test.redirectTestOutputToFile=true ' +
'-Dtycho.localArtifacts=ignore'

def DEBUG_DEPLOY_SUFFIX = ''; // should be empty string by default; use something like '_GH-1234' for debugging the publishing build itself

// the following will be set below in stage 'Compute Version':
def String LANGUAGE_VERSION;
def String N4JS_LIBS_DIST_TAG;
Expand Down Expand Up @@ -70,24 +72,32 @@ pipeline {
dir('n4js') {
sh "mvn $MAVEN_OPTIONS -PbuildProduct -DlanguageVersion=${LANGUAGE_VERSION} -DexcludeJRE -DnoTests clean verify"
}

// assert that the headless builder reports the correct version
script {
def versionOfHeadlessBuilder = sh(returnStdout: true, script: 'java -jar n4js/plugins/org.eclipse.n4js.cli/target/new_n4jsc.jar --version').trim();
if (versionOfHeadlessBuilder != LANGUAGE_VERSION) {
error "headless builder reported an incorrect version: ${versionOfHeadlessBuilder}"
}
}
}
}

stage('Sign macOS product') {
steps {
dir('n4js/builds/org.eclipse.n4js.product.build/target/products') {
dir('n4js/target/products') {
sh "pwd"

// sign the macOS product
sh """\
mv n4jside-macosx.cocoa.x86_64.zip unsigned_n4jside-macosx.cocoa.x86_64.zip
curl -o n4jside-macosx.cocoa.x86_64.zip -F file=@unsigned_n4jside-macosx.cocoa.x86_64.zip http://build.eclipse.org:31338/macsign.php
rm unsigned_n4jside-macosx.cocoa.x86_64.zip
mv n4js-ide_macos.zip unsigned_n4js-ide_macos.zip
curl -o n4js-ide_macos.zip -F file=@unsigned_n4js-ide_macos.zip http://build.eclipse.org:31338/macsign.php
rm unsigned_n4js-ide_macos.zip
"""

// bundle macOS product into a signed .dmg image
sh """\
curl -o n4jside-macosx.cocoa.x86_64.dmg -F sign=true -F source=@n4jside-macosx.cocoa.x86_64.zip http://build.eclipse.org:31338/dmg-packager
curl -o n4js-ide_macos.dmg -F sign=true -F source=@n4js-ide_macos.zip http://build.eclipse.org:31338/dmg-packager
"""
}
}
Expand All @@ -96,6 +106,17 @@ pipeline {
stage('Deploy') {
steps {
script {
// append language version to file names of products
sh """
cd n4js/target
mv n4jsc.jar n4jsc_${LANGUAGE_VERSION}.jar
cd products
mv n4js-ide_win.zip n4js-ide_win_${LANGUAGE_VERSION}.zip
mv n4js-ide_macos.zip n4js-ide_macos_${LANGUAGE_VERSION}.zip
mv n4js-ide_macos.dmg n4js-ide_macos_${LANGUAGE_VERSION}.dmg
mv n4js-ide_linux-gtk.zip n4js-ide_linux-gtk_${LANGUAGE_VERSION}.zip
"""

echo "Deploying version ${LANGUAGE_VERSION} of N4JS IDE and headless builder ..."

echo 'START: moving old products / update sites to archive.eclipse.org (those that are older than a week):'
Expand All @@ -107,27 +128,39 @@ pipeline {
echo 'END: moving old products / update sites to archive.eclipse.org'

echo 'START: copying new products / update site to download.eclipse.org:'
def pathProductsVersion = "/home/data/httpd/download.eclipse.org/n4js/products$DEBUG_DEPLOY_SUFFIX/nightly/$LANGUAGE_VERSION";
def pathUpdatesVersion = "/home/data/httpd/download.eclipse.org/n4js/updates$DEBUG_DEPLOY_SUFFIX/nightly/$LANGUAGE_VERSION";
sh """\
mkdir /home/data/httpd/download.eclipse.org/n4js/products/nightly/$LANGUAGE_VERSION
cp n4js/builds/org.eclipse.n4js.product.build/target/products/*.zip /home/data/httpd/download.eclipse.org/n4js/products/nightly/$LANGUAGE_VERSION
cp n4js/builds/org.eclipse.n4js.product.build/target/products/*.dmg /home/data/httpd/download.eclipse.org/n4js/products/nightly/$LANGUAGE_VERSION
cp n4js/tools/org.eclipse.n4js.hlc/target/n4jsc.jar /home/data/httpd/download.eclipse.org/n4js/products/nightly/$LANGUAGE_VERSION
mkdir -p ${pathProductsVersion}
cp n4js/target/products/*.zip ${pathProductsVersion}
cp n4js/target/products/*.dmg ${pathProductsVersion}
cp n4js/target/*.jar ${pathProductsVersion}

mkdir /home/data/httpd/download.eclipse.org/n4js/updates/nightly/$LANGUAGE_VERSION
cp -R n4js/builds/org.eclipse.n4js.product.build/target/repository/* /home/data/httpd/download.eclipse.org/n4js/updates/nightly/$LANGUAGE_VERSION
mkdir -p ${pathUpdatesVersion}
cp -R n4js/builds/org.eclipse.n4js.product.build/target/repository/* ${pathUpdatesVersion}
"""
echo 'END: copying new products / update site to download.eclipse.org'

echo 'START: updating folder LATEST on download.eclipse.org:'
if (N4JS_LIBS_DIST_TAG == 'latest') {
def pathProductsLatest = "/home/data/httpd/download.eclipse.org/n4js/products$DEBUG_DEPLOY_SUFFIX/nightly/LATEST";
def pathUpdatesLatest = "/home/data/httpd/download.eclipse.org/n4js/updates$DEBUG_DEPLOY_SUFFIX/nightly/LATEST";
sh """\
rm -rf /home/data/httpd/download.eclipse.org/n4js/products/nightly/LATEST
mkdir /home/data/httpd/download.eclipse.org/n4js/products/nightly/LATEST
cp -R /home/data/httpd/download.eclipse.org/n4js/products/nightly/$LANGUAGE_VERSION/* /home/data/httpd/download.eclipse.org/n4js/products/nightly/LATEST
rm -rf ${pathProductsLatest}
mkdir -p ${pathProductsLatest}
cp -R ${pathProductsVersion}/* ${pathProductsLatest}

rm -rf /home/data/httpd/download.eclipse.org/n4js/updates/nightly/LATEST
mkdir /home/data/httpd/download.eclipse.org/n4js/updates/nightly/LATEST
cp -R /home/data/httpd/download.eclipse.org/n4js/updates/nightly/$LANGUAGE_VERSION/* /home/data/httpd/download.eclipse.org/n4js/updates/nightly/LATEST
rm -rf ${pathUpdatesLatest}
mkdir -p ${pathUpdatesLatest}
cp -R ${pathUpdatesVersion}/* ${pathUpdatesLatest}
"""

// create copies of products without version suffix
sh """\
cp ${pathProductsLatest}/n4jsc_${LANGUAGE_VERSION}.jar ${pathProductsLatest}/n4jsc.jar
cp ${pathProductsLatest}/n4js-ide_win_${LANGUAGE_VERSION}.zip ${pathProductsLatest}/n4js-ide_win.zip
cp ${pathProductsLatest}/n4js-ide_macos_${LANGUAGE_VERSION}.zip ${pathProductsLatest}/n4js-ide_macos.zip
cp ${pathProductsLatest}/n4js-ide_linux-gtk_${LANGUAGE_VERSION}.zip ${pathProductsLatest}/n4js-ide_linux-gtk.zip
"""
} else {
echo 'SKIPPED! (because publishing with a dist-tag other than "latest")'
Expand All @@ -137,9 +170,9 @@ pipeline {
// show a summary
def output = sh returnStdout: true, script: """\
echo ******************** Products on download server:
ls -l /home/data/httpd/download.eclipse.org/n4js/products/nightly
ls -l /home/data/httpd/download.eclipse.org/n4js/products$DEBUG_DEPLOY_SUFFIX/nightly
echo ******************** Update sites on download server:
ls -l /home/data/httpd/download.eclipse.org/n4js/updates/nightly
ls -l /home/data/httpd/download.eclipse.org/n4js/updates$DEBUG_DEPLOY_SUFFIX/nightly
echo ******************** Products on archive server:
ls -l /home/data/httpd/archive.eclipse.org/n4js/products/nightly
echo ******************** Update sites on archive server:
Expand Down
2 changes: 1 addition & 1 deletion plugins/org.eclipse.n4js/res/language-version.properties
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
#
# This file will be overwritten at the beginning of the publishing builds by script 'compute-version.sh'.
#
language.version = 0.0.1.20130726_0941
language.version = 0.0.0.v19990101_0000
Loading

0 comments on commit e784c75

Please sign in to comment.