Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve support for builds with multiple configurations #249

Merged
merged 1 commit into from
Nov 14, 2020

Conversation

keithc-ca
Copy link
Member

  • move j9jcl preprocessing to a separate makefile to enable reuse to ensure module-deps.gmk is based on the right set of modules
  • remove redundant definition of FixPath
  • remove unused macro OPENJ9_BIN_OR_LIB_DIR

The OpenJDK build system supports multiple configurations.

One example use might be building both with UMA and with cmake (while cmake development is ongoing):

$ bash configure --with-cmake --with-conf-name=cmake
$ bash configure --with-freemarker-jar=$HOME/freemarker.jar --with-conf-name=uma

Or you might be working on code sensitive to how references are represented [1]:

$ bash configure --with-cmake --with-conf-name=normal
$ bash configure --with-cmake --with-noncompressedrefs --with-conf-name=xl
$ bash configure --with-cmake --with-mixedrefs=dynamic --with-conf-name=dynamic
$ bash configure --with-cmake --with-mixedrefs=static --with-conf-name=static

With multiple configurations, you can say make CONF= reconfigure or make CONF= clean, but make CONF= images would fail. You could say (for the first example):

$ make -C build/cmake images && make -C build/uma images

but that is somewhat tedious, and gets worse as the number of configuration grows. With these changes the shorter make CONF= images command will work.

[1] see ibmruntimes/openj9-openjdk-jdk11#359

@pshipton
Copy link
Member

jenkins compile aix,osx,win,plinux jdknext

@pshipton
Copy link
Member

pshipton commented Nov 12, 2020

The osx build hung for no obvious reason. https://ci.eclipse.org/openj9/job/Build_JDKnext_x86-64_mac_Personal/45

@pshipton
Copy link
Member

jenkins compile osx jdknext

@pshipton
Copy link
Member

@pshipton
Copy link
Member

The second one found some diagnostic files
23:42:30 make/javacore.20201111.220425.97239.0010.txt
23:42:30 make/javacore.20201111.220425.97239.0006.txt
23:42:30 make/javacore.20201111.220440.97239.0007.txt
23:42:30 make/Snap.20201111.220425.97239.0009.trc
23:42:30 make/javacore.20201111.220425.97239.0008.txt
23:42:30 make/core.20201111.220425.97239.0002.dmp

@keithc-ca
Copy link
Member Author

Jenkins compile osx jdknext depends eclipse-omr/omr#5668

@keithc-ca
Copy link
Member Author

The second one found some diagnostic files

I don't think those are from the same build. If you look through the log at the corresponding time (2020-11-12T03:04:25) you won't find any messages that should accompany creation of those files.

On the other hand, this can't be good (I believe this is the bootjdk talking):

[2020-11-12T03:05:03.758Z] The system is out of resources.
[2020-11-12T03:05:03.758Z] Consult the following stack trace for details.
[2020-11-12T03:05:03.758Z] java.lang.OutOfMemoryError: Java heap space
[2020-11-12T03:05:03.758Z] 	at java.base/java.util.Arrays.copyOfRange(Arrays.java:3852)
[2020-11-12T03:05:03.758Z] 	at java.base/java.lang.StringUTF16.newString(StringUTF16.java:1139)
[2020-11-12T03:05:03.758Z] 	at java.base/java.lang.StringBuilder.toString(StringBuilder.java:448)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavaTokenizer.checkIdent(JavaTokenizer.java:654)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavaTokenizer.scanIdent(JavaTokenizer.java:720)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavaTokenizer.readToken(JavaTokenizer.java:820)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.Scanner.nextToken(Scanner.java:115)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.nextToken(JavacParser.java:274)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.accept(JavacParser.java:467)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.accept(JavacParser.java:459)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.catchClause(JavacParser.java:2878)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.parseSimpleStatement(JavacParser.java:2774)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.blockStatement(JavacParser.java:2565)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.blockStatements(JavacParser.java:2502)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.block(JavacParser.java:2472)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.block(JavacParser.java:2486)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.parseSimpleStatement(JavacParser.java:2711)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.blockStatement(JavacParser.java:2565)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.parseStatementAsBlock(JavacParser.java:2527)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.parseSimpleStatement(JavacParser.java:2715)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.blockStatement(JavacParser.java:2565)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.blockStatements(JavacParser.java:2502)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.block(JavacParser.java:2472)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.block(JavacParser.java:2486)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.methodDeclaratorRest(JavacParser.java:4317)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.classOrInterfaceOrRecordBodyDeclaration(JavacParser.java:4182)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.classInterfaceOrRecordBody(JavacParser.java:4065)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.classDeclaration(JavacParser.java:3776)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.classOrRecordOrInterfaceOrEnumDeclaration(JavacParser.java:3717)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.typeDeclaration(JavacParser.java:3706)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.parser.JavacParser.parseCompilationUnit(JavacParser.java:3550)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:639)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.main.JavaCompiler.readSourceFile(JavaCompiler.java:816)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.main.JavaCompiler.readSourceFile(JavaCompiler.java:797)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.main.JavaCompiler.lambda$new$0(JavaCompiler.java:356)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.main.JavaCompiler$$Lambda$88/0000000010FEB1C8.complete(Unknown Source)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.code.ClassFinder.fillIn(ClassFinder.java:366)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.code.ClassFinder.complete(ClassFinder.java:292)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.code.ClassFinder$$Lambda$90/0000000010FF7E00.complete(Unknown Source)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.code.Symbol.complete(Symbol.java:681)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.code.Symbol$ClassSymbol.complete(Symbol.java:1409)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.code.ClassFinder.loadClass(ClassFinder.java:435)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.comp.Resolve.loadClass(Resolve.java:2014)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.comp.Resolve.findIdentInPackageInternal(Resolve.java:2425)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.comp.Resolve.findIdentInPackage(Resolve.java:2413)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.comp.Attr.selectSym(Attr.java:4248)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.comp.Attr.visitSelect(Attr.java:4133)
[2020-11-12T03:05:03.758Z] 	at jdk.compiler.interim/com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2273)

@pshipton
Copy link
Member

The build is hung again. I didn't stop it.
https://ci.eclipse.org/openj9/job/Build_JDKnext_x86-64_mac_Personal/48

10:32:56  Generating javadoc for openj9 source files
10:33:43  Generating JAVASE_API javadoc for 21 modules
10:35:00  Generating REFERENCE_API javadoc for 21 modules

@pshipton
Copy link
Member

Given the builds last night were all fine, and osx is failing to compile with this change for jdk11 and jdk15, plus the hangs for jdk16, I'd say this change is problematic for osx.

@keithc-ca
Copy link
Member Author

Given the builds last night were all fine, and osx is failing to compile with this change for jdk11 and jdk15, plus the hangs for jdk16, I'd say this change is problematic for osx.

Agreed, although it's not immediately clear why only OSX has trouble with this.

@keithc-ca keithc-ca changed the title Improve support for builds with multiple configurations WIP: Improve support for builds with multiple configurations Nov 12, 2020
* define J9JCL_SOURCES_DIR & J9JCL_SOURCES_DONEFILE in custom-spec.gmk
  and use as appropriate
* move j9jcl preprocessing to a separate makefile
* remove redundant definition of FixPath
* remove unused macro OPENJ9_BIN_OR_LIB_DIR

Signed-off-by: Keith W. Campbell <[email protected]>
@keithc-ca
Copy link
Member Author

As I explored a fix for this, I encountered the symptoms on Linux previously only seen on OSX. The updated code has been seen to work on Linux and OSX.

@keithc-ca
Copy link
Member Author

Jenkins compile aix,osx,plinux,win jdknext

@keithc-ca keithc-ca changed the title WIP: Improve support for builds with multiple configurations Improve support for builds with multiple configurations Nov 13, 2020
@keithc-ca keithc-ca requested a review from pshipton November 13, 2020 23:50
@pshipton pshipton merged commit e2649ca into ibmruntimes:openj9 Nov 14, 2020
@keithc-ca keithc-ca deleted the multi_config branch November 14, 2020 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants