ParameterType values not resolved when cucumber is run from a fat jar

[mjbingen]

Describe the bug
When a cucumber project is packaged into a standard fat or uber jar, parameter types are not registered properly and fail during execution. If parameter types are not used or the application is run by adding cucumber jars to the classpath, this works correctly and cucumber can be executed via a fat jar.

To Reproduce
Steps to reproduce the behavior:

1. Create a step definition that uses an @ParameterType
2. Package the cucumber application as a far Jar such that all dependencies (classes/resources) are copied into the jar.
3. Execute the jar (java -jar mycukes.jar)
4. Observe parameter type registration issue

I have provided a simple example built with gradle here: https://github.com/mjbingen/cucumber-sandbox/tree/fatjar-bug

1. git clone [email protected]:mjbingen/cucumber-sandbox.git
2. git checkout fatjar-bug
3. ./gradlew cucumber (this should work as cucumber + dependencies are simply added to the classpath)
4. ./gradlew fatJar (fat jar looks good)
5. java -jar build/libs/run-cukes.jar
6. See failure:
   SEVERE: Exception while executing pickle
   java.util.concurrent.ExecutionException: io.cucumber.core.exception.CucumberException: Could not create a cucumber expression for 'we have {triple}'.
   It appears you did not register a parameter type.
   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
   at io.cucumber.core.runtime.Runtime.run(Runtime.java:93)
   at io.cucumber.core.cli.Main.run(Main.java:92)
   at io.cucumber.core.cli.Main.main(Main.java:34)
   at com.foo.RunCukes.main(RunCukes.java:23)

Note that the code on the master branch of this repo does not have the ParameterType definition and runs just fine from the fat jar.

Expected behavior
The packaged cucumber application should execute and resolve parameter type definitions whether glue code is added to the classpath or directly embedded into a fat jar executable jar.

Context & Motivation
While packaging a cucumber based app as a "java application" is relatively easy in maven/gradle, it still requires managing all the dependent jars and runtime scripts to set classpaths properly. Sometimes when deploying a test binary to many environments its helpful just to have a fat jar that can be executed.

Your Environment

• Versions used: 5.0.0 - 6.8.1 (tried various versions in between, but seems like it's existed since ParameterType was introduced)
• Operating Systems: Mac OS, Linux RedHat 7.7
• Gradle: 6.2.1 and 5.6.2
• Java: openjdk version 1.8.0_161; oracle version 1.8.0_131

Additional context
I've tried things like shadow (https://github.com/johnrengelman/shadow) and manual jar creation, but the issue persists. This could be somewhat related to the issue with Spring Jars (#1931) but know they are special in their embedding structure. This is more around the difference between loading jars via classpath vs. embedded in the runnable jar.

[mpkorstanje]

In your reproducer you are using both cucumber-java and cucumber-java8. This means that your reproducer is not quite as minimal as it could theoretically be and that should give you a hint as to what is going wrong.

Cucumber uses Java's Service Provider Interfaces to register backend implementations and it would appear that only the Java 8 Backend implementation is loaded but not the Java backend. And indeed, if we look at the contents of the fat-jar we can see that the service registrations are overlapping rather then merged:

mpkorstanje@logarithmicwhale:~/Projects/cucumber/cucumber-sandbox$ unzip -l build/libs/run-cukes.jar  | grep META-INF/services
        0  2020-10-07 10:12   META-INF/services/
       43  2020-10-07 10:12   META-INF/services/io.cucumber.core.backend.BackendProviderService
       45  2020-10-07 10:15   META-INF/services/io.cucumber.core.backend.BackendProviderService
       63  2020-10-07 10:09   META-INF/services/io.cucumber.core.gherkin.FeatureParser

You'll have to fix the process by which you create a fat jar.

[mpkorstanje]

Note: If you are using shadow you may also have to use mergeServiceFiles.

Though I'm surprised it is not enabled by default.

[mjbingen]

Many thanks for the quick reply! This makes a lot of sense. Merging the service files did the trick.

The reason for including both cucumber-java and cucumber-java8 is because the ParameterType definition is in the io.cucumber.java package. So if you want to write lambda style step defs and use the ParameterType annotation, it looks like you need both. Perhaps the ParameterType could move to io.cucumber.cucumberexpressions? Or an explicit redefine in the java8 package?

I didn't see that these implementations should be mutually exclusive, as we've had good luck mixing classic and lambda style step defs. Some of our projects are quite large (and old) and added lambda style step defs when they were introduced, but never fully updated the old definitions. If it is not recommended to mix, it'd be nice to only need to consume one dependency or the other.

[mpkorstanje]

You can mix them just fine. However minimizing the components involved is a the part of creating and MCVE that usually allows you to discover the problem on your own.

There should also be a lambda equivalent of Parameter type. Though beware of:

#1817