NIFI-14910 NIFI-14911 NiFi - Upgrade Jetty to 12.1.1 and move to Jakarta EE11

[pvillard31]

Summary

NIFI-14910 NIFI-14911 NIFI-14913 NiFi - Upgrade Jetty to 12.1.1 and move to Jakarta EE11

Important notes:

• Given that Spring Boot 3.x relies on EE10, it is not possible to upgrade NiFi Registry to use EE10. This will be a separate effort that is tracked by NIFI-14912 & NIFI-14914. In the meantime, the pom dependencies have been adjusted to move NiFi to Jetty 12.1.1 and Jakarta EE11 while NiFi Registry remains on Jakarta EE10.
• This PR is currently as a draft to allow some time for reviewers. But also because it is worth mentioning that:
  • the Jersey dependency is currently set to 4.0.0-M4. No clarity as to when the final 4.0.0 release is expected.
  • the swagger codegen plugin in Toolkit API is still using Jersey 3 as the configured library (https://github.com/apache/nifi/blob/main/nifi-toolkit/nifi-toolkit-api/pom.xml#L89) given that Jersey 4 is not an option yet - I do not believe however that this is an issue
• I have not moved the JMS bundle from javax.jms-api to jakarta.jms-api as this would be a breaking change for users with existing JMS components where a specific JMS client is provided as part of the configuration. This is something we can evaluate later.
• Jakarta EE11 brings support for Virtual Threads with Java 21. This could be evaluated and leveraged in follow up efforts.
• Went through NOTICE files / docs to clean up references and make sure we are aligned with latest. There is a lot of changes related to NOTICE files... sorry to the reviewers, I tried to do my best to make sure we are back to a good state

Testing:

• Beyond confirming that the full build with system tests is green, some additional manual testing has been done to confirm proper behavior of the UI with custom UIs, content viewer, documentation rendering, etc.
• Confirmed that NiFi Registry is working with Jetty 12.1/EE10

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using ./mvnw clean install -P contrib-check
  • JDK 21

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for the thorough work on these changes @pvillard31. The major of the adjustments appear to be straightforward.

Aside from the thread pool handling, the only code-related adjusted that raised questions is in JettyServer, with the associated changes to add the jetty-deploy dependency.

Regarding Spring Boot and NiFi Registry, with the way the application is structured, it seems like it should be possible to use 12.1 and override some references. Is the problem there related to Jakarta EE 11 as opposed to Jetty 12.1 itself? It is fine to address that separately, but it would be helpful to have more details.

[exceptionfactory]

The jetty-deploy library brings in several other dependencies. Is there a reason this is necessary in Jetty 12.1 and not 12.0?

[pvillard31]

Yes, this is one of the changes with Jetty 12.1: DeploymentManager no longer exists and I'm replacing it by StandardDeployer which comes from jetty-deploy in JettyServer

[pvillard31]

For NiFi Registry, reviewing further, the problem is specifically related to EE11. I have made the changes to also uses Jetty 12.1.1 in NiFi Registry but keep the ee10 layer.

[pvillard31]

OK... so while using Jetty 12.1.1 works for NiFi Registry (I was able to properly build it, run it, and use it with NiFi), the integration tests are failing because of the way Spring Boot is used there (I remember now :)). And we can also see the failure on the system test that I was mentioning with the changes on FlowSynchronizationIT.

[exceptionfactory]

OK... so while using Jetty 12.1.1 works for NiFi Registry (I was able to properly build it, run it, and use it with NiFi), the integration tests are failing because of the way Spring Boot is used there (I remember now :)). And we can also see the failure on the system test that I was mentioning with the changes on FlowSynchronizationIT.

Thanks @pvillard31.

As far as the NiFI Registry integration tests, I am open to removing those because they do not follow the actual pattern of a running NiFi Registry. The NiFi Registry Docker tests also provide some basic checks for a running NiFi Registry. What do you think?

On the system test failure, it is a bit strange to see the change, but if the previous adjustments corrected it and were otherwise reasonable, that sounds good.

[pvillard31]

Yeah so for NiFi Registry, I noticed that we were bringing Jetty through Spring Boot for running the tests. I removed this import to instead used the embedded Tomcat to still execute those tests successfully. I'm not against completely removing those.

[exceptionfactory]

Yeah so for NiFi Registry, I noticed that we were bringing Jetty through Spring Boot for running the tests. I removed this import to instead used the embedded Tomcat to still execute those tests successfully. I'm not against completely removing those.

That looks like a reasonable and minimal change to keep those tests passing. If that works and doesn't impact runtime behavior for NiFi Registry, that sounds good.

[exceptionfactory]

Thanks for the extensive work here @pvillard31.

On final validation, I noticed an exception thrown from LogoutCompleteRedirectFilter related to the Fragment portion of the redirect URI when passed to sendRedirect(). Changing the approach to set the Location header and send an HTTP 302 as in QueryStringToFragmentFilter worked as expected, so I will include that change in the merge commit.