NIFI-15209 JoltTransformRecord should not only take schema from first record

[sammu97]

This PR aims to fix an unwanted behaviour of having fields omitted after a JoltTransformRecord on a batch of records within the same FlowFile, due to multiple outputs having more than 1 schema. The current implementation of the processor retrieves the schema of the FIRST transformed record, and abides by that schema throughout the rest of the transformations. A new property is introduced for the JoltTransformRecord, where users can decide to either keep the same behaviour, or utilize the new PARTITION_BY_SCHEMA strategy, which will split the transformations into separate FlowFIles, according to the number of schemas.

Summary

NIFI-15209

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using ./mvnw clean install -P contrib-check
  • JDK 21
  • JDK 25

Licensing

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[exceptionfactory]

Thanks for working on this issue @sammu97. As a general note, it seems like it would be cleaner to avoid moving all of the test schema and JSON files to a new directory, in order to focus on the actual changes proposed. Can that be adjusted?

[sammu97]

Yes sure @exceptionfactory , will handle this as soon as i can.

Also, I'm seeing that some checks are failing on code checkout, is this due to the Cloudflare outage?

[exceptionfactory]

Yes sure @exceptionfactory , will handle this as soon as i can.

Also, I'm seeing that some checks are failing on code checkout, is this due to the Cloudflare outage?

Yes, they were due to the outage, I have restarted the checks.

[sammu97]

Looks like the build failed on some of the OSs, im suspecting a file ordering issue. Will investigate and update the PR accordingly

[sammu97]

@exceptionfactory Had to make some fixes for Windows as the checks are usually omitted. However, any idea about the error for the Mac tests?

The template is not valid. .github/workflows/ci-workflow.yml (Line: 224, Col: 16): hashFiles('**/package-lock.json') failed. Fail to hash files under directory '/Users/runner/work/nifi/nifi'

[ChrisSamo632]

@exceptionfactory Had to make some fixes for Windows as the checks are usually omitted. However, any idea about the error for the Mac tests?

The template is not valid. .github/workflows/ci-workflow.yml (Line: 224, Col: 16): hashFiles('**/package-lock.json') failed. Fail to hash files under directory '/Users/runner/work/nifi/nifi'

@sammu97 the node cache issue in the build appears to have been an intermittent problem over the weekend. I spotted other PRs with similar errors, but then things seem to be working again this morning. I've restarted the failed job on your PR and so far things like happier 🤞

[sammu97]

@exceptionfactory Had to make some fixes for Windows as the checks are usually omitted. However, any idea about the error for the Mac tests?
The template is not valid. .github/workflows/ci-workflow.yml (Line: 224, Col: 16): hashFiles('**/package-lock.json') failed. Fail to hash files under directory '/Users/runner/work/nifi/nifi'

@sammu97 the node cache issue in the build appears to have been an intermittent problem over the weekend. I spotted other PRs with similar errors, but then things seem to be working again this morning. I've restarted the failed job on your PR and so far things like happier 🤞

@ChrisSamo632 Yep, seems like it's already past the step that was failing. Thanks!

[sammu97]

@exceptionfactory Just a small note too. I've also amended some logic for the testNoRecords() test, as I have put out a small change that if the Jolt has no records to transform, in my opinion there should be no resulting flowfile as there is nothing to write. Not sure what you think about this, should I be leaving the old logic?