Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
0a08fb2
PDP-536: Adding copyright check
SameeraPriyathamTadikonda Aug 27, 2025
4717b79
Multiple upgrades to prep for Kafka 4
BillFarber Sep 30, 2025
9c0696d
Merge pull request #219 from BillFarber/projectUpgrades
BillFarber Oct 1, 2025
66d6013
MLE-12345 Fixing weird plugin classpath issue
rjrudin Oct 1, 2025
f853116
Merge pull request #220 from marklogic/feature/dhf
BillFarber Oct 1, 2025
b5416f4
Moving the docker file
BillFarber Oct 1, 2025
43b9049
Merge pull request #221 from BillFarber/reorgDocker
BillFarber Oct 1, 2025
ce285f3
Upgrade to the latest MarkLogic 12 Docker image.
BillFarber Oct 1, 2025
350237a
Merge pull request #222 from BillFarber/upgradeDockerImage
BillFarber Oct 2, 2025
1fc4f0e
Merge pull request #216 from SameeraPriyathamTadikonda/develop
BillFarber Oct 2, 2025
6cdb3f9
Changing the MarkLogic 12 Docker image again.
BillFarber Oct 2, 2025
b4bce2e
Merge pull request #223 from BillFarber/updateMarkLogic12Image
BillFarber Oct 2, 2025
eac8e3c
Trying to figure out the intermittent Jenkins failures.
BillFarber Oct 2, 2025
72f037d
Merge pull request #224 from BillFarber/jenkinsReliability
BillFarber Oct 2, 2025
d4b7ab6
Upgrade to Kafka 4.1.0 and Confluent Platform 8.0
BillFarber Oct 3, 2025
962a78a
MLE-24529 - Force newer jetty and netty libraries to avoid CVEs
BillFarber Oct 7, 2025
d69f7de
MLE-24717 Bumping logback
rjrudin Oct 13, 2025
bae80ad
MLE-24717 Bumping to ml-gradle 6.1 snapshot
rjrudin Oct 13, 2025
20f849e
MLE-24640 Bumping Jackson and NOTICE file
rjrudin Oct 14, 2025
6a4e5bb
MLE-24640 Updating Jekyll file to latest action versions
rjrudin Oct 14, 2025
0ea40ba
MLE-24747 Bumped ml-gradle
rjrudin Oct 16, 2025
b18239f
MLE-24747 Slightly better way of forcing ml-gradle 6.1
rjrudin Oct 16, 2025
8afe3f5
MLE-24747 Updated NOTICE
rjrudin Oct 17, 2025
b3edddf
MLE-24747 Bumping netty to address CVE
rjrudin Oct 17, 2025
aba2973
MLE-24747 Bumping version
rjrudin Oct 17, 2025
a4d0ae9
MLE-24747 Small docs updates for versions
rjrudin Oct 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .copyrightconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# COPYRIGHT VALIDATION CONFIG
# ---------------------------------
# Required start year (keep fixed; end year auto-updates in check output)
startyear: 2019

# Optional exclusions list (comma-separated). Leave commented if none.
# Rules:
# - Relative paths (no leading ./)
# - Simple * wildcard only (no recursive **)
# - Use sparingly (third_party, generated, binary assets)
# - Dotfiles already skipped automatically
# Enable by removing the leading '# ' from the next line and editing values.
# filesexcluded: third_party/*, docs/generated/*.md, assets/*.png, scripts/temp_*.py, vendor/lib.js
filesexcluded: .github/*, README.md, CONTRIBUTING.md, Jenkinsfile, gradle/*, docker-compose.yml, *.gradle, gradle.properties, gradlew, gradlew.bat, **/test/resources/**, docs/**, test-app/docker-compose.yml, docker/**, *.txt
2 changes: 2 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
MARKLOGIC_IMAGE=ml-docker-db-dev-tierpoint.bed-artifactory.bedford.progress.com/marklogic/marklogic-server-ubi:latest-12
MARKLOGIC_LOGS_VOLUME=./docker/marklogic/logs
12 changes: 5 additions & 7 deletions .github/workflows/jekyll-gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,8 @@
name: Deploy Jekyll with GitHub Pages dependencies preinstalled

on:
# Runs on pushes targeting the default branch
push:
branches: ["develop"]
branches: ["main"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
Expand All @@ -21,21 +20,20 @@ concurrency:
cancel-in-progress: true

jobs:
# Build job
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v3
uses: actions/configure-pages@v5
- name: Build with Jekyll
uses: actions/jekyll-build-pages@v1
with:
source: ./docs/
destination: ./_site
- name: Upload artifact
uses: actions/upload-pages-artifact@v1
uses: actions/upload-pages-artifact@v3

# Deployment job
deploy:
Expand All @@ -47,4 +45,4 @@ jobs:
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1
uses: actions/deploy-pages@v4
9 changes: 8 additions & 1 deletion .github/workflows/pr-workflow.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: 🏷️ JIRA ID Validator
name: PR Workflow

on:
# Using pull_request_target instead of pull_request to handle PRs from forks
Expand All @@ -14,3 +14,10 @@ jobs:
with:
# Pass the PR title from the event context
pr-title: ${{ github.event.pull_request.title }}
copyright-validation:
name: © Validate Copyright Headers
uses: marklogic/pr-workflows/.github/workflows/copyright-check.yml@main
permissions:
contents: read
pull-requests: write
issues: write
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
build
out
gradle-local.properties
docker
docker/confluent-marklogic-components/marklogic-kafka-marklogic-connector*
docker/marklogic

bin
.vscode
Expand Down
178 changes: 84 additions & 94 deletions CONTRIBUTING.md

Large diffs are not rendered by default.

85 changes: 59 additions & 26 deletions Jenkinsfile
Original file line number Diff line number Diff line change
@@ -1,41 +1,74 @@
@Library('shared-libraries') _

def runtests(String marklogicVersion) {
cleanupDocker()
sh label:'mlsetup', script: '''#!/bin/bash
echo "Removing any running MarkLogic server and clean up MarkLogic data directory"
sudo /usr/local/sbin/mladmin remove
docker-compose down -v || true
sudo /usr/local/sbin/mladmin cleandata
cd kafka-connector
MARKLOGIC_LOGS_VOLUME=/tmp MARKLOGIC_IMAGE='''+marklogicVersion+''' docker-compose up -d --build
sleep 60s;
'''
sh label:'deploy project', script: '''#!/bin/bash
export JAVA_HOME=$JAVA17_HOME_DIR
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
cd kafka-connector
./gradlew hubInit
./gradlew mlTestConnections
./gradlew -i mlDeploy
'''
sh label:'test', script: '''#!/bin/bash
export JAVA_HOME=$JAVA17_HOME_DIR
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
cd kafka-connector
./gradlew test || true
'''
junit '**/build/**/*.xml'
}

pipeline{
agent {label 'devExpLinuxPool'}
options {
checkoutToSubdirectory 'kafka-connector'
buildDiscarder logRotator(artifactDaysToKeepStr: '7', artifactNumToKeepStr: '', daysToKeepStr: '30', numToKeepStr: '')
}
environment{
JAVA_HOME_DIR="/home/builder/java/jdk-11.0.2"
JAVA17_HOME_DIR="/home/builder/java/jdk-17.0.2"
GRADLE_DIR =".gradle"
DMC_USER = credentials('MLBUILD_USER')
DMC_PASSWORD = credentials('MLBUILD_PASSWORD')
}
stages{
stage('tests'){
stage('test-ML12'){
steps{
runtests("ml-docker-db-dev-tierpoint.bed-artifactory.bedford.progress.com/marklogic/marklogic-server-ubi:latest-12")
}
post{
always{
updateWorkspacePermissions()
sh label:'mlcleanup', script: '''#!/bin/bash
cd kafka-connector
docker-compose down -v || true
'''
cleanupDocker()
}
}
}
stage('test-ML11'){
steps{
copyRPM 'Release','11.3.0'
setUpML '$WORKSPACE/xdmp/src/Mark*.rpm'
sh label:'setup', script: '''#!/bin/bash
cd kafka-connector/test-app
echo mlPassword=admin >> gradle-local.properties
'''
sh label:'deploy project', script: '''#!/bin/bash
export JAVA_HOME=$JAVA_HOME_DIR
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
cd kafka-connector/test-app
./gradlew hubInit
./gradlew mlDeploy -PmlPassword=admin
'''
sh label:'test', script: '''#!/bin/bash
export JAVA_HOME=$JAVA_HOME_DIR
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
cd kafka-connector
./gradlew test || true
'''
junit '**/build/**/*.xml'
runtests("ml-docker-db-dev-tierpoint.bed-artifactory.bedford.progress.com/marklogic/marklogic-server-ubi:latest-11")
}
post{
always{
updateWorkspacePermissions()
sh label:'mlcleanup', script: '''#!/bin/bash
cd kafka-connector
docker-compose down -v || true
'''
cleanupDocker()
}
}
}
}
Expand Down
38 changes: 24 additions & 14 deletions NOTICE.txt
Original file line number Diff line number Diff line change
@@ -1,38 +1,47 @@
MarkLogic® Connector for Kafka
Progress® MarkLogic® Connector for Kafka v1.11.0

Copyright © 2019-2025 MarkLogic Corporation. All Rights Reserved.
Copyright (c) 2019-2025 Progress Software Corporation and/or its subsidiaries or affiliates. All Rights Reserved.

To the extent required by the applicable open-source license, a complete machine-readable copy of the source code corresponding to such code is available upon request. This offer is valid to anyone in receipt of this information and shall expire three years following the date of the final distribution of this product version by Progress Software Corporation. To obtain such source code, send an email to [email protected]. Please specify the product and version for which you are requesting source code.

Third Party Notices
-----------------------------------------------------------------------

marklogic-data-hub 6.1.1 (Apache-2.0)
ml-app-deployer 5.0.0 (Apache-2.0)
jackson-dataformat-csv 2.17.2 (Apache-2.0)
The following is a list of the third-party components used by Progress® MarkLogic® Connector for Kafka v1.11.0 (last updated October 16, 2025):

Summary

marklogic-data-hub 6.2.1 (Apache-2.0)
ml-gradle 6.1.0 (Apache-2.0)
jackson-dataformat-csv 2.20.2 (Apache-2.0)

-----------------------------------------------------------------------

Common Licenses

Apache License 2.0 (Apache-2.0)

Third-Party Components
-----------------------------------------------------------------------

The following is a list of the third-party components used by the MarkLogic® Connector for Kafka 1.11.0 (last updated August 12, 2025):

marklogic-data-hub 6.1.1 (Apache-2.0)
marklogic-data-hub 6.2.1 (Apache-2.0)
https://repo1.maven.org/maven2/com/marklogic/marklogic-data-hub/

For the full text of the Apache-2.0 license, see Apache License 2.0 (Apache-2.0)

ml-app-deployer 5.0.0 (Apache-2.0)
https://repo1.maven.org/maven2/com/marklogic/ml-app-deployer/
ml-gradle 6.1.0 (Apache-2.0)
https://repo1.maven.org/maven2/com/marklogic/ml-gradle/

For the full text of the Apache-2.0 license, see Apache License 2.0 (Apache-2.0)

jackson-dataformat-csv 2.17.2 (Apache-2.0)
jackson-dataformat-csv 2.20.2 (Apache-2.0)
https://repo1.maven.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-csv/

For the full text of the Apache-2.0 license, see Apache License 2.0 (Apache-2.0)

-----------------------------------------------------------------------

Common Licenses

This section shows the text of common third-party licenses used by MarkLogic® Connector for Kafka 1.11.0 (last updated August 12, 2025):
This section shows the text of common third-party licenses used by Progress® MarkLogic® Connector for Kafka v1.11.0 (last updated October 16, 2025):

Apache License 2.0 (Apache-2.0)
https://spdx.org/licenses/Apache-2.0.html
Expand Down Expand Up @@ -110,3 +119,4 @@ distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Loading