Skip to content

MLE-24829 Fixing cloud auth for 1.11.1 release #243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 27, 2025
Merged

MLE-24829 Fixing cloud auth for 1.11.1 release #243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/configuring-the-connector.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@ Regardless of the required authentication strategy, you must configure the follo
- `ml.connection.port` = the port of the MarkLogic app server you wish to connect to
- `ml.connection.securityContextType` = the authentication strategy required by the MarkLogic app server; defaults to DIGEST

The choices for `ml.connection.securityContextType` are DIGEST, BASIC, CERTIFICATE, KERBEROS, and NONE. The additional
properties required for each are described in the following sections.
The choices for `ml.connection.securityContextType` are `DIGEST`, `BASIC`, `CERTIFICATE`, `KERBEROS`, `CLOUD`, and
`NONE`. The additional properties required for each are described in the following sections.

### Configuring digest and basic authentication

Expand All @@ -61,6 +61,7 @@ Both digest and basic authentication require the following properties to be conf

Progress Data Cloud authentication requires the following properties to be configured:

- `ml.connection.securityContextType=CLOUD`
- `ml.connection.basePath` = the base path in your Progress Data Cloud instance that points to the REST API server you
wish to connect to
- `ml.connection.cloudApiKey` = the API key for authenticating with your Progress Data Cloud instance
Expand All @@ -71,13 +72,15 @@ You should also set `ml.connection.port` to 443 for connecting to Progress Data

Certificate authentication requires the following properties to be configured:

- `ml.connection.securityContextType=CERTIFICATE`
- `ml.connection.certFile` = path to a PKCS12 certificate file
- `ml.connection.certPassword` = password for the PKCS12 certificate file

### Configuring Kerberos authentication

Kerberos authentication requires the following property to be configured:

- `ml.connection.securityContextType=KERBEROS`
- `ml.connection.externalName` = the name of the principal to be used in Kerberos authentication

### Configuring no authentication
Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
group=com.marklogic
version=1.11.0
version=1.11.1

# For the Confluent Connector Archive
componentOwner=marklogic
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/com/marklogic/kafka/connect/MarkLogicConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ public class MarkLogicConfig extends AbstractConfig {
public static final String SSL_HOST_VERIFIER = "ml.connection.customSsl.hostNameVerifier";
public static final String SSL_MUTUAL_AUTH = "ml.connection.customSsl.mutualAuth";

private static final CustomRecommenderAndValidator CONNECTION_SECURITY_CONTEXT_TYPE_RV = new CustomRecommenderAndValidator("DIGEST", "BASIC", "CERTIFICATE", "KERBEROS", "NONE");
private static final CustomRecommenderAndValidator CONNECTION_SECURITY_CONTEXT_TYPE_RV = new CustomRecommenderAndValidator("DIGEST", "BASIC", "CERTIFICATE", "KERBEROS", "CLOUD", "NONE");
private static final CustomRecommenderAndValidator CONNECTION_TYPE_RV = new CustomRecommenderAndValidator("DIRECT", "GATEWAY", "");
private static final CustomRecommenderAndValidator SSL_HOST_VERIFIER_RV = new CustomRecommenderAndValidator("ANY", "COMMON", "STRICT");

Expand Down Expand Up @@ -73,7 +73,7 @@ public static void addDefinitions(ConfigDef configDef) {
"External name for 'KERBEROS' authentication",
GROUP, -1, ConfigDef.Width.MEDIUM, "Kerberos External Name")
.define(CONNECTION_CLOUD_API_KEY, Type.STRING, null, Importance.MEDIUM,
"API key for connecting to MarkLogic Cloud. Should set port to 443 when connecting to MarkLogic Cloud.",
"API key for connecting to Progress Data Cloud. Should set port to 443 when connecting to Progress Data Cloud.",
GROUP, -1, ConfigDef.Width.MEDIUM, "Cloud API Key")
.define(CONNECTION_TYPE, Type.STRING, "", CONNECTION_TYPE_RV, Importance.MEDIUM,
"Set to 'GATEWAY' when the host identified by ml.connection.host is a load balancer. See https://docs.marklogic.com/guide/java/data-movement#id_26583 for more information.",
Expand Down
2 changes: 1 addition & 1 deletion src/test/java/com/marklogic/kafka/connect/BuildDatabaseClientConfigTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ void testInvalidAuthentication() {
securityContextConfig.put(MarkLogicSinkConfig.CONNECTION_SECURITY_CONTEXT_TYPE, "IncorrectValue");
ConfigException ex = assertThrows(ConfigException.class, () -> MarkLogicSinkConfig.CONFIG_DEF.parse(securityContextConfig),
"Should throw ConfigException when an invalid authentication type is provided.");
assertEquals("Invalid value: IncorrectValue; must be one of: [DIGEST, BASIC, CERTIFICATE, KERBEROS, NONE]", ex.getMessage());
assertEquals("Invalid value: IncorrectValue; must be one of: [DIGEST, BASIC, CERTIFICATE, KERBEROS, CLOUD, NONE]", ex.getMessage());
}

@Test
Expand Down
21 changes: 18 additions & 3 deletions src/test/java/com/marklogic/kafka/connect/source/ReadRowsViaOpticDslTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,7 @@
import java.util.List;
import java.util.Map;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.*;
Copy link

Copilot AI Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Using wildcard imports for static assertions reduces code readability. Consider keeping the explicit imports for assertEquals, assertNull, assertTrue, and assertThrows (newly used in the test) to make dependencies clear.

Suggested change
import static org.junit.jupiter.api.Assertions.*;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.assertThrows;

Copilot uses AI. Check for mistakes.

class ReadRowsViaOpticDslTest extends AbstractIntegrationSourceTest {

Expand Down Expand Up @@ -46,6 +44,23 @@ void readFifteenAuthorsAsJson() throws InterruptedException {
verifyRecordKeysAreSetToIDColumn(records);
}

@Test
void cloudAuth() {
RuntimeException ex = assertThrows(RuntimeException.class, () -> startSourceTask(
MarkLogicSourceConfig.CONNECTION_SECURITY_CONTEXT_TYPE, "cloud",
Copy link

Copilot AI Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The security context type is specified as lowercase 'cloud', but the documentation and validator define it as uppercase 'CLOUD'. While this may work if the validation is case-insensitive, using the canonical uppercase form would improve consistency with the documented configuration values.

Suggested change
MarkLogicSourceConfig.CONNECTION_SECURITY_CONTEXT_TYPE, "cloud",
MarkLogicSourceConfig.CONNECTION_SECURITY_CONTEXT_TYPE, "CLOUD",

Copilot uses AI. Check for mistakes.
MarkLogicSourceConfig.CONNECTION_CLOUD_API_KEY, "abc123",
MarkLogicSourceConfig.DSL_QUERY, AUTHORS_ORDERED_BY_ID_OPTIC_DSL,
MarkLogicSourceConfig.TOPIC, AUTHORS_TOPIC,
MarkLogicSourceConfig.KEY_COLUMN, "Medical.Authors.ID"
));

String message = ex.getMessage();
assertTrue(message.contains("Unable to call token endpoint"),
"We expect this test to fail because it can't talk to PDC, and that's fine. What this verifies " +
"is that the user can configure the connector to talk to PDC - i.e. 'cloud' is accepted as a " +
"security context type. Actual error: " + message);
}

@Test
void includeColumnTypes() throws InterruptedException {
loadFifteenAuthorsIntoMarkLogic();
Expand Down