Skip to content
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
be48131
OpaPolarisAuthorizer
sungwy Sep 20, 2025
317cdc4
add CDI AuthorizerProducer
sungwy Sep 20, 2025
0739855
inject polarisAuthorizer in ServiceProducers CDI
sungwy Sep 22, 2025
7be0482
add integration test
sungwy Sep 23, 2025
c18d4d2
Merge branch 'main' into opa-authorizer
sungwy Sep 23, 2025
c7701cb
license
sungwy Sep 23, 2025
ec3c142
add integration tests
sungwy Sep 24, 2025
eec60c2
Merge branch 'main' into opa-authorizer
sungwy Sep 25, 2025
ed6f265
minor fixes
sungwy Sep 25, 2025
5caf1f4
adopt review feedback
sungwy Sep 26, 2025
3935c6a
remove comment
sungwy Sep 26, 2025
5ad1030
support https and bearer token authz
sungwy Sep 30, 2025
0785bdb
file token provider and token refresh
sungwy Sep 30, 2025
4b950e3
Merge branch 'main' into opa-authorizer
sungwy Sep 30, 2025
85baedc
fix
sungwy Sep 30, 2025
6421275
refactoring
sungwy Oct 1, 2025
36f687c
refactor tests, disable ssl verification in integration tests
sungwy Oct 1, 2025
c1ae608
use http in integration tests
sungwy Oct 1, 2025
6516726
remove properties from initial implementation
sungwy Oct 8, 2025
edfe61a
remove unused ssl dependencies
sungwy Oct 8, 2025
723dec1
adopt review feedback
sungwy Oct 9, 2025
479ac60
Notes about Beta
sungwy Oct 9, 2025
c946d0d
Merge branch 'main' into opa-authorizer
sungwy Oct 9, 2025
f46f97b
adopt more feedback
sungwy Oct 9, 2025
81de61e
remove JwtDecoder in favor of auth0 java-jwt
sungwy Oct 9, 2025
d014a97
use httpclient 5
sungwy Oct 9, 2025
944d005
opa http client factory refactoring
sungwy Oct 9, 2025
477839a
extensions/auth/opa refactoring
sungwy Oct 10, 2025
4252a44
fix opa tests
sungwy Oct 10, 2025
c0053f9
lint
sungwy Oct 10, 2025
0eb0a97
refactoring and cleaning up dependencies
sungwy Oct 11, 2025
7b61eee
remove old integration test files
sungwy Oct 11, 2025
44921ea
adopt review feedback and move integration tests into extensions/auth…
sungwy Oct 18, 2025
652f827
fix tests
sungwy Oct 20, 2025
30bd623
adopt review feedback
sungwy Oct 21, 2025
00d2e5d
fix tests
sungwy Oct 21, 2025
0b030e6
fix
sungwy Oct 22, 2025
5e02465
fix regtest
sungwy Oct 22, 2025
80ec27c
adopt more feedback
sungwy Oct 22, 2025
fd38aa3
add comment
sungwy Oct 22, 2025
fe8e450
adopt feedback
sungwy Oct 22, 2025
615cd9e
Make token-refresh asynchronous ...
snazy Oct 23, 2025
9f6eda6
Merge pull request #1 from snazy/opa-authorizer-file-refresh
sungwy Oct 24, 2025
615d731
thanks snazy
sungwy Oct 24, 2025
c2fe9fb
thanks snazy!
sungwy Oct 24, 2025
306d180
add polaris-extensions-auth-opa to bom
sungwy Oct 26, 2025
9ba71ec
Merge branch 'opa-authorizer' of https://github.com/sungwy/polaris in…
sungwy Oct 26, 2025
2aeb57b
remove cdi dependency in polaris-core
sungwy Oct 28, 2025
32e9951
spotlessApply
sungwy Oct 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
#

[versions]
apache-httpclient5 = "5.5"
checkstyle = "10.25.0"
hadoop = "3.4.2"
hive = "3.1.3"
Expand All @@ -41,6 +42,7 @@ swagger = "1.6.16"
#
quarkus-amazon-services-bom = { module = "io.quarkus.platform:quarkus-amazon-services-bom", version.ref="quarkus" }
antlr4-runtime = { module = "org.antlr:antlr4-runtime", version.strictly = "4.9.3" } # spark integration tests
apache-httpclient5 = { module = "org.apache.httpcomponents.client5:httpclient5", version.ref = "apache-httpclient5" }
assertj-core = { module = "org.assertj:assertj-core", version = "3.27.6" }
auth0-jwt = { module = "com.auth0:java-jwt", version = "4.5.0" }
awssdk-bom = { module = "software.amazon.awssdk:bom", version = "2.35.0" }
Expand Down
2 changes: 2 additions & 0 deletions polaris-core/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ plugins {

dependencies {
implementation(project(":polaris-api-management-model"))
implementation(libs.apache.httpclient5)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be preferable to move OPA stuff out of "core". I propose extensions/opa (include into runtime/server).

Rationale:

  1. the new authorizer is pluggable, but not all downstream projects may want to have it by default.
  2. reducing core dependencies (IIRC, Quarkus runtime already has httpclient5, but core does not have to depend on it).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that makes a lot of sense as well. I was curious to hear opinions on the new dependencies and OPA plugin being introduced as a part of polaris-core.

Let me take a stab at separating out these changes as a separate build in extensions/auth/opa

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it would be valuable to isolate OPA from core and from runtime/service and only include it into runtime/server.
the new authorizer is pluggable, but not all downstream projects may want to have it by default.

I'm OK to move OPA related classes from core to runtime/service, but I don't think we should put it in runtime/server. Here are the reason:

  1. I think it's perfect fine to have another implementation of authorizer, we did that in multiple places.
  2. The native RBAC is still the default.
  3. There is no single extra lib dependencies introduced when it landed in the service module. http and jwt libs are there already.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this dep. in core?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flyrain : if OPA is not added to runtime/server users of the OSS distributions will not be able to use it. I personally think it's a valuable feature for OSS users... Let's sync up on this aspect on the dev ML thread for visibility.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me review this dependency again.

I've moved most of the OPA factory and configuration classes into extensions/auth/opa as suggested, but now I'm seeing a test failure at PolarisEventListenerTest that's related to the Jandex index. I'll look into this a bit more and get this resolved.

PolarisEventListenerTest > testAllEventTypesHandled() FAILED

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this dep. in core?

No. I think it’s perfectly fine to keep the OPA authorizer within the polaris-runtime-service module rather than moving it to a separate extension module.
The rationale for separating components like hive or hadoop federation is that they introduce heavy and intrusive Hadoop dependencies that we don’t want to pull into the service module.
The OPA authorizer, on the other hand, adds no additional dependencies, it integrates cleanly within the service module, so there’s no downside to keeping it there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreover, OpaFileTokenIntegrationTest and OpaIntegrationTest are still be part of the polaris-runtime-service module. What's the point of having the tested classes(like OpaPolarisAuthorizer) in an extension module?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pluggability aspect is not about "heavy" or "light" depenpencies per se, but about giving downstream projects control over what to enable/bundle or not. As this appears to be a point where a bit of misunderstanding developed, let's move the discussion about this particular aspect to the dev ML for visibility and reaching consensus within the broader community (not everyone watches PRs).


implementation(platform(libs.iceberg.bom))
implementation("org.apache.iceberg:iceberg-api")
Expand All @@ -44,6 +45,7 @@ dependencies {
implementation(libs.caffeine)
implementation(libs.guava)
implementation(libs.slf4j.api)
implementation(libs.auth0.jwt)

compileOnly(project(":polaris-immutables"))
annotationProcessor(project(":polaris-immutables", configuration = "processor"))
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.polaris.core.auth;

import jakarta.annotation.Nullable;

/**
* Interface for providing bearer tokens for authentication.
*
* <p>Implementations can provide tokens from various sources such as:
*
* <ul>
* <li>Static string values
* <li>Files (with automatic reloading)
* <li>External token services
* </ul>
*/
public interface BearerTokenProvider {

/**
* Get the current bearer token.
*
* @return the bearer token, or null if no token is available
*/
@Nullable
String getToken();

/**
* Clean up any resources used by this token provider. Should be called when the provider is no
* longer needed.
*/
default void close() {
// Default implementation does nothing
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,248 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.apache.polaris.core.auth;

import com.auth0.jwt.JWT;
import com.auth0.jwt.exceptions.JWTDecodeException;
import com.auth0.jwt.interfaces.DecodedJWT;
import jakarta.annotation.Nullable;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Duration;
import java.time.Instant;
import java.util.Date;
import java.util.Optional;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* A token provider that reads tokens from a file and automatically reloads them based on a
* configurable refresh interval or JWT expiration timing.
*
* <p>This is particularly useful in Kubernetes environments where tokens are mounted as files and
* refreshed by external systems (e.g., service account tokens, projected volumes, etc.).
*
* <p>The token file is expected to contain the bearer token as plain text. Leading and trailing
* whitespace will be trimmed.
*
* <p>If JWT expiration refresh is enabled and the token is a valid JWT with an 'exp' claim, the
* provider will automatically refresh the token based on the expiration time minus a configurable
* buffer, rather than using the fixed refresh interval.
*/
public class FileBearerTokenProvider implements BearerTokenProvider {

private static final Logger logger = LoggerFactory.getLogger(FileBearerTokenProvider.class);

private final Path tokenFilePath;
private final Duration refreshInterval;
private final boolean jwtExpirationRefresh;
private final Duration jwtExpirationBuffer;
private final ReadWriteLock lock = new ReentrantReadWriteLock();

private volatile String cachedToken;
private volatile Instant lastRefresh;
private volatile Instant nextRefresh;
private volatile boolean closed = false;

/**
* Create a new file-based token provider with basic refresh interval.
*
* @param tokenFilePath path to the file containing the bearer token
* @param refreshInterval how often to check for token file changes
*/
public FileBearerTokenProvider(String tokenFilePath, Duration refreshInterval) {
this(tokenFilePath, refreshInterval, true, Duration.ofSeconds(60));
}

/**
* Create a new file-based token provider with JWT expiration support.
*
* @param tokenFilePath path to the file containing the bearer token
* @param refreshInterval how often to check for token file changes (fallback for non-JWT tokens)
* @param jwtExpirationRefresh whether to use JWT expiration for refresh timing
* @param jwtExpirationBuffer buffer time before JWT expiration to refresh the token
*/
public FileBearerTokenProvider(
String tokenFilePath,
Duration refreshInterval,
boolean jwtExpirationRefresh,
Duration jwtExpirationBuffer) {
this.tokenFilePath = Paths.get(tokenFilePath);
this.refreshInterval = refreshInterval;
this.jwtExpirationRefresh = jwtExpirationRefresh;
this.jwtExpirationBuffer = jwtExpirationBuffer;
this.lastRefresh = Instant.MIN; // Force initial load
this.nextRefresh = Instant.MIN; // Force initial calculation

logger.info(
"Created file token provider for path: {} with refresh interval: {}, JWT expiration refresh: {}, JWT buffer: {}",
tokenFilePath,
refreshInterval,
jwtExpirationRefresh,
jwtExpirationBuffer);
}

@Override
@Nullable
public String getToken() {
if (closed) {
logger.warn("Token provider is closed, returning null");
return null;
}

// Check if we need to refresh
if (shouldRefresh()) {
refreshToken();
}

lock.readLock().lock();
try {
return cachedToken;
} finally {
lock.readLock().unlock();
}
}

@Override
public void close() {
closed = true;
lock.writeLock().lock();
try {
cachedToken = null;
logger.info("File token provider closed");
} finally {
lock.writeLock().unlock();
}
}

private boolean shouldRefresh() {
return Instant.now().isAfter(nextRefresh);
}

private void refreshToken() {
lock.writeLock().lock();
try {
// Double-check pattern - another thread might have refreshed while we waited for the lock
if (!shouldRefresh()) {
return;
}

String newToken = loadTokenFromFile();
cachedToken = newToken;
lastRefresh = Instant.now();

// Calculate next refresh time based on JWT expiration or fixed interval
nextRefresh = calculateNextRefresh(newToken);

logger.debug(
"Token refreshed from file: {} (token present: {}), next refresh: {}",
tokenFilePath,
newToken != null && !newToken.isEmpty(),
nextRefresh);

} finally {
lock.writeLock().unlock();
}
}

/** Calculate when the next refresh should occur based on JWT expiration or fixed interval. */
private Instant calculateNextRefresh(@Nullable String token) {
if (token == null || !jwtExpirationRefresh) {
// Use fixed interval
return lastRefresh.plus(refreshInterval);
}

// Attempt to parse as JWT and extract expiration
Optional<Instant> expiration = getJwtExpirationTime(token);

if (expiration.isPresent()) {
// Refresh before expiration minus buffer
Instant refreshTime = expiration.get().minus(jwtExpirationBuffer);

// Ensure refresh time is in the future and not too soon (at least 1 second)
Instant minRefreshTime = Instant.now().plus(Duration.ofSeconds(1));
if (refreshTime.isBefore(minRefreshTime)) {
logger.warn(
"JWT expires too soon ({}), using minimum refresh interval instead", expiration.get());
return lastRefresh.plus(refreshInterval);
}

logger.debug(
"Using JWT expiration-based refresh: token expires at {}, refreshing at {}",
expiration.get(),
refreshTime);
return refreshTime;
}

// Fall back to fixed interval (token is not a valid JWT or has no expiration)
logger.debug("Token is not a valid JWT or has no expiration, using fixed refresh interval");
return lastRefresh.plus(refreshInterval);
}

@Nullable
private String loadTokenFromFile() {
try {
if (!Files.exists(tokenFilePath)) {
logger.warn("Token file does not exist: {}", tokenFilePath);
return null;
}

if (!Files.isReadable(tokenFilePath)) {
logger.warn("Token file is not readable: {}", tokenFilePath);
return null;
}

String content = Files.readString(tokenFilePath, StandardCharsets.UTF_8);
String token = content.trim();

if (token.isEmpty()) {
logger.warn("Token file is empty: {}", tokenFilePath);
return null;
}

return token;

} catch (IOException e) {
logger.error("Failed to read token from file: {}", tokenFilePath, e);
return null;
}
}

/**
* Extract the expiration time from a JWT token without signature verification.
*
* @param token the JWT token string
* @return the expiration time as an Instant, or empty if not present or invalid
*/
private Optional<Instant> getJwtExpirationTime(String token) {
try {
DecodedJWT decodedJWT = JWT.decode(token);
Date expiresAt = decodedJWT.getExpiresAt();
return expiresAt != null ? Optional.of(expiresAt.toInstant()) : Optional.empty();
} catch (JWTDecodeException e) {
logger.debug("Failed to decode JWT token: {}", e.getMessage());
return Optional.empty();
}
}
}
Loading