Describe the bug

When two or more plugins both extend the same module or plugin, OpenSearch's jar hell checker incorrectly throws an error claiming they have "duplicate codebases with each other," even though they're simply sharing a common transitive dependency. This prevents valid plugin configurations from loading.

Example:

• opensearch-knn extends transport-grpc
• opensearch-neural-search extends both opensearch-knn and transport-grpc (added in https://github.com/opensearch-project/neural-search/pull/1665/files)

Building neural-search gives plugin installation error:

Exception in thread "main" java.lang.IllegalStateException: failed to load plugin opensearch-neural-search due to jar hell
Caused by: java.lang.IllegalStateException: jar hell! extended plugins [opensearch-knn, transport-grpc] have duplicate codebases with each other: [
file:/path/to/modules/transport-grpc/grpc-stub-1.75.0.jar,
file:/path/to/modules/transport-grpc/guava-33.2.1-jre.jar,
...
]

To Reproduce

1. Create Plugin A that extends a module/plugin (e.g., transport-grpc)
2. Create Plugin B that also extends the same module/plugin (e.g., transport-grpc)
3. Install both Plugin A and Plugin B in OpenSearch
4. Observe error from jarhell from plugin installation

Expected behavior

Both plugins should load successfully. When multiple plugins extend the same module/plugin, the shared dependencies from that module should be recognized as a common ancestor, not as duplicate codebases.

Root Cause Analysis

The issue is in PluginsService#checkBundleJarHell():

Set<URL> intersection = new HashSet<>(urls);
intersection.retainAll(pluginUrls);
if (intersection.isEmpty() == false) {
    throw new IllegalStateException(
        "jar hell! extended plugins " + exts + " have duplicate codebases with each other: " + intersection
    );
}

This check compares the URLs from all extended plugins and throws an exception if there's any overlap. However, when multiple plugins extend the same module/plugin, they will naturally have overlapping URLs from the shared transitive dependency. This is not "jar hell" - it's the expected behavior of a shared module.

Impact

This prevents plugins from:

1. Extending the same OpenSearch module/plugin when providing different functionality
2. Using ServiceLoader-based extension mechanisms (like gRPC query converters) that require direct extension of a module
3. Following the dependency injection pattern where multiple plugins depend on a common infrastructure module

Proposed Solution

Modify the jar hell check to recognize when overlapping URLs come from a shared transitive dependency. Instead of throwing an exception, log a debug message indicating this is expected behavior:

Set<URL> intersection = new HashSet<>(urls);
intersection.retainAll(pluginUrls);
if (intersection.isEmpty() == false) {
    // Allow intersections when they represent shared transitive dependencies
    // This happens when multiple extended plugins both extend the same module/plugin
    logger.debug(
        "Plugin [{}] extends multiple plugins/modules that share common dependencies: {}. " +
        "This is expected when extended plugins share common ancestors.",
        bundle.plugin.getName(),
        intersection
    );
}

The subsequent JarHell.checkJarHell(urls, logger::debug) call after this block above, will still catch genuine jar hell issues (conflicting class versions, duplicate classes, etc.), so this change doesn't compromise safety.

Additional Context

This issue was discovered while implementing gRPC support for hybrid queries in the neural-search plugin opensearch-project/neural-search#1665. The HybridQueryBuilderProtoConverter needs to be discovered via ServiceLoader, which requires neural-search to extend transport-grpc. However, neural-search also extends opensearch-knn, which itself extends transport-grpc, causing the jar hell check to fail.

Why Neural-Search Must Extend Transport-gRPC Directly

The ServiceLoader mechanism in the transport-grpc module only discovers converters from plugins that directly declare transport-grpc in their extendedPlugins. Transitive extension (neural-search → knn → transport-grpc) is not sufficient for ServiceLoader discovery, requiring neural-search to extend transport-grpc directly, which creates the diamond dependency.