Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-16078: Create SolrJ sub-modules, including solrj-core #945

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .github/workflows/solrj-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,4 @@ jobs:
- name: Initialize gradle settings
run: ./gradlew localSettings
- name: Test the SolrJ Package
run: ./gradlew solr:solrj:test
run: ./gradlew -p solr/solrj test
2 changes: 0 additions & 2 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -184,8 +184,6 @@ apply from: file('gradle/lucene-dev/lucene-dev-repo-composite.gradle')
// of potential problems with the build conventions, dependencies, etc.
apply from: file('gradle/ant-compat/misc.gradle')

apply from: file('gradle/ant-compat/artifact-naming.gradle')

apply from: file('gradle/documentation/pull-lucene-javadocs.gradle')
apply from: file('gradle/documentation/documentation.gradle')
apply from: file('gradle/documentation/changes-to-html.gradle')
Expand Down
5 changes: 3 additions & 2 deletions gradle/documentation/render-javadoc.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ configure(project(":solr:modules:langid")) {
}
}

configure(project(":solr:solrj")) {
configure(project(":solr:solrj:core")) {
project.tasks.withType(RenderJavadocTask) {
// TODO: clean up split packages
javadocMissingIgnore = ["org.apache.solr.client.solrj.embedded"]
Expand Down Expand Up @@ -172,7 +172,8 @@ configure(subprojects.findAll {
it.path in [
':solr:solr-ref-guide',
':solr:server',
':solr:webapp']
':solr:webapp',
':solr:solrj']
}) {
project.tasks.withType(RenderJavadocTask) {
enabled = false
Expand Down
2 changes: 1 addition & 1 deletion gradle/maven/defaults-maven.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ configure(rootProject) {
":solr:solrj",
":solr:prometheus-exporter",
":solr:test-framework",
] + project(":solr:modules").getChildProjects().values().path
] + project(":solr:modules").getChildProjects().values().path + project(":solr:solrj").getChildProjects().values().path

apacheNexusSnapshots = "https://repository.apache.org/content/repositories/snapshots"
}
Expand Down
14 changes: 1 addition & 13 deletions gradle/solr/packaging.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
// There doesn't seem to be one ideal set of rules on how these should be created, but
// I tried to imitate the current (master) logic present in ivy and ant files in this way:
//
// The "solr platform" set of dependencies is a union of all deps for (core, solrj, server).
// The "solr platform" set of dependencies is a union of all deps for (core, server).
//
// Then:
// lib - these are module's "own" dependencies.
Expand Down Expand Up @@ -57,7 +57,6 @@ configure(allprojects.findAll {project -> project.path.startsWith(":solr:modules

dependencies {
solrPlatformLibs project(":solr:core")
solrPlatformLibs project(":solr:solrj")
// libExt has logging libs, which we don't want. Lets users decide what they want.
solrPlatformLibs project(path: ":solr:server", configuration: 'libExt')
}
Expand Down Expand Up @@ -178,14 +177,3 @@ configure(project(":solr:core")) {
into "lib"
}
}

configure(project(":solr:solrj")) {
evaluationDependsOn(":solr:solrj")

task resolve(type: Sync) {
from({ configurations.runtimeClasspath }, {
})

into "lib"
}
}
2 changes: 1 addition & 1 deletion gradle/validation/jar-checks.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ subprojects {
def dep = queue.removeFirst()

// Skip any artifacts from other Solr Modules (they will be resolved there).
if (dep.moduleGroup == "org.apache.solr") {
if (dep.moduleGroup.startsWith("org.apache.solr")) {
continue
}

Expand Down
2 changes: 1 addition & 1 deletion gradle/validation/rat-sources.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ allprojects {
exclude "films/README.md"
break

case ":solr:solrj":
case ":solr:solrj:core":
exclude "src/**/*.json"
exclude "src/test-files/**/*.cfg"
exclude "src/test-files/**/*.xml"
Expand Down
6 changes: 3 additions & 3 deletions help/dependencies.txt
Original file line number Diff line number Diff line change
Expand Up @@ -148,20 +148,20 @@ Inspecting current dependencies
The tree of dependencies of a project (in all configurations) can
be dumped by the following command (example):

gradlew -p solr/solrj dependencies
gradlew -p solr/solrj/core dependencies

But this can be a bit overwhelming; we will most likely be interested
in just the "publicly visible" and "classpath-visible" configurations.

The publicly visible project dependencies (classes shared by other
modules importing our module) can be displayed with:

gradlew -p solr/solrj dependencies --configuration api
gradlew -p solr/solrj/core dependencies --configuration api

And the "private" set of dependencies (real classpath) can be dumped
with:

gradlew -p solr/solrj dependencies --configuration runtimeClasspath
gradlew -p solr/solrj/core dependencies --configuration runtimeClasspath


Excluding a transitive dependency
Expand Down
2 changes: 1 addition & 1 deletion help/tests.txt
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Randomization
To run tests with the given starting seed pass 'tests.seed'
property:

gradlew :solr:solrj:test -Ptests.seed=DEADBEEF
gradlew :solr:solrj:core:test -Ptests.seed=DEADBEEF

There are a lot of other test randomization properties
available. To list them, their defaults and current values
Expand Down
3 changes: 2 additions & 1 deletion settings.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,10 @@ rootProject.name = "solr-root"

includeBuild("dev-tools/solr-missing-doclet")

include "solr:solrj"
include "solr:core"
include "solr:server"
include "solr:solrj"
include "solr:solrj:core"
include "solr:modules:analysis-extras"
include "solr:modules:analytics"
include "solr:modules:clustering"
Expand Down
2 changes: 1 addition & 1 deletion solr/benchmark/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ task echoCp {
dependencies {
implementation project(':solr:test-framework')
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
2 changes: 1 addition & 1 deletion solr/core/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ dependencies {
// We export logging api with dependencies, which is useful for all modules
api 'org.slf4j:slf4j-api'

api project(':solr:solrj')
api project(':solr:solrj:core')

api 'io.dropwizard.metrics:metrics-core'
implementation ('io.dropwizard.metrics:metrics-graphite', {
Expand Down
2 changes: 1 addition & 1 deletion solr/modules/analysis-extras/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ description = 'Additional analysis components'
dependencies {
api project(':solr:core')

implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'commons-io:commons-io'
implementation 'com.ibm.icu:icu4j'
Expand Down
2 changes: 1 addition & 1 deletion solr/modules/analytics/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'Analytics Package'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@
*/

// Stick to previous artifact names (not identical to path/ folders).
configure(subprojects.findAll { it.path.contains(':solr:modules:') }) {
project.archivesBaseName = project.archivesBaseName.replace("-modules-", "-")
subprojects {
archivesBaseName = archivesBaseName.replace("-modules-", "-")
}
2 changes: 1 addition & 1 deletion solr/modules/clustering/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'Search Results Clustering Integraton'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/extraction/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'Solr Integration with Tika for extracting content from binary fil

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'
implementation 'org.slf4j:slf4j-api'
Expand Down
2 changes: 1 addition & 1 deletion solr/modules/gcs-repository/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'GCS Backup Repository'

dependencies {
api project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'
implementation 'org.slf4j:slf4j-api'
Expand Down
2 changes: 1 addition & 1 deletion solr/modules/hadoop-auth/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ dependencies {
}

implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.slf4j:slf4j-api'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/hdfs/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'HDFS Module'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/jaegertracer-configurator/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ dependencies {
})

testImplementation project(':solr:test-framework')
testImplementation project(':solr:solrj')
testImplementation project(':solr:solrj:core')
testImplementation 'com.carrotsearch.randomizedtesting:randomizedtesting-runner'
testImplementation 'junit:junit'
}
2 changes: 1 addition & 1 deletion solr/modules/jwt-auth/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'JWT / OpenID Connect / OAuth2 authentication plugin'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.bitbucket.b_c:jose4j'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/langid/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'Language Identifier module for extracting language from a documen

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation ('org.apache.tika:tika-core') { transitive = false }
implementation 'commons-io:commons-io'
Expand Down
2 changes: 1 addition & 1 deletion solr/modules/ltr/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'Learning to Rank Package'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/s3-repository/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'S3 Repository'

dependencies {
api project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/scripting/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'Scripting Package'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.apache.lucene:lucene-core'

Expand Down
2 changes: 1 addition & 1 deletion solr/modules/sql/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ description = 'SQL Module'

dependencies {
implementation project(':solr:core')
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')

implementation 'org.slf4j:slf4j-api'

Expand Down
2 changes: 1 addition & 1 deletion solr/prometheus-exporter/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ apply plugin: 'java-library'
description = 'Prometheus exporter for exposing metrics from Solr using Metrics API and Search API'

dependencies {
implementation project(':solr:solrj')
implementation project(':solr:solrj:core')
implementation('org.apache.zookeeper:zookeeper') { transitive = false } // ideally remove ZK dep

implementation ('io.prometheus:simpleclient')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The chosen replica acts as an aggregator: it creates internal requests to random
In a SolrCloud cluster each individual node load balances read requests across all the replicas in a collection.
You may still need a load balancer on the 'outside' that talks to the cluster.
Or you need a client which understands how to read and interact with Solr's metadata in ZooKeeper and only requests the ZooKeeper ensemble's address to discover which nodes should receive requests.
Solr provides a smart Java SolrJ client called {solr-javadocs}/solrj/org/apache/solr/client/solrj/impl/CloudSolrClient.html[CloudSolrClient] which is capable of this.
Solr provides a smart Java SolrJ client called {solr-javadocs}/solrj/core/org/apache/solr/client/solrj/impl/CloudSolrClient.html[CloudSolrClient] which is capable of this.

Even if some nodes in the cluster are offline or unreachable, a Solr node will be able to correctly respond to a search request as long as it can communicate with at least one replica of every shard, or one replica of every _relevant_ shard if the user limited the search via the `shards` or `\_route_` parameters.
The more replicas there are of every shard, the more likely that the Solr cluster will be able to handle search results in the event of node failures.
Expand Down
Loading