Add adaptive merge policy to reduce benchmark variance #19352
Conversation
rgsriram
requested review from
a team,
Bukhtawar,
CEHENKLE,
Rishikesh1159,
anasalkouz,
andrross,
ashking94,
cwperks,
dbwiddis,
gbbafna,
jed326,
kotwanikunal,
mch2,
msfroh,
owaiskazi19,
reta,
sachinpkale,
saratvemulapalli,
shwetathareja and
sohami
as code owners
github-actions
bot
added
enhancement
|
❌ Gradle check result for 2b398ff: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
rgsriram
force-pushed
the
issue-11163
branch
from
2b398ff to
e88f69b
Compare
|
❌ Gradle check result for abe1e49: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
rgsriram
force-pushed
the
issue-11163
branch
from
abe1e49 to
5076ee5
Compare
|
❌ Gradle check result for 5076ee5: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
rgsriram
force-pushed
the
issue-11163
branch
from
5076ee5 to
6f0df5f
Compare
server/src/main/java/org/opensearch/index/AdaptiveTieredMergePolicyProvider.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/rest/action/admin/RestSegmentTopologyAction.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/AdaptiveTieredMergePolicyProvider.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/index/AdaptiveTieredMergePolicyProvider.java
Show resolved
Hide resolved
|
❌ Gradle check result for 8711bea: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❕ Gradle check result for cd0e959: UNSTABLE Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure. |
|
❌ Gradle check result for 12f34d9: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 95b404d: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 3b40f4c: TIMEOUT Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Sriram Ganesh <[email protected]>
Signed-off-by: Sriram Ganesh <[email protected]>
Signed-off-by: Sriram Ganesh <[email protected]>
|
❌ Gradle check result for 3a05b2a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 3a05b2a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
2 participants
Description
Introduced an adaptive merge policy system in OpenSearch that dynamically tunes Lucene segment merge settings based on shard size. The solution replaces static defaults with context-aware configurations, improving benchmark consistency and search performance across varying index sizes. Key additions include an
AdaptiveTieredMergePolicyProviderfor shard-specific optimisations, aSegmentTopologyAnalyzerfor segment distribution analysis, and a REST API for real-time monitoring. This change delivers more predictable performance and optimised search behaviour for users.Related Issues
Resolves #11163
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.