Skip to content

fix(api): preserve a zero annotation-reply score threshold - #42640

Open
JessYanCoding wants to merge 1 commit into
langgenius:mainfrom
JessYanCoding:fix/annotation-zero-score-threshold
Open

JessYanCoding wants to merge 1 commit into
langgenius:mainfrom
JessYanCoding:fix/annotation-zero-score-threshold

Conversation

@JessYanCoding

Copy link
Copy Markdown
Contributor

Summary

Fixes #42639.

AnnotationReplyFeature.query read the app's configured similarity threshold as:

score_threshold = enabled_config["score_threshold"] or 1

That is not a null guard. The key is declared score_threshold: float on AnnotationReplyEnabledConfig (api/models/model.py:127), AppAnnotationSetting.score_threshold is Float, nullable=False (:1997), load_annotation_reply_config populates it unconditionally (:2036), and enable_annotation_reply_task writes the submitted value verbatim (api/tasks/annotation/enable_annotation_reply_task.py:85,93). The only input the or can ever intercept is the legitimate value 0.0.

0.0 is the loosest setting the feature offers — the config modal sends val / 100 from a 0-100 slider whose left stop is labelled 0.0 / Easy Match. Coercing it to 1 applies a threshold stricter than the slider's own right-hand Accurate Match stop, so an app configured for the loosest matching stops replying with annotations entirely.

Why this is the odd one out

Every sibling threshold path treats None as the disabled sentinel and keeps 0.0 as an active value:

Path Shape
core/rag/retrieval/dataset_retrieval.py:751, :1208 retrieval_model.get("score_threshold", 0.0)
core/rag/retrieval/dataset_retrieval.py:1490 if score_threshold is None or ... >= score_threshold
core/app/app_config/easy_ui_based_app/dataset/manager.py:108-121 if score_threshold_val is not None
core/tools/utils/dataset_retriever/dataset_retriever_tool.py:151 .get("score_threshold", 0.0)

The annotation path was the only one coercing.

The existing test encoded the bug

tests/unit_tests/core/app/features/test_annotation_reply.py persisted score_threshold=0 at line 102 and then asserted the vector search received score_threshold=1 at line 123. That assertion is corrected here, which is also why reverting only the test file leaves the suite red:

1 failed, 4 passed   # tests reverted, fix kept

A parametrized regression test now pins both 0.0 and a non-zero threshold to the value that reaches search_by_vector.

Verification

On 22aff66, from the repo root:

  • pytest api/tests/unit_tests/core/app/features/test_annotation_reply.py7 passed (5 on main, so +2)
  • pytest api/tests/unit_tests/core/app/features/ api/tests/unit_tests/controllers/service_api/app/test_annotation.py api/tests/unit_tests/tasks — 392 passed
  • pytest api/tests/unit_tests19502 passed, 6 skipped. The single failure, libs/test_zip_filename_recovery.py::test_zip64_extensible_data_is_not_part_of_central_directory, reproduces on a clean checkout of the same commit and is unrelated.
  • restoring or 1 with the new tests in place — 2 failed (the [0.0] case and the corrected assertion); the [0.5] case still passes, so the test is specific to the zero path
  • ruff format --check / ruff check on ./api — clean
  • pyrefly check and mypy --check-untyped-defs on the changed source — clean
  • pyrefly --config=tests/unit_tests/pyrefly.toml on the changed test file — 5 diagnostics before and after, so no new strict-mode debt
  • api/uv.lock unchanged

Screenshots

N/A (backend-only fix).

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.
  • I ran make lint && make type-check (backend) and vp staged (frontend) to appease the lint gods

From Claude Code

`AnnotationReplyFeature.query` read the app's configured similarity threshold as
`enabled_config["score_threshold"] or 1`. That is not a null guard: the key is
declared `score_threshold: float` on the `AnnotationReplyEnabledConfig` TypedDict,
`AppAnnotationSetting.score_threshold` is `Float, nullable=False`, and
`enable_annotation_reply_task` always writes the value the user submitted. The only
input the `or` can ever intercept is the legitimate value `0.0`.

`0.0` is the loosest setting the feature offers — the config slider runs 0 to 100
and sends `val / 100`, labelling its left stop "0.0 / Easy Match". Coercing it to
`1` applies a threshold stricter than the slider's own right-hand stop, so an app
configured for the loosest matching stops replying with annotations altogether,
which is the opposite of what was asked for.

Every sibling threshold path already treats `None` as the disabled sentinel and
keeps `0.0` as an active value — `dataset_retrieval` uses
`retrieval_model.get("score_threshold", 0.0)` and guards with
`score_threshold is None or ... >= score_threshold`, and the easy-UI dataset config
manager checks `score_threshold_val is not None`. The annotation path was the only
one coercing.

The existing test encoded the bug: it persisted `score_threshold=0` and asserted
the vector search received `score_threshold=1`. That assertion is corrected, and a
parametrized regression test now pins both `0.0` and a non-zero threshold to the
value that reaches `search_by_vector`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 65.04% 65.04% +0.00%
Strict coverage 64.65% 64.65% +0.00%
Typed symbols 47,326 47,329 +3
Untyped symbols 25,590 25,590 0
Modules 3423 3423 0

@codecov

codecov Bot commented Sep 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.57%. Comparing base (22aff66) to head (0540a73).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main   #42640    +/-   ##
========================================
  Coverage   87.57%   87.57%            
========================================
  Files        5467     5467            
  Lines      317685   317820   +135     
  Branches    63688    63718    +30     
========================================
+ Hits       278208   278329   +121     
- Misses      34112    34118     +6     
- Partials     5365     5373     +8     
Flag Coverage Δ
api 87.25% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Annotation Reply: a stored score threshold of 0.0 is silently replaced with 1, disabling the feature

1 participant