PYTHON-5569: [Build Failure] Spec Resync job is failing silently #2553

sleepyStick · 2025-09-22T21:26:42Z

Summary

Resync Spec task was silently failing and not generating any PRs.
https://jira.mongodb.org/browse/PYTHON-5569

Changes in this PR

I added a set -eu to the beginning of the bash file which should fail the task if the python script fails.
The cause of this specific PR was from one (or more) of the patch files being out of date. Going forward, this error is caught and the error will be put into the body of the generated PR (if all other steps succeed).
updated appropriate patch files and ran the script locally to sync the remaining specs.

Test Plan

I ran the script locally and verified desired behaviour. See below:
before my changes:

>$ ./.evergreen/scripts/resync-all-specs.sh 
Beginning to sync specs
Done syncing specs
Beginning to apply patches
rm: /Users/iris.ho/GitHub/mongo-python-driver/test/transactions/legacy/errors-client.json: No such file or directory
rm: /Users/iris.ho/GitHub/mongo-python-driver/test/index_management/index-rawdata.json: No such file or directory
Done removing unimplemented tests
error: patch failed: test/discovery_and_monitoring/unified/serverMonitoringMode.json:5
error: test/discovery_and_monitoring/unified/serverMonitoringMode.json: patch does not apply
Traceback (most recent call last):
  File "/Users/iris.ho/GitHub/mongo-python-driver/./.evergreen/scripts/resync-all-specs.py", line 122, in <module>
    main(args)
    ~~~~^^^^^^
  File "/Users/iris.ho/GitHub/mongo-python-driver/./.evergreen/scripts/resync-all-specs.py", line 109, in main
    apply_patches()
    ~~~~~~~~~~~~~^^
  File "/Users/iris.ho/GitHub/mongo-python-driver/./.evergreen/scripts/resync-all-specs.py", line 36, in apply_patches
    subprocess.run(
    ~~~~~~~~~~~~~~^
        ["git apply -R --allow-empty --whitespace=fix ./.evergreen/spec-patch/*"],  # noqa: S607
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        shell=True,  # noqa: S602
        ^^^^^^^^^^^^^^^^^^^^^^^^^
        check=True,
        ^^^^^^^^^^^
    )
    ^
  File "/Users/iris.ho/.pyenv/versions/3.13.5t/lib/python3.13t/subprocess.py", line 577, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git apply -R --allow-empty --whitespace=fix ./.evergreen/spec-patch/*']' returned non-zero exit status 1.

after my changes:

>$ Beginning to sync specs
Done syncing specs
Beginning to apply patches
rm: /Users/iris.ho/GitHub/mongo-python-driver/test/transactions/legacy/errors-client.json: No such file or directory
rm: /Users/iris.ho/GitHub/mongo-python-driver/test/index_management/index-rawdata.json: No such file or directory
Done removing unimplemented tests

The following specs were changed:
 -connection_logging
 -connection_monitoring
 -csot
 -discovery_and_monitoring
 -scripts
 -server_selection_logging


The following spec syncs encountered errors:
 -applying patches
```b'error: patch failed: test/discovery_and_monitoring/unified/serverMonitoringMode.json:5\nerror: test/discovery_and_monitoring/unified/serverMonitoringMode.json: patch does not apply\n'```

Screenshots (optional)

See above?

Checklist

Do not delete the items provided on this checklist
Checklist for Author

Did you update the changelog (if necessary)? (not necessary this time since no public facing changes)
Is the intention of the code captured in relevant tests? (i think the script now behavious more like we expected it to)
If there are new TODOs, has a related JIRA ticket been created? (no new todos)

Checklist for Reviewer {@primary_reviewer}

Does the title of the PR reference a JIRA Ticket?
Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
Have you checked for spelling & grammar errors?
Is all relevant documentation (README or docstring) updated?

Focus Areas for Reviewer

List any complex portion of code you believe needs additional scrutiny and explain why.
see comments :)

… 4.0)

…ust omitting the tests)

sleepyStick · 2025-09-23T20:38:48Z

.evergreen/remove-unimplemented-tests.sh


 # PYTHON-5248 - Remove support for MongoDB 4.0
 rm $PYMONGO/test/**/pre-42-*.json
+rm $PYMONGO/test/**/**/pre-42-*.json


this is to remove files in test/discovery_and_monitoring/errors/pre-42-*.json
(I thought the ** was a recursive wildcard? but every time i ran the resync spec script those files would pop up so i added it explicitly?)

I recommend using find instead. I'm not a bash expert but the rules for ** aren't like in Path/.glob.

Try: find /$PYMONGO /test -type f -name 'pre-42-*.json' -delete

Note that -f means that this isn't going to match directories.

ooo good idea! I've replaced the two rms with the find instead. Thanks~

sleepyStick · 2025-09-23T20:39:49Z

.evergreen/scripts/resync-all-specs.sh

@@ -1,5 +1,6 @@
 #!/usr/bin/env bash
 # Run spec syncing script and create PR
+set -eu


this should make errors more visible to us

sleepyStick · 2025-09-23T20:42:00Z

.evergreen/spec-patch/PYTHON-3712.patch

         "single",
-+        "sharded"
-        "sharded",
-        "sharded-replicaset"


We already did this ticket but for whatever reason the spec repo still has this one instance of sharded-replicaset so we're applying this patch to keep it out of our tests. (I've made a comment on the drivers ticket asking why its still in the spec repo)

caseyclements

Apologies for the delay. I had a cluster of partial seizures last week so needed some rest.

caseyclements · 2025-09-28T11:17:24Z

.evergreen/remove-unimplemented-tests.sh


 # PYTHON-5248 - Remove support for MongoDB 4.0
 rm $PYMONGO/test/**/pre-42-*.json
+rm $PYMONGO/test/**/**/pre-42-*.json


I recommend using find instead. I'm not a bash expert but the rules for ** aren't like in Path/.glob.

Try: find /$PYMONGO /test -type f -name 'pre-42-*.json' -delete

Note that -f means that this isn't going to match directories.

caseyclements · 2025-09-28T16:53:18Z

.evergreen/spec-patch/PYTHON-5052.patch

Do these patches belong in this PR?

The spec sync was failing and I figured it was easier to fix it in this PR than to put it off and fix it in the next spec sync PR. (running the spec sync script locally naturally produced these changes and since I was on greener build last week I decided to take the time to straighten them out.) I saw it as "catching" up on what the spec sync for the past few weeks should've been like and was hoping to make it easier for the next person on greener build.
If you prefer that they not be added in this PR, I'm happy to take them out.

I'm personally ok with this approach. I'm not a stickler and think velocity is often more important than paper trail.

caseyclements · 2025-09-28T16:55:18Z

test/unified-test-format/invalid/runOnRequirement-csfle-minLibmongocryptVersion-pattern.json

Same question on these new tests. I take it that the sync was failing and these just piled up?

sleepyStick · 2025-09-29T16:28:35Z

Apologies for the delay. I had a cluster of partial seizures last week so needed some rest.

All good, hope you go the rest you needed and that you're feeling better now!

caseyclements

LGTM!

caseyclements · 2025-09-29T16:47:24Z

.evergreen/spec-patch/PYTHON-5052.patch

I'm personally ok with this approach. I'm not a stickler and think velocity is often more important than paper trail.

sleepyStick added 9 commits September 22, 2025 14:14

catch patch errors and show them in the PR

16a485c

python 3712 was completed (i think)

b8fdcc1

script should exit if fail

313120a

not sure why this was changed? but i think we still need this patch

ffdaba0

ran resync-all-specs

4322cc5

create patch for python 5529

a4664e2

wait these shouldn't have been added back (remove support for mongodb…

1042b82

… 4.0)

wait this shouldn't be here (python 3712)?

20cb829

open telemetry stuff (i dont think we do anything for it yet so i'm j…

fb37dc7

…ust omitting the tests)

sleepyStick commented Sep 23, 2025

View reviewed changes

sleepyStick marked this pull request as ready for review September 23, 2025 20:42

sleepyStick requested a review from a team as a code owner September 23, 2025 20:42

sleepyStick changed the title ~~PYTHON-5569~~ PYTHON-5569: [Build Failure] Spec Resync job is failing silently Sep 23, 2025

sleepyStick requested a review from caseyclements September 23, 2025 21:34

caseyclements requested changes Sep 28, 2025

View reviewed changes

use recursive find instead of two rms

b1297dd

caseyclements approved these changes Sep 29, 2025

View reviewed changes

sleepyStick merged commit 4839e52 into mongodb:master Sep 29, 2025
70 of 73 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

PYTHON-5569: [Build Failure] Spec Resync job is failing silently #2553

PYTHON-5569: [Build Failure] Spec Resync job is failing silently #2553

Uh oh!

sleepyStick commented Sep 22, 2025 •

edited by caseyclements

Loading

Uh oh!

sleepyStick Sep 23, 2025

Uh oh!

caseyclements Sep 28, 2025

Uh oh!

sleepyStick Sep 29, 2025

Uh oh!

sleepyStick Sep 23, 2025

Uh oh!

sleepyStick Sep 23, 2025

Uh oh!

caseyclements left a comment

Uh oh!

caseyclements Sep 28, 2025

Uh oh!

caseyclements Sep 28, 2025

Uh oh!

sleepyStick Sep 29, 2025

Uh oh!

caseyclements Sep 29, 2025

Uh oh!

caseyclements Sep 28, 2025

Uh oh!

sleepyStick commented Sep 29, 2025

Uh oh!

caseyclements left a comment

Uh oh!

caseyclements Sep 29, 2025

Uh oh!

Uh oh!

Uh oh!

PYTHON-5569: [Build Failure] Spec Resync job is failing silently #2553

PYTHON-5569: [Build Failure] Spec Resync job is failing silently #2553

Uh oh!

Conversation

sleepyStick commented Sep 22, 2025 • edited by caseyclements Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes in this PR

Test Plan

Screenshots (optional)

Checklist

Focus Areas for Reviewer

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

caseyclements left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sleepyStick commented Sep 29, 2025

Uh oh!

caseyclements left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sleepyStick commented Sep 22, 2025 •

edited by caseyclements

Loading