indexer sync commits contain updates already merged into master

[ineednewpajamas]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

The indexer sync process used to be mainly run from the pipeline and merged and handled by a user with write access to the repo. Issues with the commits containing out of date updates were handled by just deleting the automated-indexer-sync branch and letting it run again. Now that the process is mainly being done by people without write access its more noticeable that the github action kicked off branch contains commits with changes already in master and the default behavior of the script for users with their own fork also doesn't update off prowlarr/indexers master.

This is leading to

1. for the automated-indexer-sync it just keeps amending it's commit and it often contains changes already merged into master (see master...automated-indexer-sync)
2. for users with their own branch, if they don't update their master branch manually before running the script, it will base the changes on their version of master. This can be overcome by setting -r and creating a remove for prowlarr/indexers but that isn't the default behavior.

Expected Behavior

Only new commits from jackett that the master branch doesn't have should be reflected in both the commit message and file diff

Steps To Reproduce

set prowlarr/indexers as a remote named 'upstream' or run it from this repo and omit -r upstream
indexer-sync-v2.sh -a -r upstream -b automated-indexer-sync

or

have an out of date master on your fork and run the indexer sync , for example
indexer-sync-v2.sh -a

Environment

- OS: n/a
- Prowlarr: n/a
- Docker Install: n/a
- Using Reverse Proxy: n/a
- Browser: n/a

What branch are you running?

Master

Trace Logs? Not Optional

resulted in ineednewpajamas@afb3f40

indexer-sync-v2 has no trace logs, but debug and verbose enabled

2026-05-29T15:59:25-0500|INFO|VERBOSE logging enabled
2026-05-29T15:59:25-0500|INFO|DEBUG logging enabled
2026-05-29T15:59:25-0500|DEBUG|Using Python command: python3
2026-05-29T15:59:25-0500|INFO|Using Python validation
2026-05-29T15:59:25-0500|DEBUG|Loading blocklist from scripts/blocklist.txt
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: uniongang.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: uniongangcookie.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: ygg-api.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: yggtorrent.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: yggcookie.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: yggtorrent-turbo.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: yggcookie-turbo.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: u2p.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: sharewood.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: anirena.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: torrentgalaxy.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: scenelinks.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: therarbg.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: ildragonero.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: rutracker-ru.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: magnetz.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: tvchaosuk.yml
2026-05-29T15:59:25-0500|DEBUG|Added to blocklist: fearnopeer.yml
2026-05-29T15:59:25-0500|INFO|Loaded 18 entries from blocklist
2026-05-29T15:59:25-0500|DEBUG|Blocklist contents: uniongang.yml uniongangcookie.yml ygg-api.yml yggtorrent.yml yggcookie.yml yggtorrent-turbo.yml yggcookie-turbo.yml u2p.yml sharewood.yml anirena.yml torrentgalaxy.yml scenelinks.yml therarbg.yml ildragonero.yml rutracker-ru.yml magnetz.yml tvchaosuk.yml fearnopeer.yml
2026-05-29T15:59:25-0500|INFO|Configured Git
Fetching origin
Fetching upstream
remote: Enumerating objects: 33, done.
remote: Counting objects: 100% (33/33), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 21 (delta 16), reused 15 (delta 12), pack-reused 0 (from 0)
Unpacking objects: 100% (21/21), 6.74 KiB | 986.00 KiB/s, done.
From https://github.com/Prowlarr/Indexers
 + c98054703...c86346db7 automated-indexer-sync                       -> upstream/automated-indexer-sync  (forced update)
 + f0e78d318...31b41a768 dependabot/pip/master/python-deps-21dc72ea84 -> upstream/dependabot/pip/master/python-deps-21dc72ea84  (forced update)
   618920c75..2032e5039  master                                       -> upstream/master
Fetching z_Jackett
remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 20 (delta 11), reused 8 (delta 8), pack-reused 5 (from 1)
Unpacking objects: 100% (20/20), 22.38 KiB | 1.72 MiB/s, done.
From https://github.com/Jackett/Jackett
   7c2b5a83b..04dfea95a  master     -> z_Jackett/master
 * [new tag]             v0.24.1970 -> v0.24.1970
2026-05-29T15:59:27-0500|INFO|local branch [automated-indexer-sync] does exist
2026-05-29T15:59:27-0500|INFO|remote repo/branch [upstream/automated-indexer-sync] does exist
Switched to and reset branch 'automated-indexer-sync'
Your branch is based on 'origin/automated-indexer-sync', but the upstream is gone.
2026-05-29T15:59:27-0500|INFO|Checked out out local branch [automated-indexer-sync]
HEAD is now at c86346db7 jackett indexers as of ef4d22850d76b26630df2693fc8f4b4f95cebf4c [2026-05-29T18:26:44Z]
Current branch automated-indexer-sync is up to date.
2026-05-29T15:59:27-0500|INFO|local [automated-indexer-sync] reset and rebased on [upstream/master]
2026-05-29T15:59:27-0500|INFO|Reviewing Commits
2026-05-29T15:59:27-0500|DEBUG|Searching for commits with template: 'jackett indexers as of'
2026-05-29T15:59:27-0500|DEBUG|Searching in last 100 commits
2026-05-29T15:59:27-0500|DEBUG|Found prowlarr commits count: 91
2026-05-29T15:59:27-0500|DEBUG|First prowlarr commit found: jackett indexers as of ef4d22850d76b26630df2693fc8f4b4f95cebf4c [2026-05-29T18:26:44Z]
2026-05-29T15:59:27-0500|DEBUG|Prowlarr jackett commit message: 'jackett indexers as of ef4d22850d76b26630df2693fc8f4b4f95cebf4c [2026-05-29T18:26:44Z]'
2026-05-29T15:59:27-0500|DEBUG|Jackett recent commit: '04dfea95ac85778e91b47e7b2361dde4582b7f17'
2026-05-29T15:59:27-0500|INFO|Jackett commit message: 'gay-torrentsorg: removed. resolves #15022'
2026-05-29T15:59:27-0500|DEBUG|Recent pulled commit (field 5): 'ef4d22850d76b26630df2693fc8f4b4f95cebf4c'
2026-05-29T15:59:27-0500|DEBUG|Full first commit line: 'jackett indexers as of ef4d22850d76b26630df2693fc8f4b4f95cebf4c [2026-05-29T18:26:44Z]'
2026-05-29T15:59:27-0500|INFO|[04dfea95ac85778e91b47e7b2361dde4582b7f17] is the most recent Jackett commit as per branch [z_Jackett/master]
2026-05-29T15:59:27-0500|INFO|[ef4d22850d76b26630df2693fc8f4b4f95cebf4c] is the most recent Prowlarr/Indexer commit pulled from Jackett as per branch [upstream/automated-indexer-sync]
2026-05-29T15:59:27-0500|INFO|There are [1] commits to cherry-pick
2026-05-29T15:59:27-0500|INFO|Commit Range is: [
04dfea95ac85778e91b47e7b2361dde4582b7f17]
2026-05-29T15:59:27-0500|INFO|-- Beginning Cherrypicking ---
2026-05-29T15:59:29-0500|INFO|cherrypicking Jackett commit [04dfea95ac85778e91b47e7b2361dde4582b7f17]
2026-05-29T15:59:29-0500|INFO|Commit message: 'gay-torrentsorg: removed. resolves #15022'
Auto-merging README.md
CONFLICT (content): Merge conflict in README.md
CONFLICT (rename/delete): src/Jackett.Common/Definitions/gay-torrentsorg.yml renamed to definitions/v11/gay-torrentsorg.yml in HEAD, but deleted in 04dfea95a (gay-torrentsorg: removed. resolves #15022).
CONFLICT (modify/delete): src/Jackett.Updater/Program.cs deleted in HEAD and modified in 04dfea95a (gay-torrentsorg: removed. resolves #15022).  Version 04dfea95a (gay-torrentsorg: removed. resolves #15022) of src/Jackett.Updater/Program.cs left in tree.
error: could not apply 04dfea95a... gay-torrentsorg: removed. resolves #15022
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
2026-05-29T15:59:29-0500|WARN|conflicts exist
2026-05-29T15:59:29-0500|DEBUG|README conflict exists; using Prowlarr README
Updated 1 path from the index
2026-05-29T15:59:29-0500|DEBUG|Non-YML conflicts exist; removing [
src/Jackett.Updater/Program.cs
] files and restoring [package.json package-lock.json .editorconfig]
Updated 0 paths from the index
2026-05-29T15:59:29-0500|DEBUG|YML conflict exists; [definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|DEBUG|YML Definitions Process: [UD definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|DEBUG|YML files to add/process: []
2026-05-29T15:59:29-0500|DEBUG|YML files to delete: [definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|DEBUG|Removing definitions Jackett deleted; [definitions/v11/gay-torrentsorg.yml]
rm 'definitions/v11/gay-torrentsorg.yml'
2026-05-29T15:59:29-0500|INFO|SUCCESS|--- Completed cherry picking ---
2026-05-29T15:59:29-0500|INFO|Evaluating and Reviewing Changes
2026-05-29T15:59:29-0500|DEBUG|All staged files: [definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|DEBUG|All staged yml files: [definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|DEBUG|New indexers detected (AR filter + v[digit]+): []
2026-05-29T15:59:29-0500|INFO|No new indexers
2026-05-29T15:59:29-0500|INFO|SUCCESS|--- completed changed indexers ---
2026-05-29T15:59:29-0500|INFO|SUCCESS|--- completed backporting indexers ---
2026-05-29T15:59:29-0500|INFO|SUCCESS|Removed Indexers are [definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|INFO|Automation mode: Proceeding with commit and push automatically
[automated-indexer-sync afb3f4045] jackett indexers as of 04dfea95ac85778e91b47e7b2361dde4582b7f17 [2026-05-29T20:59:29Z]
 Author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 Date: Wed May 27 02:38:33 2026 +0000
 4 files changed, 39 insertions(+), 398 deletions(-)
 delete mode 100644 definitions/v11/bjshare.yml
 delete mode 100644 definitions/v11/gay-torrentsorg.yml
2026-05-29T15:59:29-0500|INFO|Commit Appended - [jackett indexers as of 04dfea95ac85778e91b47e7b2361dde4582b7f17 [2026-05-29T20:59:29Z]

Removed Indexers: definitions/v11/gay-torrentsorg.yml]
2026-05-29T15:59:29-0500|INFO|Evaluating for Push to Remote
2026-05-29T15:59:29-0500|DEBUG| Push Modes for Branch: automated-indexer-sync
2026-05-29T15:59:29-0500|DEBUG|Push To Remote: true with Force Push With Lease: true
2026-05-29T15:59:29-0500|DEBUG|Push To Remote: true with Force Push With Lease: true
Enumerating objects: 11, done.
Counting objects: 100% (11/11), done.
Delta compression using up to 8 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.38 KiB | 100.00 KiB/s, done.
Total 6 (delta 5), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (5/5), completed with 5 local objects.
remote: 
remote: Create a pull request for 'automated-indexer-sync' on GitHub by visiting:
remote:      https://github.com/ineednewpajamas/Prowlarr_Indexers/pull/new/automated-indexer-sync
remote: 
To https://github.com/ineednewpajamas/Prowlarr_Indexers.git
 * [new branch]          automated-indexer-sync -> automated-indexer-sync
branch 'automated-indexer-sync' set up to track 'origin/automated-indexer-sync'.
2026-05-29T15:59:30-0500|WARN|[origin automated-indexer-sync] Branch Force Pushed
2026-05-29T15:59:30-0500|INFO|SUCCESS|Create pull request: https://github.com/Prowlarr/Indexers/compare/master...ineednewpajamas:automated-indexer-sync

Trace Logs have been provided as applicable. Reports will be closed if the required logs are not provided.

• I have read and followed the steps in the wiki link above and provided the required trace logs - the logs contain trace - that are relevant and show this issue.

[ineednewpajamas]

I'm not sure in what scenario you would want to rebase instead of reset from prowlarr/indexers:master. I was thinking of checking if origin is prowlarr/indexers and if not adding a remote like it does for jackett, and then reset the branch instead of rebase using the prowlarr/indexers:master as the source (unless the dev flags are passed).