Skip to content

Adding functionality to set files_per_task per sample type#85

Merged
nshadskiy merged 5 commits intomainfrom
dev_custom_files_per_task
May 5, 2026
Merged

Adding functionality to set files_per_task per sample type#85
nshadskiy merged 5 commits intomainfrom
dev_custom_files_per_task

Conversation

@a-monsch
Copy link
Copy Markdown
Contributor

This change adds the functionality in KingMaker_luigi.cfg to further customize files_per_task argument to optionally set it per sample_type as specified in datasets.json of sample_database

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for overriding files_per_task based on a dataset’s sample_type (as defined in the sample database), configured via KingMaker_luigi.cfg.

Changes:

  • Introduce a custom_files_per_task luigi parameter (dict) to support per-sample_type overrides.
  • Apply custom_files_per_task during branch map creation in CROWNRun to adjust task splitting.
  • Update KingMaker_luigi.cfg with a new custom_files_per_task option and related documentation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
processor/tasks/CROWNRun.py Applies the per-sample_type override when determining files_per_task for branch splitting.
processor/tasks/CROWNBase.py Adds the custom_files_per_task luigi parameter to workflow tasks.
processor/framework.py Changes default nanoAOD_version value (appears unrelated to PR goal).
lawluigi_configs/KingMaker_luigi.cfg Adds config option and docs for custom_files_per_task; also adjusts a section header line.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread processor/framework.py
Comment thread processor/tasks/CROWNBase.py Outdated
Comment thread lawluigi_configs/KingMaker_luigi.cfg Outdated
Comment thread processor/tasks/CROWNRun.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread processor/framework.py
Comment thread lawluigi_configs/KingMaker_luigi.cfg Outdated
Comment thread processor/tasks/CROWNRun.py Outdated
Comment thread processor/framework.py
Comment thread lawluigi_configs/KingMaker_luigi.cfg Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nshadskiy nshadskiy merged commit 2306d88 into main May 5, 2026
5 checks passed
@nshadskiy nshadskiy deleted the dev_custom_files_per_task branch May 5, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants