Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make coprdirs have their repo available in the buildroot #3369

Merged
merged 2 commits into from
Mar 3, 2025

Conversation

FrostyX
Copy link
Member

@FrostyX FrostyX commented Aug 11, 2024

This is useful in general but especially for pull requests that build multiple packages which depend on each other (e.g. python-copr and copr-cli). Up until now, it had to be workarounded by building the dependency into the main copr repository.

@praiskup
Copy link
Member

This would bring security issue into the src.fedoraproject.org integration (the pagure events handler).... a single Copr project may integrate multiple packages from s.f.o. being developed by multiple packagers from s.f.o, and all such integrated packages can build from the corresponding "pull-request 1" (separate PRs, heading the same CoprDir!) and then, the package builds are done against each other.

We need to solve the s.f.o integration first, somehow.

Copy link

github-actions bot commented Feb 10, 2025

Pull Request validation

Failed

🔴 Review - Missing review from a member (2 required)

Success

🟢 CI - All checks have passed
🟢 Review - Reviewed by undefined

@FrostyX FrostyX force-pushed the coprdir-buildroot-repo branch from ff65588 to df12563 Compare February 20, 2025 12:29
@FrostyX FrostyX removed the blocked label Feb 20, 2025
@FrostyX FrostyX force-pushed the coprdir-buildroot-repo branch 2 times, most recently from 764cf2d to a14af3f Compare February 20, 2025 12:41
@FrostyX
Copy link
Member Author

FrostyX commented Feb 20, 2025

Took me only half a year but updated, PTAL.

@@ -692,8 +694,7 @@ def validate(cls, copr, dirname):
f"Please use directory format {copr.name}:custom:<SUFFIX_OF_CHOICE> "
f"or {copr.name}:pr:<ID> (for automatically removed directories)"
)

if not all(x.isalnum() for x in dirname.split(":")[1:]):
if not all(x.isalnum() for x in re.split(r"[:-]+", dirname)[1:]):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this deserves an in-line note :-) I'm not sure this is correct

We basically only want to allow one more :-separated field, right? I'd wish we had a test-case for the validation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then, I'm a bit scare of creating custom dirnames, because we don't have a method for deleting them.

Then, note #820 -> will that still work & remove PR dirs after certain period of time?

Copy link
Member

@praiskup praiskup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I overall like where this PR goes, thank you for working on this!

@FrostyX
Copy link
Member Author

FrostyX commented Feb 24, 2025

Mtg: Test for validating dir names and make sure the Cron for cleaning up old dirs still work.

Alternatively, we could separate then through `event_info.user` which would
produce CoprDir names like `foocopr:pr:jdoe-1`. This wouldn't separate the
different packages but it would be good enough to fix the security issue. And it
would produce shorter names than the project URL paths.
This is useful in general but especially for pull requests that build
multiple packages which depend on each other (e.g. `python-copr` and
`copr-cli`). Up until now, it had to be workarounded by building the
dependency into the main copr repository.
@FrostyX FrostyX force-pushed the coprdir-buildroot-repo branch from a14af3f to 34e4ca4 Compare February 26, 2025 16:53
@FrostyX
Copy link
Member Author

FrostyX commented Feb 26, 2025

Test for validating dir names

I didn't change any code but I added a test for the validation method. All patterns that I would expect to fail, fail. @praiskup were you worried about some different ones?

make sure the Cron for cleaning up old dirs still work.

I submitted a build:

copr --config ~/.config/copr.docker build-distgit frostyx/test-coprdir-deletion:pr:foo-bar-1 --name hello

and tried:

[copr-fe@frontend /]$ copr-frontend delete-dirs

which returned nothing. Then I did:

coprdb=# update build set submitted_on=1 where id=8561786;

and tried again:

[copr-fe@frontend /]$ copr-frontend delete-dirs
frostyx/test-coprdir-deletion:pr:foo-bar-1 is going to be deleted

Also looking at the deletion code, there doesn't seem to be any logic related to the dir name. So I think the cleanups should still work as expected.

Copy link
Member

@praiskup praiskup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you!

@praiskup
Copy link
Member

praiskup commented Mar 3, 2025

Only a nit; there are no "Fixes: " labels in git commit messages; but this certainly has some issue counterparts.

@FrostyX FrostyX merged commit d19d2ab into fedora-copr:main Mar 3, 2025
7 checks passed
@hroncok
Copy link
Contributor

hroncok commented Mar 3, 2025

This means we can no longer submit all builds to $COPR:custom:isolated and assume they are isolated from each other. We must now send them to $COPR:custom:isolated-$NUMBER hen each NUMBER is unique, right?

@FrostyX
Copy link
Member Author

FrostyX commented Mar 4, 2025

Hello @hroncok,
I think the answer to your question is "no".

Sorry, I implemented two unrelated things in this PR, to make it more confusing. The "isolation" part of this PR fixes #767, and it is only relevant for :pr: CoprDirs that are automatically created from pagure pull requests. This change should be NOOP for everybody who uses :custom:.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants