-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add multiple mirror sync with extensive include/exclude pattern matching #639
Draft
YYYasin19
wants to merge
36
commits into
mamba-org:main
Choose a base branch
from
YYYasin19:mirror-sync
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
cb4f948
start: allow multiple mirrors per package, extensive include/exclude …
YYYasin19 267b94a
use condas MatchSpec utility
YYYasin19 557cec0
package spec relies on = while our repo package names use '-'
YYYasin19 df8a217
validate that package was therefore before delete attempt is issued
YYYasin19 5815902
update: include condas matchspec
YYYasin19 de27969
refactor: extract repodata retrieval || add: package matching on more…
YYYasin19 dbd3f75
add: support for excludelist
YYYasin19 1d4af06
add: new database model for accessing channel urls
YYYasin19 0f07838
fix: not set mirror_channel_url
YYYasin19 a5e3847
fix: mirror_channel_urlS
YYYasin19 8217368
add: new tests for validating pattern checking
YYYasin19 c3c5467
refactor: channel membership is completely refactored
YYYasin19 b2af339
add + refactor: option to remove local packages that would _not_ fit …
YYYasin19 76e7c29
refactoring: clean up code, extract methods, document some others
YYYasin19 1f6be80
debug: update version to validate deployment
YYYasin19 8b4b965
add custom URLType for validating urls
YYYasin19 223e29a
fix rest models
YYYasin19 3ca604c
Update docs/source/using/mirroring.rst
YYYasin19 854d471
Update quetz/dao.py
YYYasin19 cf5cc7a
fix
YYYasin19 f6480b7
validate that proxy mode only allows a single mirror url
YYYasin19 e0f3bf1
add more commments
YYYasin19 ce554b6
add docs
YYYasin19 299d89b
add comment
YYYasin19 1afd522
rename method
YYYasin19 0c83779
add type hints
YYYasin19 a3b39db
use dist for parsing
YYYasin19 fc5c4bf
sooner package creation based on cond
YYYasin19 ff90938
simplify package removal from database
YYYasin19 5f951d0
update comments
YYYasin19 4108791
replaced Pydantic Field Validation with Annotated by constr
YYYasin19 2ed0e86
use enum instead of concrete values for mirror mode
YYYasin19 183f482
use enum instead of concrete values for mirror mode
YYYasin19 b51ef67
add docstring
YYYasin19 5ded7dd
add docstring
YYYasin19 c0acdbc
fix import
YYYasin19 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,12 +7,19 @@ | |
import uuid | ||
from datetime import date, datetime | ||
from enum import Enum | ||
from typing import Annotated, Dict, Generic, List, Optional, TypeVar, Union | ||
from typing import Dict, Generic, List, NewType, Optional, TypeVar, Union | ||
|
||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator | ||
from pydantic import ( | ||
BaseModel, | ||
ConfigDict, | ||
Field, | ||
constr, | ||
field_validator, | ||
model_validator, | ||
) | ||
|
||
T = TypeVar('T') | ||
URLType = Annotated[str, Field(pattern="^(http|https)://.+")] | ||
URLType = NewType('URLType', constr(pattern="^(http|https)://.+|None|^(\\[.*\\])+$")) | ||
|
||
|
||
class BaseProfile(BaseModel): | ||
|
@@ -66,9 +73,7 @@ class ChannelBase(BaseModel): | |
private: bool = Field(True, title="channel should be private") | ||
size_limit: Optional[int] = Field(None, title="size limit of the channel") | ||
ttl: int = Field(36000, title="ttl of the channel") | ||
mirror_channel_url: Optional[Union[URLType, List[URLType]]] = Field( | ||
None, | ||
) | ||
mirror_channel_url: Optional[Union[URLType, List[URLType]]] = Field(None) | ||
mirror_mode: Optional[MirrorMode] = Field(None, nullable=True) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we make this an enum so that only There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this not already an enum..? |
||
|
||
@field_validator("size_limit") | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you have a URL type and define a pattern? https://docs.pydantic.dev/latest/usage/types/urls/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, my thought was that due to implementation of
AnyHttpUrl
in Pydanticthis won't work as well.
But afaik, they only require Python 3.7 which we do as well.