-
Notifications
You must be signed in to change notification settings - Fork 2
file_mount validation limits for python client #660
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
48a3c0f
61c00ca
cccf210
5266abb
640c4e9
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,6 +23,7 @@ | |
| async_to_raw_response_wrapper, | ||
| async_to_streamed_response_wrapper, | ||
| ) | ||
| from .._constants import FILE_MOUNT_MAX_SIZE_BYTES, FILE_MOUNT_TOTAL_MAX_SIZE_BYTES | ||
| from ..pagination import SyncBlueprintsCursorIDPage, AsyncBlueprintsCursorIDPage | ||
| from .._exceptions import RunloopError | ||
| from ..lib.polling import PollingConfig, poll_until | ||
|
|
@@ -50,6 +51,33 @@ class BlueprintRequestArgs(TypedDict, total=False): | |
| __all__ = ["BlueprintsResource", "AsyncBlueprintsResource", "BlueprintRequestArgs"] | ||
|
|
||
|
|
||
| def _validate_file_mounts(file_mounts: Optional[Dict[str, str]] | Omit) -> None: | ||
| """Validate file_mounts are within size constraints. | ||
|
|
||
| Currently enforces a maximum per-file size to avoid server-side issues with | ||
| large inline file contents. Also enforces a maximum total size across all | ||
| file_mounts. | ||
| """ | ||
|
|
||
| if file_mounts is omit or file_mounts is None: | ||
| return | ||
|
|
||
| total_size_bytes = 0 | ||
| for mount_path, content in file_mounts.items(): | ||
| # Measure size in bytes using UTF-8 encoding since payloads are JSON strings | ||
| size_bytes = len(content.encode("utf-8")) | ||
| if size_bytes > FILE_MOUNT_MAX_SIZE_BYTES: | ||
| raise ValueError( | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might be a good idea to catch all large files instead of just the first one encountered.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Great feedback! Making this change now. Thank you |
||
| f"file_mount '{mount_path}' exceeds maximum size of {FILE_MOUNT_MAX_SIZE_BYTES} bytes. Use object_mounts instead." | ||
| ) | ||
| total_size_bytes += size_bytes | ||
|
|
||
| if total_size_bytes > FILE_MOUNT_TOTAL_MAX_SIZE_BYTES: | ||
| raise ValueError( | ||
| f"total file_mounts size exceeds maximum of {FILE_MOUNT_TOTAL_MAX_SIZE_BYTES} bytes. Use object_mounts instead." | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will we expect users to use file_mounts in combination with object_mounts? If so, it might be helpful to display what their total size is or how much they've exceeded the MAX_SIZE by, so they can make an informed partition of file_mounts vs object_mounts.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the API needs to be flexible enough to support arbitrary combinations of I think this change is worth making, but I wanted to provide more context as to why. |
||
| ) | ||
|
|
||
|
|
||
| class BlueprintsResource(SyncAPIResource): | ||
| @cached_property | ||
| def with_raw_response(self) -> BlueprintsResourceWithRawResponse: | ||
|
|
@@ -144,6 +172,8 @@ def create( | |
|
|
||
| idempotency_key: Specify a custom idempotency key for this request | ||
| """ | ||
| _validate_file_mounts(file_mounts) | ||
|
|
||
| return self._post( | ||
| "/v1/blueprints", | ||
| body=maybe_transform( | ||
|
|
@@ -758,6 +788,8 @@ async def create( | |
|
|
||
| idempotency_key: Specify a custom idempotency key for this request | ||
| """ | ||
| _validate_file_mounts(file_mounts) | ||
|
|
||
| return await self._post( | ||
| "/v1/blueprints", | ||
| body=await async_maybe_transform( | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Might be a good idea to test that we don't reject file mounts when under or precisely at the limit
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These are just client-side validations. There's no point being hyper-precise about these limits since they're really enforced on the server side anyway -- there's nothing to stop a caller ignoring our client & using our API endpoints directly.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Makes sense! |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we keep these limits static or adjust them based on Devbox resource allocation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This limit isn't a devbox limitation; it's a blueprint dockerfile length constraint. I think that each
file_mountperforms a text substitution inside the dockerfile, so the problem is that the dockerfile becomes too massive; not that the file_mounts exhaust resources for the devbox the dockerfile gets compiled onThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see, makes sense!