JEP-202 updates, esp.: remove FilePath.copyFromRemotely from API #117
Conversation
jglick
changed the title
| Going through the master as files are copied from/to the agent cause load, cpu, network issues that are hard to diagnose and recover from. | ||
| * Causing a big performance hit due to the traffic and disk access, preventing the usage of distributed filesystems such as NFS, GlusterFS,... | ||
| * Providing limited functionality compared to a first class blob store (AWS S3, Azure Blob Storage, Google Blobstore) or artifact repository (Nexus, Artifactory). | ||
| * Providing limited functionality compared to a first class blob store (AWS S3, Azure Blob Storage, Google Blobstore). |
There was a problem hiding this comment.
@svanoort read the original version to imply that we would advocate using JEP-202 infrastructure to connect to artifact repositories in addition to blob stores, which is misleading. As noted in jenkinsci/workflow-api-plugin#67 (comment), Jenkins artifacts were never really “competing” with artifact repositories to begin with: people with a reason to use a (particular) artifactory repository should do so and ignore the simplistic notion of “artifact” built into Jenkins. Probably this should be expanded on in another section of the JEP as background.
There was a problem hiding this comment.
@jglick Sorry, but I'm afraid that's putting incorrect words in my mouth -- what I said is that they were "mentioned" originally in JEP-202 which is 100% true. I never said you were advocating for it, but that you seem to be making a completely arbitrary distinction for what can and cannot integrate with the Jenkins artifact mechanism as an external store. As far as I'm concerned the reason seems to be more implementation convenience than anything in the user experience.
I will remind you that you made similar arbitrary assumptions about what users would be doing with Pipeline files, and that they would be quite simple -- assumptions that have proven incorrect.
To me, if feels obvious to ask "okay, so you support S3 now, when can I just use Artifactory/Nexus/whatever and drop the custom steps to upload artifacts?" It lets people simplify their CI/CD cycle by making the actual artifact store a mere configuration detail.
There was a problem hiding this comment.
the reason seems to be more implementation convenience than anything in the user experience
No, the reason is very much user experience. If someone does ask
when can I just use Artifactory/Nexus/whatever and drop the custom steps to upload artifacts?
then the answer will be “never”. It would be a step backwards.
| It was important to implement this in the S3 plugin in order to ensure that Classic or Blue Ocean UI flows relating to artifacts, including simply opening the index page of a build, did not make a large number of network calls. | ||
| The cache system allows a single S3 metadata call (possibly one HTTP request, depending on page size) to retrieve sufficient information for a complete page render of some or all of the artifact tree, without the rendering code (e.g., in Jenkins core) knowing anything about the storage system. | ||
|
|
||
| The `apache-httpcomponents-client-4-api` plugin is extended with a `RobustHTTPClient` |
There was a problem hiding this comment.
This has been released (thanks @dwnusbaum) but unfortunately (INFRA-1600) Javadoc is not published in an unpacked format.
There was a problem hiding this comment.
| Going through the master as files are copied from/to the agent cause load, cpu, network issues that are hard to diagnose and recover from. | ||
| * Causing a big performance hit due to the traffic and disk access, preventing the usage of distributed filesystems such as NFS, GlusterFS,... | ||
| * Providing limited functionality compared to a first class blob store (AWS S3, Azure Blob Storage, Google Blobstore) or artifact repository (Nexus, Artifactory). | ||
| * Providing limited functionality compared to a first class blob store (AWS S3, Azure Blob Storage, Google Blobstore). |
There was a problem hiding this comment.
the reason seems to be more implementation convenience than anything in the user experience
No, the reason is very much user experience. If someone does ask
when can I just use Artifactory/Nexus/whatever and drop the custom steps to upload artifacts?
then the answer will be “never”. It would be a step backwards.
|
Once this is merged—which is on hold, perversely, ultimately waiting for an unrelated jenkinsci/copyartifact-plugin#105 to be fixed up—then #96 should no longer be considered on hold and is mergeable from my PoV. |
|
This is now mergeable. |
6 participants
In case jenkinsci/jenkins#3480 is merged.