Skip to content

Conversation

@jglick
Copy link
Member

@jglick jglick commented Jan 23, 2019

While discussing an OutOfMemoryError whose investigation led to jenkinsci/workflow-durable-task-step-plugin#96, @dwnusbaum pointed out that FilePath.readToString winds up allocating a IOUtils.DEFAULT_BUFFER_SIZE ~ 4Kib buffer. This is clearly inefficient for the common case of reading a small text file, not to mention the overhead of using ProxyOutputStream in the Remoting channel if the file is remote. Better in typical cases to just read the whole file at once remotely and send back one packet with the result. (The current implementation would only make sense for huge text files, and using this method for such cases is likely a sign of poor code anyway.)

Note that I avoided FileUtils.readFileToString from Commons IO since it has the same 4Kib buffer size, perversely.

Also note that the rewritten implementation alters the behavior slightly: the default charset being used is now that of the agent, rather than the master. This is generally what you would want, and matches the behavior of FilePath.write(content, null).

Proposed changelog entries

  • Internal optimization in the method to read a remote file as text.

@jglick jglick requested a review from dwnusbaum January 23, 2019 19:53
Copy link
Member

@oleg-nenashev oleg-nenashev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@oleg-nenashev oleg-nenashev merged commit 8b61d46 into jenkinsci:master Jan 27, 2019
@jglick jglick deleted the FilePath.readToString branch January 28, 2019 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants