Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor S3Writer class to improve code structure #440

Merged
merged 2 commits into from
Sep 19, 2024

Conversation

jfzunigac
Copy link
Contributor

S3Writer refactor changes included:

  • Delete unused and redundant helper methods and class fields
  • Introduce helper functions to reduce code duplicates
  • Change File.renameTo() to Files.move() for buffer file renaming. The latter is much more reliable and offers better error handling
  • Consolidate upload logic to one single method
  • Add more comments and javadoc
  • Add metric for number of messages written to buffer (so it can be compared with upstream logger metric)
  • Introduce "logName" derived from singer log config name to use instead of logStream.getLogStreamName()

@jfzunigac jfzunigac requested a review from a team as a code owner September 17, 2024 23:17
@jfzunigac jfzunigac changed the title refactor S3Writer code Refactor S3Writer class to improve code structure Sep 17, 2024
@jfzunigac jfzunigac merged commit 2d17e71 into pinterest:master Sep 19, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants