Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements in download module, merging folders in sftp and log_summary implementation #255

Merged
merged 23 commits into from
Mar 14, 2024

Conversation

Shettland
Copy link
Member

@Shettland Shettland commented Mar 5, 2024

This pull request has two main objectives.

The first one is to provide the first steps to close #245. For this task, i implemented two small methods (update_summary and create_error_summary) to generate a custom log file in json format using two auxiliar methods (include_error and include_warning) several times along the code which will be emailed to the user after execution.
This first approach might be developed further into a new module which will create the same custom log summary for the rest of the modules in the tools.

The second one is a big enhancement on the code that aims to help this previously mentioned task by merging all the remote subfolders in the sftp repository with batches of samples and metadata from each user into a single one, so each user receives only one single report instead of one for each batch of samples.

@saramonzon
Copy link
Member

Good job! Some minor comments, and a suggestion about how to avoid using log for including sample-based error. I think if the problem is that you don't have the sample_name and only the filename it's easy to create a self.sample_files dict with this information, as this is the json you are going to print as output, and use it when you need it to include the errors. If I'm not mistaken you can use a list comprenhension to add the error to all corrupted files in the list.

@saramonzon saramonzon changed the title Big changes in download module, merging folders in sftp and log_summary implementation Improvements in download module, merging folders in sftp and log_summary implementation Mar 14, 2024
@saramonzon saramonzon merged commit 09c00c1 into BU-ISCIII:develop Mar 14, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New module required to process logs and send email
2 participants