Skip to content

Conversation

@lennijusten
Copy link
Contributor

I described issue #9 a while ago and here is the PR with the fix. The fix is a two-part solution.

First, I open the picks.csv file in the write w mode with the csv.writer from the csv library and write the header row. The write w mode will overwrite any existing file with the same filename.

I then open picks.csv in the append a mode and write the results to picks.csv by looping through each batch. The results are converted from arrays to lists with the tolist() method which removes the excess white spaces. I then append the results to the csv with the writerow() method by passing the results in list format. This automatically delimits the csv columns into fname, itp, tp_prob, its, ts_prob and adresses any issues like overfull lines automatically (see issue #6).

I have tried and tested the method and it works as hoped. When reading the csv files with pandas, one should still note that the pandas.read_csv() method will default to reading all entries as strings. This is described fully in a stackoverflow post. An easy fix is using the literal_eval method from ast.

>>> from ast import literal_eval
>>> literal_eval('[1.23, 2.34]')
[1.23, 2.34]

Best,
Lenni

I described this issue in under AI4EPS#9 and here is the PR with the fix. The fix is a two-part solution. 

First, I open the picks.csv file as fclog with the csv library and write the header row. I then open picks.csv in the append mode and write the results to picks.csv batch by batch. The results are converted from arrays to lists (this removes the empty white spaces), and then written to the row in a list of results instead of a single string formatted with the results. This also fixes the new-line issue I had opened and resolved earlier. 

I have tried and tested the method and it works as hoped. 

Best,
Lenni
@lennijusten
Copy link
Contributor Author

Here is an excel copy of an example output.

picks.xlsx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant