Errors with certain wordlists prevent entire wordlist from merging

[TheBugFather]

During some testing I noticed some wordlists merge great while others seem to completely fail. In the attached screenshot there are 4 wordlists where 1 & 2 were merged to result_1.txt and 3 & 4 were merged to result_2.txt in separate runs. By analyzing the sizes of the wordlists and the resulting files, the first two wordlists failed to merge and based on the result the first large file (rockyou.txt) was skipped entirely and the second two merged perfectly on the other run. Based on the error in the log it seems to be UTF-8 encoding related error but it does say much or what is causing the issue. Overall its an awesome tool but it currently is not as stable using cat to sort -u in terms of the resulting wordlist, it ends up being much smaller and should be able to merge common wordlists without issues. Also for reference these are the wordlists used, I had renamed them for ease of use:

wordlist1 - rockyou.txt
wordlist2 - xato-net-10-million-passwords-1000000.txt
wordlist3 - Lizard-Squad.txt
wordlist4 - Ashley-Madison.txt