[Request] Add Verbose Dev-Logging

[import-pandas-as-numpy]

I have a bug to report in Chainsaw, however it is transient and difficult to reproduce; often manifesting when parsing ~5GB+ event logs (which means testing data is also difficult to supply.)

I would appreciate a way to gain insight into where this may be hanging so I can raise a more comprehensive bug report, as at the moment I can simply state that large event logs cause Chainsaw to occasionally simply not continue parsing.

[reece394]

I can confirm I have been seeing similar issues but it happens randomly. I found that reducing the number of rules that are put into memory helps reduce this though particularly sliming down the sigma rules folder to just windows rules

[alexkornitzer]

That is very reasonable i'll try an action that this weekend.

I assume from the comments its hanging and not crashing?

[reece394]

Yeah it gets to a certain point and just hangs. Sometimes rerunning it gets it to complete the second time no issues which makes it more annoying to troubleshoot.

I will put this in here as well just in case. I have noticed that if you run an EDR product/ Antivirus sometimes it reads the rules in memory and likely treats the behaviour as malicious and that can cause a hang. When I whitelisted the binary it reduced it but not completely eliminated the problem so it seems multifaceted.

[alexkornitzer]

Ta for the extra info, i'll wire chainsaw up with -v so then when it happens again we can get some better output. I'll litter debug prints where I think they are needed and we can refine as required.

[alexkornitzer]

Finally got around to adding in logging where I hope will allow us to get to the bottom of the above. If you manage to replicate please try with -v or -vv if we can't work it out with that I will add more logging in. Sorry for the delay.