Crash Count

[rmachnee]

Hi,
I was comparing the performance between Jackalope and WinAFL with DynamoRIO and found when fuzzing the same target with the same corpus and offset that Jackalope found around 30 crashes while WinAFL only found 7.

Some of the crashes from Jackalope had access addresses that are close like:
access_violation_xxxxx76A_xxxxx861_1
access_violation_xxxxx76A_xxxxx873_1

OR

access_violation_xxxxx76A_xxxxx3DE_1
access_violation_xxxxx76A_xxxxx3C0_1

Just wondering why Jackalope is finding 4x more crashes under the same conditions?
Is this a sensitivity thing where maybe its picking up the same crash but labeling it differently?

[ifratric]

Assuming one fuzzer isn't finding bugs other isn't (which can happen due to very different approaches used in both), it could be just the way these fuzzers are deduplicating crashes, which is also very different. WinAFL, like AFL, uses coverage for crash deduplication, while Jackalope isn't using coverage at all and instead is using exception information. Both approaches can produce duplicates.

In Jackalope, for most cases, deduplication is based on the instruction address (value of instruction pointer) that caused the exception, except for access violation which also includes the access address. Addresses are "anonymized" with 'x' characters to account for ASLR. Access address is included to differentiate cases like null-pointer dereferences (where address would be 0) from something that looks exploitable. It is true that this can cause duplicates where the same instruction address causes exceptions at multiple access addresses, but in practice this is usually an indication of a "good" bug so I consider it an acceptable tradeoff.

Usually it is a good practice to rerun the crashes under page heap (on windows) or guard malloc (on mac) which might help deduplicate issues further and point to the actual root cause of the bugs.