Repository navigation
Replies: 1 comment 2 replies
|
A big part of the gap is that you're not comparing the same thing. Your 2000 SonarQube issues are mostly reliability findings plus dependency issues, while security-extended only runs security queries, mostly taint-tracking ones that need a real path from untrusted input to a dangerous sink. CodeQL also doesn't do dependency scanning at all, that's Dependabot's job on GitHub. 21 real security alerts on a mature 500k LOC codebase isn't unusual. If you want something closer to Sonar's scope, run the security-and-quality suites instead (codeql/csharp-queries:codeql-suites/csharp-security-and-quality.qls and the cpp equivalent). They add the null dereference, disposal, useless assignment and similar checks. Two sanity checks worth doing first:
|
|
Thanks @tk1475 . That helps explain the differences I'm seeing.
Ah, yes, in reality I did scan the correct databases. The differences you see in my original post were caused by me making personal notes at different times throughout the investigation.
Thanks, for the hint. I've just now ran this for both the C++ and C# code and together they total about 500k LOC, so it looks like everything was scanned. |
|
👋 PM from GitHub here. Another thing to note is that every scanner (ours vs Sonar vs Snyk vs anyone else out there) is going to find slightly different sets of things. What we prioritize at GitHub is high quality, low-false-positive findings that are relevant and actionable and are less likely to clutter up your backlog/dashboard. In the product itself (which uses CodeQL to perform scans), we have separate security and quality products to surface different types of findings for different workflows (versus Sonar which does it all together in the same view). |
Uh oh!
There was an error while loading. Please reload this page.
Hi,
I’m a complete newbie with CodeQL. I’ve been experimenting with a local scan, and I’ve some doubts about the results I’m seeing, so I’d really appreciate some feedback as to whether I’m following roughly the right steps.
Our intention is to use SAST to help us identify security issues. I’m attempting to run a comparison with SonarQube. Our code is hosted on an Azure DevOps server, and we are using the SonarQube SAST to analyse it. However, it has flagged up over 2000 issues, counting all security, reliability and dependency issues (the majority being reliability issues). My instinct is that these contain a lot of noise: false positives and issues that are theoretical rather than actual real-world issues. The software being scanned is generally stable and has been in use by a large customer base for many years, and our experience tells us that the software is largely reliable.
So, I’ve ran a local CodeQL scan as a test. It reported 21 issues, which seems too low in comparison with the SonarQube results. In summary, SonarQube feels too noisy to be useful and the CodeQL results seems suspiciously low.
Our codebase is roughly half a million lines of C# and C++ code, built with Visual Studio.
I’ve generated the CodeQL database with this command line (the build.cmd runs a Nuke script) …
I’ve downloaded the C++ and C# query packs ...
I’ve generated the sarif files with these …
I’ve used Visual Studio Code with the SARIF Explorer extension to view the sarif files.
As an experiment, I’ve also tried generating the database with a direct call to msbuild …
I’d be very interested to hear your opinions as to why we are only seeing 21 issues raised, when SonarQube raises over 2000. Thanks for your thoughts.
All reactions