Repository navigation
fix: [scanmalware] list only browser-contacted hosts, report a degraded CT answer, trailing dot - #935
Merged
Conversation
_is_scan_of_host stripped a trailing dot from the attribute's host but not from the scanned URL's host, so a URL on example.com. never matched the host's own scan of https://example.com./ and the module reported no result. Both hosts are now normalized the same way.
The /hosts endpoint's default sources are observed, scripts and stream. A stream host was never fetched: a third party asked about it and DNS confirms it exists. The module labelled every returned host "contacted by a browser during a scan", so for github.com 156 of the 174 hosts it added carried a false label. It now asks for sources=observed,scripts, the two runtime-observed sources, which is what the label and the module description say.
/ct/dns sheds load under pressure and then answers 200 with degraded: true. The API documents that as "we could NOT look": an empty subdomain list is an absence of data, not a fact about the domain. The module read it as an answer, so a lookup that never happened came back as "No ScanMalware results for this attribute". It now adds a partial result note saying the data could not be read and to retry.
Member
|
Thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three small fixes to the ScanMalware expansion module (#803, #806, #916), one commit each. Disclosure: I run ScanMalware and wrote the module.
/hostsendpoint's default sources areobserved,scriptsandstream, and astreamhost was never fetched (a third party asked about it and DNS confirms it exists). The module now asks forsources=observed,scripts, which is what the label and the module description say. Live today, forgithub.com, 174 hosts carried the label and 18 had been contacted; forexample.com, 4 and 0./ct/dnsanswers 200 withdegraded: true, which the API documents as "we could NOT look": an empty list is an absence of data, not a fact about the domain. The module now adds a partial-result note saying the data could not be read and to retry._is_scan_of_hoststripped the dot from the attribute's host but not from the scanned URL's host, so a URL onexample.com.never matched the host's own scan ofhttps://example.com./. Both sides are now normalized.Tests: 21 pass (3 new, each failing without its fix). black, isort and flake8 clean.