Repository navigation
Add phishunt Lookup expansion/hover module - #926
Conversation
Queries https://phishunt.io/api/v1/search.json (no API key) for a domain, hostname or url attribute (only the host of a url is used) and returns misp_standard results. The API does a substring match, so I keep only the rows whose host is exactly the queried one. Each detection becomes a `phishing` object (url, hostname, ip, targeted brand, first seen, phishunt uuid) referenced with 'related-to' from the queried attribute, with hosting, TLS issuer, provider IP range and the other feeds that flag the URL in the object comment, plus one link attribute to the phishunt search page. At most 25 objects are returned. HTTP 429 (20 requests per 10 s per IP) is retried once after a backoff. Only the read-only search endpoint is used: the phishunt analyze endpoints are never called, since they would submit the queried value for analysis. Nothing is tagged, no confidence score is asserted and url/ip are not flagged for IDS export. Tests mock requests.get with verbatim API captures (48 cases). Docs blocks added by hand following the generator layout. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 377f697fa7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| value = str(value).strip().replace("[.]", ".") | ||
| if attr_type != "url": | ||
| return _normalise_host(value) | ||
| if "://" not in value and not value.startswith("//"): |
There was a problem hiding this comment.
For a supported schemeless URL such as example.com/login?next=https://target.test, the embedded redirect URL puts :// in the query even though the outer URL has no scheme. This condition therefore skips prepending //, so urlsplit finds no hostname and the handler returns “Could not extract a host” instead of looking up example.com; detect a scheme only at the start of the value rather than anywhere in it.
Useful? React with 👍 / 👎.
|
Thank you for the contribution! |
What
New
expansion+hovermodule phishunt Lookup (misp_modules/modules/expansion/phishunt.py).phishunt is a free, CC0 feed of live phishing domains, updated every hour. MISP already lists the phishunt feed in the default feeds. This module adds the per-attribute lookup: given a
domain,hostnameorurlattribute (for a URL, only the host is used), it callsGET https://phishunt.io/api/v1/search.json?q=<host>&limit=200(no API key) and returnsmisp_standardresults.The search endpoint matches substrings, so the module keeps only rows whose host is an exact match of the queried host. The module uses only this read-only endpoint. It never calls the phishunt
/analyzeendpoints, because those submit the value for analysis.Mapping
phishingobject (url,hostname,ip,targetbrand,submission-timefrom first seen,internal-referencewith the phishunt id), referenced to the queried attribute withrelated-to;linkattribute to the phishunt search page for the host.At most 25 objects are returned (most recent first). All attributes have
to_idsfalse. Deliberately no tag and no confidence score are set.Rate limit
The phishunt API allows 20 requests per 10 seconds per IP on
/api/*. On HTTP 429 the module waits once (Retry-After, clamped to 1-15 s) and retries one time, then returns a clear error.Tests and docs
tests/test_phishunt.py: 48 pytest cases (33 functions, some parametrized) that mockrequests.getwith captures of the live API: exact-match filtering, URL to host, 429 retry and double 429, the 25 cap, the 200-row search cap, empty result, malformed JSON, unsupported types, hover call without uuid, introspection/version. Same style astests/test_tweetfeed.py.black,isortandflake8clean with the project configuration.README.mdbullet and thedocumentation/README.md/documentation/mkdocs/expansion.mdblocks were added by hand following the generator layout; happy to regenerate if you prefer.Author: Daniel López (phishunt maintainer, also of the TweetFeed module, #898).