Skip to content

Commit 7bf821d

Browse files
adulauadulau
committed
chg: [doc] updated
1 parent 45e9037 commit 7bf821d

File tree

1 file changed

+72
-7
lines changed

1 file changed

+72
-7
lines changed

‎README.md

+72-7
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,59 @@
11
# analyzer-d4-passivedns
22

3-
analyzer-d4-passivedns is an analyzer for a D4 network sensor. The analyser can process data produced by D4 sensors (in [passivedns](https://github.com/gamelinux/passivedns) CSV format (more to come)) and
4-
ingest these into a Passive DNS server which can be queried later to search for the Passive DNS records.
3+
analyzer-d4-passivedns is an analyzer for a D4 network sensor including a complete Passive DNS server. The analyser can process data produced by D4 sensors (in [passivedns](https://github.com/gamelinux/passivedns) CSV format (more to come)) or independently from D4 using [COF websocket](https://datatracker.ietf.org/doc/html/draft-dulaunoy-dnsop-passive-dns-cof) streams.
4+
5+
The package include a Passive DNS server which can be queried later to search for the Passive DNS records.
56

67
# Features
78

8-
- A D4 analyzer which can be plugged to one or more [D4 servers](https://github.com/D4-project/d4-core) to get a stream of DNS records
9-
- A compliant [Passive DNS ReST server compliant to Common Output Format](https://tools.ietf.org/html/draft-dulaunoy-dnsop-passive-dns-cof-04)
9+
- [Input stream] - A D4 analyzer which can be plugged to one or more [D4 servers](https://github.com/D4-project/d4-core) to get a stream of DNS records
10+
- [Input Stream] - A websocket stream (or a file stream) in NDJSON [COF format](https://datatracker.ietf.org/doc/html/draft-dulaunoy-dnsop-passive-dns-cof)
11+
- [Output API] A compliant [Passive DNS ReST server compliant to Common Output Format](https://tools.ietf.org/html/draft-dulaunoy-dnsop-passive-dns-cof)
1012
- A flexible and simple analyser which can be configured to collect the required records from DNS records
1113

1214
# Overview
1315

1416
## Requirements
1517

1618
- Python 3
17-
- Redis >5.0
19+
- Redis >5.0 or [kvrocks](https://github.com/apache/incubator-kvrocks)
1820
- Tornado
1921
- iptools
2022

2123
## Install
2224

25+
### Redis
26+
2327
~~~~
2428
./install_server.sh
2529
~~~~
2630

2731
All the Python 3 code will be installed in a virtualenv (PDNSENV).
2832

33+
### Kvrocks
34+
35+
~~~
36+
./install_server_kvrocks.sh
37+
~~~
38+
39+
All the Python 3 code will be installed in a virtualenv (PDNSENV).
40+
2941
## Running
3042

31-
### Start the redis server
43+
### Start the redis server or kvrocks server
3244

3345
Don't forget to set the DB directory in the redis.conf configuration. By default, the redis for Passive DNS is running on TCP port 6400
3446

3547
~~~~
3648
./redis/src/redis-server ./etc/redis.conf
3749
~~~~
3850

51+
or
52+
53+
~~~~
54+
./kvrocks/src/kvrocks -c ./etc/kvrocks.conf
55+
~~~~
56+
3957
### Start the Passive DNS COF server
4058

4159
~~~~
@@ -44,7 +62,17 @@ cd ./bin/
4462
python3 ./pdns-cof-server.py
4563
~~~~
4664

47-
### Configure and start the D4 analyzer
65+
## Feeding the Passive DNS server
66+
67+
You have two ways to feed the Passive DNS server. You can combine multiple streams. A sample public COF stream is available from CIRCL with the newly seen IPv6 addresses and DNS records.
68+
69+
### (via COF websocket stream) start the importer
70+
71+
~~~~
72+
python3 pdns-import-cof.py --websocket ws://crh.circl.lu:8888
73+
~~~~
74+
75+
### (via D4) Configure and start the D4 analyzer
4876

4977
~~~~
5078
cd ./etc
@@ -70,6 +98,43 @@ cd ./bin/
7098
python3 pdns-ingestion.py
7199
~~~~
72100

101+
## Usage
102+
103+
### Querying the server
104+
105+
~~~~shell
106+
adulau@kolmogorov ~/git/analyzer-d4-passivedns (master)$ curl -s http://127.0.0.1:8400/query/xn--ihuvudetpevap-xfb.se | jq .
107+
{
108+
"time_first": 1657878272,
109+
"time_last": 1657878272,
110+
"count": 1,
111+
"rrtype": "AAAA",
112+
"rrname": "xn--ihuvudetpevap-xfb.se",
113+
"rdata": "2a02:250:0:8::53",
114+
"origin": "origin not configured"
115+
}
116+
~~~~
117+
118+
~~~~shell
119+
curl -s http://127.0.0.1:8400/query/2a02:250:0:8::53
120+
{"time_first": 1657878141, "time_last": 1657878141, "count": 1, "rrtype": "AAAA", "rrname": "media.vastporten.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
121+
{"time_first": 1657878929, "time_last": 1657878929, "count": 1, "rrtype": "AAAA", "rrname": "www.folkinitiativetarjeplog.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
122+
{"time_first": 1657878272, "time_last": 1657878272, "count": 1, "rrtype": "AAAA", "rrname": "xn--ihuvudetpevap-xfb.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
123+
{"time_first": 1657878189, "time_last": 1657878189, "count": 1, "rrtype": "AAAA", "rrname": "media.primesteps.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
124+
{"time_first": 1657878986, "time_last": 1657878986, "count": 1, "rrtype": "AAAA", "rrname": "media.skellefteaadventurepark.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
125+
{"time_first": 1657874940, "time_last": 1657874940, "count": 1, "rrtype": "AAAA", "rrname": "galleri.torsaspaintball.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
126+
{"time_first": 1657874205, "time_last": 1657874205, "count": 1, "rrtype": "AAAA", "rrname": "www.media1.harlaut.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
127+
{"time_first": 1657878165, "time_last": 1657878165, "count": 1, "rrtype": "AAAA", "rrname": "www.sd-nekretnine.rs", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
128+
{"time_first": 1657878678, "time_last": 1657878678, "count": 1, "rrtype": "AAAA", "rrname": "www.www2.resultat-balans.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
129+
{"time_first": 1657874288, "time_last": 1657874288, "count": 1, "rrtype": "AAAA", "rrname": "www.assistanshemtjanst.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
130+
{"time_first": 1657878943, "time_last": 1657878943, "count": 1, "rrtype": "AAAA", "rrname": "kafekultur.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
131+
{"time_first": 1657878141, "time_last": 1657878141, "count": 1, "rrtype": "AAAA", "rrname": "media1.rlab.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
132+
{"time_first": 1657878997, "time_last": 1657878997, "count": 1, "rrtype": "AAAA", "rrname": "serbiagreenbuildingexpo.com", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
133+
{"time_first": 1657879064, "time_last": 1657879064, "count": 1, "rrtype": "AAAA", "rrname": "www.framtro.nu", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
134+
{"time_first": 1657874285, "time_last": 1657874285, "count": 1, "rrtype": "AAAA", "rrname": "www.twotheartist.com", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
135+
{"time_first": 1657878774, "time_last": 1657878774, "count": 1, "rrtype": "AAAA", "rrname": "media.narkesten.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
136+
~~~~
137+
73138
# License
74139

75140
The software is free software/open source released under the GNU Affero General Public License version 3.

0 commit comments

Comments
 (0)