Trying to get exporting to prometheus working

[mikhael47]

I have a couple of issues,

First, my nfdump records all have timestamp of 1969-12-31 19:00:00.000

Second I am getting the following error when trying to export to Prometheus

/usr/local/bin/nfcapd -z -m /data/metrics -p 3456 -M /data/nsx2 -D
<this command is actually dumping flows into /data/nsx2/

/usr/local/bin/nfcapd -l /data/tmpflows/ -S2 -y -p 9999 -m /data/metrics
-l is a legacy option and may get removed in future. Please use -w to set output directory
Bound to IPv4 host/IP: any, Port: 9999
Init v5/v7: Default sampling: 1
Init v9: Max number of v9 tags enabled: 105, default sampling: 1
Init IPFIX: Max number of ipfix tags enabled: 88, default sampling: 1
connect() failed on /data/metrics: Connection refused
metric socket unreachable
Metric initialized
Startup nfcapd.
Ident: 'none' Flows: 0, Packets: 0, Bytes: 0, Sequence Errors: 0, Bad Packets: 0

[phaag]

The timestamps reflects what your device sends you. I assume it is not sending any timestamps at all. To debug that issue, I would need to have a cap of the stream sent to the collector nfcapd. Furthermore, I assume you have a TZ issue, as 1969 is actually not possible with UNIX timestamps.

As of the metric - You need to have nfexporter https://github.com/phaag/nfexporter running, which reads data from the metric socket and feeds it into prometheus.

[mikhael47]

There is a screen capture of the timestamp and a "no template found" warning.

I was under the impression that nfdump 1.7 had metric export baked into it.

I really do appreciate your time here! I've been using nfdump for years. Management has asked me to get the metrics into prometheus for them.

[phaag]

You need to collect more data, until the required template is available. Then I would need this template record. Alternatively you can email me the pacp to the email address in the AUTHORS file.

[mikhael47]

Here is what nfcap is seeing

Flow Record:
Flags = 0x00 NETFLOW v10, Unsampled
Elements = 4: 1 2 4 12
size = 104
engine type = 0
engine ID = 0
export sysid = 3
first = 0 [.000]
last = 0 [.000]
received at = 1737572801425 [2025-01-22 14:06:41.425]
proto = 17 UDP
tcp flags = 0x00 ........
src port = 49034
dst port = 53
src tos = 0
in packets = 1
in bytes = 172
src addr = 10.32.5.240
dst addr = 10.32.3.6
input = 0
output = 0
src mask = 0 /0
dst mask = 0 /0
fwd status = 0
dst tos = 0
direction = 0
biFlow Dir = 0x00
end reason = 0x00
ip exporter = 10.0.41.52

[mikhael47]

Ok, I switched gears a bit and instead of exporting for NSX-T's Distributed Firewall, I changed it over to the Distributed Switch and all the fields are coming through. Also, later on in my firewall exports, I do see some flows with time stamps, so I suspect you are right, there is something going on with the template export frequency.