I have a freebsd box, which serves as a router from LAN to the outside world. It has several Internet providers, OpenVPN servers and clients, NAT and so on.
I would like to have complete statistics on when, who, to whom, via which provider, via which protocol, and how many bytes have been sent.
I agree that the minimal time scale will be an hour, i.e. no need for millisecond precision.
I tried vnstat2, ntopng and some other programs, but they don't seemed to have what I need or it is not obvious.
I don't understand, why this feature is not easily available. But I am not good in freebsd, so please, give me some clues.
Best Answer
For getting details of network transactions, you have got a implementation of a Netflow generator for FreeBSD or Linux:
ng_netflow
As for NetFlow itself:
also rfc 3954 - NetFlow Services Export Version 9
For storing the Netflow data you also need what is know as a server collector. It can be either a Linux or a FreeBSD box. It should not be installed on the actual router. One such known implementation is nfsen
Be aware that, depending on your available bandwidth, generating NetFlows can be taxing on the CPU. A known strategy in some cases is doing a mirror of the switch port of the router, and using another machine for those operations.
After a certain threshold of bandwidth it probably makes more sense going for a professional router if generating NetFlows is a requirement.
As a final alert, having NAT, the NetFlows have to be captured in the inside/LAN interface, as otherwise you will lose the sense of whom is doing what.
I use up around 100GB of data for 5-6 months of traffic, using NfSen collecting NetFlow data from Cisco equipment, your mileage may vary.