Python – Measuring Python script bandwidth usage

bandwidthmonitoringpython

I have a python script that uses urllibs to parse some web pages, and uses selenium to scrape pages with javascript, and I would like to know its cumulative bandwidth usage. I've looked at nethogs, but its tells me the usage per second, but won't tell me how much data the script has received or sent on eth0, for example.

Ideally, I would want it to track all new processes after I start monitoring, and then keep a table of what each process has used as a total.

Does such a tool exist? If not, is there a library in Python that I can implement in the script to track this?

Best Answer

If you are looking for a network monitor that runs as an external process and can provide per-connection statistics on the number of transferred bytes, then IPTraf can do that.

Take a look at the example screenshots, especially the TCP/UDP statistical breakdown.

Related Solutions

Monitoring – Tool to Monitor Bandwidth Usage of a Single Process

something to get you started (just in case you want to write it yourself):

#!/bin/bash
#
# usage: bwmon PID

IN=0; OUT=0; TIME=0

get_traffic() {
    t=`awk '/eth0:/ { printf("%s,%d,%d\n",strftime("%s"),$2,$10); }' < /proc/$1/net/dev`
    IN=${t#*,}; IN=${IN%,*}
    OUT=${t##*,};
    TIME=${t%%,*};
}

get_traffic $1
while true
do
    _IN=$IN; _OUT=$OUT; _TIME=$TIME
    get_traffic $1
    echo "$TIME,$(( $TIME - $_TIME )),$IN,$(( $IN - $_IN )),$OUT,$(( $OUT - $_OUT))"
    sleep 1
done

comments:

checks only eth0
checks every 1 second
works only under linux, but other unixes work similar (procfs or whatever)
output could be stored into a sqlite.db with stat --printf="%N\n" /proc/PID/exe | cut -d ' ' -f 3

Record daily internet bandwidth usage

I use vnstat, which keeps track of daily stats for the last 30 days, and is available in the Ubuntu/Debian (and probably many more) repos.

Just install it and use it like vnstat -i wlan0 -h:

 wlan0                                                                    14:47 
  ^                                                           r                 
  |                                                           r                 
  |                                                           r                 
  |                                                           r                 
  |                                                           r                 
  |                                                           r                 
  |                                                           r                 
  |                                                           r                 
  |                                                     r     r  r     r        
  |                                                     rt r  rt rt    r        
 -+---------------------------------------------------------------------------> 
  |  15 16 17 18 19 20 21 22 23 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14    

 h  rx (KiB)   tx (KiB)      h  rx (KiB)   tx (KiB)      h  rx (KiB)   tx (KiB) 
15          0          0    23          0          0    07          0          0
16          0          0    00          0          0    08     19,287      7,859
17          0          0    01          0          0    09      6,550      3,231
18          0          0    02          0          0    10     65,500      9,216
19          0          0    03          0          0    11     17,491      7,502
20          0          0    04          0          0    12      5,158      2,503
21          0          0    05          0          0    13     15,034      3,493
22          0          0    06          0          0    14      4,284      2,503

Best Answer

Related Solutions

Monitoring – Tool to Monitor Bandwidth Usage of a Single Process

Record daily internet bandwidth usage

Related Question