Command Line – Resolve All IP Addresses in Command Output

command linehostnamepipesed

I have several log files that contain a bunch of ip addresses. I would love to be able to pipe the data through a program that would match and resolve ip addresses.

I.E.
cat /var/log/somelogfile | host

which would turn a line like

10:45 accessed by 10.13.13.10

into

10:45 accessed by myhostname.intranet

My thought is there might be a way to do this with a combination of sed and host, but I have no idea how to do so. I know that I could write a simple script that would do it, but I would rather be able to use built in tools if possible. Any suggestions?

Best Answer

Here's a quick and dirty solution to this in Python. It does caching (including negative caching), but no threading and isn't the fastest thing you've seen. If you save it as something like rdns, you can call it like this:

zcat /var/log/some-file.gz | rdns
# ... or ...
rdns /var/log/some-file /var/log/some-other-file # ...

Running it will annotate the IP addresses with their PTR records in-place:

$ echo "74.125.132.147, 64.34.119.12." | rdns
74.125.132.147 (rdns: wb-in-f147.1e100.net), 64.34.119.12 (rdns: stackoverflow.com).

And here's the source:

#!/usr/bin/env python

import sys, re, socket

cache = dict()

def resolve(x):
    key = x.group(0)
    try:
        return "%s (rdns: %s)" % (key, cache[key])
    except KeyError:
        try:
            cache[key] = socket.gethostbyaddr(key)[0]
        except socket.herror:
            cache[key] = '?'
        return "%s (rdns: %s)" % (key, cache[key])

for f in [open(x) for x in sys.argv[1:]] or [sys.stdin]:
    for line in f:
        sys.stdout.write(re.sub("\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", resolve, line))

# End of file.

Please note: this isn't quite what you're after to the letter (using ‘standard tools’). But it probably helps you more than a hack that resolves every IP address every time it's encountered. With a few more lines, you can even make it cache its results persistently, which would help with repeat invocations.

Related Question