Linux – Ubuntu Linux Router: Set outbound NAT on 2 interfaces

Here is a similar setup from one of our routers (with some irrelevant stuff snipped). Note that this handles incoming connections as well.

Note the use of variables instead of hard-coded mark numbers. So much easier to maintain! They're stored in a separate script, and sourced in. Table names are configured in /etc/iproute2/rt_tables. Interface names are set in /etc/udev/rules.d/70-persistent-net.rules.

##### fwmark ######
iptables -t mangle -F
iptables -t mangle -X

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j RETURN # if already set, we're done
iptables -t mangle -A PREROUTING -i wan      -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A PREROUTING -i comcast  -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A PREROUTING -i vz-dsl   -j MARK --set-mark $MARK_VZDSL

iptables -t mangle -A POSTROUTING -o wan     -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A POSTROUTING -o comcast -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A POSTROUTING -o vz-dsl  -j MARK --set-mark $MARK_VZDSL
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark

##### NAT ######
iptables -t nat -F
iptables -t nat -X
for local in «list of internal IP/netmask combos»; do
    iptables -t nat -A POSTROUTING -s $local -o wan     -j SNAT --to-source «IP»
    iptables -t nat -A POSTROUTING -s $local -o comcast -j SNAT --to-source «IP»
    iptables -t nat -A POSTROUTING -s $local -o vz-dsl  -j SNAT --to-source «IP»
done

# this is an example of what the incoming traffic rules look like
for extip in «list of external IPs»; do
    iptables -t nat -A PREROUTING   -p tcp -d $extip --dport «port» -j DNAT --to-destination «internal-IP»:443
done

And the rules:

ip rule flush
ip rule add from all               pref 1000  lookup main 
ip rule add from A.B.C.D/29        pref 1500  lookup comcast # these IPs are the external ranges (we have multiple IPs on each connection)
ip rule add from E.F.G.H/29        pref 1501  lookup cavtel
ip rule add from I.J.K.L/31        pref 1502  lookup vzdsl
ip rule add from M.N.O.P/31        pref 1502  lookup vzdsl # yes, you can have multiple ranges
ip rule add fwmark $MARK_COMCAST   pref 2000  lookup comcast
ip rule add fwmark $MARK_CAVTEL    pref 2001  lookup cavtel
ip rule add fwmark $MARK_VZDSL     pref 2002  lookup vzdsl
ip rule add                        pref 2500  lookup comcast # the pref order here determines the default—we default to Comcast.
ip rule add                        pref 2501  lookup cavtel
ip rule add                        pref 2502  lookup vzdsl
ip rule add                        pref 32767 lookup default

The routing tables get set up in /etc/network/interfaces, so that taking down an interface makes it switch to using a different one:

iface comcast inet static
        address A.B.C.Q
        netmask 255.255.255.248
        up ip route add table comcast default via A.B.C.R dev comcast
        down ip route flush table comcast

Note: If you're doing filtering as well (which you probably are) you'll also need to add the appropriate rules to FORWARD to ACCEPT the traffic. Especially for any incoming traffic.

I got it that I had to flush the device before bringing it up:

# ip addr flush dev eth1

Clearing manually set interface configuration information like this is mentioned in the Ubuntu Server Guide.