I have a slightly more complicated configuration, which also includes NAT and dynamic (internal) routing on the machine.
There are several parts to it. Let's start with the rules:
Maginot:~# ip rule ls
0: from all lookup local
1000: from all lookup main
1500: from 173.167.51.136/29 lookup comcast
1501: from 76.160.165.106/29 lookup cavtel
1502: from 151.200.251.90/31 lookup vzdsl
1502: from 151.200.251.92/31 lookup vzdsl
1502: from 151.200.251.94 lookup vzdsl
2000: from all fwmark 0x1 lookup comcast
2001: from all fwmark 0x2 lookup cavtel
2002: from all fwmark 0x3 lookup vzdsl
2500: from all lookup comcast
2501: from all lookup cavtel
2502: from all lookup vzdsl
32767: from all lookup default
(The tables are given names in /etc/iproute2/rt_tables
)
As you can see, we have three ISPs at the moment, and we a couple static IP addresses on each. Rules 1500–1502 send the traffic from those source IP addresses out the appropriate interface. Rules 2000–2002 send traffic with a given firewall mark (we'll get to that) out the appropriate interface. Rules 2500–2502 give the preference order of our ISPs, for traffic that hasn't been assigned to an ISP already. If one goes down, you remove the rule for it, and the next one on the list is used.
Each routing table is fairly trivial:
Maginot:~# ip route ls table comcast
default via 173.167.51.142 dev comcast
(Local and main have a lot more stuff, but that's directly connected and internal routes, respectively. Table default is empty).
Next, its important that once a connection is assigned to an ISP, it stays there (including incoming connections). Trying to move it is impossible, since we've got different IP addresses on each interface, and our ISPs actually have reverse-path filtering. We keep them on the same ISP with firewall rules.
Remember that SNAT is done after routing, so the ip rule
policies will not help. You have to use something else (maybe this is your problem?)
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j RETURN # if already set, we're done
iptables -t mangle -A PREROUTING -i wan -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A PREROUTING -i comcast -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A PREROUTING -i vz-dsl -j MARK --set-mark $MARK_VZDSL
iptables -t mangle -A POSTROUTING -o wan -j MARK --set-mark $MARK_CAVTEL
iptables -t mangle -A POSTROUTING -o comcast -j MARK --set-mark $MARK_COMCAST
iptables -t mangle -A POSTROUTING -o vz-dsl -j MARK --set-mark $MARK_VZDSL
iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
This is from the shell script that sets it up; the $MARK_…
variables match the marks you see in the rules above. Fairly simple: restore the connection mark (remember, marks are per-packet); if there is a mark now (from the connection mark), we're done. Otherwise, set the mark to the one based on the interface involved.
Note it restores the mark before routing, and only saves it afterwards. And the mark is set to the outgoing interface regardless (which is arguable).
Finally, there are the actual NAT rules. We have a several local prefixes, this code is run in a for loop with $local
set to each:
iptables -t nat -A POSTROUTING -s $local -o wan -j SNAT --to-source 76.160.165.106
iptables -t nat -A POSTROUTING -s $local -o comcast -j SNAT --to-source 173.167.51.137
iptables -t nat -A POSTROUTING -s $local -o vz-dsl -j SNAT --to-source 151.200.251.90
(Note: There are a lot more rules, including some DNAT for DMZ servers, etc... I think I've copied all the relevant ones.)
Best Answer
The plain protocol is specifically designed to do what you request. With MPTCP you can
ppp0
andeth0
)If one of your paths fails, MPTCP will enable you to transparently fail-over to another known path (your peer needs to be aware of this path, of course). If your prior failed path becomes available again, you can switch back. If you addresses change in between, you can tell your peer it happened, as long as one usable path stays available (since you'll need a channel to communicate the change).
But you'll have to keep in mind that this only works if not only your client but also
In a perfect world, we'd all be using SCTP, *sigh*.