source: http://www.rjsystems.nl/en/2100-adv-routing.php
Linux has very advanced routing, filtering and traffic shaping options.
Here is how to configure a system with two default routes.
1. Installation
To activate Linux advanced routing on a Debian GNU/Linux system, install the
iproute package:
~# apt-get install iproute
This will create the
/etc/iproute2/ directory. It also installs some new executables, including
ip.
2. The ip command
From a command line on any Linux system, you can see the existing routing table by simply typing
route at the prompt
(or
/sbin/route if
/sbin is not in your path). Your routing table will be similar to this:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 * 255.255.255.0 U 0 0 0 eth1
default my.host.com 0.0.0.0 UG 0 0 0 eth1
Advanced routing commands are issued as arguments to the
ip command. To see the routing table with iproute2, you can
use the long version
ip route show table main or the shortcut version
ip route like this:
192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.20
default via 192.168.1.10 dev eth1
3. (Hot) Potato routing
Typically, a host connected to a network, such as the Internet, will
have one default route and one Internet (WAN) interface. However, if a
second connection to the Internet is added and both are accessed, you
can end up with a situation referred to as (hot) potato routing, or
deflection routing.
Normally, when a packet, such as an ICMP ping, arrives at the primary
interface, it is examined by the host, after which a reply packet is
generated, the routing table is consulted and the packet it sent back
via the default route. On the other hand, if a ping packet arrives at
the secondary WAN interface, the same thing happens: it is examined by
the host, a reply packet is generated, the routing table is consulted
and the packet it sent back via the default route. In other words: a
packet is received on one interface and the reply is sent back via the
other.
In theory such packets can be routed, but are dropped by ISPs. That's
because these packets have a source address that is not part of the
network that they are being routed from -- they look like they've been
forged. Indeed, such forged packet headers are often used in DOS
attacks.
However, if you have more than one connection to the Internet and you
want to use them all despite the fact that you have only one default
route, what can you do? The answer is advanced routing.
4. Configuring two default routes
With advanced routing, you can have as many routing tables as you want.
In the example below, we add just one for an extra DSL line
from an ISP called "cheapskate."
First add a name for the new routing table to the
/etc/iproute2/rt_tables file. This can be appended to it with
the command
echo 2 cheapskate >> /etc/iproute2/rt_tables. The result looks like this:
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
2 cheapskate
Above I mentioned that the command
ip route is actually a shortcut for the longer command
ip route
show table main. Since there is no shortcut to list the new routing table, you have no choice but to use the long form:
ip route show table cheapskate. Entering this command now will reveal that this new table is still empty.
All that is necessary is to add the new default route to the
cheapskate table -- the old
main table
will continue to handle the rest. The reason for this will soon become clear. Here is the existing
main table:
~# ip route show table main
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.10
192.168.2.0/24 dev eth1 proto kernel scope link src 192.168.2.10
default via 192.168.1.1 dev eth0
~# _
As follows, add the new default route to table
cheapskate and then display it:
~# ip route add default via 192.168.2.1 dev eth1 table cheapskate
~# ip route show table cheapskate
default via 192.168.2.1 dev eth1
~# _
As you can see, the entire table consists of a single line. However, it is not yet being used. To implement it, the command,
ip rule is required. Routing tables determine packet destinations, but now we need the kernel to use different
routing tables depending on their source addresses. The existing set of ip rules is very simple:
~# ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
~# _
At this point you need to add a new rule:
~# ip rule add from 192.168.2.10 lookup cheapskate prio 1000
This command
adds a rule for when a packet has a
from pattern of
192.168.2.10 in which case the routing table
cheapskate
should be used with a priority level of
1000. In this example the pattern only needs to match one address, but you can set patterns
in a Linux router to match different sets of addresses.
~# ip rule
0: from all lookup local
1000: from 192.168.2.10 lookup cheapskate
32766: from all lookup main
32767: from all lookup default
~# _
The kernel searches the list of ip rules starting with the lowest
priority number, processing each routing table until the packet has been
routed successfully.
The default ruleset always has a
local table with a match pattern of
all. The
local table (priority 0) handles traffic
that is supposed to stay on the localhost, as well as broadcast traffic.
After the
local rule comes our new rule with a priority of 1000. This priority number is arbitrary, but makes it easy to add other
rules before and after it later on.
Our new rule comes before the
main table, which is the one that is modified by the old
route command. The last
rule is for the
default table. I'm not certain what it's for, as I've always found it to be empty, and seeing as there is a
default
route in the table
main, no traffic ever gets to the table
default.
** Warning **
When working with more than one routing table, never forget to add the
table part of the command. If you do forget, rule changes in the
wrong table (
main) can seem awfully mysterious. When learning the ropes and working remotely, you will probably lock yourself out a few
times this way: the changes happen very quickly, so it may be wise to use a console instead.
Another important point to remember is that routes are cached. In other
words, if you update a routing table and nothing seems to happen, it's
because the table is still in memory. The solution is simply to flush
the cache with
ip route flush table cache. In this
manner it is possible to first make a number of changes and then flush
the cache so that all of the changes will be implemented simultaneously.
This is actually convenient when working on an active router.
5. Example configuration
When a secondary WAN interface became available at a client site, I
wanted to allow their remote users to use this connection as an
alternative in case
their primary Internet connection ever went down. With Linux, we now
know that this is possible.
First, some background information. The client's router had the following interfaces:
- eth0 Primary Internet connection (Versatel).
inet addr: 87.215.195.178
Bcast: 87.215.195.183
Mask: 255.255.255.248
- eth0:0 Private net behind the Versatel ADSL modem/router.
inet addr: 192.168.1.1
Bcast: 192.168.1.255
Mask: 255.255.255.0
- eth0:1 Private net behind the Zonnet ADSL modem/router.
inet addr: 10.0.0.100
Bcast: 10.255.255.255
Mask: 255.0.0.0
- eth1 Private net -- main internal network segment.
inet addr: 192.168.13.1
Bcast: 192.168.13.255
Mask: 255.255.255.0
- eth2 Private net -- wireless internal network segment.
inet addr: 192.168.14.1
Bcast: 192.168.14.255
Mask: 255.255.255.0
- lo Loopback interface.
inet addr: 127.0.0.1
Mask: 255.0.0.0
- ppp0 Secondary Internet connection (Zonnet).
inet addr: 62.58.236.234
P-t-P: 195.190.250.17
Mask: 255.255.255.255
The main routing table displayed with
route -n:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
195.190.250.17 0.0.0.0 255.255.255.255 UH 0 0 0 ppp0
87.215.195.176 0.0.0.0 255.255.255.248 U 0 0 0 eth0
62.58.50.0 62.58.236.234 255.255.255.128 UG 0 0 0 ppp0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.14.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.13.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
62.58.232.0 62.58.236.234 255.255.248.0 UG 0 0 0 ppp0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
0.0.0.0 87.215.195.177 0.0.0.0 UG 0 0 0 eth0
The route for 62.58.232.0/21 via ppp0 may be unnecessary, but I figured it would be 'cheaper' because the IP address for ppp0
is part of the same network. The route to 62.58.50.0/25 via the ppp0, on the other hand, is a network segment that includes an
SMTP relay that is not be available via any other route.
The list of interfaces displayed with
ip link list:
1: lo: mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:00:24:c1:10:5c brd ff:ff:ff:ff:ff:ff
3: eth1: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:00:24:c1:10:5d brd ff:ff:ff:ff:ff:ff
4: eth2: mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:00:24:c1:10:5e brd ff:ff:ff:ff:ff:ff
5: sit0: mtu 1480 qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
6: ppp0: mtu 1500 qdisc pfifo_fast
qlen 3 link/ppp
NB: sit0 is an IPv6-IPv4 tunnel.
The main routing table displayed with
ip route show table main:
195.190.250.17 dev ppp0 proto kernel scope link src 62.58.236.234
87.215.195.176/29 dev eth0 proto kernel scope link src 87.215.195.178
62.58.50.0/25 via 62.58.236.234 dev ppp0 scope link
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.1
192.168.14.0/24 dev eth2 proto kernel scope link src 192.168.14.1
192.168.13.0/24 dev eth1 proto kernel scope link src 192.168.13.1
62.58.232.0/21 via 62.58.236.234 dev ppp0 scope link
10.0.0.0/8 dev eth0 proto kernel scope link src 10.0.0.100
default via 87.215.195.177 dev eth0
The idea was to create a second routing table for the second Internet
connection (ppp0) with its own default route. This can be done
in only three steps. First, after installing the necessary software (see
above), I created a second routing table (after the existing
main routing table) called 'zonnet':
~# echo 2 zonnet >> /etc/iproute2/rt_tables
Second, I added a default route to the zonnet routing table using the ppp0 interface and its IP address:
~# ip route add default via 62.58.236.234 dev ppp0 table zonnet
Third, I added a new rule to the kernel that tell it to use the new
routing table when packets (connections) originate from the second
interface:
~# ip rule add from 62.58.236.234 lookup zonnet prio 1000
Thus, the new zonnet routing table looks like this (just one line):
~# ip route show table zonnet
default via 62.58.236.234 dev ppp0
~# _
And the routing rule looks like this:
~# ip rule
0: from all lookup local
1000: from 62.58.236.234 lookup zonnet
32766: from all lookup main
32767: from all lookup default
~# _
So far, the result of all this is that all requests destined for the
firewall coming in from eth0 are sent back out eth0 (the main default
gateway; 87.215.195.177), while requests destined for the firewall
coming in from ppp0 are sent back out ppp0 (the secondary default
gateway; 62.58.236.234). However, if the server responds to any requests
that are forwarded to it, those responses will still be routed out the
main default gateway regardless.
The first step towards a solution was to define a second network,
192.168.15.0/24, on the UTP segment that the server is attached to.
Luckily,
Windows server 2003 allows you to bind additional IP addresses to its
interfaces. In this case, only the server and the firewall (via eth1)
have
addresses on this network.
- eth1:0 Private net -- additional internal network segment.
inet addr: 192.168.15.1
Bcast: 192.168.13.255
Mask: 255.255.255.0
On this network, the server is defined as 192.168.15.2 and the firewall
is configured to forward all requests for it that arrive via ppp0 on to
this address. Naturally, the responses come out this way too.
Second, since all of the packets moving from 192.168/.15.0/24 into the
firewall are responses to requests that arrive via the secondary
Internet
connection (and should be sent back that way) anyway, I could use this
one routing rule:
~# ip rule add from 192.168.15.0/24 lookup secnet prio 990
The routing rule now looks like this:
~# ip rule
0: from all lookup local
990: from 192.168.15.0/24 lookup zonnet
1000: from 62.58.236.234 lookup zonnet
32766: from all lookup main
32767: from all lookup default
~# _
Now if a request is sent in via ppp0 and forwarded on to the server (via
192.168.15.0/24), its response will also be sent back via ppp0.
