Server with two dsl-lines
evets dranem
edranem "at" dranem.org
Wed Mar 9 12:20:01 2005
Windows <http://www.winguides.com/registry/display.php/951/>
Load Balance Network Adapters /(Windows NT/2000/XP)/ /Popular/
If you have two or more network cards in your system this setting allows
you to distribute the number of connections, or sessions among the
adapters according to a randomizing algorithm.
Open your registry
<http://www.winguides.com/registry/article.php?id=1&page=3> and find or
create the key below.
Create a new DWORD value, or modify the existing value, called
"RandomAdapter" and set it according to the value data below.
Exit your registry, you may need to restart or log out of Windows for
the change to take effect.
/*Note:* The Single Response
<http://www.winguides.com/registry/display.php/952/> setting should also
be enable for this setting to be effective. /
Registry Editor Example
| Name Type Data |
| (Default) REG_SZ (value not set) |
| RandomAdapter REG_DWORD 0x00000001 (1) |
-
| HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NetBT\P... |
-
*_Registry Settings_*
*System Key:* <javascript:popuphelp('registry-settings','system_key')>
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NetBT\Parameters]
*Value Name:* <javascript:popuphelp('registry-settings','value_name')>
RandomAdapter
*Data Type:* <javascript:popuphelp('registry-settings','date_type')>
REG_DWORD (DWORD Value)
*Value Data:* <javascript:popuphelp('registry-settings','value_data')>
(0 = disabled, 1 = enabled)
*_Related Tweaks_*
* *Control Network Adapter Responses to a WINS Query*
<http://www.winguides.com/registry/display.php/952/> /(Windows
NT/2000/XP)/
*Disclaimer:* Modifying the registry can cause serious problems that may
require you to reinstall your operating system. We cannot guarantee that
problems resulting from modifications to the registry can be solved. Use
the information provided at your own risk.
Linux
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking>
<http://www.lartc.org/howto/> linux Advanced Routing& Traffic
as mentioned BY DEFAULT this is not an easy out of box experience ;-)
manual routes may be your current best choice for low cost [whether time
or Ca$h]
although I saw this
4.2.2. Load balancing
The second question is how to balance traffic going out over the two
providers. This is actually not hard if you already have set up split
access as above.
Instead of choosing one of the two providers as your default route, you
now set up the default route to be a multipath route. In the default
kernel this will balance routes over the two providers. It is done as
follows (once more building on the example in the section on split-access):
ip route add default scope global nexthop via $P1 dev $IF1 weight 1 \
nexthop via $P2 dev $IF2 weight 1"
This will balance the routes over both providers. The *weight*
parameters can be tweaked to favor one provider over the other.
Note that balancing will not be perfect, as it is route based, and
routes are cached. This means that routes to often-used sites will
always be over the same provider.
Furthermore, if you really want to do this, you probably also want to
look at Julian Anastasov's patches at http://www.ssi.bg/~ja/#routes
<http://www.ssi.bg/%7Eja/#routes>, Julian's route patch page. They will
make things nicer to work with.
-=-=-=-=-=-=-=-=-=-=
Networking.MultihomedLinuxNetworkingr1.8 - 06 Mar 2005 - 15:36 -
NguyenDinhNam
<http://selab.edu.ms/twiki/bin/view/Main/NguyenDinhNam>topic end
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking#TopicEnd>
------------------------------------------------------------------------
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking#Actions>
------------------------------------------------------------------------
*/Note: This article is intended to be written for lartc
<http://www.lartc.org/howto/> so the numbering is from 4./*
4. Rules - routing policy database
4.1. Simple source policy routing
4.2. Multihomed Linux Networking
This article tries to show you the "prove of concept" solution for the
problem: We have a multihomed host - the host with multiple up-links (I
use "link" to mention the connection to the internet to avoid confusing
with TCP connections), how can we use these links efficiently? This is
the basic article so some issues could be missed, most of them are
intended to be so, in exchange for simplicity. I have an Advanced
Multihomed Linux Networking^?
<http://selab.edu.ms/twiki/bin/edit/Networking/AdvancedMultihomedLinuxNetworking?topicparent=Networking.MultihomedLinuxNetworking>
article covering missing issues. The techniques discussed here *doesn't
require you to patch the kernel* (unless you have a
not-so-normal-network like described in 4.2.5) so it's a big plus for
manageability - stock kernels are always more stable than customized one.
4.2.1. Network topology
Let's assume that we have 2 DSL links, each comes with its own modem
that allows us to connect to the internet by PPPOE. Setting up PPPOE
connections is easy, by using rp-pppoe or pppoe kernel module, if you
want to know more about PPPOE, go to PPPoEWithLinux
<http://selab.edu.ms/twiki/bin/view/Networking/PPPoEWithLinux> to
consult my PPPOE experiences.
After setting up PPPOE, we have 2 ppp connections: ppp0 with IP address
of 11.1.1.1 and ppp1 with IP address of 22.2.2.2 but we have to config
PPPOE to *not setup the default route*. We will load balance 2 links
using special routes so it's not desired if PPPOE setup a default
gateway automatically.
/
+-----------------+ +---------+ DSL link X / +------
| 11.1.1.1 ppp0 +---+ Modem X +--------------+ ISP
| | +---------+ | +------
| Linux host | | Internet
| | +---------+ | +------
| 22.2.2.2 ppp1 +---+ Modem Y +--------------+ ISP
+-----------------+ +---------+ DSL link Y \ +------
\
4.2.2. Incoming connections
A friend from the IP address of X.X.X.X wants to connect to 11.1.1.1
(link X), another friend, from Y.Y.Y.Y wants to connect to 22.2.2.2
(link Y), now we want to work with both of them simultaneously. it's
clear that ISPs have the responsibility to route packets coming from
them to our host via the correct link, so our job is just to route
outgoing packets from us to them via the corresponding interface. Note:
We have to route to corresponding because most ISPs will DROP ip
spoofing packets.
Outgoing packets belong to the connection from X.X.X.X has the source of
11.1.1.1, and outgoing packets of the connection from Y.Y.Y.Y has the
source of 22.2.2.2. Thus we need to create 2 tables - one for each of
the sources:
# ip rule add prio 1 from 11.1.1.1 lookup 1
# ip rule add prio 2 from 22.2.2.2 lookup 2
Then set default routes for packets from each source
# ip route add table 1 to default dev ppp0
# ip route add table 2 to default dev ppp1
Now we can handle connections from both interfaces, if we want to run a
web server on the host, we can setup a round robin DNS to load balance
requests effectively now.
4.2.3. Load balancing outgoing connections
Now we try to ping gnu.org to see what we have:
# ping 199.232.76.164
connect: Network is unreachable
another try:
# ping -I 11.1.1.1 199.232.76.164
PING 199.232.76.164 from 11.1.1.1 : 56(84) bytes of data.
64 bytes from 199.232.76.164: icmp_seq=1 ttl=47 time=151 ms
We realize that we can ping only if we bind the ICMP packet to a proper
IP address! It's because we have only created routes for packets having
source addresses of 11.1.1.1 and 22.2.2.2, while packets initializing
new outgoing connections don't have any source address yet.
We can solve this by creating a generic route for all packets that don't
match any of our 2 source addresses. In this case, we will spread these
packets equally to our links to balance the loads, this technique is
called multipath route:
# ip route add to default nexthop dev ppp0 nexthop dev ppp1
Now we have finished load balancing our host to the internet using 2
different links.
Note that this type of load balancing will not be perfect, as it is
connections based and route based, some connections could use more
bandwidth than the others, and routes are cached. Caching makes routes
to often-used sites may always be over the same link (this route based
problem is discussed in the advanced article^?
<http://selab.edu.ms/twiki/bin/edit/Networking/AdvancedMultihomedLinuxNetwroking?topicparent=Networking.MultihomedLinuxNetworking>,
in that article, I also propose a decent solutions to trick the cache to
make multipath routes more balance). But in most cases, in the long run,
we will have an acceptable balance between 2 links.
4.2.4. Acting as an internet gateway
Usually, only 1 broadband link is enough for most single hosts. Here we
have a multihomed host, usually, it's because that host will serve as
the internet gateway for a big LAN. In this section, we will discuss
about MASQUERADE/SNAT in a multihomed server. Firstly - IP forwarding is
required:
# echo 1 > /proc/sys/net/ipv4/ip_forward
Secondly - binding each connection to just one link. It's important
because usually, ISPs avoid IP address spoofing. Thus no matter how many
links we have, each connection can only choose just one link and stick
with that link until it's closed.
The first packet of each outgoing connection is free to choose it's link
(the decision is made by the multipath route), then all following
packets of that connection must respect that decision. With techniques
for stateless firewalls, we have no clue to recognize which link is
chosen for the first packet. But a stateful firewall of liunux can do
it. CONNMARK can store a connection based value in ip_conntrack so we
can use it to mark the link chosen for the first packet. Based on that
value, the routing system will route following packets to the correct
link. Note: From kernel 2.6.10, CONNMARK is included in the vanilla
kernel, prior than that, you have to patch the kernel with patch-o-matic
to get CONNMARK.
Now, mark first packets based on the interface selectected by the
multipath route
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 1 \
-m state --state NEW -o ppp0
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 2 \
-m state --state NEW -o ppp1
# iptables -A POSTROUTING -t mangle -j CONNMARK --save-mark \
-m state --state NEW
We restore the saved mark for following packets before it reach the
routing system
# iptables -A PREROUTING -t mangle -j CONNMARK --restore-mark
Route following packets based on the restored mark
# ip rule add fwmark 1 lookup 1
# ip rule add fwmark 2 lookup 2
The last step - Source-NAT them to the public IP addresses before send
to the internet so the returning packets know where to go back to us
# iptables -A POSTROUTING -t nat -m mark --mark 1 \
-j SNAT --to-source 11.1.1.1
# iptables -A POSTROUTING -t nat -m mark --mark 2 \
-j SNAT --to-source 22.2.2.2
Note: Some people know SNAT as the name of MASQUERADE but actually,
MASQUERADE is just a special case of SNAT, don't try to use MASQUERADE
with multipath routes, you may get undesired results such as Route sent
us somewhere else
<https://lists.netfilter.org/pipermail/netfilter-devel/2004-January/013699.html>
problem
Now you we'll see that the computers in LAN can access to the internet
via both 2 links perfectly, congratulations!
4.2.5. Dealing with DHCP
<http://selab.edu.ms/twiki/bin/view/Networking/DHCP> links and
routers
Sometimes, you may want to use existing routers instead of PPPOE. For
example, when you have internet gateways available in your LAN already,
and you want to route packets there to make sure that you do everything
correctly before changing the topo of your network. The other example is
that your ISPs assign IP to your eth interface by DHCP
<http://selab.edu.ms/twiki/bin/view/Networking/DHCP>.
If you have 1 NIC for each router, everything is too simple, just
replace ppp interfaces by eth interfaces. For example, if we have 2
routers 10.0.0.1 and 10.0.0.2 connected to eth1, then we replace
# ip route add table 1 to default dev ppp0
# ip route add table 2 to default dev ppp1
# ip route add to default nexthop dev ppp0 nexthop dev ppp1
by
# ip route add table 1 to default via 10.0.0.1
# ip route add table 2 to default via 10.0.0.2
# ip route add to default nexthop via 10.0.0.1 nexthop via 10.0.0.2
In the previous scenario, we have rules:
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 1 \
-m state --state NEW -o ppp0
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 2 \
-m state --state NEW -o ppp1
the conditions "-o ppp0" or "-o ppp1" help MARK-ing different links with
different values. When there are more than one router wired to one
network interface (all connected to a switch) we will have a trouble
because although links have different via values - "via 10.0.0.1" and
"via 10.0.0.2", outgoing interface is the same - eth1. Thus just
replacing outgoing interface (-o match) will make 2 identical conditions.
In this case, we will need rules matching based on the "via" values
("via 10.0.0.1" or "via 10.0.0.2"). Samuel Jean made a patch-o-matic to
do it, but the guys at netfilter team refuses to add it to the POM
repository so you can't find it on netfilter's website, Samuel makes it
available here: download nexthop POM
<http://svn.cookinglinux.org/view/netfilter/browser/nexthop/trunk>
With nexthop patch applied, we replace
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 1 \
-m state --state NEW -o ppp0
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 2 \
-m state --state NEW -o ppp1
by
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 1 \
-m state --state NEW -m nexthop --nexthop-ip 10.0.0.1
# iptables -A POSTROUTING -t mangle -j MARK --set-mark 2 \
-m state --state NEW -m nexthop --nexthop-ip 10.0.0.2
4.2.6. Port Forwarding (or DNAT in general)
... I'm busy for sometime, I'll continue writing soon ...
4.2.7. A real world application
Now you known the concept, but you will soon know that you want more - a
complete solution that can solve all kind of multihomed networks. For
example, the links fail quite often, or PPPOE links have dynamic IP
addresses, or you have cable links where they assign your dynamic IP
addresses by DHCP <http://selab.edu.ms/twiki/bin/view/Networking/DHCP>,
then every SNAT rules need to be changed frequently, then you need a
daemon to watch over everything and tune the network settings dynamically.
I've been working on a GPL licensed daemon that do everything necessary
for a multihomed host, it includes all techniques discussed here as well
as countless advanced features. Look at RoutesKeeper Project
<http://selab.edu.ms/twiki/bin/view/Networking/RoutesKeeperProject> for
more info.
to top
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking#PageTop>
------------------------------------------------------------------------
End of topic
Skip to action links
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking#Actions>
| Back to top
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking#PageTop>
------------------------------------------------------------------------
*Edit*
<http://selab.edu.ms/twiki/bin/edit/Networking/MultihomedLinuxNetworking?t=1110369884>
| Attach image or document
<http://selab.edu.ms/twiki/bin/attach/Networking/MultihomedLinuxNetworking>
| Printable version
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking?skin=print.pattern>
| Raw text
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking?raw=on>
| More topic actions
<http://selab.edu.ms/twiki/bin/oops/Networking/MultihomedLinuxNetworking?template=oopsmore¶m1=1.8¶m2=1.8>
Revisions: | r1.8 | >
<http://selab.edu.ms/twiki/bin/rdiff/Networking/MultihomedLinuxNetworking?rev1=1.8&rev2=1.7>
| r1.7
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking?rev=1.7>
| >
<http://selab.edu.ms/twiki/bin/rdiff/Networking/MultihomedLinuxNetworking?rev1=1.7&rev2=1.6>
| r1.6
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking?rev=1.6>
| Total page history
<http://selab.edu.ms/twiki/bin/rdiff/Networking/MultihomedLinuxNetworking>
| Backlinks
<http://selab.edu.ms/twiki/bin/search/Networking/SearchResult?scope=text®ex=on&excludetopic=MultihomedLinuxNetworking&search=Multihomed%20*Linux%20*Networking%5B%5EA-Za-z0-9%5D>
You are here: Networking
<http://selab.edu.ms/twiki/bin/view/Networking/WebHome> >
NetworkingTutorial
<http://selab.edu.ms/twiki/bin/view/Networking/NetworkingTutorial> >
MultihomedLinuxNetworking
to top
<http://selab.edu.ms/twiki/bin/view/Networking/MultihomedLinuxNetworking#PageTop>
Copyright ) 2004-2005 by SELAB and contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
<mailto:webmaster "at" selab.edu.ms?subject=TWiki%20Feedback%20on%20Networking.MultihomedLinuxNetworking>