* [LARTC] TEQL question
@ 2000-12-17 1:00 Rene
2000-12-17 19:39 ` bert
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Rene @ 2000-12-17 1:00 UTC (permalink / raw)
To: lartc
<PRE>Hello, LARTC!
I have a question concerning TEQL. I have set up two Linux routers for
experimental purposes (kernel 2.4.0-test9). Every router controls 8 modems, one
of the machines is configured as a PPP dial-in server (ppp 2.4b4).
The idea is to increase throughput between both machines via TEQL queuing. This
works quite well, but unfortunately TEQL distributes the network load not as
equal as the name TEQL might suggest. If I use ICMP floods all 8 links are
uniformly under load. The same is true for UDP floods. When using single TCP
transfers (for example SCP from the SSH package) not all 8 links are used. The
packets use 3 or 4 links of the 8.
The questions is why. I have done some experiments with various applications
(FTP, NFS, SMTP, Real Audio/Video, Netmeeting, ELSA Vision). So far TEQL uses
only all 8 links when I flood the routers with packets.
Any ideas? I would like to distribute the load over all 8 links.
Here are the configs (brief, note that some options are heavily influenced by
my experiments and not meant to be used in "production" environments):
PPP options dial-in router:
----------------------------
# PPPD options
SERVER_PPP="192.168.20.1 192.168.20.2 192.168.20.3 192.168.20.4
192.168.20.5 192.168.20.6 192.168.20.7 192.168.20.8"
SERVER_NET="192.168.20.0/24"
CLIENT_PPP="192.168.00"
CLIENT_NET="192.168.0.0/24"
CLIENT="192.168.10.230"
MTU="1500"
SERVER_OPT="lock asyncmap 0 maxfail 0 idle 300 proxyarp
logfile /var/log/ppp noauth debug kdebug 3 noipdefault
persist crtscts"
MULTILINK="multilink"
IPCP_OPTS="lcp-max-configure 30 lcp-max-failure 30 ipcp-max-terminate 10
ipcp-max-configure 20 ipcp-max-failure 20"
COMP_OPTS="novj novjccomp nobsdcomp nodeflate"
SPEEDW600
After all 8 PPP daemons are initialized I set the routing:
# Activate teql0 and set routing
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
$TCC qdisc add dev lo root teql0
$IPC link set teql0 mtu $MTU up
$TCC qdisc del dev lo root teql0
$IPC route add $CLIENT_NET dev teql0
$IPC route add $CLIENT dev teql0
$IPC and $TCC is the path to the ip and the tc binary. The pppX devices are
automatically added to the TEQL queue via the PPP daemon scripts
/etc/ppp/if-up.local and /etc/ppp/if-down.local.
/etc/ppp/if-up.local (basically):
# Put link to teql0 heap
[ ${DEBUG} = "yes" ] && logger "Adding $1 to teql0 queue"
$TC qdisc add dev $1 root teql0
[ ${DEBUG} = "yes" ] && logger "Adding local address $4 to teql0"
$IP addr add $4 dev teql0
[ ${DEBUG} = "yes" ] && logger "Deleting route to remote address $5"
$IP route del $5
/etc/ppp/if-down.local:
# Remove link from teql0 heap
[ ${DEBUG} = "yes" ] && logger "Removing $1 from teql0 queue"
$TC qdisc del dev $1 root teql0
PPP client machine:
--------------------
# PPPD options
SERVER_PPP="192.168.0.1 192.168.0.2 192.168.0.3 192.168.0.4
192.168.0.5 192.168.0.6 192.168.0.7 192.168.0.8"
SERVER_NET="192.168.0.0/24"
SERVER="192.168.10.231"
CLIENT_PPP="192.168.20"
CLIENT_NET="192.168.20.0/24"
MTU="1500"
#CLIENT_OPT="-detach lock crtscts persist demand idle 1200
noipdefault init /etc/ppp/chat/init.sh debug kdebug 2
holdoff 5 noauth novj nobsdcomp nodeflate multilink
logfile /var/log/ppp"
CLIENT_OPT="nolock asyncmap 0 maxfail 0 holdoff 2 demand persist
crtscts passive noipdefault proxyarp logfile /var/log/ppp
noauth debug kdebug 3"
PCP_OPT="lcp-max-configure 20 lcp-max-failure 20"
COMPT_OPT=""
SPEED\x19200
# Get all active PPP links
cd /proc/sys/net/ipv4/conf/
PPPDEVS=`ls -1d ppp*`
# Disable RP filter
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
# Equalize links
for LINK in $PPPDEVS
do
echo "Adding $LINK..."
$TCC qdisc add dev $LINK root teql0
$IPC addr add `/usr/local/bin/getpppinfo -l $LINK` dev teql0
$IPC route del `/usr/local/bin/getpppinfo -r $LINK`
done
# Activate teql0 and set routing
$IPC link set teql0 mtu $MTU up
$IPC route add $SERVER_NET dev teql0
$IPC route add $SERVER dev teql0
Sorry for the script mess. I was glad to see it work and did not have much time
for code clean-ups.
Best regards,
René
GNU/Linux Manages! - Support, Administration, Consulting
RP3191-RIPE - Networking, Programming, Installation
</PRE>
^ permalink raw reply [flat|nested] 4+ messages in thread* [LARTC] TEQL question 2000-12-17 1:00 [LARTC] TEQL question Rene @ 2000-12-17 19:39 ` bert 2003-02-19 23:14 ` [LARTC] teql question Bartlomiej Zarzecki 2003-02-20 14:54 ` Bartek Krajnik 2 siblings, 0 replies; 4+ messages in thread From: bert @ 2000-12-17 19:39 UTC (permalink / raw) To: lartc <PRE>On Sun, Dec 17, 2000 at 02:00:45AM +0100, Rene 'Lynx' Pfeiffer wrote: ><i> uniformly under load. The same is true for UDP floods. When using single TCP </I>><i> transfers (for example SCP from the SSH package) not all 8 links are used. The </I>><i> packets use 3 or 4 links of the 8. </I> I think this is how it is supposed to work, I'm afraid. What happens is this. The outgoing side initially queues a few packets, but not too much. That is how TCP works - by receiving ACKnowledgement packets from the remote, the sending side knows how much data it can send. It will now wait until the remote ACKs the first three packets, say, and it will then send a couple more. The problem is that while your bandwidth has gone up, your latency hasn't. This means that in order to use all 8 of your links, the sending side should ship 8 packets 'into the blind', without acknowledgement. You might make this possible by raising the window size on the receiving side. However, then there is the dark problem of packet reordering. Lets say 8 packets are sent simultaneously, they can arrive in any order and it is highly unlikely that this order is 1 2 3 4 5 6 7 8. Lets say the order is 1 3 8 2 6 5 4 7, the receiving side might interpret it as follows: 1: ok -> ack up to 1 3: missed packet 2 -> ack up to 1 8: missed packet 2, 4, 5, 6, 7 -> ack up to 1 2: missed packet 4, 5, 6, 7 -> ack up to 3 6: missed packet 4, 5, 7 -> ack 3 5: missed packet 4, 7 -> ack up to 3 4: missed packet 7 -> ack up to 6 7: ack up to 7 The sending side receives the following: ack 1, ack 1, ack 1, ack 3, ack 3, ack 3, ack 6, ack 7 This will probably be interpreted as 'packet 1 got lost, lets resend it', 'packet 3 got lost, lets resend it'. This is not the case however, but all this resending makes the sending kernel decide to slow down, and lower the 'congestion window'. This means that the kernel will not allow a lot of packets to be 'in flight' simultaneously. ><i> Any ideas? I would like to distribute the load over all 8 links. </I> If you have a lot of separate streams going, it gets better. Also, if the receiving side is Linux, it might get less confused by this packet reordering. I think regular Linux is able to reorder up to 3 out of place packets. ><i> GNU/Linux Manages! - Support, Administration, Consulting </I>><i> RP3191-RIPE - Networking, Programming, Installation </I> Good luck! Regards, bert hubert -- PowerDNS Versatile DNS Services Trilab The Technology People 'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet </PRE> ^ permalink raw reply [flat|nested] 4+ messages in thread
* [LARTC] teql question 2000-12-17 1:00 [LARTC] TEQL question Rene 2000-12-17 19:39 ` bert @ 2003-02-19 23:14 ` Bartlomiej Zarzecki 2003-02-20 14:54 ` Bartek Krajnik 2 siblings, 0 replies; 4+ messages in thread From: Bartlomiej Zarzecki @ 2003-02-19 23:14 UTC (permalink / raw) To: lartc Hello lartc, Is it possible to create more than one teql device ? If yes how to do it ? -- Pozdrawiam, Bartlomiej mailto:4bart@astercity.net _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [LARTC] teql question 2000-12-17 1:00 [LARTC] TEQL question Rene 2000-12-17 19:39 ` bert 2003-02-19 23:14 ` [LARTC] teql question Bartlomiej Zarzecki @ 2003-02-20 14:54 ` Bartek Krajnik 2 siblings, 0 replies; 4+ messages in thread From: Bartek Krajnik @ 2003-02-20 14:54 UTC (permalink / raw) To: lartc [-- Attachment #1: Type: text/plain, Size: 286 bytes --] ----- Forwarded message from kuznet@ms2.inr.ac.ru ----- Hello! > Is it possible to create more than one teql device ? If yes how to > do it ? Insert another sch_teql module under a different name. This will create device teql1 and so on. Alexey ----- End forwarded message ----- [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2003-02-20 14:54 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2000-12-17 1:00 [LARTC] TEQL question Rene 2000-12-17 19:39 ` bert 2003-02-19 23:14 ` [LARTC] teql question Bartlomiej Zarzecki 2003-02-20 14:54 ` Bartek Krajnik
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.