From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carl-Daniel Hailfinger Date: Tue, 26 Jul 2005 12:45:02 +0000 Subject: Re: [LARTC] Latency of Linux Bridge Message-Id: <42E6304E.5050208@gmx.net> MIME-Version: 1 Content-Type: multipart/mixed; boundary="------------050609050307050403090006" List-Id: References: In-Reply-To: To: lartc@vger.kernel.org This is a multi-part message in MIME format. --------------050609050307050403090006 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Christian Konecny (VI/SEA) schrieb: >=20 > Carl-Daniel wrote: >> That's strange. On my bridge with 4 network interfaces, >> the additional latency is always below 0.5 ms, even if >> I'm pushing 400 MBit/s through the machine and a kernel >> compile is running at 100% CPU. Network interfaces are >> PCIe GBit from Syskonnect, the machine is an Athlon64 >> at 2 GHz. Even if the clock speed is halved by powersave >> the additional latency will not go above 0.9 ms. >> Kernel is vanilla 2.6.11.x. >=20 > Did you use certain specific compile options? Depends. I chose to use a slightly modified .config from SUSE 9.3 (attached). > I have changed now from (Knoppix) Debian to (Slax) > Slackware running now 2.6.12.2 and have exactly the > same on each machine. > top shows me a CPU usage less than 0.5%, load average > 0.1 while the bridge is handling roughly 64kBit/sec. > I can still measure a variable delta between packets > of 5ms-10ms. > How did you measure your latency values? (with bridge in between) linux:~ # ping -f -c 1000 192.168.0.1 PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data. --- 192.168.0.1 ping statistics --- 1000 packets transmitted, 1000 received, 0% packet loss, time 963ms rtt min/avg/max/mdev =3D 0.642/0.914/2.487/0.277 ms, ipg/ewma 0.964/1.023 ms (same config without bridge in between) linux:~ # ping -f -c 1000 192.168.0.1 PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data. --- 192.168.0.1 ping statistics --- 1000 packets transmitted, 1000 received, 0% packet loss, time 883ms rtt min/avg/max/mdev =3D 0.767/0.834/4.925/0.173 ms, ipg/ewma 0.884/0.814 ms > I am using 3 PCs, one is the Linux-Box acting as bridge > (brctl addbr br0; brctl addif br0 eth0; brctl addif br0 > eth1), the other 2 PCs are connected to either side of > the bridge, running ethereal. I'm using 2 PCs (one as bridge, one as local client) and a Cisco Pix (slow, insecure, unstable) as remote router (192.168.0.1). The measaurements were done from the local client. > In my case I have 2 Phone systems generating continous traffic. Each > system is sending out packets every 30ms. > If I compare the 2 traces I can see that after the bridge the timings > are different then before the bridge. It is very possible that a bridge changes the timing distribtion (but the drastic effect you're seeing shouldn't happen). > The difference is always in steps of 5ms.=20 > so, on sending side is always 30ms difference between each packet > on receiving side - after the bridge - the delta is then 25,30,35, or 40m= s. >=20 > Is this really different in your setup? Since the remote side is not a linux box, my chances to measure the timing distribution are somewhat limited. Please try the flood ping I did above (ping -f -c 1000 re.mo.te.ip) and report your results. If you can see latencies above 2 ms something is definitely going wrong. Could be the nic, the nic settings (NAPI), the timing source or packet type (ICMP vs. IP). 1. Check ping 2a. If ping looks wrong -> try my .config and check again 2b. If ping looks OK -> try the phone while running (ping -f re.mo.te.ip) 3. Report back. Gr=FC=DFe aus T=FCbingen Carl-Daniel --=20 http://www.hailfinger.org/ --------------050609050307050403090006 Content-Type: application/x-gzip; name="config.gz" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="config.gz" H4sIAFkf5kICA4xcW3PbOLJ+31/Bmnk4SdUmkWRbY09VHkAQlDAiCBoAdZkXlmIziU5kySvL M/G/3waoC0CAmn2IbX7duDUafcElv/7r1wi97rdPy/3qYblev0Xf6k29W+7rx+hp+aOOHrab r6tvv0eP283/7aP6cbWHEtlq8/oz+lHvNvU6+qvevay2m9+jwcfhx37/42/AkO5WEXreRf2b qHf9e//295ubaNDr3fzr139hnqd0VM1vh9Xw+vPb8Xt4HVN1/gTy+YOx8vwhZpKwakRyIiiu ZEHzjOPJmX6kYJTRWCBFqoRkaOFUXWFWzPF4dAYJEtmiKgTNlVXXuCCqUpQR0cIIKzNdtVDY b5lKVCUMBQicoQJgkMKvEd4+1iDi/etutX+L1vVfIMrt8x4k+XKWEpkXUJKRXKHsXB/OCMor zFlBM3KGY8EnJK94Xkl2amZkpnMdvdT71+dzxSAzlE2JkJTnn3/58H25eqkfev1fjnQ5Mz09 fi3klBbWUAsu6bxi9yUp7Q7IBETIMZGyQhirbko1vXJqx8oaHioTWxXMp+ZBmcWUcai1TCs5 pqn63L89TQ9XRVZaMzvh8R8EWizJFMR4xumk+cNHTE8Bhjk6zAKLSZKQJFq9RJvtXsvyVDt0 Si6YtNmPWAW/7SI+A5krgaoCSRmoOi0VmVsaWnAz/lMtGFe8AOWkf5Iq5aKS8EegGjlmhFm6 g6FbdJRD9TlWMPvyc8+jZSgmWZDAeRHC/yiZwU+dUzRfNE3bXTIamW2Xj8sva1D+7eMr/Hp5 fX7e7vZn3WQ8KTMirfVvgKqElY4SD4axY5/IY8kzAksUuAokmFPsoPh+E1LgA61ytG0C9OOK Knbbh/rlZbuL9m/PdbTcPEZfa72Oa2vhssmtPVcgHRxUhZO1KsrA3GlTlfXBlOExqeKFAqEM r4PEZhkMbZqS2DV6I86TChXUghnFoOs8IZ+fbE4mhQtA92jiQpS36jL90Wal3YYSwpYFGKeg KApBCCtUkJaXDAXkM0JCaatq+wc5o1xlsdsvhokHVGDrSeZb6TESCRX3MmC/xb22DTE5qcL2 73oHRnyz/FY/1Zu9b8ALZg+9YOCM4nIUWqY8VTMkYCZLWZDc0mQoJBWYTxgrVcZc/9K0jnBB o3fo8a/l5gHcNTae+hV8N3TBqGXTPbrZ17uvy4f6fSTbS01XYVla+KpizlUL0oISMD3K9oOG IjNCihBmbGiVyhYN4bMSNa0hBbUu2mipFAzUBac0IbyFpajNdfAyXLRwNSaCgRN1UQTSbkE0 Zu2iHFZXjAA9TaTBcSkVhxmViQpMaDOQDOFJRqWqFhBh2AbSkLuU4SCEtvQIbgEFn3lTUuD2 jILvVLYLaDrPIaagoNcwrCYeAQuU7ur/vNabh7foBYLB1ebbWVOAXKWC3Dse6IB1juPEIBVS Z6k6MBSGjmTWWK1qU1RmCgzXtII4CAw9Q7lZyaEunHlLCTa8QJhc6pJfaZBDS1iiKQn03mmq g87zhED9SYCsUJwRS/pa+NHzybk87lY6snYjZtOdnM8qcC9PHYTWotZUsCgkASUoKgwxEAS5 3C3t0wOVNGoD/dRacOj2y/flDkzP2eydJqZdojEIYElSLyCIX1+ONUTvQH2jev/w8b1lQ22N ho8KzDOxo0uNMXYK5059KHMuEiJIAj4ioAqmmKSOgQYoIyOEF6a/YT8FPDliRHoDKTAG36HH wDBFn/By9whje29FOFYlhtWv4cMDlIq+7FaP30w8cYBpNN7un9ev33wbfoh4jZieAiAYyQkJ U8wkd1JcU2hTsTMlbUr1p7q5uel1Fz061DCHHBfYtrVtWqUD4cpE8zpsI54Qyc/64XVvQsyv K/1ju4M86yX6FJGn1/Wy5aFjmqcMMrostdKVBmNUWn2k6GpwSPy0F35zcMRLSyF1aoaUlz95 OARV/eExmEjqv1YPdZSc1v05f1s9HOCI+ysNjGieoIznIXMHXtsIKqWCmfAiLmlmBRfprNJx s3EDpxqNOa8SQSEW9qTL6qft7i1S9cP3zXa9/fZ26DisXaaS93bP4NvX8CVko2vIdbUSB1cG EgUXyi+old9ENevlW7BgXvimZb19+BE9Nh205jybwCCnVZrYw9YoLsCVoOC6P5IxhST2Ao+u OEH4bti7yFK2ciOPAYM1B4VhPA/M65FJJ2SW3h6LikWhuKE9tWl5nPignN+2JWH6GF9oWiDm twwgdLqERLs/DNF0kvq5P7jt9XptMs2pEpZqmm/EUgmxcQkpno57j8l/bPHhREAUVkwUTqa2 n7VhvVWQgqf+fBsmz0wWdVyJ4C3kw/daZ6e2A6ZcAlNS5Y7MjyiSPpYQlGQQZPkUnDqRFFKo 4rDaKqLGnhID8RP8K+gnlrJPIsv8lJnaGdxJoMkpvCjW9fIFUu0a7Mv24VXnKsYOflo91h/3 P/faRkbf6/Xzp9Xm6zaC/AEKNyGIY2usqisJfbqowONE813QIKAmVE4sE9sAFRhZRY1x9zUM eHASHCyIKCyFNONFsQiSJJbU1nw9MIWgC5RjlXlTocfz8H31DMBxEj59ef32dfXTNi+6Es/F ndYmS4bXvcCaNXhF8rEOR5PgwB2323yDS9RmHVJTvwRP05gjEajr3Dtv1vSe0nDQv7Tw/+zD 8g2KM2HoEK6FVEZTzW5NaCftXLpCpeJ2FQcSz7OFVo8LXUMEDwfzud83lNH+zfzKHvAswUf4 Uo0s+e06WKOidF50zG+AH8LqNCMBAl7cDvDw7ipAkTc3g14Yvwrg40Jd2U0330aczn60zT8c BrwB7g9CE1zAiAMORd7+dt2/CbAneNCD2ah4Zi3YExqXQiq/Uyd6TmaBUkZ//FJyOpvIwEgo ZWhEQgSQYT8gc5nhux4xUvFUWAk2uOtdUJYpRTCXc6MujklBgv2DHTTz09Z4Oo27V0p7lZzN rme2tJU7REG+7ziYQOvL2rw5Fz+UazY93z2uXn78O9ovn+t/Rzj5AL70vR9eSSe8wmPRoOEt viOZS6kuyEqK0MxIUUGYm3ARyviP7Y6O3lBun2pbJhC61h+/fYSBRP//+qP+sv15ytkiyBj2 q2fIIrIyf3GFdnBTQLAifY3D3zooV7KFZ3w0ovnIEavaLTcvpiW03+9WX173dbsZqTeqlBKy NUspDsLU/DxSzi2tt39/aI6ZzlsLniCvZhWo7xxiNZoEp8lUDlx3wNXNgNrZbYuM8OUGEMW/ XW6gYdCm5TLTnVvLMTOBHF/3QBsZ8IXWZvWRwKzU9wwimsV83hK4DoRaM62hCo0xDbBqW9WC Neoa1zN3Pg2hsEIhOyUh0r1U9u6n1e78OgjTLAzLEFxmwTGB5QvDikjTTXdm4lLCYqC4e+4S Nr/q3/VDkYKhEy30Jw+C4G00gvi6dTJ5pkM4PtHxNyRWkDbLEAtDc12NtFKYM7Gg3BtNWqoS YrCEg37kXR0eJWrslaSF7BaBToDcjSuPjiAS62aQC3ZzhW9hmQwu1FKIbuK9mSUwKcU/8qT4 H1l03tclnvsMDVqO84T3L1kCzTD4J4arS2IyDIPBRYbhVf8CQ4Kv7m5+Xqb3VNfQzXbL0VQz 7W8+uL46emespd78yKa2o2WB0N7GWKL1XZ88PFmQrqznIX0f8ZluPGToIMYjFsho+hk1O/OB /CuxbGzCml0LB5E5KuSYuyCjQtgGDqA/ieBHAaav+jZKpE8SvYjnNCtpqc94/fSOEBL1r+6u o3fpalfP4F9w91bzGTavggHvjLOAZmW6A14dNqwdLDaXWRzIjc80Yu4nHIab1/u/t7sfq803 v8WcqGOsYbH5Z5QIT4gl4+YbPKB9BQTq0qZTz9uZs8zp3GGpJsSaZ9p04PhVNOqBkXRRlExN ylsJXjpHjMcSRUaa0xLp0Ax7lc4YEpMA4ayLPm1KRMwlcShFXrS/q2SMfVCfkPqoQMIVFxCo lR82yEiQAKSv7qDEEwwznXVbokyyatoPgQNnR0kUQd+50BeH+IQ6stQdQWN34ioiixZCC72B 3QJVmefEiiCgPwoXCUWjEAZ/Tk873rT4PZqudvvX5TqS9U7vbzuH2NbaKaqpdJIqDXSeOzZU UJkqphCF9wfH9qYy2p8C7ufddr992K4hMl4+Rl+W6+XmIbiOmupg1Smux+HI/kQokw6ClmOQ gMZnQUDHXo5bju2GhXAkCcjMhzLsMflQFocxr7Zk3Eakj5CkDeWng0EzouXz83r10NxH0JuL /tBSZauTVg3nS1/ImyK8cFV16Onq0FfWYVBbhwF1nfogcKY0a2zR+Q7KEexUu1jQZESc0ic9 BwP8dbXeX1TxPNU+IVcCGR9gK7smKXN1LqzudtHKM4hN6UbrJHZlrompD1GB25AKsCEdRaM2 2kTf7RqLgw1v4QwpPIY4hVEVJkGEivIRCRMZwmFCMVFqUXSWEpMOivEaXHT0RfGO/guij87D NILzMCGRuAhT0Lil0raoSD5S447+qayDgAsmO/o+JllBRJimr2h0CNHW1QCZz/KuSovxQjqB hDP0JBHdMycIylhHVwOafewpY52zo4fRrQ9jJMdB3Tws8fYaQWIETlEQfee0gwjJUAel7CbB 0jfHamFqeHZzpAIQ2CeS2AcMTk0MSVi9AiWkc2iH6xdhMhg/HTeGiRIxEuqRzBmEVUhSHKIG TJGGA9ZIw6oDD1sqAEdZ11ADi/1ACazoAyW0pE+i9ZXsQMIZkpKmiy5yh4qeSpcSdJF6DQs0 65omHly5EDWGjTQQwgoPhLMQD97ur+H/5O+GQVcx7PIVwwvOYtjpECyK6CrCC9XVUircONYi jbOuHoRcyPCCYRx2O6ah7QanwzE53NUIMaBxy2UML/kMi0hKOrzuoAVM9TBs/obdJm4YXmrD C4tjeNbf5nYMwZt6f0mvrOsrWCcmUH4kUKzv63D/Gktsblj9D3p6DOrSisRtpTzQgKAzuNJ2 lBZJecJyiI6Vtii3vUF1FaQgxvNRmCKKIE7DcGsdWRR3OiyCF1ZZNKnCzUwz+5qu211BimwR JCZdgtF9q8Ik3znZ3euq0NFWC3fd8Ty1XxHoL3P783zGoorWHex39vOd906of4hVrG1CVVRJ PKqYHHXsIx4YePwHzlU3zxgMIJh39zZYgEWOUT+4I3lkYMmNe0WFdaY8ZyHpqdaa27+3NujM enQvmJkV2uz3BGrNMuumM3wMbMswdz6ay+nWCQ4CWU/sxqcVKoqMGNiKeZLEsZXwWZEco6LF o/tqg/OBtQeaoSK2x6UHlehbc6EsjcBvVwwNUqEyL5PwBvuBAxQXag3UOQNpH3bNDmp4v5V6 o/jTdhd9Xa520X9e69faucmtKzSXkFrJrQYhlJhUf9A0pST0JMnmAqOtX8bxNDGP64I1hV+1 nDjie3cvTINjFQfA1L6IeUQLQbmPCntb4gjKNNCUIvdZAI1THxwFa02kayaPOPwmzIdpDtXY Ubom3HPp7hoSCWLVLsuFcSY9AMJ+mif2nZIjwejEdQfu15POfNbyylp1B8BcmG9PtqZBN0In 74dS7n7EqS9yWgR6COjQh5F939ssGr2hwTOKW/upGh8h2wOMULP3EfsV6KOE9sRqnNrnEKfW SGKcpb2fHu3rl723usBH6kNlu4bD4z0XqMTcxyAub+0jG1i/1xDwh/MUFTFI1Cg/PUdaPvyo 95FYPq62p91Na6MfOcZLf1UJpEWVzPQDB7tNwS39FVyegns0/zi4iTaH0R+OOrynCmxCbT0f Fk6qERf3RD/HsRC0gBS9Am9apck8iI9tfIHsLQBh0rom99glKDRyKpwQhIrcPvCABJOzVo3g /GEmTjt4ul7v4MTwNe/vIH+DOEKaqTvv22l6qikifMZqGLzdxKbFzdedflvxwWxVe4JuQg4q OqeACojTKuA4BdDbzbfQa8uEm7D8/M5Un0hyN1QH9adyIT1ckYm+1NuGM6pIc1jUIjA07PU8 dERFTDOfGReD/sBn51lSxSSb0DzUz0GvZ1VlRs7Xj/8gKp2s2bIyJerdarnW795dsZ2DGNNI +wZKcwZ5obVSxkeVPYZcdAQGn2T6kr4dSembBTYS29EraNsA2xstOsDF3PkWqavXJ6gC3bBO jiF7yYlblQZgBqrTRnaL1OwlB6hjauKqw+3713q/3e6/dwpDF8AUROI03kBmOG8tGAnlsepH oOPrEGsVY/vwpUWo1BwL3CbHmA16V/N2K3GB+j0fTQN9nzr3fsDSQAwKfsry30nJmJ308Dyh dkZH7kuU0T9tE6lKy34Z86nQSdBI6OTYf+cAuGPomu8KVlXfB3s3PuhsIB0wR+uOGGd3vZ8/ u3A7TjvWTFnl3Lw+l4AlPOj5r3r23/X/N7GP3vV7EUS3/V6PfVnt37uu10jGOXZm1H7iBKnA ghH7Hags81E7WmvuElZX0JlzRVMwEnasBWnwmJtXqqZ/6nW9eoaY+2m1fms7yNDJPMxoRi3l TMig37Nv7zZ0S0IGqNiMBn3JgcqCT90aYo7sN31nrBrPQEyKjlovmxJyPbcChhnNtaZWt9c9 +/bFXb83sO8c9+2rwyZ+cd/vAWSHloglt/1+X8+cow4NTEwHQ09kE1QogvX5hAAraTuBFqU1 7Pjaioub2+jE3hlLRvaNSkLA0jkjIt5Xq4EU1Cu3Y3KkJGGORRhM3Nd6t/2rO3vbWH8r7lw0 O0D6/llw+o90sBOkUjMqw5n1ke22P7izQxWNG88q5pBBSRK8rETlnTP2gmJHGLCUEvOW1t7Y OGBdmjmlqBJj/UbGKnQCvVKeKej1opYZQJjk7tOGBqk46HgFWs7zKqRRSTaYuBPbmuf2ROfy 9urWvp8P8TgEg9a8LkiW8VlKLdMtbvvDO7tzBujS8oZo9s/OFxUnd7eZXaWW1pRkHFP7kpUZ 6NXROrki82RG5yPLjemv1ljlwLbh+uvIcBrIQL+xCr/IVNsf9SYS+nKF56OUfeEnZtwyzVf4 6ubOGhGTV7c9JRzEWUcoXsjTpWd9e2td/7exa1tuW0e2v6LKy55TdXKsu6iZygOvEiPeQpCS nBeVYmsc1diWy5Jrj//+dAOk2ACbdh6SMtcCIAAEGw2gu3E+96D3ev94Pj1//b1/esW1kTFn aGuo9Nf59Hi4HJrs6JB7bkzxXl4PX63+8P8GA1KMKNTpi/7DKtmvXijsG3Th1UoiDrx+HtNd 5SzatrAi9trp3LCF2UVMDRmkN00FVcFLjuen3qK48UA1g1aqKv5jf/Pr5uF/0M9Yei3/emOq meWhiCfGpsIGBGyEmxpV9232z71jHXCCvOUNHcGecF1SjnzEqorbxDVSyRZofiLwNmNYiVP9 GbLJ5Qb+kWVEji49au+CT1KjeTcROfnoKNSGqp8SC3IDwB/TEVxhU7900GWgVmQOtZOt9n5G VMrIx10caeUGdi47pumviI55fKpcLodjDYztbePv2OwUVo69NODCrRvZni92+u8gnMZbvU24 5Wms3nEfwRZE9AgnCfUndSamu0IGnhdSXSOgXlZiFdD3GGZUGKGxXI4mOzndpKgxqGK+8EHB TBakjdq4gIdrjVZamtaSBjFb7xeEEJGrKA1FV0PNXAJBR3i6IRKAKbXcj+jpDD5JP3HcKqSn 9JLADcfCwFDiyr+m+rZRQL4lAYov7S3g0UklJae2wkvgm/91fj9fDk+asSsyLYkOEu7l9+n5 ncQjaDyxl8bxQ7Wj8fJ26baOTbLyaq1ang+vj2juzIkSmRJaXUL3+GtqGkbxXSbsctvJCjf3 /WS3/QbK6/jjNLffZlNLT/I9vdXPRCVaiC6w+bnhuP8Bf/ttNDR4f60KbV5HBfObRtjF4U3a Xmkv7NjXz6QFyAyPwWsE9JvJxGLwaMyAflwO+qsBw5jrpisRxFafy+CK8XRAZYGfh0QUy8dd aPXHQxOE/402StgtrKE7o+bsCgf5nImhiUaw1hetoslRtOrlVrgG7f2s/FvD9bZGYD5eORwO mvLK0TzmrlS0AoaLPlcn2BYr6tZ/xRN/owVSIiOcxj6Dx53WZgVhy+VSuYnoIPG12G63NheQ 6/p5iCKkQRlrZGcntnao2hAjj0O9kEHd1MltBl8EVINvYM0yRoN3McuUYRT5MfU7uHLyuFE7 X7hSIvT8DR7I5AwJCpzLFaf8WbsI3C1kerEih6MhQ27sHAYsVwd0hY20jfCm7hjDKM2dLsrB cHgMV8Asy7d3E3pexFIoPUvdMldKwrR0l0oQkk2KBoTXkvz0W2M5pGHuFJa5IlvlJlqqKaYO uPR7/7q/Q7uP1opkTWbYtTS+khNlo+ht2pj6VPDcXYVjybXTVZzPy4KP0JKn6G2hi61aIdPc Z29/4uDjDxK8cBH6mWt3kiwRp1tbRcGL2LW55KXKoTu9hSLkXQukGt5WzOm7TmqlvFkAo6Yd W7y5Qua70K8h58MptnopXAvgZUReTpQEBKJQE4kEUqcq1ca/KdirV2wNJ/3We0eQDIqm/oSW 4fqYKpIkSb7D3WnxbcwX4W8LPzECkarjSFjdYgpAZOWN4Dp6UW6at0cugu1hjQ4+c2uXFbdE la4DZHWA9WJkMq3tAeOQ2r7EsGqFTyTSrccAzWzcpanDtBLr9yuHy+yEd/iI0Y8Ot9mVhA60 kGyS1ryoENiggZunW6rFVVC9NAiuy/n95e73/emhhytjbUlbZSd1rTEQBBv7FpZbrTd1La1L f8BAkEzVbPdHiZuZG4/u56PnVOG7SwOOwngwGU3aKChBAx0V7mTY1yG/zNN2oaEz6xsJN3YA yy09FXqbFvQcSgWTMH/0vy1s2oecGuJmpdGEjTWaDmfLoIVas5kB4k7czwqqY6h9/bU/H+47 Xyiqg26756EMbZMJnncT6Ea1ra3WK8L5pGxIwZUNU0kO3wI9bU/WWmAlLcK0V1DTpnw0p2ab aOUUurIodQipojNcfh96/348vby8y3AN+lGc5hOJ8XO4nfYFjfC5yHBrXkbubvZXAZTTyC52 l9zOak4dPfN4V3jBVkfywdAyEJgAqZxGLF7YOqCc1JsJbWOzZlK4xaN2PehBha+JGRnXG3ob Q+kwRUhad4lEP3FZsr8Gid2fT5t94GThLn3UgmTs8HrdMHQ5r1Bqrjx0dy7Gn5VLvmsm+/Hh 9Hq8/H46a/ngi16kjmYIXYGZG3CgTQtdwgD9e/96kOEXuWqp4DijiVkSgNMRA25NMPZmkymH 7cTYsoYtBs97dDDUFogKocs4iQhbBxKpHBvFV1HlYHG3WBodluWpsNdalBiEFTZuCS3bdYyE GEpmPmmB01G/hc2nWwMrSqM4LZJCBWR5amBp6qWp0eHwgqvekC/5+oLF4fl8ej2Dynt84d+0 8BMtYq56FjC/xIP+cNBFTLqIaQcx4ooSjn6IVOGeGEy5nw7QdpGp6yKaDCwRt4mwsGZtNIrp IG7Q2YRF2RJmFodafRZlf81if42v75wpF0TQYDqYt4nMtWajKZNDxMIdz5zRfNbFxUynwxic WlO7TcDcO7MGHktEM2tCHSsIJWfxq7UNnk1JmcSP0DqfPIuNubYG1mQ27iDmTHNAzFsTOXc2 WiZ8BiruJWrCvLXWNQmK00+SOKX4JIW7DNsBK7394+P+/Ne5N/j69xG+3V9vVOMkTdkM0EEj p8FoAPLEfEzHyRVqGWoDI0Nia4iI/av6EB/Pd2Rt3DTAieGbbYf1l4Erng73xz2XSwbvxpmy 3eLjw/ECS5b18f5w6jmvp/393V4aVdaBRTWjKz0ilgqR+rp/+X28Y2axgGyeBI7aKVfOjkIj ijDynSgsChWdqSHcMM9LTcUAMIuH3HUNkPrW8fOhOjemGWwRRqHdYbQPfBiLouCLXC/sAZGm iPiCRBtS/Sr8SAuIDMmWVFVSz+hDSAPeA4ru1VoyBKRO8q7VUMJdbs/Ys8ZQvEK7GLfSkrCM 9TdRkbgj9IMG6mm4BQdq2hIpB9S+hCXwkAedpsz2SOXStN0zeb1nVBambxTxQe/Yxa2m3irI fN4ZbxCh+uaByPVa3DaSEYLaebYaZOo2V6jVmRWsRolG0FBQ6nmH55bvJjaYaFjip7G9CPVf Wd1SZQaAkbYYqABmrCqdZ6AlLUCqaY69DcaVEOZFaV/Pwt3TM5719+6P5xeMKayWRG0pAt9g e2dEmvC14QAWb74KNsvtBwVpUrB3vQC+s/5rkYIU0oSHjk4Pp+o2JsbnLUoX3P6XI68fAJUX R9BOlJncZ3xv09IBtdpBofbXNV8xzXYPFMfnQeaaugq39/Z8r0WsxpOnlhiHJXK775V5ZRPR XTgd9tI4XR1g7nw+nGA9g2W1zLVVZqxZoN3Xg7gDq/BN6OnRd/WfvU3sGD7FOExAhehMVl3f 0cmnBSckkLFz5W6+W9KPXWPSpbbn0FuezhccxpfX0+MjDF2PMzb2l2gnQmPmXlGRwaQHa4ZU /z3J5Wla7JYlfE+FzqZseSWLiggWdRV8rXW18+WConM2TdqRb8yGi+K2d7d/7p2eH997vw69 N9xi+fuIBsXHMwZ4vyeJ6eE17Tw31uskL/lqVuVJWvj/7MnKFmkOArB3eMaizzJq4v/KqFt/ qWCRx/N/6hH6V+8JJMb+8XzCej0fDveH+3/J+Mq0pOXh8UWGVn46gS6HoZUxEr3mXkqSmwO9 gjtnFpomx22IQlMCtCLswg5shyeD3Pc1S1dKhsIbUlGvlZq5HbmWmQW5DjwpPC/vz7u5yYTn 5CVTy7SgXwA9izXG/TI0BiMAtZXB1a7E2wWB/pyFnnbLWCi3QmmqYgk6YRHbaGKpEdWPKKuE 5iMNvZ4DIxQvjlLxhpj4m5jVOOPVOHnw2iU4wqzwV3pbN7abGuN+he7LxqeAW92xZrEif0ye uhnyYmFH9lbHtpltdLFdgODAk1OffvDh0/6Btf2TP+a5Fl3/SwyvpVJtuhbCri+ofLYdNVnI HHI1MVb3FName9h78kOVnyRbJpmArj9cu3rZ9/uXS0u6unbhGv1sbzSDYtnR0H3aVUMI5gVI x4nRdPinTPY1If/1cvqqhL0UTYaEi8JdPJnSw2AJJ8PZ0Phyld+MjqGOKehRDoJQh0HfMnJn 0XDUH+jYyh0O5v2rn9TzBSbh4wPehdWI+vPNYn//cLiY9c7jKEyokY4caJlIk6FlWTr80y7z 0qik67nKFO39T9+WvbWswWxLMyj/qvYsVB0DwuwKzIXTC6Vsqk0kWtg1xvo7w1Xvgcu2WYLE gSVLwbJ4lqs0W8Nqh6Rxb6Xx2C62WNqPYTSyTFB4eEyWsuQ61LYICRNm9g+e4NP73qK78jUJ a3K+V2HUdPRcmG1YfOXfisxOdhl1umrzH+aNs5x9kTVfCntofZ5i+wdJ7D9I43yWZjD/NMXn lRnMN58n+fEnacLP0ow//ylIEvFf0yoS/GBZpehYKFx+qMV4K+iQ+qIQUsq6EUsJO/A7iR3M iVp0JcJ3DGm5bfRdc5El7FZ6p7JUGieh5t8l5edPZTPUTJzayoj1R/TjcGp0A0B0G18Kz9LP xcaOjDk/D9OJOYFH/iIt5OV0OmyuDyLfANxbY1dSzjtL3LssVmHB4tAba0NmhZ6ycTM06sIX BbvutIv4xhMRNw+oqYszc5WTp401aK9GY/dGePIQmHMdBZpz+D0+Hx1cbnARZ+H/JMRVatvZ 9Pj6JM95Wqs/37uuuwK8CEstj7TrhYuhdkljBey2GD6+Datrf203alPCd8s8pG6lwIyw8CcD YAofdRc+4gv/Ts0d4aEKotxsn8D059SX+tSnwH4IQyUQqlKN5VANyyNbVgO/JqkusQu4vRZS vNlESjHNpDRp6vXnv0uK2zbSi5NvegspmQG07SoDZlMogA6CPI3rTqqQH2VKQ9z/wMvR1mQn TgFDI4N2qbMH62wMcEaudS5S42cUNFaYqry8WuLGW3tyBLcGMOgk8+m0r1X/exqF1AH0JySi vHrWspReoFUEn5Po6tTipeImsIubpOBrEWA0OpI9FpBDQ9ZmkkBefKmurMT7dzNc/49HM44P U7QJENCmL8fzybIm86+DLyQGR9F6tcrg5Xx4uz/JK/BaNW7dziqBlWEZdiv0jTJY2HaNIqCy ghv6DdH+iIs4o3WQj2Z+BbbzLkuQu5FD81fQTvZlM5rxCrH6Tepzod41jTmL12plwwXd3PJD KovKTtrxu7M63dQHuVzZbJZabz+oZ9bN/Ui2424Wrzjs4kp+gNZLNTk3CXOIJsY3is/01nj5 PNafVfAv6iwAKI3vAk+qkObrkdCYC1OEjBbuWW6JJrRSGK3dMx61WvlbXMzQL1+USZ5R1xz5 vFsIKoEzF0Y8YrtV7kxYQmSrmNoUxI4mYPAZRFgtRMjvVUS1TfPl7sWa9K8367khLQSfZARY 6iwWtj9SicmZSLN6RpSGQOOv9nazzkGcYvyUrg+RF3r718tRBl8r3l8OWpD76vrsa7h5zYIN JF/SpGFGQyoCcgU3zRqHC/vDrHZh56GWubFRZGE5fTCEAwMSL0CLbEeebDX6pJzDRel8VA2R RlAPoa4LbheOpr7yhlD6C/W49mK+8Uh03mi+YBuNcXWgsgwDXwIH+4FejpId+wsou71o//zw tn84tG/FTbR4WeRD+PJ2+bf1hTL1/LuD+ZfICsrMuhlqMKQxFt3aM5hhJ9NdWlcNrGnn70wH nUxnDahZlMGMO5nOWtPr5Qxm3sHMR1155p09Oh91tWc+7voda2a0B/RCVK52VkeGwbDz94Ey utoWLg08Qssf8PCQh0c83FH3CQ9PeXjGw/OOendUZdBRl4FRmVUaWrucwUodK4vAavzVT6Cl sbcygMoaoAMvWfWlCrsePqww5sFj7/f+7j9akDZlgLXCSDERFcd4SREoMrl2N6uMlSgnQd67 RpYlQGJ+RGdhgsrBHySBonw/+yDhKnUwuPgHKToWqIrUVXq55jV8KsimQ1i0xXtDp3FcfsCv MOgec026ONy9vR4v79y18bjfxzrJmPp/jeBJxSal8QWvjGtntgMjpAh9zQqxolGbw4vFGQr+ iNYRSxgnFiajLoexNRP+7jS7tR2V/rd+Z0qYizHk8AdlwfrKj+h9xK0U9tq96nDEq8hIFUft F+W+vr9cTg/KvK5tT6FumSZaoHyWwWNbYFJS/8EKjL0xg01amTFUbSshgENqXt7Ak8GwBW8y DvVoUK4Kc2TEFnrdSUXAMGNxdDrWbJAq3KajrsLwip8Ji7ZbUvh2u8zcbXfZamn/pAexddqk dELBtNmIUVB3fOgubXThoPZadQVzdzR0mRqKomVtficHDTmjVdZLx1+v+9f33uvp7XJ8Pmij yN25blhoHejSw8wodMwK/AQMxZxsybuGNu37f4H2BkHcjgAA --------------050609050307050403090006 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc --------------050609050307050403090006--