* Re: [PATCH 0/3] netfilter : 3 patches to boost ip_tables performance
From: Andi Kleen @ 2005-09-27 16:23 UTC (permalink / raw)
To: Harald Welte; +Cc: Eric Dumazet, linux-kernel, netfilter-devel, netdev
In-Reply-To: <20050923170911.GN731@sunbeam.de.gnumonks.org>
On Friday 23 September 2005 19:09, Harald Welte wrote:
> On Thu, Sep 22, 2005 at 03:03:21PM +0200, Andi Kleen wrote:
> > > 1) No more central rwlock protecting each table (filter, nat, mangle,
> > > raw), but one lock per CPU. It avoids cache line ping pongs for each
> > > packet.
> >
> > Another useful change would be to not take the lock when there are no
> > rules. Currently just loading iptables has a large overhead.
>
> This is partially due to the netfilter hooks that are registered (so we
> always take nf_hook_slow() in the NF_HOOK() macro).
Not sure it's that. nf_hook_slow uses RCU, so it should be quite
fast.
> The default policies inside an iptables chain are internally implemented
> as a rule. Thus, policies as built-in rules have packet/byte counters.
That could be special cased and done lockless, with the counting
done per CPU.
-Andi
^ permalink raw reply
* Idea for packet classification.
From: jordi @ 2005-09-27 16:17 UTC (permalink / raw)
To: linux-kernel, linux-net, netdev
The idea is to create a set of iptables TARGETS that classifies the packets.
When a packet is classified, a classification / Values is associated with
the packet.
This classifications can then be used on an iptable filter rule, in a
routing table selection rule or in a tc classification filter.
For example:
#iptables –A INPUT –j CLS user --classifier tcfilter --filtername u32
…
#iptables –A INPUT –j CLS quota_plan --classifier hash --table
user_to_quota --input cls user
#iptables –A INPUT –j CLS tos --classifier tos
#iptables –A FORWARD –p tcp –port 5343 –cls quota_plan=1 –j DROP
So in this example when a packet arrives, the source address is taken and
translated directly to a user, and the packet is marked with the userid.
I.e. The packed has an associated classification user = 23
In the second line a hash table classifies the packet. The user is taken
from input and a quota plan is taken as an output.
So after the second rule, the packet has associated 2 classifications:
user=23
quota_plan=2
The 3rd line classifies the packet by TOS so the packet has 3
classifications
User=23
Quota_plan=2
Tos=0
Once a packet is classified, those classifications can be used in a filter
rule or can be used in a routing rule or in a traffic shaping queue
classification.
A packet can have many classifications
Those classifications can be used any time in the packet live.
In the 4th line in th example, the rule drops all tcp packets with port 5343
and had been classified as quota_plan
The 1st line in the rule uses a tc filter wrapper to classify the packet.
This idea would be an extesion of the MARK target.
I am planning to make a patch to implement a couple of functions to insert
classifications to the sk_buff structure and to consult classifications of
a sk_buff.
Do you believe that it is interesting or are you planning to do packet
classifications in another way and doing that I would lose the time.
Thank you,
Jordi
^ permalink raw reply
* Re: [Bonding-devel] Re: [PATCH] channel bonding: add support for device-indexed parameters
From: Florin Malita @ 2005-09-27 14:10 UTC (permalink / raw)
To: Eric Paris
Cc: fubar, nsxfreddy, akpm, davem, ctindel, linux-kernel, netdev,
bonding-devel
In-Reply-To: <1127829269.4560.4.camel@localhost.localdomain>
On Tue, 27 Sep 2005 09:54:29 -0400
Eric Paris <eparis@redhat.com> wrote:
> It dos work with RHEL3. In modules.conf you just need
[...]
> You can add your mode and mii_mon and such on the options lines. It
> does work I've used it.
Verified - I completely missed "-o" functionality in insmod & friends.
Sorry for the noise, I'll crawl back under my rock now :)
^ permalink raw reply
* Re: [Bonding-devel] Re: [PATCH] channel bonding: add support for device-indexed parameters
From: Eric Paris @ 2005-09-27 13:54 UTC (permalink / raw)
To: Florin Malita
Cc: Jay Vosburgh, nsxfreddy, akpm, davem, ctindel, linux-kernel,
netdev, bonding-devel
In-Reply-To: <20050927094055.7953a832.fmalita@gmail.com>
It dos work with RHEL3. In modules.conf you just need
alias bond0 bonding
options bond0 -o bonding0
alias bond1 bonding
options bond1 -o bonding1
You can add your mode and mii_mon and such on the options lines. It
does work I've used it.
-Eric
On Tue, 2005-09-27 at 09:40 -0400, Florin Malita wrote:
> On Tue, 27 Sep 2005 00:11:56 -0700
> Jay Vosburgh <fubar@us.ibm.com> wrote:
> >
> > Florin Malita <fmalita@gmail.com> wrote:
> > [...]
> > >How can you load a module multiple times on _any_ distro?
> >
> > modprobe -obond0 bonding mode=your-favorite-mode
> > modprobe -obond1 bonding mode=some-other-mode
> >
> > and so on. This is in the modprobe man page, and is described
> > in the bonding documentation (found in the kernel documentation or at
> > http://sourceforge.net/projects/bonding). It is admittedly somewhat
> > grotty, but it works.
>
> OK, I see this capability has been in module-init-tools since the 0.8
> days. Doesn't apply to any 2.4/modutils based system tough.
>
> >
> > >Not being able to set a (different) preferred
> > >interface/primary for each bond device makes it unacceptable for
> > >deployment in our environment.
> >
> > How are you configuring bonding? The current SuSE distros, for
> > example, will do the multiple module load stuff automatically in the
> > sysconfig scripts. This is described in the current bonding
> > documentation.
>
> Our systems are RHEL3 based so unfortunately the naming trick above
> doesn't work.
>
> But it does work on RHEL4 so admittedly, having this workaround
> available for recent distros removes the urgency for a fix.
>
> Thanks
> Florin
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by:
> Tame your development challenges with Apache's Geronimo App Server.
> Download it for free - -and be entered to win a 42" plasma tv or your very
> own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
> _______________________________________________
> Bonding-devel mailing list
> Bonding-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bonding-devel
^ permalink raw reply
* Re: [PATCH] channel bonding: add support for device-indexed parameters
From: Florin Malita @ 2005-09-27 13:40 UTC (permalink / raw)
To: Jay Vosburgh
Cc: nsxfreddy, akpm, davem, ctindel, linux-kernel, netdev,
bonding-devel
In-Reply-To: <200509270711.j8R7BunP014387@death.nxdomain.ibm.com>
On Tue, 27 Sep 2005 00:11:56 -0700
Jay Vosburgh <fubar@us.ibm.com> wrote:
>
> Florin Malita <fmalita@gmail.com> wrote:
> [...]
> >How can you load a module multiple times on _any_ distro?
>
> modprobe -obond0 bonding mode=your-favorite-mode
> modprobe -obond1 bonding mode=some-other-mode
>
> and so on. This is in the modprobe man page, and is described
> in the bonding documentation (found in the kernel documentation or at
> http://sourceforge.net/projects/bonding). It is admittedly somewhat
> grotty, but it works.
OK, I see this capability has been in module-init-tools since the 0.8
days. Doesn't apply to any 2.4/modutils based system tough.
>
> >Not being able to set a (different) preferred
> >interface/primary for each bond device makes it unacceptable for
> >deployment in our environment.
>
> How are you configuring bonding? The current SuSE distros, for
> example, will do the multiple module load stuff automatically in the
> sysconfig scripts. This is described in the current bonding
> documentation.
Our systems are RHEL3 based so unfortunately the naming trick above
doesn't work.
But it does work on RHEL4 so admittedly, having this workaround
available for recent distros removes the urgency for a fix.
Thanks
Florin
^ permalink raw reply
* Re: [PATCH] channel bonding: add support for device-indexed parameters
From: Jay Vosburgh @ 2005-09-27 7:11 UTC (permalink / raw)
To: Florin Malita
Cc: nsxfreddy, akpm, davem, ctindel, linux-kernel, netdev,
bonding-devel
In-Reply-To: <20050927012444.be5d5311.fmalita@gmail.com>
Florin Malita <fmalita@gmail.com> wrote:
[...]
>How can you load a module multiple times on _any_ distro?
modprobe -obond0 bonding mode=your-favorite-mode
modprobe -obond1 bonding mode=some-other-mode
and so on. This is in the modprobe man page, and is described
in the bonding documentation (found in the kernel documentation or at
http://sourceforge.net/projects/bonding). It is admittedly somewhat
grotty, but it works. The current planned API path that allow one
module load (or compiling bonding into the kernel) to create multiple
discrete bonding devices is sysfs.
[...]
>> In any event, your patch does not provide enough flexibility to
>> allow the distro scripts to switch to it (it omits arp_ip_target), so
>> the init scripts will be unable to switch.
>
>The patch is not forcing the scripts to switch since the old syntax/ABI
>still works (one reason I didn't touch arp_ip_target was to preserve
>that). It's simply providing an additional (saner) approach to bonding
>configuration which can easily co-exist with the sysfs solution.
It's not about forcing use; it's about adding an incomplete and
inconsistent feature, and the attendant maintenance and documentation
load.
In general, I can see the value of allowing users to specify all
of the parameters for some number of bonding devices on a single module
load line. However, the syntax provided by your patch does not parse
the arp_ip_target parameter consistently with the other parameters, and
has no backwards-compatible way to add support for all possible bonding
configurations that I can think of right offhand.
I don't see the value in a new module parameter syntax that is
not both backwards compatible and allows for specifying any possible set
of bonding module options.
>Not being able to set a (different) preferred
>interface/primary for each bond device makes it unacceptable for
>deployment in our environment.
How are you configuring bonding? The current SuSE distros, for
example, will do the multiple module load stuff automatically in the
sysconfig scripts. This is described in the current bonding
documentation.
-J
---
-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
^ permalink raw reply
* Re: [PATCH] channel bonding: add support for device-indexed parameters
From: Florin Malita @ 2005-09-27 5:24 UTC (permalink / raw)
To: Jay Vosburgh
Cc: nsxfreddy, akpm, davem, ctindel, linux-kernel, netdev,
bonding-devel
In-Reply-To: <200509262358.j8QNwM8N012009@death.nxdomain.ibm.com>
On Mon, 26 Sep 2005 16:58:22 -0700
Jay Vosburgh <fubar@us.ibm.com> wrote:
> Florin Malita <fmalita@gmail.com> wrote:
> >IMHO the "primary" semantics are completely broken right now and this
> >is a possible fix for it.
>
> The distro provided network init scripts are, as far as I know,
> really the main user of the bonding module parameters. Right now, the
> init scripts will generally load the bonding module multiple times when
> creating multiple bonds with differing parameters. This works tolerably
> well, although I recall that some users have run afoul of Fedora Core
> kernels that could or would not load any module multiple times. I don't
> know if that's still the case today.
How can you load a module multiple times on _any_ distro?
More specifically, how exactly do you get past this check in module.c:
http://lxr.linux.no/source/kernel/module.c#L1534
if (find_module(mod->name)) {
err = -EEXIST;
goto free_mod;
}
That means there can only be one bonding instance => there's only one
"primary" parameter for all bonding devices => the semantics are broken.
eth0 \ eth2 \
bond0 bond1
eth1 / eth3 /
current semantics: modprobe bonding max_bonds=2 primary=eth0 ...
=> bond0(primary==eth0), bond1(primary==eth0)(!!!)
new/patch semantics: modprobe bonding max_bonds=2 primary=eth0,eth2 ...
=> bond0(primary==eth0), bond1(primary==eth2)
Currently it's impossible to implement multiple
prioritized active/backup bonds - or am I missing something?
> In any event, your patch does not provide enough flexibility to
> allow the distro scripts to switch to it (it omits arp_ip_target), so
> the init scripts will be unable to switch.
The patch is not forcing the scripts to switch since the old syntax/ABI
still works (one reason I didn't touch arp_ip_target was to preserve
that). It's simply providing an additional (saner) approach to bonding
configuration which can easily co-exist with the sysfs solution.
> I'm not sure I see the real value.
I'm not sure I see the real value in bonding _without_ the capabilities
provided by this patch 8)
Not being able to set a (different) preferred
interface/primary for each bond device makes it unacceptable for
deployment in our environment.
Florin
^ permalink raw reply
* Re: [PATCH] sys_sendmsg() alignment bug fix
From: David S. Miller @ 2005-09-27 4:50 UTC (permalink / raw)
To: coywolf; +Cc: alex.williamson, akpm, linux-kernel, netdev
In-Reply-To: <2cd57c90050926204022fb22ca@mail.gmail.com>
From: Coywolf Qi Hunt <coywolf@gmail.com>
Date: Tue, 27 Sep 2005 11:40:38 +0800
> On 9/27/05, Alex Williamson <alex.williamson@hp.com> wrote:
> > The patch below adds an alignment attribute to the buffer used in
> > sys_sendmsg(). This eliminates an unaligned access warning on ia64.
>
> Is this a warning fix or bug fix?
It doesn't generate a warning, but it does generate totally
unnecessary unaligned access traps on RISC systems.
^ permalink raw reply
* Re: [PATCH] sys_sendmsg() alignment bug fix
From: Alex Williamson @ 2005-09-27 3:59 UTC (permalink / raw)
To: Coywolf Qi Hunt; +Cc: akpm, linux-kernel, netdev
In-Reply-To: <2cd57c90050926204022fb22ca@mail.gmail.com>
On Tue, 2005-09-27 at 11:40 +0800, Coywolf Qi Hunt wrote:
> On 9/27/05, Alex Williamson <alex.williamson@hp.com> wrote:
> > The patch below adds an alignment attribute to the buffer used in
> > sys_sendmsg(). This eliminates an unaligned access warning on ia64.
>
> Is this a warning fix or bug fix?
Guess I'd have to classify it as a warning fix. ia64 is fairly
verbose about unaligned accesses by default, so the warning is printed
out runtime.
Alex
^ permalink raw reply
* Re: [PATCH] sys_sendmsg() alignment bug fix
From: Coywolf Qi Hunt @ 2005-09-27 3:40 UTC (permalink / raw)
To: Alex Williamson; +Cc: akpm, linux-kernel, netdev
In-Reply-To: <1127764921.6529.60.camel@tdi>
On 9/27/05, Alex Williamson <alex.williamson@hp.com> wrote:
> The patch below adds an alignment attribute to the buffer used in
> sys_sendmsg(). This eliminates an unaligned access warning on ia64.
Is this a warning fix or bug fix?
--
Coywolf Qi Hunt
http://sosdg.org/~coywolf/
^ permalink raw reply
* Re: Debugging neighbour.c: timers
From: David S. Miller @ 2005-09-27 0:48 UTC (permalink / raw)
To: kuznet; +Cc: greearb, ja, netdev, linux-kernel, akpm
In-Reply-To: <20050927003845.GA8273@ms2.inr.ac.ru>
From: "A.N.Kuznetsov" <kuznet@ms2.inr.ac.ru>
Date: Tue, 27 Sep 2005 04:38:45 +0400
> Actually, when this code was written, add_timer() behaved differently,
> double add_timer() was prohibited and it printked
> "bug: kernel timer added twice at...".
Yes, it calls __mod_timer() now. That's quite bad and
the debugging check should be re-added I think.
Is there some reason why we want add_timer() to behave like
__mod_timer(timer, timer->expires)? I can't see any...
So perhaps we should add the simple BUG_ON(timer->pending) check to
include/linux/timer.h:add_timer()
add_timer_on() does this check btw....
^ permalink raw reply
* Re: [PATCH] channel bonding: add support for device-indexed parameters
From: Jay Vosburgh @ 2005-09-26 23:58 UTC (permalink / raw)
To: Florin Malita
Cc: Jason R. Martin, akpm, davem, ctindel, linux-kernel, netdev,
bonding-devel
In-Reply-To: <20050922091608.5ec2724c.fmalita@gmail.com>
Florin Malita <fmalita@gmail.com> wrote:
>On Wed, 21 Sep 2005 23:03:53 -0700
>"Jason R. Martin" <nsxfreddy@gmail.com> wrote:
>> Personally I think working to get the sysfs support finished in
>> bonding and stop relying on module parameters to configure bonds would
>> be better
[...]
>Agreed - that would be a better configuration interface, but I don't see
>why we couldn't support module parameter arrays too. Especially since
>the changes are minimal and don't break the ABI/ifenslave
>compatibility/etc.
>
>IMHO the "primary" semantics are completely broken right now and this
>is a possible fix for it.
The distro provided network init scripts are, as far as I know,
really the main user of the bonding module parameters. Right now, the
init scripts will generally load the bonding module multiple times when
creating multiple bonds with differing parameters. This works tolerably
well, although I recall that some users have run afoul of Fedora Core
kernels that could or would not load any module multiple times. I don't
know if that's still the case today.
In any event, your patch does not provide enough flexibility to
allow the distro scripts to switch to it (it omits arp_ip_target), so
the init scripts will be unable to switch. Given that, I'm not sure I
see the real value.
Additionally, as Jason mentions, the sysfs API is looming, and
I'd frankly rather have the init scripts switch to sysfs as it provide
greater flexibility (in particular, it removes the dependency on bonding
being compiled as a module).
-J
---
-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
^ permalink raw reply
* Re: [PATCH/RFC] [NETFILTER]: Fix invalid module autoloading by splitting iptable_nat
From: David S. Miller @ 2005-09-26 22:25 UTC (permalink / raw)
To: laforge; +Cc: netdev, netfilter-devel, kaber
In-Reply-To: <20050925164423.GM731@sunbeam.de.gnumonks.org>
From: Harald Welte <laforge@netfilter.org>
Date: Sun, 25 Sep 2005 18:44:23 +0200
> Dave, please apply
>
> (sorry for the missing Signed-off-by: Harald Welte <laforge@netfilter.org>)
Done, thanks a lot.
^ permalink raw reply
* Re: [PATCH] sys_sendmsg() alignment bug fix
From: David S. Miller @ 2005-09-26 21:28 UTC (permalink / raw)
To: akpm; +Cc: alex.williamson, linux-kernel, netdev
In-Reply-To: <20050926133634.657ef4a3.akpm@osdl.org>
From: Andrew Morton <akpm@osdl.org>
Date: Mon, 26 Sep 2005 13:36:34 -0700
> OK, thanks - I'll send this on to davem. It seems odd to be using
> __kernel_size_t rather than size_t, but that's what struct cmshdr does
> (also oddly).
Applied, thanks.
^ permalink raw reply
* Re: [PATCH] sys_sendmsg() alignment bug fix
From: Andrew Morton @ 2005-09-26 20:36 UTC (permalink / raw)
To: Alex Williamson; +Cc: linux-kernel, netdev, David S. Miller
In-Reply-To: <1127764921.6529.60.camel@tdi>
Alex Williamson <alex.williamson@hp.com> wrote:
>
> The patch below adds an alignment attribute to the buffer used in
> sys_sendmsg(). This eliminates an unaligned access warning on ia64.
>
Vaguely surprised that the compiler cannot be taught to do this.
>
> diff -r db9b9552a2b4 net/socket.c
> --- a/net/socket.c Sat Sep 24 23:56:08 2005
> +++ b/net/socket.c Mon Sep 26 13:44:09 2005
> @@ -1700,7 +1700,9 @@
> struct socket *sock;
> char address[MAX_SOCK_ADDR];
> struct iovec iovstack[UIO_FASTIOV], *iov = iovstack;
> - unsigned char ctl[sizeof(struct cmsghdr) + 20]; /* 20 is size of ipv6_pktinfo */
> + unsigned char ctl[sizeof(struct cmsghdr) + 20]
> + __attribute__ ((aligned (sizeof(__kernel_size_t))));
> + /* 20 is size of ipv6_pktinfo */
> unsigned char *ctl_buf = ctl;
> struct msghdr msg_sys;
> int err, ctl_len, iov_size, total_len;
OK, thanks - I'll send this on to davem. It seems odd to be using
__kernel_size_t rather than size_t, but that's what struct cmshdr does
(also oddly).
^ permalink raw reply
* [PATCH] sys_sendmsg() alignment bug fix
From: Alex Williamson @ 2005-09-26 20:02 UTC (permalink / raw)
To: akpm; +Cc: linux-kernel, netdev
The patch below adds an alignment attribute to the buffer used in
sys_sendmsg(). This eliminates an unaligned access warning on ia64.
Signed-off-by: Alex Williamson <alex.williamson@hp.com>
diff -r db9b9552a2b4 net/socket.c
--- a/net/socket.c Sat Sep 24 23:56:08 2005
+++ b/net/socket.c Mon Sep 26 13:44:09 2005
@@ -1700,7 +1700,9 @@
struct socket *sock;
char address[MAX_SOCK_ADDR];
struct iovec iovstack[UIO_FASTIOV], *iov = iovstack;
- unsigned char ctl[sizeof(struct cmsghdr) + 20]; /* 20 is size of ipv6_pktinfo */
+ unsigned char ctl[sizeof(struct cmsghdr) + 20]
+ __attribute__ ((aligned (sizeof(__kernel_size_t))));
+ /* 20 is size of ipv6_pktinfo */
unsigned char *ctl_buf = ctl;
struct msghdr msg_sys;
int err, ctl_len, iov_size, total_len;
^ permalink raw reply
* runt Tactful Small Carton, International Courier. alleviate
From: fritz floren @ 2005-09-26 19:16 UTC (permalink / raw)
To: Clifton Ashworth
Cc: glover, terry, mendoza, lowe, netdev, daniel, linux-xfs-outgoing
Set a snug e zonne setting.
Together we eliminate malady on diarrhoea.
Straightforward to tracks your delivery with our system.
One big sells. Sale to all remedial.
Online physician can consult for zero pennies.
I am very suggested purchasing from this dealer. Kath J --NJ.
http://uk.geocities.com/wonderfulcomplimentwell/?atednkky
I gathered from the conversation that Mr. Skimpole had been educated for
the medical profession and had once lived, in his professional capacity, in
the household of a German prince. He told us, however, that as he had always
been a mere child in point of weights and measures and had never known
anything about them (except that they disgusted him), he had never been able
to prescribe with the requisite accuracy of detail. In fact, he said, he had
no head for detail. And he told us, with great humour, that when he was
wanted to bleed the prince or physic any of his people, he was generally
found lying on his back in bed, reading the newspapers or making
fancy-sketches in pencil, and couldn't come. The prince, at last, objecting
to this, "in which," said Mr. Skimpole, in the frankest manner, "he was
perfectly right," the engagement terminated, and Mr. Skimpole having (as he
added with delightful gaiety) "nothing to live upon but love, fell in love,
and married, and surrounded himself with rosy cheeks." His good friend
Jarndyce and some other of his good friends then helped him, in quicker or
slower succession, to several openings in life, but to no purpose, for he
must confess to two of the oldest infirmities in the world: one was that he
had no idea of time, the other that he had no idea of money. In consequence
of which he never kept an appointment, never could transact any business,
and never knew the value of anything. Well. So he had got on in life, and
here he was. He was very fond of reading the papers, very fond of making
fancy-sketches with a pencil, very fond of nature, very fond of art. All he
asked of society was to let him live. THAT wasn't much. His wants were few.
Give him the papers, conversation, music, mutton, coffee, landscape, fruit
in the season, a few sheets of Bristol-board, and a little claret, and he
asked no more. He was a mere child in the world, but he didn't cry for the
moon. He said to the world, "Go your several ways in peace. Wear red coats,
blue coats, lawn sleeves; put pens behind your ears, wear aprons; go after
glory, holiness, commerce, trade, any object you prefer; only--let Harold
Skimpole live."
efficientgnicer emxs flchargefee AH02 filinge filejet
"Who?"
"Oh, it doesn't matter much. One of them. Only not Mac. I'm too fond of
him."
^ permalink raw reply
* Re: ipvs_syncmaster brings cpu to 100%
From: Nishanth Aravamudan @ 2005-09-26 17:51 UTC (permalink / raw)
To: Luca Maranzano; +Cc: LinuxVirtualServer.org users mailing list., netdev
In-Reply-To: <68559cef05092607441dd8e961@mail.gmail.com>
On 26.09.2005 [16:44:09 +0200], Luca Maranzano wrote:
> On 26/09/05, Nishanth Aravamudan <nacc@us.ibm.com> wrote:
> > On 26.09.2005 [15:52:02 +0200], Luca Maranzano wrote:
> > > On 26/09/05, Nishanth Aravamudan <nacc@us.ibm.com> wrote:
> > > > On 26.09.2005 [17:12:32 +0900], Horms wrote:
> > > > > On Mon, Sep 26, 2005 at 05:05:10PM +0900, Horms wrote:
> > > > >
> > > > > [snip]
> > > > >
> > > > > > > > > > Furthermore, if I make an "rgrep" in the source tree of kernel 2.6.12
> > > > > > > > > > the function schedule_timeout() is more used than the ssleep() (517
> > > > > > > > > > occurrencies vs. 43), so why in ip_vs_sync.c there was this change?
> > > > > > > > > >
> > > > > > > > > > The other oddity is that Horms reported on this list that on non Xeon
> > > > > > > > > > CPU the same version of kernel of mine does not present the problem.
> > > > > > > > > >
> > > > > > > > > > I'm getting crazy :-)
> > > > > > > >
> > > > > > > > I've prepared a patch, which reverts the change which was introduced
> > > > > > > > by Nishanth Aravamudan in February.
> > > > > > >
> > > > > > > Was the 100% cpu utilization only occurring on Xeon processors?
> > > > > >
> > > > > > That seems to be the only case where were this problem has been
> > > > > > observed. I don't have such a processor myself, so I haven't actually
> > > > > > been able to produce the problem locally.
> > > > > >
> > > > > > One reason I posted this issue to netdev was to get some more
> > > > > > eyes on the problem as it is puzzling to say the least.
> > > > > >
> > > > > > > Care to try to use msleep_interruptible() instead of ssleep(), as
> > > > > > > opposed to schedule_timeout()?
> > > > > >
> > > > > > I will send a version that does that shortly, Luca, can
> > > > > > you plase check that too?
> > > > >
> > > > > Here is that version of the patch. Nishanth, I take it that I do not
> > > > > need to set TASK_INTERRUPTABLE before calling msleep_interruptible(),
> > > > > please let me know if I am wrong.
> > > >
> > > > Yes, exactly. I'm just trying to narrow it down to see if it's the task
> > > > state that's causing the issue (which, to be honest, doesn't make a lot
> > > > of sense to me -- with ssleep() your load average will go up as the task
> > > > will be UNINTERRUPTIBLE state, but I am not sure why utilisation would
> > > > rise, as you are still sleeping...)
> >
> > [trimmed lvs-users from my reply, as it is a closed list]
> >
> > > Just to add more info, please note the output of "ps":
> > >
> > > debld1:~# ps aux|grep ipvs
> > > root 3748 0.0 0.0 0 0 ? D 12:09 0:00
> > > [ipvs_syncmaster]
> > > root 3757 0.0 0.0 0 0 ? D 12:09 0:00
> > > [ipvs_syncbackup]
> > >
> > > Note the D status, i.e. (from ps(1) man page): Uninterruptible sleep
> > > (usually IO)
> >
> > The msleep_interruptible() change should fix that.
> >
> > But that does not show 100% CPU utilisation at all, it shows 0. Did you
> > mean to say your load increases?
> >
> > I'm still unclear what the problem is. Horms initial Cc trimmed some
> > important information. It would be very useful to "start over" -- at
> > least from the perspective of what the problem actually is.
> >
> > > I hope to have a Xeon machine to make some more tests in the next
> > > days, in the mean time I'll try to reproduce my setup on a couple of
> > > VMWare Workstation machines.
> >
> > Please don't top-most. It makes it really hard to write sane replies...
>
> [trimmed Cc to avoid spamming...]
>
> Ok, just to summarize the long thread from the beginning:
>
> The goal: setting up a Local Director with IPVS with state
> synchronization, failover and failback.
>
> The hardware: 1 CPU Intel Xeon 3,4 Ghz - HP DL380G4 on 2 identical boxes
>
> The problems (please note that all kernel versions are *Debian* kernels):
> 1. Kernel 2.6.8: got a system lock of the standby node when simulating
> a failover. The load average as reported from "top" or "w" is always
> 0.00.
>
> 2. Kernel 2.6.11 and Kernel 2.6.12: failover and failback works fine,
> but the load average as reported from "top" or "w" is always
> systematically at 2.00 or more with both sync thread started
> (ipvs_syncmaster and ipvs_syncbackup). Load average from top is 1.00
> or mroe with only one thread (i.e. ipvs_syncmaster). Horms reported
> that he was not able to reproduce this on a non-Xeon system.
Ok, so when whomever mentioned "CPU utilisation" they were mistaken. The
load average being 2 is due to ssleep(). The msleep_interruptible()
version of the patch should fix that up. It really doesn't make any
difference in the code, except that your load average will go back to
0.00 and the ipvs threads can be interrupted by signals.
I would expect the load average to be 2.00 for all systems, not just
Xeon. The system lock has nothing to do with the patch, though.
Something else fixed it.
Thanks,
Nish
P.S. Again, please don't top-post, it makes it harder for me to reply
(and disinclines me to do so).
^ permalink raw reply
* Re: [ANNOUNCE] Release of nf-HiPAC 0.9.0
From: Emmanuel Fleury @ 2005-09-26 16:31 UTC (permalink / raw)
To: Michael Bellion; +Cc: linux-kernel, netdev
In-Reply-To: <200509261803.28150.mbellion@hipac.org>
Michael Bellion wrote:
>
> Sorry, but this is far away from the worst case for your scheme. Actually it
> is a quite good case for your compiler, because every rule is fully specified
> (meaning there are no wildcards in any rule) and there are no ranges or masks
> involved.
> Try using a mixed rule set that contains rules that only specify certain
> dimensions and have wildcards on the other dimensions. Try using rules with
> ranges and masks.
> Try using overlapping rules, meaning rules that completely or partly overlap
> other rules in certain dimensions.
> This will make your data structure grow!
I think you misunderstood our experiment. In fact, we were trying to
generate as much possible different singletons on the domain (each of
our rule was the header of a packet which have not been seen before),
because if we can group these rules into intervals, then our scheme is
having some advantages.
We were using IDD (Interval Decision Diagrams) which is a kind of
extended BDD (Binary Decision Diagrams) where you take your decision by
looking at a partition of the possible values of the variable. For
example, looking at the value x in [0,1024] where [0,128] leads to one
node in the decision tree, [129,256] to another and [257,1024] to a last
one. More this partition is fragmented more you increase the size of the
structure. Having a lot of overlap does certainly increase the number of
partitions, but adding singletons is the simplest way to increase the
number of partitions.
Take a look at this paper, maybe you can get some idea for your scheme
(it might be that some hybrid between your ideas and ours can make it):
http://www.cs.aau.dk/~fleury/download/papers/tc04.pdf
> Yes, you are right. The HiPAC project has gone through some tough times over
> the last 2 years. With MARA Systems the HiPAC Project has finally found a
> strong partner that is fully committed to the concept of Open Source
> Software. This allows me to continue the development of HiPAC under the GNU
> GPL license.
I'm always happy to see a firm funding some Open Source project. So, I
can do anything else but wishing you good luck for the future. :)
> Ok, I'll do that :)
Good. :)
Regards
--
Emmanuel Fleury
As usual, goodness hardly puts up a fight.
-- Calvin & Hobbes (Bill Waterson)
^ permalink raw reply
* Re: [ANNOUNCE] Release of nf-HiPAC 0.9.0
From: Michael Bellion @ 2005-09-26 16:03 UTC (permalink / raw)
To: Emmanuel Fleury; +Cc: linux-kernel, netdev
In-Reply-To: <43380E4A.1060604@cs.aau.dk>
Hi,
> > But your performance tests have a serious flaw:
> > You construct your rule set by creating one rule for each entry in your
> > packet header trace. This results in an completely artificial rule set
> > that creates a lot of redundancy in the nf-HiPAC lookup data structure
> > making it much larger than the Compact Filter data structure.
>
> Yes, it was intended to be a worst case for our scheme (not realistic
> but worst case)..
Sorry, but this is far away from the worst case for your scheme. Actually it
is a quite good case for your compiler, because every rule is fully specified
(meaning there are no wildcards in any rule) and there are no ranges or masks
involved.
Try using a mixed rule set that contains rules that only specify certain
dimensions and have wildcards on the other dimensions. Try using rules with
ranges and masks.
Try using overlapping rules, meaning rules that completely or partly overlap
other rules in certain dimensions.
This will make your data structure grow!
> > I am currently working on a new improved version of the algorithm used in
> > nf-HiPAC. The new algorithmic core will reduce memory usage while at the
> > same time improving the running time of insert and delete operations. The
> > lookup performance will be improved too, especially for bigger rulesets.
> > The concepts and the design are already developed, but the implementation
> > is still in its early stages.
> >
> > The new algorithmic core will make sure that the lookup data structure in
> > the kernel is always fully optimized while at the same time allowing very
> > fast dynamic updates.
> >
> > At that point Compact Filter will not be able to win in any performance
> > test against nf-HiPAC anymore, simply because there is no way to
> > optimize the lookup data structure any further.
>
> Well, you already said this last time we had exchanged some mails
> (it was more than one year ago if I count well).
Yes, you are right. The HiPAC project has gone through some tough times over
the last 2 years. With MARA Systems the HiPAC Project has finally found a
strong partner that is fully committed to the concept of Open Source
Software. This allows me to continue the development of HiPAC under the GNU
GPL license.
> Anyway, I doubt you can get something that you can update dynamically
> AND small in size following your way of doing. But, prove me wrong and
> I'll be happy. :)
Ok, I'll do that :)
Regards,
+---------------------------+
| Michael Bellion |
| <mbellion@hipac.org> |
+---------------------------+
^ permalink raw reply
* Re: [ANNOUNCE] Release of nf-HiPAC 0.9.0
From: Emmanuel Fleury @ 2005-09-26 15:05 UTC (permalink / raw)
To: Michael Bellion, linux-kernel, netdev
In-Reply-To: <200509261638.12731.mbellion@hipac.org>
Michael Bellion wrote:
>
> The current version of the algorithm used in nf-HiPAC does not optimize
> certain aspects of the lookup data structure in order to increase the speed
> of dynamic rule set updates.
> This means that the lookup data structure is larger than it really needs to be
> because it contains some unnecessary redundancy.
Could you quantify how much this "unnecessary redundancy" does hit the
size of the filter. Because last time I looked it was quite huge (you
may have improve it). And having a fat kernel does not help in backbones.
> But your performance tests have a serious flaw:
> You construct your rule set by creating one rule for each entry in your packet
> header trace. This results in an completely artificial rule set that creates
> a lot of redundancy in the nf-HiPAC lookup data structure making it much
> larger than the Compact Filter data structure.
Yes, it was intended to be a worst case for our scheme (not realistic
but worst case). We were more interested in comparing the complexity of
the different algorithms better than the efficiency of several
implementations.
I don't consider this as a flaw in our experiment because our goal was
different from having a real proof of concept (kind of having an
empirical evidence of a theoretical result).
> You have to understand that with real world rule sets the size of the computed
> lookup data structure will not be much different for Compact Filter and
> nf-HiPAC. This means that when you use real world rule sets there shouldn't
> be any noticeable difference in lookup performance betweeen Compact Filter
> and nf-HiPAC.
Might be right, but admit that the big problem of your algorithm is the
size of your data-structure in kernel-space. What you gain in speed, you
loose it in memory. And this IS an issue on routers (IMHO).
> I am currently working on a new improved version of the algorithm used in
> nf-HiPAC. The new algorithmic core will reduce memory usage while at the same
> time improving the running time of insert and delete operations. The lookup
> performance will be improved too, especially for bigger rulesets. The
> concepts and the design are already developed, but the implementation is
> still in its early stages.
>
> The new algorithmic core will make sure that the lookup data structure in the
> kernel is always fully optimized while at the same time allowing very fast
> dynamic updates.
>
> At that point Compact Filter will not be able to win in any performance test
> against nf-HiPAC anymore, simply because there is no way to optimize the
> lookup data structure any further.
Well, you already said this last time we had exchanged some mails
(it was more than one year ago if I count well).
Anyway, I doubt you can get something that you can update dynamically
AND small in size following your way of doing. But, prove me wrong and
I'll be happy. :)
Regards
--
Emmanuel Fleury
Ideals are dangerous things. Realities are better.
They wound but they are better.
-- Oscar Wilde
^ permalink raw reply
* Re: ipvs_syncmaster brings cpu to 100%
From: Luca Maranzano @ 2005-09-26 14:44 UTC (permalink / raw)
To: Nishanth Aravamudan, LinuxVirtualServer.org users mailing list.; +Cc: netdev
In-Reply-To: <20050926142109.GD7532@us.ibm.com>
[trimmed Cc to avoid spamming...]
Ok, just to summarize the long thread from the beginning:
The goal: setting up a Local Director with IPVS with state
synchronization, failover and failback.
The hardware: 1 CPU Intel Xeon 3,4 Ghz - HP DL380G4 on 2 identical boxes
The problems (please note that all kernel versions are *Debian* kernels):
1. Kernel 2.6.8: got a system lock of the standby node when simulating
a failover. The load average as reported from "top" or "w" is always
0.00.
2. Kernel 2.6.11 and Kernel 2.6.12: failover and failback works fine,
but the load average as reported from "top" or "w" is always
systematically at 2.00 or more with both sync thread started
(ipvs_syncmaster and ipvs_syncbackup). Load average from top is 1.00
or mroe with only one thread (i.e. ipvs_syncmaster). Horms reported
that he was not able to reproduce this on a non-Xeon system.
That's all, let me know if you need more info.
Regards,
Luca
On 26/09/05, Nishanth Aravamudan <nacc@us.ibm.com> wrote:
> On 26.09.2005 [15:52:02 +0200], Luca Maranzano wrote:
> > On 26/09/05, Nishanth Aravamudan <nacc@us.ibm.com> wrote:
> > > On 26.09.2005 [17:12:32 +0900], Horms wrote:
> > > > On Mon, Sep 26, 2005 at 05:05:10PM +0900, Horms wrote:
> > > >
> > > > [snip]
> > > >
> > > > > > > > > Furthermore, if I make an "rgrep" in the source tree of kernel 2.6.12
> > > > > > > > > the function schedule_timeout() is more used than the ssleep() (517
> > > > > > > > > occurrencies vs. 43), so why in ip_vs_sync.c there was this change?
> > > > > > > > >
> > > > > > > > > The other oddity is that Horms reported on this list that on non Xeon
> > > > > > > > > CPU the same version of kernel of mine does not present the problem.
> > > > > > > > >
> > > > > > > > > I'm getting crazy :-)
> > > > > > >
> > > > > > > I've prepared a patch, which reverts the change which was introduced
> > > > > > > by Nishanth Aravamudan in February.
> > > > > >
> > > > > > Was the 100% cpu utilization only occurring on Xeon processors?
> > > > >
> > > > > That seems to be the only case where were this problem has been
> > > > > observed. I don't have such a processor myself, so I haven't actually
> > > > > been able to produce the problem locally.
> > > > >
> > > > > One reason I posted this issue to netdev was to get some more
> > > > > eyes on the problem as it is puzzling to say the least.
> > > > >
> > > > > > Care to try to use msleep_interruptible() instead of ssleep(), as
> > > > > > opposed to schedule_timeout()?
> > > > >
> > > > > I will send a version that does that shortly, Luca, can
> > > > > you plase check that too?
> > > >
> > > > Here is that version of the patch. Nishanth, I take it that I do not
> > > > need to set TASK_INTERRUPTABLE before calling msleep_interruptible(),
> > > > please let me know if I am wrong.
> > >
> > > Yes, exactly. I'm just trying to narrow it down to see if it's the task
> > > state that's causing the issue (which, to be honest, doesn't make a lot
> > > of sense to me -- with ssleep() your load average will go up as the task
> > > will be UNINTERRUPTIBLE state, but I am not sure why utilisation would
> > > rise, as you are still sleeping...)
>
> [trimmed lvs-users from my reply, as it is a closed list]
>
> > Just to add more info, please note the output of "ps":
> >
> > debld1:~# ps aux|grep ipvs
> > root 3748 0.0 0.0 0 0 ? D 12:09 0:00
> > [ipvs_syncmaster]
> > root 3757 0.0 0.0 0 0 ? D 12:09 0:00
> > [ipvs_syncbackup]
> >
> > Note the D status, i.e. (from ps(1) man page): Uninterruptible sleep
> > (usually IO)
>
> The msleep_interruptible() change should fix that.
>
> But that does not show 100% CPU utilisation at all, it shows 0. Did you
> mean to say your load increases?
>
> I'm still unclear what the problem is. Horms initial Cc trimmed some
> important information. It would be very useful to "start over" -- at
> least from the perspective of what the problem actually is.
>
> > I hope to have a Xeon machine to make some more tests in the next
> > days, in the mean time I'll try to reproduce my setup on a couple of
> > VMWare Workstation machines.
>
> Please don't top-most. It makes it really hard to write sane replies...
>
> Thanks,
> Nish
>
^ permalink raw reply
* Re: [ANNOUNCE] Release of nf-HiPAC 0.9.0
From: Michael Bellion @ 2005-09-26 14:38 UTC (permalink / raw)
To: Emmanuel Fleury; +Cc: linux-kernel, linux-net, netdev, jamal
In-Reply-To: <4337DA7C.2000804@cs.aau.dk>
Hi
> I'm still not convinced by your approach. :-/
You really should have a closer look at nf-HiPAC so that you know what you are
talking about!
Your Compact Filter takes a completely different approach than nf-HiPAC to
build the data structure used in the kernel for the packet classification
lookup.
Your Compact Filter uses a static compiler in user space. That compiler
transforms the rule set into boolean expressions and than uses operations
from predicate logic to optimize the rule set.
This has the big drawback that whenever only a single rule changes you have to
recompile the complete lookup data structure. So this approach is clearly not
suitable for scenarios depending on dynamic rule sets.
nf-HiPAC uses a completely different approach to build the lookup data
structure in the kernel. It is based on geometry.
This approach allows completely dynamic updates. During an update of the rules
only the required changes of the lookup data structure are made. The data
structure is NOT rebuild from scratch. This guarantees that the packet
processing is only affected to the least possible amount during updates.
Although nf-HiPAC and Compact Filter use completely different approaches and
algorithms to build the lookup data structure it is important that you
understand the following:
nf-HiPAC and Compact filter end up with a very very similar lookup data
structure in the kernel.
> These experiments have to be updated but can you comment on this:
> http://www.cs.aau.dk/~mixxel/cf/experiments.html
The current version of the algorithm used in nf-HiPAC does not optimize
certain aspects of the lookup data structure in order to increase the speed
of dynamic rule set updates.
This means that the lookup data structure is larger than it really needs to be
because it contains some unnecessary redundancy.
This explains your test results.
Compact Filter and nf-HiPAC perform the same when they are both able to keep
their lookup data structure in the CPU caches and when they are both not able
to do so anymore.
Compact Filter is currently able to perform better in the area where it is
able to keep its data structure still in the caches while nf-HiPAC is not
able to do so anymore.
Most aspects of your performance tests are quite nice (e.g. the generating the
traffic by replaying a packet header trace).
But your performance tests have a serious flaw:
You construct your rule set by creating one rule for each entry in your packet
header trace. This results in an completely artificial rule set that creates
a lot of redundancy in the nf-HiPAC lookup data structure making it much
larger than the Compact Filter data structure.
You have to understand that with real world rule sets the size of the computed
lookup data structure will not be much different for Compact Filter and
nf-HiPAC. This means that when you use real world rule sets there shouldn't
be any noticeable difference in lookup performance betweeen Compact Filter
and nf-HiPAC.
-----------------
I am currently working on a new improved version of the algorithm used in
nf-HiPAC. The new algorithmic core will reduce memory usage while at the same
time improving the running time of insert and delete operations. The lookup
performance will be improved too, especially for bigger rulesets. The
concepts and the design are already developed, but the implementation is
still in its early stages.
The new algorithmic core will make sure that the lookup data structure in the
kernel is always fully optimized while at the same time allowing very fast
dynamic updates.
At that point Compact Filter will not be able to win in any performance test
against nf-HiPAC anymore, simply because there is no way to optimize the
lookup data structure any further.
Regards,
+---------------------------+
| Michael Bellion |
| <mbellion@hipac.org> |
+---------------------------+
^ permalink raw reply
* Out of Office AutoReply: [FMG-SPAM] - AL J RAJITDQTQH - Bayesian Filter detected spam
From: Liesbet Nyssen @ 2005-09-26 14:30 UTC (permalink / raw)
To: netdev
I'm out of office until October 25th and will have only limited access to my email. Please, contact me again later. Liesbet Nyssen
^ permalink raw reply
* RE: [FMG-SPAM] - AL J RAJITDQTQH - Bayesian Filter detected spam
From: Liesbet Nyssen @ 2005-09-26 14:30 UTC (permalink / raw)
To: netdev
I'm out of office until October 25th and will have only limited access to my email. Please contact me again later. Liesbet Nyssen
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox