* Re: netif_tx_disable and lockless TX
From: Herbert Xu @ 2006-05-31 6:31 UTC (permalink / raw)
To: David Miller; +Cc: mchan, jgarzik, netdev
In-Reply-To: <20060530.232626.00456312.davem@davemloft.net>
On Tue, May 30, 2006 at 11:26:26PM -0700, David Miller wrote:
>
> I don't think it will be worthwhile to keep NAPI around just for
> TX completion. Sure the dev_kfree_skb() will schedule software
> interrupt work to do the actual free, but the TX ring walking
> and dev_kfree_skb() calls will be in hard IRQ context.
Sure we won't need all of NAPI. But would it be that bad to schedule
a tasklet to do the TX completion?
> > > The only regret I have about that is we will go back to not being able
> > > to profile ->hard_start_xmit() very well in such drivers.
> >
> > Can you elborate on that? I think I've already removed all references
> > to this in my memory :)
>
> If you disable IRQs in the ->hard_start_xmit() handler, you don't
> get timer based profiling ticks. Currently we do.
Oh, I'm not proposing that we disable IRQs on xmit_lock at all. What
I'm saying is that for tg3, since both xmit_lock and tx_lock are BH-
disabling locks, they are equivalent and therefore we can replace
tx_lock with xmit_lock.
For other lockless NICs that have IRQ disabling tx_locks, they can
be converted to BH-disabling ones and then be able to use xmit_lock.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: netif_tx_disable and lockless TX
From: David Miller @ 2006-05-31 7:08 UTC (permalink / raw)
To: herbert; +Cc: mchan, jgarzik, netdev
In-Reply-To: <20060531063152.GA8032@gondor.apana.org.au>
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Wed, 31 May 2006 16:31:52 +1000
> On Tue, May 30, 2006 at 11:26:26PM -0700, David Miller wrote:
> >
> > I don't think it will be worthwhile to keep NAPI around just for
> > TX completion. Sure the dev_kfree_skb() will schedule software
> > interrupt work to do the actual free, but the TX ring walking
> > and dev_kfree_skb() calls will be in hard IRQ context.
>
> Sure we won't need all of NAPI. But would it be that bad to schedule
> a tasklet to do the TX completion?
It seems just an extra level of indirection. :)
> Oh, I'm not proposing that we disable IRQs on xmit_lock at all.
...
I understand what you're saying. What I'm talking about is that in
net channel drivers we might go back to IRQ disabling locks again.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: David Miller @ 2006-05-31 7:10 UTC (permalink / raw)
To: bidulock; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060530235525.A30563@openss7.org>
From: "Brian F. G. Bidulock" <bidulock@openss7.org>
Date: Tue, 30 May 2006 23:55:26 -0600
> For example, it goes to great pains to permute upper order bits in
> the local address, which for most connections will be a constant
> value.
Consider an apache server hosting thousands of virtual
hosts. The local address will be different for every
such host.
Please drop linux-kernel in any future replies, this
discussion belongs on netdev not linux-kernel. Thanks.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Brian F. G. Bidulock @ 2006-05-31 7:45 UTC (permalink / raw)
To: David Miller; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531.001027.60486156.davem@davemloft.net>
David,
On Wed, 31 May 2006, David Miller wrote:
> From: "Brian F. G. Bidulock" <bidulock@openss7.org>
> Date: Tue, 30 May 2006 23:55:26 -0600
>
> > For example, it goes to great pains to permute upper order bits in
> > the local address, which for most connections will be a constant
> > value.
>
> Consider an apache server hosting thousands of virtual
> hosts. The local address will be different for every
> such host.
>
If you mean named virtual hosts, no. They have the same
addresses.
If you mean actual hosts (with an IP address), perhaps in
the low order bits (host number), but unlikely in the high
order bits of the local address (network mask bits).
Also, in such a case the local port number will be rather
constant (80, etc); a condition also not exploited by the
function.
Also consider that the function simply folds the values
rather than permuting bits across the key field by shifting
by some other value than a multiple of 8 between XOR
operations. This will result in a longer collision list
because the entropy of the key value has not been
sufficiently reduced.
It might sound like I'm complaining, but I'm not. The
function works for me. But from a purist point of view,
the hash function is not as efficient as it could be and
there is room for improvement.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: David Miller @ 2006-05-31 7:49 UTC (permalink / raw)
To: bidulock; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531014540.A1319@openss7.org>
From: "Brian F. G. Bidulock" <bidulock@openss7.org>
Date: Wed, 31 May 2006 01:45:40 -0600
> It might sound like I'm complaining, but I'm not. The
> function works for me. But from a purist point of view,
> the hash function is not as efficient as it could be and
> there is room for improvement.
For sure and there are plans afoot to move over to
dynamic table sizing and the Jenkins hash function.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Brian F. G. Bidulock @ 2006-05-31 8:00 UTC (permalink / raw)
To: David Miller; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531.004953.91760903.davem@davemloft.net>
David,
On Wed, 31 May 2006, David Miller wrote:
>
> For sure and there are plans afoot to move over to
> dynamic table sizing and the Jenkins hash function.
Yes, that could be far more efficient.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Brian F. G. Bidulock @ 2006-05-31 8:49 UTC (permalink / raw)
To: David Miller; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531.004953.91760903.davem@davemloft.net>
David,
On Wed, 31 May 2006, David Miller wrote:
>
> For sure and there are plans afoot to move over to
> dynamic table sizing and the Jenkins hash function.
Just a suggestion, but I have an approach that stands to be
faster than Jenkins deriving from the verification tag approach
that I took for SCTP (OpenSS7 SCTP not lksctp).
TCP uses a cryptographic hash function to select its initial
sequence number (SCTP does the same for vtag). Last I checked
it was taken from an MD4 swirled entropy pool and further
combined with the local and remote IP addresses and port
numbers.
Each received segment contains a sequence number that is offset
from the initial sequence number but is expected to appear
within the current window. Most of the high order bits of an
in-window sequence number are predicatable at any point in time
and, due to cryptographic strength, are more efficient than
Jenkins, ... and they are right there in the received packet.
The approach would take the high order bits of the received
sequence number and use them to index a separate sequence number
keyed established hash (which could be dynamic). As the window
changes, the socket would have to be removed and reinserted into
this hash, but the repositioning would be infrequent. Out of
window segments would fail to find a socket, but could fall back
to the current established hash, or even the bind hash. A layer
of caching could increase the hash lookup speed further for
noisy senders.
This would be faster than a Jenkins hash approach because it
would not be necessary to calculate the hash function at all for
in-window segments. Per packet overheads would decrease and
better small packet performance would be experienced (i.e. your
http server). It has better hash coverage because MD4 and other
cryptographic algorithms used for initial sequence number
selection have far better properties than Jenkins.
What do you think?
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: David Miller @ 2006-05-31 9:02 UTC (permalink / raw)
To: bidulock; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531024954.A2458@openss7.org>
From: "Brian F. G. Bidulock" <bidulock@openss7.org>
Date: Wed, 31 May 2006 02:49:54 -0600
> This would be faster than a Jenkins hash approach because it
> would not be necessary to calculate the hash function at all for
> in-window segments. Per packet overheads would decrease and
> better small packet performance would be experienced (i.e. your
> http server). It has better hash coverage because MD4 and other
> cryptographic algorithms used for initial sequence number
> selection have far better properties than Jenkins.
>
> What do you think?
I don't know how practical this is. The 4GB sequence space
wraps very fast on 10 gigabit, so we'd be rehashing a bit
and 100 gigabit would make things worse whenever that shows
up.
It is, however, definitely an interesting idea.
We also need the pure traditional hashes for net channels. I don't
see how we could use your scheme for net channels, because we are just
hashing in the interrupt handler of the network device driver in order
to get a queue to tack the packet onto, we're not interpreting the
sequence numbers and thus would not able to maintain the sequence
space based hashing state.
On a 3ghz cpu, the jenkins hash is essentially free. Even on slower
cpus, jhash_2words for example is just 20 cycles on a sparc64 chip.
It's ~40 integer instructions and they all pair up perfectly to
dual issue. We'd probably use jhash_3words() for TCP ipv4 which
would get us into the 30 cycle range.
A few years ago when I introduced jenkins into the tree, I thought
it's execution cost might be an issue. I really don't anymore.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Evgeniy Polyakov @ 2006-05-31 9:03 UTC (permalink / raw)
To: David Miller, draghuram, linux-kernel, netdev
In-Reply-To: <20060531020009.A1868@openss7.org>
On Wed, May 31, 2006 at 02:00:09AM -0600, Brian F. G. Bidulock (bidulock@openss7.org) wrote:
> > For sure and there are plans afoot to move over to
> > dynamic table sizing and the Jenkins hash function.
>
> Yes, that could be far more efficient.
In theory practice and theory are the same, but in practice they are
different.
Current simple XOR hash used in socket selection code is just bloody good!
Jenkins hash unfortunately has _significant_ artefacts which were found
in netchannel [1] hash selection analysis [2].
And Jenkins hash is far too slower.
1. Netchannel.
http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
2. Compared Jenkins hash with XOR hash used in TCP socket selection
code.
http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14
--
Evgeniy Polyakov
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: David Miller @ 2006-05-31 9:12 UTC (permalink / raw)
To: johnpol; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531090301.GA26782@2ka.mipt.ru>
From: Evgeniy Polyakov <johnpol@2ka.mipt.ru>
Date: Wed, 31 May 2006 13:03:02 +0400
> Current simple XOR hash used in socket selection code is just bloody
> good! Jenkins hash unfortunately has _significant_ artefacts which
> were found in netchannel [1] hash selection analysis [2]. And
> Jenkins hash is far too slower.
Yes, it wins from a simplicity standpoint and it performs quite well.
It was tuned from real socket data sets, but from systems running many
many years ago :)
FreeBSD even adopted this hash into their PCB hashing code at one
point.
I think it will need to be changed nevertheless. Even though this
hash works on established sockets, it is attackable just like the
routing hash used to be. If an attacker has sufficient resources, he
can make hash chains in the TCP established hash table very long. As
the years pass, it becomes easier and easier for one to have enough
computing power at their disposal to carry out this kind of attack.
So something like Jenkins with a random hash input become more and
more critical. And this requires some kind of way to rehash, RCU
table locking opens the door for that. Current locking scheme is
too tightly bound for that kind of flexibility.
I wish Ben L. would resubmit the TCP hash locking stuff he said he'd
work on. :)
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Brian F. G. Bidulock @ 2006-05-31 9:39 UTC (permalink / raw)
To: David Miller; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531.020239.00305778.davem@davemloft.net>
David,
On Wed, 31 May 2006, David Miller wrote:
>
> I don't know how practical this is. The 4GB sequence space
> wraps very fast on 10 gigabit, so we'd be rehashing a bit
> and 100 gigabit would make things worse whenever that shows
> up.
It works better for SCTP, because the vtags are constant. No
rehashing is required there.
But, also consider that rehashing is only required for senders
sending at a high data rate. (http clients will likely never
have to be rehashed.) These packets will typically be large and
per-packet overheads will be overwhelmed by per-byte overheads.
Also, the rehashing is orderly and simple, the entry is simply
bumped to the next sequential hash slot and the socket hash
structure can already be cached at the time the action is
performed. Rehashing, although a bother, would take little
time, and could simply be added as part of TCP's existing window
calculations.
>
> It is, however, definitely an interesting idea.
>
> We also need the pure traditional hashes for net channels. I don't
> see how we could use your scheme for net channels, because we are just
> hashing in the interrupt handler of the network device driver in order
> to get a queue to tack the packet onto, we're not interpreting the
> sequence numbers and thus would not able to maintain the sequence
> space based hashing state.
Under SCTP I still have the traditional established hash for
lookups of out of the blue packets and packets containing
invalid verification tags. Really long lookups would invite DoS
attacks on these.
>
> On a 3ghz cpu, the jenkins hash is essentially free. Even on slower
> cpus, jhash_2words for example is just 20 cycles on a sparc64 chip.
> It's ~40 integer instructions and they all pair up perfectly to
> dual issue. We'd probably use jhash_3words() for TCP ipv4 which
> would get us into the 30 cycle range.
But you could throw away all 30 cycles, plus the stacking and
unstacking of registers to get in and out of the algorithm.
Some architectures might benefit more.
Well, I thought you might find it interesting. Perhaps somebody
reading this will experiment with it. For SCTP it is one of a
number of techniques that allows OpenSS7 SCTP to drastically
outperform lksctp.
--brian
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Evgeniy Polyakov @ 2006-05-31 9:44 UTC (permalink / raw)
To: David Miller; +Cc: draghuram, linux-kernel, netdev
In-Reply-To: <20060531.021243.122061524.davem@davemloft.net>
On Wed, May 31, 2006 at 02:12:43AM -0700, David Miller (davem@davemloft.net) wrote:
> I think it will need to be changed nevertheless. Even though this
> hash works on established sockets, it is attackable just like the
> routing hash used to be. If an attacker has sufficient resources, he
> can make hash chains in the TCP established hash table very long. As
> the years pass, it becomes easier and easier for one to have enough
> computing power at their disposal to carry out this kind of attack.
Jenkins hash is very complex compared to tcp XOR one, and even simple
test showed it's bottlenecks in some setups, so it's tuning will be
quite challenging both from mathematical and engineering points of view.
And socket code actually differs from routing cache, since the former is
limited to 64k connections in a time, while routing cache can grow to
unpredictibe sizes.
> So something like Jenkins with a random hash input become more and
> more critical. And this requires some kind of way to rehash, RCU
> table locking opens the door for that. Current locking scheme is
> too tightly bound for that kind of flexibility.
>
> I wish Ben L. would resubmit the TCP hash locking stuff he said he'd
> work on. :)
It was good approach indeed.
--
Evgeniy Polyakov
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Brian F. G. Bidulock @ 2006-05-31 9:51 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: David Miller, draghuram, linux-kernel, netdev
In-Reply-To: <20060531090301.GA26782@2ka.mipt.ru>
Evgeniy,
On Wed, 31 May 2006, Evgeniy Polyakov wrote:
> 2. Compared Jenkins hash with XOR hash used in TCP socket selection code.
> http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14
Two problems with the comparison:
Port numbers can be collected into a 32 bit register in network
byte order directly from the TCP packet without taking two 16 bit
values and shifting and or'ing them.
Worse: he folded the jenkins algorith result with
h ^= h >> 16;
h ^= h >> 8;
Destroying the coverage of the function.
I, for one, am not suprised that artifacts appeared in the comparison
as a result of this destruction of the coverage of the hashing function.
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Brian F. G. Bidulock @ 2006-05-31 9:52 UTC (permalink / raw)
To: Evgeniy Polyakov; +Cc: David Miller, draghuram, linux-kernel, netdev
In-Reply-To: <20060531090301.GA26782@2ka.mipt.ru>
Evgeniy,
On Wed, 31 May 2006, Evgeniy Polyakov wrote:
>
> 1. Netchannel.
> http://tservice.net.ru/~s0mbre/old/?section=projects&item=netchannel
This one refers to the erroneous result below.
>
> 2. Compared Jenkins hash with XOR hash used in TCP socket selection
> code.
> http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Evgeniy Polyakov @ 2006-05-31 10:58 UTC (permalink / raw)
To: David Miller, draghuram, linux-kernel, netdev
In-Reply-To: <20060531035124.B3065@openss7.org>
On Wed, May 31, 2006 at 03:51:24AM -0600, Brian F. G. Bidulock (bidulock@openss7.org) wrote:
> Evgeniy,
>
> On Wed, 31 May 2006, Evgeniy Polyakov wrote:
> > 2. Compared Jenkins hash with XOR hash used in TCP socket selection code.
> > http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14
>
> Two problems with the comparison:
>
> Port numbers can be collected into a 32 bit register in network
> byte order directly from the TCP packet without taking two 16 bit
> values and shifting and or'ing them.
They are.
u32 ports;
ports = lport;
ports <<= 16;
ports |= fport;
> Worse: he folded the jenkins algorith result with
>
> h ^= h >> 16;
> h ^= h >> 8;
>
> Destroying the coverage of the function.
It was done to simulate socket code which uses the same folding.
Leaving 32bit space is just wrong, consider hash table size with that
index.
> I, for one, am not suprised that artifacts appeared in the comparison
> as a result of this destruction of the coverage of the hashing function.
It is comparison of the approach used in TCP hashing code, it is not full
mathematical analysis. And in that case jenkins hash already not good.
I'm sure it can be tuned, but it does require a lot of iterations, while
XOR one "just works".
--
Evgeniy Polyakov
^ permalink raw reply
* Re: Question about tcp hash function tcp_hashfn()
From: Evgeniy Polyakov @ 2006-05-31 11:04 UTC (permalink / raw)
To: David Miller, draghuram, linux-kernel, netdev; +Cc: Brian F. G. Bidulock
In-Reply-To: <20060531105814.GB7806@2ka.mipt.ru>
On Wed, May 31, 2006 at 02:58:18PM +0400, Evgeniy Polyakov (johnpol@2ka.mipt.ru) wrote:
> On Wed, May 31, 2006 at 03:51:24AM -0600, Brian F. G. Bidulock (bidulock@openss7.org) wrote:
> > Evgeniy,
> >
> > On Wed, 31 May 2006, Evgeniy Polyakov wrote:
> > > 2. Compared Jenkins hash with XOR hash used in TCP socket selection code.
> > > http://tservice.net.ru/~s0mbre/blog/2006/05/14#2006_05_14
> >
> > Two problems with the comparison:
> >
> > Port numbers can be collected into a 32 bit register in network
> > byte order directly from the TCP packet without taking two 16 bit
> > values and shifting and or'ing them.
>
> They are.
>
> u32 ports;
>
> ports = lport;
> ports <<= 16;
> ports |= fport;
Using network or host byte order does not affect hash distribution,
that shifting was coded to simulate other types of mixing ports,
which actually never showed different results.
> > Worse: he folded the jenkins algorith result with
> >
> > h ^= h >> 16;
> > h ^= h >> 8;
> >
> > Destroying the coverage of the function.
>
> It was done to simulate socket code which uses the same folding.
> Leaving 32bit space is just wrong, consider hash table size with that
> index.
>
> > I, for one, am not suprised that artifacts appeared in the comparison
> > as a result of this destruction of the coverage of the hashing function.
>
> It is comparison of the approach used in TCP hashing code, it is not full
> mathematical analysis. And in that case jenkins hash already not good.
> I'm sure it can be tuned, but it does require a lot of iterations, while
> XOR one "just works".
--
Evgeniy Polyakov
^ permalink raw reply
* Re: skge driver oops
From: Beschorner Daniel @ 2006-05-31 11:05 UTC (permalink / raw)
To: 'Stephen Hemminger'
Cc: 'Krzysztof Oledzki', netdev, 'david@mantara.com'
> I assume you sk98lin version downloaded from their web site.
Right.
sk98lin: Network Device Driver v8.32.2.3
(C)Copyright 1999-2006 Marvell(R).
ACPI: PCI Interrupt 0000:00:0b.0[A] -> GSI 16 (level, low) -> IRQ 19
eth0: DGE-530T Gigabit Ethernet Adapter
PrefPort:A RlmtMode:Check Link State
eth0: network connection up using port A
speed: 1000
autonegotiation: yes
duplex mode: full
flowctrl: symmetric
role: slave
irq moderation: disabled
scatter-gather: enabled
tx-checksum: enabled
rx-checksum: enabled
^ permalink raw reply
* Re: Refactor Netlink connector?
From: jamal @ 2006-05-31 12:00 UTC (permalink / raw)
To: James Morris; +Cc: johnpol, netdev, tgraf, sds, David Miller
In-Reply-To: <Pine.LNX.4.64.0605301019180.25929@d.namei>
On Tue, 2006-30-05 at 10:22 -0400, James Morris wrote:
> On Mon, 29 May 2006, jamal wrote:
>
> > If SELinux should provide ways to add "filters" more dynamically at its
> > hooks - instead of having people go and look for that table and update
[..]
>
> This is similar to what the secmark stuff does, allows selection and
> labeling to be done via iptables, so the SELinux kernel stuff then just
> needs to look at the labels.
hopefully SELinux is taught about such labels semantics at runtime.
> In this case, I'm not sure it's worthwhile adding a filtering layer to
> Netlink, probably simpler just to have the different Netlink protocols
> define whether each command is one of 'read', 'write' and 'readpriv' (the
> latter is pretty rare), so nothing has to be scanned on the fly at all.
>
We could start by just adding a check for NETLINK_GENERIC in your table
(as is done generally for other netlink families/protocols with SELinux)
and then do the fine-grained stuff. I think that checking for attributes
instead of types will need to be generic for all of netlink.
Thomas?
cheers,
jamal
^ permalink raw reply
* orinoco encodeext + auth patch on lkml
From: Dan Williams @ 2006-05-31 12:03 UTC (permalink / raw)
To: netdev; +Cc: proski
Pavel,
Unsure if you saw this. Slightly annoying he didn't send to netdev, but
would be good to take a look at and possibly integrate. Looks sane to
me from 10,000ft.
http://lkml.org/lkml/2006/5/30/433
Dan
^ permalink raw reply
* Re: netif_tx_disable and lockless TX
From: Herbert Xu @ 2006-05-31 12:06 UTC (permalink / raw)
To: David Miller; +Cc: mchan, jgarzik, netdev
In-Reply-To: <20060531.000818.78646242.davem@davemloft.net>
On Wed, May 31, 2006 at 12:08:18AM -0700, David Miller wrote:
>
> I understand what you're saying. What I'm talking about is that in
> net channel drivers we might go back to IRQ disabling locks again.
OK, let's assume that the TX completion will go back into the IRQ
handler. I contend that we can still get by with a BH-disabling
xmit_lock. This is how it would work:
The IRQ handler would look like
if (!spin_trylock(&dev->xmit_lock)) {
tasklet_schedule(&priv->tx_completion_tasklet);
return;
}
handle_tx_completion();
spin_unlock(&dev->xmit_lock);
Where the TX completion tasklet would simply do
spin_lock(&dev->xmit_lock);
handle_tx_completion();
spin_unlock(&dev->xmit_lock);
What do you think?
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: Refactor Netlink connector?
From: jamal @ 2006-05-31 12:20 UTC (permalink / raw)
To: Thomas Graf
Cc: Stephen Smalley, David S. Miller, netdev, Evgeniy Polyakov,
James Morris
In-Reply-To: <20060531030046.GC7844@postel.suug.ch>
On Wed, 2006-31-05 at 05:00 +0200, Thomas Graf wrote:
> * James Morris <jmorris@namei.org> 2006-05-27 13:21
> > Actually, a possible solution here is to completely remove all internal
> > knowledge of netlink messages from SELinux and have the netfilter
> > framework and protocols provide methods to determine message types and
> > permissions.
>
> Right, regarding generic netlink we can extend struct genl_ops to
> include a policy stating what permissions are required.
The challenge is how to inform SELinux of these permissions.
The access limit could be done by putting a SELinux hook at the time the
skb gets to the generic netlink code?
Note: There's actually two things that can be classified for access
control, the genl family as well as the ops.
> Besides
> that we can extend struct nla_policy to support validating of
> attributes.
This is even further granularity that opens a whole new can of worms.
BTW, I abused the term "attribute" in my other email to James. In that
context it means metadata for the command and in the above case it means
the "T" in TLV. Despite that they are strongly related, just that
the packet offsets are different and the checks are for different
things: SELinux policy is a simple accept/deny based on some policy
(provisioned in user space??) and nla_policy is richer with a range
check for sanity reasons as opposed to access control and then
accept/deny.
cheers,
jamal
^ permalink raw reply
* Re: netif_tx_disable and lockless TX
From: jamal @ 2006-05-31 12:36 UTC (permalink / raw)
To: Herbert Xu
Cc: netdev, jgarzik, mchan, David Miller, Andi Kleen, Robert Olsson
In-Reply-To: <20060531120626.GA11925@gondor.apana.org.au>
On Wed, 2006-31-05 at 22:06 +1000, Herbert Xu wrote:
> On Wed, May 31, 2006 at 12:08:18AM -0700, David Miller wrote:
> >
> > I understand what you're saying. What I'm talking about is that in
> > net channel drivers we might go back to IRQ disabling locks again.
>
> OK, let's assume that the TX completion will go back into the IRQ
> handler. I contend that we can still get by with a BH-disabling
> xmit_lock. This is how it would work:
>
> The IRQ handler would look like
>
> if (!spin_trylock(&dev->xmit_lock)) {
> tasklet_schedule(&priv->tx_completion_tasklet);
> return;
> }
>
> handle_tx_completion();
> spin_unlock(&dev->xmit_lock);
>
> Where the TX completion tasklet would simply do
>
> spin_lock(&dev->xmit_lock);
> handle_tx_completion();
> spin_unlock(&dev->xmit_lock);
>
> What do you think?
Been done in the past, bad numbers especially in SMP for reasons of
latency and likelihood that a tasklet will run in a totally different
CPU.
Latency-wise: TX completion interrupt provides the best latency.
Processing in the poll() -aka softirq- was almost close to the hardirq
variant. So if you can make things run in a softirq such as transmit
one, then the numbers will likely stay the same.
Sorry, I havent been following discussions on netchannels[1] so i am not
qualified to comment on the "replacement" part Dave mentioned earlier.
What I can say is the tx processing doesnt have to be part of the NAPI
poll() and still use hardirq.
If you look at the earlier NAPI drivers such as the tulip or the
broadcom 1250, the TX EOL was always made an exception and was never
touched by the ->poll() code rather by the main interrupt routine. I do
prefer the scheme of leaving the TX EOL out of the NAPI poll because i
believe that it provides better performance. The caveat is there's a lot
of fscked hardware out there that does unconditional clear-on-read of
the int status register;-> I had a lot of fun with bcm1250 and ended
discovering that there were infact two status regs ;-> (it pays to know
people for undocumented things;->) one was for debug version which
doesnt clear on writing. So in one spot i would read the real register
and in the other the debug version. I had numbers which showed doing it
this way provided better performance than doing it in the poll() routine
(scouring the code just now it seems my version never made it in, so you
will see something along the lines of e1000/tg3).
BTW, Andi Kleen on CC is the person who posted some numbers on the LLTX
(as well as the patch) at some point.
I have also CCed Robert who may have comments on the use of tasklets.
cheers,
jamal
[1] Such a shame, such an exciting topic, such few cycles. I will have
time RealSoonNow and you will probably see a_patch_from_me_too.
^ permalink raw reply
* Re: netif_tx_disable and lockless TX
From: Herbert Xu @ 2006-05-31 12:40 UTC (permalink / raw)
To: jamal; +Cc: netdev, jgarzik, mchan, David Miller, Andi Kleen, Robert Olsson
In-Reply-To: <1149078972.5462.72.camel@jzny2>
On Wed, May 31, 2006 at 08:36:12AM -0400, jamal wrote:
>
> > The IRQ handler would look like
> >
> > if (!spin_trylock(&dev->xmit_lock)) {
> > tasklet_schedule(&priv->tx_completion_tasklet);
> > return;
> > }
> >
> > handle_tx_completion();
> > spin_unlock(&dev->xmit_lock);
> >
> > Where the TX completion tasklet would simply do
> >
> > spin_lock(&dev->xmit_lock);
> > handle_tx_completion();
> > spin_unlock(&dev->xmit_lock);
> >
> > What do you think?
>
> Been done in the past, bad numbers especially in SMP for reasons of
> latency and likelihood that a tasklet will run in a totally different
> CPU.
Was the bad latency measured with code like the above or was it with
an unconditional tasklet schedule?
Please note that my code will do the completion in the IRQ handler
most of the time. Only if there is contention for the spin lock will
it defer work to the softirq.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: netif_tx_disable and lockless TX
From: jamal @ 2006-05-31 13:03 UTC (permalink / raw)
To: Herbert Xu
Cc: Robert Olsson, Andi Kleen, David Miller, mchan, jgarzik, netdev
In-Reply-To: <20060531124036.GA12171@gondor.apana.org.au>
On Wed, 2006-31-05 at 22:40 +1000, Herbert Xu wrote:
> On Wed, May 31, 2006 at 08:36:12AM -0400, jamal wrote:
> >
[..]
> >
> > Been done in the past, bad numbers especially in SMP for reasons of
> > latency and likelihood that a tasklet will run in a totally different
> > CPU.
>
> Was the bad latency measured with code like the above or was it with
> an unconditional tasklet schedule?
>
The idea was to prune those skbs for reuse within reason so as to
not sacrifice perfomance. Perfomance is sacrificed if you have too many
TX EOLs, or you dont prune the descriptors fast enough. The dynamics of
proved to be hard to nail in a generic way because behavior varies based
on traffic patterns.
So the "within reason" part will unconditionally schedule a tasklet on
the tx if certain DMA ring thresholds are crossed, way after the tx lock
has been grabbed which is different from your suggestion to do it when
you cant grab a lock.
Otherwise you let the hardirq take care of things. Tasklets proved to be
a bad idea and instead we ended reclaiming the skb/descriptors right at
the point where we initially had scheduled tasklets. I dont know what
happened to using that technique - Robert can testify better than i.
> Please note that my code will do the completion in the IRQ handler
> most of the time. Only if there is contention for the spin lock will
> it defer work to the softirq.
that part is different and I think there may be potential - it will
require experimenting with a variety of traffic patterns. I wish i had
time to help.
cheers,
jamal
^ permalink raw reply
* Re: Refactor Netlink connector?
From: Thomas Graf @ 2006-05-31 13:06 UTC (permalink / raw)
To: jamal
Cc: Stephen Smalley, David S. Miller, netdev, Evgeniy Polyakov,
James Morris
In-Reply-To: <1149078055.5462.55.camel@jzny2>
* jamal <hadi@cyberus.ca> 2006-05-31 08:20
> The challenge is how to inform SELinux of these permissions.
> The access limit could be done by putting a SELinux hook at the time the
> skb gets to the generic netlink code?
> Note: There's actually two things that can be classified for access
> control, the genl family as well as the ops.
We already have the flag GENL_ADMIN_PERM which when set for a
struct genl_ops calls security_netlink_recv(). It's not as
fine grained as it could be though. The point is that adding
fine grained SELinux support is no problem and even easier than
for casual netlink families.
> This is even further granularity that opens a whole new can of worms.
I agree, the advantage is that the genetlink code already takes
care of validating the attributes, all we have to do is allow
genetlink families to provide a policy.
> BTW, I abused the term "attribute" in my other email to James. In that
> context it means metadata for the command and in the above case it means
> the "T" in TLV. Despite that they are strongly related, just that
> the packet offsets are different and the checks are for different
> things: SELinux policy is a simple accept/deny based on some policy
> (provisioned in user space??) and nla_policy is richer with a range
> check for sanity reasons as opposed to access control and then
> accept/deny.
Right, the important point is that for genetlink we already have
a point where we peek at the attributes and adding a hook is
trivial unlike for other netlink families where they'd have to be
spread in the code.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox