* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Herbert Xu @ 2010-03-30 14:30 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB1FDC0.5020200@iki.fi>
On Tue, Mar 30, 2010 at 04:33:52PM +0300, Timo Teräs wrote:
> Herbert Xu wrote:
>> On Tue, Mar 30, 2010 at 03:41:02PM +0300, Timo Teräs wrote:
>>> So it'd make more sense to nuke the hashes entirely for
>>> per-socket policies?
>>
>> Absolutely.
>
> I checked now the xfrm_user, and mostly it seems to prevent
> modification to per-socket policies.
>
> The only exception is XFRM_MSG_POLEXPIRE handler
> xfrm_add_pol_expire(). It calls xfrm_policy_byid() without
> verifying the direction, and can thus complete successfully on
> a per-socket policy. This can actually result in per-socket
> policy deletion via netlink.
That shouldn't be possible since the directions used by socket
policies cannot be set through xfrm_user.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Herbert Xu @ 2010-03-30 14:29 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB2044B.5080704@iki.fi>
On Tue, Mar 30, 2010 at 05:01:47PM +0300, Timo Teräs wrote:
>
> Since the exported function xfrm_policy_byid() can result in deletion
> of socket policy, it's safer to leave this change in. This is can be
> even triggered via xfrm_user since it does not check 'dir' for the
> policy expired message it handles. Any custom module could do similar
> harm.
That's a bug. Socket policies should never be deleted by anyone
other than the socket owner through a setsockopt.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: debugging kernel during packet drops
From: Patrick McHardy @ 2010-03-30 14:12 UTC (permalink / raw)
To: Jan Engelhardt; +Cc: Eric Dumazet, Jorrit Kronjee, netfilter-devel, netdev
In-Reply-To: <alpine.LSU.2.01.1003301405490.10802@obet.zrqbmnf.qr>
Jan Engelhardt wrote:
> On Thursday 2010-03-25 11:35, Patrick McHardy wrote:
>
>> This clashes with some recent cleanups in nf-next-2.6.git. I'm
>> also expecting a patch from Jan to remove the old v0 revision
>> very soon (probably today). Please rediff once I've pushed that out.
>
> One 12-series request has been sitting there for a while. Was there
> something not in order with it?
No, I'm currently a bit backlogged due to getting sick. I'm running
your series for testing though, I'll probably push it out tommorrow.
^ permalink raw reply
* Re: UDP path MTU discovery
From: Andi Kleen @ 2010-03-30 14:12 UTC (permalink / raw)
To: Edgar E. Iglesias
Cc: Andi Kleen, Templin, Fred L, Eric Dumazet, Rick Jones,
Glen Turner, netdev@vger.kernel.org
In-Reply-To: <20100330082013.GA18300@edde.se.axis.com>
> Where does the application controllable per socket MTU come into the
> picture?
To set the minimum path MTU so that there is a guarantee that IPv6 routers
(which are unable to fragment themselves) will never drop it.
-Andi
--
ak@linux.intel.com -- Speaking for myself only.
^ permalink raw reply
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Timo Teräs @ 2010-03-30 14:01 UTC (permalink / raw)
To: Herbert Xu; +Cc: netdev
In-Reply-To: <4BB1E8B1.4030604@iki.fi>
Timo Teräs wrote:
> Herbert Xu wrote:
>> On Tue, Mar 30, 2010 at 07:55:07AM +0300, Timo Teräs wrote:
>>> Herbert Xu wrote:
>>>> On Mon, Mar 29, 2010 at 05:12:38PM +0300, Timo Teras wrote:
>>>>> @@ -1132,7 +1119,7 @@ int xfrm_sk_policy_insert(struct sock *sk,
>>>>> int dir, struct xfrm_policy *pol)
>>>>> __xfrm_policy_link(pol, XFRM_POLICY_MAX+dir);
>>>>> }
>>>>> if (old_pol)
>>>>> - __xfrm_policy_unlink(old_pol, XFRM_POLICY_MAX+dir);
>>>>> + old_pol = __xfrm_policy_unlink(old_pol, XFRM_POLICY_MAX+dir);
>>>>> write_unlock_bh(&xfrm_policy_lock);
>>>>> if (old_pol) {
>>>> So when can this actually fail?
>>> Considering that the socket reference is received from the
>>> sk->sk_policy,
>>> and the hash bucket we use is "XFRM_POLICY_MAX+dir", it's non-obvious if
>>> it can fail or not.
>>>
>>> It would look like the timer can kill a policy and unlink it, but it
>>> would still be found from sk_policy.
>>
>> Socket policies cannot expire.
>
> Was not aware of that. The above is not needed then.
Since the exported function xfrm_policy_byid() can result in deletion
of socket policy, it's safer to leave this change in. This is can be
even triggered via xfrm_user since it does not check 'dir' for the
policy expired message it handles. Any custom module could do similar
harm.
^ permalink raw reply
* Re: linux-next: manual merge of the net tree with the wireless-current tree
From: Stephen Rothwell @ 2010-03-30 13:51 UTC (permalink / raw)
To: John W. Linville
Cc: David Miller, netdev, linux-next, linux-kernel, Shanyu Zhao,
Zhu Yi, Reinette Chatre, Johannes Berg
In-Reply-To: <20100330131220.GA13120@tuxdriver.com>
[-- Attachment #1: Type: text/plain, Size: 335 bytes --]
Hi John,
On Tue, 30 Mar 2010 09:12:21 -0400 "John W. Linville" <linville@tuxdriver.com> wrote:
>
> Yes, that looks correct to me -- at least, that's what I did in
> wireless-testing. :-)
Thanks for the confirmation.
--
Cheers,
Stephen Rothwell sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Timo Teräs @ 2010-03-30 13:33 UTC (permalink / raw)
To: Herbert Xu; +Cc: netdev
In-Reply-To: <20100330124815.GA6378@gondor.apana.org.au>
Herbert Xu wrote:
> On Tue, Mar 30, 2010 at 03:41:02PM +0300, Timo Teräs wrote:
>> So it'd make more sense to nuke the hashes entirely for
>> per-socket policies?
>
> Absolutely.
I checked now the xfrm_user, and mostly it seems to prevent
modification to per-socket policies.
The only exception is XFRM_MSG_POLEXPIRE handler
xfrm_add_pol_expire(). It calls xfrm_policy_byid() without
verifying the direction, and can thus complete successfully on
a per-socket policy. This can actually result in per-socket
policy deletion via netlink.
I guess the proper thing is to add the direction check there.
It also seems that the by-index hash is also used when
generating new index. It's to double check that the index
is unique. So deleting the by-index hash from per-socket
policies seems tricky.
Removing bydst hashing should be trivial.
^ permalink raw reply
* Re: XT_ALIGN changed to use ALIGN breaks iproute2
From: Alexey Dobriyan @ 2010-03-30 13:15 UTC (permalink / raw)
To: Andreas Henriksson; +Cc: Patrick McHardy, Stephen Hemminger, jamal, netdev
In-Reply-To: <20100330092829.GA3106@amd64.fatal.se>
On Tue, Mar 30, 2010 at 12:28 PM, Andreas Henriksson <andreas@fatal.se> wrote:
> You updated the kernel header include/linux/netfilter/x_tables.h
> in torvalds/linux-2.6.git commit 42107f5009da223daa800d6da6904d77297ae829
> with the comment "Use ALIGN() macro while I'm at it for same types.".
>
> When this header was synced into iproute2 the build broke because the
> ALIGN macro apparently only is defined in kernel headers.
>
> (For iproute2 the problem was introduced in
> 8ecdcce08319d0e39b0d32c1d17db3f69d85a35c and found by Stephen
> and worked around in 609ceb807deba8e23 and edaaa11e5a3cf2c9c1a39)
>
> I'm guessing the problem in the iproute2 header sync is just a heads
> up for what's going to happen when distributions updates their
> system headers to match linux 2.6.33.
>
>
> Could someone who knows how the userspace version of the kernel
> headers are generated please find a suitable solution?
We can export ALIGN to userspace, but the name is so generic,
so it's not clear what breakage more risky.
XT_ALIGN is a macro so breakage will appear only when it's used,
not when header is included directly or indirectly.
We have tc, iptables, both carry their own copy of headers, what else?
Right now, I'd say, do nothing, and iptables will carry fixlet as in tc,
eventually.
^ permalink raw reply
* Re: linux-next: manual merge of the net tree with the wireless-current tree
From: John W. Linville @ 2010-03-30 13:12 UTC (permalink / raw)
To: Stephen Rothwell
Cc: David Miller, netdev, linux-next, linux-kernel, Shanyu Zhao,
Zhu Yi, Reinette Chatre, Johannes Berg
In-Reply-To: <20100330133745.27295ee0.sfr@canb.auug.org.au>
On Tue, Mar 30, 2010 at 01:37:45PM +1100, Stephen Rothwell wrote:
> Hi all,
>
> Today's linux-next merge of the net tree got a conflict in
> drivers/net/wireless/iwlwifi/iwl-agn.c between commit
> d4dca4e53fde2953d74198a2c445db1d36ed9fd2 ("iwlwifi: clear unattended
> interrupts in tasklet") from the wireless-current tree and commit
> a4c8b2a692601de0a7bcb032b69f806050944dff ("iwlwifi: move ICT data to agn
> part of union") from the net tree.
>
> I fixed it up (see below) and can carry the fix for a while.
Yes, that looks correct to me -- at least, that's what I did in
wireless-testing. :-)
John
--
John W. Linville Someday the world will need a hero, and you
linville@tuxdriver.com might be all we have. Be ready.
^ permalink raw reply
* [PATCH] skb_put: remove not needed check for skb linearity
From: Paulius Zaleckas @ 2010-03-30 13:01 UTC (permalink / raw)
To: davem; +Cc: netdev
It is safe to call skb_put() on packets containing fragments.
Actually I have a case where I allocate packet header with some
extra headroom and then I dynamically add data as frag_list. After
adding frags I have to add more data to header and skb_put()
just BUG's on me :)
And we will save couple instructions for CPU.
Signed-off-by: Paulius Zaleckas <paulius.zaleckas@gmail.com>
---
include/linux/skbuff.h | 1 -
net/core/skbuff.c | 1 -
2 files changed, 0 insertions(+), 2 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 124f90c..194e9fa 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -1108,7 +1108,6 @@ extern unsigned char *skb_put(struct sk_buff *skb, unsigned int len);
static inline unsigned char *__skb_put(struct sk_buff *skb, unsigned int len)
{
unsigned char *tmp = skb_tail_pointer(skb);
- SKB_LINEAR_ASSERT(skb);
skb->tail += len;
skb->len += len;
return tmp;
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 93c4e06..ea1ca61 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -1011,7 +1011,6 @@ EXPORT_SYMBOL(skb_pad);
unsigned char *skb_put(struct sk_buff *skb, unsigned int len)
{
unsigned char *tmp = skb_tail_pointer(skb);
- SKB_LINEAR_ASSERT(skb);
skb->tail += len;
skb->len += len;
if (unlikely(skb->tail > skb->end))
^ permalink raw reply related
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Herbert Xu @ 2010-03-30 12:48 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB1F15E.3030402@iki.fi>
On Tue, Mar 30, 2010 at 03:41:02PM +0300, Timo Teräs wrote:
>
> So it'd make more sense to nuke the hashes entirely for
> per-socket policies?
Absolutely.
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 4/7] flow: delayed deletion of flow cache entries
From: Timo Teräs @ 2010-03-30 12:43 UTC (permalink / raw)
To: Herbert Xu; +Cc: netdev
In-Reply-To: <20100330123612.GB6174@gondor.apana.org.au>
Herbert Xu wrote:
> On Tue, Mar 30, 2010 at 03:32:58PM +0300, Timo Teräs wrote:
>> If flow cache regeneration or shrinking is triggered in lookup,
>> it would previously free it in place. Now that is deferred. But
>> yes, it's more useful after the next patches that call the
>> virtual destructor. Should have explained this better.
>
> Any chance you can refactor them so that this comes after the
> virtual get/put patch?
>
> That way can evaluate this on its own merit rather than being
> a prerequisite for the more important stuff.
I thought it's not good to have possible speed regressions even
temporarily in the tree, so I figured this should go first.
But sure, I'll refactor this to be a later commit for the next
iteration.
^ permalink raw reply
* Re: Network performance - iperf
From: Eric Dumazet @ 2010-03-30 12:41 UTC (permalink / raw)
To: michal.simek
Cc: LKML, John Williams, netdev, Grant Likely, John Linn,
Steven J. Magnani, Arnd Bergmann, akpm
In-Reply-To: <4BB1C5A5.5070202@petalogix.com>
Le mardi 30 mars 2010 à 11:34 +0200, Michal Simek a écrit :
> 2233 packets pruned from receive queue because of socket buffer overrun
> TCPRcvCollapsed: 207654
Thats a problem. A big one :(
If I remember, you use LL_TEMAC driver.
This drivers allocates big skbs for RX ring (more than 9000 bytes each
skb). Given your 32 Mbytes kernel size, this seems plain wrong.
You might try to copybreak them before giving skb to network stack,
consuming the minimum space.
This would also help this driver to survive in low memory conditions,
avoiding death if high order pages are not available.
I cannot even compile this driver on my x86 platform, but here is a
preliminar patch to give you the idea :
[PATCH] ll_temac: Fix some memory allocation problems
Driver use high order allocations that might fail after a while.
When receiving a buffer from card, try to copy it to keep a pool of
pre-allocated high order buffers.
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
Please please please note I did not test this patch.
drivers/net/ll_temac_main.c | 48 +++++++++++++++-------------------
1 file changed, 22 insertions(+), 26 deletions(-)
diff --git a/drivers/net/ll_temac_main.c b/drivers/net/ll_temac_main.c
index a18e348..412b72e 100644
--- a/drivers/net/ll_temac_main.c
+++ b/drivers/net/ll_temac_main.c
@@ -155,8 +155,8 @@ static int temac_dma_bd_init(struct net_device *ndev)
lp->rx_bd_v[i].next = lp->rx_bd_p +
sizeof(*lp->rx_bd_v) * ((i + 1) % RX_BD_NUM);
- skb = alloc_skb(XTE_MAX_JUMBO_FRAME_SIZE
- + XTE_ALIGN, GFP_ATOMIC);
+ skb = alloc_skb(XTE_MAX_JUMBO_FRAME_SIZE + XTE_ALIGN,
+ GFP_KERNEL);
if (skb == 0) {
dev_err(&ndev->dev, "alloc_skb error %d\n", i);
return -1;
@@ -625,34 +625,30 @@ static void ll_temac_recv(struct net_device *ndev)
skb = lp->rx_skb[lp->rx_bd_ci];
length = cur_p->app4 & 0x3FFF;
- skb_vaddr = virt_to_bus(skb->data);
+ new_skb = netdev_alloc_skb_ip_align(length);
+ if (new_skb) {
+ skb_copy_to_linear_data(new_skb, skb->data, length);
+ skb_put(new_skb, length);
+ skb_vaddr = virt_to_bus(skb->data);
+ dma_sync_single_for_device(ndev->dev.parent,
+ skb_vaddr,
+ XTE_MAX_JUMBO_FRAME_SIZE,
+ PCI_DMA_FROMDEVICE);
+ new_skb->dev = ndev;
+ new_skb->protocol = eth_type_trans(new_skb, ndev);
+ new_skb->ip_summed = CHECKSUM_NONE;
+
+ netif_rx(new_skb);
+
+ ndev->stats.rx_packets++;
+ ndev->stats.rx_bytes += length;
+ } else
+ ndev->stats.rx_dropped++;
+
dma_unmap_single(ndev->dev.parent, skb_vaddr, length,
DMA_FROM_DEVICE);
- skb_put(skb, length);
- skb->dev = ndev;
- skb->protocol = eth_type_trans(skb, ndev);
- skb->ip_summed = CHECKSUM_NONE;
-
- netif_rx(skb);
-
- ndev->stats.rx_packets++;
- ndev->stats.rx_bytes += length;
-
- new_skb = alloc_skb(XTE_MAX_JUMBO_FRAME_SIZE + XTE_ALIGN,
- GFP_ATOMIC);
- if (new_skb == 0) {
- dev_err(&ndev->dev, "no memory for new sk_buff\n");
- spin_unlock_irqrestore(&lp->rx_lock, flags);
- return;
- }
-
- skb_reserve(new_skb, BUFFER_ALIGN(new_skb->data));
-
cur_p->app0 = STS_CTRL_APP0_IRQONEND;
- cur_p->phys = dma_map_single(ndev->dev.parent, new_skb->data,
- XTE_MAX_JUMBO_FRAME_SIZE,
- DMA_FROM_DEVICE);
cur_p->len = XTE_MAX_JUMBO_FRAME_SIZE;
lp->rx_skb[lp->rx_bd_ci] = new_skb;
^ permalink raw reply related
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Timo Teräs @ 2010-03-30 12:41 UTC (permalink / raw)
To: Herbert Xu; +Cc: netdev
In-Reply-To: <20100330122301.GG5731@gondor.apana.org.au>
Herbert Xu wrote:
> On Tue, Mar 30, 2010 at 03:21:19PM +0300, Timo Teräs wrote:
>> Hmm... is it possible to modify/delete per-socket policies from
>> userland via xfrm_user? That would be also another race why
>> we'd need to check the unlinking result.
>
> How would the user be able to specify the socket?
>
> The answer is no.
I thought it was possible since they can be dumped.
But yes, I now see that the policy direction check does not
allow the per-socket policies to be specified.
So it'd make more sense to nuke the hashes entirely for
per-socket policies?
^ permalink raw reply
* Re: [PATCH 4/7] flow: delayed deletion of flow cache entries
From: Herbert Xu @ 2010-03-30 12:36 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB1EF7A.7030407@iki.fi>
On Tue, Mar 30, 2010 at 03:32:58PM +0300, Timo Teräs wrote:
> If flow cache regeneration or shrinking is triggered in lookup,
> it would previously free it in place. Now that is deferred. But
> yes, it's more useful after the next patches that call the
> virtual destructor. Should have explained this better.
Any chance you can refactor them so that this comes after the
virtual get/put patch?
That way can evaluate this on its own merit rather than being
a prerequisite for the more important stuff.
Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH RFC] inetpeer: Support ipv6 addresses.
From: Herbert Xu @ 2010-03-30 12:34 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20100329.151539.216335891.davem@davemloft.net>
On Mon, Mar 29, 2010 at 03:15:39PM -0700, David Miller wrote:
>
> Interesting idea, but there is the issue of how to fill in
> new metrics cache entries when these requests come in later.
You just perform an rt lookup as usual. That'll give you the
metrics either from the route cache, or from scratch if we get
a cache miss.
> We'd have to retain a pointer to the routing table fib entry.
> This is because the fib entry states what the initial metric
> values need to be for cached routes.
We can keep the static metrics in the dst/rt entry. For me
shrinking the dst entry isn't such a big deal, but avoiding touching
global state when constructing a new rt entry is the deal-breaker.
But if you're really into dieting then we can have that fib
pointer :)
> So we'd need a pointer to the fib_info in the routing cache entry, and
> this pointer would need to grab a reference to the fib_info. And this
> subsequently leads to the question of what to do for route changes
> (f.e. hold the fib_info around until all the route cache entries drop
> their references and have a dead state in the fib_info struct that can
> be checked, and if we find it dead what can we even do as the route
> we're working with might be cached in a socket or similar)
AFAIK on route changes we always flush the route cache.
> The other option is to relookup the FIB, but we'd then have to
> validate that the route cache entry we're working with matches
> precisely still, and also this lookup alone going to have non-trivial
> cost :-)
>
> It's really depressing how hard it is to untangle the way we have
> things currently setup, isn't it. :-)
Yeah it's like the Gordian Knot :)
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 4/7] flow: delayed deletion of flow cache entries
From: Timo Teräs @ 2010-03-30 12:32 UTC (permalink / raw)
To: Herbert Xu; +Cc: netdev
In-Reply-To: <20100330122217.GF5731@gondor.apana.org.au>
Herbert Xu wrote:
> On Mon, Mar 29, 2010 at 05:12:41PM +0300, Timo Teras wrote:
>> Speed up lookups by freeing flow cache entries later. This is also in
>> preparation to have virtual entry destructor that might do more
>> work.
>
> So how does this speed up lookups exactly?
If flow cache regeneration or shrinking is triggered in lookup,
it would previously free it in place. Now that is deferred. But
yes, it's more useful after the next patches that call the
virtual destructor. Should have explained this better.
Like said in the general description, patches 4-7 go together
and still have some problem cases. But should show where I'm
trying to go.
I'd be interested hear if the idea of patches 4-7 is good
or we could things somehow better.
^ permalink raw reply
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Herbert Xu @ 2010-03-30 12:23 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB1ECBF.50000@iki.fi>
On Tue, Mar 30, 2010 at 03:21:19PM +0300, Timo Teräs wrote:
>
> Hmm... is it possible to modify/delete per-socket policies from
> userland via xfrm_user? That would be also another race why
> we'd need to check the unlinking result.
How would the user be able to specify the socket?
The answer is no.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 4/7] flow: delayed deletion of flow cache entries
From: Herbert Xu @ 2010-03-30 12:22 UTC (permalink / raw)
To: Timo Teras; +Cc: netdev
In-Reply-To: <1269871964-5412-5-git-send-email-timo.teras@iki.fi>
On Mon, Mar 29, 2010 at 05:12:41PM +0300, Timo Teras wrote:
> Speed up lookups by freeing flow cache entries later. This is also in
> preparation to have virtual entry destructor that might do more
> work.
So how does this speed up lookups exactly?
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Timo Teräs @ 2010-03-30 12:21 UTC (permalink / raw)
To: Herbert Xu; +Cc: netdev
In-Reply-To: <20100330121427.GD5731@gondor.apana.org.au>
Herbert Xu wrote:
> On Tue, Mar 30, 2010 at 03:04:01PM +0300, Timo Teräs wrote:
>>> I think it's hashed so socket policies are included in the policy
>> db dumps and counts.
>
> No we use a linked list for that.
Ah, yes. It's just all done together in the __xfrm_link_policy().
Hmm... is it possible to modify/delete per-socket policies from
userland via xfrm_user? That would be also another race why
we'd need to check the unlinking result.
^ permalink raw reply
* Re: [PATCH 2/7] flow: structurize flow cache
From: Herbert Xu @ 2010-03-30 12:15 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB1E862.8090302@iki.fi>
On Tue, Mar 30, 2010 at 03:02:42PM +0300, Timo Teräs wrote:
>
> If we want generic flow cache, we might want to have multiple
> instance in future. It might also make sense to have per-net
> flow cache.
OK that makes sense.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 1/7] xfrm: remove policy lock when accessing policy->walk.dead
From: Herbert Xu @ 2010-03-30 12:14 UTC (permalink / raw)
To: Timo Teräs; +Cc: netdev
In-Reply-To: <4BB1E8B1.4030604@iki.fi>
On Tue, Mar 30, 2010 at 03:04:01PM +0300, Timo Teräs wrote:
>> I think it's hashed so socket policies are included in the policy
> db dumps and counts.
No we use a linked list for that.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 3/7] flow: allocate hash table for online cpus only
From: Herbert Xu @ 2010-03-30 12:12 UTC (permalink / raw)
To: Timo Teras; +Cc: netdev, Rusty Russell
In-Reply-To: <1269871964-5412-4-git-send-email-timo.teras@iki.fi>
On Mon, Mar 29, 2010 at 05:12:40PM +0300, Timo Teras wrote:
> Instead of unconditionally allocating hash table for all possible
> cpu's, allocate it only for online cpu's and release related
> memory if cpu goes down.
>
> Signed-off-by: Timo Teras <timo.teras@iki.fi>
Hmm that's where we started but then Rusty changed it back in 2004:
commit 0a32dc4d8e83c48f7535d66731eb35d1916b39a8
Author: Rusty Russell <rusty@rustcorp.com.au>
Date: Wed Jan 21 18:14:37 2004 -0800
[NET]: Simplify net/flow.c per-cpu handling.
The cpu handling in net/core/flow.c is complex: it tries to allocate
flow cache as each CPU comes up. It might as well allocate them for
each possible CPU at boot.
So I'd like to hear his opinion on changing it back again.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: Network performance - iperf
From: Steve Magnani @ 2010-03-30 12:11 UTC (permalink / raw)
To: Michal Simek, Eric Dumazet
Cc: LKML, John Williams, netdev, Grant Likely, John Linn,
Arnd Bergmann, akpm
In-Reply-To: <4BB1C5A5.5070202@petalogix.com>
Looks like a lot of time spent in tcp_collapse(). I think there is assumption that the memcpys
involved in collapsing lead to better throughput than packet retransmit, which on our platforms
may not be the case. If so is there a way to tune when tcp_collapse gets invoked? I haven't
seen one.
Steve
-----Original Message-----
From: Michal Simek <michal.simek@petalogix.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: LKML <linux-kernel@vger.kernel.org>, John Williams <john.williams@petalogix.com>,
netdev@vger.kernel.org, Grant Likely <grant.likely@secretlab.ca>, John Linn
<John.Linn@xilinx.com>, "Steven J. Magnani" <steve@digidescorp.com>, Arnd Bergmann
<arnd@arndb.de>, akpm@linux-foundation.org
Date: Tue, 30 Mar 2010 11:34:29 +0200
Subject: Re: Network performance - iperf
> Eric Dumazet wrote:
> > Le lundi 29 mars 2010 à 17:27 +0200, Michal Simek a écrit :
> >> Michal Simek wrote:
> >>> Eric Dumazet wrote:
> >>>> Le lundi 29 mars 2010 à 13:33 +0200, Michal Simek a écrit :
> >>>>
> >>>>> Do you have any idea howto improve TCP/UDP performance in
> general?
> >>>>> Or tests which can point me on weak places.
> >>>> Could you post "netstat -s" on your receiver, after fresh boot and
> your
> >>>> iperf session, for 32 MB and 256 MB ram case ?
> >>>>
> >>> I am not sure if is helpful but look below.
> >>>
> >> Sorry I forget to c&p that second part. :-(
> >>
> >
> > Sorry, your netstat is not up2date.
>
> I am afraid that is up2date.
>
> >
> > If you cannot correct it to last version
> > [ net-tools 1.60 , netstat 1.42 ], please send
> >
> > cat /proc/net/snmp
> > cat /proc/net/netstat
>
> There is small buffer for parsing /proc/net/netstat.
> There is necessary to extend buffer size because one line is greater
> than 1024 chars.
>
> ~ # head -n 1 /proc/net/netstat
> TcpExt: SyncookiesSent SyncookiesRecv SyncookiesFailed EmbryonicRsts
> PruneCalled RcvPruned OfoPruned OutOfWindowIcmps LockDroppedIcmps
> ArpFilter TW TWRecycled TWKilled PAWSPassive PAWSActive PAWSEstab
> DelayedACKs DelayedACKLocked DelayedACKLost ListenOverflows ListenDrops
> TCPPrequeued TCPDirectCopyFromBacklog TCPDirectCopyFromPrequeue
> TCPPrequeueDropped TCPHPHits TCPHPHitsToUser TCPPureAcks TCPHPAcks
> TCPRenoRecovery TCPSackRecovery TCPSACKReneging TCPFACKReorder
> TCPSACKReorder TCPRenoReorder TCPTSReorder TCPFullUndo TCPPartialUndo
> TCPDSACKUndo TCPLossUndo TCPLoss TCPLostRetransmit TCPRenoFailures
> TCPSackFailures TCPLossFailures TCPFastRetrans TCPForwardRetrans
> TCPSlowStartRetrans TCPTimeouts TCPRenoRecoveryFail TCPSackRecoveryFail
> TCPSchedulerFailed TCPRcvCollapsed TCPDSACKOldSent TCPDSACKOfoSent
> TCPDSACKRecv TCPDSACKOfoRecv TCPAbortOnSyn TCPAbortOnData
> TCPAbortOnClose TCPAbortOnMemory TCPAbortOnTimeout TCPAbortOnLinger
> TCPAbortFailed TCPMemoryPressures TCPSACKDiscard TCPDSACKIgnoredOld
> TCPDSACKIgnoredNoUndo TCPSpuriousRTOs TCPMD5NotFound TCPMD5Unexpected
> TCPSackShifted TCPSackMerged TCPSackShiftFallback TCPBacklogDrop
> TCPMinTTLDrop
>
> Look at attached patch.
> And updated results are below.
>
> Thanks,
> Michal
>
>
>
>
>
> 256M
>
> ~ # iperf -s
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> [ 6] local 192.168.0.10 port 5001 connected with 192.168.0.101 port
> 33261
> [ ID] Interval Transfer Bandwidth
> [ 6] 0.0-50.2 sec 22.9 MBytes 3.83 Mbits/sec
> ~ # ./netstat -s
> Ip:
> 16618 total packets received
> 0 forwarded
> 0 incoming packets discarded
> 16618 incoming packets delivered
> 6490 requests sent out
> Icmp:
> 0 ICMP messages received
> 0 input ICMP message failed.
> ICMP input histogram:
> 0 ICMP messages sent
> 0 ICMP messages failed
> ICMP output histogram:
> Tcp:
> 0 active connections openings
> 1 passive connection openings
> 0 failed connection attempts
> 0 connection resets received
> 0 connections established
> 16618 segments received
> 6490 segments send out
> 0 segments retransmited
> 0 bad segments received.
> 0 resets sent
> Udp:
> 0 packets received
> 0 packets to unknown port received.
> 0 packet receive errors
> 0 packets sent
> RcvbufErrors: 0
> SndbufErrors: 0
> UdpLite:
> InDatagrams: 0
> NoPorts: 0
> InErrors: 0
> OutDatagrams: 0
> RcvbufErrors: 0
> SndbufErrors: 0
> TcpExt:
> 2233 packets pruned from receive queue because of socket buffer
> overrun
> ArpFilter: 0
> 1 delayed acks sent
> 5519 packets header predicted
> TCPPureAcks: 2
> TCPHPAcks: 0
> TCPRenoRecovery: 0
> TCPSackRecovery: 0
> TCPSACKReneging: 0
> TCPFACKReorder: 0
> TCPSACKReorder: 0
> TCPRenoReorder: 0
> TCPTSReorder: 0
> TCPFullUndo: 0
> TCPPartialUndo: 0
> TCPDSACKUndo: 0
> TCPLossUndo: 0
> TCPLoss: 0
> TCPLostRetransmit: 0
> TCPRenoFailures: 0
> TCPSackFailures: 0
> TCPLossFailures: 0
> TCPFastRetrans: 0
> TCPForwardRetrans: 0
> TCPSlowStartRetrans: 0
> TCPTimeouts: 0
> TCPRenoRecoveryFail: 0
> TCPSackRecoveryFail: 0
> TCPSchedulerFailed: 0
> TCPRcvCollapsed: 207654
> TCPDSACKOldSent: 0
> TCPDSACKOfoSent: 0
> TCPDSACKRecv: 0
> TCPDSACKOfoRecv: 0
> TCPAbortOnSyn: 0
> TCPAbortOnData: 0
> TCPAbortOnClose: 0
> TCPAbortOnMemory: 0
> TCPAbortOnTimeout: 0
> TCPAbortOnLinger: 0
> TCPAbortFailed: 0
> TCPMemoryPressures: 0
> TCPSACKDiscard: 0
> TCPDSACKIgnoredOld: 0
> TCPDSACKIgnoredNoUndo: 0
> TCPSpuriousRTOs: 0
> TCPMD5NotFound: 0
> TCPMD5Unexpected: 0
> TCPSackShifted: 0
> TCPSackMerged: 0
> TCPSackShiftFallback: 0
> TCPBacklogDrop: 0
> TCPMinTTLDrop: 0
> IpExt:
> InNoRoutes: 0
> InTruncatedPkts: 0
> InMcastPkts: 0
> OutMcastPkts: 0
> InBcastPkts: 0
> OutBcastPkts: 0
> InOctets: 24915880
> OutOctets: 337488
> InMcastOctets: 0
> OutMcastOctets: 0
> InBcastOctets: 0
> OutBcastOctets: 0
> ~ # ./netstat --version
> net-tools 1.60
> netstat 1.42 (2001-04-15)
> Fred Baumgarten, Alan Cox, Bernd Eckenfels, Phil Blundell, Tuan Hoang
> and others
> +NEW_ADDRT +RTF_IRTT +RTF_REJECT +FW_MASQUERADE -I18N
> AF: (inet) +UNIX +INET -INET6 -IPX -AX25 -NETROM -X25 -ATALK -ECONET
> -ROSE
> HW: +ETHER -ARC +SLIP +PPP -TUNNEL -TR -AX25 -NETROM -X25 -FR -ROSE
> -ASH -SIT -FDDI -HIPPI -HDLC/LAPB
> ~ # head -n 1 /proc/meminfo
> MemTotal: 257108 kB
>
>
>
> 32MB
>
> ~ # head -n 1 /proc/meminfo
> MemTotal: 29920 kB
> ~ # iperf -s
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 85.3 KByte (default)
> ------------------------------------------------------------
> [ 6] local 192.168.0.10 port 5001 connected with 192.168.0.101 port
> 50088
> [ ID] Interval Transfer Bandwidth
> [ 6] 0.0-50.0 sec 109 MBytes 18.3 Mbits/sec
> ~ # ./netstat -s
> Ip:
> 79040 total packets received
> 0 forwarded
> 0 incoming packets discarded
> 79040 incoming packets delivered
> 29655 requests sent out
> Icmp:
> 0 ICMP messages received
> 0 input ICMP message failed.
> ICMP input histogram:
> 0 ICMP messages sent
> 0 ICMP messages failed
> ICMP output histogram:
> Tcp:
> 0 active connections openings
> 1 passive connection openings
> 0 failed connection attempts
> 0 connection resets received
> 0 connections established
> 79040 segments received
> 29655 segments send out
> 0 segments retransmited
> 0 bad segments received.
> 0 resets sent
> Udp:
> 0 packets received
> 0 packets to unknown port received.
> 0 packet receive errors
> 0 packets sent
> RcvbufErrors: 0
> SndbufErrors: 0
> UdpLite:
> InDatagrams: 0
> NoPorts: 0
> InErrors: 0
> OutDatagrams: 0
> RcvbufErrors: 0
> SndbufErrors: 0
> TcpExt:
> 9773 packets pruned from receive queue because of socket buffer
> overrun
> ArpFilter: 0
> 1 delayed acks sent
> 101 packets directly queued to recvmsg prequeue.
> 558928 packets directly received from prequeue
> 33274 packets header predicted
> 378 packets header predicted and directly queued to user
> TCPPureAcks: 2
> TCPHPAcks: 0
> TCPRenoRecovery: 0
> TCPSackRecovery: 0
> TCPSACKReneging: 0
> TCPFACKReorder: 0
> TCPSACKReorder: 0
> TCPRenoReorder: 0
> TCPTSReorder: 0
> TCPFullUndo: 0
> TCPPartialUndo: 0
> TCPDSACKUndo: 0
> TCPLossUndo: 0
> TCPLoss: 0
> TCPLostRetransmit: 0
> TCPRenoFailures: 0
> TCPSackFailures: 0
> TCPLossFailures: 0
> TCPFastRetrans: 0
> TCPForwardRetrans: 0
> TCPSlowStartRetrans: 0
> TCPTimeouts: 0
> TCPRenoRecoveryFail: 0
> TCPSackRecoveryFail: 0
> TCPSchedulerFailed: 0
> TCPRcvCollapsed: 120195
> TCPDSACKOldSent: 0
> TCPDSACKOfoSent: 0
> TCPDSACKRecv: 0
> TCPDSACKOfoRecv: 0
> TCPAbortOnSyn: 0
> TCPAbortOnData: 0
> TCPAbortOnClose: 0
> TCPAbortOnMemory: 0
> TCPAbortOnTimeout: 0
> TCPAbortOnLinger: 0
> TCPAbortFailed: 0
> TCPMemoryPressures: 0
> TCPSACKDiscard: 0
> TCPDSACKIgnoredOld: 0
> TCPDSACKIgnoredNoUndo: 0
> TCPSpuriousRTOs: 0
> TCPMD5NotFound: 0
> TCPMD5Unexpected: 0
> TCPSackShifted: 0
> TCPSackMerged: 0
> TCPSackShiftFallback: 0
> TCPBacklogDrop: 0
> TCPMinTTLDrop: 0
> IpExt:
> InNoRoutes: 0
> InTruncatedPkts: 0
> InMcastPkts: 0
> OutMcastPkts: 0
> InBcastPkts: 0
> OutBcastPkts: 0
> InOctets: 118232864
> OutOctets: 1542068
> InMcastOctets: 0
> OutMcastOctets: 0
> InBcastOctets: 0
> OutBcastOctets: 0
> ~ #
>
>
>
>
>
>
>
>
>
>
> --
> Michal Simek, Ing. (M.Eng)
> PetaLogix - Linux Solutions for a Reconfigurable World
> w: www.petalogix.com p: +61-7-30090663,+42-0-721842854 f:
> +61-7-30090663
^ permalink raw reply
* Re: debugging kernel during packet drops
From: Jan Engelhardt @ 2010-03-30 12:06 UTC (permalink / raw)
To: Patrick McHardy; +Cc: Eric Dumazet, Jorrit Kronjee, netfilter-devel, netdev
In-Reply-To: <4BAB3C8B.3030104@trash.net>
On Thursday 2010-03-25 11:35, Patrick McHardy wrote:
>Eric Dumazet wrote:
>> Here is patch I cooked for xt_hashlimit (on top of net-next-2.6) to make
>> it use RCU and scale better in your case (allowing several concurrent
>> cpus once RPS is activated), but also on more general cases.
>>
>> [PATCH] xt_hashlimit: RCU conversion
>>
>> xt_hashlimit uses a central lock per hash table and suffers from
>> contention on some workloads.
>>
>> After RCU conversion, central lock is only used when a writer wants to
>> add or delete an entry. For 'readers', updating an existing entry, they
>> use an individual lock per entry.
>
>This clashes with some recent cleanups in nf-next-2.6.git. I'm
>also expecting a patch from Jan to remove the old v0 revision
>very soon (probably today). Please rediff once I've pushed that out.
One 12-series request has been sitting there for a while. Was there
something not in order with it?
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox