From: Eric Dumazet <eric.dumazet@gmail.com>
To: Changli Gao <xiaosuo@gmail.com>
Cc: Benny Amorsen <benny+usenet@amorsen.dk>,
zhigang gong <zhigang.gong@gmail.com>,
netdev@vger.kernel.org
Subject: Re: Strange packet drops with heavy firewalling
Date: Tue, 13 Apr 2010 07:56:26 +0200 [thread overview]
Message-ID: <1271138186.16881.168.camel@edumazet-laptop> (raw)
In-Reply-To: <u2y412e6f7f1004121618p6d6eff30q8a45a03faa59a912@mail.gmail.com>
Le mardi 13 avril 2010 à 07:18 +0800, Changli Gao a écrit :
> On Tue, Apr 13, 2010 at 1:06 AM, Benny Amorsen <benny+usenet@amorsen.dk> wrote:
> >
> > 99: 24 1306226 3 2 PCI-MSI-edge eth1-tx-0
> > 100: 15735 1648774 3 7 PCI-MSI-edge eth1-tx-1
> > 101: 8 11 9 1083022 PCI-MSI-edge eth1-tx-2
> > 102: 0 0 0 0 PCI-MSI-edge eth1-tx-3
> > 103: 18 15 6131 1095383 PCI-MSI-edge eth1-rx-0
> > 104: 217 32 46544 1335325 PCI-MSI-edge eth1-rx-1
> > 105: 154 1305595 218 16 PCI-MSI-edge eth1-rx-2
> > 106: 17 16 8229 1467509 PCI-MSI-edge eth1-rx-3
> > 107: 0 0 1 0 PCI-MSI-edge eth1
> > 108: 2 14 15 1003053 PCI-MSI-edge eth0-tx-0
> > 109: 8226 1668924 478 487 PCI-MSI-edge eth0-tx-1
> > 110: 3 1188874 17 12 PCI-MSI-edge eth0-tx-2
> > 111: 0 0 0 0 PCI-MSI-edge eth0-tx-3
> > 112: 203 185 5324 1015263 PCI-MSI-edge eth0-rx-0
> > 113: 4141 1600793 153 159 PCI-MSI-edge eth0-rx-1
> > 114: 16242 1210108 436 3124 PCI-MSI-edge eth0-rx-2
> > 115: 267 4173 19471 1321252 PCI-MSI-edge eth0-rx-3
> > 116: 0 1 0 0 PCI-MSI-edge eth0
> >
> >
> > irqbalanced seems to have picked CPU1 and CPU3 for all the interrupts,
> > which to my mind should cause the same problem as before (where CPU1 and
> > CPU3 was handling all packets). Yet the box clearly works much better
> > than before.
>
> irqbalanced? I don't think it can work properly. Try RPS in netdev and
> linux-next tree, and if cpu load isn't even, try this patch:
> http://patchwork.ozlabs.org/patch/49915/ .
>
>
Dont try RPS on multiqueue devices !
If number of queues matches CPU numbers, it brings nothing but extra
latencies !
Benny, I am not sure your irqbalance is up2date with multiqueue devices,
you might need to disable it and manually irqaffine each interrupt
echo 01 >/proc/irq/100/smp_affinity
echo 02 >/proc/irq/101/smp_affinity
echo 04 >/proc/irq/102/smp_affinity
echo 08 >/proc/irq/103/smp_affinity
echo 10 >/proc/irq/104/smp_affinity
echo 20 >/proc/irq/105/smp_affinity
echo 40 >/proc/irq/106/smp_affinity
echo 80 >/proc/irq/107/smp_affinity
echo 01 >/proc/irq/108/smp_affinity
echo 02 >/proc/irq/109/smp_affinity
echo 04 >/proc/irq/110/smp_affinity
echo 08 >/proc/irq/111/smp_affinity
echo 10 >/proc/irq/112/smp_affinity
echo 20 >/proc/irq/113/smp_affinity
echo 40 >/proc/irq/114/smp_affinity
echo 80 >/proc/irq/115/smp_affinity
next prev parent reply other threads:[~2010-04-13 5:56 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-04-09 9:56 Strange packet drops with heavy firewalling Benny Amorsen
2010-04-09 11:47 ` Eric Dumazet
2010-04-09 12:33 ` Benny Amorsen
2010-04-09 13:29 ` Eric Dumazet
2010-04-12 6:20 ` Benny Amorsen
[not found] ` <q2v40c9f5b21004120116p766df82dj88c6af4e4cad55f@mail.gmail.com>
2010-04-12 14:44 ` Benny Lyne Amorsen
[not found] ` <p2x40c9f5b21004120833jd7a749cak6ea69cebd28f8352@mail.gmail.com>
2010-04-12 17:06 ` Benny Amorsen
2010-04-12 23:18 ` Changli Gao
2010-04-13 5:56 ` Eric Dumazet [this message]
2010-04-13 7:56 ` Benny Amorsen
2010-04-15 13:23 ` Benny Amorsen
2010-04-15 13:42 ` Eric Dumazet
2010-04-13 12:33 ` Paweł Staszewski
2010-04-13 12:53 ` Eric Dumazet
2010-04-13 13:39 ` Paweł Staszewski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1271138186.16881.168.camel@edumazet-laptop \
--to=eric.dumazet@gmail.com \
--cc=benny+usenet@amorsen.dk \
--cc=netdev@vger.kernel.org \
--cc=xiaosuo@gmail.com \
--cc=zhigang.gong@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox