From: Eric Dumazet <eric.dumazet@gmail.com>
To: Herbert Xu <herbert@gondor.apana.org.au>
Cc: David Miller <davem@davemloft.net>,
netdev <netdev@vger.kernel.org>, Jesse Gross <jesse@nicira.com>
Subject: [RFC] GRO scalability
Date: Fri, 05 Oct 2012 16:52:27 +0200 [thread overview]
Message-ID: <1349448747.21172.113.camel@edumazet-glaptop> (raw)
In-Reply-To: <CAEP_g=_nSb-ite51PM-E8SY53yOPiZs8N3gDrYNc0L4OU2Ht=A@mail.gmail.com>
Current GRO cell is somewhat limited :
- It uses a single list (napi->gro_list) of pending skbs
- This list has a limit of 8 skbs (MAX_GRO_SKBS)
- Workloads with lot of concurrent flows have small GRO hit rate but
pay high overhead (in inet_gro_receive())
- Increasing MAX_GRO_SKBS is not an option, because GRO
overhead becomes too high.
- Packets can stay a long time held in GRO cell (there is
no flush if napi never completes on a stressed cpu)
Some elephant flows can stall interactive ones (if we receive
flood of non TCP frames, we dont flush tcp packets waiting in
gro_list)
What we could do :
1) Use a hash to avoid expensive gro_list management and allow
much more concurrent flows.
Use skb_get_rxhash(skb) to compute rxhash
If l4_rxhash not set -> not a GRO candidate.
If l4_rxhash set, use a hash lookup to immediately finds a 'same flow'
candidates.
(tcp stack could eventually use rxhash instead of its custom hash
computation ...)
2) Use a LRU list to eventually be able to 'flush' too old packets,
even if the napi never completes. Each time we process a new packet,
being a GRO candidate or not, we increment a napi->sequence, and we
flush the oldest packet in gro_lru_list if its own sequence is too
old.
That would give a latency guarantee.
next prev parent reply other threads:[~2012-10-05 14:52 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-27 12:48 [PATCH net-next 3/3] ipv4: gre: add GRO capability Eric Dumazet
2012-09-27 17:52 ` Jesse Gross
2012-09-27 18:08 ` Eric Dumazet
2012-09-27 18:19 ` Eric Dumazet
2012-09-27 22:03 ` Jesse Gross
2012-09-28 14:04 ` Eric Dumazet
2012-10-01 20:56 ` Jesse Gross
2012-10-05 14:52 ` Eric Dumazet [this message]
2012-10-05 18:16 ` [RFC] GRO scalability Rick Jones
2012-10-05 19:00 ` Eric Dumazet
2012-10-05 19:35 ` Rick Jones
2012-10-05 20:06 ` Eric Dumazet
2012-10-08 16:40 ` Rick Jones
2012-10-08 16:59 ` Eric Dumazet
2012-10-08 17:49 ` Rick Jones
2012-10-08 17:55 ` Eric Dumazet
2012-10-08 17:56 ` Eric Dumazet
2012-10-08 18:58 ` [RFC] napi: limit GRO latency Stephen Hemminger
2012-10-08 19:10 ` David Miller
2012-10-08 19:12 ` Stephen Hemminger
2012-10-08 19:30 ` Eric Dumazet
2012-10-08 19:40 ` Stephen Hemminger
2012-10-08 19:46 ` Eric Dumazet
2012-10-08 19:21 ` Eric Dumazet
2012-10-08 18:21 ` [RFC] GRO scalability Rick Jones
2012-10-08 18:28 ` Eric Dumazet
2012-10-06 4:11 ` Herbert Xu
2012-10-06 5:08 ` Eric Dumazet
2012-10-06 5:14 ` Herbert Xu
2012-10-06 6:22 ` Eric Dumazet
2012-10-06 7:00 ` Eric Dumazet
2012-10-06 10:56 ` Herbert Xu
2012-10-06 18:08 ` [PATCH] net: gro: selective flush of packets Eric Dumazet
2012-10-07 0:32 ` Herbert Xu
2012-10-07 5:29 ` Eric Dumazet
2012-10-08 7:39 ` Eric Dumazet
2012-10-08 16:42 ` Rick Jones
2012-10-08 17:10 ` Eric Dumazet
2012-10-08 18:52 ` David Miller
2012-09-27 22:03 ` [PATCH net-next 3/3] ipv4: gre: add GRO capability Jesse Gross
2012-10-01 21:04 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1349448747.21172.113.camel@edumazet-glaptop \
--to=eric.dumazet@gmail.com \
--cc=davem@davemloft.net \
--cc=herbert@gondor.apana.org.au \
--cc=jesse@nicira.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox