netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Xin Long <lucien.xin@gmail.com>
To: network dev <netdev@vger.kernel.org>, linux-sctp@vger.kernel.org
Cc: mleitner@redhat.com, vyasevic@redhat.com, daniel@iogearbox.net,
	davem@davemloft.net
Subject: [PATCH net-next 0/5] sctp: use transport hashtable to replace association's with rhashtable
Date: Wed, 30 Dec 2015 23:50:45 +0800	[thread overview]
Message-ID: <cover.1451490447.git.lucien.xin@gmail.com> (raw)

for telecom center, the usual case is that a server is connected by thousands
of clients. but if the server with only one enpoint(udp style) use the same
sport and dport to communicate with every clients, and every assoc in server
will be hashed in the same chain of global assoc hashtable due to currently we
choose dport and sport as the hash key.

when a packet is received, sctp_rcv try to find the assoc with sport and dport,
since that chain is too long to find it fast, it make the performance turn to
very low, some test data is as follow:

in server:
$./ss [start a udp style server there]
in client:
$./cc [start 2500 sockets to connect server with same port and different ip,
       and use one of them to send data to server]

===== test on net-next
-- perf top
server:
  55.73%  [kernel]             [k] sctp_assoc_is_match
   6.80%  [kernel]             [k] sctp_assoc_lookup_paddr
   4.81%  [kernel]             [k] sctp_v4_cmp_addr
   3.12%  [kernel]             [k] _raw_spin_unlock_irqrestore
   1.94%  [kernel]             [k] sctp_cmp_addr_exact

client:
  46.01%  [kernel]                    [k] sctp_endpoint_lookup_assoc
   5.55%  libc-2.17.so                [.] __libc_calloc
   5.39%  libc-2.17.so                [.] _int_free
   3.92%  libc-2.17.so                [.] _int_malloc
   3.23%  [kernel]                    [k] __memset

-- spent time
time is 487s, send pkt is 10000000

we need to change the way to calculate the hash key, to use lport +
rport + paddr as the hash key can avoid this issue.

besides, this patchset will use transport hashtable to replace
association hashtable to lookup with rhashtable api. get transport
first then get association by t->asoc. and also it will make tcp
style work better.

===== test with this patchset:
-- perf top
server:
  15.98%  [kernel]                 [k] _raw_spin_unlock_irqrestore
   9.92%  [kernel]                 [k] __pv_queued_spin_lock_slowpath
   7.22%  [kernel]                 [k] copy_user_generic_string
   2.38%  libpthread-2.17.so       [.] __recvmsg_nocancel
   1.88%  [kernel]                 [k] sctp_recvmsg

client:
  11.90%  [kernel]                   [k] sctp_hash_cmp
   8.52%  [kernel]                   [k] rht_deferred_worker
   4.94%  [kernel]                   [k] __pv_queued_spin_lock_slowpath
   3.95%  [kernel]                   [k] sctp_bind_addr_match
   2.49%  [kernel]                   [k] __memset

-- spent time
time is 22s, send pkt is 10000000

Xin Long (5):
  sctp: add the rhashtable apis for sctp global transport hashtable
  sctp: apply rhashtable api to send/recv path
  sctp: apply rhashtable api to sctp procfs
  sctp: drop the old assoc hashtable of sctp
  sctp: remove the local_bh_disable/enable in sctp_endpoint_lookup_assoc

 include/net/sctp/sctp.h    |  32 ++---
 include/net/sctp/structs.h |  10 +-
 net/sctp/associola.c       |   5 +
 net/sctp/endpointola.c     |  52 ++------
 net/sctp/input.c           | 187 +++++++++++++++++----------
 net/sctp/proc.c            | 316 +++++++++++++++++++++++++--------------------
 net/sctp/protocol.c        |  36 ++----
 net/sctp/sm_sideeffect.c   |   2 -
 net/sctp/socket.c          |   6 +-
 9 files changed, 331 insertions(+), 315 deletions(-)

-- 
2.1.0

             reply	other threads:[~2015-12-30 15:50 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-12-30 15:50 Xin Long [this message]
2015-12-30 15:50 ` [PATCH net-next 1/5] sctp: add the rhashtable apis for sctp global transport hashtable Xin Long
2015-12-30 15:50   ` [PATCH net-next 2/5] sctp: apply rhashtable api to send/recv path Xin Long
2015-12-30 15:50     ` [PATCH net-next 3/5] sctp: apply rhashtable api to sctp procfs Xin Long
2015-12-30 15:50       ` [PATCH net-next 4/5] sctp: drop the old assoc hashtable of sctp Xin Long
2015-12-30 15:50         ` [PATCH net-next 5/5] sctp: remove the local_bh_disable/enable in sctp_endpoint_lookup_assoc Xin Long
2016-01-05 19:07     ` [PATCH net-next 2/5] sctp: apply rhashtable api to send/recv path Vlad Yasevich
2016-01-06 16:18       ` Xin Long
2016-01-06 17:42       ` mleitner
2016-01-11 15:00         ` Vlad Yasevich
2015-12-30 16:57   ` [PATCH net-next 1/5] sctp: add the rhashtable apis for sctp global transport hashtable Eric Dumazet
2015-12-30 17:50     ` David Miller
2016-01-11  9:32       ` Herbert Xu
2016-01-11 16:33         ` Marcelo Ricardo Leitner
2016-01-11 18:08           ` Vlad Yasevich
2016-01-11 18:19             ` Marcelo Ricardo Leitner
2015-12-30 17:41   ` Marcelo Ricardo Leitner
2016-01-05 10:10     ` Xin Long
2016-01-11  9:22       ` Herbert Xu
2016-01-05 18:38   ` Vlad Yasevich
2016-01-06 17:01     ` Xin Long
2016-01-06 18:19       ` Marcelo Ricardo Leitner
2016-01-07 17:23         ` Marcelo Ricardo Leitner
2016-01-07 20:28       ` Vlad Yasevich
2016-01-11  9:30   ` Herbert Xu
2016-01-11 16:00     ` mleitner
2016-01-11 17:20       ` Vlad Yasevich
2016-01-11 18:09         ` mleitner
2016-01-11 21:35           ` David Miller
2016-01-11 21:31         ` David Miller
2015-12-30 17:19 ` [PATCH net-next 0/5] sctp: use transport hashtable to replace association's with rhashtable Eric Dumazet
2015-12-30 17:32   ` Marcelo Ricardo Leitner
2015-12-30 19:11     ` Eric Dumazet
2015-12-30 20:44       ` David Miller
2015-12-30 21:57         ` Eric Dumazet
2015-12-30 22:29           ` Marcelo Ricardo Leitner
2015-12-30 17:52   ` David Miller
2015-12-30 19:03     ` Eric Dumazet
2015-12-30 20:40       ` David Miller
2016-01-04 22:30 ` David Miller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1451490447.git.lucien.xin@gmail.com \
    --to=lucien.xin@gmail.com \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=linux-sctp@vger.kernel.org \
    --cc=mleitner@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=vyasevic@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).