Ethernet Bridge development
 help / color / mirror / Atom feed
From: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
To: netdev@vger.kernel.org
Cc: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>,
	roopa@cumulusnetworks.com, bridge@lists.linux-foundation.org,
	davem@davemloft.net
Subject: [Bridge] [PATCH net-next 3/4] bridge: move write-heavy fdb members in their own cache line
Date: Sat,  4 Feb 2017 18:05:08 +0100	[thread overview]
Message-ID: <1486227909-18319-4-git-send-email-nikolay@cumulusnetworks.com> (raw)
In-Reply-To: <1486227909-18319-1-git-send-email-nikolay@cumulusnetworks.com>

Fdb's used and updated fields are written to on every packet forward and
packet receive respectively. Thus if we are receiving packets from a
particular fdb, they'll cause false-sharing with everyone who has looked
it up (even if it didn't match, since mac/vid share cache line!). The
"used" field is even worse since it is updated on every packet forward
to that fdb, thus the standard config where X ports use a single gateway
results in 100% fdb false-sharing. Note that this patch does not prevent
the last scenario, but it makes it better for other bridge participants
which are not using that fdb (and are only doing lookups over it).
The point is with this move we make sure that only communicating parties
get the false-sharing, in a later patch we'll show how to avoid that too.

Signed-off-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com>
---
 net/bridge/br_private.h | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h
index 47fd64bf5022..1cbbf63a5ef7 100644
--- a/net/bridge/br_private.h
+++ b/net/bridge/br_private.h
@@ -160,19 +160,21 @@ struct net_bridge_vlan_group {
 	u16				pvid;
 };
 
-struct net_bridge_fdb_entry
-{
+struct net_bridge_fdb_entry {
 	struct hlist_node		hlist;
 	struct net_bridge_port		*dst;
 
-	unsigned long			updated;
-	unsigned long			used;
 	mac_addr			addr;
 	__u16				vlan_id;
 	unsigned char			is_local:1,
 					is_static:1,
 					added_by_user:1,
 					added_by_external_learn:1;
+
+	/* write-heavy members should not affect lookups */
+	unsigned long			updated ____cacheline_aligned_in_smp;
+	unsigned long			used;
+
 	struct rcu_head			rcu;
 };
 
-- 
2.1.4


  parent reply	other threads:[~2017-02-04 17:05 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-02-04 17:05 [Bridge] [PATCH net-next 0/4] bridge: improve cache utilization Nikolay Aleksandrov
2017-02-04 17:05 ` [Bridge] [PATCH net-next 1/4] bridge: modify bridge and port to have often accessed fields in one cache line Nikolay Aleksandrov
2017-02-04 17:05 ` [Bridge] [PATCH net-next 2/4] bridge: move to workqueue gc Nikolay Aleksandrov
2017-02-04 17:05 ` Nikolay Aleksandrov [this message]
2017-02-04 17:05 ` [Bridge] [PATCH net-next 4/4] bridge: fdb: write to used and updated at most once per jiffy Nikolay Aleksandrov
2017-02-07 16:46   ` [Bridge] [PATCH net-next] bridge: avoid unnecessary read of jiffies Stephen Hemminger
2017-02-07 16:56     ` Nikolay Aleksandrov
2017-02-07 19:16     ` David Miller
2017-02-04 21:46 ` [Bridge] [PATCH net-next 0/4] bridge: improve cache utilization Stephen Hemminger
2017-02-04 21:58   ` Nikolay Aleksandrov
2017-02-07  3:53 ` David Miller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1486227909-18319-4-git-send-email-nikolay@cumulusnetworks.com \
    --to=nikolay@cumulusnetworks.com \
    --cc=bridge@lists.linux-foundation.org \
    --cc=davem@davemloft.net \
    --cc=netdev@vger.kernel.org \
    --cc=roopa@cumulusnetworks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox