[PATCH v3 03/12] NFSv4/flexfiles: cacheline align the per-stripe layoutstats

[Mike Snitzer <snitzer@kernel.org>]

From: Mike Snitzer <snitzer@hammerspace.com>

With a lock per stripe, stripes no longer share a lock, but they still
share cachelines.  struct nfs4_ff_layout_ds_stripe is 296 bytes with
8-byte alignment.  So in mirror->dss[], each stripe starts 40 bytes
further into a cacheline than the one before.  Taking stripe i's lock
invalidates the line with stripe i+1's lock and counters.

Cacheline align the lock.  The guarded members then start on a line
boundary.  The struct's alignment of 64 also rounds sizeof() up to a
multiple of 64.  So each stripe in the array starts on its own line.

The stripe grows from 296 to 384 bytes, 2816 bytes for a 32-stripe
layout.  The array base is aligned too: kvmalloc() gets it from a
kmalloc bucket of 512 bytes or more, or from vmalloc().

Signed-off-by: Mike Snitzer <snitzer@hammerspace.com>
---
 fs/nfs/flexfilelayout/flexfilelayout.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/flexfilelayout/flexfilelayout.h b/fs/nfs/flexfilelayout/flexfilelayout.h
index 0e94bc289746c..9ce09e02f1ad9 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.h
+++ b/fs/nfs/flexfilelayout/flexfilelayout.h
@@ -14,6 +14,7 @@
 #define FF_FLAGS_NO_IO_THRU_MDS  2
 #define FF_FLAGS_NO_READ_IO      4
 
+#include <linux/cache.h>
 #include <linux/refcount.h>
 #include "../pnfs.h"
 
@@ -87,7 +88,7 @@ struct nfs4_ff_layout_ds_stripe {
 	const struct cred __rcu		*rw_cred;
 	struct nfs_file_localio		nfl;
 	/* Protects read_stat, write_stat and start_time below */
-	spinlock_t			lock;
+	spinlock_t			lock ____cacheline_aligned_in_smp;
 	struct nfs4_ff_layoutstat	read_stat;
 	struct nfs4_ff_layoutstat	write_stat;
 	ktime_t				start_time;
-- 
2.52.0