[PATCH] reorder struct files_struct

[Eric Dumazet <dada1@cosmosbay.com>]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

Hi

Browsing (and using) the excellent RCU infrastructure for files that was 
adopted for 2.6.14-rc1, I noticed that the file_lock spinlock sit close to 
mostly read fields of 'struct files_struct'

In SMP (and NUMA) environnements, each time a thread wants to open or close a 
file, it has to acquire the spinlock, thus invalidating the cache line 
containing this spinlock on other CPUS. So other threads doing 
read()/write()/... calls that use RCU to access the file table are going to 
ask further memory (possibly NUMA) transactions to read again this memory line.

Please consider applying this patch. It moves the spinlock to another cache 
line, so that concurrent threads can share the cache line containing 'count' 
and 'fdt' fields.


Thank you

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>



[-- Attachment #2: reorder_files_struct --]
[-- Type: text/plain, Size: 660 bytes --]

--- linux-2.6.14-rc1/include/linux/file.h	2005-09-13 05:12:09.000000000 +0200
+++ linux-2.6.14-rc1-ed/include/linux/file.h	2005-09-15 01:09:13.000000000 +0200
@@ -34,12 +34,12 @@
  */
 struct files_struct {
         atomic_t count;
-        spinlock_t file_lock;     /* Protects all the below members.  Nests inside tsk->alloc_lock */
 	struct fdtable *fdt;
 	struct fdtable fdtab;
         fd_set close_on_exec_init;
         fd_set open_fds_init;
         struct file * fd_array[NR_OPEN_DEFAULT];
+	spinlock_t file_lock;     /* Protects concurrent writers.  Nests inside tsk->alloc_lock */
 };
 
 #define files_fdtable(files) (rcu_dereference((files)->fdt))