Re: [PATCH] reorder struct files_struct

[Dipankar Sarma <dipankar@in.ibm.com>]

On Thu, Sep 15, 2005 at 12:42:03AM +0200, Eric Dumazet wrote:
> Dipankar Sarma a écrit :
> >On Wed, Sep 14, 2005 at 11:17:42PM +0200, Eric Dumazet wrote:
> >
> >>--- linux-2.6.14-rc1/include/linux/file.h	2005-09-13 
> >>05:12:09.000000000 +0200
> >>+++ linux-2.6.14-rc1-ed/include/linux/file.h	2005-09-15 
> >>01:09:13.000000000 +0200
> >>@@ -34,12 +34,12 @@
> >> */
> >>struct files_struct {
> >>        atomic_t count;
> >>-        spinlock_t file_lock;     /* Protects all the below members.  
> >>Nests inside tsk->alloc_lock */
> >>	struct fdtable *fdt;
> >>	struct fdtable fdtab;
> >>        fd_set close_on_exec_init;
> >>        fd_set open_fds_init;
> >>        struct file * fd_array[NR_OPEN_DEFAULT];
> >>+	spinlock_t file_lock;     /* Protects concurrent writers.  Nests 
> >>inside tsk->alloc_lock */
> >>};
> >>
> >>#define files_fdtable(files) (rcu_dereference((files)->fdt))
> >
> >
> >For most apps without too many open fds, the embedded fd_sets
> >are going to be used. Wouldn't that mean that open()/close() will
> >invalidate the cache line containing fdt, fdtab by updating
> >the fd_sets ? If so, you optimization really doesn't help.
> >
> 
> If the embedded struct fdtable is used, then the only touched field is 
> 'next_fd', so we could also move this field at the end of 'struct fdtable'
> 

Not just embedded fdtable, but also the embedded fdsets. I would expect
count, fdt, fdtab and the fdsets to fit into one cache line in 
some archs.


> But I wonder if 'next_fd' really has to be in 'struct fdtable', maybe it 
> could be moved to 'struct files_struct' close to file_lock ?

next_fd has to be in struct fdtable. It needs to be consistent
with whichever fdtable a lock-free reader sees.

> 
> If yes, the whole embedded struct fdtable is readonly.

But not close_on_exec_init or open_fds_init. We would update them
on open/close.

Some benchmarking would be useful here.

Thanks
Dipankar