Re: [PATCH 2/4] Convert epoll to a bitlock

[Eric Dumazet <dada1@cosmosbay.com>]

Davide Libenzi a écrit :
> On Tue, 3 Feb 2009, Eric Dumazet wrote:
> 
>> Andrew Morton a écrit :
>>> On Mon,  2 Feb 2009 11:20:09 -0700
>>> Jonathan Corbet <corbet@lwn.net> wrote:
>>>
>>>> Matt Mackall suggested converting epoll's ep_lock to a bitlock as a way of
>>>> saving space in struct file.  This patch makes that change.
>>> hrm.  bit_spin_lock() makes people upset (large penguiny people).  iirc
>>> it doesn't have all the correct/well-understood memory/compiler
>>> ordering semantics which spinlocks have.  And lockdep doesn't know about
>>> it.
>>>
>> In a previous attempt (2005), I suggested using a single global lock.
>>
>> http://search.luky.org/linux-kernel.2005/msg50862.html
>>
>> Probably an array of hashed spinlocks would be more than enough.
> 
> That could be done, although I'm not sure it's worth going that way to 
> save 4 bytes. The effective saving rate is not even 4/sizeof(struct file) 
> since struct file never comes alone, and when you allocate a struct file 
> you always carry more allocations behind (at least for the cases where you 
> tend to have a lot of them around, so size would matter).
> The add/remove path in epoll is not a super-hot one, so it could be done. 
> I dunno how this change matter with the patchset though.

Back in 2005, I saved 4 bytes per file, and because of HWCACHE alignment, sizeof(struct file)
shrinked by 64 bytes. With more than 1.000.000 sockets opened on a busy server, it saved
64 MB of ram. At that time, this mattered (8GB of ram), but in 2009, 64 MB is so small
I dont care anymore about sizeof(struct file)

AFAIK, I just checked on x86_64 and got : sizeof(struct file)=0xc0 , so thats perfect :)

(Only thing I still do is to move private_data in the first cache line of struct file, because
it speedups a lot socket operation, when dealing with 1.000.000 sockets : one cache line miss
avoided per socket syscall)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 6022f44..03b2227 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -842,6 +842,8 @@ struct file {
 #define f_dentry       f_path.dentry
 #define f_vfsmnt       f_path.mnt
        const struct file_operations    *f_op;
+       /* needed for tty driver, and maybe others */
+       void                    *private_data;
        atomic_long_t           f_count;
        unsigned int            f_flags;
        fmode_t                 f_mode;
@@ -854,8 +856,6 @@ struct file {
 #ifdef CONFIG_SECURITY
        void                    *f_security;
 #endif
-       /* needed for tty driver, and maybe others */
-       void                    *private_data;

 #ifdef CONFIG_EPOLL
        /* Used by fs/eventpoll.c to link all the hooks to this file */