Fwd: cifs ignores sysct setting

Fwd: cifs ignores sysct setting

[Alexander Swen]

Hi Steve and other CIFS developers, 

please help me with the following: 
1. fix the bug I explain below ;-) 
2. tell me to what bug tracking system I should to file it to. 

so, imho this is a bug: when I load the cifs module it insists in setting oplockenabled to 1 while we want to have locking disabled (due to our poormen's nas who's performance sinks to subzero when we do serious io ;-) 

I've filed this bug at Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644135 

I paste all details over here as well: 
<paste> 
Package: linux-mage
Version: 2.6.32-5-amd64

cifs.ko module (which is included in the kernel pkg as far as i know)
cifs driver version 1.61

when cifs module loads oplock is enabled by default.
so I can do
echo 0 >/proc/fs/cifs/OplockEnabled
but that won't persist through a driver reload. so we tried adding this to /etc/sysctl.conf:
fs.cifs.OplockEnabled = 0
however:

root@nl14s0008-vm1:~# echo 0 >/proc/fs/cifs/OplockEnabled
root@nl14s0008-vm1:~# cat /proc/fs/cifs/OplockEnabled
0
root@nl14s0008-vm1:~# modprobe -r cifs
root@nl14s0008-vm1:~# tail -1 /etc/sysctl.conf
fs.cifs.OplockEnabled = 0
root@nl14s0008-vm1:~# lsmod|grep cifs
root@nl14s0008-vm1:~# modprobe cifs
root@nl14s0008-vm1:~# !l
lsmod |grep cifs
cifs 262528 0
nls_base 6377 3 cifs,nls_utf8,usbcore
root@nl14s0008-vm1:~# !cat
cat /proc/fs/cifs/OplockEnabled
1

So, when I read the modinfo of cifs, there are a few module options, but no oplock option.
I would like to add something like:
options cifs oplocksenabled=0
to /etc/modprobe.d/aliases.conf

Is it feasible to add that param as an option?

I use the cifs 1.61 module on 2.6.32-5-amd64 kernel on Debian 6.0 (Debian 2.6.32-35squeeze2)

</paste> 
so, Ben is partly right, me request to add a param option is a wishlist thing, but the behaviour of the sysctl setting is a bug imho. 

thanks for your response! 
Alex 




Met vriendelijke groet, 
Alexander Swen 

Met vriendelijke groet, Alexander Swen 


Met vriendelijke groet, 
Alexander Swen 

----- Original Message ----- 


From: "Ben Hutchings" <ben-/+tVBieCtBitmTQ+vhA3Yw@public.gmane.org> 
To: "Alexander Swen" <alex-UE4+9DRHXtc@public.gmane.org> 
Cc: 644135-61a8vm9lEZVf4u+23C9RwQ@public.gmane.org 
Sent: Friday, 7 October, 2011 7:03:59 AM 
Subject: Re: cifs ignores sysct setting 

On Mon, 2011-10-03 at 09:46 +0200, Alexander Swen wrote: 
> Package: linux-mage 
> Version: 2.6.32-5-amd64 
> 
> cifs.ko module (which is included in the kernel pkg as far as i know) 
> cifs driver version 1.61 
> 
> when cifs module loads oplock is enabled by default. 
> so I can do 
> echo 0 >/proc/fs/cifs/OplockEnabled 
> but that won't persist through a driver reload. so we tried adding 
> this to /etc/sysctl.conf: 
> fs.cifs.OplockEnabled = 0 
[...] 
> So, when I read the modinfo of cifs, there are a few module options, 
> but no oplock option. 
> I would like to add something like: 
> options cifs oplocksenabled=0 
> to /etc/modprobe.d/aliases.conf 
> 
> Is it feasible to add that param as an option? 
[...] 

You should ask the upstream developers (Steve French <sfrench-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> 
and linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org) for this feature. 

Ben. 

-- 
Ben Hutchings 
For every action, there is an equal and opposite criticism. - Harrison 

Re: Fwd: cifs ignores sysct setting

[Suresh Jayaraman]

On 10/10/2011 03:02 PM, Alexander Swen wrote:
> 
> Hi Steve and other CIFS developers, 
> 
> please help me with the following: 
> 1. fix the bug I explain below ;-) 
> 2. tell me to what bug tracking system I should to file it to. 
> 
> so, imho this is a bug: when I load the cifs module it insists in setting oplockenabled to 1 while we want to have locking disabled (due to our poormen's nas who's performance sinks to subzero when we do serious io ;-) 
> 
> I've filed this bug at Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644135 
> 
> I paste all details over here as well: 
> <paste> 
> Package: linux-mage
> Version: 2.6.32-5-amd64
> 
> cifs.ko module (which is included in the kernel pkg as far as i know)
> cifs driver version 1.61
> 
> when cifs module loads oplock is enabled by default.
> so I can do
> echo 0 >/proc/fs/cifs/OplockEnabled
> but that won't persist through a driver reload. so we tried adding this to /etc/sysctl.conf:
> fs.cifs.OplockEnabled = 0
> however:
> 
> root@nl14s0008-vm1:~# echo 0 >/proc/fs/cifs/OplockEnabled
> root@nl14s0008-vm1:~# cat /proc/fs/cifs/OplockEnabled
> 0
> root@nl14s0008-vm1:~# modprobe -r cifs
> root@nl14s0008-vm1:~# tail -1 /etc/sysctl.conf
> fs.cifs.OplockEnabled = 0

IIUC, CIFS OplockEnabled is not a proper sysctl, it is just a proc
entry. Sysctls are listed below /proc/sys/ directory.

For e.g. try
   sysctl -w fs.cifs.OplockEnabled=0

you'll see the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
directory". Also,

   sysctl -p

will throw the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
directory"

It is not clear to me why do you want to unload and reload cifs module
often? The general usage is load module during boot or on first use and
the module lives on till the machine goes down.

I'm not seeing an justification for making it an module parameter. Could
you please explain more on how it will be useful?


-Suresh

Re: cifs ignores sysct setting

[Jeff Layton]

On Mon, 10 Oct 2011 18:36:15 +0530
Suresh Jayaraman <sjayaraman-IBi9RG/b67k@public.gmane.org> wrote:

> On 10/10/2011 03:02 PM, Alexander Swen wrote:
> > 
> > Hi Steve and other CIFS developers, 
> > 
> > please help me with the following: 
> > 1. fix the bug I explain below ;-) 
> > 2. tell me to what bug tracking system I should to file it to. 
> > 
> > so, imho this is a bug: when I load the cifs module it insists in setting oplockenabled to 1 while we want to have locking disabled (due to our poormen's nas who's performance sinks to subzero when we do serious io ;-) 
> > 
> > I've filed this bug at Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644135 
> > 
> > I paste all details over here as well: 
> > <paste> 
> > Package: linux-mage
> > Version: 2.6.32-5-amd64
> > 
> > cifs.ko module (which is included in the kernel pkg as far as i know)
> > cifs driver version 1.61
> > 
> > when cifs module loads oplock is enabled by default.
> > so I can do
> > echo 0 >/proc/fs/cifs/OplockEnabled
> > but that won't persist through a driver reload. so we tried adding this to /etc/sysctl.conf:
> > fs.cifs.OplockEnabled = 0
> > however:
> > 
> > root@nl14s0008-vm1:~# echo 0 >/proc/fs/cifs/OplockEnabled
> > root@nl14s0008-vm1:~# cat /proc/fs/cifs/OplockEnabled
> > 0
> > root@nl14s0008-vm1:~# modprobe -r cifs
> > root@nl14s0008-vm1:~# tail -1 /etc/sysctl.conf
> > fs.cifs.OplockEnabled = 0
> 
> IIUC, CIFS OplockEnabled is not a proper sysctl, it is just a proc
> entry. Sysctls are listed below /proc/sys/ directory.
> 
> For e.g. try
>    sysctl -w fs.cifs.OplockEnabled=0
> 
> you'll see the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
> directory". Also,
> 
>    sysctl -p
> 
> will throw the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
> directory"
> 
> It is not clear to me why do you want to unload and reload cifs module
> often? The general usage is load module during boot or on first use and
> the module lives on till the machine goes down.
> 
> I'm not seeing an justification for making it an module parameter. Could
> you please explain more on how it will be useful?
> 

I agree with Alexander on this one...

Making that a module parm would allow you to set that parameter at boot
time without needing to add special startup scripts. IMO, all of the
procfile "switches" under /proc/fs/cifs should be module parms
instead.

That would also allow us to get rid of some of the special procfile
handling code for those files as well.

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Re: cifs ignores sysct setting

[Suresh Jayaraman]

On 10/10/2011 07:07 PM, Jeff Layton wrote:
> On Mon, 10 Oct 2011 18:36:15 +0530
> Suresh Jayaraman <sjayaraman-IBi9RG/b67k@public.gmane.org> wrote:
> 
>> On 10/10/2011 03:02 PM, Alexander Swen wrote:
>>>
>>> Hi Steve and other CIFS developers, 
>>>
>>> please help me with the following: 
>>> 1. fix the bug I explain below ;-) 
>>> 2. tell me to what bug tracking system I should to file it to. 
>>>
>>> so, imho this is a bug: when I load the cifs module it insists in setting oplockenabled to 1 while we want to have locking disabled (due to our poormen's nas who's performance sinks to subzero when we do serious io ;-) 
>>>
>>> I've filed this bug at Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644135 
>>>
>>> I paste all details over here as well: 
>>> <paste> 
>>> Package: linux-mage
>>> Version: 2.6.32-5-amd64
>>>
>>> cifs.ko module (which is included in the kernel pkg as far as i know)
>>> cifs driver version 1.61
>>>
>>> when cifs module loads oplock is enabled by default.
>>> so I can do
>>> echo 0 >/proc/fs/cifs/OplockEnabled
>>> but that won't persist through a driver reload. so we tried adding this to /etc/sysctl.conf:
>>> fs.cifs.OplockEnabled = 0
>>> however:
>>>
>>> root@nl14s0008-vm1:~# echo 0 >/proc/fs/cifs/OplockEnabled
>>> root@nl14s0008-vm1:~# cat /proc/fs/cifs/OplockEnabled
>>> 0
>>> root@nl14s0008-vm1:~# modprobe -r cifs
>>> root@nl14s0008-vm1:~# tail -1 /etc/sysctl.conf
>>> fs.cifs.OplockEnabled = 0
>>
>> IIUC, CIFS OplockEnabled is not a proper sysctl, it is just a proc
>> entry. Sysctls are listed below /proc/sys/ directory.
>>
>> For e.g. try
>>    sysctl -w fs.cifs.OplockEnabled=0
>>
>> you'll see the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
>> directory". Also,
>>
>>    sysctl -p
>>
>> will throw the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
>> directory"
>>
>> It is not clear to me why do you want to unload and reload cifs module
>> often? The general usage is load module during boot or on first use and
>> the module lives on till the machine goes down.
>>
>> I'm not seeing an justification for making it an module parameter. Could
>> you please explain more on how it will be useful?
>>
> 
> I agree with Alexander on this one...
> 
> Making that a module parm would allow you to set that parameter at boot
> time without needing to add special startup scripts. IMO, all of the
> procfile "switches" under /proc/fs/cifs should be module parms
> instead.

Making it a module parm would mean that it would no longer be possible
to dynamically change OplockEnabled. But, that might be Ok, because it
is quite unusual to change this setting once the module is loaded,
right?. Also, someone won't be able to see what is the current setting.
Again, not a big deal.

> That would also allow us to get rid of some of the special procfile
> handling code for those files as well.
> 

Agreed.

I'll whip up a patch and post it soon.


-Suresh

Re: cifs ignores sysct setting

[Steve French]

Actually, it seems more likely that you would turn off oplock temporarily
(ie via /proc) rather than permanently (until cifs is reloaded or the
system reboots).   Wouldn't you only turn oplock off to temporarily
work around a buggy server?

On Tue, Oct 11, 2011 at 4:14 AM, Suresh Jayaraman <sjayaraman-IBi9RG/b67k@public.gmane.org> wrote:
> On 10/10/2011 07:07 PM, Jeff Layton wrote:
>> On Mon, 10 Oct 2011 18:36:15 +0530
>> Suresh Jayaraman <sjayaraman-IBi9RG/b67k@public.gmane.org> wrote:
>>
>>> On 10/10/2011 03:02 PM, Alexander Swen wrote:
>>>>
>>>> Hi Steve and other CIFS developers,
>>>>
>>>> please help me with the following:
>>>> 1. fix the bug I explain below ;-)
>>>> 2. tell me to what bug tracking system I should to file it to.
>>>>
>>>> so, imho this is a bug: when I load the cifs module it insists in setting oplockenabled to 1 while we want to have locking disabled (due to our poormen's nas who's performance sinks to subzero when we do serious io ;-)
>>>>
>>>> I've filed this bug at Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=644135
>>>>
>>>> I paste all details over here as well:
>>>> <paste>
>>>> Package: linux-mage
>>>> Version: 2.6.32-5-amd64
>>>>
>>>> cifs.ko module (which is included in the kernel pkg as far as i know)
>>>> cifs driver version 1.61
>>>>
>>>> when cifs module loads oplock is enabled by default.
>>>> so I can do
>>>> echo 0 >/proc/fs/cifs/OplockEnabled
>>>> but that won't persist through a driver reload. so we tried adding this to /etc/sysctl.conf:
>>>> fs.cifs.OplockEnabled = 0
>>>> however:
>>>>
>>>> root@nl14s0008-vm1:~# echo 0 >/proc/fs/cifs/OplockEnabled
>>>> root@nl14s0008-vm1:~# cat /proc/fs/cifs/OplockEnabled
>>>> 0
>>>> root@nl14s0008-vm1:~# modprobe -r cifs
>>>> root@nl14s0008-vm1:~# tail -1 /etc/sysctl.conf
>>>> fs.cifs.OplockEnabled = 0
>>>
>>> IIUC, CIFS OplockEnabled is not a proper sysctl, it is just a proc
>>> entry. Sysctls are listed below /proc/sys/ directory.
>>>
>>> For e.g. try
>>>    sysctl -w fs.cifs.OplockEnabled=0
>>>
>>> you'll see the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
>>> directory". Also,
>>>
>>>    sysctl -p
>>>
>>> will throw the error: "/proc/sys/fs/cifs/OplockEnabled: No such file or
>>> directory"
>>>
>>> It is not clear to me why do you want to unload and reload cifs module
>>> often? The general usage is load module during boot or on first use and
>>> the module lives on till the machine goes down.
>>>
>>> I'm not seeing an justification for making it an module parameter. Could
>>> you please explain more on how it will be useful?
>>>
>>
>> I agree with Alexander on this one...
>>
>> Making that a module parm would allow you to set that parameter at boot
>> time without needing to add special startup scripts. IMO, all of the
>> procfile "switches" under /proc/fs/cifs should be module parms
>> instead.
>
> Making it a module parm would mean that it would no longer be possible
> to dynamically change OplockEnabled. But, that might be Ok, because it
> is quite unusual to change this setting once the module is loaded,
> right?. Also, someone won't be able to see what is the current setting.
> Again, not a big deal.
>
>> That would also allow us to get rid of some of the special procfile
>> handling code for those files as well.
>>
>
> Agreed.
>
> I'll whip up a patch and post it soon.
>
>
> -Suresh
> --
> To unsubscribe from this list: send the line "unsubscribe linux-cifs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Thanks,

Steve

Re: cifs ignores sysct setting

[Steve French]

On Tue, Oct 11, 2011 at 9:14 AM, Alexander Swen <alex-UE4+9DRHXtc@public.gmane.org> wrote:
> Hi Steve,
>
> thanks for your response.
>
> no, we do permanently turn off oplocks due to some performance issues (with
> some NAS device where our colleagues can't (don't want) configure nfs
> apparently ;-(.
> we do this on several (>20) servers and across reboots...

Isn't it easier just to turn this (oplocks) off on the server?  At
least with Linux cifs.ko
you can turn off oplocks at runtime, with some clients it is much harder.


-- 
Thanks,

Steve

Re: cifs ignores sysct setting

[Steve French]

On Tue, Oct 11, 2011 at 9:22 AM, Alexander Swen <alex-UE4+9DRHXtc@public.gmane.org> wrote:
>
>
> Met vriendelijke groet,
> Alexander Swen
> ________________________________
>
> From: "Steve French" <smfrench-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> To: "Alexander Swen" <alex-UE4+9DRHXtc@public.gmane.org>
> Cc: "Jeff Layton" <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>, linux-cifs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "Suresh
> Jayaraman" <sjayaraman-IBi9RG/b67k@public.gmane.org>
> Sent: Tuesday, 11 October, 2011 4:20:17 PM
> Subject: Re: cifs ignores sysct setting
>
> On Tue, Oct 11, 2011 at 9:14 AM, Alexander Swen <alex-UE4+9DRHXtc@public.gmane.org> wrote:
>> Hi Steve,
>>
>> thanks for your response.
>>
>> no, we do permanently turn off oplocks due to some performance issues
>> (with
>> some NAS device where our colleagues can't (don't want) configure nfs
>> apparently ;-(.
>> we do this on several (>20) servers and across reboots...
>
> Isn't it easier just to turn this (oplocks) off on the server?  At
> least with Linux cifs.ko
> you can turn off oplocks at runtime, with some clients it is much harder.
>
>
> ahhh. no, we are not the only users on the nas, our clientservers should
> just turn off oplocks. it is, we make backups to the nas device and those
> files grow quite big.

If you are trying to save RAM on the client, and don't want to cache,
and are just using the linux cifs client for backup, you could mount
with "forcedirectio" (which turns off caching a little more easily).


-- 
Thanks,

Steve

Re: cifs ignores sysct setting

[Alexander Swen]

> >
> >
> > ahhh. no, we are not the only users on the nas, our clientservers
> > should
> > just turn off oplocks. it is, we make backups to the nas device and
> > those
> > files grow quite big.

> If you are trying to save RAM on the client, and don't want to cache,
> and are just using the linux cifs client for backup, you could mount
> with "forcedirectio" (which turns off caching a little more easily).

Hi Steve, it was that we had i/o timeouts on our cifs mounted volume while doing rsyncs to that volume. since we disabled oplock the timeouts vanished. however, I will suggest your idea to my colleague who had that problem and he might give it a go as well.

thanks for your thoughts!
Alex

> Thanks,

> Steve

Re: cifs ignores sysct setting

[Jeff Layton]

On Wed, 12 Oct 2011 09:10:52 +0200 (CEST)
Alexander Swen <alex-UE4+9DRHXtc@public.gmane.org> wrote:

> 
> > >
> > >
> > > ahhh. no, we are not the only users on the nas, our clientservers
> > > should
> > > just turn off oplocks. it is, we make backups to the nas device and
> > > those
> > > files grow quite big.
> 
> > If you are trying to save RAM on the client, and don't want to cache,
> > and are just using the linux cifs client for backup, you could mount
> > with "forcedirectio" (which turns off caching a little more easily).
> 
> Hi Steve, it was that we had i/o timeouts on our cifs mounted volume while doing rsyncs to that volume. since we disabled oplock the timeouts vanished. however, I will suggest your idea to my colleague who had that problem and he might give it a go as well.
> 

The 2.6.32 kernel you're using also doesn't support async writes,
unless the debian maintainers backported that work, which they don't
typically do. If you move to a newer kernel you may get better
performance as well...

-- 
Jeff Layton <jlayton-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org>

Fwd: cifs ignores sysct setting

[Alexander Swen]

From: "Alexander Swen" <alex-UE4+9DRHXtc@public.gmane.org> 
To: "Suresh Jayaraman" <sjayaraman-IBi9RG/b67k@public.gmane.org> 
Sent: Monday, 10 October, 2011 3:44:47 PM 
Subject: Re: cifs ignores sysct setting 

Hi Suresh, thanks for your re, 

> It is not clear to me why do you want to unload and reload cifs 
> module 
> often? The general usage is load module during boot or on first use 
> and 
> the module lives on till the machine goes down. 

I don't (want to load cifs often ;-). but I don't want to exec a command to configure cifs once a server starts up). Now, I know I can automate such thing using rc.local etc... but I rather configure those things just as I would configure any other kernel module settings (preferably using modprobe.d/cifs.conf or something similar) 

> I'm not seeing an justification for making it an module parameter. 

I do: when I load for example a module like bonding I tell it how to behave in with settings which i put into a file called /etc/modprobe.d/aliases: 
alias bond0 bonding 
options bonding mode=1 miimon=100 


> Could you please explain more on how it will be useful? 

i would like to change the behaviour of cifs so that oplock is disabled when it loads. 

can you follow my way of thinking? 

thanks for your cooperation, 

Alex