is flock broken in 2.4 or 2.5 kernels or what does this mean?

is flock broken in 2.4 or 2.5 kernels or what does this mean?

[John Covici]

In the latest release notes of sendmail I have read the following:

		NOTE: Linux appears to have broken flock() again.  Unless
			the bug is fixed before sendmail 8.13 is shipped,
			8.13 will change the default locking method to
			fcntl() for Linux kernel 2.4 and later.  You may
			want to do this in 8.12 by compiling with
			-DHASFLOCK=0.  Be sure to update other sendmail
			related programs to match locking techniques.

Can anyone tell me what this is all about -- is there any basis in
reality for what they are saying?

Thanks.

-- 
         John Covici
         covici@ccs.covici.com

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[Richard A Nelson]

On 23 Jul 2002, Alan Cox wrote:

> Date: 23 Jul 2002 17:31:12 +0100
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: John Covici <covici@ccs.covici.com>
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?
>
> On Tue, 2002-07-23 at 15:41, John Covici wrote:
> > In the latest release notes of sendmail I have read the following:
> >
> > 		NOTE: Linux appears to have broken flock() again.  Unless
> > 			the bug is fixed before sendmail 8.13 is shipped,
> > 			8.13 will change the default locking method to
> > 			fcntl() for Linux kernel 2.4 and later.  You may
> > 			want to do this in 8.12 by compiling with
> > 			-DHASFLOCK=0.  Be sure to update other sendmail
> > 			related programs to match locking techniques.
> >
> > Can anyone tell me what this is all about -- is there any basis in
> > reality for what they are saying?
>
> First I've heard of it, so it would be useful if someone has access to
> the sendmail problem report/test in question that shows it and I'll go
> find out.

Indeed it is, this message explains it fairly well:
	Message-ID: <20020702160118.C4711@redhat.com>
	Date: Tue, 2 Jul 2002 16:01:18 +0100
	From: Stephen C. Tweedie <sct@redhat.com>

I haven't heard of any plans to incorporate the removal of accounting
info in either 2.4 or 2.5 ;(

-- 
Rick Nelson
Your job is being a professor and researcher: That's one hell of a good excuse
for some of the brain-damages of minix.
(Linus Torvalds to Andrew Tanenbaum)

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[Jirka Kosina]

On 23 Jul 2002, Alan Cox wrote:

> > Can anyone tell me what this is all about -- is there any basis in
> > reality for what they are saying?
> First I've heard of it, so it would be useful if someone has access to
> the sendmail problem report/test in question that shows it and I'll go
> find out.

Quoting Stephen Tweedie's earlier post to the list:

==
It really is broken, and sendmail triggers it (at least their
commercial binaries do).  I've already been talking to willy about the
problem. The trouble is the accounting: if one process opens a fd and then
fork()s, it is possible for the lock to be taken in the parent and
released in the child (or vice versa) --- unless there's an explicit
flock(LOCK_UN), then the lock will be released implicitly when the
last reference to the fd is closed. When this happens, we get the lock 
count incremented in one task and
decremented in another.  That can wrap the lock count backwards to -1
(or rather ~0UL), which causes the locks rlimit check to think we've
exceeded the lock quota and new lock requests will fail.  It's easy to
reproduce this: try the attached prog.  It produces an erroneous
ENOLCK due to the bug.
==

-- 
JiKos.

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[Alan Cox]

On Tue, 2002-07-23 at 15:41, John Covici wrote:
> In the latest release notes of sendmail I have read the following:
> 
> 		NOTE: Linux appears to have broken flock() again.  Unless
> 			the bug is fixed before sendmail 8.13 is shipped,
> 			8.13 will change the default locking method to
> 			fcntl() for Linux kernel 2.4 and later.  You may
> 			want to do this in 8.12 by compiling with
> 			-DHASFLOCK=0.  Be sure to update other sendmail
> 			related programs to match locking techniques.
> 
> Can anyone tell me what this is all about -- is there any basis in
> reality for what they are saying?

First I've heard of it, so it would be useful if someone has access to
the sendmail problem report/test in question that shows it and I'll go
find out.

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[J.A. Magallon]

On 2002.07.23 Alan Cox wrote:
>On Tue, 2002-07-23 at 15:41, John Covici wrote:
>> In the latest release notes of sendmail I have read the following:
>> 
>> 		NOTE: Linux appears to have broken flock() again.  Unless
>> 			the bug is fixed before sendmail 8.13 is shipped,
>> 			8.13 will change the default locking method to
>> 			fcntl() for Linux kernel 2.4 and later.  You may
>> 			want to do this in 8.12 by compiling with
>> 			-DHASFLOCK=0.  Be sure to update other sendmail
>> 			related programs to match locking techniques.
>> 
>> Can anyone tell me what this is all about -- is there any basis in
>> reality for what they are saying?
>
>First I've heard of it, so it would be useful if someone has access to
>the sendmail problem report/test in question that shows it and I'll go
>find out.
>

Perhaps if you have your spool over nfs:

man flock:

NOTES
       flock(2) does not  lock  files  over  NFS.   Use  fcntl(2)
       instead:  that  does  work  over NFS, given a sufficiently
       recent version of Linux and a server which supports  lock­
       ing.

       flock(2)  and fcntl(2) locks have different semantics with
       respect to forked processes and dup(2).

-- 
J.A. Magallon             \   Software is like sex: It's better when it's free
mailto:jamagallon@able.es  \                    -- Linus Torvalds, FSF T-shirt
Linux werewolf 2.4.19-rc3-jam1, Mandrake Linux 9.0 (Cooker) for i586
gcc (GCC) 3.1.1 (Mandrake Linux 8.3 3.1.1-0.10mdk)

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[Andrea Arcangeli]

On Tue, Jul 23, 2002 at 05:31:12PM +0100, Alan Cox wrote:
> On Tue, 2002-07-23 at 15:41, John Covici wrote:
> > In the latest release notes of sendmail I have read the following:
> > 
> > 		NOTE: Linux appears to have broken flock() again.  Unless
> > 			the bug is fixed before sendmail 8.13 is shipped,
> > 			8.13 will change the default locking method to
> > 			fcntl() for Linux kernel 2.4 and later.  You may
> > 			want to do this in 8.12 by compiling with
> > 			-DHASFLOCK=0.  Be sure to update other sendmail
> > 			related programs to match locking techniques.
> > 
> > Can anyone tell me what this is all about -- is there any basis in
> > reality for what they are saying?
> 
> First I've heard of it, so it would be useful if someone has access to
> the sendmail problem report/test in question that shows it and I'll go
> find out.

fix is in -aa from Matthew Wilcox:

	http://www.us.kernel.org/pub/linux/kernel/people/andrea/kernels/v2.4/2.4.19rc3aa1/00_drop-broken-flock-account-1

fcntl API never obeyed to the accounting anyways.

Andrea

re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[dank]

To save others a bit of searching:
the original thread was 'broken flock()' on 28 June 2002.

Steven's post and regression test:
http://marc.theaimsgroup.com/?l=linux-kernel&m=102562226210792&w=2
Matthew Wilcox's rip-it-out patch:
http://marc.theaimsgroup.com/?l=linux-fsdevel&m=102562469813922&w=2

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[dank]

dank@kegel.com wrote:
> 
> To save others a bit of searching:
> the original thread was 'broken flock()' on 28 June 2002.
> 
> Steven's post and regression test:
> http://marc.theaimsgroup.com/?l=linux-kernel&m=102562226210792&w=2
> Matthew Wilcox's rip-it-out patch:
> http://marc.theaimsgroup.com/?l=linux-fsdevel&m=102562469813922&w=2

See also
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=59992

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[David Ford]

(Alan)
Actually I posted a "is sendmail on crack?" question last month when 
8.12.5 was released and this was in the release notes.  We got a bunch 
of confirmations that Linux flock() was broken due to the accounting.

David

J.A. Magallon wrote:

>On 2002.07.23 Alan Cox wrote:
>  
>
>>On Tue, 2002-07-23 at 15:41, John Covici wrote:
>>    
>>
>>>In the latest release notes of sendmail I have read the following:
>>>
>>>		NOTE: Linux appears to have broken flock() again.  Unless
>>>			the bug is fixed before sendmail 8.13 is shipped,
>>>			8.13 will change the default locking method to
>>>			fcntl() for Linux kernel 2.4 and later.  You may
>>>			want to do this in 8.12 by compiling with
>>>			-DHASFLOCK=0.  Be sure to update other sendmail
>>>			related programs to match locking techniques.
>>>
>>>Can anyone tell me what this is all about -- is there any basis in
>>>reality for what they are saying?
>>>      
>>>
>>First I've heard of it, so it would be useful if someone has access to
>>the sendmail problem report/test in question that shows it and I'll go
>>find out.
>>
>>    
>>
>
>Perhaps if you have your spool over nfs:
>
>man flock:
>
>NOTES
>       flock(2) does not  lock  files  over  NFS.   Use  fcntl(2)
>       instead:  that  does  work  over NFS, given a sufficiently
>       recent version of Linux and a server which supports  lock­
>       ing.
>
>       flock(2)  and fcntl(2) locks have different semantics with
>       respect to forked processes and dup(2).
>
>  
>

-- 
I may have the information you need and I may choose only HTML.  It's up to
you. Disclaimer: I am not responsible for any email that you send me nor am
I bound to any obligation to deal with any received email in any given
fashion.  If you send me spam or a virus, I may in whole or part send you
50,000 return copies of it. I may also publically announce any and all
emails and post them to message boards, news sites, and even parody sites. 
I may also mark them up, cut and paste, print, and staple them to telephone
poles for the enjoyment of people without internet access.  This is not a
confidential medium and your assumption that your email can or will be
handled confidentially is akin to baring your backside, burying your head in
the ground, and thinking nobody can see you butt nekkid and in plain view
for miles away.  Don't be a cluebert, buy one from K-mart today.

Re: is flock broken in 2.4 or 2.5 kernels or what does this mean?

[Alan Cox]

On Wed, 2002-07-24 at 17:08, David Ford wrote:
> (Alan)
> Actually I posted a "is sendmail on crack?" question last month when 
> 8.12.5 was released and this was in the release notes.  We got a bunch 
> of confirmations that Linux flock() was broken due to the accounting.

Thanks. I now have the original discussion, the demonstration and a
patch Andrea believes fixes it.