What protects f_pos?

What protects f_pos?

[David Wragg]

Since f_pos of struct file is a loff_t, on 32-bit architectures it
needs a lock to make accesses atomic (or some more sophisticated form
of protection).  But looking in 2.4.0-test10, there doesn't seem to be
any such lock.

The llseek op is called with the Big Kernel Lock, but unlike in 2.2,
the read and write ops are called without any locks held, and so
generic_file_{read|write} make unprotected accesses to f_pos (through
their ppos argument).

Is this something for Ted's todo list?

David
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What protects f_pos?

[tytso]

   From: David Wragg <dpw@doc.ic.ac.uk>
   Date: 	04 Nov 2000 22:16:18 +0000

   Since f_pos of struct file is a loff_t, on 32-bit architectures it
   needs a lock to make accesses atomic (or some more sophisticated form
   of protection).  But looking in 2.4.0-test10, there doesn't seem to be
   any such lock.

   The llseek op is called with the Big Kernel Lock, but unlike in 2.2,
   the read and write ops are called without any locks held, and so
   generic_file_{read|write} make unprotected accesses to f_pos (through
   their ppos argument).

This looks like it's a bug to me....  although if you have multiple
threads hitting a file descriptor at the same time, you're pretty much
asking for trouble.

						- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What protects f_pos?

[David Wragg]

tytso@mit.edu writes:
> This looks like it's a bug to me....  although if you have multiple
> threads hitting a file descriptor at the same time, you're pretty much
> asking for trouble.

Yes, I haven't been able to come up with an example that might trigger
this that wasn't dubious to begin with.  I'll raise this again at a
convenient time during 2.5.

David

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

RE: What protects f_pos?

[David Schwartz]

> tytso@mit.edu writes:
> > This looks like it's a bug to me....  although if you have multiple
> > threads hitting a file descriptor at the same time, you're pretty much
> > asking for trouble.
>
> Yes, I haven't been able to come up with an example that might trigger
> this that wasn't dubious to begin with.  I'll raise this again at a
> convenient time during 2.5.
>
> David

	Suppose you had a multithreaded program that used a configuration file with
a group of fixed-length blocks indicating what work it had to do. Each
thread read a block from the file and then did the work. One might think
that there's no need to protect the file descriptor with a mutex.

	DS

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What protects f_pos?

[David Wragg]

"David Schwartz" <davids@webmaster.com> writes:
> 	Suppose you had a multithreaded program that used a
> configuration file with a group of fixed-length blocks indicating what
> work it had to do. Each thread read a block from the file and then did
> the work. One might think that there's no need to protect the file
> descriptor with a mutex.

I don't think that this will work, due to a separate non-atomicity
issue with f_pos.  The generic file read and write implementations do
not atomically update f_pos.  They read f_pos to determine the file
offset to use, then manipulate the page cache (possibly sleeping on
I/O), and only then set f_pos to the appropriate updated value.  So
the example you suggest, with two threads, could do something like:

          Thread 1                           Thread 2

   sys_read(fd, buf1, len)

      off = file->f_pos                sys_read(fd, buf2, len)

         read to buf1                     off = file->f_pos

    file->f_pos = off + len                 read to buf2

                                        file->f_pos = off + len


So both threads read the same block, and f_pos only gets incremented
once.

(Pipes and sockets are a different matter, of course.)

2.2 has the same issue, since although the BKL is held, it will get
dropped if one of the threads sleeps on I/O.  (Earlier Linux versions
might well have the same issue, but I don't have the source around to
check.)

POSIX doesn't seem to bar this behaviour.  From 6.4.1.2:

    On a regular file or other file capable of seeking, read() shall
    start at a position in the file given by the file offset
    associated with fildes.  Before successful return from read(), the
    file offset shall be incremented by the number of bytes actually
    read.

Which is exactly what Linux does.  I can't find text anywhere else in
POSIX.1 that strengthens that condition for the case of multiple
processes/threads reading from the same file.  I'll try to find out
what the Austin Group has to say about this.


David
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: What protects f_pos?

[Richard B. Johnson]

On 13 Nov 2000, David Wragg wrote:

> "David Schwartz" <davids@webmaster.com> writes:
[SNIPPED....]


> 
> So both threads read the same block, and f_pos only gets incremented
> once.
> 
> (Pipes and sockets are a different matter, of course.)
> 
> 2.2 has the same issue, since although the BKL is held, it will get
> dropped if one of the threads sleeps on I/O.  (Earlier Linux versions
> might well have the same issue, but I don't have the source around to
> check.)
> 
> POSIX doesn't seem to bar this behaviour.  From 6.4.1.2:
> 
>     On a regular file or other file capable of seeking, read() shall
>     start at a position in the file given by the file offset
>     associated with fildes.  Before successful return from read(), the
>     file offset shall be incremented by the number of bytes actually
>     read.
> 
> Which is exactly what Linux does.  I can't find text anywhere else in
> POSIX.1 that strengthens that condition for the case of multiple
> processes/threads reading from the same file.  I'll try to find out
> what the Austin Group has to say about this.
> 

Anything that uses f_pos has a problem with multiple readers or multiple
writers. That's one of the problems that file locking is supposed to
handle. An application has got to know how to access a file when it can
be changed by another.

There is a more far-reaching problem that sooner or later will get us.
That's the built-in problem with lseek. Lseek is supposed to return -1
if it failed, with errno being set appropriately. This, in principle,
does not prevent seeking to the last byte of the file as long as errno
is checked by the caller if the function returns -1.

Linux, internally, uses "-errno" as a return value to show an
error. So what happens if we `lseek` to -EBADF or -EFAULT, etc.

An otherwise empty lseek routine in a module, that does nothing except
return the input value, will show errors once the offsets corresponding
to negative errno values are reached.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/