pread() and pwrite() system calls

pread() and pwrite() system calls

[David Laight]

We have a system with linux 2.6.32 and the somewhat archaic
uClibc 0.9.27 (but I'm not sure the current version is
any better, and I think there are binary compatibility
if we update).

I've just discovered that pread() is 'implemented'
by using 3 lseek() system calls and read().
(the same is true for the 64bit versions).

I thought that pread() was supposed to be atomic
(so usable concurrently by multiple threads) which
means that this implementation is completely broken.

I've not looked to see what glibc does.

I can see that part of the problem is the alignment
of the 64bit value on the argument list of syscall()
(when the register save area is cast to a sycall
argument structure).
But it also looks as though the uClibc syscall()
stub can only pass 5 arguments in registers, and
pread() (with an aligned 64bit offset) requires 6.

The ucLibc source seems to be predicated by __NR_pread,
but if that were defined it would try to call
__syscall_pread() and I can't find that anywhere.

A special pread/pwrite asm stub that just copies
r7 to r0 could be used.

Would it be enough to do:
syscall_pread_pwrite:
	mov 0,7
	sc
	blr
and handle the -ve -> errno in C?

I've seen other parts of linux pass structures to
avoid issues with excessive number of arguments.
Unlike things like NetBSD amd64 which will read
additional args off the user stack (and has per-system
call stubs in libc so doesn't lose one argument).
(I rearranged the NetBSD amd64 kernel trap frame
layout so all the args ended up adjacent...)

	David

Re: pread() and pwrite() system calls

[Gabriel Paubert]

On Fri, May 25, 2012 at 02:29:06PM +0100, David Laight wrote:
> We have a system with linux 2.6.32 and the somewhat archaic
> uClibc 0.9.27 (but I'm not sure the current version is
> any better, and I think there are binary compatibility
> if we update).
> 
> I've just discovered that pread() is 'implemented'
> by using 3 lseek() system calls and read().
> (the same is true for the 64bit versions).
> 
> I thought that pread() was supposed to be atomic
> (so usable concurrently by multiple threads) which
> means that this implementation is completely broken.
> 
> I've not looked to see what glibc does.
> 
> I can see that part of the problem is the alignment
> of the 64bit value on the argument list of syscall()
> (when the register save area is cast to a sycall
> argument structure).
> But it also looks as though the uClibc syscall()
> stub can only pass 5 arguments in registers, and
> pread() (with an aligned 64bit offset) requires 6.
> 
> The ucLibc source seems to be predicated by __NR_pread,
> but if that were defined it would try to call
> __syscall_pread() and I can't find that anywhere.
> 
> A special pread/pwrite asm stub that just copies
> r7 to r0 could be used.
> 
> Would it be enough to do:
> syscall_pread_pwrite:
> 	mov 0,7
> 	sc
> 	blr
> and handle the -ve -> errno in C?

Huh? Won't fly, r0 is used for the system call number!

On the other hand, I believed PPC had no problems passing
up to 8 32 bit arguments in registers (r3 to r10), but
I may be confusing with the standard ABI for function calls.

Hmm, a quick look at kernel/entry_32.s shows that it should 
be able to use at least r3 to r8, which should be sufficient.

I think that it is an uClibc problem.

	Gabriel

RE: pread() and pwrite() system calls

[David Laight]

> > A special pread/pwrite asm stub that just copies
> > r7 to r0 could be used.
> >=20
> > Would it be enough to do:
> > syscall_pread_pwrite:
> > 	mov 0,7
> > 	sc
> > 	blr
> > and handle the -ve -> errno in C?
>=20
> Huh? Won't fly, r0 is used for the system call number!

I was copying that from r7!
Actually I have a much better stub by copying the one
used for mmap().
The system call itself is fine.
Using the system call almost halved the time taken
for a 4-byte read.

> On the other hand, I believed PPC had no problems passing
> up to 8 32 bit arguments in registers (r3 to r10), but
> I may be confusing with the standard ABI for function calls.
>=20
> Hmm, a quick look at kernel/entry_32.s shows that it should=20
> be able to use at least r3 to r8, which should be sufficient.
>=20
> I think that it is an uClibc problem.

True, in that it isn't a kernel bug.
OTOH the kernel is likely to get blamed for non-atomic pread.
I've found the same user-space code in newlib as well.
glibc may be ok, some code I've found implies it only
uses the 'emulation' when the system call returns ENOSYS.

	David

RE: pread() and pwrite() system calls

[Michael Ellerman]

On Tue, 2012-05-29 at 09:28 +0100, David Laight wrote:
> > > A special pread/pwrite asm stub that just copies
> > > r7 to r0 could be used.
> > > 
> > > Would it be enough to do:
> > > syscall_pread_pwrite:
> > > 	mov 0,7
> > > 	sc
> > > 	blr
> > > and handle the -ve -> errno in C?
> > 
> > Huh? Won't fly, r0 is used for the system call number!
> 
> I was copying that from r7!

Using the non-existant 'mov' instruction!

or 0,7,7

is much clearer :)

cheers

RE: pread() and pwrite() system calls

[David Laight]

> > We have a system with linux 2.6.32 and the somewhat archaic
> > uClibc 0.9.27 (but I'm not sure the current version is
> > any better, and I think there are binary compatibility
> > if we update).
> >=20
> > I've just discovered that pread() is 'implemented'
> > by using 3 lseek() system calls and read().
> > (the same is true for the 64bit versions).
...
> I think that it is an uClibc problem.

It seems that uClibc hadn't been changed when the
names of the constants used for the system calls
were changed from __NR_pread to __NR_pread64.

	David