new syscalls

new syscalls

[Andrew Morton]

Verious new syscalls which are trickling their way Linuswards and will need
testing and wiring up once they get there:

sys_migrate_pages
	Merged

sys_openat
sys_mkdirat
sys_mknodat
sys_fchownat
sys_futimesat
sys_newfstatat
sys_unlinkat
sys_renameat
sys_linkat
sys_symlinkat
sys_readlinkat
sys_fchmodat
sys_faccessat
	Probably this week

sys_pselect6
sys_ppoll
	Probably this week

sys_unshare
	Not sure yet.  These are a bit scary and I haven't seen any
	evidence that they've had the appropriate level of review.

sys_pepoll_wait
	Just queued this in -mm.  Conceivably 2.6.16, more likely 2.6.17, or
	never.

Re: new syscalls

[Andi Kleen]

On Wednesday 18 January 2006 04:29, Andrew Morton wrote:
> 
> Verious new syscalls which are trickling their way Linuswards and will need
> testing and wiring up once they get there:
> 
> sys_migrate_pages
> 	Merged

#define __NR_migrate_pages      256
__SYSCALL(__NR_migrate_pages, sys_migrate_pages)


We first have to wait if everybody survived crossing the 8bit boundary @)

-Andi

RE: new syscalls

[Luck, Tony]

> Various new syscalls which are trickling their way Linuswards and will need
> testing and wiring up once they get there:

Are there test-suites for any/all of these new calls?

-Tony

Re: new syscalls

[Andrew Morton]

"Luck, Tony" <tony.luck@intel.com> wrote:
>
> > Various new syscalls which are trickling their way Linuswards and will need
> > testing and wiring up once they get there:
> 
> Are there test-suites for any/all of these new calls?
> 

That's an excellent point.

Guys, architecture maintainers need test suites to verify that the syscalls
actually work as they wire them up.  Please share.  A stable URL would be
preferred - something which can go into the changlog or conceivably into
the kernel source.

Thanks.

Re: new syscalls

[Ulrich Drepper]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

Andrew Morton wrote:
> That's an excellent point.
> 
> Guys, architecture maintainers need test suites to verify that the syscalls
> actually work as they wire them up.

glibc has test for all the *at syscalls, pselect.  ppoll and epoll_pwait
will follow as soon as the syscalls are upstream and I can add the
interfaces.  unshare testing is harder, because it likely need
privileges.  Using the tests outside glibc isn't easy because of the
framework for tests we are using.

-- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 251 bytes --]

Re: new syscalls

[David Woodhouse]

On Wed, 2006-01-18 at 13:36 -0800, Andrew Morton wrote:
> That's an excellent point.
> 
> Guys, architecture maintainers need test suites to verify that the syscalls
> actually work as they wire them up.  Please share.  A stable URL would be
> preferred - something which can go into the changlog or conceivably into
> the kernel source.

http://david.woodhou.se/sigmasking.c is a good one for testing
sigsuspend after hooking up TIF_RESTORE_SIGMASK stuff and the generic
sys_rt_sigsuspend(), although it's not actually my code. There were a
bunch of other tests in a tarball with that (on linux-arch a year or two
ago) which arch maintainers should also use occasionally when they play
with signal or ptrace code.

When I get back home I'll tidy up and publish the hacks I used for
testing pselect() and ppoll() so that those can be tested without
actually rebuilding glibc.

-- 
dwmw2

Re: new syscalls

[David S. Miller]

From: Ulrich Drepper <drepper@redhat.com>
Date: Wed, 18 Jan 2006 13:48:22 -0800

> glibc

This puts an unnecessary burdon upon kernel platform maintainers.
They should not be required to go through the work and time of getting
together a sane enough build environment to build the current glibc
sources.

What we want is a self contained directory of small tests that
exercise the syscalls directly and do not have any dependency
on any large source base or installed libraries.

If I remember correctly, the folks who did EPOLL did exactly this
and the syscall additions were trivial to test as a result.

Re: new syscalls

[David S. Miller]

From: David Woodhouse <dwmw2@infradead.org>
Date: Thu, 19 Jan 2006 08:53:07 +1100

> When I get back home I'll tidy up and publish the hacks I used for
> testing pselect() and ppoll() so that those can be tested without
> actually rebuilding glibc.

Thanks David.

Re: new syscalls

[Davide Libenzi]

On Wed, 18 Jan 2006, Andrew Morton wrote:

> "Luck, Tony" <tony.luck@intel.com> wrote:
>>
>>> Various new syscalls which are trickling their way Linuswards and will need
>>> testing and wiring up once they get there:
>>
>> Are there test-suites for any/all of these new calls?
>>
>
> That's an excellent point.
>
> Guys, architecture maintainers need test suites to verify that the syscalls
> actually work as they wire them up.  Please share.  A stable URL would be
> preferred - something which can go into the changlog or conceivably into
> the kernel source.

How sophisticated this has to be? And what's the expected ETA? The 
epoll_pwait is really a wrapper around epoll_wait (that did not change at 
all), so the test in this case should just make sure that the signal 
behaviour is the one expected.



- Davide

Re: new syscalls

[Christoph Lameter]

On Wed, 18 Jan 2006, Andrew Morton wrote:

> Guys, architecture maintainers need test suites to verify that the syscalls
> actually work as they wire them up.  Please share.  A stable URL would be
> preferred - something which can go into the changlog or conceivably into
> the kernel source.

sys_migrate_pages will be supported by libnuma in Andi Kleen's numactl 
package. There is no URL that I am aware of.

Re: new syscalls

[David S. Miller]

From: Davide Libenzi <davidel@xmailserver.org>
Date: Wed, 18 Jan 2006 17:17:20 -0800 (PST)

> How sophisticated this has to be? And what's the expected ETA? The 
> epoll_pwait is really a wrapper around epoll_wait (that did not change at 
> all), so the test in this case should just make sure that the signal 
> behaviour is the one expected.

Something along the lines of a smoke test is probably sufficient.
The platform folks just want to make sure they wired up the
syscall tables correctly, for the most part.

Re: new syscalls

[Andrew Morton]

Davide Libenzi <davidel@xmailserver.org> wrote:
>
> On Wed, 18 Jan 2006, Andrew Morton wrote:
> 
> > "Luck, Tony" <tony.luck@intel.com> wrote:
> >>
> >>> Various new syscalls which are trickling their way Linuswards and will need
> >>> testing and wiring up once they get there:
> >>
> >> Are there test-suites for any/all of these new calls?
> >>
> >
> > That's an excellent point.
> >
> > Guys, architecture maintainers need test suites to verify that the syscalls
> > actually work as they wire them up.  Please share.  A stable URL would be
> > preferred - something which can go into the changlog or conceivably into
> > the kernel source.
> 
> How sophisticated this has to be?

umm

- test basic functionality

- test things which should fail: syscall arguments out-of-bounds,
  negative syscall args, etc.  invalid fd.  valid fd but for the wrong type
  of file.

- if you can, something which will test the 32bit->64bit->32bit
  conversions which 32-bit userspace on 64-bit kernel needs to do.

> And what's the expected ETA?

Well the patch isn't even in -mm yet and you still have an argument to win ;)

Once it's been in -mm for a week or so it'd be nice to have a little test
app please.


> epoll_pwait is really a wrapper around epoll_wait (that did not change at 
> all), so the test in this case should just make sure that the signal 
> behaviour is the one expected.

Re: new syscalls

[Andi Kleen]

On Thursday 19 January 2006 02:19, Christoph Lameter wrote:
> On Wed, 18 Jan 2006, Andrew Morton wrote:
> > Guys, architecture maintainers need test suites to verify that the
> > syscalls actually work as they wire them up.  Please share.  A stable URL
> > would be preferred - something which can go into the changlog or
> > conceivably into the kernel source.
>
> sys_migrate_pages will be supported by libnuma in Andi Kleen's numactl
> package. There is no URL that I am aware of.

The home is ftp://ftp.suse.com/pub/people/ak/numa/

The main server is a bit slow, best use a mirror like
ftp://ftp.gwde.de/pub/linux/suse/ftp.suse.com/people/ak/numa

But I haven't pushed out a  update with Christoph's patch yet
(will do ASAP)

And of course you'll need a CONFIG_NUMA=y multi node system to actually try 
it, that probably rules it out on most architectures. On the others 
it will always -ENOSYS.

-Andi

Re: new syscalls

[David Woodhouse]

On Wed, 2006-01-18 at 17:26 -0800, David S. Miller wrote:
> From: Davide Libenzi <davidel@xmailserver.org>
> Date: Wed, 18 Jan 2006 17:17:20 -0800 (PST)
> 
> > How sophisticated this has to be? And what's the expected ETA? The 
> > epoll_pwait is really a wrapper around epoll_wait (that did not change at 
> > all), so the test in this case should just make sure that the signal 
> > behaviour is the one expected.
> 
> Something along the lines of a smoke test is probably sufficient.
> The platform folks just want to make sure they wired up the
> syscall tables correctly, for the most part.

In the case of syscalls using TIF_RESTORE_SIGMASK we want to make sure
the architecture maintainer got a little bit more right than that -- but
the sigmasking.c test case I posted earlier ought to be mostly
sufficient for testing that, if the arch also switches to the generic
sys_rt_sigsuspend().

-- 
dwmw2

Re: new syscalls

[JANAK DESAI]

Andrew Morton wrote:

>"Luck, Tony" <tony.luck@intel.com> wrote:
>  
>
>>>Various new syscalls which are trickling their way Linuswards and will need
>>>testing and wiring up once they get there:
>>>      
>>>
>>Are there test-suites for any/all of these new calls?
>>
>>    
>>
>
>That's an excellent point.
>
>Guys, architecture maintainers need test suites to verify that the syscalls
>actually work as they wire them up.  Please share.  A stable URL would be
>preferred - something which can go into the changlog or conceivably into
>the kernel source.
>
>Thanks.
>
>  
>
Hi Andrew,

I have to finish up a couple of things today, but will post the unshare 
test by tomorrow.

Thanks.

-Janak

Re: new syscalls

[Davide Libenzi]

On Wed, 18 Jan 2006, Andrew Morton wrote:

> Davide Libenzi <davidel@xmailserver.org> wrote:
>>
>> How sophisticated this has to be?
>
> umm
>
> - test basic functionality
>
> - test things which should fail: syscall arguments out-of-bounds,
>  negative syscall args, etc.  invalid fd.  valid fd but for the wrong type
>  of file.
>
> - if you can, something which will test the 32bit->64bit->32bit
>  conversions which 32-bit userspace on 64-bit kernel needs to do.

Somehow, I knew I didn't have to ask :)



- Davide

Re: new syscalls

[Davide Libenzi]

On Thu, 19 Jan 2006, Davide Libenzi wrote:

> On Wed, 18 Jan 2006, Andrew Morton wrote:
>
>> Davide Libenzi <davidel@xmailserver.org> wrote:
>>> 
>>> How sophisticated this has to be?
>> 
>> umm
>> 
>> - test basic functionality
>> 
>> - test things which should fail: syscall arguments out-of-bounds,
>>  negative syscall args, etc.  invalid fd.  valid fd but for the wrong type
>>  of file.
>> 
>> - if you can, something which will test the 32bit->64bit->32bit
>>  conversions which 32-bit userspace on 64-bit kernel needs to do.
>
> Somehow, I knew I didn't have to ask :)

Here you go:

http://www.xmailserver.org/epoll_pwait_test.c

There is a problem with the test we have inside the kernel about the size 
of sigset_t, from the kernel (8 bytes on i386) and the glibc one (128 bytes):

         if (sigmask) {
                 if (sigsetsize != sizeof(sigset_t))
                         return -EINVAL;
                 if (copy_from_user(&ksigmask, sigmask, sizeof(ksigmask)))
                         return -EFAULT;
                 sigdelsetmask(&ksigmask, sigmask(SIGKILL) | sigmask(SIGSTOP));
                 sigprocmask(SIG_SETMASK, &ksigmask, &sigsaved);
         }

ATM I left the check and I bolt 8 inside the test software, but this needs 
nicer handling. Typical run will show:

davide@debstar:~/tmp$ ./epoll_pwait_test
Testing from wrong epfd type            : OK
Testing from closed epfd type           : OK
Testing numevents <= 0                  : OK
Testing numevents too big               : OK
Testing wrong sigmask size              : OK
Testing getevents when none available   : OK
Testing getevents when one available    : OK
Testing signal when blocked             : OK
Testing signal when non blocked         : OK




- Davide

Re: new syscalls

[Ulrich Drepper]

[-- Attachment #1: Type: text/plain, Size: 342 bytes --]

Davide Libenzi wrote:
> There is a problem with the test we have inside the kernel about the
> size of sigset_t, from the kernel (8 bytes on i386) and the glibc one
> (128 bytes):

Not really a problem.  We always pass down _NSIG/8 as the size.

-- 
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 251 bytes --]

Re: new syscalls

[Davide Libenzi]

On Thu, 19 Jan 2006, Ulrich Drepper wrote:

> Davide Libenzi wrote:
>> There is a problem with the test we have inside the kernel about the
>> size of sigset_t, from the kernel (8 bytes on i386) and the glibc one
>> (128 bytes):
>
> Not really a problem.  We always pass down _NSIG/8 as the size.

Ok, I use _NSIG/8 inside the test software now.



- Davide

Re: new syscalls

[Heiko Carstens]

> > Verious new syscalls which are trickling their way Linuswards and will need
> > testing and wiring up once they get there:
> > 
> > sys_migrate_pages
> > 	Merged
> 
> #define __NR_migrate_pages      256
> __SYSCALL(__NR_migrate_pages, sys_migrate_pages)
> 
> 
> We first have to wait if everybody survived crossing the 8bit boundary @)

Any reason why arch/x86_64/ia32/ia32entry.S has sys_futimesat in its syscall
table instead of compat_sys_futimesat? Looks wrong to me...

Thanks,
Heiko

Re: new syscalls

[Andi Kleen]

On Friday 20 January 2006 10:43, Heiko Carstens wrote:
> > > Verious new syscalls which are trickling their way Linuswards and will need
> > > testing and wiring up once they get there:
> > > 
> > > sys_migrate_pages
> > > 	Merged
> > 
> > #define __NR_migrate_pages      256
> > __SYSCALL(__NR_migrate_pages, sys_migrate_pages)
> > 
> > 
> > We first have to wait if everybody survived crossing the 8bit boundary @)
> 
> Any reason why arch/x86_64/ia32/ia32entry.S has sys_futimesat in its syscall
> table instead of compat_sys_futimesat? Looks wrong to me...

Yes, seems wrong. Thanks for catching. I will submit a patch.

-Andi

Re: new syscalls

[David S. Miller]

From: Heiko Carstens <heiko.carstens@de.ibm.com>
Date: Fri, 20 Jan 2006 10:43:09 +0100

> Any reason why arch/x86_64/ia32/ia32entry.S has sys_futimesat in its syscall
> table instead of compat_sys_futimesat? Looks wrong to me...

I made the same mistake on sparc64 (because I used the x86_64 code as
a guide), thanks for pointing it out :)

Re: new syscalls

[Andi Kleen]

On Wednesday 18 January 2006 22:53, David Woodhouse wrote:
> On Wed, 2006-01-18 at 13:36 -0800, Andrew Morton wrote:
> > That's an excellent point.
> > 
> > Guys, architecture maintainers need test suites to verify that the syscalls
> > actually work as they wire them up.  Please share.  A stable URL would be
> > preferred - something which can go into the changlog or conceivably into
> > the kernel source.
> 
> http://david.woodhou.se/sigmasking.c is a good one for testing
> sigsuspend after hooking up TIF_RESTORE_SIGMASK stuff and the generic
> sys_rt_sigsuspend(), although it's not actually my code. There were a
> bunch of other tests in a tarball with that (on linux-arch a year or two
> ago) which arch maintainers should also use occasionally when they play
> with signal or ptrace code.

Are you sure it's even working? I implemented TIF_RESTORE_SIGMASK
for x86-64 and the SEGV part of the test fails. But it even fails
without any of my patches applied, so if it's really broken
it was always like this. I suspect i would have heard if such
a fundamental thing was really broken on x86-64.

===== Test 2: signal SIGALRM while waiting in nanosleep() =====

Unblocking all signals
Calling alarm(2)
Calling nanosleep( 20s)
After return from nanosleep(), blocked all signals, saved returned mask
OK: nanosleep(): Interrupted system call
OK: sighandler for SIGALRM (14) didn't run (invalid stack)
OK: SIGSEGV is unmasked at end of test
ERROR: sighandler for SIGSEGV (11) didn't run
OK: SIGUSR1 is unmasked
OK: SIGUSR2 is unmasked
SIGSEGV is pending at end of test, unblocking it now ...
Signal 11 caught
... done

-Andi

Re: new syscalls

[David S. Miller]

From: Andi Kleen <ak@suse.de>
Date: Fri, 20 Jan 2006 11:04:50 +0100

> Are you sure it's even working? I implemented TIF_RESTORE_SIGMASK
> for x86-64 and the SEGV part of the test fails. But it even fails
> without any of my patches applied, so if it's really broken
> it was always like this. I suspect i would have heard if such
> a fundamental thing was really broken on x86-64.

It fails similarly on sparc.

Signal stacks and SA_ONSTACK are not a very well tested feature, and I
think that is playing a part in this.

Re: new syscalls

[JANAK DESAI]

Andrew Morton wrote:

>"Luck, Tony" <tony.luck@intel.com> wrote:
>  
>
>>>Various new syscalls which are trickling their way Linuswards and will need
>>>testing and wiring up once they get there:
>>>      
>>>
>>Are there test-suites for any/all of these new calls?
>>
>>    
>>
>
>That's an excellent point.
>
>Guys, architecture maintainers need test suites to verify that the syscalls
>actually work as they wire them up.  Please share.  A stable URL would be
>preferred - something which can go into the changlog or conceivably into
>the kernel source.
>
>Thanks.
>
>  
>
Hi Andrew,

I apologize for not sending out the unshare test program by Friday as I had
promised. I got distracted with a strange problem with rmdir. Steve Grubb
first noticed it while testing the pam module that uses unshare. I was 
able to
reproduce the problem with one of my unshare testcases as well. Initially
I suspected the unshare code, however the problem appears to be in the
base code. I am doing some additional debugging now, so I can narrow
down the release/patch that introduced the problem. Once I gather useful
information, I will email Ram Pai and Al Viro with the appropriate
information (hopefully by tonight) and a small test program that can
reproduce the problem. My new ETA for unshare test is by tomorrow
night.

Thanks.

-Janak

Re: new syscalls

[David Woodhouse]

On Fri, 2006-01-20 at 02:15 -0800, David S. Miller wrote:
> From: Andi Kleen <ak@suse.de>
> Date: Fri, 20 Jan 2006 11:04:50 +0100
> 
> > Are you sure it's even working? I implemented TIF_RESTORE_SIGMASK
> > for x86-64 and the SEGV part of the test fails. But it even fails
> > without any of my patches applied, so if it's really broken
> > it was always like this. I suspect i would have heard if such
> > a fundamental thing was really broken on x86-64.
> 
> It fails similarly on sparc.
> 
> Signal stacks and SA_ONSTACK are not a very well tested feature, and I
> think that is playing a part in this.

Indeed -- it's a fairly esoteric test case. It was part of a tarball of
similar tests posted by either Bodo Stroesser or Jeff Dike to
linux-arch, probably some time in or around 2003, when UML tripped over
the bug. I fixed it on PPC and PPC64, and I thought it got fixed on
other architectures at the same time.

I'd be more specific about the archive reference, but unfortunately the
PC with my archives on it is refusing to boot at the moment so I can't
be more helpful until I get home from LCA. But when I wanted this, I was
able to find it by grepping my archives starting from little more -- it
shouldn't be hard to find.

-- 
dwmw2

Re: new syscalls

[Heiko Carstens]

> > Are you sure it's even working? I implemented TIF_RESTORE_SIGMASK
> > for x86-64 and the SEGV part of the test fails. But it even fails
> > without any of my patches applied, so if it's really broken
> > it was always like this. I suspect i would have heard if such
> > a fundamental thing was really broken on x86-64.
> 
> It fails similarly on sparc.
> 
> Signal stacks and SA_ONSTACK are not a very well tested feature, and I
> think that is playing a part in this.

Failed with a lot of errors on s390 too. Strange enough it works well
after if implemented TIF_RESTORE_SIGMASK. Worse things could happen :)

Heiko

Re: new syscalls

[JANAK DESAI]

Andrew Morton wrote:

>"Luck, Tony" <tony.luck@intel.com> wrote:
>  
>
>>>Various new syscalls which are trickling their way Linuswards and will need
>>>testing and wiring up once they get there:
>>>      
>>>
>>Are there test-suites for any/all of these new calls?
>>
>>    
>>
>
>That's an excellent point.
>
>Guys, architecture maintainers need test suites to verify that the syscalls
>actually work as they wire them up.  Please share.  A stable URL would be
>preferred - something which can go into the changlog or conceivably into
>the kernel source.
>
>Thanks.
>
>  
>
Here is the link to a program for testing unshare system call.

http://prdownloads.sourceforge.net/audit/unshare_test.c?download

Please note that because of a problem in rmdir associated with
bind mounts and clone with CLONE_NEWNS, the test fails while
trying to remove temporary test directory. You can remove that
temporary directory by doing rmdir, twice, from the command line.
The first will fail with EBUSY, but the second will succeed. I have
reported the problem to Ram Pai and Al Viro with a small program
which reproduces the problem. Al told us yesterday that he will
be looking at the problem soon. I have tried multiple rmdirs from
the unshare_test program itself, but for some reason that is not
working. Doing two rmdirs from command line does seem to
remove the directory.

Please let me know if you have trouble downloading/running
the program.

-Janak