[Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Peter Maydell]

Unfortunately it turns out that libseccomp 2.2 still does not work
correctly on non-x86 architectures; return to the previous configure
setup of insisting on libseccomp 2.1 or better and i386/x86_64 and
disabling seccomp support in all other situations.

This reverts the two commits:
 * "seccomp: libseccomp version varying according to arch"
   (commit 896848f0d3e2393905845ef2b244bb2601f9df0c)
 * "seccomp: update libseccomp version and remove arch restriction"
   (commit 8e27fc200457e3f2473d0069263774d4ba17bd85)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
With QEMU 2.3 release so close this seems the safest approach.

 configure | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/configure b/configure
index 8b673fd..6969f6f 100755
--- a/configure
+++ b/configure
@@ -1848,19 +1848,14 @@ fi
 # libseccomp check
 
 if test "$seccomp" != "no" ; then
-    if $pkg_config --atleast-version=2.2.0 libseccomp ||
-        (test "$cpu" = "i386" || test "$cpu" = "x86_64" &&
-        $pkg_config --atleast-version=2.1.1 libseccomp); then
+    if test "$cpu" = "i386" || test "$cpu" = "x86_64" &&
+        $pkg_config --atleast-version=2.1.1 libseccomp; then
         libs_softmmu="$libs_softmmu `$pkg_config --libs libseccomp`"
         QEMU_CFLAGS="$QEMU_CFLAGS `$pkg_config --cflags libseccomp`"
 	seccomp="yes"
     else
 	if test "$seccomp" = "yes"; then
-        if test "$cpu" = "i386" || test "$cpu" = "x86_64"; then
             feature_not_found "libseccomp" "Install libseccomp devel >= 2.1.1"
-        else
-            feature_not_found "libseccomp" "Install libseccomp devel >= 2.2.0"
-        fi
 	fi
 	seccomp="no"
     fi
-- 
1.9.1

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Andrew Jones]

On Fri, Apr 10, 2015 at 01:58:01PM +0100, Peter Maydell wrote:
> Unfortunately it turns out that libseccomp 2.2 still does not work
> correctly on non-x86 architectures; return to the previous configure
> setup of insisting on libseccomp 2.1 or better and i386/x86_64 and
> disabling seccomp support in all other situations.
> 
> This reverts the two commits:
>  * "seccomp: libseccomp version varying according to arch"
>    (commit 896848f0d3e2393905845ef2b244bb2601f9df0c)
>  * "seccomp: update libseccomp version and remove arch restriction"
>    (commit 8e27fc200457e3f2473d0069263774d4ba17bd85)
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> With QEMU 2.3 release so close this seems the safest approach.
> 
>  configure | 9 ++-------
>  1 file changed, 2 insertions(+), 7 deletions(-)
> 
> diff --git a/configure b/configure
> index 8b673fd..6969f6f 100755
> --- a/configure
> +++ b/configure
> @@ -1848,19 +1848,14 @@ fi
>  # libseccomp check
>  
>  if test "$seccomp" != "no" ; then
> -    if $pkg_config --atleast-version=2.2.0 libseccomp ||
> -        (test "$cpu" = "i386" || test "$cpu" = "x86_64" &&
> -        $pkg_config --atleast-version=2.1.1 libseccomp); then
> +    if test "$cpu" = "i386" || test "$cpu" = "x86_64" &&
> +        $pkg_config --atleast-version=2.1.1 libseccomp; then
>          libs_softmmu="$libs_softmmu `$pkg_config --libs libseccomp`"
>          QEMU_CFLAGS="$QEMU_CFLAGS `$pkg_config --cflags libseccomp`"
>  	seccomp="yes"
>      else
>  	if test "$seccomp" = "yes"; then
> -        if test "$cpu" = "i386" || test "$cpu" = "x86_64"; then
>              feature_not_found "libseccomp" "Install libseccomp devel >= 2.1.1"
> -        else
> -            feature_not_found "libseccomp" "Install libseccomp devel >= 2.2.0"
> -        fi
>  	fi
>  	seccomp="no"
>      fi
> -- 
> 1.9.1
> 
> 
>

Hi Peter,

Can we now revert this revert, along with bumping the non-x86 arch
atleast-version to v2.2.1?

Thanks,
drew

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Peter Maydell]

On 16 June 2015 at 14:12, Andrew Jones <drjones@redhat.com> wrote:
> Can we now revert this revert, along with bumping the non-x86 arch
> atleast-version to v2.2.1

Probably. I suggest you submit a patch and test it on the
relevant architectures and seccomp versions.

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Andrew Jones]

On Tue, Jun 16, 2015 at 02:16:03PM +0100, Peter Maydell wrote:
> On 16 June 2015 at 14:12, Andrew Jones <drjones@redhat.com> wrote:
> > Can we now revert this revert, along with bumping the non-x86 arch
> > atleast-version to v2.2.1
> 
> Probably. I suggest you submit a patch and test it on the
> relevant architectures and seccomp versions.
>

I don't see any problems with the light testing (booting a guest)
I've done on my mustang, but AArch64 worked with libseccomp 2.2.0
too. So I dusted off my Midway (updated to Fedora 21 that has
libseccomp 2.2.1 packaged), and gave it a try, but unfortunately
it still doesn't work...

I found that we needed to add another syscall to the whitelist;
the arm-private 'cacheflush', as it's used by __builtin___clear_cache.
And, from libseccomp's git history it appears that syscall is known

commit a710a2d246bdc73ba77e3ff5624e790688cc51fd
Author: Paul Moore <pmoore@redhat.com>
Date:   Wed May 6 12:05:45 2015 -0400

    arm: add some missing syscalls
    
    Add the following syscalls to the ARM arch/ABI and update the syscall
    validation script.
    
     * breakpoint()
     * cacheflush()
     * usr26()
     * usr32()
     * set_tls()
    
    Reported-by: Purcareata Bogdan <b43198@freescale.com>
    Signed-off-by: Paul Moore <pmoore@redhat.com>


And also appears to be in 2.2.1
$ git describe a710a2d246bdc73ba77e3ff5624e790688cc51fd
v2.2.0-10-ga710a2d246bdc

However the qemu thread that makes that syscall still dies, even
with this patch

diff --git a/qemu-seccomp.c b/qemu-seccomp.c
index f9de0d3390feb..33644a4e3c3d3 100644
--- a/qemu-seccomp.c
+++ b/qemu-seccomp.c
@@ -237,7 +237,8 @@ static const struct QemuSeccompSyscall
seccomp_whitelist[] = {
     { SCMP_SYS(fadvise64), 240 },
     { SCMP_SYS(inotify_init1), 240 },
     { SCMP_SYS(inotify_add_watch), 240 },
-    { SCMP_SYS(mbind), 240 }
+    { SCMP_SYS(mbind), 240 },
+    { SCMP_SYS(cacheflush), 240 },
 };
 
 int seccomp_start(void)


Paul, can you help me figure out what I'm missing?

Thanks,
drew

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Paul Moore]

On Friday, June 26, 2015 06:03:18 PM Andrew Jones wrote:
> On Tue, Jun 16, 2015 at 02:16:03PM +0100, Peter Maydell wrote:
> > On 16 June 2015 at 14:12, Andrew Jones <drjones@redhat.com> wrote:
> > > Can we now revert this revert, along with bumping the non-x86 arch
> > > atleast-version to v2.2.1
> > 
> > Probably. I suggest you submit a patch and test it on the
> > relevant architectures and seccomp versions.
> 
> I don't see any problems with the light testing (booting a guest)
> I've done on my mustang, but AArch64 worked with libseccomp 2.2.0
> too. So I dusted off my Midway (updated to Fedora 21 that has
> libseccomp 2.2.1 packaged), and gave it a try, but unfortunately
> it still doesn't work...
> 
> I found that we needed to add another syscall to the whitelist;
> the arm-private 'cacheflush', as it's used by __builtin___clear_cache.
> And, from libseccomp's git history it appears that syscall is known
> 
> commit a710a2d246bdc73ba77e3ff5624e790688cc51fd
> Author: Paul Moore <pmoore@redhat.com>
> Date:   Wed May 6 12:05:45 2015 -0400
> 
>     arm: add some missing syscalls
> 
>     Add the following syscalls to the ARM arch/ABI and update the syscall
>     validation script.
> 
>      * breakpoint()
>      * cacheflush()
>      * usr26()
>      * usr32()
>      * set_tls()
> 
>     Reported-by: Purcareata Bogdan <b43198@freescale.com>
>     Signed-off-by: Paul Moore <pmoore@redhat.com>
> 
> 
> And also appears to be in 2.2.1
> $ git describe a710a2d246bdc73ba77e3ff5624e790688cc51fd
> v2.2.0-10-ga710a2d246bdc
> 
> However the qemu thread that makes that syscall still dies, even
> with this patch
> 
> diff --git a/qemu-seccomp.c b/qemu-seccomp.c
> index f9de0d3390feb..33644a4e3c3d3 100644
> --- a/qemu-seccomp.c
> +++ b/qemu-seccomp.c
> @@ -237,7 +237,8 @@ static const struct QemuSeccompSyscall
> seccomp_whitelist[] = {
>      { SCMP_SYS(fadvise64), 240 },
>      { SCMP_SYS(inotify_init1), 240 },
>      { SCMP_SYS(inotify_add_watch), 240 },
> -    { SCMP_SYS(mbind), 240 }
> +    { SCMP_SYS(mbind), 240 },
> +    { SCMP_SYS(cacheflush), 240 },
>  };
> 
>  int seccomp_start(void)
> 
> 
> Paul, can you help me figure out what I'm missing?

Perhaps a stupid question, but you did verify that it is cacheflush that is 
causing the problem?  The seccomp filter code will emit a message to syslog or 
the audit log, depending on your configuration, with the syscall number.

 #./tools/scmp_sys_resolver -a arm cacheflush
 983042
 #./tools/scmp_sys_resolver -a arm 983042

-- 
paul moore
security @ redhat

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Andrew Jones]

On Fri, Jun 26, 2015 at 04:26:22PM -0400, Paul Moore wrote:
> On Friday, June 26, 2015 06:03:18 PM Andrew Jones wrote:
> > On Tue, Jun 16, 2015 at 02:16:03PM +0100, Peter Maydell wrote:
> > > On 16 June 2015 at 14:12, Andrew Jones <drjones@redhat.com> wrote:
> > > > Can we now revert this revert, along with bumping the non-x86 arch
> > > > atleast-version to v2.2.1
> > > 
> > > Probably. I suggest you submit a patch and test it on the
> > > relevant architectures and seccomp versions.
> > 
> > I don't see any problems with the light testing (booting a guest)
> > I've done on my mustang, but AArch64 worked with libseccomp 2.2.0
> > too. So I dusted off my Midway (updated to Fedora 21 that has
> > libseccomp 2.2.1 packaged), and gave it a try, but unfortunately
> > it still doesn't work...
> > 
> > I found that we needed to add another syscall to the whitelist;
> > the arm-private 'cacheflush', as it's used by __builtin___clear_cache.
> > And, from libseccomp's git history it appears that syscall is known
> > 
> > commit a710a2d246bdc73ba77e3ff5624e790688cc51fd
> > Author: Paul Moore <pmoore@redhat.com>
> > Date:   Wed May 6 12:05:45 2015 -0400
> > 
> >     arm: add some missing syscalls
> > 
> >     Add the following syscalls to the ARM arch/ABI and update the syscall
> >     validation script.
> > 
> >      * breakpoint()
> >      * cacheflush()
> >      * usr26()
> >      * usr32()
> >      * set_tls()
> > 
> >     Reported-by: Purcareata Bogdan <b43198@freescale.com>
> >     Signed-off-by: Paul Moore <pmoore@redhat.com>
> > 
> > 
> > And also appears to be in 2.2.1
> > $ git describe a710a2d246bdc73ba77e3ff5624e790688cc51fd
> > v2.2.0-10-ga710a2d246bdc
> > 
> > However the qemu thread that makes that syscall still dies, even
> > with this patch
> > 
> > diff --git a/qemu-seccomp.c b/qemu-seccomp.c
> > index f9de0d3390feb..33644a4e3c3d3 100644
> > --- a/qemu-seccomp.c
> > +++ b/qemu-seccomp.c
> > @@ -237,7 +237,8 @@ static const struct QemuSeccompSyscall
> > seccomp_whitelist[] = {
> >      { SCMP_SYS(fadvise64), 240 },
> >      { SCMP_SYS(inotify_init1), 240 },
> >      { SCMP_SYS(inotify_add_watch), 240 },
> > -    { SCMP_SYS(mbind), 240 }
> > +    { SCMP_SYS(mbind), 240 },
> > +    { SCMP_SYS(cacheflush), 240 },
> >  };
> > 
> >  int seccomp_start(void)
> > 
> > 
> > Paul, can you help me figure out what I'm missing?
> 
> Perhaps a stupid question, but you did verify that it is cacheflush that is 
> causing the problem?  The seccomp filter code will emit a message to syslog or 
> the audit log, depending on your configuration, with the syscall number.
> 
>  #./tools/scmp_sys_resolver -a arm cacheflush
>  983042
>  #./tools/scmp_sys_resolver -a arm 983042

I hadn't before (didn't know about the logging). I had determined the
problem by running qemu in gdb. I just checked now though and confirmed
it

type=SECCOMP msg=audit(1435563996.731:2032): auid=1001 uid=1001 gid=1001
ses=157 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
pid=27059 comm="qemu-system-arm"
exe="/home/drjones/code/qemu/arm-softmmu/qemu-system-arm" sig=31
arch=40000028 syscall=983042 compat=0 ip=0xb6b43164 code=0x0

This log was generated even with the above patch applied to qemu.

Thanks,
drew

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Paul Moore]

On Monday, June 29, 2015 09:50:17 AM Andrew Jones wrote:
> On Fri, Jun 26, 2015 at 04:26:22PM -0400, Paul Moore wrote:
> > Perhaps a stupid question, but you did verify that it is cacheflush that
> > is causing the problem?  The seccomp filter code will emit a message to
> > syslog or the audit log, depending on your configuration, with the
> > syscall number.
> >
> >  #./tools/scmp_sys_resolver -a arm cacheflush
> >  983042
> >  #./tools/scmp_sys_resolver -a arm 983042
> 
> I hadn't before (didn't know about the logging). I had determined the
> problem by running qemu in gdb. I just checked now though and confirmed
> it
> 
> type=SECCOMP msg=audit(1435563996.731:2032): auid=1001 uid=1001 gid=1001
> ses=157 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> pid=27059 comm="qemu-system-arm"
> exe="/home/drjones/code/qemu/arm-softmmu/qemu-system-arm" sig=31
> arch=40000028 syscall=983042 compat=0 ip=0xb6b43164 code=0x0
> 
> This log was generated even with the above patch applied to qemu.

The only thing that comes to mind quickly is that the cacheflush() call is 
being done by a thread that was created before the seccomp filter was loaded 
into the kernel; although I believe you said you already checked that.

If you are using a recent kernel and libseccomp you can try enabling the 
SCMP_FLTATR_CTL_TSYNC attribute to apply the filter to all running threads in 
the process.

	rc = seccomp_attr_set(ctx, SCMP_FLTATR_CTL_TSYNC, 1);
	if (rc)
		/* error */

... although that may have unintended consequences since threads which were 
never filtered are not getting caught up in the seccomp filter.  Although, the 
current QEMU seccomp filter is so permissive it might not be a real concern.

Anyway, (double) check the thread creation and seccomp_load() ordering.

-- 
paul moore
security @ redhat

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Andrew Jones]

On Mon, Jun 29, 2015 at 10:53:14AM -0400, Paul Moore wrote:
> On Monday, June 29, 2015 09:50:17 AM Andrew Jones wrote:
> > On Fri, Jun 26, 2015 at 04:26:22PM -0400, Paul Moore wrote:
> > > Perhaps a stupid question, but you did verify that it is cacheflush that
> > > is causing the problem?  The seccomp filter code will emit a message to
> > > syslog or the audit log, depending on your configuration, with the
> > > syscall number.
> > >
> > >  #./tools/scmp_sys_resolver -a arm cacheflush
> > >  983042
> > >  #./tools/scmp_sys_resolver -a arm 983042
> > 
> > I hadn't before (didn't know about the logging). I had determined the
> > problem by running qemu in gdb. I just checked now though and confirmed
> > it
> > 
> > type=SECCOMP msg=audit(1435563996.731:2032): auid=1001 uid=1001 gid=1001
> > ses=157 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> > pid=27059 comm="qemu-system-arm"
> > exe="/home/drjones/code/qemu/arm-softmmu/qemu-system-arm" sig=31
> > arch=40000028 syscall=983042 compat=0 ip=0xb6b43164 code=0x0
> > 
> > This log was generated even with the above patch applied to qemu.
> 
> The only thing that comes to mind quickly is that the cacheflush() call is 
> being done by a thread that was created before the seccomp filter was loaded 
> into the kernel; although I believe you said you already checked that.

Nope, I hadn't, but I have now. I went back to my friend gdb and set a couple
breakpoints

Breakpoint 1, seccomp_start () at qemu-seccomp.c:246
246	    int rc = 0;
(gdb) info threads 
  Id   Target Id         Frame 
  2    Thread 0xb6a81130 (LWP 11351) "qemu-system-arm" 0xb6beebe0 in nanosleep ()
    at ../sysdeps/unix/syscall-template.S:81
* 1    Thread 0xb6a83000 (LWP 11348) "qemu-system-arm" seccomp_start () at qemu-seccomp.c:246
(gdb) c
Continuing.

Breakpoint 2, __clear_cache () at ../../../libgcc/config/arm/lib1funcs.S:1348
1348		movw	r7, #2
(gdb) info threads 
  Id   Target Id         Frame 
  2    Thread 0xb6a81130 (LWP 11351) "qemu-system-arm" 0xb6beebe0 in nanosleep ()
    at ../sysdeps/unix/syscall-template.S:81
* 1    Thread 0xb6a83000 (LWP 11348) "qemu-system-arm" __clear_cache ()
    at ../../../libgcc/config/arm/lib1funcs.S:1348
(gdb) s
1349		movt	r7, #0xf
(gdb) 
1354		mov	r2, #0
(gdb) 
1355		swi	0
(gdb) 
[Thread 0xb6a83000 (LWP 11348) exited]
No unwaited-for children left.


So we're calling __clear_cache from the same thread that called
seccomp_start, and that thread dies the moment it calls the syscall.
No other threads except id(2) at this time, which appears to be
something created by __libc_start_main before main() runs.

> 
> If you are using a recent kernel and libseccomp you can try enabling the 
> SCMP_FLTATR_CTL_TSYNC attribute to apply the filter to all running threads in 
> the process.
> 
> 	rc = seccomp_attr_set(ctx, SCMP_FLTATR_CTL_TSYNC, 1);
> 	if (rc)
> 		/* error */

I tried this, but it error'ed out with rc == -95 (EOPNOTSUPP ?)
My kernel version is 4.0.5-200.fc21.armv7hl+lpae

Thanks,
drew

> 
> ... although that may have unintended consequences since threads which were 
> never filtered are not getting caught up in the seccomp filter.  Although, the 
> current QEMU seccomp filter is so permissive it might not be a real concern.
> 
> Anyway, (double) check the thread creation and seccomp_load() ordering.
> 
> -- 
> paul moore
> security @ redhat
> 
> 

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Paul Moore]

On Monday, June 29, 2015 07:47:29 PM Andrew Jones wrote:
> On Mon, Jun 29, 2015 at 10:53:14AM -0400, Paul Moore wrote:
> > On Monday, June 29, 2015 09:50:17 AM Andrew Jones wrote:
> > > On Fri, Jun 26, 2015 at 04:26:22PM -0400, Paul Moore wrote:
> > > > Perhaps a stupid question, but you did verify that it is cacheflush
> > > > that
> > > > is causing the problem?  The seccomp filter code will emit a message
> > > > to
> > > > syslog or the audit log, depending on your configuration, with the
> > > > syscall number.
> > > > 
> > > >  #./tools/scmp_sys_resolver -a arm cacheflush
> > > >  983042
> > > >  #./tools/scmp_sys_resolver -a arm 983042
> > > 
> > > I hadn't before (didn't know about the logging). I had determined the
> > > problem by running qemu in gdb. I just checked now though and confirmed
> > > it
> > > 
> > > type=SECCOMP msg=audit(1435563996.731:2032): auid=1001 uid=1001 gid=1001
> > > ses=157 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> > > pid=27059 comm="qemu-system-arm"
> > > exe="/home/drjones/code/qemu/arm-softmmu/qemu-system-arm" sig=31
> > > arch=40000028 syscall=983042 compat=0 ip=0xb6b43164 code=0x0
> > > 
> > > This log was generated even with the above patch applied to qemu.
> > 
> > The only thing that comes to mind quickly is that the cacheflush() call is
> > being done by a thread that was created before the seccomp filter was
> > loaded into the kernel; although I believe you said you already checked
> > that.
>
> Nope, I hadn't, but I have now ...

Actually, never mind on that, I was being stupid.  If it was a different 
thread it wouldn't be impacted by the seccomp filter at all ...

> ... So we're calling __clear_cache from the same thread that called
> seccomp_start, and that thread dies the moment it calls the syscall.
> No other threads except id(2) at this time, which appears to be
> something created by __libc_start_main before main() runs.

Hmm, so either the kernel is screwing up with the seccomp filter for this 
particular syscall (unlikely) or libseccomp is screwing up the filter creation 
(more likely).  I don't have an ARM system handy at the moment, but could you 
use the seccomp_export_pfc() and seccomp_export_bpf() functions to dump the 
PFC/BPF filter code to a file and send it out?

> > If you are using a recent kernel and libseccomp you can try enabling the
> > SCMP_FLTATR_CTL_TSYNC attribute to apply the filter to all running threads
> > in the process.
> > 
> > 	rc = seccomp_attr_set(ctx, SCMP_FLTATR_CTL_TSYNC, 1);
> > 	if (rc)
> > 	
> > 		/* error */
> 
> I tried this, but it error'ed out with rc == -95 (EOPNOTSUPP ?)
> My kernel version is 4.0.5-200.fc21.armv7hl+lpae

That should be a recent enough kernel, but perhaps your version of libseccomp 
was built against an older version of the kernel that didn't have the 
necessary support (and it was disabled at compile time)?

-- 
paul moore
security @ redhat

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Andrew Jones]

[-- Attachment #1: Type: text/plain, Size: 3828 bytes --]

On Mon, Jun 29, 2015 at 04:24:55PM -0400, Paul Moore wrote:
> On Monday, June 29, 2015 07:47:29 PM Andrew Jones wrote:
> > On Mon, Jun 29, 2015 at 10:53:14AM -0400, Paul Moore wrote:
> > > On Monday, June 29, 2015 09:50:17 AM Andrew Jones wrote:
> > > > On Fri, Jun 26, 2015 at 04:26:22PM -0400, Paul Moore wrote:
> > > > > Perhaps a stupid question, but you did verify that it is cacheflush
> > > > > that
> > > > > is causing the problem?  The seccomp filter code will emit a message
> > > > > to
> > > > > syslog or the audit log, depending on your configuration, with the
> > > > > syscall number.
> > > > > 
> > > > >  #./tools/scmp_sys_resolver -a arm cacheflush
> > > > >  983042
> > > > >  #./tools/scmp_sys_resolver -a arm 983042
> > > > 
> > > > I hadn't before (didn't know about the logging). I had determined the
> > > > problem by running qemu in gdb. I just checked now though and confirmed
> > > > it
> > > > 
> > > > type=SECCOMP msg=audit(1435563996.731:2032): auid=1001 uid=1001 gid=1001
> > > > ses=157 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> > > > pid=27059 comm="qemu-system-arm"
> > > > exe="/home/drjones/code/qemu/arm-softmmu/qemu-system-arm" sig=31
> > > > arch=40000028 syscall=983042 compat=0 ip=0xb6b43164 code=0x0
> > > > 
> > > > This log was generated even with the above patch applied to qemu.
> > > 
> > > The only thing that comes to mind quickly is that the cacheflush() call is
> > > being done by a thread that was created before the seccomp filter was
> > > loaded into the kernel; although I believe you said you already checked
> > > that.
> >
> > Nope, I hadn't, but I have now ...
> 
> Actually, never mind on that, I was being stupid.  If it was a different 
> thread it wouldn't be impacted by the seccomp filter at all ...
> 
> > ... So we're calling __clear_cache from the same thread that called
> > seccomp_start, and that thread dies the moment it calls the syscall.
> > No other threads except id(2) at this time, which appears to be
> > something created by __libc_start_main before main() runs.
> 
> Hmm, so either the kernel is screwing up with the seccomp filter for this 
> particular syscall (unlikely) or libseccomp is screwing up the filter creation 
> (more likely).  I don't have an ARM system handy at the moment, but could you 
> use the seccomp_export_pfc() and seccomp_export_bpf() functions to dump the 
> PFC/BPF filter code to a file and send it out?

Attached

> 
> > > If you are using a recent kernel and libseccomp you can try enabling the
> > > SCMP_FLTATR_CTL_TSYNC attribute to apply the filter to all running threads
> > > in the process.
> > > 
> > > 	rc = seccomp_attr_set(ctx, SCMP_FLTATR_CTL_TSYNC, 1);
> > > 	if (rc)
> > > 	
> > > 		/* error */
> > 
> > I tried this, but it error'ed out with rc == -95 (EOPNOTSUPP ?)
> > My kernel version is 4.0.5-200.fc21.armv7hl+lpae
> 
> That should be a recent enough kernel, but perhaps your version of libseccomp 
> was built against an older version of the kernel that didn't have the 
> necessary support (and it was disabled at compile time)?
>

I looked at the pfc file and compared all the syscalls in it vs.  the list
in qemu-seccomp.c. The pfc file is missing cacheflush, and has an 'UNKNOWN'
instead. Also, I think there may be another problem with the filter (or pfc)
generation. Several of the syscalls have weird syscall numbers. For example,
I would expect mmap to be 90, but instead it's -10181.

And, since there was something weird, and not related to cacheflush, in the
arm32 pfc, I decided to check it on my mustang too. The output there gets
"cacheflush" for the name instead of UNKNOWN, but has the same weird
number (-10104) that the midway has. It also has several other weird
numbers. The output from the mustang is in the attached tarball as well.

Thanks,
drew

[-- Attachment #2: seccomp_filters.tar.xz --]
[-- Type: application/x-xz, Size: 4732 bytes --]

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Paul Moore]

On Tuesday, June 30, 2015 10:39:34 AM Andrew Jones wrote:
> On Mon, Jun 29, 2015 at 04:24:55PM -0400, Paul Moore wrote:
> > Hmm, so either the kernel is screwing up with the seccomp filter for this
> > particular syscall (unlikely) or libseccomp is screwing up the filter
> > creation (more likely).  I don't have an ARM system handy at the moment,
> > but could you use the seccomp_export_pfc() and seccomp_export_bpf()
> > functions to dump the PFC/BPF filter code to a file and send it out?
> 
> Attached

Thanks.

> I looked at the pfc file and compared all the syscalls in it vs.  the list
> in qemu-seccomp.c. The pfc file is missing cacheflush, and has an 'UNKNOWN'
> instead.

Yeah, the associated syscall number is showing -10104 which is the pseudo 
syscall number for architectures that don't support cacheflush().  That is 
obviously wrong

> Also, I think there may be another problem with the filter (or pfc)
> generation. Several of the syscalls have weird syscall numbers. For
> example, I would expect mmap to be 90, but instead it's -10181.

That is intentional.  According to the Linux kernel sources mmap() isn't 
defined for 32-bit ARM EABI so you are seeing libseccomp's pseudo syscall 
number for mmap().

> And, since there was something weird, and not related to cacheflush, in the
> arm32 pfc, I decided to check it on my mustang too. The output there gets
> "cacheflush" for the name instead of UNKNOWN, but has the same weird
> number (-10104) that the midway has. It also has several other weird
> numbers. The output from the mustang is in the attached tarball as well.

I would expect aarch64 to have a pseudo syscall number for cacheflush() as 
that syscall isn't defined for 64-bit ARM.  What I don't understand is why 
libseccomp doesn't recognize cacheflush() in this particular case.

I'm starting to wonder if the 32-bit ARM build system didn't have 
__NR_cacheflush defined in the system headers; that might explain some of the 
behavior.  Could you check your system to see if it has __NR_cacheflush 
defined (try /usr/include/asm/unistd.h)?  If it does, could you try rebuilding 
the libseccomp package on your system and seeing if it resolves the problem?

-- 
paul moore
security @ redhat

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Peter Maydell]

On 30 June 2015 at 18:01, Paul Moore <pmoore@redhat.com> wrote:
> I'm starting to wonder if the 32-bit ARM build system didn't have
> __NR_cacheflush defined in the system headers; that might explain some of the
> behavior.  Could you check your system to see if it has __NR_cacheflush
> defined (try /usr/include/asm/unistd.h)?

The constant name is __ARM_NR_cacheflush, not __NR_cacheflush
(all the ARM-specific syscalls are __ARM_NR_*). See
http://lxr.free-electrons.com/source/arch/arm/include/uapi/asm/unistd.h#L418

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Paul Moore]

On Tuesday, June 30, 2015 06:07:40 PM Peter Maydell wrote:
> On 30 June 2015 at 18:01, Paul Moore <pmoore@redhat.com> wrote:
> > I'm starting to wonder if the 32-bit ARM build system didn't have
> > __NR_cacheflush defined in the system headers; that might explain some of
> > the behavior.  Could you check your system to see if it has
> > __NR_cacheflush defined (try /usr/include/asm/unistd.h)?
> 
> The constant name is __ARM_NR_cacheflush, not __NR_cacheflush
> (all the ARM-specific syscalls are __ARM_NR_*). See
> http://lxr.free-electrons.com/source/arch/arm/include/uapi/asm/unistd.h#L418

/me smacks his forehead

Of course it is.  We already work around that in arch-syscall-validate.  D'oh!

Good news though, I think we just found the bug ;)

I'm currently trying to put out another fire in a different project; as soon 
as I've got that done I'll fix this.  However, if somebody wants to play, I'm 
always happy to accept patches :)

-- 
paul moore
security @ redhat

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Andrew Jones]

On Tue, Jun 30, 2015 at 01:18:49PM -0400, Paul Moore wrote:
> On Tuesday, June 30, 2015 06:07:40 PM Peter Maydell wrote:
> > On 30 June 2015 at 18:01, Paul Moore <pmoore@redhat.com> wrote:
> > > I'm starting to wonder if the 32-bit ARM build system didn't have
> > > __NR_cacheflush defined in the system headers; that might explain some of
> > > the behavior.  Could you check your system to see if it has
> > > __NR_cacheflush defined (try /usr/include/asm/unistd.h)?
> > 
> > The constant name is __ARM_NR_cacheflush, not __NR_cacheflush
> > (all the ARM-specific syscalls are __ARM_NR_*). See
> > http://lxr.free-electrons.com/source/arch/arm/include/uapi/asm/unistd.h#L418
> 
> /me smacks his forehead
> 
> Of course it is.  We already work around that in arch-syscall-validate.  D'oh!
> 
> Good news though, I think we just found the bug ;)
> 
> I'm currently trying to put out another fire in a different project; as soon 
> as I've got that done I'll fix this.  However, if somebody wants to play, I'm 
> always happy to accept patches :)

Sent: https://groups.google.com/forum/#!topic/libseccomp/RD9RTmc2Lxo

I'll send the patch for qemu to add cacheflush to the whitelist shortly.

drew

> 
> -- 
> paul moore
> security @ redhat
> 
> 
> 

Re: [Qemu-devel] [PATCH for-2.3] Revert seccomp tests that allow it to be used on non-x86 architectures

[Paul Moore]

On Wednesday, July 01, 2015 02:07:49 PM Andrew Jones wrote:
> On Tue, Jun 30, 2015 at 01:18:49PM -0400, Paul Moore wrote:
> > On Tuesday, June 30, 2015 06:07:40 PM Peter Maydell wrote:
> > > On 30 June 2015 at 18:01, Paul Moore <pmoore@redhat.com> wrote:
> > > > I'm starting to wonder if the 32-bit ARM build system didn't have
> > > > __NR_cacheflush defined in the system headers; that might explain some
> > > > of
> > > > the behavior.  Could you check your system to see if it has
> > > > __NR_cacheflush defined (try /usr/include/asm/unistd.h)?
> > > 
> > > The constant name is __ARM_NR_cacheflush, not __NR_cacheflush
> > > (all the ARM-specific syscalls are __ARM_NR_*). See
> > > http://lxr.free-electrons.com/source/arch/arm/include/uapi/asm/unistd.h#
> > > L418> 
> > /me smacks his forehead
> > 
> > Of course it is.  We already work around that in arch-syscall-validate. 
> > D'oh!
> > 
> > Good news though, I think we just found the bug ;)
> > 
> > I'm currently trying to put out another fire in a different project; as
> > soon as I've got that done I'll fix this.  However, if somebody wants to
> > play, I'm always happy to accept patches :)
> 
> Sent: https://groups.google.com/forum/#!topic/libseccomp/RD9RTmc2Lxo

Applied, thanks.

> I'll send the patch for qemu to add cacheflush to the whitelist shortly.

-- 
paul moore
security @ redhat