syscall problem on Android x86

syscall problem on Android x86

[Akers, Jason B]

Fio hangs when run on the Android x86 emulator.

Tracking through, I found that the last call made is shmget() (in init.c - setup_thread_area()).
I believe that the hang is related to the fio syscall implementation for x86.

os-android.h defines shmget as: syscall(__NR_shmget, __key, __size, __shmflg);

In arch-x86.h __NR_shmget is defined to 29. This was added on April 11th (a415b2cc).

Looking deeper, I see that syscall 29 is actually mapped to pause(). (see SYSCALLS.TXT in bionic/libc) I confirmed that sys_pause() was being called by using a kernel breakpoint. This explains why fio hangs.

Now the question is: What to do about it?
For some reason shmget is only exposed for Android ARM targets even though it is part of the kernel (system.map) for x86.

I see two options:
1. add shmget to SYSCALLS.TXT for x86 and recompile. Remove the hardcoded __NR_shmget from arch-x86.h and put a #error instead to warn others that a kernel patch is necessary for Android x86. 

2. use a "blessed" shared memory allocation method for Android targets (like ashmem / mmap??) Not sure how difficult it would be to make this work with the existing FIO architecture.

Any other ideas / thoughts / feedback / suggestions are appreciated.

Thanks,
Jason Akers

Re: syscall problem on Android x86

[Jens Axboe]

On Fri, Apr 26 2013, Akers, Jason B wrote:
> Fio hangs when run on the Android x86 emulator.
> 
> Tracking through, I found that the last call made is shmget() (in init.c - setup_thread_area()).
> I believe that the hang is related to the fio syscall implementation for x86.
> 
> os-android.h defines shmget as: syscall(__NR_shmget, __key, __size, __shmflg);
> 
> In arch-x86.h __NR_shmget is defined to 29. This was added on April 11th (a415b2cc).
> 
> Looking deeper, I see that syscall 29 is actually mapped to pause(). (see SYSCALLS.TXT in bionic/libc) I confirmed that sys_pause() was being called by using a kernel breakpoint. This explains why fio hangs.
> 
> Now the question is: What to do about it?
> For some reason shmget is only exposed for Android ARM targets even though it is part of the kernel (system.map) for x86.
> 
> I see two options:
> 1. add shmget to SYSCALLS.TXT for x86 and recompile. Remove the hardcoded __NR_shmget from arch-x86.h and put a #error instead to warn others that a kernel patch is necessary for Android x86. 
> 
> 2. use a "blessed" shared memory allocation method for Android targets (like ashmem / mmap??) Not sure how difficult it would be to make this work with the existing FIO architecture.
> 
> Any other ideas / thoughts / feedback / suggestions are appreciated.

So sounds like you are catching the x86 definition of __NR_shmget which
is indeed 29, but you are building on Arm. That sounds pretty weird. The
below patch should wire up the right Arm syscalls, but I think you need
into why it's claiming to be x86 while it is in fact building for arm.


diff --git a/arch/arch-arm.h b/arch/arch-arm.h
index 7cd9502..8a7398e 100644
--- a/arch/arch-arm.h
+++ b/arch/arch-arm.h
@@ -18,6 +18,13 @@
 #define __NR_sys_vmsplice	343
 #endif
 
+#ifndef __NR_shmget
+#define __NR_shmat		305
+#define __NR_shmdt		306
+#define __NR_shmget		307
+#define __NR_shmctl		308
+#endif
+
 #if defined (__ARM_ARCH_4__) || defined (__ARM_ARCH_4T__) \
 	|| defined (__ARM_ARCH_5__) || defined (__ARM_ARCH_5T__) || defined (__ARM_ARCH_5TE__) || defined (__ARM_ARCH_5TEJ__) \
 	|| defined(__ARM_ARCH_6__)  || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__)
diff --git a/arch/arch-x86.h b/arch/arch-x86.h
index 49e64dd..ce2414d 100644
--- a/arch/arch-x86.h
+++ b/arch/arch-x86.h
@@ -30,9 +30,10 @@ static inline void do_cpuid(unsigned int *eax, unsigned int *ebx,
 #endif
 
 #ifndef __NR_shmget
-#define __NR_shmget 29
-#define __NR_shmat 30
-#define __NR_shmctl 31
+#define __NR_shmget		 29
+#define __NR_shmat		 30
+#define __NR_shmctl		 31
+#define __NR_shmdt		 67
 #endif
 
 #define	FIO_HUGE_PAGE		4194304

-- 
Jens Axboe

RE: syscall problem on Android x86

[Akers, Jason B]

Hi Jens,

Sorry for the confusion. I'm building for x86 Android, not arm.

The issue is that Android x86 does not define __NR_shmget. And the fact that it's hardcoded to 29 in arch-x86.h is causing problems. On x86 Android syscall 29 is actually the sys_pause() function. Therefore, fio compiles but it just pauses when shmget() is called.

Thanks,
Jason Akers

-----Original Message-----
From: Jens Axboe [mailto:axboe@kernel.dk] 
Sent: Friday, April 26, 2013 11:18 AM
To: Akers, Jason B
Cc: fio@vger.kernel.org
Subject: Re: syscall problem on Android x86

On Fri, Apr 26 2013, Akers, Jason B wrote:
> Fio hangs when run on the Android x86 emulator.
> 
> Tracking through, I found that the last call made is shmget() (in init.c - setup_thread_area()).
> I believe that the hang is related to the fio syscall implementation for x86.
> 
> os-android.h defines shmget as: syscall(__NR_shmget, __key, __size, 
> __shmflg);
> 
> In arch-x86.h __NR_shmget is defined to 29. This was added on April 11th (a415b2cc).
> 
> Looking deeper, I see that syscall 29 is actually mapped to pause(). (see SYSCALLS.TXT in bionic/libc) I confirmed that sys_pause() was being called by using a kernel breakpoint. This explains why fio hangs.
> 
> Now the question is: What to do about it?
> For some reason shmget is only exposed for Android ARM targets even though it is part of the kernel (system.map) for x86.
> 
> I see two options:
> 1. add shmget to SYSCALLS.TXT for x86 and recompile. Remove the hardcoded __NR_shmget from arch-x86.h and put a #error instead to warn others that a kernel patch is necessary for Android x86. 
> 
> 2. use a "blessed" shared memory allocation method for Android targets (like ashmem / mmap??) Not sure how difficult it would be to make this work with the existing FIO architecture.
> 
> Any other ideas / thoughts / feedback / suggestions are appreciated.

So sounds like you are catching the x86 definition of __NR_shmget which is indeed 29, but you are building on Arm. That sounds pretty weird. The below patch should wire up the right Arm syscalls, but I think you need into why it's claiming to be x86 while it is in fact building for arm.


diff --git a/arch/arch-arm.h b/arch/arch-arm.h index 7cd9502..8a7398e 100644
--- a/arch/arch-arm.h
+++ b/arch/arch-arm.h
@@ -18,6 +18,13 @@
 #define __NR_sys_vmsplice	343
 #endif
 
+#ifndef __NR_shmget
+#define __NR_shmat		305
+#define __NR_shmdt		306
+#define __NR_shmget		307
+#define __NR_shmctl		308
+#endif
+
 #if defined (__ARM_ARCH_4__) || defined (__ARM_ARCH_4T__) \
 	|| defined (__ARM_ARCH_5__) || defined (__ARM_ARCH_5T__) || defined (__ARM_ARCH_5TE__) || defined (__ARM_ARCH_5TEJ__) \
 	|| defined(__ARM_ARCH_6__)  || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__) diff --git a/arch/arch-x86.h b/arch/arch-x86.h index 49e64dd..ce2414d 100644
--- a/arch/arch-x86.h
+++ b/arch/arch-x86.h
@@ -30,9 +30,10 @@ static inline void do_cpuid(unsigned int *eax, unsigned int *ebx,  #endif
 
 #ifndef __NR_shmget
-#define __NR_shmget 29
-#define __NR_shmat 30
-#define __NR_shmctl 31
+#define __NR_shmget		 29
+#define __NR_shmat		 30
+#define __NR_shmctl		 31
+#define __NR_shmdt		 67
 #endif
 
 #define	FIO_HUGE_PAGE		4194304

--
Jens Axboe

Re: syscall problem on Android x86

[Jens Axboe]

On Fri, Apr 26 2013, Akers, Jason B wrote:
> Hi Jens,
> 
> Sorry for the confusion. I'm building for x86 Android, not arm.
> 
> The issue is that Android x86 does not define __NR_shmget. And the
> fact that it's hardcoded to 29 in arch-x86.h is causing problems. On
> x86 Android syscall 29 is actually the sys_pause() function.
> Therefore, fio compiles but it just pauses when shmget() is called.

Ah I see, so android x86  You could hack around it a little bit. In
arch-x86.h, something ala:

#ifndef __NR_shmget
#if defined(__ANDROID__)
#define __NR_shmget		 a
#define __NR_shmat		 b
#define __NR_shmctl		 c
#define __NR_shmdt		 d
#else
#define __NR_shmget		 29
#define __NR_shmat		 30
#define __NR_shmctl		 31
#define __NR_shmdt		 67
#endif
#endif

with a/b/c/d being the right values for the android abi.

-- 
Jens Axboe

RE: syscall problem on Android x86

[Akers, Jason B]

Unfortunately android x86 doesn't have a syscall number for the shm* functions. They are not exposed by x86 bionic libc. So there's nothing to fill in for a/b/c/d.

The only way I see to get syscall numbers is to patch x86 android libc and recompile. This should work for me today but, since the syscall numbers are not standardized, it will not work for anyone with an unmodified libc.

So I propose something like:

#ifndef __NR_shmget
#if defined(__ANDROID__)
#error "__NR_shmget not defined. See bionic libc SYSCALLS.TXT for more information."
#else
#define __NR_shmget		 29
#define __NR_shmat		 30
#define __NR_shmctl		 31
#define __NR_shmdt		 67
#endif
#endif

This will "break the build" for unmodified x86 android platforms, but I think that's better than compiling and not working.

The other (probably better long-term) option is to use ashmem for allocating shared memory on android.

Jason Akers


-----Original Message-----
From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On Behalf Of Jens Axboe
Sent: Friday, April 26, 2013 1:04 PM
To: Akers, Jason B
Cc: fio@vger.kernel.org
Subject: Re: syscall problem on Android x86

On Fri, Apr 26 2013, Akers, Jason B wrote:
> Hi Jens,
> 
> Sorry for the confusion. I'm building for x86 Android, not arm.
> 
> The issue is that Android x86 does not define __NR_shmget. And the 
> fact that it's hardcoded to 29 in arch-x86.h is causing problems. On
> x86 Android syscall 29 is actually the sys_pause() function.
> Therefore, fio compiles but it just pauses when shmget() is called.

Ah I see, so android x86  You could hack around it a little bit. In arch-x86.h, something ala:

#ifndef __NR_shmget
#if defined(__ANDROID__)
#define __NR_shmget		 a
#define __NR_shmat		 b
#define __NR_shmctl		 c
#define __NR_shmdt		 d
#else
#define __NR_shmget		 29
#define __NR_shmat		 30
#define __NR_shmctl		 31
#define __NR_shmdt		 67
#endif
#endif

with a/b/c/d being the right values for the android abi.

--
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: syscall problem on Android x86

[Jens Axboe]

On Fri, Apr 26 2013, Akers, Jason B wrote:
> Unfortunately android x86 doesn't have a syscall number for the shm*
> functions. They are not exposed by x86 bionic libc. So there's nothing
> to fill in for a/b/c/d.
> 
> The only way I see to get syscall numbers is to patch x86 android libc
> and recompile. This should work for me today but, since the syscall
> numbers are not standardized, it will not work for anyone with an
> unmodified libc.
> 
> So I propose something like:
> 
> #ifndef __NR_shmget
> #if defined(__ANDROID__)
> #error "__NR_shmget not defined. See bionic libc SYSCALLS.TXT for more information."
> #else
> #define __NR_shmget		 29
> #define __NR_shmat		 30
> #define __NR_shmctl		 31
> #define __NR_shmdt		 67
> #endif
> #endif
> 
> This will "break the build" for unmodified x86 android platforms, but
> I think that's better than compiling and not working.

The syscall has nothing to do with the libc installed. The libc headers
might not DEFINE the numbers. As long as the syscall is wired up in the
given kernel (which I'm assuming it is, but I really know nothing about
android), then that is enough.

> The other (probably better long-term) option is to use ashmem for
> allocating shared memory on android.

I'll gladly take patches :-)

Please stop top-posting, btw, it goes against the normal net etiquette
on this forum (and most others in the open source world).

-- 
Jens Axboe

Re: syscall problem on Android x86

[Aaron Carroll]

On 27 April 2013 03:43, Akers, Jason B <jason.b.akers@intel.com> wrote:
> Fio hangs when run on the Android x86 emulator.
>
> Tracking through, I found that the last call made is shmget() (in init.c - setup_thread_area()).
> I believe that the hang is related to the fio syscall implementation for x86.
>
> os-android.h defines shmget as: syscall(__NR_shmget, __key, __size, __shmflg);
>
> In arch-x86.h __NR_shmget is defined to 29. This was added on April 11th (a415b2cc).
>
> Looking deeper, I see that syscall 29 is actually mapped to pause(). (see SYSCALLS.TXT in bionic/libc) I confirmed that sys_pause() was being called by using a kernel breakpoint. This explains why fio hangs.

You are right, I added the 64bit syscall number.

> Now the question is: What to do about it?

 * Either junk the __NR_shm* definitions, or move them to x86-64 for
future 64bit Android compatibility
 * The shm* functions need different definitions for x86-32, because
there are no shm* syscalls. They appear to be multiplexed over an IPC
syscall.

As Jens said this has nothing to do with Android per se, only that its
libc does not implement shm*() so we have to do it ourselves.


  -- Aaron

Re: syscall problem on Android x86

[Jens Axboe]

On Sat, Apr 27 2013, Aaron Carroll wrote:
> On 27 April 2013 03:43, Akers, Jason B <jason.b.akers@intel.com> wrote:
> > Fio hangs when run on the Android x86 emulator.
> >
> > Tracking through, I found that the last call made is shmget() (in init.c - setup_thread_area()).
> > I believe that the hang is related to the fio syscall implementation for x86.
> >
> > os-android.h defines shmget as: syscall(__NR_shmget, __key, __size, __shmflg);
> >
> > In arch-x86.h __NR_shmget is defined to 29. This was added on April 11th (a415b2cc).
> >
> > Looking deeper, I see that syscall 29 is actually mapped to pause(). (see SYSCALLS.TXT in bionic/libc) I confirmed that sys_pause() was being called by using a kernel breakpoint. This explains why fio hangs.
> 
> You are right, I added the 64bit syscall number.
> 
> > Now the question is: What to do about it?
> 
>  * Either junk the __NR_shm* definitions, or move them to x86-64 for
> future 64bit Android compatibility
>  * The shm* functions need different definitions for x86-32, because
> there are no shm* syscalls. They appear to be multiplexed over an IPC
> syscall.

Hmm good point, I had (wrongly) assumed that the abi was in sync for 32
and 64-bit x86, but that is not the case. For x86, you have to do
ipc(SHMGET, ...) etc to get at it.

I'll move the definitions to x86-64 where they belong.

> As Jens said this has nothing to do with Android per se, only that its
> libc does not implement shm*() so we have to do it ourselves.

Yep. Jason, it should be fairly easy to wire up ipc(). Or you can do the
android variant, I'm fine with either.

-- 
Jens Axboe

RE: syscall problem on Android x86

[Akers, Jason B]

> > As Jens said this has nothing to do with Android per se, only that its
> > libc does not implement shm*() so we have to do it ourselves.
> 
> Yep. Jason, it should be fairly easy to wire up ipc(). Or you can do the android
> variant, I'm fine with either.
> 
> --
> Jens Axboe

Thanks for the feedback. I'm leaning toward using ashmem in an attempt to make it forward-compatible with future android NDKs. We'll see how ugly it gets.
I'll submit a patch review request when it's ready.

Jason Akers