[Qemu-devel] [PATCH, RFC] BSD user emulator

[Qemu-devel] [PATCH, RFC] BSD user emulator

[Blue Swirl]

[-- Attachment #1: Type: text/plain, Size: 2569 bytes --]

Hi,

This is a preliminary implementation of BSD user emulator. Some files
inside bsd-user contain a lot of unwanted cruft copied from linux-user
and ldscript is not used. But it can already run a handcrafted Sparc64
helloworld file (on OpenBSD/Sparc64 host):

LD_BIND_NOW=y ./qemu-sparc64 -d in_asm -strace ../helloworld.sparc64_obsd
write(1,0x2001f8,13)Hello World!
 = 13
exit(13)

cat /tmp/qemu.log
start    end      size     prot
00100000-00102000 00002000 r-x
00200000-00202000 00002000 r--
00300000-00302000 00002000 rw-
00700000-00702000 00002000 rw-
40000000-40080000 00080000 rw-
40080000-40082000 00002000 ---
40d32000-40d3a000 00008000 ---
418e8000-418f0000 00008000 ---
41b84000-41b8c000 00008000 ---
4325a000-43262000 00008000 ---
4357c000-4357e000 00002000 ---
436ca000-436d2000 00008000 ---
43758000-43760000 00008000 ---
44202000-4420a000 00008000 ---
44dce000-44dd6000 00008000 ---
4643e000-46448000 0000a000 ---
466da000-466e2000 00008000 ---
467d0000-467d8000 00008000 ---
48492000-4849a000 00008000 ---
487b0000-487b8000 00008000 ---
48e8a000-48e92000 00008000 ---
49136000-4913e000 00008000 ---
4941a000-49422000 00008000 ---
494ec000-494ee000 00002000 ---
498b4000-498bc000 00008000 ---
4a950000-4a958000 00008000 ---
4ade4000-4adec000 00008000 ---
4bae8000-4baf0000 00008000 ---
4bb86000-4bb88000 00002000 ---
4d542000-4d55c000 0001a000 ---
4df92000-4df9a000 00008000 ---
4e10a000-4e112000 00008000 ---
4e212000-4e21a000 00008000 ---
4eaca000-4ead2000 00008000 ---
4f392000-4f39a000 00008000 ---
4f4ae000-4f4b6000 00008000 ---
4f6da000-4f6e2000 00008000 ---
500ba000-544bc000 04402000 ---
start_brk   0x0000000000700208
end_code    0x00000000001001f8
start_code  0x0000000000100000
start_data  0x0000000000700208
end_data    0x0000000000700208
start_stack 0x000000004007fb70
brk         0x0000000000700208
entry       0x00000000001001c8
--------------
IN:
0x00000000001001c8:  mov  0xd, %o2
0x00000000001001cc:  sethi  %hi(0), %o0
0x00000000001001d0:  sethi  %hi(0x200000), %o1
0x00000000001001d4:  mov  %o0, %o0
0x00000000001001d8:  or  %o1, 0x1f8, %o1
0x00000000001001dc:  sllx  %o0, 0x20, %o0
0x00000000001001e0:  or  %o1, %o0, %o1
0x00000000001001e4:  mov  1, %o0
0x00000000001001e8:  mov  4, %g1
0x00000000001001ec:  ta  0

--------------
IN:
0x00000000001001f0:  mov  1, %g1
0x00000000001001f4:  ta  0

I think in BSD (at least OpenBSD), system call numbers, system call
parameters, ioctls, and signal numbers are shared across
architectures, maybe even between *BSDs. If it's true, it should make
the emulator much simpler than Linux one.

[-- Attachment #2: bsd_user_emu.diff.bz2 --]
[-- Type: application/x-bzip2, Size: 43852 bytes --]

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[M. Warner Losh]

In message: <f43fc5580808251145u51499e6fsda35d805514fd82b@mail.gmail.com>
            "Blue Swirl" <blauwirbel@gmail.com> writes:
: I think in BSD (at least OpenBSD), system call numbers, system call
: parameters, ioctls, and signal numbers are shared across
: architectures, maybe even between *BSDs. If it's true, it should make
: the emulator much simpler than Linux one.

Yes.  FreeBSD, DragonFlyBSD and NetBSD all share this trait.  There's
some minor ioctl variances between architectures, however, since the
size of the data of the ioctl is encoded in the ioctl request
number/#define.  Error numbers are also the same.

The only time that system call numbers are translated is for the
different emulation layers that the BSDs implement.  So for doing
OSF/1 emulation something different than Linux emulation is done.
However, for each of the emulators, the system calls, signals, etc is
consistent across architectures for everything except Linux...

And before I forget, way way way cool!

Warner

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Andreas Färber]

Hi,

Am 25.08.2008 um 20:45 schrieb Blue Swirl:

> This is a preliminary implementation of BSD user emulator. Some files
> inside bsd-user contain a lot of unwanted cruft copied from linux-user
> and ldscript is not used. But it can already run a handcrafted Sparc64
> helloworld file (on OpenBSD/Sparc64 host)
[...]
> I think in BSD (at least OpenBSD), system call numbers, system call
> parameters, ioctls, and signal numbers are shared across
> architectures, maybe even between *BSDs. If it's true, it should make
> the emulator much simpler than Linux one.

Recently I had investigated the idea of a Haiku user emulator. Looking  
at your patch, I see a lot of similarities with what I saw in the  
linux-user dir, but your patch is not an svn diff and marks all files  
as fully new against /dev/null, so it's hard to spot the actual  
differences to linux-user...

In light of getting more userland emulations, do you see possibilities  
to move shared stuff to a new "user" dir rather than copying them to  
each *-user dir? The ELF loading for instance?

Andreas

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Blue Swirl]

On 8/27/08, Andreas Färber <andreas.faerber@web.de> wrote:
> Hi,
>
>  Am 25.08.2008 um 20:45 schrieb Blue Swirl:
>
>
> > This is a preliminary implementation of BSD user emulator. Some files
> > inside bsd-user contain a lot of unwanted cruft copied from linux-user
> > and ldscript is not used. But it can already run a handcrafted Sparc64
> > helloworld file (on OpenBSD/Sparc64 host)
> >
>  [...]
>
> > I think in BSD (at least OpenBSD), system call numbers, system call
> > parameters, ioctls, and signal numbers are shared across
> > architectures, maybe even between *BSDs. If it's true, it should make
> > the emulator much simpler than Linux one.
> >
>
>  Recently I had investigated the idea of a Haiku user emulator. Looking at
> your patch, I see a lot of similarities with what I saw in the linux-user
> dir, but your patch is not an svn diff and marks all files as fully new
> against /dev/null, so it's hard to spot the actual differences to
> linux-user...
>
>  In light of getting more userland emulations, do you see possibilities to
> move shared stuff to a new "user" dir rather than copying them to each
> *-user dir? The ELF loading for instance?

I don't think so, Fabrice was strongly against such merges:
http://article.gmane.org/gmane.comp.emulators.qemu/20864

[Qemu-devel] [PATCH, RFC] BSD user emulator

[Blue Swirl]

[-- Attachment #1: Type: text/plain, Size: 1388 bytes --]

Hi,

This version can actually execute some programs, for example:

./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /etc/motd
OpenBSD 4.4 (GENERIC) #1715: Mon Aug 11 17:55:10 MDT 2008

Welcome to OpenBSD: The proactively secure Unix-like operating system.

Please use the sendbug(1) utility to report bugs in the system.
Before reporting a bug, please try to reproduce it with the latest
version of the code.  With bug reports, please try to ensure that
enough information to reproduce the problem is enclosed, and if a
known fix for it exists, include that as well.

Something is not correct with errno handling, it could also be a bug
in CPU emulation:
/sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /foobar
cat: /foobar: Undefined error: 0

Some other programs just fail:
./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ls
ls: Cannot allocate memory
./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ksh
ksh in malloc(): error: chunk overflow
./sparc64-bsd-user/qemu-sparc64 -d in_asm /usr/bin/id
Segmentation fault
./sparc64-bsd-user/qemu-sparc64 -d in_asm /sbin/ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 0¸
ifconfig: getifgroups: Cannot allocate memory

I've cleaned up the patch a bit. Signals, ioctls and structure
conversions are still unimplemented and as usual, no docs.

I think this is getting ready to be committed pretty soon. Any
comments, objections?

[-- Attachment #2: bsd_user_emu.diff.bz2 --]
[-- Type: application/x-bzip2, Size: 31630 bytes --]

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Alexander Graf]

On Aug 30, 2008, at 10:31 AM, Blue Swirl wrote:

> Hi,
>
> This version can actually execute some programs, for example:
>
> ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /etc/motd
> OpenBSD 4.4 (GENERIC) #1715: Mon Aug 11 17:55:10 MDT 2008
>
> Welcome to OpenBSD: The proactively secure Unix-like operating system.
>
> Please use the sendbug(1) utility to report bugs in the system.
> Before reporting a bug, please try to reproduce it with the latest
> version of the code.  With bug reports, please try to ensure that
> enough information to reproduce the problem is enclosed, and if a
> known fix for it exists, include that as well.
>
> Something is not correct with errno handling, it could also be a bug
> in CPU emulation:
> /sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /foobar
> cat: /foobar: Undefined error: 0
>
> Some other programs just fail:
> ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ls
> ls: Cannot allocate memory
> ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ksh
> ksh in malloc(): error: chunk overflow
> ./sparc64-bsd-user/qemu-sparc64 -d in_asm /usr/bin/id
> Segmentation fault
> ./sparc64-bsd-user/qemu-sparc64 -d in_asm /sbin/ifconfig
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 0¸
> ifconfig: getifgroups: Cannot allocate memory
>
> I've cleaned up the patch a bit. Signals, ioctls and structure
> conversions are still unimplemented and as usual, no docs.
>
> I think this is getting ready to be committed pretty soon. Any
> comments, objections?

Just off the top of my head: Shouldn't BSD userspace and Darwin  
userspace emulation be able to share a good bunch of syscalls?
IIRC the positive Darwin syscalls should be some recent FreeBSD  
syscalls that should look pretty similar to OpenBSD syscalls. I might  
be totally wrong here though ;-).

Alex

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Andreas Färber]

Am 01.09.2008 um 10:58 schrieb Alexander Graf:

> Just off the top of my head: Shouldn't BSD userspace and Darwin  
> userspace emulation be able to share a good bunch of syscalls?

Earlier in this thread, Blue Swirl said Fabrice was opposed to code  
sharing in the user emulators. I didn't even think of syscalls at the  
time, I would've been happy sharing helpers such as gemu_log and the  
like...

Andreas

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[M. Warner Losh]

In message: <3ACB420B-8C07-4417-80F5-1B74896D1533@suse.de>
            Alexander Graf <agraf@suse.de> writes:
: 
: On Aug 30, 2008, at 10:31 AM, Blue Swirl wrote:
: 
: > Hi,
: >
: > This version can actually execute some programs, for example:
: >
: > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /etc/motd
: > OpenBSD 4.4 (GENERIC) #1715: Mon Aug 11 17:55:10 MDT 2008
: >
: > Welcome to OpenBSD: The proactively secure Unix-like operating system.
: >
: > Please use the sendbug(1) utility to report bugs in the system.
: > Before reporting a bug, please try to reproduce it with the latest
: > version of the code.  With bug reports, please try to ensure that
: > enough information to reproduce the problem is enclosed, and if a
: > known fix for it exists, include that as well.
: >
: > Something is not correct with errno handling, it could also be a bug
: > in CPU emulation:
: > /sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /foobar
: > cat: /foobar: Undefined error: 0
: >
: > Some other programs just fail:
: > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ls
: > ls: Cannot allocate memory
: > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ksh
: > ksh in malloc(): error: chunk overflow
: > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /usr/bin/id
: > Segmentation fault
: > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /sbin/ifconfig
: > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 0¸
: > ifconfig: getifgroups: Cannot allocate memory
: >
: > I've cleaned up the patch a bit. Signals, ioctls and structure
: > conversions are still unimplemented and as usual, no docs.
: >
: > I think this is getting ready to be committed pretty soon. Any
: > comments, objections?
: 
: Just off the top of my head: Shouldn't BSD userspace and Darwin  
: userspace emulation be able to share a good bunch of syscalls?
: IIRC the positive Darwin syscalls should be some recent FreeBSD  
: syscalls that should look pretty similar to OpenBSD syscalls. I might  
: be totally wrong here though ;-).

Kinda.

The system calls are mostly similar between the BSDs, but do vary for
syscalls added since 4.4BSD Lite-2.

Warner

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Alexander Graf]

Am 01.09.2008 um 18:25 schrieb "M. Warner Losh" <imp@bsdimp.com>:

> In message: <3ACB420B-8C07-4417-80F5-1B74896D1533@suse.de>
>            Alexander Graf <agraf@suse.de> writes:
> :
> : On Aug 30, 2008, at 10:31 AM, Blue Swirl wrote:
> :
> : > Hi,
> : >
> : > This version can actually execute some programs, for example:
> : >
> : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /etc/motd
> : > OpenBSD 4.4 (GENERIC) #1715: Mon Aug 11 17:55:10 MDT 2008
> : >
> : > Welcome to OpenBSD: The proactively secure Unix-like operating  
> system.
> : >
> : > Please use the sendbug(1) utility to report bugs in the system.
> : > Before reporting a bug, please try to reproduce it with the latest
> : > version of the code.  With bug reports, please try to ensure that
> : > enough information to reproduce the problem is enclosed, and if a
> : > known fix for it exists, include that as well.
> : >
> : > Something is not correct with errno handling, it could also be a  
> bug
> : > in CPU emulation:
> : > /sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /foobar
> : > cat: /foobar: Undefined error: 0
> : >
> : > Some other programs just fail:
> : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ls
> : > ls: Cannot allocate memory
> : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ksh
> : > ksh in malloc(): error: chunk overflow
> : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /usr/bin/id
> : > Segmentation fault
> : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /sbin/ifconfig
> : > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 0¸
> : > ifconfig: getifgroups: Cannot allocate memory
> : >
> : > I've cleaned up the patch a bit. Signals, ioctls and structure
> : > conversions are still unimplemented and as usual, no docs.
> : >
> : > I think this is getting ready to be committed pretty soon. Any
> : > comments, objections?
> :
> : Just off the top of my head: Shouldn't BSD userspace and Darwin
> : userspace emulation be able to share a good bunch of syscalls?
> : IIRC the positive Darwin syscalls should be some recent FreeBSD
> : syscalls that should look pretty similar to OpenBSD syscalls. I  
> might
> : be totally wrong here though ;-).
>
> Kinda.
>
> The system calls are mostly similar between the BSDs, but do vary for
> syscalls added since 4.4BSD Lite-2.
>

So darwin and all bsds could share most 4.4 bsd syscalls and implement  
the others depending on their specific target? Sounds like a big win  
to me.

Too bad fabrice doesn't agree to this.

Alex

> Warner

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[M. Warner Losh]

In message: <5EF2D60B-861A-4306-A553-15626B92C463@suse.de>
            Alexander Graf <agraf@suse.de> writes:
: 
: 
: 
: 
: Am 01.09.2008 um 18:25 schrieb "M. Warner Losh" <imp@bsdimp.com>:
: 
: > In message: <3ACB420B-8C07-4417-80F5-1B74896D1533@suse.de>
: >            Alexander Graf <agraf@suse.de> writes:
: > :
: > : On Aug 30, 2008, at 10:31 AM, Blue Swirl wrote:
: > :
: > : > Hi,
: > : >
: > : > This version can actually execute some programs, for example:
: > : >
: > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /etc/motd
: > : > OpenBSD 4.4 (GENERIC) #1715: Mon Aug 11 17:55:10 MDT 2008
: > : >
: > : > Welcome to OpenBSD: The proactively secure Unix-like operating  
: > system.
: > : >
: > : > Please use the sendbug(1) utility to report bugs in the system.
: > : > Before reporting a bug, please try to reproduce it with the latest
: > : > version of the code.  With bug reports, please try to ensure that
: > : > enough information to reproduce the problem is enclosed, and if a
: > : > known fix for it exists, include that as well.
: > : >
: > : > Something is not correct with errno handling, it could also be a  
: > bug
: > : > in CPU emulation:
: > : > /sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /foobar
: > : > cat: /foobar: Undefined error: 0
: > : >
: > : > Some other programs just fail:
: > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ls
: > : > ls: Cannot allocate memory
: > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ksh
: > : > ksh in malloc(): error: chunk overflow
: > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /usr/bin/id
: > : > Segmentation fault
: > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /sbin/ifconfig
: > : > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 0¸
: > : > ifconfig: getifgroups: Cannot allocate memory
: > : >
: > : > I've cleaned up the patch a bit. Signals, ioctls and structure
: > : > conversions are still unimplemented and as usual, no docs.
: > : >
: > : > I think this is getting ready to be committed pretty soon. Any
: > : > comments, objections?
: > :
: > : Just off the top of my head: Shouldn't BSD userspace and Darwin
: > : userspace emulation be able to share a good bunch of syscalls?
: > : IIRC the positive Darwin syscalls should be some recent FreeBSD
: > : syscalls that should look pretty similar to OpenBSD syscalls. I  
: > might
: > : be totally wrong here though ;-).
: >
: > Kinda.
: >
: > The system calls are mostly similar between the BSDs, but do vary for
: > syscalls added since 4.4BSD Lite-2.
: >
: 
: So darwin and all bsds could share most 4.4 bsd syscalls and implement  
: the others depending on their specific target? Sounds like a big win  
: to me.
: 
: Too bad fabrice doesn't agree to this.

Well, it isn't quite *that* simple.  If you look at the syscall lists,
you'll see they are mostly in common.  However, theres been some
divergence so you'll need multiple tables, one for each BSD.  Maybe
90-95% of each of them will be the same.

I've not specifically looked at Darwin system calls, but NetBSD did a
Darwin emulator without a lot of hassle.  They had to do a lot for
Mach-O image activation, of course, and fill in a number of missing
system calls.

I think that Fabrice likely will see the light when the tangible
benefits are shown.  He's likely worried about the same problem that
the FreeBSD Linux ABI implementation suffered from: lots of special
cases because Linux returns this error rather than that error, or
treats group wheel differently in this case or that, etc.  These
problems have been mostly solved in the FreeBSD implementation in the
kernel by having the emulation layers use a common set of core kernel
routines to implement things, arranging for the right set of flags and
such to be passed down.

Anyway, that's just my two cents and speculation...  Others, including
those speculated about, may differ...  Objects in the mirror are
closer than they appear...  If mail reader catches fire, leave the
room and call for help...

Warner

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Blue Swirl]

On 9/1/08, M. Warner Losh <imp@bsdimp.com> wrote:
> In message: <5EF2D60B-861A-4306-A553-15626B92C463@suse.de>
>
>             Alexander Graf <agraf@suse.de> writes:
>  :
>  :
>
> :
>  :
>  : Am 01.09.2008 um 18:25 schrieb "M. Warner Losh" <imp@bsdimp.com>:
>
> :
>  : > In message: <3ACB420B-8C07-4417-80F5-1B74896D1533@suse.de>
>  : >            Alexander Graf <agraf@suse.de> writes:
>  : > :
>  : > : On Aug 30, 2008, at 10:31 AM, Blue Swirl wrote:
>  : > :
>  : > : > Hi,
>  : > : >
>  : > : > This version can actually execute some programs, for example:
>  : > : >
>  : > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /etc/motd
>  : > : > OpenBSD 4.4 (GENERIC) #1715: Mon Aug 11 17:55:10 MDT 2008
>  : > : >
>  : > : > Welcome to OpenBSD: The proactively secure Unix-like operating
>  : > system.
>  : > : >
>  : > : > Please use the sendbug(1) utility to report bugs in the system.
>  : > : > Before reporting a bug, please try to reproduce it with the latest
>  : > : > version of the code.  With bug reports, please try to ensure that
>  : > : > enough information to reproduce the problem is enclosed, and if a
>  : > : > known fix for it exists, include that as well.
>  : > : >
>  : > : > Something is not correct with errno handling, it could also be a
>  : > bug
>  : > : > in CPU emulation:
>  : > : > /sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/cat /foobar
>  : > : > cat: /foobar: Undefined error: 0
>  : > : >
>  : > : > Some other programs just fail:
>  : > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ls
>  : > : > ls: Cannot allocate memory
>  : > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /bin/ksh
>  : > : > ksh in malloc(): error: chunk overflow
>  : > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /usr/bin/id
>  : > : > Segmentation fault
>  : > : > ./sparc64-bsd-user/qemu-sparc64 -d in_asm /sbin/ifconfig
>  : > : > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 0¸
>  : > : > ifconfig: getifgroups: Cannot allocate memory
>  : > : >
>  : > : > I've cleaned up the patch a bit. Signals, ioctls and structure
>  : > : > conversions are still unimplemented and as usual, no docs.
>  : > : >
>  : > : > I think this is getting ready to be committed pretty soon. Any
>  : > : > comments, objections?
>  : > :
>  : > : Just off the top of my head: Shouldn't BSD userspace and Darwin
>  : > : userspace emulation be able to share a good bunch of syscalls?
>  : > : IIRC the positive Darwin syscalls should be some recent FreeBSD
>  : > : syscalls that should look pretty similar to OpenBSD syscalls. I
>  : > might
>  : > : be totally wrong here though ;-).
>  : >
>  : > Kinda.
>  : >
>  : > The system calls are mostly similar between the BSDs, but do vary for
>  : > syscalls added since 4.4BSD Lite-2.
>  : >
>  :
>  : So darwin and all bsds could share most 4.4 bsd syscalls and implement
>  : the others depending on their specific target? Sounds like a big win
>  : to me.
>  :
>  : Too bad fabrice doesn't agree to this.
>
>
> Well, it isn't quite *that* simple.  If you look at the syscall lists,
>  you'll see they are mostly in common.  However, theres been some
>  divergence so you'll need multiple tables, one for each BSD.  Maybe
>  90-95% of each of them will be the same.
>
>  I've not specifically looked at Darwin system calls, but NetBSD did a
>  Darwin emulator without a lot of hassle.  They had to do a lot for
>  Mach-O image activation, of course, and fill in a number of missing
>  system calls.
>
>  I think that Fabrice likely will see the light when the tangible
>  benefits are shown.  He's likely worried about the same problem that
>  the FreeBSD Linux ABI implementation suffered from: lots of special
>  cases because Linux returns this error rather than that error, or
>  treats group wheel differently in this case or that, etc.  These
>  problems have been mostly solved in the FreeBSD implementation in the
>  kernel by having the emulation layers use a common set of core kernel
>  routines to implement things, arranging for the right set of flags and
>  such to be passed down.
>
>  Anyway, that's just my two cents and speculation...  Others, including
>  those speculated about, may differ...  Objects in the mirror are
>  closer than they appear...  If mail reader catches fire, leave the
>  room and call for help...

Time for some statistics:

grep 'SYS_[a-z_]' fbsd-syscall.h |sed 's/[    ][      ]*/ /g'|sort -k
3 -n >fbsd-syscalls
grep 'SYS_[a-z_]' nbsd-syscall.h |sed 's/[    ][      ]*/ /g'|sort -k
2 -n >nbsd-syscalls
grep 'SYS_[a-z_]' obsd-syscall.h |sed 's/[    ][      ]*/ /g'|sort -k
3 -n >obsd-syscalls

wc -l *bsd-syscalls
  365 fbsd-syscalls
  366 nbsd-syscalls
  187 obsd-syscalls
  918 total

diff -u nbsd-syscalls fbsd-syscalls |diffstat
 fbsd-syscalls |  475 ++++++++++++++++++++++++++++------------------------------
 1 file changed, 237 insertions(+), 238 deletions(-)

diff -u obsd-syscalls fbsd-syscalls |diffstat
 fbsd-syscalls |  298 ++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 238 insertions(+), 60 deletions(-)

diff -u obsd-syscalls nbsd-syscalls |diffstat
 nbsd-syscalls |  297 ++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 238 insertions(+), 59 deletions(-)

Some are just renames, but those are still large numbers. I have to
add syscall number conversion tables, I thought host SYS_xx were
almost always equal to target SYS_xx.

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[M. Warner Losh]

In message: <f43fc5580809011121m397096f1m7e8f29e67189a636@mail.gmail.com>
            "Blue Swirl" <blauwirbel@gmail.com> writes:
: diff -u nbsd-syscalls fbsd-syscalls |diffstat
:  fbsd-syscalls |  475 ++++++++++++++++++++++++++++------------------------------
:  1 file changed, 237 insertions(+), 238 deletions(-)
: 
: diff -u obsd-syscalls fbsd-syscalls |diffstat
:  fbsd-syscalls |  298 ++++++++++++++++++++++++++++++++++++++++++++++------------
:  1 file changed, 238 insertions(+), 60 deletions(-)
: 
: diff -u obsd-syscalls nbsd-syscalls |diffstat
:  nbsd-syscalls |  297 ++++++++++++++++++++++++++++++++++++++++++++++------------
:  1 file changed, 238 insertions(+), 59 deletions(-)
: 
: Some are just renames, but those are still large numbers. I have to
: add syscall number conversion tables, I thought host SYS_xx were
: almost always equal to target SYS_xx.

These numbers seem high to me...  The generated files between the
different BSDs are somewhat different, so I'm not sure that these
simple tests will tell the whole story...

Warner

Re: [Qemu-devel] [PATCH, RFC] BSD user emulator

[Blue Swirl]

On 9/1/08, M. Warner Losh <imp@bsdimp.com> wrote:
> In message: <f43fc5580809011121m397096f1m7e8f29e67189a636@mail.gmail.com>
>             "Blue Swirl" <blauwirbel@gmail.com> writes:
>  : diff -u nbsd-syscalls fbsd-syscalls |diffstat
>
> :  fbsd-syscalls |  475 ++++++++++++++++++++++++++++------------------------------
>  :  1 file changed, 237 insertions(+), 238 deletions(-)
>  :
>  : diff -u obsd-syscalls fbsd-syscalls |diffstat
>  :  fbsd-syscalls |  298 ++++++++++++++++++++++++++++++++++++++++++++++------------
>  :  1 file changed, 238 insertions(+), 60 deletions(-)
>  :
>  : diff -u obsd-syscalls nbsd-syscalls |diffstat
>  :  nbsd-syscalls |  297 ++++++++++++++++++++++++++++++++++++++++++++++------------
>  :  1 file changed, 238 insertions(+), 59 deletions(-)
>  :
>  : Some are just renames, but those are still large numbers. I have to
>  : add syscall number conversion tables, I thought host SYS_xx were
>  : almost always equal to target SYS_xx.
>
>
> These numbers seem high to me...  The generated files between the
>  different BSDs are somewhat different, so I'm not sure that these
>  simple tests will tell the whole story...

Taking OpenBSD vs. NetBSD as an example: the first 150 calls have not
changed much, then there are some changes and starting from 256 calls
diverge. There are even some different calls using the same number.

diff -u obsd-syscalls nbsd-syscalls
--- obsd-syscalls       2008-09-01 18:10:23.000000000 +0000
+++ nbsd-syscalls       2008-09-01 18:10:17.000000000 +0000
@@ -14,6 +14,7 @@
 #define SYS_chmod 15
 #define SYS_chown 16
 #define SYS_break 17
+#define SYS_freebsd4_getfsstat 18
 #define SYS_getpid 20
 #define SYS_mount 21
 #define SYS_unmount 22
@@ -34,18 +35,15 @@
 #define SYS_kill 37
 #define SYS_getppid 39
 #define SYS_dup 41
-#define SYS_opipe 42
+#define SYS_pipe 42
 #define SYS_getegid 43
 #define SYS_profil 44
 #define SYS_ktrace 45
-#define SYS_sigaction 46
 #define SYS_getgid 47
-#define SYS_sigprocmask 48
 #define SYS_getlogin 49
 #define SYS_setlogin 50
 #define SYS_acct 51
-#define SYS_sigpending 52
-#define SYS_osigaltstack 53
+#define SYS_sigaltstack 53
 #define SYS_ioctl 54
 #define SYS_reboot 55
 #define SYS_revoke 56
@@ -54,9 +52,11 @@
 #define SYS_execve 59
 #define SYS_umask 60
 #define SYS_chroot 61
+#define SYS_msync 65
 #define SYS_vfork 66
 #define SYS_sbrk 69
 #define SYS_sstk 70
+#define SYS_vadvise 72
 #define SYS_munmap 73
 #define SYS_mprotect 74
 #define SYS_madvise 75
@@ -66,7 +66,9 @@
 #define SYS_getpgrp 81
 #define SYS_setpgid 82
 #define SYS_setitimer 83
+#define SYS_swapon 85
 #define SYS_getitimer 86
+#define SYS_getdtablesize 89
 #define SYS_dup2 90
 #define SYS_fcntl 92
 #define SYS_select 93
@@ -75,11 +77,9 @@
 #define SYS_socket 97
 #define SYS_connect 98
 #define SYS_getpriority 100
-#define SYS_sigreturn 103
 #define SYS_bind 104
 #define SYS_setsockopt 105
 #define SYS_listen 106
-#define SYS_sigsuspend 111
 #define SYS_gettimeofday 116
 #define SYS_getrusage 117
 #define SYS_getsockopt 118
@@ -102,86 +102,265 @@
 #define SYS_adjtime 140
 #define SYS_setsid 147
 #define SYS_quotactl 148
+#define SYS_nlm_syscall 154
 #define SYS_nfssvc 155
+#define SYS_freebsd4_statfs 157
+#define SYS_freebsd4_fstatfs 158
+#define SYS_lgetfh 160
 #define SYS_getfh 161
+#define SYS_getdomainname 162
+#define SYS_setdomainname 163
+#define SYS_uname 164
 #define SYS_sysarch 165
-#define SYS_pread 173
-#define SYS_pwrite 174
+#define SYS_rtprio 166
+#define SYS_semsys 169
+#define SYS_msgsys 170
+#define SYS_shmsys 171
+#define SYS_freebsd6_pread 173
+#define SYS_freebsd6_pwrite 174
+#define SYS_setfib 175
+#define SYS_ntp_adjtime 176
 #define SYS_setgid 181
 #define SYS_setegid 182
 #define SYS_seteuid 183
-#define SYS_lfs_bmapv 184
-#define SYS_lfs_markv 185
-#define SYS_lfs_segclean 186
-#define SYS_lfs_segwait 187
+#define SYS_stat 188
+#define SYS_fstat 189
+#define SYS_lstat 190
 #define SYS_pathconf 191
 #define SYS_fpathconf 192
-#define SYS_swapctl 193
 #define SYS_getrlimit 194
 #define SYS_setrlimit 195
 #define SYS_getdirentries 196
-#define SYS_mmap 197
+#define SYS_freebsd6_mmap 197
 #define SYS___syscall 198
-#define SYS_lseek 199
-#define SYS_truncate 200
-#define SYS_ftruncate 201
+#define SYS_freebsd6_lseek 199
+#define SYS_freebsd6_truncate 200
+#define SYS_freebsd6_ftruncate 201
 #define SYS___sysctl 202
 #define SYS_mlock 203
 #define SYS_munlock 204
+#define SYS_undelete 205
 #define SYS_futimes 206
 #define SYS_getpgid 207
-#define SYS_xfspioctl 208
+#define SYS_poll 209
+#define SYS___semctl 220
 #define SYS_semget 221
+#define SYS_semop 222
+#define SYS_msgctl 224
 #define SYS_msgget 225
 #define SYS_msgsnd 226
 #define SYS_msgrcv 227
 #define SYS_shmat 228
+#define SYS_shmctl 229
 #define SYS_shmdt 230
+#define SYS_shmget 231
 #define SYS_clock_gettime 232
 #define SYS_clock_settime 233
 #define SYS_clock_getres 234
+#define SYS_ktimer_create 235
+#define SYS_ktimer_delete 236
+#define SYS_ktimer_settime 237
+#define SYS_ktimer_gettime 238
+#define SYS_ktimer_getoverrun 239
 #define SYS_nanosleep 240
+#define SYS_ntp_gettime 248
 #define SYS_minherit 250
 #define SYS_rfork 251
-#define SYS_poll 252
+#define SYS_openbsd_poll 252
 #define SYS_issetugid 253
 #define SYS_lchown 254
-#define SYS_getsid 255
-#define SYS_msync 256
-#define SYS_pipe 263
-#define SYS_fhopen 264
-#define SYS_preadv 267
-#define SYS_pwritev 268
-#define SYS_kqueue 269
-#define SYS_kevent 270
-#define SYS_mlockall 271
-#define SYS_munlockall 272
-#define SYS_getpeereid 273
-#define SYS_getresuid 281
-#define SYS_setresuid 282
-#define SYS_getresgid 283
-#define SYS_setresgid 284
-#define SYS_mquery 286
-#define SYS_closefrom 287
-#define SYS_sigaltstack 288
-#define SYS_shmget 289
-#define SYS_semop 290
-#define SYS_stat 291
-#define SYS_fstat 292
-#define SYS_lstat 293
-#define SYS_fhstat 294
-#define SYS___semctl 295
-#define SYS_shmctl 296
-#define SYS_msgctl 297
-#define SYS_sched_yield 298
-#define SYS_getthrid 299
-#define SYS_thrsleep 300
-#define SYS_thrwakeup 301
-#define SYS_threxit 302
-#define SYS_thrsigdivert 303
-#define SYS___getcwd 304
-#define SYS_adjfreq 305
-#define SYS_getfsstat 306
-#define SYS_statfs 307
-#define SYS_fstatfs 308
-#define SYS_fhstatfs 309
+#define SYS_aio_read 255
+#define SYS_aio_write 256
+#define SYS_lio_listio 257
+#define SYS_getdents 272
+#define SYS_lchmod 274
+#define SYS_netbsd_lchown 275
+#define SYS_lutimes 276
+#define SYS_netbsd_msync 277
+#define SYS_nstat 278
+#define SYS_nfstat 279
+#define SYS_nlstat 280
+#define SYS_preadv 289
+#define SYS_pwritev 290
+#define SYS_freebsd4_fhstatfs 297
+#define SYS_fhopen 298
+#define SYS_fhstat 299
+#define SYS_modnext 300
+#define SYS_modstat 301
+#define SYS_modfnext 302
+#define SYS_modfind 303
+#define SYS_kldload 304
+#define SYS_kldunload 305
+#define SYS_kldfind 306
+#define SYS_kldnext 307
+#define SYS_kldstat 308
+#define SYS_kldfirstmod 309
+#define SYS_getsid 310
+#define SYS_setresuid 311
+#define SYS_setresgid 312
+#define SYS_aio_return 314
+#define SYS_aio_suspend 315
+#define SYS_aio_cancel 316
+#define SYS_aio_error 317
+#define SYS_oaio_read 318
+#define SYS_oaio_write 319
+#define SYS_olio_listio 320
+#define SYS_yield 321
+#define SYS_mlockall 324
+#define SYS_munlockall 325
+#define SYS___getcwd 326
+#define SYS_sched_setparam 327
+#define SYS_sched_getparam 328
+#define SYS_sched_setscheduler 329
+#define SYS_sched_getscheduler 330
+#define SYS_sched_yield 331
+#define SYS_sched_get_priority_max 332
+#define SYS_sched_get_priority_min 333
+#define SYS_sched_rr_get_interval 334
+#define SYS_utrace 335
+#define SYS_freebsd4_sendfile 336
+#define SYS_kldsym 337
+#define SYS_jail 338
+#define SYS_sigprocmask 340
+#define SYS_sigsuspend 341
+#define SYS_freebsd4_sigaction 342
+#define SYS_sigpending 343
+#define SYS_freebsd4_sigreturn 344
+#define SYS_sigtimedwait 345
+#define SYS_sigwaitinfo 346
+#define SYS___acl_get_file 347
+#define SYS___acl_set_file 348
+#define SYS___acl_get_fd 349
+#define SYS___acl_set_fd 350
+#define SYS___acl_delete_file 351
+#define SYS___acl_delete_fd 352
+#define SYS___acl_aclcheck_file 353
+#define SYS___acl_aclcheck_fd 354
+#define SYS_extattrctl 355
+#define SYS_extattr_set_file 356
+#define SYS_extattr_get_file 357
+#define SYS_extattr_delete_file 358
+#define SYS_aio_waitcomplete 359
+#define SYS_getresuid 360
+#define SYS_getresgid 361
+#define SYS_kqueue 362
+#define SYS_kevent 363
+#define SYS_extattr_set_fd 371
+#define SYS_extattr_get_fd 372
+#define SYS_extattr_delete_fd 373
+#define SYS___setugid 374
+#define SYS_nfsclnt 375
+#define SYS_eaccess 376
+#define SYS_nmount 378
+#define SYS___mac_get_proc 384
+#define SYS___mac_set_proc 385
+#define SYS___mac_get_fd 386
+#define SYS___mac_get_file 387
+#define SYS___mac_set_fd 388
+#define SYS___mac_set_file 389
+#define SYS_kenv 390
+#define SYS_lchflags 391
+#define SYS_uuidgen 392
+#define SYS_sendfile 393
+#define SYS_mac_syscall 394
+#define SYS_getfsstat 395
+#define SYS_statfs 396
+#define SYS_fstatfs 397
+#define SYS_fhstatfs 398
+#define SYS_ksem_close 400
+#define SYS_ksem_post 401
+#define SYS_ksem_wait 402
+#define SYS_ksem_trywait 403
+#define SYS_ksem_init 404
+#define SYS_ksem_open 405
+#define SYS_ksem_unlink 406
+#define SYS_ksem_getvalue 407
+#define SYS_ksem_destroy 408
+#define SYS___mac_get_pid 409
+#define SYS___mac_get_link 410
+#define SYS___mac_set_link 411
+#define SYS_extattr_set_link 412
+#define SYS_extattr_get_link 413
+#define SYS_extattr_delete_link 414
+#define SYS___mac_execve 415
+#define SYS_sigaction 416
+#define SYS_sigreturn 417
+#define SYS_getcontext 421
+#define SYS_setcontext 422
+#define SYS_swapcontext 423
+#define SYS_swapoff 424
+#define SYS___acl_get_link 425
+#define SYS___acl_set_link 426
+#define SYS___acl_delete_link 427
+#define SYS___acl_aclcheck_link 428
+#define SYS_sigwait 429
+#define SYS_thr_create 430
+#define SYS_thr_exit 431
+#define SYS_thr_self 432
+#define SYS_thr_kill 433
+#define SYS__umtx_lock 434
+#define SYS__umtx_unlock 435
+#define SYS_jail_attach 436
+#define SYS_extattr_list_fd 437
+#define SYS_extattr_list_file 438
+#define SYS_extattr_list_link 439
+#define SYS_ksem_timedwait 441
+#define SYS_thr_suspend 442
+#define SYS_thr_wake 443
+#define SYS_kldunloadf 444
+#define SYS_audit 445
+#define SYS_auditon 446
+#define SYS_getauid 447
+#define SYS_setauid 448
+#define SYS_getaudit 449
+#define SYS_setaudit 450
+#define SYS_getaudit_addr 451
+#define SYS_setaudit_addr 452
+#define SYS_auditctl 453
+#define SYS__umtx_op 454
+#define SYS_thr_new 455
+#define SYS_sigqueue 456
+#define SYS_kmq_open 457
+#define SYS_kmq_setattr 458
+#define SYS_kmq_timedreceive 459
+#define SYS_kmq_timedsend 460
+#define SYS_kmq_notify 461
+#define SYS_kmq_unlink 462
+#define SYS_abort2 463
+#define SYS_thr_set_name 464
+#define SYS_aio_fsync 465
+#define SYS_rtprio_thread 466
+#define SYS_sctp_peeloff 471
+#define SYS_sctp_generic_sendmsg 472
+#define SYS_sctp_generic_sendmsg_iov 473
+#define SYS_sctp_generic_recvmsg 474
+#define SYS_pread 475
+#define SYS_pwrite 476
+#define SYS_mmap 477
+#define SYS_lseek 478
+#define SYS_truncate 479
+#define SYS_ftruncate 480
+#define SYS_thr_kill2 481
+#define SYS_shm_open 482
+#define SYS_shm_unlink 483
+#define SYS_cpuset 484
+#define SYS_cpuset_setid 485
+#define SYS_cpuset_getid 486
+#define SYS_cpuset_getaffinity 487
+#define SYS_cpuset_setaffinity 488
+#define SYS_faccessat 489
+#define SYS_fchmodat 490
+#define SYS_fchownat 491
+#define SYS_fexecve 492
+#define SYS_fstatat 493
+#define SYS_futimesat 494
+#define SYS_linkat 495
+#define SYS_mkdirat 496
+#define SYS_mkfifoat 497
+#define SYS_mknodat 498
+#define SYS_openat 499
+#define SYS_readlinkat 500
+#define SYS_renameat 501
+#define SYS_symlinkat 502
+#define SYS_unlinkat 503
+#define SYS_posix_openpt 504