[Qemu-devel] user emulation status?

[Qemu-devel] user emulation status?

[Tomasz Chmielewski]

According to the list of supported guest OSes on 
http://bellard.org/qemu/status.html, these architectures have "OK/green" 
status for user emulation:

x86, ARM, SPARC, PPC, MIPS, m68k, SH-4.

If these architectures are really fully supported by Qemu, it should be 
possible to run a chroot of another architecture (i.e. on x86 we chroot 
to ARM base filesystem/pcakages).

I thought I'd give it a try, with current Qemu SVN, by installing Debian 
for different architectures with debootstrap command (debootstrap allows 
to install base filesystem/packages).


Debian Etch supports ARM, SPARC, PPC and MIPS, so I only tested those on 
x86 PC.


Results:

1. ARM - support is really great.
debootstrap installs base filesystem/packages without any errors in an 
ARM chroot made on x86 PC.
It's even possible to start some GUI applications.


2. MIPS - support is not so great.
It's not possible to install a base filesystem.
Chrooting from x86 to an already existing MIPS filesystem works, but 
lots of commands just break (i.e., dpkg exits with "Invalid argument"; 
some stracing reveals more details).


3. PPC
Chrooting from x86 to a PPC filesystem ends with a segmentation fault.


4. SPARC
Chrooting from x86 to a SPARC filesystem works, but it's only possible 
to start one command. After that, qemu-sparc process dies with HELPME... 
(I used the default --sparc_cpu).


Are you people interested in running processes in a foreign chroot (i.e. 
ARM chroot on x86 PC)?


I wanted to set up some daily automated tests which would fetch current 
SVN, build for ARM, MIPS, PPC and SPARC targets, try to install Debian 
with debootstrap in a chroot.


So far I see though, that only ARM would get "OK" result, MIPS would get 
"partial"; PPC and SPARC would fail.

Does it make sense to trace it on a daily basis?



-- 
Tomasz Chmielewski
http://wpkg.org

Re: [Qemu-devel] user emulation status?

[Kirill A. Shutemov]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

On Sun, Sep 07, 2008 at 10:28:54PM +0200, Tomasz Chmielewski wrote:
> Are you people interested in running processes in a foreign chroot (i.e. 
> ARM chroot on x86 PC)?

Yes. I port ALT Linux to ARM using qemu-arm.

> I wanted to set up some daily automated tests which would fetch current 
> SVN, build for ARM, MIPS, PPC and SPARC targets, try to install Debian 
> with debootstrap in a chroot.
> 
> 
> So far I see though, that only ARM would get "OK" result, MIPS would get 
> "partial"; PPC and SPARC would fail.
> 
> Does it make sense to trace it on a daily basis?

Better weekly basis.

linux-user emulation has no active maintainer, so I don't think that
something change dramatically in one day. I have posted patches some week
ago but have no answer. :(

-- 
Regards,  Kirill A. Shutemov
 + Belarus, Minsk
 + ALT Linux Team, http://www.altlinux.com/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [Qemu-devel] user emulation status?

[Thiemo Seufer]

Tomasz Chmielewski wrote:
> According to the list of supported guest OSes on  
> http://bellard.org/qemu/status.html, these architectures have "OK/green"  
> status for user emulation:
>
> x86, ARM, SPARC, PPC, MIPS, m68k, SH-4.
>
> If these architectures are really fully supported by Qemu, it should be  
> possible to run a chroot of another architecture (i.e. on x86 we chroot  
> to ARM base filesystem/pcakages).
>
> I thought I'd give it a try, with current Qemu SVN, by installing Debian  
> for different architectures with debootstrap command (debootstrap allows  
> to install base filesystem/packages).
>
>
> Debian Etch supports ARM, SPARC, PPC and MIPS, so I only tested those on  
> x86 PC.
>
>
> Results:
>
> 1. ARM - support is really great.
> debootstrap installs base filesystem/packages without any errors in an  
> ARM chroot made on x86 PC.
> It's even possible to start some GUI applications.
>
>
> 2. MIPS - support is not so great.
> It's not possible to install a base filesystem.
> Chrooting from x86 to an already existing MIPS filesystem works, but  
> lots of commands just break (i.e., dpkg exits with "Invalid argument";  
> some stracing reveals more details).

I concentrate mostly on the MIPS system emulation. I figure the usermode
emulation needs a fair bit of debugging, particularily the 64-bit
variants.


Thiemo

Re: [Qemu-devel] user emulation status?

[Riku Voipio]

On Sun, Sep 07, 2008 at 10:28:54PM +0200, Tomasz Chmielewski wrote:
> Debian Etch supports ARM, SPARC, PPC and MIPS, so I only tested those on 
> x86 PC.

Notice that etch doesn't use many of new syscalls introduced recently,
and doesn't use TLS/NPTL on many archs. So it's not really a very hard
test for qemu-user.

> Are you people interested in running processes in a foreign chroot (i.e. 
> ARM chroot on x86 PC)?

Yes. And I need to get my lazy ass off and start cleaning maemo/debian
linux-user patches and propose them here.

> I wanted to set up some daily automated tests which would fetch current 
> SVN, build for ARM, MIPS, PPC and SPARC targets, try to install Debian 
> with debootstrap in a chroot.

Since what we are actually emulating, is linux kernel syscall interface,
the most exhaustive test would be LTP. but debootstrapping debian is
good smoketest. Other nice tests could be perl/python/glibc testsuites.

-- 
"rm -rf" only sounds scary if you don't have backups

Re: [Qemu-devel] user emulation status?

[Tomasz Chmielewski]

Riku Voipio schrieb:
> On Sun, Sep 07, 2008 at 10:28:54PM +0200, Tomasz Chmielewski wrote:
>> Debian Etch supports ARM, SPARC, PPC and MIPS, so I only tested those on 
>> x86 PC.
> 
> Notice that etch doesn't use many of new syscalls introduced recently,
> and doesn't use TLS/NPTL on many archs. So it's not really a very hard
> test for qemu-user.

Even then, ARM user emulation seem to be the only one fully working.
I tried also Lenny on ARM, it had some minor problems, but was working.

I could add Lenny to the testing suite, it's not a problem.


>> Are you people interested in running processes in a foreign chroot (i.e. 
>> ARM chroot on x86 PC)?
> 
> Yes. And I need to get my lazy ass off and start cleaning maemo/debian
> linux-user patches and propose them here.
> 
>> I wanted to set up some daily automated tests which would fetch current 
>> SVN, build for ARM, MIPS, PPC and SPARC targets, try to install Debian 
>> with debootstrap in a chroot.
> 
> Since what we are actually emulating, is linux kernel syscall interface,
> the most exhaustive test would be LTP. but debootstrapping debian is
> good smoketest. Other nice tests could be perl/python/glibc testsuites.

My motivation for running a "foreign chroot" is basically "apt-get 
update; apt-get upgrade" for some filesystem images I use on embedded 
devices running Debian.

It's easier and faster to do so on a PC than on real hardware or in 
qemu-system emulation.


Running LTP might be a good idea for running on those architectures that 
pass the "debootstrap smoketest".



-- 
Tomasz Chmielewski
http://wpkg.org

Re: [Qemu-devel] user emulation status?

[Stuart Anderson]

On Mon, 8 Sep 2008, Riku Voipio wrote:

> Since what we are actually emulating, is linux kernel syscall interface,
> the most exhaustive test would be LTP. but debootstrapping debian is
> good smoketest. Other nice tests could be perl/python/glibc testsuites.

You can find statically linked LTP binaries for a few architectures 
at http://qemu.olmeclinux.com/ltp/. These are what I have been using for
qemu debugging for a long time now. These are statically linked so they
can be run standalone and not require setting up the chroot, etc.


                                 Stuart

Stuart R. Anderson                               anderson@netsweng.com
Network & Software Engineering                   http://www.netsweng.com/
1024D/37A79149:                                  0791 D3B8 9A4C 2CDC A31F
                                                  BD03 0A62 E534 37A7 9149

Re: [Qemu-devel] user emulation status?

[Martin Mohring]

Hi,

is there a place to put testresults for svn snapshots?

Martin

Martin Mohring wrote:
> Hi,
>
> I am one of the developers of the openSUSE Buildservice (OBS), and 
> also the maintainer of the developer/test packages for OBS. OBS is an 
> automated system for compiling, testing and distributing packages, 
> projects and even complete distribtions on basis of .rpm or .deb 
> architectures (even windows/solaris/macos planned). Currently, we 
> support build for all major linux distribtions Fedora, openSUSE, 
> Debian, RHEL, SLES and Ubuntu.
>
> Recently, I have added support for cross-build to OBS, our first 
> target was Debin:Etch/arm and Maemo 4.1. I could now successfully 
> setup a buildenvironment for Debian:Etch/arm and compile lots of the 
> packages. The cross-build support we have implemented is based on: see 
> these two articles here 
> <http://lizards.opensuse.org/2008/08/31/hackweek-day-2-cross-build-with-obs-part-1/> 
> and here 
> <http://lizards.opensuse.org/2008/09/01/hackweek-day-3-cross-build-with-obs-part-2/>. 
> In the meantime, we have now a completely automated patch, so we can 
> use the normal OBS scheduler and so on (the only remaining issues were 
> the fakeroot-tcp vs. fakeroot-sysv and the /proc/sys/vm/mmap_min_addr 
> issue).
>
> To sum it up, our OBS can instantiate all linux distributions you can 
> think of in a chroot (even on basis of the original prebuild binary 
> .rpm or .deb), and can though be ideally be used for testing qemu in 
> all situations you can think of. Since we do not want to use qemu 
> system emulation for the obvious reasons, we would like qemu user mode 
> to just work.
>
> I would be happy to share with you your patches for qemu-arm user mode 
> to get Debian/Maemo builds working.
>
> I wrote already an e-mail on this here, but no reaction. Also, I am 
> too busy atm with development.
>
> Martin
>
> Riku Voipio wrote:
>> On Sun, Sep 07, 2008 at 10:28:54PM +0200, Tomasz Chmielewski wrote:
>>   
>>> Debian Etch supports ARM, SPARC, PPC and MIPS, so I only tested those on 
>>> x86 PC.
>>>     
>>
>> Notice that etch doesn't use many of new syscalls introduced recently,
>> and doesn't use TLS/NPTL on many archs. So it's not really a very hard
>> test for qemu-user.
>>
>>   
>>> Are you people interested in running processes in a foreign chroot (i.e. 
>>> ARM chroot on x86 PC)?
>>>     
>>
>> Yes. And I need to get my lazy ass off and start cleaning maemo/debian
>> linux-user patches and propose them here.
>>
>>   
>>> I wanted to set up some daily automated tests which would fetch current 
>>> SVN, build for ARM, MIPS, PPC and SPARC targets, try to install Debian 
>>> with debootstrap in a chroot.
>>>     
>>
>> Since what we are actually emulating, is linux kernel syscall interface,
>> the most exhaustive test would be LTP. but debootstrapping debian is
>> good smoketest. Other nice tests could be perl/python/glibc testsuites.
>>
>>   
>

Re: [Qemu-devel] user emulation status?

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 428 bytes --]

On Sun, Sep 07, 2008 at 10:28:54PM +0200, Tomasz Chmielewski <mangoo@wpkg.org> wrote:
> 3. PPC
> Chrooting from x86 to a PPC filesystem ends with a segmentation fault.

It works fine for me with non-NPTL binaries. Sadly this is not too
useful, since most PPC binary nowadays uses NPTL.

This issue was already reported by me a few weeks ago, as well as by the
SuSE guys a bit later.

AFAIK NPTL only works fine only on ARM ATM.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: [Qemu-devel] user emulation status?

[Martin Mohring]

Miklos Vajna wrote:
> On Sun, Sep 07, 2008 at 10:28:54PM +0200, Tomasz Chmielewski <mangoo@wpkg.org> wrote:
>   
>> 3. PPC
>> Chrooting from x86 to a PPC filesystem ends with a segmentation fault.
>>     
>
> It works fine for me with non-NPTL binaries. Sadly this is not too
> useful, since most PPC binary nowadays uses NPTL.
>
> This issue was already reported by me a few weeks ago, as well as by the
> SuSE guys a bit later.
>
> AFAIK NPTL only works fine only on ARM ATM.
>   
The guys porting BSD user mode told us that was astonished that syscall 
handling in linux does not use common code. Now I am a bit stonished 
too. Are things like syscalls and even more nptl not handled in common 
code? Or is the non common code just not implemented?

Re: [Qemu-devel] user emulation status?

[Paul Brook]

> > AFAIK NPTL only works fine only on ARM ATM.
>
> The guys porting BSD user mode told us that was astonished that syscall
> handling in linux does not use common code. Now I am a bit stonished
> too. Are things like syscalls and even more nptl not handled in common
> code? Or is the non common code just not implemented?

The kenrel/userspace interface for NPTL is actually very small.  There's a few 
extra arguments to clone, TLS register handling and synchronization 
primitives.

The clone arguments are pretty much the same on all architectures. Variations 
tend to be minor tweaks to do with passing a large number of arguments to a 
syscall.

Pretty much every architecture has its own mechanism for providing a TLS 
register. Some have dedicated hardware registers for this purpose, others use 
a variety of tricks to provide the value.  qemu needs to support this before 
even single threaded NPTL applications will work.

Synchronisation primitives operations come in two halves. futex syscalls which 
are the same on all targets, and atomic operations which are different for 
each architecture.  To make multithreaded applications work you need to 
ensure these atomic operations actually appear atomic.

Paul

Re: [Qemu-devel] user emulation status?

[michael]

Hi,
Paul Brook wrote:
> The kenrel/userspace interface for NPTL is actually very small.  There's a few 
> extra arguments to clone, TLS register handling and synchronization 
> primitives.
>   

> The clone arguments are pretty much the same on all architectures. Variations 
> tend to be minor tweaks to do with passing a large number of arguments to a 
> syscall.
>   
I change a little bit the qemu code for support nptl on the sh4. I try 
to send a patch for
the clone system call in sh4 (maybe is wrong !?!) and a have a simple 
filesystem to doing test
on this architecture.
I may test nptl support for sh4 or contribute on it?

Regards Michael

Re: [Qemu-devel] user emulation status?

[Martin Mohring]

Paul Brook wrote:
>>> AFAIK NPTL only works fine only on ARM ATM.
>>>       
>> The guys porting BSD user mode told us that was astonished that syscall
>> handling in linux does not use common code. Now I am a bit stonished
>> too. Are things like syscalls and even more nptl not handled in common
>> code? Or is the non common code just not implemented?
>>     
Btw, I was one of the "SUSE guys". I can offer testing qemu user by 
setting up every single chroot you whish (with every single linux distro 
you whish, not only static chroots). We can do that with our buildsystem 
now. Do you have a place for testresults, which help the developers?
>
> The kenrel/userspace interface for NPTL is actually very small.  There's a few 
> extra arguments to clone, TLS register handling and synchronization 
> primitives.
>
> The clone arguments are pretty much the same on all architectures. Variations 
> tend to be minor tweaks to do with passing a large number of arguments to a 
> syscall.
>
> Pretty much every architecture has its own mechanism for providing a TLS 
> register. Some have dedicated hardware registers for this purpose, others use 
> a variety of tricks to provide the value.  qemu needs to support this before 
> even single threaded NPTL applications will work.
>   
OK, understood. Its using the hardware optimizations provided by the 
processor isa.
> Synchronisation primitives operations come in two halves. futex syscalls which 
> are the same on all targets, and atomic operations which are different for 
> each architecture.  To make multithreaded applications work you need to 
> ensure these atomic operations actually appear atomic.
>   
Ok, understood. Another argument was: If multi-threading is provided, 
qemu user must be thread-safe itself.