[Qemu-devel] chroot to ARM Gentoo install.

[Qemu-devel] chroot to ARM Gentoo install.

[James Pellow]

Hi All,

Thanks for a truely awesome tool!  I have some custom ARM hardware that I have 
ported the linux kernel to, and for fun I want to install Gentoo on it.  
Since gentoo requires a fair piece of compiling (gasp...) and since my board 
only runs at 74MHz, I thought I would try chroot'ing into ARM gentoo using 
qemu on my Athlon system.  (Yes, I have tried distcc...  Its still slow.)  I 
have been able to set up binfmt_misc and a statically compiled qemu-arm to 
chroot into an arm busybox install (uClibc), but for some reason I get the 
following error when I try to chroot into a gentoo root (full glibc):

cherypit / # chroot ./gentoo /bin/bash
Inconsistency detected by ld.so: dynamic-link.h: 150: elf_get_dynamic_info: 
Assertion `info[20]->d_un.d_val == 17 || info[20]->d_un.d_val == 7' failed!

The gentoo install is mature enough to boot normally, and all works fine on 
real hardware, so I am thinking this may be a qemu problem.

So, about my host system:

Athlon 1.4GHz, 768M ram.

$ cat /proc/version:
Linux version 2.6.7 (root@cherypit) (gcc version 3.3.3 20040412 (Gentoo Linux 
3.3.3-r6, ssp-3.3.2-2, pie-8.7.6)) #2 Wed Jul 21 16:01:54 PDT 2004

libc-2.3.3

qemu-arm version 0.6.0 (statically linked to the above libc and compiled with 
gcc 3.3.3)
(Compiled lib search path to '/' IIRC)

Any ideas?  Thanks for your time.

Regards,

-- 
----------------------------
James Pellow, President
Alent Design Solutions
www.alentdesignsolutions.com
509.526.0682
----------------------------

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Paul Brook]

On Tuesday 24 August 2004 20:38, James Pellow wrote:
> Hi All,
>
> Thanks for a truely awesome tool!  I have some custom ARM hardware that I
> have ported the linux kernel to, and for fun I want to install Gentoo on
> it. Since gentoo requires a fair piece of compiling (gasp...) and since my
> board only runs at 74MHz, I thought I would try chroot'ing into ARM gentoo
> using qemu on my Athlon system.  (Yes, I have tried distcc...  Its still
> slow.)  I have been able to set up binfmt_misc and a statically compiled
> qemu-arm to chroot into an arm busybox install (uClibc), but for some
> reason I get the following error when I try to chroot into a gentoo root
> (full glibc):

You might see if the this patch helps:
http://lists.gnu.org/archive/html/qemu-devel/2004-08/msg00015.html

You don't mention what sort of ARM chip you are using. qemu only supports 
armv4. It does not currently implement the armv4t or armv5 instruction sets. 
I have patches implementing armv5te, but they still need testing.

Paul

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Lennert Buytenhek]

On Tue, Aug 24, 2004 at 09:06:57PM +0100, Paul Brook wrote:

> You don't mention what sort of ARM chip you are using. qemu only supports 
> armv4. It does not currently implement the armv4t or armv5 instruction sets. 
> I have patches implementing armv5te, but they still need testing.

Can I volunteer?


--L

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Paul Brook]

[-- Attachment #1: Type: text/plain, Size: 619 bytes --]

On Tuesday 24 August 2004 22:26, Lennert Buytenhek wrote:
> On Tue, Aug 24, 2004 at 09:06:57PM +0100, Paul Brook wrote:
> > You don't mention what sort of ARM chip you are using. qemu only supports
> > armv4. It does not currently implement the armv4t or armv5 instruction
> > sets. I have patches implementing armv5te, but they still need testing.
>
> Can I volunteer?

Sure. Patch attached. This includes a couple of other minor changes I've got 
in my local tree.

I've run the gcc testsuite in arm mode (cross-compiling with an an armv4 libc) 
and the results look reasonable. Thumb mode is totally untested.

Paul

[-- Attachment #2: patch.qemu_armv5te.bz2 --]
[-- Type: application/x-bzip2, Size: 7237 bytes --]

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Fabrice Bellard]

Paul Brook wrote:
> On Tuesday 24 August 2004 22:26, Lennert Buytenhek wrote:
> 
>>On Tue, Aug 24, 2004 at 09:06:57PM +0100, Paul Brook wrote:
>>
>>>You don't mention what sort of ARM chip you are using. qemu only supports
>>>armv4. It does not currently implement the armv4t or armv5 instruction
>>>sets. I have patches implementing armv5te, but they still need testing.
>>
>>Can I volunteer?
> 
> 
> Sure. Patch attached. This includes a couple of other minor changes I've got 
> in my local tree.
> 
> I've run the gcc testsuite in arm mode (cross-compiling with an an armv4 libc) 
> and the results look reasonable. Thumb mode is totally untested.

Now you can begin Strong ARM system mode emulation ! Simulating a 
Netwinder hardware should be possible with the existing PC devices.

Fabrice.

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Paul Brook]

On Tuesday 24 August 2004 23:14, Fabrice Bellard wrote:
> Paul Brook wrote:
> > On Tuesday 24 August 2004 22:26, Lennert Buytenhek wrote:
> >>On Tue, Aug 24, 2004 at 09:06:57PM +0100, Paul Brook wrote:
> >>>You don't mention what sort of ARM chip you are using. qemu only
> >>> supports armv4. It does not currently implement the armv4t or armv5
> >>> instruction sets. I have patches implementing armv5te, but they still
> >>> need testing.
> >>
> >>Can I volunteer?
> >
> > Sure. Patch attached. This includes a couple of other minor changes I've
> > got in my local tree.
> >
> > I've run the gcc testsuite in arm mode (cross-compiling with an an armv4
> > libc) and the results look reasonable. Thumb mode is totally untested.
>
> Now you can begin Strong ARM system mode emulation ! Simulating a
> Netwinder hardware should be possible with the existing PC devices.

First we'd need to implement privileged CPU modes, system control coprocessor, 
and maybe MMU...

I'm a toolchain developer, not a kernel hacker, so I'm mostly interested in 
user-mode emulation. Full system emulation is probably not gonna happen 
unless I get really bored, or someone else does it ;)

Armv6 (ie arm11 series cores) and VFP coprocessor are higher priority for me.

Paul

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Fabrice Bellard]

Paul Brook wrote:

>>Now you can begin Strong ARM system mode emulation ! Simulating a
>>Netwinder hardware should be possible with the existing PC devices.
> 
> 
> First we'd need to implement privileged CPU modes, system control coprocessor, 
> and maybe MMU...
> 
> I'm a toolchain developer, not a kernel hacker, so I'm mostly interested in 
> user-mode emulation. Full system emulation is probably not gonna happen 
> unless I get really bored, or someone else does it ;)
> 
> Armv6 (ie arm11 series cores) and VFP coprocessor are higher priority for me.

OK. I'll try to merge your last ARM patch ASAP. I am curious to know 
what speed you get in user mode emulation.

Fabrice.

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Paul Brook]

On Wednesday 25 August 2004 23:36, Fabrice Bellard wrote:
> I am curious to know what speed you get in user mode emulation.

I did some benchmarking on on a 2GHz amd64 machine running in 32-bit mode.
I don't have a real arm machine for comparison. 

I did two runs. One native (i486), the other soft-float armv4 using qemu. 

First I ran the SPECint benchmarks. These mainly stress integer CPU and memory 
performance. I'm not sure if I'm allowed to give actual results, so I'll just 
give a summary for now.

Five tests fail to execute properly. I haven't tried the executables on real 
arm hardware, so these may be compiler/libc bugs.

One test runs a mere 60% slower. The other six tests take between 3 and 7 
times as long to run.

I also ran the bonnie++ disk IO benchmark. The machine was under intermittent 
light load while the tests were run, so there's probably some variation in 
the numbers.

Native:
 1.03       ------Sequential Output------ --Sequential Input- --Random-
            -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
       Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
         4G 30667  75 31279  12 14788   5 21951  60 34922   5 161.0   0
            ------Sequential Create------ --------Random Create--------
            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
      files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
         16  2296  98 +++++ +++ +++++ +++  2267  97 +++++ +++ 10236  99

qemu-arm:
 1.03       ------Sequential Output------ --Sequential Input- --Random-
            -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
       Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
         4G     0   0 31734  10     0   0 +++++ +++ 36888   5 167.8   0
            ------Sequential Create------ --------Random Create--------
            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
      files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
         16  2733  97 +++++ +++ +++++ +++  2863  98 +++++ +++  9947  93

Strangely the qemu results are showing zero for per-char IO. I guess this is a 
bug somewhere. The per-char write results should be ~6000K/s

Paul

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Lennert Buytenhek]

On Tue, Aug 24, 2004 at 10:44:39PM +0100, Paul Brook wrote:

> > > You don't mention what sort of ARM chip you are using. qemu only supports
> > > armv4. It does not currently implement the armv4t or armv5 instruction
> > > sets. I have patches implementing armv5te, but they still need testing.
> >
> > Can I volunteer?
> 
> Sure. Patch attached. This includes a couple of other minor changes
> I've got in my local tree.
> 
> I've run the gcc testsuite in arm mode (cross-compiling with an an
> armv4 libc) and the results look reasonable. Thumb mode is totally
> untested.

Do you happen to have a newer version of this patch somewhere?  I tried
applying this against a qemu CVS snapshot made today and qemu-arm just
seems to go into an infinite loop.

I think there's but one or two issues preventing a 'native' ARM glibc
build under qemu.  Something in my 'make' binary is driving qemu-arm
crazy, because I consistently get this (without your v5 patch):

[...]
              Trying implicit prerequisite `config.sym'.
              Trying pattern rule with stem `config'.
              Trying rule prerequisite `scripts/gen-as-const.awk'.
              Trying implicit prerequisite `config.sym'.
              Looking for a rule with intermediate file `config.sym'.
qemu: uncaught target signal 11 (Segmentation fault) - exiting
sh-2.05b# 

(I did apply the patch.qemu_selfmodify patch you posted a while ago.)


--L

Re: [Qemu-devel] chroot to ARM Gentoo install.

[James Pellow]

On Tuesday 24 August 2004 01:06 pm, Paul Brook wrote:
> On Tuesday 24 August 2004 20:38, James Pellow wrote:
> > Hi All,
> >
> > Thanks for a truely awesome tool!  I have some custom ARM hardware that I
> > have ported the linux kernel to, and for fun I want to install Gentoo on
> > it. Since gentoo requires a fair piece of compiling (gasp...) and since
> > my board only runs at 74MHz, I thought I would try chroot'ing into ARM
> > gentoo using qemu on my Athlon system.  (Yes, I have tried distcc...  Its
> > still slow.)  I have been able to set up binfmt_misc and a statically
> > compiled qemu-arm to chroot into an arm busybox install (uClibc), but for
> > some reason I get the following error when I try to chroot into a gentoo
> > root (full glibc):
>
> You might see if the this patch helps:
> http://lists.gnu.org/archive/html/qemu-devel/2004-08/msg00015.html
>
> You don't mention what sort of ARM chip you are using. qemu only supports
> armv4. It does not currently implement the armv4t or armv5 instruction
> sets. I have patches implementing armv5te, but they still need testing.
>
> Paul

Hi Paul,

Thanks for the quick reply.  I am using an armv4 chip, specifically an 
arm720tdmi.  All code has been built for armv4.

I tried the patch you pointed me to.  Unfortunately I am still seeing the same 
results as previously noted.  Looking at the source for ld.so, I note that 
this is all happening in the elf parsing code.  The actual assert looks like:

      assert (info[DT_PLTREL]->d_un.d_val == DT_REL
              || info[DT_PLTREL]->d_un.d_val == DT_RELA);
(elf/dynamic-link.h:126)

for the case where !ELF_MACHINE_NO_RELA && !ELF_MACHINE_NO_REL are not 
defined...  That didn't make much sense to me yet, as I haven't poked around 
in glibc much.  Its hard for me to imagine that the problem is in glibc 
though since it shouldn't know the difference between running natively and 
running under qemu. (Or is there a subtle difference?)  It more obviously 
seems like elf should parse same either in emulation or not.  Since qemu 
doesn't signal, how should I procede in debugging this?  Any further ideas?  
Thanks for your help.

Regards,

-- 
----------------------------
James Pellow, President
Alent Design Solutions
www.alentdesignsolutions.com
509.526.0682
----------------------------

Re: [Qemu-devel] chroot to ARM Gentoo install.

[Paul Brook]

On Tuesday 24 August 2004 23:44, James Pellow wrote:
> Thanks for the quick reply.  I am using an armv4 chip, specifically an
> arm720tdmi.  All code has been built for armv4.
>
> I tried the patch you pointed me to.  Unfortunately I am still seeing the
> same results as previously noted.  Looking at the source for ld.so, I note
> that this is all happening in the elf parsing code.  The actual assert
> looks like:
>
>       assert (info[DT_PLTREL]->d_un.d_val == DT_REL
>               || info[DT_PLTREL]->d_un.d_val == DT_RELA);
>
> (elf/dynamic-link.h:126)
>
> for the case where !ELF_MACHINE_NO_RELA && !ELF_MACHINE_NO_REL are not
> defined...  That didn't make much sense to me yet, as I haven't poked
> around in glibc much.  Its hard for me to imagine that the problem is in
> glibc though since it shouldn't know the difference between running
> natively and running under qemu. (Or is there a subtle difference?)  It
> more obviously seems like elf should parse same either in emulation or not.
>  Since qemu doesn't signal, how should I procede in debugging this?  Any
> further ideas? Thanks for your help.

No idea, sorry.

Paul