odd(?) behaviour with qemu arm image and "runqemu-extract-sdk"

odd(?) behaviour with qemu arm image and "runqemu-extract-sdk"

[Robert P. J. Day]

  first time perusing these sections in the yocto docs, kind of
puzzling result so i'm prepared to believe i did something silly.
reading:

http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#workflow-using-stand-alone-cross-development-toolchains

"Workflow Using Stand-alone Cross-development Toolchains", and i
already had an OE qemuarm core-image-minimal built that i can run
just fine with:

$ runqemu qemuarm

so i was reading that section and followed the link over to the ADT
manual:

http://www.yoctoproject.org/docs/1.4/adt-manual/adt-manual.html#extracting-the-root-filesystem

where it talked about "Extracting the Root Filesystem", so i figured,
what the heck, i'll give that a shot, so i ran "runqemu-extract-sdk"
on the very rootfs tarball that had been created by my build and
extracted it under ~/rootfs, then without reading any further, tried
this variation of runqemu for the first time:

$ runqemu qemuarm ~/rootfs
Assuming /home/rpjday/rootfs is an nfs rootfs
Continuing with the following parameters:
KERNEL:
[/home/rpjday/oe/builds/oe/qemuarm/tmp-eglibc/deploy/images/zImage-qemuarm.bin]
ROOTFS: [/home/rpjday/rootfs]
FSTYPE: [nfs]
Setting up tap interface under sudo
[sudo] password for rpjday:
Acquiring lockfile for tap0...
runqemu-export-rootfs restart /home/rpjday/rootfs
Error: Unable to find rpc.mountd binary in
/home/rpjday/oe/builds/oe/qemuarm/tmp-eglibc/sysroots/x86_64-linux/usr/sbin/
Have you run 'bitbake meta-ide-support'?
Set 'tap0' nonpersistent
Releasing lockfile of preconfigured tap device 'tap0'
$

  so ... how many things did i do wrong?  some random thoughts on this
before i dig back into the docs and code:

1) why the name "runqemu-extract-sdk"?  it seems that the point of
that script is to just untar a root filesystem.  i don't see anything
being "extracted" per se.


2) i notice that the "dev" directory created by that script contains
only regular files instead of actual special device files.  but if i
use "sudo" to just untar exactly the same tarball, i get real device
files.  is that the expected behaviour of that script?

3) what should i have done earlier to have the apparently necessary
usr/sbin/rpc.mountd in my sysroot?  it seems odd that i can run my
built qemu image just fine if i run it normally, but if i
runqemu-extract-sdk to unload a rootfs, suddenly my sysroot is missing
that utility.  is the documentation simply missing some simple but
critical details about what types of images are required for this?

  and one more thing.  from that first link in the dev manual, there's
that note that takes you over to the ADT manual.  and it's entirely
possible that there some critical info *before* the section
"Extracting the Root Filesystem" but given how one got there (by
following the link), the reader might miss it.

  anyway, thoughts?

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: odd(?) behaviour with qemu arm image and "runqemu-extract-sdk"

[Scott Garman]

On 12/29/2012 09:33 AM, Robert P. J. Day wrote:
>
>    first time perusing these sections in the yocto docs, kind of
> puzzling result so i'm prepared to believe i did something silly.
> reading:
>
> http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#workflow-using-stand-alone-cross-development-toolchains
>
> "Workflow Using Stand-alone Cross-development Toolchains", and i
> already had an OE qemuarm core-image-minimal built that i can run
> just fine with:
>
> $ runqemu qemuarm
>
> so i was reading that section and followed the link over to the ADT
> manual:
>
> http://www.yoctoproject.org/docs/1.4/adt-manual/adt-manual.html#extracting-the-root-filesystem
>
> where it talked about "Extracting the Root Filesystem", so i figured,
> what the heck, i'll give that a shot, so i ran "runqemu-extract-sdk"
> on the very rootfs tarball that had been created by my build and
> extracted it under ~/rootfs, then without reading any further, tried
> this variation of runqemu for the first time:
>
> $ runqemu qemuarm ~/rootfs
> Assuming /home/rpjday/rootfs is an nfs rootfs
> Continuing with the following parameters:
> KERNEL:
> [/home/rpjday/oe/builds/oe/qemuarm/tmp-eglibc/deploy/images/zImage-qemuarm.bin]
> ROOTFS: [/home/rpjday/rootfs]
> FSTYPE: [nfs]
> Setting up tap interface under sudo
> [sudo] password for rpjday:
> Acquiring lockfile for tap0...
> runqemu-export-rootfs restart /home/rpjday/rootfs
> Error: Unable to find rpc.mountd binary in
> /home/rpjday/oe/builds/oe/qemuarm/tmp-eglibc/sysroots/x86_64-linux/usr/sbin/
> Have you run 'bitbake meta-ide-support'?
> Set 'tap0' nonpersistent
> Releasing lockfile of preconfigured tap device 'tap0'
> $
>
>    so ... how many things did i do wrong?

Hi Robert,

When you built your original image, it generated a filesystem in a file 
(in this case, an ext3 one) which the runqemu script used to boot with.

But booting an nfs-based image with runqemu requires our userspace NFS 
server. This is a new requirement that requires you to build the 
meta-ide-support target.

So in this case, what you're seeing is exactly what is intended - the 
runqemu script tells you what you need to do to proceed. You did nothing 
wrong.

There was a time (a long time ago) when meta-ide-support was built 
anytime qemu was built, but we ended up splitting it out since it added 
an unnecessary build dependency on meta-ide-support when most users 
never needed to boot an NFS image.

   some random thoughts on this
> before i dig back into the docs and code:
>
> 1) why the name "runqemu-extract-sdk"?  it seems that the point of
> that script is to just untar a root filesystem.  i don't see anything
> being "extracted" per se.

I suppose runqemu-extract-rootfs might be a better name for it. When the 
script was first developed, the main use case for it was extracting the 
-sdk images so they could be used with our application developer tools.

The key thing this script does is extract the rootfs tarball under 
pseudo, which creates a virtual mapping of file ownership that is 
necessary for the rootfs to be usable but not require root access from 
the end user's perspective.

> 2) i notice that the "dev" directory created by that script contains
> only regular files instead of actual special device files.  but if i
> use "sudo" to just untar exactly the same tarball, i get real device
> files.  is that the expected behaviour of that script?

Yes - since the invokation of pseudo from the runqemu-extract-rootfs 
script creates a pseudo database that will later be used when runqemu 
boots the rootfs to read the files.

> 3) what should i have done earlier to have the apparently necessary
> usr/sbin/rpc.mountd in my sysroot?

Just what the script said: bitbake meta-ide-support


   it seems odd that i can run my
> built qemu image just fine if i run it normally, but if i
> runqemu-extract-sdk to unload a rootfs, suddenly my sysroot is missing
> that utility.

By now I hope it should be clear that the rpc.mountd was not needed to 
boot a .ext3 image. It's only needed for NFS-based booting. So the file 
didn't suddenly disappear when you went to boot the NFS-based rootfs. It 
was just needed for the first time then.

>    and one more thing.  from that first link in the dev manual, there's
> that note that takes you over to the ADT manual.  and it's entirely
> possible that there some critical info *before* the section
> "Extracting the Root Filesystem" but given how one got there (by
> following the link), the reader might miss it.

I'll have to leave this one to our app dev folks and ScottR if they 
agree something should be changed.

Scott

-- 
Scott Garman
Embedded Linux Engineer - Yocto Project
Intel Open Source Technology Center

Re: odd(?) behaviour with qemu arm image and "runqemu-extract-sdk"

[Robert P. J. Day]

On Tue, 1 Jan 2013, Scott Garman wrote:

> On 12/29/2012 09:33 AM, Robert P. J. Day wrote:
> >
> >    first time perusing these sections in the yocto docs, kind of
> > puzzling result so i'm prepared to believe i did something silly.
> > reading:
> >
> > http://www.yoctoproject.org/docs/latest/dev-manual/dev-manual.html#workflow-using-stand-alone-cross-development-toolchains
> >
> > "Workflow Using Stand-alone Cross-development Toolchains", and i
> > already had an OE qemuarm core-image-minimal built that i can run
> > just fine with:
> >
> > $ runqemu qemuarm
> >
> > so i was reading that section and followed the link over to the ADT
> > manual:
> >
> > http://www.yoctoproject.org/docs/1.4/adt-manual/adt-manual.html#extracting-the-root-filesystem
> >
> > where it talked about "Extracting the Root Filesystem", so i figured,
> > what the heck, i'll give that a shot, so i ran "runqemu-extract-sdk"
> > on the very rootfs tarball that had been created by my build and
> > extracted it under ~/rootfs, then without reading any further, tried
> > this variation of runqemu for the first time:
> >
> > $ runqemu qemuarm ~/rootfs
> > Assuming /home/rpjday/rootfs is an nfs rootfs
> > Continuing with the following parameters:
> > KERNEL:
> > [/home/rpjday/oe/builds/oe/qemuarm/tmp-eglibc/deploy/images/zImage-qemuarm.bin]
> > ROOTFS: [/home/rpjday/rootfs]
> > FSTYPE: [nfs]
> > Setting up tap interface under sudo
> > [sudo] password for rpjday:
> > Acquiring lockfile for tap0...
> > runqemu-export-rootfs restart /home/rpjday/rootfs
> > Error: Unable to find rpc.mountd binary in
> > /home/rpjday/oe/builds/oe/qemuarm/tmp-eglibc/sysroots/x86_64-linux/usr/sbin/
> > Have you run 'bitbake meta-ide-support'?
> > Set 'tap0' nonpersistent
> > Releasing lockfile of preconfigured tap device 'tap0'
> > $
> >
> >    so ... how many things did i do wrong?
>
> Hi Robert,
>
> When you built your original image, it generated a filesystem in a
> file (in this case, an ext3 one) which the runqemu script used to
> boot with.
>
> But booting an nfs-based image with runqemu requires our userspace
> NFS server. This is a new requirement that requires you to build the
> meta-ide-support target.
>
> So in this case, what you're seeing is exactly what is intended -
> the runqemu script tells you what you need to do to proceed. You did
> nothing wrong.
>
> There was a time (a long time ago) when meta-ide-support was built
> anytime qemu was built, but we ended up splitting it out since it
> added an unnecessary build dependency on meta-ide-support when most
> users never needed to boot an NFS image.

  that's fine ... it's probably worth adding that tidbit to the docs
since the only thing that was concerning me was trying something
according to the docs and have it come back saying, "wait, did you do
X first?" when i didn't recall anything suggesting i do X in the first
place.  and i didn't want to simply follow the advice and do X until i
verified that i didn't miss the advice to run X earlier.

  ok, that's the mimosas talking ...

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================

Re: odd(?) behaviour with qemu arm image and "runqemu-extract-sdk"

[Robert P. J. Day]

On Tue, 1 Jan 2013, Scott Garman wrote:

> On 12/29/2012 09:33 AM, Robert P. J. Day wrote:

> >    and one more thing.  from that first link in the dev manual,
> > there's that note that takes you over to the ADT manual.  and it's
> > entirely possible that there some critical info *before* the
> > section "Extracting the Root Filesystem" but given how one got
> > there (by following the link), the reader might miss it.
>
> I'll have to leave this one to our app dev folks and ScottR if they
> agree something should be changed.

  that's the danger of linking from one manual to another -- it's
sometimes hard to tell if you're linking *after* fairly important
info at the destination.

rday

-- 

========================================================================
Robert P. J. Day                                 Ottawa, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================