[Buildroot] RFC: "make emu" build and start QEMU target

[Buildroot] RFC: "make emu" build and start QEMU target

[Spenser Gilliland]

Hi,

I'm proposing a project where a "make emu" target would be added to
Buildroot in order to start a virtualized version of the target.  This
would provide developers with the ability to quickly test and validate
their configuration.

To configure the emu target a new top-level menu, "Target Emulation,"
would be added.  This menu would contain the following options.

Emulation Method -> QEMU
QEMU Version -> Custom Git Tree | Custom tarball | Custom Version | X.Y.Z
... Source Configuration similar to Kernel ...
Machine -> list of machines
Memory -> Memory in Megabytes
Additional QEMU Options -> (additional options for command line)

This will start the qemu-system-$(ARCH) version of QEMU with the
appropriate configuration.

Thanks,
Spenser

--
Spenser Gilliland
Computer Engineer
Doctoral Candidate

[Buildroot] RFC: "make emu" build and start QEMU target

[Jeremy Rosen]

----- Mail original -----
> Hi,
> 
> I'm proposing a project where a "make emu" target would be added to
> Buildroot in order to start a virtualized version of the target.
>  This
> would provide developers with the ability to quickly test and
> validate
> their configuration.


I'd be interested in that, I have a few patch for qemu flying around
and having a quick way to test them would help

(I know testing qemu is not the point of your patch, but it would 
be usefull for that)

> 
> To configure the emu target a new top-level menu, "Target Emulation,"
> would be added.  This menu would contain the following options.
> 
> Emulation Method -> QEMU
> QEMU Version -> Custom Git Tree | Custom tarball | Custom Version |
> X.Y.Z
> ... Source Configuration similar to Kernel ...
> Machine -> list of machines
> Memory -> Memory in Megabytes
> Additional QEMU Options -> (additional options for command line)
> 
> This will start the qemu-system-$(ARCH) version of QEMU with the
> appropriate configuration.
> 

hmm... not sure if target emulation makes sense as a separate entry 
or if the qemu targets should be simply different machines.


> Thanks,
> Spenser
> 
> --
> Spenser Gilliland
> Computer Engineer
> Doctoral Candidate
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

[Buildroot] RFC: "make emu" build and start QEMU target

[Spenser Gilliland]

Jeremy,

Thanks for responding.

> I'd be interested in that, I have a few patch for qemu flying around
> and having a quick way to test them would help
>
> (I know testing qemu is not the point of your patch, but it would
> be usefull for that)

Very much so this is a good use case.  I looked in patchwork but
couldn't find any patches related to QEMU.  Can you point me to them?

> hmm... not sure if target emulation makes sense as a separate entry
> or if the qemu targets should be simply different machines.

I'm not sure what you mean by this can you explain further?

Spenser

--
Spenser Gilliland
Computer Engineer
Doctoral Candidate

[Buildroot] RFC: "make emu" build and start QEMU target

[Jeremy Rosen]

    Cordialement

    J?r?my Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> Jeremy,
> 
> Thanks for responding.
> 
> > I'd be interested in that, I have a few patch for qemu flying
> > around
> > and having a quick way to test them would help
> >
> > (I know testing qemu is not the point of your patch, but it would
> > be usefull for that)
> 
> Very much so this is a good use case.  I looked in patchwork but
> couldn't find any patches related to QEMU.  Can you point me to them?
> 

they are not in patchwork we work on QEMU and submit upstream stuff that is 
not buildroot specific and shouldn't be a buildroot patch

> > hmm... not sure if target emulation makes sense as a separate entry
> > or if the qemu targets should be simply different machines.
> 
> I'm not sure what you mean by this can you explain further?
> 


I was commenting on the idea of having an emulation sub-menu

> Spenser
> 
> --
> Spenser Gilliland
> Computer Engineer
> Doctoral Candidate
> 

[Buildroot] RFC: "make emu" build and start QEMU target

[Arnout Vandecappelle]

On 17/04/13 20:32, Spenser Gilliland wrote:
> Hi,
>
> I'm proposing a project where a "make emu" target would be added to
> Buildroot in order to start a virtualized version of the target.  This
> would provide developers with the ability to quickly test and validate
> their configuration.

  Great idea!

> To configure the emu target a new top-level menu, "Target Emulation,"
> would be added.  This menu would contain the following options.

  I think  a single 'QEMU emulation of target' menu is sufficient. Can 
you think of another emulator?

  And maybe it belongs in the host-tools menu, rather than top-level.

> Emulation Method -> QEMU
> QEMU Version -> Custom Git Tree | Custom tarball | Custom Version | X.Y.Z
> ... Source Configuration similar to Kernel ...
> Machine -> list of machines

  I think the number of machines is a bit too large to list them. Just 
specify a string, like a kernel defconfig or a u-boot config.

> Memory -> Memory in Megabytes
> Additional QEMU Options -> (additional options for command line)

  Probably more options (devices, graphics output, #CPUs) should be 
configurable, but that can be added in later steps.

> This will start the qemu-system-$(ARCH) version of QEMU with the
> appropriate configuration.

  I'm not a big fan of starting tools like qemu or a debugger or whatever 
from the buildroot 'make' command. I would prefer to generate a script 
that runs qemu with the correct arguments. You can easily start that 
script either with 'make && output/images/run_qemu.sh' or from a 
post-build script. But others may disagree with me.


  Would you generate the correct -drive option automatically from the 
selected rootfs, or select the rootfs automatically?

  Regards,
  Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] RFC: "make emu" build and start QEMU target

[Thomas Petazzoni]

Dear Spenser Gilliland,

On Wed, 17 Apr 2013 13:32:07 -0500, Spenser Gilliland wrote:

> I'm proposing a project where a "make emu" target would be added to
> Buildroot in order to start a virtualized version of the target.  This
> would provide developers with the ability to quickly test and validate
> their configuration.

Thanks for your proposal!

> To configure the emu target a new top-level menu, "Target Emulation,"
> would be added.  This menu would contain the following options.
> 
> Emulation Method -> QEMU
> QEMU Version -> Custom Git Tree | Custom tarball | Custom Version |
> X.Y.Z ... Source Configuration similar to Kernel ...
> Machine -> list of machines
> Memory -> Memory in Megabytes
> Additional QEMU Options -> (additional options for command line)
> 
> This will start the qemu-system-$(ARCH) version of QEMU with the
> appropriate configuration.

I am not sure to understand the reasoning behind this proposal.
Buildroot is a tool that, given a .config file, generates a
toolchain+root filesystem+kernel+bootloader for a given platform. We
already have defconfigs for various platforms, some of them being real
hardware platforms, some others being emulated platforms in Qemu.

We've worked really hard to get rid of all the board-specific code
that used to be in Buildroot, and what you're proposing seems like
reintroducing this.

I have nothing against adding a host-qemu package, but for the rest, I
don't understand the need. Each Qemu defconfig is associated with a
readme.txt file in board/qemu/<something>/ that explains how to start
the emulation. Just like we have a readme.txt file for some platforms
in board/<foo>/<bar> that explain how to format the SD card or other
platform-specific details.

I don't see why Qemu platforms should be handled differently.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] RFC: "make emu" build and start QEMU target

[Spenser Gilliland]

Thomas & Arnout,

> I am not sure to understand the reasoning behind this proposal.
> Buildroot is a tool that, given a .config file, generates a
> toolchain+root filesystem+kernel+bootloader for a given platform. We
> already have defconfigs for various platforms, some of them being real
> hardware platforms, some others being emulated platforms in Qemu.

QEMU is such a large part of embedded development.  It's an integral
part of most work flows.  In many cases there is a one-to-one
relationship between a machine configuration in QEMU and an embedded
target.  Thus, creating a configuration for a board to specify it's
QEMU configuration ensures that it's easy to emulate.  Sometimes, the
vendor provides their own QEMU fork; thus, the need for a more "kernel
like" configuration menu.

> We've worked really hard to get rid of all the board-specific code
> that used to be in Buildroot, and what you're proposing seems like
> reintroducing this.

I anticipate that most configuration would occur in the additional
options.  Thus, we avoid creating to many board specific attributes
and would use defconfigs for board specific configuration.

> I have nothing against adding a host-qemu package, but for the rest, I
> don't understand the need. Each Qemu defconfig is associated with a
> readme.txt file in board/qemu/<something>/ that explains how to start
> the emulation. Just like we have a readme.txt file for some platforms
> in board/<foo>/<bar> that explain how to format the SD card or other
> platform-specific details.

It seems like the "make emu" option is what is your disagreeing with.
I think Arnout's idea of generating a script may be more appropriate.


>  I think  a single 'QEMU emulation of target' menu is sufficient. Can
> you think of another emulator?

There's Dynampis, and Virtualbox but I don't think many people use
them for embedded.  Perhaps it's better to go with "QEMU Target
Emulation."

>  And maybe it belongs in the host-tools menu, rather than top-level.

I agree.

> I think the number of machines is a bit too large to list them. Just
> specify a string, like a kernel defconfig or a u-boot config.

Good idea and avoids the maintenance hassle of updating this entry.

> I'm not a big fan of starting tools like qemu or a debugger or whatever
> from the buildroot 'make' command. I would prefer to generate a script
> that runs qemu with the correct arguments. You can easily start that
> script either with 'make && output/images/run_qemu.sh' or from a
> post-build script. But others may disagree with me.

Good point.  See above.

> Would you generate the correct -drive option automatically from the
> selected rootfs, or select the rootfs automatically?

I don't think we should go this far.  The -drive option should be an
optional attribute.

Spenser






--
Spenser Gilliland
Computer Engineer
Doctoral Candidate

[Buildroot] RFC: "make emu" build and start QEMU target

[Rob Landley]

On 04/19/2013 05:56:05 PM, Thomas Petazzoni wrote:
> Dear Spenser Gilliland,
> 
> On Wed, 17 Apr 2013 13:32:07 -0500, Spenser Gilliland wrote:
> 
> > I'm proposing a project where a "make emu" target would be added to
> > Buildroot in order to start a virtualized version of the target.   
> This
> > would provide developers with the ability to quickly test and  
> validate
> > their configuration.
> 
> Thanks for your proposal!
> 
> > To configure the emu target a new top-level menu, "Target  
> Emulation,"
> > would be added.  This menu would contain the following options.
> >
> > Emulation Method -> QEMU
> > QEMU Version -> Custom Git Tree | Custom tarball | Custom Version |
> > X.Y.Z ... Source Configuration similar to Kernel ...
> > Machine -> list of machines
> > Memory -> Memory in Megabytes
> > Additional QEMU Options -> (additional options for command line)
> >
> > This will start the qemu-system-$(ARCH) version of QEMU with the
> > appropriate configuration.
> 
> I am not sure to understand the reasoning behind this proposal.
> Buildroot is a tool that, given a .config file, generates a
> toolchain+root filesystem+kernel+bootloader for a given platform. We
> already have defconfigs for various platforms, some of them being real
> hardware platforms, some others being emulated platforms in Qemu.
> 
> We've worked really hard to get rid of all the board-specific code
> that used to be in Buildroot, and what you're proposing seems like
> reintroducing this.
> 
> I have nothing against adding a host-qemu package, but for the rest, I
> don't understand the need. Each Qemu defconfig is associated with a
> readme.txt file in board/qemu/<something>/ that explains how to start
> the emulation. Just like we have a readme.txt file for some platforms
> in board/<foo>/<bar> that explain how to format the SD card or other
> platform-specific details.
> 
> I don't see why Qemu platforms should be handled differently.

I also note that my Aboriginal Linux project already does this for the  
targets it supports. It creates the simplest native development  
environment capable of rebuilding itself under itself, and then boots  
it under QEMU to perform automated builds under the control of a build  
control image:

   http://landley.net/aboriginal/about.html

I also did a presentation about this back in 2008 explaining why it was  
all set up that way. The slides from that are available at:

    
https://speakerdeck.com/mirell/developing-for-non-x86-targets-using-qemu

Within that context, buildroot would be considered a Linux  
distribution. It's something you'd bootstrap natively on target once  
you had a development environment running under the emulator.

I haven't made a build control image for it yet, but can try if  
somebody can tell me how to convince buildroot "build for whatever the  
host is and do all the supported packages". (If you _really_ need to  
know your host tuple you can do
"gcc -dumpmachine", but generally if you shouldn't need to. There's an  
existing toolchain in a running system...)

Rob