[Qemu-devel] config file support

[Qemu-devel] config file support

[Chuck Brazie]

Is there any work going on now to add config file support?

Chuck Brazie 
brazie@us.ibm.com

[Qemu-devel] Re: config file support

[Anthony Liguori]

Chuck Brazie wrote:
> Is there any work going on now to add config file support?

It's been discussed.  I think Fabrice had some ideas too.  AFAIK, noone
is actively working on implementing it though.

Regards,

Anthony Liguori

> Chuck Brazie 
> brazie@us.ibm.com

Re: [Qemu-devel] Re: config file support

[Fabrice Bellard]

Anthony Liguori wrote:
> Chuck Brazie wrote:
> 
>> Is there any work going on now to add config file support?
> 
> 
> It's been discussed.  I think Fabrice had some ideas too.  AFAIK, noone
> is actively working on implementing it though.

I want to implement it ASAP and it is likely to be the next feature I 
will add in QEMU.

Regards,

Fabrice.

[Qemu-devel] Config file support

[Chuck Brazie]

Fabrice,

What are your ideas on the syntax of the config file? Will it be XML based 
or similar to the current options?

Chuck Brazie 
brazie@us.ibm.com

Re: [Qemu-devel] Config file support

[Johannes Schindelin]

Hi,

On Fri, 20 Oct 2006, Chuck Brazie wrote:

> What are your ideas on the syntax of the config file? Will it be XML 
> based or similar to the current options?

Oh yes! *claps his hands* XML! *gets shiny eyes* We absolutely have not 
enough bloat in QEmu! You could integrate libxml2. Or even better, write 
yet-another XML parser!

And encodings! XML has automatic encodings! We got to have them. Yes, 
encodings. It is international!

And you can put blobs into an XML! Isn't that a great feature we could 
use?

The best of all, it's a Best Practice! Plus, you can write a DTD for it 
and _validate_ it! Or even an _XML Schema_!

...

You get the idea.

Ciao,
Dscho

Re: [Qemu-devel] Config file support

[Ricardo Almeida]

Hi,

Comments are always welcome, I guess, but since there is someone with
interest to implement config files, maybe constructive comments are
better :p

I don't dislike the use of xml nor I think is bloat. This subject as
been discuted in the list,
http://lists.gnu.org/archive/html/qemu-devel/2006-01/msg00030.html ,
xml format has already been proposed,
http://lists.gnu.org/archive/html/qemu-devel/2005-08/msg00034.html ,
promptly followed by key/value pair proposal...

Maybe the best is writing a tool like vqemu,
http://lists.gnu.org/archive/html/qemu-devel/2006-01/msg00063.html
that can be easily updated to support the different file formats and
see what people prefer...

Regards,
Ricardo Almeida

On 10/21/06, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Fri, 20 Oct 2006, Chuck Brazie wrote:
>
> > What are your ideas on the syntax of the config file? Will it be XML
> > based or similar to the current options?
>
> Oh yes! *claps his hands* XML! *gets shiny eyes* We absolutely have not
> enough bloat in QEmu! You could integrate libxml2. Or even better, write
> yet-another XML parser!
>
> And encodings! XML has automatic encodings! We got to have them. Yes,
> encodings. It is international!
>
> And you can put blobs into an XML! Isn't that a great feature we could
> use?
>
> The best of all, it's a Best Practice! Plus, you can write a DTD for it
> and _validate_ it! Or even an _XML Schema_!
>
> ...
>
> You get the idea.
>
> Ciao,
> Dscho
>
>
>
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
>

Re: [Qemu-devel] Config file support

[Stefan Weil]

Hello,

maybe we should start with a collection of configuration options.
Of course, most command line options are configuration options,
but there are many more which are needed for special applications.
Different users of QEMU have quite different needs.

A simple example: I need to configure the names and load addresses of the
(PC) bios files - not only bios.bin and vgabios.bin or  vgabios-cirrus.bin.
Today, I have to modify QEMU's code and recompile...

After a while, we'll have collected enough interesting options, so
it will be easier to see how they might be structured and which
representation (XML, key/value list, shell script, ...) is suited best.

Could we use the Wiki (http://kidsquid.com/cgi-bin/moin.cgi)
to create a page where QEMU users can add their special
needs for configuration options?

Regards
Stefan


Ricardo Almeida schrieb:
> Hi,
>
> Comments are always welcome, I guess, but since there is someone with
> interest to implement config files, maybe constructive comments are
> better

Re: [Qemu-devel] Config file support

[David Baird]

On 10/20/06, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> The best of all, it's a Best Practice! Plus, you can write a DTD for it
> and _validate_ it! Or even an _XML Schema_!

Don't forget: RelaxNG Compact Syntax :-)

Re: [Qemu-devel] Config file support

[Johannes Schindelin]

Hi,

On Sat, 21 Oct 2006, Ricardo Almeida wrote:

> Comments are always welcome, I guess, but since there is someone with
> interest to implement config files, maybe constructive comments are
> better :p
> 
> I don't dislike the use of xml nor I think is bloat.

You are free not to dislike the use of xml, but believe me: in most cases 
it is bloat. As in this case.

As for constructive ideas: I doubt that for a simple key/value store it is 
worth proposing a file format. It is very clear how such a file should 
look like.

And the most constructive ideas do not concern the "in what form?" 
question, but the "what?" and "how?".

So _I_ think that it is just a matter of extending the command line option 
parsing to also be able to parse a config file (in which case -- I am sure 
even you agree) it is easier if one line holds a complete key/value pair.

If I found config files for QEmu being a worthwhile feature, I'd just 
implement it. But I don't, so I don't ;-)

Hth,
Dscho

Re: [Qemu-devel] Config file support

[Flavio Visentin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Johannes Schindelin wrote:
> So _I_ think that it is just a matter of extending the command line option 
> parsing to also be able to parse a config file (in which case -- I am sure 
> even you agree) it is easier if one line holds a complete key/value pair.

I think reinventing the wheel is not so clever.
VMWare's config file style is really simple and it would be possible to
use VMWare's files with few or no changes.

One useful thing is to be able to use the #!/usr/bin/qemu feature :-)

- --
Flavio Visentin
GPG Key: http://www.zipman.it/gpgkey.asc

There are only 10 types of people in this world:
those who understand binary, and those who don't.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFO6PRusUmHkh1cnoRAtHyAJ9CfcDVPYTr43F0as/5xwJ4S6EnvgCePC1U
M1Bub93EY4kKg2TjbMUVEvc=
=fECK
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Config file support

[Martin Guy]

> VMWare's config file style is really simple

ethernet0.present = "TRUE"
ethernet0.virtualDev = "e1000"
e1000bios.filename = "path/etherboot-for-E1000"

> and it would be possible to
> use VMWare's files with few or no changes.

Would that be enough to be able to move the emulated system
description into config files rather than having the set of hard-coded
machine alternatives we have at present? If so it would be a boon to
anyone wanting to emulate, frinstance, any ARM board other than those
manufactured by ARM Corp.

Re: [Qemu-devel] Config file support

[Paul Brook]

> Would that be enough to be able to move the emulated system
> description into config files rather than having the set of hard-coded
> machine alternatives we have at present? If so it would be a boon to
> anyone wanting to emulate, frinstance, any ARM board other than those
> manufactured by ARM Corp.

To a first approximation the machine description probably wants to be separate 
from the user config file. There is some overlap and interlinking, but they 
tend to be aimed at different users. eg. the machine description will specify 
a particular scsi HBA with a MMIO address and wired to a particular IRQ, 
whereas the user just specifies they want a cdrom using a particular image.

I've been considering a machine config file for a while, but haven't come up 
with a coherent way of representing everything yet.

Paul

Re: [Qemu-devel] config file support

[Rob Landley]

On Wednesday 18 October 2006 2:42 pm, Chuck Brazie wrote:
> Is there any work going on now to add config file support?
> 
> Chuck Brazie 
> brazie@us.ibm.com

As a random end-user, I really like being able to run qemu without a config 
file, configuring it entirely on the command line.  I'd be highly 
disappointed if qemu turned into another Wine.

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] config file support

[Christian MICHON]

On 10/22/06, Rob Landley <rob@landley.net> wrote:
> As a random end-user, I really like being able to run qemu without a config
> file, configuring it entirely on the command line.  I'd be highly
> disappointed if qemu turned into another Wine.
>
> Rob

we've a lot to gain from it. Think twice: the shell and the host you use
behaves in a certain way because of the way you write the command line.

Migrate to a different OS for the host and you might be done for: I
believe the config file support will help solving problems and debug,
but most important will help promote qemu usage to the rest of
the community (understand: non-developpers).

Rob: you're far from being a random user, right ? :)

-- 
Christian

Re: [Qemu-devel] config file support

[Jan Marten Simons]

In my opinion config files should _always only_ be *an alternative* to a
long command line.

Basically you should be able to do anything with both configuration
options, be it command line or a config file (or a combination of both).

Ciao,
Jan

Christian MICHON schrieb:
> On 10/22/06, Rob Landley <rob@landley.net> wrote:
>> As a random end-user, I really like being able to run qemu without a
>> config
>> file, configuring it entirely on the command line.  I'd be highly
>> disappointed if qemu turned into another Wine.
>>
>> Rob
>
> we've a lot to gain from it. Think twice: the shell and the host you use
> behaves in a certain way because of the way you write the command line.
>
> Migrate to a different OS for the host and you might be done for: I
> believe the config file support will help solving problems and debug,
> but most important will help promote qemu usage to the rest of
> the community (understand: non-developpers).
>
> Rob: you're far from being a random user, right ? :)
>

Re: [Qemu-devel] config file support

[Paul Brook]

On Monday 23 October 2006 12:48, Jan Marten Simons wrote:
> In my opinion config files should _always only_ be *an alternative* to a
> long command line.
>
> Basically you should be able to do anything with both configuration
> options, be it command line or a config file (or a combination of both).

Part of the motivation for wanting a config file is that it's more flexible 
and allows more complicated configs than just commandline options.
For that reason I'd expect the commandline to only offer a subset of the 
features available via a config file.

You can always write a wrapper script that generates a config file.

Paul

Re: [Qemu-devel] config file support

[K. Richard Pixley]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

Rob Landley wrote:
> On Wednesday 18 October 2006 2:42 pm, Chuck Brazie wrote:
>   
>> Is there any work going on now to add config file support?
>>
>> Chuck Brazie 
>> brazie@us.ibm.com
>>     
> As a random end-user, I really like being able to run qemu without a config 
> file, configuring it entirely on the command line.  I'd be highly 
> disappointed if qemu turned into another Wine.
Except that I never do.  Instead, I write a trivial shell script since I 
can never remember the command line options, much less type them in 
consistently.

What's the difference between a shell script to cover qemu and a 
#!/bin/qemu config file?  Seems to me they both address roughly the same 
issues with roughly the same considerations.  Am I missing any 
significant functionality differences?

--rich

[-- Attachment #2: Type: text/html, Size: 1294 bytes --]

Re: [Qemu-devel] config file support

[Ben Taylor]

---- "K. Richard Pixley" <rich.pixley@palmsource.com> wrote: 
> Rob Landley wrote:
> > On Wednesday 18 October 2006 2:42 pm, Chuck Brazie wrote:
> >   
> >> Is there any work going on now to add config file support?
> >>
> >> Chuck Brazie 
> >> brazie@us.ibm.com
> >>     
> > As a random end-user, I really like being able to run qemu without a config 
> > file, configuring it entirely on the command line.  I'd be highly 
> > disappointed if qemu turned into another Wine.
> Except that I never do.  Instead, I write a trivial shell script since I 
> can never remember the command line options, much less type them in 
> consistently.
> 
> What's the difference between a shell script to cover qemu and a 
> #!/bin/qemu config file?  Seems to me they both address roughly the same 
> issues with roughly the same considerations.  Am I missing any 
> significant functionality differences?

There are probably no functional differences.  A shell script with a config
file (which is how I wrote one 18 months ago) should be able to handle 
almost any kind of config, without adding to code bloat to qemu.  

The script I wrote is probably horribly out of date, and has some holes
(and probably some logic flaws yet unfound), but for the most part 
replicated most of the things I wanted to do (networking, tun/tap or 
redirs, samba etc ) without having to type  command lines in excess of 200 
characters.  Given that I had probably 8-10 qemu guest systems, it was 
a huge time saver when it came to starting up a brand new guest.

Ben

Re: [Qemu-devel] Config file support

[Rob Landley]

On Sunday 22 October 2006 2:27 pm, Paul Brook wrote:
> I've been considering a machine config file for a while, but haven't come up 
> with a coherent way of representing everything yet.

Do you at least have a list of everything that needs to be represented?  (I 
have a list but am fairly certain it's not complete.)

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Paul Brook]

On Monday 23 October 2006 21:01, Rob Landley wrote:
> On Sunday 22 October 2006 2:27 pm, Paul Brook wrote:
> > I've been considering a machine config file for a while, but haven't come
> > up with a coherent way of representing everything yet.
>
> Do you at least have a list of everything that needs to be represented?  (I
> have a list but am fairly certain it's not complete.)

Not really. I guess a generic key/value pair is sufficient for most things 
(base address, model number, etc).

The trickier cases are where devices refer to each other: PCI, usb, IRQ lines, 
DMA channels, etc. ie. anything where the existing _init function returns a 
value.
I guess you could use the same key/value mechanism, and allow the value to be 
an "object".

The really OTT method is to embed a python interpreter or something and do the 
machine init that way. I'm not seriously suggesting that though.

Paul

Re: [Qemu-devel] config file support

[Rob Landley]

On Monday 23 October 2006 1:50 pm, K. Richard Pixley wrote:
> Rob Landley wrote:
> > On Wednesday 18 October 2006 2:42 pm, Chuck Brazie wrote:
> >   
> >> Is there any work going on now to add config file support?
> >>
> >> Chuck Brazie 
> >> brazie@us.ibm.com
> >>     
> > As a random end-user, I really like being able to run qemu without a 
config 
> > file, configuring it entirely on the command line.  I'd be highly 
> > disappointed if qemu turned into another Wine.
> Except that I never do.  Instead, I write a trivial shell script since I 
> can never remember the command line options, much less type them in 
> consistently.
> 
> What's the difference between a shell script to cover qemu and a 
> #!/bin/qemu config file?

The shell script works now, and you're proposing breaking it?

> Seems to me they both address roughly the same  
> issues with roughly the same considerations.

Using a *.PIF file is the Windows way.  Using the command line is Linux.

> Am I missing any significant functionality differences?

So you'd have no trouble configuring kde's file type associations to open 
arbitrary "*.img" files with qemu when you click on them if you couldn't do 
this entirely from the command line?
 
> --rich

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] config file support

[André Braga]

On 10/23/06, K. Richard Pixley <rich.pixley@palmsource.com> wrote:
>  What's the difference between a shell script to cover qemu and a
> #!/bin/qemu config file?

Not everyone may run QEMU under a POSIX-ish command-line shell. There
are several active operating systems in the world, and several people
fiddling with the porting of QEMU to those systems.

What I'd really like to see, for example, would be the possibility of
dumping the options active when QEMU was started to a config file. It
could be then tweaked at will, but the skeleton would be there
already, customized to a pretty comfortable point even.

Re: [Qemu-devel] config file support

[Paul Brook]

> > Seems to me they both address roughly the same
> > issues with roughly the same considerations.
>
> Using a *.PIF file is the Windows way.  Using the command line is Linux.

There's plenty of prior art for using config files on unix/linux systems.

I'm not saying we should remove all commandline options. However qemu is 
growing sufficiently many user-adjustable knobs that manipulating them all 
solely via the commandline gets unwieldy. IMHO there's a limit to how much 
information can/should be usefully expressed on the commandline, and qemu is 
approaching that limit.

Paul

Re: [Qemu-devel] config file support

[K. Richard Pixley]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

Rob Landley wrote:
>> What's the difference between a shell script to cover qemu and a 
>> #!/bin/qemu config file?
>>     
> The shell script works now, and you're proposing breaking it?
>   
No, I'm not.  I'm genuinely asking about functional differences.
>> Am I missing any significant functionality differences?
>>     
> So you'd have no trouble configuring kde's file type associations to open 
> arbitrary "*.img" files with qemu when you click on them if you couldn't do 
> this entirely from the command line?
>   
I don't use kde.  And *.img is used for far too many file types to 
assume that this alone indicates that any foo.img is a qemu related file.

But yes, I could configure gnome to do this.  Or firefox.  It's pretty 
much trivial.  MacosX has file system metadata to help out so that it 
doesn't have to rely solely on the irrelevant file name suffixes.

--rich

[-- Attachment #2: Type: text/html, Size: 1496 bytes --]

Re: [Qemu-devel] config file support

[M. Warner Losh]

In message: <200610231639.00717.rob@landley.net>
            Rob Landley <rob@landley.net> writes:
: > Seems to me they both address roughly the same  
: > issues with roughly the same considerations.
: 
: Using a *.PIF file is the Windows way.  Using the command line is Linux.

Except for complicated things, like, say, X's config file.

Warner

Re: [Qemu-devel] Config file support

[Rob Landley]

On Monday 23 October 2006 4:29 pm, Paul Brook wrote:
> On Monday 23 October 2006 21:01, Rob Landley wrote:
> > On Sunday 22 October 2006 2:27 pm, Paul Brook wrote:
> > > I've been considering a machine config file for a while, but haven't 
come
> > > up with a coherent way of representing everything yet.
> >
> > Do you at least have a list of everything that needs to be represented?  
(I
> > have a list but am fairly certain it's not complete.)
> 
> Not really. I guess a generic key/value pair is sufficient for most things 
> (base address, model number, etc).

The things are what I was asking about.  Assuming that QEMU has support for 
the appropriate processor type, support for the right bus controller(s), and 
support for various devices that can attach to that bus, what other 
information is needed to completely specify a machine?  (You mention IRQ 
lines and DMA channels...)

I'm still a little fuzzy about basic questions like "How much information is 
in 'processor type'?"  (Does that include cache size?  Floating point 
support?  Has mmu flag?  Are these separate processors with their own names, 
or are they options to a base processor type?)

> The really OTT method is to embed a python interpreter or something and do
> the machine init that way. I'm not seriously suggesting that though.

Thank goodness. :)

I'm generally not worried about parsing data files being hard, I just don't 
currently know what's involved in adding a new machine type to QEMU anyway.  	 
don't know what all the data _is_ let alone what to do with it once it's read 
in.

Fabrice did a good job explaining the CPU part in 
http://www.usenix.org/events/usenix05/tech/freenix/full_papers/bellard/bellard_html/index.html
but that sort of glosses over the support chips, bus, devices, interrupts, 
dma...

Last I checked, each processor was in its own directory (at the top level, not 
under any kind of processors/ directory), the devices were under "hw", and 
the motherboards gluing together a bunch of devices were _also_ under "hw".

Sorting through that, I read bits of files like "hw/pc.c" where it defines 
QEMUMachine structures like "pc_machine" where the important thing seems to 
be pc_init_pci which takes ten arguments and is a wrapper around pc_init1 
(which takes eleven).

That calls lots of functions to init cpu emulators (_which_ cpu is being 
initalized seems to be specified elsewhere, possibly by the makefile linking 
in the right objects), map memory into the virtual MMU, initialize the bios, 
possibly load a kernel, init the virtual PCI bus (and the PCI to ISA bridge), 
whatever the heck the two calls to register_ioport_write() do (0x80 and 
0xf0...  Let's see, http://www.cs.cmu.edu/~ralf/interrupt-list/inter61d.zip 
file PORT.A.  PORT 0080 is "manufacturing diagnostics port", sometimes used 
for a hex display, and I vaguely remember from way back some linux kernel 
thread about this (it was being used as a delay, or...?)  Bug can't seem to 
google for it right now.  0xf0 is the math coprocessor.), init the virtual 
graphics card, init the realtime clock, more I/O port stuff (0x92...  PS/2 
system control port a, controls the A20 line among other things), setup the 
ioapic, and the ISA Programmable interrupt controller, the programmable 
interval timer, pci speaker, do something else with the interrupts, init 
serial and parallel ports, network cards, ide controllers, keyboard, dma, 
sound card, floppy controller, cmos, usb, power management, a commented out 
block of scsi devices, and the pci and acpi bioses.

Currently, this is all hard-wired together into a big blob.  Step one of 
untangling it would probably be moving the device files and the motherboard 
files to separate directories...

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Paul Brook]

> > Not really. I guess a generic key/value pair is sufficient for most
> > things (base address, model number, etc).
>
> The things are what I was asking about.  Assuming that QEMU has support for
> the appropriate processor type, support for the right bus controller(s),
> and support for various devices that can attach to that bus, what other
> information is needed to completely specify a machine?  (You mention IRQ
> lines and DMA channels...)

A good first guess is to look at the the *_init functions in the hw/ 
directory. They should tell you what parameters a device has.

> I'm still a little fuzzy about basic questions like "How much information
> is in 'processor type'?"  (Does that include cache size?  Floating point
> support?  Has mmu flag?  Are these separate processors with their own
> names, or are they options to a base processor type?)
>
> I'm generally not worried about parsing data files being hard, I just don't
> currently know what's involved in adding a new machine type to QEMU anyway.
> don't know what all the data _is_ let alone what to do with it once it's
> read in.

This is why I suggested a *generic* key/value system. Basically each "device" 
registers itself with qemu, and provides an initialisation function and a 
list of properties. qemu doesn't know the meaning of a particular key, just 
its name and type (number/string/whatever).

The machine config file instantiates particular devices (explicitly or 
implicitly one per section). qemu validates+parses the keys in the config 
file against the list provided by the device. Then the init function is 
called.

> Last I checked, each processor was in its own directory (at the top level,
> not under any kind of processors/ directory), 

qemu doesn't support different CPUs in the same machine. That's a whole other 
problem.

> the devices were under "hw", 
> and the motherboards gluing together a bunch of devices were _also_ under
> "hw".
>...
> Currently, this is all hard-wired together into a big blob.  Step one of
> untangling it would probably be moving the device files and the motherboard
> files to separate directories...

My intention is that a machine config file would remove the "motherboard" bits 
altogether. ie. the config file describes everything that pc_init_1 does. The 
first half of pc.c would remain because that's device emulation.

For things like network/serial/disks we need to figure out how to make the 
machine description adapt to the config the user requested. Proably want to 
replace the fixed tables eg. bs_table with some mechanism for 
identifying/requesting disks by name.

Likewise if you identify PCI busses and IRQs by name/location this provides a 
way for the user to wire them up.

Most of the code is already fairly well separated. It's just that the glue is 
hardcoded in C and parameters passed as function arguments rather than being 
something that is determined at runtime.

Take the Integrator/CP board as an example. I'd expect the machine config to 
look something like:

ram {base=0; size=RAM_SIZE, physaddr=0}
ram {base=0x80000000; size=RAM_SIZE, physaddr=0}
integrator_core{ram_size=RAM_SIZE};
arm_cpu_pic {cpu_index=0, pic_name="CPU0"}
integrator_pic {pic_name="PRIMARY", base=0x14000000,parent="CPU0", 
parent_irq=0, parent_fiq=1}
integrator_pic {pic_name="SECONDARY", base=0xca000000, pic="PRIMARY",irq=0, 
fiq=1}
integrator_pit{base=0x13000000, pic="PRIMARY", irq=5}
pl011{base=0x16000000, name="serial0", pic="PRIMARY", irq=1}
etc.

The syntax I just made up, and there are the issues I mentioned above, but 
hopefully you get the idea.

Paul

Re: [Qemu-devel] Config file support

[andrzej zaborowski]

On 24/10/06, Rob Landley <rob@landley.net> wrote:
> On Monday 23 October 2006 4:29 pm, Paul Brook wrote:
> > On Monday 23 October 2006 21:01, Rob Landley wrote:
> > > On Sunday 22 October 2006 2:27 pm, Paul Brook wrote:
> > > > I've been considering a machine config file for a while, but haven't
> come
> > > > up with a coherent way of representing everything yet.
> > >
> > > Do you at least have a list of everything that needs to be represented?
> (I
> > > have a list but am fairly certain it's not complete.)
> >
> > Not really. I guess a generic key/value pair is sufficient for most things
> > (base address, model number, etc).
>
> The things are what I was asking about.  Assuming that QEMU has support for
> the appropriate processor type, support for the right bus controller(s), and
> support for various devices that can attach to that bus, what other
> information is needed to completely specify a machine?  (You mention IRQ
> lines and DMA channels...)

I'm pessimistic about machine config file support. I know little about
the PC-like machines in qemu but I've been playing with embedded
(system-on-chip) hw emulation and every new piece of hardware required
changes (even if very small) in the bus or cpu code as well, the
reason being that manufacturers are allowed to do any kind of tricks
in their hardware knowing that it doesn't need to be configurable,
being sold together as a single board. For example chips with totally
contrasting functions (take keypad input and LCD) are allowed to
communicate between themselves for good synchronisation, without
poking the main processor. A different example is a single device
occupying multiple "slots" on a given bus, or multiple busses.

Basically I expect only things that are "pluggable" in the real world
to be practically configurable through something that is not C code.
So for example PCI or USB, but these are already configurable.

>
> I'm still a little fuzzy about basic questions like "How much information is
> in 'processor type'?"  (Does that include cache size?  Floating point
> support?  Has mmu flag?  Are these separate processors with their own names,
> or are they options to a base processor type?)
>
> > The really OTT method is to embed a python interpreter or something and do
> > the machine init that way. I'm not seriously suggesting that though.

A python (or other) interpreter would be cool in place of qemu monitor though :)

Regards,

Re: [Qemu-devel] Config file support

[Paul Brook]

> > The things are what I was asking about.  Assuming that QEMU has support
> > for the appropriate processor type, support for the right bus
> > controller(s), and support for various devices that can attach to that
> > bus, what other information is needed to completely specify a machine? 
> > (You mention IRQ lines and DMA channels...)
>
> I'm pessimistic about machine config file support. I know little about
> the PC-like machines in qemu but I've been playing with embedded
> (system-on-chip) hw emulation and every new piece of hardware required
> changes (even if very small) in the bus or cpu code as well, the
> reason being that manufacturers are allowed to do any kind of tricks
> in their hardware knowing that it doesn't need to be configurable,
> being sold together as a single board. For example chips with totally
> contrasting functions (take keypad input and LCD) are allowed to
> communicate between themselves for good synchronisation, without
> poking the main processor. A different example is a single device
> occupying multiple "slots" on a given bus, or multiple busses.

I'm more optimistic. Even SoC designs tend to be built up from modular 
components. While adding support for a totally new system may require 
changes, I think there's a good chance of being able to describe different 
variants of the similar devices (eg. all the different PrimeCell based 
integrator/versatile/Realview boards, or different members of the OMAP 
family).

Paul

Re: [Qemu-devel] Config file support

[Paul Brook]

On Tuesday 24 October 2006 01:12, Paul Sokolovsky wrote:
> Hello Paul,
>
> Monday, October 23, 2006, 11:29:52 PM, you wrote:
> > On Monday 23 October 2006 21:01, Rob Landley wrote:
> >> On Sunday 22 October 2006 2:27 pm, Paul Brook wrote:
> >> > I've been considering a machine config file for a while, but haven't
> >> > come up with a coherent way of representing everything yet.
>
>    I'm glad this discussion was brought up on the list. And I'd like
> to also bring back another related issue - what about providing
> "plugin" system for device (chip) implementation, in addition to
> flexible-format machine config allowing to construct "virtual boards"
> out of them?

IMHO we already have a fairly good device model, and it's not hard to add new 
devices.

If you mean putting individual devices in shared libraries and dlopen'ing them 
at runtime then I have no interest in that. AFAICS the only reason for 
wanting to do this is to use closed-source device models.

Paul

Re: [Qemu-devel] Config file support

[Paul Brook]

> >> >> > I've been considering a machine config file for a while, but
> >> >> > haven't come up with a coherent way of representing everything yet.
> >>
> >>    I'm glad this discussion was brought up on the list. And I'd like
> >> to also bring back another related issue - what about providing
> >> "plugin" system for device (chip) implementation, in addition to
> >> flexible-format machine config allowing to construct "virtual boards"
> >> out of them?
> >
> > IMHO we already have a fairly good device model, and it's not hard to add
> > new devices.
>
>   Maybe. But where are new chips in qemu? 

You mean like m68k I just committed, and sh4 that got added not so long ago?

Or the LSI SCSI emulation? Or the PCnet and rtl8139 NIC emulation?

>   Why there're still only 2 ARM boards?

It's actually 3 (another with 2 minor variants of those boards), and there are 
at least 3 other (incomplete) ports have been posted on the list but not 
merged.

> How do I "stick" wi-fi card in one of them?

Huh? Same way you add any other device. Call the init function and get it to 
register its resources.

> So the concern  
> is not just if it's easy to add new devices or not, but if there're means
> to actually support appearance and growth of device library. Plugin system
> would be a "decree" that there's a stable API to define devices and
> welcome for 3rd-party developers to develop them.
>
>   And well, patching source is not really that easy a way to "add new
> devices".

I don't believe that. Adding 2 lines to a Makefile is probably easier than 
building an independent plugin.

Paul

Re: [Qemu-devel] Config file support

[Christian MICHON]

IMHO, I believe:
- python inside monitor is uncalled for (average python installation size is
big, no ?)
- xml is still too big a format for something we can do by shell script
(joke: why not yaml ?)

-- 
Christian

Re: [Qemu-devel] Config file support

[Rob Landley]

On Monday 23 October 2006 7:33 pm, Paul Brook wrote:

> My intention is that a machine config file would remove the "motherboard" 
bits 
> altogether. ie. the config file describes everything that pc_init_1 does. 
The 
> first half of pc.c would remain because that's device emulation.

Sounds highly cool.  I'm quite in favor of _that_ kind of config file.

> For things like network/serial/disks we need to figure out how to make the 
> machine description adapt to the config the user requested. Proably want to 
> replace the fixed tables eg. bs_table with some mechanism for 
> identifying/requesting disks by name.

If some of the hardware could be hotpluggable, that would be cool.  (I've 
hotpluged real IDE disks, ill-advised as that is.)  I dunno what has ordering 
requirements (or more specifically, dependencies on previous hardware) 
though.

> Take the Integrator/CP board as an example. I'd expect the machine config to 
> look something like:
> 
> ram {base=0; size=RAM_SIZE, physaddr=0}
> ram {base=0x80000000; size=RAM_SIZE, physaddr=0}
> integrator_core{ram_size=RAM_SIZE};
> arm_cpu_pic {cpu_index=0, pic_name="CPU0"}
> integrator_pic {pic_name="PRIMARY", base=0x14000000,parent="CPU0", 
> parent_irq=0, parent_fiq=1}
> integrator_pic {pic_name="SECONDARY", base=0xca000000, pic="PRIMARY",irq=0, 
> fiq=1}
> integrator_pit{base=0x13000000, pic="PRIMARY", irq=5}
> pl011{base=0x16000000, name="serial0", pic="PRIMARY", irq=1}
> etc.
> 
> The syntax I just made up, and there are the issues I mentioned above, but 
> hopefully you get the idea.

The syntax looks fine to me, and I can see where bits of that come from 
hw/integratorcp.c intergratorcp_init(), but when in that file I also see 
things like struct integratorcm_state and icp_pic_read() in there, and I 
don't know how they relate.  The "here's a new device: it's a DMA controller" 
and "here's a new motherboard that has all these chips and devices on it 
wired together this way" is all mixed together in the same files.  I have 
trouble figure out which bits belong to which categories.

Possibly I should be poking at application emulation first, rather than system 
emulation.  Easier to follow what's happening when you run "hello world"...

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Flavio Visentin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Paul Brook wrote:
> ram {base=0; size=RAM_SIZE, physaddr=0}
> ram {base=0x80000000; size=RAM_SIZE, physaddr=0}
> integrator_core{ram_size=RAM_SIZE};
> arm_cpu_pic {cpu_index=0, pic_name="CPU0"}
> integrator_pic {pic_name="PRIMARY", base=0x14000000,parent="CPU0", 
> parent_irq=0, parent_fiq=1}
> integrator_pic {pic_name="SECONDARY", base=0xca000000, pic="PRIMARY",irq=0, 
> fiq=1}
> integrator_pit{base=0x13000000, pic="PRIMARY", irq=5}
> pl011{base=0x16000000, name="serial0", pic="PRIMARY", irq=1}

At this point it's really cleaner and maybe simpler to use XML, as
someone suggested, also because you can have more information on how the
components relates to each other.

<motherboard type="x86", ...>
   <cpu type="AMDK7", ... />
   <memory type="ram", size="64M", ... />
   <memory type="ram", size="64M", ... />
   <bus type="PCI", ... >
      <controller type="EIDE", ... />
      <controller type="SCSI", ... />
   </bus>
   <bus type="USB", ...>
       ...
   </bus>
</motherboard>

- --
Flavio Visentin
GPG Key: http://www.zipman.it/gpgkey.asc

There are only 10 types of people in this world:
those who understand binary, and those who don't.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFPe8nusUmHkh1cnoRAu3ZAJ40KCZqn6qxTVNZH7xNGFkr+2XSsQCaAjww
5eoh+FWayk4ZEuCLWlPuqOQ=
=76Qs
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Config file support

[Christian MICHON]

how about this ? (it's yaml, not xml)

the idea would be to store all cfg in one file and switch at boot time
which guest you want to boot...

This is just a draft, and your mileage may vary. More readings at:
http://yaml.org
http://www-128.ibm.com/developerworks/library/x-matters23.html

---
guests:
  -
    name:  win30
    kqemu: no
    ram:   16
    boot:  c
    fda:   none
    hda:   win30.img
    cdrom: none
    type:  isapc
    net:   none
    audio: none
  -
    name:  rh72
    kqemu: kernel
    ram:   128
    type:  pc
    fda:   none
    hda:   rh72.qcow
    cdrom: enigma-disc1.iso
    boot:  c
    net:   user,...,...
  -
    name:  bartpe
    kqemu: user
    ram:   256
    type:  pc
    fda:   none
    hda:   none
    cdrom: pebuilder.iso
    boot:  d
    net:   none
  -
    name:  xp_lite
    kqemu: kernel
    ram:   384
    type:  pc
    fda:   none
    hda:   nliteos.qcow
    cdrom: nliteos.iso
    boot:  c
    net:   user,...,...

-- 
Christian

Re: [Qemu-devel] Config file support

[Blue Swirl]

I'd recommend Qemu Launcher (https://gna.org/projects/qemulaunch).

If Qemu gets a config file and a configuration utility, it should be similar 
in my opinion.

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

Re: [Qemu-devel] Config file support

[Christian MICHON]

On 10/24/06, Blue Swirl <blueswir1@hotmail.com> wrote:
> I'd recommend Qemu Launcher (https://gna.org/projects/qemulaunch).
>
> If Qemu gets a config file and a configuration utility, it should be similar
> in my opinion.
>

I thought the qemu config file could be having the noble aim to be
multi-host. Using such gtk/gui interface would work on win32 hosts too ?

-- 
Christian

Re: [Qemu-devel] Config file support

[Rob Landley]

On Monday 23 October 2006 8:12 pm, Paul Sokolovsky wrote:
>   Yes, machine config apparently would be a hierarchical structure,
> with cross-references. And well, there's an industrial standard to
> represent that - XML.

There's an interesting sort of natural selection at work in open source.  
Anybody clueful enough to actually make the necessary changes to qemu to 
implement that idea is also clueful enough to know precisely how bad an idea 
it is.

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Rob Landley]

On Monday 23 October 2006 9:38 pm, Paul Sokolovsky wrote:
>   Maybe. But where are new chips in qemu? Why there're still only 2
> ARM boards? How do I "stick" wi-fi card in one of them? So the concern
> is not just if it's easy to add new devices or not, but if there're means
> to actually support appearance and growth of device library. Plugin system
> would be a "decree" that there's a stable API to define devices and
> welcome for 3rd-party developers to develop them.

Because the lack of a stable internal API has completely prevented Linux from 
getting any sort of device support.  It runs on far less hardware than things 
like Solaris, with such a stable API...

> P.S. This is not a troll

People who are not trolling generally don't have to _say_ they aren't 
trolling.

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Rob Landley]

On Tuesday 24 October 2006 6:47 am, Flavio Visentin wrote:
> At this point it's really cleaner and maybe simpler to use XML

Have you ever implemented a validating XML parser?  I have.  It only 
_looks_ "clean and simple".

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Johannes Schindelin]

Hi,

On Tue, 24 Oct 2006, Rob Landley wrote:

> On Tuesday 24 October 2006 6:47 am, Flavio Visentin wrote:
> > At this point it's really cleaner and maybe simpler to use XML
> 
> Have you ever implemented a validating XML parser?  I have.  It only 
> _looks_ "clean and simple".

+1

Re: [Qemu-devel] Config file support

[Paul Brook]

>   Oh, c'mon, Rob! I really didn't want to ask Paul Brook that, but
> sure you'll fix my cluelessness right here, right now - tell me, tell me,
> why Linux has dynamic-loadable modules support, which clueless passers-by
> like me call "plugins"? It must be closed-source diversion, no?

Linux has genuine reasons for wanting modules.
Kernel size is important because (a) it has to be loaded by the bootloader, 
often from a small, slow device (eg. floppy, flash or network).
(b) The whole kernel is permanently locked into ram. It you've ever tried to 
build a kernel with everything enable you'll know the result is unreasonably 
large. Modules allow the same kernel to work on a wide variety of large and 
small machines.

It's also a fairly convenient way of allowing userspace to disable a 
particular set of drivers.

Closed source kernel modules are explicitly *not* supported by kernel 
developers.

A typical qemu process already uses well over a hundred Mb of memory. Saving a 
few hundred k of code at runtime isn't going to make any difference to 
anything.

Paul

Re: [Qemu-devel] Config file support

[Rob Landley]

On Wednesday 25 October 2006 11:01 am, Paul Brook wrote:
> >   Oh, c'mon, Rob! I really didn't want to ask Paul Brook that, but
> > sure you'll fix my cluelessness right here, right now - tell me, tell me,
> > why Linux has dynamic-loadable modules support, which clueless passers-by
> > like me call "plugins"? It must be closed-source diversion, no?
> 
> Linux has genuine reasons for wanting modules.
> Kernel size is important because (a) it has to be loaded by the bootloader, 
> often from a small, slow device (eg. floppy, flash or network).
> (b) The whole kernel is permanently locked into ram. It you've ever tried to 
> build a kernel with everything enable you'll know the result is unreasonably 
> large. Modules allow the same kernel to work on a wide variety of large and 
> small machines.

It also avoids a reboot cycle when you want to debug small changes to drivers 
(assuming you didn't crash).  Restarting a userspace app (like qemu) takes 
five seconds.  Restarting the kernel can take a minute and change, and often 
involves pressing a button on a machine that's shoved under a desk and hard 
to get at.

I've found avoiding the reboot cycle to be a nice thing with qemu (and User 
Mode Linux), but alas you can't test a driver for hardware qemu doesn't 
emulate.  Nice for filesystems and VM stuff, though...

Rob
-- 
"Perfection is reached, not when there is no longer anything to add, but
when there is no longer anything to take away." - Antoine de Saint-Exupery

Re: [Qemu-devel] Config file support

[Paul Brook]

> > Linux has genuine reasons for wanting modules.
> > Kernel size is important because (a) it has to be loaded by the
> > bootloader, often from a small, slow device (eg. floppy, flash or
> > network).
> > (b) The whole kernel is permanently locked into ram. It you've ever tried
> > to build a kernel with everything enable you'll know the result is
> > unreasonably large. Modules allow the same kernel to work on a wide
> > variety of large and small machines.
>
>   Thanks for your response. But I hope none of us take the discussion
> too seriously to consider the arguments like above are all-convincing.
> They can be easily reversed by simple replacements of notions. To not
> just do s///, how about such questions: when do you think QEMU will
> support all (or any, to not sound that naughty ;-) ) of 1K ARM boards
> in mach-types will be supported? And if that ever happen, will that
> support be in QEMU mainline?

All the boards qemu emulates are supported out the box by mainline linux 
kernels.

>   So well, if there was a plugin support with a nice kind of SDK, I
> would for sure already hacked emulation for some chip found in
> embedded ARM systems (even mock one). But for now, I just wait for
> next time I'll be able to setup session to peer into QEMU source.

You seem to be confusing a binary plugin interface with documentation.
The two are independent.  I really don't buy "I would have developed stuff if 
there was a plugin interface" as an argument. If you bothered to look you'd 
find the qemu fairly modular.

>   What applies to me likely will apply to many more people (it's
> *socio*psychological matter). So yes, the question is: do you care
> enough to support QEMU-extension community up to the level of making
> its life easier? Yes, sure, real men can hack new device support in
> QEMU the way it is now. But even real men have constraints on time and
> effort involved, so maybe won't have patience to push changes back to
> QEMU, and will just leave random snapshots and forks around. And
> that's already starting, e.g. http://www.bitmux.org/qemu.html

I don't think a binary plugin interface will help with this.
How are abandoned binary plugins better than abandoned open source projects?
At least with the latter interested third parties have the option of picking 
them up and making them work.

> > A typical qemu process already uses well over a hundred Mb of memory.
> > Saving a few hundred k of code at runtime isn't going to make any
> > difference to anything.
>
>   Yes, as I told, "memory" is not a keyword here. "number of files in QEMU
> distribution" and "ease of their maintenance" are.

Binary plugins don't make things easier to maintain. They just mean you're 
locked in to a particular interface, and can't change anything.

The kernel manages perfectly well with everything in the same tree.

Paul