Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paolo Bonzini]

> The magic I'm preferring to is precisely things like pci="on".

Hmm, pci=on is magic indeed. :-)

> What I'm objecting to is making machine construction be controlled by an 
> arbitrary set of hardcoded parameters.
> My argument is that in the short term this parameterization provides limited 
> benefit, and longer term it's going to be obsolete and probably painful to 
> support.

Then it would be enough to have these options available internally for use
in the machine-core but not export them to the command-line.  If we only
aim at using management interfaces to use the interface we can be very
conservative and not export even things like default_drive.

Paolo

[Qemu-devel] [PATCH 0/22] Refactor machine support

[Anthony Liguori]

This series introduces a rather radical change in the way we deal with machine
definitions in QEMU.  Here are the features this series introduces:

 - Machines are definable via a -machine-def option or a [machine-def] config
   stanza
 - Parameters such as -kernel, -initrd, -append, -m, -acpi-enable, -enable-kvm,
   and potentially many more are now support via a config
 - It's possible to set a default block format (ide, scsi, virtio) for a machine
 - It's possible to make kvm enablement a property of a machine type.  The
   support modes are KVM enabled, TCG enabled, and KVM enabled with TCG
   fallback.
 - All properties of the default machine type are overridable via global config.
   IOW, you can make KVM enablement default without touching the source code.
   You can also increase the default ram size without touching any code.

Conceptually, it works by introducing a MachineCore concept.  A MachineCore is
a function that is able to create a machine based on a set of user specified
parameters.  These parameters may be high level properties (like ram size) or
qdev global properties (virtio-blk-pci.vectors).

A builtin Machine is an instance of a MachineCore that has a set of default
parameters set.

The concept of MachineCore is *not* a stop-gap measure for proper qdev
conversion.  Rather, the idea behind MachineCore is that it's a device tree
builder.

You certainly could still have a machine described purely by a device tree and
qdev properties but that's not enough to represent the UI that we currently
expose.  We need a way convert user visible operations like enable usb to
platform specific operations like add a PIIX3 USB hub.  This is what a
MachineCore does.

Re: [Qemu-devel] [PATCH 0/22] Refactor machine support

[Paul Brook]

> This series introduces a rather radical change in the way we deal with
> machine definitions in QEMU.

I think we should aim to eliminate machine_register_core, and design 
appropriately.  In particular I'd try and avoid adding options that become 
trivially redundant once you can easily change the device trees.

I'm undecided how much parameterisation it's worth trying to support in the 
device tree. IMO the current code has way too much magic, because creating a 
new variant involves hacking and rebuilding pc.c.

I'm extremely tempted by the extreme approach of punting everything to an 
external device tree builder. i.e. remove automagical device reation 
altogether. If you want USB and 6 virtio disks then ask your device tree 
builder to create a machine description that includes pci-ohci and a bunch of 
virtio-blk-pci devices. The virtio-blk devices reference "drives" by id, so 
the same host configuration can be used with a scsi HBA. A small shell script 
should be sufficient to replicate the limited machine variants currently 
supported.

The problem with doing clever device tree generation/manipulation inside qemu 
is that for anything vaguely nonstandard you're likely to have to hack your 
own device tree anyway. Even ram allocation is nontrivial, e.g. the hole below 
4G on the PC, and that's before you start with numa-like layouts.

On a similar note, I don't see any point having machine_create_from_core,
we can just ship appropriate config files.

Paul

[Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paolo Bonzini]

On 06/08/2010 05:12 AM, Paul Brook wrote:
>> This series introduces a rather radical change in the way we deal with
>> machine definitions in QEMU.
>
> I think we should aim to eliminate machine_register_core, and design
> appropriately.  In particular I'd try and avoid adding options that become
> trivially redundant once you can easily change the device trees.

I agree I don't like patch 13 (the parallel/virtcon/vga/floppy/cdrom 
QemuOpts).  Maybe those QemuOpts can be marked in a way that they're not 
user-accessible (and though in some cases customizing max_cpus may make 
sense, that option could also fall in that category).  Everything else 
is very much reasonable in Anthony's patch series, I think.

> I'm undecided how much parameterisation it's worth trying to support in the
> device tree. IMO the current code has way too much magic, because creating a
> new variant involves hacking and rebuilding pc.c.

See patch 22/22.  There is really no magic involved, even though the 
compat machines are not yet as config files.

> I'm extremely tempted by the extreme approach of punting everything to an
> external device tree builder. i.e. remove automagical device reation
> altogether.

I think this should have been raised when the -readconfig/-writeconfig 
scheme was proposed and committed.  I don't think it's reasonable to 
block this patch series (or something very much like it) on the grounds 
that a better device tree description model than QemuOpts can be designed.

> The problem with doing clever device tree generation/manipulation inside qemu
> is that for anything vaguely nonstandard you're likely to have to hack your
> own device tree anyway. Even ram allocation is nontrivial, e.g. the hole below
> 4G on the PC, and that's before you start with numa-like layouts.

Again, the patch is not making anything worse.

> On a similar note, I don't see any point having machine_create_from_core,
> we can just ship appropriate config files.

I think that's the obvious next step, but there's no reason not to split 
it at this point.

Paolo

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paul Brook]

> > I'm undecided how much parameterisation it's worth trying to support in
> > the device tree. IMO the current code has way too much magic, because
> > creating a new variant involves hacking and rebuilding pc.c.
> 
> See patch 22/22.  There is really no magic involved, even though the
> compat machines are not yet as config files.

The magic I'm preferring to is precisely things like pci="on".
This is an artifact of only having a hardcoded single device tree (defined by 
pc_init), so we need magic parameters to pick between different variants.

If this were just a cleanup of how we implement the various machines that 
would be harmless. However these are now a user-visible interface.

It implies that qemu is expected to know how to add/remove PCI capabilities 
from a machine. Once you eliminate machine_register_core that knowledge has 
somehow got to come from your device tree description file.  Having a single 
device tree that can morph into significantly different machines seems like 
unnecessary complexity given this is a user-specified file.

> > I'm extremely tempted by the extreme approach of punting everything to an
> > external device tree builder. i.e. remove automagical device reation
> > altogether.
> 
> I think this should have been raised when the -readconfig/-writeconfig
> scheme was proposed and committed.  I don't think it's reasonable to
> block this patch series (or something very much like it) on the grounds
> that a better device tree description model than QemuOpts can be designed.

I don't see how the introduction of QemuOpts is relevant. That's just a 
flexible commandline option handling system, which makes a lot of sense.

What I'm objecting to is making machine construction be controlled by an 
arbitrary set of hardcoded parameters. One of the goals of the qdev work is 
that you don't need to have hardcoded all the interesting ways a machine can 
vary. Instead you can directly specify how to construct the machine.

My argument is that in the short term this parameterization provides limited 
benefit, and longer term it's going to be obsolete and probably painful to 
support.

Paul

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Anthony Liguori]

On 06/08/2010 09:30 AM, Paul Brook wrote:
>>> I'm undecided how much parameterisation it's worth trying to support in
>>> the device tree. IMO the current code has way too much magic, because
>>> creating a new variant involves hacking and rebuilding pc.c.
>>>        
>> See patch 22/22.  There is really no magic involved, even though the
>> compat machines are not yet as config files.
>>      
> The magic I'm preferring to is precisely things like pci="on".
> This is an artifact of only having a hardcoded single device tree (defined by
> pc_init), so we need magic parameters to pick between different variants.
>
> If this were just a cleanup of how we implement the various machines that
> would be harmless. However these are now a user-visible interface.
>
> It implies that qemu is expected to know how to add/remove PCI capabilities
> from a machine.

No, this implies that the 'pc' MachineCore is capable of removing PCI 
from a machine and that's all.  'pci' is a MachineCore specific option.

>   Once you eliminate machine_register_core that knowledge has
> somehow got to come from your device tree description file.  Having a single
> device tree that can morph into significantly different machines seems like
> unnecessary complexity given this is a user-specified file.
>    

99% of qemu users will never touch a device tree.  The practical matter 
is, we're going to have to provide higher level interfaces that allow a 
user to significantly morph a base device tree into something different.

> What I'm objecting to is making machine construction be controlled by an
> arbitrary set of hardcoded parameters. One of the goals of the qdev work is
> that you don't need to have hardcoded all the interesting ways a machine can
> vary. Instead you can directly specify how to construct the machine.
>    

MachineCore is orthogonal to constructing a device tree from scratch.  
If you want to express the types of things we allow users to do with 
machines, you are going to need a C function that can manipulate these 
devices trees based on higher level constructs (like PCI enabled and a 
scalar RAM size).

IMHO, the only real question is whether this should be built into QEMU 
or we should rely on an external script to do this work.  Since the vast 
majority of users are going to want this higher level interface, I feel 
pretty confident that we should build this into QEMU.

> My argument is that in the short term this parameterization provides limited
> benefit, and longer term it's going to be obsolete and probably painful to
> support.
>    

And how do you expect this to work for x86 machines?  Even if you had a 
pc-0.13.fdt that represented the base pc-0.13, how do we translate -hda 
foo.img into "add an IDE disk to the primary slot as master to the IDE 
controller on the PIIX3 IDE bus".  Sure, you can say -device 
ide-disk,parent=/piix3/ide0 or something like that but something needs 
to make that translation and it's entirely machine specific.  Because 
with the arm boards, -hda foo.img translates to -device scsi-disk,...

A more complicated example is how you support the behavior of something 
like -usbdevice which adds a usb controller if one isn't already present.

This is the logic that MachineCore implements and it will always be 
needed.  The MachineCores expose whatever tunables make sense for the 
machines they represent.

Regards,

Anthony Liguori

> Paul
>
>    

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paul Brook]

> >   Once you eliminate machine_register_core that knowledge has
> > 
> > somehow got to come from your device tree description file.  Having a
> > single device tree that can morph into significantly different machines
> > seems like unnecessary complexity given this is a user-specified file.
> 
> 99% of qemu users will never touch a device tree.  The practical matter
> is, we're going to have to provide higher level interfaces that allow a
> user to significantly morph a base device tree into something different.

That's the bit I don't get.  Why are we significantly morphing a base device 
tree?  Surely it's easier to just use a different base tree.

Paul

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paolo Bonzini]

On 06/08/2010 05:36 PM, Paul Brook wrote:
>>>    Once you eliminate machine_register_core that knowledge has
>>>
>>> somehow got to come from your device tree description file.  Having a
>>> single device tree that can morph into significantly different machines
>>> seems like unnecessary complexity given this is a user-specified file.
>>
>> 99% of qemu users will never touch a device tree.  The practical matter
>> is, we're going to have to provide higher level interfaces that allow a
>> user to significantly morph a base device tree into something different.
>
> That's the bit I don't get.  Why are we significantly morphing a base device
> tree?  Surely it's easier to just use a different base tree.

Because at some point the base tree will have to be written in C.  Of 
course you have to decide where to stop allowing customization, but 
anyway it will be significantly more complex than a QemuOpts config 
file.  QemuOpts obviously doesn't have the flexibility to create an 
entire device tree.

Want to let the user add an IDE controller?  Fine, you need to provide 
him with a way of writing the following logic:

     for(i = 0; i < MAX_IDE_BUS * MAX_IDE_DEVS; i++) {
         hd[i] = drive_get(IF_IDE, i / MAX_IDE_DEVS, i % MAX_IDE_DEVS);
     }
     pci_piix3_ide_init(pci_bus, hd, piix3_devfn + 1);

What do you do?  Invent your own language (another YaST? no thanks)? 
Massive refactoring of QEMU's bowels (cannot solve all the problems, 
Anthony gave more examples in the email I replied to)?  Keep C code 
(then you're not doing anything different than what Anthony's 
MachineCore is doing)?


I think there are four choices:

1) only merge the first 5-7 patches of the series, basically up to the 
creation of -machine and conversion of -kernel/-initrd/-M/-m.  This is 
also what is needed for KVM's irqchip.  Think about some other smart way 
to implement default devices.

2) do not expose some things to the config files.  This requires 
implementing an additional QemuOpts feature for internal options, plus 
the current patchset.  It means that moving machine types out to config 
files is not possible.

3) put some things into an implementation-dependent namespace, i.e. 
__pci or __default_device or __serial.  In the future it's possible to 
these things in config files, just nobody has to worry about users 
relying on them and can refactor to death.  This is the current patchset 
plus some search and replace, and is my favorite.

4) expose everything to the user, at the cost of regretting it later. 
This is the current patchset.


One "smart way to implement default devices" could be an inclusion 
mechanism where a machine can ask qemu to read other config files.  Then 
you'd have config files for the default serial/parallel/etc.  This could 
also be implemented on top of choices 2/3/4 though.

Paolo

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Anthony Liguori]

On 06/08/2010 10:58 AM, Paolo Bonzini wrote:
> On 06/08/2010 05:36 PM, Paul Brook wrote:
>>>>    Once you eliminate machine_register_core that knowledge has
>>>>
>>>> somehow got to come from your device tree description file.  Having a
>>>> single device tree that can morph into significantly different 
>>>> machines
>>>> seems like unnecessary complexity given this is a user-specified file.
>>>
>>> 99% of qemu users will never touch a device tree.  The practical matter
>>> is, we're going to have to provide higher level interfaces that allow a
>>> user to significantly morph a base device tree into something 
>>> different.
>>
>> That's the bit I don't get.  Why are we significantly morphing a base 
>> device
>> tree?  Surely it's easier to just use a different base tree.
>
> Because at some point the base tree will have to be written in C.  Of 
> course you have to decide where to stop allowing customization, but 
> anyway it will be significantly more complex than a QemuOpts config 
> file.  QemuOpts obviously doesn't have the flexibility to create an 
> entire device tree.
>
> Want to let the user add an IDE controller?  Fine, you need to provide 
> him with a way of writing the following logic:
>
>     for(i = 0; i < MAX_IDE_BUS * MAX_IDE_DEVS; i++) {
>         hd[i] = drive_get(IF_IDE, i / MAX_IDE_DEVS, i % MAX_IDE_DEVS);
>     }
>     pci_piix3_ide_init(pci_bus, hd, piix3_devfn + 1);
>
> What do you do?  Invent your own language (another YaST? no thanks)? 
> Massive refactoring of QEMU's bowels (cannot solve all the problems, 
> Anthony gave more examples in the email I replied to)?  Keep C code 
> (then you're not doing anything different than what Anthony's 
> MachineCore is doing)?

Or punt all of the complexity to the user such that have to manually 
figure out where in the device tree a drive has to be inserted.  That's 
not really reasonable IMHO.

> I think there are four choices:
>
> 1) only merge the first 5-7 patches of the series, basically up to the 
> creation of -machine and conversion of -kernel/-initrd/-M/-m.  This is 
> also what is needed for KVM's irqchip.  Think about some other smart 
> way to implement default devices.

Or don't declare MachineCore options stable across releases.  I concur 
that we probably don't want to support some of these interfaces long 
term but I think we can pretty quickly figure out some better interfaces.

The advantage of a full conversion is that it allows us to focus on 
improving the interfaces since we'll have the architecture in place.

> 2) do not expose some things to the config files.  This requires 
> implementing an additional QemuOpts feature for internal options, plus 
> the current patchset.  It means that moving machine types out to 
> config files is not possible.

I dislike having internal only options as it means we have to keep 
things within qemu.  For things like max_cpus, I think we just need to 
validate that we've got passed sane values.  Setting max_cpus in a 
global config is actually a really useful thing for downstreams to do 
since they are all having a fixed support statement of X VCPUs per VM.

> 3) put some things into an implementation-dependent namespace, i.e. 
> __pci or __default_device or __serial.  In the future it's possible to 
> these things in config files, just nobody has to worry about users 
> relying on them and can refactor to death.  This is the current 
> patchset plus some search and replace, and is my favorite.

Yeah, I'm good with being explicit.  I'd prefer x-pci or x-serial since 
we've already established that as the experimential namespace.

> 4) expose everything to the user, at the cost of regretting it later. 
> This is the current patchset.
>
>
> One "smart way to implement default devices" could be an inclusion 
> mechanism where a machine can ask qemu to read other config files.  
> Then you'd have config files for the default serial/parallel/etc.  
> This could also be implemented on top of choices 2/3/4 though.

Default devices are a real pain.  Fortunately, we only mess with it in 
s390 so I'm fairly certain we can simplify things considerable.  It's 
just not something I wanted to tackle in this series.

Regards,

Anthony Liguori

> Paolo

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Alexander Graf]

On 08.06.2010, at 18:15, Anthony Liguori wrote:

> 
>> 4) expose everything to the user, at the cost of regretting it later. This is the current patchset.
>> 
>> 
>> One "smart way to implement default devices" could be an inclusion mechanism where a machine can ask qemu to read other config files.  Then you'd have config files for the default serial/parallel/etc.  This could also be implemented on top of choices 2/3/4 though.
> 
> Default devices are a real pain.  Fortunately, we only mess with it in s390 so I'm fairly certain we can simplify things considerable.  It's just not something I wanted to tackle in this series.

Wait, what? The only "default" device we have on s390 is virtio-console - and that's basically the same as having VGA output on x86 enabled by default, no?

But either way, I agree that we should make things easy for 99% of the users. And taking incremental improvements is useful too. pc.c as it is right now can only be improved, no matter what you do with it.


Alex

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Anthony Liguori]

On 06/08/2010 04:05 PM, Alexander Graf wrote:
> On 08.06.2010, at 18:15, Anthony Liguori wrote:
>
>    
>>      
>>> 4) expose everything to the user, at the cost of regretting it later. This is the current patchset.
>>>
>>>
>>> One "smart way to implement default devices" could be an inclusion mechanism where a machine can ask qemu to read other config files.  Then you'd have config files for the default serial/parallel/etc.  This could also be implemented on top of choices 2/3/4 though.
>>>        
>> Default devices are a real pain.  Fortunately, we only mess with it in s390 so I'm fairly certain we can simplify things considerable.  It's just not something I wanted to tackle in this series.
>>      
> Wait, what? The only "default" device we have on s390 is virtio-console - and that's basically the same as having VGA output on x86 enabled by default, no?
>    

What I was saying is that all the .no_vga/.no_serial/.use_virtcon bits 
in QEMUMachine are strictly due to s390.  Since the use of these are so 
specific, it'll be fairly easy to eliminate them in a more natural way.

What most other boards do is simply ignore whatever vl.c does FWIW.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Anthony Liguori]

> One "smart way to implement default devices" could be an inclusion 
> mechanism where a machine can ask qemu to read other config files.  
> Then you'd have config files for the default serial/parallel/etc.  
> This could also be implemented on top of choices 2/3/4 though.

Defaults are tough because there are multiple ways to create devices 
today.  Here's something that could potentially address the problem.  
I'll focus on -serial but it's equally applicable to the other default 
devices.

We would introduce serial0, serial1, serial2, serial3 MachineCore 
properties.  These properties would be chardev names.

The pc machine types would do:
    QOPT_VALUE("serial0", "__default_serial0");

In vl.c, we would always create a __default_serial0 chardev of 'vc'.

The -serial option would keep a counter and:

   -serial mon:stdio -serial vc

Would be equivalent to:

  -chardev mon:stdio,id=__serial0 -machine serial0=__serial0 \
  -chardev vc,id=__serial1 -machine serial1=__serial1

-nodefaults simply suppresses the creation of the default serial devices.

It's the job of the MachineCore to try to find each of the serialN 
properties and if the chardevs have been created, use qdev to create an 
appropriate serial port device.

This would apply equally well to parallel and virtcon.

Networking is a bit of a nightmare as many boards simply ignore the 
networking options.  Floppy and cdrom can work in this way too.

Regards,

Anthony Liguori

> Paolo

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paul Brook]

> >>>    Once you eliminate machine_register_core that knowledge has
> >>> 
> >>> somehow got to come from your device tree description file.  Having a
> >>> single device tree that can morph into significantly different machines
> >>> seems like unnecessary complexity given this is a user-specified file.
> >> 
> >> 99% of qemu users will never touch a device tree.  The practical matter
> >> is, we're going to have to provide higher level interfaces that allow a
> >> user to significantly morph a base device tree into something different.
> > 
> > That's the bit I don't get.  Why are we significantly morphing a base
> > device tree?  Surely it's easier to just use a different base tree.
> 
> Because at some point the base tree will have to be written in C.

No. You can start with a completely empty machine.
We don't/shouldn't need any machine specific C code.
When I submitted the original qdev code I also posted a proof-of-concept patch 
that built the whole machine based on a config file with no hardcoded 
knowledge of the machine structure.

> Of course you have to decide where to stop allowing customization, but
> anyway it will be significantly more complex than a QemuOpts config
> file.  QemuOpts obviously doesn't have the flexibility to create an
> entire device tree.

Why not?
The current -device option already provides the majority of the functionality 
required.  Most of the missing bits are incomplete qdev conversion.
I'm not saying that creating a whole machine via -device options is an elegant 
solution (hence the interest in things like FDT) but in I see no reason why it 
shouldn't work.

IMO all uses of the no_user flag are bugs resulting from incomplete/incorrect 
qdev conversions.
 
> Want to let the user add an IDE controller?  Fine, you need to provide
> him with a way of writing the following logic:
> 
>      for(i = 0; i < MAX_IDE_BUS * MAX_IDE_DEVS; i++) {
>          hd[i] = drive_get(IF_IDE, i / MAX_IDE_DEVS, i % MAX_IDE_DEVS);
>      }
>      pci_piix3_ide_init(pci_bus, hd, piix3_devfn + 1);
> 
> What do you do?

-device piix3-ide. See also docs/qdev-device-use.txt

Paul

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Anthony Liguori]

On 06/08/2010 09:11 PM, Paul Brook wrote:
>>
>> Because at some point the base tree will have to be written in C.
>>      
> No. You can start with a completely empty machine.
> We don't/shouldn't need any machine specific C code.
>    

I think you're missing the argument.  I should be possible to create a 
machine entirely from a FDT or via -device options.

However, to continue to support the interfaces that we support today, it 
will be necessary to have C code that manipulates a base device tree.

When a user specifies '-M pc -hda foo.img'  verses '-M versatilepb -hda 
foo.img', the equivalent are two very different operations on a device 
tree.  The former adds an ide disk to the default controller and the 
later potentially creates a new scsi bus and then adds a disk to a 
specific bus.

We could force users to these explicit operations instead of the generic 
options we provide today but we would certain end up having our users 
hate us as a result.

> When I submitted the original qdev code I also posted a proof-of-concept patch
> that built the whole machine based on a config file with no hardcoded
> knowledge of the machine structure.
>    

And in your proof-of-concept patches, you couldn't use -hda or any of 
the other command line options that are currently used.  To support 
those things, something would have to understand the device tree well 
enough to know what -hda is supposed to mean.  That's the point of 
MachineCore.

>> Of course you have to decide where to stop allowing customization, but
>> anyway it will be significantly more complex than a QemuOpts config
>> file.  QemuOpts obviously doesn't have the flexibility to create an
>> entire device tree.
>>      
> Why not?
> The current -device option already provides the majority of the functionality
> required.  Most of the missing bits are incomplete qdev conversion.
> I'm not saying that creating a whole machine via -device options is an elegant
> solution (hence the interest in things like FDT) but in I see no reason why it
> shouldn't work.
>
> IMO all uses of the no_user flag are bugs resulting from incomplete/incorrect
> qdev conversions.
>    

I don't disagree.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paul Brook]

> >> Because at some point the base tree will have to be written in C.
> > 
> > No. You can start with a completely empty machine.
> > We don't/shouldn't need any machine specific C code.
> 
> I think you're missing the argument.  I should be possible to create a
> machine entirely from a FDT or via -device options.
> 
> However, to continue to support the interfaces that we support today, it
> will be necessary to have C code that manipulates a base device tree.
>
> When a user specifies '-M pc -hda foo.img'  verses '-M versatilepb -hda
> foo.img', the equivalent are two very different operations on a device
> tree.  The former adds an ide disk to the default controller and the
> later potentially creates a new scsi bus and then adds a disk to a
> specific bus.

AFAICS the current commandline options only result in simple addition of 
devices. They might add slightly different devices in slightly different 
places, but that's easy to accommodate by having the machine define a few 
standard device/bus IDs.

IMO it's even more lame if -hda shops working when you supply a device tree.

Paul

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Blue Swirl]

On Wed, Jun 9, 2010 at 2:30 PM, Paul Brook <paul@codesourcery.com> wrote:
>> >> Because at some point the base tree will have to be written in C.
>> >
>> > No. You can start with a completely empty machine.
>> > We don't/shouldn't need any machine specific C code.
>>
>> I think you're missing the argument.  I should be possible to create a
>> machine entirely from a FDT or via -device options.
>>
>> However, to continue to support the interfaces that we support today, it
>> will be necessary to have C code that manipulates a base device tree.
>>
>> When a user specifies '-M pc -hda foo.img'  verses '-M versatilepb -hda
>> foo.img', the equivalent are two very different operations on a device
>> tree.  The former adds an ide disk to the default controller and the
>> later potentially creates a new scsi bus and then adds a disk to a
>> specific bus.
>
> AFAICS the current commandline options only result in simple addition of
> devices. They might add slightly different devices in slightly different
> places, but that's easy to accommodate by having the machine define a few
> standard device/bus IDs.
>
> IMO it's even more lame if -hda shops working when you supply a device tree.

The tree supplied by the user should label a bus node with a property
'QEMU,hda'. The C version (called by the board) would be something
like setprop("/i440fx/pci.0/ide.0", "QEMU,hda"). QEMU should search
the device tree for such labels at startup.

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Anthony Liguori]

On 06/09/2010 03:47 PM, Blue Swirl wrote:
> On Wed, Jun 9, 2010 at 2:30 PM, Paul Brook<paul@codesourcery.com>  wrote:
>    
>>>>> Because at some point the base tree will have to be written in C.
>>>>>            
>>>> No. You can start with a completely empty machine.
>>>> We don't/shouldn't need any machine specific C code.
>>>>          
>>> I think you're missing the argument.  I should be possible to create a
>>> machine entirely from a FDT or via -device options.
>>>
>>> However, to continue to support the interfaces that we support today, it
>>> will be necessary to have C code that manipulates a base device tree.
>>>
>>> When a user specifies '-M pc -hda foo.img'  verses '-M versatilepb -hda
>>> foo.img', the equivalent are two very different operations on a device
>>> tree.  The former adds an ide disk to the default controller and the
>>> later potentially creates a new scsi bus and then adds a disk to a
>>> specific bus.
>>>        
>> AFAICS the current commandline options only result in simple addition of
>> devices. They might add slightly different devices in slightly different
>> places, but that's easy to accommodate by having the machine define a few
>> standard device/bus IDs.
>>
>> IMO it's even more lame if -hda shops working when you supply a device tree.
>>      
> The tree supplied by the user should label a bus node with a property
> 'QEMU,hda'. The C version (called by the board) would be something
> like setprop("/i440fx/pci.0/ide.0", "QEMU,hda"). QEMU should search
> the device tree for such labels at startup.
>    

That works for the very simple case of -hda, but what do you do for:

-drive file=foo.img,bus=0,index=2

I think it's possible to come up with some form of solution once you 
start adding synthetic properties or dummy devices but ultimately I 
think it pollutes the device tree.  I think the easiest way to support 
the interfaces we provide now is to just code the device tree 
manipulation in C.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Blue Swirl]

On Wed, Jun 9, 2010 at 8:52 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 06/09/2010 03:47 PM, Blue Swirl wrote:
>>
>> On Wed, Jun 9, 2010 at 2:30 PM, Paul Brook<paul@codesourcery.com>  wrote:
>>
>>>>>>
>>>>>> Because at some point the base tree will have to be written in C.
>>>>>>
>>>>>
>>>>> No. You can start with a completely empty machine.
>>>>> We don't/shouldn't need any machine specific C code.
>>>>>
>>>>
>>>> I think you're missing the argument.  I should be possible to create a
>>>> machine entirely from a FDT or via -device options.
>>>>
>>>> However, to continue to support the interfaces that we support today, it
>>>> will be necessary to have C code that manipulates a base device tree.
>>>>
>>>> When a user specifies '-M pc -hda foo.img'  verses '-M versatilepb -hda
>>>> foo.img', the equivalent are two very different operations on a device
>>>> tree.  The former adds an ide disk to the default controller and the
>>>> later potentially creates a new scsi bus and then adds a disk to a
>>>> specific bus.
>>>>
>>>
>>> AFAICS the current commandline options only result in simple addition of
>>> devices. They might add slightly different devices in slightly different
>>> places, but that's easy to accommodate by having the machine define a few
>>> standard device/bus IDs.
>>>
>>> IMO it's even more lame if -hda shops working when you supply a device
>>> tree.
>>>
>>
>> The tree supplied by the user should label a bus node with a property
>> 'QEMU,hda'. The C version (called by the board) would be something
>> like setprop("/i440fx/pci.0/ide.0", "QEMU,hda"). QEMU should search
>> the device tree for such labels at startup.
>>
>
> That works for the very simple case of -hda, but what do you do for:
>
> -drive file=foo.img,bus=0,index=2

This is a host device, so the device tree does not address that. But
previously I proposed that also the host devices (drives, chardev,
netdev etc.) could be considered to be qdev like devices and qemu_irqs
could be used to communicate between the host devices and
machine/board. Maybe that should be reconsidered.

> I think it's possible to come up with some form of solution once you start
> adding synthetic properties or dummy devices but ultimately I think it
> pollutes the device tree.  I think the easiest way to support the interfaces
> we provide now is to just code the device tree manipulation in C.

I don't think pollution is a problem with careful prefixing, also
Linux adds some properties to its device tree. We could also prune our
properties from the tree before passing it around if necessary.

Re: [Qemu-devel] Re: [PATCH 0/22] Refactor machine support

[Paul Brook]

> >> AFAICS the current commandline options only result in simple addition of
> >> devices. They might add slightly different devices in slightly different
> >> places, but that's easy to accommodate by having the machine define a
> >> few standard device/bus IDs.
> >> 
> >> IMO it's even more lame if -hda shops working when you supply a device
> >> tree.
> > 
> > The tree supplied by the user should label a bus node with a property
> > 'QEMU,hda'. The C version (called by the board) would be something
> > like setprop("/i440fx/pci.0/ide.0", "QEMU,hda"). QEMU should search
> > the device tree for such labels at startup.
> 
> That works for the very simple case of -hda, but what do you do for:
> 
> -drive file=foo.img,bus=0,index=2

I don't see how that is any harder. I'm assuming that -hda is a trivial alias 
for -drive bus=0,index=0. Fact is that -drive supports a very limited set of 
operations. Figuring out the default interface is easy - use whatever exists 
on the machine. Creating IDE disks is easy, just add them to the existing IDE 
bus.  For SCSI/virtio you might need to add a HBA first, but that's also 
fairly trivial.  Most other interfaces (sd, pflash) don't do automagical 
device creation to start with - if you don't specify an image then you get an 
empty/disconnected device.

> I think it's possible to come up with some form of solution once you
> start adding synthetic properties or dummy devices but ultimately I
> think it pollutes the device tree.  I think the easiest way to support
> the interfaces we provide now is to just code the device tree
> manipulation in C.

I disagree. If programmatic machine construction/manipulation was sufficient 
then we wouldn't need device trees at all.  Current qemu supports an extremely 
limited set of machines.  Bear in mind that x86 PCs are, pretty much by 
definition, approximately all the same to start with.  Once you start looking 
at other platforms (e.g. arm boards) you realise just how cripplingly 
restrictive the this is.  Restricting -hda, -drive, etc. and similar to a 
small subset of built-in machines is IMO and unacceptably poor solution.

Some of your arguments seem to be self-fulfilling prophecies. If you design 
your new option structure around a magical self-morphing device tree then it's 
unlikely that this will work with user specified device trees.  I believe the 
current set of commandline options are sufficient abstract that they do not 
dictate a particular implementation.

Paul