[Qemu-devel] Machine description, an alternativ using XML

[Qemu-devel] Machine description, an alternativ using XML

[Torbjörn Andersson]

Greetings!

I, and my collegues, have been using QEMU for some time and we had from the
beginning a need for a more flexible and readable machine definition.
Therefore we developed a solution based on XML with a parser using expat.

I will try to exemplify our solution with a "snip" a ARM INTEGRATOR machine.

<MACHINE name=ARM-INTEGRATOR>
  <OBJECT name="cpu" class="ARM926ej">
    <ATTRIBUTE name="mhz">	<value integer="208"/>  </ATTRIBUTE>
    <ATTRIBUTE name="dtcm_size">	<value integer="8192"/> </ATTRIBUTE>
	<snipp...>
    <ATTRIBUTE name="memmap">	<value obj_ref=":memmap"/>  </ATTRIBUTE>
  </OBJECT>
  <OBJECT name="memmap" class="MEMMAP">
  </OBJECT>
    ....
  <OBJECT name="pic" class="pl190">
    <ATTRIBUTE name="pic">	<value obj_ref=":cpu"/>  </ATTRIBUTE>
    <ATTRIBUTE name="memmap">	<value obj_ref=":memmap"/>  </ATTRIBUTE>
    <ATTRIBUTE name="base">	<value integer="0xc0010000"/>  </ATTRIBUTE>
  </OBJECT>
<MACHINE/>

Firstly, in our solution we call properties attributes. We define
relationships between objects using textual references, with a naming
convention where a name with ':' signals that the name of the target is
relative to the object.
We also have a explicit memmap, all the global variables in exec.c are moved
into a specific QEMU class. There could be several parallel memmaps and
"multi-homed" devices could be attached to several address-spaces.

Our XML files are at QEMU startup parsed by expat and our custom code and we
create an instance of a tree, containing all the information. At the time
the machine is created, a copy of the "template" tree is create and we start
a two-pass traversal of the new tree. We start with creating all the
objects, which enables us to replace all obj_ref values with obj_ptr values
(Pointers to real objects). From this point we can continue with
initializing all our objects, by updating their attributes with the values
in the machine configuration.

I our case, we have found that our "targets" are quite similar, and
therefore we support "clusters" to be defined in XML. Clusters are not
machines but very much alike, an instance of a cluster cannot be the top of
a tree.

All interactions between objects are done through interfaces defined in
h-files. The interfaces and attributes are registered in a "CLASS and
OBJECT" registry that we have added. An object can ask for an interface for
a specific object and the interface is returned, a struct with function
pointers.

I will not try to describe the solution more, because I believe that you now
see what we have built. There is a lot more details that are quite nice. We
defined a inheritance scheme that allows to share even more between
different machines. 

Our question is now: are you interested in this solution? I think we can
quite swiftly provide a patch-set to QEMU where we show the solution in the
ARM-target domain.

I know you are looking at a solution based on FDT. My belief is that the XML
solution is more flexible, but I admit that I know very little about FDT.
Further, I believe that one can create FDTs, from the XML machine
definitions, in runtime and pass them to the target-os if required.

The strong point with the XML solution is that it is very suitable for
modeling embedded systems where lots of GPIOs, interrupts, dma-channels,
i2c, spi, i2s/pcm etc.

I know that this is will result in a large patch set but I think the FDT is
equally big. Further I believe we can have both schemes in QEMU in parallel,
if necessary.

Please take this into consideration. We would be happy to share our solution
with QEMU.

/Best Regards Torbjörn Andersson

Re: [Qemu-devel] Machine description, an alternativ using XML

[Paul Brook]

>   <OBJECT name="pic" class="pl190">
>     <ATTRIBUTE name="pic">	<value obj_ref=":cpu"/>  </ATTRIBUTE>
>     <ATTRIBUTE name="memmap">	<value obj_ref=":memmap"/>  </ATTRIBUTE>
>     <ATTRIBUTE name="base">	<value integer="0xc0010000"/>  </ATTRIBUTE>
>   </OBJECT>
> <MACHINE/>

It looks like you're just using XML to encapsulate simple <key,value> pairs, 
which IMHO is completely the wrong way to use XML. Qemu already has to know 
about things like IO regions, IRQs, bus bindings. XML gives you the power to 
describe these things properly, rather than relying on clumsy naming 
conventions.

The current device implementations already know two much about this kind of 
detail. There's no reason[1] why every device needs to know how to map IO 
regions. They should just tell qemu what IO regions they have, and generic 
code will sort out the rest. Similarly for IRQs and bus bindings the device 
just registers that it provides particular functionality (IRQ source, I2c 
master/slave) and doesn't know or care how these are connected. PCI devices 
already have some of this abstraction, IMHO we need to go further (e.g. by 
removing the map_func callback), and extend this to other systems.

I think one of the most important goals of qemu machine description 
infrastructure is to abstract this kind of detail away from the devices.

Paul

[1] Ignoring the VGA hacks, which need to go way anyway.

SV: [Qemu-devel] Machine description, an alternativ using XML

[Torbjörn Andersson]

Thanks for your replies. I do not really aim to force my solution into QEMU,
but I try visualize the good parts. It would be a bonus if my code end up
contributed.

I recognize that my XML usage is quite poor. I will tight it up according to
Jamie's suggestion. I solved my problem at the time and it have been good
enough since then. We have now been using it with lots of targets, >20. We
have been able to identify the common parts of our machines and decreased
our maintenance code and duplication. 

Paul B wrote:
> It looks like you're just using XML to encapsulate simple <key,value>
> pairs,
> which IMHO is completely the wrong way to use XML.

I don't see the fundamental difference compared to FDT. Does it mean that
FDT is wrong too? I guess you can explain a bit more.

mpic: pic@40000 {
  interrupt-controller;
  #address-cells = <0>;
  #interrupt-cells = <2>;
  reg = <0x40000 0x40000>;
  compatible = "chrp,open-pic";
  device_type = "open-pic";
};

Note: if FDT fulfills our requirements, I expect that we will move to FDT
too.

All our QEMU knows about is classes, attributes and interfaces. Clusters and
Machines are simply specialized classes. Defining machines/clusters in our
solution is like picking building blocks and sticking them together in XML.
A building block could be a PIC or an entire cluster.

There is no "naming-convention" in our XML solution what so ever. The reason
for why we have object reference described as strings like ":pic" are two.
(pic is simply a suitable name for a pl190)

First we have a layered design, where a cluster/machine describes one layer.
A cluster in our design could as an example be the OMAP SOC. The OMAP SOC is
alone not a machine. It needs to be complemented with RAM and other stuff
and that defines the machine and hooks it together with known parts in the
OMAP cluster. The OMAP itself could also be defined with other clusters.
Therefore the ':' in the object reference tells QEMU where to look.  

The second reason is that we do not want to use fully qualified names
because we might want to run two or more machines within the same QEMU
instance. We prefer to have all machines in one QEMU instance because it is
easier to control and deterministic. (I can submit patches for this too,
including CPU threading support)

In the end we got object names like this
<instance name of machine>
<instance name of machine>_<OMAP name defined in machine>
<instance name of machine>_<OMAP name defined in machine >_<name within the
omap cluster>

I recommend having named objects because it makes life easier when debugging
the state of the machine. Imagine having several objects of the same class
and only be able to separate them with the address to the state-struct. We
also have a object browser, where we can browse the machine and all its
components with current values on attributes, quite powerful and useful for
both users and QEMU developers.

I want to mention on more thing about our clusters and machines. In our XML
solution a cluster can be coupled with a specific "builder" class. The
standard builder, used when none is specified, simple creates all objects,
resolves all object references (obj_ref -> obj. pointers) and sets all
attributes. A specialized "builder" can do more, like adding interfaces for
the entire cluster. I.e. things that are not suitable or impossible to be
defined in XML.

Paul wrote:
> I think one of the most important goals of qemu machine description
> infrastructure is to abstract this kind of detail away from the
> devices.

Such centralization of logic is well suited in a class with the specific
role of easing the "load" of its children. In this case a "builder" class
described above. 

Andreas F wrote:
>True, but XML itself does not allow for arbitrary graphs, only trees.  

Hmm, what kind of graphs could be needed? (Out of curiosity) 

Blue S. wrote:
> OK, so now we have:
> 1 - Fabrice's original proposal
> 2 - Paul's FDT based work
> 3 - Mark's FDT based work
> 4 - Torbjörn's XML based configuration
> 5 - Microsoft XML configuration

I'm honored to be on the list and not last! :)

/BR Torbjörn

Re: [Qemu-devel] Machine description, an alternativ using XML

[Anthony Liguori]

Paul Brook wrote:
>>   <OBJECT name="pic" class="pl190">
>>     <ATTRIBUTE name="pic">	<value obj_ref=":cpu"/>  </ATTRIBUTE>
>>     <ATTRIBUTE name="memmap">	<value obj_ref=":memmap"/>  </ATTRIBUTE>
>>     <ATTRIBUTE name="base">	<value integer="0xc0010000"/>  </ATTRIBUTE>
>>   </OBJECT>
>> <MACHINE/>
>>     
>
> It looks like you're just using XML to encapsulate simple <key,value> pairs, 
> which IMHO is completely the wrong way to use XML. Qemu already has to know 
> about things like IO regions, IRQs, bus bindings. XML gives you the power to 
> describe these things properly, rather than relying on clumsy naming 
> conventions.
>   

I'm really don't want to see this get furthered rat holed.  Since we all 
seemed relatively happy with the device tree as a configuration file, 
can we agree to that firmly and avoid this discussion?

Regards,

Anthony Liguori