Re: [Qemu-devel] QEmu as a Device Software Optimization tool

Re: [Qemu-devel] QEmu as a Device Software Optimization tool

[n schembr]

 
-snap-


>Personally though I don't see much benefit to simple syntax config 

>files over C files, that are being used now. 



Config files implies a self check process.  A better question might be, has qemu grown to the point where an outsider is going to define a new  platform?  

Could an arm based cell phone manufacture use the config files to define the hardware for the emulator without a full port? Iphone anyone :)

Nicholas A. Schembri 
state  college pa usa 


 










          

[Qemu-devel] Re: QEmu as a Device Software Optimization tool

[Hollis Blanchard]

On Thu, 26 Jul 2007 21:00:31 +0200, andrzej zaborowski wrote:
> 
> There is some interesting work being done on a similar project by Paul
> Sokolovsky for his and Maria Zabolotnaya's Google Summer Of Project.
> In their approach machine descriptions (but not only) are written in
> python, which lets you hope that they can have a simple syntax,
> practically like a config file, if desired, or a much more powerful
> one, so it seems to be a better idea than plain config files.
> 
> Personally though I don't see much benefit to simple syntax config
> files over C files, that are being used now.

Really? Take a look at ppc_prep_init()...

I would love to see machine definition done in python.

-- 
Hollis Blanchard
IBM Linux Technology Center

Re: [Qemu-devel] QEmu as a Device Software Optimization tool

[Paul Borman]

The embedded space contains a vast number of boards, often only  
different by what devices are use, where they are located, etc.   
Building a new version of qemu for each board would be burdensome.   
The hope would be that we could build a generic qemu (for an  
architecture family), say, ppc_generic, which then read an external  
file (which, in our case, will be generated from board description)  
and configure itself according to that.  Further, by allowing devices  
to be loaded vi dlopen(), devices can be added after qemu has been  
built.  This would also allow the developer to specify the exact  
location of the device, rather than having a list of IRQ's and base  
ports, etc.

For the embedded world, yes, "outsiders" are certainly going to want  
to define new platforms and more than likely, provide their own  
custom devices.  They key is they do not need to muck with the real  
internals of qemu, just the part that picks the bits and pieces to use.

Oh, and if I am an outsider, then absolutely the answer is yes!, I  
already have had the need :-)

	-Paul

On Jul 26, 2007, at 3:48 PM, n schembr wrote:

>
> -snap-
>
>
>> Personally though I don't see much benefit to simple syntax config
>
>> files over C files, that are being used now.
>
>
>
> Config files implies a self check process.  A better question might  
> be, has qemu grown to the point where an outsider is going to  
> define a new  platform?
>
> Could an arm based cell phone manufacture use the config files to  
> define the hardware for the emulator without a full port? Iphone  
> anyone :)
>
> Nicholas A. Schembri
> state  college pa usa
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Re: [Qemu-devel] Re: QEmu as a Device Software Optimization tool

[Paul Borman]

The platform I started with was ppc_prep :-)

So, is Python already part of qemu?  Personally, I would have a goal  
that the definition of a device would be simple enough that there  
would be no reason to have anything more complicated than a simple  
text file.  While I would not suggest that the BSD config file syntax  
should be used, I do think it is instructive.

It has a section for defining macros, really, the standard name/value  
pairs

	options         KTR                     # kernel wraparound trace  
buffer support
	options "KTR_SIZE=32768"        # trace buffer in bytes (def=8K)
	options "KTR_COMPILE='~0'"      # compile in all traces
	options "KTR_MASK=0"            # but turn none of them on

Clearly there would be cleaner ways to specify this for QEmu

	set memory_size	0x10000000

There is also a section that specifies the various devices.  Here is  
a snippet of a kernel for an embedded PPC board I used to work on:


	# main (on-board io) bus
	obio0   at root
	pci0  at obio0

	# COM ports
	# Flags are 0xF00000 for fifo flags F; see com(4).
	# Default is 0; 1, 2, 3, 4 set receive fifo trigger to 1, 4, 8, 14;
	# 9 ignores the FIFO on 16550/16550AF.
	com0    at obio? addr "0xff600300" irq 3
	com1    at obio? addr "0xff600400" irq 4

	# Crystal Semiconductor Ethernet
	ce0     at obio? addr "0x7fe00c00" irq 24

	# Disk On Chip 2000 driver
	dc0    at obio? iomem "0x800d0000" iosiz 0x2000

This particular syntax specifies several things:

1)  The devices in a generic way
2)  How devices are connected
3)  The order to search for the devices

Again, QEmu would need something slightly different, but at least  
this demonstrates how a similar problem has been previous solved.

There are also pseudo devices

	pseudo-device   loop
	pseudo-device   pty
	pseudo-device sl      1

QEmu might be able to treat pseudo and "physical" devices the same,  
or maybe keeping them independent would be good.

Using python, or some general purpose scripting language, to  
create .c files for inclusion in the compilation seems like it may  
have some merit, though I see these as different issues.  I wrote a  
PPC virtual machine to run on PPC hosts (i.e., more like vmware).   
When I found that I wanted a little more than a simple ascii file, I  
settled on pretty much the standard sh syntax.  I have written (and  
am willing to contribute) an embedded version of sh which only calls  
builtin functions.  I used it both for my monitor as well as my  
config syntax (config files were essentially read by the monitor)

		-Paul



>> Personally though I don't see much benefit to simple syntax config
>> files over C files, that are being used now.
>
> Really? Take a look at ppc_prep_init()...
>
> I would love to see machine definition done in python.
>
> -- 
> Hollis Blanchard
> IBM Linux Technology Center
>
>
>
>

Re: [Qemu-devel] Re: QEmu as a Device Software Optimization tool

[Alexander Voropay]

"Paul Borman" wrote:

>Personally, I would have a goal  
> that the definition of a device would be simple enough that there  
> would be no reason to have anything more complicated than a simple  
> text file.

 Look at the Virtutech Simics emulator. (It's closed source product but free
for academic use). Their config file allows to "construct" a whole machine
from the "processor", "bus", PCI devices e.t.c. components.

http://www.virtutech.com/

script file to crate a Pentium machine with 440BX chipset:
===========
if not defined freq_mhz {$freq_mhz = 20}
if not defined cpi {$cpi = 1}
if not defined disk_size {$disk_size = 20496236544}
if not defined disk_image {}
if not defined rtc_time {$rtc_time = "2002-09-18 10:00:00 UTC"}
if not defined num_cpus {$num_cpus = 1}
if not defined memory_megs {$memory_megs = 256}
if not defined cpu_class {$cpu_class = "pentium-4"}
if not defined text_console {$text_console = "no"}
if not defined use_acpi {$use_acpi = TRUE}
###
add-directory "%simics%/targets/x86-440bx/images"
import-isa-components
import-pci-components
import-std-components
import-x86-components

$nb = (create-north-bridge-443bx)
$vga = (create-pci-voodoo3)
$nb.connect pci-slot5 $vga
$sb = (create-south-bridge-piix4)
$sio = (create-std-super-io)

$console.connect keyboard $sio kbd-console
$console.connect $vga
$system.connect chipset $nb
$system.connect interrupt $sb
$system.connect reset $sio
$nb.connect pci-slot7 $sb
$sb.connect $sio
$sb.connect ide0-master $disk

e.t.c.

--
-=AV=-

*******************************************************************************************************
This message and any attachments (the "message") are confidential and intended solely for the addressees. 
Any unauthorised use or dissemination is prohibited.
Messages are susceptible to alteration. Orange Business Services shall not be liable for the message if altered, changed or
falsified. If you are not the intended addressee of this message, please cancel it immediately and inform 
the sender.
*******************************************************************************************************

[Qemu-devel] Re: QEmu as a Device Software Optimization tool

[Brian Johnson]

n schembr wrote:
> 
> -snap-
> 
>> Personally though I don't see much benefit to simple syntax config
>> files over C files, that are being used now.
> 
> Config files implies a self check process.  A better question might
> be, has qemu grown to the point where an outsider is going to define
> a new platform?

Absolutely.  I've been using it to prototype firmware for a NUMA 
chipset.  I've had to invent ugly command-line hacks for specifying the 
number of nodes, per-node consoles, etc.  And I still need to use a 
debugger to override variable values in my init. code when I want to 
test "odd" cases which my hacks don't support.  I could make good use of 
a generalized configuration file.

Brian

Re: [Qemu-devel] Re: QEmu as a Device Software Optimization tool

[andrzej zaborowski]

On 27/07/07, Paul Borman <paul.borman@windriver.com> wrote:
> The platform I started with was ppc_prep :-)
>
> So, is Python already part of qemu?  Personally, I would have a goal
> that the definition of a device would be simple enough that there
> would be no reason to have anything more complicated than a simple
> text file.  While I would not suggest that the BSD config file syntax
> should be used, I do think it is instructive.

The python thing is not in qemu and is (I think) a long term project
in not much sync with qemu now, but I don't know the details.

>
> It has a section for defining macros, really, the standard name/value
> pairs
>
>         options         KTR                     # kernel wraparound trace
> buffer support
>         options "KTR_SIZE=32768"        # trace buffer in bytes (def=8K)
>         options "KTR_COMPILE='~0'"      # compile in all traces
>         options "KTR_MASK=0"            # but turn none of them on
>
> Clearly there would be cleaner ways to specify this for QEmu
>
>         set memory_size 0x10000000
>
> There is also a section that specifies the various devices.  Here is
> a snippet of a kernel for an embedded PPC board I used to work on:
>
>
>         # main (on-board io) bus
>         obio0   at root
>         pci0  at obio0
>
>         # COM ports
>         # Flags are 0xF00000 for fifo flags F; see com(4).
>         # Default is 0; 1, 2, 3, 4 set receive fifo trigger to 1, 4, 8, 14;
>         # 9 ignores the FIFO on 16550/16550AF.
>         com0    at obio? addr "0xff600300" irq 3
>         com1    at obio? addr "0xff600400" irq 4
>
>         # Crystal Semiconductor Ethernet
>         ce0     at obio? addr "0x7fe00c00" irq 24
>
>         # Disk On Chip 2000 driver
>         dc0    at obio? iomem "0x800d0000" iosiz 0x2000
>
> This particular syntax specifies several things:
>
> 1)  The devices in a generic way
> 2)  How devices are connected
> 3)  The order to search for the devices
>
> Again, QEmu would need something slightly different, but at least
> this demonstrates how a similar problem has been previous solved.
>
> There are also pseudo devices
>
>         pseudo-device   loop
>         pseudo-device   pty
>         pseudo-device sl      1
>
> QEmu might be able to treat pseudo and "physical" devices the same,
> or maybe keeping them independent would be good.
>
> Using python, or some general purpose scripting language, to
> create .c files for inclusion in the compilation seems like it may
> have some merit, though I see these as different issues.  I wrote a
> PPC virtual machine to run on PPC hosts (i.e., more like vmware).
> When I found that I wanted a little more than a simple ascii file, I
> settled on pretty much the standard sh syntax.  I have written (and
> am willing to contribute) an embedded version of sh which only calls
> builtin functions.  I used it both for my monitor as well as my
> config syntax (config files were essentially read by the monitor)

I think python or other scripting language are a much better idea than
a custom syntax with only variables and static content. I would hate
to write 121 lines to set up initial states of the 121 gpios present
in pxa-based machines, I would use a loop.

In my experience embedded machines even those with very similar
capabilities and coming from the same manufacturer have huge hardware
differences between models. My experience is only with ARM. The common
parts between the different ARM-based computers are usually only the
CPU (which fortunately embeds many peripherals) and if there's an
external chip onboard that two devices use in common then it usually
is wired differently or there are other quirks that will force you
dirty your hands in C code and rebuild (which doesn't really take
significantly more time than editing a config file which is loaded in
runtime). Take as an example a peripheral that has one of the signals
connecting it with the cpu/bus, inverted/pulled up/unconnected, it
would be very hard to make a simple text syntax that accounts for
that, and python might potentially do it without much pain (this
example is a very usual case between PDAs).

Regards

Re: [Qemu-devel] QEmu as a Device Software Optimization tool

[Paul Sokolovsky]

Hello Paul,

Friday, July 27, 2007, 12:25:13 PM, you wrote:

> The embedded space contains a vast number of boards, often only  
> different by what devices are use, where they are located, etc.   
> Building a new version of qemu for each board would be burdensome.   
> The hope would be that we could build a generic qemu (for an  
> architecture family), say, ppc_generic, which then read an external  
> file (which, in our case, will be generated from board description)  
> and configure itself according to that.  Further, by allowing devices
> to be loaded vi dlopen(), devices can be added after qemu has been  
> built.  This would also allow the developer to specify the exact  
> location of the device, rather than having a list of IRQ's and base  
> ports, etc.

> For the embedded world, yes, "outsiders" are certainly going to want  
> to define new platforms and more than likely, provide their own  
> custom devices.  They key is they do not need to muck with the real  
> internals of qemu, just the part that picks the bits and pieces to use.

  Funnily, this is the same argumentation as I used more than
half-year ago when there was last big discussion on the need of
flexible and generalized config file:
http://lists.gnu.org/archive/html/qemu-devel/2006-10/msg00249.html
Then, it received rather cool response, I actually don't remember any
single vote for flexible/structural config and plugins. Well, I always
thought that just a small time will be required for more wider
audience to chime in and come back to this idea ;-).

> Oh, and if I am an outsider, then absolutely the answer is yes!, I  
> already have had the need :-)

>         -Paul

[]

-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com

Re: [Qemu-devel] Re: QEmu as a Device Software Optimization tool

[Paul Sokolovsky]

Hello,

Andrzej Zaborowski wrote:

> There is some interesting work being done on a similar project by Paul
> Sokolovsky for his and Maria Zabolotnaya's Google Summer Of Project.

  Yes, there's such project, being done from Handhelds.org, with the
motivation that the project works on porting Linux to several dozens of
PDA, while having less than a dozen people active in any given time,
and concentrating on small subset of devices. Thus, to leverage
emulation technology to our devices (and this brings obvious
benefits), there would need to be rapid prototyping technology
allowing easy reuse of already existing components.

  The project itself is more proof-of-concept though.


Friday, July 27, 2007, 12:46:29 PM, Paul Borman wrote:

> The platform I started with was ppc_prep :-)



> So, is Python already part of qemu?

  No, but fortunately it's not required at all. The idea is to have
Python wrapper for entire QEMU functionality, so that QEMU can be
completely controlled from Python code. Its relation to QEMU is the
same as that of PyGTK+ and GTK+ - the former should not be part of the
latter.

  Such a design besides being a technical best practice, also rooted in
the outcome of the mentioned Oct 2006 discussion, when it became clear
that flexible configuration won't be a part of QEMY any time soon ;-).

[]

> Using python, or some general purpose scripting language, to  
> create .c files for inclusion in the compilation seems like it may  
> have some merit, though I see these as different issues.

  That's not purpose of the Python module in question. Being a wrapper
around QEMU, it would allow to use Python's dynamical and high-level nature
to easily "construct" virtual board as well as handle "boring" tasks
like configuration file parsing, logging etc. So, the module itself
allows to specify machine in "imperative" description. I'm great
proponent of declarative descriptions myself, but designing such would
need more work. Plus, as the old discussion showed, there's no
unanimity even regarding what syntax it should use! (I for one find it
amusing that in 2006 people could think that writing XML parser would
be a prerequisite for XML use, and in even in 2007 come with idea of
"simple ascii files" ;-)).

>   I wrote a  
> PPC virtual machine to run on PPC hosts (i.e., more like vmware).   
> When I found that I wanted a little more than a simple ascii file, I  
> settled on pretty much the standard sh syntax.  I have written (and  
> am willing to contribute) an embedded version of sh which only calls  
> builtin functions.  I used it both for my monitor as well as my  
> config syntax (config files were essentially read by the monitor)

  Good luck pushing it upstream! As I wrote yet back in that thread,
there're more political than technical reasons why QEMU still doesn't
have it, and won't have it for some time more. But of course, raising
awareness and lobbying actions should help with this.

>                 -Paul



>>> Personally though I don't see much benefit to simple syntax config
>>> files over C files, that are being used now.
>>
>> Really? Take a look at ppc_prep_init()...
>>
>> I would love to see machine definition done in python.
>>
>> -- 
>> Hollis Blanchard
>> IBM Linux Technology Center
>>
>>
>>
>>





-- 
Best regards,
 Paul                            mailto:pmiscml@gmail.com