[Qemu-devel] [PATCH] Add support for a configuration file

[Qemu-devel] [PATCH] Add support for a configuration file

[Anthony Liguori]

There has been an awful lot of discussion about a configuration file with
almost no general agreement except that one is strongly desired.

I thought about it a bit, and I think a nice step would be to simply allow
the current configuration parameters to be stored in a file using a pretty
familiar format.

I think this is pretty useful as-is.  I think it also gives us a reasonable
way to move forward that will keep everyone pretty happy.

Here's a short example:

qemu-system-x86_64 -hda ~/images/linux.img -snapshot -vnc :2

Would become `foo.qemu':

 # Main disk image
 hda=/home/anthony/images/linux.img

 # Redirect disk writes to a temporary image
 snapshot

 # Make the graphical display available on port 5902
 vnc=:2

With:

qemu-system-x86_64 -config foo.qemu

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

diff --git a/qemu-doc.texi b/qemu-doc.texi
index cca483c..4861fc0 100644
--- a/qemu-doc.texi
+++ b/qemu-doc.texi
@@ -395,6 +395,12 @@ Sets the @var{name} of the guest.
 This name will be display in the SDL window caption.
 The @var{name} will also be used for the VNC server.
 
+@item -config @var{file}
+Reads configuration options from @var{file}.  The format of @var{file} is the
+same as the command line options, except no dash ``-'' is required.  Options
+that take an argument are in the format @var{option=value}.  A pound ``#''
+character can be used as a comment.
+
 @end table
 
 Display options:
diff --git a/vl.c b/vl.c
index 67712f0..2eb39dd 100644
--- a/vl.c
+++ b/vl.c
@@ -7276,6 +7276,7 @@ static void help(int exitcode)
            "-clock          force the use of the given methods for timer alarm.\n"
            "                To see what timers are available use -clock ?\n"
            "-startdate      select initial date of the clock\n"
+	   "-config FILE    read command line options from FILE\n"
            "\n"
            "During emulation, the following keys are useful:\n"
            "ctrl-alt-f      toggle full screen\n"
@@ -7379,6 +7380,7 @@ enum {
     QEMU_OPTION_old_param,
     QEMU_OPTION_clock,
     QEMU_OPTION_startdate,
+    QEMU_OPTION_config,
 };
 
 typedef struct QEMUOption {
@@ -7490,6 +7492,7 @@ const QEMUOption qemu_options[] = {
 #endif
     { "clock", HAS_ARG, QEMU_OPTION_clock },
     { "startdate", HAS_ARG, QEMU_OPTION_startdate },
+    { "config", HAS_ARG, QEMU_OPTION_config },
     { NULL },
 };
 
@@ -7665,9 +7668,106 @@ static BOOL WINAPI qemu_ctrl_handler(DWORD type)
 }
 #endif
 
+static char **insert_opts(char **old_argv, int *old_argc, int index,
+			  char **argv, int argc)
+{
+    char **new_argv;
+
+    /* Allocate larger array */
+    new_argv = realloc(old_argv, (*old_argc + argc) * sizeof(old_argv[0]));
+    if (new_argv == NULL) {
+	fprintf(stderr, "allocate failed in insert_opts\n");
+	exit(1);
+    }
+
+    /* move elements after insertion point to end of array */
+    memmove(new_argv+index + argc, new_argv + index,
+	    (*old_argc - index) * sizeof(argv[0]));
+
+    /* copy in new elements */
+    memcpy(new_argv + index, argv, argc * sizeof(argv[0]));
+
+    *old_argc += argc;
+
+    if (0) { /* for debugging */
+	int i;
+	printf("argv[] = {");
+	for (i = 0; i < *old_argc; i++) {
+	    if (i)
+		printf(", ");
+	    printf("\"%s\"", new_argv[i]);
+	}
+	printf("}\n");
+    }
+
+    return new_argv;
+}
+
+static char **parse_config_file(const char *file, int *pargc)
+{
+    FILE *f;
+    char buffer[4096];
+    char **argv = NULL;
+    int argc = 0;
+
+    f = fopen(file, "r");
+    if (f == NULL)
+	return NULL;
+
+    while (fgets(buffer, sizeof(buffer), f)) {
+	char *ptr = buffer;
+	char *tok, *key, *val;
+	char *targv[2];
+	int targc = 0;
+
+	/* skip whitespace */
+	while (isspace(*ptr)) ptr++;
+
+	/* skip comments or empty lines */
+	if (*ptr == '#' || *ptr == 0)
+	    continue;
+
+	/* trim new line and carriage return if necessary */
+	tok = strchr(ptr, '\n');
+	if (tok)
+	    *tok = 0;
+	tok = strchr(ptr, '\r');
+	if (tok)
+	    *tok = 0;
+
+	/* check if it has an argument */
+	tok = strchr(ptr, '=');
+	if (tok)
+	    *tok = 0;
+
+	/* add key */
+	if (asprintf(&key, "--%s", ptr) == -1)
+	    return NULL;
+	targv[targc++] = key;
+
+	/* add argument (optionally) */
+	if (tok) {
+	    if (asprintf(&val, "%s", tok + 1) == -1)
+		return NULL;
+	    targv[targc++] = val;
+	}
+
+	/* insert new arguments */
+	argv = insert_opts(argv, &argc, argc, targv, targc);
+	if (argv == NULL)
+	    return NULL;
+    }
+
+    fclose(f);
+
+    *pargc = argc;
+
+    return argv;
+}
+
 #define MAX_NET_CLIENTS 32
 
-int main(int argc, char **argv)
+int main(int orig_argc, char **orig_argv)
 {
 #ifdef CONFIG_GDBSTUB
     int use_gdbstub;
@@ -7700,6 +7800,10 @@ int main(int argc, char **argv)
     int fds[2];
     const char *pid_file = NULL;
     VLANState *vlan;
+    char **argv = NULL;
+    int argc = 0;
+
+    argv = insert_opts(argv, &argc, 0, orig_argv, orig_argc);
 
     LIST_INIT (&vm_change_state_head);
 #ifndef _WIN32
@@ -8297,6 +8401,20 @@ int main(int argc, char **argv)
                     }
                 }
                 break;
+	    case QEMU_OPTION_config: {
+		char **config_argv;
+		int config_argc;
+
+		config_argv = parse_config_file(optarg, &config_argc);
+		if (config_argv == NULL) {
+		    fprintf(stderr, "failed to parse config file `%s'\n", optarg);
+		    exit(1);
+		}
+
+		argv = insert_opts(argv, &argc, optind,
+				   config_argv, config_argc);
+		free(config_argv);
+		} break;
             }
         }
     }

[Qemu-devel] Re: [PATCH] Add support for a configuration file

[Anthony Liguori]

Anthony Liguori wrote:
> I think this is pretty useful as-is.  I think it also gives us a reasonable
> way to move forward that will keep everyone pretty happy.
> 
> Here's a short example:
> 
> qemu-system-x86_64 -hda ~/images/linux.img -snapshot -vnc :2
> 
> Would become `foo.qemu':
> 
>  # Main disk image
>  hda=/home/anthony/images/linux.img
> 
>  # Redirect disk writes to a temporary image
>  snapshot
> 
>  # Make the graphical display available on port 5902
>  vnc=:2
> 
> With:
> 
> qemu-system-x86_64 -config foo.qemu

One thought I had, is that it would be very nice to break up the -drive 
file=foo.img,if=scsi syntax within the config file.  In general, I'm 
thinking something like:

[drive]
file=foo.img
if=scsi

or:

drive {
   file=foo.img
   if=scsi
}

or even:

drive:
   file=foo.img
   if=scsi

Basically, I'm looking for a syntax for sub-options.  This would be 
useful for -drive or -net but I also think would lay the foundations for 
specifying a full machine config.

It would get very unwieldy on the command line to have a large number of 
suboptions but it works reasonably well within a config.

Regards,

Anthony Liguori

[Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Tue, May 13, 2008 at 06:07:08PM -0500, Anthony Liguori wrote:
> Anthony Liguori wrote:
> > I think this is pretty useful as-is.  I think it also gives us a reasonable
> > way to move forward that will keep everyone pretty happy.
> > 
> > Here's a short example:
> > 
> > qemu-system-x86_64 -hda ~/images/linux.img -snapshot -vnc :2
> > 
> > Would become `foo.qemu':
> > 
> >  # Main disk image
> >  hda=/home/anthony/images/linux.img
> > 
> >  # Redirect disk writes to a temporary image
> >  snapshot
> > 
> >  # Make the graphical display available on port 5902
> >  vnc=:2
> > 
> > With:
> > 
> > qemu-system-x86_64 -config foo.qemu
> 
> One thought I had, is that it would be very nice to break up the -drive 
> file=foo.img,if=scsi syntax within the config file.  In general, I'm 
> thinking something like:

Yes, that would be the main concern I have with the plain conversion
of existing command line args. It would essentially be limiting the 
expressiveness of the config file to that of the command line - flat
key,value pairs. All we'd be gaining  is avoidance of command line 
length limits and persistent storage. Two worthy goals, but IMHO it
could be worth striving for more structure, so the config can explicitly
represent arrays and hashes as concepts.

> [drive]
> file=foo.img
> if=scsi

This just feels like a bad 1/2 house compromise. Adds the complexity
of a more structured config file without giving the full benefits of
a more expressive format such as the 2 you show below.

> drive {
>    file=foo.img
>    if=scsi
> }

I like both this & the next format because they're very expressive.

> or even:
> 
> drive:
>    file=foo.img
>    if=scsi

That's very nearly YAML format[1], which is attractive because parsers
are available in every major programming language, and it is still
pretty human friendly.

So my preference would be to go with the last option and make sure
it really is YAML compliant so people can use standard tools for 
generating and parsing the format.

Regards,
Daniel

[1]  http://yaml.org/spec/1.2/
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

[Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Colin Adams]

>  That's very nearly YAML format[1], which is attractive because parsers
>  are available in every major programming language,

Really?

I can't find one for Eiffel. Can you give me a pointer please?

Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Avi Kivity]

Daniel P. Berrange wrote:
> That's very nearly YAML format[1], which is attractive because parsers
> are available in every major programming language, and it is still
> pretty human friendly.
>
> So my preference would be to go with the last option and make sure
> it really is YAML compliant so people can use standard tools for 
> generating and parsing the format.
>   

Using a standard format has the added benefit that things like quoting 
are taken care of.

Filenames with leading and trailing spaces, anyone?  Embedded control 
characters?

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Dor Laor]

On Wed, 2008-05-14 at 17:41 +0300, Avi Kivity wrote:
> Daniel P. Berrange wrote:
> > That's very nearly YAML format[1], which is attractive because parsers
> > are available in every major programming language, and it is still
> > pretty human friendly.
> >
> > So my preference would be to go with the last option and make sure
> > it really is YAML compliant so people can use standard tools for 
> > generating and parsing the format.
> >   
> 
> Using a standard format has the added benefit that things like quoting 
> are taken care of.
> 
> Filenames with leading and trailing spaces, anyone?  Embedded control 
> characters?
> 

Please don't jump over me but I think it is worth mentioning OVF, at
least for to know what's you opinions.

Open Virtualization Format -
http://www.vmware.com/appliances/learn/ovf.html

It's xml based, supported by all major hypervisors, so qemu/kvm/xen
users might eventually use a product that support OVF.
Since its a new format it is open for changes and has lots of
flexibility. As a start we don't have to be completely compatible with
it.
It supports definition cpus, startup options, various devices (nic, ide,
scsi,...). For example:
"
<Item>
  <rasd:Caption>Ethernet adapter on "VM Network"</rasd:Caption>
  <rasd:InstanceId>4000</rasd:InstanceId>
  <rasd:ResourceType>10</rasd:ResourceType>
  <rasd:ResourceSubType>VmxNet, E1000</rasd:ResourceSubType>
  <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
  <rasd:Connection>VM Network</rasd:Connection>
</Item>
<Item>
  <rasd:Caption>SCSI Controller 0</rasd:Caption>
  <rasd:InstanceId>1000</rasd:InstanceId>
  <rasd:ResourceType>6</rasd:ResourceType>
  <rasd:ResourceSubType>LsiLogic, BusLogic</rasd:ResourceSubType>
</Item>
<Item>
  <rasd:Caption>Harddisk 1</rasd:Caption>
  <rasd:InstanceId>22001</rasd:InstanceId>
  <rasd:ResourceType>17</rasd:ResourceType>
  <rasd:HostResource>disk/vmdisk1</rasd:HostResource>
  <rasd:Parent>1000</rasd:Parent>
</Item>
<Item>
</Item>
"

One can claim to xml is bad and ovf is outside of the scope and if one
wants ovf, mgmt tool can wrap it around qemu. Nevertheless why doubling
the effort? Qemu can reuse it and its mgmt tools.

Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Wed, May 14, 2008 at 05:52:56PM +0300, Dor Laor wrote:
> 
> On Wed, 2008-05-14 at 17:41 +0300, Avi Kivity wrote:
> > Daniel P. Berrange wrote:
> > > That's very nearly YAML format[1], which is attractive because parsers
> > > are available in every major programming language, and it is still
> > > pretty human friendly.
> > >
> > > So my preference would be to go with the last option and make sure
> > > it really is YAML compliant so people can use standard tools for 
> > > generating and parsing the format.
> > >   
> > 
> > Using a standard format has the added benefit that things like quoting 
> > are taken care of.
> > 
> > Filenames with leading and trailing spaces, anyone?  Embedded control 
> > characters?
> > 
> 
> Please don't jump over me but I think it is worth mentioning OVF, at
> least for to know what's you opinions.

OVF is insanely overcomplicated. It is also addressing a different problem
space, that of virtual machine applinance interchange / distribution. And 
it is a disgusting format for users to deal with. 

> Open Virtualization Format -
> http://www.vmware.com/appliances/learn/ovf.html
> 
> It's xml based, supported by all major hypervisors, so qemu/kvm/xen
> users might eventually use a product that support OVF.
> Since its a new format it is open for changes and has lots of
> flexibility. As a start we don't have to be completely compatible with
> it.

It is a 'open' format defined in secret invitation only cabal by
a bunch of proprietry software vendors. No thanks.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Anthony Liguori]

Dor Laor wrote:
> On Wed, 2008-05-14 at 17:41 +0300, Avi Kivity wrote:
>   
> Please don't jump over me but I think it is worth mentioning OVF, at
> least for to know what's you opinions.
>
> Open Virtualization Format -
> http://www.vmware.com/appliances/learn/ovf.html
>
> It's xml based, supported by all major hypervisors, so qemu/kvm/xen
> users might eventually use a product that support OVF.
>   

xml is a non-starter for QEMU.  We go out of our way to be portable.  
Having an XML dependency that could be satisfied on Windows and Linux 
would probably require more code to support the dependency than all of 
QEMU itself.


Regards,

Anthony Liguori

Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Anthony Liguori]

Avi Kivity wrote:
> Daniel P. Berrange wrote:
>> That's very nearly YAML format[1], which is attractive because parsers
>> are available in every major programming language, and it is still
>> pretty human friendly.
>>
>> So my preference would be to go with the last option and make sure
>> it really is YAML compliant so people can use standard tools for 
>> generating and parsing the format.
>>   
>
> Using a standard format has the added benefit that things like quoting 
> are taken care of.
>
> Filenames with leading and trailing spaces, anyone?  Embedded control 
> characters?

YAML is a bad choice though.  It's purpose is to model data structures 
of embedded languages (similar to JSON).  The syntax would get out of 
hand quickly because what we've been talking about so far would be 
modeled as an association whereas semantically, we want it to be a 
sequence.  To make it a sequence, we would have to prefix every line 
with '-'.

I'm not against following some sort of standard (or even best 
practice).  I just don't like YAML.

Regards,

Anthony Liguori

Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Andreas Färber]

Am 14.05.2008 um 16:41 schrieb Avi Kivity:

> Daniel P. Berrange wrote:
>> That's very nearly YAML format[1], which is attractive because  
>> parsers
>> are available in every major programming language, and it is still
>> pretty human friendly.
>>
>> So my preference would be to go with the last option and make sure
>> it really is YAML compliant so people can use standard tools for  
>> generating and parsing the format.
>>
>
> Using a standard format has the added benefit that things like  
> quoting are taken care of.
>
> Filenames with leading and trailing spaces, anyone?  Embedded  
> control characters?

XML parsers are certainly much more widespread than YAML parsers,  
especially if this format it intended for use by frontends. XML allows  
to define entities as a way to avoid copy-and-paste, and frontends  
could easily extend the file with their own configuration data.

Proposing your own QEMU config file format might make sense for  
defining pins and complex peripheral setups but not for the usual  
options such as disks and RAM. On the other hand, passing a config  
file to the qemu app already requires the user to know which one to  
invoke it with, so its not suited for 'dumb' users anyway. ;)

Andreas

Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a configuration file

[Ian Jackson]

Andreas Färber writes ("Re: [Qemu-devel] Re: [kvm-devel] [PATCH] Add support for a	configuration file"):
> XML

Absolutely not!

I don't care if an invented little language or YAML or raw shell
script fragments or an embedded Tcl interpreter or Scheme.  Just so
long as it's not XML.

XML for configuration files is an abomination.  XML is utterly awful
to edit by hand, doesn't diff well, is vastly overcomplex, encourages
overcomplex configuration structures, requires a huge amount of
parsing infrastructure, and is just plain ugly.  Also, while it
doesn't matter here, it's very slow to parse too.

Ian.

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Fabrice Bellard]

Anthony Liguori wrote:
> One thought I had, is that it would be very nice to break up the -drive 
> file=foo.img,if=scsi syntax within the config file.  In general, I'm 
> thinking something like:
> 
> [drive]
> file=foo.img
> if=scsi
> 
> or:
> 
> drive {
>   file=foo.img
>   if=scsi
> }
> 
> or even:
> 
> drive:
>   file=foo.img
>   if=scsi
> 
> Basically, I'm looking for a syntax for sub-options.  This would be 
> useful for -drive or -net but I also think would lay the foundations for 
> specifying a full machine config.
> 
> It would get very unwieldy on the command line to have a large number of 
> suboptions but it works reasonably well within a config.

I prefer:

drive.file=foo.img
drive.if=scsi

Regards,

Fabrice.

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Avi Kivity]

Fabrice Bellard wrote:
>
> I prefer:
>
> drive.file=foo.img
> drive.if=scsi
>

That doesn't support multiple drives very well.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Fabrice Bellard]

Avi Kivity wrote:
> Fabrice Bellard wrote:
>>
>> I prefer:
>>
>> drive.file=foo.img
>> drive.if=scsi
>>
> 
> That doesn't support multiple drives very well.

Right, I realized it afterwards !

I suggested it because my original plan for the configuration file was 
based on this syntax with a strong inspiration from the OpenFirmware 
device tree. The idea was that the object name ("drive" here) had no 
hardcoded meaning, except for some predefined object names in order to 
keep a kind of backward compatibility with the current QEMU options. In 
order to create a new drive for example, you just have to do:

mydrive.class=drive
mydrive.if=scsi
mydrive.file=abc.img

the "class" field is used to select the device model. Then all the other 
parameters are used to initialize the device model. That way it is 
possible to keep the compatibility with the existing options and add a 
provision to instanciate arbitrary new device models, such as:

mynetworkcard.class="ne2000pci"
mynetworkcard.bus=1 # pci bus selection
mynetworkcard.macaddr=00:01:02:03:04:05
mynetworkcard.vlan=1

I will strongly support configuration file formats having this property.

Regards,

Fabrice.

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Wed, May 14, 2008 at 02:26:40PM +0200, Fabrice Bellard wrote:
> Avi Kivity wrote:
> > Fabrice Bellard wrote:
> >>
> >> I prefer:
> >>
> >> drive.file=foo.img
> >> drive.if=scsi
> >>
> > 
> > That doesn't support multiple drives very well.
> 
> Right, I realized it afterwards !
> 
> I suggested it because my original plan for the configuration file was 
> based on this syntax with a strong inspiration from the OpenFirmware 
> device tree. The idea was that the object name ("drive" here) had no 
> hardcoded meaning, except for some predefined object names in order to 
> keep a kind of backward compatibility with the current QEMU options. In 
> order to create a new drive for example, you just have to do:
> 
> mydrive.class=drive
> mydrive.if=scsi
> mydrive.file=abc.img

With this kind of syntax, now tools generating config files need to make
up unique names for each drive. So you'll probably end up with them just
naming things based on the class name + a number appended.

   drive0.class=drive
   drive0.if=scsi
   drive0.file=foo.img

   drive1.class=drive
   drive1.if=scsi
   drive1.file=bar.img

   drive2.class=drive
   ...

Which suggests it'd be better to take your original previous syntax example
and using an explicit numeric component to represent lists of drives, eg

   drive.0.file=foo.img
   drive.0.if=scsi

   drive.1.file=bar.img
   drive.1.if=scsi

   drive.2.file=wiz.img
   drive.2.if=scsi

Thus avoiding the need for adding the 'class' setting at all

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Avi Kivity]

Daniel P. Berrange wrote:
> With this kind of syntax, now tools generating config files need to make
> up unique names for each drive. So you'll probably end up with them just
> naming things based on the class name + a number appended.
>   

I would hope that tools don't have to resort to reading and writing 
these config files.  Usually a management system would prefer storing 
parameters in its own database, and writing a temporary config file just 
to pass the data seems awkward.  I would much prefer to see the command 
line and monitor retain full control over every configurable parameter.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Thu, May 15, 2008 at 11:04:47AM +0300, Avi Kivity wrote:
> Daniel P. Berrange wrote:
> >With this kind of syntax, now tools generating config files need to make
> >up unique names for each drive. So you'll probably end up with them just
> >naming things based on the class name + a number appended.
> >  
> 
> I would hope that tools don't have to resort to reading and writing 
> these config files.  Usually a management system would prefer storing 
> parameters in its own database, and writing a temporary config file just 
> to pass the data seems awkward.  I would much prefer to see the command 
> line and monitor retain full control over every configurable parameter.

I expect that libvirt will create config files - it is only a matter of
time before we hit the command line ARGV length limits - particularly
with the -net and -drive syntax. People already requesting that we support
guests with > 16 disks, and > 8 network cards so command lines get very
long. 

I wouldn't write out the config file to disk though - I'd just send it
on the fly on stdin -, eg   'qemu -config -'  to tell it to read the config
on its stdin.

Regards,
Daniel.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Avi Kivity]

Daniel P. Berrange wrote:
> On Thu, May 15, 2008 at 11:04:47AM +0300, Avi Kivity wrote:
>   
>> Daniel P. Berrange wrote:
>>     
>>> With this kind of syntax, now tools generating config files need to make
>>> up unique names for each drive. So you'll probably end up with them just
>>> naming things based on the class name + a number appended.
>>>  
>>>       
>> I would hope that tools don't have to resort to reading and writing 
>> these config files.  Usually a management system would prefer storing 
>> parameters in its own database, and writing a temporary config file just 
>> to pass the data seems awkward.  I would much prefer to see the command 
>> line and monitor retain full control over every configurable parameter.
>>     
>
> I expect that libvirt will create config files - it is only a matter of
> time before we hit the command line ARGV length limits - particularly
> with the -net and -drive syntax. People already requesting that we support
> guests with > 16 disks, and > 8 network cards so command lines get very
> long. 
>   

What are those limits, btw? ISTR 10240 words, but how many chars?

> I wouldn't write out the config file to disk though - I'd just send it
> on the fly on stdin -, eg   'qemu -config -'  to tell it to read the config
> on its stdin.
>   

That's fine from my point of view.

-- 
error compiling committee.c: too many arguments to function

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Andreas Färber]

Am 15.05.2008 um 14:04 schrieb Avi Kivity:

> Daniel P. Berrange wrote:
>> it is only a matter of
>> time before we hit the command line ARGV length limits - particularly
>> with the -net and -drive syntax.
>
> What are those limits, btw? ISTR 10240 words, but how many chars?

It varies between platforms; autoconf usually checks it and prints its  
findings.

Andreas

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Laurent Vivier]

Le jeudi 15 mai 2008 à 15:04 +0300, Avi Kivity a écrit :
> Daniel P. Berrange wrote:
> > On Thu, May 15, 2008 at 11:04:47AM +0300, Avi Kivity wrote:
> >   
> >> Daniel P. Berrange wrote:
> >>     
> >>> With this kind of syntax, now tools generating config files need to make
> >>> up unique names for each drive. So you'll probably end up with them just
> >>> naming things based on the class name + a number appended.
> >>>  
> >>>       
> >> I would hope that tools don't have to resort to reading and writing 
> >> these config files.  Usually a management system would prefer storing 
> >> parameters in its own database, and writing a temporary config file just 
> >> to pass the data seems awkward.  I would much prefer to see the command 
> >> line and monitor retain full control over every configurable parameter.
> >>     
> >
> > I expect that libvirt will create config files - it is only a matter of
> > time before we hit the command line ARGV length limits - particularly
> > with the -net and -drive syntax. People already requesting that we support
> > guests with > 16 disks, and > 8 network cards so command lines get very
> > long. 
> >   
> 
> What are those limits, btw? ISTR 10240 words, but how many chars?

ARG_MAX - _SC_ARG_MAX
        The  maximum  length  of  the arguments to the exec(3) family of
        functions.  Must not be less than _POSIX_ARG_MAX (4096).

getconf ARG_MAX
131072

And from a configure log I have:

checking the maximum length of command line arguments: 98304

Regards,
Laurent
-- 
------------- Laurent.Vivier@bull.net ---------------
"The best way to predict the future is to invent it."
- Alan Kay

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Anthony Liguori]

Fabrice Bellard wrote:
> Avi Kivity wrote:
>> Fabrice Bellard wrote:
>>>
>>> I prefer:
>>>
>>> drive.file=foo.img
>>> drive.if=scsi
>>>
>>
>> That doesn't support multiple drives very well.
>
> Right, I realized it afterwards !
>
> I suggested it because my original plan for the configuration file was 
> based on this syntax with a strong inspiration from the OpenFirmware 
> device tree. The idea was that the object name ("drive" here) had no 
> hardcoded meaning, except for some predefined object names in order to 
> keep a kind of backward compatibility with the current QEMU options. 
> In order to create a new drive for example, you just have to do:
>
> mydrive.class=drive
> mydrive.if=scsi
> mydrive.file=abc.img
>
> the "class" field is used to select the device model. Then all the 
> other parameters are used to initialize the device model. That way it 
> is possible to keep the compatibility with the existing options and 
> add a provision to instanciate arbitrary new device models, such as:

I like this syntax primarily because it provides a means to associate 
arbitrary data with a VM.  It also provides a sane way to keep track of 
which device is which so that the "config" can be updated while the VM 
is running.  I'll update the patch.

Regards,

Anthony Liguori

> mynetworkcard.class="ne2000pci"
> mynetworkcard.bus=1 # pci bus selection
> mynetworkcard.macaddr=00:01:02:03:04:05
> mynetworkcard.vlan=1
>
> I will strongly support configuration file formats having this property.
>
> Regards,
>
> Fabrice.
>

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Paul Brook]

> I suggested it because my original plan for the configuration file was
> based on this syntax with a strong inspiration from the OpenFirmware
> device tree. The idea was that the object name ("drive" here) had no
> hardcoded meaning, except for some predefined object names in order to
> keep a kind of backward compatibility with the current QEMU options. In
> order to create a new drive for example, you just have to do:
>
> mydrive.class=drive
> mydrive.if=scsi
> mydrive.file=abc.img
>
> the "class" field is used to select the device model. Then all the other
> parameters are used to initialize the device model. That way it is
> possible to keep the compatibility with the existing options and add a
> provision to instanciate arbitrary new device models, such as:

I like the idea, but I'm not so keen on the automatic allocation. I generally 
prefer explicit declaration over implicit things. The latter makes it very 
easy to not notice when you make a typo.

It sounds like what you really want is something similar to an OF device tree.  
So you have something like:

# pciide0 may be an alias (possibly provided by qemu)
# e.g. pci0.slot1.func1.ide
alias hda ide0.primary.master

hda.type=disk
hda.file=foo.img

You can then define some form of magic aliases that select the next unused 
device. e.g.

alias mydrive $next_ide_disk

IMHO This provides the flexibility and structure that Fabrice is talking 
about, and with suitable aliases can be made to look a lot like the existing 
options.

This may require some internal restructuring to allow the machine descriptions 
to feed into the user config file.

Thoughts?

Paul

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Javier Guerra]

What about Lua?  (http://www.lua.org)

it started up as a configuration language, and evolved into a full
programming language, while remaining _very_ light (less than 200K
with all libraries), and wonderfully easy to embed into C programs.

it lets you write things like:

drives = {
  hda = {if='scsi', file='hda.img'},
  hdb = {if='ide', file='hdb.img'},
}
mem = 512*MB

or, equivalently:

drives={}
drives.hda={}
drives.hda.if='scsi'
drives.hda.file='hda.img'
drives.hdb={}
drives.hdb.if='ide'
drives.hdb.file='hdb.img'
mem=512*MB

or, if you want:

drive {if='scsi', file='hda.img'}
drive {if='ide', file='hdb.img'}
mem(512*MB)

or even:

for img in lfs.dir("*.img") do
   drive{if='scsi', file=img}
end



-- 
Javier

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Johannes Schindelin]

Hi,

On Wed, 14 May 2008, Javier Guerra wrote:

> What about Lua?  (http://www.lua.org)
> 
> it started up as a configuration language, and evolved into a full
> programming language, while remaining _very_ light (less than 200K
> with all libraries), and wonderfully easy to embed into C programs.

Okay, so much for the upsides.  Now for the downsides: a language that 
nearly nobody knows, for something that is not meant to be executed (think 
security implications).

Hth,
Dscho

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Javier Guerra]

On Wed, May 14, 2008 at 10:13 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>  On Wed, 14 May 2008, Javier Guerra wrote:
>  > What about Lua?  (http://www.lua.org)
>  >
>  > it started up as a configuration language, and evolved into a full
>  > programming language, while remaining _very_ light (less than 200K
>  > with all libraries), and wonderfully easy to embed into C programs.
>
>  Okay, so much for the upsides.  Now for the downsides: a language that
>  nearly nobody knows, for something that is not meant to be executed (think
>  security implications).

when embedded, you get to choose what libraries are available. there
are several examples of fairly secure settings.

personally, i find shell scripts enough for setting up parameters.  a
static config wouldn't bring much advantages.

-- 
Javier

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Johannes Schindelin]

Hi,

On Wed, 14 May 2008, Javier Guerra wrote:

> On Wed, May 14, 2008 at 10:13 AM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >  On Wed, 14 May 2008, Javier Guerra wrote:
> >  > What about Lua?  (http://www.lua.org)
> >  >
> >  > it started up as a configuration language, and evolved into a full
> >  > programming language, while remaining _very_ light (less than 200K
> >  > with all libraries), and wonderfully easy to embed into C programs.
> >
> >  Okay, so much for the upsides.  Now for the downsides: a language that
> >  nearly nobody knows, for something that is not meant to be executed (think
> >  security implications).
> 
> when embedded, you get to choose what libraries are available. there
> are several examples of fairly secure settings.

Why artificially make it complicated, and then have to take care of such 
issues?

That's like an ex-colleague of mine, who absolutely had to rewrite the 
database engine in-RAM, and when the application was too slow over modem 
(leeching in megabytes, where it got bytes from the SQL database before), 
he tried to force Windows Terminal Services, instead of reverting his 
change.

Simplicity is underrated.

Ciao,
Dscho

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Javier Guerra]

On Wed, May 14, 2008 at 10:37 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>  On Wed, 14 May 2008, Javier Guerra wrote:
>  > when embedded, you get to choose what libraries are available. there
>  > are several examples of fairly secure settings.
>
>  Why artificially make it complicated, and then have to take care of such
>  issues?

i should have said "Lua is secure by default, all libraries are
optional", but it's quickly turning into a bikeshed
(http://www.bikeshed.com)

my own personal preference would be to just stuff the whole command
line parameters into the qcow2 image file.

-- 
Javier

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Kelly French]

On Wed, 14 May 2008, Johannes Schindelin wrote:

> Hi,
>
> On Wed, 14 May 2008, Javier Guerra wrote:
>
>> On Wed, May 14, 2008 at 10:13 AM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>>>  On Wed, 14 May 2008, Javier Guerra wrote:
>>> > What about Lua?  (http://www.lua.org)
>>> >
>>> > it started up as a configuration language, and evolved into a full
>>> > programming language, while remaining _very_ light (less than 200K
>>> > with all libraries), and wonderfully easy to embed into C programs.
>>>
>>>  Okay, so much for the upsides.  Now for the downsides: a language that
>>>  nearly nobody knows, for something that is not meant to be executed (think
>>>  security implications).
>>
>> when embedded, you get to choose what libraries are available. there
>> are several examples of fairly secure settings.
>
> Why artificially make it complicated, and then have to take care of such
> issues?
>
> That's like an ex-colleague of mine, who absolutely had to rewrite the
> database engine in-RAM, and when the application was too slow over modem
> (leeching in megabytes, where it got bytes from the SQL database before),
> he tried to force Windows Terminal Services, instead of reverting his
> change.
>
> Simplicity is underrated.
>
> Ciao,
> Dscho

Why not just bypass the whole config file idea and just use enviornment 
variables?  No more parsing or dependencies on the language of the day. 
Yes, you wouldn't have the tree format that some people are asking for. 
You'd get all of the power of your favorite shell, plus maybe some 
benefits when migrating a VM to another machine.

 	-kf

#!/bin/sh

drive_0_file=foo.img
drive_0_if=scsi

drive_1_file=bar.img
drive_1_if=scsi

drive_2_file=wiz.img
drive_2_if=scsi

exec qemu

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Paul Brook]

> Why not just bypass the whole config file idea and just use enviornment
> variables?  

Absolutely not. Environment variables are a horrid way of configuring things.

Paul

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Andreas Färber]

Am 14.05.2008 um 17:30 schrieb Javier Guerra:

> personally, i find shell scripts enough for setting up parameters.  a
> static config wouldn't bring much advantages.

For personal use they are okay, but for distributing a machine, you  
may need two or three platform-specific scripts for QEMU, or even  
worse one config file per frontend.

VMware has the advantage of allowing to distribute one .vmx file along  
with the image to facilitate its use. QEMU users can be happy if a raw  
or qcow2 image is provided at all and then need to figure out useful  
command line parameters (and whether to use qemu or qemu-system-x86_64  
or whichever).

Andreas

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Wed, May 14, 2008 at 06:01:43PM +0200, Andreas F?rber wrote:
> 
> Am 14.05.2008 um 17:30 schrieb Javier Guerra:
> 
> >personally, i find shell scripts enough for setting up parameters.  a
> >static config wouldn't bring much advantages.
> 
> For personal use they are okay, but for distributing a machine, you  
> may need two or three platform-specific scripts for QEMU, or even  
> worse one config file per frontend.
> 
> VMware has the advantage of allowing to distribute one .vmx file along  
> with the image to facilitate its use. QEMU users can be happy if a raw  
> or qcow2 image is provided at all and then need to figure out useful  
> command line parameters (and whether to use qemu or qemu-system-x86_64  
> or whichever).

Re-distribution of virtual machine images is a problem that's tangential
to configuration file. 

There is alot of data you want in the configuration file for a specific
deployment of a VM, that is simply not relevant or desired when distributing
appliances. As such you don't typically want to simply distribute the config
file. Furthermore, the format for distributing appliances can & shouuld be
independant of the virtualization technology used for a specific deployment.

So an appliance format would specify *requirements*, eg a virtualization
product capable of fully virtualizing a x86_64 CPU.  The tool to deploy
an instace of the appliance examines the requirements to decide hwo to
satisfy them, eg it might decide qemu-system-x86_64 will do, or it might
decide to use  'kvm' instead. 

You simply do not want to tie the deployment configuration to the metadata
for redistribution. It happened to more or less work for VMWare with VMDK
and VMX file, only because VMWare were the only game in town with no real
widely deployed alternatives. That time has passed.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Andreas Färber]

Am 14.05.2008 um 18:21 schrieb Daniel P. Berrange:

> On Wed, May 14, 2008 at 06:01:43PM +0200, Andreas F?rber wrote:
>>
>> Am 14.05.2008 um 17:30 schrieb Javier Guerra:
>>
>>> personally, i find shell scripts enough for setting up  
>>> parameters.  a
>>> static config wouldn't bring much advantages.
>>
>> For personal use they are okay, but for distributing a machine, you
>> may need two or three platform-specific scripts for QEMU, or even
>> worse one config file per frontend.
>>
>> VMware has the advantage of allowing to distribute one .vmx file  
>> along
>> with the image to facilitate its use. QEMU users can be happy if a  
>> raw
>> or qcow2 image is provided at all and then need to figure out useful
>> command line parameters (and whether to use qemu or qemu-system- 
>> x86_64
>> or whichever).
>
> Re-distribution of virtual machine images is a problem that's  
> tangential
> to configuration file.
>
> There is alot of data you want in the configuration file for a  
> specific
> deployment of a VM, that is simply not relevant or desired when  
> distributing
> appliances. As such you don't typically want to simply distribute  
> the config
> file. Furthermore, the format for distributing appliances can &  
> shouuld be
> independant of the virtualization technology used for a specific  
> deployment.
>
> So an appliance format would specify *requirements*, eg a  
> virtualization
> product capable of fully virtualizing a x86_64 CPU.  The tool to  
> deploy
> an instace of the appliance examines the requirements to decide hwo to
> satisfy them, eg it might decide qemu-system-x86_64 will do, or it  
> might
> decide to use  'kvm' instead.
>
> You simply do not want to tie the deployment configuration to the  
> metadata
> for redistribution. It happened to more or less work for VMWare with  
> VMDK
> and VMX file, only because VMWare were the only game in town with no  
> real
> widely deployed alternatives. That time has passed.

Yeah, and the ongoing discussion of a format to connect disk images to  
QEMU is in no way specifying such cross-tool requirements...

If QEMU wants to stay out of that game, then specify machine-level  
things in this config file format of yours but not the user-level  
stuff. Otherwise the format you are choosing for portability reasons  
is going to make the resulting config files unportable.

Andreas

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Fabrice Bellard]

Johannes Schindelin wrote:
> Hi,
> 
> On Wed, 14 May 2008, Javier Guerra wrote:
> 
>> What about Lua?  (http://www.lua.org)
>>
>> it started up as a configuration language, and evolved into a full
>> programming language, while remaining _very_ light (less than 200K
>> with all libraries), and wonderfully easy to embed into C programs.
> 
> Okay, so much for the upsides.  Now for the downsides: a language that 
> nearly nobody knows, for something that is not meant to be executed (think 
> security implications).

What is sure is that I don't wish to use a language as configuration 
file. The configuration file must be seen as a serialization of 
variables. It is simple to understand, can be processed automatically in 
case of a GUI and its format can be easily changed.

Regards,

Fabrice.

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Luca Barbato]

Fabrice Bellard wrote:
> What is sure is that I don't wish to use a language as configuration 
> file. The configuration file must be seen as a serialization of 
> variables. It is simple to understand, can be processed automatically in 
> case of a GUI and its format can be easily changed.

lighttpd current configuration looks like what you want even if it could 
be overkill already (and they are thinking about using lua since gives 
even more rope)

lu

-- 

Luca Barbato
Gentoo Council Member
Gentoo/linux Gentoo/PPC
http://dev.gentoo.org/~lu_zero

Re: [kvm-devel] [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Wed, May 14, 2008 at 09:45:02AM -0500, Javier Guerra wrote:
> What about Lua?  (http://www.lua.org)
> 
> it started up as a configuration language, and evolved into a full
> programming language, while remaining _very_ light (less than 200K
> with all libraries), and wonderfully easy to embed into C programs.

Config files are data, not programs. Xen made this mistake originally
too just having python config files that were eval'd, but thankfully
they've defined a sensible data format now.

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Anthony Liguori]

Paul Brook wrote:
>> the "class" field is used to select the device model. Then all the other
>> parameters are used to initialize the device model. That way it is
>> possible to keep the compatibility with the existing options and add a
>> provision to instanciate arbitrary new device models, such as:
>>     
>
> I like the idea, but I'm not so keen on the automatic allocation. I generally 
> prefer explicit declaration over implicit things. The latter makes it very 
> easy to not notice when you make a typo.
>
> It sounds like what you really want is something similar to an OF device tree.  
> So you have something like:
>
> # pciide0 may be an alias (possibly provided by qemu)
> # e.g. pci0.slot1.func1.ide
> alias hda ide0.primary.master
>   

What I don't like about the ide0.primary.master syntax is that there 
isn't enough structure.  I would prefer:

alias hda ide,bus=0,primary,master

If you combine this with your magic variable idea, you could also do:

alias hda ide,bus=0,unit=$next

But you could also just fold that into Fabrice's syntax (which I prefer):

hda.class = ide,bus=0,unit=$next
hda.type = disk
hda.file = foo.img

If you then default bus, and unit, you can just write:

hda.class = ide
hda.type = disk
hda.file = foo.img

And better yet, you could even allow for something like:

hda.class = ide
hda.bus = 0
hda.unit = 0
hda.type = disk
hda.file = foo.img

So I really actually prefer Fabrice's syntax because there is much more 
structure.  I think it's a good idea to allow .class to contain multiple 
properties and to basically establish an alias.  This way, you could 
predefine a bunch of aliases for today's parameters like hda, hdb, etc.

> hda.type=disk
> hda.file=foo.img
>
> You can then define some form of magic aliases that select the next unused 
> device. e.g.
>
> alias mydrive $next_ide_disk
>
> IMHO This provides the flexibility and structure that Fabrice is talking 
> about, and with suitable aliases can be made to look a lot like the existing 
> options.
>
> This may require some internal restructuring to allow the machine descriptions 
> to feed into the user config file.
>   

I think if we choose a syntax we like, we can start using that pretty 
easily.  We'll have to start adjusting option names keeping them only 
valid on the command line (for things like -m).  However, I think it 
would grow pretty well into a machine description mechanism.

Regards,

Anthony Liguori

> Thoughts?
>
> Paul
>   

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Paul Brook]

On Wednesday 14 May 2008, Anthony Liguori wrote:
> Paul Brook wrote:
> >> the "class" field is used to select the device model. Then all the other
> >> parameters are used to initialize the device model. That way it is
> >> possible to keep the compatibility with the existing options and add a
> >> provision to instanciate arbitrary new device models, such as:
> >
> > I like the idea, but I'm not so keen on the automatic allocation. I
> > generally prefer explicit declaration over implicit things. The latter
> > makes it very easy to not notice when you make a typo.
> >
> > It sounds like what you really want is something similar to an OF device
> > tree. So you have something like:
> >
> > # pciide0 may be an alias (possibly provided by qemu)
> > # e.g. pci0.slot1.func1.ide
> > alias hda ide0.primary.master
>
> What I don't like about the ide0.primary.master syntax is that there
> isn't enough structure.  I would prefer:
>
> alias hda ide,bus=0,primary,master
>
> If you combine this with your magic variable idea, you could also do:
>
> alias hda ide,bus=0,unit=$next
>
> But you could also just fold that into Fabrice's syntax (which I prefer):

What I dislike about this is that it's a flat format, where you identify 
things by setting some combination of attributes. I really like the idea of 
having a tree structure.

Paul

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Fabrice Bellard]

Paul Brook wrote:
>> I suggested it because my original plan for the configuration file was
>> based on this syntax with a strong inspiration from the OpenFirmware
>> device tree. The idea was that the object name ("drive" here) had no
>> hardcoded meaning, except for some predefined object names in order to
>> keep a kind of backward compatibility with the current QEMU options. In
>> order to create a new drive for example, you just have to do:
>>
>> mydrive.class=drive
>> mydrive.if=scsi
>> mydrive.file=abc.img
>>
>> the "class" field is used to select the device model. Then all the other
>> parameters are used to initialize the device model. That way it is
>> possible to keep the compatibility with the existing options and add a
>> provision to instanciate arbitrary new device models, such as:
> 
> I like the idea, but I'm not so keen on the automatic allocation. I generally 
> prefer explicit declaration over implicit things. The latter makes it very 
> easy to not notice when you make a typo.
> 
> It sounds like what you really want is something similar to an OF device tree.  
> So you have something like:
> 
> # pciide0 may be an alias (possibly provided by qemu)
> # e.g. pci0.slot1.func1.ide
> alias hda ide0.primary.master
> 
> hda.type=disk
> hda.file=foo.img
> 
> You can then define some form of magic aliases that select the next unused 
> device. e.g.
> 
> alias mydrive $next_ide_disk
> 
> IMHO This provides the flexibility and structure that Fabrice is talking 
> about, and with suitable aliases can be made to look a lot like the existing 
> options.

Right. It is my intent too to allow aliases to keep the same "familiar" 
names as the command line.

Moreover the tree you suggest is necessary in order to derive the device 
instanciation order. In my idea, the tree has no relation with the 
actual device connections which are specified by explicit fields such as 
slots, functions, interface index, disk indexes or anything else.

An interesting shortcut can be to automatically define a field "index" 
if the device name terminates with a number (if I remember correctly 
OpenFirmware does something like this).

The initialization phase would consist in traversing the tree 
recursively and by instanciating a device for all nodes containing a 
"class" (or "type" if you prefer) field. The parents would be 
instanciated before the children to ensure a coherent initialization order.

Regarding the syntax, quoted strings must be supported of course. I 
don't think there is a great complexity in that :-) A cpp like 
preprocessing can be added, but it can be done later.

> This may require some internal restructuring to allow the machine descriptions 
> to feed into the user config file.

Hopefully it is not necessary to fully implement the proposal now. But 
ultimately, each QEMU device would have to register its class name and 
an instanciation function. The machine descriptions would have to 
predefine some object names so that the user can modify parameters.

Regards,

Fabrice.

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[andrzej zaborowski]

On 14/05/2008, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Anthony Liguori wrote:
>
> > I think this is pretty useful as-is.  I think it also gives us a
> reasonable
> > way to move forward that will keep everyone pretty happy.
> >
> > Here's a short example:
> >
> > qemu-system-x86_64 -hda ~/images/linux.img -snapshot -vnc :2
> >
> > Would become `foo.qemu':
> >
> >  # Main disk image
> >  hda=/home/anthony/images/linux.img
> >
> >  # Redirect disk writes to a temporary image
> >  snapshot
> >
> >  # Make the graphical display available on port 5902
> >  vnc=:2
> >
> > With:
> >
> > qemu-system-x86_64 -config foo.qemu
> >
>
>  One thought I had, is that it would be very nice to break up the -drive
> file=foo.img,if=scsi syntax within the config file.  In general, I'm
> thinking something like:
>
>  [drive]
>  file=foo.img
>  if=scsi
>
>  or:
>
>  drive {
>   file=foo.img
>   if=scsi
>  }

I like this one and it would be nice to be able to replace the equal
sign with whitespace.

What I'd love, though, but expect others will consider bloat, is that
files are passed through cpp before interpreting.  This way the syntax
becomes not (much) less powerful than the current command line
invocation where you get variable expansion for free.  I would hate to
type in some things in as many times as there are instances of a given
hardware.  Consider some disk arrays or the once discussed -pins
switch for initial state of pins or jumpers (some machines have
hundreds of them).
-- 
Please do not print this email unless absolutely necessary. Spread
environmental awareness.

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Johannes Schindelin]

Hi,

On Wed, 14 May 2008, andrzej zaborowski wrote:

> What I'd love, though, but expect others will consider bloat, is that
> files are passed through cpp before interpreting.

This will add a dependency to a developer's tool on an application that 
has not much to do with development for most users.

Ciao,
Dscho

Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file

[Ian Jackson]

andrzej zaborowski writes ("Re: [Qemu-devel] Re: [PATCH] Add support for a configuration file"):
> What I'd love, though, but expect others will consider bloat, is that
> files are passed through cpp before interpreting.

cpp is a terrible preprocessor.  It mostly works for C source code
(although it has some serious deficiences there - for example, that
you can't have macros that generate macros).  But for anything whose
lexical structure is not much like C it causes terrible trouble -
randomly inserting spaces, need for explicit token pasting (dependent
on the C operator set), etc.  It breaks even something as simple as
#-comments, which are obviously what we would want in our plain text
config file.

If you really want a preprocessor, use m4 (one that supports -p and
using a suitable m4_changequote).  m4 isn't particularly pretty but it
doesn't interfere with the core of the syntax.

Better, make the configuration defaults sufficiently good that it is
not necessary to reproduce lots of boilerplate to get things done.
Then macros aren't needed for human-written files - and of course
program-written config files can don't need a macro system anyway.

Ian.

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[Erik de Castro Lopo]

Anthony Liguori wrote:

> There has been an awful lot of discussion about a configuration file with
> almost no general agreement except that one is strongly desired.

One thing I'd like to see is the ability to include one config file
inside another.

The use case for this is to have something like:

    my-machine-base.conf

which contains base configuration stuff. I might then have a couple
of other configs which include the above file:

   my-machine-use-vde-networking.conf
   my-machine-standard-networking.conf

The base config might not even be a valid config by itself, but is
only useful when included with one of these other configs.

Comments?

Cheers,
Erik
-- 
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"... there's something really scary about a language (C++) where
copying state from one object to another is this complicated"
-- Richard Gillam

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[andrzej zaborowski]

On 15/05/2008, Erik de Castro Lopo <mle+tools@mega-nerd.com> wrote:
> One thing I'd like to see is the ability to include one config file
>  inside another.

That's a good idea, I suggested preprocessing with a cpp (-like) tool
with this in mind.  This lets you kindof "make your own" syntax for
the config file, redefine instead of search&replace, and allows a lib
like libvirt to have its standard "base" config / "template" config
that would be included by the generated files and used.
-- 
Please do not print this email unless absolutely necessary. Spread
environmental awareness.

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[Daniel P. Berrange]

On Thu, May 15, 2008 at 02:36:43PM +0200, andrzej zaborowski wrote:
> On 15/05/2008, Erik de Castro Lopo <mle+tools@mega-nerd.com> wrote:
> > One thing I'd like to see is the ability to include one config file
> >  inside another.
> 
> That's a good idea, I suggested preprocessing with a cpp (-like) tool
> with this in mind.  This lets you kindof "make your own" syntax for
> the config file, redefine instead of search&replace, and allows a lib
> like libvirt to have its standard "base" config / "template" config
> that would be included by the generated files and used.

We don't really have a 'base' template for QEMU - we only specify the args
that are explicitly required based on the user's libvirt XML config. Plus
couple of extra ones to turn off default settings (eg -serial none) if the
user didn't ask for a serial device. So I'd probably not need includes
for the QEMU config. Definitely useful for people not using libvirt though

Dan.
-- 
|: Red Hat, Engineering, Boston   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[andrzej zaborowski]

On 15/05/2008, Daniel P. Berrange <berrange@redhat.com> wrote:
> On Thu, May 15, 2008 at 02:36:43PM +0200, andrzej zaborowski wrote:
>  > On 15/05/2008, Erik de Castro Lopo <mle+tools@mega-nerd.com> wrote:
>  > > One thing I'd like to see is the ability to include one config file
>  > >  inside another.
>  >
>  > That's a good idea, I suggested preprocessing with a cpp (-like) tool
>  > with this in mind.  This lets you kindof "make your own" syntax for
>  > the config file, redefine instead of search&replace, and allows a lib
>  > like libvirt to have its standard "base" config / "template" config
>  > that would be included by the generated files and used.
>
> We don't really have a 'base' template for QEMU - we only specify the args

Obviously, because there was no config files until now.  What I mean is that
instead of writing the full "net {\n  user;\n  vlan 0;\n"... in every
config file you generate, you'd write user_net(0);, where the user_net
macro is #defined in the template.  With a powerful enough
preprocessor you could even have qemu read your XML configuratoins
directly, given their DTDs or schemas.  With a powerful enough
processor qemu could read the raw binary representation of a libvirt
internal struct.  (I'm not trying to suggest doing any of these of
course)
-- 
Please do not print this email unless absolutely necessary. Spread
environmental awareness.

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[Anthony Liguori]

Erik de Castro Lopo wrote:
> Anthony Liguori wrote:
>
>   
>> There has been an awful lot of discussion about a configuration file with
>> almost no general agreement except that one is strongly desired.
>>     
>
> One thing I'd like to see is the ability to include one config file
> inside another.
>   

The astute reader will notice that my patch already includes that 
functionality :-)

Regards,

Anthony Liguori

> The use case for this is to have something like:
>
>     my-machine-base.conf
>
> which contains base configuration stuff. I might then have a couple
> of other configs which include the above file:
>
>    my-machine-use-vde-networking.conf
>    my-machine-standard-networking.conf
>
> The base config might not even be a valid config by itself, but is
> only useful when included with one of these other configs.
>
> Comments?
>
> Cheers,
> Erik
>   

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[Avi Kivity]

Anthony Liguori wrote:
>>
>> One thing I'd like to see is the ability to include one config file
>> inside another.
>>   
>
> The astute reader will notice that my patch already includes that 
> functionality :-)
>

Yes!  Neat :)

Maybe you can make the comma a line continuation character:

drive file=blah.img,
         if=scsi,
         format=raw,
         index=3

Though using {} is sufficient IMO.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] [PATCH] Add support for a configuration file

[Anthony Liguori]

Avi Kivity wrote:
> Anthony Liguori wrote:
>>>
>>> One thing I'd like to see is the ability to include one config file
>>> inside another.
>>>   
>>
>> The astute reader will notice that my patch already includes that 
>> functionality :-)
>>
>
> Yes!  Neat :)
>
> Maybe you can make the comma a line continuation character:
>
> drive file=blah.img,
>         if=scsi,
>         format=raw,
>         index=3
>
> Though using {} is sufficient IMO.

Yeah, I was planning on using {}.  The comma thing is interesting though.

Regards,

Anthony Liguori