[Qemu-devel] KVM call agenda for 2013-05-28

[Qemu-devel] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

Juan is not available now, and Anthony asked for
agenda to be sent early.
So here comes:

Agenda for the meeting Tue, May 28:

- Generating acpi tables

- Switching the call to a bi-weekly schedule

Please, send any topic that you are interested in covering.

Thanks, MST

-- 
MST

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[li guang]

在 2013-05-23四的 15:41 +0300，Michael S. Tsirkin写道：
> Juan is not available now, and Anthony asked for
> agenda to be sent early.
> So here comes:
> 
> Agenda for the meeting Tue, May 28:
> 
> - Generating acpi tables
> 
> - Switching the call to a bi-weekly schedule
> 
> Please, send any topic that you are interested in covering.

"add ACPI Embedded Controller"

it can work as an event carrier, so ACPI events 
can be passed between platform(like QEMU emulated)
and OS, OS can execute _Qxx method which defined
by yourself.
this mechanism can be very flexible, and you can
avoid bothering to create some special devices
for QEMU and drivers for linux kernel.
I don't have many examples,
e.g. CPU hotplug,
we can pass hotplug event by _Qxx method, and,
Embedded Controller driver of linux kernel can
get this event, so, it can trigger cpu_up process,
and you'll don't need 
"echo 1 > /sys/devices/system/cpu/cpu1/online"

may you think of it,

Thanks!

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Kevin O'Connor]

On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> Juan is not available now, and Anthony asked for
> agenda to be sent early.
> So here comes:
> 
> Agenda for the meeting Tue, May 28:
> 
> - Generating acpi tables

I didn't see any meeting notes, but I thought it would be worthwhile
to summarize the call.  This is from memory so correct me if I got
anything wrong.

Anthony believes that the generation of ACPI tables is the task of the
firmware.  Reasons cited include security implications of running more
code in qemu vs the guest context, complexities in running iasl on
big-endian machines, possible complexity of having to regenerate
tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
that QOM interface should be sufficient.

Kevin believes that the bios table code should be moved up into QEMU.
Reasons cited include the churn rate in SeaBIOS for this QEMU feature
(15-20% of all SeaBIOS commits since integrating with QEMU have been
for bios tables; 20% of SeaBIOS commits in last year), complexity of
trying to pass all the content needed to generate the tables (eg,
device details, power tree, irq routing), complexity of scheduling
changes across different repos and synchronizing their rollout,
complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
wasn't aware of a requirement to regenerate acpi tables on a vm
reboot.

There were discussions on potentially introducing a middle component
to generate the tables.  Coreboot was raised as a possibility, and
David thought it would be okay to use coreboot for both OVMF and
SeaBIOS.  The possibility was also raised of a "rom" that lives in the
qemu repo, is run in the guest, and generates the tables (which is
similar to the hvmloader approach that Xen uses).

Anthony requested that patches be made that generate the ACPI tables
in QEMU for the upcoming hotplug work, so that they could be evaluated
to see if they truly do need to live in QEMU or if the code could live
in the firmware.  There were no objections.

-Kevin

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> > Juan is not available now, and Anthony asked for
> > agenda to be sent early.
> > So here comes:
> > 
> > Agenda for the meeting Tue, May 28:
> > 
> > - Generating acpi tables
> 
> I didn't see any meeting notes, but I thought it would be worthwhile
> to summarize the call.  This is from memory so correct me if I got
> anything wrong.
> 
> Anthony believes that the generation of ACPI tables is the task of the
> firmware.  Reasons cited include security implications of running more
> code in qemu vs the guest context, complexities in running iasl on
> big-endian machines, possible complexity of having to regenerate
> tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> that QOM interface should be sufficient.
> 
> Kevin believes that the bios table code should be moved up into QEMU.
> Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> (15-20% of all SeaBIOS commits since integrating with QEMU have been
> for bios tables; 20% of SeaBIOS commits in last year), complexity of
> trying to pass all the content needed to generate the tables (eg,
> device details, power tree, irq routing), complexity of scheduling
> changes across different repos and synchronizing their rollout,
> complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> wasn't aware of a requirement to regenerate acpi tables on a vm
> reboot.

I think this last one is based on a misunderstanding: it's based
on assumption that we we change hardware by hotplug
we should regenerate the tables to match.
But there's no management that can take advantage of
this.
Two possible reasonable things we can tell management:
- hotplug for device XXX is not supported: restart qemu
  to make guest use the device
- hotplug for device XXX is supported

What is proposed here instead is a third option:
- hotplug is supported but device is not functional.
  reboot guest to make it fully functional

This will naturally lead to requirement to regenerate tables on reset.

And this is what would happen with guest-generated
tables, and I consider this a bug, not a feature.

If you really wanted to update tables dynamically, without restarting
qemu, don't stop there, add an interface for guest to update them
without reset. I think that's over-endineering and a
requirement that's best avoided.


> There were discussions on potentially introducing a middle component
> to generate the tables.  Coreboot was raised as a possibility, and
> David thought it would be okay to use coreboot for both OVMF and
> SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> qemu repo, is run in the guest, and generates the tables (which is
> similar to the hvmloader approach that Xen uses).
> 
> Anthony requested that patches be made that generate the ACPI tables
> in QEMU for the upcoming hotplug work, so that they could be evaluated
> to see if they truly do need to live in QEMU or if the code could live
> in the firmware.  There were no objections.
> 
> -Kevin

I volunteered to implement this.

It was also mentioned that this patch does not yet have to fix the
cross-version migration issue with fw_cfg. If we agree on a direction,
we will fix it then.

Lastly, a proposal was made by Michael to make the call bi-weekly
instead of weekly, as we were cancelling it too much.
There were no objections.

Thus, the next call is planned for June 11, 2013.

-- 
MST

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

"Michael S. Tsirkin" <mst@redhat.com> writes:

> On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
>> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
>> > Juan is not available now, and Anthony asked for
>> > agenda to be sent early.
>> > So here comes:
>> > 
>> > Agenda for the meeting Tue, May 28:
>> > 
>> > - Generating acpi tables
>> 
>> I didn't see any meeting notes, but I thought it would be worthwhile
>> to summarize the call.  This is from memory so correct me if I got
>> anything wrong.
>> 
>> Anthony believes that the generation of ACPI tables is the task of the
>> firmware.  Reasons cited include security implications of running more
>> code in qemu vs the guest context, complexities in running iasl on
>> big-endian machines, possible complexity of having to regenerate
>> tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
>> that QOM interface should be sufficient.
>> 
>> Kevin believes that the bios table code should be moved up into QEMU.
>> Reasons cited include the churn rate in SeaBIOS for this QEMU feature
>> (15-20% of all SeaBIOS commits since integrating with QEMU have been
>> for bios tables; 20% of SeaBIOS commits in last year), complexity of
>> trying to pass all the content needed to generate the tables (eg,
>> device details, power tree, irq routing), complexity of scheduling
>> changes across different repos and synchronizing their rollout,
>> complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
>> wasn't aware of a requirement to regenerate acpi tables on a vm
>> reboot.
>
> I think this last one is based on a misunderstanding: it's based
> on assumption that we we change hardware by hotplug
> we should regenerate the tables to match.
> But there's no management that can take advantage of
> this.
> Two possible reasonable things we can tell management:
> - hotplug for device XXX is not supported: restart qemu
>   to make guest use the device
> - hotplug for device XXX is supported

This introduces an assumption: that the device model never radically
changes across resets.

Why should this be true?  Shouldn't we be allowed to increase the amount
of memory the guest has across reboots?  That's equivalent to adding
another DIMM after power off.

Not generating tables on reset does limit what we can do in a pretty
fundamental way.  Even if you can argue it in the short term, I don't
think it's viable in the long term.

Regards,

Anthony Liguori

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Wed, May 29, 2013 at 11:12:06AM -0500, Anthony Liguori wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
> 
> > On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> >> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> >> > Juan is not available now, and Anthony asked for
> >> > agenda to be sent early.
> >> > So here comes:
> >> > 
> >> > Agenda for the meeting Tue, May 28:
> >> > 
> >> > - Generating acpi tables
> >> 
> >> I didn't see any meeting notes, but I thought it would be worthwhile
> >> to summarize the call.  This is from memory so correct me if I got
> >> anything wrong.
> >> 
> >> Anthony believes that the generation of ACPI tables is the task of the
> >> firmware.  Reasons cited include security implications of running more
> >> code in qemu vs the guest context, complexities in running iasl on
> >> big-endian machines, possible complexity of having to regenerate
> >> tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> >> that QOM interface should be sufficient.
> >> 
> >> Kevin believes that the bios table code should be moved up into QEMU.
> >> Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> >> (15-20% of all SeaBIOS commits since integrating with QEMU have been
> >> for bios tables; 20% of SeaBIOS commits in last year), complexity of
> >> trying to pass all the content needed to generate the tables (eg,
> >> device details, power tree, irq routing), complexity of scheduling
> >> changes across different repos and synchronizing their rollout,
> >> complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> >> wasn't aware of a requirement to regenerate acpi tables on a vm
> >> reboot.
> >
> > I think this last one is based on a misunderstanding: it's based
> > on assumption that we we change hardware by hotplug
> > we should regenerate the tables to match.
> > But there's no management that can take advantage of
> > this.
> > Two possible reasonable things we can tell management:
> > - hotplug for device XXX is not supported: restart qemu
> >   to make guest use the device
> > - hotplug for device XXX is supported
> 
> This introduces an assumption: that the device model never radically
> changes across resets.
> 
> Why should this be true?  Shouldn't we be allowed to increase the amount
> of memory the guest has across reboots?  That's equivalent to adding
> another DIMM after power off.
> 

You can argue the same thing about non hotpluggable devices:
you might be able to replace them when guest is powered off.

It's not supported ATM and if/when it is, there's a bunch of
code to be written.

> Not generating tables on reset does limit what we can do in a pretty
> fundamental way.  Even if you can argue it in the short term, I don't
> think it's viable in the long term.
> 
> Regards,
> 
> Anthony Liguori

No because that's not "at reset".
We need a separate state for power off.

You power off the machine, add DIMM, restart it.

Its not something you can do from inside the guest,
unlike reset.

At the moment, the only way to implement this is by
exiting from QEMU.
So we are not introducing any regressions here.
When qemu gains power off state we can add a handler
and regenerate the tables.

-- 
MST

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

  Hi,

> Why should this be true?  Shouldn't we be allowed to increase the amount
> of memory the guest has across reboots?  That's equivalent to adding
> another DIMM after power off.

poweroff is equivalent to exiting qemu, not to guest reset.

> Not generating tables on reset does limit what we can do in a pretty
> fundamental way.  Even if you can argue it in the short term, I don't
> think it's viable in the long term.

I don't think so.  The procedure for adding/removing non-hotpluggable
hardware is:  poweroff, plugin/-out hardware (change config in qemu),
boot.  Hotpluggable hardware doesn't need acpi table updates.

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gleb Natapov]

On Wed, May 29, 2013 at 11:45:44AM +0300, Michael S. Tsirkin wrote:
> On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> > On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> > > Juan is not available now, and Anthony asked for
> > > agenda to be sent early.
> > > So here comes:
> > > 
> > > Agenda for the meeting Tue, May 28:
> > > 
> > > - Generating acpi tables
> > 
> > I didn't see any meeting notes, but I thought it would be worthwhile
> > to summarize the call.  This is from memory so correct me if I got
> > anything wrong.
> > 
> > Anthony believes that the generation of ACPI tables is the task of the
> > firmware.  Reasons cited include security implications of running more
> > code in qemu vs the guest context, complexities in running iasl on
> > big-endian machines, possible complexity of having to regenerate
> > tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> > that QOM interface should be sufficient.
> > 
> > Kevin believes that the bios table code should be moved up into QEMU.
> > Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> > (15-20% of all SeaBIOS commits since integrating with QEMU have been
> > for bios tables; 20% of SeaBIOS commits in last year), complexity of
> > trying to pass all the content needed to generate the tables (eg,
> > device details, power tree, irq routing), complexity of scheduling
> > changes across different repos and synchronizing their rollout,
> > complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> > wasn't aware of a requirement to regenerate acpi tables on a vm
> > reboot.
> 
> I think this last one is based on a misunderstanding: it's based
> on assumption that we we change hardware by hotplug
> we should regenerate the tables to match.
> But there's no management that can take advantage of
> this.
> Two possible reasonable things we can tell management:
> - hotplug for device XXX is not supported: restart qemu
>   to make guest use the device
> - hotplug for device XXX is supported
> 
> What is proposed here instead is a third option:
> - hotplug is supported but device is not functional.
>   reboot guest to make it fully functional
> 
> This will naturally lead to requirement to regenerate tables on reset.
> 
> And this is what would happen with guest-generated
> tables, and I consider this a bug, not a feature.
> 
+1. This will probably break guest resume too.

> If you really wanted to update tables dynamically, without restarting
> qemu, don't stop there, add an interface for guest to update them
> without reset. I think that's over-endineering and a
> requirement that's best avoided.
> 
> 
> > There were discussions on potentially introducing a middle component
> > to generate the tables.  Coreboot was raised as a possibility, and
> > David thought it would be okay to use coreboot for both OVMF and
> > SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> > qemu repo, is run in the guest, and generates the tables (which is
> > similar to the hvmloader approach that Xen uses).
> > 
> > Anthony requested that patches be made that generate the ACPI tables
> > in QEMU for the upcoming hotplug work, so that they could be evaluated
> > to see if they truly do need to live in QEMU or if the code could live
> > in the firmware.  There were no objections.
> > 
> > -Kevin
> 
> I volunteered to implement this.
Why hotplug should generate ACPI code? It does not do so on real HW.

> 
> It was also mentioned that this patch does not yet have to fix the
> cross-version migration issue with fw_cfg. If we agree on a direction,
> we will fix it then.
> 
> Lastly, a proposal was made by Michael to make the call bi-weekly
> instead of weekly, as we were cancelling it too much.
> There were no objections.
> 
> Thus, the next call is planned for June 11, 2013.
> 
> -- 
> MST
> 
> _______________________________________________
> SeaBIOS mailing list
> SeaBIOS@seabios.org
> http://www.seabios.org/mailman/listinfo/seabios

--
			Gleb.

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Sun, Jun 02, 2013 at 06:05:42PM +0300, Gleb Natapov wrote:
> On Wed, May 29, 2013 at 11:45:44AM +0300, Michael S. Tsirkin wrote:
> > On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> > > On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> > > > Juan is not available now, and Anthony asked for
> > > > agenda to be sent early.
> > > > So here comes:
> > > > 
> > > > Agenda for the meeting Tue, May 28:
> > > > 
> > > > - Generating acpi tables
> > > 
> > > I didn't see any meeting notes, but I thought it would be worthwhile
> > > to summarize the call.  This is from memory so correct me if I got
> > > anything wrong.
> > > 
> > > Anthony believes that the generation of ACPI tables is the task of the
> > > firmware.  Reasons cited include security implications of running more
> > > code in qemu vs the guest context, complexities in running iasl on
> > > big-endian machines, possible complexity of having to regenerate
> > > tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> > > that QOM interface should be sufficient.
> > > 
> > > Kevin believes that the bios table code should be moved up into QEMU.
> > > Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> > > (15-20% of all SeaBIOS commits since integrating with QEMU have been
> > > for bios tables; 20% of SeaBIOS commits in last year), complexity of
> > > trying to pass all the content needed to generate the tables (eg,
> > > device details, power tree, irq routing), complexity of scheduling
> > > changes across different repos and synchronizing their rollout,
> > > complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> > > wasn't aware of a requirement to regenerate acpi tables on a vm
> > > reboot.
> > 
> > I think this last one is based on a misunderstanding: it's based
> > on assumption that we we change hardware by hotplug
> > we should regenerate the tables to match.
> > But there's no management that can take advantage of
> > this.
> > Two possible reasonable things we can tell management:
> > - hotplug for device XXX is not supported: restart qemu
> >   to make guest use the device
> > - hotplug for device XXX is supported
> > 
> > What is proposed here instead is a third option:
> > - hotplug is supported but device is not functional.
> >   reboot guest to make it fully functional
> > 
> > This will naturally lead to requirement to regenerate tables on reset.
> > 
> > And this is what would happen with guest-generated
> > tables, and I consider this a bug, not a feature.
> > 
> +1. This will probably break guest resume too.
> 
> > If you really wanted to update tables dynamically, without restarting
> > qemu, don't stop there, add an interface for guest to update them
> > without reset. I think that's over-endineering and a
> > requirement that's best avoided.
> > 
> > 
> > > There were discussions on potentially introducing a middle component
> > > to generate the tables.  Coreboot was raised as a possibility, and
> > > David thought it would be okay to use coreboot for both OVMF and
> > > SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> > > qemu repo, is run in the guest, and generates the tables (which is
> > > similar to the hvmloader approach that Xen uses).
> > > 
> > > Anthony requested that patches be made that generate the ACPI tables
> > > in QEMU for the upcoming hotplug work, so that they could be evaluated
> > > to see if they truly do need to live in QEMU or if the code could live
> > > in the firmware.  There were no objections.
> > > 
> > > -Kevin
> > 
> > I volunteered to implement this.
> Why hotplug should generate ACPI code? It does not do so on real HW.

Hotplug should not generate ACPI code.
What is meant here is adding ACPI code to support hotplug
of devices behind a PCI to PCI bridge.


> > 
> > It was also mentioned that this patch does not yet have to fix the
> > cross-version migration issue with fw_cfg. If we agree on a direction,
> > we will fix it then.
> > 
> > Lastly, a proposal was made by Michael to make the call bi-weekly
> > instead of weekly, as we were cancelling it too much.
> > There were no objections.
> > 
> > Thus, the next call is planned for June 11, 2013.
> > 
> > -- 
> > MST
> > 
> > _______________________________________________
> > SeaBIOS mailing list
> > SeaBIOS@seabios.org
> > http://www.seabios.org/mailman/listinfo/seabios
> 
> --
> 			Gleb.

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gleb Natapov]

On Sun, Jun 02, 2013 at 06:09:50PM +0300, Michael S. Tsirkin wrote:
> On Sun, Jun 02, 2013 at 06:05:42PM +0300, Gleb Natapov wrote:
> > On Wed, May 29, 2013 at 11:45:44AM +0300, Michael S. Tsirkin wrote:
> > > On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> > > > On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> > > > > Juan is not available now, and Anthony asked for
> > > > > agenda to be sent early.
> > > > > So here comes:
> > > > > 
> > > > > Agenda for the meeting Tue, May 28:
> > > > > 
> > > > > - Generating acpi tables
> > > > 
> > > > I didn't see any meeting notes, but I thought it would be worthwhile
> > > > to summarize the call.  This is from memory so correct me if I got
> > > > anything wrong.
> > > > 
> > > > Anthony believes that the generation of ACPI tables is the task of the
> > > > firmware.  Reasons cited include security implications of running more
> > > > code in qemu vs the guest context, complexities in running iasl on
> > > > big-endian machines, possible complexity of having to regenerate
> > > > tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> > > > that QOM interface should be sufficient.
> > > > 
> > > > Kevin believes that the bios table code should be moved up into QEMU.
> > > > Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> > > > (15-20% of all SeaBIOS commits since integrating with QEMU have been
> > > > for bios tables; 20% of SeaBIOS commits in last year), complexity of
> > > > trying to pass all the content needed to generate the tables (eg,
> > > > device details, power tree, irq routing), complexity of scheduling
> > > > changes across different repos and synchronizing their rollout,
> > > > complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> > > > wasn't aware of a requirement to regenerate acpi tables on a vm
> > > > reboot.
> > > 
> > > I think this last one is based on a misunderstanding: it's based
> > > on assumption that we we change hardware by hotplug
> > > we should regenerate the tables to match.
> > > But there's no management that can take advantage of
> > > this.
> > > Two possible reasonable things we can tell management:
> > > - hotplug for device XXX is not supported: restart qemu
> > >   to make guest use the device
> > > - hotplug for device XXX is supported
> > > 
> > > What is proposed here instead is a third option:
> > > - hotplug is supported but device is not functional.
> > >   reboot guest to make it fully functional
> > > 
> > > This will naturally lead to requirement to regenerate tables on reset.
> > > 
> > > And this is what would happen with guest-generated
> > > tables, and I consider this a bug, not a feature.
> > > 
> > +1. This will probably break guest resume too.
> > 
> > > If you really wanted to update tables dynamically, without restarting
> > > qemu, don't stop there, add an interface for guest to update them
> > > without reset. I think that's over-endineering and a
> > > requirement that's best avoided.
> > > 
> > > 
> > > > There were discussions on potentially introducing a middle component
> > > > to generate the tables.  Coreboot was raised as a possibility, and
> > > > David thought it would be okay to use coreboot for both OVMF and
> > > > SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> > > > qemu repo, is run in the guest, and generates the tables (which is
> > > > similar to the hvmloader approach that Xen uses).
> > > > 
> > > > Anthony requested that patches be made that generate the ACPI tables
> > > > in QEMU for the upcoming hotplug work, so that they could be evaluated
> > > > to see if they truly do need to live in QEMU or if the code could live
> > > > in the firmware.  There were no objections.
> > > > 
> > > > -Kevin
> > > 
> > > I volunteered to implement this.
> > Why hotplug should generate ACPI code? It does not do so on real HW.
> 
> Hotplug should not generate ACPI code.
> What is meant here is adding ACPI code to support hotplug
> of devices behind a PCI to PCI bridge.
> 
Ah, OK. This one does not change on reset.

--
			Gleb.

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Sun, Jun 02, 2013 at 06:40:43PM +0300, Gleb Natapov wrote:
> On Sun, Jun 02, 2013 at 06:09:50PM +0300, Michael S. Tsirkin wrote:
> > On Sun, Jun 02, 2013 at 06:05:42PM +0300, Gleb Natapov wrote:
> > > On Wed, May 29, 2013 at 11:45:44AM +0300, Michael S. Tsirkin wrote:
> > > > On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> > > > > On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> > > > > > Juan is not available now, and Anthony asked for
> > > > > > agenda to be sent early.
> > > > > > So here comes:
> > > > > > 
> > > > > > Agenda for the meeting Tue, May 28:
> > > > > > 
> > > > > > - Generating acpi tables
> > > > > 
> > > > > I didn't see any meeting notes, but I thought it would be worthwhile
> > > > > to summarize the call.  This is from memory so correct me if I got
> > > > > anything wrong.
> > > > > 
> > > > > Anthony believes that the generation of ACPI tables is the task of the
> > > > > firmware.  Reasons cited include security implications of running more
> > > > > code in qemu vs the guest context, complexities in running iasl on
> > > > > big-endian machines, possible complexity of having to regenerate
> > > > > tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> > > > > that QOM interface should be sufficient.
> > > > > 
> > > > > Kevin believes that the bios table code should be moved up into QEMU.
> > > > > Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> > > > > (15-20% of all SeaBIOS commits since integrating with QEMU have been
> > > > > for bios tables; 20% of SeaBIOS commits in last year), complexity of
> > > > > trying to pass all the content needed to generate the tables (eg,
> > > > > device details, power tree, irq routing), complexity of scheduling
> > > > > changes across different repos and synchronizing their rollout,
> > > > > complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> > > > > wasn't aware of a requirement to regenerate acpi tables on a vm
> > > > > reboot.
> > > > 
> > > > I think this last one is based on a misunderstanding: it's based
> > > > on assumption that we we change hardware by hotplug
> > > > we should regenerate the tables to match.
> > > > But there's no management that can take advantage of
> > > > this.
> > > > Two possible reasonable things we can tell management:
> > > > - hotplug for device XXX is not supported: restart qemu
> > > >   to make guest use the device
> > > > - hotplug for device XXX is supported
> > > > 
> > > > What is proposed here instead is a third option:
> > > > - hotplug is supported but device is not functional.
> > > >   reboot guest to make it fully functional
> > > > 
> > > > This will naturally lead to requirement to regenerate tables on reset.
> > > > 
> > > > And this is what would happen with guest-generated
> > > > tables, and I consider this a bug, not a feature.
> > > > 
> > > +1. This will probably break guest resume too.
> > > 
> > > > If you really wanted to update tables dynamically, without restarting
> > > > qemu, don't stop there, add an interface for guest to update them
> > > > without reset. I think that's over-endineering and a
> > > > requirement that's best avoided.
> > > > 
> > > > 
> > > > > There were discussions on potentially introducing a middle component
> > > > > to generate the tables.  Coreboot was raised as a possibility, and
> > > > > David thought it would be okay to use coreboot for both OVMF and
> > > > > SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> > > > > qemu repo, is run in the guest, and generates the tables (which is
> > > > > similar to the hvmloader approach that Xen uses).
> > > > > 
> > > > > Anthony requested that patches be made that generate the ACPI tables
> > > > > in QEMU for the upcoming hotplug work, so that they could be evaluated
> > > > > to see if they truly do need to live in QEMU or if the code could live
> > > > > in the firmware.  There were no objections.
> > > > > 
> > > > > -Kevin
> > > > 
> > > > I volunteered to implement this.
> > > Why hotplug should generate ACPI code? It does not do so on real HW.
> > 
> > Hotplug should not generate ACPI code.
> > What is meant here is adding ACPI code to support hotplug
> > of devices behind a PCI to PCI bridge.
> > 
> Ah, OK. This one does not change on reset.

It wouldn't if QEMU generates it.
With bios generating the tables it might depending
on how it's implemented.
To make it not change across resets we'd need
an interface in QEMU to tell guest whether a
device was added since QEMU start.

> --
> 			Gleb.

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Paolo Bonzini]

Il 02/06/2013 17:05, Gleb Natapov ha scritto:
>>> Anthony requested that patches be made that generate the ACPI tables
>>> in QEMU for the upcoming hotplug work, so that they could be evaluated
>>> to see if they truly do need to live in QEMU or if the code could live
>>> in the firmware.  There were no objections.
>>
>> I volunteered to implement this.
>
> Why hotplug should generate ACPI code? It does not do so on real HW.

Hotplug can do a LoadTable and merge it into the existing ones.  But
then you do not need QEMU-time generation of tables to do the same thing
for cold-plug.

Paolo

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

On 05/29/13 01:53, Kevin O'Connor wrote:
> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
>> Juan is not available now, and Anthony asked for
>> agenda to be sent early.
>> So here comes:
>>
>> Agenda for the meeting Tue, May 28:
>>
>> - Generating acpi tables
> 
> I didn't see any meeting notes, but I thought it would be worthwhile
> to summarize the call.  This is from memory so correct me if I got
> anything wrong.
> 
> Anthony believes that the generation of ACPI tables is the task of the
> firmware.  Reasons cited include security implications of running more
> code in qemu vs the guest context,

I fail to see the security issues here.  It's not like the apci table
generation code operates on untrusted input from the guest ...

> complexities in running iasl on
> big-endian machines,

We already have a bunch of prebuilt blobs in the qemu repo for simliar
reasons, we can do that with iasl output too.

> possible complexity of having to regenerate
> tables on a vm reboot,

Why tables should be regenerated at reboot?  I remember hotplug being
mentioned in the call.  Hmm?  Which hotplugged component needs acpi
table updates to work properly?  And what is the point of hotplugging if
you must reboot the guest anyway to get the acpi updates needed?
Details please.

Also mentioned in the call: "architectural reasons", which I understand
as "real hardware works that way".  Correct.  But qemu's virtual
hardware is configurable in more ways than real hardware, so we have
different needs.  For example: pci slots can or can't be hotpluggable.
On real hardware this is fixed.  IIRC this is one of the reasons why we
have to patch acpi tables.

> overall sloppiness of doing it in QEMU.

/me gets the feeling that this is the *main* reason, given that the
other ones don't look very convincing to me.

> Raised
> that QOM interface should be sufficient.

Agree on this one.  Ideally the acpi table generation code should be
able to gather all information it needs from the qom tree, so it can be
a standalone C file instead of being scattered over all qemu.

> There were discussions on potentially introducing a middle component
> to generate the tables.  Coreboot was raised as a possibility, and
> David thought it would be okay to use coreboot for both OVMF and
> SeaBIOS.

Certainly an option, but that is a long-term project.

> The possibility was also raised of a "rom" that lives in the
> qemu repo, is run in the guest, and generates the tables (which is
> similar to the hvmloader approach that Xen uses).

Also simliar to the coreboot idea.

Also in the call: The idea of having some library for acpi table
generation provided by qemu which the firmware can use.  Has license
compatibility issues.  Also difficult due to the fact that there is no
libc in firmware, so such a library would need firmware-specific
abstraction layers even for simple stuff such as memory allocation.

> Anthony requested that patches be made that generate the ACPI tables
> in QEMU for the upcoming hotplug work, so that they could be evaluated
> to see if they truly do need to live in QEMU or if the code could live
> in the firmware.

Good.  I think having qemu generate the tables is also quite useful for
evaluating the move to coreboot:

  (1)  make qemu generate the acpi tables.
  (2a) make seabios use the qemu-generated tables.
  (2b) make ovmf use the qemu-generated tables.
  (2c) make coreboot use the qemu-generated tables.

Now we can look where we stand when using coreboot+seabios or
coreboot+tianocore compared to bare seabios / bare ovmf.  I expect there
are quite a few things to fix until the coreboot+seabios combo runs
without regressions compared to bare seabios.  But maybe not when qemu
provides the acpi tables to coreboot.

In case the coreboot testdrive works out well we can continue with:

  (3)  use coreboot+seabios by default.
  (4)  move acpi table generation from qemu to coreboot.

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Wed, May 29, 2013 at 10:49:27AM +0200, Gerd Hoffmann wrote:
> On 05/29/13 01:53, Kevin O'Connor wrote:
> > On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> >> Juan is not available now, and Anthony asked for
> >> agenda to be sent early.
> >> So here comes:
> >>
> >> Agenda for the meeting Tue, May 28:
> >>
> >> - Generating acpi tables
> > 
> > I didn't see any meeting notes, but I thought it would be worthwhile
> > to summarize the call.  This is from memory so correct me if I got
> > anything wrong.
> > 
> > Anthony believes that the generation of ACPI tables is the task of the
> > firmware.  Reasons cited include security implications of running more
> > code in qemu vs the guest context,
> 
> I fail to see the security issues here.  It's not like the apci table
> generation code operates on untrusted input from the guest ...
> 
> > complexities in running iasl on
> > big-endian machines,
> 
> We already have a bunch of prebuilt blobs in the qemu repo for simliar
> reasons, we can do that with iasl output too.
> 
> > possible complexity of having to regenerate
> > tables on a vm reboot,
> 
> Why tables should be regenerated at reboot?  I remember hotplug being
> mentioned in the call.  Hmm?  Which hotplugged component needs acpi
> table updates to work properly?  And what is the point of hotplugging if
> you must reboot the guest anyway to get the acpi updates needed?
> Details please.

I think it's a mistake. I sent a mail explaining this part.

> Also mentioned in the call: "architectural reasons", which I understand
> as "real hardware works that way".  Correct.

Not exactly. Real hardware is very likely to have
most of the tables pre-generated in ROM, load
them and tweak them in the minor way.

That's exactly what patches I sent do.

>  But qemu's virtual
> hardware is configurable in more ways than real hardware, so we have
> different needs.  For example: pci slots can or can't be hotpluggable.
> On real hardware this is fixed.  IIRC this is one of the reasons why we
> have to patch acpi tables.
> 
> > overall sloppiness of doing it in QEMU.
> 
> /me gets the feeling that this is the *main* reason, given that the
> other ones don't look very convincing to me.
> 
> > Raised
> > that QOM interface should be sufficient.
> 
> Agree on this one.  Ideally the acpi table generation code should be
> able to gather all information it needs from the qom tree, so it can be
> a standalone C file instead of being scattered over all qemu.

Did you look at the patchset I posted?
Generation is in a standalone C file there.


However, if you mean we should do things like

if (Device_id == foobar) {
}

in once central place, I disagree.
I think that's nasty, adding devices would
mean touching this central registry.


> > There were discussions on potentially introducing a middle component
> > to generate the tables.  Coreboot was raised as a possibility, and
> > David thought it would be okay to use coreboot for both OVMF and
> > SeaBIOS.
> 
> Certainly an option, but that is a long-term project.
> 
> > The possibility was also raised of a "rom" that lives in the
> > qemu repo, is run in the guest, and generates the tables (which is
> > similar to the hvmloader approach that Xen uses).
> 
> Also simliar to the coreboot idea.
> 
> Also in the call: The idea of having some library for acpi table
> generation provided by qemu which the firmware can use.  Has license
> compatibility issues.  Also difficult due to the fact that there is no
> libc in firmware, so such a library would need firmware-specific
> abstraction layers even for simple stuff such as memory allocation.
> 
> > Anthony requested that patches be made that generate the ACPI tables
> > in QEMU for the upcoming hotplug work, so that they could be evaluated
> > to see if they truly do need to live in QEMU or if the code could live
> > in the firmware.
> 
> Good.  I think having qemu generate the tables is also quite useful for
> evaluating the move to coreboot:
> 
>   (1)  make qemu generate the acpi tables.
>   (2a) make seabios use the qemu-generated tables.
>   (2b) make ovmf use the qemu-generated tables.
>   (2c) make coreboot use the qemu-generated tables.
> 
> Now we can look where we stand when using coreboot+seabios or
> coreboot+tianocore compared to bare seabios / bare ovmf.  I expect there
> are quite a few things to fix until the coreboot+seabios combo runs
> without regressions compared to bare seabios.  But maybe not when qemu
> provides the acpi tables to coreboot.
> 
> In case the coreboot testdrive works out well we can continue with:
> 
>   (3)  use coreboot+seabios by default.
>   (4)  move acpi table generation from qemu to coreboot.
> 
> cheers,
>   Gerd
> 

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

  Hi,

>>> possible complexity of having to regenerate
>>> tables on a vm reboot,
>>
>> Why tables should be regenerated at reboot?  I remember hotplug being
>> mentioned in the call.  Hmm?  Which hotplugged component needs acpi
>> table updates to work properly?  And what is the point of hotplugging if
>> you must reboot the guest anyway to get the acpi updates needed?
>> Details please.
> 
> I think it's a mistake. I sent a mail explaining this part.

Saw it meanwhile.

>> Also mentioned in the call: "architectural reasons", which I understand
>> as "real hardware works that way".  Correct.
> 
> Not exactly. Real hardware is very likely to have
> most of the tables pre-generated in ROM, load
> them and tweak them in the minor way.

>From a quick look it seems coreboot has a static (iasl-compiled) dsdt
and generates everything else.

http://review.coreboot.org/gitweb?p=coreboot.git;a=blob;f=src/mainboard/emulation/qemu-x86/acpi_tables.c

>> Agree on this one.  Ideally the acpi table generation code should be
>> able to gather all information it needs from the qom tree, so it can be
>> a standalone C file instead of being scattered over all qemu.
> 
> Did you look at the patchset I posted?

Very briefly only.

> Generation is in a standalone C file there.

Good.

> However, if you mean we should do things like
> 
> if (Device_id == foobar) {
> }
> in once central place, I disagree.
> I think that's nasty, adding devices would
> mean touching this central registry.

No, I mean more "lookup PIIX4_PM object + check disable_s3 property"
instead of having code for it in hw/acpi/piix4.c or using global variables.

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Wed, May 29, 2013 at 11:42:34AM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> >>> possible complexity of having to regenerate
> >>> tables on a vm reboot,
> >>
> >> Why tables should be regenerated at reboot?  I remember hotplug being
> >> mentioned in the call.  Hmm?  Which hotplugged component needs acpi
> >> table updates to work properly?  And what is the point of hotplugging if
> >> you must reboot the guest anyway to get the acpi updates needed?
> >> Details please.
> > 
> > I think it's a mistake. I sent a mail explaining this part.
> 
> Saw it meanwhile.
> 
> >> Also mentioned in the call: "architectural reasons", which I understand
> >> as "real hardware works that way".  Correct.
> > 
> > Not exactly. Real hardware is very likely to have
> > most of the tables pre-generated in ROM, load
> > them and tweak them in the minor way.
> 
> >From a quick look it seems coreboot has a static (iasl-compiled) dsdt
> and generates everything else.
> 
> http://review.coreboot.org/gitweb?p=coreboot.git;a=blob;f=src/mainboard/emulation/qemu-x86/acpi_tables.c
> 
> >> Agree on this one.  Ideally the acpi table generation code should be
> >> able to gather all information it needs from the qom tree, so it can be
> >> a standalone C file instead of being scattered over all qemu.
> > 
> > Did you look at the patchset I posted?
> 
> Very briefly only.
> 
> > Generation is in a standalone C file there.
> 
> Good.
> 
> > However, if you mean we should do things like
> > 
> > if (Device_id == foobar) {
> > }
> > in once central place, I disagree.
> > I think that's nasty, adding devices would
> > mean touching this central registry.
> 
> No, I mean more "lookup PIIX4_PM object + check disable_s3 property"
> instead of having code for it in hw/acpi/piix4.c or using global variables.
> 
> cheers,
>   Gerd

So that would make code PIIX specific.
Instead I'm passing in guest_info structure to each object
and that describes itself, e.g. sets disable_s3.

-- 
MST

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Anthony Liguori]

Gerd Hoffmann <kraxel@redhat.com> writes:

> On 05/29/13 01:53, Kevin O'Connor wrote:
>> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
>>> Juan is not available now, and Anthony asked for
>>> agenda to be sent early.
>>> So here comes:
>>>
>>> Agenda for the meeting Tue, May 28:
>>>
>>> - Generating acpi tables
>> 
>> I didn't see any meeting notes, but I thought it would be worthwhile
>> to summarize the call.  This is from memory so correct me if I got
>> anything wrong.
>> 
>> Anthony believes that the generation of ACPI tables is the task of the
>> firmware.  Reasons cited include security implications of running more
>> code in qemu vs the guest context,
>
> I fail to see the security issues here.  It's not like the apci table
> generation code operates on untrusted input from the guest ...

But possibly untrusted input from a malicious user.  You can imagine
something like a IaaS provider that let's a user input arbitrary values
for memory, number of nics, etc.

It's a stretch of an example, I agree, but the general principle I think
is sound:  we should push as much work as possible to the least
privileged part of the stack.  In this case, firmware has much less
privileges than QEMU.

>> complexities in running iasl on
>> big-endian machines,
>
> We already have a bunch of prebuilt blobs in the qemu repo for simliar
> reasons, we can do that with iasl output too.
>
>> possible complexity of having to regenerate
>> tables on a vm reboot,
>
> Why tables should be regenerated at reboot?  I remember hotplug being
> mentioned in the call.  Hmm?  Which hotplugged component needs acpi
> table updates to work properly?  And what is the point of hotplugging if
> you must reboot the guest anyway to get the acpi updates needed?
> Details please.

See my response to Michael.

> Also mentioned in the call: "architectural reasons", which I understand
> as "real hardware works that way".  Correct.  But qemu's virtual
> hardware is configurable in more ways than real hardware, so we have
> different needs.  For example: pci slots can or can't be hotpluggable.
> On real hardware this is fixed.  IIRC this is one of the reasons why we
> have to patch acpi tables.

It's not really fixed.  Hardware supports PCI expansion chassises.
Multi-node NUMA systems also affect the ACPI tables.

>> overall sloppiness of doing it in QEMU.
>
> /me gets the feeling that this is the *main* reason, given that the
> other ones don't look very convincing to me.
>
>> Raised
>> that QOM interface should be sufficient.
>
> Agree on this one.  Ideally the acpi table generation code should be
> able to gather all information it needs from the qom tree, so it can be
> a standalone C file instead of being scattered over all qemu.

Ack.  So my basic argument is why not expose the QOM interfaces to
firmware and move the generation code there?  Seems like it would be
more or less a copy/paste once we had a proper implementation in QEMU.

>> There were discussions on potentially introducing a middle component
>> to generate the tables.  Coreboot was raised as a possibility, and
>> David thought it would be okay to use coreboot for both OVMF and
>> SeaBIOS.
>
> Certainly an option, but that is a long-term project.

Out of curiousity, are there other benefits to using coreboot as a core
firmware in QEMU?

Is there a payload we would ever plausibly use besides OVMF and SeaBIOS?

Regards,

Anthony Liguori

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Wed, May 29, 2013 at 11:18:03AM -0500, Anthony Liguori wrote:
> Gerd Hoffmann <kraxel@redhat.com> writes:
> 
> > On 05/29/13 01:53, Kevin O'Connor wrote:
> >> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> >>> Juan is not available now, and Anthony asked for
> >>> agenda to be sent early.
> >>> So here comes:
> >>>
> >>> Agenda for the meeting Tue, May 28:
> >>>
> >>> - Generating acpi tables
> >> 
> >> I didn't see any meeting notes, but I thought it would be worthwhile
> >> to summarize the call.  This is from memory so correct me if I got
> >> anything wrong.
> >> 
> >> Anthony believes that the generation of ACPI tables is the task of the
> >> firmware.  Reasons cited include security implications of running more
> >> code in qemu vs the guest context,
> >
> > I fail to see the security issues here.  It's not like the apci table
> > generation code operates on untrusted input from the guest ...
> 
> But possibly untrusted input from a malicious user.  You can imagine
> something like a IaaS provider that let's a user input arbitrary values
> for memory, number of nics, etc.
> 
> It's a stretch of an example, I agree, but the general principle I think
> is sound:  we should push as much work as possible to the least
> privileged part of the stack.  In this case, firmware has much less
> privileges than QEMU.

It's a big stretch. We have to draw the line somewhere, and I think
when *all* firmware people tell us that QEMU is a pain to work
with and should just supply ACPI table to BIOS, that line
has been crossed.

> >> complexities in running iasl on
> >> big-endian machines,
> >
> > We already have a bunch of prebuilt blobs in the qemu repo for simliar
> > reasons, we can do that with iasl output too.
> >
> >> possible complexity of having to regenerate
> >> tables on a vm reboot,
> >
> > Why tables should be regenerated at reboot?  I remember hotplug being
> > mentioned in the call.  Hmm?  Which hotplugged component needs acpi
> > table updates to work properly?  And what is the point of hotplugging if
> > you must reboot the guest anyway to get the acpi updates needed?
> > Details please.
> 
> See my response to Michael.
> 
> > Also mentioned in the call: "architectural reasons", which I understand
> > as "real hardware works that way".  Correct.  But qemu's virtual
> > hardware is configurable in more ways than real hardware, so we have
> > different needs.  For example: pci slots can or can't be hotpluggable.
> > On real hardware this is fixed.  IIRC this is one of the reasons why we
> > have to patch acpi tables.
> 
> It's not really fixed.  Hardware supports PCI expansion chassises.

These normally aren't reported in ACPI, so no hotplug,
or only native hotplug.

> Multi-node NUMA systems also affect the ACPI tables.

In a very minor way.

> >> overall sloppiness of doing it in QEMU.
> >
> > /me gets the feeling that this is the *main* reason, given that the
> > other ones don't look very convincing to me.
> >
> >> Raised
> >> that QOM interface should be sufficient.
> >
> > Agree on this one.  Ideally the acpi table generation code should be
> > able to gather all information it needs from the qom tree, so it can be
> > a standalone C file instead of being scattered over all qemu.
> 
> Ack.  So my basic argument is why not expose the QOM interfaces to
> firmware and move the generation code there?  Seems like it would be
> more or less a copy/paste once we had a proper implementation in QEMU.

Because that's just insanely rick interface we have no chance to
keep stable across versions.
Because it's a ton of QEMU specific firmware.
Because firmware devs don't want to maintain the ACPI that *is* there either.

> >> There were discussions on potentially introducing a middle component
> >> to generate the tables.  Coreboot was raised as a possibility, and
> >> David thought it would be okay to use coreboot for both OVMF and
> >> SeaBIOS.
> >
> > Certainly an option, but that is a long-term project.
> 
> Out of curiousity, are there other benefits to using coreboot as a core
> firmware in QEMU?
> 
> Is there a payload we would ever plausibly use besides OVMF and SeaBIOS?
> 
> Regards,
> 
> Anthony Liguori

The easier it is to switch firmware the better.

Gives us choice, we switched firmware several times,
we will do it again.

If firmware only has a simple loader for QEMU specific
stuff and is mostly generic, then it's easy.
If there's a lot of code for walking QOM, etc - it's
very painful.

-- 
MST

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Wed, May 29, 2013 at 07:28:05PM +0300, Michael S. Tsirkin wrote:
> Because that's just insanely rick interface

s/rick/rich/. Sorry about the typo.

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Markus Armbruster]

Anthony Liguori <anthony@codemonkey.ws> writes:

> Gerd Hoffmann <kraxel@redhat.com> writes:
>
>> On 05/29/13 01:53, Kevin O'Connor wrote:
>>> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
>>>> Juan is not available now, and Anthony asked for
>>>> agenda to be sent early.
>>>> So here comes:
>>>>
>>>> Agenda for the meeting Tue, May 28:
>>>>
>>>> - Generating acpi tables
>>> 
>>> I didn't see any meeting notes, but I thought it would be worthwhile
>>> to summarize the call.  This is from memory so correct me if I got
>>> anything wrong.
>>> 
>>> Anthony believes that the generation of ACPI tables is the task of the
>>> firmware.  Reasons cited include security implications of running more
>>> code in qemu vs the guest context,
>>
>> I fail to see the security issues here.  It's not like the apci table
>> generation code operates on untrusted input from the guest ...
>
> But possibly untrusted input from a malicious user.  You can imagine
> something like a IaaS provider that let's a user input arbitrary values
> for memory, number of nics, etc.
>
> It's a stretch of an example, I agree, but the general principle I think
> is sound:  we should push as much work as possible to the least
> privileged part of the stack.  In this case, firmware has much less
> privileges than QEMU.

Firmware runs in a primitive, unforgiving environment, and should do as
little as humanly possible.  For an instructive example of deviating
from that rule, check out UEFI.

[...]

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Kevin O'Connor]

On Wed, May 29, 2013 at 11:18:03AM -0500, Anthony Liguori wrote:
> Gerd Hoffmann <kraxel@redhat.com> writes:
> > On 05/29/13 01:53, Kevin O'Connor wrote:
> >> Raised
> >> that QOM interface should be sufficient.
> >
> > Agree on this one.  Ideally the acpi table generation code should be
> > able to gather all information it needs from the qom tree, so it can be
> > a standalone C file instead of being scattered over all qemu.
> 
> Ack.  So my basic argument is why not expose the QOM interfaces to
> firmware and move the generation code there?

I remain doubtful that QOM has all the info needed to generate the
BIOS tables.  Does QOM describe how the 5th pci device uses global
interrupt 11 when using global interrupts, legacy interrupt 5 when not
using global interrupts, and that the legacy interrupt can be changed
by writing to the 0x60 address of the 1st pci device's config space?
Does QOM state that the machine supports S3 sleep mode?  Does QOM
indicate that an IPMI device supports the 3rd version of the IPMI
device specification?

I don't see how exporting QOM to the firmware will help.  I predict we
would continue to see most of the BIOS tables hardcoded in the
firmware and that all but the most minor changes to those tables would
require synchronizing code patches to both QEMU and the firmware.  I
also suspect we would end up adding fields to QOM that only the BIOS
tables cared about, and that ever increasing code would be needed in
both QEMU and the firmware to juggle to/from QOM so that the BIOS
tables could be created.

-Kevin

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1366 bytes --]

On Wed, 2013-05-29 at 21:12 -0400, Kevin O'Connor wrote:
> 
> I remain doubtful that QOM has all the info needed to generate the
> BIOS tables.  Does QOM describe how the 5th pci device uses global
> interrupt 11 when using global interrupts, legacy interrupt 5 when not
> using global interrupts, and that the legacy interrupt can be changed
> by writing to the 0x60 address of the 1st pci device's config space?
> Does QOM state that the machine supports S3 sleep mode?  Does QOM
> indicate that an IPMI device supports the 3rd version of the IPMI
> device specification?

Does it indicate whether this particular version of qemu has correctly
implemented the hard reset at 0xcf9? If so, we need to put that in as
the ACPI RESET_REG.

It seems that there's a *lot* which isn't fully described in the QOM
tree. Do we really want to add it all, just so that ACPI tables can be
reliably generated from it? 

As we add new types of hardware and even fix/adjust features like the
examples above, we'll also have to implement the translation from QOM to
ACPI tables. And we'll have to do so in more than one place, in projects
with a completely different release cycle. This would be *so* much
easier if the code which actually generates the ACPI tables was *in* the
qemu tree along with the "hardware" that those tables describe.

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

  Hi,

>>> Raised
>>> that QOM interface should be sufficient.
>>
>> Agree on this one.  Ideally the acpi table generation code should be
>> able to gather all information it needs from the qom tree, so it can be
>> a standalone C file instead of being scattered over all qemu.
> 
> Ack.  So my basic argument is why not expose the QOM interfaces to
> firmware and move the generation code there?  Seems like it would be
> more or less a copy/paste once we had a proper implementation in QEMU.

Well, no.  Firmware is a quite simple environment without standard libc
etc, so moving code from qemu to firmware certainly isn't as easy as
copying over a file.

>>> There were discussions on potentially introducing a middle component
>>> to generate the tables.  Coreboot was raised as a possibility, and
>>> David thought it would be okay to use coreboot for both OVMF and
>>> SeaBIOS.
>>
>> Certainly an option, but that is a long-term project.
> 
> Out of curiousity, are there other benefits to using coreboot as a core
> firmware in QEMU?

Short-term it's alot of work as we have to bring coreboot's qemu support
to feature parity with seabios.  I suspect most of this is acpi related
though, so when qemu provides the tables and coreboot uses them we could
be pretty close already.

Long-term it should simplify firmware maintainance as we have only *one*
place which handles the hardware bringup, and seabios/ovmf have less
work to do.

> Is there a payload we would ever plausibly use besides OVMF and SeaBIOS?

I wouldn't be surprised if people start using other coreboot payloads
and/or features such as direct linux kernel boot once it works well on qemu.

We might even run qemu test suites as coreboot payload.

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

On Wed, 2013-05-29 at 11:18 -0500, Anthony Liguori wrote:
> 
> > Certainly an option, but that is a long-term project.
> 
> Out of curiousity, are there other benefits to using coreboot as a core
> firmware in QEMU?
> 
> Is there a payload we would ever plausibly use besides OVMF and SeaBIOS?

I like the idea of using Coreboot on the UEFI side — if the most
actively used TianoCore platform is CorebootPkg instead of OvmfPkg, that
makes it a lot easier for people using *real* hardware with Coreboot to
use TianoCore.

And it helps to dispel the stupid misconception in some quarters that
Coreboot *competes* with UEFI and thus cannot possibly be supported
because helping something that competes with UEFI would be bad.

> Is there a payload we would ever plausibly use besides OVMF and
> SeaBIOS?

For my part I want to get to the point where the default firmware
shipped with qemu can be OVMF. We have SeaBIOS-as-CSM working, which was
one of the biggest barriers. There are a few more things (like NV
variable storage, in particular) that I need to fix before I can
actually make that suggestion with a straight face though...

-- 
dwmw2


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/30/13 11:23, David Woodhouse wrote:
> On Wed, 2013-05-29 at 11:18 -0500, Anthony Liguori wrote:
>>
>>> Certainly an option, but that is a long-term project.
>>
>> Out of curiousity, are there other benefits to using coreboot as a core
>> firmware in QEMU?
>>
>> Is there a payload we would ever plausibly use besides OVMF and SeaBIOS?
> 
> I like the idea of using Coreboot on the UEFI side — if the most
> actively used TianoCore platform is CorebootPkg instead of OvmfPkg, that
> makes it a lot easier for people using *real* hardware with Coreboot to
> use TianoCore.

Where is CorebootPkg available from?

> And it helps to dispel the stupid misconception in some quarters that
> Coreboot *competes* with UEFI and thus cannot possibly be supported
> because helping something that competes with UEFI would be bad.

I'm not sure who do you mean by "some quarters", but for some
distributions Coreboot would be yet another component (package) to
support, for no obvious benefit.

(Gerd said it better than I possibly could:
<http://thread.gmane.org/gmane.comp.bios.coreboot.seabios/5685/focus=5705>.)

> 
>> Is there a payload we would ever plausibly use besides OVMF and
>> SeaBIOS?
> 
> For my part I want to get to the point where the default firmware
> shipped with qemu can be OVMF.

For some distributions this is a licensing question as well. See
"FatBinPkg/License.txt". (The same applies if you rebuild it from source
(FatPkgDev), based on "FatBinPkg/ReadMe.txt".)  For example Fedora can't
ship OVMF because of it.

If you implement a UEFI FAT driver based on Microsoft's official
specification, you're bound by the same restrictions on use and
redistribution.

If you implement a private UEFI FAT driver from scratch, or port a free
software FAT implementation (eg. the r/o one in grub or the r/w one in
mtools), you could still run into legal problems, I've been told.

If you rip out the FAT driver, then OVMF won't be UEFI compliant and no
installer media will boot on it.

Interestingly, Ubuntu has OVMF in "Universe"
<http://packages.ubuntu.com/raring/ovmf>. I think they missed the
FatBinPkg license (I would have missed it too, after all you have to
track down the licenses of every module included in the FDF file -- it
was Paolo who told me about it) and I believe they should actually ship
OVMF in Multiverse or Restricted
<https://help.ubuntu.com/community/Repositories/Ubuntu>.

> We have SeaBIOS-as-CSM working, which was
> one of the biggest barriers.

Agreed, and I could have never done that. Thanks for implementing it
with Kevin.

We need at least one out-of-tree edk2 patch for now (from you) but
apparently that's no problem.

> There are a few more things (like NV
> variable storage, in particular) that I need to fix before I can
> actually make that suggestion with a straight face though...

As far as I understand:
- Jordan is nearing completion of flash support on KVM,
- he also has WIP NvVar storage on top of qemu flash.

http://thread.gmane.org/gmane.comp.emulators.qemu/213690
http://thread.gmane.org/gmane.comp.bios.tianocore.devel/2781/focus=2798

("Please coordinate" I guess :))

Laszlo

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2563 bytes --]

On Thu, 2013-05-30 at 13:13 +0200, Laszlo Ersek wrote:
> Where is CorebootPkg available from?

https://github.com/pgeorgi/edk2/tree/coreboot-pkg

> > And it helps to dispel the stupid misconception in some quarters that
> > Coreboot *competes* with UEFI and thus cannot possibly be supported
> > because helping something that competes with UEFI would be bad.
> 
> I'm not sure who do you mean by "some quarters", but for some
> distributions Coreboot would be yet another component (package) to
> support, for no obvious benefit.
> 
> (Gerd said it better than I possibly could:
> <http://thread.gmane.org/gmane.comp.bios.coreboot.seabios/5685/focus=5705>.)

Yeah, but if we're shoving a lot of hardware-specific ACPI table
generation into the guest's firmware, instead of just doing it on the
qemu side where a number of us seem to think it belongs, then there *is*
a benefit to using Coreboot. When stuff changes on the qemu side and we
have to update the table generation to match, you end up having to
update just the Coreboot package, and *not* having to patch both SeaBIOS
and OVMF.

The extra package in the distro really isn't painful to handle, and I
suspect it would end up *reducing* the amount of work that we have to do
to update. You update *just* Coreboot, not *both* of SeaBIOS and OVMF.

> If you implement a private UEFI FAT driver from scratch, or port a free
> software FAT implementation (eg. the r/o one in grub or the r/w one in
> mtools), you could still run into legal problems, I've been told.

That has been said, and it's been said for the FAT implementation in the
kernel too. If a distribution is happy to ship the kernel without
ripping out its FAT support, then I see no reason why that distribution
wouldn't also be happy to ship a version of OVMF with a clean
implementation of FAT support. It doesn't make sense to be happy with
one but not the other.

> We need at least one out-of-tree edk2 patch for now (from you) but
> apparently that's no problem.

That'll get merged soon. We are working on the corresponding spec
update...

> As far as I understand:
> - Jordan is nearing completion of flash support on KVM,
> - he also has WIP NvVar storage on top of qemu flash.
> 
> http://thread.gmane.org/gmane.comp.emulators.qemu/213690
> http://thread.gmane.org/gmane.comp.bios.tianocore.devel/2781/focus=2798
> 
> ("Please coordinate" I guess :))

Ooh, shiny. Yeah, when I get back to actually working on it rather than
just heckling, I'll make sure I do :)

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Thu, May 30, 2013 at 01:19:18PM +0100, David Woodhouse wrote:
> Yeah, but if we're shoving a lot of hardware-specific ACPI table
> generation into the guest's firmware, instead of just doing it on the
> qemu side where a number of us seem to think it belongs,

Hopefully this is not yet set in stone.

> then there *is*
> a benefit to using Coreboot. When stuff changes on the qemu side and we
> have to update the table generation to match, you end up having to
> update just the Coreboot package, and *not* having to patch both SeaBIOS
> and OVMF.

We have all kind of logic in qemu. Some of it can thinkably
be moved to a separate VM - it doesn't even need to
run in the same VM as the guest - we could do it e.g. like
kvm unit-test does, with less pain than running it in firmware.
Not clear why would generating ACPI tables - which merely
fills up an array of bytes from internal QEMU
datastructures - should be the part where we start
this modularization.

-- 
MST

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/30/13 14:19, David Woodhouse wrote:

> Yeah, but if we're shoving a lot of hardware-specific ACPI table
> generation into the guest's firmware, instead of just doing it on the
> qemu side where a number of us seem to think it belongs, then there *is*
> a benefit to using Coreboot. When stuff changes on the qemu side and we
> have to update the table generation to match, you end up having to
> update just the Coreboot package, and *not* having to patch both SeaBIOS
> and OVMF.
> 
> The extra package in the distro really isn't painful to handle, and I
> suspect it would end up *reducing* the amount of work that we have to do
> to update. You update *just* Coreboot, not *both* of SeaBIOS and OVMF.

I can't deny there's logic in that, but it still feels like tying
ourselves up in some intricate bondage choreography. "We'd like to move
ACPI tables out of firmware, but we can't move them to qemu due to
project direction disagreement, so let's adopt a middleman." (I'm not
trying to denigrate coreboot -- I don't know it at all --, but
introducing it in a (granted, distro-specific) stack just for this
purpose seems quite arbitrary.)

>> If you implement a private UEFI FAT driver from scratch, or port a free
>> software FAT implementation (eg. the r/o one in grub or the r/w one in
>> mtools), you could still run into legal problems, I've been told.
> 
> That has been said, and it's been said for the FAT implementation in the
> kernel too. If a distribution is happy to ship the kernel without
> ripping out its FAT support, then I see no reason why that distribution
> wouldn't also be happy to ship a version of OVMF with a clean
> implementation of FAT support. It doesn't make sense to be happy with
> one but not the other.

Under my *personal* impression, logic and Common Law don't really mix,
especially not in the US. Still under my *personal* impression, someone
might not feel convenient suing you for redistributing code that already
exists in the upstream Linux kernel, but might happily drag you to court
for an original clean implementation, and then you can explain it's
illogical for them to do so.

The best I can do with your suggestion is to take it to our legal dept.
I would be happy to work on a new FAT driver. (I used to feel
differently earlier but I've changed my mind.)


>> We need at least one out-of-tree edk2 patch for now (from you) but
>> apparently that's no problem.
> 
> That'll get merged soon. We are working on the corresponding spec
> update...

Great news!

Thanks,
Laszlo

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Jordan Justen]

On Thu, May 30, 2013 at 5:19 AM, David Woodhouse <dwmw2@infradead.org> wrote:
> On Thu, 2013-05-30 at 13:13 +0200, Laszlo Ersek wrote:
>> Where is CorebootPkg available from?
>
> https://github.com/pgeorgi/edk2/tree/coreboot-pkg

Is the license on this actually BSD as the License.txt indicates?

Is this planned to be upstreamed?

Does this support UEFI variables?

Does this support UEFI IA32 / X64?

>> > And it helps to dispel the stupid misconception in some quarters that
>> > Coreboot *competes* with UEFI and thus cannot possibly be supported
>> > because helping something that competes with UEFI would be bad.

Coreboot and EDK II both provide a good infrastructure for
initializing hardware. So, they compete on that point.

Coreboot then focuses on booting coreboot payloads, while EDK II
focuses on UEFI support. On that point they don't compete, but the
focus is different.

Of course, you can build a layer of EDK II => Coreboot payload
support, or Coreboot => EDK II (CorebootPkg, I guess?), but the match
will not be perfect. (That is not to say it can't work.)

>> I'm not sure who do you mean by "some quarters", but for some
>> distributions Coreboot would be yet another component (package) to
>> support, for no obvious benefit.
>>
>> (Gerd said it better than I possibly could:
>> <http://thread.gmane.org/gmane.comp.bios.coreboot.seabios/5685/focus=5705>.)
>
> Yeah, but if we're shoving a lot of hardware-specific ACPI table
> generation into the guest's firmware, instead of just doing it on the
> qemu side where a number of us seem to think it belongs, then there *is*
> a benefit to using Coreboot. When stuff changes on the qemu side and we
> have to update the table generation to match, you end up having to
> update just the Coreboot package, and *not* having to patch both SeaBIOS
> and OVMF.

I think ACPI table generation lives in firmware on real products,
because on real products the firmware is the point that best
understands the actual hardware layout for the machine. In qemu, I
would say that qemu best knows the hardware layout, given that the
firmware is generally a slightly separate project from qemu.

I don't think adding a coreboot layer into the picture helps, if it
brings along the coreboot payload boot interface as a requirement.

Then again, I don't really understand how firmware could be swapped
out in this case. What would -bios do? How would the coreboot ACPI
shim layer be specified to qemu?

-Jordan

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/30/13 18:20, Jordan Justen wrote:

> I think ACPI table generation lives in firmware on real products,
> because on real products the firmware is the point that best
> understands the actual hardware layout for the machine. In qemu, I
> would say that qemu best knows the hardware layout, given that the
> firmware is generally a slightly separate project from qemu.
> 
> I don't think adding a coreboot layer into the picture helps, if it
> brings along the coreboot payload boot interface as a requirement.
> 
> Then again, I don't really understand how firmware could be swapped
> out in this case. What would -bios do? How would the coreboot ACPI
> shim layer be specified to qemu?

I guess -bios would load coreboot. Coreboot would siphon the data
necessary for ACPI table building through the current (same) fw_cfg
bottleneck, build the tables, load the boot firmware (SeaBIOS or OVMF or
something else -- not sure how to configure that), and pass down the
tables to the firmware (through a now unspecified interface -- perhaps
the tables could even be installed at this point). This could introduce
another interface (fw_cfg+something rather than just fw_cfg), but ACPI
table preparation would be concentrated in one project.

I guess.

Laszlo

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Jordan Justen]

On Thu, May 30, 2013 at 9:41 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> On 05/30/13 18:20, Jordan Justen wrote:
>> I think ACPI table generation lives in firmware on real products,
>> because on real products the firmware is the point that best
>> understands the actual hardware layout for the machine. In qemu, I
>> would say that qemu best knows the hardware layout, given that the
>> firmware is generally a slightly separate project from qemu.
>>
>> I don't think adding a coreboot layer into the picture helps, if it
>> brings along the coreboot payload boot interface as a requirement.
>>
>> Then again, I don't really understand how firmware could be swapped
>> out in this case. What would -bios do? How would the coreboot ACPI
>> shim layer be specified to qemu?
>
> I guess -bios would load coreboot. Coreboot would siphon the data
> necessary for ACPI table building through the current (same) fw_cfg
> bottleneck, build the tables, load the boot firmware (SeaBIOS or OVMF or
> something else -- not sure how to configure that), and pass down the
> tables to the firmware (through a now unspecified interface -- perhaps
> the tables could even be installed at this point). This could introduce
> another interface (fw_cfg+something rather than just fw_cfg), but ACPI
> table preparation would be concentrated in one project.
>
> I guess.

For reference, I believe that both Xen and virtualbox build ACPI table
in the VMM rather than firmware. They both dump the tables into the
0xe000 segment (yuck) where firmware finds and publishes it to the OS.
I think fw-cfg would be a reasonable alternative to this for
communicating the tables.

-Jordan

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/30/13 18:57, Jordan Justen wrote:
> On Thu, May 30, 2013 at 9:41 AM, Laszlo Ersek <lersek@redhat.com> wrote:
>> On 05/30/13 18:20, Jordan Justen wrote:
>>> I think ACPI table generation lives in firmware on real products,
>>> because on real products the firmware is the point that best
>>> understands the actual hardware layout for the machine. In qemu, I
>>> would say that qemu best knows the hardware layout, given that the
>>> firmware is generally a slightly separate project from qemu.
>>>
>>> I don't think adding a coreboot layer into the picture helps, if it
>>> brings along the coreboot payload boot interface as a requirement.
>>>
>>> Then again, I don't really understand how firmware could be swapped
>>> out in this case. What would -bios do? How would the coreboot ACPI
>>> shim layer be specified to qemu?
>>
>> I guess -bios would load coreboot. Coreboot would siphon the data
>> necessary for ACPI table building through the current (same) fw_cfg
>> bottleneck, build the tables, load the boot firmware (SeaBIOS or OVMF or
>> something else -- not sure how to configure that), and pass down the
>> tables to the firmware (through a now unspecified interface -- perhaps
>> the tables could even be installed at this point). This could introduce
>> another interface (fw_cfg+something rather than just fw_cfg), but ACPI
>> table preparation would be concentrated in one project.
>>
>> I guess.
> 
> For reference, I believe that both Xen and virtualbox build ACPI table
> in the VMM rather than firmware. They both dump the tables into the
> 0xe000 segment (yuck) where firmware finds and publishes it to the OS.
> I think fw-cfg would be a reasonable alternative to this for
> communicating the tables.

I think Xen uses a separate utility called "hvmloader" that runs in the
domU.

- http://thread.gmane.org/gmane.comp.bios.coreboot.seabios/5453/focus=5668
- http://xenbits.xen.org/gitweb/?p=xen.git;a=tree;f=tools/firmware/hvmloader
- http://thread.gmane.org/gmane.comp.bios.coreboot.seabios/6255/focus=110562

Laszlo

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Thu, May 30, 2013 at 09:57:10AM -0700, Jordan Justen wrote:
> On Thu, May 30, 2013 at 9:41 AM, Laszlo Ersek <lersek@redhat.com> wrote:
> > On 05/30/13 18:20, Jordan Justen wrote:
> >> I think ACPI table generation lives in firmware on real products,
> >> because on real products the firmware is the point that best
> >> understands the actual hardware layout for the machine. In qemu, I
> >> would say that qemu best knows the hardware layout, given that the
> >> firmware is generally a slightly separate project from qemu.
> >>
> >> I don't think adding a coreboot layer into the picture helps, if it
> >> brings along the coreboot payload boot interface as a requirement.
> >>
> >> Then again, I don't really understand how firmware could be swapped
> >> out in this case. What would -bios do? How would the coreboot ACPI
> >> shim layer be specified to qemu?
> >
> > I guess -bios would load coreboot. Coreboot would siphon the data
> > necessary for ACPI table building through the current (same) fw_cfg
> > bottleneck, build the tables, load the boot firmware (SeaBIOS or OVMF or
> > something else -- not sure how to configure that), and pass down the
> > tables to the firmware (through a now unspecified interface -- perhaps
> > the tables could even be installed at this point). This could introduce
> > another interface (fw_cfg+something rather than just fw_cfg), but ACPI
> > table preparation would be concentrated in one project.
> >
> > I guess.
> 
> For reference, I believe that both Xen and virtualbox build ACPI table
> in the VMM rather than firmware. They both dump the tables into the
> 0xe000 segment (yuck) where firmware finds and publishes it to the OS.
> I think fw-cfg would be a reasonable alternative to this for
> communicating the tables.
> 
> -Jordan

Want to review/ack the patches I sent? That's exactly what they do.

-- 
MST

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

  Hi,

> I guess -bios would load coreboot. Coreboot would siphon the data
> necessary for ACPI table building through the current (same) fw_cfg
> bottleneck, build the tables,

Yes.

> load the boot firmware (SeaBIOS or OVMF or
> something else -- not sure how to configure that),

The coreboot rom has named sections (this is called cbfs which stands
for coreboot filesystem IIRC):

rincewind kraxel ~# cbfstool /usr/share/coreboot.git/bios.bin print
bios.bin: 256 kB, bootblocksize 848, romsize 262144, offset 0x0
alignment: 64 bytes

Name                           Offset     Type         Size
cmos_layout.bin                0x0        cmos_layout  1160
fallback/romstage              0x4c0      stage        14419
fallback/coreboot_ram          0x3d80     stage        37333
config                         0xcfc0     raw          2493
fallback/payload               0xd9c0     payload      56969
vgabios/sgabios                0x1b8c0    raw          4096
(empty)                        0x1c900    null         144216

where "fallback/payload" is seabios.

> and pass down the
> tables to the firmware (through a now unspecified interface -- perhaps
> the tables could even be installed at this point).

As far I know coreboot can add more stuff such as acpi tables to cbfs at
runtime and seabios able to access cbfs too and pull informations from
coreboot that way.

HTH,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Peter Stuge]

Gerd Hoffmann wrote:
> > and pass down the
> > tables to the firmware (through a now unspecified interface -- perhaps
> > the tables could even be installed at this point).
> 
> As far I know coreboot can add more stuff such as acpi tables to cbfs at
> runtime and seabios able to access cbfs too and pull informations from
> coreboot that way.

Only a minor correction - cbfs is the flash image, which so far
doesn't really change at runtime. Stuff added at runtime goes into
"coreboot tables" which is a coreboot-specified data structure which
SeaBIOS finds and uses to know things like the memory map.

When using coreboot+SeaBIOS on real hardware, ACPI tables are built
and put in place by coreboot, and never modified by SeaBIOS AFAIK.


//Peter

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Jordan Justen]

On Fri, May 31, 2013 at 2:32 AM, Gerd Hoffmann <kraxel@redhat.com> wrote:
>   Hi,
>
>> I guess -bios would load coreboot. Coreboot would siphon the data
>> necessary for ACPI table building through the current (same) fw_cfg
>> bottleneck, build the tables,
>
> Yes.

So, this is really about making coreboot+seabios the default QEMU
firmware, and making seabios depend on being a coreboot payload?

>> load the boot firmware (SeaBIOS or OVMF or
>> something else -- not sure how to configure that),

It wouldn't be loading OVMF. It would be loading CorebootPkg.

OVMF is a better sample platform for EDK II since it shows a more
realistic view of what an EDK II based platform looks like on real
hardware.

Thus, if the ACPI tables are just being added to a new coreboot layer
with coreboot becoming the default QEMU firmware, then it doesn't help
OVMF (or other non-coreboot payloads).

Well, it could if the table code was BSD licensed, but only so we
could then merge them into OVMF. Then again, why not just provide a
set of suitably licensed ACPI source files within the QEMU tree that
firmware projects could use? QEMU doesn't necessarily need to
build/link them, or attempt to communicate them at runtime.

-Jordan

> The coreboot rom has named sections (this is called cbfs which stands
> for coreboot filesystem IIRC):
>
> rincewind kraxel ~# cbfstool /usr/share/coreboot.git/bios.bin print
> bios.bin: 256 kB, bootblocksize 848, romsize 262144, offset 0x0
> alignment: 64 bytes
>
> Name                           Offset     Type         Size
> cmos_layout.bin                0x0        cmos_layout  1160
> fallback/romstage              0x4c0      stage        14419
> fallback/coreboot_ram          0x3d80     stage        37333
> config                         0xcfc0     raw          2493
> fallback/payload               0xd9c0     payload      56969
> vgabios/sgabios                0x1b8c0    raw          4096
> (empty)                        0x1c900    null         144216
>
> where "fallback/payload" is seabios.
>
>> and pass down the
>> tables to the firmware (through a now unspecified interface -- perhaps
>> the tables could even be installed at this point).
>
> As far I know coreboot can add more stuff such as acpi tables to cbfs at
> runtime and seabios able to access cbfs too and pull informations from
> coreboot that way.
>
> HTH,
>   Gerd
>
>

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

On 06/01/13 01:01, Jordan Justen wrote:
> On Fri, May 31, 2013 at 2:32 AM, Gerd Hoffmann <kraxel@redhat.com> wrote:
>>   Hi,
>>
>>> I guess -bios would load coreboot. Coreboot would siphon the data
>>> necessary for ACPI table building through the current (same) fw_cfg
>>> bottleneck, build the tables,
>>
>> Yes.
> 
> So, this is really about making coreboot+seabios the default QEMU
> firmware, and making seabios depend on being a coreboot payload?

I still think it's better to simply have qemu generate the acpi tables,
but if that isn't going to be accepted we should seriously consider &
evaluate switching to coreboot.

>>> load the boot firmware (SeaBIOS or OVMF or
>>> something else -- not sure how to configure that),
> 
> It wouldn't be loading OVMF. It would be loading CorebootPkg.

Yep.  Some OVMF bits would be needed though (virtio drivers, qemu boot
priority support, ...).

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Thu, May 30, 2013 at 09:20:42AM -0700, Jordan Justen wrote:
> On Thu, May 30, 2013 at 5:19 AM, David Woodhouse <dwmw2@infradead.org> wrote:
> > On Thu, 2013-05-30 at 13:13 +0200, Laszlo Ersek wrote:
> >> Where is CorebootPkg available from?
> >
> > https://github.com/pgeorgi/edk2/tree/coreboot-pkg
> 
> Is the license on this actually BSD as the License.txt indicates?
> 
> Is this planned to be upstreamed?
> 
> Does this support UEFI variables?
> 
> Does this support UEFI IA32 / X64?
> 
> >> > And it helps to dispel the stupid misconception in some quarters that
> >> > Coreboot *competes* with UEFI and thus cannot possibly be supported
> >> > because helping something that competes with UEFI would be bad.
> 
> Coreboot and EDK II both provide a good infrastructure for
> initializing hardware. So, they compete on that point.
> 
> Coreboot then focuses on booting coreboot payloads, while EDK II
> focuses on UEFI support. On that point they don't compete, but the
> focus is different.
> 
> Of course, you can build a layer of EDK II => Coreboot payload
> support, or Coreboot => EDK II (CorebootPkg, I guess?), but the match
> will not be perfect. (That is not to say it can't work.)
> 
> >> I'm not sure who do you mean by "some quarters", but for some
> >> distributions Coreboot would be yet another component (package) to
> >> support, for no obvious benefit.
> >>
> >> (Gerd said it better than I possibly could:
> >> <http://thread.gmane.org/gmane.comp.bios.coreboot.seabios/5685/focus=5705>.)
> >
> > Yeah, but if we're shoving a lot of hardware-specific ACPI table
> > generation into the guest's firmware, instead of just doing it on the
> > qemu side where a number of us seem to think it belongs, then there *is*
> > a benefit to using Coreboot. When stuff changes on the qemu side and we
> > have to update the table generation to match, you end up having to
> > update just the Coreboot package, and *not* having to patch both SeaBIOS
> > and OVMF.
> 
> I think ACPI table generation lives in firmware on real products,
> because on real products the firmware is the point that best
> understands the actual hardware layout for the machine. In qemu, I
> would say that qemu best knows the hardware layout, given that the
> firmware is generally a slightly separate project from qemu.

Of course ACPI tables are firmware.
Please note that what my patches do is simply supply
templates for ACPI tables on a ROM device separate
from where bios code resides.
bios still tweaks them in minor ways.
I am guessing this is what happens on real hardware too:
most tables pre-generated in ROM, firmware shadows
them and tweaks them in minor ways.

> I don't think adding a coreboot layer into the picture helps, if it
> brings along the coreboot payload boot interface as a requirement.
> 
> Then again, I don't really understand how firmware could be swapped
> out in this case. What would -bios do? How would the coreboot ACPI
> shim layer be specified to qemu?
> 
> -Jordan

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 551 bytes --]

On Thu, 2013-05-30 at 09:20 -0700, Jordan Justen wrote:
> On Thu, May 30, 2013 at 5:19 AM, David Woodhouse <dwmw2@infradead.org>
> wrote:
> > On Thu, 2013-05-30 at 13:13 +0200, Laszlo Ersek wrote:
> >> Where is CorebootPkg available from?
> >
> > https://github.com/pgeorgi/edk2/tree/coreboot-pkg
> 
> Is the license on this actually BSD as the License.txt indicates?
> 
> Is this planned to be upstreamed?
> 
> Does this support UEFI variables?
> 
> Does this support UEFI IA32 / X64?

Those are questions for Patrick.

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Patrick Georgi]

Am 31.05.2013 14:09, schrieb David Woodhouse:
> On Thu, 2013-05-30 at 09:20 -0700, Jordan Justen wrote:
>> On Thu, May 30, 2013 at 5:19 AM, David Woodhouse <dwmw2@infradead.org>
>> wrote:
>>> https://github.com/pgeorgi/edk2/tree/coreboot-pkg
>> Is the license on this actually BSD as the License.txt indicates?
Yes. All code is either Stefan's or my own work or taken from Tiano and
adapted. We will probably import some libpayload code, but that is
BSD-l, too.
>> Is this planned to be upstreamed?
Yes, once it's ready.
>> Does this support UEFI variables?
Not yet, planned.
>> Does this support UEFI IA32 / X64?
Both, no mixed mode.
> Those are questions for Patrick. 
HTH,
Patrick

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> On Thu, May 23, 2013 at 03:41:32PM +0300, Michael S. Tsirkin wrote:
> > Juan is not available now, and Anthony asked for
> > agenda to be sent early.
> > So here comes:
> > 
> > Agenda for the meeting Tue, May 28:
> > 
> > - Generating acpi tables
> 
> I didn't see any meeting notes, but I thought it would be worthwhile
> to summarize the call.  This is from memory so correct me if I got
> anything wrong.
> 
> Anthony believes that the generation of ACPI tables is the task of the
> firmware.  Reasons cited include security implications of running more
> code in qemu vs the guest context, complexities in running iasl on
> big-endian machines,

Forgot to mention: my patchset actually solves this by keeping
pre-generated ACPI tables in QEMU.  This means you need to have iasl to
do ACPI development but that's nothing new.

However, generating the tables in QEMU actually opens up the possibility
of linking in a library for generating ACPI tables, if such surfaces,
and dropping the iasl dependency.

While my patchset does not do this, it's not unheard of.

This would not be practical for bios.

> possible complexity of having to regenerate
> tables on a vm reboot, overall sloppiness of doing it in QEMU.  Raised
> that QOM interface should be sufficient.
> 
> Kevin believes that the bios table code should be moved up into QEMU.
> Reasons cited include the churn rate in SeaBIOS for this QEMU feature
> (15-20% of all SeaBIOS commits since integrating with QEMU have been
> for bios tables; 20% of SeaBIOS commits in last year), complexity of
> trying to pass all the content needed to generate the tables (eg,
> device details, power tree, irq routing), complexity of scheduling
> changes across different repos and synchronizing their rollout,
> complexity of implemeting the code in both OVMF and SeaBIOS.  Kevin
> wasn't aware of a requirement to regenerate acpi tables on a vm
> reboot.
> 
> There were discussions on potentially introducing a middle component
> to generate the tables.  Coreboot was raised as a possibility, and
> David thought it would be okay to use coreboot for both OVMF and
> SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> qemu repo, is run in the guest, and generates the tables (which is
> similar to the hvmloader approach that Xen uses).
> 
> Anthony requested that patches be made that generate the ACPI tables
> in QEMU for the upcoming hotplug work, so that they could be evaluated
> to see if they truly do need to live in QEMU or if the code could live
> in the firmware.  There were no objections.
> 
> -Kevin

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Kevin O'Connor]

On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> There were discussions on potentially introducing a middle component
> to generate the tables.  Coreboot was raised as a possibility, and
> David thought it would be okay to use coreboot for both OVMF and
> SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> qemu repo, is run in the guest, and generates the tables (which is
> similar to the hvmloader approach that Xen uses).

Given the objections to implementing ACPI directly in QEMU, one
possible way forward would be to split the current SeaBIOS rom into
two roms: "qvmloader" and "seabios".  The "qvmloader" would do the
qemu specific platform init (pci init, smm init, mtrr init, bios
tables) and then load and run the regular seabios rom.  With this
split, qvmloader could be committed into the QEMU repo and maintained
there.  This would be analogous to Xen's hvmloader with the seabios
code used as a starting point to implement it.

With both the hardware implementation and acpi descriptions for that
hardware in the same source code repository, it would be possible to
implement changes to both in a single patch series.  The fwcfg entries
used to pass data between qemu and qvmloader could also be changed in
a single patch and thus those fwcfg entries would not need to be
considered a stable interface.  The qvmloader code also wouldn't need
the 16bit handlers that seabios requires and thus wouldn't need the
full complexity of the seabios build.  Finally, it's possible that
both ovmf and seabios could use a single qvmloader implementation.

On the down side, reboots can be a bit goofy today in kvm, and that
would need to be settled before something like qvmloader could be
implemented.  Also, it may be problematic to support passing of bios
tables from qvmloader to seabios for guests with only 1 meg of ram.

Thoughts?
-Kevin

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Jordan Justen]

On Thu, May 30, 2013 at 7:34 PM, Kevin O'Connor <kevin@koconnor.net> wrote:
> On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
>> There were discussions on potentially introducing a middle component
>> to generate the tables.  Coreboot was raised as a possibility, and
>> David thought it would be okay to use coreboot for both OVMF and
>> SeaBIOS.  The possibility was also raised of a "rom" that lives in the
>> qemu repo, is run in the guest, and generates the tables (which is
>> similar to the hvmloader approach that Xen uses).
>
> Given the objections to implementing ACPI directly in QEMU, one
> possible way forward would be to split the current SeaBIOS rom into
> two roms: "qvmloader" and "seabios".  The "qvmloader" would do the
> qemu specific platform init (pci init, smm init, mtrr init, bios
> tables) and then load and run the regular seabios rom.  With this
> split, qvmloader could be committed into the QEMU repo and maintained
> there.  This would be analogous to Xen's hvmloader with the seabios
> code used as a starting point to implement it.

I think hvmloader is more closely tied to Xen, than the Xen firmware.
I could be wrong, but thought it could do things like add memory to
guest machine. ?? I don't think this model is analogous to Xen's
model. I view the hvmloader as just a part of Xen. (Not part of the
'firmware' stack.)

In adding this pre-firmware firmware, wouldn't Anthony's concern of
iasl still be an issue?

Why is updating the ACPI tables in seabios viewed as such a burden?
Either qemu does it, or seabios... (And, OVMF too, but I don't think
you guys are concerned with that. :)

On the flip side, why is moving the ACPI tables to QEMU such an issue?
It seems like Xen and virtualbox both already do this. Why is running
iasl not an issue for them?

I think overall I prefer the tables being built in the firmware,
despite the extra thrash. Some things, such as the addresses where
devices are configured at are re-programmable in QEMU, so a firmware
can decide to use a different address, and thus invalidate the address
qvmloader had set in the tables.

Maybe we are doing lots of things horribly wrong in our OVMF ACPI
tables :), but I haven't seen it as much of a burden. (Of course,
Laszlo has helped out with many of the ACPI changes in OVMF, so his
opinion should be taken into consideration too. :)

-Jordan

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Peter Stuge]

Kevin O'Connor wrote:
> one possible way forward would be to split the current SeaBIOS rom
> into two roms: "qvmloader" and "seabios".  The "qvmloader" would do
> the qemu specific platform init (pci init, smm init, mtrr init, bios
> tables) and then load and run the regular seabios rom.

qvmloader sounds a lot like coreboot.


> qvmloader could be committed into the QEMU repo and maintained there.

If QEMU really doesn't want anything besides quacking like a PC with
BIOS or UEFI (such as quacking like a PC *without* a particular
firmware) it makes perfect sense to me to put the complete firmware
code into the QEMU repo and never reuse anything else. After all,
that's how the proprietary firmware products are managed.


Jordan Justen wrote:
> Why is updating the ACPI tables in seabios viewed as such a burden?

I don't know about burden but to me it just doesn't make any sense
to generate ACPI in one component (SeaBIOS) based on configuration
for another component (QEMU).

ACPI bytes are obviously a function of QEMU configuration. QEMU
configuration can be changed through a great many channels, so it
makes sense to me that QEMU itself would take care of generating
correct ACPI, rather than exporting it's own data structures and
pushing the ACPI problem onto the firmware, especially considering
the desire for multiple independent firmware implementations.

There's some code for dynamic ACPI generation in coreboot already,
maybe that can be reused in QEMU to save some effort..


> On the flip side, why is moving the ACPI tables to QEMU such an issue?

Maybe because it is such a steaming pile that even the place where it
belongs doesn't really want it..


> I think overall I prefer the tables being built in the firmware,
> despite the extra thrash.

That doesn't make sense to me. :\

Keep in mind: there is firmware and there is firmware..


> Some things, such as the addresses where devices are configured at
> are re-programmable in QEMU, so a firmware can decide to use a
> different address, and thus invalidate the address qvmloader had
> set in the tables.

..there is now talk about a first-stage firmware (qvmloader) which
does only hardware init, and then jumps into a second-stage firmware
(SeaBIOS) which starts the operating system.

I don't expect that anyone would argue for the second-stage firmware
to generate ACPI tables if the first-stage firmware would be shared
across different second-stage implementations.

The above is by the way *exactly* the model coreboot uses since 14 years.

Please make an ernest effort to *look into and try to reuse* what *is
already there* ..

The fear of coreboot is truly amazing.


//Peter

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Gerd Hoffmann]

On 05/31/13 10:13, Peter Stuge wrote:
> Kevin O'Connor wrote:
>> one possible way forward would be to split the current SeaBIOS rom
>> into two roms: "qvmloader" and "seabios".  The "qvmloader" would do
>> the qemu specific platform init (pci init, smm init, mtrr init, bios
>> tables) and then load and run the regular seabios rom.
> 
> qvmloader sounds a lot like coreboot.

Indeed.

> ACPI bytes are obviously a function of QEMU configuration. QEMU
> configuration can be changed through a great many channels, so it
> makes sense to me that QEMU itself would take care of generating
> correct ACPI, rather than exporting it's own data structures and
> pushing the ACPI problem onto the firmware, especially considering
> the desire for multiple independent firmware implementations.

Can't agree more.

I still think the best solution is to have qemu generate the acpi tables
and all firmware can just grab them.

Second best option would be to have coreboot generate them and
everything else go on top of coreboot then.

Third best option is to duplicate the acpi generation code in all
firmware variants (this is what we have today).

IMO qvmloader would be even worse than these three.  Writing a piece of
firmware is alot more tricky than a linux userspace app, especially in
x86 land with the funky mode switching and assembler modes.

cheers,
  Gerd

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 10:13, Peter Stuge wrote:

> ACPI bytes are obviously a function of QEMU configuration.

Precisely!

When we evaluate that (mathematical-sense) function in boot firmware, we
need to retrieve the function's arguments. Those arguments are bits of
QEMU configuration, as you say, and fw_cfg is extremely inconvenient for
fetching them. Whenever the domain or arity of said "mathematical"
function changes (we need more arguments, or different kinds of
arguments), we have to extend fw_cfg with yet another ad-hoc key or file
entry.

The set of arguments going into ACPI tables *is* ad-hoc and arbitrary,
there's nothing to do about it. But at least we shouldn't impede the
retrieval of said arguments with artificial obstacles, such as
half-assedly serializing them over fw_cfg (and not documenting them,
naturally). In qemu the entire pool of arguments, current and future,
would be at just an arm's length, in immediately consumable form.

I've said the same about the acpi_build_madt() prototype that was
proposed for qemu:
<http://thread.gmane.org/gmane.comp.emulators.qemu/207171/focus=208719>.

Thanks,
Laszlo

Re: [Qemu-devel] [SeaBIOS] KVM call agenda for 2013-05-28

[Kevin O'Connor]

On Fri, May 31, 2013 at 10:13:34AM +0200, Peter Stuge wrote:
> Kevin O'Connor wrote:
> > one possible way forward would be to split the current SeaBIOS rom
> > into two roms: "qvmloader" and "seabios".  The "qvmloader" would do
> > the qemu specific platform init (pci init, smm init, mtrr init, bios
> > tables) and then load and run the regular seabios rom.
> 
> qvmloader sounds a lot like coreboot.

Agreed.  I don't much like the qvmloader idea.  I did want to open up
discussion on the possibility, however.  The only advantage it has
over coreboot is that it could reasonably live in the qemu repo, and I
do think that the hardware descriptions should like in the same code
repo as the hardware implementation.

-Kevin

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 09:09, Jordan Justen wrote:

> Why is updating the ACPI tables in seabios viewed as such a burden?
> Either qemu does it, or seabios... (And, OVMF too, but I don't think
> you guys are concerned with that. :)

I am :)

> On the flip side, why is moving the ACPI tables to QEMU such an issue?
> It seems like Xen and virtualbox both already do this. Why is running
> iasl not an issue for them?

I think something was mentioned about iasl having problems on BE
machines? I could be easily wrong but I *guess* qemu's hosts x targets
(emulate what on what) set is a proper superset of xen's and
virtualbox's. Presumably if you want to run an x86 guest on a MIPS host,
and also want to build qemu on the same MIPS (or SPARC) host, you'd have
to run iasl there too.

> Maybe we are doing lots of things horribly wrong in our OVMF ACPI
> tables :)

Impossible. :)

In earnest, I think what we have now is (mostly) correct, just not
extensive / flexible enough. No support for PCI hotplug or CPU hotplug,
none for S3 (although all of these tie into UEFI deeply), no MTRR setup,
no MPTABLE; let alone a non-PIIX chipset. (Well maybe I shouldn't lump
these under the "ACPI umbrella".)

> but I haven't seen it as much of a burden. (Of course,
> Laszlo has helped out with many of the ACPI changes in OVMF, so his
> opinion should be taken into consideration too. :)

It hasn't been a "burden" in the sense of me not liking the activity; I
actually like fiddling with knobs. It has certainly been extra work to
bring OVMF's ACPI tables closer to SeaBIOS's functionality / flexibility
(and we still lag behind it quite.).

Due to licensing differences I can't just port code from SeaBIOS to OVMF
(and I never have without explicit permission), so it's been a lot of
back and forth with acpidump / iasl -d in guests (massage OVMF, boot
guest, check guest dmesg / lspci, dump tables, compare, repeat), brain
picking colleagues, the ACPI and PIIX specs and so on. I have a page on
the RH intranet dedicated to this. When something around these parts is
being changed (or looks like it could be changed) in SeaBIOS, or between
qemu and SeaBIOS, I always must be alert and consider reimplementing it
in, or porting it with permission to, OVMF. (Most recent example:
pvpanic device -- currently only in SeaBIOS.)

It worries me that if I slack off, or am busy with something else, or
simply don't notice, then the gap will widen again. I appreciate
learning a bunch about ACPI, and don't mind the days of work that went
into some of my simple-looking ACPI patches for OVMF, but had the tables
come from a common (programmatic) source, none of this would have been
an issue, and I wouldn't have felt even occasionally that ACPI patches
for OVMF were both duplicate work *and* futile (considering how much
ahead SeaBIOS was).

I don't mind reimplementing stuff, or porting it with permission, going
forward, but the sophisticated parts in SeaBIOS are a hard nut. For
example I'll never be able to auto-extract offsets from generated AML
and patch the AML using those offsets; the edk2 build tools (a project
separate from edk2) don't support this, and it takes several months to
get a thing as simple as gcc-47 build flags into edk2-buildtools.

Instead I have to write template ASL, compile it to AML, hexdump the
result, verify it against the AML grammar in the ACPI spec (offsets
aren't obvious, BytePrefix and friends are a joy), define & initialize a
packed struct or array in OVMF, and patch the template AML using fixed
field names or array subscripts. Workable, but dog slow. If the ACPI
payload came from up above, we might be as well provided with a list of
(canonical name, offset, size) triplets, and could perhaps blindly patch
the contents. (Not unlike Michael's linker code for connecting tables
into a hierarchy.)

AFAIK most recently iasl got built-in support for offset extraction (and
in the process the current SeaBIOS build method was broken...), so that
part might get easier in the future.

Oh well it's Friday, sorry about this rant! :) I'll happily do what I
can in the current status quo, but frequently, it won't amount to much.

Thanks,
Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Laszlo Ersek <lersek@redhat.com> writes:

> On 05/31/13 09:09, Jordan Justen wrote:
>
> Due to licensing differences I can't just port code from SeaBIOS to
> OVMF

<soapbox>

Fork OVMF, drop the fat module, and just add GPL code.  It's an easily
solvable problem.

Rewriting BSD implementations of everything is silly.  Every other
vendor that uses TianoCore has a proprietary fork.  Maintaining a GPL
fork seems just as reasonable.

</soapbox>

Regards,

Anthony Liguori

> (and I never have without explicit permission), so it's been a lot of
> back and forth with acpidump / iasl -d in guests (massage OVMF, boot
> guest, check guest dmesg / lspci, dump tables, compare, repeat), brain
> picking colleagues, the ACPI and PIIX specs and so on. I have a page on
> the RH intranet dedicated to this. When something around these parts is
> being changed (or looks like it could be changed) in SeaBIOS, or between
> qemu and SeaBIOS, I always must be alert and consider reimplementing it
> in, or porting it with permission to, OVMF. (Most recent example:
> pvpanic device -- currently only in SeaBIOS.)
>
> It worries me that if I slack off, or am busy with something else, or
> simply don't notice, then the gap will widen again. I appreciate
> learning a bunch about ACPI, and don't mind the days of work that went
> into some of my simple-looking ACPI patches for OVMF, but had the tables
> come from a common (programmatic) source, none of this would have been
> an issue, and I wouldn't have felt even occasionally that ACPI patches
> for OVMF were both duplicate work *and* futile (considering how much
> ahead SeaBIOS was).
>
> I don't mind reimplementing stuff, or porting it with permission, going
> forward, but the sophisticated parts in SeaBIOS are a hard nut. For
> example I'll never be able to auto-extract offsets from generated AML
> and patch the AML using those offsets; the edk2 build tools (a project
> separate from edk2) don't support this, and it takes several months to
> get a thing as simple as gcc-47 build flags into edk2-buildtools.
>
> Instead I have to write template ASL, compile it to AML, hexdump the
> result, verify it against the AML grammar in the ACPI spec (offsets
> aren't obvious, BytePrefix and friends are a joy), define & initialize a
> packed struct or array in OVMF, and patch the template AML using fixed
> field names or array subscripts. Workable, but dog slow. If the ACPI
> payload came from up above, we might be as well provided with a list of
> (canonical name, offset, size) triplets, and could perhaps blindly patch
> the contents. (Not unlike Michael's linker code for connecting tables
> into a hierarchy.)
>
> AFAIK most recently iasl got built-in support for offset extraction (and
> in the process the current SeaBIOS build method was broken...), so that
> part might get easier in the future.
>
> Oh well it's Friday, sorry about this rant! :) I'll happily do what I
> can in the current status quo, but frequently, it won't amount to much.
>
> Thanks,
> Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Fri, 2013-05-31 at 08:04 -0500, Anthony Liguori wrote:
> 
> <soapbox>
> 
> Fork OVMF, drop the fat module, and just add GPL code.  It's an easily
> solvable problem.

Heh. Actually it doesn't need to be a fork. It's modular, and the FAT
driver is just a single module. Which is actually included in *binary*
form in the EDK2 repository, I believe, and its source code is
elsewhere.

We could happily make a GPL¹ or LGPL implementation of a FAT module and
build our OVMF with that instead, and we wouldn't need to fork OVMF at
all.

-- 
dwmw2

¹ If it's GPL, of course, then we mustn't include any *other* binary
blobs in our OVMF build. But the whole point in this conversation is
that we don't *want* to do that. So that's fine.


[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 16:08, David Woodhouse wrote:
> On Fri, 2013-05-31 at 08:04 -0500, Anthony Liguori wrote:
>>
>> <soapbox>
>>
>> Fork OVMF, drop the fat module, and just add GPL code.  It's an easily
>> solvable problem.
> 
> Heh. Actually it doesn't need to be a fork. It's modular, and the FAT
> driver is just a single module. Which is actually included in *binary*
> form in the EDK2 repository, I believe, and its source code is
> elsewhere.

Correct.

> We could happily make a GPL¹ or LGPL implementation of a FAT module and
> build our OVMF with that instead, and we wouldn't need to fork OVMF at
> all.

Yes, that's one plan, *if* someone can sort out, or is willing to
shoulder, the perhaps illogical but still worrisome surroundings of
FatPkg / FatBinPkg.

(I don't intend to spread FUD!)

For example, if your employer authorizes you to implement GplFatPkg from
scratch, and distribute it as an external module, I -- as someone
without any education in law though -- will give you a standing ovation
and buy you a case of beer at KVM Forum 2013. Deal? :)

(You proved to have great leverage by getting the efi compat table
extended, so... :))

> ¹ If it's GPL, of course, then we mustn't include any *other* binary
> blobs in our OVMF build. But the whole point in this conversation is
> that we don't *want* to do that. So that's fine.

Right. Eg. Shell1 is embedded as a pre-built binary, but that's just
"convenience", you can build the in-tree Shell2 from source afresh and
embed that instead (and ship its source too).

Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

David Woodhouse <dwmw2@infradead.org> writes:

> On Fri, 2013-05-31 at 08:04 -0500, Anthony Liguori wrote:
>> 
>> <soapbox>
>> 
>> Fork OVMF, drop the fat module, and just add GPL code.  It's an easily
>> solvable problem.
>
> Heh. Actually it doesn't need to be a fork. It's modular, and the FAT
> driver is just a single module. Which is actually included in *binary*
> form in the EDK2 repository, I believe, and its source code is
> elsewhere.
>
> We could happily make a GPL¹ or LGPL implementation of a FAT module and
> build our OVMF with that instead, and we wouldn't need to fork OVMF at
> all.

So can't we have GPL virtio modules too?  I don't think there's any
problem there except for the FAT module.

I would propose more of a virtual fork.  It could consist of a git repo with
the GPL modules + a submodule for edk2.  Ideally, there would be no need
to actually fork edk2.

My assumption is that edk2 won't take GPL code.  But does ovmf really
need to live in the edk2 tree?

If we're going to get serious about supporting OVMF, it we need
something that isn't proprietary.

> -- 
> dwmw2
>
> ¹ If it's GPL, of course, then we mustn't include any *other* binary
> blobs in our OVMF build. But the whole point in this conversation is
> that we don't *want* to do that. So that's fine.

It's even more fundamental.  OVMF as a whole (at least in it's usable
form) is not Open Source.  Without even tackling the issue of GPL code
sharing, that is a fundamental problem that needs to be solved if we're
going to serious about making changes to QEMU to support it.

I think solving the general problem will also enable GPL code sharing
though.

Regards,

Anthony Liguori

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

On Fri, 2013-05-31 at 10:43 -0500, Anthony Liguori wrote:
> It's even more fundamental.  OVMF as a whole (at least in it's usable
> form) is not Open Source. 

The FAT module is required to make EDK2 usable, and yes, that's not Open
Source. So in a sense you're right.

But we're talking here about *replacing* the FAT module with something
that *is* open source. And the FAT module isn't a fundamental part of
EDK2; it's just an optional module that happens to be bundled with the
repository.

So I think you're massively overstating the issue. OVMF/EDK2 *is* Open
Source, and replacing the FAT module really isn't that hard.

We can only bury our heads in the sand and ship qemu with
non-EFI-capable firmware for so long...

I *know* there's more work to be done. We have SeaBIOS-as-CSM, Jordan
has mostly sorted out the NV variable storage, and now the FAT issue is
coming up to the top of the pile. But we aren't far from the point where
we can realistically say that we want the Open Source OVMF to be the
default firmware shipped with qemu.

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 18:33, David Woodhouse wrote:
> On Fri, 2013-05-31 at 10:43 -0500, Anthony Liguori wrote:
>> It's even more fundamental.  OVMF as a whole (at least in it's usable
>> form) is not Open Source. 
> 
> The FAT module is required to make EDK2 usable, and yes, that's not Open
> Source. So in a sense you're right.
> 
> But we're talking here about *replacing* the FAT module with something
> that *is* open source. And the FAT module isn't a fundamental part of
> EDK2; it's just an optional module that happens to be bundled with the
> repository.

Yes. *Some* FAT module is a hard requirement.

> So I think you're massively overstating the issue. OVMF/EDK2 *is* Open
> Source,

Agreed,

> and replacing the FAT module really isn't that hard.

technically it's not hard; for a seasoned file system developer (which
I'm not, of course), even possibly missing UEFI bits, it should be
children's play actually, considering the high quality of UEFI
documentation and the responsiveness of edk2-devel.

Considering US legal climate however, it appears *extremely* hard to
replace the FAT module, in my unwashed personal opinion.

Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

David Woodhouse <dwmw2@infradead.org> writes:

> On Fri, 2013-05-31 at 10:43 -0500, Anthony Liguori wrote:
>> It's even more fundamental.  OVMF as a whole (at least in it's usable
>> form) is not Open Source. 
>
> The FAT module is required to make EDK2 usable, and yes, that's not Open
> Source. So in a sense you're right.
>
> But we're talking here about *replacing* the FAT module with something
> that *is* open source. And the FAT module isn't a fundamental part of
> EDK2; it's just an optional module that happens to be bundled with the
> repository.

So *if* we replace the FAT module *and* that replacement was GPL, would
there be any objects to having more GPL modules for things like virtio,
ACPI, etc?

And would that be doable in the context of OVMF or would another project
need to exist for this purpose?

> So I think you're massively overstating the issue. OVMF/EDK2 *is* Open
> Source, and replacing the FAT module really isn't that hard.
>
> We can only bury our heads in the sand and ship qemu with
> non-EFI-capable firmware for so long...

Which is why I think we need to solve the real problem here.

> I *know* there's more work to be done. We have SeaBIOS-as-CSM, Jordan
> has mostly sorted out the NV variable storage, and now the FAT issue is
> coming up to the top of the pile. But we aren't far from the point where
> we can realistically say that we want the Open Source OVMF to be the
> default firmware shipped with qemu.

Yes, that's why I'm raising this now.  We all knew that we'd have to
talk about this eventually.

Regards,

Anthony Liguori

>
> -- 
> dwmw2

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Paolo Bonzini]

Il 31/05/2013 19:06, Anthony Liguori ha scritto:
> David Woodhouse <dwmw2@infradead.org> writes:
> 
>> On Fri, 2013-05-31 at 10:43 -0500, Anthony Liguori wrote:
>>> It's even more fundamental.  OVMF as a whole (at least in it's usable
>>> form) is not Open Source. 
>>
>> The FAT module is required to make EDK2 usable, and yes, that's not Open
>> Source. So in a sense you're right.
>>
>> But we're talking here about *replacing* the FAT module with something
>> that *is* open source. And the FAT module isn't a fundamental part of
>> EDK2; it's just an optional module that happens to be bundled with the
>> repository.
> 
> So *if* we replace the FAT module *and* that replacement was GPL, would
> there be any objects to having more GPL modules for things like virtio,
> ACPI, etc?
> 
> And would that be doable in the context of OVMF or would another project
> need to exist for this purpose?

I don't think it would be doable in TianoCore.  I think it would end up
either in distros, or in QEMU.

A separate question is whether OVMF makes more sense as part of
TianoCore or rather as part of QEMU.  With 75% of the free hypervisors
now reunited under the same source repository, the balance is tilting...

Paolo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Paolo Bonzini <pbonzini@redhat.com> writes:

> Il 31/05/2013 19:06, Anthony Liguori ha scritto:
>> David Woodhouse <dwmw2@infradead.org> writes:
>> 
>>> On Fri, 2013-05-31 at 10:43 -0500, Anthony Liguori wrote:
>>>> It's even more fundamental.  OVMF as a whole (at least in it's usable
>>>> form) is not Open Source. 
>>>
>>> The FAT module is required to make EDK2 usable, and yes, that's not Open
>>> Source. So in a sense you're right.
>>>
>>> But we're talking here about *replacing* the FAT module with something
>>> that *is* open source. And the FAT module isn't a fundamental part of
>>> EDK2; it's just an optional module that happens to be bundled with the
>>> repository.
>> 
>> So *if* we replace the FAT module *and* that replacement was GPL, would
>> there be any objects to having more GPL modules for things like virtio,
>> ACPI, etc?
>> 
>> And would that be doable in the context of OVMF or would another project
>> need to exist for this purpose?
>
> I don't think it would be doable in TianoCore.  I think it would end up
> either in distros, or in QEMU.

As I think more about it, I think forking edk2 is inevitable.  We need a
clean repo that doesn't include the proprietary binaries.  I doubt
upstream edk2 is willing to remove the binaries.

But this can be quite simple using a combination of git-svn and a
rewriting script.  We did exactly this to pull out the VGABios from
Bochs and remove the binaries associated with it.  It's 100% automated
and can be kept in sync via a script on qemu.org.

> A separate question is whether OVMF makes more sense as part of
> TianoCore or rather as part of QEMU.

I'm not sure if qemu.git is the right location, but we can certainly
host an ovmf.git on qemu.git that embeds the scrubbed version of
edk2.git.

Of course, this would enable us to add GPL code (including a FAT module)
to ovmf.git without any impact on upstream edk2.

> With 75% of the free hypervisors
> now reunited under the same source repository, the balance is
> tilting...

<insert evil laugh> :-)

Regards,

Anthony Liguori

>
> Paolo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Jordan Justen]

On Fri, May 31, 2013 at 11:35 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> As I think more about it, I think forking edk2 is inevitable.  We need a
> clean repo that doesn't include the proprietary binaries.  I doubt
> upstream edk2 is willing to remove the binaries.

No, probably not unless a BSD licensed alternative was available. :)

But, in thinking about what might make sense for EDK II with git, one
option that should be considered is breaking the top-level 'packages'
into separate sub-modules. I had gone so far as to start pushing repos
as sub-modules.

But, as the effort to convert EDK II to git has stalled (actually
never even thought about leaving the ground), I abandoned that
approach and went back to just mirroring one EDK II.

I could fairly easily re-enable mirror the sub-set of packages needed
for OVMF. So, in that case, the FatBinPkg sub-module could easily be
dropped from a tree.

> But this can be quite simple using a combination of git-svn and a
> rewriting script.  We did exactly this to pull out the VGABios from
> Bochs and remove the binaries associated with it.  It's 100% automated
> and can be kept in sync via a script on qemu.org.

I would love to mirror the BaseTools as a sub-package without all the
silly windows binaries... What script did you guys use?

-Jordan

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Jordan Justen <jljusten@gmail.com> writes:

> On Fri, May 31, 2013 at 11:35 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
>> As I think more about it, I think forking edk2 is inevitable.  We need a
>> clean repo that doesn't include the proprietary binaries.  I doubt
>> upstream edk2 is willing to remove the binaries.
>
> No, probably not unless a BSD licensed alternative was available. :)
>
> But, in thinking about what might make sense for EDK II with git, one
> option that should be considered is breaking the top-level 'packages'
> into separate sub-modules. I had gone so far as to start pushing repos
> as sub-modules.
>
> But, as the effort to convert EDK II to git has stalled (actually
> never even thought about leaving the ground), I abandoned that
> approach and went back to just mirroring one EDK II.
>
> I could fairly easily re-enable mirror the sub-set of packages needed
> for OVMF. So, in that case, the FatBinPkg sub-module could easily be
> dropped from a tree.
>
>> But this can be quite simple using a combination of git-svn and a
>> rewriting script.  We did exactly this to pull out the VGABios from
>> Bochs and remove the binaries associated with it.  It's 100% automated
>> and can be kept in sync via a script on qemu.org.
>
> I would love to mirror the BaseTools as a sub-package without all the
> silly windows binaries... What script did you guys use?

We did this in git pre-history, now git has a fancy git-filter-branch
command that makes it a breeze:

http://git-scm.com/book/ch6-4.html

Regards,

Anthony Liguori

>
> -Jordan

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 17:43, Anthony Liguori wrote:
> David Woodhouse <dwmw2@infradead.org> writes:
> 
>> On Fri, 2013-05-31 at 08:04 -0500, Anthony Liguori wrote:
>>>
>>> <soapbox>
>>>
>>> Fork OVMF, drop the fat module, and just add GPL code.  It's an easily
>>> solvable problem.
>>
>> Heh. Actually it doesn't need to be a fork. It's modular, and the FAT
>> driver is just a single module. Which is actually included in *binary*
>> form in the EDK2 repository, I believe, and its source code is
>> elsewhere.
>>
>> We could happily make a GPL¹ or LGPL implementation of a FAT module and
>> build our OVMF with that instead, and we wouldn't need to fork OVMF at
>> all.
> 
> So can't we have GPL virtio modules too?  I don't think there's any
> problem there except for the FAT module.

I share your assessment.

> I would propose more of a virtual fork.  It could consist of a git repo with
> the GPL modules + a submodule for edk2.  Ideally, there would be no need
> to actually fork edk2.

Indeed. edk2 is extremely modular. But in order to get a useful firmware
image ultimately, you need a FAT driver.

> My assumption is that edk2 won't take GPL code.

Correct, see eg. OvmfPkg/Contributions.txt.

Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Laszlo Ersek <lersek@redhat.com> writes:

> On 05/31/13 15:04, Anthony Liguori wrote:
>> Laszlo Ersek <lersek@redhat.com> writes:
>> 
>>> On 05/31/13 09:09, Jordan Justen wrote:
>>>
>>> Due to licensing differences I can't just port code from SeaBIOS to
>>> OVMF
>> 
>> <soapbox>
>
> :)
>
>> Fork OVMF, drop the fat module, and just add GPL code.  It's an easily
>> solvable problem.
>
> It's not optimal for the "upstream first" principle;

<still on soapbox>

OVMF is not Open Source so "upstream first" doesn't apply.  At least,
the FAT module is not Open Source.

Bullet 8 from the Open Source Definition[1]

"8. License Must Not Be Specific to a Product

The rights attached to the program must not depend on the program's
being part of a particular software distribution. If the program is
extracted from that distribution and used or distributed within the
terms of the program's license, all parties to whom the program is
redistributed should have the same rights as those that are granted in
conjunction with the original software distribution."

License from OVMF FAT module[2]:

"Additional terms: In addition to the forgoing, redistribution and use
of the code is conditioned upon the FAT 32 File System Driver and all
derivative works thereof being used for and designed only to read and/or
write to a file system that is directly managed by: Intel’s Extensible
Firmware Initiative (EFI) Specification v. 1.0 and later and/or the
Unified Extensible Firmware Interface (UEFI) Forum’s UEFI Specifications
v.2.0 and later (together the “UEFI Specifications”); only as necessary
to emulate an implementation of the UEFI Specifications; and to create
firmware, applications, utilities and/or drivers."

[1] http://opensource.org/osd-annotated
[2] http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=Edk2-fat-driver

AFAIK, for the systems that we'd actually want to use OVMF for, a FAT
module is a hard requirement.

> we'd have to
> backport upstream edk2 patches forever (there's a whole lot of edk2
> modules outside of direct OvmfPkg that get built into OVMF.fd -- OvmfPkg
> "only" customizes / cherry-picks the full edk2 tree for virtual
> machines), or to periodically rebase an ever-increasing set of patches.
>
> Independently, we need *some* FAT driver (otherwise you can't even boot
> most installer media), which is where the already discussed worries lie.
> Whatever solves this aspect is independent of forking all of edk2.

It's either Open Source or it's not.  It's currently not.  I have a hard
time sympathesizing with trying to work with a proprietary upstream.

>> Rewriting BSD implementations of everything is silly.  Every other
>> vendor that uses TianoCore has a proprietary fork.
>
> Correct, but they (presumably) keep rebasing their ever accumulating
> stuff at least on the periodically refreshed "stable edk2 subset"
> (UDK2010, which BTW doesn't include OvmfPkg). This must be horrible for
> them, but in exchange they get to remain proprietary (which may benefit
> them commercially).
>
>> Maintaining a GPL
>> fork seems just as reasonable.
>
> Perhaps; diverging from "upstream first" would hurt for certain.

Well I'm suggesting creating a real upstream (that is actually Open
Source).  Then I'm all for upstream first.

In terms of creating a FAT module, the most likely source would seem to
be the kernel code and since that's GPL, I don't think it's terribly
avoidable to end up with a GPL'd uefi implementation.

If that's inevitable, then we're wasting effort by rewriting stuff under
a BSD license.

Regards,

Anthony Liguori

>
>> </soapbox>
>
> Thanks for the suggestion :)
> Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 16:38, Anthony Liguori wrote:

> It's either Open Source or it's not.  It's currently not.

I disagree with this binary representation of Open Source or Not. If it
weren't (mostly) Open Source, how could we fork (most of) it as you're
suggesting (from the soapbox :))?

> I have a hard
> time sympathesizing with trying to work with a proprietary upstream.

My experience has been positive.

First of all, whether UEFI is a good thing or not is controversial. I
won't try to address that.

However UEFI is here to stay, machines are being shipped with it, Linux
and other OSen try to support it. Developing (or running) an OS in
combination with a specific firmware is sometimes easier / more economic
in a virtual environment, hence there should be support for qemu + UEFI.
It is this mindset that I operate in. (Oh, I also forgot to mention that
this task has been assigned to me by my superiors as well :))

Jordan, the OvmfPkg maintainer is responsive and progressive in the true
FLOSS manner (*), which was a nice surprise for a project whose coding
standards for example are made 100% after Windows source code, and whose
mailing list is mostly subscribed to by proprietary vendors. Really when
it comes to OvmfPkg patches the process follows the "normal" FLOSS
development model.

(*) Jordan, I hope this will prompt you to merge VirtioNetDxe v4 real
soon now :)

I personally think the 2-clause BSDL for 99% of the project was a very
sane and practical one from Intel et al.

FatPkg is a sad exception. One might even consider it a bad accident:
http://thread.gmane.org/gmane.comp.bios.tianocore.devel/1861/focus=1878

I have no idea how that selection process went down, but I assume if
FLOSS people had been screaming very loud at that time and had offered a
*simple* (which ext2 is not, I gather), wide-spread and unencumbered
filesystem, things would be different today.

> In terms of creating a FAT module, the most likely source would seem to
> be the kernel code and since that's GPL, I don't think it's terribly
> avoidable to end up with a GPL'd uefi implementation.
> 
> If that's inevitable, then we're wasting effort by rewriting stuff under
> a BSD license.

Please ask your employer if they'd be willing to put their name on an
original, clean-room, GPL-licensed implementation of FAT32 for UEFI.


Thus far we've been talking copyright rather than patents, but there's
also this:

http://en.wikipedia.org/wiki/FAT_filesystem#Challenge
http://en.wikipedia.org/wiki/FAT_filesystem#Patent_infringement_lawsuits

It almost doesn't matter who prevails in such a lawsuit; the
*possibility* of such a lawsuit gives people cold feet. Blame the USPTO.

Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Laszlo Ersek <lersek@redhat.com> writes:

> On 05/31/13 16:38, Anthony Liguori wrote:
>
>> It's either Open Source or it's not.  It's currently not.
>
> I disagree with this binary representation of Open Source or Not. If it
> weren't (mostly) Open Source, how could we fork (most of) it as you're
> suggesting (from the soapbox :))?
>
>> I have a hard
>> time sympathesizing with trying to work with a proprietary upstream.
>
> My experience has been positive.
>
> First of all, whether UEFI is a good thing or not is controversial. I
> won't try to address that.
>
> However UEFI is here to stay, machines are being shipped with it, Linux
> and other OSen try to support it. Developing (or running) an OS in
> combination with a specific firmware is sometimes easier / more economic
> in a virtual environment, hence there should be support for qemu + UEFI.
> It is this mindset that I operate in. (Oh, I also forgot to mention that
> this task has been assigned to me by my superiors as well :))
>
> Jordan, the OvmfPkg maintainer is responsive and progressive in the true
> FLOSS manner (*), which was a nice surprise for a project whose coding
> standards for example are made 100% after Windows source code, and whose
> mailing list is mostly subscribed to by proprietary vendors. Really when
> it comes to OvmfPkg patches the process follows the "normal" FLOSS
> development model.
>
> (*) Jordan, I hope this will prompt you to merge VirtioNetDxe v4 real
> soon now :)

(Removing seabios from the CC as we've moved far away from seabios as a topic)

Just so no one gets the wrong idea, the OVMF team is now a victim of
their own success.  I had hoped that no one would do the work necessary
to get us to the point where we had to seriously think about UEFI
support but that's where we are now :-)

> Thus far we've been talking copyright rather than patents, but there's
> also this:
>
> http://en.wikipedia.org/wiki/FAT_filesystem#Challenge
> http://en.wikipedia.org/wiki/FAT_filesystem#Patent_infringement_lawsuits
>
> It almost doesn't matter who prevails in such a lawsuit; the
> *possibility* of such a lawsuit gives people cold feet. Blame the
> USPTO.

Just to say it once so I don't have to ever say it again.

I'm not going to discuss anything relating to patents and FAT publicly.
Everyone should consult with their respective lawyers on such issues.

Copyright is straight forward.  Patents are not.

Regards,

Anthony Liguori

>
> Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Jordan Justen]

On Fri, May 31, 2013 at 7:38 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> In terms of creating a FAT module, the most likely source would seem to
> be the kernel code and since that's GPL, I don't think it's terribly
> avoidable to end up with a GPL'd uefi implementation.

Why would OpenBSD not be a potential source?

http://www.openbsd.org/cgi-bin/cvsweb/src/sys/msdosfs/

We have a half-done ext2 fs from GSoC2011 that started with OpenBSD.

https://github.com/the-ridikulus-rat/Tianocore_Ext2Pkg

> If that's inevitable, then we're wasting effort by rewriting stuff under
> a BSD license.
>
> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Jordan Justen <jljusten@gmail.com> writes:

> On Fri, May 31, 2013 at 7:38 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
>> In terms of creating a FAT module, the most likely source would seem to
>> be the kernel code and since that's GPL, I don't think it's terribly
>> avoidable to end up with a GPL'd uefi implementation.
>
> Why would OpenBSD not be a potential source?
>
> http://www.openbsd.org/cgi-bin/cvsweb/src/sys/msdosfs/

If someone is going to do it, that's fine.

But if me, it's going to be a GPL base.  Actually, enabling GPL
contributions to OVMF is a major motivating factor for me in this whole
discussion.

Regards,

Anthony Liguori

>
> We have a half-done ext2 fs from GSoC2011 that started with OpenBSD.
>
> https://github.com/the-ridikulus-rat/Tianocore_Ext2Pkg
>
>> If that's inevitable, then we're wasting effort by rewriting stuff under
>> a BSD license.
>>
>> Regards,
>>
>> Anthony Liguori

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Jordan Justen]

On Fri, May 31, 2013 at 1:27 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Jordan Justen <jljusten@gmail.com> writes:
>
>> On Fri, May 31, 2013 at 7:38 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
>>> In terms of creating a FAT module, the most likely source would seem to
>>> be the kernel code and since that's GPL, I don't think it's terribly
>>> avoidable to end up with a GPL'd uefi implementation.
>>
>> Why would OpenBSD not be a potential source?
>>
>> http://www.openbsd.org/cgi-bin/cvsweb/src/sys/msdosfs/
>
> If someone is going to do it, that's fine.
>
> But if me, it's going to be a GPL base.

Of potential modules for GPL, this wouldn't be my first choice. For
EDK II it would be nice if all the core essential pieces were BSD
licensed. This allows more flexibility for those that don't want to
use GPL.

Of course, the fact that the current FAT driver is exclusionary for
free software projects is a point that is not lost on me. I just don't
agree that the best response to this is a GPL'd FAT driver. (But, it
does seem fair. :)

> Actually, enabling GPL
> contributions to OVMF is a major motivating factor for me in this whole
> discussion.

I wouldn't mind figuring out a way to allow GPL components for people
that prefer that. EDK II has thus far not proved very welcoming on
this front. I think the main repo will remain BSD though.

I think that the sub-modules option is the best way to address this.
But, I'm not going to bother with creating the sub-module repos if no
one is going to use them. (As it was in the past.)

-Jordan

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Laszlo Ersek]

On 05/31/13 23:03, Jordan Justen wrote:

> Of course, the fact that the current FAT driver is exclusionary for
> free software projects is a point that is not lost on me. I just don't
> agree that the best response to this is a GPL'd FAT driver.

What would you suggest?

Thank you,
Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Jordan Justen]

On Fri, May 31, 2013 at 5:01 PM, Laszlo Ersek <lersek@redhat.com> wrote:
> On 05/31/13 23:03, Jordan Justen wrote:
>
>> Of course, the fact that the current FAT driver is exclusionary for
>> free software projects is a point that is not lost on me. I just don't
>> agree that the best response to this is a GPL'd FAT driver.
>
> What would you suggest?

Wasn't that a few levels up in this thread? (And properly phased in
the form of a question, no less! :)

-Jordan

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Fri, May 31, 2013 at 01:45:55PM +0200, Laszlo Ersek wrote:
> On 05/31/13 09:09, Jordan Justen wrote:
> 
> > Why is updating the ACPI tables in seabios viewed as such a burden?
> > Either qemu does it, or seabios... (And, OVMF too, but I don't think
> > you guys are concerned with that. :)
> 
> I am :)
> 
> > On the flip side, why is moving the ACPI tables to QEMU such an issue?
> > It seems like Xen and virtualbox both already do this. Why is running
> > iasl not an issue for them?
> 
> I think something was mentioned about iasl having problems on BE
> machines? I could be easily wrong but I *guess* qemu's hosts x targets
> (emulate what on what) set is a proper superset of xen's and
> virtualbox's. Presumably if you want to run an x86 guest on a MIPS host,
> and also want to build qemu on the same MIPS (or SPARC) host, you'd have
> to run iasl there too.

You guys should take a look at the patch series I posted.

That's solved there by the means of keeping iasl output in qemu git tree.
configure checks for a working iasl and enables/disables
using this pre-processed output accordingly.
Everyone developing ASL code would still need working iasl
but that's already the case today.

> > tables :)
> 
> Impossible. :)
> 
> In earnest, I think what we have now is (mostly) correct, just not
> extensive / flexible enough. No support for PCI hotplug or CPU hotplug,
> none for S3 (although all of these tie into UEFI deeply), no MTRR setup,
> no MPTABLE; let alone a non-PIIX chipset. (Well maybe I shouldn't lump
> these under the "ACPI umbrella".)
> 
> > but I haven't seen it as much of a burden. (Of course,
> > Laszlo has helped out with many of the ACPI changes in OVMF, so his
> > opinion should be taken into consideration too. :)
> 
> It hasn't been a "burden" in the sense of me not liking the activity; I
> actually like fiddling with knobs. It has certainly been extra work to
> bring OVMF's ACPI tables closer to SeaBIOS's functionality / flexibility
> (and we still lag behind it quite.).
> 
> Due to licensing differences I can't just port code from SeaBIOS to OVMF
> (and I never have without explicit permission), so it's been a lot of
> back and forth with acpidump / iasl -d in guests (massage OVMF, boot
> guest, check guest dmesg / lspci, dump tables, compare, repeat), brain
> picking colleagues, the ACPI and PIIX specs and so on. I have a page on
> the RH intranet dedicated to this. When something around these parts is
> being changed (or looks like it could be changed) in SeaBIOS, or between
> qemu and SeaBIOS, I always must be alert and consider reimplementing it
> in, or porting it with permission to, OVMF. (Most recent example:
> pvpanic device -- currently only in SeaBIOS.)
> 
> It worries me that if I slack off, or am busy with something else, or
> simply don't notice, then the gap will widen again. I appreciate
> learning a bunch about ACPI, and don't mind the days of work that went
> into some of my simple-looking ACPI patches for OVMF, but had the tables
> come from a common (programmatic) source, none of this would have been
> an issue, and I wouldn't have felt even occasionally that ACPI patches
> for OVMF were both duplicate work *and* futile (considering how much
> ahead SeaBIOS was).
> 
> I don't mind reimplementing stuff, or porting it with permission, going
> forward, but the sophisticated parts in SeaBIOS are a hard nut. For
> example I'll never be able to auto-extract offsets from generated AML
> and patch the AML using those offsets; the edk2 build tools (a project
> separate from edk2) don't support this, and it takes several months to
> get a thing as simple as gcc-47 build flags into edk2-buildtools.
> 
> Instead I have to write template ASL, compile it to AML, hexdump the
> result, verify it against the AML grammar in the ACPI spec (offsets
> aren't obvious, BytePrefix and friends are a joy), define & initialize a
> packed struct or array in OVMF, and patch the template AML using fixed
> field names or array subscripts. Workable, but dog slow. If the ACPI
> payload came from up above, we might be as well provided with a list of
> (canonical name, offset, size) triplets, and could perhaps blindly patch
> the contents. (Not unlike Michael's linker code for connecting tables
> into a hierarchy.)
> 
> AFAIK most recently iasl got built-in support for offset extraction (and
> in the process the current SeaBIOS build method was broken...), so that
> part might get easier in the future.
> 
> Oh well it's Friday, sorry about this rant! :) I'll happily do what I
> can in the current status quo, but frequently, it won't amount to much.
> 
> Thanks,
> Laszlo

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Jordan Justen]

On Sun, Jun 2, 2013 at 2:43 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Fri, May 31, 2013 at 01:45:55PM +0200, Laszlo Ersek wrote:
>> On 05/31/13 09:09, Jordan Justen wrote:
>>
>> > Why is updating the ACPI tables in seabios viewed as such a burden?
>> > Either qemu does it, or seabios... (And, OVMF too, but I don't think
>> > you guys are concerned with that. :)
>>
>> I am :)
>>
>> > On the flip side, why is moving the ACPI tables to QEMU such an issue?
>> > It seems like Xen and virtualbox both already do this. Why is running
>> > iasl not an issue for them?
>>
>> I think something was mentioned about iasl having problems on BE
>> machines? I could be easily wrong but I *guess* qemu's hosts x targets
>> (emulate what on what) set is a proper superset of xen's and
>> virtualbox's. Presumably if you want to run an x86 guest on a MIPS host,
>> and also want to build qemu on the same MIPS (or SPARC) host, you'd have
>> to run iasl there too.
>
> You guys should take a look at the patch series I posted.
>
> That's solved there by the means of keeping iasl output in qemu git tree.
> configure checks for a working iasl and enables/disables
> using this pre-processed output accordingly.
> Everyone developing ASL code would still need working iasl
> but that's already the case today.

I'm sorry the I haven't had time to review your series yet. But, from
what you saying about it in this thread, it sounds like a good plan.

-Jordan

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Anthony Liguori]

Kevin O'Connor <kevin@koconnor.net> writes:

> On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
>> There were discussions on potentially introducing a middle component
>> to generate the tables.  Coreboot was raised as a possibility, and
>> David thought it would be okay to use coreboot for both OVMF and
>> SeaBIOS.  The possibility was also raised of a "rom" that lives in the
>> qemu repo, is run in the guest, and generates the tables (which is
>> similar to the hvmloader approach that Xen uses).
>
> Given the objections to implementing ACPI directly in QEMU, one
> possible way forward would be to split the current SeaBIOS rom into
> two roms: "qvmloader" and "seabios".  The "qvmloader" would do the
> qemu specific platform init (pci init, smm init, mtrr init, bios
> tables) and then load and run the regular seabios rom.  With this
> split, qvmloader could be committed into the QEMU repo and maintained
> there.  This would be analogous to Xen's hvmloader with the seabios
> code used as a starting point to implement it.

What about a small change to the SeaBIOS build system to allow ACPI
table generation to be done via a "plugin".

This could be as simple as moving acpi.c and *.dsl into the QEMU build
tree and then having a way to point the SeaBIOS makefiles to our copy of
it.

Then the logic is maintained stays in firmware but the churn happens in
the QEMU tree instead of the SeaBIOS tree.

Regards,

Anthony Liguori

>
> With both the hardware implementation and acpi descriptions for that
> hardware in the same source code repository, it would be possible to
> implement changes to both in a single patch series.  The fwcfg entries
> used to pass data between qemu and qvmloader could also be changed in
> a single patch and thus those fwcfg entries would not need to be
> considered a stable interface.  The qvmloader code also wouldn't need
> the 16bit handlers that seabios requires and thus wouldn't need the
> full complexity of the seabios build.  Finally, it's possible that
> both ovmf and seabios could use a single qvmloader implementation.
>
> On the down side, reboots can be a bit goofy today in kvm, and that
> would need to be settled before something like qvmloader could be
> implemented.  Also, it may be problematic to support passing of bios
> tables from qvmloader to seabios for guests with only 1 meg of ram.
>
> Thoughts?
> -Kevin

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 850 bytes --]

On Fri, 2013-05-31 at 07:58 -0500, Anthony Liguori wrote:
> What about a small change to the SeaBIOS build system to allow ACPI
> table generation to be done via a "plugin".

SeaBIOS already accepts ACPI tables from Coreboot or UEFI, and queries
them to find things that it needs.

> This could be as simple as moving acpi.c and *.dsl into the QEMU build
> tree and then having a way to point the SeaBIOS makefiles to our copy
> of it.
> 
> Then the logic is maintained stays in firmware but the churn happens
> in the QEMU tree instead of the SeaBIOS tree.

Even if you get this working such that SeaBIOS and OVMF can both be
built with ACPI tables that match the last qemu you built, that doesn't
solve the issue of running a firmware that *wasn't* built to precisely
match the version of qemu you're running today.

-- 
dwmw2

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Kevin O'Connor]

On Fri, May 31, 2013 at 07:58:36AM -0500, Anthony Liguori wrote:
> Kevin O'Connor <kevin@koconnor.net> writes:
> > Given the objections to implementing ACPI directly in QEMU, one
> > possible way forward would be to split the current SeaBIOS rom into
> > two roms: "qvmloader" and "seabios".  The "qvmloader" would do the
> > qemu specific platform init (pci init, smm init, mtrr init, bios
> > tables) and then load and run the regular seabios rom.
> What about a small change to the SeaBIOS build system to allow ACPI
> table generation to be done via a "plugin".

Using a runtime plugin (eg, "qplugin") would require a more complex
handoff then qvmloader.  With qplugin, seabios would need to know what
memory qplugin is compiled to run in and make sure it didn't allocate
anything there.  Similarly, qplugin would need to not stomp on seabios
while it runs, and it would need to coordinate with seabios where to
place the final tables.  With qvmloader, there is no need to
coordinate memory addresses, so it can run anywhere, deploy the tables
in their final location, and then launch seabios.

> This could be as simple as moving acpi.c and *.dsl into the QEMU build
> tree and then having a way to point the SeaBIOS makefiles to our copy of
> it.

I don't see how that would work.  It would complicate the seabios
build (as it would require a copy of qemu source to compile), and the
resulting seabios binary would be strongly tied to the qemu version it
was compiled with and vice-versa.  This would break distro seabios
rpms.  It would also cause great pain when bisecting and would be
confusing even during regular compile/debug cycles.  Internal seabios
calls (eg, memory allocations, pci config accesses) would need to be
static interfaces, etc.

-Kevin

Re: [Qemu-devel] KVM call agenda for 2013-05-28

[Michael S. Tsirkin]

On Thu, May 30, 2013 at 10:34:26PM -0400, Kevin O'Connor wrote:
> On Tue, May 28, 2013 at 07:53:09PM -0400, Kevin O'Connor wrote:
> > There were discussions on potentially introducing a middle component
> > to generate the tables.  Coreboot was raised as a possibility, and
> > David thought it would be okay to use coreboot for both OVMF and
> > SeaBIOS.  The possibility was also raised of a "rom" that lives in the
> > qemu repo, is run in the guest, and generates the tables (which is
> > similar to the hvmloader approach that Xen uses).
> 
> Given the objections to implementing ACPI directly in QEMU,

I don't think that's a given, just yet.

So far Anthony asked to be shown the kind of project that
ACPI generation in QEMU would enable. Since qemu community wasn't
directly exposed to the ACPI-related patches it's easy to see how qemu
maintainers won't be aware of the churn and maintainance overhead caused by
generating them on the guest side.

That seems reasonable, so please hang on just a little bit longer
until I post acpi hotplug support for pci bridges
based on this code.

Then we can discuss.

-- 
MST