[Qemu-devel] Release plan for 0.12.0

[Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Hi,

Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

I'd like to do a few things different this time around.  I don't think 
the -rc process went very well as I don't think we got more testing out 
of it.  I'd like to shorten the timeline for 0.12.0 a good bit.  The 
0.10 stable tree got pretty difficult to maintain toward the end of the 
cycle.  We also had a pretty huge amount of change between 0.10 and 0.11 
so I think a shorter cycle is warranted.

I think aiming for early to mid-December would give us roughly a 3 month 
cycle and would align well with some of the Linux distribution cycles.  
I'd like to limit things to a single -rc that lasted only for about a 
week.  This is enough time to fix most of the obvious issues I think.

I'd also like to try to enumerate some features for this release.  
Here's a short list of things I expect to see for this release 
(target-i386 centric).  Please add or comment on items that you'd either 
like to see in the release or are planning on working on.

 o VMState conversion -- I expect most of the pc target to be completed
 o qdev conversion -- I hope that we'll get most of the pc target 
completely converted to qdev
 o storage live migration
 o switch to SeaBIOS (need to finish porting features from Bochs)
 o switch to gPXE (need to resolve slirp tftp server issue)
 o KSM integration
 o in-kernel APIC support for KVM
 o guest SMP support for KVM
 o updates to the default pc machine type

Please add to this list and I'll collect it all and post it somewhere.

Thanks!

-- 
Regards,

Anthony Liguori

[Qemu-devel] Re: Release plan for 0.12.0

[Dustin Kirkland]

On Tue, Sep 29, 2009 at 6:54 PM, Anthony Liguori <aliguori@us.ibm.com> wrote:
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>
> I'd like to do a few things different this time around.  I don't think the
> -rc process went very well as I don't think we got more testing out of it.
>  I'd like to shorten the timeline for 0.12.0 a good bit.  The 0.10 stable
> tree got pretty difficult to maintain toward the end of the cycle.  We also
> had a pretty huge amount of change between 0.10 and 0.11 so I think a
> shorter cycle is warranted.
>
> I think aiming for early to mid-December would give us roughly a 3 month
> cycle and would align well with some of the Linux distribution cycles.  I'd
> like to limit things to a single -rc that lasted only for about a week.
>  This is enough time to fix most of the obvious issues I think.

As a downstream packager of qemu-kvm, I thought I'd mention that the
next Ubuntu cycle is now public:
 * https://wiki.ubuntu.com/LucidReleaseSchedule

The key date here is Feature Freeze, which is February 25, 2010.
That's the point by which we'd need to have a new qemu-kvm (which of
course is downstream of qemu) package in Ubuntu for the LTS 10.04
release in April 2010.

I'll gladly track the release candidate(s) in the Lucid development
tree, and hopefully pull 0.12 as soon as its available.

And we also provide daily snapshots of qemu builds at:
 * https://edge.launchpad.net/~ubuntu-virt/+archive/virt-daily-upstream

:-Dustin

[Qemu-devel] Re: Release plan for 0.12.0

[Anthony Liguori]

Dustin Kirkland wrote:
> On Tue, Sep 29, 2009 at 6:54 PM, Anthony Liguori <aliguori@us.ibm.com> wrote:
>   
>> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>>
>> I'd like to do a few things different this time around.  I don't think the
>> -rc process went very well as I don't think we got more testing out of it.
>>  I'd like to shorten the timeline for 0.12.0 a good bit.  The 0.10 stable
>> tree got pretty difficult to maintain toward the end of the cycle.  We also
>> had a pretty huge amount of change between 0.10 and 0.11 so I think a
>> shorter cycle is warranted.
>>
>> I think aiming for early to mid-December would give us roughly a 3 month
>> cycle and would align well with some of the Linux distribution cycles.  I'd
>> like to limit things to a single -rc that lasted only for about a week.
>>  This is enough time to fix most of the obvious issues I think.
>>     
>
> As a downstream packager of qemu-kvm, I thought I'd mention that the
> next Ubuntu cycle is now public:
>  * https://wiki.ubuntu.com/LucidReleaseSchedule
>
> The key date here is Feature Freeze, which is February 25, 2010.
> That's the point by which we'd need to have a new qemu-kvm (which of
> course is downstream of qemu) package in Ubuntu for the LTS 10.04
> release in April 2010.
>   

If we did a December release, then the 0.13 release would probably be in 
the April time frame.  Not really ideal for Lucid so I'd recommend that 
you ship 0.12.  The good news would be that 0.12 should be very stable 
by Feb 25th and since Lucid is an LTS, that's probably a Good Thing.

-- 
Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Isaku Yamahata]

On Tue, Sep 29, 2009 at 06:54:53PM -0500, Anthony Liguori wrote:
> Hi,
> 
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
> 
> I'd like to do a few things different this time around.  I don't think 
> the -rc process went very well as I don't think we got more testing out 
> of it.  I'd like to shorten the timeline for 0.12.0 a good bit.  The 
> 0.10 stable tree got pretty difficult to maintain toward the end of the 
> cycle.  We also had a pretty huge amount of change between 0.10 and 0.11 
> so I think a shorter cycle is warranted.
> 
> I think aiming for early to mid-December would give us roughly a 3 month 
> cycle and would align well with some of the Linux distribution cycles.  
> I'd like to limit things to a single -rc that lasted only for about a 
> week.  This is enough time to fix most of the obvious issues I think.
> 
> I'd also like to try to enumerate some features for this release.  
> Here's a short list of things I expect to see for this release 
> (target-i386 centric).  Please add or comment on items that you'd either 
> like to see in the release or are planning on working on.
> 
> o VMState conversion -- I expect most of the pc target to be completed
> o qdev conversion -- I hope that we'll get most of the pc target 
> completely converted to qdev
> o storage live migration
> o switch to SeaBIOS (need to finish porting features from Bochs)
> o switch to gPXE (need to resolve slirp tftp server issue)
> o KSM integration
> o in-kernel APIC support for KVM
> o guest SMP support for KVM
> o updates to the default pc machine type
> 
> Please add to this list and I'll collect it all and post it somewhere.

 o newer chipset (which is based on Q35 chipset)
 o multiple pci bus 
 o PCI express (MMCONFIG)
 o PCI express hot plug (not acpi based)
 o PCI express switch emulator

Although there is no PCIe emulated device at the moment, 
this will be a fundamental infrastructure for PCI express native
direct attach.

thanks,
-- 
yamahata

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Hi Isaku,

Isaku Yamahata wrote:
>  o newer chipset (which is based on Q35 chipset)
>  o multiple pci bus 
>  o PCI express (MMCONFIG)
>  o PCI express hot plug (not acpi based)
>  o PCI express switch emulator
>
> Although there is no PCIe emulated device at the moment, 
> this will be a fundamental infrastructure for PCI express native
> direct attach.
>   

Your patches definitely deserve review/commit.  I'll make sure that 
happens for the 0.12 time frame.

Michael, could you help review some of the PCI patches?

Thanks,

-- 
Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Michael S. Tsirkin]

On Wed, Sep 30, 2009 at 08:03:20AM -0500, Anthony Liguori wrote:
> Hi Isaku,
>
> Isaku Yamahata wrote:
>>  o newer chipset (which is based on Q35 chipset)
>>  o multiple pci bus  o PCI express (MMCONFIG)
>>  o PCI express hot plug (not acpi based)
>>  o PCI express switch emulator
>>
>> Although there is no PCIe emulated device at the moment, this will be a 
>> fundamental infrastructure for PCI express native
>> direct attach.
>>   
>
> Your patches definitely deserve review/commit.  I'll make sure that  
> happens for the 0.12 time frame.
>
> Michael, could you help review some of the PCI patches?

Yes, I am doing this sent comments already.
The only thing I have not looked at yet is the new express file.

> Thanks,
>
> -- 
> Regards,
>
> Anthony Liguori

[Qemu-devel] Re: Release plan for 0.12.0

[Amit Shah]

On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
> Hi,
>
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>
> I'd like to do a few things different this time around.  I don't think  
> the -rc process went very well as I don't think we got more testing out  
> of it.  I'd like to shorten the timeline for 0.12.0 a good bit.  The  
> 0.10 stable tree got pretty difficult to maintain toward the end of the  
> cycle.  We also had a pretty huge amount of change between 0.10 and 0.11  
> so I think a shorter cycle is warranted.
>
> I think aiming for early to mid-December would give us roughly a 3 month  
> cycle and would align well with some of the Linux distribution cycles.   
> I'd like to limit things to a single -rc that lasted only for about a  
> week.  This is enough time to fix most of the obvious issues I think.
>
> I'd also like to try to enumerate some features for this release.   
> Here's a short list of things I expect to see for this release  
> (target-i386 centric).  Please add or comment on items that you'd either  
> like to see in the release or are planning on working on.
>
> o VMState conversion -- I expect most of the pc target to be completed
> o qdev conversion -- I hope that we'll get most of the pc target  
> completely converted to qdev
> o storage live migration
> o switch to SeaBIOS (need to finish porting features from Bochs)
> o switch to gPXE (need to resolve slirp tftp server issue)
> o KSM integration
> o in-kernel APIC support for KVM
> o guest SMP support for KVM
> o updates to the default pc machine type

  o multiport virtio-console support

> Please add to this list and I'll collect it all and post it somewhere.

Thanks,
		Amit

[Qemu-devel] Re: Release plan for 0.12.0

[Anthony Liguori]

Amit Shah wrote:
> On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
>   
>   o multiport virtio-console support
>   

Assuming we can get the kernel drivers straightened out, I think it's 
certainly reasonable for 0.12.

-- 
Regards,

Anthony Liguori

[Qemu-devel] Re: Release plan for 0.12.0

[Amit Shah]

On (Wed) Sep 30 2009 [08:04:17], Anthony Liguori wrote:
> Amit Shah wrote:
>> On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
>>     o multiport virtio-console support
>>   
>
> Assuming we can get the kernel drivers straightened out, I think it's  
> certainly reasonable for 0.12.

The kernel drivers are in fine shape.

		Amit

[Qemu-devel] Re: Release plan for 0.12.0

[Anthony Liguori]

Amit Shah wrote:
> On (Wed) Sep 30 2009 [08:04:17], Anthony Liguori wrote:
>   
>> Amit Shah wrote:
>>     
>>> On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
>>>     o multiport virtio-console support
>>>   
>>>       
>> Assuming we can get the kernel drivers straightened out, I think it's  
>> certainly reasonable for 0.12.
>>     
>
> The kernel drivers are in fine shape.
>   

I meant on track for including into the appropriate tree.  Looking for 
an Ack/Nack from Rusty.  That's been the general policy for all virtio 
changes btw.  Nothing specific to virtio-console.

> 		Amit
>   


-- 
Regards,

Anthony Liguori

[Qemu-devel] Re: Release plan for 0.12.0

[Amit Shah]

On (Wed) Sep 30 2009 [09:47:22], Anthony Liguori wrote:
> Amit Shah wrote:
>> On (Wed) Sep 30 2009 [08:04:17], Anthony Liguori wrote:
>>   
>>> Amit Shah wrote:
>>>     
>>>> On (Tue) Sep 29 2009 [18:54:53], Anthony Liguori wrote:
>>>>     o multiport virtio-console support
>>>>         
>>> Assuming we can get the kernel drivers straightened out, I think it's 
>>>  certainly reasonable for 0.12.
>>
>> The kernel drivers are in fine shape.
>
> I meant on track for including into the appropriate tree.  Looking for  
> an Ack/Nack from Rusty.  That's been the general policy for all virtio  
> changes btw.  Nothing specific to virtio-console.

That's fine.

		Amit

Re: [Qemu-devel] Release plan for 0.12.0

[Avi Kivity]

On 09/30/2009 01:54 AM, Anthony Liguori wrote:
> Hi,
>
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>
> I'd like to do a few things different this time around.  I don't think 
> the -rc process went very well as I don't think we got more testing 
> out of it.  I'd like to shorten the timeline for 0.12.0 a good bit.  
> The 0.10 stable tree got pretty difficult to maintain toward the end 
> of the cycle.  We also had a pretty huge amount of change between 0.10 
> and 0.11 so I think a shorter cycle is warranted.
>
> I think aiming for early to mid-December would give us roughly a 3 
> month cycle and would align well with some of the Linux distribution 
> cycles.  I'd like to limit things to a single -rc that lasted only for 
> about a week.  This is enough time to fix most of the obvious issues I 
> think.
>
> I'd also like to try to enumerate some features for this release.  
> Here's a short list of things I expect to see for this release 
> (target-i386 centric).  Please add or comment on items that you'd 
> either like to see in the release or are planning on working on.
>
> o VMState conversion -- I expect most of the pc target to be completed
> o qdev conversion -- I hope that we'll get most of the pc target 
> completely converted to qdev
> o storage live migration
> o switch to SeaBIOS (need to finish porting features from Bochs)
> o switch to gPXE (need to resolve slirp tftp server issue)
> o KSM integration
> o in-kernel APIC support for KVM
> o guest SMP support for KVM
> o updates to the default pc machine type

Machine monitor protocol.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Avi Kivity wrote:
> On 09/30/2009 01:54 AM, Anthony Liguori wrote:
>> Hi,
>>
>> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>>
>> I'd like to do a few things different this time around.  I don't 
>> think the -rc process went very well as I don't think we got more 
>> testing out of it.  I'd like to shorten the timeline for 0.12.0 a 
>> good bit.  The 0.10 stable tree got pretty difficult to maintain 
>> toward the end of the cycle.  We also had a pretty huge amount of 
>> change between 0.10 and 0.11 so I think a shorter cycle is warranted.
>>
>> I think aiming for early to mid-December would give us roughly a 3 
>> month cycle and would align well with some of the Linux distribution 
>> cycles.  I'd like to limit things to a single -rc that lasted only 
>> for about a week.  This is enough time to fix most of the obvious 
>> issues I think.
>>
>> I'd also like to try to enumerate some features for this release.  
>> Here's a short list of things I expect to see for this release 
>> (target-i386 centric).  Please add or comment on items that you'd 
>> either like to see in the release or are planning on working on.
>>
>> o VMState conversion -- I expect most of the pc target to be completed
>> o qdev conversion -- I hope that we'll get most of the pc target 
>> completely converted to qdev
>> o storage live migration
>> o switch to SeaBIOS (need to finish porting features from Bochs)
>> o switch to gPXE (need to resolve slirp tftp server issue)
>> o KSM integration
>> o in-kernel APIC support for KVM
>> o guest SMP support for KVM
>> o updates to the default pc machine type
>
> Machine monitor protocol.

If we're going to support the protocol for 0.12, I'd like to most of the 
code merged by the end of October.

-- 
Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Luiz Capitulino]

On Wed, 30 Sep 2009 08:05:16 -0500
Anthony Liguori <aliguori@us.ibm.com> wrote:

> Avi Kivity wrote:
> > On 09/30/2009 01:54 AM, Anthony Liguori wrote:
> >> Hi,
> >>
> >> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
> >>
> >> I'd like to do a few things different this time around.  I don't 
> >> think the -rc process went very well as I don't think we got more 
> >> testing out of it.  I'd like to shorten the timeline for 0.12.0 a 
> >> good bit.  The 0.10 stable tree got pretty difficult to maintain 
> >> toward the end of the cycle.  We also had a pretty huge amount of 
> >> change between 0.10 and 0.11 so I think a shorter cycle is warranted.
> >>
> >> I think aiming for early to mid-December would give us roughly a 3 
> >> month cycle and would align well with some of the Linux distribution 
> >> cycles.  I'd like to limit things to a single -rc that lasted only 
> >> for about a week.  This is enough time to fix most of the obvious 
> >> issues I think.
> >>
> >> I'd also like to try to enumerate some features for this release.  
> >> Here's a short list of things I expect to see for this release 
> >> (target-i386 centric).  Please add or comment on items that you'd 
> >> either like to see in the release or are planning on working on.
> >>
> >> o VMState conversion -- I expect most of the pc target to be completed
> >> o qdev conversion -- I hope that we'll get most of the pc target 
> >> completely converted to qdev
> >> o storage live migration
> >> o switch to SeaBIOS (need to finish porting features from Bochs)
> >> o switch to gPXE (need to resolve slirp tftp server issue)
> >> o KSM integration
> >> o in-kernel APIC support for KVM
> >> o guest SMP support for KVM
> >> o updates to the default pc machine type
> >
> > Machine monitor protocol.
> 
> If we're going to support the protocol for 0.12, I'd like to most of the 
> code merged by the end of October.

 Four weeks.. Not so much time, but let's try.

 There are two major issues that may delay QMP.

 Firstly, we are still on the infrastructure/design phase, which
is natural to take time. Maybe when handlers start getting converted
en masse things will be faster.

 Secondly: testing. I have a very ugly python script to test the
already converted handlers. The problem is not only the ugliness,
the right way to do this would be to use kvm-autotest. So, I was
planning to take a detailed look at it and perhaps start writing
tests for QMP right when each handler is converted. Right Thing,
but takes time.

Re: [Qemu-devel] Release plan for 0.12.0

[Avi Kivity]

On 10/01/2009 11:13 PM, Luiz Capitulino wrote:
>> If we're going to support the protocol for 0.12, I'd like to most of the
>> code merged by the end of October.
>>      
>   Four weeks.. Not so much time, but let's try.
>
>   There are two major issues that may delay QMP.
>
>   Firstly, we are still on the infrastructure/design phase, which
> is natural to take time. Maybe when handlers start getting converted
> en masse things will be faster.
>    

I sure hope so.  Maybe someone can pitch in if not.

>   Secondly: testing. I have a very ugly python script to test the
> already converted handlers. The problem is not only the ugliness,
> the right way to do this would be to use kvm-autotest. So, I was
> planning to take a detailed look at it and perhaps start writing
> tests for QMP right when each handler is converted. Right Thing,
> but takes time.
>    

I think this could be done by having autotest use two monitors, one with 
the machine protocol and one with the human protocol, trying first the 
machine protocol and falling back if the command is not supported.

Hopefully we can get the autotest people to work on it so we parallelize 
development.  They'll also give user-oriented feedback which can be 
valuable.

Are you using a standard json parser with your test script?  That's an 
additional validation.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Qemu-devel] Release plan for 0.12.0

[Luiz Capitulino]

On Sat, 03 Oct 2009 12:04:57 +0200
Avi Kivity <avi@redhat.com> wrote:

> On 10/01/2009 11:13 PM, Luiz Capitulino wrote:
> >> If we're going to support the protocol for 0.12, I'd like to most of the
> >> code merged by the end of October.
> >>      
> >   Four weeks.. Not so much time, but let's try.
> >
> >   There are two major issues that may delay QMP.
> >
> >   Firstly, we are still on the infrastructure/design phase, which
> > is natural to take time. Maybe when handlers start getting converted
> > en masse things will be faster.
> >    
> 
> I sure hope so.  Maybe someone can pitch in if not.

 I've written a TODO list if someone is willing to help:

http://tinyurl.com/ya7l6bo

> >   Secondly: testing. I have a very ugly python script to test the
> > already converted handlers. The problem is not only the ugliness,
> > the right way to do this would be to use kvm-autotest. So, I was
> > planning to take a detailed look at it and perhaps start writing
> > tests for QMP right when each handler is converted. Right Thing,
> > but takes time.
> >    
> 
> I think this could be done by having autotest use two monitors, one with 
> the machine protocol and one with the human protocol, trying first the 
> machine protocol and falling back if the command is not supported.

 Yes, sounds a good idea.

> Hopefully we can get the autotest people to work on it so we parallelize 
> development.  They'll also give user-oriented feedback which can be 
> valuable.

 I will talk to them about that.

> Are you using a standard json parser with your test script?  That's an 
> additional validation.

 I'm using Python's json module, but I could run one of the checkers
listed in the json's page for each test, before the Python's module
kicks in.

Re: [Qemu-devel] Release plan for 0.12.0

[Avi Kivity]

On 10/05/2009 02:43 PM, Luiz Capitulino wrote:
>> Are you using a standard json parser with your test script?  That's an
>> additional validation.
>>      
>   I'm using Python's json module, but I could run one of the checkers
> listed in the json's page for each test, before the Python's module
> kicks in.
>    

python-json should be sufficient, I think.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Release plan for 0.12.0

[Luiz Capitulino]

On Wed, 30 Sep 2009 08:41:23 +0200
Avi Kivity <avi@redhat.com> wrote:

> On 09/30/2009 01:54 AM, Anthony Liguori wrote:
> > Hi,
> >
> > Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
> >
> > I'd like to do a few things different this time around.  I don't think 
> > the -rc process went very well as I don't think we got more testing 
> > out of it.  I'd like to shorten the timeline for 0.12.0 a good bit.  
> > The 0.10 stable tree got pretty difficult to maintain toward the end 
> > of the cycle.  We also had a pretty huge amount of change between 0.10 
> > and 0.11 so I think a shorter cycle is warranted.
> >
> > I think aiming for early to mid-December would give us roughly a 3 
> > month cycle and would align well with some of the Linux distribution 
> > cycles.  I'd like to limit things to a single -rc that lasted only for 
> > about a week.  This is enough time to fix most of the obvious issues I 
> > think.
> >
> > I'd also like to try to enumerate some features for this release.  
> > Here's a short list of things I expect to see for this release 
> > (target-i386 centric).  Please add or comment on items that you'd 
> > either like to see in the release or are planning on working on.
> >
> > o VMState conversion -- I expect most of the pc target to be completed
> > o qdev conversion -- I hope that we'll get most of the pc target 
> > completely converted to qdev
> > o storage live migration
> > o switch to SeaBIOS (need to finish porting features from Bochs)
> > o switch to gPXE (need to resolve slirp tftp server issue)
> > o KSM integration
> > o in-kernel APIC support for KVM
> > o guest SMP support for KVM
> > o updates to the default pc machine type
> 
> Machine monitor protocol.

 Yeah, I was going to suggest it as well.

Re: [Qemu-devel] Release plan for 0.12.0

[Michael Tokarev]

Anthony Liguori wrote:
[]
> Here's a short list of things I expect to see for this release 
> (target-i386 centric).  Please add or comment on items that you'd either 
> like to see in the release or are planning on working on.
[..]
> o guest SMP support for KVM

Hmm.  What is this, can you elaborate a bit more please?
-smp nn is already here, no?

Thanks!

/mjt

Re: [Qemu-devel] Release plan for 0.12.0

[Avi Kivity]

On 09/30/2009 10:53 AM, Michael Tokarev wrote:
> Anthony Liguori wrote:
> []
>> Here's a short list of things I expect to see for this release 
>> (target-i386 centric).  Please add or comment on items that you'd 
>> either like to see in the release or are planning on working on.
> [..]
>> o guest SMP support for KVM
>
> Hmm.  What is this, can you elaborate a bit more please?
> -smp nn is already here, no?
>

Only in qemu-kvm.git.  This is about qemu.git (which supports -smp, but 
not with kvm).

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

Hi,

On 30.09.2009 01:54, Anthony Liguori wrote:
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>
> I'd also like to try to enumerate some features for this release. 
> Here's a short list of things I expect to see for this release
> (target-i386 centric).
>
> o switch to SeaBIOS (need to finish porting features from Bochs)

That switch is much appreciated because it also reduces the testing
matrix of those coreboot developers who boot test every commit with Qemu.

However, to run coreboot on Qemu with the same init sequence as on
simplified real hardware, we need Cache-as-RAM (CAR) support. This is
basically a mode where sizeof(cacheable area) <= sizeof (L2 cache) and
causes the processor to lock the cache and not pass any reads/writes
through to the RAM behind the cached area. The easiest way to implement
this would be to check the cache size criterion upon every MTRR
manipulation and either map a chunk of fresh memory on top of the
existing memory (which may be RAM, ROM or unmapped) for every cacheable
area, and if the cacheable area starts to exceed the L2 cache size,
discard all memory contents of the memory mapped on top.
For additional correctness, the memory shoud not be discarded and
written back to the lower layer of memory if WBINVD (instead of INVD) or
CLFLUSH are called. That one is mostly sugar, though, and coreboot can
do without.

Right now coreboot sets up the MTRRs correctly, but then (conditional on
Qemu) only uses areas which are known to be backed by RAM instead of the
areas designated by CAR.

I'd like to implement CAR support which builds on top of my MTRR code
which was merged some months ago (and I already have code to check for
total cacheable area size), but I need help with the memory mapping
stuff. How do I proceed? Clean up what I have and insert "FIXME"
comments where I don't know how to implement stuff so others can see the
code and comment on it?

Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Carl-Daniel Hailfinger wrote:
> Hi,
>
> On 30.09.2009 01:54, Anthony Liguori wrote:
>   
>> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>>
>> I'd also like to try to enumerate some features for this release. 
>> Here's a short list of things I expect to see for this release
>> (target-i386 centric).
>>
>> o switch to SeaBIOS (need to finish porting features from Bochs)
>>     
>
> That switch is much appreciated because it also reduces the testing
> matrix of those coreboot developers who boot test every commit with Qemu.
>
> However, to run coreboot on Qemu with the same init sequence as on
> simplified real hardware, we need Cache-as-RAM (CAR) support. This is
> basically a mode where sizeof(cacheable area) <= sizeof (L2 cache) and
> causes the processor to lock the cache and not pass any reads/writes
> through to the RAM behind the cached area. The easiest way to implement
> this would be to check the cache size criterion upon every MTRR
> manipulation and either map a chunk of fresh memory on top of the
> existing memory (which may be RAM, ROM or unmapped) for every cacheable
> area, and if the cacheable area starts to exceed the L2 cache size,
> discard all memory contents of the memory mapped on top.
> For additional correctness, the memory shoud not be discarded and
> written back to the lower layer of memory if WBINVD (instead of INVD) or
> CLFLUSH are called. That one is mostly sugar, though, and coreboot can
> do without.
>   

Do we really need coreboot to use the same init sequence?   coreboot is 
firmware and we don't necessarily run real firmware under QEMU.  It's a 
short cut that lets us avoid a lot of complexity.

> Right now coreboot sets up the MTRRs correctly, but then (conditional on
> Qemu) only uses areas which are known to be backed by RAM instead of the
> areas designated by CAR.
>
> I'd like to implement CAR support which builds on top of my MTRR code
> which was merged some months ago (and I already have code to check for
> total cacheable area size), but I need help with the memory mapping
> stuff. How do I proceed? Clean up what I have and insert "FIXME"
> comments where I don't know how to implement stuff so others can see the
> code and comment on it?
>   

You could start there.  But from a higher level, I'm not sure I think a 
partial implementation of something like CAR is all that valuable since 
coreboot already runs under QEMU.

> Regards,
> Carl-Daniel
>
>   
-- 

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 30.09.2009 15:07, Anthony Liguori wrote:
> Carl-Daniel Hailfinger wrote:
>> However, to run coreboot on Qemu with the same init sequence as on
>> simplified real hardware, we need Cache-as-RAM (CAR) support. [...] 
>
> Do we really need coreboot to use the same init sequence?   coreboot
> is firmware and we don't necessarily run real firmware under QEMU. 
> It's a short cut that lets us avoid a lot of complexity.

I know that some people were running 440BX BIOS images for real hardware
on Qemu and they got pretty far.

The complexity would be limited to the MTRR code and unless there were
major architectural changes in mapping RAM to address ranges, no other
code (except VM save and VM restore) should get even a single line changed.

>> Right now coreboot sets up the MTRRs correctly, but then (conditional on
>> Qemu) only uses areas which are known to be backed by RAM instead of the
>> areas designated by CAR.
>>
>> I'd like to implement CAR support which builds on top of my MTRR code
>> which was merged some months ago (and I already have code to check for
>> total cacheable area size), but I need help with the memory mapping
>> stuff. How do I proceed? Clean up what I have and insert "FIXME"
>> comments where I don't know how to implement stuff so others can see the
>> code and comment on it?   
>
> You could start there.  But from a higher level, I'm not sure I think
> a partial implementation of something like CAR is all that valuable
> since coreboot already runs under QEMU.

It only runs if WORKAROUND_QEMU is defined (maybe not exactly that name,
but you get the point). The code in coreboot calculates MTRR settings to
cover the place where the stack will be. To workaround missing CAR in
Qemu, it then has to recalculate the stack location to be able to
actually use the stack. That forces coreboot to keep two stack base
variables and to completely replace the generic logic which switches off
CAR.

I hope the explanation above didn't offend you, I just tried to clarify
why working CAR is such a big deal for coreboot.

If you want either a full CAR implementation or no CAR implementation, I
can write a patch which implements full CAR, but then I need to hook
WBINVD, INVD and CLFLUSH. Neither instruction is executed often enough
to show up in any profile. Besides that, for anything not using CAR
(everything after the firmware), the penalty is a simple test of a
boolean variable per WBINVD/INVD/CLFLUSH.

If you have further questions, please don't hesitate to ask.

Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/

Re: [Qemu-devel] Release plan for 0.12.0

[Blue Swirl]

On Wed, Sep 30, 2009 at 6:59 PM, Carl-Daniel Hailfinger
<c-d.hailfinger.devel.2006@gmx.net> wrote:
> On 30.09.2009 15:07, Anthony Liguori wrote:
>> Carl-Daniel Hailfinger wrote:
>>> However, to run coreboot on Qemu with the same init sequence as on
>>> simplified real hardware, we need Cache-as-RAM (CAR) support. [...]
>>
>> Do we really need coreboot to use the same init sequence?   coreboot
>> is firmware and we don't necessarily run real firmware under QEMU.
>> It's a short cut that lets us avoid a lot of complexity.
>
> I know that some people were running 440BX BIOS images for real hardware
> on Qemu and they got pretty far.
>
> The complexity would be limited to the MTRR code and unless there were
> major architectural changes in mapping RAM to address ranges, no other
> code (except VM save and VM restore) should get even a single line changed.
>
>>> Right now coreboot sets up the MTRRs correctly, but then (conditional on
>>> Qemu) only uses areas which are known to be backed by RAM instead of the
>>> areas designated by CAR.
>>>
>>> I'd like to implement CAR support which builds on top of my MTRR code
>>> which was merged some months ago (and I already have code to check for
>>> total cacheable area size), but I need help with the memory mapping
>>> stuff. How do I proceed? Clean up what I have and insert "FIXME"
>>> comments where I don't know how to implement stuff so others can see the
>>> code and comment on it?
>>
>> You could start there.  But from a higher level, I'm not sure I think
>> a partial implementation of something like CAR is all that valuable
>> since coreboot already runs under QEMU.
>
> It only runs if WORKAROUND_QEMU is defined (maybe not exactly that name,
> but you get the point). The code in coreboot calculates MTRR settings to
> cover the place where the stack will be. To workaround missing CAR in
> Qemu, it then has to recalculate the stack location to be able to
> actually use the stack. That forces coreboot to keep two stack base
> variables and to completely replace the generic logic which switches off
> CAR.
>
> I hope the explanation above didn't offend you, I just tried to clarify
> why working CAR is such a big deal for coreboot.
>
> If you want either a full CAR implementation or no CAR implementation, I
> can write a patch which implements full CAR, but then I need to hook
> WBINVD, INVD and CLFLUSH. Neither instruction is executed often enough
> to show up in any profile. Besides that, for anything not using CAR
> (everything after the firmware), the penalty is a simple test of a
> boolean variable per WBINVD/INVD/CLFLUSH.

The CAR mode could affect only translation so that special CAR
versions of the WBINVD etc. instructions are selected. On switch to
normal mode, the TBs need to be flushed.

Instead of your memory mapping approach (which should work) you could
also try using different memory access functions in CAR mode. It may
be more difficult, though.

Re: [Qemu-devel] Release plan for 0.12.0

[Luiz Capitulino]

On Tue, 29 Sep 2009 18:54:53 -0500
Anthony Liguori <aliguori@us.ibm.com> wrote:

> I think aiming for early to mid-December would give us roughly a 3 month 
> cycle and would align well with some of the Linux distribution cycles.  
> I'd like to limit things to a single -rc that lasted only for about a 
> week.  This is enough time to fix most of the obvious issues I think.

 How do you plan to do it? I mean, are you going to create a separate branch
or make master the -rc?

 Creating a separate branch (which is what we do today, iiuc) makes it
get less attention, freezing master for a certain period is the best
way to stabilize.

 Is this what you had in mind?

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Luiz Capitulino wrote:
> On Tue, 29 Sep 2009 18:54:53 -0500
> Anthony Liguori <aliguori@us.ibm.com> wrote:
>
>   
>> I think aiming for early to mid-December would give us roughly a 3 month 
>> cycle and would align well with some of the Linux distribution cycles.  
>> I'd like to limit things to a single -rc that lasted only for about a 
>> week.  This is enough time to fix most of the obvious issues I think.
>>     
>
>  How do you plan to do it? I mean, are you going to create a separate branch
> or make master the -rc?
>
>  Creating a separate branch (which is what we do today, iiuc) makes it
> get less attention, freezing master for a certain period is the best
> way to stabilize.
>
>  Is this what you had in mind?
>   
What do people think?

One reason I branch is because some people care a bit less about 
releases so it makes the process non-disruptive to them.  If the other 
maintainers agreed though, I would certainly like to have the master 
branch essentially frozen for the week before the release.

-- 
Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Fred Leeflang]

[-- Attachment #1: Type: text/plain, Size: 1433 bytes --]

2009/9/30 Anthony Liguori <aliguori@us.ibm.com>

> Luiz Capitulino wrote:
>
>> On Tue, 29 Sep 2009 18:54:53 -0500
>> Anthony Liguori <aliguori@us.ibm.com> wrote:
>>
>>
>>
>>> I think aiming for early to mid-December would give us roughly a 3 month
>>> cycle and would align well with some of the Linux distribution cycles.  I'd
>>> like to limit things to a single -rc that lasted only for about a week.
>>>  This is enough time to fix most of the obvious issues I think.
>>>
>>>
>>
>>  How do you plan to do it? I mean, are you going to create a separate
>> branch
>> or make master the -rc?
>>
>>  Creating a separate branch (which is what we do today, iiuc) makes it
>> get less attention, freezing master for a certain period is the best
>> way to stabilize.
>>
>>  Is this what you had in mind?
>>
>>
> What do people think?
>
> One reason I branch is because some people care a bit less about releases
> so it makes the process non-disruptive to them.  If the other maintainers
> agreed though, I would certainly like to have the master branch essentially
> frozen for the week before the release.
>

freezing is only neccesary if you need time to gather all the patches, build
and test them together etc. If you don't feel you or the developers need to
do that to get a reliable release out I think it only halts developers
without any clear reason to do so. Calling 'attention' to a release is not a
clear reason IMO.

-Fred

[-- Attachment #2: Type: text/html, Size: 2139 bytes --]

Re: [Qemu-devel] Release plan for 0.12.0

[Luiz Capitulino]

On Wed, 30 Sep 2009 17:03:23 +0200
Fred Leeflang <fredl@dutchie.org> wrote:

> 2009/9/30 Anthony Liguori <aliguori@us.ibm.com>
> 
> > Luiz Capitulino wrote:
> >
> >> On Tue, 29 Sep 2009 18:54:53 -0500
> >> Anthony Liguori <aliguori@us.ibm.com> wrote:
> >>
> >>
> >>
> >>> I think aiming for early to mid-December would give us roughly a 3 month
> >>> cycle and would align well with some of the Linux distribution cycles.  I'd
> >>> like to limit things to a single -rc that lasted only for about a week.
> >>>  This is enough time to fix most of the obvious issues I think.
> >>>
> >>>
> >>
> >>  How do you plan to do it? I mean, are you going to create a separate
> >> branch
> >> or make master the -rc?
> >>
> >>  Creating a separate branch (which is what we do today, iiuc) makes it
> >> get less attention, freezing master for a certain period is the best
> >> way to stabilize.
> >>
> >>  Is this what you had in mind?
> >>
> >>
> > What do people think?
> >
> > One reason I branch is because some people care a bit less about releases
> > so it makes the process non-disruptive to them.  If the other maintainers
> > agreed though, I would certainly like to have the master branch essentially
> > frozen for the week before the release.
> >
> 
> freezing is only neccesary if you need time to gather all the patches, build
> and test them together etc. 

 Not exactly, freezing is done to stop/slowdown writing new code and focus
on bug fixing for a period of time.

 This is not only needed for a release, but projects should always try
to find the best balance between 'number of bugs' and 'feature addition rate'.

> If you don't feel you or the developers need to
> do that to get a reliable release out I think it only halts developers
> without any clear reason to do so. Calling 'attention' to a release is not a
> clear reason IMO.

 Having a functional and relatively stable release is not only
important, but it's the ultimate goal IMO.

 Obviously we should take care not to take extremes. No QEMU release
will be 100% bug free, that's why we have stables.

Re: [Qemu-devel] Release plan for 0.12.0

[Gerd Hoffmann]

On 09/30/09 16:45, Anthony Liguori wrote:
> One reason I branch is because some people care a bit less about
> releases so it makes the process non-disruptive to them. If the other
> maintainers agreed though, I would certainly like to have the master
> branch essentially frozen for the week before the release.

We had much longer disruptions without a release freeze, so why worry 
about a single week?  One week freeze is short enougth that the 
disruption isn't a big issue.  It will help testing the to-be-released 
code.  Go for it.

cheers,
   Gerd

Re: [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

for target-i386 and target-x86_64:

o EFI firmware emulation (as a machine or command line option)
o OS/2 1.x support

for target-m68k

o Macintosh support at least to boot an Apple Toolbox (the firmware)

for target-ppc

o Mac OS X support (there is already a open source emulator able to  
boot it shouldn't be so difficult to correct QEMU bugs on it)

for target-mips

o Windows NT support

integrating target-alpha and target-zxspectrum in the branch tree
a new official os support list design (please, give me ideas, css,  
help, anything!) with a better captcha

El 30/09/2009, a las 00:54, Anthony Liguori escribió:

> Hi,
>
> Now that 0.11.0 is behind us, it's time to start thinking about  
> 0.12.0.
>
> I'd like to do a few things different this time around.  I don't  
> think the -rc process went very well as I don't think we got more  
> testing out of it.  I'd like to shorten the timeline for 0.12.0 a  
> good bit.  The 0.10 stable tree got pretty difficult to maintain  
> toward the end of the cycle.  We also had a pretty huge amount of  
> change between 0.10 and 0.11 so I think a shorter cycle is warranted.
>
> I think aiming for early to mid-December would give us roughly a 3  
> month cycle and would align well with some of the Linux distribution  
> cycles.  I'd like to limit things to a single -rc that lasted only  
> for about a week.  This is enough time to fix most of the obvious  
> issues I think.
>
> I'd also like to try to enumerate some features for this release.   
> Here's a short list of things I expect to see for this release  
> (target-i386 centric).  Please add or comment on items that you'd  
> either like to see in the release or are planning on working on.
>
> o VMState conversion -- I expect most of the pc target to be completed
> o qdev conversion -- I hope that we'll get most of the pc target  
> completely converted to qdev
> o storage live migration
> o switch to SeaBIOS (need to finish porting features from Bochs)
> o switch to gPXE (need to resolve slirp tftp server issue)
> o KSM integration
> o in-kernel APIC support for KVM
> o guest SMP support for KVM
> o updates to the default pc machine type
>
> Please add to this list and I'll collect it all and post it somewhere.
>
> Thanks!
>
> -- 
> Regards,
>
> Anthony Liguori
>
>
>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 01.10.2009 03:55, Natalia Portillo wrote:
> for target-i386 and target-x86_64:
>
> o EFI firmware emulation (as a machine or command line option)

Are there any opensource EFI implementations with hardware init? I know
that an opensource EFI glue layer exists, but AFAIK hardware init is
still closed source. That leaves coreboot+UEFI (works somewhat) or
SeaBIOS+UEFI (no idea if this is possible). If I missed an announcement
about opensource hw init for EFI, I wish to apologize.

Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

I work for Intel on the edk2.tianocore.org project.  Related
to QEMU & UEFI, I am working on this project:
https://edk2.tianocore.org/OVMF.html

So, I would like to see 0.12 support UEFI, and it'd be great
if our OVMF project could enable this.

Related to OVMF I submitted this patch in August:
http://patchwork.ozlabs.org/patch/31009/
I hope this patch would enable QEMU to ship the legacy
bochs BIOS, and OVMF.  The user would be able to select
the OVMF UEFI firmware via the -M command line parameter.

-Jordan

On Thu, Oct 1, 2009 at 01:07, Carl-Daniel Hailfinger
<c-d.hailfinger.devel.2006@gmx.net> wrote:
>
> On 01.10.2009 03:55, Natalia Portillo wrote:
> > for target-i386 and target-x86_64:
> >
> > o EFI firmware emulation (as a machine or command line option)
>
> Are there any opensource EFI implementations with hardware init? I know
> that an opensource EFI glue layer exists, but AFAIK hardware init is
> still closed source. That leaves coreboot+UEFI (works somewhat) or
> SeaBIOS+UEFI (no idea if this is possible). If I missed an announcement
> about opensource hw init for EFI, I wish to apologize.
>
> Regards,
> Carl-Daniel
>
> --
> http://www.hailfinger.org/
>
>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

The only opensource EFI implementation I know about is the Intel EFI  
Reference.

Dunno if it includes hardware initialization.

El 01/10/2009, a las 09:07, Carl-Daniel Hailfinger escribió:

> On 01.10.2009 03:55, Natalia Portillo wrote:
>> for target-i386 and target-x86_64:
>>
>> o EFI firmware emulation (as a machine or command line option)
>
> Are there any opensource EFI implementations with hardware init? I  
> know
> that an opensource EFI glue layer exists, but AFAIK hardware init is
> still closed source. That leaves coreboot+UEFI (works somewhat) or
> SeaBIOS+UEFI (no idea if this is possible). If I missed an  
> announcement
> about opensource hw init for EFI, I wish to apologize.
>
> Regards,
> Carl-Daniel
>
> -- 
> http://www.hailfinger.org/
>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

I work for Intel on the edk2.tianocore.org project.  (Compared to the
original edk) edk2 may be of interest to those on this list since it
supports building on several OS's with several different toolchains.
In other words, it also supports building under Linux and OS X with
the GNU compiler and binutils.

Within our edk2 tree, we have two projects that relate to QEMU.

The DUET platform is a UEFI emulator that boots like a legacy OS on
top of a legacy BIOS.  It contains various hardware initialization
drivers for several legacy hardware devices, but it also will call
into the legacy BIOS and make use of certain items from the legacy
BIOS.  DUET can boot from the QEMU legacy BIOS, but it does require a
disk to be setup to have the DUET image on it.  I am not sure if all
of DUET's code is currently safe for a UEFI OS to be able to access
UEFI runtime services.

The OVMF platform is a project to build a (mostly) UEFI compatible
firmware for virtual machines.  QEMU support is one of the main goals
for OVMF.  The OVMF rom image completely replaces QEMU's standard
bios.bin, and therefore we must have hardware drivers for any hardware
within the QEMU VM that we want to make use of.  The project also has
the goal to support UEFI OS's at runtime.

-Jordan

On Thu, Oct 1, 2009 at 21:38, Natalia Portillo <claunia@claunia.com> wrote:
>
> The only opensource EFI implementation I know about is the Intel EFI Reference.
>
> Dunno if it includes hardware initialization.
>
> El 01/10/2009, a las 09:07, Carl-Daniel Hailfinger escribió:
>
>> On 01.10.2009 03:55, Natalia Portillo wrote:
>>>
>>> for target-i386 and target-x86_64:
>>>
>>> o EFI firmware emulation (as a machine or command line option)
>>
>> Are there any opensource EFI implementations with hardware init? I know
>> that an opensource EFI glue layer exists, but AFAIK hardware init is
>> still closed source. That leaves coreboot+UEFI (works somewhat) or
>> SeaBIOS+UEFI (no idea if this is possible). If I missed an announcement
>> about opensource hw init for EFI, I wish to apologize.
>>
>> Regards,
>> Carl-Daniel
>>
>> --
>> http://www.hailfinger.org/
>>
>>
>
>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

Hi Jordan,

thanks for the detailed explanation. It certainly helped me understand
some parts of the tianocore umbrealla better.

On 02.10.2009 07:37, Jordan Justen wrote:
> I work for Intel on the edk2.tianocore.org project.  (Compared to the
> original edk) edk2 may be of interest to those on this list since it
> supports building on several OS's with several different toolchains.
> In other words, it also supports building under Linux and OS X with
> the GNU compiler and binutils.
>   

Neat. I remember the toolchain issues Linux-based developers had in the
past, so I think this is a good thing.


> Within our edk2 tree, we have two projects that relate to QEMU.
>
> The DUET platform is a UEFI emulator that boots like a legacy OS on
> top of a legacy BIOS.  It contains various hardware initialization
> drivers for several legacy hardware devices, but it also will call
> into the legacy BIOS and make use of certain items from the legacy
> BIOS.  DUET can boot from the QEMU legacy BIOS, but it does require a
> disk to be setup to have the DUET image on it.  I am not sure if all
> of DUET's code is currently safe for a UEFI OS to be able to access
> UEFI runtime services.
>   

By the way, SeaBIOS can boot virtual floppy images stored in the ROM (at
least when SeaBIOS runs as coreboot payload), so if you can fit DUET
into a floppy image (or if Kevin adds support for virtual harddisk
images), you can have a coreboot+SeaBIOS+DUET ROM right now.


> The OVMF platform is a project to build a (mostly) UEFI compatible
> firmware for virtual machines.  QEMU support is one of the main goals
> for OVMF.  The OVMF rom image completely replaces QEMU's standard
> bios.bin, and therefore we must have hardware drivers for any hardware
> within the QEMU VM that we want to make use of.  The project also has
> the goal to support UEFI OS's at runtime.
>   

So OVMF is not just hardware init, but a complete package of hardware
init and UEFI interface?


Regards,
Carl-Daniel

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Natalia Portillo wrote:
> for target-i386 and target-x86_64:
>
> o EFI firmware emulation (as a machine or command line option)
> o OS/2 1.x support

AFAIK, no one is actually working on these.  EFI is also not terribly 
useful without a CSM and I suspect porting SeaBIOS to tiano core is 
going to be a big effort.

-- 
Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Thu, Oct 1, 2009 at 05:45, Anthony Liguori <aliguori@us.ibm.com> wrote:
> Natalia Portillo wrote:
>>
>> for target-i386 and target-x86_64:
>>
>> o EFI firmware emulation (as a machine or command line option)
>> o OS/2 1.x support
>
> AFAIK, no one is actually working on these.  EFI is also not terribly useful
> without a CSM and I suspect porting SeaBIOS to tiano core is going to be a
> big effort.

Several Linux distributions are getting UEFI support ironed out now,
so UEFI can be used without a CSM.  QEMU support for UEFI could
help the Linux distributions to enable UEFI support...

Windows Vista and 7 support UEFI on x64-64, but I've not been able
to boot either with the OVMF project yet.  (I suspect that the video driver
might not be initializing correctly, but the root cause of the failure is not
yet known.)

>
> --
> Regards,
>
> Anthony Liguori
>
>
>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Jordan Justen wrote:
> On Thu, Oct 1, 2009 at 05:45, Anthony Liguori <aliguori@us.ibm.com> wrote:
>   
>> Natalia Portillo wrote:
>>     
>>> for target-i386 and target-x86_64:
>>>
>>> o EFI firmware emulation (as a machine or command line option)
>>> o OS/2 1.x support
>>>       
>> AFAIK, no one is actually working on these.  EFI is also not terribly useful
>> without a CSM and I suspect porting SeaBIOS to tiano core is going to be a
>> big effort.
>>     
>
> Several Linux distributions are getting UEFI support ironed out now,
> so UEFI can be used without a CSM.  QEMU support for UEFI could
> help the Linux distributions to enable UEFI support...
>   

I see a CSM as a pre-requisite for merging a uefi rom.  A user can 
already use a uefi rom by simply using the -bios parameter so there's 
nothing inhibiting testing.  My concern about introducing a new machine 
type is that it would require duplicate testing and force the selection 
of uefi up through the management tool stack.

OTH, uefi + a CSM based on SeaBIOS could be used as the default 
firmware.  I don't think it's a reasonable target for 0.12 but I think 
if someone actively worked on it, it would be possible for future releases.

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Thu, Oct 1, 2009 at 14:23, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Jordan Justen wrote:
>>
>> On Thu, Oct 1, 2009 at 05:45, Anthony Liguori <aliguori@us.ibm.com> wrote:
>>
>>>
>>> Natalia Portillo wrote:
>>>
>>>>
>>>> for target-i386 and target-x86_64:
>>>>
>>>> o EFI firmware emulation (as a machine or command line option)
>>>> o OS/2 1.x support
>>>>
>>>
>>> AFAIK, no one is actually working on these.  EFI is also not terribly
>>> useful
>>> without a CSM and I suspect porting SeaBIOS to tiano core is going to be
>>> a
>>> big effort.
>>>
>>
>> Several Linux distributions are getting UEFI support ironed out now,
>> so UEFI can be used without a CSM.  QEMU support for UEFI could
>> help the Linux distributions to enable UEFI support...
>>
>
> I see a CSM as a pre-requisite for merging a uefi rom.  A user can already
> use a uefi rom by simply using the -bios parameter so there's nothing
> inhibiting testing.  My concern about introducing a new machine type is that
> it would require duplicate testing and force the selection of uefi up
> through the management tool stack.

I understand your points.  You want to keep a single machine type and
try to support UEFI and legacy boots with that firmware.
Unfortunately, I don't know that we will be able to provide that via
OVMF.  There is no open source CSM for us to make use of.

And, it is true that -bios can be used for OVMF by those with a
particular interest in UEFI.  But for a more general audience, I think
without the -M switch a Linux distribution couldn't package qemu in
such a way that both a legacy bios and the OVMF firmware would be
available.

> OTH, uefi + a CSM based on SeaBIOS could be used as the default firmware.  I
> don't think it's a reasonable target for 0.12 but I think if someone
> actively worked on it, it would be possible for future releases.

I don't know of any efforts to create an open source CSM.  Is someone
working on converting SeaBIOS to act as a CSM?

For tianocore.org & OVMF we would further be restricted by requiring a
BSD licensed CSM.  Of course, if there was a SeaBIOS based CSM, then
I'm sure OVMF could be modified to make use of it easily enough.  We
just wouldn't be able to have the SeaBIOS CSM on tianocore.org and
part of the normal tianocore.org OVMF releases.

>
> Regards,
>
> Anthony Liguori
>

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Jordan Justen wrote:
> On Thu, Oct 1, 2009 at 14:23, Anthony Liguori <anthony@codemonkey.ws> wrote:
>   
>> I see a CSM as a pre-requisite for merging a uefi rom.  A user can already
>> use a uefi rom by simply using the -bios parameter so there's nothing
>> inhibiting testing.  My concern about introducing a new machine type is that
>> it would require duplicate testing and force the selection of uefi up
>> through the management tool stack.
>>     
>
> I understand your points.  You want to keep a single machine type and
> try to support UEFI and legacy boots with that firmware.
> Unfortunately, I don't know that we will be able to provide that via
> OVMF.  There is no open source CSM for us to make use of.
>
> And, it is true that -bios can be used for OVMF by those with a
> particular interest in UEFI.  But for a more general audience, I think
> without the -M switch a Linux distribution couldn't package qemu in
> such a way that both a legacy bios and the OVMF firmware would be
> available.
>   

My concern is that if we provide separate machine types, it doubles our 
testing since we have to test guests with both the BIOS firmware and the 
UEFI firmware.  It also hurts usability since a user now has to make the 
decision as to whether they want to use UEFI or not.  A user should not 
have to even know what EFI is.

So I think the best way forward is to hold off on UEFI in mainline until 
we can provide a single unified stack.

>> OTH, uefi + a CSM based on SeaBIOS could be used as the default firmware.  I
>> don't think it's a reasonable target for 0.12 but I think if someone
>> actively worked on it, it would be possible for future releases.
>>     
>
> I don't know of any efforts to create an open source CSM.  Is someone
> working on converting SeaBIOS to act as a CSM?
>   

There are multiple parties that seem to be interested in uefi for 
guests.  I expect that one of those parties will end up doing the work 
if that's the requirement that we put down.

> For tianocore.org & OVMF we would further be restricted by requiring a
> BSD licensed CSM.  Of course, if there was a SeaBIOS based CSM, then
> I'm sure OVMF could be modified to make use of it easily enough.  We
> just wouldn't be able to have the SeaBIOS CSM on tianocore.org and
> part of the normal tianocore.org OVMF releases.
>   

I don't think that's a big problem for us.

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Fri, Oct 2, 2009 at 06:29, Anthony Liguori <anthony@codemonkey.ws> wrote:
> Jordan Justen wrote:
>>
>> On Thu, Oct 1, 2009 at 14:23, Anthony Liguori <anthony@codemonkey.ws>
>> wrote:
>>
>>>
>>> I see a CSM as a pre-requisite for merging a uefi rom.  A user can
>>> already
>>> use a uefi rom by simply using the -bios parameter so there's nothing
>>> inhibiting testing.  My concern about introducing a new machine type is
>>> that
>>> it would require duplicate testing and force the selection of uefi up
>>> through the management tool stack.
>>>
>>
>> I understand your points.  You want to keep a single machine type and
>> try to support UEFI and legacy boots with that firmware.
>> Unfortunately, I don't know that we will be able to provide that via
>> OVMF.  There is no open source CSM for us to make use of.
>>
>> And, it is true that -bios can be used for OVMF by those with a
>> particular interest in UEFI.  But for a more general audience, I think
>> without the -M switch a Linux distribution couldn't package qemu in
>> such a way that both a legacy bios and the OVMF firmware would be
>> available.
>>
>
> My concern is that if we provide separate machine types, it doubles our
> testing since we have to test guests with both the BIOS firmware and the
> UEFI firmware.  It also hurts usability since a user now has to make the
> decision as to whether they want to use UEFI or not.  A user should not have
> to even know what EFI is.
>
> So I think the best way forward is to hold off on UEFI in mainline until we
> can provide a single unified stack.

Ok.  I understand if this is the way the QEMU project wants to
proceed.  I think it might delay UEFI support somewhat, but it
certainly would be better for the user if they did not have to provide
the correct command line to boot.

While it is true that a separate machine type could potentially be
viewed as increasing the testing requirements, I am not so sure.  If
QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI
based OS boot and the CSM based legacy OS boot?

>>> OTH, uefi + a CSM based on SeaBIOS could be used as the default firmware.
>>>  I
>>> don't think it's a reasonable target for 0.12 but I think if someone
>>> actively worked on it, it would be possible for future releases.
>>>
>>
>> I don't know of any efforts to create an open source CSM.  Is someone
>> working on converting SeaBIOS to act as a CSM?
>>
>
> There are multiple parties that seem to be interested in uefi for guests.  I
> expect that one of those parties will end up doing the work if that's the
> requirement that we put down.
>
>> For tianocore.org & OVMF we would further be restricted by requiring a
>> BSD licensed CSM.  Of course, if there was a SeaBIOS based CSM, then
>> I'm sure OVMF could be modified to make use of it easily enough.  We
>> just wouldn't be able to have the SeaBIOS CSM on tianocore.org and
>> part of the normal tianocore.org OVMF releases.
>>
>
> I don't think that's a big problem for us.

Yes, I agree that this is would not be a problem for QEMU.  In fact,
I'd like to take back my opinion regarding how this may or may not be
usable by tianocore.org and OVMF.  Although we don't have GPL code
hosted within any of our tianocore.org projects at this time, maybe
this could change...

I do know that we don't intend to start an open source CSM project on
tianocore.org at this time.

But, if such a project was started, I think we would support
development via email on our edk2 dev list, and potentially even host
the project on tianocore.org.

So, I would invite anyone thinking of tackling such a task to join the
edk2 dev list to discuss your work.  Furthermore, if you'd like to
request the project to be hosted on tianocore.org, feel free to ask
our administrators.  (As mentioned previously, we usually deal with
the BSD license on tianocore.org, but please don't let this hold you
back from asking questions, or requesting hosting of your project.)

Thanks,

-Jordan

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 02.10.2009 18:58, Jordan Justen wrote:
> On Fri, Oct 2, 2009 at 06:29, Anthony Liguori <anthony@codemonkey.ws> wrote:
>   
>> So I think the best way forward is to hold off on UEFI in mainline until we
>> can provide a single unified stack.
>>     
>
> While it is true that a separate machine type could potentially be
> viewed as increasing the testing requirements, I am not so sure.  If
> QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI
> based OS boot and the CSM based legacy OS boot?
>   

Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and
have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI
interface), wouldn't that solve all problems in one go? You get an
unified stack and don't have to mess around with different firmware
files because coreboot can read in a Qemu machine config option (or
NVRAM/CMOS) and select the interface based on that.

The hardware init part would be identical for all variants, only the
interface would differ. coreboot works now and has the benefit of
supporting real hardware as well, so the difference between a setup
inside Qemu and a setup on real hardware is minimal.


>>> I don't know of any efforts to create an open source CSM.  Is someone
>>> working on converting SeaBIOS to act as a CSM?
>>>       

The coreboot solution would also avoid converting SeaBIOS because
SeaBIOS already works as coreboot payload (that's how coreboot
developers call the CSM).


> I do know that we don't intend to start an open source CSM project on
> tianocore.org at this time.
>
> But, if such a project was started, I think we would support
> development via email on our edk2 dev list, and potentially even host
> the project on tianocore.org.
>
> So, I would invite anyone thinking of tackling such a task to join the
> edk2 dev list to discuss your work.  Furthermore, if you'd like to
> request the project to be hosted on tianocore.org, feel free to ask
> our administrators.  (As mentioned previously, we usually deal with
> the BSD license on tianocore.org, but please don't let this hold you
> back from asking questions, or requesting hosting of your project.)
>   

I do invite everyone who is interested in having one single ROM file
with EFI+SeaBIOS to join the coreboot list as well. We already have
coreboot+SeaBIOS running (both on hardware and inside Qemu) and
coreboot+tianocore works as well. It has the advantage of requiring zero
changes in coreboot, zero changes in SeaBIOS and (I think) zero changes
in Tianocore.

As a nice side benefit for the EFI specialists, you get the ability to
run EFI on every coreboot supported platform.


If you have any questions, please ask.

Regards,
Carl-Daniel

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Carl-Daniel Hailfinger wrote:
> On 02.10.2009 18:58, Jordan Justen wrote:
>   
>> On Fri, Oct 2, 2009 at 06:29, Anthony Liguori <anthony@codemonkey.ws> wrote:
>>   
>>     
>>> So I think the best way forward is to hold off on UEFI in mainline until we
>>> can provide a single unified stack.
>>>     
>>>       
>> While it is true that a separate machine type could potentially be
>> viewed as increasing the testing requirements, I am not so sure.  If
>> QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI
>> based OS boot and the CSM based legacy OS boot?
>>   
>>     
>
> Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and
> have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI
> interface), wouldn't that solve all problems in one go? You get an
> unified stack and don't have to mess around with different firmware
> files because coreboot can read in a Qemu machine config option (or
> NVRAM/CMOS) and select the interface based on that.
>   

If you want to to work seamlessly, you need to check for the EFI system 
partition to see whether it's an EFI capable OS and then fallback 
gracefully to SeaBIOS boot.

> The hardware init part would be identical for all variants, only the
> interface would differ. coreboot works now and has the benefit of
> supporting real hardware as well, so the difference between a setup
> inside Qemu and a setup on real hardware is minimal.
>   

Tianocore's OVMF project should provide all the required initialization 
for EFI on QEMU.

> The coreboot solution would also avoid converting SeaBIOS because
> SeaBIOS already works as coreboot payload (that's how coreboot
> developers call the CSM).
>   

I'll bite, what's the advantage of doing coreboot + SeaBIOS vs. SeaBIOS 
alone?  Forget about EFI for the moment, should be considering switching 
to coreboot + SeaBIOS for 0.12?

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

[Adding the coreboot list to CC. Please ignore the moderation messages,
your addresses will be whitelisted ASAP.]

On 02.10.2009 20:53, Anthony Liguori wrote:
> Carl-Daniel Hailfinger wrote:
>> Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and
>> have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI
>> interface), wouldn't that solve all problems in one go? You get an
>> unified stack and don't have to mess around with different firmware
>> files because coreboot can read in a Qemu machine config option (or
>> NVRAM/CMOS) and select the interface based on that.
>>   
>
> If you want to to work seamlessly, you need to check for the EFI
> system partition to see whether it's an EFI capable OS and then
> fallback gracefully to SeaBIOS boot.

Hm. Wouldn't that be a layering violation (hw init != reading the disk)
and also cause problems if you want to boot an EFI capable OS from
SeaBIOS? I can think of someone having an EFI bootable disk image who
wants to boot that disk image with EFI and BIOS without having to
repartition.


>> The hardware init part would be identical for all variants, only the
>> interface would differ. coreboot works now and has the benefit of
>> supporting real hardware as well, so the difference between a setup
>> inside Qemu and a setup on real hardware is minimal.   
>
> Tianocore's OVMF project should provide all the required
> initialization for EFI on QEMU.

Yes, but then we'd have two sets of hardware init: OVMF for EFI, SeaBIOS
for SeaBIOS. That also means each hw init codebase has to support the
new Q35 chipset proposed for Qemu.

It's not any better if SeaBIOS gets changed into a CSM for EFI. Then you
have SeaBIOS on top of EFI on top of OVMF. As I understand it, OVMF had
less testing than the Bochs BIOS or SeaBIOS on Qemu.

Jordan, I have to admit that I'm surprised OVMF was apparently created
from scratch although quite a few (established) hardware init solutions
already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old
hobbyist projects. I'd like to understand the reasons for that and fix
them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS). If
you ported/modified existing code, I'd be interested in the original
codebase to learn from it (especially what you had to change).


>> The coreboot solution would also avoid converting SeaBIOS because
>> SeaBIOS already works as coreboot payload (that's how coreboot
>> developers call the CSM).
>>   
>
> I'll bite, what's the advantage of doing coreboot + SeaBIOS vs.
> SeaBIOS alone?  Forget about EFI for the moment, should be considering
> switching to coreboot + SeaBIOS for 0.12?

Advantages:
- Code coverage increase: SeaBIOS is used with coreboot on real
hardware, so the BIOS interface part of SeaBIOS gets a lot more testing
than the Qemu hardware init part of SeaBIOS.

- coreboot supports real 440BX hardware besides Qemu, so the coreboot
init code is exercised more (and there is still a sizable number of such
machines around (clusters), many of them running coreboot).

- Only one ROM image needed. A coreboot ROM can pack the VGA BIOS into
the ROM image and SeaBIOS will automatically load it from there. Same
for network card ROMs (with gPXE etc.).

- coreboot ROMs (including those with SeaBIOS and/or EFI and/or VGABIOS)
are archives and can be listed/edited with cbfstool if you want to know
what's in there.

- coreboot ROMs use compression, so you can stuff more code (and PCI
option ROMs) into smaller ROMs.

- coreboot has pretty verbose hardware init messages (if you want that)
and can be totally silent as well. The messages end up in a log buffer
which can be read by the OS (experimental feature, not available by
default).

There are a lot more advantages, but I want to give other coreboot
developers a chance to chime in. If you add EFI to the mix, the
synergies increase.


Regards,
Carl-Daniel

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Fri, Oct 2, 2009 at 14:39, Carl-Daniel Hailfinger
<c-d.hailfinger.devel.2006@gmx.net> wrote:
> [Adding the coreboot list to CC. Please ignore the moderation messages,
> your addresses will be whitelisted ASAP.]
>
> On 02.10.2009 20:53, Anthony Liguori wrote:
>> Carl-Daniel Hailfinger wrote:
>>> Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and
>>> have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI
>>> interface), wouldn't that solve all problems in one go? You get an
>>> unified stack and don't have to mess around with different firmware
>>> files because coreboot can read in a Qemu machine config option (or
>>> NVRAM/CMOS) and select the interface based on that.
>>>
>>
>> If you want to to work seamlessly, you need to check for the EFI
>> system partition to see whether it's an EFI capable OS and then
>> fallback gracefully to SeaBIOS boot.
>
> Hm. Wouldn't that be a layering violation (hw init != reading the disk)
> and also cause problems if you want to boot an EFI capable OS from
> SeaBIOS? I can think of someone having an EFI bootable disk image who
> wants to boot that disk image with EFI and BIOS without having to
> repartition.
>
>
>>> The hardware init part would be identical for all variants, only the
>>> interface would differ. coreboot works now and has the benefit of
>>> supporting real hardware as well, so the difference between a setup
>>> inside Qemu and a setup on real hardware is minimal.
>>
>> Tianocore's OVMF project should provide all the required
>> initialization for EFI on QEMU.
>
> Yes, but then we'd have two sets of hardware init: OVMF for EFI, SeaBIOS
> for SeaBIOS. That also means each hw init codebase has to support the
> new Q35 chipset proposed for Qemu.
>
> It's not any better if SeaBIOS gets changed into a CSM for EFI. Then you
> have SeaBIOS on top of EFI on top of OVMF. As I understand it, OVMF had
> less testing than the Bochs BIOS or SeaBIOS on Qemu.
>
> Jordan, I have to admit that I'm surprised OVMF was apparently created
> from scratch although quite a few (established) hardware init solutions
> already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old
> hobbyist projects.

The edk2 project provides an infrastructure for building complete UEFI
firmware solutions.  OVMF is a sample platform which demonstrates how
edk2 can be used to build firmware to boot a UEFI platform.

Aside from just this, OVMF is an effort to support VMs on edk2.  (Ie,
more than just QEMU.)  Ideally the project, and code of OVMF could be
used by any VM vendor as a sample for building UEFI compatible
firmware.

Most of the code required to support QEMU was already open sourced on
edk2 before OVMF was launched.  At the time that we started OVMF, it
did not seem like any project was targeting UEFI support on QEMU.
Tristan Gingold had done a port for UEFI QEMU support, but at that
time it did not seem to be functional with the current QEMU, nor
actively developed.

> I'd like to understand the reasons for that and fix
> them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS).

If you want to remove any need for OVMF on QEMU, then I think all that
is needed is to support booting UEFI OS's for both i386 and x86-64.
Of course, we may still have some use for the OVMF project on edk2 as
a sample platform.

> If
> you ported/modified existing code, I'd be interested in the original
> codebase to learn from it (especially what you had to change).

As I mentioned above, a good portion of the code was already available
in edk2.tianocore.org.  Edk2 is BSD licensed, and therefore from a
licensing perspective should be easy for your project to utilize.

>
>>> The coreboot solution would also avoid converting SeaBIOS because
>>> SeaBIOS already works as coreboot payload (that's how coreboot
>>> developers call the CSM).
>>>
>>
>> I'll bite, what's the advantage of doing coreboot + SeaBIOS vs.
>> SeaBIOS alone?  Forget about EFI for the moment, should be considering
>> switching to coreboot + SeaBIOS for 0.12?
>
> Advantages:
> - Code coverage increase: SeaBIOS is used with coreboot on real
> hardware, so the BIOS interface part of SeaBIOS gets a lot more testing
> than the Qemu hardware init part of SeaBIOS.
>
> - coreboot supports real 440BX hardware besides Qemu, so the coreboot
> init code is exercised more (and there is still a sizable number of such
> machines around (clusters), many of them running coreboot).
>
> - Only one ROM image needed. A coreboot ROM can pack the VGA BIOS into
> the ROM image and SeaBIOS will automatically load it from there. Same
> for network card ROMs (with gPXE etc.).
>
> - coreboot ROMs (including those with SeaBIOS and/or EFI and/or VGABIOS)
> are archives and can be listed/edited with cbfstool if you want to know
> what's in there.
>
> - coreboot ROMs use compression, so you can stuff more code (and PCI
> option ROMs) into smaller ROMs.
>
> - coreboot has pretty verbose hardware init messages (if you want that)
> and can be totally silent as well. The messages end up in a log buffer
> which can be read by the OS (experimental feature, not available by
> default).
>
> There are a lot more advantages, but I want to give other coreboot
> developers a chance to chime in. If you add EFI to the mix, the
> synergies increase.
>
>
> Regards,
> Carl-Daniel
>

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 03.10.2009 00:28, Jordan Justen wrote:
> On Fri, Oct 2, 2009 at 14:39, Carl-Daniel Hailfinger
> <c-d.hailfinger.devel.2006@gmx.net> wrote:
>   
>> Jordan, I have to admit that I'm surprised OVMF was apparently created
>> from scratch although quite a few (established) hardware init solutions
>> already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old
>> hobbyist projects.
>>     
>
> The edk2 project provides an infrastructure for building complete UEFI
> firmware solutions.  OVMF is a sample platform which demonstrates how
> edk2 can be used to build firmware to boot a UEFI platform.
>   

I wish to apologize for the misunderstanding. I thought OVMF was only
hardware init. Thanks for correcting me.


> Aside from just this, OVMF is an effort to support VMs on edk2.  (Ie,
> more than just QEMU.)  Ideally the project, and code of OVMF could be
> used by any VM vendor as a sample for building UEFI compatible
> firmware.
>   

How does it relate to real hardware? You mention OVMF as an effort to
support VMs on edk2. Does this mean that VMs need special support or
that real hardware has different needs?


> Most of the code required to support QEMU was already open sourced on
> edk2 before OVMF was launched.  At the time that we started OVMF, it
> did not seem like any project was targeting UEFI support on QEMU.
> Tristan Gingold had done a port for UEFI QEMU support, but at that
> time it did not seem to be functional with the current QEMU, nor
> actively developed.
>   

I see.


>> I'd like to understand the reasons for that and fix
>> them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS).
>>     
>
> If you want to remove any need for OVMF on QEMU, then I think all that
> is needed is to support booting UEFI OS's for both i386 and x86-64.
> Of course, we may still have some use for the OVMF project on edk2 as
> a sample platform.
>   

Given that my original statement incorrectly assumed OVMF was only about
hardware init, please let me try to express what I originally meant (and
which came across with unintended meaning).
I hope to use all EFI support code from OVMF, keep SeaBIOS, and have
coreboot as common hardware init base.


>> If
>> you ported/modified existing code, I'd be interested in the original
>> codebase to learn from it (especially what you had to change).
>>     
>
> As I mentioned above, a good portion of the code was already available
> in edk2.tianocore.org.

Thanks for the pointer.


> Edk2 is BSD licensed, and therefore from a
> licensing perspective should be easy for your project to utilize.
>   

(Speaking with my coreboot hat on.)
The goal of coreboot is not to assimilate and change other projects, but
to provide hardware init for any code which needs it.

After hardware init, it passes control to a payload (SeaBIOS, UEFI,
GRUB2, FILO, Linux, Plan9...). There are no callbacks to coreboot, each
payload is expected to talk to the hardware on its own. Except for ACPI
tables (and a memory map), nothing of coreboot stays in memory after
passing control to that payload. If you want some basic hardware drivers
for your favourite payload, you can use the BSD-licensed libpayload
library in your code, but most payloads (SeaBIOS, GRUB2, ...) have their
own drivers anyway.
Operating systems like Linux and Plan9 which do not need any BIOS/EFI
interface can be stored in the ROM and will be booted directly by
coreboot (if in ROM). Boot loaders like GRUB2 or FILO don't need
BIOS/EFI either, can be stored in ROM and will then be booted directly.
BIOS and EFI code can be used as a coreboot payload in ROM as well. Some
people are even working on making U-Boot available as coreboot payload.

The idea is to have coreboot handle all the hardware-specific
initialization and allow all other code to be mostly hardware-agnostic
(unless said code wants to implement drivers). The clean interface
(well, no interface, coreboot just passes control to the payload) allows
payloads to do whatever they want as long as they provide a single
32-bit entry point coreboot can jump to.


Regards,
Carl-Daniel

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

Carl-Daniel,

It sounds like there is a whole lot of overlap in what coreboot and
tianocore are trying to enable.

The key difference is that tianocore is focused on enabling the UEFI
interfaces within platforms.

OS loaders in UEFI are UEFI applications, and therefore just like in
the case of the UEFI shell (which is a UEFI application), they could
be embedded in the ROM.  So, this is similar to the coreboot payloads
concept.  But, it is not quite as central of a design goal for UEFI
systems as it appears to be within coreboot.

UEFI does provide runtime services that an OS can make use of, so that
sounds a bit different that coreboot.  (Linux does have support for
these interfaces.)  Essentially you can think of UEFI as a better
spec'd version of what the legacy BIOS was, but with more
extensibility designed into the interfaces.

Anyway, it sounds like a useful project might be to develop a UEFI
coreboot payload based on the tianocore.org code.  Our DUET platform
would only work on top of seabios, and OVMF has some overlap in
hardware initialization along with being a VM/QEMU specific solution.
Of course, we are always glad to hear if people are able to make use
of our code, and we like to encourage any UEFI or tianocore related
contributions to tianocore.org.

Thanks,

-Jordan

On Fri, Oct 2, 2009 at 16:05, Carl-Daniel Hailfinger
<c-d.hailfinger.devel.2006@gmx.net> wrote:
> On 03.10.2009 00:28, Jordan Justen wrote:
>> On Fri, Oct 2, 2009 at 14:39, Carl-Daniel Hailfinger
>> <c-d.hailfinger.devel.2006@gmx.net> wrote:
>>
>>> Jordan, I have to admit that I'm surprised OVMF was apparently created
>>> from scratch although quite a few (established) hardware init solutions
>>> already exist for Qemu: SeaBIOS, Bochs BIOS, coreboot, and some old
>>> hobbyist projects.
>>>
>>
>> The edk2 project provides an infrastructure for building complete UEFI
>> firmware solutions.  OVMF is a sample platform which demonstrates how
>> edk2 can be used to build firmware to boot a UEFI platform.
>>
>
> I wish to apologize for the misunderstanding. I thought OVMF was only
> hardware init. Thanks for correcting me.
>
>
>> Aside from just this, OVMF is an effort to support VMs on edk2.  (Ie,
>> more than just QEMU.)  Ideally the project, and code of OVMF could be
>> used by any VM vendor as a sample for building UEFI compatible
>> firmware.
>>
>
> How does it relate to real hardware? You mention OVMF as an effort to
> support VMs on edk2. Does this mean that VMs need special support or
> that real hardware has different needs?
>
>
>> Most of the code required to support QEMU was already open sourced on
>> edk2 before OVMF was launched.  At the time that we started OVMF, it
>> did not seem like any project was targeting UEFI support on QEMU.
>> Tristan Gingold had done a port for UEFI QEMU support, but at that
>> time it did not seem to be functional with the current QEMU, nor
>> actively developed.
>>
>
> I see.
>
>
>>> I'd like to understand the reasons for that and fix
>>> them in coreboot (Kevin O'Connor will probably fix them in SeaBIOS).
>>>
>>
>> If you want to remove any need for OVMF on QEMU, then I think all that
>> is needed is to support booting UEFI OS's for both i386 and x86-64.
>> Of course, we may still have some use for the OVMF project on edk2 as
>> a sample platform.
>>
>
> Given that my original statement incorrectly assumed OVMF was only about
> hardware init, please let me try to express what I originally meant (and
> which came across with unintended meaning).
> I hope to use all EFI support code from OVMF, keep SeaBIOS, and have
> coreboot as common hardware init base.
>
>
>>> If
>>> you ported/modified existing code, I'd be interested in the original
>>> codebase to learn from it (especially what you had to change).
>>>
>>
>> As I mentioned above, a good portion of the code was already available
>> in edk2.tianocore.org.
>
> Thanks for the pointer.
>
>
>> Edk2 is BSD licensed, and therefore from a
>> licensing perspective should be easy for your project to utilize.
>>
>
> (Speaking with my coreboot hat on.)
> The goal of coreboot is not to assimilate and change other projects, but
> to provide hardware init for any code which needs it.
>
> After hardware init, it passes control to a payload (SeaBIOS, UEFI,
> GRUB2, FILO, Linux, Plan9...). There are no callbacks to coreboot, each
> payload is expected to talk to the hardware on its own. Except for ACPI
> tables (and a memory map), nothing of coreboot stays in memory after
> passing control to that payload. If you want some basic hardware drivers
> for your favourite payload, you can use the BSD-licensed libpayload
> library in your code, but most payloads (SeaBIOS, GRUB2, ...) have their
> own drivers anyway.
> Operating systems like Linux and Plan9 which do not need any BIOS/EFI
> interface can be stored in the ROM and will be booted directly by
> coreboot (if in ROM). Boot loaders like GRUB2 or FILO don't need
> BIOS/EFI either, can be stored in ROM and will then be booted directly.
> BIOS and EFI code can be used as a coreboot payload in ROM as well. Some
> people are even working on making U-Boot available as coreboot payload.
>
> The idea is to have coreboot handle all the hardware-specific
> initialization and allow all other code to be mostly hardware-agnostic
> (unless said code wants to implement drivers). The clean interface
> (well, no interface, coreboot just passes control to the payload) allows
> payloads to do whatever they want as long as they provide a single
> 32-bit entry point coreboot can jump to.
>
>
> Regards,
> Carl-Daniel
>

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Peter Stuge]

Jordan Justen wrote:
> Anyway, it sounds like a useful project might be to develop a UEFI
> coreboot payload based on the tianocore.org code.

I believe it might have been done already.

http://www.coreboot.org/File:Tianocoreboot.png


//Peter

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Sat, Oct 3, 2009 at 10:30, Peter Stuge <peter@stuge.se> wrote:
> Jordan Justen wrote:
>> Anyway, it sounds like a useful project might be to develop a UEFI
>> coreboot payload based on the tianocore.org code.
>
> I believe it might have been done already.
>
> http://www.coreboot.org/File:Tianocoreboot.png

That screenshot mentions DUET which is the tianocore.org UEFI emulator
that boots on top of a legacy BIOS.  But, it's unclear if it was just
DUET, or something based modified specifically for coreboot based on
DUET.

I will not dispute that DUET might be a potential solution to achieve
UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not
be able to boot UEFI OS's at this time.)  However, we thought a
project such as OVMF was a more direct approach to achieve UEFI
compatibility for QEMU.

I see that there was a gnufi project that is mentioned on
http://www.coreboot.org/Payloads, but it looks like this
project may be stalled.  ??

-Jordan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Patrick Georgi]

Am Samstag, den 03.10.2009, 14:49 -0700 schrieb Jordan Justen:
> On Sat, Oct 3, 2009 at 10:30, Peter Stuge <peter@stuge.se> wrote:
> > Jordan Justen wrote:
> >> Anyway, it sounds like a useful project might be to develop a UEFI
> >> coreboot payload based on the tianocore.org code.
> >
> > I believe it might have been done already.
> >
> > http://www.coreboot.org/File:Tianocoreboot.png
> 
> That screenshot mentions DUET which is the tianocore.org UEFI emulator
> that boots on top of a legacy BIOS.  But, it's unclear if it was just
> DUET, or something based modified specifically for coreboot based on
> DUET.
Modified: The 16bit loader stub was dropped and replaced by 32bit C code
that did mostly the same (pushing code to the right address, jump to the
real entry point, as determined by some PE data structures).
With some minor modification to Duet it's even possible to remove the
470k limit Duet suffers from, by moving the data chunk to >1MB.

Interaction with BIOS seems to be restricted to expecting the right
tables (ACPI, MP table, SMBIOS stuff), which coreboot can provide.

> be able to boot UEFI OS's at this time.)  However, we thought a
> project such as OVMF was a more direct approach to achieve UEFI
> compatibility for QEMU.
With coreboot, seabios and Duet, it should be reasonably simple to
provide a single BIOS image that selects (based on nvram - ie.
configuration) which interface to provide: PCBIOS or UEFI.

If that is just as simple with OVMF, and easier to maintain, then it's
hard to argue with that.


Patrick

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Patrick Georgi wrote:
> With coreboot, seabios and Duet, it should be reasonably simple to
> provide a single BIOS image that selects (based on nvram - ie.
> configuration) which interface to provide: PCBIOS or UEFI.
>   

That isn't terribly useful for QEMU because it implies that QEMU needs 
to make the decision about which one to use.  If QEMU was capable of 
making that decision without user intervention, then we could just 
select to load PCBIOS or UEFI.

> If that is just as simple with OVMF, and easier to maintain, then it's
> hard to argue with that.
>   

I don't know enough about EFI, but my concern is that down the road, 
hardware devices will start to exist that require EFI because they 
implement EFI modules and not legacy option ROMs.  In order to support 
PCI passthrough of such devices, we will need to provide an EFI capable 
firmware.

It's really just a matter of how long before this becomes a problem for us.

-- 
Regards,

Anthony Liguori

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Stefan Reinauer]

Anthony Liguori wrote:
> Patrick Georgi wrote:
>> With coreboot, seabios and Duet, it should be reasonably simple to
>> provide a single BIOS image that selects (based on nvram - ie.
>> configuration) which interface to provide: PCBIOS or UEFI.
>>   
>
> That isn't terribly useful for QEMU because it implies that QEMU needs
> to make the decision about which one to use.  If QEMU was capable of
> making that decision without user intervention, then we could just
> select to load PCBIOS or UEFI.
Are you guys sure you want to go down that road rather than using an
UEFI with a CSM? There may be cases when you really want to boot legacy
even though your client can do UEFI.

> I don't know enough about EFI, but my concern is that down the road,
> hardware devices will start to exist that require EFI because they
> implement EFI modules and not legacy option ROMs.  In order to support
> PCI passthrough of such devices, we will need to provide an EFI
> capable firmware.
>
> It's really just a matter of how long before this becomes a problem
> for us.
This was attempted for Open Firmware and never happened during its life
time. I doubt there is any incentive for vendors to ship devices without
legacy option roms, especially since there is no advantage in doing so.

Stefan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Stefan Reinauer wrote:
> Anthony Liguori wrote:
>   
>> Patrick Georgi wrote:
>>     
>>> With coreboot, seabios and Duet, it should be reasonably simple to
>>> provide a single BIOS image that selects (based on nvram - ie.
>>> configuration) which interface to provide: PCBIOS or UEFI.
>>>   
>>>       
>> That isn't terribly useful for QEMU because it implies that QEMU needs
>> to make the decision about which one to use.  If QEMU was capable of
>> making that decision without user intervention, then we could just
>> select to load PCBIOS or UEFI.
>>     
> Are you guys sure you want to go down that road rather than using an
> UEFI with a CSM?

We'll I've been advocating UEFI + CSM (based on SeaBIOS) so I'm not sure 
what you meant by 'that road'.

> This was attempted for Open Firmware and never happened during its life
> time. I doubt there is any incentive for vendors to ship devices without
> legacy option roms, especially since there is no advantage in doing so.
>   

We'll be stuck with legacy option roms for a long, long time.  But I 
also expect there will be a few devices out there that only provide EFI 
modules.

-- 
Regards,

Anthony Liguori

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 05.10.2009 15:03, Anthony Liguori wrote:
> Stefan Reinauer wrote:
>> Anthony Liguori wrote:
>>  
>>> Patrick Georgi wrote:
>>>    
>>>> With coreboot, seabios and Duet, it should be reasonably simple to
>>>> provide a single BIOS image that selects (based on nvram - ie.
>>>> configuration) which interface to provide: PCBIOS or UEFI.
>>>>         
>>> That isn't terribly useful for QEMU because it implies that QEMU needs
>>> to make the decision about which one to use.  If QEMU was capable of
>>> making that decision without user intervention, then we could just
>>> select to load PCBIOS or UEFI.
>>>     
>> Are you guys sure you want to go down that road rather than using an
>> UEFI with a CSM?
>
> We'll I've been advocating UEFI + CSM (based on SeaBIOS) so I'm not
> sure what you meant by 'that road'.

What about SeaBIOS + CSM (based on DUET)? That allows you to continue
using all the Qemu specific init code in SeaBIOS and optionally load
UEFI if some hardware needs it.
A SeaBIOS based CSM will probably never get merged/supported at
tianocore.org because of licensing, but the SeaBIOS + DUET solution
should be supportable by upstream.

I can't speak for Patrick, but he probably was concerned about making
EFI the default with BIOS as fallback instead of the other way round.
Forcing any EFI capable (or semi-capable) OS to be booted with EFI
instead of leaving the choice in the hand of the user (NVRAM) or picking
the sane default (what almost all boards out there are doing) sounds
like a non-sustainable way for Qemu.


>> This was attempted for Open Firmware and never happened during its life
>> time. I doubt there is any incentive for vendors to ship devices without
>> legacy option roms, especially since there is no advantage in doing so.
>>   
>
> We'll be stuck with legacy option roms for a long, long time.  But I
> also expect there will be a few devices out there that only provide
> EFI modules.

I expect that it will be some time before we see such devices (maybe
only at trade show demos if at all). It will start to get interesting
once such EFI modules have to interact with classic option ROMs.

Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Carl-Daniel Hailfinger wrote:
> What about SeaBIOS + CSM (based on DUET)?

That's not quite the same thing.

In EFI, CSM is a specification that defines how to port a legacy BIOS 
such that it runs as basically an EFI module providing the old legacy 
BIOS interfaces that OSes support.  If you have a set of legacy option 
roms and efi modules, it defines how all of those things interact with 
each other to provide a consistent experience.

It's is not at all the same as just switching between EFI and BIOS.  
It's much more tightly integrated than that.

> I can't speak for Patrick, but he probably was concerned about making
> EFI the default with BIOS as fallback instead of the other way round.
> Forcing any EFI capable (or semi-capable) OS to be booted with EFI
> instead of leaving the choice in the hand of the user (NVRAM) or picking
> the sane default (what almost all boards out there are doing) sounds
> like a non-sustainable way for Qemu.
>   

Why?  As long as it Just Works, I don't think it will ever even cross a 
users mind.

>> We'll be stuck with legacy option roms for a long, long time.  But I
>> also expect there will be a few devices out there that only provide
>> EFI modules.
>>     
>
> I expect that it will be some time before we see such devices (maybe
> only at trade show demos if at all). It will start to get interesting
> once such EFI modules have to interact with classic option ROMs.
>   

I think at the high end, we'll see these sooner than you think.

Regards,

Anthony Liguori

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 05.10.2009 15:51, Anthony Liguori wrote:
> Carl-Daniel Hailfinger wrote:
>> What about SeaBIOS + CSM (based on DUET)?
>
> That's not quite the same thing.
>
> In EFI, CSM is a specification that defines how to port a legacy BIOS
> such that it runs as basically an EFI module providing the old legacy
> BIOS interfaces that OSes support.  If you have a set of legacy option
> roms and efi modules, it defines how all of those things interact with
> each other to provide a consistent experience.

That's the design, but how do the implementations hold up? If we ignore
the "it's a spec" vs. "it's an industry standard not formalized
anywhere" distinction, DUET is an EFI compatibility layer/plugin/module
on top of a BIOS and a CSM is a BIOS compatibility layer/plugin/module
on top of EFI.
If DUET still uses the BIOS as claimed in the FAQ, is should be able to
reuse the handlers installed by classic option ROMs without problems.


> It's is not at all the same as just switching between EFI and BIOS. 
> It's much more tightly integrated than that.

OK, and how does a user specify "do not, under any circumstance, try to
boot with EFI" if a theoretically EFI capable OS (with broken EFI
support) is on disk? With current Qemu firmware, it just works (because
there is no EFI support). AFAICS EFI by default breaks such installations.


>> I can't speak for Patrick, but he probably was concerned about making
>> EFI the default with BIOS as fallback instead of the other way round.
>> Forcing any EFI capable (or semi-capable) OS to be booted with EFI
>> instead of leaving the choice in the hand of the user (NVRAM) or picking
>> the sane default (what almost all boards out there are doing) sounds
>> like a non-sustainable way for Qemu.  
>
> Why?  As long as it Just Works, I don't think it will ever even cross
> a users mind.

EFI support in enterprise Linux distributions is often not really good.
If the firmware tries EFI booting "just because it can", such
distributions will be booted with an almost untested path.


>>> We'll be stuck with legacy option roms for a long, long time.  But I
>>> also expect there will be a few devices out there that only provide
>>> EFI modules.
>>>     
>>
>> I expect that it will be some time before we see such devices (maybe
>> only at trade show demos if at all). It will start to get interesting
>> once such EFI modules have to interact with classic option ROMs.
>>   
>
> I think at the high end, we'll see these sooner than you think.

I just noticed that one big (>4000 nodes) Dell S1850 cluster deployment
(sort of mentioned as EFI example in older Intel literature) is
considering to drop EFI and use coreboot, maybe with SeaBIOS on top. The
journey ahead will be really interesting, regardless of where it ends.


Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Patrick Georgi]

Am Sonntag, den 04.10.2009, 14:31 -0500 schrieb Anthony Liguori:
> I don't know enough about EFI, but my concern is that down the road, 
> hardware devices will start to exist that require EFI because they 
> implement EFI modules and not legacy option ROMs.  In order to support 
> PCI passthrough of such devices, we will need to provide an EFI capable 
> firmware.
Why? They're initialized at boot time (by the host's EFI firmware, if
necessary), and are usable by the OS then.

Look at OpenFirmware, the drivers initialize the hardware and provide
just enough support to be able to start an operating system, but as soon
as the operating system runs, it takes over as much as possible, as
quickly as possible.

For good reasons, as external drivers (no matter if OF or EFI) impose
the same flaw as ACPI: Hardware state changes outside the control of the
operating system.

Beyond boot loaders and toy kernels, firmware level drivers hurt more
than they help. The world changed since the invention of BIOS in CP/M.


Regards,
Patrick Georgi

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Patrick Georgi wrote:
> Am Sonntag, den 04.10.2009, 14:31 -0500 schrieb Anthony Liguori:
>   
>> I don't know enough about EFI, but my concern is that down the road, 
>> hardware devices will start to exist that require EFI because they 
>> implement EFI modules and not legacy option ROMs.  In order to support 
>> PCI passthrough of such devices, we will need to provide an EFI capable 
>> firmware.
>>     
> Why? They're initialized at boot time (by the host's EFI firmware, if
> necessary), and are usable by the OS then.
>   

EFI defines new interfaces for OSes to use.  A device is going to want 
to provide drivers for those new interfaces.

For instance, EFI defines a SCSI passthrough protocol such that a RAID 
controller could install an EFI SCSI driver and that would allow OSes to 
install to it without requiring special drivers.  Of course, I'm sure 
we'll still see special drivers.

I assume that future RAID controllers will provide both a legacy option 
ROM (int13 hook) and an EFI SCSI driver.  However, I'm also pretty sure 
that we'll see some subset of devices that only provide the EFI modules 
because they target a very specific class of hardware/software that will 
be EFI enabled.

> Beyond boot loaders and toy kernels, firmware level drivers hurt more
> than they help. The world changed since the invention of BIOS in CP/M.
>   

Except that VESA and VGA are still pretty popular.  It's not at all 
uncommon to either use a VESA driver or at least to complete an install 
with just a VESA driver.

-- 
Regards,

Anthony Liguori

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Stefan Reinauer]

Jordan Justen wrote:
> On Sat, Oct 3, 2009 at 10:30, Peter Stuge <peter@stuge.se> wrote:
>   
>> Jordan Justen wrote:
>>     
>>> Anyway, it sounds like a useful project might be to develop a UEFI
>>> coreboot payload based on the tianocore.org code.
>>>       
>> I believe it might have been done already.
>>
>> http://www.coreboot.org/File:Tianocoreboot.png
>>     
>
> That screenshot mentions DUET which is the tianocore.org UEFI emulator
> that boots on top of a legacy BIOS.  But, it's unclear if it was just
> DUET, or something based modified specifically for coreboot based on
> DUET.
>
> I will not dispute that DUET might be a potential solution to achieve
> UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not
> be able to boot UEFI OS's at this time.)  However, we thought a
> project such as OVMF was a more direct approach to achieve UEFI
> compatibility for QEMU.
>   

We have DUET running as a coreboot payload with a small coreboot
specific PE payload loader.

DUET is, however, not an emulator, it is executing much of the same code
as all other TianoCore based UEFI implementations.

It is possible to boot an OS just fine with DUET.

Can you explain what you think would be more direct about OVMF than
about DUET? As far as I understand it's another build target of EDK2 but
besides that shares exactly the same design and even 99% of the code.

Stefan


-- 
coresystems GmbH • Brahmsstr. 16 • D-79104 Freiburg i. Br.
      Tel.: +49 761 7668825 • Fax: +49 761 7664613
Email: info@coresystems.de  • http://www.coresystems.de/
Registergericht: Amtsgericht Freiburg • HRB 7656
Geschäftsführer: Stefan Reinauer • Ust-IdNr.: DE245674866

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Sat, Oct 3, 2009 at 15:02, Stefan Reinauer <stepan@coresystems.de> wrote:
> Jordan Justen wrote:
>> On Sat, Oct 3, 2009 at 10:30, Peter Stuge <peter@stuge.se> wrote:
>>
>>> Jordan Justen wrote:
>>>
>>>> Anyway, it sounds like a useful project might be to develop a UEFI
>>>> coreboot payload based on the tianocore.org code.
>>>>
>>> I believe it might have been done already.
>>>
>>> http://www.coreboot.org/File:Tianocoreboot.png
>>>
>>
>> That screenshot mentions DUET which is the tianocore.org UEFI emulator
>> that boots on top of a legacy BIOS.  But, it's unclear if it was just
>> DUET, or something based modified specifically for coreboot based on
>> DUET.
>>
>> I will not dispute that DUET might be a potential solution to achieve
>> UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not
>> be able to boot UEFI OS's at this time.)  However, we thought a
>> project such as OVMF was a more direct approach to achieve UEFI
>> compatibility for QEMU.
>>
>
> We have DUET running as a coreboot payload with a small coreboot
> specific PE payload loader.

Meaning you bring up a legacy BIOS compatible interface before
starting DUET?  DUET depends on a legacy BIOS.  My point is that a
tianocore.org based coreboot payload ought be able to do away with
this legacy BIOS dependency.

> DUET is, however, not an emulator, it is executing much of the same code
> as all other TianoCore based UEFI implementations.

According to the ReadMe.txt for our edk2 DuetPkg, DUET stands for
Developer's UEFI Emulation.  (Seems a bit of a stretch as an acronym.
:)  But, 'emulation' is a very squishy word at times, and this doesn't
imply that it can't accomplish a lot in terms of UEFI compatibility.

> It is possible to boot an OS just fine with DUET.
>
> Can you explain what you think would be more direct about OVMF than
> about DUET? As far as I understand it's another build target of EDK2 but
> besides that shares exactly the same design and even 99% of the code.

DUET expects that you boot a legacy BIOS, and then you load DUET off
the disk.  Once DUET is loaded, there is a (mostly) UEFI compatible
environment.

OVMF initializes the VM hardware and at the same time brings up the
(mostly) UEFI compatible environment.  This is basically the same
thing that would normally be done in most UEFI compatible systems.  To
me, this is a more direct approach to UEFI compatibility for QEMU.

Both DUET and OVMF have some slight issues with providing a fully
compatible UEFI variable interface.  But I know OVMF can still boot a
UEFI OS, and I thought DUET might have a problem in this area.  But, I
was fairly sure this could have been fixed in DUET, and it appears
that perhaps it has been.

Yes, DUET and OVMF likely share 90+% of the same code.

-Jordan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Stefan Reinauer]

Jordan Justen wrote:
> On Sat, Oct 3, 2009 at 15:02, Stefan Reinauer <stepan@coresystems.de> wrote:
>   
>> Jordan Justen wrote:
>>     
>>> On Sat, Oct 3, 2009 at 10:30, Peter Stuge <peter@stuge.se> wrote:
>>>
>>>       
>>>> Jordan Justen wrote:
>>>>
>>>>         
>>>>> Anyway, it sounds like a useful project might be to develop a UEFI
>>>>> coreboot payload based on the tianocore.org code.
>>>>>
>>>>>           
>>>> I believe it might have been done already.
>>>>
>>>> http://www.coreboot.org/File:Tianocoreboot.png
>>>>
>>>>         
>>> That screenshot mentions DUET which is the tianocore.org UEFI emulator
>>> that boots on top of a legacy BIOS.  But, it's unclear if it was just
>>> DUET, or something based modified specifically for coreboot based on
>>> DUET.
>>>
>>> I will not dispute that DUET might be a potential solution to achieve
>>> UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not
>>> be able to boot UEFI OS's at this time.)  However, we thought a
>>> project such as OVMF was a more direct approach to achieve UEFI
>>> compatibility for QEMU.
>>>
>>>       
>> We have DUET running as a coreboot payload with a small coreboot
>> specific PE payload loader.
>>     
>
> Meaning you bring up a legacy BIOS compatible interface before
> starting DUET? 
No.
>  DUET depends on a legacy BIOS.  
It did not for us, except for the loader which was replaced. We might
have been lucky though...

> My point is that a tianocore.org based coreboot payload ought be able to do away with
> this legacy BIOS dependency.
>   
Absolutely agreed.


At some point it might make sense to have a coreboot specific target
next to OVMF and DUET, some corebootPkg with specific adaptions and the
loader integrated.

The requirements for a coreboot target are very similar to those of OVMF
and/or Duet, I guess. No hardware specific code is required, but in
addition to what OVMF provides, we feature an in-flash filesystem and we
export a coreboot table which contains memory information, cmos layout
among other things...

What are the chances to get something like this integrated upstream
TianoCore.org?

>> Can you explain what you think would be more direct about OVMF than
>> about DUET? As far as I understand it's another build target of EDK2 but
>> besides that shares exactly the same design and even 99% of the code.
>>     
>
> DUET expects that you boot a legacy BIOS, and then you load DUET off
> the disk. 
It does expect a few tables, but does not seem to make any 16bit calls
once loaded.
>  Once DUET is loaded, there is a (mostly) UEFI compatible
> environment.
>   
I'm curious on the "mostly" here... what's missing? We certainly want to
make sure what we do is fully UEFI compatible.
> Both DUET and OVMF have some slight issues with providing a fully
> compatible UEFI variable interface. 
Is that about saving settings in an NVRAM/flash memory?



Stefan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Sat, Oct 3, 2009 at 16:03, Stefan Reinauer <stepan@coresystems.de> wrote:
> Jordan Justen wrote:
>> On Sat, Oct 3, 2009 at 15:02, Stefan Reinauer <stepan@coresystems.de> wrote:
>>
>>> Jordan Justen wrote:
>>>
>>>> On Sat, Oct 3, 2009 at 10:30, Peter Stuge <peter@stuge.se> wrote:
>>>>
>>>>
>>>>> Jordan Justen wrote:
>>>>>
>>>>>
>>>>>> Anyway, it sounds like a useful project might be to develop a UEFI
>>>>>> coreboot payload based on the tianocore.org code.
>>>>>>
>>>>>>
>>>>> I believe it might have been done already.
>>>>>
>>>>> http://www.coreboot.org/File:Tianocoreboot.png
>>>>>
>>>>>
>>>> That screenshot mentions DUET which is the tianocore.org UEFI emulator
>>>> that boots on top of a legacy BIOS.  But, it's unclear if it was just
>>>> DUET, or something based modified specifically for coreboot based on
>>>> DUET.
>>>>
>>>> I will not dispute that DUET might be a potential solution to achieve
>>>> UEFI compatibility for QEMU.  (I'm not sure, but I think DUET may not
>>>> be able to boot UEFI OS's at this time.)  However, we thought a
>>>> project such as OVMF was a more direct approach to achieve UEFI
>>>> compatibility for QEMU.
>>>>
>>>>
>>> We have DUET running as a coreboot payload with a small coreboot
>>> specific PE payload loader.
>>>
>>
>> Meaning you bring up a legacy BIOS compatible interface before
>> starting DUET?
> No.
>>  DUET depends on a legacy BIOS.
> It did not for us, except for the loader which was replaced. We might
> have been lucky though...
>
>> My point is that a tianocore.org based coreboot payload ought be able to do away with
>> this legacy BIOS dependency.
>>
> Absolutely agreed.
>
>
> At some point it might make sense to have a coreboot specific target
> next to OVMF and DUET, some corebootPkg with specific adaptions and the
> loader integrated.
>LegacyBiosInt86
> The requirements for a coreboot target are very similar to those of OVMF
> and/or Duet, I guess. No hardware specific code is required, but in
> addition to what OVMF provides, we feature an in-flash filesystem and we
> export a coreboot table which contains memory information, cmos layout
> among other things...
>
> What are the chances to get something like this integrated upstream
> TianoCore.org?

I'll refrain from making a prediction on this.  But, by my view,
tianocore.org is supposed to be an open source community that
encourages UEFI related contributions.

However, (at this time) it is going to tougher to get it included at
tianocore.org if it is not BSD licensed.  I would also suggest
discussing the situation on the tianocore.org email lists beforehand
to get somewhat of a confirmation before investing a lot into it.

>>> Can you explain what you think would be more direct about OVMF than
>>> about DUET? As far as I understand it's another build target of EDK2 but
>>> besides that shares exactly the same design and even 99% of the code.
>>>
>>
>> DUET expects that you boot a legacy BIOS, and then you load DUET off
>> the disk.
> It does expect a few tables, but does not seem to make any 16bit calls
> once loaded.
>>  Once DUET is loaded, there is a (mostly) UEFI compatible
>> environment.
>>
> I'm curious on the "mostly" here... what's missing? We certainly want to
> make sure what we do is fully UEFI compatible.

I think tianocore.org will not call these fully UEFI compatible
projects, since that implies a lot.  Normally these platforms are not
run through the UEFI Self-Certification Tests, for example.
(Although, this is something I plan to try on OVMF at some point.)

Also, there is the variable situation.  (see below)

>> Both DUET and OVMF have some slight issues with providing a fully
>> compatible UEFI variable interface.
> Is that about saving settings in an NVRAM/flash memory?

Yes.  Neither platform provides proper UEFI NV variables support.  The
NV variables can be lost depending on when the platform is shut off,
and when the variable has been written to.

But in addition, for DUET, I thought that accessing the NV variable
services at OS runtime might not work correctly.  (Perhaps cause an
exception.  Perhaps just always return an error.)

-Jordan

Re: [Qemu-devel] Release plan for 0.12.0

[Gleb Natapov]

On Fri, Oct 02, 2009 at 11:39:50PM +0200, Carl-Daniel Hailfinger wrote:
> > I'll bite, what's the advantage of doing coreboot + SeaBIOS vs.
> > SeaBIOS alone?  Forget about EFI for the moment, should be considering
> > switching to coreboot + SeaBIOS for 0.12?
> 
> Advantages:
> - Code coverage increase: SeaBIOS is used with coreboot on real
> hardware, so the BIOS interface part of SeaBIOS gets a lot more testing
> than the Qemu hardware init part of SeaBIOS.
> 
> - coreboot supports real 440BX hardware besides Qemu, so the coreboot
> init code is exercised more (and there is still a sizable number of such
> machines around (clusters), many of them running coreboot).
> 
QEMU goal is to run popular OSes and OS usually don't care about low level
chipset details. That's way QEMU's chipset emulation implements only
bare minimum that is needed for OS to run: no need to initialize memory
controller for instance. Coreboot goal, on the other hand, is to talk to
real HW, so to use it on QEMU will require implementing chipset features
that are not needed to accomplish QEMU goal. So for me real 440BX
hardware support of coreboot is actually disadvantage. QEMU don't
have real 440BX hardware and there is not point in having one. It is
possible to implement 440BX-qemu support in coreboot of course if there
are other advantages worth having.

> - Only one ROM image needed. A coreboot ROM can pack the VGA BIOS into
> the ROM image and SeaBIOS will automatically load it from there. Same
> for network card ROMs (with gPXE etc.).
> 
> - coreboot ROMs (including those with SeaBIOS and/or EFI and/or VGABIOS)
> are archives and can be listed/edited with cbfstool if you want to know
> what's in there.
> 
> - coreboot ROMs use compression, so you can stuff more code (and PCI
> option ROMs) into smaller ROMs.
> 
> - coreboot has pretty verbose hardware init messages (if you want that)
> and can be totally silent as well. The messages end up in a log buffer
> which can be read by the OS (experimental feature, not available by
> default).
> 
> There are a lot more advantages, but I want to give other coreboot
> developers a chance to chime in. If you add EFI to the mix, the
> synergies increase.
> 
> 
> Regards,
> Carl-Daniel
> 

--
			Gleb.

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Peter Stuge]

Hi Gleb,

Gleb Natapov wrote:
> So for me real 440BX hardware support of coreboot is actually
> disadvantage. QEMU don't have real 440BX hardware and there is not
> point in having one. It is possible to implement 440BX-qemu support
> in coreboot of course if there are other advantages worth having.

coreboot supports both Intel 440BX and QEMU 440BX since many years.
The QEMU support in particular is used intensively by many coreboot
developers.


//Peter

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[ron minnich]

I use qemu for a lot of coreboot work. I really depend on qemu for
many things I do, not just coreboot related. The qemu target in
coreboot has been very heavily used by us to test out new ideas.

That said, I don't see a compelling need to augment seabios with
coreboot on qemu *in the standard distribution*. If seabios gets the
job done, and gets OSes booted, I think that's sufficient. I don't see
a need to complicate anyone's life with something that is, after all,
a sideshow for qemu users.

Conversely, I don't see the need to add the huge pile of stuff that
comes with UEFI/OVMF/whatever to qemu either. One might argue that
having any BIOS callbacks in the OS is a huge mistake, and certainly
I've learned in practice that this argument is true.

thanks

ron

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Gleb Natapov]

On Sat, Oct 03, 2009 at 10:40:35AM -0700, ron minnich wrote:
> I use qemu for a lot of coreboot work. I really depend on qemu for
> many things I do, not just coreboot related. The qemu target in
> coreboot has been very heavily used by us to test out new ideas.
> 
> That said, I don't see a compelling need to augment seabios with
> coreboot on qemu *in the standard distribution*. If seabios gets the
> job done, and gets OSes booted, I think that's sufficient. I don't see
> a need to complicate anyone's life with something that is, after all,
> a sideshow for qemu users.
> 
Exactly. I am glad to hear that coreboot has support for QEMU,
but seabios does the job already, so why add more layers?

> Conversely, I don't see the need to add the huge pile of stuff that
> comes with UEFI/OVMF/whatever to qemu either. One might argue that
> having any BIOS callbacks in the OS is a huge mistake, and certainly
> I've learned in practice that this argument is true.
> 
> thanks
> 
> ron

--
			Gleb.

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Peter Stuge]

Gleb Natapov wrote:
> Exactly. I am glad to hear that coreboot has support for QEMU,
> but seabios does the job already, so why add more layers?

If SeaBIOS does not need any code at all for QEMU machine init I
agree there's no point in considering coreboot.

If QEMU machine specific init is in fact needed in that SeaBIOS, then
the answer isn't as obvious..


//Peter

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Kevin O'Connor]

On Sat, Oct 03, 2009 at 08:30:30PM +0200, Peter Stuge wrote:
> Gleb Natapov wrote:
> > Exactly. I am glad to hear that coreboot has support for QEMU,
> > but seabios does the job already, so why add more layers?
> 
> If SeaBIOS does not need any code at all for QEMU machine init I
> agree there's no point in considering coreboot.

SeaBIOS contains the code necessary to initialize the qemu hardware.

For the details see src/mtrr.c, src/pciinit.c, src/shadow.c, and
src/smm.c.  These codepaths are disabled when SeaBIOS is compiled for
coreboot.

-Kevin

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Gleb Natapov]

On Sat, Oct 03, 2009 at 08:30:30PM +0200, Peter Stuge wrote:
> Gleb Natapov wrote:
> > Exactly. I am glad to hear that coreboot has support for QEMU,
> > but seabios does the job already, so why add more layers?
> 
> If SeaBIOS does not need any code at all for QEMU machine init I
> agree there's no point in considering coreboot.
> 
The code is there already.

> If QEMU machine specific init is in fact needed in that SeaBIOS, then
> the answer isn't as obvious..
> 
May be it is possible to drop seabios init code and use coreboot, but I
prefer to dial with one codebase for all BIOS needs. By moving to seabios
we try to improve situation not make it worse.

--
			Gleb.

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Sat, Oct 3, 2009 at 10:40, ron minnich <rminnich@gmail.com> wrote:
> I use qemu for a lot of coreboot work. I really depend on qemu for
> many things I do, not just coreboot related. The qemu target in
> coreboot has been very heavily used by us to test out new ideas.
>
> That said, I don't see a compelling need to augment seabios with
> coreboot on qemu *in the standard distribution*. If seabios gets the
> job done, and gets OSes booted, I think that's sufficient. I don't see
> a need to complicate anyone's life with something that is, after all,
> a sideshow for qemu users.
>
> Conversely, I don't see the need to add the huge pile of stuff that
> comes with UEFI/OVMF/whatever to qemu either. One might argue that

This is a valid argument right now.  OS X is the only OS today that
targets UEFI, and specifically not legacy BIOS.  But, in 5 ~ 10 years
that might not be the case.

I'll admit that this is a fairly dumb argument to make while we are
talking about a QEMU release only a few months from now.  But, as UEFI
seems to be gaining ground in the industry, I think the sooner QEMU
can get this support, the better.

We're specifically trying to help out with this with OVMF.  But if a
better solution is developed, then so be it.

> having any BIOS callbacks in the OS is a huge mistake, and certainly
> I've learned in practice that this argument is true.

I'm not going to take a side on this matter.  But, I think what will
be more important is what is used in the majority of OS's and systems.
 This is why we still put the 16-bit legacy BIOS as the #1 priority
after ~30 years.  But, like I mention, I think there are signs that
this may shift towards UEFI at some point.

-Jordan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Patrick Georgi]

Am Samstag, den 03.10.2009, 15:13 -0700 schrieb Jordan Justen:
> I'll admit that this is a fairly dumb argument to make while we are
> talking about a QEMU release only a few months from now.  But, as UEFI
> seems to be gaining ground in the industry, I think the sooner QEMU
> can get this support, the better.
This smells like self-fulfulling prophecy: Let QEmu support EFI in the
hope that EFI actually gains ground (for example by better testability
due to available emulation environments)

So you want QEmu as a marketing device - nothing wrong with saying that,
right?


Regards,
Patrick

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

On Sat, Oct 3, 2009 at 15:19, Patrick Georgi <patrick@georgi-clan.de> wrote:
> Am Samstag, den 03.10.2009, 15:13 -0700 schrieb Jordan Justen:
>> I'll admit that this is a fairly dumb argument to make while we are
>> talking about a QEMU release only a few months from now.  But, as UEFI
>> seems to be gaining ground in the industry, I think the sooner QEMU
>> can get this support, the better.
> This smells like self-fulfulling prophecy: Let QEmu support EFI in the
> hope that EFI actually gains ground (for example by better testability
> due to available emulation environments)
>
> So you want QEmu as a marketing device - nothing wrong with saying that,
> right?

I'm not in marketing. :)  But, I do work for Intel, on tianocore.org
and thus UEFI.  I've been working with Tiano/EFI since ~2003, when
Intel started converting its desktop motherboards over to a Tiano code
base.  So yes, I have a bias.

I think (but no, I cannot back this up) that tens of millions of UEFI
compatible motherboards are shipping out each year now.  Microsoft has
implemented UEFI support in Vista and Win7.  Several Linux vendor have
or are enabling UEFI support now.  Mac OS X implements UEFI.

My point?  Well, while I think QEMU support for UEFI is still valuable
to help support UEFI adoption, I think it could have done a lot more
for UEFI if it was done several years ago. :)

On the flip side, I also think it could have done a lot more to assist
Linux to have first rate UEFI support now if it was done several years
ago. :(

-Jordan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Jordan Justen wrote:
>> So you want QEmu as a marketing device - nothing wrong with saying that,
>> right?
>>     
>
> I'm not in marketing. :)  But, I do work for Intel, on tianocore.org
> and thus UEFI.  I've been working with Tiano/EFI since ~2003, when
> Intel started converting its desktop motherboards over to a Tiano code
> base.  So yes, I have a bias.
>
> I think (but no, I cannot back this up) that tens of millions of UEFI
> compatible motherboards are shipping out each year now.  Microsoft has
> implemented UEFI support in Vista and Win7.  Several Linux vendor have
> or are enabling UEFI support now.  Mac OS X implements UEFI.
>
> My point?  Well, while I think QEMU support for UEFI is still valuable
> to help support UEFI adoption, I think it could have done a lot more
> for UEFI if it was done several years ago. :)
>   

Many major vendors (like IBM) are starting to ship (and will eventually 
exclusively ship) UEFI firmwares.  Hardware, particularly the kind 
targeted at higher end systems, will eventually start to assume the 
presence of EFI.  If we want to support PCI passthrough of these 
devices, then we'll have to support EFI in some form.

-- 
Regards,

Anthony Liguori

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

I don't think that's the case.

EFI actually DOES HAVE ground.

Itanium machines are still worldwide used, manufactured and sold.
Intel-based Macintoshes used them and ALL of their facilities.

The disadvantages can be a lot.
The advantages that I see, are the following (implemented by Apple's  
EFI):
Hardware drivers, so the OS loader can use ANY hardware present.
Hardware testing easily programable, as you can use the EFI to call  
the hardware (unlike PC diagnostics that makes direct and real mode  
calls to the hardware, making them imposible to test different  
hardware without implementing all variations -SCSI cards, wifi cards,  
so on-)
Filesystem independent bootloader (it just expects the EFI to have the  
filesystem and disk driver, then searches the disk for the OS loader)
Upgradable drivers without firmware patching (so if I add a wifi card,  
I can put a driver for netbooting it)
Extensive input devices support (USB keyb and mice, bluetooth keyb and  
mice and infrared remote)

I think that this is impossible (if not nearly to) make using BIOS.
I think it is only possible with OpenFirmware or EFI.

And I prefer EFI for the matters of being programable in C (personal  
distaste for Forth) and the EFI System Partition usability.

A bootloader can fuckinly easy put a good splash without limiting to  
12 colors or making calls to the VGA system (for example). What will  
happen with the GRUB if tomorrow VGA disappears? What a mess...

And I don't work for Intel, IBM, HP, Apple, etc.

In QEMU' side I see that maybe not for 0.12.0 but anyway, EFI is a  
MUST have if we want to emulate, support, test, patch, anything that  
uses it, starting with Itanium, continuing with Intel Macintosh, and  
finishing with all the thousands of PCs (not only from Intel, as I've  
seen a bunch from Asus and Asrock) that instead of using a BIOS are  
using a hidden EFI with a boot-by-default CSM.

I'm sure, EFI will prevail over BIOS and sooner or later, also over  
OpenFirmware.
And we need to move with the world, not against it.

Regards,
Natalia Portillo

El 03/10/2009, a las 23:19, Patrick Georgi escribió:

> Am Samstag, den 03.10.2009, 15:13 -0700 schrieb Jordan Justen:
>> I'll admit that this is a fairly dumb argument to make while we are
>> talking about a QEMU release only a few months from now.  But, as  
>> UEFI
>> seems to be gaining ground in the industry, I think the sooner QEMU
>> can get this support, the better.
> This smells like self-fulfulling prophecy: Let QEmu support EFI in the
> hope that EFI actually gains ground (for example by better testability
> due to available emulation environments)
>
> So you want QEmu as a marketing device - nothing wrong with saying  
> that,
> right?
>
>
> Regards,
> Patrick
>
>
>
>

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

Hi Natalia,

I tried to understand your points. Please correct me where I'm wrong.

On 04.10.2009 06:10, Natalia Portillo wrote:
> EFI actually DOES HAVE ground.
>
> Itanium machines are still worldwide used, manufactured and sold.

Unless I'm mistaken, Itanium is not x86. Now if we consider EFI for x86
because it has solid standing on Itanium, shouldn't we also consider
U-Boot and OpenFirmware for x86 because they have solid standing on
ARM/PowerPC?


> Intel-based Macintoshes used them and ALL of their facilities.

True.


> The disadvantages can be a lot.
> The advantages that I see, are the following (implemented by Apple's
> EFI):
> Hardware drivers, so the OS loader can use ANY hardware present.

Same for BIOS.


> Hardware testing easily programable, as you can use the EFI to call
> the hardware (unlike PC diagnostics that makes direct and real mode
> calls to the hardware, making them imposible to test different
> hardware without implementing all variations -SCSI cards, wifi cards,
> so on-)

Same for BIOS (the only difference is that the interface to the BIOS
provides a 16bit interface and EFI provides a 32bit interface).


> Filesystem independent bootloader (it just expects the EFI to have the
> filesystem and disk driver, then searches the disk for the OS loader)

Burn a BIOS together with DOS in the ROM and you get the same
functionality. I have seen youtube videos of such systems.


> Upgradable drivers without firmware patching (so if I add a wifi card,
> I can put a driver for netbooting it)

Install DOS with a boot agent on the disk and you get exactly the same
feature.


> Extensive input devices support (USB keyb and mice, bluetooth keyb and
> mice and infrared remote)

AFAIK USB keyboard+mouse is supported in pretty much every BIOS out
there. Not sure about bluetooth keyboard and infrared remote, but the
boards with builtin bluetooth support probably also have a bluetooth
keyboard driver in the BIOS.


> I think that this is impossible (if not nearly to) make using BIOS. I
> think it is only possible with OpenFirmware or EFI. 

Except for bluetooth keyboard and infrared remote which I'm not sure
about, everything you listed above is the same for BIOS and EFI.

> And I prefer EFI for the matters of being programable in C (personal
> distaste for Forth)

Now the Forth argument is something I can agree with. ;-)


> and the EFI System Partition usability.

I don't understand what an EFI System Partition is used for. The
wikipedia entry suggests that it is essentially a special partition
where bootloaders live (instead of using a normal partition for that)
and where EFI loads drivers and DOS/Windows PE commandline executables
from. That sounds suspiciously like a DOS partition with NTLDR and loadlin.


> A bootloader can fuckinly easy put a good splash without limiting to
> 12 colors or making calls to the VGA system (for example). What will
> happen with the GRUB if tomorrow VGA disappears? What a mess...

I've seen quite a few GRUB installations with 24-bit color splash
screens in high resolutions. Graphics hardware won't disappear from
computers any time soon, only the interfaces may change.


> And I don't work for Intel, IBM, HP, Apple, etc.
>
> In QEMU' side I see that maybe not for 0.12.0 but anyway, EFI is a
> MUST have if we want to emulate, support, test, patch, anything that
> uses it, starting with Itanium, continuing with Intel Macintosh, and
> finishing with all the thousands of PCs (not only from Intel, as I've
> seen a bunch from Asus and Asrock) that instead of using a BIOS are
> using a hidden EFI with a boot-by-default CSM.

Interesting. Do these boards still have a 20-30 second wait time from
poweron to bootloader? I've seen coreboot+SeaBIOS achieve 1 second from
poweron to bootloader on real x86 hardware, so I always wonder what the
commercial BIOSes do during the time we wait for them.


> I'm sure, EFI will prevail over BIOS and sooner or later, also over
> OpenFirmware.

I'm not so sure about that. EFI is "cool", but so were Linux netbooks
and nowadays people buy mostly Windows netbooks.


> And we need to move with the world, not against it.

Fully agreed.


Many years ago, some EFI proponent told me: "EFI is great. It is a
32-bit BIOS with builtin 32-bit DOS. Now every operating system can use
EFI drivers in compatibility mode like Windows 95 used DOS drivers in
compatibility mode."
That statement had a lasting impression on me and I've yet to see any
explanation why that statement would be incorrect. I'm very willing to
learn, so if I got something wrong, please educate me.


Regards,
Carl-Daniel

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

El 04/10/2009, a las 12:16, Carl-Daniel Hailfinger escribió:

> Hi Natalia,
>
> I tried to understand your points. Please correct me where I'm wrong.
>
> On 04.10.2009 06:10, Natalia Portillo wrote:
>> EFI actually DOES HAVE ground.
>>
>> Itanium machines are still worldwide used, manufactured and sold.
>
> Unless I'm mistaken, Itanium is not x86. Now if we consider EFI for  
> x86
> because it has solid standing on Itanium, shouldn't we also consider
> U-Boot and OpenFirmware for x86 because they have solid standing on
> ARM/PowerPC?
EFI is used in x86 as well, contrary to OF.

>
>> Intel-based Macintoshes used them and ALL of their facilities.
>
> True.
>
>
>> The disadvantages can be a lot.
>> The advantages that I see, are the following (implemented by Apple's
>> EFI):
>> Hardware drivers, so the OS loader can use ANY hardware present.
>
> Same for BIOS.
BIOS searches for them in ROM space.
No manufacturer ROM, no driver.

>
>> Hardware testing easily programable, as you can use the EFI to call
>> the hardware (unlike PC diagnostics that makes direct and real mode
>> calls to the hardware, making them imposible to test different
>> hardware without implementing all variations -SCSI cards, wifi cards,
>> so on-)
>
> Same for BIOS (the only difference is that the interface to the BIOS
> provides a 16bit interface and EFI provides a 32bit interface).
Not only a 16 bit interface, REAL MODE interface.
Your driver can crash my driver easily.

>
>> Filesystem independent bootloader (it just expects the EFI to have  
>> the
>> filesystem and disk driver, then searches the disk for the OS loader)
>
> Burn a BIOS together with DOS in the ROM and you get the same
> functionality. I have seen youtube videos of such systems.
DOS is not a filesystem independent bootloader.
And filesystem drivers for DOS are not drivers, hacks and traps to INT  
21h.

>
>> Upgradable drivers without firmware patching (so if I add a wifi  
>> card,
>> I can put a driver for netbooting it)
>
> Install DOS with a boot agent on the disk and you get exactly the same
> feature.
>
>> Extensive input devices support (USB keyb and mice, bluetooth keyb  
>> and
>> mice and infrared remote)
>
> AFAIK USB keyboard+mouse is supported in pretty much every BIOS out
> there. Not sure about bluetooth keyboard and infrared remote, but the
> boards with builtin bluetooth support probably also have a bluetooth
> keyboard driver in the BIOS.
Take any of it, no support.
I know a couple of Asus and Abit motherboards with integrated Wifi and  
Bluetooth.

Yet to see wifi netboot in ANY wifi card with bios.
Seen EFI netboot in Apple wifis.

>
>> I think that this is impossible (if not nearly to) make using BIOS. I
>> think it is only possible with OpenFirmware or EFI.
>
> Except for bluetooth keyboard and infrared remote which I'm not sure
> about, everything you listed above is the same for BIOS and EFI.
>
>> And I prefer EFI for the matters of being programable in C (personal
>> distaste for Forth)
>
> Now the Forth argument is something I can agree with. ;-)
>
>
>> and the EFI System Partition usability.
>
> I don't understand what an EFI System Partition is used for. The
> wikipedia entry suggests that it is essentially a special partition
> where bootloaders live (instead of using a normal partition for that)
> and where EFI loads drivers and DOS/Windows PE commandline executables
> from. That sounds suspiciously like a DOS partition with NTLDR and  
> loadlin.
The partition is empty by default.
EFI expects a directory structure and loads drivers from this  
partition if present, without loading any bootloader.
That allows it to load a bootloader from an ext3 filesystem, a direct  
ELF kernel from XFS, or a netboot updated driver for a WiMAX (for  
example).

>
>> A bootloader can fuckinly easy put a good splash without limiting to
>> 12 colors or making calls to the VGA system (for example). What will
>> happen with the GRUB if tomorrow VGA disappears? What a mess...
>
> I've seen quite a few GRUB installations with 24-bit color splash
> screens in high resolutions. Graphics hardware won't disappear from
> computers any time soon, only the interfaces may change.
So, say me how!

>
>> And I don't work for Intel, IBM, HP, Apple, etc.
>>
>> In QEMU' side I see that maybe not for 0.12.0 but anyway, EFI is a
>> MUST have if we want to emulate, support, test, patch, anything that
>> uses it, starting with Itanium, continuing with Intel Macintosh, and
>> finishing with all the thousands of PCs (not only from Intel, as I've
>> seen a bunch from Asus and Asrock) that instead of using a BIOS are
>> using a hidden EFI with a boot-by-default CSM.
>
> Interesting. Do these boards still have a 20-30 second wait time from
> poweron to bootloader? I've seen coreboot+SeaBIOS achieve 1 second  
> from
> poweron to bootloader on real x86 hardware, so I always wonder what  
> the
> commercial BIOSes do during the time we wait for them.
Nope. The only "wait" is the normal Apple POST. Then the CSM takes 1  
or 2 seconds to load the OS.

>
>> I'm sure, EFI will prevail over BIOS and sooner or later, also over
>> OpenFirmware.
>
> I'm not so sure about that. EFI is "cool", but so were Linux netbooks
> and nowadays people buy mostly Windows netbooks.
>
>
>> And we need to move with the world, not against it.
>
> Fully agreed.
>
>
> Many years ago, some EFI proponent told me: "EFI is great. It is a
> 32-bit BIOS with builtin 32-bit DOS. Now every operating system can  
> use
> EFI drivers in compatibility mode like Windows 95 used DOS drivers in
> compatibility mode."
> That statement had a lasting impression on me and I've yet to see any
> explanation why that statement would be incorrect. I'm very willing to
> learn, so if I got something wrong, please educate me.

EFI is not a 32-bit BIOS, it is a 32 or 64 bit protected mode hardware  
initialization and booting interface.

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

On 04.10.2009 18:06, Natalia Portillo wrote:
>
> El 04/10/2009, a las 12:16, Carl-Daniel Hailfinger escribió:
>
>> Hi Natalia,
>>
>> I tried to understand your points. Please correct me where I'm wrong.
>>
>> On 04.10.2009 06:10, Natalia Portillo wrote:
>>> EFI actually DOES HAVE ground.
>>>
>>> Itanium machines are still worldwide used, manufactured and sold.
>>
>> Unless I'm mistaken, Itanium is not x86. Now if we consider EFI for x86
>> because it has solid standing on Itanium, shouldn't we also consider
>> U-Boot and OpenFirmware for x86 because they have solid standing on
>> ARM/PowerPC?
> EFI is used in x86 as well, contrary to OF.

All OLPC laptops (which are x86) have OF. Not sure how many of them are
already deployed, but I think it's roughly a million. (Side note: That
OF implementation has a builtin BIOS emulator/CSM in newer versions to
be able to boot Windows).


>>> The disadvantages can be a lot.
>>> The advantages that I see, are the following (implemented by Apple's
>>> EFI):
>>> Hardware drivers, so the OS loader can use ANY hardware present.
>>
>> Same for BIOS.
> BIOS searches for them in ROM space.
> No manufacturer ROM, no driver.

EFI searches the drivers on disk, so it's like booting DOS from disk
which has drivers.


>>> Hardware testing easily programable, as you can use the EFI to call
>>> the hardware (unlike PC diagnostics that makes direct and real mode
>>> calls to the hardware, making them imposible to test different
>>> hardware without implementing all variations -SCSI cards, wifi cards,
>>> so on-)
>>
>> Same for BIOS (the only difference is that the interface to the BIOS
>> provides a 16bit interface and EFI provides a 32bit interface).
> Not only a 16 bit interface, REAL MODE interface.
> Your driver can crash my driver easily.

OK, now I'm intrigued. This implies that an EFI driver can not crash the
OS easily. Given that programming bugs sometimes are really nasty, does
the "not crashing easily" promise apply even if it deliberately tries to
overwrite all of the physical RAM (such driver bugs have existed in the
past, I even have a link somewhere). But that also means (if we ignore
virtualization for a moment) that any OS using an EFI driver has to call
the EFI driver in ring 3 because the calling the EFI driver in ring 0
would negate any memory protection. Is that really the case?


>>> Filesystem independent bootloader (it just expects the EFI to have the
>>> filesystem and disk driver, then searches the disk for the OS loader)
>>
>> Burn a BIOS together with DOS in the ROM and you get the same
>> functionality. I have seen youtube videos of such systems.
> DOS is not a filesystem independent bootloader.

Neither is EFI.
Linux and DOS and EFI can have file system drivers for various file
systems. They all can execute binaries and those binaries can be
bootloaders. Such bootloaders have the option of either using the
existing VFS interface (to use a Linux term for the generic interface to
file systems) or to implement their own file system drivers to search
the various file systems on disk for bootable operating systems.
loadlin is an example which uses the file system drivers present in DOS
and (assuming that DOS has NTFS or ext2 drivers installed) fits your
description of a file system independent bootloader.


> And filesystem drivers for DOS are not drivers, hacks and traps to INT
> 21h.

Erm. Which filesystem driver for DOS is not hooking INT 21h?


>>> Extensive input devices support (USB keyb and mice, bluetooth keyb and
>>> mice and infrared remote)
>>
>> AFAIK USB keyboard+mouse is supported in pretty much every BIOS out
>> there. Not sure about bluetooth keyboard and infrared remote, but the
>> boards with builtin bluetooth support probably also have a bluetooth
>> keyboard driver in the BIOS.
> Take any of it, no support.
> I know a couple of Asus and Abit motherboards with integrated Wifi and
> Bluetooth.
>
> Yet to see wifi netboot in ANY wifi card with bios.
> Seen EFI netboot in Apple wifis.

Given that EFI loads these wifi drivers for netbooting from disk as per
your earlier statement about on-disk EFI drivers, does that mean wifi
netbooting breaks once you remove the disk?


>> I don't understand what an EFI System Partition is used for. The
>> wikipedia entry suggests that it is essentially a special partition
>> where bootloaders live (instead of using a normal partition for that)
>> and where EFI loads drivers and DOS/Windows PE commandline executables
>> from. That sounds suspiciously like a DOS partition with NTLDR and
>> loadlin.
> The partition is empty by default.
> EFI expects a directory structure and loads drivers from this
> partition if present, without loading any bootloader.
> That allows it to load a bootloader from an ext3 filesystem, a direct
> ELF kernel from XFS, or a netboot updated driver for a WiMAX (for
> example).

Wikipedia says the EFI System Partition is a FAT variant. "load a
bootloader" implies that the thing loading the bootloader is a
bootloader as well. Such a construct is not uncommon, take a look at
GRUB booting Windows where it just acts as a bootloader for the Windows
bootloader.


>>> A bootloader can fuckinly easy put a good splash without limiting to
>>> 12 colors or making calls to the VGA system (for example). What will
>>> happen with the GRUB if tomorrow VGA disappears? What a mess...
>>
>> I've seen quite a few GRUB installations with 24-bit color splash
>> screens in high resolutions. Graphics hardware won't disappear from
>> computers any time soon, only the interfaces may change.
> So, say me how!

What do you want to know?

How to use 24-bit color for boot loader splash screens on systems with
BIOS? I think there are patches floating around for GRUB and LILO, and
some other bootloaders sport graphical interfaces by default.

Or how to deal with systems that don't have a VGA realmode interface
anymore? Such systems already exist (VIA EPIA has that option), and let
me tell you that they run coreboot+Linux instead of EFI.


>> Interesting. Do these boards still have a 20-30 second wait time from
>> poweron to bootloader? I've seen coreboot+SeaBIOS achieve 1 second from
>> poweron to bootloader on real x86 hardware, so I always wonder what the
>> commercial BIOSes do during the time we wait for them.
> Nope. The only "wait" is the normal Apple POST. Then the CSM takes 1
> or 2 seconds to load the OS.

How long does the normal Apple POST take? Longer than a second? If yes,
where does it spend that time?


>> Many years ago, some EFI proponent told me: "EFI is great. It is a
>> 32-bit BIOS with builtin 32-bit DOS. Now every operating system can use
>> EFI drivers in compatibility mode like Windows 95 used DOS drivers in
>> compatibility mode."
>> That statement had a lasting impression on me and I've yet to see any
>> explanation why that statement would be incorrect. I'm very willing to
>> learn, so if I got something wrong, please educate me.
>
> EFI is not a 32-bit BIOS, it is a 32 or 64 bit protected mode hardware
> initialization and booting interface.

I see. Thanks for clarifying this.

Looking at the wikipedia definition for operating systems, EFI fits that
definition pretty well. It has drivers, hardware abstraction, and runs
applications.

There are good reasons to have full operating systems in the firmware.
The programming model is easier, the environment is userfriendly, and if
there are good recovery and diagnosis tools available for that operating
system, you can simply reuse them.

And since having an OS in the firmware is good, people have created
systems which have coreboot+Linux in the firmware flash chip. Some
variants with coreboot+Linux+KVM (the virtualization solution) and
coreboot+Linux+busybox+X(Kdrive)+windowmanager(matchbox)+terminal(rxvt)
are already out there. A boot demo video of the latter variant is here:
http://www.youtube.com/watch?v=nuzRsXKm_NQ

Of course, Linux can act as a boot loader as well (see kboot and
petitboot and runnix), and I've heard of people using Linux in firmware
to boot some Windows variant over Fibrechannel.


Regards,
Carl-Daniel

-- 
http://www.hailfinger.org/

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Stefan Reinauer]

Jordan Justen wrote:
> I'm not going to take a side on this matter.  But, I think what will
> be more important is what is used in the majority of OS's and systems.
>  This is why we still put the 16-bit legacy BIOS as the #1 priority
> after ~30 years.  But, like I mention, I think there are signs that
> this may shift towards UEFI at some point.
>   
Looking at today's OSes, the BIOS "interface" is basically completely
unused except for loading the OS kernel.
For future generations of firmware and computers and virtual machines it
might make sense to optimize with that situation in mind.

The industry we're working in does not see UEFI can cope with the
problems firmware has today, unfortunately. After seeing EFI fail so
badly with the Itanium it would certainly be nice to see if something
can evolve from it in a decade, or two. We're watching excitedly.

All the best,
Stefan

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

Correct me if I'm wrong but most the IA64 opcodes can be derived from  
IA32 emulation isn't it why both are "Intel Architecture"?

El 04/10/2009, a las 01:01, Jordan Justen escribió:

> On Sat, Oct 3, 2009 at 16:39, Natalia Portillo <claunia@claunia.com>  
> wrote:
>>
>> El 03/10/2009, a las 23:13, Jordan Justen escribió:
>>
>>> On Sat, Oct 3, 2009 at 10:40, ron minnich <rminnich@gmail.com>  
>>> wrote:
>>>>
>>>> I use qemu for a lot of coreboot work. I really depend on qemu for
>>>> many things I do, not just coreboot related. The qemu target in
>>>> coreboot has been very heavily used by us to test out new ideas.
>>>>
>>>> That said, I don't see a compelling need to augment seabios with
>>>> coreboot on qemu *in the standard distribution*. If seabios gets  
>>>> the
>>>> job done, and gets OSes booted, I think that's sufficient. I  
>>>> don't see
>>>> a need to complicate anyone's life with something that is, after  
>>>> all,
>>>> a sideshow for qemu users.
>>>>
>>>> Conversely, I don't see the need to add the huge pile of stuff that
>>>> comes with UEFI/OVMF/whatever to qemu either. One might argue that
>>>
>>> This is a valid argument right now.  OS X is the only OS today that
>>> targets UEFI, and specifically not legacy BIOS.  But, in 5 ~ 10  
>>> years
>>> that might not be the case.
>>
>> Wrong case.
>>
>> Itanium architecture ONLY uses UEFI.
>> Having already got UEFI compatibility in QEMU is a step forware  
>> implemente
>> ia64-machine.
>
> Good point.  For OVMF we only support i386/x86-64 right now, but, we
> would like to add ia64 support.  I think the lack of ia64 support in
> QEMU is one of the big things holding this back.
>

Re: [coreboot] [Qemu-devel] Release plan for 0.12.0

[Lennart Sorensen]

On Sun, Oct 04, 2009 at 04:55:10AM +0100, Natalia Portillo wrote:
> Correct me if I'm wrong but most the IA64 opcodes can be derived from  
> IA32 emulation isn't it why both are "Intel Architecture"?

IA64 and IA32 have nothing in common other than being from intel.
That's why they are called Intel Architecture.  x86-64 on the other hand
is based on x86 (IA32).  IA64 is totally different.  I think intel has
refered to x86-64 as IA-32e, EM64T, Intel 64, and probably other
things too.  IA-32e does appear to have refered to intel's own 64bit
extension to x86 before they licensed the AMD64 instructions instead.

IA64 has fixed length instructions, with 3 operations per instruction
word.  IA32 has variable length instructions, with one operation per
instruction "word".

IA64 is in order execution.  Most IA32 chips are out of order execution
these days.  Not that that is really anything to do with the instruction
set though.

-- 
Len Sorensen

Re: [Qemu-devel] Release plan for 0.12.0

[Jordan Justen]

Carl-Daniel,

Interesting.  So, where can I download a QEMU bios rom to boot a UEFI
OS with this solution?

Can you explain what coreboot+tianocore means?  Which parts of
tianocore do you use in this situation?

If your solution can accomplish all of what you say, I'm not sure how
OVMF would be able to compete against in terms of being included with
QEMU.  Part of the reason for starting OVMF was to help enable UEFI
support within VM's such as QEMU.  If OVMF was utilized by QEMU it
definitely would have been a bit of a boost for our efforts, but if
QEMU accomplishes UEFI support via another path, then overall we will
still be happy.

Thanks,

-Jordan

On Fri, Oct 2, 2009 at 11:45, Carl-Daniel Hailfinger
<c-d.hailfinger.devel.2006@gmx.net> wrote:
> On 02.10.2009 18:58, Jordan Justen wrote:
>> On Fri, Oct 2, 2009 at 06:29, Anthony Liguori <anthony@codemonkey.ws> wrote:
>>
>>> So I think the best way forward is to hold off on UEFI in mainline until we
>>> can provide a single unified stack.
>>>
>>
>> While it is true that a separate machine type could potentially be
>> viewed as increasing the testing requirements, I am not so sure.  If
>> QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI
>> based OS boot and the CSM based legacy OS boot?
>>
>
> Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and
> have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI
> interface), wouldn't that solve all problems in one go? You get an
> unified stack and don't have to mess around with different firmware
> files because coreboot can read in a Qemu machine config option (or
> NVRAM/CMOS) and select the interface based on that.
>
> The hardware init part would be identical for all variants, only the
> interface would differ. coreboot works now and has the benefit of
> supporting real hardware as well, so the difference between a setup
> inside Qemu and a setup on real hardware is minimal.
>
>
>>>> I don't know of any efforts to create an open source CSM.  Is someone
>>>> working on converting SeaBIOS to act as a CSM?
>>>>
>
> The coreboot solution would also avoid converting SeaBIOS because
> SeaBIOS already works as coreboot payload (that's how coreboot
> developers call the CSM).
>
>
>> I do know that we don't intend to start an open source CSM project on
>> tianocore.org at this time.
>>
>> But, if such a project was started, I think we would support
>> development via email on our edk2 dev list, and potentially even host
>> the project on tianocore.org.
>>
>> So, I would invite anyone thinking of tackling such a task to join the
>> edk2 dev list to discuss your work.  Furthermore, if you'd like to
>> request the project to be hosted on tianocore.org, feel free to ask
>> our administrators.  (As mentioned previously, we usually deal with
>> the BSD license on tianocore.org, but please don't let this hold you
>> back from asking questions, or requesting hosting of your project.)
>>
>
> I do invite everyone who is interested in having one single ROM file
> with EFI+SeaBIOS to join the coreboot list as well. We already have
> coreboot+SeaBIOS running (both on hardware and inside Qemu) and
> coreboot+tianocore works as well. It has the advantage of requiring zero
> changes in coreboot, zero changes in SeaBIOS and (I think) zero changes
> in Tianocore.
>
> As a nice side benefit for the EFI specialists, you get the ability to
> run EFI on every coreboot supported platform.
>
>
> If you have any questions, please ask.
>
> Regards,
> Carl-Daniel
>

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Jordan Justen wrote:
> Carl-Daniel,
>
> Interesting.  So, where can I download a QEMU bios rom to boot a UEFI
> OS with this solution?
>
> Can you explain what coreboot+tianocore means?  Which parts of
> tianocore do you use in this situation?
>
> If your solution can accomplish all of what you say, I'm not sure how
> OVMF would be able to compete against in terms of being included with
> QEMU.  Part of the reason for starting OVMF was to help enable UEFI
> support within VM's such as QEMU.  If OVMF was utilized by QEMU it
> definitely would have been a bit of a boost for our efforts, but if
> QEMU accomplishes UEFI support via another path, then overall we will
> still be happy.
>   

FWIW, I looked more closely at Coreboot + SeaBIOS today.  It's much less 
functional than just SeaBIOS alone.  There really is no additional 
functionality because all payloads that Coreboot can run already run 
directly under QEMU (or under SeaBIOS).

With respect to Coreboot + SeaBIOS + UEFI, AFAIK, you cannot have 
multiple payloads without using essentially a payload switcher (like 
Bayou).  The problem with this though is your just deferring the 
EFI/BIOS choice to the user in a different place.  What we need is a 
mechanism to transparently choose UEFI or SeaBIOS depending on whether 
the guest is EFI aware.

I think option roms further complicate the matter because we would need 
a gPXE EFI module to support network boot.  That makes the switch logic 
even more complicated.  For PCI passthrough, I assume that the legacy 
option ROMs have to be loaded through a CSM so if we want to enable PCI 
passthrough, we really need a proper CSM.

-- 
Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Carl-Daniel Hailfinger]

Hi Jordan,

I have to admit that I'm not the expert for coreboot+EFI solutions. I
know they exist and work, but I'll defer to other developers on the
coreboot list to give you detailed answers. Please see below for what I
think I know (but I'll gladly accept any corrections).


On 02.10.2009 22:57, Jordan Justen wrote:
> Carl-Daniel,
>
> Interesting.  So, where can I download a QEMU bios rom to boot a UEFI
> OS with this solution?
>
> Can you explain what coreboot+tianocore means?  Which parts of
> tianocore do you use in this situation?
>   

This mail is CC: to the coreboot list and I'm sure someone who knows
more than I can respond.

The documentation about Tianocore/UEFI/EFI is not entirely clear about
the names of each part of the stack, but if I understand correctly,
there is a hardware init part (OVMF for Qemu) and parts which boot EFI
enabled operating systems, provide EFI services to the operating system,
or give users a shell to work from.

coreboot is pure hardware init (CPU, RAM, chipset, PCI), it has no
drivers whatsoever. It does provide ACPI tables for the hardware and a
memory map. Once coreboot finishes hardware initialization, it passes
control to a payload (which has drivers, boots the OS, provides
services). That payload is hardware-independent unless it explicitly
wants to contain non-generic drivers. It is possible to use the exact
same SeaBIOS binary as coreboot payload on Qemu, Pentium III systems
with i440BX chipset, AMD Phenom systems with 690G chipset, Via Nano
systems with VX800 chipset, and Intel Core 2 Duo systems with i945 chipset.
Unless I'm mistaken, OVMF and coreboot have pretty much the same purpose.


> If your solution can accomplish all of what you say, I'm not sure how
> OVMF would be able to compete against in terms of being included with
> QEMU.  Part of the reason for starting OVMF was to help enable UEFI
> support within VM's such as QEMU.  If OVMF was utilized by QEMU it
> definitely would have been a bit of a boost for our efforts, but if
> QEMU accomplishes UEFI support via another path, then overall we will
> still be happy.
>   

Please don't get me wrong. The number of developers in the open source
firmware space is small and I'm happy that you're such a developer. My
main worry is splitting one task among too many projects, with the
danger of frustation if only one project is accepted into Qemu. I don't
have an axe to grind against EFI (which is desired by many people) or
OVMF (which is purpose-built for UEFI in Qemu).
That's why I want to understand your goals and hope to find a way to
accomplish your goals, coreboot goals and at the same time have Qemu get
the best firmware possible.


> Thanks,
>
> -Jordan
>   

Regards,
Carl-Daniel

> On Fri, Oct 2, 2009 at 11:45, Carl-Daniel Hailfinger
> <c-d.hailfinger.devel.2006@gmx.net> wrote:
>   
>> On 02.10.2009 18:58, Jordan Justen wrote:
>>     
>>> On Fri, Oct 2, 2009 at 06:29, Anthony Liguori <anthony@codemonkey.ws> wrote:
>>>
>>>       
>>>> So I think the best way forward is to hold off on UEFI in mainline until we
>>>> can provide a single unified stack.
>>>>
>>>>         
>>> While it is true that a separate machine type could potentially be
>>> viewed as increasing the testing requirements, I am not so sure.  If
>>> QEMU has a UEFI+CSM solution, wouldn't we still have to test both UEFI
>>> based OS boot and the CSM based legacy OS boot?
>>>
>>>       
>> Given that you can easily pack coreboot+SeaBIOS+UEFI into one ROM and
>> have coreboot boot either SeaBIOS (BIOS interface) or UEFI (EFI
>> interface), wouldn't that solve all problems in one go? You get an
>>     

Re: [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

Mac OS X / Intel also depends on EFI, even if netkas and others hacked  
it to run on BIOS it would be great to have an emulator capable of  
running it.

About license terms Apple states:
Mac OS X Server can be run emulated as long as they are run inside a  
Macintosh.
That is, we can have a Linux in an iMac running QEMU for emulating Mac  
OS X Server.
It is forbidden to run Mac OS X (client) in anything other than a real  
Apple hardware (nothing forbides having an emulator capable of doing  
so).

El 01/10/2009, a las 22:10, Jordan Justen escribió:

> On Thu, Oct 1, 2009 at 05:45, Anthony Liguori <aliguori@us.ibm.com>  
> wrote:
>> Natalia Portillo wrote:
>>>
>>> for target-i386 and target-x86_64:
>>>
>>> o EFI firmware emulation (as a machine or command line option)
>>> o OS/2 1.x support
>>
>> AFAIK, no one is actually working on these.  EFI is also not  
>> terribly useful
>> without a CSM and I suspect porting SeaBIOS to tiano core is going  
>> to be a
>> big effort.
>
> Several Linux distributions are getting UEFI support ironed out now,
> so UEFI can be used without a CSM.  QEMU support for UEFI could
> help the Linux distributions to enable UEFI support...
>
> Windows Vista and 7 support UEFI on x64-64, but I've not been able
> to boot either with the OVMF project yet.  (I suspect that the video  
> driver
> might not be initializing correctly, but the root cause of the  
> failure is not
> yet known.)
>
>>
>> --
>> Regards,
>>
>> Anthony Liguori
>>
>>
>>
>>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Stuart Brady]

On Thu, Oct 01, 2009 at 02:55:05AM +0100, Natalia Portillo wrote:

> integrating target-alpha and target-zxspectrum in the branch tree

Heh. :)

z80-softmmu now has MSX1 emulation, with thanks to Juha Riihimäki,
and I've also added the beginnings of ZX Spectrum 128K and SAM Coupé
emulation.  It still needs quite a few cleanups, though.

When you speak of target-alpha, I presume you're referring to
Tristan Gingold's system emulation patches... it would certainly be
nice to see these applied.

Same also for Laurent Vivier's m68k patches -- I would certainly have a
go at implementing Atari ST emulation, and would also be interested in
helping with Amiga, 68K Mac and perhaps even NeXT, SUN3, etc.

> a new official os support list design (please, give me ideas, css,
> help, anything!) with a better captcha

For the OS support list, I'd perhaps suggest something similar to the
Wine AppDB -- one page per OS version per architecture, containing
details of each tester's QEMU configuration (KVM/TCG, host OS/arch,
emulated devices, etc.) and guest OS configuration (drivers/kernel used,
service packs applied, etc.), together with any extra notes.

If it's of any use: http://source.winehq.org/git/appdb.git/

Cheers,
-- 
Stuart Brady

Re: [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

Hi Stuart!

Nice to read you!

El 01/10/2009, a las 21:50, Stuart Brady escribió:

> On Thu, Oct 01, 2009 at 02:55:05AM +0100, Natalia Portillo wrote:
>
>> integrating target-alpha and target-zxspectrum in the branch tree
>
> Heh. :)
>
> z80-softmmu now has MSX1 emulation, with thanks to Juha Riihimäki,
> and I've also added the beginnings of ZX Spectrum 128K and SAM Coupé
> emulation.  It still needs quite a few cleanups, though.
Great great!!!
If you need any software for testing them just email me.

Could you please email me the public domain ROMs?
I've lost them

> When you speak of target-alpha, I presume you're referring to
> Tristan Gingold's system emulation patches... it would certainly be
> nice to see these applied.
Yes I indeed didn't remember his name.

> Same also for Laurent Vivier's m68k patches -- I would certainly  
> have a
> go at implementing Atari ST emulation, and would also be interested in
> helping with Amiga, 68K Mac and perhaps even NeXT, SUN3, etc.
If operating systems or real systems are need for that I have almost  
all.
Atari ST512FM, ST1024, Amiga 500, a pletory of Macs.
And of course, Atari TOS, Amiga Kickstart, OS and Workbench, Mac OS, A/ 
UX, NeXTStep and OpenStep, and SunOS/sun3.

>> a new official os support list design (please, give me ideas, css,
>> help, anything!) with a better captcha
>
> For the OS support list, I'd perhaps suggest something similar to the
> Wine AppDB -- one page per OS version per architecture, containing
> details of each tester's QEMU configuration (KVM/TCG, host OS/arch,
> emulated devices, etc.) and guest OS configuration (drivers/kernel  
> used,
> service packs applied, etc.), together with any extra notes.
>
> If it's of any use: http://source.winehq.org/git/appdb.git/
A GREAT idea I'll check on it.

Regards,
Natalia Portillo

Re: [Qemu-devel] Release plan for 0.12.0

[Stuart Brady]

On Fri, Oct 02, 2009 at 05:51:37AM +0100, Natalia Portillo wrote:
> Hi Stuart!
> 
> Nice to read you!

And you, Natalia! :-)

> El 01/10/2009, a las 21:50, Stuart Brady escribió:
> 
> >z80-softmmu now has MSX1 emulation, with thanks to Juha Riihimäki,
> >and I've also added the beginnings of ZX Spectrum 128K and SAM Coupé
> >emulation.  It still needs quite a few cleanups, though.

> Great great!!!
> If you need any software for testing them just email me.

I've no real problems getting hold of SAM and Spectrum software, but I
expect there are still bugs in the Z80 emulation... certainly, support
for the Z80's undocumented flags would be useful!

> Could you please email me the public domain ROMs?
> I've lost them

The ZX Spectrum ROMs are available here:

   http://www.shadowmagic.org.uk/spectrum/roms.html

For 48K emulation, you'll likely want 48.rom, renamed to zx-rom.bin...
For 128K emulation, you'd need to join 128-0.rom and 128-1.rom together,
and named zx128-rom.bin.

The SAM Coupe ROM is available here:

   ftp://ftp.nvg.ntnu.no/pub/sam-coupe/emulation/ROMs/samroms.zip

I'd recommend using ROM30 (renaming it to sam-rom.bin).

(Note to all: these are not truly 'free software', but their owners have
given permission for them to be used for emulation.  The MSX ROMs are a
different matter, though.)

The Z80 target is available at:

   http://repo.or.cz/w/qemu/z80.git

BTW, one thing that looks interesting is TAKEDA Toshiya's PC-98
emulation.  If support for the NEC v20's 8080 mode (!) would be useful,
then perhaps parts of the Z80 target might serve as a starting point...

> >Same also for Laurent Vivier's m68k patches -- I would certainly have a
> >go at implementing Atari ST emulation, and would also be interested in
> >helping with Amiga, 68K Mac and perhaps even NeXT, SUN3, etc.

> If operating systems or real systems are need for that I have almost all.
> Atari ST512FM, ST1024, Amiga 500, a pletory of Macs.
> And of course, Atari TOS, Amiga Kickstart, OS and Workbench, Mac OS,
> A/UX, NeXTStep and OpenStep, and SunOS/sun3.

Those would certainly be useful for testing!  For the time being,
Laurent's plan to boot linux-m68k sounds like a good approach.

> >For the OS support list, I'd perhaps suggest something similar to the
> >Wine AppDB [...]
> >If it's of any use: http://source.winehq.org/git/appdb.git/

> A GREAT idea I'll check on it.

Thanks!  I'm eager to hear what conclusion you come to. :-)

Cheers,
-- 
Stuart Brady

Re: [Qemu-devel] Release plan for 0.12.0

[Natalia Portillo]

[-- Attachment #1: Type: text/plain, Size: 530 bytes --]

>
>>> For the OS support list, I'd perhaps suggest something similar to  
>>> the
>>> Wine AppDB [...]
>>> If it's of any use: http://source.winehq.org/git/appdb.git/
>
>> A GREAT idea I'll check on it.
>
> Thanks!  I'm eager to hear what conclusion you come to. :-)

I've got the GIT source but the question is how to modify it to handle  
the device list.
As wine just needs a version, but QEMU needs version + chosen system  
emulation + chosen machine + chosen graphics + chosen audio + chosen  
network + chosen mass storage.

[-- Attachment #2: Type: text/html, Size: 1109 bytes --]

Re: [Qemu-devel] Release plan for 0.12.0

[Stefan Weil]

Anthony Liguori schrieb:
> Hi,
>
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>
> I'd like to do a few things different this time around.  I don't think
> the -rc process went very well as I don't think we got more testing
> out of it.  I'd like to shorten the timeline for 0.12.0 a good bit. 
> The 0.10 stable tree got pretty difficult to maintain toward the end
> of the cycle.  We also had a pretty huge amount of change between 0.10
> and 0.11 so I think a shorter cycle is warranted.
>
> I think aiming for early to mid-December would give us roughly a 3
> month cycle and would align well with some of the Linux distribution
> cycles.  I'd like to limit things to a single -rc that lasted only for
> about a week.  This is enough time to fix most of the obvious issues I
> think.
>
> I'd also like to try to enumerate some features for this release. 
> Here's a short list of things I expect to see for this release
> (target-i386 centric).  Please add or comment on items that you'd
> either like to see in the release or are planning on working on.
>
> o VMState conversion -- I expect most of the pc target to be completed
> o qdev conversion -- I hope that we'll get most of the pc target
> completely converted to qdev
> o storage live migration
> o switch to SeaBIOS (need to finish porting features from Bochs)
> o switch to gPXE (need to resolve slirp tftp server issue)
> o KSM integration
> o in-kernel APIC support for KVM
> o guest SMP support for KVM
> o updates to the default pc machine type
>
> Please add to this list and I'll collect it all and post it somewhere.
>
> Thanks!
>

* VDI block driver
* Update eepro100 driver
* Fix VNC support for Windows

Stefan

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Stefan Weil wrote:
> * VDI block driver
>   

If a new rev is submitted in time, sure.

> * Update eepro100 driver
>   

I have some of the patches that Blue Swirl didn't apply in my staging 
tree so they should get included assuming all goes well.

> * Fix VNC support for Windows
>   

I wasn't aware this was broken, can you file a bug report?

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Stefan Weil]

Anthony Liguori schrieb:
> Stefan Weil wrote:
>> * VDI block driver
>>   
>
> If a new rev is submitted in time, sure.

The one from qemu master? There is no plan for a new revision.

>
>> * Update eepro100 driver
>>   
>
> I have some of the patches that Blue Swirl didn't apply in my staging
> tree so they should get included assuming all goes well.

Some more patches are in my local queue.
I'm waiting for the last ones to get included before I'll send new ones.

>
>> * Fix VNC support for Windows
>>   
>
> I wasn't aware this was broken, can you file a bug report?

A patch was just sent to qemu-devel.

>
> Regards,
>
> Anthony Liguori
>


Regards

Stefan Weil

Re: [Qemu-devel] Release plan for 0.12.0

[TAKEDA, toshiya]

Anthony Liguori さんは書きました:
>Hi,
>
>Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.
>
>I'd like to do a few things different this time around.  I don't think 
>the -rc process went very well as I don't think we got more testing out 
>of it.  I'd like to shorten the timeline for 0.12.0 a good bit.  The 
>0.10 stable tree got pretty difficult to maintain toward the end of the 
>cycle.  We also had a pretty huge amount of change between 0.10 and 0.11 
>so I think a shorter cycle is warranted.
>
>I think aiming for early to mid-December would give us roughly a 3 month 
>cycle and would align well with some of the Linux distribution cycles.  
>I'd like to limit things to a single -rc that lasted only for about a 
>week.  This is enough time to fix most of the obvious issues I think.
>
>I'd also like to try to enumerate some features for this release.  
>Here's a short list of things I expect to see for this release 
>(target-i386 centric).  Please add or comment on items that you'd either 
>like to see in the release or are planning on working on.
>
> o VMState conversion -- I expect most of the pc target to be completed
> o qdev conversion -- I hope that we'll get most of the pc target 
>completely converted to qdev
> o storage live migration
> o switch to SeaBIOS (need to finish porting features from Bochs)
> o switch to gPXE (need to resolve slirp tftp server issue)
> o KSM integration
> o in-kernel APIC support for KVM
> o guest SMP support for KVM
> o updates to the default pc machine type
>
>Please add to this list and I'll collect it all and post it somewhere.

o NEC PC-9821 family support on target-i386

In the last patch MS-DOS can boot on QEMU.
I think I can add support nic (LGY-98) and IDE in 0.12.0
and I hope I can boot FreeBSD/pc98 on it.

PS.
I will repost v3 patch in the next week, please wait reviewing v2 patch I post Oct.1.

Thanks,
TAKEDA, toshiya

>Thanks!
>
>-- 
>Regards,
>
>Anthony Liguori
>
>
>
>

Re: [Qemu-devel] Release plan for 0.12.0

[Jens Osterkamp]

On Wednesday 30 September 2009, Anthony Liguori wrote:

>  o VMState conversion -- I expect most of the pc target to be completed
>  o qdev conversion -- I hope that we'll get most of the pc target
> completely converted to qdev
>  o storage live migration
>  o switch to SeaBIOS (need to finish porting features from Bochs)
>  o switch to gPXE (need to resolve slirp tftp server issue)
>  o KSM integration
>  o in-kernel APIC support for KVM
>  o guest SMP support for KVM
>  o updates to the default pc machine type
>
> Please add to this list and I'll collect it all and post it somewhere.

What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did 
I miss something ?

Jens

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Jens Osterkamp wrote:
> On Wednesday 30 September 2009, Anthony Liguori wrote:
>
>   
>>  o VMState conversion -- I expect most of the pc target to be completed
>>  o qdev conversion -- I hope that we'll get most of the pc target
>> completely converted to qdev
>>  o storage live migration
>>  o switch to SeaBIOS (need to finish porting features from Bochs)
>>  o switch to gPXE (need to resolve slirp tftp server issue)
>>  o KSM integration
>>  o in-kernel APIC support for KVM
>>  o guest SMP support for KVM
>>  o updates to the default pc machine type
>>
>> Please add to this list and I'll collect it all and post it somewhere.
>>     
>
> What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did 
> I miss something ?
>   

The patch seems to have not been updated after the initial posting and 
the first feedback cycle.

I'm generally inclined to oppose the functionality as I don't think it 
offers any advantages over the existing backends.

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Arnd Bergmann]

On Thursday 08 October 2009, Anthony Liguori wrote:
> Jens Osterkamp wrote:
> > On Wednesday 30 September 2009, Anthony Liguori wrote:
> >
> >> Please add to this list and I'll collect it all and post it somewhere.
> >>     
> >
> > What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did 
> > I miss something ?
> >   
> 
> The patch seems to have not been updated after the initial posting and 
> the first feedback cycle.
> 
> I'm generally inclined to oppose the functionality as I don't think it 
> offers any advantages over the existing backends.

There are two reasons why I think this backend is important:

- As an easy way to provide isolation between guests (private ethernet
  port aggregator, PEPA) and external enforcement of network priviledges
  (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.

- As a counterpart to the vhost_net driver, providing an identical
  user interface with or without vhost_net acceleration in the kernel.

	Arnd <><

Re: [Qemu-devel] Release plan for 0.12.0

[Anthony Liguori]

Arnd Bergmann wrote:
> There are two reasons why I think this backend is important:
>
> - As an easy way to provide isolation between guests (private ethernet
>   port aggregator, PEPA) and external enforcement of network priviledges
>   (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.
>   

Can't this all be done with tap and a bridge?

Regards,

Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Michael S. Tsirkin]

On Wed, Oct 14, 2009 at 08:53:55AM -0500, Anthony Liguori wrote:
> Arnd Bergmann wrote:
>> There are two reasons why I think this backend is important:
>>
>> - As an easy way to provide isolation between guests (private ethernet
>>   port aggregator, PEPA) and external enforcement of network priviledges
>>   (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.
>>   
>
> Can't this all be done with tap and a bridge?

Not with existing kernels, I think.

> Regards,
>
> Anthony Liguori

Re: [Qemu-devel] Release plan for 0.12.0

[Michael S. Tsirkin]

On Wed, Oct 14, 2009 at 03:09:28PM +0200, Arnd Bergmann wrote:
> On Thursday 08 October 2009, Anthony Liguori wrote:
> > Jens Osterkamp wrote:
> > > On Wednesday 30 September 2009, Anthony Liguori wrote:
> > >
> > >> Please add to this list and I'll collect it all and post it somewhere.
> > >>     
> > >
> > > What about Or Gerlitz' raw backend driver ? I did not see it go in yet, or did 
> > > I miss something ?
> > >   
> > 
> > The patch seems to have not been updated after the initial posting and 
> > the first feedback cycle.
> > 
> > I'm generally inclined to oppose the functionality as I don't think it 
> > offers any advantages over the existing backends.
> 
> There are two reasons why I think this backend is important:
> 
> - As an easy way to provide isolation between guests (private ethernet
>   port aggregator, PEPA) and external enforcement of network priviledges
>   (virtual ethernet port aggregator, VEPA) using the macvlan subsystem.
> 
> - As a counterpart to the vhost_net driver, providing an identical
>   user interface with or without vhost_net acceleration in the kernel.
> 
> 	Arnd <><

I think raw sockets also support RX mac/vlan filtering in kernel, which
might be faster than doing it in virtio in userspace as it's done now.

-- 
MST

[Qemu-devel] Re: Release plan for 0.12.0

[Michael S. Tsirkin]

On Thu, Oct 08, 2009 at 09:21:04AM -0500, Anthony Liguori wrote:
> Jens Osterkamp wrote:
>> On Wednesday 30 September 2009, Anthony Liguori wrote:
>>
>>   
>>>  o VMState conversion -- I expect most of the pc target to be completed
>>>  o qdev conversion -- I hope that we'll get most of the pc target
>>> completely converted to qdev
>>>  o storage live migration
>>>  o switch to SeaBIOS (need to finish porting features from Bochs)
>>>  o switch to gPXE (need to resolve slirp tftp server issue)
>>>  o KSM integration
>>>  o in-kernel APIC support for KVM
>>>  o guest SMP support for KVM
>>>  o updates to the default pc machine type
>>>
>>> Please add to this list and I'll collect it all and post it somewhere.
>>>     
>>
>> What about Or Gerlitz' raw backend driver ? I did not see it go in yet, 
>> or did I miss something ?
>>   
>
> The patch seems to have not been updated after the initial posting and  
> the first feedback cycle.

Looks like Or has abandoned it.  I have an updated version which works
with new APIs, etc.  Let me post it and we'll go from there.

> I'm generally inclined to oppose the functionality as I don't think it  
> offers any advantages over the existing backends.

I patch it in and use it all the time.  It's much easier to setup
on a random machine than a bridged config.

-- 
MST

[Qemu-devel] Re: Release plan for 0.12.0

[Anthony Liguori]

Michael S. Tsirkin wrote:
> Looks like Or has abandoned it.  I have an updated version which works
> with new APIs, etc.  Let me post it and we'll go from there.
>
>   
>> I'm generally inclined to oppose the functionality as I don't think it  
>> offers any advantages over the existing backends.
>>     
>
> I patch it in and use it all the time.  It's much easier to setup
> on a random machine than a bridged config.
>   

Having two things that do the same thing is just going to lead to user 
confusion.  If the problem is tap is too hard to setup, we should try to 
simplify tap configuration.

Regards,

Anthony Liguori

[Qemu-devel] Re: Release plan for 0.12.0

[Michael S. Tsirkin]

On Wed, Oct 14, 2009 at 09:17:15AM -0500, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> Looks like Or has abandoned it.  I have an updated version which works
>> with new APIs, etc.  Let me post it and we'll go from there.
>>
>>   
>>> I'm generally inclined to oppose the functionality as I don't think 
>>> it  offers any advantages over the existing backends.
>>>     
>>
>> I patch it in and use it all the time.  It's much easier to setup
>> on a random machine than a bridged config.
>>   
>
> Having two things that do the same thing is just going to lead to user  
> confusion.

They do not do the same thing. With raw socket you can use windows
update without a bridge in the host, with tap you can't.

> If the problem is tap is too hard to setup, we should try to  
> simplify tap configuration.

The problem is bridge is too hard to setup.
Simplifying that is a good idea, but outside the scope
of the qemu project.


> Regards,
>
> Anthony Liguori
>

Re: [Qemu-devel] Re: Release plan for 0.12.0

[Jamie Lokier]

Michael S. Tsirkin wrote:
> On Wed, Oct 14, 2009 at 09:17:15AM -0500, Anthony Liguori wrote:
> > Michael S. Tsirkin wrote:
> >> Looks like Or has abandoned it.  I have an updated version which works
> >> with new APIs, etc.  Let me post it and we'll go from there.
> >>
> >>   
> >>> I'm generally inclined to oppose the functionality as I don't think 
> >>> it  offers any advantages over the existing backends.
> >>>     
> >>
> >> I patch it in and use it all the time.  It's much easier to setup
> >> on a random machine than a bridged config.
> >>   
> >
> > Having two things that do the same thing is just going to lead to user  
> > confusion.
> 
> They do not do the same thing. With raw socket you can use windows
> update without a bridge in the host, with tap you can't.

On the other hand, with raw socket, guest Windows can't access files
on the host's Samba share can it?  So it's not that useful even for
Windows guests.

> > If the problem is tap is too hard to setup, we should try to  
> > simplify tap configuration.
> 
> The problem is bridge is too hard to setup.
> Simplifying that is a good idea, but outside the scope
> of the qemu project.

I venture it's important enough for qemu that it's worth working on
that.  Something that looks like the raw socket but behaves like an
automatically instantiated bridge attached to the bound interface
would be a useful interface.

I don't have much time, but I'll help anybody who wants to do that.

-- Jamie

Re: [Qemu-devel] Re: Release plan for 0.12.0

[Michael S. Tsirkin]

On Wed, Oct 14, 2009 at 04:19:17PM +0100, Jamie Lokier wrote:
> Michael S. Tsirkin wrote:
> > On Wed, Oct 14, 2009 at 09:17:15AM -0500, Anthony Liguori wrote:
> > > Michael S. Tsirkin wrote:
> > >> Looks like Or has abandoned it.  I have an updated version which works
> > >> with new APIs, etc.  Let me post it and we'll go from there.
> > >>
> > >>   
> > >>> I'm generally inclined to oppose the functionality as I don't think 
> > >>> it  offers any advantages over the existing backends.
> > >>>     
> > >>
> > >> I patch it in and use it all the time.  It's much easier to setup
> > >> on a random machine than a bridged config.
> > >>   
> > >
> > > Having two things that do the same thing is just going to lead to user  
> > > confusion.
> > 
> > They do not do the same thing. With raw socket you can use windows
> > update without a bridge in the host, with tap you can't.
> 
> On the other hand, with raw socket, guest Windows can't access files
> on the host's Samba share can it?  So it's not that useful even for
> Windows guests.

I guess this depends on whether you use the same host for samba :)

> > > If the problem is tap is too hard to setup, we should try to  
> > > simplify tap configuration.
> > 
> > The problem is bridge is too hard to setup.
> > Simplifying that is a good idea, but outside the scope
> > of the qemu project.
> 
> I venture it's important enough for qemu that it's worth working on
> that.  Something that looks like the raw socket but behaves like an
> automatically instantiated bridge attached to the bound interface
> would be a useful interface.

I agree, that would be good to have.

> I don't have much time, but I'll help anybody who wants to do that.
> 
> -- Jamie

Re: [Qemu-devel] Re: Release plan for 0.12.0

[Sridhar Samudrala]

On Wed, 2009-10-14 at 17:50 +0200, Michael S. Tsirkin wrote:
> On Wed, Oct 14, 2009 at 04:19:17PM +0100, Jamie Lokier wrote:
> > Michael S. Tsirkin wrote:
> > > On Wed, Oct 14, 2009 at 09:17:15AM -0500, Anthony Liguori wrote:
> > > > Michael S. Tsirkin wrote:
> > > >> Looks like Or has abandoned it.  I have an updated version which works
> > > >> with new APIs, etc.  Let me post it and we'll go from there.
> > > >>
> > > >>   
> > > >>> I'm generally inclined to oppose the functionality as I don't think 
> > > >>> it  offers any advantages over the existing backends.
> > > >>>     
> > > >>
> > > >> I patch it in and use it all the time.  It's much easier to setup
> > > >> on a random machine than a bridged config.
> > > >>   
> > > >
> > > > Having two things that do the same thing is just going to lead to user  
> > > > confusion.
> > > 
> > > They do not do the same thing. With raw socket you can use windows
> > > update without a bridge in the host, with tap you can't.
> > 
> > On the other hand, with raw socket, guest Windows can't access files
> > on the host's Samba share can it?  So it's not that useful even for
> > Windows guests.
> 
> I guess this depends on whether you use the same host for samba :)
> 
> > > > If the problem is tap is too hard to setup, we should try to  
> > > > simplify tap configuration.
> > > 
> > > The problem is bridge is too hard to setup.
> > > Simplifying that is a good idea, but outside the scope
> > > of the qemu project.
> > 
> > I venture it's important enough for qemu that it's worth working on
> > that.  Something that looks like the raw socket but behaves like an
> > automatically instantiated bridge attached to the bound interface
> > would be a useful interface.
> 
> I agree, that would be good to have.

Can't we bind the raw socket to the tap interface instead of the
physical interface and allow the bridge config to work.

Thanks
Sridhar


> 
> > I don't have much time, but I'll help anybody who wants to do that.
> > 
> > -- Jamie
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Raw vs. tap (was: Re: [Qemu-devel] Re: Release plan for 0.12.0)

[Anthony Liguori]

Sridhar Samudrala wrote:
> Can't we bind the raw socket to the tap interface instead of the
> physical interface and allow the bridge config to work.
>   

But why use the raw interface instead of tap directly.

Let me summarize the discussion so far:

Raw sockets
Pros:
 o User specifies a network interface to bind to
 o External traffic Just Works, guest-to-guest traffic Just Works

Cons:
 o Requires root (cannot chmod)
 o Guest<->host traffic does not work
 o No support for GSO/checksum offload

Some things that I'm not sure will work or not:
 o guest with a bridge (sending traffic with multiple mac addresses)
 o guest trying to enter promiscuous mode

Tap
Pros:
 o All types of networking works when configured
 o Supports non-root users via tunctl
 o Supports GSO/checksum offload

Cons:
 o Requires configuring a bridge which can be difficult for some users

Since I don't see any clear features in raw sockets that aren't present 
in tap, the argument really boils down to two things.  First, we should 
take any feature in qemu and let the user decide whether or not they 
want to use it.  I strongly feel this is a bad philosophy that will lead 
to increased user confusion and a poor user experience.

Second, even though raw looses performance and requires root, since it 
requires no external configuration it is easier to use and therefore 
should be an option for users.  I dislike this argument because it 
tricks a user into thinking that raw is a viable replacement for tap.  
It certainly isn't performance wise but most importantly, it isn't from 
a functional perspective.  I would be much more inclined to consider 
taking raw and improving the performance long term if guest<->host 
networking worked.  This appears to be a fundamental limitation though 
and I think it's something that will forever plague users if we include 
this feature.

So at this point, I think it's a mistake to include raw socket support.  
If the goal is to improve networking usability such that it just works 
as a root user, let's incorporate a default network script that creates 
a bridge or something like that.  There are better ways to achieve that 
goal.

Regards,

Anthony Liguori

Re: Raw vs. tap (was: Re: [Qemu-devel] Re: Release plan for 0.12.0)

[Mark McLoughlin]

On Wed, 2009-10-14 at 17:53 -0500, Anthony Liguori wrote:

> So at this point, I think it's a mistake to include raw socket support.  
> If the goal is to improve networking usability such that it just works 
> as a root user, let's incorporate a default network script that creates 
> a bridge or something like that.  There are better ways to achieve that 
> goal.

FWIW, I haven't really played with the raw backend yet, but my initial
thought was also "what exactly does this gain us apart from yet more
confusion for users?".

So, I tend to agree, but I'm not so hung up on the "user confusion"
aspect - the users that I worry about confusing (e.g. virt-manager
users) would never even know the backend exists, even if qemu did
support it.

The one hope I had for raw is that it might allow us to get closer to
the NIC, get more details on the NIC tx queue and have more intelligent
tx mitigation. This is probably better explored in the context of
vhost-net, though.

Wrt. to configuring bridges, libvirt comes with a good default setup - a
bridge without any physical NICs connected, but NAT set up for access to
the outside world.

For bridging to a physical NIC, our plan continues to be that
NetworkManager will eventually make this trivial for users, but that's
still in progress. In the meantime, the config isn't all that complex:

  http://wiki.libvirt.org/page/Networking#Fedora.2FRHEL_Bridging

Cheers,
Mark.

Re: Raw vs. tap (was: Re: [Qemu-devel] Re: Release plan for 0.12.0)

[Michael S. Tsirkin]

On Wed, Oct 14, 2009 at 05:53:56PM -0500, Anthony Liguori wrote:
> I would be much more inclined to consider  
> taking raw and improving the performance long term if guest<->host  
> networking worked.  This appears to be a fundamental limitation though  
> and I think it's something that will forever plague users if we include  
> this feature.

In fact, I think it's fixable with a raw socket bound to a macvlan.
Would that be enough?

-- 
MST

[Qemu-devel] Re: Raw vs. tap

[Anthony Liguori]

Michael S. Tsirkin wrote:
> On Wed, Oct 14, 2009 at 05:53:56PM -0500, Anthony Liguori wrote:
>   
>> I would be much more inclined to consider  
>> taking raw and improving the performance long term if guest<->host  
>> networking worked.  This appears to be a fundamental limitation though  
>> and I think it's something that will forever plague users if we include  
>> this feature.
>>     
>
> In fact, I think it's fixable with a raw socket bound to a macvlan.
> Would that be enough?
>   

What setup does that entail on the part of a user?  Wouldn't we be back 
to square one wrt users having to run archaic networking commands in 
order to set things up?

Regards,

Anthony Liguori

[Qemu-devel] Re: Raw vs. tap

[Michael S. Tsirkin]

On Thu, Oct 15, 2009 at 08:32:03AM -0500, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> On Wed, Oct 14, 2009 at 05:53:56PM -0500, Anthony Liguori wrote:
>>   
>>> I would be much more inclined to consider  taking raw and improving 
>>> the performance long term if guest<->host  networking worked.  This 
>>> appears to be a fundamental limitation though  and I think it's 
>>> something that will forever plague users if we include  this feature.
>>>     
>>
>> In fact, I think it's fixable with a raw socket bound to a macvlan.
>> Would that be enough?
>>   
>
> What setup does that entail on the part of a user?  Wouldn't we be back  
> to square one wrt users having to run archaic networking commands in  
> order to set things up?

Unlike bridge, qemu could set up macvlan without disrupting
host networking. The only issue would be cleanup if qemu
is killed.

> Regards,
>
> Anthony Liguori
>

[Qemu-devel] Re: Raw vs. tap

[Anthony Liguori]

Michael S. Tsirkin wrote:
> On Thu, Oct 15, 2009 at 08:32:03AM -0500, Anthony Liguori wrote:
>   
>> Michael S. Tsirkin wrote:
>>     
>>> On Wed, Oct 14, 2009 at 05:53:56PM -0500, Anthony Liguori wrote:
>>>   
>>>       
>>>> I would be much more inclined to consider  taking raw and improving 
>>>> the performance long term if guest<->host  networking worked.  This 
>>>> appears to be a fundamental limitation though  and I think it's 
>>>> something that will forever plague users if we include  this feature.
>>>>     
>>>>         
>>> In fact, I think it's fixable with a raw socket bound to a macvlan.
>>> Would that be enough?
>>>   
>>>       
>> What setup does that entail on the part of a user?  Wouldn't we be back  
>> to square one wrt users having to run archaic networking commands in  
>> order to set things up?
>>     
>
> Unlike bridge, qemu could set up macvlan without disrupting
> host networking. The only issue would be cleanup if qemu
> is killed.
>   

But this would require additional features in macvlan, correct?

This also only works if a guest uses the mac address assigned to it, 
correct?  If a guest was bridging the virtual nic, this would all come 
apart?

Regards,

Anthony Liguori

[Qemu-devel] Re: Raw vs. tap

[Michael S. Tsirkin]

On Thu, Oct 15, 2009 at 10:18:18AM -0500, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> On Thu, Oct 15, 2009 at 08:32:03AM -0500, Anthony Liguori wrote:
>>   
>>> Michael S. Tsirkin wrote:
>>>     
>>>> On Wed, Oct 14, 2009 at 05:53:56PM -0500, Anthony Liguori wrote:
>>>>         
>>>>> I would be much more inclined to consider  taking raw and 
>>>>> improving the performance long term if guest<->host  networking 
>>>>> worked.  This appears to be a fundamental limitation though  and 
>>>>> I think it's something that will forever plague users if we 
>>>>> include  this feature.
>>>>>             
>>>> In fact, I think it's fixable with a raw socket bound to a macvlan.
>>>> Would that be enough?
>>>>         
>>> What setup does that entail on the part of a user?  Wouldn't we be 
>>> back  to square one wrt users having to run archaic networking 
>>> commands in  order to set things up?
>>>     
>>
>> Unlike bridge, qemu could set up macvlan without disrupting
>> host networking. The only issue would be cleanup if qemu
>> is killed.
>>   
>
> But this would require additional features in macvlan, correct?

Not sure: what is the "this" that you are talking about.
It can already be set up without disturbing host networking.

> This also only works if a guest uses the mac address assigned to it,  
> correct?  If a guest was bridging the virtual nic, this would all come  
> apart?

Hmm, you could enable promisc mode, but generally this is true:
if you require bridging, use a bridge.

> Regards,
>
> Anthony Liguori

[Qemu-devel] Re: Raw vs. tap

[Anthony Liguori]

Michael S. Tsirkin wrote:
> Not sure: what is the "this" that you are talking about.
>   

I meant, fixing guest<->host traffic.  You're former argument hinged on 
being able to having networking that Just Worked without adding new 
things.  If this doesn't work today with macvlan, then the argument is 
invalid.

Regards,

Anthony Liguori

[Qemu-devel] Re: Raw vs. tap

[Michael S. Tsirkin]

On Thu, Oct 15, 2009 at 01:37:20PM -0500, Anthony Liguori wrote:
> Michael S. Tsirkin wrote:
>> Not sure: what is the "this" that you are talking about.
>>   
>
> I meant, fixing guest<->host traffic.

This needs to be supported in networking core.

> You're former argument hinged on  
> being able to having networking that Just Worked without adding new  
> things.  If this doesn't work today with macvlan, then the argument is  
> invalid.
>
> Regards,
>
> Anthony Liguori

[Qemu-devel] Re: Raw vs. tap

[Michael S. Tsirkin]

On Thu, Oct 15, 2009 at 05:48:19PM +0200, Michael S. Tsirkin wrote:
> On Thu, Oct 15, 2009 at 10:18:18AM -0500, Anthony Liguori wrote:
> > Michael S. Tsirkin wrote:
> >> On Thu, Oct 15, 2009 at 08:32:03AM -0500, Anthony Liguori wrote:
> >>   
> >>> Michael S. Tsirkin wrote:
> >>>     
> >>>> On Wed, Oct 14, 2009 at 05:53:56PM -0500, Anthony Liguori wrote:
> >>>>         
> >>>>> I would be much more inclined to consider  taking raw and 
> >>>>> improving the performance long term if guest<->host  networking 
> >>>>> worked.  This appears to be a fundamental limitation though  and 
> >>>>> I think it's something that will forever plague users if we 
> >>>>> include  this feature.
> >>>>>             
> >>>> In fact, I think it's fixable with a raw socket bound to a macvlan.
> >>>> Would that be enough?
> >>>>         
> >>> What setup does that entail on the part of a user?  Wouldn't we be 
> >>> back  to square one wrt users having to run archaic networking 
> >>> commands in  order to set things up?
> >>>     
> >>
> >> Unlike bridge, qemu could set up macvlan without disrupting
> >> host networking. The only issue would be cleanup if qemu
> >> is killed.
> >>   
> >
> > But this would require additional features in macvlan, correct?
> 
> Not sure: what is the "this" that you are talking about.
> It can already be set up without disturbing host networking.
> 
> > This also only works if a guest uses the mac address assigned to it,  
> > correct?  If a guest was bridging the virtual nic, this would all come  
> > apart?
> 
> Hmm, you could enable promisc mode, but generally this is true:
> if you require bridging, use a bridge.

Just to clarify this statement: if bridge in guest does not
configure mac filtering on guest uplink (linux
bridge by default does not do this now, OTOH macvlan does),
bridge in host will perform better in this setup
because it learns macs from traffic and does mac filtering in host.

> > Regards,
> >
> > Anthony Liguori

Re: [Qemu-devel] Re: Release plan for 0.12.0

[Michael S. Tsirkin]

On Wed, Oct 14, 2009 at 02:10:00PM -0700, Sridhar Samudrala wrote:
> On Wed, 2009-10-14 at 17:50 +0200, Michael S. Tsirkin wrote:
> > On Wed, Oct 14, 2009 at 04:19:17PM +0100, Jamie Lokier wrote:
> > > Michael S. Tsirkin wrote:
> > > > On Wed, Oct 14, 2009 at 09:17:15AM -0500, Anthony Liguori wrote:
> > > > > Michael S. Tsirkin wrote:
> > > > >> Looks like Or has abandoned it.  I have an updated version which works
> > > > >> with new APIs, etc.  Let me post it and we'll go from there.
> > > > >>
> > > > >>   
> > > > >>> I'm generally inclined to oppose the functionality as I don't think 
> > > > >>> it  offers any advantages over the existing backends.
> > > > >>>     
> > > > >>
> > > > >> I patch it in and use it all the time.  It's much easier to setup
> > > > >> on a random machine than a bridged config.
> > > > >>   
> > > > >
> > > > > Having two things that do the same thing is just going to lead to user  
> > > > > confusion.
> > > > 
> > > > They do not do the same thing. With raw socket you can use windows
> > > > update without a bridge in the host, with tap you can't.
> > > 
> > > On the other hand, with raw socket, guest Windows can't access files
> > > on the host's Samba share can it?  So it's not that useful even for
> > > Windows guests.
> > 
> > I guess this depends on whether you use the same host for samba :)
> > 
> > > > > If the problem is tap is too hard to setup, we should try to  
> > > > > simplify tap configuration.
> > > > 
> > > > The problem is bridge is too hard to setup.
> > > > Simplifying that is a good idea, but outside the scope
> > > > of the qemu project.
> > > 
> > > I venture it's important enough for qemu that it's worth working on
> > > that.  Something that looks like the raw socket but behaves like an
> > > automatically instantiated bridge attached to the bound interface
> > > would be a useful interface.
> > 
> > I agree, that would be good to have.
> 
> Can't we bind the raw socket to the tap interface instead of the
> physical interface and allow the bridge config to work.


We can, kind of (e.g. with veth) but what's the point then?

> Thanks
> Sridhar
> 
> 
> > 
> > > I don't have much time, but I'll help anybody who wants to do that.
> > > 
> > > -- Jamie
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Qemu-devel] Re: Release plan for 0.12.0

[Takahiro Hirofuchi]

Hello,


2009/9/30 Anthony Liguori <aliguori@us.ibm.com>:
> Hi,
>
> Now that 0.11.0 is behind us, it's time to start thinking about 0.12.0.

> o storage live migration

Sorry for a bit off topic. But, my special NBD server can do this
independently of VMM implementations.
See http://bitbucket.org/hirofuchi/xnbd/wiki/Home if interested.


Takahiro