[Qemu-devel] OVMF Google Summer of Code ideas

[Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

I know you already have your own ideas to think of. :)
http://wiki.qemu.org/Google_Summer_of_Code_2011

But, I am wondering if any of you might have some UEFI or OVMF ideas for us:
http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=GSOC2011

Thanks,

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Michal Suchanek]

Hello

On 6 March 2011 05:12, Jordan Justen <jljusten@gmail.com> wrote:
> I know you already have your own ideas to think of. :)
> http://wiki.qemu.org/Google_Summer_of_Code_2011
>
> But, I am wondering if any of you might have some UEFI or OVMF ideas for us:
> http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=GSOC2011
>

I would like to see the ability to run x86 on arm.

I think there were some attempts to implement this and an outdated?
support for arm on x86 exists but x86 cannot be emulated on arm so
far.

That said I don't really understand arm so I probably won't be writing
the support myself and can't help much anybody trying to implement it,
at least not anytime soon.

Is there any work going on adding this support?
Would it be suitable for GSoC?

Thanks

Michal

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

On Sun, Mar 6, 2011 at 07:29, Michal Suchanek <hramrach@centrum.cz> wrote:
> On 6 March 2011 05:12, Jordan Justen <jljusten@gmail.com> wrote:
>> I know you already have your own ideas to think of. :)
>> http://wiki.qemu.org/Google_Summer_of_Code_2011
>>
>> But, I am wondering if any of you might have some UEFI or OVMF ideas for us:
>> http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=GSOC2011
>
> I would like to see the ability to run x86 on arm.

Hi.  Was this an idea for the qemu project and GSoC?

In this thread I was trying to get GSoC ideas related to a separate
(but qemu dependent) project:
https://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF

Sorry for the confusion.  I think another thread previously discussed
qemu GSoC ideas.

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Stefan Hajnoczi]

On Sun, Mar 6, 2011 at 9:15 PM, Jordan Justen <jljusten@gmail.com> wrote:
> On Sun, Mar 6, 2011 at 07:29, Michal Suchanek <hramrach@centrum.cz> wrote:
>> On 6 March 2011 05:12, Jordan Justen <jljusten@gmail.com> wrote:
>>> I know you already have your own ideas to think of. :)
>>> http://wiki.qemu.org/Google_Summer_of_Code_2011
>>>
>>> But, I am wondering if any of you might have some UEFI or OVMF ideas for us:
>>> http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=GSOC2011
>>
>> I would like to see the ability to run x86 on arm.
>
> Hi.  Was this an idea for the qemu project and GSoC?
>
> In this thread I was trying to get GSoC ideas related to a separate
> (but qemu dependent) project:
> https://sourceforge.net/apps/mediawiki/tianocore/index.php?title=OVMF

Jordan, UEFI guest firmware comes up periodically.  It seems to be a
feature that will be required in the future but as of today I don't
know of QEMU developers who are working on or using it.

Maybe you can start by giving an update on where OVMF stands today,
what the roadmap is, and where collaboration/integration with QEMU is
necessary?

Stefan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

On Mon, Mar 7, 2011 at 01:48, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> Jordan, UEFI guest firmware comes up periodically.  It seems to be a
> feature that will be required in the future but as of today I don't
> know of QEMU developers who are working on or using it.
>
> Maybe you can start by giving an update on where OVMF stands today,
> what the roadmap is, and where collaboration/integration with QEMU is
> necessary?

The status today is that we can boot UEFI Linux with qemu or kvm on
IA-32 & X64 pretty reliably.

The biggest issue for UEFI Linux seems to be getting distributions to
ship UEFI compatible ISO images.  However, there seems to be some
progress here.  For instance 64-bit Ubuntu 10.10's ISO is UEFI
bootable.

UEFI Windows 7 still does not boot on OVMF, and this is still a big open issue.

One other OVMF issue that requires some help on the VM side is that we
cannot support true non-volatile variables in OVMF due to a lack of
flash support in QEMU/KVM.

The biggest issue that you guys care about for OVMF is a lack of
legacy bios compatibility (CSM).  There has been no progress in this
area.  (However, it has now been suggested by someone on our dev list
for a potential Google Summer of Code project.  We will have to see if
anything comes of this...)

In terms of a forward looking 'roadmap,' I'm sure that the UEFI Linux
situation should only improve as more and more x86/x86-64 systems are
shipped with UEFI available.  We will continue to try to improve UEFI
Linux, and hopefully distributions will be able to leverage OVMF for
UEFI compatibility testing.

Clearly we hope to resolve the UEFI Win7 boot issue, or possibly
address UEFI booting with Win8.

Regarding the non-volatile variables issue, I have been trying to
develop a proposal for addressing this with a change to QEMU's
hardware support of bios.bin.  But, I don't have the suggestion (or
implementation) ready at this time.

Thanks,

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Stefan Hajnoczi]

On Tue, Mar 8, 2011 at 12:54 AM, Jordan Justen <jljusten@gmail.com> wrote:
> On Mon, Mar 7, 2011 at 01:48, Stefan Hajnoczi <stefanha@gmail.com> wrote:
>> Jordan, UEFI guest firmware comes up periodically.  It seems to be a
>> feature that will be required in the future but as of today I don't
>> know of QEMU developers who are working on or using it.
>>
>> Maybe you can start by giving an update on where OVMF stands today,
>> what the roadmap is, and where collaboration/integration with QEMU is
>> necessary?
>
> The status today is that we can boot UEFI Linux with qemu or kvm on
> IA-32 & X64 pretty reliably.
>
> The biggest issue for UEFI Linux seems to be getting distributions to
> ship UEFI compatible ISO images.  However, there seems to be some
> progress here.  For instance 64-bit Ubuntu 10.10's ISO is UEFI
> bootable.
>
> UEFI Windows 7 still does not boot on OVMF, and this is still a big open issue.
>
> One other OVMF issue that requires some help on the VM side is that we
> cannot support true non-volatile variables in OVMF due to a lack of
> flash support in QEMU/KVM.
>
> The biggest issue that you guys care about for OVMF is a lack of
> legacy bios compatibility (CSM).  There has been no progress in this
> area.  (However, it has now been suggested by someone on our dev list
> for a potential Google Summer of Code project.  We will have to see if
> anything comes of this...)

That would be cool.  Is there a way to just embed SeaBIOS?  Or do the
UEFI APIs still need to be around when CSM kicks in?

> In terms of a forward looking 'roadmap,' I'm sure that the UEFI Linux
> situation should only improve as more and more x86/x86-64 systems are
> shipped with UEFI available.  We will continue to try to improve UEFI
> Linux, and hopefully distributions will be able to leverage OVMF for
> UEFI compatibility testing.
>
> Clearly we hope to resolve the UEFI Win7 boot issue, or possibly
> address UEFI booting with Win8.
>
> Regarding the non-volatile variables issue, I have been trying to
> develop a proposal for addressing this with a change to QEMU's
> hardware support of bios.bin.  But, I don't have the suggestion (or
> implementation) ready at this time.

Sounds like something to keep discussing with the QEMU and SeaBIOS
communities.  Gleb Natapov and Kevin O'Connor have done a lot of the
recent BIOS and firmware interface work.  I think persistent CMOS has
come up several times and might be similar to non-volatile UEFI
storage.

Stefan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Gleb Natapov]

On Tue, Mar 08, 2011 at 07:18:09AM +0000, Stefan Hajnoczi wrote:
> > Regarding the non-volatile variables issue, I have been trying to
> > develop a proposal for addressing this with a change to QEMU's
> > hardware support of bios.bin.  But, I don't have the suggestion (or
> > implementation) ready at this time.
> 
> Sounds like something to keep discussing with the QEMU and SeaBIOS
> communities.  Gleb Natapov and Kevin O'Connor have done a lot of the
> recent BIOS and firmware interface work.  I think persistent CMOS has
> come up several times and might be similar to non-volatile UEFI
> storage.
> 
What kind of information OVMF stores on a persistent storage? CMOS
memory is less them 512 byte IIRC and this may not be enough. What OVMF
uses on real HW for non-volatile storage?

--
			Gleb.

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

2011/3/8 Gleb Natapov <gleb@redhat.com>:
> On Tue, Mar 08, 2011 at 07:18:09AM +0000, Stefan Hajnoczi wrote:
>> > Regarding the non-volatile variables issue, I have been trying to
>> > develop a proposal for addressing this with a change to QEMU's
>> > hardware support of bios.bin.  But, I don't have the suggestion (or
>> > implementation) ready at this time.
>>
>> Sounds like something to keep discussing with the QEMU and SeaBIOS
>> communities.  Gleb Natapov and Kevin O'Connor have done a lot of the
>> recent BIOS and firmware interface work.  I think persistent CMOS has
>> come up several times and might be similar to non-volatile UEFI
>> storage.
>>
> What kind of information OVMF stores on a persistent storage?

Non-volatile variables are a general system wide environment variable
storage facility, but one key thing to store (for instance) is the
path to the boot image.

> CMOS
> memory is less them 512 byte IIRC and this may not be enough. What OVMF
> uses on real HW for non-volatile storage?

Regarding CMOS, I think qemu exposes 128 bytes of non persistent CMOS
RAM.  Many current chipsets provide another bank of 128 bytes, meaning
a total of 256 bytes (minus the RTC registers).

Regarding UEFI non-volatile variables on real HW:
Most systems today have at least 1MB of flash storage located just
below 4GB.  The entire contents can be modified, which is how firmware
updates happen.

For a UEFI based system, the non-volatile variables generally occupy
8KB~64KB of the flash depending on flash space availability.

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Gleb Natapov]

On Tue, Mar 08, 2011 at 09:13:38AM -0800, Jordan Justen wrote:
> 2011/3/8 Gleb Natapov <gleb@redhat.com>:
> > On Tue, Mar 08, 2011 at 07:18:09AM +0000, Stefan Hajnoczi wrote:
> >> > Regarding the non-volatile variables issue, I have been trying to
> >> > develop a proposal for addressing this with a change to QEMU's
> >> > hardware support of bios.bin.  But, I don't have the suggestion (or
> >> > implementation) ready at this time.
> >>
> >> Sounds like something to keep discussing with the QEMU and SeaBIOS
> >> communities.  Gleb Natapov and Kevin O'Connor have done a lot of the
> >> recent BIOS and firmware interface work.  I think persistent CMOS has
> >> come up several times and might be similar to non-volatile UEFI
> >> storage.
> >>
> > What kind of information OVMF stores on a persistent storage?
> 
> Non-volatile variables are a general system wide environment variable
> storage facility, but one key thing to store (for instance) is the
> path to the boot image.
> 
> > CMOS
> > memory is less them 512 byte IIRC and this may not be enough. What OVMF
> > uses on real HW for non-volatile storage?
> 
> Regarding CMOS, I think qemu exposes 128 bytes of non persistent CMOS
> RAM.  Many current chipsets provide another bank of 128 bytes, meaning
> a total of 256 bytes (minus the RTC registers).
> 
> Regarding UEFI non-volatile variables on real HW:
> Most systems today have at least 1MB of flash storage located just
> below 4GB.  The entire contents can be modified, which is how firmware
> updates happen.
> 
How this flash storage is programmed? May be we can emulate something
similar.

> For a UEFI based system, the non-volatile variables generally occupy
> 8KB~64KB of the flash depending on flash space availability.
> 
> -Jordan

--
			Gleb.

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

2011/3/9 Gleb Natapov <gleb@redhat.com>:
> On Tue, Mar 08, 2011 at 09:13:38AM -0800, Jordan Justen wrote:
>> Regarding UEFI non-volatile variables on real HW:
>> Most systems today have at least 1MB of flash storage located just
>> below 4GB.  The entire contents can be modified, which is how firmware
>> updates happen.
>>
> How this flash storage is programmed? May be we can emulate something
> similar.

I am familiar with two common flash types used in this area.

The older firmware hub (FWH) devices, which were common around 10
years ago.  These devices would be reasonable to emulate, but for real
devices they topped out at 1MB.

More recently SPI flash devices are used, and the chipset has an SPI
controller to talk to it.  Emulating this like real hardware would be
a bit challenging.  (First you'd need to emulate the chipset SPI
controller, and then you'd need to emulate the SPI device.)

I would like to make a proposal for something more simplistic than
either of these, but which would be much easier to implement and would
better fit the more flexible size of bios.bin in qemu/kvm today.

Is it acceptable to document a proposal for a change to qemu on wiki.qemu.org?

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

On Mon, Mar 7, 2011 at 23:18, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> On Tue, Mar 8, 2011 at 12:54 AM, Jordan Justen <jljusten@gmail.com> wrote:
>> The biggest issue that you guys care about for OVMF is a lack of
>> legacy bios compatibility (CSM).  There has been no progress in this
>> area.  (However, it has now been suggested by someone on our dev list
>> for a potential Google Summer of Code project.  We will have to see if
>> anything comes of this...)
>
> That would be cool.  Is there a way to just embed SeaBIOS?  Or do the
> UEFI APIs still need to be around when CSM kicks in?

Yes, the UEFI system is still in place.  The UEFI part still handles
the majority of platform init, and calls into the CSM at various
points.  The CSM returns back to UEFI for all CSM calls, except the
legacy boot.

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Kevin O'Connor]

On Tue, Mar 08, 2011 at 09:00:05AM -0800, Jordan Justen wrote:
> On Mon, Mar 7, 2011 at 23:18, Stefan Hajnoczi <stefanha@gmail.com> wrote:
> > On Tue, Mar 8, 2011 at 12:54 AM, Jordan Justen <jljusten@gmail.com> wrote:
> >> The biggest issue that you guys care about for OVMF is a lack of
> >> legacy bios compatibility (CSM).  There has been no progress in this
> >> area.  (However, it has now been suggested by someone on our dev list
> >> for a potential Google Summer of Code project.  We will have to see if
> >> anything comes of this...)
> >
> > That would be cool.  Is there a way to just embed SeaBIOS?  Or do the
> > UEFI APIs still need to be around when CSM kicks in?
> 
> Yes, the UEFI system is still in place.  The UEFI part still handles
> the majority of platform init, and calls into the CSM at various
> points.  The CSM returns back to UEFI for all CSM calls, except the
> legacy boot.

Is there a concise list of these various callbacks between UEFI and
CSM?

If SeaBIOS just needs to be loaded up for legacy boots, that doesn't
sound too difficult.  However, if SeaBIOS would need to translate
various BIOS calls into UEFI calls - that sounds like it could be
complex.

-Kevin

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

On Tue, Mar 8, 2011 at 18:23, Kevin O'Connor <kevin@koconnor.net> wrote:
> On Tue, Mar 08, 2011 at 09:00:05AM -0800, Jordan Justen wrote:
>> Yes, the UEFI system is still in place.  The UEFI part still handles
>> the majority of platform init, and calls into the CSM at various
>> points.  The CSM returns back to UEFI for all CSM calls, except the
>> legacy boot.
>
> Is there a concise list of these various callbacks between UEFI and
> CSM?
>
> If SeaBIOS just needs to be loaded up for legacy boots, that doesn't
> sound too difficult.  However, if SeaBIOS would need to translate
> various BIOS calls into UEFI calls - that sounds like it could be
> complex.

A CSM does not really know about UEFI for the most part.  Rather it
carries out some tasks when request by the UEFI environment.  The UEFI
side still manages the high level boot flow (even for legacy boots).
The CSM does not call into UEFI services, but just returns back to
whoever invoked the CSM call.

The 16-bit CSM component interface is described in this file:
https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2/IntelFrameworkPkg/Include/Protocol/LegacyBios.h

The full CSM specification document is available here:
http://www.intel.com/technology/framework/spec.htm

Thanks,

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Natalia Portillo]

Hi all,

This may come late in the discussion, but, has OVMF been tested with Mac OS X?

A decent Intel Macintosh emulation requires of course EFI + HFS.

Regards,
Natalia Portillo

El 09/03/2011, a las 05:34, Jordan Justen escribió:

> On Tue, Mar 8, 2011 at 18:23, Kevin O'Connor <kevin@koconnor.net> wrote:
>> On Tue, Mar 08, 2011 at 09:00:05AM -0800, Jordan Justen wrote:
>>> Yes, the UEFI system is still in place.  The UEFI part still handles
>>> the majority of platform init, and calls into the CSM at various
>>> points.  The CSM returns back to UEFI for all CSM calls, except the
>>> legacy boot.
>> 
>> Is there a concise list of these various callbacks between UEFI and
>> CSM?
>> 
>> If SeaBIOS just needs to be loaded up for legacy boots, that doesn't
>> sound too difficult.  However, if SeaBIOS would need to translate
>> various BIOS calls into UEFI calls - that sounds like it could be
>> complex.
> 
> A CSM does not really know about UEFI for the most part.  Rather it
> carries out some tasks when request by the UEFI environment.  The UEFI
> side still manages the high level boot flow (even for legacy boots).
> The CSM does not call into UEFI services, but just returns back to
> whoever invoked the CSM call.
> 
> The 16-bit CSM component interface is described in this file:
> https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2/IntelFrameworkPkg/Include/Protocol/LegacyBios.h
> 
> The full CSM specification document is available here:
> http://www.intel.com/technology/framework/spec.htm
> 
> Thanks,
> 
> -Jordan
> 

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Jordan Justen]

On Wed, Mar 9, 2011 at 05:43, Natalia Portillo <claunia@claunia.com> wrote:
> This may come late in the discussion, but, has OVMF been tested with Mac OS X?

No.

I don't think Apple considers VMs an acceptable usage environment for
OS X.  Please correct me if I am mistaken.

Thanks,

-Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Natalia Portillo]

Hi,
El 09/03/2011, a las 18:44, Jordan Justen escribió:

> On Wed, Mar 9, 2011 at 05:43, Natalia Portillo <claunia@claunia.com> wrote:
>> This may come late in the discussion, but, has OVMF been tested with Mac OS X?
> 
> No.
> 
> I don't think Apple considers VMs an acceptable usage environment for
> OS X.  Please correct me if I am mistaken.

Their EULA explicitly says that OS X Server can be run in VMs, as long as the VM is running on Apple hardware (VMWare Fusion, Parallels Desktop and VMWare ESXi for Mac are clearly authorized).
For OS X non-server it just says "must run on Apple hardware" nothing specific as on VM, but none of the previously named took the risk.

VirtualBox is almost able to run OS X now but with some tricks and no "officially supported" statement.

As currently OS X searches for an encryption key on the hardware, and there has been code for providing it if being known in QEMU, as long as the requirement is reading that code on runtime on the real hardware, there should be no legal problems.
MacOnLinux allowed to virtualize OS X on PowerPC and never received communications from Apple, even if it allowed to run on non-Apple PowerPC machines.
PearPC emulated enough of a PowerMac to make OS X run and also never received communications from Apple.

In any case, IMHO, it's up to the user to respect or violate the EULA, we just provide the knife, it's not our fault it's used for assassination and not for cutting food.

> Thanks,
> 
> -Jordan

Re: [Qemu-devel] OVMF Google Summer of Code ideas

[Peter Maydell]

On 6 March 2011 15:29, Michal Suchanek <hramrach@centrum.cz> wrote:
> I would like to see the ability to run x86 on arm.
>
> I think there were some attempts to implement this and an outdated?
> support for arm on x86 exists but x86 cannot be emulated on arm so
> far.

ARM on x86 isn't outdated; it has bugs (what code doesn't?) but we're
working on squashing those; I definitely care about A-profile ARM
targets, so if you have specific bugs please report them and I will
investigate.

x86 on ARM should in theory work, because all the major pieces are
there (ie there is an x86 TCG frontend and an ARM backend), and I
believe Aurelien fixed a number of bugs in it before the 0.14
release. I just tried the linux-user mode and this seems to work:

root@localhost:~/qemu/qemu/qemu# uname -a
Linux localhost.localdomain 2.6.35-1006-linaro-omap #12-Ubuntu Tue Sep
21 20:09:17 UTC 2010 armv7l GNU/Linux
root@localhost:~/qemu/qemu/qemu# file /tmp/hello
/tmp/hello: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
statically linked, for GNU/Linux 2.6.15, not stripped
root@localhost:~/qemu/qemu/qemu# ./i386-linux-user/qemu-i386 /tmp/hello
Hello, x86 on ARM world!

I haven't tested the system-emulation mode but I don't think there
is anything known to be missing that would make it not work. So I
think this is also in the "report bugs for things which don't work"
phase.

One caveat: I had to configure with '--extra-cflags=-marm', otherwise
qemu segfaulted; I guess we aren't properly coping with the C code
being compiled for Thumb...

-- PMM