Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f

On Mon, Sep 07, Laszlo Ersek wrote:

> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
> decompressor source code that edk2 had flattened into BaseTools was
> replaced with a git submodule.
> 
> This means we have to initialize edk2's own submodules before building
> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile" (for
> the sake of the "efirom" target) and "tests/uefi-test-tools/Makefile" as
> well.

> +++ b/roms/Makefile
>  edk2-basetools:
> +	cd edk2/BaseTools && git submodule update --init --force
>  build-edk2-tools:
> +	cd $(edk2_dir)/BaseTools && git submodule update --init --force


This change can not possibly be correct.

With current qemu.git#master one is forced to have network access to
build the roms. This fails with exported (and complete) sources in an
offline environment.


Please revert this patch.


Olaf

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Philippe Mathieu-Daudé]

Hi Olaf,

On 10/20/20 11:16 AM, Olaf Hering wrote:
> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
> 
> On Mon, Sep 07, Laszlo Ersek wrote:
> 
>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
>> decompressor source code that edk2 had flattened into BaseTools was
>> replaced with a git submodule.
>>
>> This means we have to initialize edk2's own submodules before building
>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile" (for
>> the sake of the "efirom" target) and "tests/uefi-test-tools/Makefile" as
>> well.
> 
>> +++ b/roms/Makefile
>>   edk2-basetools:
>> +	cd edk2/BaseTools && git submodule update --init --force
>>   build-edk2-tools:
>> +	cd $(edk2_dir)/BaseTools && git submodule update --init --force
> 
> 
> This change can not possibly be correct.
> 
> With current qemu.git#master one is forced to have network access to
> build the roms. This fails with exported (and complete) sources in an
> offline environment.

The EDK2 roms are only used for testing, we certainly don't want them
to be used by distributions. I suppose the question is "why is this
rule called if tests are not built?".

> Please revert this patch.
> 
> 
> Olaf
> 

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 275 bytes --]

Am Tue, 20 Oct 2020 11:29:01 +0200
schrieb Philippe Mathieu-Daudé <philmd@redhat.com>:

> The EDK2 roms are only used for testing, we certainly don't want them
> to be used by distributions. 

You mean 'make -C rom efirom' is something that must not be done?

Olaf

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Philippe Mathieu-Daudé]

On 10/20/20 11:29 AM, Philippe Mathieu-Daudé wrote:
> Hi Olaf,
> 
> On 10/20/20 11:16 AM, Olaf Hering wrote:
>> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
>>
>> On Mon, Sep 07, Laszlo Ersek wrote:
>>
>>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
>>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
>>> decompressor source code that edk2 had flattened into BaseTools was
>>> replaced with a git submodule.
>>>
>>> This means we have to initialize edk2's own submodules before building
>>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile" (for
>>> the sake of the "efirom" target) and "tests/uefi-test-tools/Makefile" as
>>> well.
>>
>>> +++ b/roms/Makefile
>>>   edk2-basetools:
>>> +    cd edk2/BaseTools && git submodule update --init --force
>>>   build-edk2-tools:
>>> +    cd $(edk2_dir)/BaseTools && git submodule update --init --force
>>
>>
>> This change can not possibly be correct.
>>
>> With current qemu.git#master one is forced to have network access to
>> build the roms. This fails with exported (and complete) sources in an
>> offline environment.
> 
> The EDK2 roms are only used for testing, we certainly don't want them
> to be used by distributions. I suppose the question is "why is this
> rule called if tests are not built?".
> 
>> Please revert this patch.

Maybe to clarify the use of this submodule:

-- >8 --
Author: Philippe Mathieu-Daudé <philmd@redhat.com>
Date:   Tue Oct 20 11:33:22 2020 +0200

     EDK2 submodule should only be used for testing QEMU

     Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>

diff --git a/.gitmodules b/.gitmodules
index 2bdeeacef88..dd9b5ed03e1 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -47,7 +47,7 @@
         path = tests/fp/berkeley-softfloat-3
         url = https://git.qemu.org/git/berkeley-softfloat-3.git
  [submodule "roms/edk2"]
-       path = roms/edk2
+       path = 
roms/edk2-unsafe-with-all-security-features-disabled-only-usable-for-testing
         url = https://git.qemu.org/git/edk2.git
  [submodule "slirp"]
         path = slirp
diff --git a/MAINTAINERS b/MAINTAINERS
index a7f0acf8663..35f977efbcc 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2732,7 +2732,7 @@ S: Supported
  F: pc-bios/descriptors/??-edk2-*.json
  F: pc-bios/edk2-*
  F: roms/Makefile.edk2
-F: roms/edk2
+F: 
roms/edk2-unsafe-with-all-security-features-disabled-only-usable-for-testing
  F: roms/edk2-*
  F: tests/data/uefi-boot-images/
  F: tests/uefi-test-tools/
diff --git a/roms/edk2 
b/roms/edk2-unsafe-with-all-security-features-disabled-only-usable-for-testing
similarity index 100%
rename from roms/edk2
rename to 
roms/edk2-unsafe-with-all-security-features-disabled-only-usable-for-testing
---

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Daniel P. Berrangé]

On Tue, Oct 20, 2020 at 11:29:01AM +0200, Philippe Mathieu-Daudé wrote:
> Hi Olaf,
> 
> On 10/20/20 11:16 AM, Olaf Hering wrote:
> > This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
> > 
> > On Mon, Sep 07, Laszlo Ersek wrote:
> > 
> > > In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
> > > 2020-04-16), part of edk2-stable202005, the Brotli compressor /
> > > decompressor source code that edk2 had flattened into BaseTools was
> > > replaced with a git submodule.
> > > 
> > > This means we have to initialize edk2's own submodules before building
> > > BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile" (for
> > > the sake of the "efirom" target) and "tests/uefi-test-tools/Makefile" as
> > > well.
> > 
> > > +++ b/roms/Makefile
> > >   edk2-basetools:
> > > +	cd edk2/BaseTools && git submodule update --init --force
> > >   build-edk2-tools:
> > > +	cd $(edk2_dir)/BaseTools && git submodule update --init --force
> > 
> > 
> > This change can not possibly be correct.
> > 
> > With current qemu.git#master one is forced to have network access to
> > build the roms. This fails with exported (and complete) sources in an
> > offline environment.
> 
> The EDK2 roms are only used for testing, we certainly don't want them
> to be used by distributions. I suppose the question is "why is this
> rule called if tests are not built?".

I don't believe that is correct - the pc-bios/edk*  ROMs and the
corresponding  pc-bios/descriptor files are there for real world
end user consumption.   roms/edk2 should (must) match / reflect
the content used to build the pci-bios/edk* blobs.

Many distros have a policy requiring them to build everything
from source, so they will ignore the pre-built edk2 ROMs, but
regular end users taking QEMU directly from upstream can certainly
use our edk2 ROMs.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Philippe Mathieu-Daudé]

On 10/20/20 11:44 AM, Daniel P. Berrangé wrote:
> On Tue, Oct 20, 2020 at 11:29:01AM +0200, Philippe Mathieu-Daudé wrote:
>> Hi Olaf,
>>
>> On 10/20/20 11:16 AM, Olaf Hering wrote:
>>> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
>>>
>>> On Mon, Sep 07, Laszlo Ersek wrote:
>>>
>>>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
>>>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
>>>> decompressor source code that edk2 had flattened into BaseTools was
>>>> replaced with a git submodule.
>>>>
>>>> This means we have to initialize edk2's own submodules before building
>>>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile" (for
>>>> the sake of the "efirom" target) and "tests/uefi-test-tools/Makefile" as
>>>> well.
>>>
>>>> +++ b/roms/Makefile
>>>>    edk2-basetools:
>>>> +	cd edk2/BaseTools && git submodule update --init --force
>>>>    build-edk2-tools:
>>>> +	cd $(edk2_dir)/BaseTools && git submodule update --init --force
>>>
>>>
>>> This change can not possibly be correct.
>>>
>>> With current qemu.git#master one is forced to have network access to
>>> build the roms. This fails with exported (and complete) sources in an
>>> offline environment.
>>
>> The EDK2 roms are only used for testing, we certainly don't want them
>> to be used by distributions. I suppose the question is "why is this
>> rule called if tests are not built?".
> 
> I don't believe that is correct - the pc-bios/edk*  ROMs and the
> corresponding  pc-bios/descriptor files are there for real world
> end user consumption.   roms/edk2 should (must) match / reflect
> the content used to build the pci-bios/edk* blobs.
> 
> Many distros have a policy requiring them to build everything
> from source, so they will ignore the pre-built edk2 ROMs, but
> regular end users taking QEMU directly from upstream can certainly
> use our edk2 ROMs.

Well I'm lost (and I don't think mainstream QEMU have the
bandwidth to follow mainstream EDK2 security fixes) so I'm
giving up, waiting for clarification from Laszlo.

> 
> 
> Regards,
> Daniel
> 

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Laszlo Ersek]

On 10/20/20 11:54, Philippe Mathieu-Daudé wrote:
> On 10/20/20 11:44 AM, Daniel P. Berrangé wrote:
>> On Tue, Oct 20, 2020 at 11:29:01AM +0200, Philippe Mathieu-Daudé wrote:
>>> Hi Olaf,
>>>
>>> On 10/20/20 11:16 AM, Olaf Hering wrote:
>>>> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
>>>>
>>>> On Mon, Sep 07, Laszlo Ersek wrote:
>>>>
>>>>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
>>>>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
>>>>> decompressor source code that edk2 had flattened into BaseTools was
>>>>> replaced with a git submodule.
>>>>>
>>>>> This means we have to initialize edk2's own submodules before building
>>>>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile"
>>>>> (for
>>>>> the sake of the "efirom" target) and
>>>>> "tests/uefi-test-tools/Makefile" as
>>>>> well.
>>>>
>>>>> +++ b/roms/Makefile
>>>>>    edk2-basetools:
>>>>> +    cd edk2/BaseTools && git submodule update --init --force
>>>>>    build-edk2-tools:
>>>>> +    cd $(edk2_dir)/BaseTools && git submodule update --init --force
>>>>
>>>>
>>>> This change can not possibly be correct.
>>>>
>>>> With current qemu.git#master one is forced to have network access to
>>>> build the roms. This fails with exported (and complete) sources in an
>>>> offline environment.
>>>
>>> The EDK2 roms are only used for testing, we certainly don't want them
>>> to be used by distributions. I suppose the question is "why is this
>>> rule called if tests are not built?".
>>
>> I don't believe that is correct - the pc-bios/edk*  ROMs and the
>> corresponding  pc-bios/descriptor files are there for real world
>> end user consumption.   roms/edk2 should (must) match / reflect
>> the content used to build the pci-bios/edk* blobs.
>>
>> Many distros have a policy requiring them to build everything
>> from source, so they will ignore the pre-built edk2 ROMs, but
>> regular end users taking QEMU directly from upstream can certainly
>> use our edk2 ROMs.
> 
> Well I'm lost (and I don't think mainstream QEMU have the
> bandwidth to follow mainstream EDK2 security fixes) so I'm
> giving up, waiting for clarification from Laszlo.

I definitely don't have time for keeping the edk2 blobs bundled with
QEMU fresh wrt. security fixes in upstream edk2, so anyone expecting
that is in for a bad surprise. The blobs are provided, from my
perspective, (a) for some tests in the test suite (such as
bios-tables-test for the aarch64 target), (b) as a convenience for
end-users that desire to build QEMU from source, without wanting to
build OVMF from source.

I don't understand the particular problem (or rather: use case) that
Olaf is reporting (and this is not the first time). I see four classes
of people here:

(1) end-users described above, in point (b) -- then, there is no need
for rebuilding the bundled edk2 binaries using the QEMU build infrastructure

(2) end-users building everything from source (genuine standalone clones
/ checkouts), for themselves

(3) distributors building everything from source (genuine, standalone
clones / checkouts), for their users

(4) QEMU co-maintainers that sometimes refresh the binaries -- this is
the only group that the build infra *needs* to work for (in the future,
the edk2 build infra should actually target a remote build system, but
we're not there yet -- and even in that case, the edk2 build scripts
inside the QEMU tree will only have to work for *that* environment)

Olaf: if you build QEMU from source, why don't you build SeaBIOS, iPXE,
edk2 etc *also* from their corresponding pristine upstream clones /
checkouts, using your own dedicated build scripts / packagings?

... On the technical side, I guess the problem is that edk2, unlike some
other submodules of QEMU, has its own submodules (meaning that, from the
QEMU superproject's perspective, edk2 creates recusrive submodules). I
have really zero idea how to deal with that (or more precisely, what the
grander impact of that would be); but importantly, it does not *matter*,
in my opinion. If you don't co-maintain the edk2 binaries bundled with
QEMU, then the edk2 build stuff present in QEMU is not *required* to
work for you.

If you don't like that, feel free to post patches, or I can quit even
this level of maintenance for the bundled edk2 binaries. I will
absolutely not consider downstream packaging needs with *how* the
bundled edk2 binaries are built.

Thanks
Laszlo

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]

Am Wed, 21 Oct 2020 14:05:18 +0200
schrieb Laszlo Ersek <lersek@redhat.com>:

> Olaf: if you build QEMU from source, why don't you build SeaBIOS, iPXE,
> edk2 etc *also* from their corresponding pristine upstream clones /
> checkouts, using your own dedicated build scripts / packagings?

From my perspective it is like that:

I export xen/qemu/libivrt into an offline environment for building.
The "git clone/git export" is done without submodules, but each required submodule is of course cloned/exported as well into the target directory. In the end it is me who decides what is required or not, which means only a subset of all submodules need to be provided. The build process sees the complete source, and as a result nothing needs to be downloaded.

With current master there are these two offending git commands. In my environment they can not do anything but fail. I guess once the next qemu-X.Y release becomes available as the usual "qemu-X.Y.tar.xz" release these git commands will fail as well with 'make -C roms efirom'.

As said elsewhere, the correct approach might be to check what is missing and download only these submodules. This should take the existing configure knobs into account.


Olaf

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Laszlo Ersek]

On 10/21/20 14:30, Olaf Hering wrote:
> Am Wed, 21 Oct 2020 14:05:18 +0200
> schrieb Laszlo Ersek <lersek@redhat.com>:
> 
>> Olaf: if you build QEMU from source, why don't you build SeaBIOS, iPXE,
>> edk2 etc *also* from their corresponding pristine upstream clones /
>> checkouts, using your own dedicated build scripts / packagings?
> 
> From my perspective it is like that:
> 
> I export xen/qemu/libivrt into an offline environment for building.

Makes sense.

> The "git clone/git export" is done without submodules, but each required submodule is of course cloned/exported as well into the target directory.

The "roms/edk2" submodule of QEMU is *not required* for building QEMU.

> In the end it is me who decides what is required or not, which means only a subset of all submodules need to be provided. The build process sees the complete source, and as a result nothing needs to be downloaded.
> 
> With current master there are these two offending git commands. In my environment they can not do anything but fail. I guess once the next qemu-X.Y release becomes available as the usual "qemu-X.Y.tar.xz" release these git commands will fail as well with 'make -C roms efirom'.

I can't fathom why someone would want to run "make -C roms efirom"
against a tarball release of QEMU. That command does not participate in
building QEMU proper. It participates in refreshing some binaries that
serve convenience and/or self-check purposes.

(In case of edk2, there isn't even a legal/licensing requirement for
distributing the source of the firmware, along with the binary. See
"pc-bios/edk2-licenses.txt".)

> As said elsewhere, the correct approach might be to check what is missing and download only these submodules. This should take the existing configure knobs into account.

The "make -C roms efirom" command was never meant to work in the
environment -- such as an offline environment -- in which you have been
running it. I have not once tested the build scripts like that.

I'm OK to review and regression-test patches to this end, but I'm not
interested in authoring them (nor in testing them in an offline env).

Thanks
Laszlo

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Daniel P. Berrangé]

On Wed, Oct 21, 2020 at 02:05:18PM +0200, Laszlo Ersek wrote:
> On 10/20/20 11:54, Philippe Mathieu-Daudé wrote:
> > On 10/20/20 11:44 AM, Daniel P. Berrangé wrote:
> >> On Tue, Oct 20, 2020 at 11:29:01AM +0200, Philippe Mathieu-Daudé wrote:
> >>> Hi Olaf,
> >>>
> >>> On 10/20/20 11:16 AM, Olaf Hering wrote:
> >>>> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
> >>>>
> >>>> On Mon, Sep 07, Laszlo Ersek wrote:
> >>>>
> >>>>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
> >>>>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
> >>>>> decompressor source code that edk2 had flattened into BaseTools was
> >>>>> replaced with a git submodule.
> >>>>>
> >>>>> This means we have to initialize edk2's own submodules before building
> >>>>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile"
> >>>>> (for
> >>>>> the sake of the "efirom" target) and
> >>>>> "tests/uefi-test-tools/Makefile" as
> >>>>> well.
> >>>>
> >>>>> +++ b/roms/Makefile
> >>>>>    edk2-basetools:
> >>>>> +    cd edk2/BaseTools && git submodule update --init --force
> >>>>>    build-edk2-tools:
> >>>>> +    cd $(edk2_dir)/BaseTools && git submodule update --init --force
> >>>>
> >>>>
> >>>> This change can not possibly be correct.
> >>>>
> >>>> With current qemu.git#master one is forced to have network access to
> >>>> build the roms. This fails with exported (and complete) sources in an
> >>>> offline environment.
> >>>
> >>> The EDK2 roms are only used for testing, we certainly don't want them
> >>> to be used by distributions. I suppose the question is "why is this
> >>> rule called if tests are not built?".
> >>
> >> I don't believe that is correct - the pc-bios/edk*  ROMs and the
> >> corresponding  pc-bios/descriptor files are there for real world
> >> end user consumption.   roms/edk2 should (must) match / reflect
> >> the content used to build the pci-bios/edk* blobs.
> >>
> >> Many distros have a policy requiring them to build everything
> >> from source, so they will ignore the pre-built edk2 ROMs, but
> >> regular end users taking QEMU directly from upstream can certainly
> >> use our edk2 ROMs.
> > 
> > Well I'm lost (and I don't think mainstream QEMU have the
> > bandwidth to follow mainstream EDK2 security fixes) so I'm
> > giving up, waiting for clarification from Laszlo.
> 
> I definitely don't have time for keeping the edk2 blobs bundled with
> QEMU fresh wrt. security fixes in upstream edk2, so anyone expecting
> that is in for a bad surprise. The blobs are provided, from my
> perspective, (a) for some tests in the test suite (such as
> bios-tables-test for the aarch64 target), (b) as a convenience for
> end-users that desire to build QEMU from source, without wanting to
> build OVMF from source.

The issue with security is not unique to EDK2. Essentially all the
binary blob firmwares that QEMU distributes have this problem, since
we dont update any of them in response to upstream security issues
in any reliable timeframe.  EDK2 is probably most dangerous since
its code base is relatively larger than other firmwares, but they
are all essentially doomed.

This is why distros should generall ybuild as many of the ROMs as
possible from scratch using latest available upstream source, not
what QEMU distributes.

I wish we would actually ship a qemu tarball which excluded all the
pre-built ROMs and bundle them in a separate add-on tarball, with a
warning that they shouldn't be used in any "virtualization" use case
in production, only for non-virtualization use cases, as described in

  https://www.qemu.org/docs/master/system/security.html

because the latter is where security does not matter.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Laszlo Ersek]

On 10/21/20 15:46, Daniel P. Berrangé wrote:
> On Wed, Oct 21, 2020 at 02:05:18PM +0200, Laszlo Ersek wrote:
>> On 10/20/20 11:54, Philippe Mathieu-Daudé wrote:
>>> On 10/20/20 11:44 AM, Daniel P. Berrangé wrote:
>>>> On Tue, Oct 20, 2020 at 11:29:01AM +0200, Philippe Mathieu-Daudé wrote:
>>>>> Hi Olaf,
>>>>>
>>>>> On 10/20/20 11:16 AM, Olaf Hering wrote:
>>>>>> This is about qemu.git#ec87b5daca761039bbcf781eedbe4987f790836f
>>>>>>
>>>>>> On Mon, Sep 07, Laszlo Ersek wrote:
>>>>>>
>>>>>>> In edk2 commit 06033f5abad3 ("BaseTools: Make brotli a submodule",
>>>>>>> 2020-04-16), part of edk2-stable202005, the Brotli compressor /
>>>>>>> decompressor source code that edk2 had flattened into BaseTools was
>>>>>>> replaced with a git submodule.
>>>>>>>
>>>>>>> This means we have to initialize edk2's own submodules before building
>>>>>>> BaseTools not just in "roms/Makefile.edk2", but in "roms/Makefile"
>>>>>>> (for
>>>>>>> the sake of the "efirom" target) and
>>>>>>> "tests/uefi-test-tools/Makefile" as
>>>>>>> well.
>>>>>>
>>>>>>> +++ b/roms/Makefile
>>>>>>>    edk2-basetools:
>>>>>>> +    cd edk2/BaseTools && git submodule update --init --force
>>>>>>>    build-edk2-tools:
>>>>>>> +    cd $(edk2_dir)/BaseTools && git submodule update --init --force
>>>>>>
>>>>>>
>>>>>> This change can not possibly be correct.
>>>>>>
>>>>>> With current qemu.git#master one is forced to have network access to
>>>>>> build the roms. This fails with exported (and complete) sources in an
>>>>>> offline environment.
>>>>>
>>>>> The EDK2 roms are only used for testing, we certainly don't want them
>>>>> to be used by distributions. I suppose the question is "why is this
>>>>> rule called if tests are not built?".
>>>>
>>>> I don't believe that is correct - the pc-bios/edk*  ROMs and the
>>>> corresponding  pc-bios/descriptor files are there for real world
>>>> end user consumption.   roms/edk2 should (must) match / reflect
>>>> the content used to build the pci-bios/edk* blobs.
>>>>
>>>> Many distros have a policy requiring them to build everything
>>>> from source, so they will ignore the pre-built edk2 ROMs, but
>>>> regular end users taking QEMU directly from upstream can certainly
>>>> use our edk2 ROMs.
>>>
>>> Well I'm lost (and I don't think mainstream QEMU have the
>>> bandwidth to follow mainstream EDK2 security fixes) so I'm
>>> giving up, waiting for clarification from Laszlo.
>>
>> I definitely don't have time for keeping the edk2 blobs bundled with
>> QEMU fresh wrt. security fixes in upstream edk2, so anyone expecting
>> that is in for a bad surprise. The blobs are provided, from my
>> perspective, (a) for some tests in the test suite (such as
>> bios-tables-test for the aarch64 target), (b) as a convenience for
>> end-users that desire to build QEMU from source, without wanting to
>> build OVMF from source.
> 
> The issue with security is not unique to EDK2. Essentially all the
> binary blob firmwares that QEMU distributes have this problem, since
> we dont update any of them in response to upstream security issues
> in any reliable timeframe.  EDK2 is probably most dangerous since
> its code base is relatively larger than other firmwares, but they
> are all essentially doomed.
> 
> This is why distros should generall ybuild as many of the ROMs as
> possible from scratch using latest available upstream source, not
> what QEMU distributes.
> 
> I wish we would actually ship a qemu tarball which excluded all the
> pre-built ROMs and bundle them in a separate add-on tarball, with a
> warning that they shouldn't be used in any "virtualization" use case
> in production, only for non-virtualization use cases, as described in
> 
>   https://www.qemu.org/docs/master/system/security.html
> 
> because the latter is where security does not matter.

Can we implement the lightweight variant of this, namely extend
"pc-bios/README" to communicate this limitation? What you describe above
could be the very first paragraph (a file-top banner) in that file.


Also, it seems like "pc-bios/README" does not get installed under
$PREFIX/share/qemu. That looks like a general omission around
INSTALL_BLOBS / BLOBS to me. I can't remember -- was this the case
(i.e., "pc-bios/README" not getting installed) before the Meson
conversion too? (CC Paolo, Marc-André.)

Hm... even at the time of commit 26ce90fde5ce ("Makefile: install the
edk2 firmware images and their descriptors", 2019-04-17), BLOBS didn't
cover the "pc-bios/README" file.

Thanks,
Laszlo

Re: roms/efirom, tests/uefi-test-tools: update edk2's own submodules first

[Olaf Hering]

[-- Attachment #1: Type: text/plain, Size: 342 bytes --]

Am Tue, 20 Oct 2020 11:16:18 +0200
schrieb Olaf Hering <olaf@aepfle.de>:

> > +	cd edk2/BaseTools && git submodule update --init --force

After looking further, there are more bugs:

configure has a --with-git knob, which this patch ignores.
configure has a --enable-git-update knob, which this patch should most likely use.

Olaf

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]