Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[Heinrich Schuchardt]

The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:

  Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
-0400)

are available in the Git repository at:

  https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2

for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:

  efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
+0200)

----------------------------------------------------------------
Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

Up to now UEFI variables where stored in U-Boot environment variables.
Saving UEFI variables was not possible without saving the U-Boot
environment variables. With this patch series file ubootefi.var in the
EFI system partition is used for saving UEFI variables. Furthermore the
UEFI variables are exposed for reading at runtime.

Code corrections for UEFI secure boot are provided.

A buffer overrun in the RSA library is fixed.

----------------------------------------------------------------
AKASHI Takahiro (13):
      efi_loader: image_loader: add a check against certificate type of
authenticode
      efi_loader: image_loader: retrieve authenticode only if it exists
      efi_loader: signature: fix a size check against revocation list
      efi_loader: signature: make efi_hash_regions more generic
      efi_loader: image_loader: verification for all signatures should pass
      efi_loader: image_loader: add digest-based verification for signed
image
      test/py: efi_secboot: apply autopep8
      test/py: efi_secboot: more fixes against pylint
      test/py: efi_secboot: split "signed image" test case-1 into two cases
      test/py: efi_secboot: add a test against certificate revocation
      test/py: efi_secboot: add a test for multiple signatures
      test/py: efi_secboot: add a test for verifying with digest of
signed image
      lib: rsa: export rsa_verify_with_pkey()

Heinrich Schuchardt (22):
      lib/crypto: use qualified path for x509_parser.h
      efi_loader: wrong printf format in efi_image_parse
      efi_loader: fix efi_get_child_controllers()
      efi_loader: NULL dereference in efi_convert_pointer
      fs/fat: reduce data size for FAT_WRITE
      efi_loader: prepare for read only OP-TEE variables
      efi_loader: display RO attribute in printenv -e
      efi_loader: separate UEFI variable API from implemementation
      efi_loader: OsIndicationsSupported, PlatformLangCodes
      efi_loader: simplify boot manager
      efi_loader: keep attributes in efi_set_variable_int
      efi_loader: value of VendorKeys
      efi_loader: read-only AuditMode and DeployedMode
      efi_loader: secure boot flag
      efi_loader: UEFI variable persistence
      efi_loader: export efi_convert_pointer()
      efi_loader: optional pointer for ConvertPointer
      efi_loader: new function efi_memcpy_runtime()
      efi_loader: memory buffer for variables
      efi_loader: use memory based variable storage
      efi_loader: enable UEFI variables at runtime
      efi_selftest: adjust runtime test for variables

Ilias Apalodimas (1):
      efi_loader: cleanup for tee backed variables

 cmd/nvedit_efi.c                                  |  24 +-
 doc/api/efi.rst                                   |   2 +
 fs/fat/fat_write.c                                |   9 +-
 include/crypto/pkcs7_parser.h                     |   2 +-
 include/efi_api.h                                 |   2 +
 include/efi_loader.h                              |  21 +-
 include/efi_variable.h                            | 198 ++++++
 include/u-boot/rsa.h                              |   3 +
 lib/efi_loader/Kconfig                            |   8 +
 lib/efi_loader/Makefile                           |   3 +
 lib/efi_loader/efi_bootmgr.c                      |  28 +-
 lib/efi_loader/efi_boottime.c                     |  12 +-
 lib/efi_loader/efi_image_loader.c                 | 164 +++--
 lib/efi_loader/efi_runtime.c                      |  38 +-
 lib/efi_loader/efi_setup.c                        |  59 +-
 lib/efi_loader/efi_signature.c                    | 435 ++++++------
 lib/efi_loader/efi_var_common.c                   | 140 ++++
 lib/efi_loader/efi_var_file.c                     | 239 +++++++
 lib/efi_loader/efi_var_mem.c                      | 266 +++++++
 lib/efi_loader/efi_variable.c                     | 830
+++++-----------------
 lib/efi_loader/efi_variable_tee.c                 | 138 +---
 lib/efi_selftest/efi_selftest_variables_runtime.c |  13 +-
 lib/rsa/rsa-verify.c                              |   8 +-
 test/py/tests/test_efi_secboot/conftest.py        | 104 +--
 test/py/tests/test_efi_secboot/defs.py            |  14 +-
 test/py/tests/test_efi_secboot/test_authvar.py    |  92 +--
 test/py/tests/test_efi_secboot/test_signed.py     | 206 +++++-
 test/py/tests/test_efi_secboot/test_unsigned.py   |  66 +-
 28 files changed, 1867 insertions(+), 1257 deletions(-)
 create mode 100644 include/efi_variable.h
 create mode 100644 lib/efi_loader/efi_var_common.c
 create mode 100644 lib/efi_loader/efi_var_file.c
 create mode 100644 lib/efi_loader/efi_var_mem.c

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[Tom Rini]

On Thu, Jul 09, 2020 at 06:12:02PM +0200, Heinrich Schuchardt wrote:

> The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:
> 
>   Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
> -0400)
> 
> are available in the Git repository at:
> 
>   https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2
> 
> for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:
> 
>   efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
> +0200)
> 

NAK.  This is reliably failing here:
https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018

I see it passed Azure, and hasn't run through Travis yet.  Maybe it
needs to be run repeatedly to fail and we just got "lucky" ?

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200710/cfc6be73/attachment.sig>

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[Heinrich Schuchardt]

On 7/10/20 8:09 PM, Tom Rini wrote:
> On Thu, Jul 09, 2020 at 06:12:02PM +0200, Heinrich Schuchardt wrote:
>
>> The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:
>>
>>   Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
>> -0400)
>>
>> are available in the Git repository at:
>>
>>   https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2
>>
>> for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:
>>
>>   efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
>> +0200)
>>
>
> NAK.  This is reliably failing here:
> https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
>
> I see it passed Azure, and hasn't run through Travis yet.  Maybe it
> needs to be run repeatedly to fail and we just got "lucky" ?
>

Hello Tom,

you saw unreproducible results with multiple runs failing and one run
succeeding. The reason is that when signing with sign-efi-sig-list in
out Python tests without passing a timestamp two signatures may be in
the same second or not.

When using the signed files to set UEFI variables a variable can only be
overwritten by a file with a newer timestamp. But without setting
timestamps explicitly using parameter -t passed to sign-efi-sig-list we
have no control.

I already fixed this for some elder tests but missed to fix this for the
merged patches from Takahiro.

Best regards

Heinrich

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[Tom Rini]

On Sat, Jul 11, 2020 at 09:00:16AM +0200, Heinrich Schuchardt wrote:
> On 7/10/20 8:09 PM, Tom Rini wrote:
> > On Thu, Jul 09, 2020 at 06:12:02PM +0200, Heinrich Schuchardt wrote:
> >
> >> The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:
> >>
> >>   Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
> >> -0400)
> >>
> >> are available in the Git repository at:
> >>
> >>   https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2
> >>
> >> for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:
> >>
> >>   efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
> >> +0200)
> >>
> >
> > NAK.  This is reliably failing here:
> > https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
> >
> > I see it passed Azure, and hasn't run through Travis yet.  Maybe it
> > needs to be run repeatedly to fail and we just got "lucky" ?
> >
> 
> Hello Tom,
> 
> you saw unreproducible results with multiple runs failing and one run
> succeeding. The reason is that when signing with sign-efi-sig-list in
> out Python tests without passing a timestamp two signatures may be in
> the same second or not.
> 
> When using the signed files to set UEFI variables a variable can only be
> overwritten by a file with a newer timestamp. But without setting
> timestamps explicitly using parameter -t passed to sign-efi-sig-list we
> have no control.
> 
> I already fixed this for some elder tests but missed to fix this for the
> merged patches from Takahiro.

Ah, thanks for the explanation.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200711/d11c47dc/attachment.sig>

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[Heinrich Schuchardt]

On 7/11/20 2:16 PM, Tom Rini wrote:
> On Sat, Jul 11, 2020 at 09:00:16AM +0200, Heinrich Schuchardt wrote:
>> On 7/10/20 8:09 PM, Tom Rini wrote:
>>> On Thu, Jul 09, 2020 at 06:12:02PM +0200, Heinrich Schuchardt wrote:
>>>
>>>> The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:
>>>>
>>>>   Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
>>>> -0400)
>>>>
>>>> are available in the Git repository at:
>>>>
>>>>   https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2
>>>>
>>>> for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:
>>>>
>>>>   efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
>>>> +0200)
>>>>
>>>
>>> NAK.  This is reliably failing here:
>>> https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
>>>
>>> I see it passed Azure, and hasn't run through Travis yet.  Maybe it
>>> needs to be run repeatedly to fail and we just got "lucky" ?
>>>
>>
>> Hello Tom,
>>
>> you saw unreproducible results with multiple runs failing and one run
>> succeeding. The reason is that when signing with sign-efi-sig-list in
>> out Python tests without passing a timestamp two signatures may be in
>> the same second or not.
>>
>> When using the signed files to set UEFI variables a variable can only be
>> overwritten by a file with a newer timestamp. But without setting
>> timestamps explicitly using parameter -t passed to sign-efi-sig-list we
>> have no control.
>>
>> I already fixed this for some elder tests but missed to fix this for the
>> merged patches from Takahiro.
>
> Ah, thanks for the explanation.
>

Hello Tom,

what I still do not understand why tests are sometimes skipped and
sometimes not for the same source code:

https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
Commit 7068e523

140 test/py/tests/test_efi_secboot/test_authvar.py .....
141 test/py/tests/test_efi_secboot/test_signed.py .....F
142 test/py/tests/test_efi_secboot/test_unsigned.py ...

https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/122155
Commit 7068e523

148 test/py/tests/test_efi_secboot/test_authvar.py sssss
149 test/py/tests/test_efi_secboot/test_signed.py ssssss
150 test/py/tests/test_efi_secboot/test_unsigned.py sss

Both runs used the same Docker image
trini/u-boot-gitlab-ci-runner:bionic-20200526-18Jun2020

What influence have different versions of the Gitlab runner?

gitlab-runner 13.1.1
gitlab-runner 12.2.0

Some of our tests create and delete files in /tmp. How are parallel jobs
separated in Gitlab?

Best regards

Heinrich

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[Heinrich Schuchardt]

On 7/12/20 12:05 AM, Heinrich Schuchardt wrote:
> On 7/11/20 2:16 PM, Tom Rini wrote:
>> On Sat, Jul 11, 2020 at 09:00:16AM +0200, Heinrich Schuchardt wrote:
>>> On 7/10/20 8:09 PM, Tom Rini wrote:
>>>> On Thu, Jul 09, 2020 at 06:12:02PM +0200, Heinrich Schuchardt wrote:
>>>>
>>>>> The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:
>>>>>
>>>>>   Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
>>>>> -0400)
>>>>>
>>>>> are available in the Git repository at:
>>>>>
>>>>>   https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2
>>>>>
>>>>> for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:
>>>>>
>>>>>   efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
>>>>> +0200)
>>>>>
>>>>
>>>> NAK.  This is reliably failing here:
>>>> https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
>>>>
>>>> I see it passed Azure, and hasn't run through Travis yet.  Maybe it
>>>> needs to be run repeatedly to fail and we just got "lucky" ?
>>>>
>>>
>>> Hello Tom,
>>>
>>> you saw unreproducible results with multiple runs failing and one run
>>> succeeding. The reason is that when signing with sign-efi-sig-list in
>>> out Python tests without passing a timestamp two signatures may be in
>>> the same second or not.
>>>
>>> When using the signed files to set UEFI variables a variable can only be
>>> overwritten by a file with a newer timestamp. But without setting
>>> timestamps explicitly using parameter -t passed to sign-efi-sig-list we
>>> have no control.
>>>
>>> I already fixed this for some elder tests but missed to fix this for the
>>> merged patches from Takahiro.
>>
>> Ah, thanks for the explanation.
>>
>
> Hello Tom,
>
> what I still do not understand why tests are sometimes skipped and
> sometimes not for the same source code:
>
> https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
> Commit 7068e523
>
> 140 test/py/tests/test_efi_secboot/test_authvar.py .....
> 141 test/py/tests/test_efi_secboot/test_signed.py .....F
> 142 test/py/tests/test_efi_secboot/test_unsigned.py ...
>
> https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/122155
> Commit 7068e523
>
> 148 test/py/tests/test_efi_secboot/test_authvar.py sssss
> 149 test/py/tests/test_efi_secboot/test_signed.py ssssss
> 150 test/py/tests/test_efi_secboot/test_unsigned.py sss
>
> Both runs used the same Docker image
> trini/u-boot-gitlab-ci-runner:bionic-20200526-18Jun2020
>
> What influence have different versions of the Gitlab runner?
>
> gitlab-runner 13.1.1
> gitlab-runner 12.2.0
>
> Some of our tests create and delete files in /tmp. How are parallel jobs
> separated in Gitlab?
>
> Best regards
>
> Heinrich
>

This information I received on the #gitlab IRC channel:

Q:
When using Gitlab CI for building the U-Boot project many jobs run in
parallel. Some results are irreproducible. How are parallel jobs
separated in Gitlab. E.g. if parallel jobs write and delete files in
/tmp are these in separate Docker containers or are they in the same
Docker container accessing the same directory?

A:
Yes, each job is a distinct docker container on a transient VM . Nothing
is shared unless you use the https://docs.gitlab.com/ee/ci/yaml/#cache
capability, but that wouldn't help for parallel jobs. At least: on
gitlab.com, that's how it's implemented. On self-managed: depends a
little on how you choose to operate your runners (e.g. shell-executor vs
docker etc).

Best regards

Heinrich

Pull request for UEFI sub-system for efi-2020-10-rc1 (2)

[AKASHI Takahiro]

Heinrich,

On Sun, Jul 12, 2020 at 12:05:32AM +0200, Heinrich Schuchardt wrote:
> On 7/11/20 2:16 PM, Tom Rini wrote:
> > On Sat, Jul 11, 2020 at 09:00:16AM +0200, Heinrich Schuchardt wrote:
> >> On 7/10/20 8:09 PM, Tom Rini wrote:
> >>> On Thu, Jul 09, 2020 at 06:12:02PM +0200, Heinrich Schuchardt wrote:
> >>>
> >>>> The following changes since commit 61608f395e7dcb2be6060407a72a1149b046430a:
> >>>>
> >>>>   Merge branch '2020-07-08-misc-features-and-fixes' (2020-07-08 20:20:24
> >>>> -0400)
> >>>>
> >>>> are available in the Git repository at:
> >>>>
> >>>>   https://gitlab.denx.de/u-boot/custodians/u-boot-efi.git efi-2020-10-rc1-2
> >>>>
> >>>> for you to fetch changes up to f4cef8e7585c268f05a8c39e368ca115c25e40d5:
> >>>>
> >>>>   efi_selftest: adjust runtime test for variables (2020-07-09 12:08:41
> >>>> +0200)
> >>>>
> >>>
> >>> NAK.  This is reliably failing here:
> >>> https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
> >>>
> >>> I see it passed Azure, and hasn't run through Travis yet.  Maybe it
> >>> needs to be run repeatedly to fail and we just got "lucky" ?
> >>>
> >>
> >> Hello Tom,
> >>
> >> you saw unreproducible results with multiple runs failing and one run
> >> succeeding. The reason is that when signing with sign-efi-sig-list in
> >> out Python tests without passing a timestamp two signatures may be in
> >> the same second or not.
> >>
> >> When using the signed files to set UEFI variables a variable can only be
> >> overwritten by a file with a newer timestamp. But without setting
> >> timestamps explicitly using parameter -t passed to sign-efi-sig-list we
> >> have no control.
> >>
> >> I already fixed this for some elder tests but missed to fix this for the
> >> merged patches from Takahiro.
> >
> > Ah, thanks for the explanation.
> >
> 
> Hello Tom,
> 
> what I still do not understand why tests are sometimes skipped and
> sometimes not for the same source code:
> 
> https://gitlab.denx.de/u-boot/u-boot/-/jobs/122018
> Commit 7068e523
> 
> 140 test/py/tests/test_efi_secboot/test_authvar.py .....
> 141 test/py/tests/test_efi_secboot/test_signed.py .....F
> 142 test/py/tests/test_efi_secboot/test_unsigned.py ...
> 
> https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/122155
> Commit 7068e523
> 
> 148 test/py/tests/test_efi_secboot/test_authvar.py sssss
> 149 test/py/tests/test_efi_secboot/test_signed.py ssssss
> 150 test/py/tests/test_efi_secboot/test_unsigned.py sss
> 
> Both runs used the same Docker image
> trini/u-boot-gitlab-ci-runner:bionic-20200526-18Jun2020
> 
> What influence have different versions of the Gitlab runner?
> 
> gitlab-runner 13.1.1
> gitlab-runner 12.2.0
> 
> Some of our tests create and delete files in /tmp. How are parallel jobs
> separated in Gitlab?

No longer true.
You fixed it with the commit b32ac16f9a32 ("test/py: fix
test_efi_secboot/conftest.py").

-Takahiro Akashi



> Best regards
> 
> Heinrich