[xen-unstable-smoke test] 171511: regressions - FAIL

[xen-unstable-smoke test] 171511: regressions - FAIL

[osstest service owner]

flight 171511 xen-unstable-smoke real [real]
flight 171517 xen-unstable-smoke real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/171511/
http://logs.test-lab.xenproject.org/osstest/logs/171517/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 171486

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt     15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          15 migrate-support-check        fail   never pass
 test-armhf-armhf-xl          16 saverestore-support-check    fail   never pass

version targeted for testing:
 xen                  61ff2733221e3b5bae5f647d9a460c7a68a5ace8
baseline version:
 xen                  4df2e99d731402da48afb19dc970ccab5a0814d6

Last test of basis   171486  2022-07-04 13:00:25 Z    1 days
Failing since        171501  2022-07-05 12:03:08 Z    0 days    3 attempts
Testing same since   171506  2022-07-05 17:00:29 Z    0 days    2 attempts

------------------------------------------------------------
People who touched revisions under test:
  Anthony PERARD <anthony.perard@citrix.com>
  Demi Marie Obenour <demi@invisiblethingslab.com>
  Jan Beulich <jbeulich@suse.com>
  Julien Grall <jgrall@amazon.com>
  Luca Fancellu <luca.fancellu@arm.com>
  Michal Orzel <michal.orzel@arm.com>
  Roger Pau Monne <roger.pau@citrix.com>
  Roger Pau Monné <roger.pau@citrix.com>
  Wei Chen <wei.chen@arm.com>

jobs:
 build-arm64-xsm                                              pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-amd64-libvirt                                          pass    
 test-armhf-armhf-xl                                          pass    
 test-arm64-arm64-xl-xsm                                      fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-amd64-libvirt                                     pass    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

------------------------------------------------------------
commit 61ff2733221e3b5bae5f647d9a460c7a68a5ace8
Author: Michal Orzel <michal.orzel@arm.com>
Date:   Mon Jun 27 15:15:39 2022 +0200

    xen/common: Use unsigned int instead of plain unsigned
    
    This is just for the style and consistency reasons as the former is
    being used more often than the latter.
    
    Signed-off-by: Michal Orzel <michal.orzel@arm.com>
    Reviewed-by: Juergen Gross <jgross@suse.com>
    Acked-by: Jan Beulich <jbeulich@suse.com>

commit 54d8f27d0477937e1f99a414fc1ffd93d184b38a
Author: Roger Pau Monne <roger.pau@citrix.com>
Date:   Fri Apr 8 10:21:11 2022 +0200

    tools/libxl: report trusted backend status to frontends
    
    Allow administrators to notify a frontend driver that it's backend
    counterpart is not to be trusted, so the frontend can deploy whatever
    mitigations required in order to secure itself.
    
    Allow such option for disk and network frontends only, as those are
    the only hardened ones currently supported.
    
    This is part of XSA-403
    
    Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
    Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

commit a4d4c541f58b378bc9d499dcb554eb9fe22312c8
Author: Wei Chen <wei.chen@arm.com>
Date:   Tue Jul 5 13:12:15 2022 +0200

    xen/arm32: avoid EFI stub wchar_t size linker warning
    
    Xen uses "-fshort-wchar" in CFLAGS for EFI common code. Arm32
    is using stub.c of EFI common code for EFI stub functions. But
    "-fshort-wchar" CFLAG will cause a warning when build stub.c
    for Arm32:
    "arm-linux-gnueabihf-ld: warning: arch/arm/efi/built_in.o uses
    2-byte wchar_t yet the output is to use 4-byte wchar_t; use of
    wchar_t values across objects may fail"
    
    This is because the "-fshort-wchar" flag causes GCC to generate
    code that is not binary compatible with code generated without
    that flag. Why this warning hasn't been triggered in Arm64 is
    because Arm64 does not use wchar type directly in any code for
    parameters, variables and return values. And in EFI code, wchar
    has been replaced by CHAR16 (the UEFI "abstraction" of wchar_t).
    CHAR16 has been specified as unsigned short type in typedef, the
    "-fshort-wchar" flag will not affect CHAR16. So Arm64 object
    files are exactly the same with "-fshort-wchar" and without
    "-fshort-wchar".
    
    We are also not using wchar in Arm32 codes, but Arm32 will embed
    ABI information in ".ARM.attributes" section. This section stores
    some object file attributes, like ABI version, CPU arch and etc.
    And wchar size is described in this section by "Tag_ABI_PCS_wchar_t"
    too. Tag_ABI_PCS_wchar_t is 2 for object files with "-fshort-wchar",
    but for object files without "-fshort-wchar" is 4. Arm32 GCC
    ld will check this tag, and throw above warning when it finds
    the object files have different Tag_ABI_PCS_wchar_t values.
    
    Xen need to keep "-fshort-wchar" in EFI code to force wchar to use
    short integers (2 bytes) instead of integers (4 bytes), but this is
    unnecessary for code out of EFI. So in this patch, we add
    "-fno-short-wchar" to override "-fshort-wchar" for Arm architectures
    without EFI enabled to remove above warning."
    
    Reported-and-Suggested-by: Jan Beulich <jbeulich@suse.com>
    Tested-by: Jan Beulich <jbeulich@suse.com>
    Signed-off-by: Wei Chen <wei.chen@arm.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
    Acked-by: Julien Grall <jgrall@amazon.com>

commit c4184bf305dc14c3e150617904c40b120664efe6
Author: Jan Beulich <jbeulich@suse.com>
Date:   Tue Jul 5 13:11:51 2022 +0200

    public: constify xsd_errors[]
    
    While in principle this could break existing users, I think such users
    deserve to be put in trouble. After all the table should have been const
    from the very beginning.
    
    Signed-off-by: Jan Beulich <jbeulich@suse.com>
    Reviewed-by: Juergen Gross <jgross@suse.com>

commit 2b1ee386122a6e8bf66f5163cbda51084af6e0f4
Author: Luca Fancellu <luca.fancellu@arm.com>
Date:   Tue Jul 5 13:11:25 2022 +0200

    tools/helpers: fix snprintf argument in init-dom0less.c
    
    Fix snprintf argument in init-dom0less.c because two instances of
    the function are using libxl_dominfo struct members that are uint64_t
    types, so change "%lu" to "%"PRIu64 to handle it properly when
    building on arm32 and arm64.
    
    Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
    Reviewed-by: Bertrand Marquis <bertrand.marquis@arm.com>
    Acked-by: Anthony PERARD <anthony.perard@citrix.com>

commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
Author: Demi Marie Obenour <demi@invisiblethingslab.com>
Date:   Tue Jul 5 13:10:46 2022 +0200

    EFI: preserve the System Resource Table for dom0
    
    The EFI System Resource Table (ESRT) is necessary for fwupd to identify
    firmware updates to install.  According to the UEFI specification §23.4,
    the ESRT shall be stored in memory of type EfiBootServicesData.  However,
    memory of type EfiBootServicesData is considered general-purpose memory
    by Xen, so the ESRT needs to be moved somewhere where Xen will not
    overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
    which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
    in memory of type EfiRuntimeServicesData.
    
    Earlier versions of this patch reserved the memory in which the ESRT was
    located.  This created awkward alignment problems, and required either
    splitting the E820 table or wasting memory.  It also would have required
    a new platform op for dom0 to use to indicate if the ESRT is reserved.
    By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
    does not need to be modified, and dom0 can just check the type of the
    memory region containing the ESRT.  The copy is only done if the ESRT is
    not already in EfiRuntimeServicesData memory, avoiding memory leaks on
    repeated kexec.
    
    See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
    for details.
    
    Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
    Reviewed-by: Jan Beulich <jbeulich@suse.com>
(qemu changes not included)

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Jan Beulich]

On 06.07.2022 05:39, osstest service owner wrote:
> flight 171511 xen-unstable-smoke real [real]
> flight 171517 xen-unstable-smoke real-retest [real]
> http://logs.test-lab.xenproject.org/osstest/logs/171511/
> http://logs.test-lab.xenproject.org/osstest/logs/171517/
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 171486

Looking at what's under test, I guess ...

> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
> Date:   Tue Jul 5 13:10:46 2022 +0200
> 
>     EFI: preserve the System Resource Table for dom0
>     
>     The EFI System Resource Table (ESRT) is necessary for fwupd to identify
>     firmware updates to install.  According to the UEFI specification §23.4,
>     the ESRT shall be stored in memory of type EfiBootServicesData.  However,
>     memory of type EfiBootServicesData is considered general-purpose memory
>     by Xen, so the ESRT needs to be moved somewhere where Xen will not
>     overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
>     which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
>     in memory of type EfiRuntimeServicesData.
>     
>     Earlier versions of this patch reserved the memory in which the ESRT was
>     located.  This created awkward alignment problems, and required either
>     splitting the E820 table or wasting memory.  It also would have required
>     a new platform op for dom0 to use to indicate if the ESRT is reserved.
>     By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
>     does not need to be modified, and dom0 can just check the type of the
>     memory region containing the ESRT.  The copy is only done if the ESRT is
>     not already in EfiRuntimeServicesData memory, avoiding memory leaks on
>     repeated kexec.
>     
>     See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
>     for details.
>     
>     Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>     Reviewed-by: Jan Beulich <jbeulich@suse.com>

... this is the most likely candidate, considering in the log all we
see is:

Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.

But I guess we'll want to wait for the bi-sector to give us a more
solid indication ...

Jan

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Demi Marie Obenour]

[-- Attachment #1: Type: text/plain, Size: 2532 bytes --]

> > commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
> > Author: Demi Marie Obenour <demi@invisiblethingslab.com>
> > Date:   Tue Jul 5 13:10:46 2022 +0200
> > 
> >     EFI: preserve the System Resource Table for dom0
> >     
> >     The EFI System Resource Table (ESRT) is necessary for fwupd to identify
> >     firmware updates to install.  According to the UEFI specification §23.4,
> >     the ESRT shall be stored in memory of type EfiBootServicesData.  However,
> >     memory of type EfiBootServicesData is considered general-purpose memory
> >     by Xen, so the ESRT needs to be moved somewhere where Xen will not
> >     overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
> >     which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
> >     in memory of type EfiRuntimeServicesData.
> >     
> >     Earlier versions of this patch reserved the memory in which the ESRT was
> >     located.  This created awkward alignment problems, and required either
> >     splitting the E820 table or wasting memory.  It also would have required
> >     a new platform op for dom0 to use to indicate if the ESRT is reserved.
> >     By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
> >     does not need to be modified, and dom0 can just check the type of the
> >     memory region containing the ESRT.  The copy is only done if the ESRT is
> >     not already in EfiRuntimeServicesData memory, avoiding memory leaks on
> >     repeated kexec.
> >     
> >     See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
> >     for details.
> >     
> >     Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> >     Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> ... this is the most likely candidate, considering in the log all we
> see is:
> 
> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
> Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
> Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
> Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
> Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
> Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.

This would not surprise me at all.  I was hoping that Jan would be able
to test this before he merged it, especially the ARM-specific stuff.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Jan Beulich]

On 06.07.2022 08:56, Demi Marie Obenour wrote:
>>> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
>>> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
>>> Date:   Tue Jul 5 13:10:46 2022 +0200
>>>
>>>     EFI: preserve the System Resource Table for dom0
>>>     
>>>     The EFI System Resource Table (ESRT) is necessary for fwupd to identify
>>>     firmware updates to install.  According to the UEFI specification §23.4,
>>>     the ESRT shall be stored in memory of type EfiBootServicesData.  However,
>>>     memory of type EfiBootServicesData is considered general-purpose memory
>>>     by Xen, so the ESRT needs to be moved somewhere where Xen will not
>>>     overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
>>>     which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
>>>     in memory of type EfiRuntimeServicesData.
>>>     
>>>     Earlier versions of this patch reserved the memory in which the ESRT was
>>>     located.  This created awkward alignment problems, and required either
>>>     splitting the E820 table or wasting memory.  It also would have required
>>>     a new platform op for dom0 to use to indicate if the ESRT is reserved.
>>>     By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
>>>     does not need to be modified, and dom0 can just check the type of the
>>>     memory region containing the ESRT.  The copy is only done if the ESRT is
>>>     not already in EfiRuntimeServicesData memory, avoiding memory leaks on
>>>     repeated kexec.
>>>     
>>>     See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
>>>     for details.
>>>     
>>>     Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>     Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>
>> ... this is the most likely candidate, considering in the log all we
>> see is:
>>
>> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
>> Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
>> Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
>> Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
>> Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
>> Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.
> 
> This would not surprise me at all.  I was hoping that Jan would be able
> to test this before he merged it, especially the ARM-specific stuff.

Jan (i.e. me)? I've never done any testing on Arm; all I do is build-test
things there. Also if you suspected there might be issues, I think you
should have arranged for someone to test this, i.e. at the very least
indicate so in a post-commit-message remark targeted at the eventual
committer.

Jan

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Demi Marie Obenour]

[-- Attachment #1: Type: text/plain, Size: 3321 bytes --]

On Wed, Jul 06, 2022 at 09:02:47AM +0200, Jan Beulich wrote:
> On 06.07.2022 08:56, Demi Marie Obenour wrote:
> >>> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
> >>> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
> >>> Date:   Tue Jul 5 13:10:46 2022 +0200
> >>>
> >>>     EFI: preserve the System Resource Table for dom0
> >>>     
> >>>     The EFI System Resource Table (ESRT) is necessary for fwupd to identify
> >>>     firmware updates to install.  According to the UEFI specification §23.4,
> >>>     the ESRT shall be stored in memory of type EfiBootServicesData.  However,
> >>>     memory of type EfiBootServicesData is considered general-purpose memory
> >>>     by Xen, so the ESRT needs to be moved somewhere where Xen will not
> >>>     overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
> >>>     which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
> >>>     in memory of type EfiRuntimeServicesData.
> >>>     
> >>>     Earlier versions of this patch reserved the memory in which the ESRT was
> >>>     located.  This created awkward alignment problems, and required either
> >>>     splitting the E820 table or wasting memory.  It also would have required
> >>>     a new platform op for dom0 to use to indicate if the ESRT is reserved.
> >>>     By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
> >>>     does not need to be modified, and dom0 can just check the type of the
> >>>     memory region containing the ESRT.  The copy is only done if the ESRT is
> >>>     not already in EfiRuntimeServicesData memory, avoiding memory leaks on
> >>>     repeated kexec.
> >>>     
> >>>     See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
> >>>     for details.
> >>>     
> >>>     Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> >>>     Reviewed-by: Jan Beulich <jbeulich@suse.com>
> >>
> >> ... this is the most likely candidate, considering in the log all we
> >> see is:
> >>
> >> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
> >> Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
> >> Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
> >> Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
> >> Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
> >> Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.
> > 
> > This would not surprise me at all.  I was hoping that Jan would be able
> > to test this before he merged it, especially the ARM-specific stuff.
> 
> Jan (i.e. me)? I've never done any testing on Arm; all I do is build-test
> things there. Also if you suspected there might be issues, I think you
> should have arranged for someone to test this, i.e. at the very least
> indicate so in a post-commit-message remark targeted at the eventual
> committer.

I don't have access to an ARM64 machine (other than a mobile device)
myself, so I can't test anything on that platform.  I did not *expect*
there to be any issues, but I am not surprised that there are.  I should
have done a basic smoke test on x86, though.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Julien Grall]

Hi Jan,

On 06/07/2022 07:44, Jan Beulich wrote:
> On 06.07.2022 05:39, osstest service owner wrote:
>> flight 171511 xen-unstable-smoke real [real]
>> flight 171517 xen-unstable-smoke real-retest [real]
>> http://logs.test-lab.xenproject.org/osstest/logs/171511/
>> http://logs.test-lab.xenproject.org/osstest/logs/171517/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking,
>> including tests which could not be run:
>>   test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 171486
> 
> Looking at what's under test, I guess ...
> 
>> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
>> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
>> Date:   Tue Jul 5 13:10:46 2022 +0200
>>
>>      EFI: preserve the System Resource Table for dom0
>>      
>>      The EFI System Resource Table (ESRT) is necessary for fwupd to identify
>>      firmware updates to install.  According to the UEFI specification §23.4,
>>      the ESRT shall be stored in memory of type EfiBootServicesData.  However,
>>      memory of type EfiBootServicesData is considered general-purpose memory
>>      by Xen, so the ESRT needs to be moved somewhere where Xen will not
>>      overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
>>      which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
>>      in memory of type EfiRuntimeServicesData.
>>      
>>      Earlier versions of this patch reserved the memory in which the ESRT was
>>      located.  This created awkward alignment problems, and required either
>>      splitting the E820 table or wasting memory.  It also would have required
>>      a new platform op for dom0 to use to indicate if the ESRT is reserved.
>>      By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
>>      does not need to be modified, and dom0 can just check the type of the
>>      memory region containing the ESRT.  The copy is only done if the ESRT is
>>      not already in EfiRuntimeServicesData memory, avoiding memory leaks on
>>      repeated kexec.
>>      
>>      See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
>>      for details.
>>      
>>      Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>>      Reviewed-by: Jan Beulich <jbeulich@suse.com>
> 
> ... this is the most likely candidate, considering in the log all we
> see is:
> 
> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
> Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
> Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
> Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
> Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
> Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.
> 
> But I guess we'll want to wait for the bi-sector to give us a more
> solid indication ...

I have tested a Xen with and without this patch this morning and can 
EFI. I haven't looked into details yet why.

Can we consider to revert it?

Cheers,

-- 
Julien Grall

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Demi Marie Obenour]

[-- Attachment #1: Type: text/plain, Size: 3625 bytes --]

On Wed, Jul 06, 2022 at 08:53:49AM +0100, Julien Grall wrote:
> Hi Jan,
> 
> On 06/07/2022 07:44, Jan Beulich wrote:
> > On 06.07.2022 05:39, osstest service owner wrote:
> > > flight 171511 xen-unstable-smoke real [real]
> > > flight 171517 xen-unstable-smoke real-retest [real]
> > > http://logs.test-lab.xenproject.org/osstest/logs/171511/
> > > http://logs.test-lab.xenproject.org/osstest/logs/171517/
> > > 
> > > Regressions :-(
> > > 
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > >   test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 171486
> > 
> > Looking at what's under test, I guess ...
> > 
> > > commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
> > > Author: Demi Marie Obenour <demi@invisiblethingslab.com>
> > > Date:   Tue Jul 5 13:10:46 2022 +0200
> > > 
> > >      EFI: preserve the System Resource Table for dom0
> > >      The EFI System Resource Table (ESRT) is necessary for fwupd to identify
> > >      firmware updates to install.  According to the UEFI specification §23.4,
> > >      the ESRT shall be stored in memory of type EfiBootServicesData.  However,
> > >      memory of type EfiBootServicesData is considered general-purpose memory
> > >      by Xen, so the ESRT needs to be moved somewhere where Xen will not
> > >      overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
> > >      which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
> > >      in memory of type EfiRuntimeServicesData.
> > >      Earlier versions of this patch reserved the memory in which the ESRT was
> > >      located.  This created awkward alignment problems, and required either
> > >      splitting the E820 table or wasting memory.  It also would have required
> > >      a new platform op for dom0 to use to indicate if the ESRT is reserved.
> > >      By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
> > >      does not need to be modified, and dom0 can just check the type of the
> > >      memory region containing the ESRT.  The copy is only done if the ESRT is
> > >      not already in EfiRuntimeServicesData memory, avoiding memory leaks on
> > >      repeated kexec.
> > >      See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
> > >      for details.
> > >      Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
> > >      Reviewed-by: Jan Beulich <jbeulich@suse.com>
> > 
> > ... this is the most likely candidate, considering in the log all we
> > see is:
> > 
> > Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
> > Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
> > Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
> > Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
> > Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
> > Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.
> > 
> > But I guess we'll want to wait for the bi-sector to give us a more
> > solid indication ...
> 
> I have tested a Xen with and without this patch this morning and can EFI. I
> haven't looked into details yet why.
> 
> Can we consider to revert it?

I'm fine with reverting it for now, but I would like to know what the
bug was.  Does a Xen with this patch boot okay on x86?  If so, could it
be temporarily turned off on ARM until the problem can be tracked down?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Julien Grall]

Hi Demi,

On 06/07/2022 09:05, Demi Marie Obenour wrote:
> On Wed, Jul 06, 2022 at 08:53:49AM +0100, Julien Grall wrote:
>> Hi Jan,
>>
>> On 06/07/2022 07:44, Jan Beulich wrote:
>>> On 06.07.2022 05:39, osstest service owner wrote:
>>>> flight 171511 xen-unstable-smoke real [real]
>>>> flight 171517 xen-unstable-smoke real-retest [real]
>>>> http://logs.test-lab.xenproject.org/osstest/logs/171511/
>>>> http://logs.test-lab.xenproject.org/osstest/logs/171517/
>>>>
>>>> Regressions :-(
>>>>
>>>> Tests which did not succeed and are blocking,
>>>> including tests which could not be run:
>>>>    test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 171486
>>>
>>> Looking at what's under test, I guess ...
>>>
>>>> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
>>>> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>> Date:   Tue Jul 5 13:10:46 2022 +0200
>>>>
>>>>       EFI: preserve the System Resource Table for dom0
>>>>       The EFI System Resource Table (ESRT) is necessary for fwupd to identify
>>>>       firmware updates to install.  According to the UEFI specification §23.4,
>>>>       the ESRT shall be stored in memory of type EfiBootServicesData.  However,
>>>>       memory of type EfiBootServicesData is considered general-purpose memory
>>>>       by Xen, so the ESRT needs to be moved somewhere where Xen will not
>>>>       overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
>>>>       which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
>>>>       in memory of type EfiRuntimeServicesData.
>>>>       Earlier versions of this patch reserved the memory in which the ESRT was
>>>>       located.  This created awkward alignment problems, and required either
>>>>       splitting the E820 table or wasting memory.  It also would have required
>>>>       a new platform op for dom0 to use to indicate if the ESRT is reserved.
>>>>       By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
>>>>       does not need to be modified, and dom0 can just check the type of the
>>>>       memory region containing the ESRT.  The copy is only done if the ESRT is
>>>>       not already in EfiRuntimeServicesData memory, avoiding memory leaks on
>>>>       repeated kexec.
>>>>       See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
>>>>       for details.
>>>>       Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>>       Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>>
>>> ... this is the most likely candidate, considering in the log all we
>>> see is:
>>>
>>> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
>>> Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
>>> Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
>>> Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
>>> Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
>>> Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.
>>>
>>> But I guess we'll want to wait for the bi-sector to give us a more
>>> solid indication ...
>>
>> I have tested a Xen with and without this patch this morning and can EFI. I
>> haven't looked into details yet why.
>>
>> Can we consider to revert it?
> 
> I'm fine with reverting it for now, but I would like to know what the
> bug was.  Does a Xen with this patch boot okay on x86?

I haven't tried and I don't know whether we have UEFI system on x86.

>  If so, could it
> be temporarily turned off on ARM until the problem can be tracked down?

I am not in favor of this approach. There are no reason for this to be 
x86-only aside there is a bug in the code.

AFAICT, this is always an issue on Arm (both QEMU and Softiron fails to 
boot). It still not clear whether it might fail on some x86 systems. So 
we first need to figure out what's happening.

I am planning to spend some time on it today (as a low priority).

Cheers,

-- 
Julien Grall

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Jan Beulich]

On 06.07.2022 10:17, Julien Grall wrote:
> Hi Demi,
> 
> On 06/07/2022 09:05, Demi Marie Obenour wrote:
>> On Wed, Jul 06, 2022 at 08:53:49AM +0100, Julien Grall wrote:
>>> Hi Jan,
>>>
>>> On 06/07/2022 07:44, Jan Beulich wrote:
>>>> On 06.07.2022 05:39, osstest service owner wrote:
>>>>> flight 171511 xen-unstable-smoke real [real]
>>>>> flight 171517 xen-unstable-smoke real-retest [real]
>>>>> http://logs.test-lab.xenproject.org/osstest/logs/171511/
>>>>> http://logs.test-lab.xenproject.org/osstest/logs/171517/
>>>>>
>>>>> Regressions :-(
>>>>>
>>>>> Tests which did not succeed and are blocking,
>>>>> including tests which could not be run:
>>>>>    test-arm64-arm64-xl-xsm       8 xen-boot                 fail REGR. vs. 171486
>>>>
>>>> Looking at what's under test, I guess ...
>>>>
>>>>> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
>>>>> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>>> Date:   Tue Jul 5 13:10:46 2022 +0200
>>>>>
>>>>>       EFI: preserve the System Resource Table for dom0
>>>>>       The EFI System Resource Table (ESRT) is necessary for fwupd to identify
>>>>>       firmware updates to install.  According to the UEFI specification §23.4,
>>>>>       the ESRT shall be stored in memory of type EfiBootServicesData.  However,
>>>>>       memory of type EfiBootServicesData is considered general-purpose memory
>>>>>       by Xen, so the ESRT needs to be moved somewhere where Xen will not
>>>>>       overwrite it.  Copy the ESRT to memory of type EfiRuntimeServicesData,
>>>>>       which Xen will not reuse.  dom0 can use the ESRT if (and only if) it is
>>>>>       in memory of type EfiRuntimeServicesData.
>>>>>       Earlier versions of this patch reserved the memory in which the ESRT was
>>>>>       located.  This created awkward alignment problems, and required either
>>>>>       splitting the E820 table or wasting memory.  It also would have required
>>>>>       a new platform op for dom0 to use to indicate if the ESRT is reserved.
>>>>>       By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
>>>>>       does not need to be modified, and dom0 can just check the type of the
>>>>>       memory region containing the ESRT.  The copy is only done if the ESRT is
>>>>>       not already in EfiRuntimeServicesData memory, avoiding memory leaks on
>>>>>       repeated kexec.
>>>>>       See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
>>>>>       for details.
>>>>>       Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>>>       Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>>>
>>>> ... this is the most likely candidate, considering in the log all we
>>>> see is:
>>>>
>>>> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
>>>> Jul  5 23:09:15.692859 Using configuration file 'xen.cfg'
>>>> Jul  5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
>>>> Jul  5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
>>>> Jul  5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
>>>> Jul  5 23:09:15.980866 Using bootargs from Xen configuration file.
>>>>
>>>> But I guess we'll want to wait for the bi-sector to give us a more
>>>> solid indication ...
>>>
>>> I have tested a Xen with and without this patch this morning and can EFI. I
>>> haven't looked into details yet why.
>>>
>>> Can we consider to revert it?
>>
>> I'm fine with reverting it for now, but I would like to know what the
>> bug was.  Does a Xen with this patch boot okay on x86?
> 
> I haven't tried and I don't know whether we have UEFI system on x86.
> 
>>  If so, could it
>> be temporarily turned off on ARM until the problem can be tracked down?
> 
> I am not in favor of this approach. There are no reason for this to be 
> x86-only aside there is a bug in the code.

I agree - either we keep the patch in anticipation of a soon-ish fix,
or we revert it.

Jan

Re: [xen-unstable-smoke test] 171511: regressions - FAIL

[Luca Fancellu]

> On 6 Jul 2022, at 09:05, Demi Marie Obenour <demi@invisiblethingslab.com> wrote:
> 
> On Wed, Jul 06, 2022 at 08:53:49AM +0100, Julien Grall wrote:
>> Hi Jan,
>> 
>> On 06/07/2022 07:44, Jan Beulich wrote:
>>> On 06.07.2022 05:39, osstest service owner wrote:
>>>> flight 171511 xen-unstable-smoke real [real]
>>>> flight 171517 xen-unstable-smoke real-retest [real]
>>>> http://logs.test-lab.xenproject.org/osstest/logs/171511/
>>>> http://logs.test-lab.xenproject.org/osstest/logs/171517/
>>>> 
>>>> Regressions :-(
>>>> 
>>>> Tests which did not succeed and are blocking,
>>>> including tests which could not be run:
>>>> test-arm64-arm64-xl-xsm 8 xen-boot fail REGR. vs. 171486
>>> 
>>> Looking at what's under test, I guess ...
>>> 
>>>> commit 8d410ac2c178e1dd1001cadddbe9ca75a9738c95
>>>> Author: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>> Date: Tue Jul 5 13:10:46 2022 +0200
>>>> 
>>>> EFI: preserve the System Resource Table for dom0
>>>> The EFI System Resource Table (ESRT) is necessary for fwupd to identify
>>>> firmware updates to install. According to the UEFI specification §23.4,
>>>> the ESRT shall be stored in memory of type EfiBootServicesData. However,
>>>> memory of type EfiBootServicesData is considered general-purpose memory
>>>> by Xen, so the ESRT needs to be moved somewhere where Xen will not
>>>> overwrite it. Copy the ESRT to memory of type EfiRuntimeServicesData,
>>>> which Xen will not reuse. dom0 can use the ESRT if (and only if) it is
>>>> in memory of type EfiRuntimeServicesData.
>>>> Earlier versions of this patch reserved the memory in which the ESRT was
>>>> located. This created awkward alignment problems, and required either
>>>> splitting the E820 table or wasting memory. It also would have required
>>>> a new platform op for dom0 to use to indicate if the ESRT is reserved.
>>>> By copying the ESRT into EfiRuntimeServicesData memory, the E820 table
>>>> does not need to be modified, and dom0 can just check the type of the
>>>> memory region containing the ESRT. The copy is only done if the ESRT is
>>>> not already in EfiRuntimeServicesData memory, avoiding memory leaks on
>>>> repeated kexec.
>>>> See https://lore.kernel.org/xen-devel/20200818184018.GN1679@mail-itl/T/
>>>> for details.
>>>> Signed-off-by: Demi Marie Obenour <demi@invisiblethingslab.com>
>>>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>> 
>>> ... this is the most likely candidate, considering in the log all we
>>> see is:
>>> 
>>> Xen 4.17-unstable (c/s Mon Jun 27 15:15:39 2022 +0200 git:61ff273322-dirty) EFI loader
>>> Jul 5 23:09:15.692859 Using configuration file 'xen.cfg'
>>> Jul 5 23:09:15.704878 vmlinuz: 0x00000083fb1ac000-0x00000083fc880a00
>>> Jul 5 23:09:15.704931 initrd.gz: 0x00000083f94b7000-0x00000083fb1ab6e8
>>> Jul 5 23:09:15.836836 xenpolicy: 0x00000083f94b4000-0x00000083f94b6a5f
>>> Jul 5 23:09:15.980866 Using bootargs from Xen configuration file.
>>> 
>>> But I guess we'll want to wait for the bi-sector to give us a more
>>> solid indication ...
>> 
>> I have tested a Xen with and without this patch this morning and can EFI. I
>> haven't looked into details yet why.
>> 
>> Can we consider to revert it?
> 
> I'm fine with reverting it for now, but I would like to know what the
> bug was. Does a Xen with this patch boot okay on x86? If so, could it
> be temporarily turned off on ARM until the problem can be tracked down?

I can test it with an arm64 machine, I will try it now, will let you know.

Cheers,
Luca

> -- 
> Sincerely,
> Demi Marie Obenour (she/her/hers)
> Invisible Things Lab