[PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Bin Meng]

From: Bin Meng <bin.meng@windriver.com>

The following error message was seen during the configure:

  "ln: failed to create symbolic link
  'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"

By default the MSYS environment variable is not defined, so the runtime
behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
At the configure phase, the qemu-system-x86_64.exe has not been built
so creation of the symbolic link fails hence the error message.

Set winsymlinks to 'native' whose behavior is most similar to the
behavior of 'ln -s' on *nix, that is:

  a) if native symlinks are enabled, and whether <target> exists
     or not, creates <destination> as a native Windows symlink;
  b) else if native symlinks are not enabled, and whether <target>
     exists or not, 'ln -s' creates as a Windows shortcut file.

Signed-off-by: Bin Meng <bin.meng@windriver.com>
---

 .gitlab-ci.d/windows.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 1b2ede49e1..0b9572a8a3 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -57,6 +57,7 @@ msys2-64bit:
       mingw-w64-x86_64-zstd "
   - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
   - $env:MSYSTEM = 'MINGW64'     # Start a 64 bit Mingw environment
+  - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
       --enable-capstone --without-default-devices'
   - .\msys64\usr\bin\bash -lc "sed -i '/^ROMS=/d' build/config-host.mak"
@@ -89,6 +90,7 @@ msys2-32bit:
       mingw-w64-i686-usbredir "
   - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
   - $env:MSYSTEM = 'MINGW32'     # Start a 32-bit MinG environment
+  - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - mkdir output
   - cd output
   - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"
-- 
2.34.1

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Alex Bennée]

Bin Meng <bmeng.cn@gmail.com> writes:

> From: Bin Meng <bin.meng@windriver.com>
>
> The following error message was seen during the configure:
>
>   "ln: failed to create symbolic link
>   'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
>
> By default the MSYS environment variable is not defined, so the runtime
> behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
> At the configure phase, the qemu-system-x86_64.exe has not been built
> so creation of the symbolic link fails hence the error message.
>
> Set winsymlinks to 'native' whose behavior is most similar to the
> behavior of 'ln -s' on *nix, that is:
>
>   a) if native symlinks are enabled, and whether <target> exists
>      or not, creates <destination> as a native Windows symlink;
>   b) else if native symlinks are not enabled, and whether <target>
>      exists or not, 'ln -s' creates as a Windows shortcut file.
>
> Signed-off-by: Bin Meng <bin.meng@windriver.com>

I'm still seeing Windows build failures such as:

  https://gitlab.com/stsquad/qemu/-/jobs/2765579269

and

  https://gitlab.com/stsquad/qemu/-/jobs/2765579267

Any idea what's falling over?

> ---
>
>  .gitlab-ci.d/windows.yml | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> index 1b2ede49e1..0b9572a8a3 100644
> --- a/.gitlab-ci.d/windows.yml
> +++ b/.gitlab-ci.d/windows.yml
> @@ -57,6 +57,7 @@ msys2-64bit:
>        mingw-w64-x86_64-zstd "
>    - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
>    - $env:MSYSTEM = 'MINGW64'     # Start a 64 bit Mingw environment
> +  - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
>    - .\msys64\usr\bin\bash -lc './configure --target-list=x86_64-softmmu
>        --enable-capstone --without-default-devices'
>    - .\msys64\usr\bin\bash -lc "sed -i '/^ROMS=/d' build/config-host.mak"
> @@ -89,6 +90,7 @@ msys2-32bit:
>        mingw-w64-i686-usbredir "
>    - $env:CHERE_INVOKING = 'yes'  # Preserve the current working directory
>    - $env:MSYSTEM = 'MINGW32'     # Start a 32-bit MinG environment
> +  - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
>    - mkdir output
>    - cd output
>    - ..\msys64\usr\bin\bash -lc "../configure --target-list=ppc64-softmmu"


-- 
Alex Bennée

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Thomas Huth]

On 25/07/2022 15.47, Alex Bennée wrote:
> 
> Bin Meng <bmeng.cn@gmail.com> writes:
> 
>> From: Bin Meng <bin.meng@windriver.com>
>>
>> The following error message was seen during the configure:
>>
>>    "ln: failed to create symbolic link
>>    'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
>>
>> By default the MSYS environment variable is not defined, so the runtime
>> behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
>> At the configure phase, the qemu-system-x86_64.exe has not been built
>> so creation of the symbolic link fails hence the error message.
>>
>> Set winsymlinks to 'native' whose behavior is most similar to the
>> behavior of 'ln -s' on *nix, that is:
>>
>>    a) if native symlinks are enabled, and whether <target> exists
>>       or not, creates <destination> as a native Windows symlink;
>>    b) else if native symlinks are not enabled, and whether <target>
>>       exists or not, 'ln -s' creates as a Windows shortcut file.
>>
>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> 
> I'm still seeing Windows build failures such as:
> 
>    https://gitlab.com/stsquad/qemu/-/jobs/2765579269
> 
> and
> 
>    https://gitlab.com/stsquad/qemu/-/jobs/2765579267
> 
> Any idea what's falling over?

No clue, but FWIW, I had the same problem in a run from last Friday here 
(without this symlink patch):

  https://gitlab.com/thuth/qemu/-/jobs/2758244223#L2817

I've never seen this failure before - so I guess it's rather something new?

  Thomas

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Bin Meng]

On Mon, Jul 25, 2022 at 9:48 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>
>
> Bin Meng <bmeng.cn@gmail.com> writes:
>
> > From: Bin Meng <bin.meng@windriver.com>
> >
> > The following error message was seen during the configure:
> >
> >   "ln: failed to create symbolic link
> >   'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
> >
> > By default the MSYS environment variable is not defined, so the runtime
> > behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
> > At the configure phase, the qemu-system-x86_64.exe has not been built
> > so creation of the symbolic link fails hence the error message.
> >
> > Set winsymlinks to 'native' whose behavior is most similar to the
> > behavior of 'ln -s' on *nix, that is:
> >
> >   a) if native symlinks are enabled, and whether <target> exists
> >      or not, creates <destination> as a native Windows symlink;
> >   b) else if native symlinks are not enabled, and whether <target>
> >      exists or not, 'ln -s' creates as a Windows shortcut file.
> >
> > Signed-off-by: Bin Meng <bin.meng@windriver.com>
>
> I'm still seeing Windows build failures such as:
>
>   https://gitlab.com/stsquad/qemu/-/jobs/2765579269

I've seen this one before. Looks like this one can be easily reproduced.

>
> and
>
>   https://gitlab.com/stsquad/qemu/-/jobs/2765579267

This one seems to be a random failure?

>
> Any idea what's falling over?
>

Regards,
Bin

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Bin Meng]

On Tue, Jul 26, 2022 at 9:38 AM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> On Mon, Jul 25, 2022 at 9:48 PM Alex Bennée <alex.bennee@linaro.org> wrote:
> >
> >
> > Bin Meng <bmeng.cn@gmail.com> writes:
> >
> > > From: Bin Meng <bin.meng@windriver.com>
> > >
> > > The following error message was seen during the configure:
> > >
> > >   "ln: failed to create symbolic link
> > >   'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
> > >
> > > By default the MSYS environment variable is not defined, so the runtime
> > > behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
> > > At the configure phase, the qemu-system-x86_64.exe has not been built
> > > so creation of the symbolic link fails hence the error message.
> > >
> > > Set winsymlinks to 'native' whose behavior is most similar to the
> > > behavior of 'ln -s' on *nix, that is:
> > >
> > >   a) if native symlinks are enabled, and whether <target> exists
> > >      or not, creates <destination> as a native Windows symlink;
> > >   b) else if native symlinks are not enabled, and whether <target>
> > >      exists or not, 'ln -s' creates as a Windows shortcut file.
> > >
> > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> >
> > I'm still seeing Windows build failures such as:
> >
> >   https://gitlab.com/stsquad/qemu/-/jobs/2765579269
>
> I've seen this one before. Looks like this one can be easily reproduced.
>
> >
> > and
> >
> >   https://gitlab.com/stsquad/qemu/-/jobs/2765579267
>
> This one seems to be a random failure?
>

Saw another random failure today in the msys2-32bit build in CI.

[313/1788] Generating texture-blit-vert.h with a custom command
(wrapped by meson to capture output)
FAILED: ui/shader/texture-blit-vert.h
"C:/GitLab-Runner/builds/lbmeng/qemu/msys64/mingw32/bin/python3.exe"
"C:/GitLab-Runner/builds/lbmeng/qemu/meson/meson.py" "--internal"
"exe" "--capture" "ui/shader/texture-blit-vert.h" "--" "perl"
"C:/GitLab-Runner/builds/lbmeng/qemu/scripts/shaderinclude.pl"
"../ui/shader/texture-blit.vert"
[314/1788] Generating texture-blit-flip-vert.h with a custom command
(wrapped by meson to capture output)
ninja: build stopped: subcommand failed.
make: *** [Makefile:162: run-ninja] Error 1

Regards,
Bin

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Daniel P. Berrangé]

On Wed, Jul 27, 2022 at 02:02:48PM +0800, Bin Meng wrote:
> On Tue, Jul 26, 2022 at 9:38 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> >
> > On Mon, Jul 25, 2022 at 9:48 PM Alex Bennée <alex.bennee@linaro.org> wrote:
> > >
> > >
> > > Bin Meng <bmeng.cn@gmail.com> writes:
> > >
> > > > From: Bin Meng <bin.meng@windriver.com>
> > > >
> > > > The following error message was seen during the configure:
> > > >
> > > >   "ln: failed to create symbolic link
> > > >   'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
> > > >
> > > > By default the MSYS environment variable is not defined, so the runtime
> > > > behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
> > > > At the configure phase, the qemu-system-x86_64.exe has not been built
> > > > so creation of the symbolic link fails hence the error message.
> > > >
> > > > Set winsymlinks to 'native' whose behavior is most similar to the
> > > > behavior of 'ln -s' on *nix, that is:
> > > >
> > > >   a) if native symlinks are enabled, and whether <target> exists
> > > >      or not, creates <destination> as a native Windows symlink;
> > > >   b) else if native symlinks are not enabled, and whether <target>
> > > >      exists or not, 'ln -s' creates as a Windows shortcut file.
> > > >
> > > > Signed-off-by: Bin Meng <bin.meng@windriver.com>
> > >
> > > I'm still seeing Windows build failures such as:
> > >
> > >   https://gitlab.com/stsquad/qemu/-/jobs/2765579269
> >
> > I've seen this one before. Looks like this one can be easily reproduced.
> >
> > >
> > > and
> > >
> > >   https://gitlab.com/stsquad/qemu/-/jobs/2765579267
> >
> > This one seems to be a random failure?
> >
> 
> Saw another random failure today in the msys2-32bit build in CI.
> 
> [313/1788] Generating texture-blit-vert.h with a custom command
> (wrapped by meson to capture output)
> FAILED: ui/shader/texture-blit-vert.h
> "C:/GitLab-Runner/builds/lbmeng/qemu/msys64/mingw32/bin/python3.exe"
> "C:/GitLab-Runner/builds/lbmeng/qemu/meson/meson.py" "--internal"
> "exe" "--capture" "ui/shader/texture-blit-vert.h" "--" "perl"
> "C:/GitLab-Runner/builds/lbmeng/qemu/scripts/shaderinclude.pl"
> "../ui/shader/texture-blit.vert"
> [314/1788] Generating texture-blit-flip-vert.h with a custom command
> (wrapped by meson to capture output)
> ninja: build stopped: subcommand failed.
> make: *** [Makefile:162: run-ninja] Error 1

IMHO we need to just drop the msys jobs from GitLab. They are too
unreliable and causing frequent failed pipelines. 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Thomas Huth]

On 27/07/2022 10.54, Daniel P. Berrangé wrote:
> On Wed, Jul 27, 2022 at 02:02:48PM +0800, Bin Meng wrote:
>> On Tue, Jul 26, 2022 at 9:38 AM Bin Meng <bmeng.cn@gmail.com> wrote:
>>>
>>> On Mon, Jul 25, 2022 at 9:48 PM Alex Bennée <alex.bennee@linaro.org> wrote:
>>>>
>>>>
>>>> Bin Meng <bmeng.cn@gmail.com> writes:
>>>>
>>>>> From: Bin Meng <bin.meng@windriver.com>
>>>>>
>>>>> The following error message was seen during the configure:
>>>>>
>>>>>    "ln: failed to create symbolic link
>>>>>    'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
>>>>>
>>>>> By default the MSYS environment variable is not defined, so the runtime
>>>>> behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
>>>>> At the configure phase, the qemu-system-x86_64.exe has not been built
>>>>> so creation of the symbolic link fails hence the error message.
>>>>>
>>>>> Set winsymlinks to 'native' whose behavior is most similar to the
>>>>> behavior of 'ln -s' on *nix, that is:
>>>>>
>>>>>    a) if native symlinks are enabled, and whether <target> exists
>>>>>       or not, creates <destination> as a native Windows symlink;
>>>>>    b) else if native symlinks are not enabled, and whether <target>
>>>>>       exists or not, 'ln -s' creates as a Windows shortcut file.
>>>>>
>>>>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
>>>>
>>>> I'm still seeing Windows build failures such as:
>>>>
>>>>    https://gitlab.com/stsquad/qemu/-/jobs/2765579269
>>>
>>> I've seen this one before. Looks like this one can be easily reproduced.
>>>
>>>>
>>>> and
>>>>
>>>>    https://gitlab.com/stsquad/qemu/-/jobs/2765579267
>>>
>>> This one seems to be a random failure?
>>>
>>
>> Saw another random failure today in the msys2-32bit build in CI.
>>
>> [313/1788] Generating texture-blit-vert.h with a custom command
>> (wrapped by meson to capture output)
>> FAILED: ui/shader/texture-blit-vert.h
>> "C:/GitLab-Runner/builds/lbmeng/qemu/msys64/mingw32/bin/python3.exe"
>> "C:/GitLab-Runner/builds/lbmeng/qemu/meson/meson.py" "--internal"
>> "exe" "--capture" "ui/shader/texture-blit-vert.h" "--" "perl"
>> "C:/GitLab-Runner/builds/lbmeng/qemu/scripts/shaderinclude.pl"
>> "../ui/shader/texture-blit.vert"
>> [314/1788] Generating texture-blit-flip-vert.h with a custom command
>> (wrapped by meson to capture output)
>> ninja: build stopped: subcommand failed.
>> make: *** [Makefile:162: run-ninja] Error 1
> 
> IMHO we need to just drop the msys jobs from GitLab. They are too
> unreliable and causing frequent failed pipelines.

IIRC they were working way more reliable 'till two or three months ago? 
Maybe this was introduced by the bump to the newest level of MSYS2 in commit 
5c570ef2f154 ? Maybe we should upgrade or downgrade the MSYS2 version again?

  Thomas

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[罗勇刚(Yonggang Luo)]

[-- Attachment #1: Type: text/plain, Size: 3215 bytes --]

I've seen the cirrus ci always succeed, maybe using cirrus instead?

On Wed, Jul 27, 2022 at 5:03 PM Thomas Huth <thuth@redhat.com> wrote:
>
> On 27/07/2022 10.54, Daniel P. Berrangé wrote:
> > On Wed, Jul 27, 2022 at 02:02:48PM +0800, Bin Meng wrote:
> >> On Tue, Jul 26, 2022 at 9:38 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> >>>
> >>> On Mon, Jul 25, 2022 at 9:48 PM Alex Bennée <alex.bennee@linaro.org>
wrote:
> >>>>
> >>>>
> >>>> Bin Meng <bmeng.cn@gmail.com> writes:
> >>>>
> >>>>> From: Bin Meng <bin.meng@windriver.com>
> >>>>>
> >>>>> The following error message was seen during the configure:
> >>>>>
> >>>>>    "ln: failed to create symbolic link
> >>>>>    'x86_64-softmmu/qemu-system-x86_64.exe': No such file or
directory"
> >>>>>
> >>>>> By default the MSYS environment variable is not defined, so the
runtime
> >>>>> behavior of winsymlinks is: if <target> does not exist, 'ln -s'
fails.
> >>>>> At the configure phase, the qemu-system-x86_64.exe has not been
built
> >>>>> so creation of the symbolic link fails hence the error message.
> >>>>>
> >>>>> Set winsymlinks to 'native' whose behavior is most similar to the
> >>>>> behavior of 'ln -s' on *nix, that is:
> >>>>>
> >>>>>    a) if native symlinks are enabled, and whether <target> exists
> >>>>>       or not, creates <destination> as a native Windows symlink;
> >>>>>    b) else if native symlinks are not enabled, and whether <target>
> >>>>>       exists or not, 'ln -s' creates as a Windows shortcut file.
> >>>>>
> >>>>> Signed-off-by: Bin Meng <bin.meng@windriver.com>
> >>>>
> >>>> I'm still seeing Windows build failures such as:
> >>>>
> >>>>    https://gitlab.com/stsquad/qemu/-/jobs/2765579269
> >>>
> >>> I've seen this one before. Looks like this one can be easily
reproduced.
> >>>
> >>>>
> >>>> and
> >>>>
> >>>>    https://gitlab.com/stsquad/qemu/-/jobs/2765579267
> >>>
> >>> This one seems to be a random failure?
> >>>
> >>
> >> Saw another random failure today in the msys2-32bit build in CI.
> >>
> >> [313/1788] Generating texture-blit-vert.h with a custom command
> >> (wrapped by meson to capture output)
> >> FAILED: ui/shader/texture-blit-vert.h
> >> "C:/GitLab-Runner/builds/lbmeng/qemu/msys64/mingw32/bin/python3.exe"
> >> "C:/GitLab-Runner/builds/lbmeng/qemu/meson/meson.py" "--internal"
> >> "exe" "--capture" "ui/shader/texture-blit-vert.h" "--" "perl"
> >> "C:/GitLab-Runner/builds/lbmeng/qemu/scripts/shaderinclude.pl"
> >> "../ui/shader/texture-blit.vert"
> >> [314/1788] Generating texture-blit-flip-vert.h with a custom command
> >> (wrapped by meson to capture output)
> >> ninja: build stopped: subcommand failed.
> >> make: *** [Makefile:162: run-ninja] Error 1
> >
> > IMHO we need to just drop the msys jobs from GitLab. They are too
> > unreliable and causing frequent failed pipelines.
>
> IIRC they were working way more reliable 'till two or three months ago?
> Maybe this was introduced by the bump to the newest level of MSYS2 in
commit
> 5c570ef2f154 ? Maybe we should upgrade or downgrade the MSYS2 version
again?
>
>   Thomas
>


--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

[-- Attachment #2: Type: text/html, Size: 4907 bytes --]

Re: [PATCH] .gitlab-ci.d/windows.yml: Enable native Windows symlink

[Thomas Huth]

On 27/07/2022 11.11, 罗勇刚(Yonggang Luo) wrote:
> I've seen the cirrus ci always succeed, maybe using cirrus instead?

IIRC the free tier of Cirrus only allows two jobs in parallel, and we're 
using that for FreeBSD and macOS already - we could maybe add another manual 
job like we do it for NetBSD and OpenBSD already, but that means that the 
MSYS2 build won't run by default anymore.

  Thomas