[PATCH 0/2] Fix win32/msys2 shader compilation

[PATCH 0/2] Fix win32/msys2 shader compilation

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

Hi,

Fix the shader compilation error on win32/msys2 and convert the related script
from perl to python.

Marc-André Lureau (2):
  build-sys: fix crlf-ending C code
  .gitlab-ci.d/windows: do not disable opengl

 meson.build              |  2 +-
 .gitlab-ci.d/windows.yml |  5 ++---
 scripts/shaderinclude.pl | 16 ----------------
 scripts/shaderinclude.py | 22 ++++++++++++++++++++++
 4 files changed, 25 insertions(+), 20 deletions(-)
 delete mode 100644 scripts/shaderinclude.pl
 create mode 100755 scripts/shaderinclude.py

-- 
2.39.0

[PATCH 1/2] build-sys: fix crlf-ending C code

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

On msys2, the shader-to-C script produces bad C:
./ui/shader/texture-blit-vert.h:2:5: error: missing terminating " character [-Werror]

Fix it by changing the line ending from crlf to lf, and convert the
script to Python (qemu build seems perl-free after that).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 meson.build              |  2 +-
 scripts/shaderinclude.pl | 16 ----------------
 scripts/shaderinclude.py | 22 ++++++++++++++++++++++
 3 files changed, 23 insertions(+), 17 deletions(-)
 delete mode 100644 scripts/shaderinclude.pl
 create mode 100755 scripts/shaderinclude.py

diff --git a/meson.build b/meson.build
index 175517eafd..b3c6db8343 100644
--- a/meson.build
+++ b/meson.build
@@ -2781,7 +2781,7 @@ config_host_data.set('CONFIG_SLIRP', slirp.found())
 genh += configure_file(output: 'config-host.h', configuration: config_host_data)
 
 hxtool = find_program('scripts/hxtool')
-shaderinclude = find_program('scripts/shaderinclude.pl')
+shaderinclude = find_program('scripts/shaderinclude.py')
 qapi_gen = find_program('scripts/qapi-gen.py')
 qapi_gen_depends = [ meson.current_source_dir() / 'scripts/qapi/__init__.py',
                      meson.current_source_dir() / 'scripts/qapi/commands.py',
diff --git a/scripts/shaderinclude.pl b/scripts/shaderinclude.pl
deleted file mode 100644
index cd3bb40b12..0000000000
--- a/scripts/shaderinclude.pl
+++ /dev/null
@@ -1,16 +0,0 @@
-#!/usr/bin/env perl
-use strict;
-use warnings;
-
-my $file = shift;
-open FILE, "<", $file or die "open $file: $!";
-my $name = $file;
-$name =~ s|.*/||;
-$name =~ s/[-.]/_/g;
-print "static GLchar ${name}_src[] =\n";
-while (<FILE>) {
-    chomp;
-    printf "    \"%s\\n\"\n", $_;
-}
-print "    \"\\n\";\n";
-close FILE;
diff --git a/scripts/shaderinclude.py b/scripts/shaderinclude.py
new file mode 100755
index 0000000000..c314b7ac63
--- /dev/null
+++ b/scripts/shaderinclude.py
@@ -0,0 +1,22 @@
+#!/usr/bin/env python3
+
+import sys
+import os
+
+
+def main(args):
+    file_path = args[1]
+    basename = os.path.basename(file_path)
+    varname = basename.replace('-', '_').replace('.', '_')
+
+    with os.fdopen(sys.stdout.fileno(), "wt", closefd=False, newline='\n') as stdout:
+        with open(file_path, "r", encoding='utf-8') as file:
+            print(f'static GLchar {varname}_src[] =', file=stdout)
+            for line in file:
+                line = line.rstrip()
+                print(f'    "{line}\\n"', file=stdout)
+            print('    "\\n";', file=stdout)
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv))
-- 
2.39.0

Re: [PATCH 1/2] build-sys: fix crlf-ending C code

[Thomas Huth]

On 09/01/2023 12.21, marcandre.lureau@redhat.com wrote:
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> On msys2, the shader-to-C script produces bad C:
> ./ui/shader/texture-blit-vert.h:2:5: error: missing terminating " character [-Werror]
> 
> Fix it by changing the line ending from crlf to lf, and convert the
> script to Python (qemu build seems perl-free after that).

If the build process does not depend on Perl anymore, would it make sense to 
also add a patch that removes perl from most of the containers (the ones 
that don't use check_patch.pl etc.)? ... that way we would make sure that 
the dependency does not creep in again later...

> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>   meson.build              |  2 +-
>   scripts/shaderinclude.pl | 16 ----------------
>   scripts/shaderinclude.py | 22 ++++++++++++++++++++++
>   3 files changed, 23 insertions(+), 17 deletions(-)
>   delete mode 100644 scripts/shaderinclude.pl
>   create mode 100755 scripts/shaderinclude.py

Acked-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 1/2] build-sys: fix crlf-ending C code

[Marc-André Lureau]

[-- Attachment #1: Type: text/plain, Size: 1343 bytes --]

Hi

On Mon, Jan 9, 2023 at 3:41 PM Thomas Huth <thuth@redhat.com> wrote:

> On 09/01/2023 12.21, marcandre.lureau@redhat.com wrote:
> > From: Marc-André Lureau <marcandre.lureau@redhat.com>
> >
> > On msys2, the shader-to-C script produces bad C:
> > ./ui/shader/texture-blit-vert.h:2:5: error: missing terminating "
> character [-Werror]
> >
> > Fix it by changing the line ending from crlf to lf, and convert the
> > script to Python (qemu build seems perl-free after that).
>
> If the build process does not depend on Perl anymore, would it make sense
> to
> also add a patch that removes perl from most of the containers (the ones
> that don't use check_patch.pl etc.)? ... that way we would make sure that
> the dependency does not creep in again later...
>

Let's try that. A quick check reveals that configure already still has perl
usage. I will take a look.


>
> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > ---
> >   meson.build              |  2 +-
> >   scripts/shaderinclude.pl | 16 ----------------
> >   scripts/shaderinclude.py | 22 ++++++++++++++++++++++
> >   3 files changed, 23 insertions(+), 17 deletions(-)
> >   delete mode 100644 scripts/shaderinclude.pl
> >   create mode 100755 scripts/shaderinclude.py
>
> Acked-by: Thomas Huth <thuth@redhat.com>
>
>

[-- Attachment #2: Type: text/html, Size: 2411 bytes --]

Re: [PATCH 1/2] build-sys: fix crlf-ending C code

[Paolo Bonzini]

On 1/9/23 12:47, Marc-André Lureau wrote:
> Let's try that. A quick check reveals that configure already still has 
> perl usage. I will take a look.

There's already a patch planned to remove it ("configure: remove 
backwards-compatibility code").

Paolo

Re: [PATCH 1/2] build-sys: fix crlf-ending C code

[Marc-André Lureau]

Hi

On Mon, Jan 9, 2023 at 7:57 PM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 1/9/23 12:47, Marc-André Lureau wrote:
> > Let's try that. A quick check reveals that configure already still has
> > perl usage. I will take a look.
>
> There's already a patch planned to remove it ("configure: remove
> backwards-compatibility code").

Ok, in the meantime, I replaced it with sed :)
The remaining one seems to be some sphinx workaround.

We'll need to update lcitool, my test CI branch is currently running from:

https://gitlab.com/marcandre.lureau/qemu/-/commits/test


-- 
Marc-André Lureau

Re: [PATCH 1/2] build-sys: fix crlf-ending C code

[Philippe Mathieu-Daudé]

On 9/1/23 12:21, marcandre.lureau@redhat.com wrote:
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> On msys2, the shader-to-C script produces bad C:
> ./ui/shader/texture-blit-vert.h:2:5: error: missing terminating " character [-Werror]
> 
> Fix it by changing the line ending from crlf to lf, and convert the
> script to Python (qemu build seems perl-free after that).
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>   meson.build              |  2 +-
>   scripts/shaderinclude.pl | 16 ----------------
>   scripts/shaderinclude.py | 22 ++++++++++++++++++++++
>   3 files changed, 23 insertions(+), 17 deletions(-)
>   delete mode 100644 scripts/shaderinclude.pl
>   create mode 100755 scripts/shaderinclude.py


> diff --git a/scripts/shaderinclude.py b/scripts/shaderinclude.py
> new file mode 100755
> index 0000000000..c314b7ac63
> --- /dev/null
> +++ b/scripts/shaderinclude.py
> @@ -0,0 +1,22 @@

Missing license boilerplate.

> +#!/usr/bin/env python3
> +
> +import sys
> +import os
> +
> +
> +def main(args):
> +    file_path = args[1]
> +    basename = os.path.basename(file_path)
> +    varname = basename.replace('-', '_').replace('.', '_')
> +
> +    with os.fdopen(sys.stdout.fileno(), "wt", closefd=False, newline='\n') as stdout:
> +        with open(file_path, "r", encoding='utf-8') as file:
> +            print(f'static GLchar {varname}_src[] =', file=stdout)
> +            for line in file:
> +                line = line.rstrip()
> +                print(f'    "{line}\\n"', file=stdout)
> +            print('    "\\n";', file=stdout)

[PATCH 2/2] .gitlab-ci.d/windows: do not disable opengl

[marcandre.lureau]

From: Marc-André Lureau <marcandre.lureau@redhat.com>

The previous patch should fix shader compilation.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
---
 .gitlab-ci.d/windows.yml | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
index 22f794e537..5e59a7cc96 100644
--- a/.gitlab-ci.d/windows.yml
+++ b/.gitlab-ci.d/windows.yml
@@ -71,7 +71,7 @@ msys2-64bit:
   # for the msys2 64-bit job, due to the build could not complete within
   # the project timeout.
   - ..\msys64\usr\bin\bash -lc '../configure --target-list=x86_64-softmmu
-      --without-default-devices --disable-opengl'
+      --without-default-devices'
   - ..\msys64\usr\bin\bash -lc 'make'
   # qTests don't run successfully with "--without-default-devices",
   # so let's exclude the qtests from CI for now.
@@ -113,7 +113,6 @@ msys2-32bit:
   - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
   - mkdir output
   - cd output
-  - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu
-        --disable-opengl'
+  - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu'
   - ..\msys64\usr\bin\bash -lc 'make'
   - ..\msys64\usr\bin\bash -lc 'make check || { cat meson-logs/testlog.txt; exit 1; } ;'
-- 
2.39.0

Re: [PATCH 2/2] .gitlab-ci.d/windows: do not disable opengl

[Thomas Huth]

On 09/01/2023 12.21, marcandre.lureau@redhat.com wrote:
> From: Marc-André Lureau <marcandre.lureau@redhat.com>
> 
> The previous patch should fix shader compilation.
> 
> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> ---
>   .gitlab-ci.d/windows.yml | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/.gitlab-ci.d/windows.yml b/.gitlab-ci.d/windows.yml
> index 22f794e537..5e59a7cc96 100644
> --- a/.gitlab-ci.d/windows.yml
> +++ b/.gitlab-ci.d/windows.yml
> @@ -71,7 +71,7 @@ msys2-64bit:
>     # for the msys2 64-bit job, due to the build could not complete within
>     # the project timeout.
>     - ..\msys64\usr\bin\bash -lc '../configure --target-list=x86_64-softmmu
> -      --without-default-devices --disable-opengl'
> +      --without-default-devices'
>     - ..\msys64\usr\bin\bash -lc 'make'
>     # qTests don't run successfully with "--without-default-devices",
>     # so let's exclude the qtests from CI for now.
> @@ -113,7 +113,6 @@ msys2-32bit:
>     - $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
>     - mkdir output
>     - cd output
> -  - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu
> -        --disable-opengl'
> +  - ..\msys64\usr\bin\bash -lc '../configure --target-list=ppc64-softmmu'
>     - ..\msys64\usr\bin\bash -lc 'make'
>     - ..\msys64\usr\bin\bash -lc 'make check || { cat meson-logs/testlog.txt; exit 1; } ;'

Reviewed-by: Thomas Huth <thuth@redhat.com>