[PATCH 0/2] Fixes docs building on msys2/mingw

[PATCH 0/2] Fixes docs building on msys2/mingw

[Yonggang Luo]

The cirrus are based on non-merged patches

Yonggang Luo (2):
  docs: Fixes build docs on msys2/mingw
  cirrus: Enable doc build on msys2/mingw

 .cirrus.yml                   |  8 ++++++++
 docs/conf.py                  |  2 +-
 docs/sphinx/kerneldoc.py      |  2 +-
 scripts/rst-sanitize.py       | 21 +++++++++++++++++++++
 tests/qapi-schema/meson.build |  5 +++--
 5 files changed, 34 insertions(+), 4 deletions(-)
 create mode 100644 scripts/rst-sanitize.py

-- 
2.28.0.windows.1

[PATCH 1/2] docs: Fixes build docs on msys2/mingw

[Yonggang Luo]

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 docs/conf.py                  |  2 +-
 docs/sphinx/kerneldoc.py      |  2 +-
 scripts/rst-sanitize.py       | 21 +++++++++++++++++++++
 tests/qapi-schema/meson.build |  5 +++--
 4 files changed, 26 insertions(+), 4 deletions(-)
 create mode 100644 scripts/rst-sanitize.py

diff --git a/docs/conf.py b/docs/conf.py
index 00e1b750e2..e584f68393 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -241,7 +241,7 @@ texinfo_documents = [
 # We use paths starting from qemu_docdir here so that you can run
 # sphinx-build from anywhere and the kerneldoc extension can still
 # find everything.
-kerneldoc_bin = os.path.join(qemu_docdir, '../scripts/kernel-doc')
+kerneldoc_bin = ['perl', os.path.join(qemu_docdir, '../scripts/kernel-doc')]
 kerneldoc_srctree = os.path.join(qemu_docdir, '..')
 hxtool_srctree = os.path.join(qemu_docdir, '..')
 qapidoc_srctree = os.path.join(qemu_docdir, '..')
diff --git a/docs/sphinx/kerneldoc.py b/docs/sphinx/kerneldoc.py
index 3e87940206..3ac277d162 100644
--- a/docs/sphinx/kerneldoc.py
+++ b/docs/sphinx/kerneldoc.py
@@ -67,7 +67,7 @@ class KernelDocDirective(Directive):
 
     def run(self):
         env = self.state.document.settings.env
-        cmd = [env.config.kerneldoc_bin, '-rst', '-enable-lineno']
+        cmd = env.config.kerneldoc_bin + ['-rst', '-enable-lineno']
 
         filename = env.config.kerneldoc_srctree + '/' + self.arguments[0]
         export_file_patterns = []
diff --git a/scripts/rst-sanitize.py b/scripts/rst-sanitize.py
new file mode 100644
index 0000000000..26060f1208
--- /dev/null
+++ b/scripts/rst-sanitize.py
@@ -0,0 +1,21 @@
+#!/usr/bin/env python3
+
+#
+# Script for remove cr line ending in file
+#
+# Authors:
+#  Yonggang Luo <luoyonggang@gmail.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or, at your option, any later version.  See the COPYING file in
+# the top-level directory.
+
+import sys
+
+def main(_program, file, *unused):
+    with open(file, 'rb') as content_file:
+        content = content_file.read()
+        sys.stdout.buffer.write(content.replace(b'\r', b''))
+
+if __name__ == "__main__":
+    main(*sys.argv)
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index f08c902911..a6832634b3 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -250,18 +250,19 @@ qapi_doc_out = custom_target('QAPI rST doc',
 # using an explicit '\' character in the command arguments to
 # a custom_target(), as Meson will unhelpfully replace it with a '/'
 # (https://github.com/mesonbuild/meson/issues/1564)
+rst_sanitize_cmd = [find_program('../../scripts/rst-sanitize.py'), '@INPUT@']
 qapi_doc_out_nocr = custom_target('QAPI rST doc newline-sanitized',
                                   output: ['doc-good.txt.nocr'],
                                   input: qapi_doc_out[0],
                                   build_by_default: build_docs,
-                                  command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
+                                  command: rst_sanitize_cmd,
                                   capture: true)
 
 qapi_doc_ref_nocr = custom_target('QAPI rST doc reference newline-sanitized',
                                   output: ['doc-good.ref.nocr'],
                                   input: files('doc-good.txt'),
                                   build_by_default: build_docs,
-                                  command: ['perl', '-pe', '$x = chr 13; s/$x$//', '@INPUT@'],
+                                  command: rst_sanitize_cmd,
                                   capture: true)
 
 if build_docs
-- 
2.28.0.windows.1

[PATCH 2/2] cirrus: Enable doc build on msys2/mingw

[Yonggang Luo]

Currently rST depends on old version sphinx-2.x.
Install it by downloading it.

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 .cirrus.yml | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/.cirrus.yml b/.cirrus.yml
index 77cfd99afe..b1725bd0c1 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -73,6 +73,7 @@ windows_msys2_task:
         bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND https://github.com/msys2/msys2-installer/releases/download/2020-09-03/msys2-base-x86_64-20200903.sfx.exe C:\tools\base.exe
         Write-Output "Download time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"
         C:\tools\base.exe -y
+        del C:\tools\base.exe
         ((Get-Content -path C:\tools\msys64\etc\\post-install\\07-pacman-key.post -Raw) -replace '--refresh-keys', '--version') | Set-Content -Path C:\tools\msys64\etc\\post-install\\07-pacman-key.post
         C:\tools\msys64\usr\bin\bash.exe -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
         C:\tools\msys64\usr\bin\bash.exe -lc "export"
@@ -112,6 +113,13 @@ windows_msys2_task:
           mingw-w64-x86_64-gnutls \
           mingw-w64-x86_64-libnfs \
           "
+        bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND `
+          https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/x86_64/mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz `
+          C:\tools\mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
+        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -U \
+          /c/tools/mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
+          "
+        del C:\tools\mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
         C:\tools\msys64\usr\bin\bash.exe -lc "rm -rf /var/cache/pacman/pkg/*"
         cd C:\tools\msys64
         echo "Start archive"
-- 
2.28.0.windows.1

Re: [PATCH 2/2] cirrus: Enable doc build on msys2/mingw

[Daniel P. Berrangé]

On Fri, Oct 09, 2020 at 04:13:53AM +0800, Yonggang Luo wrote:
> Currently rST depends on old version sphinx-2.x.
> Install it by downloading it.
> 
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> ---
>  .cirrus.yml | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/.cirrus.yml b/.cirrus.yml
> index 77cfd99afe..b1725bd0c1 100644
> --- a/.cirrus.yml
> +++ b/.cirrus.yml
> @@ -73,6 +73,7 @@ windows_msys2_task:
>          bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND https://github.com/msys2/msys2-installer/releases/download/2020-09-03/msys2-base-x86_64-20200903.sfx.exe C:\tools\base.exe
>          Write-Output "Download time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"
>          C:\tools\base.exe -y
> +        del C:\tools\base.exe
>          ((Get-Content -path C:\tools\msys64\etc\\post-install\\07-pacman-key.post -Raw) -replace '--refresh-keys', '--version') | Set-Content -Path C:\tools\msys64\etc\\post-install\\07-pacman-key.post
>          C:\tools\msys64\usr\bin\bash.exe -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
>          C:\tools\msys64\usr\bin\bash.exe -lc "export"
> @@ -112,6 +113,13 @@ windows_msys2_task:
>            mingw-w64-x86_64-gnutls \
>            mingw-w64-x86_64-libnfs \
>            "
> +        bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND `
> +          https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/x86_64/mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz `

Relying on some random university mirror site is not a nice approach IMHO.
Why isn't this using the official msys2 site like the other packages ?

> +          C:\tools\mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
> +        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -U \
> +          /c/tools/mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
> +          "
> +        del C:\tools\mingw-w64-x86_64-python-sphinx-2.3.1-1-any.pkg.tar.xz
>          C:\tools\msys64\usr\bin\bash.exe -lc "rm -rf /var/cache/pacman/pkg/*"
>          cd C:\tools\msys64
>          echo "Start archive"

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|