[PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Andrew Cooper]

Switch to using the system liblz4.

This brings libxenguest's lz4 decompression in line all the others, rather
than using the unsafe decompressor from Xen (itself a port of Linux's unsafe
decompressor).

Andrew Cooper (2):
  tools/configure: Detect the presence of liblz4
  tools/libs/guest: Use the system liblz4 in the bzimage loader

 automation/build/debian/13-arm64v8.dockerfile |   1 +
 automation/build/debian/13-x86_64.dockerfile  |   1 +
 tools/configure                               |  79 ++++++++++
 tools/configure.ac                            |   4 +
 tools/libs/guest/Makefile.common              |   2 +-
 tools/libs/guest/xg_dom_bzimageloader.c       | 128 +++++++++++++++-
 tools/libs/guest/xg_dom_decompress.h          |   6 -
 tools/libs/guest/xg_dom_decompress_lz4.c      | 143 ------------------
 tools/libs/guest/xg_dom_decompress_unsafe.h   |   2 +
 .../libs/guest/xg_dom_decompress_unsafe_lz4.c |  39 +++++
 10 files changed, 254 insertions(+), 151 deletions(-)
 delete mode 100644 tools/libs/guest/xg_dom_decompress.h
 delete mode 100644 tools/libs/guest/xg_dom_decompress_lz4.c
 create mode 100644 tools/libs/guest/xg_dom_decompress_unsafe_lz4.c


base-commit: c069c014f21fd1f5925d8c30c18adb4f26381475
-- 
2.39.5

[PATCH 1/2] tools/configure: Detect the presence of liblz4

[Andrew Cooper]

As with other compression libraries, group liblz4 into ZLIB_{CFLAGS,LIBS}.

Add the packages to the Debian Trixie build containers for coverage.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Juergen Gross <jgross@suse.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>

Rebuild the containers when pushed.
---
 automation/build/debian/13-arm64v8.dockerfile |  1 +
 automation/build/debian/13-x86_64.dockerfile  |  1 +
 tools/configure                               | 79 +++++++++++++++++++
 tools/configure.ac                            |  4 +
 4 files changed, 85 insertions(+)

diff --git a/automation/build/debian/13-arm64v8.dockerfile b/automation/build/debian/13-arm64v8.dockerfile
index b9062ee8b443..ee9bb841eb92 100644
--- a/automation/build/debian/13-arm64v8.dockerfile
+++ b/automation/build/debian/13-arm64v8.dockerfile
@@ -28,6 +28,7 @@ RUN <<EOF
         wget
         # libxenguest dombuilder
         libbz2-dev
+        liblz4-dev
         liblzma-dev
         liblzo2-dev
         libzstd-dev
diff --git a/automation/build/debian/13-x86_64.dockerfile b/automation/build/debian/13-x86_64.dockerfile
index 2c6c9d4a5098..e0a722595233 100644
--- a/automation/build/debian/13-x86_64.dockerfile
+++ b/automation/build/debian/13-x86_64.dockerfile
@@ -29,6 +29,7 @@ RUN <<EOF
         wget
         # libxenguest dombuilder
         libbz2-dev
+        liblz4-dev
         liblzma-dev
         liblzo2-dev
         libzstd-dev
diff --git a/tools/configure b/tools/configure
index 6f69b0b52634..bf14864cfb03 100755
--- a/tools/configure
+++ b/tools/configure
@@ -669,6 +669,8 @@ PTHREAD_CFLAGS
 EXTFS_LIBS
 ZLIB_LIBS
 ZLIB_CFLAGS
+liblz4_LIBS
+liblz4_CFLAGS
 libzstd_LIBS
 libzstd_CFLAGS
 FETCHER
@@ -885,6 +887,8 @@ pixman_CFLAGS
 pixman_LIBS
 libzstd_CFLAGS
 libzstd_LIBS
+liblz4_CFLAGS
+liblz4_LIBS
 libjsonc_CFLAGS
 libjsonc_LIBS
 LIBNL3_CFLAGS
@@ -1638,6 +1642,9 @@ Some influential environment variables:
               C compiler flags for libzstd, overriding pkg-config
   libzstd_LIBS
               linker flags for libzstd, overriding pkg-config
+  liblz4_CFLAGS
+              C compiler flags for liblz4, overriding pkg-config
+  liblz4_LIBS linker flags for liblz4, overriding pkg-config
   libjsonc_CFLAGS
               C compiler flags for libjsonc, overriding pkg-config
   libjsonc_LIBS
@@ -9458,6 +9465,78 @@ printf "%s\n" "yes" >&6; }
      ZLIB_LIBS="$ZLIB_LIBS $libzstd_LIBS"
 fi
 
+pkg_failed=no
+{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+printf %s "checking for liblz4... " >&6; }
+
+if test -n "$liblz4_CFLAGS"; then
+    pkg_cv_liblz4_CFLAGS="$liblz4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+    if test -n "$PKG_CONFIG" && \
+    { { printf "%s\n" "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+  ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+  ac_status=$?
+  printf "%s\n" "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+  test $ac_status = 0; }; then
+  pkg_cv_liblz4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+		      test "x$?" != "x0" && pkg_failed=yes
+else
+  pkg_failed=yes
+fi
+ else
+    pkg_failed=untried
+fi
+if test -n "$liblz4_LIBS"; then
+    pkg_cv_liblz4_LIBS="$liblz4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+    if test -n "$PKG_CONFIG" && \
+    { { printf "%s\n" "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+  ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+  ac_status=$?
+  printf "%s\n" "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+  test $ac_status = 0; }; then
+  pkg_cv_liblz4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+		      test "x$?" != "x0" && pkg_failed=yes
+else
+  pkg_failed=yes
+fi
+ else
+    pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+   	{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: no" >&5
+printf "%s\n" "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+        _pkg_short_errors_supported=yes
+else
+        _pkg_short_errors_supported=no
+fi
+        if test $_pkg_short_errors_supported = yes; then
+	        liblz4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+        else
+	        liblz4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+        fi
+	# Put the nasty error message in config.log where it belongs
+	echo "$liblz4_PKG_ERRORS" >&5
+
+	true
+elif test $pkg_failed = untried; then
+     	{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: no" >&5
+printf "%s\n" "no" >&6; }
+	true
+else
+	liblz4_CFLAGS=$pkg_cv_liblz4_CFLAGS
+	liblz4_LIBS=$pkg_cv_liblz4_LIBS
+        { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+printf "%s\n" "yes" >&6; }
+	ZLIB_CFLAGS="$ZLIB_CFLAGS -DHAVE_LZ4 $liblz4_CFLAGS"
+     ZLIB_LIBS="$ZLIB_LIBS $liblz4_LIBS"
+fi
+
 
 
 ac_fn_c_check_header_compile "$LINENO" "ext2fs/ext2fs.h" "ac_cv_header_ext2fs_ext2fs_h" "$ac_includes_default"
diff --git a/tools/configure.ac b/tools/configure.ac
index ecd45e782eb8..74b9f56025cb 100644
--- a/tools/configure.ac
+++ b/tools/configure.ac
@@ -403,6 +403,10 @@ PKG_CHECK_MODULES([libzstd], [libzstd],
     [ZLIB_CFLAGS="$ZLIB_CFLAGS -DHAVE_ZSTD $libzstd_CFLAGS"
      ZLIB_LIBS="$ZLIB_LIBS $libzstd_LIBS"],
     [true])
+PKG_CHECK_MODULES([liblz4], [liblz4],
+    [ZLIB_CFLAGS="$ZLIB_CFLAGS -DHAVE_LZ4 $liblz4_CFLAGS"
+     ZLIB_LIBS="$ZLIB_LIBS $liblz4_LIBS"],
+    [true])
 AC_SUBST([ZLIB_CFLAGS])
 AC_SUBST([ZLIB_LIBS])
 AX_CHECK_EXTFS
-- 
2.39.5

[PATCH 2/2] tools/libs/guest: Use the system liblz4 in the bzimage loader

[Andrew Cooper]

Right now lz4, unlike every other compression scheme, unconditionally uses
Xen's unsafe decompressor.  Make it consistent with all other compression
schemes by using liblz4.

The unsafe decompression is still required for the MiniOS build, so rename
xg_dom_decompress_lz4.c to xg_dom_decompress_unsafe_lz4.c and drop the
non-MiniOS content.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Juergen Gross <jgross@suse.com>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
 tools/libs/guest/Makefile.common              |   2 +-
 tools/libs/guest/xg_dom_bzimageloader.c       | 128 +++++++++++++++-
 tools/libs/guest/xg_dom_decompress.h          |   6 -
 tools/libs/guest/xg_dom_decompress_lz4.c      | 143 ------------------
 tools/libs/guest/xg_dom_decompress_unsafe.h   |   2 +
 .../libs/guest/xg_dom_decompress_unsafe_lz4.c |  39 +++++
 6 files changed, 169 insertions(+), 151 deletions(-)
 delete mode 100644 tools/libs/guest/xg_dom_decompress.h
 delete mode 100644 tools/libs/guest/xg_dom_decompress_lz4.c
 create mode 100644 tools/libs/guest/xg_dom_decompress_unsafe_lz4.c

diff --git a/tools/libs/guest/Makefile.common b/tools/libs/guest/Makefile.common
index b928a4a246a9..86b1f160e536 100644
--- a/tools/libs/guest/Makefile.common
+++ b/tools/libs/guest/Makefile.common
@@ -46,7 +46,6 @@ OBJS-y                 += xg_dom_core.o
 OBJS-y                 += xg_dom_boot.o
 OBJS-y                 += xg_dom_elfloader.o
 OBJS-$(CONFIG_X86)     += xg_dom_bzimageloader.o
-OBJS-$(CONFIG_X86)     += xg_dom_decompress_lz4.o
 OBJS-$(CONFIG_X86)     += xg_dom_hvmloader.o
 OBJS-$(CONFIG_ARM)     += xg_dom_armzimageloader.o
 OBJS-y                 += xg_dom_binloader.o
@@ -59,6 +58,7 @@ OBJS-$(CONFIG_ARM)     += xg_dom_arm.o
 ifeq ($(CONFIG_LIBXC_MINIOS),y)
 OBJS-y                 += xg_dom_decompress_unsafe.o
 OBJS-y                 += xg_dom_decompress_unsafe_bzip2.o
+OBJS-y                 += xg_dom_decompress_unsafe_lz4.o
 OBJS-y                 += xg_dom_decompress_unsafe_lzma.o
 OBJS-y                 += xg_dom_decompress_unsafe_lzo1x.o
 OBJS-y                 += xg_dom_decompress_unsafe_xz.o
diff --git a/tools/libs/guest/xg_dom_bzimageloader.c b/tools/libs/guest/xg_dom_bzimageloader.c
index 1fb4e5a1f728..32b3c682a447 100644
--- a/tools/libs/guest/xg_dom_bzimageloader.c
+++ b/tools/libs/guest/xg_dom_bzimageloader.c
@@ -32,7 +32,6 @@
 #include <inttypes.h>
 
 #include "xg_private.h"
-#include "xg_dom_decompress.h"
 
 #include <xen-tools/common-macros.h>
 
@@ -623,6 +622,133 @@ static int xc_try_zstd_decode(
 
 #endif
 
+#if defined(HAVE_LZ4)
+
+#include <lz4.h>
+
+#define ARCHIVE_MAGICNUMBER 0x184C2102
+
+static int xc_try_lz4_decode(struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    size_t outsize, insize;
+    unsigned char *outbuf = NULL, *inp = *blob, *outp;
+    uint32_t chunksize;
+
+    /* Magic, descriptor byte, and trailing size field. */
+    if ( *size <= 8 )
+    {
+        DOMPRINTF("LZ4: insufficient input data");
+        goto err;
+    }
+
+    insize = *size - 4;
+    outsize = get_unaligned_le32(*blob + insize);
+
+    if ( xc_dom_kernel_check_size(dom, outsize) )
+    {
+        DOMPRINTF("LZ4: output too large");
+        goto err;
+    }
+
+    outbuf = malloc(outsize);
+    if ( !outbuf )
+    {
+        DOMPRINTF("LZ4: failed to alloc memory");
+        goto err;
+    }
+    outp = outbuf;
+
+    chunksize = get_unaligned_le32(inp);
+    if ( chunksize == ARCHIVE_MAGICNUMBER )
+    {
+        inp    += 4;
+        insize -= 4;
+    }
+    else
+    {
+        DOMPRINTF("LZ4: invalid header");
+        goto err;
+    }
+
+    for ( ;; )
+    {
+        int dst_len, len;
+
+        if ( insize < 4 )
+        {
+            DOMPRINTF("LZ4: missing data");
+            goto err;
+        }
+
+        chunksize = get_unaligned_le32(inp);
+        inp    += 4;
+        insize -= 4;
+
+        if ( chunksize == ARCHIVE_MAGICNUMBER )
+            continue;
+
+        if ( chunksize > insize )
+        {
+            DOMPRINTF("LZ4: insufficient input data");
+            goto err;
+        }
+
+        dst_len = outsize - (outp - outbuf);
+        len = LZ4_decompress_safe((const void *)inp,
+                                  (void *)outp, chunksize, dst_len);
+
+        if ( len < 0 )
+        {
+            DOMPRINTF("LZ4: decoding failed");
+            goto err;
+        }
+
+        outp   += len;
+        inp    += chunksize;
+        insize -= chunksize;
+
+        if ( insize == 0 )
+            break;
+    }
+
+    if ( (outp - outbuf) != outsize )
+    {
+        DOMPRINTF("LZ4: got 0x%zx bytes instead of 0x%zx",
+                  outp - outbuf, outsize);
+        goto err;
+    }
+
+    if ( xc_dom_register_external(dom, outbuf, outsize) )
+    {
+        DOMPRINTF("LZ4: error registering stream output");
+        goto err;
+    }
+
+    DOMPRINTF("%s: LZ4 decompress OK, 0x%zx -> 0x%zx",
+              __FUNCTION__, insize, outsize);
+
+    *blob = outbuf;
+    *size = outsize;
+
+    return 0;
+
+ err:
+    free(outbuf);
+    return -1;
+}
+
+#else /* !defined(HAVE_LZ4) */
+
+static int xc_try_lz4_decode(struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
+                 "%s: LZ4 decompress support unavailable\n",
+                 __FUNCTION__);
+    return -1;
+}
+
+#endif
+
 #endif /* !__MINIOS__ */
 
 struct setup_header {
diff --git a/tools/libs/guest/xg_dom_decompress.h b/tools/libs/guest/xg_dom_decompress.h
deleted file mode 100644
index d7a45f730d67..000000000000
--- a/tools/libs/guest/xg_dom_decompress.h
+++ /dev/null
@@ -1,6 +0,0 @@
-#ifdef __MINIOS__
-# include "xg_dom_decompress_unsafe.h"
-#endif
-
-int xc_try_lz4_decode(struct xc_dom_image *dom, void **blob, size_t *size);
-
diff --git a/tools/libs/guest/xg_dom_decompress_lz4.c b/tools/libs/guest/xg_dom_decompress_lz4.c
deleted file mode 100644
index 53ef0bf328ed..000000000000
--- a/tools/libs/guest/xg_dom_decompress_lz4.c
+++ /dev/null
@@ -1,143 +0,0 @@
-#include <stdio.h>
-#include <stdlib.h>
-#include <inttypes.h>
-#include <stdint.h>
-
-#include INCLUDE_ENDIAN_H
-
-#define XG_NEED_UNALIGNED
-#include "xg_private.h"
-#include "xg_dom_decompress.h"
-
-#define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
-
-typedef uint8_t u8;
-typedef uint16_t u16;
-typedef uint32_t u32;
-typedef uint64_t u64;
-
-#define likely(a) a
-#define unlikely(a) a
-
-static inline uint16_t le16_to_cpu(uint16_t v)
-{
-#if BYTE_ORDER == BIG_ENDIAN
-    return __builtin_bswap16(v);
-#else
-    return v;
-#endif
-}
-
-#include "../../xen/include/xen/lz4.h"
-#include "../../xen/common/decompress.h"
-
-#ifndef __MINIOS__
-
-#include "../../xen/common/lz4/decompress.c"
-
-#define ARCHIVE_MAGICNUMBER 0x184C2102
-
-int xc_try_lz4_decode(
-	struct xc_dom_image *dom, void **blob, size_t *psize)
-{
-	int ret = -1;
-	unsigned char *inp = *blob, *output, *outp;
-	ssize_t size = *psize - 4;
-	size_t out_len, dest_len, chunksize;
-	const char *msg;
-
-	if (size < 4) {
-		msg = "input too small";
-		goto exit_0;
-	}
-
-	out_len = get_unaligned_le32(inp + size);
-	if (xc_dom_kernel_check_size(dom, out_len)) {
-		msg = "Decompressed image too large";
-		goto exit_0;
-	}
-
-	output = malloc(out_len);
-	if (!output) {
-		msg = "Could not allocate output buffer";
-		goto exit_0;
-	}
-	outp = output;
-
-	chunksize = get_unaligned_le32(inp);
-	if (chunksize == ARCHIVE_MAGICNUMBER) {
-		inp += 4;
-		size -= 4;
-	} else {
-		msg = "invalid header";
-		goto exit_2;
-	}
-
-	for (;;) {
-		if (size < 4) {
-			msg = "missing data";
-			goto exit_2;
-		}
-		chunksize = get_unaligned_le32(inp);
-		if (chunksize == ARCHIVE_MAGICNUMBER) {
-			inp += 4;
-			size -= 4;
-			continue;
-		}
-		inp += 4;
-		size -= 4;
-		if (chunksize > size) {
-			msg = "insufficient input data";
-			goto exit_2;
-		}
-
-		dest_len = out_len - (outp - output);
-		ret = lz4_decompress_unknownoutputsize(inp, chunksize, outp,
-				&dest_len);
-		if (ret < 0) {
-			msg = "decoding failed";
-			goto exit_2;
-		}
-
-		ret = -1;
-		outp += dest_len;
-		size -= chunksize;
-
-		if (size == 0)
-		{
-			if ( xc_dom_register_external(dom, output, out_len) )
-			{
-				msg = "Error registering stream output";
-				goto exit_2;
-			}
-			*blob = output;
-			*psize = out_len;
-			return 0;
-		}
-
-		if (size < 0) {
-			msg = "data corrupted";
-			goto exit_2;
-		}
-
-		inp += chunksize;
-	}
-
-exit_2:
-	free(output);
-exit_0:
-	DOMPRINTF("LZ4 decompression error: %s\n", msg);
-	return ret;
-}
-
-#else /* __MINIOS__ */
-
-#include "../../xen/common/unlz4.c"
-
-int xc_try_lz4_decode(
-    struct xc_dom_image *dom, void **blob, size_t *size)
-{
-    return xc_dom_decompress_unsafe(unlz4, dom, blob, size);
-}
-
-#endif
diff --git a/tools/libs/guest/xg_dom_decompress_unsafe.h b/tools/libs/guest/xg_dom_decompress_unsafe.h
index ac6b94288d5e..5bc222207653 100644
--- a/tools/libs/guest/xg_dom_decompress_unsafe.h
+++ b/tools/libs/guest/xg_dom_decompress_unsafe.h
@@ -16,6 +16,8 @@ int xc_dom_decompress_unsafe(
 
 int xc_try_bzip2_decode(struct xc_dom_image *dom, void **blob, size_t *size)
     __attribute__((visibility("internal")));
+int xc_try_lz4_decode(struct xc_dom_image *dom, void **blob, size_t *size)
+    __attribute__((visibility("internal")));
 int xc_try_lzma_decode(struct xc_dom_image *dom, void **blob, size_t *size)
     __attribute__((visibility("internal")));
 int xc_try_lzo1x_decode(struct xc_dom_image *dom, void **blob, size_t *size)
diff --git a/tools/libs/guest/xg_dom_decompress_unsafe_lz4.c b/tools/libs/guest/xg_dom_decompress_unsafe_lz4.c
new file mode 100644
index 000000000000..405143aa6182
--- /dev/null
+++ b/tools/libs/guest/xg_dom_decompress_unsafe_lz4.c
@@ -0,0 +1,39 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <inttypes.h>
+#include <stdint.h>
+
+#include INCLUDE_ENDIAN_H
+
+#define XG_NEED_UNALIGNED
+#include "xg_private.h"
+#include "xg_dom_decompress.h"
+
+#define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+
+typedef uint8_t u8;
+typedef uint16_t u16;
+typedef uint32_t u32;
+typedef uint64_t u64;
+
+#define likely(a) a
+#define unlikely(a) a
+
+static inline uint16_t le16_to_cpu(uint16_t v)
+{
+#if BYTE_ORDER == BIG_ENDIAN
+    return __builtin_bswap16(v);
+#else
+    return v;
+#endif
+}
+
+#include "../../xen/include/xen/lz4.h"
+#include "../../xen/common/decompress.h"
+#include "../../xen/common/unlz4.c"
+
+int xc_try_lz4_decode(
+    struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    return xc_dom_decompress_unsafe(unlz4, dom, blob, size);
+}
-- 
2.39.5

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Jan Beulich]

On 03.06.2026 10:53, Andrew Cooper wrote:
> Switch to using the system liblz4.
> 
> This brings libxenguest's lz4 decompression in line all the others, rather
> than using the unsafe decompressor from Xen (itself a port of Linux's unsafe
> decompressor).

As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"), there was
no shared library available at the time (and on the SLES versions I worked
with). Later a shared library appeared, but the -devel package still wasn't
there. On my main dev system (intentionally a relatively old SLES version) I
therefore wouldn't be able to build/test LZ4 anymore if we went this route.
(FTAOD this isn't an outright objection, as the goal of the series is
certainly good. It is mainly a data point to consider.)

Jan

> Andrew Cooper (2):
>   tools/configure: Detect the presence of liblz4
>   tools/libs/guest: Use the system liblz4 in the bzimage loader
> 
>  automation/build/debian/13-arm64v8.dockerfile |   1 +
>  automation/build/debian/13-x86_64.dockerfile  |   1 +
>  tools/configure                               |  79 ++++++++++
>  tools/configure.ac                            |   4 +
>  tools/libs/guest/Makefile.common              |   2 +-
>  tools/libs/guest/xg_dom_bzimageloader.c       | 128 +++++++++++++++-
>  tools/libs/guest/xg_dom_decompress.h          |   6 -
>  tools/libs/guest/xg_dom_decompress_lz4.c      | 143 ------------------
>  tools/libs/guest/xg_dom_decompress_unsafe.h   |   2 +
>  .../libs/guest/xg_dom_decompress_unsafe_lz4.c |  39 +++++
>  10 files changed, 254 insertions(+), 151 deletions(-)
>  delete mode 100644 tools/libs/guest/xg_dom_decompress.h
>  delete mode 100644 tools/libs/guest/xg_dom_decompress_lz4.c
>  create mode 100644 tools/libs/guest/xg_dom_decompress_unsafe_lz4.c
> 
> 
> base-commit: c069c014f21fd1f5925d8c30c18adb4f26381475

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Andrew Cooper]

On 03/06/2026 10:16 am, Jan Beulich wrote:
> On 03.06.2026 10:53, Andrew Cooper wrote:
>> Switch to using the system liblz4.
>>
>> This brings libxenguest's lz4 decompression in line all the others, rather
>> than using the unsafe decompressor from Xen (itself a port of Linux's unsafe
>> decompressor).
> As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"), there was
> no shared library available at the time (and on the SLES versions I worked
> with). Later a shared library appeared, but the -devel package still wasn't
> there. On my main dev system (intentionally a relatively old SLES version) I
> therefore wouldn't be able to build/test LZ4 anymore if we went this route.
> (FTAOD this isn't an outright objection, as the goal of the series is
> certainly good. It is mainly a data point to consider.)

That was 13 years ago.  Are you saying that there's still an in-support
version of SLES which doesn't have liblz4 ?

~Andrew

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Jan Beulich]

On 03.06.2026 11:21, Andrew Cooper wrote:
> On 03/06/2026 10:16 am, Jan Beulich wrote:
>> On 03.06.2026 10:53, Andrew Cooper wrote:
>>> Switch to using the system liblz4.
>>>
>>> This brings libxenguest's lz4 decompression in line all the others, rather
>>> than using the unsafe decompressor from Xen (itself a port of Linux's unsafe
>>> decompressor).
>> As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"), there was
>> no shared library available at the time (and on the SLES versions I worked
>> with). Later a shared library appeared, but the -devel package still wasn't
>> there. On my main dev system (intentionally a relatively old SLES version) I
>> therefore wouldn't be able to build/test LZ4 anymore if we went this route.
>> (FTAOD this isn't an outright objection, as the goal of the series is
>> certainly good. It is mainly a data point to consider.)
> 
> That was 13 years ago.  Are you saying that there's still an in-support
> version of SLES which doesn't have liblz4 ?

If "in-support" includes LTSS, then yes as far as liblz4-devel goes. If you
mean only ordinary support, then (afaict) no.

Jan

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Oleksii Kurochko]

On 6/3/26 11:16 AM, Jan Beulich wrote:
> On 03.06.2026 10:53, Andrew Cooper wrote:
>> Switch to using the system liblz4.
>>
>> This brings libxenguest's lz4 decompression in line all the others, rather
>> than using the unsafe decompressor from Xen (itself a port of Linux's unsafe
>> decompressor).

Generally, the patch series looks straightforward and low risk, so I am 
comfortable taking it for this release. Considering that...

> 
> As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"), there was
> no shared library available at the time (and on the SLES versions I worked
> with). Later a shared library appeared, but the -devel package still wasn't
> there. On my main dev system (intentionally a relatively old SLES version) I
> therefore wouldn't be able to build/test LZ4 anymore if we went this route.
> (FTAOD this isn't an outright objection, as the goal of the series is
> certainly good. It is mainly a data point to consider.)

...does not consider this an outright objection. While this may become 
an issue sooner or later on older dev systems, if the change is accepted 
into staging, we could switch to the shared library approach in 4.22.

We should also consider adding a note to CHANGELOG.md.

Thanks.

~ Oleksii

Re: [PATCH 1/2] tools/configure: Detect the presence of liblz4

[Roger Pau Monné]

On Wed, Jun 03, 2026 at 09:53:30AM +0100, Andrew Cooper wrote:
> As with other compression libraries, group liblz4 into ZLIB_{CFLAGS,LIBS}.
> 
> Add the packages to the Debian Trixie build containers for coverage.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks, Roger.

Re: [PATCH 2/2] tools/libs/guest: Use the system liblz4 in the bzimage loader

[Roger Pau Monné]

On Wed, Jun 03, 2026 at 09:53:31AM +0100, Andrew Cooper wrote:
> Right now lz4, unlike every other compression scheme, unconditionally uses
> Xen's unsafe decompressor.  Make it consistent with all other compression
> schemes by using liblz4.
> 
> The unsafe decompression is still required for the MiniOS build, so rename
> xg_dom_decompress_lz4.c to xg_dom_decompress_unsafe_lz4.c and drop the
> non-MiniOS content.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

> ---
> CC: Anthony PERARD <anthony.perard@vates.tech>
> CC: Juergen Gross <jgross@suse.com>
> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
>  tools/libs/guest/Makefile.common              |   2 +-
>  tools/libs/guest/xg_dom_bzimageloader.c       | 128 +++++++++++++++-
>  tools/libs/guest/xg_dom_decompress.h          |   6 -
>  tools/libs/guest/xg_dom_decompress_lz4.c      | 143 ------------------
>  tools/libs/guest/xg_dom_decompress_unsafe.h   |   2 +
>  .../libs/guest/xg_dom_decompress_unsafe_lz4.c |  39 +++++
>  6 files changed, 169 insertions(+), 151 deletions(-)
>  delete mode 100644 tools/libs/guest/xg_dom_decompress.h
>  delete mode 100644 tools/libs/guest/xg_dom_decompress_lz4.c
>  create mode 100644 tools/libs/guest/xg_dom_decompress_unsafe_lz4.c
> 
> diff --git a/tools/libs/guest/Makefile.common b/tools/libs/guest/Makefile.common
> index b928a4a246a9..86b1f160e536 100644
> --- a/tools/libs/guest/Makefile.common
> +++ b/tools/libs/guest/Makefile.common
> @@ -46,7 +46,6 @@ OBJS-y                 += xg_dom_core.o
>  OBJS-y                 += xg_dom_boot.o
>  OBJS-y                 += xg_dom_elfloader.o
>  OBJS-$(CONFIG_X86)     += xg_dom_bzimageloader.o
> -OBJS-$(CONFIG_X86)     += xg_dom_decompress_lz4.o
>  OBJS-$(CONFIG_X86)     += xg_dom_hvmloader.o
>  OBJS-$(CONFIG_ARM)     += xg_dom_armzimageloader.o
>  OBJS-y                 += xg_dom_binloader.o
> @@ -59,6 +58,7 @@ OBJS-$(CONFIG_ARM)     += xg_dom_arm.o
>  ifeq ($(CONFIG_LIBXC_MINIOS),y)
>  OBJS-y                 += xg_dom_decompress_unsafe.o
>  OBJS-y                 += xg_dom_decompress_unsafe_bzip2.o
> +OBJS-y                 += xg_dom_decompress_unsafe_lz4.o
>  OBJS-y                 += xg_dom_decompress_unsafe_lzma.o
>  OBJS-y                 += xg_dom_decompress_unsafe_lzo1x.o
>  OBJS-y                 += xg_dom_decompress_unsafe_xz.o
> diff --git a/tools/libs/guest/xg_dom_bzimageloader.c b/tools/libs/guest/xg_dom_bzimageloader.c
> index 1fb4e5a1f728..32b3c682a447 100644
> --- a/tools/libs/guest/xg_dom_bzimageloader.c
> +++ b/tools/libs/guest/xg_dom_bzimageloader.c
> @@ -32,7 +32,6 @@
>  #include <inttypes.h>
>  
>  #include "xg_private.h"
> -#include "xg_dom_decompress.h"
>  
>  #include <xen-tools/common-macros.h>
>  
> @@ -623,6 +622,133 @@ static int xc_try_zstd_decode(
>  
>  #endif
>  
> +#if defined(HAVE_LZ4)
> +
> +#include <lz4.h>
> +
> +#define ARCHIVE_MAGICNUMBER 0x184C2102
> +
> +static int xc_try_lz4_decode(struct xc_dom_image *dom, void **blob, size_t *size)
> +{
> +    size_t outsize, insize;
> +    unsigned char *outbuf = NULL, *inp = *blob, *outp;
> +    uint32_t chunksize;
> +
> +    /* Magic, descriptor byte, and trailing size field. */
> +    if ( *size <= 8 )
> +    {
> +        DOMPRINTF("LZ4: insufficient input data");
> +        goto err;
> +    }
> +
> +    insize = *size - 4;
> +    outsize = get_unaligned_le32(*blob + insize);
> +
> +    if ( xc_dom_kernel_check_size(dom, outsize) )
> +    {
> +        DOMPRINTF("LZ4: output too large");
> +        goto err;
> +    }
> +
> +    outbuf = malloc(outsize);

I would use calloc() or memset() the buffer, just in case part of it
is (wrongly) left uninitialized, as this is copied into guest memory.

I see this is code moment, so possibly better to adjust afterwards if
anything.

Thanks, Roger.

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Oleksii Kurochko]

On 6/3/26 3:01 PM, Oleksii Kurochko wrote:
> 
> 
> On 6/3/26 11:16 AM, Jan Beulich wrote:
>> On 03.06.2026 10:53, Andrew Cooper wrote:
>>> Switch to using the system liblz4.
>>>
>>> This brings libxenguest's lz4 decompression in line all the others, 
>>> rather
>>> than using the unsafe decompressor from Xen (itself a port of Linux's 
>>> unsafe
>>> decompressor).
> 
> Generally, the patch series looks straightforward and low risk, so I am 
> comfortable taking it for this release. Considering that...
> 
>>
>> As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"), 
>> there was
>> no shared library available at the time (and on the SLES versions I 
>> worked
>> with). Later a shared library appeared, but the -devel package still 
>> wasn't
>> there. On my main dev system (intentionally a relatively old SLES 
>> version) I
>> therefore wouldn't be able to build/test LZ4 anymore if we went this 
>> route.
>> (FTAOD this isn't an outright objection, as the goal of the series is
>> certainly good. It is mainly a data point to consider.)
> 
> ...does not consider this an outright objection. While this may become 
> an issue sooner or later on older dev systems, if the change is accepted 
> into staging, we could switch to the shared library approach in 4.22.

Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii

Re: [PATCH 2/2] tools/libs/guest: Use the system liblz4 in the bzimage loader

[Andrew Cooper]

On 08/06/2026 11:46 am, Roger Pau Monné wrote:
> On Wed, Jun 03, 2026 at 09:53:31AM +0100, Andrew Cooper wrote:
>> Right now lz4, unlike every other compression scheme, unconditionally uses
>> Xen's unsafe decompressor.  Make it consistent with all other compression
>> schemes by using liblz4.
>>
>> The unsafe decompression is still required for the MiniOS build, so rename
>> xg_dom_decompress_lz4.c to xg_dom_decompress_unsafe_lz4.c and drop the
>> non-MiniOS content.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
>
>> ---
>> CC: Anthony PERARD <anthony.perard@vates.tech>
>> CC: Juergen Gross <jgross@suse.com>
>> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
>> ---
>>  tools/libs/guest/Makefile.common              |   2 +-
>>  tools/libs/guest/xg_dom_bzimageloader.c       | 128 +++++++++++++++-
>>  tools/libs/guest/xg_dom_decompress.h          |   6 -
>>  tools/libs/guest/xg_dom_decompress_lz4.c      | 143 ------------------
>>  tools/libs/guest/xg_dom_decompress_unsafe.h   |   2 +
>>  .../libs/guest/xg_dom_decompress_unsafe_lz4.c |  39 +++++
>>  6 files changed, 169 insertions(+), 151 deletions(-)
>>  delete mode 100644 tools/libs/guest/xg_dom_decompress.h
>>  delete mode 100644 tools/libs/guest/xg_dom_decompress_lz4.c
>>  create mode 100644 tools/libs/guest/xg_dom_decompress_unsafe_lz4.c
>>
>> diff --git a/tools/libs/guest/Makefile.common b/tools/libs/guest/Makefile.common
>> index b928a4a246a9..86b1f160e536 100644
>> --- a/tools/libs/guest/Makefile.common
>> +++ b/tools/libs/guest/Makefile.common
>> @@ -46,7 +46,6 @@ OBJS-y                 += xg_dom_core.o
>>  OBJS-y                 += xg_dom_boot.o
>>  OBJS-y                 += xg_dom_elfloader.o
>>  OBJS-$(CONFIG_X86)     += xg_dom_bzimageloader.o
>> -OBJS-$(CONFIG_X86)     += xg_dom_decompress_lz4.o
>>  OBJS-$(CONFIG_X86)     += xg_dom_hvmloader.o
>>  OBJS-$(CONFIG_ARM)     += xg_dom_armzimageloader.o
>>  OBJS-y                 += xg_dom_binloader.o
>> @@ -59,6 +58,7 @@ OBJS-$(CONFIG_ARM)     += xg_dom_arm.o
>>  ifeq ($(CONFIG_LIBXC_MINIOS),y)
>>  OBJS-y                 += xg_dom_decompress_unsafe.o
>>  OBJS-y                 += xg_dom_decompress_unsafe_bzip2.o
>> +OBJS-y                 += xg_dom_decompress_unsafe_lz4.o
>>  OBJS-y                 += xg_dom_decompress_unsafe_lzma.o
>>  OBJS-y                 += xg_dom_decompress_unsafe_lzo1x.o
>>  OBJS-y                 += xg_dom_decompress_unsafe_xz.o
>> diff --git a/tools/libs/guest/xg_dom_bzimageloader.c b/tools/libs/guest/xg_dom_bzimageloader.c
>> index 1fb4e5a1f728..32b3c682a447 100644
>> --- a/tools/libs/guest/xg_dom_bzimageloader.c
>> +++ b/tools/libs/guest/xg_dom_bzimageloader.c
>> @@ -32,7 +32,6 @@
>>  #include <inttypes.h>
>>  
>>  #include "xg_private.h"
>> -#include "xg_dom_decompress.h"
>>  
>>  #include <xen-tools/common-macros.h>
>>  
>> @@ -623,6 +622,133 @@ static int xc_try_zstd_decode(
>>  
>>  #endif
>>  
>> +#if defined(HAVE_LZ4)
>> +
>> +#include <lz4.h>
>> +
>> +#define ARCHIVE_MAGICNUMBER 0x184C2102
>> +
>> +static int xc_try_lz4_decode(struct xc_dom_image *dom, void **blob, size_t *size)
>> +{
>> +    size_t outsize, insize;
>> +    unsigned char *outbuf = NULL, *inp = *blob, *outp;
>> +    uint32_t chunksize;
>> +
>> +    /* Magic, descriptor byte, and trailing size field. */
>> +    if ( *size <= 8 )
>> +    {
>> +        DOMPRINTF("LZ4: insufficient input data");
>> +        goto err;
>> +    }
>> +
>> +    insize = *size - 4;
>> +    outsize = get_unaligned_le32(*blob + insize);
>> +
>> +    if ( xc_dom_kernel_check_size(dom, outsize) )
>> +    {
>> +        DOMPRINTF("LZ4: output too large");
>> +        goto err;
>> +    }
>> +
>> +    outbuf = malloc(outsize);
> I would use calloc() or memset() the buffer, just in case part of it
> is (wrongly) left uninitialized, as this is copied into guest memory.
>
> I see this is code moment, so possibly better to adjust afterwards if
> anything.

There's a later check:

    if ( (outp - outbuf) != outsize )

which will fail and free the buffer if the output length isn't exactly
as expected.

This code is all manner of horrible, including the fact that liblz4
doesn't actually have a function to decompress a full stream; only
chunk-by-chunk with the caller providing the framing.

~Andrew

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Andrew Cooper]

On 08/06/2026 11:51 am, Oleksii Kurochko wrote:
>
>
> On 6/3/26 3:01 PM, Oleksii Kurochko wrote:
>>
>>
>> On 6/3/26 11:16 AM, Jan Beulich wrote:
>>> On 03.06.2026 10:53, Andrew Cooper wrote:
>>>> Switch to using the system liblz4.
>>>>
>>>> This brings libxenguest's lz4 decompression in line all the others,
>>>> rather
>>>> than using the unsafe decompressor from Xen (itself a port of
>>>> Linux's unsafe
>>>> decompressor).
>>
>> Generally, the patch series looks straightforward and low risk, so I
>> am comfortable taking it for this release. Considering that...
>>
>>>
>>> As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"),
>>> there was
>>> no shared library available at the time (and on the SLES versions I
>>> worked
>>> with). Later a shared library appeared, but the -devel package still
>>> wasn't
>>> there. On my main dev system (intentionally a relatively old SLES
>>> version) I
>>> therefore wouldn't be able to build/test LZ4 anymore if we went this
>>> route.
>>> (FTAOD this isn't an outright objection, as the goal of the series is
>>> certainly good. It is mainly a data point to consider.)
>>
>> ...does not consider this an outright objection. While this may
>> become an issue sooner or later on older dev systems, if the change
>> is accepted into staging, we could switch to the shared library
>> approach in 4.22.
>
> Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> 

I've included this incremental diff for CHANGELOG.

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1db3efc4864c..5cf19372a361 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -13,6 +13,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    represent a wildcard input.
  - On x86:
    - Enable pf-fixup option by default for PVH dom0.
+   - The libxenguest bzImage loader now uses the system liblz4 library.
 
 ### Added
  - Support for per-domain Xenstore quota in C xenstored (includes


~Andrew

Re: [PATCH for-4.22(?) 0/2] tools: Use the system liblz4 package

[Oleksii Kurochko]

On 6/8/26 12:57 PM, Andrew Cooper wrote:
> On 08/06/2026 11:51 am, Oleksii Kurochko wrote:
>>
>>
>> On 6/3/26 3:01 PM, Oleksii Kurochko wrote:
>>>
>>>
>>> On 6/3/26 11:16 AM, Jan Beulich wrote:
>>>> On 03.06.2026 10:53, Andrew Cooper wrote:
>>>>> Switch to using the system liblz4.
>>>>>
>>>>> This brings libxenguest's lz4 decompression in line all the others,
>>>>> rather
>>>>> than using the unsafe decompressor from Xen (itself a port of
>>>>> Linux's unsafe
>>>>> decompressor).
>>>
>>> Generally, the patch series looks straightforward and low risk, so I
>>> am comfortable taking it for this release. Considering that...
>>>
>>>>
>>>> As stated in 84f04d8f0dbf ("libxc: add LZ4 decompression support"),
>>>> there was
>>>> no shared library available at the time (and on the SLES versions I
>>>> worked
>>>> with). Later a shared library appeared, but the -devel package still
>>>> wasn't
>>>> there. On my main dev system (intentionally a relatively old SLES
>>>> version) I
>>>> therefore wouldn't be able to build/test LZ4 anymore if we went this
>>>> route.
>>>> (FTAOD this isn't an outright objection, as the goal of the series is
>>>> certainly good. It is mainly a data point to consider.)
>>>
>>> ...does not consider this an outright objection. While this may
>>> become an issue sooner or later on older dev systems, if the change
>>> is accepted into staging, we could switch to the shared library
>>> approach in 4.22.
>>
>> Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> 
> I've included this incremental diff for CHANGELOG.
> 
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index 1db3efc4864c..5cf19372a361 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -13,6 +13,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
>      represent a wildcard input.
>    - On x86:
>      - Enable pf-fixup option by default for PVH dom0.
> +   - The libxenguest bzImage loader now uses the system liblz4 library.

Looks good to me.

Thanks.

~ Oleksii