[PATCH 3/4] tests/file_filter: Add zstd test file

[PATCH 3/4] tests/file_filter: Add zstd test file

[Logan Gunthorpe via Grub-devel]

Add a file.zstd similar to the other compression methods and generate
a gpg signature with `gpg --detach-sign`

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
---
 tests/file_filter/file.zstd     | Bin 0 -> 26 bytes
 tests/file_filter/file.zstd.sig | Bin 0 -> 310 bytes
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/file_filter/file.zstd
 create mode 100644 tests/file_filter/file.zstd.sig

diff --git a/tests/file_filter/file.zstd b/tests/file_filter/file.zstd
new file mode 100644
index 0000000000000000000000000000000000000000..6ea24ccf8390f456a63f15e1d0f473d0a0ebc75e
GIT binary patch
literal 26
hcmdPcs{fZIB9no^BQ+-{Uq_*|IJHQTYg5;2b^v!n2`2ym

literal 0
HcmV?d00001

diff --git a/tests/file_filter/file.zstd.sig b/tests/file_filter/file.zstd.sig
new file mode 100644
index 0000000000000000000000000000000000000000..ab101bb6e883a09dd41ab8fe348cf8a9ce5843c7
GIT binary patch
literal 310
zcmV-60m=S}0W$;u0SW*e79j-t9c_qHaZBEqV(=dtEpNLINa*JU0%+3Wr2q;E5E?CS
zyAMd{=f%Sa0D;xR-EXs|$ZnV8EKm8`Acg4~^9(E4y~^NzaINTQYg%<WdW2aqV(Ij1
zIfbrviC5H^>`Gg2<@;Fpno=K?r5u)5TM|R$F@ovn27&G8fg-pVRvhu}ovsacO<}u{
z25i6uvgdqAkCTR$d4F3pccusj##rB9AXS&mdn)t-MYSd?m-HI2<^5Isfdl;1i)xUX
zxKdFt^#|YWXW1hao$90KM#Cx?ic2Sv^Si30&h0m4<=XFvh;2}vaiGu|h={rI3w*pU
zi7KP2UVUACi2ubxOq2^og4khH%5%bHKw9Mcwh)Sob&Oz#ur6US1>g2>8<kESSR<Uw
I4A}zsN@DJozW@LL

literal 0
HcmV?d00001

-- 
2.47.3


_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 0/4] Add zstdio support for booting Trixie kernels in Xen

[Logan Gunthorpe via Grub-devel]

This is v2 of the patch cleaning up a lot of style issues noted by
Avnish. I think I've got all the style issues that were requested fixed
up.

Thanks,

Logan

--

Changes in v2:
  - Fixed a bunch of style issues noted by Avnish.

Cover letter from v1:

Debian has started compressing the kernel images in the Trixie release.
This breaks booting xen kernels seeing grub has no support to decompress
the zstd images[1][2].

To improve this situation, this patch series adds zstdio support in
a way that's very similar to xzio. The first patch adds this support.
The remaining patchs expand the file_filter tests so this functionality
is covered.

--

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1102066
[2] https://www.mail-archive.com/grub-devel@gnu.org/msg41663.html

Logan Gunthorpe (4):
  io: Implement zstdio decompression
  tests/file_filter: Regenerate gpg keys
  tests/file_filter: Add zstd test file
  tests/file_filter: Add zstd tests

 Makefile.util.def               |   1 +
 grub-core/Makefile.core.def     |   7 +
 grub-core/io/zstdio.c           | 237 ++++++++++++++++++++++++++++++++
 include/grub/file.h             |   3 +-
 tests/file_filter/file.gz.sig   | Bin 96 -> 310 bytes
 tests/file_filter/file.lzop.sig | Bin 96 -> 310 bytes
 tests/file_filter/file.xz.sig   | Bin 96 -> 310 bytes
 tests/file_filter/file.zstd     | Bin 0 -> 26 bytes
 tests/file_filter/file.zstd.sig | Bin 0 -> 310 bytes
 tests/file_filter/keys          | Bin 994 -> 1308 bytes
 tests/file_filter/keys.pub      | Bin 990 -> 657 bytes
 tests/file_filter/test.cfg      |   1 +
 tests/file_filter_test.in       |   7 +-
 13 files changed, 253 insertions(+), 3 deletions(-)
 create mode 100644 grub-core/io/zstdio.c
 create mode 100644 tests/file_filter/file.zstd
 create mode 100644 tests/file_filter/file.zstd.sig


base-commit: 6b5c671d35b1b84468da1fef2b9bd8e05682f7e6
--
2.47.3

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 1/4] io: Implement zstdio decompression

[Logan Gunthorpe via Grub-devel]

Add zstd based io decompression.

Based largely on the existing xzio, implement the same features using
the zstd library already included in the project.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
---
 Makefile.util.def           |   1 +
 grub-core/Makefile.core.def |   7 ++
 grub-core/io/zstdio.c       | 237 ++++++++++++++++++++++++++++++++++++
 include/grub/file.h         |   3 +-
 4 files changed, 247 insertions(+), 1 deletion(-)
 create mode 100644 grub-core/io/zstdio.c

diff --git a/Makefile.util.def b/Makefile.util.def
index 038253b37a42..74786177f908 100644
--- a/Makefile.util.def
+++ b/Makefile.util.def
@@ -162,6 +162,7 @@ library = {
   common = grub-core/io/gzio.c;
   common = grub-core/io/xzio.c;
   common = grub-core/io/lzopio.c;
+  common = grub-core/io/zstdio.c;
   common = grub-core/kern/ia64/dl_helper.c;
   common = grub-core/kern/arm/dl_helper.c;
   common = grub-core/kern/arm64/dl_helper.c;
diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
index a729cb6a8e61..a2635d61b6e8 100644
--- a/grub-core/Makefile.core.def
+++ b/grub-core/Makefile.core.def
@@ -2512,6 +2512,13 @@ module = {
   cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/minilzo -DMINILZO_HAVE_CONFIG_H';
 };
 
+module = {
+  name = zstdio;
+  common = io/zstdio.c;
+  cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/zstd';
+  cflags='-Wno-unreachable-code';
+};
+
 module = {
   name = testload;
   common = commands/testload.c;
diff --git a/grub-core/io/zstdio.c b/grub-core/io/zstdio.c
new file mode 100644
index 000000000000..01a5a45587a7
--- /dev/null
+++ b/grub-core/io/zstdio.c
@@ -0,0 +1,237 @@
+/* zstdio.c - decompression support for zstd */
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2010  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <grub/err.h>
+#include <grub/mm.h>
+#include <grub/misc.h>
+#include <grub/file.h>
+#include <grub/fs.h>
+#include <grub/dl.h>
+
+GRUB_MOD_LICENSE ("GPLv3+");
+
+#include "zstd.h"
+
+#define STREAM_HEADER_SIZE 16
+
+struct grub_zstdio
+{
+  grub_file_t file;
+  ZSTD_DCtx *dctx;
+  grub_size_t insize;
+  grub_size_t outsize;
+
+  ZSTD_outBuffer output;
+  ZSTD_inBuffer input;
+
+  grub_off_t saved_offset;
+  grub_uint8_t bufs[];
+};
+
+typedef struct grub_zstdio *grub_zstdio_t;
+static struct grub_fs grub_zstdio_fs;
+
+static int
+test_header (grub_file_t file)
+{
+  grub_zstdio_t zstdio = file->data;
+  size_t zret;
+
+  zstdio->input.pos = 0;
+  zstdio->output.pos = 0;
+  zstdio->output.size = zstdio->outsize;
+  zstdio->input.size = grub_file_read (zstdio->file, zstdio->bufs,
+                                       STREAM_HEADER_SIZE);
+  if (zstdio->input.size != STREAM_HEADER_SIZE)
+    return 0;
+
+  zret = ZSTD_decompressStream (zstdio->dctx, &zstdio->output, &zstdio->input);
+  if (ZSTD_isError (zret))
+    return 0;
+
+  return 1;
+}
+static grub_file_t
+grub_zstdio_open (grub_file_t io, enum grub_file_type type)
+{
+  grub_file_t file;
+  grub_zstdio_t zstdio;
+
+  if (type & GRUB_FILE_TYPE_NO_DECOMPRESS)
+    return io;
+
+  file = (grub_file_t) grub_zalloc (sizeof (grub_file_t));
+  if (file == NULL)
+    return 0;
+
+  zstdio = grub_zalloc (sizeof (grub_zstdio_t) + ZSTD_DStreamInSize () +
+                        ZSTD_DStreamOutSize ());
+  if (zstdio == NULL)
+    {
+      grub_free (file);
+      return 0;
+    }
+
+  zstdio->file = io;
+  zstdio->insize = ZSTD_DStreamInSize ();
+  zstdio->outsize = ZSTD_DStreamOutSize ();
+  zstdio->input.src = zstdio->bufs;
+  zstdio->output.dst = &zstdio->bufs[zstdio->insize];
+
+  file->device = io->device;
+  file->data = zstdio;
+  file->fs = &grub_zstdio_fs;
+  file->size = GRUB_FILE_SIZE_UNKNOWN;
+  file->not_easily_seekable = 1;
+
+  if (grub_file_tell (zstdio->file) != 0)
+    grub_file_seek (zstdio->file, 0);
+
+  zstdio->dctx = ZSTD_createDCtx ();
+  if (zstdio->dctx == NULL)
+    {
+      grub_free (file);
+      grub_free (zstdio);
+      return 0;
+    }
+
+  if (!test_header (file))
+    {
+      grub_errno = GRUB_ERR_NONE;
+      grub_file_seek (io, 0);
+      ZSTD_freeDCtx (zstdio->dctx);
+      grub_free (zstdio);
+      grub_free (file);
+
+      return io;
+    }
+
+  return file;
+}
+
+static grub_ssize_t
+grub_zstdio_read (grub_file_t file, char *buf, grub_size_t len)
+{
+  grub_zstdio_t zstdio = file->data;
+  grub_ssize_t ret = 0;
+  grub_ssize_t readret;
+  grub_off_t current_offset;
+  grub_size_t zret;
+
+  /* If seek backward need to reset decoder and start from beginning of file. */
+  if (file->offset < zstdio->saved_offset)
+    {
+      ZSTD_initDStream (zstdio->dctx);
+      zstdio->input.pos = 0;
+      zstdio->input.size = 0;
+      zstdio->output.pos = 0;
+      zstdio->saved_offset = 0;
+      grub_file_seek (zstdio->file, 0);
+    }
+
+  current_offset = zstdio->saved_offset;
+
+  while (len > 0)
+    {
+      zstdio->output.size = file->offset + ret + len - current_offset;
+      if (zstdio->output.size > zstdio->outsize)
+        zstdio->output.size = zstdio->outsize;
+      if (zstdio->input.pos == zstdio->input.size)
+        {
+	  readret = grub_file_read (zstdio->file, zstdio->bufs,
+	                            zstdio->insize);
+	  if (readret < 0)
+	    return -1;
+
+	  zstdio->input.size = readret;
+	  zstdio->input.pos = 0;
+	}
+
+      zret = ZSTD_decompressStream (zstdio->dctx, &zstdio->output,
+                                    &zstdio->input);
+      if (ZSTD_isError (zret))
+	{
+	  grub_error (GRUB_ERR_BAD_COMPRESSED_DATA,
+		      N_("zstd file corrupted or unsupported block options"));
+	  return -1;
+        }
+
+      grub_off_t new_offset = current_offset + zstdio->output.pos;
+
+      /* Store first chunk of data in buffer.  */
+      if (file->offset <= new_offset)
+        {
+	  grub_size_t delta = new_offset - (file->offset + ret);
+	  grub_memmove (buf, (grub_uint8_t *) zstdio->output.dst +
+                        (zstdio->output.pos - delta),
+                        delta);
+	  len -= delta;
+	  buf += delta;
+	  ret += delta;
+	}
+	current_offset = new_offset;
+
+	zstdio->output.pos = 0;
+
+	if (zstdio->input.pos == 0 && zstdio->output.pos == 0)
+	  break;
+    }
+
+  if (ret >= 0)
+    zstdio->saved_offset = file->offset + ret;
+
+  return ret;
+}
+
+/* Release everything, including the underlying file object.  */
+static grub_err_t
+grub_zstdio_close (grub_file_t file)
+{
+  grub_zstdio_t zstdio = file->data;
+
+  ZSTD_freeDCtx (zstdio->dctx);
+
+  grub_file_close (zstdio->file);
+  grub_free (zstdio);
+
+  /* Device must not be closed twice.  */
+  file->device = 0;
+  file->name = 0;
+  return grub_errno;
+}
+
+static struct grub_fs grub_zstdio_fs = {
+  .name = "zstdio",
+  .fs_dir = 0,
+  .fs_open = 0,
+  .fs_read = grub_zstdio_read,
+  .fs_close = grub_zstdio_close,
+  .fs_label = 0,
+  .next = 0
+};
+
+GRUB_MOD_INIT (zstdio)
+{
+  grub_file_filter_register (GRUB_FILE_FILTER_ZSTDIO, grub_zstdio_open);
+}
+
+GRUB_MOD_FINI (zstdio)
+{
+  grub_file_filter_unregister (GRUB_FILE_FILTER_ZSTDIO);
+}
diff --git a/include/grub/file.h b/include/grub/file.h
index 16a4b7d26437..d51834e6abd2 100644
--- a/include/grub/file.h
+++ b/include/grub/file.h
@@ -190,9 +190,10 @@ typedef enum grub_file_filter_id
     GRUB_FILE_FILTER_GZIO,
     GRUB_FILE_FILTER_XZIO,
     GRUB_FILE_FILTER_LZOPIO,
+    GRUB_FILE_FILTER_ZSTDIO,
     GRUB_FILE_FILTER_MAX,
     GRUB_FILE_FILTER_COMPRESSION_FIRST = GRUB_FILE_FILTER_GZIO,
-    GRUB_FILE_FILTER_COMPRESSION_LAST = GRUB_FILE_FILTER_LZOPIO,
+    GRUB_FILE_FILTER_COMPRESSION_LAST = GRUB_FILE_FILTER_ZSTDIO,
   } grub_file_filter_id_t;
 
 typedef grub_file_t (*grub_file_filter_t) (grub_file_t in, enum grub_file_type type);
-- 
2.47.3


_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 2/4] tests/file_filter: Regenerate gpg keys

[Logan Gunthorpe via Grub-devel]

The 'keys' file is not a valid GPG secret key so it is not possible to
generate new signatures.

Create a new key and use `gpg --export-secret-key` to export the key
and `gpg --export` to export the public key. Then resign all the
signatures with `gpg --detach-sign`.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
---
 tests/file_filter/file.gz.sig   | Bin 96 -> 310 bytes
 tests/file_filter/file.lzop.sig | Bin 96 -> 310 bytes
 tests/file_filter/file.xz.sig   | Bin 96 -> 310 bytes
 tests/file_filter/keys          | Bin 994 -> 1308 bytes
 tests/file_filter/keys.pub      | Bin 990 -> 657 bytes
 5 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/tests/file_filter/file.gz.sig b/tests/file_filter/file.gz.sig
index 602e6187e22d7fc288a52ac572f38cc1066083cb..27cdcff2dac7d94c8a16468431054cc15b2c9cea 100644
GIT binary patch
literal 310
zcmV-60m=S}0W$;u0SW*e79j-t9c_qHaZBEqV(=dtEpNLINa*JU0%+3UmH-L~5E?CS
zyAMd{=bteM0Eh|rq-p-LiIvH-gwFCsL}^W9@Rh5aTnE_XNOoBFXh7bRxx!`#A<KL=
zkZOrXYM`@1e7PtCfp;^frnpi_-vBFf7IyLwhN``kZxp2p3das{ch=_aW1eC*;(@@_
zE=AZ~UBu-EK+yY+r|e=M=dhTyB;^ASNm#|vu>-OAZEX4NZ~=M6V3X+7<*;rgVRfyr
z8Lm7k0T9UYkIWu11Ac3#6y(_k#56BNOuv?{$9LqA<>a}%mZiQ%&F7=oSOzau(YL6A
zFLB;)=bq<$G0#NZo>Mon`-+Ot2p?@nQGn^qk6VBambn-bzmQ@jjqqH(l7LJ4@2aN8
ITqy*<+#TkZDF6Tf

literal 96
zcmV-m0H6PeUIYLU2ml5J0#t#n2LK8Q5M~MT@8nv<3h9^t|7f?gmaE1LNh!Sm;mO#P
zdLo*tTpkl+s!TB^f%Fh1{Q&^Tlz2`!Q4G<oG#0}qGOvRW#+GAgRi%IW@C=M*jB>iq
Ct|zAe

diff --git a/tests/file_filter/file.lzop.sig b/tests/file_filter/file.lzop.sig
index 7c68dcf931e438973f5c4bfe9a0f9d5ca68fbacc..05949ae7287ad7e102d2ada99284df99de52df8b 100644
GIT binary patch
literal 310
zcmV-60m=S}0W$;u0SW*e79j-t9c_qHaZBEqV(=dtEpNLINa*JU0%+3Ur2q;E5E?CS
zyAMd{=L9wf|5>JA(bK%c)*r6O*tJpvT^U(2FXqi_npuY=P`-}+{+F1a=QO92UWu=~
zeuC|%dIu1}w&D=S(!LK)sKS>W<^PhzyqF!p_ugD%B>IBVpY~YLkzMWYw%Y+4U)w2S
zo`%wlXlQO3_YS+Rl^)f&=>%~9cTa55Z1bDIh@%TIDFX%#t84-Ski|Y9iShWr#C<4j
z;g8z<N73<4r+A7(j21<0Gl>JouOM&dBW^7GQ|wOJ7&d(bAmkLlJb%qbv7)%1Dchll
zZo>64iqi1{uyFha3{)TW&HPgUCOu;F>jo|~r;|A)iq#{V0`lZmjjNShihg>4#gi8|
IC=bA4)%ydMUjP6A

literal 96
zcmV-m0H6PeUIYLU2ml5J0#t#n5C94Z5M~MT@8nv<3Q2ST|3TGyOWf``exmR@&j2XX
z!ZeX0$UFCBibkPCfL8Tos{sJ3$5~Z-z|*E$2u>jC(Bu@f@NKNp%ec7JvFb-Lz3O)*
CVl0pV

diff --git a/tests/file_filter/file.xz.sig b/tests/file_filter/file.xz.sig
index 57569242e4742cdfb929307ec19440cf91f49e9a..9633a2fa15b2f0d3584afacf84d55a8bd8011fab 100644
GIT binary patch
literal 310
zcmV-60m=S}0W$;u0SW*e79j-t9c_qHaZBEqV(=dtEpNLINa*JU0%+3UumB1P5E?CS
zyAMd{=XbRS|5<}>|Ee1TE0F9z)>?}P(_Z&yZ7cm#o@dkY_{O6rB<#(vL2LQY_ihb>
zUo3YVw(f+y;({{CCGMS&G}n0Vu28A*dC1faO>&3ad-EYTFviW8pB<Ip!fN!`v<g`J
zI2c0pPp|rEJ{Fcw5IiW9iTl)AlszP}_ov4nDgLdkgmIQ9)s+(m_qRG1Uq|?KS<v#Y
zZWlotcNFF(D&$dSsLxt69`)m_TF^XARjH3#g)Dj-Aow5B&DR#Ly<*iptG!s;daCan
zkg}ihL^Z<U66jMB`;N1Im6hv8-(&xWywcny_?z!bz#@?dZ;eE()werlDT+eL%tZl6
IwQRwI0x`y!o&W#<

literal 96
zcmV-m0H6PeUIYLU2ml5J0#t#n3jhiU5M~MT@8nv<3OS$v{x;4Uj6+!wt^-rP*&N$|
z+5zKQ7vOO`?M~+y=au4FasdGNRgV;Q;4TNc&v05)-JLjLl54~a(Q}0wbmOZ2ne0%w
Ci77V#

diff --git a/tests/file_filter/keys b/tests/file_filter/keys
index 1afa71382364060d08bb0c28e36a1ff98d585fe8..67f29b88e338ff6d34527f3feb7d74479b19845c 100644
GIT binary patch
delta 1279
zcmV<b1OWTu2b>Cjl>?XrXwu%?0SExHqzXGq&M-E@<DA{c!5L+o%tGw=EvmW0{ulb*
zIsg9b$q63=4Plz!P}8SvTQz^=J=%7-V%C)4?`Wx(=0~Q<azH%lPT2plgle_QuuZ~+
za(DUDP0)fsj&&s;nOBxE;2cGd`lcHn_H446gt$!>jNmALLuc;PkuiAKz$9pl5B*My
zErDt-7un4%(mMTB7=i|rVlt{d$@Ep}6n<(4K>QbQ8+R~hbEcEA2LFMcP=XX0hbTlQ
zd6SD;7SQD)j28lu@AJklal8J?@IIqDpr6L4mMx=8$UlS)Dx1vZvqeLB9@O|F?Wa?k
z+@X3Gxg`64uH-*nxDI-n?A<T?_W%(A00968{yd-H{Rlw&Davp5PieRXhi`NKj&BzT
zW^A@Gt0;M-fBqS#r^ldW>}odx?Lqc`EPUtW*#B@tgjnUqt}hVHW;5M8A+a926XT$M
znq5Qz&#wFq?NFhL&UsuPGfxWoCv}^t+>NC%bLk&{_W6<M(x(os7IAZywEdi%o_A=K
zvWTq4DTllxeKc1PR*+%k_I!fD%ZW+&HXXtM1wx18AkY5hg`uv=*DT7R#9ggq;dDJm
zJu%u>T0lZ1X4>R9QTC`0jp61Dvz1-`@a;IsHe=s5s3n;a6;4<0_&Mb9tB^TrNxsVV
zF8)1#U%<Juz*0mH0ZvleL9J+m<wNo5%5#YX0MfAv7y+AC9~{+_JUY`2rXM%MpX~bU
z#`Mg2_wCj_8TrtD9-Zj=>2v%C%ksW&-bDhCb_I^2eE>SSDko~H@Z8ZI$g|WR%05T~
z@bke>UT}`o?U5=FYdAX39K7!sPO5k}-nkfm4fpaAYg>g^53&*Ic*yfIR`7C@s1HEk
z7F7fQ*$#`j0u)ZEB5`5w0spB7DW@5m>OY#odJy5W$-N0100-tkDWkj1T;!Ux6(}`&
z2k)4NUUc^6YM4GCk!?`N3aoZu(-q;#oF$%Y5d_u*Zl@_CSfJID2F(Gs+94w}b?k|M
z95%8WYh9On|Kty$=pk;;4^VmcthBp%>E$IF8w3Eb`~;kP!1Z#bR5Mk9rpAoBav5OV
zwzUm3&{>4GfAbhR&hY0}vA#;&xD=ZJ%Qzfu4}dWkMSqo^DCXERpfI?7LgRow<Qn3~
z6ouYY2g;lcGZgC{iqiK_z~q9RtCVmeO9I<vkcyVq=y*uwB|A1eQxwwMAUAs3fprWn
zcB)7SLkW{H12BJy0Zs%H0SW*(79j-t9c_qHaZBEqV(=dtEpNLINa*JU0%+3S+yWZ|
z1q%rX2Lc8a3JC}c0t6NU0|5da0Rk6*0162Z8ZB?T4@l_eC?*I0N9CQPYLiO=znFS{
z_U;SEA!B<aJF(FxuU^sHHXaDi%7EP;q*`e`pIW4|XKR1j0S;Ig3p6<7X81rkgH}c<
z3>`6g=Ck}L4Fz9n2J%B57Hi+vtU``oc{z}nRI5>4x?1*(&a@ZE8L8j@ZIlDe7b~#!
zx&+{(f*_j-6|rsM9w5Mj61TXLL8e)<4=MbeyY;s;#|j(U;<T86Ij;Uac{J2S%sRW2
zNaqg=joVWII$B^2Bd>@(Z)T3bkIi9A{@dOOi43*5N@X-HgTkw~nF`v0d2`U`E^iJ#
puwkM8&%e5|8Tg=)pBmQLPAdn2IgOCR{iv<yGPhAWkjZgiHVox=WKRG9

delta 963
zcmV;!13dhk3gQQUnFB5aRDrEf5eNXak$CR!^i^qM-2=)M(8-`vDQdcaw{#q%DO)*>
zd7*;=eq+{7*`IvWWAkd)@wV4<6dF6}$Q$yhy|Kltkvt90iqlY|7JP5nf#rM#2DtCu
zh;9+E;AJ;0rBpA9l>pwU9}N;t?D?~oCe9{=*0AjC?Qr>j;bOq!5$Yuocx8582a-E4
zdcJPf3L`Si&UU?(3z3{?RFoaWXS7$|2nd{_-eJlv%`a=7@EDk#fE)EI@YVq?Xao9I
zbMOvW0x7t$Dprmb0WH@nYr%3ekt!nn7e-d1%82Z;yj>wC039nL$s34N?EW!-Av$so
z!`DA-$eIU#*?$H0M~=H>WfW$`PXPe<{-J%q5gI}NA3-`^az(7r5xTAwL#+mBz|^?@
zI}fu5{wxKxDZP{FUck6;a(sE4bpk=Wc0wH$W)=p%kG+MS!RmG~OWi4<;3=Oo{5BqH
zBg~!VR|NBhf=d&J|B$j25JzIBp9Kld$l#yDvuz=NdN^nA@=cwz&`Ezpv7R;6gk|My
z_?I8#%?Z*v0G=T3S|-k)ND-11n=+U)J=DUmZ`*NlWO1qRl1d|e3jZ@ioyPN?bxzIt
zo0ltfBGhMj*HF^AUdvgzSm>LzM75fVG>0caAHZcq9dkq@E1ccwLg7IkAD-Uj9eRWe
zy-8PpA1OO@l`ln1(1<?>=R<R<m1OgHqS<peDd6U@+EGe;6O#&c(k8xOIqQ+{>p8vO
zhuG2w{xno40uuPIPG?MWCC0Q2-{V6FS{Cz}EfnlhM@>u=!-(=uFastW7Q++Frx~}7
zMgL46c$uH4S-c-UuJEc>o1Z`}^Dg{%_Ev0vu#BwZ0~=9rNw3S14`YP`A(m%EqsL=|
zcV{&Z6f~)GHeZAo+I<Y(1V_kG%@0C=hL{bMW{A>DlC?FaI?=82AmOaOiEXonNmLLN
zhNI5T)a!L9eJdZ0gpLW2y;mIa;GlWR;;#iv$ja>x<l4R!%Nq(O1$G7#57e5xbu`mA
zB7MAC2>BSV4sVvx=}>7h*2=CQ4Ak3YL#Z1|T%6Q`uC%j;uNS@9<W8H^kf$-bqMTm8
zaPqU0n*%U^h<gMR5eNVy1p-uotxy6R0|yHU2nPcK0R|Na0tpHW1Qr4V0RkQY0vCV)
z3JDNq3G(mcTEq(MOaTDr(EwV(66?31G$X)Yc}L#-f1V|Ihs%o{ohXCXws^Jx{xf`J
lh8hbK7OiUdLcWgg9!aO2P9rt8vrqb7BxSl5Vz2@N00X7Yxq1Kq

diff --git a/tests/file_filter/keys.pub b/tests/file_filter/keys.pub
index 61d4e7a7bd3adf6c66a69b4e7bd10bdf7c7877c3..44418a6f4fcb2d013abe723a733bbb073d03d828 100644
GIT binary patch
delta 623
zcmV-#0+9XQ2ayGTnE?$1Xwu%?0SExHqzXGq&M-E@<DA{c!5L+o%tGw=EvmW0{ulb*
zIsg9b$q63=4Plz!P}8SvTQz^=J=%7-V%C)4?`Wx(=0~Q<azH%lPT2plgle_QuuZ~+
za(DUDP0)fsj&&s;nOBxE;2cGd`lcHn_H446gt$!>jNmALLuc;PkuiAKz$9pl5B*My
zErDt-7un4%(mMTB7=i|rVlt{d$@Ep}6n<(4K>QbQ8+R~hbEcEA2LFMcP=XX0hbTlQ
zd6SD;7SQD)j28lu@AJklal8J?@IIqDpr6L4mMx=8$UlS)Dx1vZvqeLB9@O|F?Wa?k
z+@X3Gxg`4%uH-*nxDI-n?A<T?_W%(A00EOR12BJy0Zs%H0SW*(79j-t9c_qHaZBEq
zV(=dtEpNLINa*JU0%+3S+yWZ|1q%rX2Lc8a3JC}c0t6NU0|5da0Rk6*0162Z8ZB?T
z4@l_eC?*I0N9CQPYLiO=znFS{_U;SEA!B<aJF(FxuU^sHHXaDi%7EP;q*`e`pIW4|
zXKR1j0S;Ig3p6<7X81rkgH}c<3>`6g=Ck}L4Fz9n2J%B57Hi+vtU``oc{z}nRI5>4
zx?1*(&a@ZE8L8j@ZIlDe7b~#!x&+{(f*_j-6|rsM9w5Mj61TXLL8e)<4=MbeyY;s;
z#|j(U;<T86Ij;Uac{J2S%sRW2Naqg=joVWII$B^2Bd>@(Z)T3bkIi9A{@dOOi43*5
zN@X-HgTkw~nF`v0d2`U`E^iJ#uwkM8&%e5|8Tg=)pBmQLPAdn2IgOCR{iv<yGPhAW
JkjZgiHVl>p9m4<s

delta 958
zcmV;v13~<e1>OgLnFB5aRDrEf5eNXak$CR!^i^qM-2=)M(8-`vDQdcaw{#q%DO)*>
zd7*;=eq+{7*`IvWWAkd)@wV4<6dF6}$Q$yhy|Kltkvt90iqlY|7JP5nf#rM#2DtCu
zh;9+E;AJ;0rBpA9l>pwU9}N;t?D?~oCe9{=*0AjC?Qr>j;bOq!5$Yuocx8582a-E4
zdcJPf3L`Si&UU?(3z3{?RFoaWXS7$|2nd{_-eJlv%`a=7@EDk#fE)EI@YVq?Xao9I
zbMOvW0x7t$Dprmb0WH@nYr%3ekt!nn7e-d1%82Z;yj>wC039nL$s34N?EW!-Av$so
z!`DA-$eIU#*?$H0M~=H>WfW$`PXPe<{-J%q5gI}NA3-`^az(7r5xTAwL#+mBz|^?@
zI}fu5{wxKxDZP{FUck6;a(sE4bpk=Wc0wH$W)=p%kG+MS!RmG~OWi4<;3=Oo{5BqH
zBg~!VR|NBhf=d&J|B$j25JzIBp9Kld$l#yDvuz=NdN^nA@=cwz&`Ezpv7R;6gk|My
z_?I8#%?Z*v0G=T3S|-k)ND-11n=+U)J=DUmZ`*NlWO1qRl1d|e3jZ@ioyPN?bxzIt
zo0ltfBGhMj*HF^AUdvgzSm>LzM75fVG>0caAHZcq9dkq@E1ccwLg7IkAD-Uj9eRWe
zy-8PpA1OO@l`ln1(1<?>=R<R<m1OgHqS<peDd6U@+EGe;6O#&c(k8xOIqQ+{>p8vO
zhuG2w{xno40uuPIPG?MWCC0Q2-{V6FS{Cz}EfnlhM@>u=!-(=uFastW7Q++Frx~}7
zMgL46c$uH4S-c-UuJEc>o1Z`}^Dg{%_Ev0vu#BwZ0~=9rNw3S14`YP`A(m%EqsL=|
zcV{&Z6f~)GHeZAo+I<Y(1V_kG%@0C=hL{bMW{A>DlC?FaI?=82AmOaOiEXonNmLLN
zhNI5T)a!L9eJdZ0gpLW2y;mIa;GlWR;;#iv$ja>x<l4R!%Nq(O1$G7#57e5xbu`mA
zB7MAC2>BSV4sVvx=}>7h*2=CQ4Ak3YL#Z1|T%6Q`uC%j;uNS@9<W8H^kf$-bqMTm8
zaPqU05CJfKh<gMR5eNVy1p-uotxy6R0|yHU2nPcK0R|Na0tpHW1Qr4V0RkQY0vCV)
z3JDNq3G(mcTEq(MOaTDr(EwV(66?31G$X)Yc}L#-f1V|Ihs%o{ohXCXws^Jx{xf`J
gh8hbK7OiUdLcWgg9!aO2P9rt8vrqb7BxSl5Vsmh~AOHXW

-- 
2.47.3


_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 3/4] tests/file_filter: Add zstd test file

[Logan Gunthorpe via Grub-devel]

Add a file.zstd similar to the other compression methods and generate
a gpg signature with `gpg --detach-sign`

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
---
 tests/file_filter/file.zstd     | Bin 0 -> 26 bytes
 tests/file_filter/file.zstd.sig | Bin 0 -> 310 bytes
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/file_filter/file.zstd
 create mode 100644 tests/file_filter/file.zstd.sig

diff --git a/tests/file_filter/file.zstd b/tests/file_filter/file.zstd
new file mode 100644
index 0000000000000000000000000000000000000000..6ea24ccf8390f456a63f15e1d0f473d0a0ebc75e
GIT binary patch
literal 26
hcmdPcs{fZIB9no^BQ+-{Uq_*|IJHQTYg5;2b^v!n2`2ym

literal 0
HcmV?d00001

diff --git a/tests/file_filter/file.zstd.sig b/tests/file_filter/file.zstd.sig
new file mode 100644
index 0000000000000000000000000000000000000000..ab101bb6e883a09dd41ab8fe348cf8a9ce5843c7
GIT binary patch
literal 310
zcmV-60m=S}0W$;u0SW*e79j-t9c_qHaZBEqV(=dtEpNLINa*JU0%+3Wr2q;E5E?CS
zyAMd{=f%Sa0D;xR-EXs|$ZnV8EKm8`Acg4~^9(E4y~^NzaINTQYg%<WdW2aqV(Ij1
zIfbrviC5H^>`Gg2<@;Fpno=K?r5u)5TM|R$F@ovn27&G8fg-pVRvhu}ovsacO<}u{
z25i6uvgdqAkCTR$d4F3pccusj##rB9AXS&mdn)t-MYSd?m-HI2<^5Isfdl;1i)xUX
zxKdFt^#|YWXW1hao$90KM#Cx?ic2Sv^Si30&h0m4<=XFvh;2}vaiGu|h={rI3w*pU
zi7KP2UVUACi2ubxOq2^og4khH%5%bHKw9Mcwh)Sob&Oz#ur6US1>g2>8<kESSR<Uw
I4A}zsN@DJozW@LL

literal 0
HcmV?d00001

-- 
2.47.3


_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

[PATCH 4/4] tests/file_filter: Add zstd tests

[Logan Gunthorpe via Grub-devel]

Test zstd decompression in the same way that other decompressors are
tested.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
---
 tests/file_filter/test.cfg | 1 +
 tests/file_filter_test.in  | 7 +++++--
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tests/file_filter/test.cfg b/tests/file_filter/test.cfg
index 4308aaca52ba..ec4d10ab4b23 100644
--- a/tests/file_filter/test.cfg
+++ b/tests/file_filter/test.cfg
@@ -3,4 +3,5 @@ set check_signatures=enforce
 cat /file.gz
 cat /file.xz
 cat /file.lzop
+cat /file.zstd
 set check_signatures=
diff --git a/tests/file_filter_test.in b/tests/file_filter_test.in
index ed6abcb5af43..18324f1a0933 100644
--- a/tests/file_filter_test.in
+++ b/tests/file_filter_test.in
@@ -19,14 +19,15 @@ grubshell=@builddir@/grub-shell
 
 . "@builddir@/grub-core/modinfo.sh"
 
-filters="gzio xzio lzopio pgp"
+filters="gzio xzio lzopio zstdio pgp"
 modules="cat mpi"
 
 for mod in $(cut -d ' ' -f 2 "@builddir@/grub-core/crypto.lst"  | sort -u); do
     modules="$modules $mod"
 done
 
-for file in file.gz file.xz file.lzop file.gz.sig file.xz.sig file.lzop.sig keys.pub; do
+for file in file.gz file.xz file.lzop file.zstd file.gz.sig file.xz.sig \
+            file.lzop.sig file.zstd.sig keys.pub; do
     files="$files /$file=@srcdir@/tests/file_filter/$file"
 done
 
@@ -35,6 +36,8 @@ result="Hello, user!
 
 Hello, user!
 
+Hello, user!
+
 Hello, user!"
 
 out="$("${grubshell}" --modules="$modules $filters" --files="$files" "@srcdir@/tests/file_filter/test.cfg")"
-- 
2.47.3


_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [PATCH 0/4] Add zstdio support for booting Trixie kernels in Xen

[Logan Gunthorpe via Grub-devel]

Sorry, I forgot to add the v2 flag in the PATCH subject. Hopefully it's
not too confusing.

Logan

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [PATCH 1/4] io: Implement zstdio decompression

[Daniel Kiper]

On Tue, Oct 14, 2025 at 02:14:18PM -0600, Logan Gunthorpe via Grub-devel wrote:
> Add zstd based io decompression.
>
> Based largely on the existing xzio, implement the same features using
> the zstd library already included in the project.
>
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> ---
>  Makefile.util.def           |   1 +
>  grub-core/Makefile.core.def |   7 ++
>  grub-core/io/zstdio.c       | 237 ++++++++++++++++++++++++++++++++++++
>  include/grub/file.h         |   3 +-
>  4 files changed, 247 insertions(+), 1 deletion(-)
>  create mode 100644 grub-core/io/zstdio.c
>
> diff --git a/Makefile.util.def b/Makefile.util.def
> index 038253b37a42..74786177f908 100644
> --- a/Makefile.util.def
> +++ b/Makefile.util.def
> @@ -162,6 +162,7 @@ library = {
>    common = grub-core/io/gzio.c;
>    common = grub-core/io/xzio.c;
>    common = grub-core/io/lzopio.c;
> +  common = grub-core/io/zstdio.c;
>    common = grub-core/kern/ia64/dl_helper.c;
>    common = grub-core/kern/arm/dl_helper.c;
>    common = grub-core/kern/arm64/dl_helper.c;
> diff --git a/grub-core/Makefile.core.def b/grub-core/Makefile.core.def
> index a729cb6a8e61..a2635d61b6e8 100644
> --- a/grub-core/Makefile.core.def
> +++ b/grub-core/Makefile.core.def
> @@ -2512,6 +2512,13 @@ module = {
>    cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/minilzo -DMINILZO_HAVE_CONFIG_H';
>  };
>
> +module = {
> +  name = zstdio;
> +  common = io/zstdio.c;
> +  cppflags = '-I$(srcdir)/lib/posix_wrap -I$(srcdir)/lib/zstd';
> +  cflags='-Wno-unreachable-code';
> +};
> +
>  module = {
>    name = testload;
>    common = commands/testload.c;
> diff --git a/grub-core/io/zstdio.c b/grub-core/io/zstdio.c
> new file mode 100644
> index 000000000000..01a5a45587a7
> --- /dev/null
> +++ b/grub-core/io/zstdio.c
> @@ -0,0 +1,237 @@
> +/* zstdio.c - decompression support for zstd */
> +/*
> + *  GRUB  --  GRand Unified Bootloader
> + *  Copyright (C) 2010  Free Software Foundation, Inc.

s/2010/2025/

> + *  GRUB is free software: you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License as published by
> + *  the Free Software Foundation, either version 3 of the License, or
> + *  (at your option) any later version.
> + *
> + *  GRUB is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <grub/err.h>
> +#include <grub/mm.h>
> +#include <grub/misc.h>
> +#include <grub/file.h>
> +#include <grub/fs.h>
> +#include <grub/dl.h>
> +
> +GRUB_MOD_LICENSE ("GPLv3+");
> +
> +#include "zstd.h"
> +
> +#define STREAM_HEADER_SIZE 16
> +
> +struct grub_zstdio

You do not need grub_ prefix if name is internal to the module.

> +{
> +  grub_file_t file;
> +  ZSTD_DCtx *dctx;
> +  grub_size_t insize;
> +  grub_size_t outsize;
> +
> +  ZSTD_outBuffer output;
> +  ZSTD_inBuffer input;
> +
> +  grub_off_t saved_offset;
> +  grub_uint8_t bufs[];
> +};
> +

Please drop this empty line...

> +typedef struct grub_zstdio *grub_zstdio_t;

... and add one here... :-)

> +static struct grub_fs grub_zstdio_fs;
> +
> +static int

Please use bool, including true/false, instead of int.

> +test_header (grub_file_t file)
> +{
> +  grub_zstdio_t zstdio = file->data;
> +  size_t zret;
> +
> +  zstdio->input.pos = 0;
> +  zstdio->output.pos = 0;
> +  zstdio->output.size = zstdio->outsize;
> +  zstdio->input.size = grub_file_read (zstdio->file, zstdio->bufs,
> +                                       STREAM_HEADER_SIZE);
> +  if (zstdio->input.size != STREAM_HEADER_SIZE)
> +    return 0;
> +
> +  zret = ZSTD_decompressStream (zstdio->dctx, &zstdio->output, &zstdio->input);
> +  if (ZSTD_isError (zret))
> +    return 0;
> +
> +  return 1;
> +}

Missing empty line here...

> +static grub_file_t
> +grub_zstdio_open (grub_file_t io, enum grub_file_type type)
> +{
> +  grub_file_t file;
> +  grub_zstdio_t zstdio;
> +
> +  if (type & GRUB_FILE_TYPE_NO_DECOMPRESS)
> +    return io;
> +
> +  file = (grub_file_t) grub_zalloc (sizeof (grub_file_t));
> +  if (file == NULL)
> +    return 0;

s/0/NULL/

> +  zstdio = grub_zalloc (sizeof (grub_zstdio_t) + ZSTD_DStreamInSize () +
> +                        ZSTD_DStreamOutSize ());
> +  if (zstdio == NULL)
> +    {
> +      grub_free (file);
> +      return 0;

Ditto...

In general please use NULL instead of 0.

> +    }
> +
> +  zstdio->file = io;
> +  zstdio->insize = ZSTD_DStreamInSize ();
> +  zstdio->outsize = ZSTD_DStreamOutSize ();
> +  zstdio->input.src = zstdio->bufs;
> +  zstdio->output.dst = &zstdio->bufs[zstdio->insize];
> +
> +  file->device = io->device;
> +  file->data = zstdio;
> +  file->fs = &grub_zstdio_fs;
> +  file->size = GRUB_FILE_SIZE_UNKNOWN;
> +  file->not_easily_seekable = 1;
> +
> +  if (grub_file_tell (zstdio->file) != 0)
> +    grub_file_seek (zstdio->file, 0);

grub_file_seek() may fail. Please always check for errors.

> +  zstdio->dctx = ZSTD_createDCtx ();
> +  if (zstdio->dctx == NULL)
> +    {
> +      grub_free (file);
> +      grub_free (zstdio);
> +      return 0;

Again, s/0/NULL/...

> +    }
> +
> +  if (!test_header (file))

if (test_header (file) == false)

> +    {
> +      grub_errno = GRUB_ERR_NONE;
> +      grub_file_seek (io, 0);
> +      ZSTD_freeDCtx (zstdio->dctx);
> +      grub_free (zstdio);
> +      grub_free (file);
> +
> +      return io;
> +    }
> +
> +  return file;
> +}
> +
> +static grub_ssize_t
> +grub_zstdio_read (grub_file_t file, char *buf, grub_size_t len)
> +{
> +  grub_zstdio_t zstdio = file->data;
> +  grub_ssize_t ret = 0;
> +  grub_ssize_t readret;
> +  grub_off_t current_offset;
> +  grub_size_t zret;
> +
> +  /* If seek backward need to reset decoder and start from beginning of file. */
> +  if (file->offset < zstdio->saved_offset)
> +    {
> +      ZSTD_initDStream (zstdio->dctx);
> +      zstdio->input.pos = 0;
> +      zstdio->input.size = 0;
> +      zstdio->output.pos = 0;
> +      zstdio->saved_offset = 0;
> +      grub_file_seek (zstdio->file, 0);

grub_file_seek() may fail...

> +    }
> +
> +  current_offset = zstdio->saved_offset;
> +
> +  while (len > 0)
> +    {
> +      zstdio->output.size = file->offset + ret + len - current_offset;

Is there any chance for overflow here? If yes then please use safe math
from include/grub/safemath.h.

> +      if (zstdio->output.size > zstdio->outsize)
> +        zstdio->output.size = zstdio->outsize;
> +      if (zstdio->input.pos == zstdio->input.size)
> +        {
> +	  readret = grub_file_read (zstdio->file, zstdio->bufs,
> +	                            zstdio->insize);
> +	  if (readret < 0)
> +	    return -1;
> +
> +	  zstdio->input.size = readret;
> +	  zstdio->input.pos = 0;
> +	}
> +
> +      zret = ZSTD_decompressStream (zstdio->dctx, &zstdio->output,
> +                                    &zstdio->input);
> +      if (ZSTD_isError (zret))
> +	{
> +	  grub_error (GRUB_ERR_BAD_COMPRESSED_DATA,
> +		      N_("zstd file corrupted or unsupported block options"));
> +	  return -1;
> +        }
> +
> +      grub_off_t new_offset = current_offset + zstdio->output.pos;

Please do not mix variable definitions with the code. Define all stuff
at the beginning of the function or block.

> +      /* Store first chunk of data in buffer.  */
> +      if (file->offset <= new_offset)
> +        {
> +	  grub_size_t delta = new_offset - (file->offset + ret);

OK, but missing empty line here.

> +	  grub_memmove (buf, (grub_uint8_t *) zstdio->output.dst +
> +                        (zstdio->output.pos - delta),

Safe math? In general please check all math in this patch and use safe
math macros where needed.

> +                        delta);
> +	  len -= delta;
> +	  buf += delta;
> +	  ret += delta;
> +	}
> +	current_offset = new_offset;
> +
> +	zstdio->output.pos = 0;
> +
> +	if (zstdio->input.pos == 0 && zstdio->output.pos == 0)
> +	  break;
> +    }
> +
> +  if (ret >= 0)
> +    zstdio->saved_offset = file->offset + ret;
> +
> +  return ret;
> +}

Daniel

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [PATCH 1/4] io: Implement zstdio decompression

[Logan Gunthorpe via Grub-devel]

Hi Daniel,

On 2025-10-15 09:52, Daniel Kiper wrote:
> On Tue, Oct 14, 2025 at 02:14:18PM -0600, Logan Gunthorpe via Grub-devel wrote:
>> +  while (len > 0)
>> +    {
>> +      zstdio->output.size = file->offset + ret + len - current_offset;
> 
> Is there any chance for overflow here? If yes then please use safe math
> from include/grub/safemath.h.

My read is that unless we are dealing with files greater than the two
offsets are 64bits and unlikely to ever see a file that doesn't fit.

len is going to be the size of a memory buffer and ret is going to be
less than or equal to ret. So it doesn't seem like an overflow is possible.

>> +	  grub_size_t delta = new_offset - (file->offset + ret);
>> +	  grub_memmove (buf, (grub_uint8_t *) zstdio->output.dst +
>> +                        (zstdio->output.pos - delta),
> 
> Safe math? In general please check all math in this patch and use safe
> math macros where needed.


output.pos must be less than outsize which is defined by
ZSTD_DStreamOutSize () and will be much less than even a 32bit type.

delta is by definition less than pos.

(new_offset - file->offset) must be less than len.

This code is very similar to code in xzio.c and did not need any safe
math functions.

Please let me know if you disagree, otherwise I'll send a new version
next week.

Thanks,

Logan


_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [PATCH 1/4] io: Implement zstdio decompression

[Daniel Kiper]

On Fri, Oct 17, 2025 at 05:23:48PM -0600, Logan Gunthorpe via Grub-devel wrote:
> Hi Daniel,
>
> On 2025-10-15 09:52, Daniel Kiper wrote:
> > On Tue, Oct 14, 2025 at 02:14:18PM -0600, Logan Gunthorpe via Grub-devel wrote:
> >> +  while (len > 0)
> >> +    {
> >> +      zstdio->output.size = file->offset + ret + len - current_offset;
> >
> > Is there any chance for overflow here? If yes then please use safe math
> > from include/grub/safemath.h.
>
> My read is that unless we are dealing with files greater than the two
> offsets are 64bits and unlikely to ever see a file that doesn't fit.

First of all, please remember that the GRUB can be compiled for 32-bit
targets. So, zstdio->output.size can be 32-bit...

> len is going to be the size of a memory buffer and ret is going to be
> less than or equal to ret. So it doesn't seem like an overflow is possible.

I think you meant "less than or equal to len"...

> >> +	  grub_size_t delta = new_offset - (file->offset + ret);
> >> +	  grub_memmove (buf, (grub_uint8_t *) zstdio->output.dst +
> >> +                        (zstdio->output.pos - delta),
> >
> > Safe math? In general please check all math in this patch and use safe
> > math macros where needed.
>
> output.pos must be less than outsize which is defined by
> ZSTD_DStreamOutSize () and will be much less than even a 32bit type.
>
> delta is by definition less than pos.
>
> (new_offset - file->offset) must be less than len.
>
> This code is very similar to code in xzio.c and did not need any safe
> math functions.

IIRC it predates safe math and it may not be (fully) correct..

> Please let me know if you disagree, otherwise I'll send a new version
> next week.

I am not saying I disagree. I rather have some reservations. However, if
you convince me it is not possible or it is very difficult to blow up
this code using malformed zstd archives than I am OK with it.

Daniel

PS I am going to cut GRUB 2.14~rc1 in second half of the week. So, it
   would be nice to have these patches before that...

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel

Re: [PATCH 1/4] io: Implement zstdio decompression

[Logan Gunthorpe via Grub-devel]

On 2025-10-20 10:16, Daniel Kiper wrote:
> On Fri, Oct 17, 2025 at 05:23:48PM -0600, Logan Gunthorpe via Grub-devel wrote:
>> My read is that unless we are dealing with files greater than the two
>> offsets are 64bits and unlikely to ever see a file that doesn't fit.
> 
> First of all, please remember that the GRUB can be compiled for 32-bit
> targets. So, zstdio->output.size can be 32-bit...

Yes, but zstdio->output.size is always less than the relatively small
buffer size returned by ZSTD_DStreamOutSize ().

The only values that could conceivably be larger than 32-bits are the
offsets within the file which are of type grup_off_t, which is a typedef
of grub_uint64_t and, by my read of the code, it looks like it should be
64-bits on all platforms.

>> len is going to be the size of a memory buffer and ret is going to be
>> less than or equal to ret. So it doesn't seem like an overflow is possible.
> 
> I think you meant "less than or equal to len"...

Yes, sorry.

> I am not saying I disagree. I rather have some reservations. However, if
> you convince me it is not possible or it is very difficult to blow up
> this code using malformed zstd archives than I am OK with it.

I'm just worried the change is a bit tricky to get right needing to
convert it to a string of grub_add() calls. And the conversion seems
risky especially given the existing code it was copied from has been
around for a while without issue (as far as I'm aware).


> PS I am going to cut GRUB 2.14~rc1 in second half of the week. So, it
>    would be nice to have these patches before that...
Ok, well except for this issue I should be good to send the next version
today or, at worst tomorrow.

Thanks,

Logan

_______________________________________________
Grub-devel mailing list
Grub-devel@gnu.org
https://lists.gnu.org/mailman/listinfo/grub-devel