* [PATCH] kbuild: modinst: remove the multithread option from zstd compression
@ 2024-08-04 3:33 Masahiro Yamada
2024-08-04 6:35 ` Sedat Dilek
2024-08-06 11:02 ` Nicolas Schier
0 siblings, 2 replies; 5+ messages in thread
From: Masahiro Yamada @ 2024-08-04 3:33 UTC (permalink / raw)
To: linux-kbuild
Cc: linux-kernel, Masahiro Yamada, Nathan Chancellor, Nick Terrell,
Nicolas Schier
Parallel execution is supported by GNU Make:
$ make -j<N> modules_install
It is questionable to enable multithreading within each zstd process
by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_install
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---
scripts/Makefile.modinst | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
index 0afd75472679..04f5229efa6b 100644
--- a/scripts/Makefile.modinst
+++ b/scripts/Makefile.modinst
@@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@
quiet_cmd_xz = XZ $@
cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $<
quiet_cmd_zstd = ZSTD $@
- cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
+ cmd_zstd = $(ZSTD) --rm -f -q $<
$(dst)/%.ko.gz: $(dst)/%.ko FORCE
$(call cmd,gzip)
--
2.43.0
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH] kbuild: modinst: remove the multithread option from zstd compression
2024-08-04 3:33 [PATCH] kbuild: modinst: remove the multithread option from zstd compression Masahiro Yamada
@ 2024-08-04 6:35 ` Sedat Dilek
2024-08-04 17:12 ` Masahiro Yamada
2024-08-06 11:02 ` Nicolas Schier
1 sibling, 1 reply; 5+ messages in thread
From: Sedat Dilek @ 2024-08-04 6:35 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Terrell,
Nicolas Schier
On Sun, Aug 4, 2024 at 5:33 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Parallel execution is supported by GNU Make:
>
> $ make -j<N> modules_install
>
> It is questionable to enable multithreading within each zstd process
> by default.
>
> If you still want to do it, you can use the environment variable:
>
> $ ZSTD_NBTHREADS=<N> make modules_install
>
Hi Masahiro,
I have some understanding problems.
[ start-build.txt ]
dileks 24225 24217 0 17:55 tty2 00:00:00 /usr/bin/perf stat
make V=1 -k -j4 ARCH=x86_64 LLVM=1 LLVM=/opt/llvm/bin/
PAHOLE=/opt/pahole/bin/pahole KBUILD_BUILD_HOST=iniza
KBUILD_BUILD_USER=sedat.dilek@gmail.com
KBUILD_BUILD_TIMESTAMP=2024-08-03
KDEB_PKGVERSION=6.10.3-1~trixie+dileks1
LOCALVERSION=-1-amd64-clang18-kcfi olddefconfig bindeb-pkg
^^ How shall someone pass so that ... ZSTD_NBTHREADS=<N> make
modules_install ... is used?
As far as I understood, each kernel-module file is taken - in the
Debian build-process - sequentially file for file - ZSTD compressed
and afterwards deleted.
Is there a benefit when 'make -j<N>' is used?
[ EXAMPLE - with my patch from [1] ]
zstd -T0 -19 --rm -f -q
debian/linux-image-6.10.3-1-amd64-clang18-kcfi/lib/modules/6.10.3-1-amd64-clang18-kcfi/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko
^^ How should ZSTD_NBTHREADS=<N> replace 'zstd -T0'?
When I pass '-T0 -L19' to ZSTD two cores are used on my CPU and a
better compress-level produces smaller kernel-module files.
I would like to benefit from both.
Thanks.
Best regards,
-Sedat-
Link: https://lore.kernel.org/all/CA+icZUUQadYjAXiCNx7PmKDV20WctvnzkXC3R2F7FM4Gzcm39Q@mail.gmail.com/
[1]
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> scripts/Makefile.modinst | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> index 0afd75472679..04f5229efa6b 100644
> --- a/scripts/Makefile.modinst
> +++ b/scripts/Makefile.modinst
> @@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@
> quiet_cmd_xz = XZ $@
> cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $<
> quiet_cmd_zstd = ZSTD $@
> - cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> + cmd_zstd = $(ZSTD) --rm -f -q $<
>
> $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> $(call cmd,gzip)
> --
> 2.43.0
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] kbuild: modinst: remove the multithread option from zstd compression
2024-08-04 6:35 ` Sedat Dilek
@ 2024-08-04 17:12 ` Masahiro Yamada
0 siblings, 0 replies; 5+ messages in thread
From: Masahiro Yamada @ 2024-08-04 17:12 UTC (permalink / raw)
To: sedat.dilek
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Terrell,
Nicolas Schier
On Sun, Aug 4, 2024 at 3:36 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>
> On Sun, Aug 4, 2024 at 5:33 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > Parallel execution is supported by GNU Make:
> >
> > $ make -j<N> modules_install
> >
> > It is questionable to enable multithreading within each zstd process
> > by default.
> >
> > If you still want to do it, you can use the environment variable:
> >
> > $ ZSTD_NBTHREADS=<N> make modules_install
> >
>
> Hi Masahiro,
>
> I have some understanding problems.
>
> [ start-build.txt ]
> dileks 24225 24217 0 17:55 tty2 00:00:00 /usr/bin/perf stat
> make V=1 -k -j4 ARCH=x86_64 LLVM=1 LLVM=/opt/llvm/bin/
> PAHOLE=/opt/pahole/bin/pahole KBUILD_BUILD_HOST=iniza
> KBUILD_BUILD_USER=sedat.dilek@gmail.com
> KBUILD_BUILD_TIMESTAMP=2024-08-03
> KDEB_PKGVERSION=6.10.3-1~trixie+dileks1
> LOCALVERSION=-1-amd64-clang18-kcfi olddefconfig bindeb-pkg
>
> ^^ How shall someone pass so that ... ZSTD_NBTHREADS=<N> make
> modules_install ... is used?
Option 1
$ export ZSTD_NBTHREADS=<N>
$ make V=1 -k [snip] olddefconfig bindeb-pkg
Option 2
$ ZSTD_NBTHREADS=<N> make V=1 -k [snip] olddefconfig bindeb-pkg
>
> As far as I understood, each kernel-module file is taken - in the
> Debian build-process - sequentially file for file - ZSTD compressed
> and afterwards deleted.
> Is there a benefit when 'make -j<N>' is used?
zstd --rm -f -q foo.ko
zstd --rm -f -q bar.ko
zstd --rm -f -q baz.ko
will run in parallel with Make's -j<N> option.
--
Best Regards
Masahiro Yamada
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] kbuild: modinst: remove the multithread option from zstd compression
2024-08-04 3:33 [PATCH] kbuild: modinst: remove the multithread option from zstd compression Masahiro Yamada
2024-08-04 6:35 ` Sedat Dilek
@ 2024-08-06 11:02 ` Nicolas Schier
2024-08-24 21:51 ` Sedat Dilek
1 sibling, 1 reply; 5+ messages in thread
From: Nicolas Schier @ 2024-08-06 11:02 UTC (permalink / raw)
To: Masahiro Yamada
Cc: linux-kbuild, linux-kernel, Nathan Chancellor, Nick Terrell
On Sun, Aug 04, 2024 at 12:33:07PM +0900, Masahiro Yamada wrote:
> Parallel execution is supported by GNU Make:
>
> $ make -j<N> modules_install
>
> It is questionable to enable multithreading within each zstd process
> by default.
>
> If you still want to do it, you can use the environment variable:
>
> $ ZSTD_NBTHREADS=<N> make modules_install
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
>
> scripts/Makefile.modinst | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> index 0afd75472679..04f5229efa6b 100644
> --- a/scripts/Makefile.modinst
> +++ b/scripts/Makefile.modinst
> @@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@
> quiet_cmd_xz = XZ $@
> cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $<
> quiet_cmd_zstd = ZSTD $@
> - cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> + cmd_zstd = $(ZSTD) --rm -f -q $<
>
> $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> $(call cmd,gzip)
> --
> 2.43.0
>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] kbuild: modinst: remove the multithread option from zstd compression
2024-08-06 11:02 ` Nicolas Schier
@ 2024-08-24 21:51 ` Sedat Dilek
0 siblings, 0 replies; 5+ messages in thread
From: Sedat Dilek @ 2024-08-24 21:51 UTC (permalink / raw)
To: Nicolas Schier
Cc: Masahiro Yamada, linux-kbuild, linux-kernel, Nathan Chancellor,
Nick Terrell
On Tue, Aug 6, 2024 at 1:02 PM Nicolas Schier <nicolas@fjasle.eu> wrote:
>
> On Sun, Aug 04, 2024 at 12:33:07PM +0900, Masahiro Yamada wrote:
> > Parallel execution is supported by GNU Make:
> >
> > $ make -j<N> modules_install
> >
> > It is questionable to enable multithreading within each zstd process
> > by default.
> >
> > If you still want to do it, you can use the environment variable:
> >
> > $ ZSTD_NBTHREADS=<N> make modules_install
> >
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> >
> > scripts/Makefile.modinst | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.modinst b/scripts/Makefile.modinst
> > index 0afd75472679..04f5229efa6b 100644
> > --- a/scripts/Makefile.modinst
> > +++ b/scripts/Makefile.modinst
> > @@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@
> > quiet_cmd_xz = XZ $@
> > cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $<
> > quiet_cmd_zstd = ZSTD $@
> > - cmd_zstd = $(ZSTD) -T0 --rm -f -q $<
> > + cmd_zstd = $(ZSTD) --rm -f -q $<
> >
> > $(dst)/%.ko.gz: $(dst)/%.ko FORCE
> > $(call cmd,gzip)
> > --
> > 2.43.0
> >
>
> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # Linux v6.11-rc4+ on x86-64
-Sedat-
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2024-08-24 21:52 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-08-04 3:33 [PATCH] kbuild: modinst: remove the multithread option from zstd compression Masahiro Yamada
2024-08-04 6:35 ` Sedat Dilek
2024-08-04 17:12 ` Masahiro Yamada
2024-08-06 11:02 ` Nicolas Schier
2024-08-24 21:51 ` Sedat Dilek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox