[Buildroot] [PATCH] fs/squashfs: honour the number of parallel jobs

[Buildroot] [PATCH] fs/squashfs: honour the number of parallel jobs

[Yann E. MORIN]

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
---
 fs/squashfs/squashfs.mk | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/squashfs/squashfs.mk b/fs/squashfs/squashfs.mk
index 342c7a8a37..7de7f51af1 100644
--- a/fs/squashfs/squashfs.mk
+++ b/fs/squashfs/squashfs.mk
@@ -6,6 +6,8 @@
 
 ROOTFS_SQUASHFS_DEPENDENCIES = host-squashfs
 
+ROOTFS_SQUASHFS_ARGS = -noappend -processors $(PARALLEL_JOBS)
+
 ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZ4),y)
 ROOTFS_SQUASHFS_ARGS += -comp lz4 -Xhc
 else ifeq ($(BR2_TARGET_ROOTFS_SQUASHFS4_LZO),y)
@@ -19,8 +21,7 @@ ROOTFS_SQUASHFS_ARGS += -comp gzip
 endif
 
 define ROOTFS_SQUASHFS_CMD
-	$(HOST_DIR)/bin/mksquashfs $(TARGET_DIR) $@ -noappend \
-		$(ROOTFS_SQUASHFS_ARGS)
+	$(HOST_DIR)/bin/mksquashfs $(TARGET_DIR) $@ $(ROOTFS_SQUASHFS_ARGS)
 endef
 
 $(eval $(call ROOTFS_TARGET,squashfs))
-- 
2.11.0

[Buildroot] [PATCH] fs/squashfs: honour the number of parallel jobs

[Thomas Petazzoni]

Hello,

On Wed,  2 Aug 2017 23:22:39 +0200, Yann E. MORIN wrote:
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>

This would benefit from an extended commit log. Why do we do this? Is
this fixing something? Is this reducing the duration of mksquashfs
because it leverages multiple cores when doing the compression? In the
latter case, a few numbers would be nice to have.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

[Buildroot] [PATCH] fs/squashfs: honour the number of parallel jobs

[Yann E. MORIN]

Thomas, All,

On 2017-08-03 09:25 +0200, Thomas Petazzoni spake thusly:
> On Wed,  2 Aug 2017 23:22:39 +0200, Yann E. MORIN wrote:
> > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> > Cc: Clayton Shotwell <clayton.shotwell@rockwellcollins.com>
> 
> This would benefit from an extended commit log. Why do we do this? Is
> this fixing something? Is this reducing the duration of mksquashfs
> because it leverages multiple cores when doing the compression? In the
> latter case, a few numbers would be nice to have.

By default, mksquashfs uses all the CPUs available (e.g. like us with
BR2_JLEVEL=0).

However, a user may well set BR2_JLEVEL to a specific value, to
voluntarily limit the number of CPUs to use (e.g. like us on the
autobuilders).

So, this patch just propagates that BR2_JLEVEL setting to mksquashfs,
to use the same number of jobs as the rest of the build process.

So, there is not speed improvement to expect, quite the opposite in
fact. And this is why this patch can be seen as a mitigation (but not a
fix) for #10141, as it allows to reduce the parallelism on systems with
a huge number of CPUs, which is where #10141 occurs.

I'll add a bit more explanations to this patch and respin...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'