Re: [PATCH v3] module: add in-kernel support for decompressing

[Dmitry Torokhov <dmitry.torokhov@gmail.com>]

On Mon, Dec 20, 2021 at 08:52:30AM -0800, Luis Chamberlain wrote:
> On Fri, Dec 10, 2021 at 05:09:23PM -0800, Dmitry Torokhov wrote:
> > On Fri, Dec 10, 2021 at 04:11:21PM -0800, Luis Chamberlain wrote:
> > > On Thu, Dec 09, 2021 at 10:09:17PM -0800, Dmitry Torokhov wrote:
> > > > diff --git a/init/Kconfig b/init/Kconfig
> > > > index cd23faa163d1..d90774ff7610 100644
> > > > --- a/init/Kconfig
> > > > +++ b/init/Kconfig
> > > > @@ -2305,6 +2305,19 @@ config MODULE_COMPRESS_ZSTD
> > > >  
> > > >  endchoice
> > > >  
> > > > +config MODULE_DECOMPRESS
> > > > +	bool "Support in-kernel module decompression"
> > > > +	depends on MODULE_COMPRESS_GZIP || MODULE_COMPRESS_XZ
> > > > +	select ZLIB_INFLATE if MODULE_COMPRESS_GZIP
> > > > +	select XZ_DEC if MODULE_COMPRESS_XZ
> > > 
> > > What if MODULE_COMPRESS_GZIP and MODULE_COMPRESS_XZ are enabled?
> > > These are not mutually exclusive.
> > 
> > They are mutually exclusive, the kernel uses the same (one) compression
> > method for all kernel modules that it generates (i.e we do not compress
> > drivers/usb/... with gzip while drivers/net/... with xz).
> 
> Ah yes I failed to see the choice/prompt for it.
> 
> > The idea here is to allow the kernel consume the same format that was
> > used when generating modules. Supporting multiple formats at once is
> > overkill IMO.
> 
> Indeed.
> 
> > > > +	help
> > > > +
> > > > +	  Support for decompressing kernel modules by the kernel itself
> > > > +	  instead of relying on userspace to perform this task. Useful when
> > > > +	  load pinning security policy is enabled.
> > > 
> > > Shouldn't kernel decompression be faster too? If so, what's the
> > > point of doing it in userspace?
> > 
> > Make the kernel smaller?
> 
> Yes this I buy.
> 
> > Have more flexibility with exotic compression
> > formats?
> 
> I just have a hunch that doing module decompression in the kernel will
> speed things quite a bit... any chance you can provide some before and
> after systemd-analyze ?

If you insist I can try running it, but it should be slower unless your
memory controller is so slow that reading file from disk and dealing
with page by page decompression is quicker than copying already
decompressed data from userspace. We still reading and uncompressing
file in kmod (to make sure the format is valid) and we can uncompress
using large buffers (we are not concerned with using unswappable kernel
memory).

Maybe in the future when we have streaming and accelerated in-kernel
decompression API we could optimize for that in kmod and see some
savings on very large modules.

Thanks.

-- 
Dmitry