[PATCH] initrd: support erofs as initrd

[PATCH] initrd: support erofs as initrd

[Julian Stecklina via B4 Relay]

From: Julian Stecklina <julian.stecklina@cyberus-technology.de>

Add erofs detection to the initrd mount code. This allows systems to
boot from an erofs-based initrd in the same way as they can boot from
a squashfs initrd.

Just as squashfs initrds, erofs images as initrds are a good option
for systems that are memory-constrained.

Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
---
 init/do_mounts_rd.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/init/do_mounts_rd.c b/init/do_mounts_rd.c
index ac021ae6e6fa78c7b7828a78ab2fa3af3611bef3..7c3f8b45b5ed2eea3c534d7f2e65608542009df5 100644
--- a/init/do_mounts_rd.c
+++ b/init/do_mounts_rd.c
@@ -11,6 +11,7 @@
 
 #include "do_mounts.h"
 #include "../fs/squashfs/squashfs_fs.h"
+#include "../fs/erofs/erofs_fs.h"
 
 #include <linux/decompress/generic.h>
 
@@ -47,6 +48,7 @@ static int __init crd_load(decompress_fn deco);
  *	romfs
  *	cramfs
  *	squashfs
+ *	erofs
  *	gzip
  *	bzip2
  *	lzma
@@ -63,6 +65,7 @@ identify_ramdisk_image(struct file *file, loff_t pos,
 	struct romfs_super_block *romfsb;
 	struct cramfs_super *cramfsb;
 	struct squashfs_super_block *squashfsb;
+	struct erofs_super_block *erofsb;
 	int nblocks = -1;
 	unsigned char *buf;
 	const char *compress_name;
@@ -77,6 +80,7 @@ identify_ramdisk_image(struct file *file, loff_t pos,
 	romfsb = (struct romfs_super_block *) buf;
 	cramfsb = (struct cramfs_super *) buf;
 	squashfsb = (struct squashfs_super_block *) buf;
+	erofsb = (struct erofs_super_block *) buf;
 	memset(buf, 0xe5, size);
 
 	/*
@@ -165,6 +169,21 @@ identify_ramdisk_image(struct file *file, loff_t pos,
 		goto done;
 	}
 
+	/* Try erofs */
+	pos = (start_block * BLOCK_SIZE) + EROFS_SUPER_OFFSET;
+	kernel_read(file, buf, size, &pos);
+
+	if (erofsb->magic == EROFS_SUPER_MAGIC_V1) {
+		printk(KERN_NOTICE
+		       "RAMDISK: erofs filesystem found at block %d\n",
+		       start_block);
+
+		nblocks = ((erofsb->blocks << erofsb->blkszbits) + BLOCK_SIZE - 1)
+			>> BLOCK_SIZE_BITS;
+
+		goto done;
+	}
+
 	printk(KERN_NOTICE
 	       "RAMDISK: Couldn't find valid RAM disk image starting at %d.\n",
 	       start_block);

---
base-commit: 5fc31936081919a8572a3d644f3fbb258038f337
change-id: 20250320-initrd-erofs-76e925fdf68c

Best regards,
-- 
Julian Stecklina <julian.stecklina@cyberus-technology.de>

Re: [PATCH] initrd: support erofs as initrd

[Al Viro]

On Thu, Mar 20, 2025 at 08:28:23PM +0100, Julian Stecklina via B4 Relay wrote:
> From: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> 
> Add erofs detection to the initrd mount code. This allows systems to
> boot from an erofs-based initrd in the same way as they can boot from
> a squashfs initrd.
> 
> Just as squashfs initrds, erofs images as initrds are a good option
> for systems that are memory-constrained.
> 
> Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>

>  #include "do_mounts.h"
>  #include "../fs/squashfs/squashfs_fs.h"
> +#include "../fs/erofs/erofs_fs.h"

This is getting really unpleasant...

Folks, could we do something similar to initcalls - add a section
(.init.text.rd_detect?) with array of pointers to __init functions
that would be called by that thing in turn?  With filesystems that
want to add that kind of stuff being about to do something like

static int __init detect_minix(struct file *file, void *buf, loff_t *pos, int start_block)
{
	struct minix_super_block *minixsb = buf;
	initrd_fill_buffer(file, buf, pos, (start_block + 1) * BLOCK_SIZE);
	if (minixsb->s_magic == MINIX_SUPER_MAGIC ||
	    minixsb->s_magic == MINIX_SUPER_MAGIC2) {
		printk(KERN_NOTICE
			"RAMDISK: Minix filesystem found at block %d\n",
			start_block);
		return minixsb->s_nzones << minixsb->s_log_zone_size;
	}
	return -1;
}

initrd_detect(detect_minix);

with the latter emitting a pointer to detect_minix into that new
section?

initrd_fill_buffer() would be something along the lines of

	if (*pos != wanted) {
		*pos = wanted;
		kernel_read(file, buf, 512, pos);
	}

I mean, we can keep adding those pieces there, but...

Re: [PATCH] initrd: support erofs as initrd

[Christoph Hellwig]

We've been trying to kill off initrd in favor of initramfs for about
two decades.  I don't think adding new file system support to it is
helpful.

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

Hi Christoph,

On 2025/3/21 13:01, Christoph Hellwig wrote:
> We've been trying to kill off initrd in favor of initramfs for about
> two decades.  I don't think adding new file system support to it is
> helpful.
> 

Disclaimer: I don't know the background of this effort so
more background might be helpful.

Two years ago, I once thought if using EROFS + FSDAX to directly
use the initrd image from bootloaders to avoid the original initrd
double caching issue (which is what initramfs was proposed to
resolve) and initramfs unnecessary tmpfs unpack overhead:
https://lore.kernel.org/r/ZXgNQ85PdUKrQU1j@infradead.org

Also EROFS supports xattrs so the following potential work (which
the cpio format doesn't support) is no longer needed although I
don't have any interest to follow either):
https://lore.kernel.org/r/20190523121803.21638-1-roberto.sassu@huawei.com

Anyway, personally I have no time slot or even input on those.

Thanks,
Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[Christian Brauner]

On Fri, Mar 21, 2025 at 02:08:26AM +0000, Al Viro wrote:
> On Thu, Mar 20, 2025 at 08:28:23PM +0100, Julian Stecklina via B4 Relay wrote:
> > From: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> > 
> > Add erofs detection to the initrd mount code. This allows systems to
> > boot from an erofs-based initrd in the same way as they can boot from
> > a squashfs initrd.
> > 
> > Just as squashfs initrds, erofs images as initrds are a good option
> > for systems that are memory-constrained.
> > 
> > Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> 
> >  #include "do_mounts.h"
> >  #include "../fs/squashfs/squashfs_fs.h"
> > +#include "../fs/erofs/erofs_fs.h"
> 
> This is getting really unpleasant...
> 
> Folks, could we do something similar to initcalls - add a section
> (.init.text.rd_detect?) with array of pointers to __init functions
> that would be called by that thing in turn?  With filesystems that
> want to add that kind of stuff being about to do something like
> 
> static int __init detect_minix(struct file *file, void *buf, loff_t *pos, int start_block)
> {
> 	struct minix_super_block *minixsb = buf;
> 	initrd_fill_buffer(file, buf, pos, (start_block + 1) * BLOCK_SIZE);
> 	if (minixsb->s_magic == MINIX_SUPER_MAGIC ||
> 	    minixsb->s_magic == MINIX_SUPER_MAGIC2) {
> 		printk(KERN_NOTICE
> 			"RAMDISK: Minix filesystem found at block %d\n",
> 			start_block);
> 		return minixsb->s_nzones << minixsb->s_log_zone_size;
> 	}
> 	return -1;
> }
> 
> initrd_detect(detect_minix);
> 
> with the latter emitting a pointer to detect_minix into that new
> section?
> 
> initrd_fill_buffer() would be something along the lines of
> 
> 	if (*pos != wanted) {
> 		*pos = wanted;
> 		kernel_read(file, buf, 512, pos);
> 	}
> 
> I mean, we can keep adding those pieces there, but...

Very much agreed.

Re: [PATCH] initrd: support erofs as initrd

[Christian Brauner]

On Thu, Mar 20, 2025 at 08:28:23PM +0100, Julian Stecklina via B4 Relay wrote:
> From: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> 
> Add erofs detection to the initrd mount code. This allows systems to
> boot from an erofs-based initrd in the same way as they can boot from
> a squashfs initrd.
> 
> Just as squashfs initrds, erofs images as initrds are a good option
> for systems that are memory-constrained.

I think this can be valuable and I know we've had discussion about this
before but it should please come with a detailed rationale for using
erofs and what project(s) this would be used by. Then I don't think
there's a reason per se not to do it.

Re: [PATCH] initrd: support erofs as initrd

[Thomas Weißschuh]

On Thu, Mar 20, 2025 at 08:28:23PM +0100, Julian Stecklina via B4 Relay wrote:
> From: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> 
> Add erofs detection to the initrd mount code. This allows systems to
> boot from an erofs-based initrd in the same way as they can boot from
> a squashfs initrd.
> 
> Just as squashfs initrds, erofs images as initrds are a good option
> for systems that are memory-constrained.
> 
> Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> ---
>  init/do_mounts_rd.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/init/do_mounts_rd.c b/init/do_mounts_rd.c
> index ac021ae6e6fa78c7b7828a78ab2fa3af3611bef3..7c3f8b45b5ed2eea3c534d7f2e65608542009df5 100644
> --- a/init/do_mounts_rd.c
> +++ b/init/do_mounts_rd.c
> @@ -11,6 +11,7 @@
>  
>  #include "do_mounts.h"
>  #include "../fs/squashfs/squashfs_fs.h"
> +#include "../fs/erofs/erofs_fs.h"
>  
>  #include <linux/decompress/generic.h>
>  
> @@ -47,6 +48,7 @@ static int __init crd_load(decompress_fn deco);
>   *	romfs
>   *	cramfs
>   *	squashfs
> + *	erofs
>   *	gzip
>   *	bzip2
>   *	lzma
> @@ -63,6 +65,7 @@ identify_ramdisk_image(struct file *file, loff_t pos,
>  	struct romfs_super_block *romfsb;
>  	struct cramfs_super *cramfsb;
>  	struct squashfs_super_block *squashfsb;
> +	struct erofs_super_block *erofsb;
>  	int nblocks = -1;
>  	unsigned char *buf;
>  	const char *compress_name;
> @@ -77,6 +80,7 @@ identify_ramdisk_image(struct file *file, loff_t pos,
>  	romfsb = (struct romfs_super_block *) buf;
>  	cramfsb = (struct cramfs_super *) buf;
>  	squashfsb = (struct squashfs_super_block *) buf;
> +	erofsb = (struct erofs_super_block *) buf;
>  	memset(buf, 0xe5, size);
>  
>  	/*
> @@ -165,6 +169,21 @@ identify_ramdisk_image(struct file *file, loff_t pos,
>  		goto done;
>  	}
>  
> +	/* Try erofs */
> +	pos = (start_block * BLOCK_SIZE) + EROFS_SUPER_OFFSET;
> +	kernel_read(file, buf, size, &pos);
> +
> +	if (erofsb->magic == EROFS_SUPER_MAGIC_V1) {

le32_to_cpu(erofsb->magic)

> +		printk(KERN_NOTICE
> +		       "RAMDISK: erofs filesystem found at block %d\n",
> +		       start_block);
> +
> +		nblocks = ((erofsb->blocks << erofsb->blkszbits) + BLOCK_SIZE - 1)
> +			>> BLOCK_SIZE_BITS;

le32_to_cpu(erofsb->blocks)

> +
> +		goto done;
> +	}
> +
>  	printk(KERN_NOTICE
>  	       "RAMDISK: Couldn't find valid RAM disk image starting at %d.\n",
>  	       start_block);
> 

This seems to be broken for cramfs and minix, too.


Thomas

Re: [PATCH] initrd: support erofs as initrd

[Julian Stecklina]

Hi Al!

On Fri, 2025-03-21 at 02:08 +0000, Al Viro wrote:
> On Thu, Mar 20, 2025 at 08:28:23PM +0100, Julian Stecklina via B4 Relay wrote:
> > From: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> > 
> > Add erofs detection to the initrd mount code. This allows systems to
> > boot from an erofs-based initrd in the same way as they can boot from
> > a squashfs initrd.
> > 
> > Just as squashfs initrds, erofs images as initrds are a good option
> > for systems that are memory-constrained.
> > 
> > Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> 
> >  #include "do_mounts.h"
> >  #include "../fs/squashfs/squashfs_fs.h"
> > +#include "../fs/erofs/erofs_fs.h"
> 
> This is getting really unpleasant...
> 
> Folks, could we do something similar to initcalls - add a section
> (.init.text.rd_detect?) with array of pointers to __init functions
> that would be called by that thing in turn?  With filesystems that
> want to add that kind of stuff being about to do something like

That's a great suggestion! I wonder whether it's possible to restructure the
code further, so it does something along:

1. Try to detect whether it's a (compressed) initramfs -> if so, extract and be
done.
2. Copy the initrd onto /dev/ram and just try to mount it (without the manual fs
detection)

As far as I understand it, the kernel should be able to figure out itself what
filesystem it is? If so, this removes the manual fs detection and just allows
all filesystem images to be used as initrd.

I'm a noob in this part of the code base, so feel free to tell me that/why this
is not possible. :)

Julian

Re: [PATCH] initrd: support erofs as initrd

[Julian Stecklina]

On Fri, 2025-03-21 at 13:27 +0800, Gao Xiang wrote:
> Hi Christoph,
> 
> On 2025/3/21 13:01, Christoph Hellwig wrote:
> > We've been trying to kill off initrd in favor of initramfs for about
> > two decades.  I don't think adding new file system support to it is
> > helpful.
> > 
> 
> Disclaimer: I don't know the background of this effort so
> more background might be helpful.

So erofs came up in an effort to improve the experience for users of NixOS on
smaller systems. We use erofs a lot and some people in the community just
consider it a "better" cpio at this point. A great property is that the contents
stays compressed in memory and there is no need to unpack anything at boot.
Others like that the rootfs is read-only by default. In short: erofs is a great
fit.

Of course there are some solutions to using erofs images at boot now:
https://github.com/containers/initoverlayfs

But this adds yet another step in the already complex boot process and feels
like a hack. It would be nice to just use erofs images as initrd. The other
building block to this is automatically sizing /dev/ram0:

https://lkml.org/lkml/2025/3/20/1296

I didn't pack both patches into one series, because I thought enabling erofs
itself would be less controversial and is already useful on its own. The
autosizing of /dev/ram is probably more involved than my RFC patch. I'm hoping
for some input on how to do it right. :)

> 
> Two years ago, I once thought if using EROFS + FSDAX to directly
> use the initrd image from bootloaders to avoid the original initrd
> double caching issue (which is what initramfs was proposed to
> resolve) and initramfs unnecessary tmpfs unpack overhead:
> https://lore.kernel.org/r/ZXgNQ85PdUKrQU1j@infradead.org
> 
> Also EROFS supports xattrs so the following potential work (which
> the cpio format doesn't support) is no longer needed although I
> don't have any interest to follow either):
> https://lore.kernel.org/r/20190523121803.21638-1-roberto.sassu@huawei.com

Thanks for the pointers!

Julian

Re: [PATCH] initrd: support erofs as initrd

[Julian Stecklina]

On Fri, 2025-03-21 at 10:16 +0100, Thomas Weißschuh wrote:
> On Thu, Mar 20, 2025 at 08:28:23PM +0100, Julian Stecklina via B4 Relay wrote:
> > From: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> > 
> > Add erofs detection to the initrd mount code. This allows systems to
> > boot from an erofs-based initrd in the same way as they can boot from
> > a squashfs initrd.
> > 
> > Just as squashfs initrds, erofs images as initrds are a good option
> > for systems that are memory-constrained.
> > 
> > Signed-off-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
> > ---
> >  init/do_mounts_rd.c | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> > 
> > diff --git a/init/do_mounts_rd.c b/init/do_mounts_rd.c
> > index
> > ac021ae6e6fa78c7b7828a78ab2fa3af3611bef3..7c3f8b45b5ed2eea3c534d7f2e65608542
> > 009df5 100644
> > --- a/init/do_mounts_rd.c
> > +++ b/init/do_mounts_rd.c
> > @@ -11,6 +11,7 @@
> >  
> >  #include "do_mounts.h"
> >  #include "../fs/squashfs/squashfs_fs.h"
> > +#include "../fs/erofs/erofs_fs.h"
> >  
> >  #include <linux/decompress/generic.h>
> >  
> > @@ -47,6 +48,7 @@ static int __init crd_load(decompress_fn deco);
> >   * romfs
> >   * cramfs
> >   * squashfs
> > + * erofs
> >   * gzip
> >   * bzip2
> >   * lzma
> > @@ -63,6 +65,7 @@ identify_ramdisk_image(struct file *file, loff_t pos,
> >   struct romfs_super_block *romfsb;
> >   struct cramfs_super *cramfsb;
> >   struct squashfs_super_block *squashfsb;
> > + struct erofs_super_block *erofsb;
> >   int nblocks = -1;
> >   unsigned char *buf;
> >   const char *compress_name;
> > @@ -77,6 +80,7 @@ identify_ramdisk_image(struct file *file, loff_t pos,
> >   romfsb = (struct romfs_super_block *) buf;
> >   cramfsb = (struct cramfs_super *) buf;
> >   squashfsb = (struct squashfs_super_block *) buf;
> > + erofsb = (struct erofs_super_block *) buf;
> >   memset(buf, 0xe5, size);
> >  
> >   /*
> > @@ -165,6 +169,21 @@ identify_ramdisk_image(struct file *file, loff_t pos,
> >   goto done;
> >   }
> >  
> > + /* Try erofs */
> > + pos = (start_block * BLOCK_SIZE) + EROFS_SUPER_OFFSET;
> > + kernel_read(file, buf, size, &pos);
> > +
> > + if (erofsb->magic == EROFS_SUPER_MAGIC_V1) {
> 
> le32_to_cpu(erofsb->magic)
> 
> > + printk(KERN_NOTICE
> > +        "RAMDISK: erofs filesystem found at block %d\n",
> > +        start_block);
> > +
> > + nblocks = ((erofsb->blocks << erofsb->blkszbits) + BLOCK_SIZE - 1)
> > + >> BLOCK_SIZE_BITS;
> 
> le32_to_cpu(erofsb->blocks)
> 
> > +
> > + goto done;
> > + }
> > +
> >   printk(KERN_NOTICE
> >          "RAMDISK: Couldn't find valid RAM disk image starting at %d.\n",
> >          start_block);
> > 
> 
> This seems to be broken for cramfs and minix, too.

Great observation. Will fix! 

Julian

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

Hi Julian,

On 2025/3/21 21:17, Julian Stecklina wrote:
> On Fri, 2025-03-21 at 13:27 +0800, Gao Xiang wrote:
>> Hi Christoph,
>>
>> On 2025/3/21 13:01, Christoph Hellwig wrote:
>>> We've been trying to kill off initrd in favor of initramfs for about
>>> two decades.  I don't think adding new file system support to it is
>>> helpful.
>>>
>>
>> Disclaimer: I don't know the background of this effort so
>> more background might be helpful.
> 
> So erofs came up in an effort to improve the experience for users of NixOS on
> smaller systems. We use erofs a lot and some people in the community just
> consider it a "better" cpio at this point. A great property is that the contents
> stays compressed in memory and there is no need to unpack anything at boot.
> Others like that the rootfs is read-only by default. In short: erofs is a great
> fit.
> 
> Of course there are some solutions to using erofs images at boot now:
> https://github.com/containers/initoverlayfs
> 
> But this adds yet another step in the already complex boot process and feels
> like a hack. It would be nice to just use erofs images as initrd. The other
> building block to this is automatically sizing /dev/ram0:
> 
> https://lkml.org/lkml/2025/3/20/1296
> 
> I didn't pack both patches into one series, because I thought enabling erofs
> itself would be less controversial and is already useful on its own. The
> autosizing of /dev/ram is probably more involved than my RFC patch. I'm hoping
> for some input on how to do it right. :)

Ok, my own thought is that cpio format is somewhat inflexible.  It
seems that the main original reason for introducing initramfs and
cpio was to avoid double caching, but it can be resolved with FSDAX
now and initdax totally avoids unnecessary cpio parsing and unpacking.
cpio format is much like tar which lacks of basic features like
random access and xattrs which are useful for some use cases as
I mentioned before.

The initrd image can even compressed as a whole and decompress in
the current initramfs way.  If you really need on-demand
decompression, you could leave some file compresssed since EROFS
supports per-inode compression, but those files are still double
caching since FSDAX mode should be uncompressed to support mmap.
You could leave rare-used files compressed.

Thanks,
Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[hch]

On Fri, Mar 21, 2025 at 01:17:54PM +0000, Julian Stecklina wrote:
> Of course there are some solutions to using erofs images at boot now:
> https://github.com/containers/initoverlayfs
> 
> But this adds yet another step in the already complex boot process and feels
> like a hack. It would be nice to just use erofs images as initrd. The other
> building block to this is automatically sizing /dev/ram0:
> 
> https://lkml.org/lkml/2025/3/20/1296
> 
> I didn't pack both patches into one series, because I thought enabling erofs
> itself would be less controversial and is already useful on its own. The
> autosizing of /dev/ram is probably more involved than my RFC patch. I'm hoping
> for some input on how to do it right. :)

Booting from erofs seems perfectly fine to me.  Booting from erofs on
an initrd is not.  There is no reason to fake up a block device, just
have a version of erofs that directly points to pre-loaded kernel
memory instead.  This is a bit more work, but a lot more efficient
in that it removes the block path from the I/O stack, removes the boot
time copy and allows for much more flexible memory management.

Re: [PATCH] initrd: support erofs as initrd

[Julian Stecklina]

Hi!

On Mon, 2025-04-07 at 10:57 +0200, hch@lst.de wrote:
> On Fri, Mar 21, 2025 at 01:17:54PM +0000, Julian Stecklina wrote:
> > Of course there are some solutions to using erofs images at boot now:
> > https://github.com/containers/initoverlayfs
> > 
> > But this adds yet another step in the already complex boot process and feels
> > like a hack. It would be nice to just use erofs images as initrd. The other
> > building block to this is automatically sizing /dev/ram0:
> > 
> > https://lkml.org/lkml/2025/3/20/1296
> > 
> > I didn't pack both patches into one series, because I thought enabling erofs
> > itself would be less controversial and is already useful on its own. The
> > autosizing of /dev/ram is probably more involved than my RFC patch. I'm
> > hoping
> > for some input on how to do it right. :)
> 
> Booting from erofs seems perfectly fine to me.  Booting from erofs on
> an initrd is not.  There is no reason to fake up a block device, just
> have a version of erofs that directly points to pre-loaded kernel
> memory instead.  This is a bit more work, but a lot more efficient
> in that it removes the block path from the I/O stack, removes the boot
> time copy and allows for much more flexible memory management.

Can you be more specific in how that would look like in practice? The
infrastructure for initrds is universally available and what you are proposing
sounds like adding a new mechanism entirely?

Julian

PS. Sorry for the re-send. I accidentally sent a HTML mail. :(

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

Hi Christoph,

On 2025/4/7 16:57, hch@lst.de wrote:
> On Fri, Mar 21, 2025 at 01:17:54PM +0000, Julian Stecklina wrote:
>> Of course there are some solutions to using erofs images at boot now:
>> https://github.com/containers/initoverlayfs
>>
>> But this adds yet another step in the already complex boot process and feels
>> like a hack. It would be nice to just use erofs images as initrd. The other
>> building block to this is automatically sizing /dev/ram0:
>>
>> https://lkml.org/lkml/2025/3/20/1296
>>
>> I didn't pack both patches into one series, because I thought enabling erofs
>> itself would be less controversial and is already useful on its own. The
>> autosizing of /dev/ram is probably more involved than my RFC patch. I'm hoping
>> for some input on how to do it right. :)
> 
> Booting from erofs seems perfectly fine to me.  Booting from erofs on
> an initrd is not.  There is no reason to fake up a block device, just
> have a version of erofs that directly points to pre-loaded kernel
> memory instead.  This is a bit more work, but a lot more efficient
> in that it removes the block path from the I/O stack, removes the boot
> time copy and allows for much more flexible memory management.

For unencoded (uncompressed) images, especially data is page-aligned,
FSDAX can be used directly to replace page cache, which is absolutely
fine now.

But for encoded I/Os, since it needs for somewhat data transformation,
currently BIOs (actually mainly bio_vecs, but bio_vecs are lack of
useful interfaces) is needed to get the on-disk (or original memory)
encoded data in a flexible way.

I hope BIO (or some other-like interfaces) can be eventually decoupled
from block devices, and instead form generic block-based interfaces
anyway, so that eveutually yes, we don't need to fake a block device
(like FSDAX support for brd) like the old initrd.  But I suspect if
any folk puts more development resource to work on this (I don't
have more resource from my current employer), so personally I guess
initrd could be workable for the first step for people to leverage
this feature in some way.

Thanks,
Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[Askar Safin]

> We've been trying to kill off initrd in favor of initramfs for about
> two decades.  I don't think adding new file system support to it is
> helpful.

I totally agree.

What prevents us from removing initrd right now?

The only reason is lack of volunteers?

If yes, then may I remove initrd?

--
Askar Safin

Re: [PATCH] initrd: support erofs as initrd

[Christoph Hellwig]

On Mon, Aug 25, 2025 at 09:27:13PM +0300, Askar Safin wrote:
> > We've been trying to kill off initrd in favor of initramfs for about
> > two decades.  I don't think adding new file system support to it is
> > helpful.
> 
> I totally agree.
> 
> What prevents us from removing initrd right now?
> 
> The only reason is lack of volunteers?
> 
> If yes, then may I remove initrd?

Give it a spin and see if anyone shouts.

Re: [PATCH] initrd: support erofs as initrd

[Byron Stanoszek]

On Tue, 26 Aug 2025, Christoph Hellwig wrote:

> On Mon, Aug 25, 2025 at 09:27:13PM +0300, Askar Safin wrote:
>>> We've been trying to kill off initrd in favor of initramfs for about
>>> two decades.  I don't think adding new file system support to it is
>>> helpful.
>>
>> I totally agree.
>>
>> What prevents us from removing initrd right now?
>>
>> The only reason is lack of volunteers?
>>
>> If yes, then may I remove initrd?
>
> Give it a spin and see if anyone shouts.

Well, this makes me a little sad. I run several hundred embedded systems out in
the world, and I use a combination of initrd and initramfs for booting. These
systems operate entirely in ramdisk form.

I concatenate a very large .sqfs file onto the end of "vmlinuz", which gets
loaded into initrd automatically by the bootloader. Then in my initramfs (cpio
archive that's compiled in with the kernel), my /sbin/init executable copies
/initrd.image to /dev/ram0, mounts a tmpfs overlay on top of it, then does a
pivot root to it.

This gives it the appearance of a read-write initramfs filesystem, but the
lower layer data remains compressed in RAM. This saves quite a bit of RAM
during runtime, which is still yet important on older PCs.

If there's a better (more official) way of having a real compressed initramfs
that remains compressed during runtime, I'm all for it. But until then, I would
like to ask you to please not remove the initrd functionality.

(In fact, I was actually thinking about trying this method with erofs as the
lower layer filesystem someday soon instead of squashfs. But I would still be
using an overlay to mount it, instead of the auto-detect method addressed by
this patch.)

Thank you,
  -Byron

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On 2025/8/26 22:21, Byron Stanoszek wrote:
> On Tue, 26 Aug 2025, Christoph Hellwig wrote:
> 
>> On Mon, Aug 25, 2025 at 09:27:13PM +0300, Askar Safin wrote:
>>>> We've been trying to kill off initrd in favor of initramfs for about
>>>> two decades.  I don't think adding new file system support to it is
>>>> helpful.
>>>
>>> I totally agree.
>>>
>>> What prevents us from removing initrd right now?
>>>
>>> The only reason is lack of volunteers?
>>>
>>> If yes, then may I remove initrd?
>>
>> Give it a spin and see if anyone shouts.
> 
> Well, this makes me a little sad. I run several hundred embedded systems out in
> the world, and I use a combination of initrd and initramfs for booting. These
> systems operate entirely in ramdisk form.
> 
> I concatenate a very large .sqfs file onto the end of "vmlinuz", which gets
> loaded into initrd automatically by the bootloader. Then in my initramfs (cpio
> archive that's compiled in with the kernel), my /sbin/init executable copies
> /initrd.image to /dev/ram0, mounts a tmpfs overlay on top of it, then does a
> pivot root to it.
> 
> This gives it the appearance of a read-write initramfs filesystem, but the
> lower layer data remains compressed in RAM. This saves quite a bit of RAM
> during runtime, which is still yet important on older PCs.
> 
> If there's a better (more official) way of having a real compressed initramfs
> that remains compressed during runtime, I'm all for it. But until then, I would
> like to ask you to please not remove the initrd functionality.
> 
> (In fact, I was actually thinking about trying this method with erofs as the
> lower layer filesystem someday soon instead of squashfs. But I would still be
> using an overlay to mount it, instead of the auto-detect method addressed by
> this patch.)

Something a bit out of the topic, to quota the previous reply from
Christiph:

> There is no reason to fake up a block device, just have a version
> of erofs that directly points to pre-loaded kernel memory instead. 

I completely agree with that point. However, since EROFS is a
block-based filesystem (Thanks to strictly block alignment, meta(data)
can work efficiently on both block-addressed storage
devices and byte-addressed memory devices. Also if the fsblock size
is set as the page size, the page cache itself can also be avoided
for plain inodes), I think even pre-loaded kernel memory is used,
a unified set of block-based kAPIs to access different backends
(e.g., block devices, kernel memory, etc.) is useful for block-based
filesystems instead of developing another dax_direct_access() path
just as pmem-specialized filesystems.

In short, in my opinion, the current bio interfaces fit the
requirements well for various storage for block-based filesystems.

As for EROFS initrd support, I've seen several requests on this,
although I think the interesting point is data integrity and
security since the golden image can be easier protected compared to
tmpfs-based initramfs. It is just my own opinion though, if initrd
survives in the foresee future, I do hope initrd could be an
intermediate solution to initdax support since it just needs a few
line of changes, but if initrd removes soon, just forget what I said.

Thanks,
Gao Xiang

> 
> Thank you,
>   -Byron

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On 2025/8/26 23:32, Gao Xiang wrote:
> 
> 
> On 2025/8/26 22:21, Byron Stanoszek wrote:
>> On Tue, 26 Aug 2025, Christoph Hellwig wrote:
>>
>>> On Mon, Aug 25, 2025 at 09:27:13PM +0300, Askar Safin wrote:
>>>>> We've been trying to kill off initrd in favor of initramfs for about
>>>>> two decades.  I don't think adding new file system support to it is
>>>>> helpful.
>>>>
>>>> I totally agree.
>>>>
>>>> What prevents us from removing initrd right now?
>>>>
>>>> The only reason is lack of volunteers?
>>>>
>>>> If yes, then may I remove initrd?
>>>
>>> Give it a spin and see if anyone shouts.
>>
>> Well, this makes me a little sad. I run several hundred embedded systems out in
>> the world, and I use a combination of initrd and initramfs for booting. These
>> systems operate entirely in ramdisk form.
>>
>> I concatenate a very large .sqfs file onto the end of "vmlinuz", which gets
>> loaded into initrd automatically by the bootloader. Then in my initramfs (cpio
>> archive that's compiled in with the kernel), my /sbin/init executable copies
>> /initrd.image to /dev/ram0, mounts a tmpfs overlay on top of it, then does a
>> pivot root to it.
>>
>> This gives it the appearance of a read-write initramfs filesystem, but the
>> lower layer data remains compressed in RAM. This saves quite a bit of RAM
>> during runtime, which is still yet important on older PCs.
>>
>> If there's a better (more official) way of having a real compressed initramfs
>> that remains compressed during runtime, I'm all for it. But until then, I would
>> like to ask you to please not remove the initrd functionality.
>>
>> (In fact, I was actually thinking about trying this method with erofs as the
>> lower layer filesystem someday soon instead of squashfs. But I would still be
>> using an overlay to mount it, instead of the auto-detect method addressed by
>> this patch.)
> 
> Something a bit out of the topic, to quota the previous reply from
> Christiph:
> 
>> There is no reason to fake up a block device, just have a version
>> of erofs that directly points to pre-loaded kernel memory instead. 
> 
> I completely agree with that point. However, since EROFS is a
> block-based filesystem (Thanks to strictly block alignment, meta(data)
> can work efficiently on both block-addressed storage
> devices and byte-addressed memory devices. Also if the fsblock size
> is set as the page size, the page cache itself can also be avoided
> for plain inodes), I think even pre-loaded kernel memory is used,
> a unified set of block-based kAPIs to access different backends
> (e.g., block devices, kernel memory, etc.) is useful for block-based
> filesystems instead of developing another dax_direct_access() path
> just as pmem-specialized filesystems.
> 
> In short, in my opinion, the current bio interfaces fit the
> requirements well for various storage for block-based filesystems.

refine a bit: for data/metadata direct access, dax_direct_access()
and page cache apis are simple and efficient enough, but if on-disk
(meta)data needs to be transformed ((de)compressed or {en,de}crypted),
a unified bio interface is simplier than working on these individual
storage.

> 
> As for EROFS initrd support, I've seen several requests on this,
> although I think the interesting point is data integrity and
> security since the golden image can be easier protected compared to
> tmpfs-based initramfs. It is just my own opinion though, if initrd
> survives in the foresee future, I do hope initrd could be an
> intermediate solution to initdax support since it just needs a few
> line of changes, but if initrd removes soon, just forget what I said.
> 
> Thanks,
> Gao Xiang
> 
>>
>> Thank you,
>>   -Byron
> 

Re: [PATCH] initrd: support erofs as initrd

[Askar Safin]

 ---- On Tue, 26 Aug 2025 18:21:50 +0400  Byron Stanoszek <gandalf@winds.org> wrote --- 
 > Well, this makes me a little sad. I run several hundred embedded systems out in
 > the world, and I use a combination of initrd and initramfs for booting. These
 > systems operate entirely in ramdisk form.

Put your squashfs to initramfs. Then do everything as you did before.
I. e. in your /sbin/init copy that squashfs to ramdisk (or, better, loop-mount it), etc.

If I understand you correctly, this will work after removing of initrd.

I will not remove ramdisks. I will remove initramdisks, i. e.
special mounting logic in kernel, which automatically loads and mounts
ramdisk at boot.

-- 
Askar Safin
https://types.pl/@safinaskar

Re: [PATCH] initrd: support erofs as initrd

[Askar Safin]

 ---- On Tue, 26 Aug 2025 19:32:34 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote --- 
 > I completely agree with that point. However, since EROFS is a
 > block-based filesystem (Thanks to strictly block alignment, meta(data)
 > can work efficiently on both block-addressed storage
 > devices and byte-addressed memory devices. Also if the fsblock size

As I said previously, just put your erofs image to initramfs
(or to disk) and then (in your initramfs init) create ramdisk out of it
or loop mount it (both ramdisks and loop devices are block devices).

This way you will have erofs on top of block device.

And you will not depend on initrd. (Again: I plan to remove initial ramdisk
support, not ramdisk support per se.)

-- 
Askar Safin
https://types.pl/@safinaskar

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On 2025/8/27 17:22, Askar Safin wrote:
>   ---- On Tue, 26 Aug 2025 19:32:34 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote ---
>   > I completely agree with that point. However, since EROFS is a
>   > block-based filesystem (Thanks to strictly block alignment, meta(data)
>   > can work efficiently on both block-addressed storage
>   > devices and byte-addressed memory devices. Also if the fsblock size
> 
> As I said previously, just put your erofs image to initramfs
> (or to disk) and then (in your initramfs init) create ramdisk out of it
> or loop mount it (both ramdisks and loop devices are block devices).
> 
> This way you will have erofs on top of block device.
> 
> And you will not depend on initrd. (Again: I plan to remove initial ramdisk
> support, not ramdisk support per se.)

It doesn't work if end users put `init` into erofs image and sign
the whole erofs initram images with their certifications.

And it doesn't have any relationship with cpio because users need
signed image and load from memory.

Thanks,
Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On 2025/8/27 17:48, Gao Xiang wrote:
> 
> 
> On 2025/8/27 17:22, Askar Safin wrote:
>>   ---- On Tue, 26 Aug 2025 19:32:34 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote ---
>>   > I completely agree with that point. However, since EROFS is a
>>   > block-based filesystem (Thanks to strictly block alignment, meta(data)
>>   > can work efficiently on both block-addressed storage
>>   > devices and byte-addressed memory devices. Also if the fsblock size
>>
>> As I said previously, just put your erofs image to initramfs
>> (or to disk) and then (in your initramfs init) create ramdisk out of it
>> or loop mount it (both ramdisks and loop devices are block devices).
>>
>> This way you will have erofs on top of block device.
>>
>> And you will not depend on initrd. (Again: I plan to remove initial ramdisk
>> support, not ramdisk support per se.)
> 
> It doesn't work if end users put `init` into erofs image and sign
> the whole erofs initram images with their certifications.
> 
> And it doesn't have any relationship with cpio because users need
> signed image and load from memory.

Again, I don't think initramfs is the same concept as a golden
signed initramdisk, and cpio doesn't support those advanced usage.

The additional cpio extraction destroys bit-for-bit identical data
protection, or some other new verification approach is needed for
initramfs tmpfs.

Thanks,
Gao Xiang

> 
> Thanks,
> Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[Askar Safin]

 ---- On Wed, 27 Aug 2025 13:58:02 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote --- 
 > The additional cpio extraction destroys bit-for-bit identical data
 > protection, or some other new verification approach is needed for
 > initramfs tmpfs.

Put erofs to initramfs and sign whole thing.

Also: initramfs's are concatenatable.
So, you can put erofs to cpio and sign the result.
And then concatenate that cpio with another cpio (with init).

Also, you can put erofs to cpio, then sign this thing, and then add init to kernel
built-in cpio (via INITRAMFS_SOURCE).

In fact, this built-in initramfs (INITRAMFS_SOURCE) is very powerful thing.
You can specify there arbitrary boot logic, and have that initramfs inside kernel
image.

You can even specify logic there for checking signature of erofs (not cpio, but erofs itself).

Also, if all these is still not helpful, then try to describe your use case in more details.
I still don't understand what is wrong with signing cpio, which contains erofs.
Yes, this will slightly complicate pipeline for building erofs. Now you will
have to put it to cpio and then sign that cpio. So what?

Also, if your users want to have their own init inside that erofs, then
you can just write trivial init, which calls their init.
And you can put that trivial init to cpio, which is build-in in kernel via INITRAMFS_SOURCE.

Also, when I hear "sign, signature, something something", then UKIs show up in my mind.
( https://uapi-group.org/specifications/specs/unified_kernel_image/ ).
Maybe they are somehow helpful? That page talks a lot about signatures, measuring into
TPM, etc. (Every time that page says "initrd", they mean initramfs, of course.)

-- 
Askar Safin
https://types.pl/@safinaskar

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On 2025/8/29 00:44, Askar Safin wrote:
>   ---- On Wed, 27 Aug 2025 13:58:02 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote ---
>   > The additional cpio extraction destroys bit-for-bit identical data
>   > protection, or some other new verification approach is needed for
>   > initramfs tmpfs.
> 
> Put erofs to initramfs and sign whole thing.
> 
> Also: initramfs's are concatenatable.
> So, you can put erofs to cpio and sign the result.
> And then concatenate that cpio with another cpio (with init).
> 
> Also, you can put erofs to cpio, then sign this thing, and then add init to kernel
> built-in cpio (via INITRAMFS_SOURCE).

Which part of the running system check the cpio signature.

Why users need some cpio format (which even cannot be random accessed)
since it already contains a real filesystem, also which part check the
signature of `init` itself before `init` runs?  IOWs, why `init` in
cpio can be trusted to run?

Why users need to extract the whole cpio to tmpfs just for some data
part in the erofs? even some data is never used?

Why the initrd memory cannot be used directly as the dax filesystem
instead of copying to tmpfs instead?

Thanks,
Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On 2025/8/29 01:00, Gao Xiang wrote:
> 
> 
> On 2025/8/29 00:44, Askar Safin wrote:
>>   ---- On Wed, 27 Aug 2025 13:58:02 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote ---
>>   > The additional cpio extraction destroys bit-for-bit identical data
>>   > protection, or some other new verification approach is needed for
>>   > initramfs tmpfs.
>>
>> Put erofs to initramfs and sign whole thing.
>>
>> Also: initramfs's are concatenatable.
>> So, you can put erofs to cpio and sign the result.
>> And then concatenate that cpio with another cpio (with init).
>>
>> Also, you can put erofs to cpio, then sign this thing, and then add init to kernel
>> built-in cpio (via INITRAMFS_SOURCE).
> 
> Which part of the running system check the cpio signature.

Anyway, built-in cpio may resolve the issue (honestly, I've never tried
this feature), but I'm not sure all users would like to use this way to
bind the customized `init` to the kernel.

Again, I don't have any strong opinion to kill initrd entirely because
I think initdax may be more efficient and I don't have any time to work
on this part -- it's unrelated to my job.

Personally I just don't understand why cpio stands out considering it
even the format itself doesn't support xattrs and more.

Thanks,
Gao Xiang

> 
> Why users need some cpio format (which even cannot be random accessed)
> since it already contains a real filesystem, also which part check the
> signature of `init` itself before `init` runs?  IOWs, why `init` in
> cpio can be trusted to run?
> 
> Why users need to extract the whole cpio to tmpfs just for some data
> part in the erofs? even some data is never used?
> 
> Why the initrd memory cannot be used directly as the dax filesystem
> instead of copying to tmpfs instead?
> 
> Thanks,
> Gao Xiang

Re: [PATCH] initrd: support erofs as initrd

[Askar Safin]

 ---- On Thu, 28 Aug 2025 21:14:34 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote --- 
 > Which part of the running system check the cpio signature.

You mean who checks cpio signature at boot?
Ideally, bootloader should do this.

For example, as well as I understand, UKI's EFI stub checks
initramfs signature. (See
https://github.com/systemd/systemd/blob/main/docs/ROOTFS_DISCOVERY.md
).

It seems that this document (ROOTFS_DISCOVERY) covers
zillions of use cases, so I hope you will find something for you.

I also added to CC Poettering and systemd, hopefully they have some
ideas.

 > Why users need to extract the whole cpio to tmpfs just for some data
 > part in the erofs? even some data is never used?

Initramfs should be small. Time for extracting should not matter.
If initramfs is too big, and time becomes issue, you are doing it
wrong.

Point of initramfs is to be transition stage and then get to real root.
Initramfs should not be feature-rich and should not be big.

 > Personally I just don't understand why cpio stands out considering it
 > even the format itself doesn't support xattrs and more.

As I said above, initramfs should not be feature-rich.

(But xattrs can be added to it, if needed.)

-- 
Askar Safin
https://types.pl/@safinaskar

Re: [PATCH] initrd: support erofs as initrd

[Gao Xiang]

On Sat, Aug 30, 2025 at 03:49:48PM +0400, Askar Safin wrote:
>  ---- On Thu, 28 Aug 2025 21:14:34 +0400  Gao Xiang <hsiangkao@linux.alibaba.com> wrote --- 
>  > Which part of the running system check the cpio signature.
> 
> You mean who checks cpio signature at boot?
> Ideally, bootloader should do this.

The kernel shouldn't trust the bootloader even the bootloader
checked before, it should re-check itself to re-ensure that.

Ideally, the running system should have a way to check itself
is in a safe environment and the data provided by the bootloader
is genuine, otherwise some dedicated malicious bootloader could
have a way to do some bad behavior.

> 
> For example, as well as I understand, UKI's EFI stub checks
> initramfs signature. (See
> https://github.com/systemd/systemd/blob/main/docs/ROOTFS_DISCOVERY.md
> ).
> 
> It seems that this document (ROOTFS_DISCOVERY) covers
> zillions of use cases, so I hope you will find something for you.

I've said I have no time to look into this.

> 
> I also added to CC Poettering and systemd, hopefully they have some
> ideas.
> 

...

> 
>  > Personally I just don't understand why cpio stands out considering it
>  > even the format itself doesn't support xattrs and more.
> 
> As I said above, initramfs should not be feature-rich.
> 
> (But xattrs can be added to it, if needed.)

The point is:

AFAIK, There is no formal standard for POSIX cpio to implement xattrs,
IOWs, it will be some non-standard cpio just for kernel initramfs
cases, and the new added xattr code has no other usage: just for
temporary booting use case.

Take a look at `init/initramfs.c`, the entire file is just for
unpacking an unaligned/in-random-accessable cpio even it cannot be
called as `cpiofs`.

There are many real filesystems with enough features in the kernel
can help on the same use cases and data doesn't need to be moved
from unaligned cpio to tmpfs. It just sounds like a net win to me.

As for your initramfs size assumption, I don't want to comment on
this, because I'm not distribution guy, I don't know but since
users already claimed they use compressed initrd and they don't want
to unpack them all in one shot, it sounds to me that it may not be
so small.

Thanks,
Gao Xiang

> 
> -- 
> Askar Safin
> https://types.pl/@safinaskar
>