[PATCH v2] initramfs: Expose retained initrd as sysfs file

[PATCH v2] initramfs: Expose retained initrd as sysfs file

[Alexander Graf]

When the kernel command line option "retain_initrd" is set, we do not
free the initrd memory. However, we also don't expose it to anyone for
consumption. That leaves us in a weird situation where the only user of
this feature is ppc64 and arm64 specific kexec tooling.

To make it more generally useful, this patch adds a kobject to the
firmware object that contains the initrd context when "retain_initrd"
is set. That way, we can access the initrd any time after boot from
user space and for example hand it into kexec as --initrd parameter
if we want to reboot the same initrd. Or inspect it directly locally.

With this patch applied, there is a new /sys/firmware/initrd file when
the kernel was booted with an initrd and "retain_initrd" command line
option is set.

Signed-off-by: Alexander Graf <graf@amazon.com>

---

v1 -> v2:

  - Reword commit message to explain the new file path
  - Add a Documentation/ABI/testing/sysfs-firmware-initrd file
---
 .../ABI/testing/sysfs-firmware-initrd          |  8 ++++++++
 .../admin-guide/kernel-parameters.txt          |  5 +++--
 init/initramfs.c                               | 18 +++++++++++++++++-
 3 files changed, 28 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/ABI/testing/sysfs-firmware-initrd

diff --git a/Documentation/ABI/testing/sysfs-firmware-initrd b/Documentation/ABI/testing/sysfs-firmware-initrd
new file mode 100644
index 000000000000..20bf7cf77a19
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-firmware-initrd
@@ -0,0 +1,8 @@
+What:		/sys/firmware/initrd
+Date:		December 2023
+Contact:	Alexander Graf <graf@amazon.com>
+Description:
+		When the kernel was booted with an initrd and the
+		"retain_initrd" option is set on the kernel command
+		line, /sys/firmware/initrd contains the contents of the
+		initrd that the kernel was booted with.
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 65731b060e3f..51575cd31741 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2438,7 +2438,7 @@
 			between unregistering the boot console and initializing
 			the real console.
 
-	keepinitrd	[HW,ARM]
+	keepinitrd	[HW,ARM] See retain_initrd.
 
 	kernelcore=	[KNL,X86,IA-64,PPC]
 			Format: nn[KMGTPE] | nn% | "mirror"
@@ -5580,7 +5580,8 @@
 			Useful for devices that are detected asynchronously
 			(e.g. USB and MMC devices).
 
-	retain_initrd	[RAM] Keep initrd memory after extraction
+	retain_initrd	[RAM] Keep initrd memory after extraction. After boot, it will
+			be accessible via /sys/firmware/initrd.
 
 	retbleed=	[X86] Control mitigation of RETBleed (Arbitrary
 			Speculative Code Execution with Return Instructions)
diff --git a/init/initramfs.c b/init/initramfs.c
index 8d0fd946cdd2..25244e2a5739 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -574,6 +574,16 @@ extern unsigned long __initramfs_size;
 #include <linux/initrd.h>
 #include <linux/kexec.h>
 
+static ssize_t raw_read(struct file *file, struct kobject *kobj,
+			struct bin_attribute *attr, char *buf,
+			loff_t pos, size_t count)
+{
+	memcpy(buf, attr->private + pos, count);
+	return count;
+}
+
+static BIN_ATTR(initrd, 0440, raw_read, NULL, 0);
+
 void __init reserve_initrd_mem(void)
 {
 	phys_addr_t start;
@@ -715,8 +725,14 @@ static void __init do_populate_rootfs(void *unused, async_cookie_t cookie)
 	 * If the initrd region is overlapped with crashkernel reserved region,
 	 * free only memory that is not part of crashkernel region.
 	 */
-	if (!do_retain_initrd && initrd_start && !kexec_free_initrd())
+	if (!do_retain_initrd && initrd_start && !kexec_free_initrd()) {
 		free_initrd_mem(initrd_start, initrd_end);
+	} else if (do_retain_initrd) {
+		bin_attr_initrd.size = initrd_end - initrd_start;
+		bin_attr_initrd.private = (void *)initrd_start;
+		if (sysfs_create_bin_file(firmware_kobj, &bin_attr_initrd))
+			pr_err("Failed to create initrd sysfs file");
+	}
 	initrd_start = 0;
 	initrd_end = 0;
 
-- 
2.40.1




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

Re: [PATCH v2] initramfs: Expose retained initrd as sysfs file

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 3841 bytes --]

On Wed, Dec 06, 2023 at 09:33:23PM +0000, Alexander Graf wrote:
> diff --git a/Documentation/ABI/testing/sysfs-firmware-initrd b/Documentation/ABI/testing/sysfs-firmware-initrd
> new file mode 100644
> index 000000000000..20bf7cf77a19
> --- /dev/null
> +++ b/Documentation/ABI/testing/sysfs-firmware-initrd
> @@ -0,0 +1,8 @@
> +What:		/sys/firmware/initrd
> +Date:		December 2023
> +Contact:	Alexander Graf <graf@amazon.com>
> +Description:
> +		When the kernel was booted with an initrd and the
> +		"retain_initrd" option is set on the kernel command
> +		line, /sys/firmware/initrd contains the contents of the
> +		initrd that the kernel was booted with.
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 65731b060e3f..51575cd31741 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -2438,7 +2438,7 @@
>  			between unregistering the boot console and initializing
>  			the real console.
>  
> -	keepinitrd	[HW,ARM]
> +	keepinitrd	[HW,ARM] See retain_initrd.
>  
>  	kernelcore=	[KNL,X86,IA-64,PPC]
>  			Format: nn[KMGTPE] | nn% | "mirror"
> @@ -5580,7 +5580,8 @@
>  			Useful for devices that are detected asynchronously
>  			(e.g. USB and MMC devices).
>  
> -	retain_initrd	[RAM] Keep initrd memory after extraction
> +	retain_initrd	[RAM] Keep initrd memory after extraction. After boot, it will
> +			be accessible via /sys/firmware/initrd.
>  
>  	retbleed=	[X86] Control mitigation of RETBleed (Arbitrary
>  			Speculative Code Execution with Return Instructions)
> diff --git a/init/initramfs.c b/init/initramfs.c
> index 8d0fd946cdd2..25244e2a5739 100644
> --- a/init/initramfs.c
> +++ b/init/initramfs.c
> @@ -574,6 +574,16 @@ extern unsigned long __initramfs_size;
>  #include <linux/initrd.h>
>  #include <linux/kexec.h>
>  
> +static ssize_t raw_read(struct file *file, struct kobject *kobj,
> +			struct bin_attribute *attr, char *buf,
> +			loff_t pos, size_t count)
> +{
> +	memcpy(buf, attr->private + pos, count);
> +	return count;
> +}
> +
> +static BIN_ATTR(initrd, 0440, raw_read, NULL, 0);
> +
>  void __init reserve_initrd_mem(void)
>  {
>  	phys_addr_t start;
> @@ -715,8 +725,14 @@ static void __init do_populate_rootfs(void *unused, async_cookie_t cookie)
>  	 * If the initrd region is overlapped with crashkernel reserved region,
>  	 * free only memory that is not part of crashkernel region.
>  	 */
> -	if (!do_retain_initrd && initrd_start && !kexec_free_initrd())
> +	if (!do_retain_initrd && initrd_start && !kexec_free_initrd()) {
>  		free_initrd_mem(initrd_start, initrd_end);
> +	} else if (do_retain_initrd) {
> +		bin_attr_initrd.size = initrd_end - initrd_start;
> +		bin_attr_initrd.private = (void *)initrd_start;
> +		if (sysfs_create_bin_file(firmware_kobj, &bin_attr_initrd))
> +			pr_err("Failed to create initrd sysfs file");
> +	}
>  	initrd_start = 0;
>  	initrd_end = 0;
>  

On my Arch Linux system, /sys/firmware/initrd is not same as initramfs image
from /boot partition that is uncompressed. `ls -l` listing shows
(with /tmp/initramfs-boot is unzstd'ed initramfs of the same kernel booted):

```
-r--r----- 1 root root 22967535 Dec  7 19:32 /sys/firmware/initrd
-rw------- 1 root root 40960000 Dec  7 19:26 /tmp/initramfs-boot
```

And thus, `cpio -i -v` listing differs. While in uncompressed initramfs,
I got expected initramfs contents (early userpace for booting), doing the same
to /sys/firmware/initrd only shows Intel microcode.

Regardless, exposing initramfs as advertised in the patch description works for
me.

Tested-by: Bagas Sanjaya <bagasdotme@gmail.com>

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH v2] initramfs: Expose retained initrd as sysfs file

[Alexander Graf]

Hi Bagas,

On 07.12.23 13:37, Bagas Sanjaya wrote:
> On Wed, Dec 06, 2023 at 09:33:23PM +0000, Alexander Graf wrote:
>> diff --git a/Documentation/ABI/testing/sysfs-firmware-initrd b/Documentation/ABI/testing/sysfs-firmware-initrd
>> new file mode 100644
>> index 000000000000..20bf7cf77a19
>> --- /dev/null
>> +++ b/Documentation/ABI/testing/sysfs-firmware-initrd
>> @@ -0,0 +1,8 @@
>> +What:		/sys/firmware/initrd
>> +Date:		December 2023
>> +Contact:	Alexander Graf <graf@amazon.com>
>> +Description:
>> +		When the kernel was booted with an initrd and the
>> +		"retain_initrd" option is set on the kernel command
>> +		line, /sys/firmware/initrd contains the contents of the
>> +		initrd that the kernel was booted with.
>> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
>> index 65731b060e3f..51575cd31741 100644
>> --- a/Documentation/admin-guide/kernel-parameters.txt
>> +++ b/Documentation/admin-guide/kernel-parameters.txt
>> @@ -2438,7 +2438,7 @@
>>   			between unregistering the boot console and initializing
>>   			the real console.
>>   
>> -	keepinitrd	[HW,ARM]
>> +	keepinitrd	[HW,ARM] See retain_initrd.
>>   
>>   	kernelcore=	[KNL,X86,IA-64,PPC]
>>   			Format: nn[KMGTPE] | nn% | "mirror"
>> @@ -5580,7 +5580,8 @@
>>   			Useful for devices that are detected asynchronously
>>   			(e.g. USB and MMC devices).
>>   
>> -	retain_initrd	[RAM] Keep initrd memory after extraction
>> +	retain_initrd	[RAM] Keep initrd memory after extraction. After boot, it will
>> +			be accessible via /sys/firmware/initrd.
>>   
>>   	retbleed=	[X86] Control mitigation of RETBleed (Arbitrary
>>   			Speculative Code Execution with Return Instructions)
>> diff --git a/init/initramfs.c b/init/initramfs.c
>> index 8d0fd946cdd2..25244e2a5739 100644
>> --- a/init/initramfs.c
>> +++ b/init/initramfs.c
>> @@ -574,6 +574,16 @@ extern unsigned long __initramfs_size;
>>   #include <linux/initrd.h>
>>   #include <linux/kexec.h>
>>   
>> +static ssize_t raw_read(struct file *file, struct kobject *kobj,
>> +			struct bin_attribute *attr, char *buf,
>> +			loff_t pos, size_t count)
>> +{
>> +	memcpy(buf, attr->private + pos, count);
>> +	return count;
>> +}
>> +
>> +static BIN_ATTR(initrd, 0440, raw_read, NULL, 0);
>> +
>>   void __init reserve_initrd_mem(void)
>>   {
>>   	phys_addr_t start;
>> @@ -715,8 +725,14 @@ static void __init do_populate_rootfs(void *unused, async_cookie_t cookie)
>>   	 * If the initrd region is overlapped with crashkernel reserved region,
>>   	 * free only memory that is not part of crashkernel region.
>>   	 */
>> -	if (!do_retain_initrd && initrd_start && !kexec_free_initrd())
>> +	if (!do_retain_initrd && initrd_start && !kexec_free_initrd()) {
>>   		free_initrd_mem(initrd_start, initrd_end);
>> +	} else if (do_retain_initrd) {
>> +		bin_attr_initrd.size = initrd_end - initrd_start;
>> +		bin_attr_initrd.private = (void *)initrd_start;
>> +		if (sysfs_create_bin_file(firmware_kobj, &bin_attr_initrd))
>> +			pr_err("Failed to create initrd sysfs file");
>> +	}
>>   	initrd_start = 0;
>>   	initrd_end = 0;
>>   
> On my Arch Linux system, /sys/firmware/initrd is not same as initramfs image
> from /boot partition that is uncompressed. `ls -l` listing shows
> (with /tmp/initramfs-boot is unzstd'ed initramfs of the same kernel booted):
>
> ```
> -r--r----- 1 root root 22967535 Dec  7 19:32 /sys/firmware/initrd
> -rw------- 1 root root 40960000 Dec  7 19:26 /tmp/initramfs-boot
> ```
>
> And thus, `cpio -i -v` listing differs. While in uncompressed initramfs,
> I got expected initramfs contents (early userpace for booting), doing the same
> to /sys/firmware/initrd only shows Intel microcode.
>
> Regardless, exposing initramfs as advertised in the patch description works for
> me.


Thanks a bunch for testing the patch!

The reason you're seeing microcode is that something in your boot chain 
(grub maybe? sd-boot?) sends multiple initrd blobs to Linux: One that 
contains microcode and another that contains the real initrd. Linux 
continues extracting past the first cpio archive.


Alex




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

Re: [PATCH v2] initramfs: Expose retained initrd as sysfs file

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 1281 bytes --]

On Fri, Dec 08, 2023 at 12:54:18AM +0100, Alexander Graf wrote:
> Hi Bagas,
> 
> On 07.12.23 13:37, Bagas Sanjaya wrote:
> > On my Arch Linux system, /sys/firmware/initrd is not same as initramfs image
> > from /boot partition that is uncompressed. `ls -l` listing shows
> > (with /tmp/initramfs-boot is unzstd'ed initramfs of the same kernel booted):
> > 
> > ```
> > -r--r----- 1 root root 22967535 Dec  7 19:32 /sys/firmware/initrd
> > -rw------- 1 root root 40960000 Dec  7 19:26 /tmp/initramfs-boot
> > ```
> > 
> > And thus, `cpio -i -v` listing differs. While in uncompressed initramfs,
> > I got expected initramfs contents (early userpace for booting), doing the same
> > to /sys/firmware/initrd only shows Intel microcode.
> > 
> > Regardless, exposing initramfs as advertised in the patch description works for
> > me.
> 
> 
> Thanks a bunch for testing the patch!
> 
> The reason you're seeing microcode is that something in your boot chain
> (grub maybe? sd-boot?) sends multiple initrd blobs to Linux: One that
> contains microcode and another that contains the real initrd. Linux
> continues extracting past the first cpio archive.
> 

Yes, I use grub on my setup.

Ciao!

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]