Patch : drivers/md/dm-verity-target.c

Patch : drivers/md/dm-verity-target.c

[Aaditya Raj Barnwal (QUIC)]

[-- Attachment #1.1: Type: text/plain, Size: 886 bytes --]

Hi Developers,

I have attached the patch for file : drivers/md/dm-verity-target.c

Race condition with some eMMC devices where the early_probe
to dm-verity driver try to check access of data device
fails. As this error is going to impact the bootup and mount
of rootfs. As its impact bootup we make to wait for and
then do a bail-out .
adding the error which show the that dm-init already
ack "all device available " but dm-verity returned
error .
device-mapper: init: all devices available
device-mapper: table: 252:0: verity: Data device lookup failed
device-mapper: ioctl: error adding target to table

This patch is going to wait based on module parm if not passed it
will be 1sec max.

This patch is based on
https://lore.kernel.org/lkml/7442e695-654a-59c2-e3dd-710946e6cddd@seco.com/T/
which tries to do the same to dm-int level.


Thanks
Aaditya




[-- Attachment #1.2: Type: text/html, Size: 4935 bytes --]

[-- Attachment #2: 0001-md-dm-verity-target-Wait-data-devices-giving-time-to.patch --]
[-- Type: application/octet-stream, Size: 2750 bytes --]

From 0220b1b240d60a9433da5f72f6c109e098ad1bcf Mon Sep 17 00:00:00 2001
From: Aaditya Raj Barnwal <quic_abarnwal@quicinc.com>
Date: Mon, 6 Nov 2023 14:49:45 +0530
Subject: [PATCH] md: dm-verity-target: Wait data devices giving time to probe
 ack

Race condition with some devices where the early_probe
to dm-verity driver try to check access of data device
fails. As this error is going to impact the bootup and mount
of rootfs.  As its impact bootup  we make to wait for and
then do a bail-out .
adding the error which show the that dm-init already
ack "all device available " but dm-verity  returned
error .
 device-mapper: init: all devices available
 device-mapper: table: 252:0: verity: Data device lookup failed
 device-mapper: ioctl: error adding target to table

This patch is based on
https://lore.kernel.org/lkml/
7442e695-654a-59c2-e3dd-710946e6cddd@seco.com/T/

which tries to do the same to dm-int level.
---
 drivers/md/dm-verity-target.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index 26adcfea0302..bbe027190487 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -22,6 +22,7 @@
 #include <linux/scatterlist.h>
 #include <linux/string.h>
 #include <linux/jump_label.h>
+#include <linux/delay.h>
 
 #define DM_MSG_PREFIX			"verity"
 
@@ -42,6 +43,9 @@
 #define DM_VERITY_OPTS_MAX		(4 + DM_VERITY_OPTS_FEC + \
 					 DM_VERITY_ROOT_HASH_VERIFICATION_OPTS)
 
+#define DM_DEFAULT_MAX_PROBE_DELAY_SEC 1
+#define DM_DEFAULT_PROBE_FACTOR 5
+
 static unsigned int dm_verity_prefetch_cluster = DM_VERITY_DEFAULT_PREFETCH_SIZE;
 
 module_param_named(prefetch_cluster, dm_verity_prefetch_cluster, uint, 0644);
@@ -1183,7 +1187,7 @@ static int verity_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 	sector_t hash_position;
 	char dummy;
 	char *root_hash_digest_to_validate;
-
+	int loopcntr = DM_DEFAULT_MAX_PROBE_DELAY_SEC * 1000 / DM_DEFAULT_PROBE_FACTOR;
 	v = kzalloc(sizeof(struct dm_verity), GFP_KERNEL);
 	if (!v) {
 		ti->error = "Cannot allocate verity structure";
@@ -1225,7 +1229,17 @@ static int verity_ctr(struct dm_target *ti, unsigned int argc, char **argv)
 	}
 	v->version = num;
 
-	r = dm_get_device(ti, argv[1], BLK_OPEN_READ, &v->data_dev);
+	/*
+	 * Some time we see race condition of early_device probe to
+	 * dm_get_device() leading to failure leading to  mount
+	 * failure of data device specaially when data device is
+	 * eMMC devices (with rootfs)
+	 */
+
+	while ((r = dm_get_device(ti, argv[1], FMODE_READ, &v->data_dev)) && loopcntr) {
+		loopcntr--;
+		msleep_interruptible(5);
+	}
 	if (r) {
 		ti->error = "Data device lookup failed";
 		goto bad;
-- 
2.17.1

Re: Patch : drivers/md/dm-verity-target.c

[Mikulas Patocka]

[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]



On Mon, 6 Nov 2023, Aaditya Raj Barnwal (QUIC) wrote:

> Hi Developers, 
> 
> I have attached the patch for file : drivers/md/dm-verity-target.c
> 
> Race condition with some eMMC devices where the early_probe
> to dm-verity driver try to check access of data device
> fails. As this error is going to impact the bootup and mount
> of rootfs. As its impact bootup we make to wait for and
> then do a bail-out .
> adding the error which show the that dm-init already
> ack "all device available " but dm-verity returned
> error .
> device-mapper: init: all devices available
> device-mapper: table: 252:0: verity: Data device lookup failed
> device-mapper: ioctl: error adding target to table
> 
> This patch is going to wait based on module parm if not passed it
> will be 1sec max.
> 
> This patch is based on
> https://lore.kernel.org/lkml/7442e695-654a-59c2-e3dd-710946e6cddd@seco.com/T/
> which tries to do the same to dm-int level.
> 
> Thanks
> Aaditya

Hi

dm-init already has a parameter "dm-mod.waitfor" that waits for the listed 
devices. So, you can use "dm-mod.waitfor=/dev/mmcblk0" to wait for the 
emmc device that you need.

Mikulas

Re: Patch : drivers/md/dm-verity-target.c

[Mikulas Patocka]

[-- Attachment #1: Type: text/plain, Size: 2825 bytes --]



On Mon, 6 Nov 2023, Aaditya Raj Barnwal wrote:

> thanks for  reply ,  
>
> yes we already have 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=035641b01e72af4f6c6cf22a4bdb5d7dfc4e8e8e 
> and kernel cmdline is with wait for /dev/dm-0 where i could make out 
> from the logs it was able to get /dev/dm-0 but still failed to get the 
> data-devices adding the logs below
> 
>    9.785204][    T9] mmc0: CQHCI version 5.10
> [    9.826746][    T9] mmc0: SDHCI controller on 8804000.sdhci [8804000.sdhci] using ADMA 64-bit
> [    9.835559][    T9] sdhci_msm 8804000.sdhci: mmc0: CQE init: success
> [    9.842578][    T1] device-mapper: init: waiting for device /dev/dm-0 ...

You should wait for /dev/mmcblk0 here, not for /dev/dm-0.

> [    9.849605][    T1] device-mapper: init:  exiting after  /dev/dm-0  <<<<<got  dev/dm-0 

Where does this 'exiting after' message come from? I grepped the kernel 
for it and didn't find it.

> [    9.856689][    T1] device-mapper: init: all devices available
> [    9.863061][    T1] device-mapper: table: 252:0: verity: Data device lookup failed
> [    9.870868][    T1] device-mapper: ioctl: error adding target to table
> [    9.895539][   T37] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
> [    9.898825][    T1] gcc-sdxpinn 80000.clock-controller: sync-state
> [    9.905125][   T37] cfg80211: failed to load regulatory.db
> [    9.911731][    T1] ALSA device list:
> [    9.920911][    T1]   No soundcards found.
> [    9.925133][    T1] TAP version 14
> [    9.928642][    T1] 1..0
> [    9.931645][    T1] md: Waiting for all devices to be available before autodetect
> [    9.931829][   T85] mmc0: Command Queue Engine enabled
> [    9.939343][    T1] md: If you don't use raid, use raid=noautodetect
> [    9.944631][   T85] mmc0: new HS400 MMC card at address 0001
> [    9.951158][    T1] md: Autodetecting RAID arrays.
> [    9.957520][   T85] mmcblk0: mmc0:0001 S0J35A 7.28 GiB
> [    9.961908][    T1] md: autorun ...
> [    9.961912][    T1] md: ... autorun DONE.
> [    9.970173][   T85]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 p29 p30 p31 p32 p33 p34 p35 p36
> p37 p38 p39 p40 p41 p42 p43 p44 p45 p46
> [    9.971004][    T1] Waiting for root device /dev/dm-0...
> [    9.979384][   T85] mmcblk0boot0: mmc0:0001 S0J35A 31.5 MiB
> [   10.005842][   T85] mmcblk0boot1: mmc0:0001 S0J35A 31.5 MiB
> [   10.012536][   T85] mmcblk0rpmb: mmc0:0001 S0J35A 4.00 MiB, chardev (500:0)
> [   10.901495][   T85] msm-dwc3 a600000.ssusb: DWC3 in low power mode
> << no rootfs and no further bootup  blocked at this point same as the above path >>

I see that you also use md - please describe the full storage stack that 
you use.

Mikulas

RE: Patch : drivers/md/dm-verity-target.c

[Aaditya Raj Barnwal (QUIC)]

Hi  
Sorry for my late reply 
I was not clear on the ask assuming it to be device configuration , please see below points for your query.

.	Only eMMC chip  Micron product name (S0J35A)  PNM 48(width) CID  value 8GB: 53304A333541h
.	Linux version 5.15.104 compiled for  Arm64 (armv8+ )
.	with sdhci enabled and sdcard support for auto detect

Please let me know if you have any concerns.

Thanks
Aaditya


-----Original Message-----
From: Mikulas Patocka <mpatocka@redhat.com> 
Sent: Monday, November 6, 2023 5:25 PM
To: Aaditya Raj Barnwal <abarnwal@qti.qualcomm.com>
Cc: Aaditya Raj Barnwal (QUIC) <quic_abarnwal@quicinc.com>; agk@redhat.com; snitzer@kernel.org; dm-devel@lists.linux.dev; Akshay Rukmangad <arukmang@qti.qualcomm.com>
Subject: Re: Patch : drivers/md/dm-verity-target.c

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

On Mon, 6 Nov 2023, Aaditya Raj Barnwal wrote:

> thanks for  reply ,
>
> yes we already have
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/com
> mit/?id=035641b01e72af4f6c6cf22a4bdb5d7dfc4e8e8e
> and kernel cmdline is with wait for /dev/dm-0 where i could make out 
> from the logs it was able to get /dev/dm-0 but still failed to get the 
> data-devices adding the logs below
>
>    9.785204][    T9] mmc0: CQHCI version 5.10 [    9.826746][    T9] 
> mmc0: SDHCI controller on 8804000.sdhci [8804000.sdhci] using ADMA 
> 64-bit [    9.835559][    T9] sdhci_msm 8804000.sdhci: mmc0: CQE init: 
> success [    9.842578][    T1] device-mapper: init: waiting for device /dev/dm-0 ...

You should wait for /dev/mmcblk0 here, not for /dev/dm-0.

> [    9.849605][    T1] device-mapper: init:  exiting after  /dev/dm-0  
> <<<<<got  dev/dm-0

Where does this 'exiting after' message come from? I grepped the kernel for it and didn't find it.

> [    9.856689][    T1] device-mapper: init: all devices available [    
> 9.863061][    T1] device-mapper: table: 252:0: verity: Data device 
> lookup failed [    9.870868][    T1] device-mapper: ioctl: error 
> adding target to table [    9.895539][   T37] platform regulatory.0: 
> Direct firmware load for regulatory.db failed with error -2 [    
> 9.898825][    T1] gcc-sdxpinn 80000.clock-controller: sync-state [    
> 9.905125][   T37] cfg80211: failed to load regulatory.db [    9.911731][    T1] ALSA device list:
> [    9.920911][    T1]   No soundcards found.
> [    9.925133][    T1] TAP version 14
> [    9.928642][    T1] 1..0
> [    9.931645][    T1] md: Waiting for all devices to be available 
> before autodetect [    9.931829][   T85] mmc0: Command Queue Engine 
> enabled [    9.939343][    T1] md: If you don't use raid, use 
> raid=noautodetect [    9.944631][   T85] mmc0: new HS400 MMC card at 
> address 0001 [    9.951158][    T1] md: Autodetecting RAID arrays.
> [    9.957520][   T85] mmcblk0: mmc0:0001 S0J35A 7.28 GiB [    
> 9.961908][    T1] md: autorun ...
> [    9.961912][    T1] md: ... autorun DONE.
> [    9.970173][   T85]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 
> p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28 
> p29 p30 p31 p32 p33 p34 p35 p36
> p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 [    9.971004][    T1] Waiting 
> for root device /dev/dm-0...
> [    9.979384][   T85] mmcblk0boot0: mmc0:0001 S0J35A 31.5 MiB [   
> 10.005842][   T85] mmcblk0boot1: mmc0:0001 S0J35A 31.5 MiB [   
> 10.012536][   T85] mmcblk0rpmb: mmc0:0001 S0J35A 4.00 MiB, chardev 
> (500:0) [   10.901495][   T85] msm-dwc3 a600000.ssusb: DWC3 in low 
> power mode << no rootfs and no further bootup  blocked at this point 
> same as the above path >>

I see that you also use md - please describe the full storage stack that you use.

Mikulas

RE: Patch : drivers/md/dm-verity-target.c

[Mikulas Patocka]

[-- Attachment #1: Type: text/plain, Size: 4319 bytes --]



On Fri, 10 Nov 2023, Aaditya Raj Barnwal (QUIC) wrote:

> Hi @Mikulas
> Sorry for my late reply
> I was not clear on the ask assuming it to be device configuration , please see below points for your query.
>  
>  *  Only eMMC chip  Micron product name (S0J35A)  PNM 48(width) CID  value 8GB: 53304A333541h
>  *  Linux version 5.15.104 compiled for  Arm64 (armv8+ )

I did these commands:

$ git checkout v5.15.104
$ grep -r 'exiting after' * 

and grep didn't find anything. But your log shows an 'exiting after' 
message. So, where does this message 'device-mapper: init:  exiting after' 
come from? Are you using clean v5.15.104 or did you patch the kernel with 
some external patch?

Mikulas


>  *  with sdhci enabled and sdcard support for auto detect
>  
> Please let me know if you have any concerns.
>  
> Thanks
> Aaditya
>  
> -----Original Message-----
> From: Mikulas Patocka <mpatocka@redhat.com>
> Sent: Monday, November 6, 2023 5:25 PM
> To: Aaditya Raj Barnwal <abarnwal@qti.qualcomm.com>
> Cc: Aaditya Raj Barnwal (QUIC) <quic_abarnwal@quicinc.com>; agk@redhat.com; snitzer@kernel.org; dm-devel@lists.linux.dev; Akshay Rukmangad <arukmang@qti.qualcomm.com>
> Subject: Re: Patch : drivers/md/dm-verity-target.c
>  
> WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>  
> On Mon, 6 Nov 2023, Aaditya Raj Barnwal wrote:
>  
> > thanks for  reply ,
> >
> > yes we already have
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/com
> > mit/?id=035641b01e72af4f6c6cf22a4bdb5d7dfc4e8e8e
> > and kernel cmdline is with wait for /dev/dm-0 where i could make out
> > from the logs it was able to get /dev/dm-0 but still failed to get the
> > data-devices adding the logs below
> >
> >    9.785204][    T9] mmc0: CQHCI version 5.10 [    9.826746][    T9]
> > mmc0: SDHCI controller on 8804000.sdhci [8804000.sdhci] using ADMA
> > 64-bit [    9.835559][    T9] sdhci_msm 8804000.sdhci: mmc0: CQE init:
> > success [    9.842578][    T1] device-mapper: init: waiting for device /dev/dm-0 ...
>  
> You should wait for /dev/mmcblk0 here, not for /dev/dm-0.
>  
> > [    9.849605][    T1] device-mapper: init:  exiting after  /dev/dm-0 
> > <<<<<got  dev/dm-0
>  
> Where does this 'exiting after' message come from? I grepped the kernel for it and didn't find it.
>  
> > [    9.856689][    T1] device-mapper: init: all devices available [   
> > 9.863061][    T1] device-mapper: table: 252:0: verity: Data device
> > lookup failed [    9.870868][    T1] device-mapper: ioctl: error
> > adding target to table [    9.895539][   T37] platform regulatory.0:
> > Direct firmware load for regulatory.db failed with error -2 [   
> > 9.898825][    T1] gcc-sdxpinn 80000.clock-controller: sync-state [   
> > 9.905125][   T37] cfg80211: failed to load regulatory.db [    9.911731][    T1] ALSA device list:
> > [    9.920911][    T1]   No soundcards found.
> > [    9.925133][    T1] TAP version 14
> > [    9.928642][    T1] 1..0
> > [    9.931645][    T1] md: Waiting for all devices to be available
> > before autodetect [    9.931829][   T85] mmc0: Command Queue Engine
> > enabled [    9.939343][    T1] md: If you don't use raid, use
> > raid=noautodetect [    9.944631][   T85] mmc0: new HS400 MMC card at
> > address 0001 [    9.951158][    T1] md: Autodetecting RAID arrays.
> > [    9.957520][   T85] mmcblk0: mmc0:0001 S0J35A 7.28 GiB [   
> > 9.961908][    T1] md: autorun ...
> > [    9.961912][    T1] md: ... autorun DONE.
> > [    9.970173][   T85]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> > p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28
> > p29 p30 p31 p32 p33 p34 p35 p36
> > p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 [    9.971004][    T1] Waiting
> > for root device /dev/dm-0...
> > [    9.979384][   T85] mmcblk0boot0: mmc0:0001 S0J35A 31.5 MiB [  
> > 10.005842][   T85] mmcblk0boot1: mmc0:0001 S0J35A 31.5 MiB [  
> > 10.012536][   T85] mmcblk0rpmb: mmc0:0001 S0J35A 4.00 MiB, chardev
> > (500:0) [   10.901495][   T85] msm-dwc3 a600000.ssusb: DWC3 in low
> > power mode << no rootfs and no further bootup  blocked at this point
> > same as the above path >>
>  
> I see that you also use md - please describe the full storage stack that you use.
>  
> Mikulas
>  
> 
> 

RE: Patch : drivers/md/dm-verity-target.c

[Mikulas Patocka]

[-- Attachment #1: Type: text/plain, Size: 1063 bytes --]



On Tue, 14 Nov 2023, Mikulas Patocka wrote:

> 
> 
> On Fri, 10 Nov 2023, Aaditya Raj Barnwal (QUIC) wrote:
> 
> > Hi @Mikulas
> > Sorry for my late reply
> > I was not clear on the ask assuming it to be device configuration , please see below points for your query.
> >  
> >  *  Only eMMC chip  Micron product name (S0J35A)  PNM 48(width) CID  value 8GB: 53304A333541h
> >  *  Linux version 5.15.104 compiled for  Arm64 (armv8+ )
> 
> I did these commands:
> 
> $ git checkout v5.15.104
> $ grep -r 'exiting after' * 
> 
> and grep didn't find anything. But your log shows an 'exiting after' 
> message. So, where does this message 'device-mapper: init:  exiting after' 
> come from? Are you using clean v5.15.104 or did you patch the kernel with 
> some external patch?
> 
> Mikulas

BTW. I see that the "dm-mod.waitfor" mechanism was introduced in 6.2, and 
it is not available in 5.15.104.

So I suggest that you upgrade the kernel to at least 6.2, or backport the 
upstream patch 035641b01e72af4f6c6cf22a4bdb5d7dfc4e8e8e to your kernel 
5.15.104.

Mikulas

RE: Patch : drivers/md/dm-verity-target.c

[Aaditya Raj Barnwal (QUIC)]

Sorry if that statement confused you ,  It was debugging print 
that was added to check if the tight while loop has ended or not.
 
Code : dm-init.c 
dm_init_init()
 
	wait_for_device_probe();
	for (i = 0; i < ARRAY_SIZE(waitfor); i++) {
		if (waitfor[i]) {
			dev_t dev;
 
			DMERR("waiting for device %s ...", waitfor[i]);
			while ((dev=  name_to_dev_t(waitfor[i]) !=0))
				fsleep(5000);
		}
 
	DMERR("exiting after %s ",waitfor[i]);  <<<<<<<<<<<here it was added 
	}
 
	if (waitfor[0])
		DMINFO("all devices available");
 
	list_for_each_entry(dev, &devices, list) {
		if (dm_early_create(&dev->dmi, dev->table,
				    dev->target_args_array))
			break;
	}

-----Original Message-----
From: Mikulas Patocka <mpatocka@redhat.com> 
Sent: Tuesday, November 14, 2023 8:07 PM
To: Aaditya Raj Barnwal (QUIC) <quic_abarnwal@quicinc.com>
Cc: agk@redhat.com; snitzer@kernel.org; dm-devel@lists.linux.dev; Ravi Kumar Siddojigari <rsiddoji@qti.qualcomm.com>
Subject: RE: Patch : drivers/md/dm-verity-target.c

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

On Fri, 10 Nov 2023, Aaditya Raj Barnwal (QUIC) wrote:

> Hi @Mikulas
> Sorry for my late reply
> I was not clear on the ask assuming it to be device configuration , please see below points for your query.
>
>  *  Only eMMC chip  Micron product name (S0J35A)  PNM 48(width) CID  
> value 8GB: 53304A333541h
>  *  Linux version 5.15.104 compiled for  Arm64 (armv8+ )

I did these commands:

$ git checkout v5.15.104
$ grep -r 'exiting after' *

and grep didn't find anything. But your log shows an 'exiting after'
message. So, where does this message 'device-mapper: init:  exiting after'
come from? Are you using clean v5.15.104 or did you patch the kernel with some external patch?

Mikulas


>  *  with sdhci enabled and sdcard support for auto detect
>
> Please let me know if you have any concerns.
>
> Thanks
> Aaditya
>
> -----Original Message-----
> From: Mikulas Patocka <mpatocka@redhat.com>
> Sent: Monday, November 6, 2023 5:25 PM
> To: Aaditya Raj Barnwal <abarnwal@qti.qualcomm.com>
> Cc: Aaditya Raj Barnwal (QUIC) <quic_abarnwal@quicinc.com>; 
> agk@redhat.com; snitzer@kernel.org; dm-devel@lists.linux.dev; Akshay 
> Rukmangad <arukmang@qti.qualcomm.com>
> Subject: Re: Patch : drivers/md/dm-verity-target.c
>
> WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.
>
> On Mon, 6 Nov 2023, Aaditya Raj Barnwal wrote:
>
> > thanks for  reply ,
> >
> > yes we already have
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/c
> > om mit/?id=035641b01e72af4f6c6cf22a4bdb5d7dfc4e8e8e
> > and kernel cmdline is with wait for /dev/dm-0 where i could make out 
> > from the logs it was able to get /dev/dm-0 but still failed to get 
> > the data-devices adding the logs below
> >
> >    9.785204][    T9] mmc0: CQHCI version 5.10 [    9.826746][    T9]
> > mmc0: SDHCI controller on 8804000.sdhci [8804000.sdhci] using ADMA  
> >64-bit [    9.835559][    T9] sdhci_msm 8804000.sdhci: mmc0: CQE init:
> > success [    9.842578][    T1] device-mapper: init: waiting for device /dev/dm-0 ...
>
> You should wait for /dev/mmcblk0 here, not for /dev/dm-0.
>
> > [    9.849605][    T1] device-mapper: init:  exiting after  
> > /dev/dm-0 <<<<<got  dev/dm-0
>
> Where does this 'exiting after' message come from? I grepped the kernel for it and didn't find it.
>
> > [    9.856689][    T1] device-mapper: init: all devices available [ 
> > 9.863061][    T1] device-mapper: table: 252:0: verity: Data device 
> > lookup failed [    9.870868][    T1] device-mapper: ioctl: error 
> > adding target to table [    9.895539][   T37] platform regulatory.0:
> > Direct firmware load for regulatory.db failed with error -2 [ 
> > 9.898825][    T1] gcc-sdxpinn 80000.clock-controller: sync-state [ 
> > 9.905125][   T37] cfg80211: failed to load regulatory.db [    9.911731][    T1] ALSA device list:
> > [    9.920911][    T1]   No soundcards found.
> > [    9.925133][    T1] TAP version 14 [    9.928642][    T1] 1..0 [    
> > 9.931645][    T1] md: Waiting for all devices to be available before 
> > autodetect [    9.931829][   T85] mmc0: Command Queue Engine enabled 
> > [    9.939343][    T1] md: If you don't use raid, use 
> > raid=noautodetect [    9.944631][   T85] mmc0: new HS400 MMC card at 
> > address 0001 [    9.951158][    T1] md: Autodetecting RAID arrays.
> > [    9.957520][   T85] mmcblk0: mmc0:0001 S0J35A 7.28 GiB [ 
> > 9.961908][    T1] md: autorun ...
> > [    9.961912][    T1] md: ... autorun DONE.
> > [    9.970173][   T85]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11
> > p12 p13 p14 p15 p16 p17 p18 p19 p20 p21 p22 p23 p24 p25 p26 p27 p28
> > p29 p30 p31 p32 p33 p34 p35 p36
> > p37 p38 p39 p40 p41 p42 p43 p44 p45 p46 [    9.971004][    T1] 
> > Waiting for root device /dev/dm-0...
> > [    9.979384][   T85] mmcblk0boot0: mmc0:0001 S0J35A 31.5 MiB [ 
> > 10.005842][   T85] mmcblk0boot1: mmc0:0001 S0J35A 31.5 MiB [ 
> > 10.012536][   T85] mmcblk0rpmb: mmc0:0001 S0J35A 4.00 MiB, chardev
> > (500:0) [   10.901495][   T85] msm-dwc3 a600000.ssusb: DWC3 in low 
> > power mode << no rootfs and no further bootup  blocked at this point 
> > same as the above path >>
>
> I see that you also use md - please describe the full storage stack that you use.
>
> Mikulas
>
>
>

RE: Patch : drivers/md/dm-verity-target.c

[Aaditya Raj Barnwal]

Sorry if that statement confused you ,  It was debugging print 
that was added to check if the  tight while loop has ended or not
 
Code : dm-init.c 
dm_init_init()
 
	wait_for_device_probe();
	for (i = 0; i < ARRAY_SIZE(waitfor); i++) {
		if (waitfor[i]) {
			dev_t dev;
 
			DMERR("waiting for device %s ...", waitfor[i]);
			while ((dev=  name_to_dev_t(waitfor[i]) !=0))
				fsleep(5000);
		}
 
	DMERR("exiting after %s ",waitfor[i]);  <<<<<<<<<<<here it was added 
	}
 
	if (waitfor[0])
		DMINFO("all devices available");
 
	list_for_each_entry(dev, &devices, list) {
		if (dm_early_create(&dev->dmi, dev->table,
				    dev->target_args_array))
			break;
	}


-----Original Message-----
From: Mikulas Patocka <mpatocka@redhat.com> 
Sent: Tuesday, November 14, 2023 8:23 PM
To: Aaditya Raj Barnwal (QUIC) <quic_abarnwal@quicinc.com>
Cc: agk@redhat.com; snitzer@kernel.org; dm-devel@lists.linux.dev; 
Subject: RE: Patch : drivers/md/dm-verity-target.c

WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

On Tue, 14 Nov 2023, Mikulas Patocka wrote:

>
>
> On Fri, 10 Nov 2023, Aaditya Raj Barnwal (QUIC) wrote:
>
> > Hi @Mikulas
> > Sorry for my late reply
> > I was not clear on the ask assuming it to be device configuration , please see below points for your query.
> >
> >  *  Only eMMC chip  Micron product name (S0J35A)  PNM 48(width) CID  
> > value 8GB: 53304A333541h
> >  *  Linux version 5.15.104 compiled for  Arm64 (armv8+ )
>
> I did these commands:
>
> $ git checkout v5.15.104
> $ grep -r 'exiting after' *
>
> and grep didn't find anything. But your log shows an 'exiting after'
> message. So, where does this message 'device-mapper: init:  exiting after'
> come from? Are you using clean v5.15.104 or did you patch the kernel 
> with some external patch?
>
> Mikulas

BTW. I see that the "dm-mod.waitfor" mechanism was introduced in 6.2, and it is not available in 5.15.104.

So I suggest that you upgrade the kernel to at least 6.2, or backport the upstream patch 035641b01e72af4f6c6cf22a4bdb5d7dfc4e8e8e to your kernel 5.15.104.

Mikulas

Re: Patch : drivers/md/dm-verity-target.c

[Mike Snitzer]

On Wed, Nov 15 2023 at  2:01P -0500,
Aaditya Raj Barnwal <abarnwal@qti.qualcomm.com> wrote:

> Sorry if that statement confused you ,  It was debugging print 
> that was added to check if the  tight while loop has ended or not
>  
> Code : dm-init.c 
> dm_init_init()
>  
> 	wait_for_device_probe();
> 	for (i = 0; i < ARRAY_SIZE(waitfor); i++) {
> 		if (waitfor[i]) {
> 			dev_t dev;
>  
> 			DMERR("waiting for device %s ...", waitfor[i]);
> 			while ((dev=  name_to_dev_t(waitfor[i]) !=0))
> 				fsleep(5000);
> 		}
>  
> 	DMERR("exiting after %s ",waitfor[i]);  <<<<<<<<<<<here it was added 
> 	}
>  
> 	if (waitfor[0])
> 		DMINFO("all devices available");
>  
> 	list_for_each_entry(dev, &devices, list) {
> 		if (dm_early_create(&dev->dmi, dev->table,
> 				    dev->target_args_array))
> 			break;
> 	}
> 


Mikulas wasn't confused.  He was pointing out that you have made
changes to your kernel without sharing what they are.

But your issue is that you configured the dm-mod.waitfor device
incorrectly. You must wait for the underlying physical device(s) that
the virtual DM device depends on.

Mike

Re: Patch : drivers/md/dm-verity-target.c

[Mikulas Patocka]

On Wed, 15 Nov 2023, Mike Snitzer wrote:

> On Wed, Nov 15 2023 at  2:01P -0500,
> Aaditya Raj Barnwal <abarnwal@qti.qualcomm.com> wrote:
> 
> > Sorry if that statement confused you ,  It was debugging print 
> > that was added to check if the  tight while loop has ended or not
> >  
> > Code : dm-init.c 
> > dm_init_init()
> >  
> > 	wait_for_device_probe();
> > 	for (i = 0; i < ARRAY_SIZE(waitfor); i++) {
> > 		if (waitfor[i]) {
> > 			dev_t dev;
> >  
> > 			DMERR("waiting for device %s ...", waitfor[i]);
> > 			while ((dev=  name_to_dev_t(waitfor[i]) !=0))
> > 				fsleep(5000);
> > 		}
> >  
> > 	DMERR("exiting after %s ",waitfor[i]);  <<<<<<<<<<<here it was added 
> > 	}
> >  
> > 	if (waitfor[0])
> > 		DMINFO("all devices available");
> >  
> > 	list_for_each_entry(dev, &devices, list) {
> > 		if (dm_early_create(&dev->dmi, dev->table,
> > 				    dev->target_args_array))
> > 			break;
> > 	}
> > 
> 
> 
> Mikulas wasn't confused.  He was pointing out that you have made
> changes to your kernel without sharing what they are.
> 
> But your issue is that you configured the dm-mod.waitfor device
> incorrectly. You must wait for the underlying physical device(s) that
> the virtual DM device depends on.

Yes, that's the problem.

> Mike

[    9.785204][    T9] mmc0: CQHCI version 5.10
[    9.826746][    T9] mmc0: SDHCI controller on 8804000.sdhci [8804000.sdhci] using ADMA 64-bit
[    9.835559][    T9] sdhci_msm 8804000.sdhci: mmc0: CQE init: success
[    9.842578][    T1] device-mapper: init: waiting for device /dev/dm-0 ...
[    9.849605][    T1] device-mapper: init:  exiting after  /dev/dm-0 <<<<<got  dev/dm-0
[    9.856689][    T1] device-mapper: init: all devices available
[    9.863061][    T1] device-mapper: table: 252:0: verity: Data device lookup failed
[    9.870868][    T1] device-mapper: ioctl: error adding target to table
[    9.895539][   T37]

I'm wondering about another thing - here it succeeds finding /dev/dm-0 and 
immediatelly after that fails loading /dev/dm-0 table with the dm-verity 
target.

So, how could we succeed finding /dev/dm-0 before it was created?

I think that you backported that patch incorrectly - instead of

	while ((dev=  name_to_dev_t(waitfor[i]) !=0)) fsleep(5000);

there should be

	while ((dev=  name_to_dev_t(waitfor[i]) ==0)) fsleep(5000);

There was a change in the return value of devt_from_devname - in 6.7-rc1, 
it returns 0 on success and -ERROR on error. In 5.15.104, it returns 0 on 
error and a device nubmer on success.

Mikulas