[PATCH V2 0/4] nvme: init ctrl memleak and debugfs fix

[PATCH V2 0/4] nvme: init ctrl memleak and debugfs fix

[Chaitanya Kulkarni]

Hi,

There is memleak in nvme_ctrl_dhchap_secret_store(), 
nvme_ctrl_dhchap_ctrl_secret_store(), and nvme_init_ctrl().

Fix dhchap memleaks by adding right kfree() :- 

unreferenced object 0xffff8886376ea800 (size 64):
  comm "check", pid 22048, jiffies 4344316705 (age 92.199s)
  hex dump (first 32 bytes):
    44 48 48 43 2d 31 3a 30 30 3a 6e 78 72 35 4b 67  DHHC-1:00:nxr5Kg
    75 58 34 75 6f 41 78 73 4a 61 34 63 2f 68 75 4c  uX4uoAxsJa4c/huL
  backtrace:
    [<0000000030ce5d4b>] __kmalloc+0x4b/0x130
    [<000000009be1cdc1>] nvme_ctrl_dhchap_secret_store+0x8f/0x160 [nvme_core]
    [<00000000ac06c96a>] kernfs_fop_write_iter+0x12b/0x1c0
    [<00000000437e7ced>] vfs_write+0x2ba/0x3c0
    [<00000000f9491baf>] ksys_write+0x5f/0xe0
    [<000000001c46513d>] do_syscall_64+0x3b/0x90
    [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc

Fix dev_pm_qos related memleak in nvme_init_ctrl() with adding 
dev_pm_qos_hide_latency_tolerance() in the error unwind code :-

nvme/044 (Test bi-directional authentication)                [passed]
    runtime  2.111s  ...  2.124s
unreferenced object 0xffff888110c46240 (size 96):
  comm "nvme", pid 33461, jiffies 4345365353 (age 75.586s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [<0000000069ac2cec>] kmalloc_trace+0x25/0x90
    [<000000006acc66d5>] dev_pm_qos_update_user_latency_tolerance+0x6f/0x100
    [<00000000cc376ea7>] nvme_init_ctrl+0x38e/0x410 [nvme_core]
    [<000000007df61b4b>] 0xffffffffc05e88b3
    [<00000000d152b985>] 0xffffffffc05744cb
    [<00000000f04a4041>] vfs_write+0xc5/0x3c0
    [<00000000f9491baf>] ksys_write+0x5f/0xe0
    [<000000001c46513d>] do_syscall_64+0x3b/0x90
    [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc

When blktests are running multiple times we get following messages
consistently :-

[  147.853464] debugfs: Directory 'nvme1' with parent '/' already present!
[  147.853973] nvme1: failed to create debugfs attr
[  148.802490] debugfs: Directory 'nvme1' with parent '/' already present!
[  148.803244] nvme1: failed to create debugfs attr
[  148.877304] debugfs: Directory 'nvme1' with parent '/' already present!
[  148.877775] nvme1: failed to create debugfs attr
[  149.816652] debugfs: Directory 'nvme1' with parent '/' already present!
[  149.818011] nvme1: failed to create debugfs attr

Fix those by adding nvme_fault inject_fini() call in the error unwind
code path.

Still trying to find out device_add() memleak, will send patch soon.

-ck

v2:-

1. In first two patches add kfree() call after strcmp() if block.
2. Add a patch to fix the nvme_ctrl_init() dev_pm_qos memleak.
3. Add a patch to fix the debugfs directory error message.
4. Add tested by tag on patch 1.

Chaitanya Kulkarni (4):
  nvme-core: fix memory leak in dhchap_secret_store
  nvme-core: fix memory leak in dhchap_ctrl_secret
  nvme-core: add missing fault-injection cleanup
  nvme-core: fix dev_pm_qos memleak

 drivers/nvme/host/core.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

-- 
2.40.0

[PATCH V2 1/4] nvme-core: fix memory leak in dhchap_secret_store

[Chaitanya Kulkarni]

Free dhchap_secret in nvme_ctrl_dhchap_secret_store() before we return
fix following kmemleack:-

unreferenced object 0xffff8886376ea800 (size 64):
  comm "check", pid 22048, jiffies 4344316705 (age 92.199s)
  hex dump (first 32 bytes):
    44 48 48 43 2d 31 3a 30 30 3a 6e 78 72 35 4b 67  DHHC-1:00:nxr5Kg
    75 58 34 75 6f 41 78 73 4a 61 34 63 2f 68 75 4c  uX4uoAxsJa4c/huL
  backtrace:
    [<0000000030ce5d4b>] __kmalloc+0x4b/0x130
    [<000000009be1cdc1>] nvme_ctrl_dhchap_secret_store+0x8f/0x160 [nvme_core]
    [<00000000ac06c96a>] kernfs_fop_write_iter+0x12b/0x1c0
    [<00000000437e7ced>] vfs_write+0x2ba/0x3c0
    [<00000000f9491baf>] ksys_write+0x5f/0xe0
    [<000000001c46513d>] do_syscall_64+0x3b/0x90
    [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
unreferenced object 0xffff8886376eaf00 (size 64):
  comm "check", pid 22048, jiffies 4344316736 (age 92.168s)
  hex dump (first 32 bytes):
    44 48 48 43 2d 31 3a 30 30 3a 6e 78 72 35 4b 67  DHHC-1:00:nxr5Kg
    75 58 34 75 6f 41 78 73 4a 61 34 63 2f 68 75 4c  uX4uoAxsJa4c/huL
  backtrace:
    [<0000000030ce5d4b>] __kmalloc+0x4b/0x130
    [<000000009be1cdc1>] nvme_ctrl_dhchap_secret_store+0x8f/0x160 [nvme_core]
    [<00000000ac06c96a>] kernfs_fop_write_iter+0x12b/0x1c0
    [<00000000437e7ced>] vfs_write+0x2ba/0x3c0
    [<00000000f9491baf>] ksys_write+0x5f/0xe0
    [<000000001c46513d>] do_syscall_64+0x3b/0x90
    [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc

Fixes: f50fff73d620 ("nvme: implement In-Band authentication")
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
Tested-by: Yi Zhang <yi.zhang@redhat.com>
---
 drivers/nvme/host/core.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1bfd52eae2ee..c7793991faeb 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3825,8 +3825,10 @@ static ssize_t nvme_ctrl_dhchap_secret_store(struct device *dev,
 		int ret;
 
 		ret = nvme_auth_generate_key(dhchap_secret, &key);
-		if (ret)
+		if (ret) {
+			kfree(dhchap_secret);
 			return ret;
+		}
 		kfree(opts->dhchap_secret);
 		opts->dhchap_secret = dhchap_secret;
 		host_key = ctrl->host_key;
@@ -3834,7 +3836,8 @@ static ssize_t nvme_ctrl_dhchap_secret_store(struct device *dev,
 		ctrl->host_key = key;
 		mutex_unlock(&ctrl->dhchap_auth_mutex);
 		nvme_auth_free_key(host_key);
-	}
+	} else
+		kfree(dhchap_secret);
 	/* Start re-authentication */
 	dev_info(ctrl->device, "re-authenticating controller\n");
 	queue_work(nvme_wq, &ctrl->dhchap_auth_work);
-- 
2.40.0

Re: [PATCH V2 1/4] nvme-core: fix memory leak in dhchap_secret_store

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi@grimberg.me>

[PATCH V2 2/4] nvme-core: fix memory leak in dhchap_ctrl_secret

[Chaitanya Kulkarni]

Free dhchap_secret in nvme_ctrl_dhchap_ctrl_secret_store() before we
return when nvme_auth_generate_key() returns error.

Fixes: f50fff73d620 ("nvme: implement In-Band authentication")
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/nvme/host/core.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index c7793991faeb..42e90d00fc40 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3882,8 +3882,10 @@ static ssize_t nvme_ctrl_dhchap_ctrl_secret_store(struct device *dev,
 		int ret;
 
 		ret = nvme_auth_generate_key(dhchap_secret, &key);
-		if (ret)
+		if (ret) {
+			kfree(dhchap_secret);
 			return ret;
+		}
 		kfree(opts->dhchap_ctrl_secret);
 		opts->dhchap_ctrl_secret = dhchap_secret;
 		ctrl_key = ctrl->ctrl_key;
@@ -3891,7 +3893,8 @@ static ssize_t nvme_ctrl_dhchap_ctrl_secret_store(struct device *dev,
 		ctrl->ctrl_key = key;
 		mutex_unlock(&ctrl->dhchap_auth_mutex);
 		nvme_auth_free_key(ctrl_key);
-	}
+	} else
+		kfree(dhchap_secret);
 	/* Start re-authentication */
 	dev_info(ctrl->device, "re-authenticating controller\n");
 	queue_work(nvme_wq, &ctrl->dhchap_auth_work);
-- 
2.40.0

Re: [PATCH V2 2/4] nvme-core: fix memory leak in dhchap_ctrl_secret

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi@grimberg.me>

[PATCH V2 3/4] nvme-core: add missing fault-injection cleanup

[Chaitanya Kulkarni]

Add missing fault-injection cleanup in nvme_init_ctrl() in the error
unwind path that also fixes following message for blktests:-

linux-block (for-next) # grep debugfs debugfs-err.log
[  147.853464] debugfs: Directory 'nvme1' with parent '/' already present!
[  147.853973] nvme1: failed to create debugfs attr
[  148.802490] debugfs: Directory 'nvme1' with parent '/' already present!
[  148.803244] nvme1: failed to create debugfs attr
[  148.877304] debugfs: Directory 'nvme1' with parent '/' already present!
[  148.877775] nvme1: failed to create debugfs attr
[  149.816652] debugfs: Directory 'nvme1' with parent '/' already present!
[  149.818011] nvme1: failed to create debugfs attr

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 42e90d00fc40..15607281baaa 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -5201,6 +5201,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 
 	return 0;
 out_free_cdev:
+	nvme_fault_inject_fini(&ctrl->fault_inject);
 	cdev_device_del(&ctrl->cdev, ctrl->device);
 out_free_name:
 	nvme_put_ctrl(ctrl);
-- 
2.40.0

Re: [PATCH V2 3/4] nvme-core: add missing fault-injection cleanup

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi@grimberg.me>

Re: [PATCH V2 3/4] nvme-core: add missing fault-injection cleanup

[Yi Zhang]

Confirmed the issue was fixed, feel free to add:

Tested-by: Yi Zhang <yi.zhang@redhat.com>


On Fri, Apr 28, 2023 at 3:40 PM Chaitanya Kulkarni <kch@nvidia.com> wrote:
>
> Add missing fault-injection cleanup in nvme_init_ctrl() in the error
> unwind path that also fixes following message for blktests:-
>
> linux-block (for-next) # grep debugfs debugfs-err.log
> [  147.853464] debugfs: Directory 'nvme1' with parent '/' already present!
> [  147.853973] nvme1: failed to create debugfs attr
> [  148.802490] debugfs: Directory 'nvme1' with parent '/' already present!
> [  148.803244] nvme1: failed to create debugfs attr
> [  148.877304] debugfs: Directory 'nvme1' with parent '/' already present!
> [  148.877775] nvme1: failed to create debugfs attr
> [  149.816652] debugfs: Directory 'nvme1' with parent '/' already present!
> [  149.818011] nvme1: failed to create debugfs attr
>
> Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
> ---
>  drivers/nvme/host/core.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 42e90d00fc40..15607281baaa 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -5201,6 +5201,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
>
>         return 0;
>  out_free_cdev:
> +       nvme_fault_inject_fini(&ctrl->fault_inject);
>         cdev_device_del(&ctrl->cdev, ctrl->device);
>  out_free_name:
>         nvme_put_ctrl(ctrl);
> --
> 2.40.0
>
>


--
Best Regards,
  Yi Zhang

[PATCH V2 4/4] nvme-core: fix dev_pm_qos memleak

[Chaitanya Kulkarni]

Call dev_pm_qos_hide_latency_tolerance() in the error unwind patch to
avoid following kmemleak:-

blktests (master) # kmemleak-clear; ./check nvme/044;
blktests (master) # kmemleak-scan ; kmemleak-show
nvme/044 (Test bi-directional authentication)                [passed]
    runtime  2.111s  ...  2.124s
unreferenced object 0xffff888110c46240 (size 96):
  comm "nvme", pid 33461, jiffies 4345365353 (age 75.586s)
  hex dump (first 32 bytes):
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  backtrace:
    [<0000000069ac2cec>] kmalloc_trace+0x25/0x90
    [<000000006acc66d5>] dev_pm_qos_update_user_latency_tolerance+0x6f/0x100
    [<00000000cc376ea7>] nvme_init_ctrl+0x38e/0x410 [nvme_core]
    [<000000007df61b4b>] 0xffffffffc05e88b3
    [<00000000d152b985>] 0xffffffffc05744cb
    [<00000000f04a4041>] vfs_write+0xc5/0x3c0
    [<00000000f9491baf>] ksys_write+0x5f/0xe0
    [<000000001c46513d>] do_syscall_64+0x3b/0x90
    [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc

Fixes: f50fff73d620 ("nvme: implement In-Band authentication")
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/nvme/host/core.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 15607281baaa..90dc4b832200 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -5202,6 +5202,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	return 0;
 out_free_cdev:
 	nvme_fault_inject_fini(&ctrl->fault_inject);
+	dev_pm_qos_hide_latency_tolerance(ctrl->device);
 	cdev_device_del(&ctrl->cdev, ctrl->device);
 out_free_name:
 	nvme_put_ctrl(ctrl);
-- 
2.40.0

Re: [PATCH V2 4/4] nvme-core: fix dev_pm_qos memleak

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi@grimberg.me>

Re: [PATCH V2 4/4] nvme-core: fix dev_pm_qos memleak

[Yi Zhang]

Verified the kmemleak issue I reported:
https://lore.kernel.org/linux-nvme/CAHj4cs-nDaKzMx2txO4dbE+Mz9ePwLtU0e3egz+StmzOUgWUrA@mail.gmail.com/

Tested-by: Yi Zhang <yi.zhang@redhat.com>


On Fri, Apr 28, 2023 at 3:42 PM Chaitanya Kulkarni <kch@nvidia.com> wrote:
>
> Call dev_pm_qos_hide_latency_tolerance() in the error unwind patch to
> avoid following kmemleak:-
>
> blktests (master) # kmemleak-clear; ./check nvme/044;
> blktests (master) # kmemleak-scan ; kmemleak-show
> nvme/044 (Test bi-directional authentication)                [passed]
>     runtime  2.111s  ...  2.124s
> unreferenced object 0xffff888110c46240 (size 96):
>   comm "nvme", pid 33461, jiffies 4345365353 (age 75.586s)
>   hex dump (first 32 bytes):
>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>   backtrace:
>     [<0000000069ac2cec>] kmalloc_trace+0x25/0x90
>     [<000000006acc66d5>] dev_pm_qos_update_user_latency_tolerance+0x6f/0x100
>     [<00000000cc376ea7>] nvme_init_ctrl+0x38e/0x410 [nvme_core]
>     [<000000007df61b4b>] 0xffffffffc05e88b3
>     [<00000000d152b985>] 0xffffffffc05744cb
>     [<00000000f04a4041>] vfs_write+0xc5/0x3c0
>     [<00000000f9491baf>] ksys_write+0x5f/0xe0
>     [<000000001c46513d>] do_syscall_64+0x3b/0x90
>     [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
>
> Fixes: f50fff73d620 ("nvme: implement In-Band authentication")
> Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
> ---
>  drivers/nvme/host/core.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 15607281baaa..90dc4b832200 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -5202,6 +5202,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
>         return 0;
>  out_free_cdev:
>         nvme_fault_inject_fini(&ctrl->fault_inject);
> +       dev_pm_qos_hide_latency_tolerance(ctrl->device);
>         cdev_device_del(&ctrl->cdev, ctrl->device);
>  out_free_name:
>         nvme_put_ctrl(ctrl);
> --
> 2.40.0
>
>


--
Best Regards,
  Yi Zhang

Re: [PATCH V2 0/4] nvme: init ctrl memleak and debugfs fix

[Chaitanya Kulkarni]

On 4/28/23 00:31, Chaitanya Kulkarni wrote:
> Hi,
>
> There is memleak in nvme_ctrl_dhchap_secret_store(),
> nvme_ctrl_dhchap_ctrl_secret_store(), and nvme_init_ctrl().
>
> Fix dhchap memleaks by adding right kfree() :-
>
> unreferenced object 0xffff8886376ea800 (size 64):
>    comm "check", pid 22048, jiffies 4344316705 (age 92.199s)
>    hex dump (first 32 bytes):
>      44 48 48 43 2d 31 3a 30 30 3a 6e 78 72 35 4b 67  DHHC-1:00:nxr5Kg
>      75 58 34 75 6f 41 78 73 4a 61 34 63 2f 68 75 4c  uX4uoAxsJa4c/huL
>    backtrace:
>      [<0000000030ce5d4b>] __kmalloc+0x4b/0x130
>      [<000000009be1cdc1>] nvme_ctrl_dhchap_secret_store+0x8f/0x160 [nvme_core]
>      [<00000000ac06c96a>] kernfs_fop_write_iter+0x12b/0x1c0
>      [<00000000437e7ced>] vfs_write+0x2ba/0x3c0
>      [<00000000f9491baf>] ksys_write+0x5f/0xe0
>      [<000000001c46513d>] do_syscall_64+0x3b/0x90
>      [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
>
> Fix dev_pm_qos related memleak in nvme_init_ctrl() with adding
> dev_pm_qos_hide_latency_tolerance() in the error unwind code :-
>
> nvme/044 (Test bi-directional authentication)                [passed]
>      runtime  2.111s  ...  2.124s
> unreferenced object 0xffff888110c46240 (size 96):
>    comm "nvme", pid 33461, jiffies 4345365353 (age 75.586s)
>    hex dump (first 32 bytes):
>      00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>      00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>    backtrace:
>      [<0000000069ac2cec>] kmalloc_trace+0x25/0x90
>      [<000000006acc66d5>] dev_pm_qos_update_user_latency_tolerance+0x6f/0x100
>      [<00000000cc376ea7>] nvme_init_ctrl+0x38e/0x410 [nvme_core]
>      [<000000007df61b4b>] 0xffffffffc05e88b3
>      [<00000000d152b985>] 0xffffffffc05744cb
>      [<00000000f04a4041>] vfs_write+0xc5/0x3c0
>      [<00000000f9491baf>] ksys_write+0x5f/0xe0
>      [<000000001c46513d>] do_syscall_64+0x3b/0x90
>      [<00000000ecf348fe>] entry_SYSCALL_64_after_hwframe+0x72/0xdc
>
> When blktests are running multiple times we get following messages
> consistently :-
>
> [  147.853464] debugfs: Directory 'nvme1' with parent '/' already present!
> [  147.853973] nvme1: failed to create debugfs attr
> [  148.802490] debugfs: Directory 'nvme1' with parent '/' already present!
> [  148.803244] nvme1: failed to create debugfs attr
> [  148.877304] debugfs: Directory 'nvme1' with parent '/' already present!
> [  148.877775] nvme1: failed to create debugfs attr
> [  149.816652] debugfs: Directory 'nvme1' with parent '/' already present!
> [  149.818011] nvme1: failed to create debugfs attr
>
> Fix those by adding nvme_fault inject_fini() call in the error unwind
> code path.
>
> Still trying to find out device_add() memleak, will send patch soon.
>
> -ck
>
>

I've generated this series on the linux-block/for-next, forget to
mentioned that in the cover-letter, plz let me know if I should
re-post this on linux-block mailing list...

-ck

Re: [PATCH V2 0/4] nvme: init ctrl memleak and debugfs fix

[Christoph Hellwig]

The whole series looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH V2 0/4] nvme: init ctrl memleak and debugfs fix

[Keith Busch]

Thanks, applied to nvme-6.5.