[PATCH] ref_tracker: add stack_depot_save() failure handling to ref_tracker_alloc()

[PATCH] ref_tracker: add stack_depot_save() failure handling to ref_tracker_alloc()

[Tetsuo Handa]

stack_depot_save() cannot accept __GFP_NOFAIL flag because
__stack_depot_save() drops gfp flags which are not in
GFP_KERNEL | GFP_ATOMIC | __GFP_NOWARN. Also, changing
__stack_depot_save() to accept __GFP_NOFAIL is not possible
because rmqueue() does not want __GFP_NOFAIL flag for
order == DEPOT_POOL_ORDER allocation request.

Therefore, assume that stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) from
ref_tracker_alloc() can silently fail, and emit "unreliable refcount
tracker." message.

Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
 lib/ref_tracker.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c
index dc7b14aa3431..ad48ff19adb2 100644
--- a/lib/ref_tracker.c
+++ b/lib/ref_tracker.c
@@ -84,12 +84,18 @@ int ref_tracker_alloc(struct ref_tracker_dir *dir,
 		gfp_mask |= __GFP_NOFAIL;
 	*trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask);
 	if (unlikely(!tracker)) {
+nomem:
 		pr_err_once("memory allocation failure, unreliable refcount tracker.\n");
 		refcount_inc(&dir->untracked);
 		return -ENOMEM;
 	}
 	nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 1);
 	tracker->alloc_stack_handle = stack_depot_save(entries, nr_entries, gfp);
+	if (!tracker->alloc_stack_handle) {
+		*trackerp = NULL;
+		kfree(tracker);
+		goto nomem;
+	}
 
 	spin_lock_irqsave(&dir->lock, flags);
 	list_add(&tracker->head, &dir->list);
-- 
2.18.4

Re: [PATCH] ref_tracker: add stack_depot_save() failure handling to ref_tracker_alloc()

[Jakub Kicinski]

On Sat, 27 May 2023 20:04:11 +0900 Tetsuo Handa wrote:
> stack_depot_save() cannot accept __GFP_NOFAIL flag because
> __stack_depot_save() drops gfp flags which are not in
> GFP_KERNEL | GFP_ATOMIC | __GFP_NOWARN. Also, changing
> __stack_depot_save() to accept __GFP_NOFAIL is not possible
> because rmqueue() does not want __GFP_NOFAIL flag for
> order == DEPOT_POOL_ORDER allocation request.
> 
> Therefore, assume that stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) from
> ref_tracker_alloc() can silently fail, and emit "unreliable refcount
> tracker." message.

It's probably a good idea to CC netdev@vger. I'm not sure if anyone
will pick this up from LKML. 

For the patch itself - I'm not sure it's needed, even if we don't
record the stack we'll have a tracker object and still detect the leak.
So printing the "unreliable refcount" message is not very precise. 
At least to me; Eric's opinion matters most.

Re: [PATCH] ref_tracker: add stack_depot_save() failure handling to ref_tracker_alloc()

[Eric Dumazet]

On Tue, May 30, 2023 at 4:05 AM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Sat, 27 May 2023 20:04:11 +0900 Tetsuo Handa wrote:
> > stack_depot_save() cannot accept __GFP_NOFAIL flag because
> > __stack_depot_save() drops gfp flags which are not in
> > GFP_KERNEL | GFP_ATOMIC | __GFP_NOWARN. Also, changing
> > __stack_depot_save() to accept __GFP_NOFAIL is not possible
> > because rmqueue() does not want __GFP_NOFAIL flag for
> > order == DEPOT_POOL_ORDER allocation request.
> >
> > Therefore, assume that stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) from
> > ref_tracker_alloc() can silently fail, and emit "unreliable refcount
> > tracker." message.
>
> It's probably a good idea to CC netdev@vger. I'm not sure if anyone
> will pick this up from LKML.
>
> For the patch itself - I'm not sure it's needed, even if we don't
> record the stack we'll have a tracker object and still detect the leak.
> So printing the "unreliable refcount" message is not very precise.
> At least to me; Eric's opinion matters most.

Thanks Jakub (I was on a 3-days week end, computer turned off)

This patch looks wrong to me, or at very least not complete ?

If we really want this, why not remove all the code dealing with
tracker->alloc_stack_handle
being potentially NULL ?

Note: I never assumed stack_depot_save() would enforce/use NOFAIL.

diff --git a/lib/ref_tracker.c b/lib/ref_tracker.c
index dc7b14aa3431e2bf7a97a7e78220f04da144563d..530c51ab31f227a64e1210d108e9780f0bad72f7
100644
--- a/lib/ref_tracker.c
+++ b/lib/ref_tracker.c
@@ -29,8 +29,7 @@ void ref_tracker_dir_exit(struct ref_tracker_dir *dir)
        }
        list_for_each_entry_safe(tracker, n, &dir->list, head) {
                pr_err("leaked reference.\n");
-               if (tracker->alloc_stack_handle)
-                       stack_depot_print(tracker->alloc_stack_handle);
+               stack_depot_print(tracker->alloc_stack_handle);
                leak = true;
                list_del(&tracker->head);
                kfree(tracker);
@@ -53,8 +52,7 @@ void ref_tracker_dir_print(struct ref_tracker_dir *dir,
        list_for_each_entry(tracker, &dir->list, head) {
                if (i < display_limit) {
                        pr_err("leaked reference.\n");
-                       if (tracker->alloc_stack_handle)
-                               stack_depot_print(tracker->alloc_stack_handle);
+                       stack_depot_print(tracker->alloc_stack_handle);
                        i++;
                } else {
                        break;
@@ -124,10 +122,8 @@ int ref_tracker_free(struct ref_tracker_dir *dir,
        spin_lock_irqsave(&dir->lock, flags);
        if (tracker->dead) {
                pr_err("reference already released.\n");
-               if (tracker->alloc_stack_handle) {
-                       pr_err("allocated in:\n");
-                       stack_depot_print(tracker->alloc_stack_handle);
-               }
+               pr_err("allocated in:\n");
+               stack_depot_print(tracker->alloc_stack_handle);
                if (tracker->free_stack_handle) {
                        pr_err("freed in:\n");
                        stack_depot_print(tracker->free_stack_handle);

Re: [PATCH] ref_tracker: add stack_depot_save() failure handling to ref_tracker_alloc()

[Tetsuo Handa]

On 2023/05/30 16:22, Eric Dumazet wrote:
>>> Therefore, assume that stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) from
>>> ref_tracker_alloc() can silently fail, and emit "unreliable refcount
>>> tracker." message.
>
> Note: I never assumed stack_depot_save() would enforce/use NOFAIL.

Hmm, I misread this function.

	if (gfp & __GFP_DIRECT_RECLAIM)
		gfp_mask |= __GFP_NOFAIL; // Or'ing to "gfp_mask" than "gfp".
	*trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask); // <= This is "gfp_mask".
	tracker->alloc_stack_handle = stack_depot_save(entries, nr_entries, gfp); // <= This is "gfp".

So, stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) is not happening.

Then, question becomes whether we want tracker->alloc_stack_handle != NULL or not.
If tracker->alloc_stack_handle == NULL is still useful, this patch will be useless...

Re: [PATCH] ref_tracker: add stack_depot_save() failure handling to ref_tracker_alloc()

[Eric Dumazet]

On Tue, May 30, 2023 at 11:52 AM Tetsuo Handa
<penguin-kernel@i-love.sakura.ne.jp> wrote:
>
> On 2023/05/30 16:22, Eric Dumazet wrote:
> >>> Therefore, assume that stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) from
> >>> ref_tracker_alloc() can silently fail, and emit "unreliable refcount
> >>> tracker." message.
> >
> > Note: I never assumed stack_depot_save() would enforce/use NOFAIL.
>
> Hmm, I misread this function.
>
>         if (gfp & __GFP_DIRECT_RECLAIM)
>                 gfp_mask |= __GFP_NOFAIL; // Or'ing to "gfp_mask" than "gfp".
>         *trackerp = tracker = kzalloc(sizeof(*tracker), gfp_mask); // <= This is "gfp_mask".
>         tracker->alloc_stack_handle = stack_depot_save(entries, nr_entries, gfp); // <= This is "gfp".
>
> So, stack_depot_save(GFP_KERNEL | __GFP_NOFAIL) is not happening.

Yes.1

>
> Then, question becomes whether we want tracker->alloc_stack_handle != NULL or not.
> If tracker->alloc_stack_handle == NULL is still useful, this patch will be useless...
>

I think it is useful to have the tracker (as Jakub hinted).
It is better than nothing.
We even might be able to allocate memory later for the
free_stack_handle which could give us
developers enough clues for bug hunting.

Thanks.