Re: [rcu:dev.2019.09.23a 62/77] kernel/rcu/tree.c:2882 kfree_call_rcu() warn: inconsistent returns 'irqsave:flags'.

[Joel Fernandes <joel@joelfernandes.org>]

On Tue, Sep 24, 2019 at 09:03:48AM -0700, Paul E. McKenney wrote:
> On Tue, Sep 24, 2019 at 08:15:12AM -0400, Joel Fernandes wrote:
> > On Tue, Sep 24, 2019 at 4:05 AM Dan Carpenter <dan.carpenter@oracle.com> wrote:
> > >
> > > tree:   https://kernel.googlesource.com/pub/scm/linux/kernel/git/paulmck/linux-rcu.git dev.2019.09.23a
> > > head:   97de53b94582c208ee239178b208b8e8b9472585
> > > commit: 39676bb72323718a9e7bab1ba9c4a68319a96f62 [62/77] rcu: Add support for debug_objects debugging for kfree_rcu()
> > >
> > > If you fix the issue, kindly add following tag
> > > Reported-by: kbuild test robot <lkp@intel.com>
> > > Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
> > >
> > > smatch warnings:
> > > kernel/rcu/tree.c:2882 kfree_call_rcu() warn: inconsistent returns 'irqsave:flags'.
> > >   Locked on:   line 2867
> > >   Unlocked on: line 2882
> > >
> > > git remote add rcu https://kernel.googlesource.com/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
> > > git remote update rcu
> > > git checkout 39676bb72323718a9e7bab1ba9c4a68319a96f62
> > > vim +2882 kernel/rcu/tree.c
> > >
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2850) void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2851) {
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2852)        unsigned long flags;
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2853)        struct kfree_rcu_cpu *krcp;
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2854)
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2855)        head->func = func;
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2856)
> > > 87970ccf5a9125 Paul E. McKenney        2019-09-19  2857         local_irq_save(flags);  // For safely calling this_cpu_ptr().
> > >                                                                 ^^^^^^^^^^^^^^^^^^^^^
> > > IRQs disabled.
> > >
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2858)        krcp = this_cpu_ptr(&krc);
> > > ff8db005e371cb Paul E. McKenney        2019-09-19  2859         if (krcp->initialized)
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2860)                spin_lock(&krcp->lock);
> > > 5f5046cc15d514 Joel Fernandes (Google  2019-08-05  2861)
> > > 87970ccf5a9125 Paul E. McKenney        2019-09-19  2862         // Queue the object but don't yet schedule the batch.
> > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2863)        if (debug_rcu_head_queue(head)) {
> > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2864)                // Probable double kfree_rcu(), just leak.
> > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2865)                WARN_ONCE(1, "%s(): Double-freed call. rcu_head %p\n",
> > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2866)                          __func__, head);
> > > 39676bb7232371 Joel Fernandes (Google  2019-09-22  2867)                return;
> > >
> > > Need to re-enable before returning.
> > 
> > Right. At this point the system has a bug anyway since it is a
> > double-free. But yes, no reason to introduce more bugs. Will fix.
> > Thank you, Dan!
> 
> What Joel said!
> 
> As long as I am doing several other reports anyway...
> 
> How does the to-be-squashed patch below look?

Hi Paul,

Sorry for the late reply due to the conference..

This patch looks Ok, the code in your tree though is slightly different mine
I think because you reworked the kfree_rcu() code for working during early
boot.

After my talk tomorrow, I will go through your rcu/dev tree in detail to
review the reworks you did in detail.

Let me know if you have the kfree_rcu() changes in a separate topic branch,
as well. So it will make reviewing easier.

thanks,

 - Joel


> 
> 							Thanx, Paul
> 
> ------------------------------------------------------------------------
> 
> commit 39af4c08038a8a6a7b1c9804fdd2c1921d583222
> Author: Paul E. McKenney <paulmck@kernel.org>
> Date:   Tue Sep 24 09:02:00 2019 -0700
> 
>     squash! rcu: Add support for debug_objects debugging for kfree_rcu()
>     
>     [ paulmck: Fix IRQ per kbuild test robot <lkp@intel.com> feedback. ]
>     Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 921daa7..edb7539 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -2870,7 +2870,7 @@ void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
>  		// Probable double kfree_rcu(), just leak.
>  		WARN_ONCE(1, "%s(): Double-freed call. rcu_head %p\n",
>  			  __func__, head);
> -		return;
> +		goto unlock_return;
>  	}
>  	head->func = func;
>  	head->next = krcp->head;
> @@ -2883,6 +2883,7 @@ void kfree_call_rcu(struct rcu_head *head, rcu_callback_t func)
>  		schedule_delayed_work(&krcp->monitor_work, KFREE_DRAIN_JIFFIES);
>  	}
>  
> +unlock_return:
>  	if (krcp->initialized)
>  		spin_unlock(&krcp->lock);
>  	local_irq_restore(flags);