Re: [PATCH V3] lockdep: fix deadlock issue between lockdep and rcu

Re: [PATCH V3] lockdep: fix deadlock issue between lockdep and rcu

[Carlos Llamas]

On Fri, Feb 02, 2024 at 04:14:36PM +0800, Zhiguo Niu wrote:
> There is a deadlock scenario between lockdep and rcu when
> rcu nocb feature is enabled, just as following call stack:
> 
>      rcuop/x
> -000|queued_spin_lock_slowpath(lock = 0xFFFFFF817F2A8A80, val = ?)
> -001|queued_spin_lock(inline) // try to hold nocb_gp_lock
> -001|do_raw_spin_lock(lock = 0xFFFFFF817F2A8A80)
> -002|__raw_spin_lock_irqsave(inline)
> -002|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F2A8A80)
> -003|wake_nocb_gp_defer(inline)
> -003|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F30B680)
> -004|__call_rcu_common(inline)
> -004|call_rcu(head = 0xFFFFFFC082EECC28, func = ?)
> -005|call_rcu_zapped(inline)
> -005|free_zapped_rcu(ch = ?)// hold graph lock
> -006|rcu_do_batch(rdp = 0xFFFFFF817F245680)
> -007|nocb_cb_wait(inline)
> -007|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F245680)
> -008|kthread(_create = 0xFFFFFF80803122C0)
> -009|ret_from_fork(asm)
> 
>      rcuop/y
> -000|queued_spin_lock_slowpath(lock = 0xFFFFFFC08291BBC8, val = 0)
> -001|queued_spin_lock()
> -001|lockdep_lock()
> -001|graph_lock() // try to hold graph lock
> -002|lookup_chain_cache_add()
> -002|validate_chain()
> -003|lock_acquire
> -004|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F211D80)
> -005|lock_timer_base(inline)
> -006|mod_timer(inline)
> -006|wake_nocb_gp_defer(inline)// hold nocb_gp_lock
> -006|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F2A8680)
> -007|__call_rcu_common(inline)
> -007|call_rcu(head = 0xFFFFFFC0822E0B58, func = ?)
> -008|call_rcu_hurry(inline)
> -008|rcu_sync_call(inline)
> -008|rcu_sync_func(rhp = 0xFFFFFFC0822E0B58)
> -009|rcu_do_batch(rdp = 0xFFFFFF817F266680)
> -010|nocb_cb_wait(inline)
> -010|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F266680)
> -011|kthread(_create = 0xFFFFFF8080363740)
> -012|ret_from_fork(asm)
> 
> rcuop/x and rcuop/y are rcu nocb threads with the same nocb gp thread.
> This patch release the graph lock before lockdep call_rcu.
> 
> Fixes: a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use")
> Cc: <stable@vger.kernel.org>
> Cc: Boqun Feng <boqun.feng@gmail.com>
> Cc: Waiman Long <longman@redhat.com>
> Cc: Carlos Llamas <cmllamas@google.com>
> Cc: Bart Van Assche <bvanassche@acm.org>
> Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
> Signed-off-by: Xuewen Yan <xuewen.yan@unisoc.com>
> ---
> changes of v3: correct code comments and add Cc tag.
> changes of v2: update patch according to Boqun's suggestions.
> ---

It seems v3 should have collected the review tags from Boqun and Waiman.
Also, I'm actually Cc'ing stable here. I hope that is enough.
FWIW, this looks fine to me.

Reviewed-by: Carlos Llamas <cmllamas@google.com>

Thanks

Re: [PATCH V3] lockdep: fix deadlock issue between lockdep and rcu

[Greg KH]

On Fri, Feb 02, 2024 at 07:55:48PM +0000, Carlos Llamas wrote:
> On Fri, Feb 02, 2024 at 04:14:36PM +0800, Zhiguo Niu wrote:
> > There is a deadlock scenario between lockdep and rcu when
> > rcu nocb feature is enabled, just as following call stack:
> > 
> >      rcuop/x
> > -000|queued_spin_lock_slowpath(lock = 0xFFFFFF817F2A8A80, val = ?)
> > -001|queued_spin_lock(inline) // try to hold nocb_gp_lock
> > -001|do_raw_spin_lock(lock = 0xFFFFFF817F2A8A80)
> > -002|__raw_spin_lock_irqsave(inline)
> > -002|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F2A8A80)
> > -003|wake_nocb_gp_defer(inline)
> > -003|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F30B680)
> > -004|__call_rcu_common(inline)
> > -004|call_rcu(head = 0xFFFFFFC082EECC28, func = ?)
> > -005|call_rcu_zapped(inline)
> > -005|free_zapped_rcu(ch = ?)// hold graph lock
> > -006|rcu_do_batch(rdp = 0xFFFFFF817F245680)
> > -007|nocb_cb_wait(inline)
> > -007|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F245680)
> > -008|kthread(_create = 0xFFFFFF80803122C0)
> > -009|ret_from_fork(asm)
> > 
> >      rcuop/y
> > -000|queued_spin_lock_slowpath(lock = 0xFFFFFFC08291BBC8, val = 0)
> > -001|queued_spin_lock()
> > -001|lockdep_lock()
> > -001|graph_lock() // try to hold graph lock
> > -002|lookup_chain_cache_add()
> > -002|validate_chain()
> > -003|lock_acquire
> > -004|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F211D80)
> > -005|lock_timer_base(inline)
> > -006|mod_timer(inline)
> > -006|wake_nocb_gp_defer(inline)// hold nocb_gp_lock
> > -006|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F2A8680)
> > -007|__call_rcu_common(inline)
> > -007|call_rcu(head = 0xFFFFFFC0822E0B58, func = ?)
> > -008|call_rcu_hurry(inline)
> > -008|rcu_sync_call(inline)
> > -008|rcu_sync_func(rhp = 0xFFFFFFC0822E0B58)
> > -009|rcu_do_batch(rdp = 0xFFFFFF817F266680)
> > -010|nocb_cb_wait(inline)
> > -010|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F266680)
> > -011|kthread(_create = 0xFFFFFF8080363740)
> > -012|ret_from_fork(asm)
> > 
> > rcuop/x and rcuop/y are rcu nocb threads with the same nocb gp thread.
> > This patch release the graph lock before lockdep call_rcu.
> > 
> > Fixes: a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use")
> > Cc: <stable@vger.kernel.org>
> > Cc: Boqun Feng <boqun.feng@gmail.com>
> > Cc: Waiman Long <longman@redhat.com>
> > Cc: Carlos Llamas <cmllamas@google.com>
> > Cc: Bart Van Assche <bvanassche@acm.org>
> > Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
> > Signed-off-by: Xuewen Yan <xuewen.yan@unisoc.com>
> > ---
> > changes of v3: correct code comments and add Cc tag.
> > changes of v2: update patch according to Boqun's suggestions.
> > ---
> 
> It seems v3 should have collected the review tags from Boqun and Waiman.
> Also, I'm actually Cc'ing stable here. I hope that is enough.
> FWIW, this looks fine to me.
> 
> Reviewed-by: Carlos Llamas <cmllamas@google.com>


<formletter>

This is not the correct way to submit patches for inclusion in the
stable kernel tree.  Please read:
    https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
for how to do this properly.

</formletter>

Re: [PATCH V3] lockdep: fix deadlock issue between lockdep and rcu

[Zhiguo Niu]

hi Greg,

On Sat, Feb 3, 2024 at 5:36 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Fri, Feb 02, 2024 at 07:55:48PM +0000, Carlos Llamas wrote:
> > On Fri, Feb 02, 2024 at 04:14:36PM +0800, Zhiguo Niu wrote:
> > > There is a deadlock scenario between lockdep and rcu when
> > > rcu nocb feature is enabled, just as following call stack:
> > >
> > >      rcuop/x
> > > -000|queued_spin_lock_slowpath(lock = 0xFFFFFF817F2A8A80, val = ?)
> > > -001|queued_spin_lock(inline) // try to hold nocb_gp_lock
> > > -001|do_raw_spin_lock(lock = 0xFFFFFF817F2A8A80)
> > > -002|__raw_spin_lock_irqsave(inline)
> > > -002|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F2A8A80)
> > > -003|wake_nocb_gp_defer(inline)
> > > -003|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F30B680)
> > > -004|__call_rcu_common(inline)
> > > -004|call_rcu(head = 0xFFFFFFC082EECC28, func = ?)
> > > -005|call_rcu_zapped(inline)
> > > -005|free_zapped_rcu(ch = ?)// hold graph lock
> > > -006|rcu_do_batch(rdp = 0xFFFFFF817F245680)
> > > -007|nocb_cb_wait(inline)
> > > -007|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F245680)
> > > -008|kthread(_create = 0xFFFFFF80803122C0)
> > > -009|ret_from_fork(asm)
> > >
> > >      rcuop/y
> > > -000|queued_spin_lock_slowpath(lock = 0xFFFFFFC08291BBC8, val = 0)
> > > -001|queued_spin_lock()
> > > -001|lockdep_lock()
> > > -001|graph_lock() // try to hold graph lock
> > > -002|lookup_chain_cache_add()
> > > -002|validate_chain()
> > > -003|lock_acquire
> > > -004|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F211D80)
> > > -005|lock_timer_base(inline)
> > > -006|mod_timer(inline)
> > > -006|wake_nocb_gp_defer(inline)// hold nocb_gp_lock
> > > -006|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F2A8680)
> > > -007|__call_rcu_common(inline)
> > > -007|call_rcu(head = 0xFFFFFFC0822E0B58, func = ?)
> > > -008|call_rcu_hurry(inline)
> > > -008|rcu_sync_call(inline)
> > > -008|rcu_sync_func(rhp = 0xFFFFFFC0822E0B58)
> > > -009|rcu_do_batch(rdp = 0xFFFFFF817F266680)
> > > -010|nocb_cb_wait(inline)
> > > -010|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F266680)
> > > -011|kthread(_create = 0xFFFFFF8080363740)
> > > -012|ret_from_fork(asm)
> > >
> > > rcuop/x and rcuop/y are rcu nocb threads with the same nocb gp thread.
> > > This patch release the graph lock before lockdep call_rcu.
> > >
> > > Fixes: a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use")
> > > Cc: <stable@vger.kernel.org>
> > > Cc: Boqun Feng <boqun.feng@gmail.com>
> > > Cc: Waiman Long <longman@redhat.com>
> > > Cc: Carlos Llamas <cmllamas@google.com>
> > > Cc: Bart Van Assche <bvanassche@acm.org>
> > > Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
> > > Signed-off-by: Xuewen Yan <xuewen.yan@unisoc.com>
> > > ---
> > > changes of v3: correct code comments and add Cc tag.
> > > changes of v2: update patch according to Boqun's suggestions.
> > > ---
> >
> > It seems v3 should have collected the review tags from Boqun and Waiman.
> > Also, I'm actually Cc'ing stable here. I hope that is enough.
> > FWIW, this looks fine to me.
> >
> > Reviewed-by: Carlos Llamas <cmllamas@google.com>
>
>
> <formletter>
>
> This is not the correct way to submit patches for inclusion in the
> stable kernel tree.  Please read:
>     https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
> for how to do this properly.
>
> </formletter>

I see that many commits in mainline use Cc: <stable@vger.kernel.org>
directly without other information,
and I also find this information from above link: "Note, such tagging
is unnecessary if the stable team can
derive the appropriate versions from Fixes: tags."

In addition, this fixed commit "a0b0fd53e1e6 ("locking/lockdep: Free
lock classes that are no longer in use")"
was committed in 2019, so I am not very sure which start version
should be added to stabe tag.
Do you have any good suggestions?
thanks!

Re: [PATCH V3] lockdep: fix deadlock issue between lockdep and rcu

[Greg KH]

On Tue, Feb 06, 2024 at 06:37:05PM +0800, Zhiguo Niu wrote:
> hi Greg,
> 
> On Sat, Feb 3, 2024 at 5:36 AM Greg KH <gregkh@linuxfoundation.org> wrote:
> >
> > On Fri, Feb 02, 2024 at 07:55:48PM +0000, Carlos Llamas wrote:
> > > On Fri, Feb 02, 2024 at 04:14:36PM +0800, Zhiguo Niu wrote:
> > > > There is a deadlock scenario between lockdep and rcu when
> > > > rcu nocb feature is enabled, just as following call stack:
> > > >
> > > >      rcuop/x
> > > > -000|queued_spin_lock_slowpath(lock = 0xFFFFFF817F2A8A80, val = ?)
> > > > -001|queued_spin_lock(inline) // try to hold nocb_gp_lock
> > > > -001|do_raw_spin_lock(lock = 0xFFFFFF817F2A8A80)
> > > > -002|__raw_spin_lock_irqsave(inline)
> > > > -002|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F2A8A80)
> > > > -003|wake_nocb_gp_defer(inline)
> > > > -003|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F30B680)
> > > > -004|__call_rcu_common(inline)
> > > > -004|call_rcu(head = 0xFFFFFFC082EECC28, func = ?)
> > > > -005|call_rcu_zapped(inline)
> > > > -005|free_zapped_rcu(ch = ?)// hold graph lock
> > > > -006|rcu_do_batch(rdp = 0xFFFFFF817F245680)
> > > > -007|nocb_cb_wait(inline)
> > > > -007|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F245680)
> > > > -008|kthread(_create = 0xFFFFFF80803122C0)
> > > > -009|ret_from_fork(asm)
> > > >
> > > >      rcuop/y
> > > > -000|queued_spin_lock_slowpath(lock = 0xFFFFFFC08291BBC8, val = 0)
> > > > -001|queued_spin_lock()
> > > > -001|lockdep_lock()
> > > > -001|graph_lock() // try to hold graph lock
> > > > -002|lookup_chain_cache_add()
> > > > -002|validate_chain()
> > > > -003|lock_acquire
> > > > -004|_raw_spin_lock_irqsave(lock = 0xFFFFFF817F211D80)
> > > > -005|lock_timer_base(inline)
> > > > -006|mod_timer(inline)
> > > > -006|wake_nocb_gp_defer(inline)// hold nocb_gp_lock
> > > > -006|__call_rcu_nocb_wake(rdp = 0xFFFFFF817F2A8680)
> > > > -007|__call_rcu_common(inline)
> > > > -007|call_rcu(head = 0xFFFFFFC0822E0B58, func = ?)
> > > > -008|call_rcu_hurry(inline)
> > > > -008|rcu_sync_call(inline)
> > > > -008|rcu_sync_func(rhp = 0xFFFFFFC0822E0B58)
> > > > -009|rcu_do_batch(rdp = 0xFFFFFF817F266680)
> > > > -010|nocb_cb_wait(inline)
> > > > -010|rcu_nocb_cb_kthread(arg = 0xFFFFFF817F266680)
> > > > -011|kthread(_create = 0xFFFFFF8080363740)
> > > > -012|ret_from_fork(asm)
> > > >
> > > > rcuop/x and rcuop/y are rcu nocb threads with the same nocb gp thread.
> > > > This patch release the graph lock before lockdep call_rcu.
> > > >
> > > > Fixes: a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use")
> > > > Cc: <stable@vger.kernel.org>

Oops, I missed this line ^^^

> > > > Cc: Boqun Feng <boqun.feng@gmail.com>
> > > > Cc: Waiman Long <longman@redhat.com>
> > > > Cc: Carlos Llamas <cmllamas@google.com>
> > > > Cc: Bart Van Assche <bvanassche@acm.org>
> > > > Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
> > > > Signed-off-by: Xuewen Yan <xuewen.yan@unisoc.com>
> > > > ---
> > > > changes of v3: correct code comments and add Cc tag.
> > > > changes of v2: update patch according to Boqun's suggestions.
> > > > ---
> > >
> > > It seems v3 should have collected the review tags from Boqun and Waiman.
> > > Also, I'm actually Cc'ing stable here. I hope that is enough.
> > > FWIW, this looks fine to me.
> > >
> > > Reviewed-by: Carlos Llamas <cmllamas@google.com>
> >
> >
> > <formletter>
> >
> > This is not the correct way to submit patches for inclusion in the
> > stable kernel tree.  Please read:
> >     https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
> > for how to do this properly.
> >
> > </formletter>
> 
> I see that many commits in mainline use Cc: <stable@vger.kernel.org>
> directly without other information,
> and I also find this information from above link: "Note, such tagging
> is unnecessary if the stable team can
> derive the appropriate versions from Fixes: tags."
> 
> In addition, this fixed commit "a0b0fd53e1e6 ("locking/lockdep: Free
> lock classes that are no longer in use")"
> was committed in 2019, so I am not very sure which start version
> should be added to stabe tag.
> Do you have any good suggestions?

Nope, you did this right, I missed it in the body of the changelog as
listed above, my apologies for the incorrect response here.

greg k-h