From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62DBEC433DB for ; Fri, 15 Jan 2021 00:35:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 20D0A23AA7 for ; Fri, 15 Jan 2021 00:35:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731413AbhAOAfI (ORCPT ); Thu, 14 Jan 2021 19:35:08 -0500 Received: from mail.kernel.org ([198.145.29.99]:48304 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731257AbhAOAfG (ORCPT ); Thu, 14 Jan 2021 19:35:06 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id D9D6223A9D; Fri, 15 Jan 2021 00:34:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1610670866; bh=9yGttj7wMOq1F5uCIowW+pgMdB2g42U1fvz5OyAiVq0=; h=Subject:From:To:Cc:Date:In-Reply-To:References:From; b=UpHK4McfxRAtXXiRezl767DHEE6wrvX8mhjyp2C3w8iXXnwplKbp+CfGg8m1yIWyo OqWEMGrvqNAmS37lnW2TGkM7Zm9zkKJlpqbvpUKJgT5jlVz1F6DImHIWR8ifQ69CMZ Yy4XBxF/BMHm1vFxyF21bTVpKvkd+Q8m2ij1ZCy/EXL5c3OBr4C3+ezwFwz3v8yxp/ waJTuX1uOpdzb+o5OmhHv5IAHFnkcO81VUGBBiV+jEROPnYLnDToYM08H6EJQ6j1tP j90ckcmXfXAnBkbHPmsD6jM1PPR4MAbACSTb5sDO3/AwbeoL5LsT5sydWSsY+0DfLn 4W3MQoKY5uXIw== Message-ID: <174c673c87fea15e832b795140addb70827818ff.camel@kernel.org> Subject: Re: [PATCH net] team: fix deadlock during setting of MTU From: Saeed Mahameed To: Ivan Vecera , netdev@vger.kernel.org Cc: Jiri Pirko Date: Thu, 14 Jan 2021 16:34:24 -0800 In-Reply-To: <20210114115506.1713330-1-ivecera@redhat.com> References: <20210114115506.1713330-1-ivecera@redhat.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Thu, 2021-01-14 at 12:55 +0100, Ivan Vecera wrote: > Team driver protects port list traversal in team_change_mtu() > by its team->lock mutex. This causes a deadlock with certain > devices that calls netdev_update_features() during their > .ndo_change_mtu() callback. In this case netdev_update_features() > calls team's netdevice notifier team_device_event() that in case > of NETDEV_FEAT_CHANGE tries lock team->lock mutex again. > > Example (r8169 case): > ... > [ 6391.348202] __mutex_lock.isra.6+0x2d0/0x4a0 > [ 6391.358602] team_device_event+0x9d/0x160 [team] > [ 6391.363756] notifier_call_chain+0x47/0x70 > [ 6391.368329] netdev_update_features+0x56/0x60 > [ 6391.373207] rtl8169_change_mtu+0x14/0x50 [r8169] > [ 6391.378457] dev_set_mtu_ext+0xe1/0x1d0 > [ 6391.387022] dev_set_mtu+0x52/0x90 > [ 6391.390820] team_change_mtu+0x64/0xf0 [team] > [ 6391.395683] dev_set_mtu_ext+0xe1/0x1d0 > [ 6391.399963] do_setlink+0x231/0xf50 > ... > > To fix the problem the port list traversal in team_change_mtu() can > be protected by RCU read lock. In case of failure the failing port > is marked and unwind code-path is done also under RCU read lock > protection (but not in reverse order). > > Fixes: 3d249d4ca7d0 ("net: introduce ethernet teaming device") > Cc: Jiri Pirko > Signed-off-by: Ivan Vecera > --- > drivers/net/team/team.c | 20 ++++++++++---------- > 1 file changed, 10 insertions(+), 10 deletions(-) > > diff --git a/drivers/net/team/team.c b/drivers/net/team/team.c > index c19dac21c468..69d4b28beb17 100644 > --- a/drivers/net/team/team.c > +++ b/drivers/net/team/team.c > @@ -1802,35 +1802,35 @@ static int team_set_mac_address(struct > net_device *dev, void *p) > static int team_change_mtu(struct net_device *dev, int new_mtu) > { > struct team *team = netdev_priv(dev); > - struct team_port *port; > + struct team_port *port, *fail_port; > int err; > > - /* > - * Alhough this is reader, it's guarded by team lock. It's not > possible > - * to traverse list in reverse under rcu_read_lock > - */ > - mutex_lock(&team->lock); > + rcu_read_lock(); > team->port_mtu_change_allowed = true; > - list_for_each_entry(port, &team->port_list, list) { > + list_for_each_entry_rcu(port, &team->port_list, list) { > err = dev_set_mtu(port->dev, new_mtu); some netdevs will need to sleep in their set_mtu, and you can't sleep under rcu! according to your explanation in the commit message the team->lock mutex will be also taken under this rcu lock, so this is bad even if dev_set_mtu does not sleep.