From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98BFE1F63C3 for ; Tue, 4 Mar 2025 19:41:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.45 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741117306; cv=none; b=KtRd4TpIpB6GalsGi+IzyVXhgkGt0UAejQJRBRPz2fK+iBsk9RTKkxWp8+goC44AJuSCG9933SvSf4wIcUEzszx3HN4kB4gPx8pbZHvbrHLem4JVumG7NWNdDe6H1EiKHi6dmPvIwPih9VdYFTJA12knMyvvPvoDvJsYIfqTedc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741117306; c=relaxed/simple; bh=Z1Dr2H1wZQma+yJxOpXVs8NcHO1DKsOkb0OPlMDP+XE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=H+DMF76GYWW+Jz0USb5u5o3UE2nVB4GBFluP5OqegcFGEtwuHFvr5qFypPXOEk5SYHtk/NgR3X/T28dAGlfRqkdR215lu9MWfqmNuiPAncw3EzL7Brbk7fuclBsjyc4Id5muLemf1Wl9t2YvckfTgbCSCb7/r5Kp74zxWDVq70I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=Dp0ostl+; arc=none smtp.client-ip=209.85.219.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="Dp0ostl+" Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-6e895fd83d3so46642046d6.3 for ; Tue, 04 Mar 2025 11:41:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1741117302; x=1741722102; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hniMK4SB1u1F2OOA4rSYd4xqdAr8kGaAFH0sJ2S5DiI=; b=Dp0ostl+XTTjDEM3m1SzD6qZL0bKvFcGaXEQHgrUtJ+Mr5RonLJYQ3jcyInnuALtSd UEQn/OvkSo7/odNgS8TUNQrB+7GyZTVNKZjqzVBHfy9OzsDd1h9s0XyMdiKLLsZT+uck 7VtQLP+Xh6czvHWQ4t6pKXbWPVr72SisFtfcZ/UuXrO5S1Y+4JJE6M3tcrGPWGftPD66 nWx6U7SZAhx/oZ40QYh47Dd8tHzuVDhY+RHrYlBAvVhnY96hm8+rFAtjUVBzm0zokP3j SImSxPHMkasmx0/sPaU0h96q9GXQza8442rT7xUw8HKjvYKQBwf8pZpq1DnC1gLbBPjh OifQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741117302; x=1741722102; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=hniMK4SB1u1F2OOA4rSYd4xqdAr8kGaAFH0sJ2S5DiI=; b=oEsBL5Le7IJO39OmGe9q6Piykiknzgd+Lo8/Tix6iWxg1dEBYatFDdKTyVZfVwR5jC MRNvea2gy+UdIkeoFhfgZtIe6Z5+E+egYmiGUi6hUCqTvNyoC3sCMIu2xiji9YNDx7MQ ffaszmScSP2VTjQarv2idch77vc606HwOBvJIJbVWfFlfK3sHHGmbOGDtcZJO5S+x7hq tjmUbXKHkOcNwxM2G7u6f9POKbvE+uRwBx/IMK0/IjiMHex+aYxVTG3WstuCIKCiQTvJ 1rtHf7mDdDtlmKJqfCwpCrnTwPUy/KjLfd0sqDCVJQyGd1C7UOUE7sBm/ZgBsZewIuLH HZeA== X-Forwarded-Encrypted: i=1; AJvYcCXsQRJC4gO8KwLRa0ytOww+eh8oUwoDyXX91DmV391A1mPUZg8qS+k5qfyN6KKZrqRolPNKFZw=@vger.kernel.org X-Gm-Message-State: AOJu0YwLsH8XN5C0EZzgE/d3XEaz04oQ8NByqoN7zBe2qP5FkSYECjKb fQOtzgB3yYpbgZ//qKLyqGv6v42Qt1F8DnLlVJzDHQ63oXY2cdVCh/5G0W2lCvA= X-Gm-Gg: ASbGncsYGY+fdoCPqJqPcrFWIH5AM2Xv/eVBjcDeucYyu7wrMXGL7a/loj3EJfK7Rxi 3zqUenM+lj5tXLwbU3dH4ltMD7WRoNIL3c2fxp0VNt79mN1njd+FKpt0elVx4WQC4ee0ASkdK3k OPpgq17+gI46vUiFOTgWR9/gLTkidn1/CdI3NtSCIVQyPxhtXyugJed9Ewsm3JCuiJP8tWtUcap VevC8jNJbXU1T8QRVT4UGGmdiaOvgcw1wrqQ4gwuM8sDOl0QUQA3u+ELtqdjMZEBEG8ij+N0AxZ sPQkyZKJFyVeWi08Rl7wJiOGmiVrZSjDBYI6A5CX/dhbPWFfCRtH3+pxAf/drnopryUY8h6PTdo /LF66tO6Hbgbyy7QNNg== X-Google-Smtp-Source: AGHT+IEVAKGzcENzw+iAKzItxzizgLoOPdiHsjgb9KK/h/j44uEdoziCKUzFRD74zSh7p4AQOUAf1A== X-Received: by 2002:a05:6214:e67:b0:6e8:89bd:2b50 with SMTP id 6a1803df08f44-6e8e6cc6b6emr9042306d6.7.1741117300888; Tue, 04 Mar 2025 11:41:40 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-68-128-5.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.128.5]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6e8976ec1f8sm70616526d6.121.2025.03.04.11.41.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Mar 2025 11:41:40 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1tpY95-00000001J4e-2w8n; Tue, 04 Mar 2025 15:41:39 -0400 Date: Tue, 4 Mar 2025 15:41:39 -0400 From: Jason Gunthorpe To: Long Li Cc: Ratheesh Kannoth , "longli@linuxonhyperv.com" , Leon Romanovsky , Konstantin Taranov , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , "linux-rdma@vger.kernel.org" , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-hyperv@vger.kernel.org" Subject: Re: [EXTERNAL] Re: [Patch rdma-next] RDMA/mana_ib: handle net event for pointing to the current netdev Message-ID: <20250304194139.GE5011@ziepe.ca> References: <1740782519-13485-1-git-send-email-longli@linuxonhyperv.com> <20250304063940.GA2702870@maili.marvell.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Tue, Mar 04, 2025 at 06:26:03PM +0000, Long Li wrote: > > On 2025-03-01 at 04:11:59, longli@linuxonhyperv.com > > (longli@linuxonhyperv.com) wrote: > > > From: Long Li > > > > > > When running under Hyper-V, the master device to the RDMA device is > > > always bonded to this RDMA device if it's present in the kernel. This > > > is not user-configurable. > > > > > > The master device can be unbind/bind from the kernel. During those > > > events, the RDMA device should set to the current netdev to relect the > > > change of master device from those events. > > > > > > Signed-off-by: Long Li > > > --- > > > drivers/infiniband/hw/mana/device.c | 35 > > > ++++++++++++++++++++++++++++ drivers/infiniband/hw/mana/mana_ib.h | > > > 1 + > > > 2 files changed, 36 insertions(+) > > > > > > diff --git a/drivers/infiniband/hw/mana/device.c > > > b/drivers/infiniband/hw/mana/device.c > > > index 3416a85f8738..3e4f069c2258 100644 > > > --- a/drivers/infiniband/hw/mana/device.c > > > +++ b/drivers/infiniband/hw/mana/device.c > > > @@ -51,6 +51,37 @@ static const struct ib_device_ops mana_ib_dev_ops = { > > > ib_ind_table), }; > > > > > > +static int mana_ib_netdev_event(struct notifier_block *this, > > > + unsigned long event, void *ptr) { > > > + struct mana_ib_dev *dev = container_of(this, struct mana_ib_dev, nb); > > > + struct net_device *event_dev = netdev_notifier_info_to_dev(ptr); > > > + struct gdma_context *gc = dev->gdma_dev->gdma_context; > > > + struct mana_context *mc = gc->mana.driver_data; > > > + struct net_device *ndev; > > > + > > > + if (event_dev != mc->ports[0]) > > > + return NOTIFY_DONE; > > > + > > > + switch (event) { > > > + case NETDEV_CHANGEUPPER: > > > + rcu_read_lock(); > > > + ndev = mana_get_primary_netdev_rcu(mc, 0); > > > + rcu_read_unlock(); > > ... > > > + > > > + /* > > > + * RDMA core will setup GID based on updated netdev. > > > + * It's not possible to race with the core as rtnl lock is being > > > + * held. > > > + */ > > > + ib_device_set_netdev(&dev->ib_dev, ndev, 1); > > rcu_read_unlock() should be here, right ? > > It can't. ib_device_set_netdev() is calling alloc_port_data() and may sleep. > > I think this locking is okay. This event only comes in when: > 1. the master device has changed to netvsc. In this case ndev is guaranteed to be valid as this notification is triggered by netvsc. > 2. the master device has changed to itself (the ethernet device parent for the IB device). In this case, ndev is valid because mana_ib is an auxiliary device to ndev and it can't unload itself at this time. Why not return with the netdev refcount held so you don't need this weirdo rcu thing? Jason