Netdev List
 help / color / mirror / Atom feed
From: Tony Nguyen <anthony.l.nguyen@intel.com>
To: Jacob Keller <jacob.e.keller@intel.com>,
	Jakub Kicinski <kuba@kernel.org>, <davem@davemloft.net>
Cc: <netdev@vger.kernel.org>, <edumazet@google.com>,
	<pabeni@redhat.com>, <andrew+netdev@lunn.ch>, <horms@kernel.org>,
	<michael.chan@broadcom.com>, <hkallweit1@gmail.com>,
	<maxime.chevallier@bootlin.com>, <joshwash@google.com>,
	<tariqt@nvidia.com>, <alexanderduyck@fb.com>,
	<willemb@google.com>, <kory.maincent@bootlin.com>,
	<sdf.kernel@gmail.com>, <jakub@cloudflare.com>, <nb@tipi-net.de>
Subject: Re: [PATCH net-next v2 00/12] net: ethtool: let ops locked drivers run without rtnl_lock
Date: Mon, 8 Jun 2026 16:33:31 -0700	[thread overview]
Message-ID: <bdabf4af-187c-4df3-a64a-6f41dea9bf27@intel.com> (raw)
In-Reply-To: <abceffa3-3544-4da3-83ea-e23b64b3c0a7@intel.com>



On 6/8/2026 4:01 PM, Jacob Keller wrote:
> On 6/8/2026 3:31 PM, Jacob Keller wrote:
>> On 6/4/2026 5:29 PM, Jakub Kicinski wrote:
>>> With the ethtool_get_link_ksettings() situation hopefully ironed out
>>> the previous series (commit 6a5d837f0ce2) let's return to the main
>>> part of the series.
>>>
>>> We have been slowly moving towards removing the rtnl_lock dependency
>>> in driver ops since the concept of "ops-locked" drivers have been
>>> introduced last year. Since last year will take the netdev instance
>>> lock before invoking any ndo or ethtool op of "ops-locked" drivers.
>>>
>>> We dipped our toes into rtnl_lock-less ops with the queue binding API.
>>> Queue stats, NAPI, and other netdev-netlink objects are also queried
>>> without holding rtnl_lock already. It's time to take the next logical
>>> step and lift the requirement from ethtool ops.
>>>
>>> The direct motivation for this patchset is that ethtool ops often
>>> involve communicating with device FW, and may take a long time
>>> to complete. Aggressive polling of device state on machines
>>> with 10+ NICs have been shown to significantly increase rtnl_lock
>>> pressure.
>>>
>>> There's a handful of areas which still need rtnl_lock (see below).
>>> I decided to convert everything to rtnl_lock-less by default, and
>>> add a set of flags which let the drivers request rtnl_lock to still
>>> be taken. I don't love this, but I'm worried that opt-in would be
>>> even more confusing.
>>>
>>
>> Agreed. It might be nicer to opt-out first, and then opt-in the drivers
>> that don't do the update_features.. That would make it easier to prevent
>> buggy drivers slipping in as easily... But that would result in the
>> following messy situation:
>>
>> ethtool ops for ops-locked drivers don't hold RTNL lock, *except* some
>> which do because they might call update_features.. *except* those which
>> opt-out of RTNL because they know their driver implementation doesn't
>> call update_features...
>>
>> Yea. I think that is too messy. This approach requires slightly more
>> vigilance on the part of reviewers to ensure that ops-locked drivers
>> don't make a mistake here. However, the checks to see if features
>> changed while the RTNL wasn't held should be enough to help catch most
>> cases. Ok.
>>
>>> Known issues / exclusions:
>>>   - qdiscs - qdisc configuration currently assumes rtnl_lock, this
>>>     is mostly impacting set_channels callback. qdisc config is probably
>>>     the easiest one of the exclusions to tackle, it's fairly self-contained.
>>>   - features - even tho feature changes are (correctly) plumbed to
>>>     the driver thru ndos they are part of ethtool uAPI. ethtool itself
>>>     calls netdev_features_change() if it has spotted device feature change
>>>     before vs after to the callback. Some drivers also call
>>>     netdev_features_change() directly in response to various changes,
>>>     e.g. setting priv flags.
>>>     Since features have to propagate to upper and lower devices anything
>>>     that touches features is quite hard to move from under rtnl_lock.
>>>   - phylink - phylink and SFP depend on rtnl_lock today, I suspect
>>>     that this is purely for historic reasons. I started poking at
>>>     it and don't really see a need for a global lock. But accessing
>>>     the netdev instance lock from the SFP entry points will require
>>>     some attention from the phylink folks.
>>>   - phydev - similar to phylink, looks quite doable. But no ops-locked
>>>     driver currently has a phydev (fbnic only uses phylink) so phydev
>>>     related paths retain a ASSERT_RTNL() for now.
>>>
>>
>> Makes sense. Taking some steps towards the end goal is better than
>> trying to wait until every piece here is done.
>>
>> Whole series looks good to me. I had one minor nit about one patch
>> adding a bit flag at BIT(1) and then later changing that to BIT(5) but
>> its ultimately harmless and should not force a respin of this important
>> work.
>>
>> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
>>
>>> Tested on mlx5, bnxt and fbnic.
>>>
>>
>> I think currently only iAVF is ops-locked for Intel. Tony, would it make
>> sense to see if our virtualization validation folks have cycles to give
>> this a quick pass on iAVF? (Regardless of whether it merges before or
>> after, it would be beneficial to make sure we don't have any lingering
>> issues there).
> 
> Just to clarify, I'm not suggesting a delay on merging and am 100% fine
> with merging this before such testing completes. I just want to make
> sure we (Intel) do double check the iavf code so that development team
> can try to fix any lurking bugs within the kernel cycle before they
> become wide spread.

+1. I've asked our validation to test iavf on this series.

Thanks,
Tony

      reply	other threads:[~2026-06-08 23:33 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-05  0:29 [PATCH net-next v2 00/12] net: ethtool: let ops locked drivers run without rtnl_lock Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 01/12] net: ethtool: serialize broadcast notification sequence allocation Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 02/12] net: ethtool: relax ethnl_req_get_phydev() locking assertion Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 03/12] net: ethtool: make dev->hwprov ops-protected Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 04/12] net: ethtool: optionally skip rtnl_lock on Netlink path for GET ops Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 05/12] net: ethtool: optionally skip rtnl_lock on Netlink path for SET ops Jakub Kicinski
2026-06-08 22:15   ` Jacob Keller
2026-06-08 23:58     ` Jakub Kicinski
2026-06-09  1:01       ` Jacob Keller
2026-06-05  0:29 ` [PATCH net-next v2 06/12] net: ethtool: optionally skip rtnl_lock in cable test handlers Jakub Kicinski
2026-06-08 22:17   ` Jacob Keller
2026-06-05  0:29 ` [PATCH net-next v2 07/12] net: ethtool: optionally skip rtnl_lock in ethnl_tsinfo_dumpit() Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 08/12] net: ethtool: optionally skip rtnl_lock in ethnl_act_module_fw_flash() Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 09/12] net: ethtool: optionally skip rtnl_lock in RSS context handlers Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 10/12] net: ethtool: ioctl: concentrate the locking Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 11/12] net: ethtool: optionally skip rtnl_lock on IOCTL path Jakub Kicinski
2026-06-05  0:29 ` [PATCH net-next v2 12/12] docs: net: ethtool: document ops-locked drivers and op_needs_rtnl Jakub Kicinski
2026-06-05 22:48 ` [PATCH net-next v2 00/12] net: ethtool: let ops locked drivers run without rtnl_lock Stanislav Fomichev
2026-06-06  0:09   ` Jakub Kicinski
2026-06-08 15:20     ` Stanislav Fomichev
2026-06-08 22:04     ` Jacob Keller
2026-06-08 22:31 ` Jacob Keller
2026-06-08 23:01   ` Jacob Keller
2026-06-08 23:33     ` Tony Nguyen [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=bdabf4af-187c-4df3-a64a-6f41dea9bf27@intel.com \
    --to=anthony.l.nguyen@intel.com \
    --cc=alexanderduyck@fb.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=hkallweit1@gmail.com \
    --cc=horms@kernel.org \
    --cc=jacob.e.keller@intel.com \
    --cc=jakub@cloudflare.com \
    --cc=joshwash@google.com \
    --cc=kory.maincent@bootlin.com \
    --cc=kuba@kernel.org \
    --cc=maxime.chevallier@bootlin.com \
    --cc=michael.chan@broadcom.com \
    --cc=nb@tipi-net.de \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=sdf.kernel@gmail.com \
    --cc=tariqt@nvidia.com \
    --cc=willemb@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox