netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Taehee Yoo <ap420073@gmail.com>
To: Paolo Abeni <pabeni@redhat.com>,
	Nikolay Aleksandrov <razor@blackwall.org>,
	davem@davemloft.net, kuba@kernel.org, edumazet@google.com,
	jiri@resnulli.us, j.vosburgh@gmail.com, andy@greyhouse.net,
	netdev@vger.kernel.org
Cc: jarod@redhat.com, wangyufen@huawei.com,
	syzbot+60748c96cf5c6df8e581@syzkaller.appspotmail.com
Subject: Re: [PATCH net] net: fix stack overflow when LRO is disabled for virtual interfaces
Date: Tue, 16 May 2023 20:29:39 +0900	[thread overview]
Message-ID: <41942532-462e-fa1d-d9a4-eeb26abc481f@gmail.com> (raw)
In-Reply-To: <813be3bd0823bac31dc1b018750fad29d794d9c2.camel@redhat.com>

On 5/16/23 17:34, Paolo Abeni wrote:

Hi Paolo,
Thank you so much for the review!

 > On Mon, 2023-05-15 at 18:12 +0900, Taehee Yoo wrote:
 >> On 5/15/23 15:24, Nikolay Aleksandrov wrote:
 >>   > On 15/05/2023 08:37, Taehee Yoo wrote:
 >>   >> When the virtual interface's feature is updated, it 
synchronizes the
 >>   >> updated feature for its own lower interface.
 >>   >> This propagation logic should be worked as the iteration, not
 >> recursively.
 >>   >> But it works recursively due to the netdev notification 
unexpectedly.
 >>   >> This problem occurs when it disables LRO only for the team and 
bonding
 >>   >> interface type.
 >>   >>
 >>   >>         team0
 >>   >>           |
 >>   >>    +------+------+-----+-----+
 >>   >>    |      |      |     |     |
 >>   >> team1  team2  team3  ...  team200
 >>   >>
 >>   >> If team0's LRO feature is updated, it generates the 
NETDEV_FEAT_CHANGE
 >>   >> event to its own lower interfaces(team1 ~ team200).
 >>   >> It is worked by netdev_sync_lower_features().
 >>   >> So, the NETDEV_FEAT_CHANGE notification logic of each lower 
interface
 >>   >> work iteratively.
 >>   >> But generated NETDEV_FEAT_CHANGE event is also sent to the upper
 >>   >> interface too.
 >>   >> upper interface(team0) generates the NETDEV_FEAT_CHANGE event for
 >> its own
 >>   >> lower interfaces again.
 >>   >> lower and upper interfaces receive this event and generate this
 >>   >> event again and again.
 >>   >> So, the stack overflow occurs.
 >>   >>
 >>   >> But it is not the infinite loop issue.
 >>   >> Because the netdev_sync_lower_features() updates features before
 >>   >> generating the NETDEV_FEAT_CHANGE event.
 >>   >> Already synchronized lower interfaces skip notification logic.
 >>   >> So, it is just the problem that iteration logic is changed to the
 >>   >> recursive unexpectedly due to the notification mechanism.
 >>   >>
 >>   >> Reproducer:
 >>   >>
 >>   >> ip link add team0 type team
 >>   >> ethtool -K team0 lro on
 >>   >> for i in {1..200}
 >>   >> do
 >>   >>          ip link add team$i master team0 type team
 >>   >>          ethtool -K team$i lro on
 >>   >> done
 >>   >>
 >>   >> ethtool -K team0 lro off
 >>   >>
 >>   >> In order to fix it, the priv_notifier_ctx net_device member is
 >> introduced.
 >>   >> This variable can be used by each interface in its own way in the
 >>   >> notification context. The bonding and team interface is going 
to use it
 >>   >> to avoid duplicated NETDEV_FEAT_CHANGE event handling.
 >>   >>
 >>   >> Reported-by: syzbot+60748c96cf5c6df8e581@syzkaller.appspotmail.com
 >>   >> Fixes: fd867d51f889 ("net/core: generic support for disabling 
netdev
 >> features down stack")
 >>   >> Signed-off-by: Taehee Yoo <ap420073@gmail.com>
 >>   >> ---
 >>   >>   drivers/net/bonding/bond_main.c | 6 +++++-
 >>   >>   drivers/net/team/team.c         | 6 +++++-
 >>   >>   include/linux/netdevice.h       | 1 +
 >>   >>   net/core/dev.c                  | 2 ++
 >>   >>   4 files changed, 13 insertions(+), 2 deletions(-)
 >>   >>
 >>   >
 >>   > Since you're syncing to lower devices, can't you check if the event
 >> source device
 >>   > is lower to the current one (i.e. reverse propagation has happened)
 >> in the affected
 >>   > drivers ? Adding a new struct netdevice member just for this seems
 >> unnecessary to me.
 >>   > Especially for a setup like a bond of bonds or a team of teams, 
these
 >> are corner case
 >>   > setups that shouldn't exist in general. :)
 >>   >
 >>
 >> I agree that this new variable is unnecessary right now.
 >> I tried to avoid introducing new variables, but unfortunately, I
 >> couldn't find a solution to detect duplicated notification events.
 >>
 >> The reason why I introduced the new member of the net_device is that I
 >> thought there might be similar problems in the future such as mtu.
 >> so, I hoped that it can be used as a general variable to avoid similar
 >> problems.
 >> But I really agree that this new variable is over-spec.
 >> So, adding a new boolean variable into the struct bonding and team, not
 >> net_device would be reasonable if I can't find a proper solution.
 >
 > I think adding a bool variable to bonding/team priv would be better, as
 > it looks like the issues is specific to such kind of devices.
 >

Thanks, I will add a bool variable to the bonding and team struct in the v2.

Thank you so much!
Taehee Yoo

 > Thanks!
 >
 > Paolo
 >

  reply	other threads:[~2023-05-16 11:29 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-05-15  5:37 [PATCH net] net: fix stack overflow when LRO is disabled for virtual interfaces Taehee Yoo
2023-05-15  6:24 ` Nikolay Aleksandrov
2023-05-15  9:12   ` Taehee Yoo
2023-05-16  8:34     ` Paolo Abeni
2023-05-16 11:29       ` Taehee Yoo [this message]
2023-05-15 13:11 ` Simon Horman
2023-05-15 16:21   ` Taehee Yoo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=41942532-462e-fa1d-d9a4-eeb26abc481f@gmail.com \
    --to=ap420073@gmail.com \
    --cc=andy@greyhouse.net \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=j.vosburgh@gmail.com \
    --cc=jarod@redhat.com \
    --cc=jiri@resnulli.us \
    --cc=kuba@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=razor@blackwall.org \
    --cc=syzbot+60748c96cf5c6df8e581@syzkaller.appspotmail.com \
    --cc=wangyufen@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).