Re: [patch net-next v4 2/7] switchdev: allow caller to explicitly request attr_set as deferred

[Jiri Pirko <jiri@resnulli.us>]

Tue, Oct 13, 2015 at 08:03:46AM CEST, john.fastabend@gmail.com wrote:
>On 15-10-12 10:44 PM, Jiri Pirko wrote:
>> Tue, Oct 13, 2015 at 04:52:42AM CEST, sfeldma@gmail.com wrote:
>>> On Mon, Oct 12, 2015 at 11:03 AM, Jiri Pirko <jiri@resnulli.us> wrote:
>>>> From: Jiri Pirko <jiri@mellanox.com>
>>>>
>>>> Caller should know if he can call attr_set directly (when holding RTNL)
>>>> or if he has to defer the att_set processing for later.
>>>>
>>>> This also allows drivers to sleep inside attr_set and report operation
>>>> status back to switchdev core. Switchdev core then warns if status is
>>>> not ok, instead of silent errors happening in drivers.
>>>>
>>>> Signed-off-by: Jiri Pirko <jiri@mellanox.com>
>>>> ---
>>>>  include/net/switchdev.h   |   1 +
>>>>  net/bridge/br_stp.c       |   3 +-
>>>>  net/switchdev/switchdev.c | 107 ++++++++++++++++++++++++----------------------
>>>>  3 files changed, 59 insertions(+), 52 deletions(-)
>>>>
>>>> diff --git a/include/net/switchdev.h b/include/net/switchdev.h
>>>> index d2879f2..6b109e4 100644
>>>> --- a/include/net/switchdev.h
>>>> +++ b/include/net/switchdev.h
>>>> @@ -17,6 +17,7 @@
>>>>
>>>>  #define SWITCHDEV_F_NO_RECURSE         BIT(0)
>>>>  #define SWITCHDEV_F_SKIP_EOPNOTSUPP    BIT(1)
>>>> +#define SWITCHDEV_F_DEFER              BIT(2)
>>>>
>>>>  struct switchdev_trans_item {
>>>>         struct list_head list;
>>>> diff --git a/net/bridge/br_stp.c b/net/bridge/br_stp.c
>>>> index db6d243de..80c34d7 100644
>>>> --- a/net/bridge/br_stp.c
>>>> +++ b/net/bridge/br_stp.c
>>>> @@ -41,13 +41,14 @@ void br_set_state(struct net_bridge_port *p, unsigned int state)
>>>>  {
>>>>         struct switchdev_attr attr = {
>>>>                 .id = SWITCHDEV_ATTR_ID_PORT_STP_STATE,
>>>> +               .flags = SWITCHDEV_F_DEFER,
>>>>                 .u.stp_state = state,
>>>>         };
>>>>         int err;
>>>>
>>>>         p->state = state;
>>>>         err = switchdev_port_attr_set(p->dev, &attr);
>>>> -       if (err && err != -EOPNOTSUPP)
>>>> +       if (err)
>>>
>>> This looks like a problem as now all other non-switchdev ports will
>>> get an WARN in the log when STP state changes.  We should only WARN if
>>> there was an err and the err is not -EOPNOTSUPP.
>> 
>> If SWITCHDEV_F_DEFER flag is set, there's only 0 of -ENOMEM.
>> 
>> 
>>>
>>>>                 br_warn(p->br, "error setting offload STP state on port %u(%s)\n",
>>>>                                 (unsigned int) p->port_no, p->dev->name);
>>>>  }
>>>
>>> <snip>
>>>
>>>>  struct switchdev_attr_set_work {
>>>>         struct work_struct work;
>>>>         struct net_device *dev;
>>>> @@ -183,14 +226,17 @@ static void switchdev_port_attr_set_work(struct work_struct *work)
>>>>  {
>>>>         struct switchdev_attr_set_work *asw =
>>>>                 container_of(work, struct switchdev_attr_set_work, work);
>>>> +       bool rtnl_locked = rtnl_is_locked();
>>>>         int err;
>>>>
>>>> -       rtnl_lock();
>>>> -       err = switchdev_port_attr_set(asw->dev, &asw->attr);
>>>> +       if (!rtnl_locked)
>>>> +               rtnl_lock();
>>>
>>> I'm not following this change.  If someone else has rtnl_lock, we'll
>>> not wait to grab it here ourselves, and proceed as if we have the
>>> lock.  But what if that someone else releases the lock in the middle
>>> of us doing switchdev_port_attr_set_now?  Seems we want to
>>> unconditionally wait and grab the lock.  We need to block anything
>>>from moving while we do the attr set.
>> 
>> Why would someone we call (driver) return the lock? In that case, he is
>> buggy and should be fixed.
>> 
>> This hunk only ensures we have rtnl_lock. If not, we take it here. We do
>> not take it unconditionally because we may already have it, for example
>> if caller of switchdev_flush_deferred holds rtnl_lock.
>> 
>
>This is where you lost me. How do you know another core doesn't happen
>to have the lock when you hit this code path? Maybe someone is running
>an ethtool command on another core or something.

You are right. The same problem exists currently in switchdev_port_attr_set.