Re: [RFC net-next 0/4] devlink: Add boot-time defaults

[Mark Bloch <mbloch@nvidia.com>]

On 13/05/2026 14:11, Jiri Pirko wrote:
> Wed, May 13, 2026 at 07:53:05AM CEST, mbloch@nvidia.com wrote:
>>
>>
>> On 12/05/2026 21:35, Jiri Pirko wrote:
>>> Tue, May 12, 2026 at 05:25:21PM CEST, parav@nvidia.com wrote:
>>>>
>>>>
>>>>> From: Jiri Pirko <jiri@resnulli.us>
>>>>> Sent: 12 May 2026 07:37 PM
>>>>>
>>>>> Tue, May 12, 2026 at 03:48:32PM CEST, parav@nvidia.com wrote:
>>>>>>
>>>>>>> From: Jiri Pirko <jiri@resnulli.us>
>>>>>>> Sent: 12 May 2026 02:16 PM
>>>>>>>
>>>>>>> Mon, May 11, 2026 at 08:21:37PM +0200, parav@nvidia.com wrote:
>>>>>>>>
>>>>>>>>> From: Mark Bloch <mbloch@nvidia.com>
>>>>>>>>> Sent: 10 May 2026 06:02 PM
>>>>>>>>>
>>>>>>>>
>>>>>>>> [..]
>>>>>>>>
>>>>>>>>>> I look at it from the perspective that from some CX generation,
>>>>>>>>>> switchdev mode should be default. So that is a device-based decision.
>>>>>>>>>> I believe as such it can optionally be permanenty configured (nv config)
>>>>>>>>>> on older device. Why not?
>>>>>>>>>
>>>>>>>> Because sometimes switchdev_inactive is needed and sometimes not.
>>>>>>>> Such knob is not device decision.
>>>>>>>
>>>>>>> That is what I would call corner case. In that, user can use userspace
>>>>>>> configuration to change the mode in runtime.
>>>>>>>
>>>>>> Corner vs common depends on users one talks to. :)
>>>>>> If fw has switchdev(active) as default, and then
>>>>>> And user needs to run switchdev_inactive, it will actually break their switching applications.
>>>>>
>>>>> Can you describe the actutal breakage please?
>>>>>
>>>> Driver default was switchdev so all the traffic is forwarded to the switch,
>>>> and user didn't have chance to setup the fdb rules.
>>>> So packets are dropped but user didn't expect the traffic to be forwarded.
>>>
>>> User may switch mode to switchdev_inactive early on, before any of the
>>> representors are created. What's the issue then?
>>
>> That is the ordering problem I am trying to solve.
>>
>> On a DPU, the host PF cannot finish loading until the ECPF moves the eswitch to
>> switchdev/switchdev_inactive. So we need to do that transition during ECPF
>> driver init, as early as possible. Waiting for userspace means the host PF stays
>> blocked until userspace is up and has the right logic.
>>
>> That is not always true in practice, the driver may be built in, loaded from an
>> initramfs, or the initramfs may simply not contain the devlink policy we need.
>>
>> Also, after talking with Parav, my understanding is that we need to support both
>> switchdev and switchdev_inactive, since different customers want different boot
>> behavior. Once we do the transition, the host PF can load and may start sending
>> packets. At that point the initial mode already matters: in switchdev_inactive
>> packets are dropped until userspace programs the pipeline; in switchdev they may
>> reach the FDB before the pipeline is ready.
>>
>> So I do not think an early userspace transition is equivalent here. The initial
>> mode needs to be known by the kernel before userspace runs, which is why I am
>> proposing the devlink= command line default.
> 
> Okay fair enough. Could you please at least make sure this is mode only
> config and noone would ever think about abusing this for any other
> configuration? Perhaps call it "devlink_eswitch_mode=" to remove
> the "devlink=" namespace flexibility?

Sure, something along these lines:
devlink_eswitch_mode=[*]:switchdev
devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:switchdev_inactive

The proper (not RFC) series will have 3 patches:

- devlink: add the command-line default eswitch mode handling
- mlx5: cleanup/prep patch
- mlx5: use the devlink API to apply the early eswitch mode

Since the mlx5 changes are part of the series, I suspect this will need to
go through Tariq. The patches are ready, but are currently in
our submission queue.

Mark