Re: [PATCH] RDMA/core: Fix stale RoCE GIDs during netdev events at registration

["Yanjun.Zhu" <yanjun.zhu@linux.dev>]

On 1/27/26 8:00 AM, Jason Gunthorpe wrote:
> On Tue, Jan 27, 2026 at 08:14:58PM +0900, Tetsuo Handa wrote:
>> On 2026/01/27 18:38, Jiri Pirko wrote:
>>> From: Jiri Pirko <jiri@nvidia.com>
>>>
>>> RoCE GID entries become stale when netdev properties change during the
>>> IB device registration window. This is reproducible with a udev rule
>>> that sets a MAC address when a VF netdev appears:
>>>
>>>    ACTION=="add", SUBSYSTEM=="net", KERNEL=="eth4", \
>>>      RUN+="/sbin/ip link set eth4 address 88:22:33:44:55:66"
>>>
>>> After VF creation, show_gids displays GIDs derived from the original
>>> random MAC rather than the configured one.
>>>
>>> The root cause is a race between netdev event processing and device
>>> registration:
>>>
>>>    CPU 0 (driver)                    CPU 1 (udev/workqueue)
>>>    ──────────────                    ──────────────────────
>>>    ib_register_device()
>>>      ib_cache_setup_one()
>>>        gid_table_setup_one()
>>>          _gid_table_setup_one()
>>>            ← GID table allocated
>>>          rdma_roce_rescan_device()
>>>            ← GIDs populated with
>>>              OLD MAC
>>>                                      ip link set eth4 addr NEW_MAC
>>>                                      NETDEV_CHANGEADDR queued
>>>                                      netdevice_event_work_handler()
>>>                                        ib_enum_all_roce_netdevs()
>>>                                          ← Iterates DEVICE_REGISTERED
>>>                                          ← Device NOT marked yet, SKIP!
>>>      enable_device_and_get()
>>>        xa_set_mark(DEVICE_REGISTERED)
>>>            ← Too late, event was lost
>>>
>>> The netdev event handler uses ib_enum_all_roce_netdevs() which only
>>> iterates devices marked DEVICE_REGISTERED. However, this mark is set
>>> late in the registration process, after the GID cache is already
>>> populated. Events arriving in this window are silently dropped.
>>>
>>> Fix this by introducing a new xarray mark DEVICE_GID_UPDATES that is
>>> set immediately after the GID table is allocated and initialized. Use
>>> the new mark in ib_enum_all_roce_netdevs() function to iterate devices
>>> instead of DEVICE_REGISTERED.
>>>
>>> This is safe because:
>>> - After _gid_table_setup_one(), all required structures exist (port_data,
>>>    immutable, cache.gid)
>>> - The GID table mutex serializes concurrent access between the initial
>>>    rescan and event handlers
>>> - Event handlers correctly update stale GIDs even when racing with rescan
>>> - The mark is cleared in ib_cache_cleanup_one() before teardown
>>>
>>> This also fixes similar races for IP address events (inetaddr_event,
>>> inet6addr_event) which use the same enumeration path.
>>>
>>> Fixes: 0df91bb67334 ("RDMA/devices: Use xarray to store the client_data")
>>> Signed-off-by: Jiri Pirko <jiri@nvidia.com>
>> I was thinking making the NETDEV_UNREGISTER event handler valid until
>> wait_for_completion(&device->unreg_completion) in disable_device() returns
>> ( https://lkml.kernel.org/r/b4a09ad8-97cc-4fe1-b02a-6192248694a8@I-love.SAKURA.ne.jp ).
>>
>> Since your patch includes what I was trying to address, you can add
>>
>> Reported-by: syzbot+881d65229ca4f9ae8c84@syzkaller.appspotmail.com
>> Closes: https://syzkaller.appspot.com/bug?extid=881d65229ca4f9ae8c84
>>
>> lines.
> Can we feed it to syzkaller please and see if it does actually clear
> it's repo? That particular bug already has 5 patches claiming to fix
> it.


#syz test: repository_link branch

The above command will make syzkaller test your commit.

BTW, your commit should be the topmost commit.


Thanks,

Zhu Yanjun

>
> It has become some kind of catch all of all kinds of refcounting errors
>
> [  247.188486][ T6052] unregister_netdevice: waiting for vcan0 to become free. Usage count = 2
>
> Does this actually change the refcounting around that could fix that?
> Looked like no?
>
> Jason