Re: VDPA MAC address problem

[Konstantin Shkolnyy <kshk@linux.ibm.com>]

On 3/19/2025 19:58, Jason Wang wrote:
> Adding Cindy and Eugenio
> 
> On Thu, Mar 20, 2025 at 12:34 AM Konstantin Shkolnyy <kshk@linux.ibm.com> wrote:
>>
>> I’m observing a problem while testing VDPA with Nvidia ConnectX-6 (mlx5)
>> on s390.
>>
>> Upon start, virtio_net_device_realize() tries to set a new MAC address
>> by VHOST_VDPA_SET_CONFIG which doesn’t do anything.
>>
>> Later, the VM gets started and learns about the old address from
>> virtio_net_get_config() which returns whatever VHOST_VDPA_GET_CONFIG
>> returns, unless it's "6 zero bytes", in which case it instead returns
>> the desired new address (and the problem is avoided).
>>
>> Then QEMU again tries to set the new address from vhost_net_start(), now
>> by calling vhost_vdpa_net_load_cmd(...,VIRTIO_NET_CTRL_MAC,
>> VIRTIO_NET_CTRL_MAC_ADDR_SET, ...). This time the new address is
>> successfully programmed into the NIC, but the VM doesn't know about it.
> 
> Have you enabled shadow virtqueue? If yes, does it work if you don't do that?

I only run it in the default configuration, which has SVQ enabled.

>> As it is, the only VDPA scenario that's working for me is:
>> 1) Avoid specifying the MAC address in the "vdpa dev add" command (which
>> will create the "6 zero bytes" condition on the first launch).
>> 2) Keep using the same MAC address for every subsequent VM launch on the
>> same NIC "virtual function" (so that the old and new addresses are the
>> same).
> 
> This is the way we currently use it. Is there any limitation of this?

To be honest, I don't know how severe this limitation is, because I'm 
not a sysadmin. The problem is that I was searching www for info on how 
to setup VDPA and ended up hitting this bug and investigating it. 
Because the found info sources are variable and some show the address 
specified in "vdpa dev add".