Re: pci passthrough - VF reset at boot is dropping assigned MAC

[David Ahern <dsahern@gmail.com>]

On 04/25/11 12:36, Alex Williamson wrote:
> On Mon, 2011-04-25 at 12:04 -0600, David Ahern wrote:
>>
>> On 04/25/11 11:30, Alex Williamson wrote:
>>> # modprobe -r igbvf
>>> # ip link set eth2 vf 6 mac 02:00:10:91:73:01
>>> # modprobe igbvf
>>> # dmesg | grep "igbvf 0000\:01\:11.5\: Address\:"
>>> igbvf 0000:01:11.5: Address: d2:c8:17:d6:97:f7
>>> igbvf 0000:01:11.5: Address: 4e:ee:2a:d8:12:7c
>>> igbvf 0000:01:11.5: Address: 02:00:10:91:73:01
>>> # modprobe -r igbvf
>>> # echo 1 > /sys/bus/pci/devices/0000:01:11.5/reset
>>> # modprobe igbvf
>>> # dmesg | grep "igbvf 0000\:01\:11.5\: Address\:"
>>> igbvf 0000:01:11.5: Address: d2:c8:17:d6:97:f7
>>> igbvf 0000:01:11.5: Address: 4e:ee:2a:d8:12:7c
>>> igbvf 0000:01:11.5: Address: 02:00:10:91:73:01
>>> igbvf 0000:01:11.5: Address: 02:00:10:91:73:01
>>>
>>> So now it sticks.  You're going to get random mac addresses on the VFs
>>> every time you reload the igb driver (ie. ever boot) anyway (at least
>>> with these sr-iov cards), so if you need consistent macs, they probably
>>> need to be set before launching the VM anyway.  Thanks,
>>>
>>> Alex
>>>
>>
>> Ok, I was able to repeat the above commands from the host command line.
>>
>> However, when qemu-kvm starts the MAC is reset.
>>
>> # ip link show | less
>>
>> 2: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> qlen 1000
>>     link/ether 00:1b:21:98:b7:10 brd ff:ff:ff:ff:ff:ff
>>     vf 0 MAC 02:12:34:56:80:20
>>
>> --> that's the MAC address I set
>>
>> I start qemu-kvm (unpatched version) and the host side sees the address
>> changed:
>>
>> # ip link show | less
>>
>> 2: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
>> qlen 1000
>>     link/ether 00:1b:21:98:b7:10 brd ff:ff:ff:ff:ff:ff
>>     vf 0 MAC 7a:17:3f:98:0f:db
>>
>>
>> Can you try that aspect on your end - seeing if the MAC address
>> maintains after starting qemu-kvm?
> 
> I don't see this happening on my system, once manually set the mac never
> changes.  I can restart and reset the VM and the host and guest both
> continue seeing the set mac address.  I tested it with both a recent
> rhel6.1 host kernel as well as upstream 2.6.39-rc4.  If I switch to a VF
> with an unset mac, those will change on each VM reset or restart.

Blacklist igbvf in the host and you will. That must be the difference: I
was preventing the vf driver from loading in the host -- it's not needed
there, so why load it?

I rebooted for a fresh run. Loaded the igbvf driver before starting the
VM using my tools. With the igbvf driver loaded in the host the MAC
address for the VF was not reset.

As for why I blacklisted it -- udev. What a PITA with VFs. I saw the
feature for Fedora 15 which should address this.

David