From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: pci passthrough - VF reset at boot is dropping assigned MAC Date: Mon, 25 Apr 2011 13:18:36 -0600 Message-ID: <1303759116.3431.59.camel@x201> References: <4DB5A13B.4050804@gmail.com> <1303749455.3431.21.camel@x201> <4DB5A44F.6040304@gmail.com> <1303752659.3431.31.camel@x201> <4DB5B7AE.8010900@gmail.com> <1303756594.3431.55.camel@x201> <4DB5C78A.1030505@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: KVM mailing list To: David Ahern Return-path: Received: from mx1.redhat.com ([209.132.183.28]:39546 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758774Ab1DYTSi (ORCPT ); Mon, 25 Apr 2011 15:18:38 -0400 In-Reply-To: <4DB5C78A.1030505@gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On Mon, 2011-04-25 at 13:12 -0600, David Ahern wrote: > > On 04/25/11 12:36, Alex Williamson wrote: > > On Mon, 2011-04-25 at 12:04 -0600, David Ahern wrote: > >> > >> On 04/25/11 11:30, Alex Williamson wrote: > >>> # modprobe -r igbvf > >>> # ip link set eth2 vf 6 mac 02:00:10:91:73:01 > >>> # modprobe igbvf > >>> # dmesg | grep "igbvf 0000\:01\:11.5\: Address\:" > >>> igbvf 0000:01:11.5: Address: d2:c8:17:d6:97:f7 > >>> igbvf 0000:01:11.5: Address: 4e:ee:2a:d8:12:7c > >>> igbvf 0000:01:11.5: Address: 02:00:10:91:73:01 > >>> # modprobe -r igbvf > >>> # echo 1 > /sys/bus/pci/devices/0000:01:11.5/reset > >>> # modprobe igbvf > >>> # dmesg | grep "igbvf 0000\:01\:11.5\: Address\:" > >>> igbvf 0000:01:11.5: Address: d2:c8:17:d6:97:f7 > >>> igbvf 0000:01:11.5: Address: 4e:ee:2a:d8:12:7c > >>> igbvf 0000:01:11.5: Address: 02:00:10:91:73:01 > >>> igbvf 0000:01:11.5: Address: 02:00:10:91:73:01 > >>> > >>> So now it sticks. You're going to get random mac addresses on the VFs > >>> every time you reload the igb driver (ie. ever boot) anyway (at least > >>> with these sr-iov cards), so if you need consistent macs, they probably > >>> need to be set before launching the VM anyway. Thanks, > >>> > >>> Alex > >>> > >> > >> Ok, I was able to repeat the above commands from the host command line. > >> > >> However, when qemu-kvm starts the MAC is reset. > >> > >> # ip link show | less > >> > >> 2: eth2: mtu 1500 qdisc mq state UP > >> qlen 1000 > >> link/ether 00:1b:21:98:b7:10 brd ff:ff:ff:ff:ff:ff > >> vf 0 MAC 02:12:34:56:80:20 > >> > >> --> that's the MAC address I set > >> > >> I start qemu-kvm (unpatched version) and the host side sees the address > >> changed: > >> > >> # ip link show | less > >> > >> 2: eth2: mtu 1500 qdisc mq state UP > >> qlen 1000 > >> link/ether 00:1b:21:98:b7:10 brd ff:ff:ff:ff:ff:ff > >> vf 0 MAC 7a:17:3f:98:0f:db > >> > >> > >> Can you try that aspect on your end - seeing if the MAC address > >> maintains after starting qemu-kvm? > > > > I don't see this happening on my system, once manually set the mac never > > changes. I can restart and reset the VM and the host and guest both > > continue seeing the set mac address. I tested it with both a recent > > rhel6.1 host kernel as well as upstream 2.6.39-rc4. If I switch to a VF > > with an unset mac, those will change on each VM reset or restart. > > Blacklist igbvf in the host and you will. That must be the difference: I > was preventing the vf driver from loading in the host -- it's not needed > there, so why load it? I already have it blacklisted. It's not needed if you're using the VFs they way we are, but there are other uses. > I rebooted for a fresh run. Loaded the igbvf driver before starting the > VM using my tools. With the igbvf driver loaded in the host the MAC > address for the VF was not reset. > > As for why I blacklisted it -- udev. What a PITA with VFs. I saw the > feature for Fedora 15 which should address this. Yes, my VM is up to renaming the VFs eth1340 since the mac changes every boot. I'm still confused though as I did a whole round of testing after a reboot where igbvf was never loaded and the set mac address stuck across VM restarts and resets. Alex