From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34521) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XIXrZ-0008BU-4S for qemu-devel@nongnu.org; Sat, 16 Aug 2014 02:53:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XIXrS-00064Z-O3 for qemu-devel@nongnu.org; Sat, 16 Aug 2014 02:53:17 -0400 Received: from [58.251.49.30] (port=49413 helo=mail.sangfor.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XIXrR-00063s-Ku for qemu-devel@nongnu.org; Sat, 16 Aug 2014 02:53:10 -0400 Date: Sat, 16 Aug 2014 14:48:41 +0800 From: "=?utf-8?B?WmhhbmcgSGFveXU=?=" References: <201408141622244965425@sangfor.com> Message-ID: <201408161448390411470@sangfor.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] =?utf-8?q?=5Bquestions=5D_about_using_vfio_to_assign?= =?utf-8?q?_sr-iov_vf_to_vm?= List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: =?utf-8?B?QWxleCBXaWxsaWFtc29u?= Cc: =?utf-8?B?cWVtdS1kZXZlbA==?= , =?utf-8?B?a3Zt?= >> Hi, all >> I'm using VFIO to assign intel 82599 VF to VM, now I encounter a problem, >> 82599 PF and its VFs belong to the same iommu_group, but I only want to assign some VFs to one VM, and some other VFs to another VM, ..., >> so how to only unbind (part of) the VFs but PF? >> I read the kernel doc vfio.txt, I'm not sure should I unbind all of the devices which belong to one iommu_group? >> If so, because PF and its VFs belong to the same iommu_group, if I unbind the PF, its VFs also diappeared. >> I think I misunderstand someting, >> any advises? > >This occurs when the PF is installed behind components in the system >that do not support PCIe Access Control Services (ACS). The IOMMU group >contains both the PF and the VF because upstream transactions can be >re-routed downstream by these non-ACS components before being translated >by the IOMMU. Please provide 'sudo lspci -vvv', 'lspci -n', and kernel >version and we might be able to give you some advise on how to work >around the problem. Thanks, > # lspci | grep Ether 02:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) 02:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) 08:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 08:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 09:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 09:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0a:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0a:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0b:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0b:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 0c:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8110SC/8169SC Gigabit Ethernet (rev 10) I want to direct-assign the VFs of intel 82599(02:00.0 or 02:00.1) to VM, # lspci -t -[0000:00]-+-00.0 +-01.0-[01]-- +-01.1-[02-03]--+-00.0 | \-00.1 +-02.0 +-06.0-[04]-- +-16.0 +-1a.0 +-1c.0-[05-0b]----00.0-[06-0b]--+-04.0-[07]-- | +-05.0-[08]--+-00.0 | | \-00.1 | +-06.0-[09]--+-00.0 | | \-00.1 | +-08.0-[0a]--+-00.0 | | \-00.1 | \-09.0-[0b]--+-00.0 | \-00.1 +-1d.0 +-1e.0-[0c]----00.0 +-1f.0 +-1f.2 \-1f.3 lspci -vvv -s 02.00.0 02:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- TAbort- SERR- TAbort- Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [88] Subsystem: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port Capabilities: [80] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee002f8 Data: 0000 Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us ExtTag- RBE+ FLReset- DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #3, Speed 8GT/s, Width x8, ASPM L0s L1, Latency L0 <256ns, L1 <8us ClockPM- Surprise- LLActRep- BwNot+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt+ SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #2, PowerLimit 75.000W; Interlock- NoCompl+ SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Not Supported, TimeoutDis- ARIFwd- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd- LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -3.5dB Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v1] Virtual Channel Caps: LPEVC=0 RefClk=100ns PATEntryBits=1 Arb: Fixed- WRR32- WRR64- WRR128- Ctrl: ArbSelect=Fixed Status: InProgress- VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans- Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff Status: NegoPending- InProgress- Capabilities: [140 v1] Root Complex Link Desc: PortNumber=03 ComponentID=01 EltType=Config Link0: Desc: TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+ Addr: 00000000fed19000 Capabilities: [d94 v1] #19 Kernel driver in use: pcieport The intel 82599(02:00.0 or 02:00.1) is behind the pci bridge (00:01.1), does 00:01.1 PCI bridge support ACS ? Thanks, Zhang Haoyu >Alex