PCI passthrough and 3.3.1/3.4.0

PCI passthrough and 3.3.1/3.4.0

[Andy Burns]

I've not been keeping up with developments on PCI passthrough, it last
worked for me in 3.2.0, unfortunately 3.3.0 broke my setup ("pci:
SSSS:BB:DD.F must be co-assigned to the same guest with SSSS:BB:DD.F"
and then when I did try co-assignment of the devices I got the
spurious error "unsupported format character ':' (0x3a) at index 6").

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1340
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1336

Now I'm trying to catch up with recent developments. I've tried 3.3.1
and it's still not working, but at least the spurious error is now
being reported properly as "pci: SSSS:BB:DD.F: non-page-aligned MMIO
BAR found".

I see Yuji has been doing work with reassigndev/guestdev to realign
these devices, is there anything required within xen for these to
work, or are they purely dom0 changes? I might try to apply the
patches to my CentOS 5.2 dom0.

Is there anything else related to PCI passthrough expected in 3.4.0,
is the timescale still roughly end of March? Is there a release
candidate tag for it on xenbits yet?

Re: PCI passthrough and 3.3.1/3.4.0

[Keir Fraser]

On 01/03/2009 10:22, "Andy Burns" <xen.lists@burns.me.uk> wrote:

> Is there anything else related to PCI passthrough expected in 3.4.0,
> is the timescale still roughly end of March? Is there a release
> candidate tag for it on xenbits yet?

Most passthrough work currently going on is in relation to HVM guests. The
mechanisms for PV and HVM guests are quite different, apart from a bit of
shared logic in xend and in pciback. Indeed it's probably those bits of
shared logic in xend that cause PV passthrough to periodically break, as
developers consider the HVM case but not possible PV regressions.

3.4.0 will probably get pushed back a bit to try and get some more features
in. I'm currently thinking about freezing the tree at the end of March and
releasing late April or early May.

 -- Keir

Re: PCI passthrough and 3.3.1/3.4.0

[Boris Derzhavets]

[-- Attachment #1.1: Type: text/plain, Size: 1333 bytes --]

What version and nature of linux kernel you expect to support 3.4 ?

--- On Sun, 3/1/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
From: Keir Fraser <keir.fraser@eu.citrix.com>
Subject: Re: [Xen-devel] PCI passthrough and 3.3.1/3.4.0
To: "Andy Burns" <xen.lists@burns.me.uk>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>
Date: Sunday, March 1, 2009, 5:32 AM

On 01/03/2009 10:22, "Andy Burns" <xen.lists@burns.me.uk> wrote:

> Is there anything else related to PCI passthrough expected in 3.4.0,
> is the timescale still roughly end of March? Is there a release
> candidate tag for it on xenbits yet?

Most passthrough work currently going on is in relation to HVM guests. The
mechanisms for PV and HVM guests are quite different, apart from a bit of
shared logic in xend and in pciback. Indeed it's probably those bits of
shared logic in xend that cause PV passthrough to periodically break, as
developers consider the HVM case but not possible PV regressions.

3.4.0 will probably get pushed back a bit to try and get some more features
in. I'm currently thinking about freezing the tree at the end of March and
releasing late April or early May.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



      

[-- Attachment #1.2: Type: text/html, Size: 1723 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: PCI passthrough and 3.3.1/3.4.0

[Keir Fraser]

[-- Attachment #1.1: Type: text/plain, Size: 1883 bytes --]

Well, it should work with any Linux kernel that claims to have Xen support,
of course. We will still be supporting the 2.6.18 tree, and also mainstream
kernel.org has decent domU support now, while dom0 support is still up and
coming (hopefully rather more complete in 2.6.30, but we will have to wait
and see). Linux and Xen releases are not really tied together.

 -- Keir

On 01/03/2009 10:56, "Boris Derzhavets" <bderzhavets@yahoo.com> wrote:

> What version and nature of linux kernel you expect to support 3.4 ?
> 
> --- On Sun, 3/1/09, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
>> From: Keir Fraser <keir.fraser@eu.citrix.com>
>> Subject: Re: [Xen-devel] PCI passthrough and 3.3.1/3.4.0
>> To: "Andy Burns" <xen.lists@burns.me.uk>, "xen-devel@lists.xensource.com"
>> <xen-devel@lists.xensource.com>
>> Date: Sunday, March 1, 2009, 5:32 AM
>> 
>> On 01/03/2009 10:22, "Andy Burns" <xen.lists@burns.me.uk> wrote:
>> 
>>> > Is there anything else related to PCI passthrough expected in 3.4.0,
>>> > is the timescale still roughly end of March? Is there a release
>>> > candidate tag for it on xenbits yet?
>> 
>> Most passthrough work currently going on is in
>>  relation to HVM guests. The
>> mechanisms for PV and HVM guests are quite different, apart from a bit of
>> shared logic in xend and in pciback. Indeed it's probably those bits of
>> shared logic in xend that cause PV passthrough to periodically break, as
>> developers consider the HVM case but not possible PV regressions.
>> 
>> 3.4.0 will probably get pushed back a bit to try and get some more features
>> in. I'm currently thinking about freezing the tree at the end of March and
>> releasing late April or early May.
>> 
>>  -- Keir
>> 
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
> 
>  


[-- Attachment #1.2: Type: text/html, Size: 3004 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: PCI passthrough and 3.3.1/3.4.0

[Andy Burns]

2009/3/1 Keir Fraser <keir.fraser@eu.citrix.com>:

> Most passthrough work currently going on is in relation to HVM guests.

Heh, I thought my Q9450 cpu and X38 chipset were going to be
sufficient for HVM I/O support, but ASUS have not enabled support for
it in the BIOS, I don't suppose there's a well-known bit in a register
on the X38 ICH9R that can be flipped to force VT-d on is there?

> I'm currently thinking about freezing the tree at the end of March and
> releasing late April or early May.

OK thanks, that's far enough away to tempt me to try Yuji's BAR
alignment patches on CentOS in the meantime.

Re: PCI passthrough and 3.3.1/3.4.0

[Christian Tramnitz]

Hi Boris,

As you can see in the xen-unstable tree 2.6.18 is still the official 
dom0 kernel.
Pvops will hopefully come with 2.6.30, but that will still take some 
time: from recent pvops testing I've observed that passthrough is not 
working at all (simply due to the lack of pciback, correct me if I'm 
wrong here)...


Best regards,
    Christian

Boris Derzhavets schrieb:
> What version and nature of linux kernel you expect to support 3.4 ?
> 

Re: PCI passthrough and 3.3.1/3.4.0

[Keir Fraser]

On 01/03/2009 11:16, "Andy Burns" <xen.lists@burns.me.uk> wrote:

>> Most passthrough work currently going on is in relation to HVM guests.
> 
> Heh, I thought my Q9450 cpu and X38 chipset were going to be
> sufficient for HVM I/O support, but ASUS have not enabled support for
> it in the BIOS, I don't suppose there's a well-known bit in a register
> on the X38 ICH9R that can be flipped to force VT-d on is there?

Unfortunately you need a VT-d aware BIOS because it is the BIOS which
describes the VT-d hardware to the hypervisor, via a set of in-memory info
tables. No BIOS support == no VT-d, despite the fact at the hardware level
the support may well be present and waiting to be used. Actually even some
BIOSes which do claim to support VT-d are pretty broken -- presumably it's
something vendors don't have comprehensive tests for as yet -- so you have
to be careful.

>> I'm currently thinking about freezing the tree at the end of March and
>> releasing late April or early May.
> 
> OK thanks, that's far enough away to tempt me to try Yuji's BAR
> alignment patches on CentOS in the meantime.

Yes, worth some experimentation I think. Also generally providing more info
on xen-devel about how PV passthru is failing for you may help get it fixed.
Even better if you can track down when it broke, who broke it, and try and
get help from them via direct plea. ;-)

 -- Keir

Re: Re: PCI passthrough and 3.3.1/3.4.0

[Mark Williamson]

On Sunday 01 March 2009 11:17:58 Christian Tramnitz wrote:
> Hi Boris,
>
> As you can see in the xen-unstable tree 2.6.18 is still the official
> dom0 kernel.
> Pvops will hopefully come with 2.6.30, but that will still take some
> time: from recent pvops testing I've observed that passthrough is not
> working at all (simply due to the lack of pciback, correct me if I'm
> wrong here)...

Even then, dom0 support in mainline would only be a minimal functionality.  
Complete dom0 support would wait for a later release, if it ever gets merged.

As far as I know most / all domU support is already merged upstream so at 
least unprivileged guests can benefit from recently mainline developments.

Cheers,
Mark

>
> Best regards,
>     Christian
>
> Boris Derzhavets schrieb:
> > What version and nature of linux kernel you expect to support 3.4 ?
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: PCI passthrough and 3.3.1/3.4.0

[Christian Tramnitz]

Mark Williamson schrieb:
> As far as I know most / all domU support is already merged upstream so at 
> least unprivileged guests can benefit from recently mainline developments.

Unfortunately this is not the case: as far as I know there is no 
pci-frontend yet, so pass-through is still not possible with vanilla 
linux as domU...


Best regards,
    Christian

Re: Re: PCI passthrough and 3.3.1/3.4.0

[Mark Williamson]

> Unfortunately this is not the case: as far as I know there is no
> pci-frontend yet, so pass-through is still not possible with vanilla
> linux as domU...

I was considering that (perhaps non-obviously!) as being "dom0-type" 
functionality.  But I should have said so, so thank you for pointing it out.

Part of the code required (aside from pcifront itself) would actually be code 
to make DMA work properly - code which dom0 has to have whereas an 
unprivileged domU does not.  I assume that this code will go into the kernel 
if basic dom0 support is merged; that might make a pcifront in Linux upstream 
somewhat more likely.  Fingers crossed!

Cheers,
Mark

Re: PCI passthrough and 3.3.1/3.4.0

[Andy Burns]

2009/3/1 Keir Fraser <keir.fraser@eu.citrix.com>:

> On 01/03/2009 11:16, "Andy Burns" <xen.lists@burns.me.uk> wrote:
>
>> I'll try Yuji's BAR alignment patches on CentOS
>
> Yes, worth some experimentation I think.

OK, I've built a centosplus 5.2 kernel for dom0

I have patched it to

1) re-enable my sky2 NICs

2) correctly size the saa7134 mmio area

3) include the first of Yuji's "page-aligned" patches with minor
tweaks to quirks.c to allow clean application.
http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/2b5cc22ab406

I've added "reassign_resources reassigndev=0000:08:00.0" to my grub stanza,

I do see "PCI: resource reassign ON" in the console output, but no
"PCI: Disable device and release resources" message and lspci still
shows the device on a non-page aligned address

# lspci -v -s 08:00.0

08:00.0 Multimedia controller: Philips Semiconductors SAA7130 Video
Broadcast Decoder
        Subsystem: Compro Technology, Inc. Videomate DVB-T200
        Flags: medium devsel, IRQ 16
        Memory at febffc00 (32-bit, non-prefetchable) [disabled] [size=1K]
        Capabilities: [40] Power Management version 1

Any thoughts? Do I need to reassign the other devices behind the
00.1e.0 PCI bridge or the bridge itself?

# lspci -v -s 00:1e.0

00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92) (prog-if 01 [Su)
        Flags: bus master, fast devsel, latency 0
        Bus: primary=00, secondary=08, subordinate=08, sec-latency=32
        I/O behind bridge: 0000e000-0000efff
        Memory behind bridge: feb00000-febfffff
        Capabilities: [50] #0d [0000]

# lspci -t

-[0000:00]-+-00.0
           +-01.0-[0000:01]--+-00.0
           |                 \-00.1
           +-1a.0
           +-1a.1
           +-1a.2
           +-1a.7
           +-1b.0
           +-1c.0-[0000:05-07]--+-00.0-[0000:07]----01.0
           |                    \-00.1-[0000:06]--
           +-1c.2-[0000:04]----00.0
           +-1c.3-[0000:03]----00.0
           +-1c.4-[0000:02]----00.0
           +-1d.0
           +-1d.1
           +-1d.2
           +-1d.7
           +-1e.0-[0000:08]--+-00.0
           |                 +-01.0
           |                 \-03.0
           +-1f.0
           +-1f.2
           \-1f.3

Re: PCI passthrough and 3.3.1/3.4.0

[Andy Burns]

2009/3/8 Andy Burns <xen.lists@burns.me.uk>:

> OK, I've built a centosplus 5.2 kernel for dom0
> include the first of Yuji's "page-aligned" patches with minor
> tweaks to quirks.c to allow clean application.
> http://xenbits.xensource.com/linux-2.6.18-xen.hg?rev/2b5cc22ab406

I don't see any calls to the quirk_align_mem_resources() function
within the patch, was a DECLARE_PCI_FIXUP_HEADER missing in the
original version or have I missed how it is meant to get invoked?