SATA pass-through woes

SATA pass-through woes

[Patrick Colp]

I'm trying to pass my SATA controller through to a domain and have run
into a weird situation. I have two machines with the same general
class of hardware (Intel PIIX) and the pass-through works on one
machine but not the other. On the non-working system, the SATA
controller makes port reads, but they don't return the correct values.
For example, when running it in dom0, when polling the bmdma status
register, it returns 0x60 and when polling the command status register
, it returns 0x50. However, when passing it through, the bmdma
register returns 0x0 and the command register returns 0xff (the error
register also returns 0xff, shedding no additional light on the
situation).

This is the hardware on the working system:

00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE
Controller (rev 01) (prog-if 8f [Master SecP SecO PriP PriO])
	Subsystem: Intel Corporation Device 544e
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin B routed to IRQ 19
	Region 0: I/O ports at 20c8 [size=8]
	Region 1: I/O ports at 20ec [size=4]
	Region 2: I/O ports at 20c0 [size=8]
	Region 3: I/O ports at 20e8 [size=4]
	Region 4: I/O ports at 20a0 [size=16]
	Region 5: Memory at d02c4000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [70] Power Management version 2
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: ata_piix


This is the hardware on the non-working system:

00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port
SATA IDE Controller #1 (prog-if 8f [Master SecP SecO PriP PriO])
	Subsystem: Dell Device 0293
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin C routed to IRQ 20
	Region 0: I/O ports at fe00 [size=8]
	Region 1: I/O ports at fe10 [size=4]
	Region 2: I/O ports at fe20 [size=8]
	Region 3: I/O ports at fe30 [size=4]
	Region 4: I/O ports at fec0 [size=16]
	Region 5: I/O ports at ecc0 [size=16]
	Capabilities: [70] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [b0] PCIe advanced features <?>
	Kernel driver in use: ata_piix


Does anybody have any thoughts about what might be going on here?


Patrick

Re: SATA pass-through woes

[Patrick Colp]

A quick addendum to this:

I've tried with both PVOPS (2.6.31.13 and 2.6.32.26) and 2.6.18
kernels. For 2.6.18, the SATA driver doesn't work, so I use the PIIXn
regular ATA IDE driver (in the BIOS I have the SATA set to export the
device in IDE mode rather than AHCI, as I don't think I can pass AHCI
devices through).

The issue is the same in both (constantly reading in 0xff from the
status port and SRST failing and eventually giving up).


Patrick


On 4 March 2011 18:38, Patrick Colp <pjcolp@cs.ubc.ca> wrote:
> I'm trying to pass my SATA controller through to a domain and have run
> into a weird situation. I have two machines with the same general
> class of hardware (Intel PIIX) and the pass-through works on one
> machine but not the other. On the non-working system, the SATA
> controller makes port reads, but they don't return the correct values.
> For example, when running it in dom0, when polling the bmdma status
> register, it returns 0x60 and when polling the command status register
> , it returns 0x50. However, when passing it through, the bmdma
> register returns 0x0 and the command register returns 0xff (the error
> register also returns 0xff, shedding no additional light on the
> situation).
>
> This is the hardware on the working system:
>
> 00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE
> Controller (rev 01) (prog-if 8f [Master SecP SecO PriP PriO])
>        Subsystem: Intel Corporation Device 544e
>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR- FastB2B- DisINTx-
>        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>        Latency: 0
>        Interrupt: pin B routed to IRQ 19
>        Region 0: I/O ports at 20c8 [size=8]
>        Region 1: I/O ports at 20ec [size=4]
>        Region 2: I/O ports at 20c0 [size=8]
>        Region 3: I/O ports at 20e8 [size=4]
>        Region 4: I/O ports at 20a0 [size=16]
>        Region 5: Memory at d02c4000 (32-bit, non-prefetchable) [size=1K]
>        Capabilities: [70] Power Management version 2
>                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
>                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>        Kernel driver in use: ata_piix
>
>
> This is the hardware on the non-working system:
>
> 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port
> SATA IDE Controller #1 (prog-if 8f [Master SecP SecO PriP PriO])
>        Subsystem: Dell Device 0293
>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR- FastB2B- DisINTx-
>        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>        Latency: 0
>        Interrupt: pin C routed to IRQ 20
>        Region 0: I/O ports at fe00 [size=8]
>        Region 1: I/O ports at fe10 [size=4]
>        Region 2: I/O ports at fe20 [size=8]
>        Region 3: I/O ports at fe30 [size=4]
>        Region 4: I/O ports at fec0 [size=16]
>        Region 5: I/O ports at ecc0 [size=16]
>        Capabilities: [70] Power Management version 3
>                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
>                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>        Capabilities: [b0] PCIe advanced features <?>
>        Kernel driver in use: ata_piix
>
>
> Does anybody have any thoughts about what might be going on here?
>
>
> Patrick
>

Re: SATA pass-through woes

[Patrick Colp]

It looks like I solved the problem. I turned on AHCI mode in the BIOS
and I could indeed pass through the device. Doing this, it found the
controller and the drive and connected without any trouble. There's a
chance I did some other magic-fu, but I don't think so.

I got it working with 2.6.31.13, but not with 2.6.32.26, but I think
it may just be config issues.


Patrick


On 4 March 2011 19:24, Patrick Colp <pjcolp@cs.ubc.ca> wrote:
> A quick addendum to this:
>
> I've tried with both PVOPS (2.6.31.13 and 2.6.32.26) and 2.6.18
> kernels. For 2.6.18, the SATA driver doesn't work, so I use the PIIXn
> regular ATA IDE driver (in the BIOS I have the SATA set to export the
> device in IDE mode rather than AHCI, as I don't think I can pass AHCI
> devices through).
>
> The issue is the same in both (constantly reading in 0xff from the
> status port and SRST failing and eventually giving up).
>
>
> Patrick
>
>
> On 4 March 2011 18:38, Patrick Colp <pjcolp@cs.ubc.ca> wrote:
>> I'm trying to pass my SATA controller through to a domain and have run
>> into a weird situation. I have two machines with the same general
>> class of hardware (Intel PIIX) and the pass-through works on one
>> machine but not the other. On the non-working system, the SATA
>> controller makes port reads, but they don't return the correct values.
>> For example, when running it in dom0, when polling the bmdma status
>> register, it returns 0x60 and when polling the command status register
>> , it returns 0x50. However, when passing it through, the bmdma
>> register returns 0x0 and the command register returns 0xff (the error
>> register also returns 0xff, shedding no additional light on the
>> situation).
>>
>> This is the hardware on the working system:
>>
>> 00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE
>> Controller (rev 01) (prog-if 8f [Master SecP SecO PriP PriO])
>>        Subsystem: Intel Corporation Device 544e
>>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
>> Stepping- SERR- FastB2B- DisINTx-
>>        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>        Latency: 0
>>        Interrupt: pin B routed to IRQ 19
>>        Region 0: I/O ports at 20c8 [size=8]
>>        Region 1: I/O ports at 20ec [size=4]
>>        Region 2: I/O ports at 20c0 [size=8]
>>        Region 3: I/O ports at 20e8 [size=4]
>>        Region 4: I/O ports at 20a0 [size=16]
>>        Region 5: Memory at d02c4000 (32-bit, non-prefetchable) [size=1K]
>>        Capabilities: [70] Power Management version 2
>>                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
>>                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>>        Kernel driver in use: ata_piix
>>
>>
>> This is the hardware on the non-working system:
>>
>> 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port
>> SATA IDE Controller #1 (prog-if 8f [Master SecP SecO PriP PriO])
>>        Subsystem: Dell Device 0293
>>        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
>> Stepping- SERR- FastB2B- DisINTx-
>>        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>        Latency: 0
>>        Interrupt: pin C routed to IRQ 20
>>        Region 0: I/O ports at fe00 [size=8]
>>        Region 1: I/O ports at fe10 [size=4]
>>        Region 2: I/O ports at fe20 [size=8]
>>        Region 3: I/O ports at fe30 [size=4]
>>        Region 4: I/O ports at fec0 [size=16]
>>        Region 5: I/O ports at ecc0 [size=16]
>>        Capabilities: [70] Power Management version 3
>>                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
>>                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>>        Capabilities: [b0] PCIe advanced features <?>
>>        Kernel driver in use: ata_piix
>>
>>
>> Does anybody have any thoughts about what might be going on here?
>>
>>
>> Patrick
>>
>

Re: Re: SATA pass-through woes

[Konrad Rzeszutek Wilk]

On Fri, Mar 04, 2011 at 07:24:24PM -0800, Patrick Colp wrote:
> A quick addendum to this:
> 
> I've tried with both PVOPS (2.6.31.13 and 2.6.32.26) and 2.6.18
> kernels. For 2.6.18, the SATA driver doesn't work, so I use the PIIXn
> regular ATA IDE driver (in the BIOS I have the SATA set to export the
> device in IDE mode rather than AHCI, as I don't think I can pass AHCI
> devices through).

And what does your guest config file look like? What does your serial log
look like? What version of Xen are you using (please provide c/s)?Do other
devices on the "failing" machine work with pass-through? What kind of
machine is this? Does it have an IOMMU?
> 
> The issue is the same in both (constantly reading in 0xff from the
> status port and SRST failing and eventually giving up).
> 
> 
> Patrick
> 
> 
> On 4 March 2011 18:38, Patrick Colp <pjcolp@cs.ubc.ca> wrote:
> > I'm trying to pass my SATA controller through to a domain and have run
> > into a weird situation. I have two machines with the same general
> > class of hardware (Intel PIIX) and the pass-through works on one
> > machine but not the other. On the non-working system, the SATA
> > controller makes port reads, but they don't return the correct values.
> > For example, when running it in dom0, when polling the bmdma status
> > register, it returns 0x60 and when polling the command status register
> > , it returns 0x50. However, when passing it through, the bmdma
> > register returns 0x0 and the command register returns 0xff (the error
> > register also returns 0xff, shedding no additional light on the
> > situation).
> >
> > This is the hardware on the working system:
> >
> > 00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE
> > Controller (rev 01) (prog-if 8f [Master SecP SecO PriP PriO])
> >        Subsystem: Intel Corporation Device 544e
> >        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > Stepping- SERR- FastB2B- DisINTx-
> >        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> > <TAbort- <MAbort- >SERR- <PERR- INTx-
> >        Latency: 0
> >        Interrupt: pin B routed to IRQ 19
> >        Region 0: I/O ports at 20c8 [size=8]
> >        Region 1: I/O ports at 20ec [size=4]
> >        Region 2: I/O ports at 20c0 [size=8]
> >        Region 3: I/O ports at 20e8 [size=4]
> >        Region 4: I/O ports at 20a0 [size=16]
> >        Region 5: Memory at d02c4000 (32-bit, non-prefetchable) [size=1K]
> >        Capabilities: [70] Power Management version 2
> >                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
> >                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
> >        Kernel driver in use: ata_piix
> >
> >
> > This is the hardware on the non-working system:
> >
> > 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port
> > SATA IDE Controller #1 (prog-if 8f [Master SecP SecO PriP PriO])
> >        Subsystem: Dell Device 0293
> >        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > Stepping- SERR- FastB2B- DisINTx-
> >        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> > <TAbort- <MAbort- >SERR- <PERR- INTx-
> >        Latency: 0
> >        Interrupt: pin C routed to IRQ 20
> >        Region 0: I/O ports at fe00 [size=8]
> >        Region 1: I/O ports at fe10 [size=4]
> >        Region 2: I/O ports at fe20 [size=8]
> >        Region 3: I/O ports at fe30 [size=4]
> >        Region 4: I/O ports at fec0 [size=16]
> >        Region 5: I/O ports at ecc0 [size=16]
> >        Capabilities: [70] Power Management version 3
> >                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
> >                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
> >        Capabilities: [b0] PCIe advanced features <?>
> >        Kernel driver in use: ata_piix
> >
> >
> > Does anybody have any thoughts about what might be going on here?
> >
> >
> > Patrick
> >
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Re: SATA pass-through woes

[Patrick Colp]

On 7 March 2011 09:36, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> On Fri, Mar 04, 2011 at 07:24:24PM -0800, Patrick Colp wrote:
>> A quick addendum to this:
>>
>> I've tried with both PVOPS (2.6.31.13 and 2.6.32.26) and 2.6.18
>> kernels. For 2.6.18, the SATA driver doesn't work, so I use the PIIXn
>> regular ATA IDE driver (in the BIOS I have the SATA set to export the
>> device in IDE mode rather than AHCI, as I don't think I can pass AHCI
>> devices through).
>
> And what does your guest config file look like? What does your serial log
> look like? What version of Xen are you using (please provide c/s)?Do other
> devices on the "failing" machine work with pass-through? What kind of
> machine is this? Does it have an IOMMU?

I actually fixed the issue by using AHCI mode for SATA. I could then
pass through the device and mount the drive without any issue. I
suppose the only downside is that in IDE mode, the SATA controller was
exposed as two separate devices, so I could pass through each
controller to a different domain (to give two domains raw access to
different disks). Not really a huge deal, though.

For reference, I'm running xen-unstable revision 22155, my network
device could be passed through with no issue, it's a Nehalem Xeon
W3520, and it does indeed have an IOMMU.


Patrick


>>
>> The issue is the same in both (constantly reading in 0xff from the
>> status port and SRST failing and eventually giving up).
>>
>>
>> Patrick
>>
>>
>> On 4 March 2011 18:38, Patrick Colp <pjcolp@cs.ubc.ca> wrote:
>> > I'm trying to pass my SATA controller through to a domain and have run
>> > into a weird situation. I have two machines with the same general
>> > class of hardware (Intel PIIX) and the pass-through works on one
>> > machine but not the other. On the non-working system, the SATA
>> > controller makes port reads, but they don't return the correct values.
>> > For example, when running it in dom0, when polling the bmdma status
>> > register, it returns 0x60 and when polling the command status register
>> > , it returns 0x50. However, when passing it through, the bmdma
>> > register returns 0x0 and the command register returns 0xff (the error
>> > register also returns 0xff, shedding no additional light on the
>> > situation).
>> >
>> > This is the hardware on the working system:
>> >
>> > 00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE
>> > Controller (rev 01) (prog-if 8f [Master SecP SecO PriP PriO])
>> >        Subsystem: Intel Corporation Device 544e
>> >        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
>> > Stepping- SERR- FastB2B- DisINTx-
>> >        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
>> > <TAbort- <MAbort- >SERR- <PERR- INTx-
>> >        Latency: 0
>> >        Interrupt: pin B routed to IRQ 19
>> >        Region 0: I/O ports at 20c8 [size=8]
>> >        Region 1: I/O ports at 20ec [size=4]
>> >        Region 2: I/O ports at 20c0 [size=8]
>> >        Region 3: I/O ports at 20e8 [size=4]
>> >        Region 4: I/O ports at 20a0 [size=16]
>> >        Region 5: Memory at d02c4000 (32-bit, non-prefetchable) [size=1K]
>> >        Capabilities: [70] Power Management version 2
>> >                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold-)
>> >                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>> >        Kernel driver in use: ata_piix
>> >
>> >
>> > This is the hardware on the non-working system:
>> >
>> > 00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port
>> > SATA IDE Controller #1 (prog-if 8f [Master SecP SecO PriP PriO])
>> >        Subsystem: Dell Device 0293
>> >        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
>> > Stepping- SERR- FastB2B- DisINTx-
>> >        Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
>> > <TAbort- <MAbort- >SERR- <PERR- INTx-
>> >        Latency: 0
>> >        Interrupt: pin C routed to IRQ 20
>> >        Region 0: I/O ports at fe00 [size=8]
>> >        Region 1: I/O ports at fe10 [size=4]
>> >        Region 2: I/O ports at fe20 [size=8]
>> >        Region 3: I/O ports at fe30 [size=4]
>> >        Region 4: I/O ports at fec0 [size=16]
>> >        Region 5: I/O ports at ecc0 [size=16]
>> >        Capabilities: [70] Power Management version 3
>> >                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
>> >                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>> >        Capabilities: [b0] PCIe advanced features <?>
>> >        Kernel driver in use: ata_piix
>> >
>> >
>> > Does anybody have any thoughts about what might be going on here?
>> >
>> >
>> > Patrick
>> >
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>