* Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)
@ 2013-07-22 12:55 Stefan Bader
2013-08-05 12:58 ` Jan Beulich
0 siblings, 1 reply; 6+ messages in thread
From: Stefan Bader @ 2013-07-22 12:55 UTC (permalink / raw)
To: xen-devel@lists.xensource.com
[-- Attachment #1.1: Type: text/plain, Size: 2331 bytes --]
While testing Xen 4.3 on my AMD testbox I noticed the following output from the
hypervisor (this has no visible effect on at least simple operation):
(XEN) setup 0000:00:18.0 for d0 failed (-19)
(XEN) setup 0000:00:18.1 for d0 failed (-19)
(XEN) setup 0000:00:18.2 for d0 failed (-19)
(XEN) setup 0000:00:18.3 for d0 failed (-19)
(XEN) setup 0000:00:18.4 for d0 failed (-19)
(XEN) setup 0000:00:19.0 for d0 failed (-19)
(XEN) setup 0000:00:19.1 for d0 failed (-19)
(XEN) setup 0000:00:19.2 for d0 failed (-19)
(XEN) setup 0000:00:19.3 for d0 failed (-19)
(XEN) setup 0000:00:19.4 for d0 failed (-19)
The PCI devices related to the output are all PCI host bridges:
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor HyperTransport Configuration
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor Address Map
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor Miscellaneous Control
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor Link Control
00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor HyperTransport Configuration
00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor Address Map
00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor DRAM Controller
00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor Miscellaneous Control
00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
Family 10h Processor Link Control
This all seems to be related to PCI passthrough setup started from
xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
While the Intel code skips over bridge type entries, the AMD code has no such
exception and will fail the handler with -ENODEV when find_iommu_for_device
fails. But I am not sure one can just compare implementations here.
Would someone have more insight to decide whether skipping host bridges in
amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
caused by this but it does not look good and did not happen before.
-Stefan
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)
2013-07-22 12:55 Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV) Stefan Bader
@ 2013-08-05 12:58 ` Jan Beulich
2013-08-06 2:02 ` Suravee Suthikulanit
0 siblings, 1 reply; 6+ messages in thread
From: Jan Beulich @ 2013-08-05 12:58 UTC (permalink / raw)
To: Stefan Bader; +Cc: xen-devel, suravee.suthikulpanit
>>> On 22.07.13 at 14:55, Stefan Bader <stefan.bader@canonical.com> wrote:
> While testing Xen 4.3 on my AMD testbox I noticed the following output from
> the
> hypervisor (this has no visible effect on at least simple operation):
>
> (XEN) setup 0000:00:18.0 for d0 failed (-19)
> (XEN) setup 0000:00:18.1 for d0 failed (-19)
> (XEN) setup 0000:00:18.2 for d0 failed (-19)
> (XEN) setup 0000:00:18.3 for d0 failed (-19)
> (XEN) setup 0000:00:18.4 for d0 failed (-19)
> (XEN) setup 0000:00:19.0 for d0 failed (-19)
> (XEN) setup 0000:00:19.1 for d0 failed (-19)
> (XEN) setup 0000:00:19.2 for d0 failed (-19)
> (XEN) setup 0000:00:19.3 for d0 failed (-19)
> (XEN) setup 0000:00:19.4 for d0 failed (-19)
>
> The PCI devices related to the output are all PCI host bridges:
>
> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor HyperTransport Configuration
> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor Address Map
> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor DRAM Controller
> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor Miscellaneous Control
> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor Link Control
> 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor HyperTransport Configuration
> 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor Address Map
> 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor DRAM Controller
> 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor Miscellaneous Control
> 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
> Family 10h Processor Link Control
>
> This all seems to be related to PCI passthrough setup started from
> xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
> xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
>
> While the Intel code skips over bridge type entries, the AMD code has no
> such
> exception and will fail the handler with -ENODEV when find_iommu_for_device
> fails. But I am not sure one can just compare implementations here.
>
> Would someone have more insight to decide whether skipping host bridges in
> amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
> caused by this but it does not look good and did not happen before.
Fundamentally this looks like a BIOS bug, and hence doing as you
suggest unconditionally is not an option imo. Adding code to skip
host bridges (under the assumption that they won't initiate
transactions that might require translation by the IOMMU) may be
an option if enabled only via command line option.
In any event I'd like to hear Suravee's opinion on this first.
Jan
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)
2013-08-05 12:58 ` Jan Beulich
@ 2013-08-06 2:02 ` Suravee Suthikulanit
2013-08-06 6:55 ` Jan Beulich
0 siblings, 1 reply; 6+ messages in thread
From: Suravee Suthikulanit @ 2013-08-06 2:02 UTC (permalink / raw)
To: Jan Beulich; +Cc: xen-devel, Stefan Bader
On 8/5/2013 7:58 AM, Jan Beulich wrote:
>>>> On 22.07.13 at 14:55, Stefan Bader <stefan.bader@canonical.com> wrote:
>> While testing Xen 4.3 on my AMD testbox I noticed the following output from
>> the
>> hypervisor (this has no visible effect on at least simple operation):
>>
>> (XEN) setup 0000:00:18.0 for d0 failed (-19)
>> (XEN) setup 0000:00:18.1 for d0 failed (-19)
>> (XEN) setup 0000:00:18.2 for d0 failed (-19)
>> (XEN) setup 0000:00:18.3 for d0 failed (-19)
>> (XEN) setup 0000:00:18.4 for d0 failed (-19)
>> (XEN) setup 0000:00:19.0 for d0 failed (-19)
>> (XEN) setup 0000:00:19.1 for d0 failed (-19)
>> (XEN) setup 0000:00:19.2 for d0 failed (-19)
>> (XEN) setup 0000:00:19.3 for d0 failed (-19)
>> (XEN) setup 0000:00:19.4 for d0 failed (-19)
>>
>> The PCI devices related to the output are all PCI host bridges:
>>
>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor HyperTransport Configuration
>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor Address Map
>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor DRAM Controller
>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor Miscellaneous Control
>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor Link Control
>> 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor HyperTransport Configuration
>> 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor Address Map
>> 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor DRAM Controller
>> 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor Miscellaneous Control
>> 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>> Family 10h Processor Link Control
>>
>> This all seems to be related to PCI passthrough setup started from
>> xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
>> xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
>>
>> While the Intel code skips over bridge type entries, the AMD code has no
>> such
>> exception and will fail the handler with -ENODEV when find_iommu_for_device
>> fails. But I am not sure one can just compare implementations here.
>>
>> Would someone have more insight to decide whether skipping host bridges in
>> amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
>> caused by this but it does not look good and did not happen before.
> Fundamentally this looks like a BIOS bug, and hence doing as you
> suggest unconditionally is not an option imo.
Actually, I don't think this is a BIOS bug. I have looked through the IVRS table
on the system that I have and it looks correct to me in the sense that it does not list
all these processor host bridge devices for IOMMU. As Stefan mentioned, the current
dom0 PCI setup code scans through PCI bus does not have filter for the host bridges which
do not need IOMMU. However, these error message should not be harmful. Also, in debug mode,
it clearly state that "No iommu for device 0000:00:18.0".
> Adding code to skip host bridges (under the assumption that they won't initiate
> transactions that might require translation by the IOMMU) may be
> an option if enabled only via command line option.
Currently, pdev_type() in driver/passthrough/pci.c only match the PCI_CLASS_BRIDGE_PCI (0x0604)
and not the Host/PCI bridge (0x600) which I see what the devices 0:18.x, 0x19.x are reporting in
the PCI class code structure. This might need to change?
Suravee
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)
2013-08-06 2:02 ` Suravee Suthikulanit
@ 2013-08-06 6:55 ` Jan Beulich
2013-08-06 14:11 ` Suravee Suthikulanit
0 siblings, 1 reply; 6+ messages in thread
From: Jan Beulich @ 2013-08-06 6:55 UTC (permalink / raw)
To: Suravee Suthikulanit; +Cc: xen-devel, Stefan Bader
>>> On 06.08.13 at 04:02, Suravee Suthikulanit <suravee.suthikulpanit@amd.com>
wrote:
> On 8/5/2013 7:58 AM, Jan Beulich wrote:
>>>>> On 22.07.13 at 14:55, Stefan Bader <stefan.bader@canonical.com> wrote:
>>> While testing Xen 4.3 on my AMD testbox I noticed the following output from
>>> the
>>> hypervisor (this has no visible effect on at least simple operation):
>>>
>>> (XEN) setup 0000:00:18.0 for d0 failed (-19)
>>> (XEN) setup 0000:00:18.1 for d0 failed (-19)
>>> (XEN) setup 0000:00:18.2 for d0 failed (-19)
>>> (XEN) setup 0000:00:18.3 for d0 failed (-19)
>>> (XEN) setup 0000:00:18.4 for d0 failed (-19)
>>> (XEN) setup 0000:00:19.0 for d0 failed (-19)
>>> (XEN) setup 0000:00:19.1 for d0 failed (-19)
>>> (XEN) setup 0000:00:19.2 for d0 failed (-19)
>>> (XEN) setup 0000:00:19.3 for d0 failed (-19)
>>> (XEN) setup 0000:00:19.4 for d0 failed (-19)
>>>
>>> The PCI devices related to the output are all PCI host bridges:
>>>
>>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor HyperTransport Configuration
>>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor Address Map
>>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor DRAM Controller
>>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor Miscellaneous Control
>>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor Link Control
>>> 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor HyperTransport Configuration
>>> 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor Address Map
>>> 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor DRAM Controller
>>> 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor Miscellaneous Control
>>> 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>> Family 10h Processor Link Control
>>>
>>> This all seems to be related to PCI passthrough setup started from
>>> xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
>>> xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
>>>
>>> While the Intel code skips over bridge type entries, the AMD code has no
>>> such
>>> exception and will fail the handler with -ENODEV when find_iommu_for_device
>>> fails. But I am not sure one can just compare implementations here.
>>>
>>> Would someone have more insight to decide whether skipping host bridges in
>>> amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
>>> caused by this but it does not look good and did not happen before.
>> Fundamentally this looks like a BIOS bug, and hence doing as you
>> suggest unconditionally is not an option imo.
>
> Actually, I don't think this is a BIOS bug. I have looked through the IVRS
> table
> on the system that I have and it looks correct to me in the sense that it
> does not list
> all these processor host bridge devices for IOMMU. As Stefan mentioned, the
> current
> dom0 PCI setup code scans through PCI bus does not have filter for the host
> bridges which
> do not need IOMMU. However, these error message should not be harmful.
> Also, in debug mode,
> it clearly state that "No iommu for device 0000:00:18.0".
So are you saying that you also see those messages in debug mode?
If so, i.e. if they are expected to always appear, we should indeed
try to avoid issuing them, as they may cause unnecessary worries
(as is the case for Stefan here).
>> Adding code to skip host bridges (under the assumption that they won't
> initiate
>> transactions that might require translation by the IOMMU) may be
>> an option if enabled only via command line option.
>
> Currently, pdev_type() in driver/passthrough/pci.c only match the
> PCI_CLASS_BRIDGE_PCI (0x0604)
> and not the Host/PCI bridge (0x600) which I see what the devices 0:18.x,
> 0x19.x are reporting in
> the PCI class code structure. This might need to change?
We may want to change this, but this needs to be done carefully
to not cause regressions on the VT-d side (to which host bridges
currently are the same as ordinary PCI devices/PCIe endpoints).
Jan
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)
2013-08-06 6:55 ` Jan Beulich
@ 2013-08-06 14:11 ` Suravee Suthikulanit
2013-08-06 14:53 ` Jan Beulich
0 siblings, 1 reply; 6+ messages in thread
From: Suravee Suthikulanit @ 2013-08-06 14:11 UTC (permalink / raw)
To: Jan Beulich; +Cc: xen-devel, Stefan Bader
On 8/6/2013 1:55 AM, Jan Beulich wrote:
>>>> On 06.08.13 at 04:02, Suravee Suthikulanit <suravee.suthikulpanit@amd.com>
> wrote:
>> On 8/5/2013 7:58 AM, Jan Beulich wrote:
>>>>>> On 22.07.13 at 14:55, Stefan Bader <stefan.bader@canonical.com> wrote:
>>>> While testing Xen 4.3 on my AMD testbox I noticed the following output from
>>>> the
>>>> hypervisor (this has no visible effect on at least simple operation):
>>>>
>>>> (XEN) setup 0000:00:18.0 for d0 failed (-19)
>>>> (XEN) setup 0000:00:18.1 for d0 failed (-19)
>>>> (XEN) setup 0000:00:18.2 for d0 failed (-19)
>>>> (XEN) setup 0000:00:18.3 for d0 failed (-19)
>>>> (XEN) setup 0000:00:18.4 for d0 failed (-19)
>>>> (XEN) setup 0000:00:19.0 for d0 failed (-19)
>>>> (XEN) setup 0000:00:19.1 for d0 failed (-19)
>>>> (XEN) setup 0000:00:19.2 for d0 failed (-19)
>>>> (XEN) setup 0000:00:19.3 for d0 failed (-19)
>>>> (XEN) setup 0000:00:19.4 for d0 failed (-19)
>>>>
>>>> The PCI devices related to the output are all PCI host bridges:
>>>>
>>>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor HyperTransport Configuration
>>>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor Address Map
>>>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor DRAM Controller
>>>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor Miscellaneous Control
>>>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor Link Control
>>>> 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor HyperTransport Configuration
>>>> 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor Address Map
>>>> 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor DRAM Controller
>>>> 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor Miscellaneous Control
>>>> 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>> Family 10h Processor Link Control
>>>>
>>>> This all seems to be related to PCI passthrough setup started from
>>>> xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
>>>> xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
>>>>
>>>> While the Intel code skips over bridge type entries, the AMD code has no
>>>> such
>>>> exception and will fail the handler with -ENODEV when find_iommu_for_device
>>>> fails. But I am not sure one can just compare implementations here.
>>>>
>>>> Would someone have more insight to decide whether skipping host bridges in
>>>> amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
>>>> caused by this but it does not look good and did not happen before.
>>> Fundamentally this looks like a BIOS bug, and hence doing as you
>>> suggest unconditionally is not an option imo.
>> Actually, I don't think this is a BIOS bug. I have looked through the IVRS
>> table
>> on the system that I have and it looks correct to me in the sense that it
>> does not list
>> all these processor host bridge devices for IOMMU. As Stefan mentioned, the
>> current
>> dom0 PCI setup code scans through PCI bus does not have filter for the host
>> bridges which
>> do not need IOMMU. However, these error message should not be harmful.
>> Also, in debug mode,
>> it clearly state that "No iommu for device 0000:00:18.0".
> So are you saying that you also see those messages in debug mode?
The default (stock unstable) is showing "(XEN) setup 0000:00:18.0 for d0 failed (-19)".
With the "amd-iommu-debug" boot options, it is also showing "(XEN) AMD-Vi: No iommu for device 0000:00:18.0".
> If so, i.e. if they are expected to always appear, we should indeed
> try to avoid issuing them, as they may cause unnecessary worries
> (as is the case for Stefan here).
Agree
>
>>> Adding code to skip host bridges (under the assumption that they won't
>> initiate
>>> transactions that might require translation by the IOMMU) may be
>>> an option if enabled only via command line option.
>> Currently, pdev_type() in driver/passthrough/pci.c only match the
>> PCI_CLASS_BRIDGE_PCI (0x0604)
>> and not the Host/PCI bridge (0x600) which I see what the devices 0:18.x,
>> 0x19.x are reporting in
>> the PCI class code structure. This might need to change?
> We may want to change this, but this needs to be done carefully
> to not cause regressions on the VT-d side (to which host bridges
> currently are the same as ordinary PCI devices/PCIe endpoints).
Which class code are showing on Intel system for host bridges?
Suravee
>
> Jan
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV)
2013-08-06 14:11 ` Suravee Suthikulanit
@ 2013-08-06 14:53 ` Jan Beulich
0 siblings, 0 replies; 6+ messages in thread
From: Jan Beulich @ 2013-08-06 14:53 UTC (permalink / raw)
To: Suravee Suthikulanit; +Cc: xen-devel, Stefan Bader
>>> On 06.08.13 at 16:11, Suravee Suthikulanit <suravee.suthikulpanit@amd.com>
wrote:
> On 8/6/2013 1:55 AM, Jan Beulich wrote:
>>>>> On 06.08.13 at 04:02, Suravee Suthikulanit <suravee.suthikulpanit@amd.com>
>> wrote:
>>> On 8/5/2013 7:58 AM, Jan Beulich wrote:
>>>>>>> On 22.07.13 at 14:55, Stefan Bader <stefan.bader@canonical.com> wrote:
>>>>> While testing Xen 4.3 on my AMD testbox I noticed the following output from
>>>>> the
>>>>> hypervisor (this has no visible effect on at least simple operation):
>>>>>
>>>>> (XEN) setup 0000:00:18.0 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:18.1 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:18.2 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:18.3 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:18.4 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:19.0 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:19.1 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:19.2 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:19.3 for d0 failed (-19)
>>>>> (XEN) setup 0000:00:19.4 for d0 failed (-19)
>>>>>
>>>>> The PCI devices related to the output are all PCI host bridges:
>>>>>
>>>>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor HyperTransport Configuration
>>>>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor Address Map
>>>>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor DRAM Controller
>>>>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor Miscellaneous Control
>>>>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor Link Control
>>>>> 00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor HyperTransport Configuration
>>>>> 00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor Address Map
>>>>> 00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor DRAM Controller
>>>>> 00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor Miscellaneous Control
>>>>> 00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>> Family 10h Processor Link Control
>>>>>
>>>>> This all seems to be related to PCI passthrough setup started from
>>>>> xen/drivers/passthrough/vtd/iommu.c:intel_iommu_dom0_init() or
>>>>> xen/drivers/passthrough/amd/pci_amd_iommu.c:amd_iommu_dom0_init()
>>>>>
>>>>> While the Intel code skips over bridge type entries, the AMD code has no
>>>>> such
>>>>> exception and will fail the handler with -ENODEV when find_iommu_for_device
>>>>> fails. But I am not sure one can just compare implementations here.
>>>>>
>>>>> Would someone have more insight to decide whether skipping host bridges in
>>>>> amd_iommu_setup_dom0_device would make sense? I do not see any bad effects
>>>>> caused by this but it does not look good and did not happen before.
>>>> Fundamentally this looks like a BIOS bug, and hence doing as you
>>>> suggest unconditionally is not an option imo.
>>> Actually, I don't think this is a BIOS bug. I have looked through the IVRS
>>> table
>>> on the system that I have and it looks correct to me in the sense that it
>>> does not list
>>> all these processor host bridge devices for IOMMU. As Stefan mentioned, the
>>> current
>>> dom0 PCI setup code scans through PCI bus does not have filter for the host
>>> bridges which
>>> do not need IOMMU. However, these error message should not be harmful.
>>> Also, in debug mode,
>>> it clearly state that "No iommu for device 0000:00:18.0".
>> So are you saying that you also see those messages in debug mode?
>
> The default (stock unstable) is showing "(XEN) setup 0000:00:18.0 for d0
> failed (-19)".
> With the "amd-iommu-debug" boot options, it is also showing "(XEN) AMD-Vi: No
> iommu for device 0000:00:18.0".
And both messages are a reason for the reader to expect trouble.
>> If so, i.e. if they are expected to always appear, we should indeed
>> try to avoid issuing them, as they may cause unnecessary worries
>> (as is the case for Stefan here).
> Agree
>>
>>>> Adding code to skip host bridges (under the assumption that they won't
>>> initiate
>>>> transactions that might require translation by the IOMMU) may be
>>>> an option if enabled only via command line option.
>>> Currently, pdev_type() in driver/passthrough/pci.c only match the
>>> PCI_CLASS_BRIDGE_PCI (0x0604)
>>> and not the Host/PCI bridge (0x600) which I see what the devices 0:18.x,
>>> 0x19.x are reporting in
>>> the PCI class code structure. This might need to change?
>> We may want to change this, but this needs to be done carefully
>> to not cause regressions on the VT-d side (to which host bridges
>> currently are the same as ordinary PCI devices/PCIe endpoints).
> Which class code are showing on Intel system for host bridges?
0600 of course, just like AMD's. That's why we need to be careful to
not unintentionally change behavior for VT-d.
Jan
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-08-06 14:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-07-22 12:55 Xen 4.3/AMD: setup ssss:bb:dd.f for d0 failed (-ENODEV) Stefan Bader
2013-08-05 12:58 ` Jan Beulich
2013-08-06 2:02 ` Suravee Suthikulanit
2013-08-06 6:55 ` Jan Beulich
2013-08-06 14:11 ` Suravee Suthikulanit
2013-08-06 14:53 ` Jan Beulich
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.