Re: IOMMU/AMD-Vi not working after XSA-36 with 970A-UD3

[Hans Mueller <mcbeagle@gmx.de>]

On Saturday, 4. May 2013 12:20:14 Marcus Osdoba wrote:
> Am 04.05.2013 02:09, schrieb Eric Shelton:
> > Did you try the xen boot option mentioned in XSA-36?
> > 
> > iommu=amd-iommu-perdev-intremap
> > 
> > Add it to the line in grub.conf for the hypervisor (e.g., "kernel
> > /boot/xen.gz dom0_mem=2048M iommu=1,no-amd-iommu-perdev-intremap")
> 
> Hello Eric,
> 
> Thanks for the hint. Unfortuantly this wasn't a solution as reported in
> [0]. Even with the option "no-amd-iommu-perdev-intremap" the I/O
> virtualisation remains disabled (xm dmesg output see below).

XSA-36 says that for Xen 4.1.x 'iommu=amd-iommu-global-intremap' is the 
related parameter instead of 'iommu=amd-iommu-perdev-intremap'.


In addition my remark that it would not really help was refered to problems 
regarding interrupt sharing which persisted and seem to be independent of 
XSA-36 and the disabled I/O virtualisation.


For me 'iommu=amd-iommu-perdev-intremap' (Xen 4.2.x) worked and enabled the 
I/O virtualisation with 'global vector map':

(XEN) Xen version 4.2.2-pre (@sec.chaos) (gcc (Gentoo Hardened 4.6.3 p1.11, 
pie-0.5.2) 4.6.3) Sun Mar  3 16:35:02 CET 2013
(XEN) Latest ChangeSet: Wed Feb 13 17:00:15 2013 +0000 26013:e28ffa5410df
...
(XEN) Command line: ucode=-1 dom0_mem=1024M,max:1024M com1=115200,8n1,0x3f8,4 
console=com1 cpufreq=xen:ondemand loglvl=all guest_loglvl=all 
apic_verbosity=debug e820-verbose=1 iommu=debug,verbose,no-amd-iommu-perdev-
intremap
...
(XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0xd7
(XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x8
(XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0xd7
(XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x2 handle 0x0
(XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0x0
(XEN) AMD-Vi: IVHD Special: 0000:00:00.1 variety 0x1 handle 0x8
(XEN) IVHD Error: Conflicting IO-APIC 0x8 entries
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) AMD-Vi: Enabling global vector map
(XEN) AMD-Vi: Using global interrupt remap table is not recommended (see 
XSA-36)!
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed


However, since upgrading the BIOS to version F8c this is no longer required, 
the conflicting IO-APIC entry is removed and 'per-device vector maps' are 
enabled w/o passing any IOMMU related parameter to Xen:

(XEN) Xen version 4.2.2 (@sec.chaos) (gcc (Gentoo Hardened 4.6.3 p1.11, 
pie-0.5.2) 4.6.3) Sun Apr 28 03:45:10 CEST 2013
(XEN) Latest ChangeSet: Tue Apr 23 18:42:55 2013 +0200 26064:754008dbaa6c
...
(XEN) Command line: com1=115200,8n1,0x3f8,4 console=com1 ucode=-1 
cpufreq=xen:ondemand dom0_mem=1024M,max:1024M conring_size=64k loglvl=all 
guest_loglvl=all cpuinfo=on e820-verbose=on iommu=debug apic_verbosity=debug
...
(XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0xd7
(XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x1 handle 0x8
(XEN) AMD-Vi: IVHD Device Entry: type 0x48 id 0x0 flags 0xd7
(XEN) AMD-Vi: IVHD Special: 0000:00:14.0 variety 0x2 handle 0x0
(XEN) AMD-Vi: IOMMU 0 Enabled.
(XEN) AMD-Vi: Enabling per-device vector maps
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled


Of course the northbridge IO-APIC is disabled but it wasn't properly setup and 
didn't work before, either.

Regards
Hans