Re: Some test results on Xen 4.0 and 2.6.31 / 2.6.32 pvops kernels

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Apr 08, 2010 at 09:46:58PM +0200, Carsten Schiers wrote:
> OK, I am still a bit fuzzy as I had to learn how to git and configure 
> kernels, and I am still unsure whether I did everything 
> right. Nevertheless, and even when I cannot provide a lot of details, I 
> would like to inform you about some anomalies. 
> 
> Currently, I run Xen 3.4.1 and 2.6.18.8 kernels, 64 Bit Dom0 and some 32 
> Bit and 64 Bit DomUs on a Gigabyte M56S-S3 / AMD 4050e 
> with cpufreq on powernow-k8 in cpufreq=dom0-kernel mode. I make use of 
> PCI Passthrough for DVB-C cards and onboard NIC, as well 
> as for USB controller.
> 
> 
> 
> Compile problem on xen/stable-2.6.32
> ------------------------------------
> 
> I compiled 2.6.32.10-pvops for DomU only mode, resulting in compile 
> error very similar to 
> 
>   
> http://lists.xensource.com/archives/html/xen-devel/2010-03/msg00925.html
> 
> I get a "static declaration of 'xen_register_gsi' follows a non-static 
> declaration, because I have set CONFIG_XEN_PCI_MSI and 
> CONFIG_PCI_XEN, but not CONFIG_XEN_DOM0_PCI. It will disappear when 
> compiling a DOM0 kernel.
> 
Can you do a copy-n-paste of your compile problem, please?
> 
> 
> PCI controller / 2nd Function? / working in xen/master-2.6.32.13, not 
> working on xen/stable-2.6.32 on Xen 3.4.1
> ------------------------------------------------------------------------
> ---------------------------------------
> 
> It will allow passthrough of all mentioned devices except one of the PCI 
> controllers. It will not be detected. Using 
> xm pci-attach will produce an error:
> 
>   troi kernel: [   23.862294] ehci_hcd 0000:00:01.1: device not 
> available because of BAR 0 [0xfc102000-0xfc1020ff] collisions
>   troi kernel: [   61.942809] ohci_hcd 0000:00:01.0: device not 
> available because of BAR 0 [0xfc104000-0xfc104fff] collisions

How much memory did you assign to the guest? If it is below 4GB, say 2GB
did you see these errors?

> nVidia forcedeth, working on pvops/Xen 3.4.1, not working on pvops/Xen 
> 4.0


> ------------------------------------------------------------------------
> --
> 
> The whole experiment was done because when I use Xen 4.0 with my 
> 2.6.18.8 kernels, I cannot use the onboard NIC any longer. The 
> driver forcedeth will report an error. So, as using the pvops DomU on 
> Xen 3.4.1 worked, I tried to also use a pvops Dom0 underneath

So.. DomU and Dom0 with the same kernel works?

> Xen 4.0, but the problem seems to be related to the Xen 4.0, which is 
> killing forcedeth MAC address detection. By the way: MSI has
> never worked for me on that NIC. So I had pci=nomsi on kernel.
> 
> not working example (MAC is random):
> 
>   troi kernel: [    4.335591] forcedeth: Reverse Engineered nForce 
> ethernet driver. Version 0.64.
>   troi kernel: [    4.335811] forcedeth 0000:00:00.0: enabling device 
> (0000 -> 0003)
>   troi kernel: [    4.335942] forcedeth 0000:00:00.0: Xen PCI enabling 
> IRQ: 23
>   troi kernel: [    4.336016] forcedeth 0000:00:00.0: setting latency 
> timer to 64
>   troi kernel: [    4.336271] forcedeth 0000:00:00.0: Invalid Mac 
> address detected: 00:00:00:00:00:00
>   troi kernel: [    4.336295] forcedeth 0000:00:00.0: Please complain to 
> your hardware vendor. Switching to a random MAC.
>   troi kernel: [    4.860537] forcedeth 0000:00:00.0: ifname eth0, PHY 
> OUI 0x732 @ 1, addr 5e:1a:09:69:77:5d
>   troi kernel: [    4.860593] forcedeth 0000:00:00.0: highdma pwrctl 
> mgmt gbit lnktim msi desc-v3
> 
> working example (real MAC, note different IRQ):
>  
>   troi kernel: [    1.133102] forcedeth: Reverse Engineered nForce 
> ethernet driver. Version 0.64.
>   troi kernel: [    1.133376] forcedeth 0000:00:01.0: enabling device 
> (0000 -> 0003)
>   troi kernel: [    1.133463] forcedeth 0000:00:01.0: Xen PCI enabling 
> IRQ: 19
>   troi kernel: [    1.652048] forcedeth 0000:00:01.0: ifname eth0, PHY 
> OUI 0x732 @ 1, addr 00:1d:7d:e8:97:7b
>   troi kernel: [    1.652139] forcedeth 0000:00:01.0: highdma pwrctl 
> mgmt gbit lnktim msi desc-v3

I got lost in your description. The failing case happens when
your Dom0 is 2.6.18 or the 2.6.31 (or 2.6.32?) And is the DomU pv-ops
or 2.6.18?