Xen Kernel (3.0.2) breaks b44 module

Xen Kernel (3.0.2) breaks b44 module

[xen]

I have a new laptop (Dell 9400) that I am trying to work with Xen 3.0.

Xen works fine, runs windows XP under HVM.

But, the Xen kernel on Dom0 has a few problems with some of the hardware.

The network card is a Broadcom 4400 10/100BaseT.

Normally loading the b44 module gives this:

Apr 28 20:07:53 ipanema kernel: [4294683.185000] b44.c:v0.97 (Nov 30, 2005)
Apr 28 20:07:53 ipanema kernel: [4294683.185000] ACPI: PCI Interrupt
0000:03:00.0[A] -> GSI 17 (level, low) -> IRQ 177
Apr 28 20:07:53 ipanema kernel: [4294683.189000] eth0: Broadcom 4400
10/100BaseT Ethernet 00:14:22:f2:57:36

(under an Ubuntu kernel - 2.6.15-21-686 #1 SMP PREEMPT)
Works similarly under a vanilla 2.6.16.

When running a Xen patched 2.6.16 (as per
https://wiki.ubuntu.com/XenVirtualMachine/XenOnUbuntuDapper) or on the Xen
live CD, I get:

Apr 27 00:28:31 ipanema kernel: [   20.038669] b44.c:v0.97 (Nov 30, 2005)
Apr 27 00:28:31 ipanema kernel: [   20.038700] ACPI: PCI Interrupt
0000:03:00.0[A] -> GSI 17 (level, low) -> IRQ 17
Apr 27 00:28:31 ipanema kernel: [   20.038710] b44: No usable DMA
configuration, aborting.
Apr 27 00:28:31 ipanema kernel: [   20.040248] ACPI: PCI interrupt for
device 0000:03:00.0 disabled
Apr 27 00:28:31 ipanema kernel: [   20.040251] b44: probe of 0000:03:00.0
failed with error -5

Am I doing something wrong?

The first difference here is that ACPI allocates IRQ 17 instead of IRQ
177.  Hope that rings bells for somebody :-)

This is the relevant output of lspci -vvvv while it's running under 2.6.16:
0000:03:00.0 Ethernet controller: Broadcom Corporation BCM4401-B0
100Base-TX (rev 02)
        Subsystem: Dell: Unknown device 01cd
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort-
<MAbort- >SERR- <PERR-
        Latency: 64
        Interrupt: pin A routed to IRQ 177
        Region 0: Memory at dcbfe000 (32-bit, non-prefetchable) [size=8K]
Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=2 PME-

and the first two lines of lspci -nvvvv

0000:03:00.0 0200: 14e4:170c (rev 02)
        Subsystem: 1028:01cd

Laptop is Dell 9400 Core Duo T2600 2Gb Ram , SATA 100Gb

The ultimate aim is to use the Intel VT extensions to run unmodified OSes
underneath, but that's not that useful without networking.

Other hints of problems: noticable graphics corruption under framebuffer
console while running xen kernel. Also the cdrom is inaccessible under the
Xen Kernel.

cheers,
Woody

PS: I posted this to Xen-user previously but got no answers :-(

Re: Xen Kernel (3.0.2) breaks b44 module

[Jan Beulich]

This ought to be attributed to the fact that B44 supports only 30-bit DMA addresses (and does special checking), but
XenLinux doesn't have a distinct DMA zone restricted to 24-bit addresses, and both dma_alloc_coherent() and swiotlb are
restricting physical addresses to 31 bits only (at least the former could certainly look at the device's DMA mask and
use that value rather than hard-coding 31). Jan

>>> <xen@wood.id.au> 12.05.06 09:57 >>>
I have a new laptop (Dell 9400) that I am trying to work with Xen 3.0.

Xen works fine, runs windows XP under HVM.

But, the Xen kernel on Dom0 has a few problems with some of the hardware.

The network card is a Broadcom 4400 10/100BaseT.

Normally loading the b44 module gives this:

Apr 28 20:07:53 ipanema kernel: [4294683.185000] b44.c:v0.97 (Nov 30, 2005)
Apr 28 20:07:53 ipanema kernel: [4294683.185000] ACPI: PCI Interrupt
0000:03:00.0[A] -> GSI 17 (level, low) -> IRQ 177
Apr 28 20:07:53 ipanema kernel: [4294683.189000] eth0: Broadcom 4400
10/100BaseT Ethernet 00:14:22:f2:57:36

(under an Ubuntu kernel - 2.6.15-21-686 #1 SMP PREEMPT)
Works similarly under a vanilla 2.6.16.

When running a Xen patched 2.6.16 (as per
https://wiki.ubuntu.com/XenVirtualMachine/XenOnUbuntuDapper) or on the Xen
live CD, I get:

Apr 27 00:28:31 ipanema kernel: [   20.038669] b44.c:v0.97 (Nov 30, 2005)
Apr 27 00:28:31 ipanema kernel: [   20.038700] ACPI: PCI Interrupt
0000:03:00.0[A] -> GSI 17 (level, low) -> IRQ 17
Apr 27 00:28:31 ipanema kernel: [   20.038710] b44: No usable DMA
configuration, aborting.
Apr 27 00:28:31 ipanema kernel: [   20.040248] ACPI: PCI interrupt for
device 0000:03:00.0 disabled
Apr 27 00:28:31 ipanema kernel: [   20.040251] b44: probe of 0000:03:00.0
failed with error -5

Am I doing something wrong?

The first difference here is that ACPI allocates IRQ 17 instead of IRQ
177.  Hope that rings bells for somebody :-)

This is the relevant output of lspci -vvvv while it's running under 2.6.16:
0000:03:00.0 Ethernet controller: Broadcom Corporation BCM4401-B0
100Base-TX (rev 02)
        Subsystem: Dell: Unknown device 01cd
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort-
<MAbort- >SERR- <PERR-
        Latency: 64
        Interrupt: pin A routed to IRQ 177
        Region 0: Memory at dcbfe000 (32-bit, non-prefetchable) [size=8K]
Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold+)
                Status: D0 PME-Enable- DSel=0 DScale=2 PME-

and the first two lines of lspci -nvvvv

0000:03:00.0 0200: 14e4:170c (rev 02)
        Subsystem: 1028:01cd

Laptop is Dell 9400 Core Duo T2600 2Gb Ram , SATA 100Gb

The ultimate aim is to use the Intel VT extensions to run unmodified OSes
underneath, but that's not that useful without networking.

Other hints of problems: noticable graphics corruption under framebuffer
console while running xen kernel. Also the cdrom is inaccessible under the
Xen Kernel.

cheers,
Woody

PS: I posted this to Xen-user previously but got no answers :-(

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com 
http://lists.xensource.com/xen-devel

Re: Xen Kernel (3.0.2) breaks b44 module

[Keir Fraser]

On 12 May 2006, at 09:57, Jan Beulich wrote:

> This ought to be attributed to the fact that B44 supports only 30-bit 
> DMA addresses (and does special checking), but
> XenLinux doesn't have a distinct DMA zone restricted to 24-bit 
> addresses, and both dma_alloc_coherent() and swiotlb are
> restricting physical addresses to 31 bits only (at least the former 
> could certainly look at the device's DMA mask and
> use that value rather than hard-coding 31).

If dma_alloc_coherent isn't doing that then it should be patched to do 
so. Even though it won't help right now (since Xen will reject any 
requests more restricted than 31 bits) it will be needed when Xen is 
fixed to support basket-case hardware.

  -- Keir

Re: Xen Kernel (3.0.2) breaks b44 module

[Jan Beulich]

>>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> 12.05.06 11:03 >>>
>
>On 12 May 2006, at 09:57, Jan Beulich wrote:
>
>> This ought to be attributed to the fact that B44 supports only 30-bit 
>> DMA addresses (and does special checking), but
>> XenLinux doesn't have a distinct DMA zone restricted to 24-bit 
>> addresses, and both dma_alloc_coherent() and swiotlb are
>> restricting physical addresses to 31 bits only (at least the former 
>> could certainly look at the device's DMA mask and
>> use that value rather than hard-coding 31).
>
>If dma_alloc_coherent isn't doing that then it should be patched to do 
>so. Even though it won't help right now (since Xen will reject any 
>requests more restricted than 31 bits) it will be needed when Xen is 
>fixed to support basket-case hardware.

I agree, but this wouldn't help this specific case, as the driver also (and mostly) uses pci_map_single on skb->data,
which ends up in swiotlb. Since swiotlb does its allocation and range restriction during init, adjusting things here
would likely be more complicated (or cost performance, if one wanted to further restrict the range upon use when
needed).

Jan

Re: Xen Kernel (3.0.2) breaks b44 module

[Keir Fraser]

On 12 May 2006, at 10:15, Jan Beulich wrote:

> I agree, but this wouldn't help this specific case, as the driver also 
> (and mostly) uses pci_map_single on skb->data,
> which ends up in swiotlb. Since swiotlb does its allocation and range 
> restriction during init, adjusting things here
> would likely be more complicated (or cost performance, if one wanted 
> to further restrict the range upon use when
> needed).

When we support more flexible allocation in Xen, we'll fix up the 
swiotlb to automatically try to grab some lower memory and/or allow the 
address mask for the physical memory in the swiotlb to be specified on 
the kernel command line.

It sounds like 30-bit-addressable memory will be a good thing to try 
for automatically at init. I'd thought that 31-bit was really as bad as 
it got (except for old ISA-ish hardware with 24-bit limitations, which 
we'll never support).

  -- Keir

Re: Xen Kernel (3.0.2) breaks b44 module

[Chris Wright]

* Keir Fraser (Keir.Fraser@cl.cam.ac.uk) wrote:
> When we support more flexible allocation in Xen, we'll fix up the 
> swiotlb to automatically try to grab some lower memory and/or allow the 
> address mask for the physical memory in the swiotlb to be specified on 
> the kernel command line.

What's the plan here?  I've seen a few bug reports due to 28-bit device
limitations (sound cards).

thanks,
-chris

Re: Xen Kernel (3.0.2) breaks b44 module

[Keir Fraser]

On 12 May 2006, at 19:27, Chris Wright wrote:

>> When we support more flexible allocation in Xen, we'll fix up the
>> swiotlb to automatically try to grab some lower memory and/or allow 
>> the
>> address mask for the physical memory in the swiotlb to be specified on
>> the kernel command line.
>
> What's the plan here?  I've seen a few bug reports due to 28-bit device
> limitations (sound cards).

Memory below 16MB isn't going to happen, at least on 32-bit Xen. It'd 
be easier on 64-bit Xen as we don't need separate notions of Xen heap 
and guest heap. We could support pretty much anything more than 24 bits 
though. It's just a case of making the Xen buddy allocator more 
flexible and keep separate buddy lists for different address widths. If 
we fix for things like b4400, we should be fixed for the crappy sound 
cards too.

  -- Keir

Re: Xen Kernel (3.0.2) breaks b44 module

[Keir Fraser]

On 12 May 2006, at 08:57, <xen@wood.id.au> wrote:

> I have a new laptop (Dell 9400) that I am trying to work with Xen 3.0.
>
> Xen works fine, runs windows XP under HVM.
>
> But, the Xen kernel on Dom0 has a few problems with some of the 
> hardware.
>
> The network card is a Broadcom 4400 10/100BaseT.

B4400 can apparently DMA only to low 1GB of memory. Xen does not have 
an allocation pool that guarantees to only return memory that low. 
Hence you're stuck for the time being. You could try working around by 
changing B44_DMA_MASK in drivers/net/b44.c to 0x7fffffff. Maybe it'll 
turn out that only certain broken chips have this DMA limitation. Or 
you can hack the DMA pool to only return memory <1GB rather than <2GB 
(as it does currently). That'll be a slightly bigger hack though.

  -- Keir