Re: Fusion problems with 2.6.5rc1 on AMD x86-64 III

Re: Fusion problems with 2.6.5rc1 on AMD x86-64 III

[Andi Kleen]

On Wed, 17 Mar 2004 06:03:32 +0100
Andi Kleen <ak@suse.de> wrote:


> The actual problem is the recent consistent_dma_mask change from James. For some reason
> the x86-64 pci_alloc_consistent always sees an 0xffffffffffffffff dma mask now
> and that breaks Fusion.

I found some bugs in x86-64 code now that explained the problem. For some reason
the consistent mask patchkit seems to have triggered them. It's fixed now.

-Andi

Re: Fusion problems with 2.6.5rc1 on AMD x86-64 IV

[Andi Kleen]

On Wed, 17 Mar 2004 10:00:40 +0100
Andi Kleen <ak@suse.de> wrote:

> On Wed, 17 Mar 2004 06:03:32 +0100
> Andi Kleen <ak@suse.de> wrote:
> 
> 
> > The actual problem is the recent consistent_dma_mask change from James. For some reason
> > the x86-64 pci_alloc_consistent always sees an 0xffffffffffffffff dma mask now
> > and that breaks Fusion.
> 
> I found some bugs in x86-64 code now that explained the problem. For some reason
> the consistent mask patchkit seems to have triggered them. It's fixed now.

Sorry for the many follow ups to myself :-)

Anyways the consistent mask problem is still with us. I just have a workaround that 
works (force it to 4GB) and there were related bugs in the x86-64 IOMMU code.

James, can you take a look at that please? From a first look i didn't find the place
where it is set to 0xffffffffffffffff. It isn't the driver.

-Andi

RE: Fusion problems with 2.6.5rc1 on AMD x86-64 IV

[Moore, Eric Dean]

There is a bug in the driver where we do three
seperate memory allocations for the request/reply FIFOs 
that need to be on the same 4gb boundary. This is on my todo list,
hope to fix soon.


A temporary fix is there in the driver. Please recompile
with the following line uncommented in the Makefile
will put these allocations below 4gb:

EXTRA_CFLAGS += -DMPTBASE_MEM_ALLOC_FIFO_FIX

Eric Moore


On Wednesday, March 17, 2004 2:07 AM,  Andi Kleen wrote:
> 
> On Wed, 17 Mar 2004 10:00:40 +0100
> Andi Kleen <ak@suse.de> wrote:
> 
> > On Wed, 17 Mar 2004 06:03:32 +0100
> > Andi Kleen <ak@suse.de> wrote:
> > 
> > 
> > > The actual problem is the recent consistent_dma_mask 
> change from James. For some reason
> > > the x86-64 pci_alloc_consistent always sees an 
> 0xffffffffffffffff dma mask now
> > > and that breaks Fusion.
> > 
> > I found some bugs in x86-64 code now that explained the 
> problem. For some reason
> > the consistent mask patchkit seems to have triggered them. 
> It's fixed now.
> 
> Sorry for the many follow ups to myself :-)
> 
> Anyways the consistent mask problem is still with us. I just 
> have a workaround that 
> works (force it to 4GB) and there were related bugs in the 
> x86-64 IOMMU code.
> 
> James, can you take a look at that please? From a first look 
> i didn't find the place
> where it is set to 0xffffffffffffffff. It isn't the driver.
> 
> -Andi
> 

Re: Fusion problems with 2.6.5rc1 on AMD x86-64 IV

[Andi Kleen]

On Wed, 17 Mar 2004 10:58:09 -0500
"Moore, Eric Dean" <Emoore@lsil.com> wrote:

> There is a bug in the driver where we do three
> seperate memory allocations for the request/reply FIFOs 
> that need to be on the same 4gb boundary. This is on my todo list,
> hope to fix soon.

The problem seems to be the broke consistent dma mask.

I currently just mask it with 0xffffffff in pci_alloc_consistent
and why that workaround it works.

-Andi

Re: Fusion problems with 2.6.5rc1 on AMD x86-64 IV

[James Bottomley]

> The problem seems to be the broke consistent dma mask.
> 
> I currently just mask it with 0xffffffff in pci_alloc_consistent
> and why that workaround it works.

Well, I don't understand this.

The place it's setup is in pci/probe.c:

--- 1.60/drivers/pci/probe.c    Wed Feb 18 17:41:09 2004
+++ 1.61/drivers/pci/probe.c    Sun Mar 14 14:17:06 2004
@@ -570,7 +570,6 @@
        /* Assume 32-bit PCI; let 64-bit PCI cards (which are far rarer)
           set this higher, assuming the system even supports it.  */
        dev->dma_mask = 0xffffffff;
-       dev->consistent_dma_mask = 0xffffffff;
        if (pci_setup_device(dev) < 0) {
                kfree(dev);
                return NULL;
@@ -582,6 +581,7 @@
        pci_name_device(dev);
 
        dev->dev.dma_mask = &dev->dma_mask;
+       dev->dev.coherent_dma_mask = 0xffffffffull;
 
        return dev;

The only reason I can see having a screw up is if some untoward
manipulation is going on in pci_setup_device().  This shouldn't be
because the pci->dev isn't initialised until after that (which is why
the coherent_dma_mask initialisation had to be moved further back).

You can verify this by moving the device_initialize() and the
coherent_dma_mask assignment above pci_setup_device.

James