[BUG] memory mngt > 2 Gbytes and DMA for the Alpha? pci_iommu.c

[BUG] memory mngt > 2 Gbytes and DMA for the Alpha? pci_iommu.c

[Cabaniols, Sebastien]

Hi lkml,

There is likely a bug in the management of memory above two Gigabytes and
DMA in kernel 2.4.4
(up to ac-6) with the alpha. :-(

My hardware is: an Alphaserver ES40 with 4 Cpus and 8 Gigabytes of RAM,
myrinet 2k board, SCSI,
3Com905b ethernet, DE600 ethernet and Qlogic Fiber Channel board.

The Fiber Chanel, the myrinet board and any of the two Ethernet boards all
have troubles working with 
such a big RAM.

They all produce pci_map_sg_failed errors on the logs before totally
crashing the system under load.
(That is not totally right, sometime I don't get anything into the
logs.....)

When I say under load I mean telnet is working with ethernet but massive ftp
is freezing the box.
For the Fiber Channel storage, everything is fine with little files, but
massive dd lead to freezing the box.


BUT

When I boot the system with mem=2048M, everything is back... network,
storage...

and at last, memory crunching C code (allocation,read and write to larger
and larger buffers) 
works fine with the whole RAM.



My understanding is that when the PCI boards want to switch to high
throughtput they go for DMA
and DMA is not working correctly with memory above 2 GBytes. 

The Error message comes from arch/alpha/pci_iommu.c. 

Do you think that makes sense ? What else could I do to help finding the bug
?

Thks for any help

----------------------------------------------------------------------------
--
Sebastien CABANIOLS
COMPAQ France 
HPTC Engineer
CustomSystems & Solutions  Annecy  
High Performance Technical Computing 
Office No.  +33 (0)4 50 09 44 10
Fax No.  +33 (0)4 50 64 01 39 
Email.   sebastien.cabaniols@compaq.com 
----------------------------------------------------------------------------
--

  

Re: [BUG] memory mngt > 2 Gbytes and DMA for the Alpha? pci_iommu.c

[Andrea Arcangeli]

On Wed, May 09, 2001 at 04:12:41PM +0200, Cabaniols, Sebastien wrote:
> Hi lkml,
> 
> There is likely a bug in the management of memory above two Gigabytes and
> DMA in kernel 2.4.4
> (up to ac-6) with the alpha. :-(

remeber why last yaer I was advocating a CONFIG_HIGHMEM option also in
2.4 and not only in 2.2? If we had that now I would tell you "set
HIGHMEM to y until we fix it" and you could use more up to terabyte of
ram in the meantime.

> When I boot the system with mem=2048M, everything is back... network,
> storage...

can you try to set DEBUG_NODIRECT to 1 in pci_iommu.c and then to boot
with mem=2048M, if you can reproduce I should be able to reproduce too.

Andrea