Re: X86_64 + VIA + 4g problems

[Robert Hancock <hancockr@shaw.ca>]

Andi Kleen wrote:
>>This brings up something I've been wondering. It's possible to run most 
>>64-bit capable PCI devices in a 32-bit slot (i.e. with the 64-bit part 
>>hanging out of the slot). In this configuration the device will not be 
>>able to use 64-bit DMA (unless it supports dual address cycle). However, 
>>who is supposed to detect this and know to not try to use DMA addresses 
>>above 4GB on that device? 
> 
> 
> 64bit PCI-X bus has nothing to do with the addressing capability. It
> just gives you more bandwidth.

Indeed, I misunderstood how this worked.. Accesses to addresses >4GB 
must always use DAC addressing, the address can just be decoded faster 
if the extra 64-bit data lines are connected.

I think there could still be an issue if the PCI bus does not support 
DAC and the device tries to use PCI DAC addressing causing the bus to 
choke or the wrong RAM to get accessed. Apparently this is a real 
problem according to MS:

http://www.microsoft.com/whdc/system/platform/server/PAE/PAEdrv.mspx

"Unfortunately, Microsoft is finding that not all PCI buses on a system 
board support DAC, which is required for a 32-bit PCI adapter to address 
more than 4 GB of memory. Furthermore, there is no way for a DAC-capable 
PCI device (or its associated driver) to know that it is running on a 
non-DAC-capable bus."

Microsoft's solution is to disable all memory above 4GB if 
non-DAC-capable buses are present - largely due to their API limitations 
and the number of broken drivers that don't call proper DMA functions. 
Haha to them, I guess..

I'm not sure if this would be an issue with Linux. I would suspect that 
it could be, if there are indeed such buses that can't support DAC.. In 
this case the right thing to do would be to reject requests for DMA 
masks larger than 4GB for devices located on such a bus.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/