Re: What's the SB600 64-bit DMA problem?

["George Spelvin" <linux@horizon.com>]

Tejun Heo <htejun@gmail.com> wrote:

> George Spelvin wrote:
>> Tejun Heo <htejun@gmail.com> wrote:
>>>> Q: Does the fact that I'm using the GART IOMMU mean that I'm not really using
>>>> using 64-bit DMA at all?
>>> Can you turn it off and see what happens?
>> 
>> I can if I can verify that every other bus-master PCI device in the
>> system can do 64-bit DMA.
>> 
>> Do you know how to check that?  I haven't found a file under /sys or
>> /proc that tells me what the DMA masks are of various system
>> drivers, but that doesn't mean that it doesn't exist.
>
> It doesn't really matter.  The kernel will bounce buffers as
> necessary.

Er, according to the docs I can find, that's iommu=soft, and still
doesn't guarantee 64-bit DMA.  I was thinking about using iommu=off,
which disables everything including bounce buffers, and thereby forces
64-bit DMA.

Um... actually, I just tried it, and when there's >4G of memory, the kernel
forces iommu=soft, and it works.  At least as far as e2fsck -n on every
file system (including a 1.7 TB RAID-5) can tell.

It still doesn't definitively tell me that a DMA to and from an address
>4G was done, but it's an awfully strong hint.  I was thinking of
inserting a debug message that logs every time the "highest DMA address
used" increases, than I can watch it to see for sure.

ATM I'm just recompiling the kernel from -rc6 to -rc7.

It's in single-user mode so I can't ssh over the boot messages, but
they definitely included "64bit" from ahci_print_info().

Speaking of anci_print_info(), sould something like the following be a good
thing?

diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 2e1a7cb..18137d2 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -2464,22 +2464,22 @@ static void ahci_print_info(struct ata_host *host)
 		"%s\n"
 		,
 
-		cap & (1 << 31) ? "64bit " : "",
-		cap & (1 << 30) ? "ncq " : "",
-		cap & (1 << 29) ? "sntf " : "",
-		cap & (1 << 28) ? "ilck " : "",
-		cap & (1 << 27) ? "stag " : "",
-		cap & (1 << 26) ? "pm " : "",
-		cap & (1 << 25) ? "led " : "",
-
-		cap & (1 << 24) ? "clo " : "",
-		cap & (1 << 19) ? "nz " : "",
-		cap & (1 << 18) ? "only " : "",
-		cap & (1 << 17) ? "pmp " : "",
-		cap & (1 << 15) ? "pio " : "",
-		cap & (1 << 14) ? "slum " : "",
-		cap & (1 << 13) ? "part " : "",
-		cap & (1 << 6) ? "ems ": ""
+		cap & HOST_CAP_64   ? "64bit " : "",
+		cap & HOST_CAP_NCQ  ? "ncq " : "",
+		cap & HOST_CAP_SNTF ? "sntf " : "",
+		cap & (1 << 28)     ? "ilck " : "",
+		cap & HOST_CAP_SSS  ? "stag " : "",
+		cap & HOST_CAP_ALPM ? "pm " : "",
+		cap & (1 << 25)     ? "led " : "",
+
+		cap & HOST_CAP_CLO  ? "clo " : "",
+		cap & (1 << 19)     ? "nz " : "",
+		cap & (1 << 18)     ? "only " : "",
+		cap & HOST_CAP_PMP  ? "pmp " : "",
+		cap & (1 << 15)     ? "pio " : "",
+		cap & HOST_CAP_SSC  ? "slum " : "",
+		cap & (1 << 13)     ? "part " : "",
+		cap & HOST_CAP_EMS  ? "ems ": ""
 		);
 }