[Linux-ia64] SW IO TLB buffer management in kernel test10

[Linux-ia64] SW IO TLB buffer management in kernel test10

[Hirofumi Fujita]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1254", Size: 2364 bytes --]

Hi,

there seems to be a problem on buffer management of SW IO TLB.
The contents of io_tlb_list[] become broken.

This happens when a machine has more then 2GB memory,
and CD-ROM or SCSI (>4GB addressing is disabled) are accessed.

But there may be another bug ?

I applied the attached patch and kernel panic when mke2fs /dev/sda4.
(QLogic BIOS setting: >4GB addressing is disabled)
Kernel panic: __pci_map_single: could not allocate software IO TLB (16384 bytes)
At this time, io_tlb_list[] are almost 0.

Same with swiotlb\x1024 option (which extends this bounce buffer to
8192 x 2KB).

With swiotlb92 option, mke2fs successes,
but 60% of io_tlb_list[] remain 0 after mke2fs finished.
This is because pci_unmap is not called correctly ?

Hirofumi Fujita
Hitachi, Ltd.

--- linux-2.4.0-test10-ia64-001101/arch/ia64/kernel/pci-dma.c.org	Mon Nov 13 12:05:13 2000
+++ linux-2.4.0-test10-ia64-001101/arch/ia64/kernel/pci-dma.c	Wed Nov 15 16:19:12 2000
@@ -109,7 +109,8 @@
 {
 	unsigned long flags;
 	char *dma_addr;
-	unsigned int i, nslots, stride, index, wrap;
+	unsigned int nslots, stride, index, wrap;
+	int i;
 
 	/*
 	 * For mappings greater than a page size, we limit the stride (and hence alignment)
@@ -133,7 +134,7 @@
 		wrap = index = ALIGN(io_tlb_index, stride);
 
 		if (index >= io_tlb_nslabs) 
-			index = 0;
+			wrap = index = 0;
 
 		do {
 			/*
@@ -142,8 +143,11 @@
 			 * entries as '0' indicating unavailable.
 			 */
 			if (io_tlb_list[index] >= nslots) {
+				int count = 0;
 				for (i = index; i < index + nslots; i++)
 					io_tlb_list[i] = 0;
+				for (i = index - 1; (i >= 0) && io_tlb_list[i]; i--)
+					io_tlb_list[i] = ++count;
 				dma_addr = io_tlb_start + (index << IO_TLB_SHIFT);
 
 				/*
@@ -210,14 +214,10 @@
 		int count = ((index + nslots) < io_tlb_nslabs ? io_tlb_list[index + nslots] : 0);
 		/*
 		 * Step 1: return the slots to the free list, merging the slots with superceeding slots
-		 */
-		for (i = index + nslots - 1; i >= index; i--)
-			io_tlb_list[i] = ++count;
-		/*
 		 * Step 2: merge the returned slots with the preceeding slots, if available (non zero)
 		 */
-		for (i = index - 1; (i >= 0) && io_tlb_list[i]; i--)
-			io_tlb_list[i] += io_tlb_list[index];
+		for (i = index + nslots - 1; (i >= 0) && io_tlb_list[i]; i--)
+			io_tlb_list[i] = ++count;
 	}
 	spin_unlock_irqrestore(&io_tlb_lock, flags);
 }

Re: [Linux-ia64] SW IO TLB buffer management in kernel test10

[nomura]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="windows-1252", Size: 1966 bytes --]

Hello,

From: Hirofumi Fujita <h-fujita@ebina.hitachi.co.jp>
Subject: [Linux-ia64] SW IO TLB buffer management in kernel test10
Date: Wed, 15 Nov 2000 17:15:52 +0900

> there seems to be a problem on buffer management of SW IO TLB.
> The contents of io_tlb_list[] become broken.
...
> I applied the attached patch and kernel panic when mke2fs /dev/sda4.
> (QLogic BIOS setting: >4GB addressing is disabled)
> Kernel panic: __pci_map_single: could not allocate software IO TLB (16384 bytes)
> At this time, io_tlb_list[] are almost 0.
...
> With swiotlb92 option, mke2fs successes,
> but 60% of io_tlb_list[] remain 0 after mke2fs finished.
> This is because pci_unmap is not called correctly ?

I think it's because your patch makes pci_unmap do nothing on io_tlb_list[].
In __pci_unmap_single, making 2 steps apart has its meaning.

The patch should be something like this?

--- arch/ia64/kernel/pci-dma.c	2000/11/07 06:49:39	1.1.1.4.2.1
+++ arch/ia64/kernel/pci-dma.c	2000/11/15 13:50:28
@@ -109,7 +109,8 @@
 {
 	unsigned long flags;
 	char *dma_addr;
-	unsigned int i, nslots, stride, index, wrap;
+	unsigned int nslots, stride, index, wrap, count;
+	int i;
 
 	/*
 	 * For mappings greater than a page size, we limit the stride (and hence alignment)
@@ -133,7 +134,7 @@
 		wrap = index = ALIGN(io_tlb_index, stride);
 
 		if (index >= io_tlb_nslabs) 
-			index = 0;
+			wrap = index = 0;
 
 		do {
 			/*
@@ -144,6 +145,9 @@
 			if (io_tlb_list[index] >= nslots) {
 				for (i = index; i < index + nslots; i++)
 					io_tlb_list[i] = 0;
+				count = 0;
+				for (i = index - 1; (i >= 0) && io_tlb_list[i]; i--)
+					io_tlb_list[i] = ++count;
 				dma_addr = io_tlb_start + (index << IO_TLB_SHIFT);
 
 				/*

--
NOMURA, Jun'ichi <j-nomura@ce.jp.nec.com, nomura@hpc.bs1.fc.nec.co.jp>
HPC Operating System Group, 1st Computers Software Division,
Computers Software Operations Unit, NEC Solutions.