scsi_debug issues

scsi_debug issues

[Nishanth Aravamudan]

Hi,

At the recommendation of Pat Mansfield, I'm posting this issue to
linux-scsi. I have run into a big problem with the scsi_debug driver
which is causing my machine to hang.

I was finally able to get a large number (10000) of scsi_debug disks via
modprobe (no partitioning, no mounting, no fs), so I decided to go ahead
and try to continue the experiment. I found that I was not able to ls in
a mounted directory from one of the scsi_debug disks. In the chance that
it was somehow due to the high number of disks present, I tried just

  modprobe scsi_debug

so that I would only get one disk. I then ran these commands [0]. I did
not set the SCSI logging on until the last moment, after I synced the
existent SCSI disks in the machine (there are two actual SCSI disks)
[1]. I found that the ls command never fails to hang (although sometimes
it will not hang immediately, I have to

  cd lost+found

first. Interestingly, if the machine does not have highmem (i.e. less
than 896 MB of RAM) or the kernel is booted with mem=800 (or some other
number less than 896), the problem dissappears. I had to put the check
in my patch in for the NULL dereference, as it would cause a
segmentation fault otherwise. I'm guessing that is where the problem may
be, as the NULL-check was not there before (an assumption that it
(scatg2virt) would always work?).

I'm more than happy to run further tests, post further output, apply
patches, etc. or anything else to resolve this bug. Please CC me on
reply, as I'm not subscribed to linux-scsi, thanks.

-Nish

[0] commands run to produce hang:

/etc/init.d/klogd stop
/etc/init.d/sysklogd stop
modprobe scsi_debug
fdisk /dev/sdc (primary partition, default values selected)
mkfs /dev/sdc1
mount /dev/sdc1 /home/test1
cd /home/test1
sync
echo 0xfffffff > /proc/sys/dev/scsi/logging_level
ls

[1] hw/sw description

2-way 700MHz with 2GB RAM
2.6.9-rc3 kernel with patch[3] applied

[2] output of serial console from modprobe onwards:

scsi2 : scsi_debug, version 1.73 [20040518], dev_size_mb=8, opts=0x0
 Vendor: Linux     Model: scsi_debug        Rev: 0004
 Type:   Direct-Access                      ANSI SCSI revision: 03
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:scatg2virt(&sgpnt[0] [==c2268b60]) returned f7531000
 unknown partition table
Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg3 at scsi2, channel 0, id 0, lun 0,  type 0
scatg2virt(&sgpnt[0] [==c23d6b80]) returned f7532000
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:scatg2virt(&sgpnt[0] [==c2268960]) returned f767c000
 sdc1
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:scatg2virt(&sgpnt[0] [==f76efd60]) returned f7515000
 sdc1
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7667000
scatg2virt(&sgpnt[0] [==c23d6b80]) returned f767d000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f76c6000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f74bb000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f74b7000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7ec9000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f75eb000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7452000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f74cd000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7ff8000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7e72000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f766d000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f74e4000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7689000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f76ab000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7586000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7632000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7554000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f745f000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7408000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f74ba000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7589000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7400000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f76b4000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f76b6000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f765c000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f765e000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f765f000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7674000
scatg2virt(&sgpnt[0] [==f76efe60]) returned f7675000
scatg2virt(&sgpnt[0] [==c23d6a80]) returned f7516000
scatg2virt(&sgpnt[0] [==c2268960]) returned f751b400
scatg2virt(&sgpnt[0] [==c2268960]) returned f751b800
scatg2virt(&sgpnt[0] [==c2268960]) returned f7661400
sd_init_command: disk=sdc, block=554, count=2
sdc : block=554
sdc : reading 2/2 512 byte blocks.
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 SUCCESS    70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 70000
sd_rw_intr: sdc: res=0x70000
sd_rw_intr: sb[0,2,asc,ascq]=0,0,0,0
2 sectors total, 0 bytes done.
use_sg is 1
SCSI error : <2 0 0 0> return code = 0x70000
end_request: I/O error, dev sdc, sector 554
sd_init_command: disk=sdc, block=554, count=2
sdc : block=554
sdc : reading 2/2 512 byte blocks.
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 RETRY      70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Inserting command f76e2e00 into mlqueue
scsi_delete_timer: scmd: f76e2e00, rtn: 0
scsi_add_timer: scmd: f76e2e00, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2e00                  scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
buffer = 0xf76efd60, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
scatg2virt(&sgpnt[0] [==f76efd60]) returned NULL
Invalid memory access in resp_read...
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2e00, rtn: 1
scsi <2:0:0:0> done 0xf76e2e00 SUCCESS    70000 scsi2 : destination target 0, lun 0
        command = 0x28 00 00 00 02 2a 00 00 02 00 
scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 70000
sd_rw_intr: sdc: res=0x70000
sd_rw_intr: sb[0,2,asc,ascq]=0,0,0,0
2 sectors total, 0 bytes done.
use_sg is 1
SCSI error : <2 0 0 0> return code = 0x70000
end_request: I/O error, dev sdc, sector 554
sd_init_command: disk=sdc, block=34, count=2
sdc : block=34
sdc : writing 2/2 512 byte blocks.
scsi_add_timer: scmd: f76e2ca0, time: 30000, (c023dc50)
scsi <2:0:0:0> send 0xf76e2ca0                  scsi2 : destination target 0, lun 0
        command = 0x2a 00 00 00 00 22 00 00 02 00 
buffer = 0xf76ef060, bufflen = 1024, done = 0xc0264770, queuecommand 0xf88c4030
sd_init_command: disk=sda, block=7575, count=80
sda : block=7575
sda : writing 80/80 512 byte blocks.
scsi_add_timer: scmd: f76e2880, time: 30000, (c023dc50)
scsi <0:0:0:0> send 0xf76e2880                  scsi0 : destination target 0, lun 0
        command = 0x2a 00 00 00 1d 97 00 00 50 00 
buffer = 0xc23d6180, bufflen = 40960, done = 0xc0264770, queuecommand 0xc025b350
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2880, rtn: 1
scsi <0:0:0:0> done 0xf76e2880 SUCCESS        0 scsi0 : destination target 0, lun 0
        command = 0x2a 00 00 00 1d 97 00 00 50 00 
scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 0
sd_rw_intr: sda: res=0x0
80 sectors total, 40960 bytes done.
use_sg is 9
sd_init_command: disk=sda, block=7655, count=8
sda : block=7655
sda : writing 8/8 512 byte blocks.
scsi_add_timer: scmd: f76e2880, time: 30000, (c023dc50)
scsi <0:0:0:0> send 0xf76e2880                  scsi0 : destination target 0, lun 0
        command = 0x2a 00 00 00 1d e7 00 00 08 00 
buffer = 0xc2268ce0, bufflen = 4096, done = 0xc0264770, queuecommand 0xc025b350
leaving scsi_dispatch_cmnd()
scsi_delete_timer: scmd: f76e2880, rtn: 1
scsi <0:0:0:0> done 0xf76e2880 SUCCESS        0 scsi0 : destination target 0, lun 0
        command = 0x2a 00 00 00 1d e7 00 00 08 00 
scsi host busy 1 failed 0
Notifying upper driver of completion for device 0 0
sd_rw_intr: sda: res=0x0
8 sectors total, 4096 bytes done.
use_sg is 1

[3] patch applied:

--- 2.6.9-rc3-vanilla/drivers/scsi/scsi_debug.c	2004-10-15 11:09:50.000000000 -0700
+++ 2.6.9-rc3/drivers/scsi/scsi_debug.c	2004-10-15 11:18:42.000000000 -0700
@@ -423,6 +423,10 @@ int scsi_debug_queuecommand(struct scsi_
 			num = cmd[4];
 		}
 		errsts = resp_read(SCpnt, upper_blk, block, num, devip);
+		if (errsts == (DID_ERROR << 16)) {
+			printk ("Invalid memory access in resp_read...\n");
+			break;
+		}
 		if (inj_recovered && (0 == errsts)) {
 			mk_sense_buffer(devip, RECOVERED_ERROR,
 					THRESHHOLD_EXCEEDED, 0, 18);
@@ -801,6 +805,12 @@ static int resp_read(struct scsi_cmnd * 
 		sgcount = 0;
 		sgpnt = (struct scatterlist *) buff;
 		buff = scatg2virt(&sgpnt[sgcount]);
+		printk("scatg2virt(&sgpnt[0] [==%p]) returned ", &sgpnt[sgcount]);
+		if (buff == NULL) {
+			printk("NULL\n");
+			return (DID_ERROR << 16);
+		}
+		printk("%p\n", buff);
 		bufflen = sgpnt[sgcount].length;
 	}
 	do {
@@ -1243,6 +1253,10 @@ static int schedule_resp(struct scsi_cmn
 			printk(KERN_INFO "scsi_debug: ... <%u %u %u %u> "
 			       "non-zero result=0x%x\n", sdp->host->host_no,
 			       sdp->channel, sdp->id, sdp->lun, scsi_result);
+			if (scsi_result == (DID_ERROR << 16)) {
+				printk("scsi_debug: WARNING: DID_ERROR "
+						"returned!\n");
+			}
 		}
 	}
 	if (cmnd && devip) {

Re: scsi_debug issues

[Douglas Gilbert]

[-- Attachment #1: Type: text/plain, Size: 1951 bytes --]

Nishanth Aravamudan wrote:
> Hi,
> 
> At the recommendation of Pat Mansfield, I'm posting this issue to
> linux-scsi. I have run into a big problem with the scsi_debug driver
> which is causing my machine to hang.
> 
> I was finally able to get a large number (10000) of scsi_debug disks via
> modprobe (no partitioning, no mounting, no fs), so I decided to go ahead
> and try to continue the experiment. I found that I was not able to ls in
> a mounted directory from one of the scsi_debug disks. In the chance that
> it was somehow due to the high number of disks present, I tried just
> 
>   modprobe scsi_debug
> 
> so that I would only get one disk. I then ran these commands [0]. I did
> not set the SCSI logging on until the last moment, after I synced the
> existent SCSI disks in the machine (there are two actual SCSI disks)
> [1]. I found that the ls command never fails to hang (although sometimes
> it will not hang immediately, I have to
> 
>   cd lost+found
> 
> first. Interestingly, if the machine does not have highmem (i.e. less
> than 896 MB of RAM) or the kernel is booted with mem=800 (or some other
> number less than 896), the problem dissappears. I had to put the check
> in my patch in for the NULL dereference, as it would cause a
> segmentation fault otherwise. I'm guessing that is where the problem may
> be, as the NULL-check was not there before (an assumption that it
> (scatg2virt) would always work?).

Nishanth,
So this problem seems related to highmem.

The attachment is against the current scsi_debug driver
(at least in lk 2.6.9-rc4). It uses dma_map_sg() and friends
together with phys_to_virt() which comes highly recommended:
"in almost all conceivable cases a device driver should not be
using this function" :-) Could you try the patch.

Perhaps others my be able to answer this: is setting
scsi_host_template::dma_boundary (to some figure other
than 0xffffffff) going to help in this case?

Doug Gilbert


[-- Attachment #2: sdebug2681dma1.diff --]
[-- Type: text/x-patch, Size: 4095 bytes --]

--- linux/drivers/scsi/scsi_debug.c	2004-08-14 21:12:43.000000000 +1000
+++ linux/drivers/scsi/scsi_debug.c2681dma1	2004-10-16 16:22:34.330229976 +1000
@@ -262,16 +262,6 @@
 static struct device pseudo_primary;
 static struct bus_type pseudo_lld_bus;
 
-static unsigned char * scatg2virt(const struct scatterlist * sclp)
-{
-	if (NULL == sclp)
-		return NULL;
-	else if (sclp->page)
-		return (unsigned char *)page_address(sclp->page) +
-		       sclp->offset;
-	else
-		return NULL;
-}
 
 static
 int scsi_debug_queuecommand(struct scsi_cmnd * SCpnt, done_funct_t done)
@@ -286,6 +276,7 @@
 	struct sdebug_dev_info * devip = NULL;
 	unsigned char * sbuff;
 	int inj_recovered = 0;
+        int num_elems = 0;
 
 	if (done == NULL)
 		return 0;	/* assume mid level reprocessing command */
@@ -294,8 +285,12 @@
 		struct scatterlist *sgpnt = (struct scatterlist *)
 						SCpnt->request_buffer;
 
-		buff = scatg2virt(&sgpnt[0]);
-		bufflen = sgpnt[0].length;
+		num_elems = dma_map_sg(NULL, sgpnt, SCpnt->use_sg,
+				       SCpnt->sc_data_direction);
+		buff = phys_to_virt(sg_dma_address(&sgpnt[0]));
+		bufflen = sg_dma_len(&sgpnt[0]);
+		dma_unmap_sg(NULL, sgpnt, SCpnt->use_sg,
+			     SCpnt->sc_data_direction);
 		/* READ and WRITE process scatterlist themselves */
 	}
 	else
@@ -778,6 +773,7 @@
         struct scatterlist *sgpnt = NULL;
         int bufflen = SCpnt->request_bufflen;
 	unsigned long iflags;
+        int num_elems = 0;
 
 	if (upper_blk || (block + num > sdebug_capacity)) {
 		mk_sense_buffer(devip, ILLEGAL_REQUEST, ADDR_OUT_OF_RANGE,
@@ -799,9 +795,11 @@
 	       block, bufflen); */
 	if (SCpnt->use_sg) {
 		sgcount = 0;
-		sgpnt = (struct scatterlist *) buff;
-		buff = scatg2virt(&sgpnt[sgcount]);
-		bufflen = sgpnt[sgcount].length;
+		sgpnt = (struct scatterlist *)buff;
+		num_elems = dma_map_sg(NULL, sgpnt, SCpnt->use_sg,
+				       SCpnt->sc_data_direction);
+		buff = phys_to_virt(sg_dma_address(&sgpnt[0]));
+		bufflen = sg_dma_len(&sgpnt[0]);
 	}
 	do {
 		memcpy(buff, fake_storep + (block * SECT_SIZE), bufflen);
@@ -809,14 +807,18 @@
 		if (SCpnt->use_sg) {
 			block += bufflen >> POW2_SECT_SIZE;
 			sgcount++;
-			if (nbytes) {
-				buff = scatg2virt(&sgpnt[sgcount]);
-				bufflen = sgpnt[sgcount].length;
+			if (nbytes > 0) {
+				buff = phys_to_virt(
+					sg_dma_address(&sgpnt[sgcount]));
+				bufflen = sg_dma_len(&sgpnt[sgcount]);
 			}
 		} else if (nbytes > 0)
 			printk(KERN_WARNING "scsi_debug:resp_read: unexpected "
 			       "nbytes=%d\n", nbytes);
-	} while (nbytes);
+	} while (nbytes > 0);
+	if (SCpnt->use_sg)
+		dma_unmap_sg(NULL, sgpnt, SCpnt->use_sg,
+			     SCpnt->sc_data_direction);
 	read_unlock_irqrestore(&atomic_rw, iflags);
 	return 0;
 }
@@ -829,6 +831,7 @@
         struct scatterlist *sgpnt = NULL;
         int bufflen = SCpnt->request_bufflen;
 	unsigned long iflags;
+        int num_elems = 0;
 
 	if (upper_blk || (block + num > sdebug_capacity)) {
 		mk_sense_buffer(devip, ILLEGAL_REQUEST, ADDR_OUT_OF_RANGE,
@@ -842,8 +845,10 @@
 	if (SCpnt->use_sg) {
 		sgcount = 0;
 		sgpnt = (struct scatterlist *) buff;
-		buff = scatg2virt(&sgpnt[sgcount]);
-		bufflen = sgpnt[sgcount].length;
+		num_elems = dma_map_sg(NULL, sgpnt, SCpnt->use_sg,
+				       SCpnt->sc_data_direction);
+		buff = phys_to_virt(sg_dma_address(&sgpnt[0]));
+		bufflen = sg_dma_len(&sgpnt[0]);
 	}
 	do {
 		memcpy(fake_storep + (block * SECT_SIZE), buff, bufflen);
@@ -852,14 +857,18 @@
 		if (SCpnt->use_sg) {
 			block += bufflen >> POW2_SECT_SIZE;
 			sgcount++;
-			if (nbytes) {
-				buff = scatg2virt(&sgpnt[sgcount]);
-				bufflen = sgpnt[sgcount].length;
+			if (nbytes > 0) {
+				buff = phys_to_virt(
+					sg_dma_address(&sgpnt[sgcount]));
+				bufflen = sg_dma_len(&sgpnt[sgcount]);
 			}
 		} else if (nbytes > 0)
 			printk(KERN_WARNING "scsi_debug:resp_write: "
 			       "unexpected nbytes=%d\n", nbytes);
-	} while (nbytes);
+	} while (nbytes > 0);
+	if (SCpnt->use_sg)
+		dma_unmap_sg(NULL, sgpnt, SCpnt->use_sg,
+			     SCpnt->sc_data_direction);
 	write_unlock_irqrestore(&atomic_rw, iflags);
 	return 0;
 }

Re: scsi_debug issues

[Christoph Hellwig]

A dma address is not a physical address.  You need to kmap the virtual
addresses in the sg lists to get a kernel virtual address. 

Re: scsi_debug issues

[James Bottomley]

On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> So this problem seems related to highmem.
> 
> The attachment is against the current scsi_debug driver
> (at least in lk 2.6.9-rc4). It uses dma_map_sg() and friends
> together with phys_to_virt() which comes highly recommended:
> "in almost all conceivable cases a device driver should not be
> using this function" :-) Could you try the patch.
> 
> Perhaps others my be able to answer this: is setting
> scsi_host_template::dma_boundary (to some figure other
> than 0xffffffff) going to help in this case?

No, the problem is with highmem as you correctly point out, but the
issue is that your driver cannot see beyond it.  On x86, highmem begins
at about 900Mb.  The kernel has no page tables for any memory beyond
this.  In order to see the memory you need to kmap it (i.e. ask the
kernel to create a temporary page table for it); so the virtual address
sg uses should be got by kmapping the pages in the sg list:

kaddr = kmap(sg[i].page) + sg[i].offset

If you make sure clustering is disabled, you should never get multiple
pages in the initial sg list.

When you've finished you need to release the mapping with kunmap().

There are also atomic versions of these depending on where you are.

James

[PATCH] scsi_debug [was: scsi_debug issues]

[Douglas Gilbert]

[-- Attachment #1: Type: text/plain, Size: 2470 bytes --]

James Bottomley wrote:
> On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> 
>>So this problem seems related to highmem.
>>
>>The attachment is against the current scsi_debug driver
>>(at least in lk 2.6.9-rc4). It uses dma_map_sg() and friends
>>together with phys_to_virt() which comes highly recommended:
>>"in almost all conceivable cases a device driver should not be
>>using this function" :-) Could you try the patch.
>>
>>Perhaps others my be able to answer this: is setting
>>scsi_host_template::dma_boundary (to some figure other
>>than 0xffffffff) going to help in this case?
> 
> 
> No, the problem is with highmem as you correctly point out, but the
> issue is that your driver cannot see beyond it.  On x86, highmem begins
> at about 900Mb.  The kernel has no page tables for any memory beyond
> this.  In order to see the memory you need to kmap it (i.e. ask the
> kernel to create a temporary page table for it); so the virtual address
> sg uses should be got by kmapping the pages in the sg list:
> 
> kaddr = kmap(sg[i].page) + sg[i].offset
> 
> If you make sure clustering is disabled, you should never get multiple
> pages in the initial sg list.
done

> When you've finished you need to release the mapping with kunmap().
ouch, that requires a re-org

> There are also atomic versions of these depending on where you are.
hopefully not required


The above required a fair few changes to scsi_debug.
Attached is a patch that rolls "kmap" changes with
patches I have sent recently for scsi_debug.
Attachment is gzipped (due to size) and applies against
lk 2.6.8.1 -> lk 2.6.9-rc4 .

Nishanth, could you test this with highmem?

Changelog:
    - use kmap/kunmap to handle normal and highmem cases
        - disable clustering
        - previously could only handle 'use_sg > 1' for
          read and write commands
    - VERIFY (SBC) and REWIND (SSC) command support (dummies)
    - 'dsense' parameter (sysfs + load time) for descriptor
      sense format
    - allow negative 'every_nth' to fail continually after
      |every_nth| commands (or until sysfs intervention)
    - clean up debug messages sent to the log (when opts=1)
        - correct ordering of log messages
    - set scsi_cmnd::resid when underflow on DMA_FROM_DEVICE
      operations
    - clean up std inquiry response; add version descriptors
    - reject mode sense subpage commands since no subpages
      currently supported (i.e. better command filtering)


Doug Gilbert


[-- Attachment #2: sdebug269rc4_174.diff.gz --]
[-- Type: application/x-gzip, Size: 8135 bytes --]

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Nishanth Aravamudan]

[-- Attachment #1: Type: text/plain, Size: 11468 bytes --]

On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
> James Bottomley wrote:
> >On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> >
> >>So this problem seems related to highmem.

<snip>

> The above required a fair few changes to scsi_debug.
> Attached is a patch that rolls "kmap" changes with
> patches I have sent recently for scsi_debug.
> Attachment is gzipped (due to size) and applies against
> lk 2.6.8.1 -> lk 2.6.9-rc4 .
> 
> Nishanth, could you test this with highmem?

Doug,

I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
occur (complaining about sleeping in an invalid context) and a final
panic during mkfs :) I think something is still wrong... ;) Output is
below and attached.

-Nish

scsi2 : scsi_debug, version 1.74 [20041018], dev_size_mb=8, opts=0x0
  Vendor: Linux     Model: scsi_debug        Rev: 0004
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a10d>] register_disk+0xdd/0x190
 [<c02255d0>] blk_register_region+0x40/0x50
 [<c022568c>] add_disk+0x4c/0x60
 [<c0225610>] exact_match+0x0/0x10
 [<c0225620>] exact_lock+0x0/0x20
 [<c0265e0f>] sd_probe+0x24f/0x3a0
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<c0243e52>] scsi_sysfs_add_sdev+0x52/0x200
 [<f88c5880>] scsi_debug_slave_configure+0x0/0xa0 [scsi_debug]
 [<c0242674>] scsi_add_lun+0x2f4/0x3b0
 [<c0242809>] scsi_probe_and_add_lun+0xd9/0x240
 [<c02430b8>] scsi_scan_target+0xa8/0x130
 [<c02431c9>] scsi_scan_channel+0x89/0xa0
 [<c02432e1>] scsi_scan_host_selected+0x101/0x140
 [<c0243350>] scsi_scan_host+0x30/0x40
 [<f88c6d8d>] sdebug_driver_probe+0x9d/0xd0 [scsi_debug]
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<f88c6bbf>] sdebug_add_adapter+0x12f/0x1f0 [scsi_debug]
 [<f88cf36c>] scsi_debug_init+0x16c/0x1d2 [scsi_debug]
 [<c0132f59>] sys_init_module+0x199/0x230
 [<c01060cf>] syscall_call+0x7/0xb
 unknown partition table
Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg3 at scsi2, channel 0, id 0, lun 0,  type 0
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c48ab>] fetch_to_dev_buffer+0x9b/0x140 [scsi_debug]
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221da0>] blk_unplug_work+0x10/0x20
 [<c012ab79>] worker_thread+0x1b9/0x260
 [<c0221d90>] blk_unplug_work+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c012a9c0>] worker_thread+0x0/0x260
 [<c012edba>] kthread+0xba/0xc0
 [<c012ed00>] kthread+0x0/0xc0
 [<c0104145>] kernel_thread_helper+0x5/0x10
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c015527f>] sync_buffer+0x3f/0x50
 [<c0155408>] __wait_on_buffer+0xa8/0xb0
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0157706>] __block_prepare_write+0x156/0x440
 [<c01ef2cf>] radix_tree_node_alloc+0x1f/0x70
 [<c0158224>] block_prepare_write+0x34/0x50
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c013675f>] generic_file_buffered_write+0x1cf/0x600
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c016f540>] inode_update_time+0xd0/0xe0
 [<c0136e1a>] generic_file_aio_write_nolock+0x28a/0x4c0
 [<c01370f3>] generic_file_write_nolock+0xa3/0xc0
 [<c011328c>] do_page_fault+0x1cc/0x5f1
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<c015d038>] blkdev_file_write+0x38/0x40
 [<c01540d8>] vfs_write+0xb8/0x130
 [<c0154221>] sys_write+0x51/0x80
 [<c01060cf>] syscall_call+0x7/0xb
------------[ cut here ]------------
kernel BUG at arch/i386/mm/highmem.c:14!
invalid operand: 0000 [#1]
SMP 
Modules linked in: scsi_debug
CPU:    1
EIP:    0060:[<c0114784>]    Not tainted VLI
EFLAGS: 00010006   (2.6.9-rc4) 
EIP is at kunmap+0x14/0x30
eax: f7f82000   ebx: 00001000   ecx: 00000000   edx: c16eeb00
esi: f7759000   edi: f93ea000   ebp: f75e1c00   esp: f7f83bf4
ds: 007b   es: 007b   ss: 0068
Process mkfs.ext2 (pid: 1573, threadinfo=f7f82000 task=f7694150)
Stack: f88c4906 c16eeb00 f7f83bfc f7f83bfc 00000000 00000000 00000000 00000006 
       00000000 f93e9000 0001c000 f88c563d f7e015c0 f93e9000 0001c000 00004000 
       00000000 000000e0 f7e01614 00000000 f88c44a0 f7e015c0 00000000 00003f20 
Call Trace:
 [<f88c4906>] fetch_to_dev_buffer+0xf6/0x140 [scsi_debug]
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c023b2b3>] scsi_put_command+0x73/0xa0
 [<c0221e7d>] blk_run_queue+0x2d/0x50
 [<c02405fd>] scsi_end_request+0xdd/0xf0
 [<c024092c>] scsi_io_completion+0x15c/0x4d0
 [<c013d970>] cache_alloc_refill+0x150/0x230
 [<c0264ab8>] sd_rw_intr+0x88/0x310
 [<c023deba>] scsi_delete_timer+0x1a/0x70
 [<c023bd11>] scsi_finish_command+0x81/0xe0
 [<c023bc0e>] scsi_softirq+0xbe/0xf0
 [<c011f6ba>] __do_softirq+0xba/0xd0
 [<c011f6fd>] do_softirq+0x2d/0x30
 [<c01106fd>] smp_apic_timer_interrupt+0x8d/0x100
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c0106abe>] apic_timer_interrupt+0x1a/0x20
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c023b877>] scsi_dispatch_cmd+0x187/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c0134875>] wait_on_page_bit+0xe5/0xf0
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c013437c>] wait_on_page_writeback_range+0x10c/0x150
 [<c0134541>] filemap_fdatawait+0x71/0x80
 [<c0155599>] sync_blockdev+0x39/0x50
 [<c0155aa7>] sys_fsync+0xb7/0xf0
 [<c01060cf>] syscall_call+0x7/0xb
Code: d8 8b 5c 24 08 83 c4 0c e9 6a d5 02 00 8d 76 00 8d bc 27 00 00 00 00 b8 00 e0 ff ff 8b 54 24 04 21 e0 f7 40 14 00 ff ff 00 74 08 <0f> 0b 0e 00 82 d4 30 c0 3b 15 34 cb 44 c0 73 01 c3 89 d0 e9 e4 
 <0>Kernel panic - not syncing: Fatal exception in interrupt

[-- Attachment #2: output.file --]
[-- Type: text/plain, Size: 10726 bytes --]

scsi2 : scsi_debug, version 1.74 [20041018], dev_size_mb=8, opts=0x0
  Vendor: Linux     Model: scsi_debug        Rev: 0004
  Type:   Direct-Access                      ANSI SCSI revision: 05
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a10d>] register_disk+0xdd/0x190
 [<c02255d0>] blk_register_region+0x40/0x50
 [<c022568c>] add_disk+0x4c/0x60
 [<c0225610>] exact_match+0x0/0x10
 [<c0225620>] exact_lock+0x0/0x20
 [<c0265e0f>] sd_probe+0x24f/0x3a0
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<c0243e52>] scsi_sysfs_add_sdev+0x52/0x200
 [<f88c5880>] scsi_debug_slave_configure+0x0/0xa0 [scsi_debug]
 [<c0242674>] scsi_add_lun+0x2f4/0x3b0
 [<c0242809>] scsi_probe_and_add_lun+0xd9/0x240
 [<c02430b8>] scsi_scan_target+0xa8/0x130
 [<c02431c9>] scsi_scan_channel+0x89/0xa0
 [<c02432e1>] scsi_scan_host_selected+0x101/0x140
 [<c0243350>] scsi_scan_host+0x30/0x40
 [<f88c6d8d>] sdebug_driver_probe+0x9d/0xd0 [scsi_debug]
 [<c021cc4f>] bus_match+0x3f/0x70
 [<c021cccf>] device_attach+0x4f/0xa0
 [<c01ee93a>] kobject_get+0x1a/0x30
 [<c021d00b>] bus_add_device+0x7b/0xd0
 [<c021bc33>] device_add+0x93/0x120
 [<f88c6bbf>] sdebug_add_adapter+0x12f/0x1f0 [scsi_debug]
 [<f88cf36c>] scsi_debug_init+0x16c/0x1d2 [scsi_debug]
 [<c0132f59>] sys_init_module+0x199/0x230
 [<c01060cf>] syscall_call+0x7/0xb
 unknown partition table
Attached scsi disk sdc at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg3 at scsi2, channel 0, id 0, lun 0,  type 0
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c48ab>] fetch_to_dev_buffer+0x9b/0x140 [scsi_debug]
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221da0>] blk_unplug_work+0x10/0x20
 [<c012ab79>] worker_thread+0x1b9/0x260
 [<c0221d90>] blk_unplug_work+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c0116b00>] default_wake_function+0x0/0x20
 [<c012a9c0>] worker_thread+0x0/0x260
 [<c012edba>] kthread+0xba/0xc0
 [<c012ed00>] kthread+0x0/0xc0
 [<c0104145>] kernel_thread_helper+0x5/0x10
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1
SCSI device sdc: 16384 512-byte hdwr sectors (8 MB)
SCSI device sdc: drive cache: write back
 sdc:<3>Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c01349f6>] __lock_page+0xf6/0x100
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c0136247>] read_cache_page+0x127/0x190
 [<c018a352>] read_dev_sector+0x42/0xa0
 [<c015c0d0>] blkdev_readpage+0x0/0x20
 [<c018a7f1>] msdos_partition+0x51/0x320
 [<c011aaeb>] vprintk+0x12b/0x180
 [<c0189c56>] check_partition+0xd6/0x150
 [<c018a23b>] rescan_partitions+0x7b/0x150
 [<c02fcdf6>] schedule_timeout+0x76/0xc0
 [<c0224b5e>] blkdev_reread_part+0x7e/0x90
 [<c01671cb>] sys_ioctl+0x10b/0x2a0
 [<c01060cf>] syscall_call+0x7/0xb
 sdc1
Debug: sleeping function called from invalid context at arch/i386/mm/highmem.c:5
in_atomic():0, irqs_disabled():1
 [<c0117f88>] __might_sleep+0x98/0xa0
 [<c0114740>] kmap+0x20/0x50
 [<f88c4792>] fill_from_dev_buffer+0x102/0x180 [scsi_debug]
 [<f88c5546>] resp_read+0xc6/0x100 [scsi_debug]
 [<f88c42c4>] scsi_debug_queuecommand+0x2c4/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c015527f>] sync_buffer+0x3f/0x50
 [<c0155408>] __wait_on_buffer+0xa8/0xb0
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0155200>] bh_wake_function+0x0/0x40
 [<c0157706>] __block_prepare_write+0x156/0x440
 [<c01ef2cf>] radix_tree_node_alloc+0x1f/0x70
 [<c0158224>] block_prepare_write+0x34/0x50
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c013675f>] generic_file_buffered_write+0x1cf/0x600
 [<c015bf00>] blkdev_get_block+0x0/0x80
 [<c016f540>] inode_update_time+0xd0/0xe0
 [<c0136e1a>] generic_file_aio_write_nolock+0x28a/0x4c0
 [<c01370f3>] generic_file_write_nolock+0xa3/0xc0
 [<c011328c>] do_page_fault+0x1cc/0x5f1
 [<c01183e0>] autoremove_wake_function+0x0/0x60
 [<c015d038>] blkdev_file_write+0x38/0x40
 [<c01540d8>] vfs_write+0xb8/0x130
 [<c0154221>] sys_write+0x51/0x80
 [<c01060cf>] syscall_call+0x7/0xb
------------[ cut here ]------------
kernel BUG at arch/i386/mm/highmem.c:14!
invalid operand: 0000 [#1]
SMP 
Modules linked in: scsi_debug
CPU:    1
EIP:    0060:[<c0114784>]    Not tainted VLI
EFLAGS: 00010006   (2.6.9-rc4) 
EIP is at kunmap+0x14/0x30
eax: f7f82000   ebx: 00001000   ecx: 00000000   edx: c16eeb00
esi: f7759000   edi: f93ea000   ebp: f75e1c00   esp: f7f83bf4
ds: 007b   es: 007b   ss: 0068
Process mkfs.ext2 (pid: 1573, threadinfo=f7f82000 task=f7694150)
Stack: f88c4906 c16eeb00 f7f83bfc f7f83bfc 00000000 00000000 00000000 00000006 
       00000000 f93e9000 0001c000 f88c563d f7e015c0 f93e9000 0001c000 00004000 
       00000000 000000e0 f7e01614 00000000 f88c44a0 f7e015c0 00000000 00003f20 
Call Trace:
 [<f88c4906>] fetch_to_dev_buffer+0xf6/0x140 [scsi_debug]
 [<f88c563d>] resp_write+0xbd/0x100 [scsi_debug]
 [<f88c44a0>] scsi_debug_queuecommand+0x4a0/0x600 [scsi_debug]
 [<c023b86b>] scsi_dispatch_cmd+0x17b/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c023b2b3>] scsi_put_command+0x73/0xa0
 [<c0221e7d>] blk_run_queue+0x2d/0x50
 [<c02405fd>] scsi_end_request+0xdd/0xf0
 [<c024092c>] scsi_io_completion+0x15c/0x4d0
 [<c013d970>] cache_alloc_refill+0x150/0x230
 [<c0264ab8>] sd_rw_intr+0x88/0x310
 [<c023deba>] scsi_delete_timer+0x1a/0x70
 [<c023bd11>] scsi_finish_command+0x81/0xe0
 [<c023bc0e>] scsi_softirq+0xbe/0xf0
 [<c011f6ba>] __do_softirq+0xba/0xd0
 [<c011f6fd>] do_softirq+0x2d/0x30
 [<c01106fd>] smp_apic_timer_interrupt+0x8d/0x100
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c0106abe>] apic_timer_interrupt+0x1a/0x20
 [<c02fd3b5>] _spin_unlock_irqrestore+0x5/0x10
 [<c023b877>] scsi_dispatch_cmd+0x187/0x240
 [<c023bab0>] scsi_done+0x0/0x30
 [<c023df10>] scsi_times_out+0x0/0xc0
 [<c0241243>] scsi_request_fn+0x203/0x3e0
 [<c021ff2e>] elv_next_request+0x4e/0x110
 [<c0221d21>] __generic_unplug_device+0x41/0x50
 [<c0221d4e>] generic_unplug_device+0x1e/0x30
 [<c0221d7f>] blk_backing_dev_unplug+0x1f/0x30
 [<c0159202>] block_sync_page+0x42/0x50
 [<c0134875>] wait_on_page_bit+0xe5/0xf0
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c01346c0>] page_wake_function+0x0/0x50
 [<c013437c>] wait_on_page_writeback_range+0x10c/0x150
 [<c0134541>] filemap_fdatawait+0x71/0x80
 [<c0155599>] sync_blockdev+0x39/0x50
 [<c0155aa7>] sys_fsync+0xb7/0xf0
 [<c01060cf>] syscall_call+0x7/0xb
Code: d8 8b 5c 24 08 83 c4 0c e9 6a d5 02 00 8d 76 00 8d bc 27 00 00 00 00 b8 00 e0 ff ff 8b 54 24 04 21 e0 f7 40 14 00 ff ff 00 74 08 <0f> 0b 0e 00 82 d4 30 c0 3b 15 34 cb 44 c0 73 01 c3 89 d0 e9 e4 
 <0>Kernel panic - not syncing: Fatal exception in interrupt

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Douglas Gilbert]

[-- Attachment #1: Type: text/plain, Size: 894 bytes --]

Nishanth Aravamudan wrote:
> On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
> 
>>James Bottomley wrote:
>>
>>>On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
>>>
>>>
>>>>So this problem seems related to highmem.
> 
> 
> <snip>
> 
>>The above required a fair few changes to scsi_debug.
>>Attached is a patch that rolls "kmap" changes with
>>patches I have sent recently for scsi_debug.
>>Attachment is gzipped (due to size) and applies against
>>lk 2.6.8.1 -> lk 2.6.9-rc4 .
>>
>>Nishanth, could you test this with highmem?
> 
> 
> Doug,
> 
> I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
> occur (complaining about sleeping in an invalid context) and a final
> panic during mkfs :) I think something is still wrong... ;) Output is
> below and attached.

Ok, it looks like kmap_atomic() is needed.
Could you try this additional patch.

Doug Gilbert

[-- Attachment #2: sdebug269rc4_174b.diff --]
[-- Type: text/x-patch, Size: 1534 bytes --]

--- linux/drivers/scsi/scsi_debug.c	2004-10-19 07:59:14.966673272 +1000
+++ linux/drivers/scsi/scsi_debug.c174b	2004-10-19 07:58:46.044070176 +1000
@@ -56,7 +56,7 @@
 #include "scsi_debug.h"
 
 #define SCSI_DEBUG_VERSION "1.74"
-static const char * scsi_debug_version_date = "20041018";
+static const char * scsi_debug_version_date = "20041019";
 
 /* Additional Sense Code (ASC) used */
 #define NO_ADDED_SENSE 0x0
@@ -505,7 +505,8 @@
 	active = 1;
 	for (k = 0, req_len = 0, act_len = 0; k < scp->use_sg; ++k, ++sgpnt) {
 		if (active) {
-			kaddr = (unsigned char *)kmap(sgpnt->page);
+			kaddr = (unsigned char *)
+				kmap_atomic(sgpnt->page, KM_USER0);
 			if (NULL == kaddr)
 				return (DID_ERROR << 16);
 			kaddr = (unsigned char *)kaddr + sgpnt->offset;
@@ -515,7 +516,7 @@
 				len = arr_len - req_len;
 			}
 			memcpy(kaddr, arr + req_len, len);
-			kunmap(sgpnt->page);
+			kunmap_atomic(sgpnt->page, KM_USER0);
 			act_len += len;
 		}
 		req_len += sgpnt->length;
@@ -547,7 +548,7 @@
 	}
 	sgpnt = (struct scatterlist *)scp->request_buffer;
 	for (k = 0, req_len = 0, fin = 0; k < scp->use_sg; ++k, ++sgpnt) {
-		kaddr = (unsigned char *)kmap(sgpnt->page);
+		kaddr = (unsigned char *)kmap_atomic(sgpnt->page, KM_USER0);
 		if (NULL == kaddr)
 			return -1;
 		kaddr = (unsigned char *)kaddr + sgpnt->offset;
@@ -557,7 +558,7 @@
 			fin = 1;
 		}
 		memcpy(arr + req_len, kaddr, len);
-		kunmap(sgpnt->page);
+		kunmap_atomic(sgpnt->page, KM_USER0);
 		if (fin)
 			return req_len + len;
 		req_len += sgpnt->length;

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Nishanth Aravamudan]

On Tue, Oct 19, 2004 at 08:05:53AM +1000, Douglas Gilbert wrote:
> Nishanth Aravamudan wrote:
> >On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
> >
> >>James Bottomley wrote:
> >>
> >>>On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> >>>
> >>>
> >>>>So this problem seems related to highmem.
> >
> >
> ><snip>
> >
> >>The above required a fair few changes to scsi_debug.
> >>Attached is a patch that rolls "kmap" changes with
> >>patches I have sent recently for scsi_debug.
> >>Attachment is gzipped (due to size) and applies against
> >>lk 2.6.8.1 -> lk 2.6.9-rc4 .
> >>
> >>Nishanth, could you test this with highmem?
> >
> >
> >Doug,
> >
> >I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
> >occur (complaining about sleeping in an invalid context) and a final
> >panic during mkfs :) I think something is still wrong... ;) Output is
> >below and attached.
> 
> Ok, it looks like kmap_atomic() is needed.
> Could you try this additional patch.

Great! That patch seems to have fixed it. I am able to vi files, sync
the scsi_debug 'disks' and not have any hangs. It is noticeably slower,
though, it seems. Might that just be a side effect of the atomic calls?

Thanks again!

-Nish

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Douglas Gilbert]

Nishanth Aravamudan wrote:
> On Tue, Oct 19, 2004 at 08:05:53AM +1000, Douglas Gilbert wrote:
> 
>>Nishanth Aravamudan wrote:
>>
>>>On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
>>>
>>>
>>>>James Bottomley wrote:
>>>>
>>>>
>>>>>On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
>>>>>
>>>>>
>>>>>
>>>>>>So this problem seems related to highmem.
>>>
>>>
>>><snip>
>>>
>>>>The above required a fair few changes to scsi_debug.
>>>>Attached is a patch that rolls "kmap" changes with
>>>>patches I have sent recently for scsi_debug.
>>>>Attachment is gzipped (due to size) and applies against
>>>>lk 2.6.8.1 -> lk 2.6.9-rc4 .
>>>>
>>>>Nishanth, could you test this with highmem?
>>>
>>>
>>>Doug,
>>>
>>>I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
>>>occur (complaining about sleeping in an invalid context) and a final
>>>panic during mkfs :) I think something is still wrong... ;) Output is
>>>below and attached.
>>
>>Ok, it looks like kmap_atomic() is needed.
>>Could you try this additional patch.
> 
> 
> Great! That patch seems to have fixed it. I am able to vi files, sync
> the scsi_debug 'disks' and not have any hangs. It is noticeably slower,
> though, it seems. Might that just be a side effect of the atomic calls?

One would expect it to be pretty fast since only a
ram copy is involved.

Perhaps James can comment on this strategy:

   if (( kaddr = page_address(scatp->page)) {
         kaddr += scatp->offset;
	memcpy(....);
   } else {
	kaddr = kmap_atomic(scatp->page) + scatp-offset;
	memcpy(....);
	kunmap_atomic(scatp->page);
   }

	
Doug Gilbert

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Nishanth Aravamudan]

On Tue, Oct 19, 2004 at 04:57:44PM +1000, Douglas Gilbert wrote:
> Nishanth Aravamudan wrote:
> >On Tue, Oct 19, 2004 at 08:05:53AM +1000, Douglas Gilbert wrote:
> >
> >>Nishanth Aravamudan wrote:
> >>
> >>>On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
> >>>
> >>>
> >>>>James Bottomley wrote:
> >>>>
> >>>>
> >>>>>On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>>So this problem seems related to highmem.
> >>>
> >>>
> >>><snip>
> >>>
> >>>>The above required a fair few changes to scsi_debug.
> >>>>Attached is a patch that rolls "kmap" changes with
> >>>>patches I have sent recently for scsi_debug.
> >>>>Attachment is gzipped (due to size) and applies against
> >>>>lk 2.6.8.1 -> lk 2.6.9-rc4 .
> >>>>
> >>>>Nishanth, could you test this with highmem?
> >>>
> >>>
> >>>Doug,
> >>>
> >>>I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
> >>>occur (complaining about sleeping in an invalid context) and a final
> >>>panic during mkfs :) I think something is still wrong... ;) Output is
> >>>below and attached.
> >>
> >>Ok, it looks like kmap_atomic() is needed.
> >>Could you try this additional patch.
> >
> >
> >Great! That patch seems to have fixed it. I am able to vi files, sync
> >the scsi_debug 'disks' and not have any hangs. It is noticeably slower,
> >though, it seems. Might that just be a side effect of the atomic calls?
> 
> One would expect it to be pretty fast since only a
> ram copy is involved.

I agree, and in fact I'm not seeing any perf. related problems anymore.
Now it's on to debug udev's relation to thousands of disks :)

Thanks for all your help!

-Nish

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Jens Axboe]

On Tue, Oct 19 2004, Douglas Gilbert wrote:
> Nishanth Aravamudan wrote:
> >On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
> >
> >>James Bottomley wrote:
> >>
> >>>On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> >>>
> >>>
> >>>>So this problem seems related to highmem.
> >
> >
> ><snip>
> >
> >>The above required a fair few changes to scsi_debug.
> >>Attached is a patch that rolls "kmap" changes with
> >>patches I have sent recently for scsi_debug.
> >>Attachment is gzipped (due to size) and applies against
> >>lk 2.6.8.1 -> lk 2.6.9-rc4 .
> >>
> >>Nishanth, could you test this with highmem?
> >
> >
> >Doug,
> >
> >I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
> >occur (complaining about sleeping in an invalid context) and a final
> >panic during mkfs :) I think something is still wrong... ;) Output is
> >below and attached.
> 
> Ok, it looks like kmap_atomic() is needed.
> Could you try this additional patch.
> 
> Doug Gilbert

> --- linux/drivers/scsi/scsi_debug.c	2004-10-19 07:59:14.966673272 +1000
> +++ linux/drivers/scsi/scsi_debug.c174b	2004-10-19 07:58:46.044070176 +1000
> @@ -56,7 +56,7 @@
>  #include "scsi_debug.h"
>  
>  #define SCSI_DEBUG_VERSION "1.74"
> -static const char * scsi_debug_version_date = "20041018";
> +static const char * scsi_debug_version_date = "20041019";
>  
>  /* Additional Sense Code (ASC) used */
>  #define NO_ADDED_SENSE 0x0
> @@ -505,7 +505,8 @@
>  	active = 1;
>  	for (k = 0, req_len = 0, act_len = 0; k < scp->use_sg; ++k, ++sgpnt) {
>  		if (active) {
> -			kaddr = (unsigned char *)kmap(sgpnt->page);
> +			kaddr = (unsigned char *)
> +				kmap_atomic(sgpnt->page, KM_USER0);
>  			if (NULL == kaddr)
>  				return (DID_ERROR << 16);
>  			kaddr = (unsigned char *)kaddr + sgpnt->offset;
> @@ -515,7 +516,7 @@
>  				len = arr_len - req_len;
>  			}
>  			memcpy(kaddr, arr + req_len, len);
> -			kunmap(sgpnt->page);
> +			kunmap_atomic(sgpnt->page, KM_USER0);

BUG, kunmap_atomic(kaddr) (without the offset).

Also do make sure that you stay on the same cpu from calling kmap_atomic
to kunmap_atomic(). If you are under the queue_lock you are fine, I
didn't check if this is the case for all your calls.

-- 
Jens Axboe

Re: [PATCH] scsi_debug [was: scsi_debug issues]

[Jens Axboe]

On Tue, Oct 19 2004, Douglas Gilbert wrote:
> Nishanth Aravamudan wrote:
> >On Tue, Oct 19, 2004 at 08:05:53AM +1000, Douglas Gilbert wrote:
> >
> >>Nishanth Aravamudan wrote:
> >>
> >>>On Mon, Oct 18, 2004 at 11:44:59PM +1000, Douglas Gilbert wrote:
> >>>
> >>>
> >>>>James Bottomley wrote:
> >>>>
> >>>>
> >>>>>On Sat, 2004-10-16 at 01:51, Douglas Gilbert wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>>So this problem seems related to highmem.
> >>>
> >>>
> >>><snip>
> >>>
> >>>>The above required a fair few changes to scsi_debug.
> >>>>Attached is a patch that rolls "kmap" changes with
> >>>>patches I have sent recently for scsi_debug.
> >>>>Attachment is gzipped (due to size) and applies against
> >>>>lk 2.6.8.1 -> lk 2.6.9-rc4 .
> >>>>
> >>>>Nishanth, could you test this with highmem?
> >>>
> >>>
> >>>Doug,
> >>>
> >>>I ran 2.6.9-rc4 with your patch applied and had several dump_stack()s
> >>>occur (complaining about sleeping in an invalid context) and a final
> >>>panic during mkfs :) I think something is still wrong... ;) Output is
> >>>below and attached.
> >>
> >>Ok, it looks like kmap_atomic() is needed.
> >>Could you try this additional patch.
> >
> >
> >Great! That patch seems to have fixed it. I am able to vi files, sync
> >the scsi_debug 'disks' and not have any hangs. It is noticeably slower,
> >though, it seems. Might that just be a side effect of the atomic calls?
> 
> One would expect it to be pretty fast since only a
> ram copy is involved.
> 
> Perhaps James can comment on this strategy:
> 
>   if (( kaddr = page_address(scatp->page)) {
>         kaddr += scatp->offset;
> 	memcpy(....);
>   } else {
> 	kaddr = kmap_atomic(scatp->page) + scatp-offset;
> 	memcpy(....);
> 	kunmap_atomic(scatp->page);
>   }

Don't do that, always kmap_atomic(). It already does various checks to
avoid unecessary work.

-- 
Jens Axboe