Re: BUG() in 2.4: sg direct IO + exit()

Jens Axboe
2004-03-24T13:02:15Z

On Wed, Mar 24 2004, Douglas Gilbert wrote:
> Tony Battersby wrote:
> >The following BUG() is triggered in 2.4.x when a program calls exit()
> >immediately after sending a SCSI command that uses direct IO:
> >
> >kernel BUG at page_alloc.c:98!
> >invalid operand: 0000
> >sg sym53c8xx_2 e1000
> >CPU: 0
> >EIP: 0010:[<c0133d22>] Not tainted
> >EFLAGS: 00010202
> >EIP is at __free_pages_ok+0x32/0x2b0 [kernel]
> >eax: 00000001 ebx: c12aceb0 ecx: c12aceb0 edx: 00000000
> >esi: 00000000 edi: 00000000 ebp: 00000001 esp: c02bbe94
> >ds: 0018 es: 0018 ss: 0018
> >Process swapper (pid: 0, stackpage=c02bb000)
> >Stack: cfefc640 cfdf2de8 cfbe93c0 cfefc600 00000001 00000000 cfefc600
> >c12aceb0
> > cf926fa0 00000000 cfb8b040 c0127cb0 cfb8b04c cfb8b04c cfb8b000
> >d0885db6
> > cf926fa0 00000000 cfb8b040 d0885cc9 cfb8b04c 00000001 cfd08680
> >cfd086c8
> >Call Trace:
> > [<c0127cb0>] unmap_kiobuf+0x30/0x50 [kernel]
> > [<d0885db6>] sg_unmap_and+0x26/0x50 [sg]
> > [<d0885cc9>] sg_finish_rem_req+0x39/0x70 [sg]
> > [<d0885451>] sg_cmd_done_bh+0x281/0x380 [sg]
> > [<c01ab65a>] scsi_finish_command+0xda/0xe0 [kernel]
> > [<c01ab380>] scsi_bottom_half_handler+0xc0/0x230 [kernel]
> > [<c011dacb>] bh_action+0x4b/0x90 [kernel]
> > [<c011d971>] tasklet_hi_action+0x61/0xa0 [kernel]
> > [<c011d6fb>] do_softirq+0x6b/0xd0 [kernel]
> > [<c0108baf>] do_IRQ+0xdf/0xf0 [kernel]
> > [<c01052c0>] default_idle+0x0/0x40 [kernel]
> > [<c01052c0>] default_idle+0x0/0x40 [kernel]
> > [<c010b388>] call_do_IRQ+0x5/0xd [kernel]
> > [<c01052c0>] default_idle+0x0/0x40 [kernel]
> > [<c01052c0>] default_idle+0x0/0x40 [kernel]
> > [<c01052ec>] default_idle+0x2c/0x40 [kernel]
> > [<c0105372>] cpu_idle+0x52/0x70 [kernel]
> > [<c0105000>] stext+0x0/0x50 [kernel]
> >
> >Code: 0f 0b 62 00 ab 04 23 c0 89 d8 e8 2f ed ff ff 8b 7b 28 85 ff
> > <0>Kernel panic: Aiee, killing interrupt handler!
> >In interrupt handler - not syncing
> >
> >The following program reproduces the problem:
> >
> >#include <stdlib.h>
> >#include <stdio.h>
> >#include <string.h>
> >#include <unistd.h>
> >#include <scsi/sg.h>
> >#include <limits.h>
> >#include <sys/types.h>
> >#include <sys/stat.h>
> >#include <fcntl.h>
> >
> >unsigned char mode_sense_cdb[] = { 0x1a, 0x00, 0x3f, 0x00, 0xff, 0x00 };
> >
> >int main(int argc, char *argv[])
> >{
> > int fd;
> > unsigned char sense_data[256];
> > unsigned char buf[256];
> > sg_io_hdr_t io;
> >
> > fd = open("/dev/sg0", O_RDWR);
> > if (fd == -1) {
> > perror("open(/dev/sg0)");
> > exit(1);
> > }
> >
> > memset(&io, 0, sizeof(sg_io_hdr_t));
> > io.interface_id = 'S';
> > io.timeout = UINT_MAX;
> > io.flags = SG_FLAG_DIRECT_IO;
> > io.sbp = sense_data;
> > io.mx_sb_len = 0xff;
> > io.cmdp = mode_sense_cdb;
> > io.cmd_len = 6;
> > io.dxfer_direction = SG_DXFER_FROM_DEV;
> > io.dxferp = buf;
> > io.dxfer_len = 0xff;
> > write(fd, &io, sizeof(sg_io_hdr_t));
> >
> > return 0;
> >}
> >
> >Just compile, echo 1 > /proc/scsi/sg/allow_dio, and run. The SCSI
> >device that I am using is a tape drive, but it shouldn't matter.
> >
> >I have tested this with 2.4.21+kksymoops and 2.4.26-pre5+kksymoops with
> >the same results. The above stack trace is from 2.4.26-pre5. I haven't
> >tried 2.6.
> >
> >I am running a SMP kernel on a UP Intel P4 with 256 MB RAM. The test
> >machine does not have any swap space configured (swapoff -a).
>
> Tony,
> It is not causing an oops when I try with scsi_debug and lk 2.6.5-rc2.
> Neither is there a problem with a Suse 9 stock SMP kernel
> (2.4.21-99-smp4G) on an old dual celeron (A-bit mb) box with a
> Sony SDT-7000 tape drive on /dev/sg0.
>
> I'll keep looking. The oops suggests that the memory is not being
> locked down (as you are probably aware).

Looks like an sg bug, you are doing direct io cleanup from interrupt
context if the fd has been closed (SCSI -> sg_cmd_done_bh ->
sg_finish_rem_req -> sg_unmap_and -> unmap_kiobuf).

--
Jens Axboe