Re: BIO Usage Error or Conflicting Designs

Re: BIO Usage Error or Conflicting Designs

[Manfred Spraul]

 
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

> 
> Is this with the highmem debug stuff enabled? That's the only way I can
> see this BUG triggering, otherwise q->bounce_pfn _cannot_ be smaller
> than the max_pfn.
> 
Have you tested that?

Unless I misread arch/i386/kernel/setup.c, line 740 to 760, max_pfn is
the upper end of the highmem area, if highmem is configured.
For non-highmem setup, it's set to min(system_memory, 4 GB).
It was a local variable within setup_arch, and someone made it a global
variable.

I.e. max_pfn is 1 GB with Andre's setup.

His patch doesn't touch the bounce limit, the default limit from
blk_queue_make_request() is used: BLK_BOUNCE_HIGH, which is max_low_pfn.

max_low_pfn is 896 MB.

--> BUG in create_bounce(), because a request comes in with a bounce
limit less than the total system memory, and no highmem configured.

--
	Manfred

Re: BIO Usage Error or Conflicting Designs

[Jens Axboe]

On Sun, Jan 13 2002, Manfred Spraul wrote:
> > 
> > Is this with the highmem debug stuff enabled? That's the only way I can
> > see this BUG triggering, otherwise q->bounce_pfn _cannot_ be smaller
> > than the max_pfn.
> > 
> Have you tested that?
> 
> Unless I misread arch/i386/kernel/setup.c, line 740 to 760, max_pfn is
> the upper end of the highmem area, if highmem is configured.
> For non-highmem setup, it's set to min(system_memory, 4 GB).
> It was a local variable within setup_arch, and someone made it a global
> variable.
> 
> I.e. max_pfn is 1 GB with Andre's setup.
> 
> His patch doesn't touch the bounce limit, the default limit from
> blk_queue_make_request() is used: BLK_BOUNCE_HIGH, which is max_low_pfn.
> 
> max_low_pfn is 896 MB.
> 
> --> BUG in create_bounce(), because a request comes in with a bounce
> limit less than the total system memory, and no highmem configured.

Indeed, I misread the max_pfn stuff when I added that.

--- /opt/kernel/linux-2.5.2-pre11/drivers/block/ll_rw_blk.c	Thu Jan 10 09:56:52 2002
+++ drivers/block/ll_rw_blk.c	Mon Jan 14 02:21:50 2002
@@ -1711,7 +1705,11 @@
 	printk("block: %d slots per queue, batch=%d\n", queue_nr_requests, batch_requests);
 
 	blk_max_low_pfn = max_low_pfn;
+#ifdef CONFIG_HIGHMEM
 	blk_max_pfn = max_pfn;
+#else
+	blk_max_pfn = max_low_pfn;
+#endif
 
 #if defined(CONFIG_IDE) && defined(CONFIG_BLK_DEV_IDE)
 	ide_init();		/* this MUST precede hd_init */
--- /opt/kernel/linux-2.5.2-pre11/mm/highmem.c	Thu Jan 10 09:56:53 2002
+++ mm/highmem.c	Mon Jan 14 02:20:53 2002
@@ -367,12 +367,6 @@
 		if (pfn >= blk_max_pfn)
 			return;
 
-#ifndef CONFIG_HIGHMEM
-		/*
-		 * should not hit for non-highmem case
-		 */
-		BUG();
-#endif
 		bio_gfp = GFP_NOHIGHIO;
 		pool = page_pool;
 	} else {

-- 
Jens Axboe

Re: [PATCH] 1-2-3 GB

[Albert D. Cahalan]

Andrea Arcangeli writes:
> On Sat, Jan 12, 2002 at 12:26:35PM -0500, Albert D. Cahalan wrote:

>> The numbers are wrong anyway, because of vmalloc() and PCI space.
>> The PCI space is motherboard-dependent AFAIK, but you could at
>> least account for the 128 MB vmalloc() area:
>
> looks dirty, the size of the kernel direct mapping is mainly in function
> of #defines that can be changed freely, they're not constant in function
> of CONFIG_1G etc.. and it changes also in function of smp/up/4G/64G
> options.  The 3GB/2GB/1GB/3.5GB visible into the menuconfig are exact
> instead.  So I wouldn't mention inprecise stuff that can changed under
> us (and the exact size of the kernel direct mapping doesn't matter to
> the user anyways I think [and if it matters I think it means he's
> skilled enough to know about vmalloc space ;) ]).

The problem is that the "1GB" option doesn't cover 1 GB.
It is common for people to buy 1 GB of memory (power of 2),
and then complain that Linux only sees 896 MB of memory.

So how will you make the choices clear to such people?
There are 3 options, not counting the slram block device:

a. give up 128 MB (12.5 %) of memory
b. suffer the kmap overhead
c. give up some user virtual address space

None of this is obvious. The user sees "1GB" and will
innocently believe that this is good for a 1 GB system!

BTW, do we no longer require that the kernel side of things
(whole thing, including vmalloc space) be a power of two?
There used to be a bitwise operation in the user access stuff.

BIO Usage Error or Conflicting Designs

[Andre Hedrick]

[-- Attachment #1: Type: text/PLAIN, Size: 1990 bytes --]


Jens,

Below is a single sector read using ACB.
If I do not use the code inside "#ifdef USEBIO" and run UP/SMP but no
highmem, it runs and works like a charm.  It is also 100% unchanged code
from what is in 2.4 patches.  The attached oops is generate under
SMP without highmem and running the USEBIO code.

CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set

Regards,

Andre Hedrick
Linux Disk Certification Project                Linux ATA Development


/*
 * Handler for command with PIO data-in phase
 */
ide_startstop_t task_in_intr (ide_drive_t *drive)
{
  byte stat               = GET_STAT();
  byte io_32bit           = drive->io_32bit;
  struct request *rq      = HWGROUP(drive)->rq;
  char *pBuf              = NULL;

  if (!OK_STAT(stat,DATA_READY,BAD_R_STAT)) {
    if (stat & (ERR_STAT|DRQ_STAT)) {
      return ide_error(drive, "task_in_intr", stat);
    }
    if (!(stat & BUSY_STAT)) {
      DTF("task_in_intr to Soon wait for next interrupt\n");
      ide_set_handler(drive, &task_in_intr, WAIT_CMD, NULL);
      return ide_started;
    }
  }
  drive->io_32bit = 0;
  DTF("stat: %02x\n", stat);
#ifdef USEBIO
  if (rq->flags & REQ_CMD) {
    pBuf = ide_map_buffer(rq, &flags);
  } else {
    pBuf = rq->buffer + ((rq->nr_sectors - rq->current_nr_sectors) * SECTOR_SIZE);
  }
#else
  pBuf = rq->buffer + ((rq->nr_sectors - rq->current_nr_sectors) * SECTOR_SIZE);
#endif
  DTF("Read: %p, rq->current_nr_sectors: %d\n", pBuf, (int) rq->current_nr_sectors);
  taskfile_input_data(drive, pBuf, SECTOR_WORDS);
#ifdef USEBIO
  if (rq->flags & REQ_CMD)
    ide_unmap_buffer(pBuf, &flags);
  rq->sector++;
  rq->errors = 0;
#endif
  drive->io_32bit = io_32bit;
  if (--rq->current_nr_sectors <= 0) {
    /* (hs): swapped next 2 lines */
    DTF("Request Ended stat: %02x\n", GET_STAT());
    ide_end_request(1, HWGROUP(drive));
  } else {
    ide_set_handler(drive, &task_in_intr,  WAIT_CMD, NULL);
    return ide_started;
  }
  return ide_stopped;
}


[-- Attachment #2: Type: text/PLAIN, Size: 2377 bytes --]

ksymoops 2.3.5 on i686 2.5.2-pre11.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.5.2-pre11/ (default)
     -m /usr/src/linux/System.map (specified)

No modules in ksyms, skipping objects
Warning (read_lsmod): no symbols in lsmod, is /proc/modules a valid lsmod file?
cpu: 0, clocks: 2659849, slice: 886616
cpu: 1, clocks: 2659849, slice: 886616
invalid operand: 0000
CPU:    0
EIP:    0010:[<c0136dd9>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010283
eax: 00038000   ebx: 00000070   ecx: f70e4980   edx: f70e4980
esi: c0402084   edi: 00038000   ebp: f70e4980   esp: f6a01df4
ds: 0018   es: 0018   ss: 0018
Process hdparm (pid: 822, stackpage=f6a01000)
Stack: 00000000 c0402084 00000002 f70e4980 00001000 00000001 00000000 00000217 
       00000000 00000000 00000000 00000000 c01fcafe 00038000 00000070 f6a01e68 
       00000000 f7d10640 c0402084 f70e4980 00000002 00000000 00000000 00000002 
Call Trace: [<c01fcafe>] [<c01fd036>] [<c01fd14c>] [<c01fd1f9>] [<c013be3b>] 
   [<c013f00f>] [<c013ef70>] [<c012b7ce>] [<c012bc82>] [<c012bb90>] [<c01390ce>] 
   [<c0108c7f>] 
Code: 0f 0b c7 44 24 24 70 00 00 00 8b 1d 68 08 3e c0 89 5c 24 20 

>>EIP; c0136dd9 <create_bounce+49/2c0>   <=====
Trace; c01fcafe <__make_request+5e/470>
Trace; c01fd036 <generic_make_request+126/1b0>
Trace; c01fd14c <submit_bio+4c/60>
Trace; c01fd1f9 <submit_bh+99/a0>
Trace; c013be3b <block_read_full_page+1db/1f0>
Trace; c013f00f <blkdev_readpage+f/20>
Trace; c013ef70 <blkdev_get_block+0/40>
Trace; c012b7ce <do_generic_file_read+2de/460>
Trace; c012bc82 <generic_file_read+92/190>
Trace; c012bb90 <file_read_actor+0/60>
Trace; c01390ce <sys_read+8e/d0>
Trace; c0108c7f <system_call+33/38>
Code;  c0136dd9 <create_bounce+49/2c0>
00000000 <_EIP>:
Code;  c0136dd9 <create_bounce+49/2c0>   <=====
   0:   0f 0b                     ud2a      <=====
Code;  c0136ddb <create_bounce+4b/2c0>
   2:   c7 44 24 24 70 00 00      movl   $0x70,0x24(%esp,1)
Code;  c0136de2 <create_bounce+52/2c0>
   9:   00 
Code;  c0136de3 <create_bounce+53/2c0>
   a:   8b 1d 68 08 3e c0         mov    0xc03e0868,%ebx
Code;  c0136de9 <create_bounce+59/2c0>
  10:   89 5c 24 20               mov    %ebx,0x20(%esp,1)


1 warning issued.  Results may not be reliable.

Re: BIO Usage Error or Conflicting Designs

[Jens Axboe]

On Sat, Jan 12 2002, Andre Hedrick wrote:
> 
> Jens,
> 
> Below is a single sector read using ACB.
> If I do not use the code inside "#ifdef USEBIO" and run UP/SMP but no
> highmem, it runs and works like a charm.  It is also 100% unchanged code
> from what is in 2.4 patches.  The attached oops is generate under
> SMP without highmem and running the USEBIO code.
> 
> CONFIG_NOHIGHMEM=y

Is this with the highmem debug stuff enabled? That's the only way I can
see this BUG triggering, otherwise q->bounce_pfn _cannot_ be smaller
than the max_pfn.

-- 
Jens Axboe

Re: BIO Usage Error or Conflicting Designs

[Andre Hedrick]

[-- Attachment #1: Type: text/plain, Size: 118 bytes --]


Jens,

Here is back at you sir.

Andre Hedrick
Linux Disk Certification Project                Linux ATA Development

[-- Attachment #2: Type: text/plain, Size: 3831 bytes --]

ksymoops 2.3.5 on i686 2.5.2-pre11.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.5.2-pre11/ (default)
     -m /usr/src/linux/System.map (specified)

No modules in ksyms, skipping objects
Warning (read_lsmod): no symbols in lsmod, is /proc/modules a valid lsmod file?
cpu: 0, clocks: 2660067, slice: 886689
cpu: 1, clocks: 2660067, slice: 886689
kernel BUG at highmem.c:374!
invalid operand: 0000
CPU:    1
EIP:    0010:[<c0144478>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010286
eax: 0000001d   ebx: 00000070   ecx: c03785ac   edx: 00003ce8
esi: c045ea64   edi: 00038000   ebp: f76bf2c4   esp: f6321de4
ds: 0018   es: 0018   ss: 0018
Process hdparm (pid: 924, stackpage=f6321000)
Stack: c02fe555 00000176 00000000 c045ea64 00000002 f76bf2c4 00000001 00000246 
       c1effecc 00000000 f1aabd84 00000020 00000000 00000000 c022092e 00038000 
       00000070 f6321e60 00000000 f7c33344 c045ea64 f76bf2c4 00000002 00000000 
Call Trace: [<c022092e>] [<c020f21e>] [<c0220f9f>] [<c02210f1>] [<c02211d9>] 
   [<c014b31c>] [<c014f46f>] [<c014f3d0>] [<c013477e>] [<c0134cc2>] [<c0134bd0>] 
   [<c014706e>] [<c010935f>] 
Code: 0f 0b 83 c4 08 c7 44 24 24 70 00 00 00 8b 1d 68 b9 43 c0 8b 

>>EIP; c0144478 <create_bounce+68/310>   <=====
Trace; c022092e <__make_request+5e/590>
Trace; c020f21e <set_cursor+6e/80>
Trace; c0220f9f <generic_make_request+13f/1c0>
Trace; c02210f1 <submit_bio+81/90>
Trace; c02211d9 <submit_bh+d9/e0>
Trace; c014b31c <block_read_full_page+1ec/200>
Trace; c014f46f <blkdev_readpage+f/20>
Trace; c014f3d0 <blkdev_get_block+0/40>
Trace; c013477e <do_generic_file_read+34e/560>
Trace; c0134cc2 <generic_file_read+92/190>
Trace; c0134bd0 <file_read_actor+0/60>
Trace; c014706e <sys_read+8e/d0>
Trace; c010935f <system_call+33/38>
Code;  c0144478 <create_bounce+68/310>
00000000 <_EIP>:
Code;  c0144478 <create_bounce+68/310>   <=====
   0:   0f 0b                     ud2a      <=====
Code;  c014447a <create_bounce+6a/310>
   2:   83 c4 08                  add    $0x8,%esp
Code;  c014447d <create_bounce+6d/310>
   5:   c7 44 24 24 70 00 00      movl   $0x70,0x24(%esp,1)
Code;  c0144484 <create_bounce+74/310>
   c:   00 
Code;  c0144485 <create_bounce+75/310>
   d:   8b 1d 68 b9 43 c0         mov    0xc043b968,%ebx
Code;  c014448b <create_bounce+7b/310>
  13:   8b 00                     mov    (%eax),%eax


1 warning issued.  Results may not be reliable.

[root@xathy DiskPerf-1.0.7]# hdparm -t /dev/hde

/dev/hde:
 Timing buffered disk reads:  kernel BUG at highmem.c:374!
invalid operand: 0000
CPU:    1
EIP:    0010:[<c0144478>]    Not tainted
EFLAGS: 00010286
eax: 0000001d   ebx: 00000070   ecx: c03785ac   edx: 00003ce8
esi: c045ea64   edi: 00038000   ebp: f76bf2c4   esp: f6321de4
ds: 0018   es: 0018   ss: 0018
Process hdparm (pid: 924, stackpage=f6321000)
Stack: c02fe555 00000176 00000000 c045ea64 00000002 f76bf2c4 00000001 00000246
       c1effecc 00000000 f1aabd84 00000020 00000000 00000000 c022092e 00038000
       00000070 f6321e60 00000000 f7c33344 c045ea64 f76bf2c4 00000002 00000000
Call Trace: [<c022092e>] [<c020f21e>] [<c0220f9f>] [<c02210f1>] [<c02211d9>]
   [<c014b31c>] [<c014f46f>] [<c014f3d0>] [<c013477e>] [<c0134cc2>] [<c0134bd0>]
   [<c014706e>] [<c010935f>]

Code: 0f 0b 83 c4 08 c7 44 24 24 70 00 00 00 8b 1d 68 b9 43 c0 8b
 Segmentation fault

[root@xathy DiskPerf-1.0.7]# hdparm -v /dev/hde

/dev/hde:
 multcount    =  0 (off)
 I/O support  =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  0 (off)
 keepsettings =  0 (off)
 nowerr       =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 55488/255/63, sectors = 320173056, start = 0

Re: BIO Usage Error or Conflicting Designs

[Jens Axboe]

On Sat, Jan 12 2002, Andre Hedrick wrote:
> 
> Jens,
> 
> Here is back at you sir.

Without highmem debug enabled?? I already knew this was the bug
triggered, nothing new here.

Please print the two pfn values triggering the BUG_ON, I'll take a look
at this tomorrow.

-- 
Jens Axboe

Re: BIO Usage Error or Conflicting Designs

[Andre Hedrick]

On Sun, 13 Jan 2002, Jens Axboe wrote:

> On Sat, Jan 12 2002, Andre Hedrick wrote:
> > 
> > Jens,
> > 
> > Here is back at you sir.
> 
> Without highmem debug enabled?? I already knew this was the bug
> triggered, nothing new here.
> 
> Please print the two pfn values triggering the BUG_ON, I'll take a look
> at this tomorrow.

That is with highmem debug on, the stuff at the end of the config file.
Nothing more is generated, if there are more flags to set please tell me
where.

Regards,

Andre Hedrick
Linux Disk Certification Project                Linux ATA Development

Re: BIO Usage Error or Conflicting Designs

[Jens Axboe]

On Sun, Jan 13 2002, Andre Hedrick wrote:
> On Sun, 13 Jan 2002, Jens Axboe wrote:
> 
> > On Sat, Jan 12 2002, Andre Hedrick wrote:
> > > 
> > > Jens,
> > > 
> > > Here is back at you sir.
> > 
> > Without highmem debug enabled?? I already knew this was the bug
> > triggered, nothing new here.
> > 
> > Please print the two pfn values triggering the BUG_ON, I'll take a look
> > at this tomorrow.
> 
> That is with highmem debug on, the stuff at the end of the config file.
> Nothing more is generated, if there are more flags to set please tell me
> where.

Sorry if I wasn't clear, I mean the emulate highmem debug patch I
forwarded to you. I'll look into Manfred's post right now, you can
simply remove the

#ifndef CONFIG_HIGHMEM
	BUG();
#endif

test for now, for testing.

-- 
Jens Axboe