[PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Joe Korty]

Hi Alan, Everyone,
The bh_map_irq/bh_unmap_irq functions are broken in 2.4.21-pre5.
However no symptoms occur unless the preemption patch is applied.

The bug is that bh_map_irq *conditionally* calls kmap_atomic (which
disables preemption as one of its functions), while bh_unmap_irq
*unconditionally* calls kunmap_atomic (which enables it).  This
imbalance results in a occasional off-by-one preempt_count, which in
turn causes IDE PIO mode interrupt code (specifically, read_intr) to
erronously invoke preempt_schedule while at interrupt level.

The below patch compiles and boots ide=nodma on my preempt 2.4 kernel
on the one motherboard that had the problem.  Before this patch, the
kernel would not even boot for that motherboard.  I also applied and
test booted a pure 2.4.21-pre5 kernel with this patch.

The patch implements my preference for simplicity, so you may want to
take some other approach if maximal performance is what you want.

Joe




--- include/linux/highmem.h.orig	2003-03-12 05:01:56.000000000 -0500
+++ include/linux/highmem.h	2003-03-12 16:07:04.000000000 -0500
@@ -33,22 +33,10 @@
 {
 	unsigned long addr;
 
-	__save_flags(*flags);
-
-	/*
-	 * could be low
-	 */
-	if (!PageHighMem(bh->b_page))
-		return bh->b_data;
-
-	/*
-	 * it's a highmem page
-	 */
-	__cli();
+	local_irq_save(*flags);
 	addr = (unsigned long) kmap_atomic(bh->b_page, KM_BH_IRQ);
 
-	if (addr & ~PAGE_MASK)
-		BUG();
+	BUG_ON (addr & ~PAGE_MASK);
 
 	return (char *) addr + bh_offset(bh);
 }
@@ -58,7 +46,7 @@
 	unsigned long ptr = (unsigned long) buffer & PAGE_MASK;
 
 	kunmap_atomic((void *) ptr, KM_BH_IRQ);
-	__restore_flags(*flags);
+	local_irq_restore(*flags);
 }
 
 #else /* CONFIG_HIGHMEM */

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Alan Cox]

On Wed, 2003-03-12 at 22:13, Joe Korty wrote:
> The bug is that bh_map_irq *conditionally* calls kmap_atomic (which
> disables preemption as one of its functions), while bh_unmap_irq
> *unconditionally* calls kunmap_atomic (which enables it).  This

Thats a pre-empt bug ont a bh_map_irq bug. I'm glad you've found it
however. It explains a few things and will be useful for people wanting
pre-empt 2.4 .

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Jens Axboe]

On Wed, Mar 12 2003, Joe Korty wrote:
> Hi Alan, Everyone,
> The bh_map_irq/bh_unmap_irq functions are broken in 2.4.21-pre5.
> However no symptoms occur unless the preemption patch is applied.
> 
> The bug is that bh_map_irq *conditionally* calls kmap_atomic (which
> disables preemption as one of its functions), while bh_unmap_irq
> *unconditionally* calls kunmap_atomic (which enables it).  This
> imbalance results in a occasional off-by-one preempt_count, which in
> turn causes IDE PIO mode interrupt code (specifically, read_intr) to
> erronously invoke preempt_schedule while at interrupt level.
> 
> The below patch compiles and boots ide=nodma on my preempt 2.4 kernel
> on the one motherboard that had the problem.  Before this patch, the
> kernel would not even boot for that motherboard.  I also applied and
> test booted a pure 2.4.21-pre5 kernel with this patch.
> 
> The patch implements my preference for simplicity, so you may want to
> take some other approach if maximal performance is what you want.

I fixed this in 2.5 ages ago, just didn't get it in 2.4 block-highmem...
There's a tiny bit missing from your patch:

> --- include/linux/highmem.h.orig	2003-03-12 05:01:56.000000000 -0500
> +++ include/linux/highmem.h	2003-03-12 16:07:04.000000000 -0500
> @@ -33,22 +33,10 @@
>  {
>  	unsigned long addr;
>  
> -	__save_flags(*flags);
> -
> -	/*
> -	 * could be low
> -	 */
> -	if (!PageHighMem(bh->b_page))
> -		return bh->b_data;
> -
> -	/*
> -	 * it's a highmem page
> -	 */
> -	__cli();
> +	local_irq_save(*flags);
	local_irq_disable();

>  	addr = (unsigned long) kmap_atomic(bh->b_page, KM_BH_IRQ);
>  
> -	if (addr & ~PAGE_MASK)
> -		BUG();
> +	BUG_ON (addr & ~PAGE_MASK);
>  
>  	return (char *) addr + bh_offset(bh);
>  }
> @@ -58,7 +46,7 @@
>  	unsigned long ptr = (unsigned long) buffer & PAGE_MASK;
>  
>  	kunmap_atomic((void *) ptr, KM_BH_IRQ);
> -	__restore_flags(*flags);
> +	local_irq_restore(*flags);
>  }

other than that it's fine. See 2.5 for reference.

-- 
Jens Axboe

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Marc-Christian Petersen]

On Thursday 13 March 2003 10:26, Jens Axboe wrote:

Hi Jens,

> There's a tiny bit missing from your patch:
> > -	 * it's a highmem page
> > -	 */
> > -	__cli();
> > +	local_irq_save(*flags);
>
> 	local_irq_disable();
>
> >  	addr = (unsigned long) kmap_atomic(bh->b_page, KM_BH_IRQ);
> >
> > -	if (addr & ~PAGE_MASK)
> > -		BUG();
> > +	BUG_ON (addr & ~PAGE_MASK);
> >
> >  	return (char *) addr + bh_offset(bh);
> >  }
> > @@ -58,7 +46,7 @@
> >  	unsigned long ptr = (unsigned long) buffer & PAGE_MASK;
> >
> >  	kunmap_atomic((void *) ptr, KM_BH_IRQ);
> > -	__restore_flags(*flags);
> > +	local_irq_restore(*flags);
	local_irq_enable();

^ isn't this missing too with your suggested one-liner?

ciao, Marc

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Jens Axboe]

On Thu, Mar 13 2003, Marc-Christian Petersen wrote:
> On Thursday 13 March 2003 10:26, Jens Axboe wrote:
> 
> Hi Jens,
> 
> > There's a tiny bit missing from your patch:
> > > -	 * it's a highmem page
> > > -	 */
> > > -	__cli();
> > > +	local_irq_save(*flags);
> >
> > 	local_irq_disable();
> >
> > >  	addr = (unsigned long) kmap_atomic(bh->b_page, KM_BH_IRQ);
> > >
> > > -	if (addr & ~PAGE_MASK)
> > > -		BUG();
> > > +	BUG_ON (addr & ~PAGE_MASK);
> > >
> > >  	return (char *) addr + bh_offset(bh);
> > >  }
> > > @@ -58,7 +46,7 @@
> > >  	unsigned long ptr = (unsigned long) buffer & PAGE_MASK;
> > >
> > >  	kunmap_atomic((void *) ptr, KM_BH_IRQ);
> > > -	__restore_flags(*flags);
> > > +	local_irq_restore(*flags);
> 	local_irq_enable();
> 
> ^ isn't this missing too with your suggested one-liner?

no, the local_irq_restore() brings back the irq flags from before we did
the irq disable. if interrupts were disabled before bh_kmap_irq() was
called, we must not enable them. basically, maintain the same flags.

-- 
Jens Axboe

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Marc-Christian Petersen]

On Thursday 13 March 2003 11:28, Jens Axboe wrote:

Hi Jens,

> > 	local_irq_enable();
> > ^ isn't this missing too with your suggested one-liner?
> no, the local_irq_restore() brings back the irq flags from before we did
> the irq disable. if interrupts were disabled before bh_kmap_irq() was
> called, we must not enable them. basically, maintain the same flags.
hmm, then I am blind 8-) ... Thanks.

ciao, Marc

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Joe Korty]

> I fixed this in 2.5 ages ago, just didn't get it in 2.4 block-highmem...
> There's a tiny bit missing from your patch:
> 
> > +	local_irq_save(*flags);
> 	local_irq_disable();
> 
> other than that it's fine. See 2.5 for reference.

local_irq_save() does a local_irq_disable() as part of its function.
Joe

Re: [PATCH] bug in 2.4 bh_kmap_irq() breaks IDE under preempt patch

[Jens Axboe]

On Thu, Mar 13 2003, Joe Korty wrote:
> > I fixed this in 2.5 ages ago, just didn't get it in 2.4 block-highmem...
> > There's a tiny bit missing from your patch:
> > 
> > > +	local_irq_save(*flags);
> > 	local_irq_disable();
> > 
> > other than that it's fine. See 2.5 for reference.
> 
> local_irq_save() does a local_irq_disable() as part of its function.

indeed, you are right.

-- 
Jens Axboe