[PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Jakub W. Jozwicki]

Change locking surrounding kmap_atomic from local_irqsave to 
local_irqsave_nort. This fixes issues with PREEMPT_RT.
 
Signed-off-by:  Jakub Jozwicki <jozwicki@aster.pl>

--- linux-2.6.25.10/drivers/ata/libata-core.c	2008-07-03 05:46:47.000000000 
+0200
+++ linux-2.6.25.10-rt7/drivers/ata/libata-core.c	2008-07-12 
23:59:33.132140258 +0200
@@ -5157,14 +5157,14 @@
 		unsigned long flags;
 
 		/* FIXME: use a bounce buffer */
-		local_irq_save(flags);
+		local_irq_save_nort(flags);
 		buf = kmap_atomic(page, KM_IRQ0);
 
 		/* do the actual data transfer */
 		ap->ops->data_xfer(qc->dev, buf + offset, qc->sect_size, do_write);
 
 		kunmap_atomic(buf, KM_IRQ0);
-		local_irq_restore(flags);
+		local_irq_restore_nort(flags);
 	} else {
 		buf = page_address(page);
 		ap->ops->data_xfer(qc->dev, buf + offset, qc->sect_size, do_write);
@@ -5294,14 +5294,14 @@
 		unsigned long flags;
 
 		/* FIXME: use bounce buffer */
-		local_irq_save(flags);
+		local_irq_save_nort(flags);
 		buf = kmap_atomic(page, KM_IRQ0);
 
 		/* do the actual data transfer */
 		consumed = ap->ops->data_xfer(dev,  buf + offset, count, rw);
 
 		kunmap_atomic(buf, KM_IRQ0);
-		local_irq_restore(flags);
+		local_irq_restore_nort(flags);
 	} else {
 		buf = page_address(page);
 		consumed = ap->ops->data_xfer(dev,  buf + offset, count, rw);

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Andrew Morton]

On Sun, 13 Jul 2008 01:27:36 +0200 "Jakub W. Jozwicki" <jozwicki@aster.pl> wrote:

> Change locking surrounding kmap_atomic from local_irqsave to 
> local_irqsave_nort. This fixes issues with PREEMPT_RT.
>  

I assume this is a -rt kernel patch.

> 
> --- linux-2.6.25.10/drivers/ata/libata-core.c	2008-07-03 05:46:47.000000000 
> +0200
> +++ linux-2.6.25.10-rt7/drivers/ata/libata-core.c	2008-07-12 
> 23:59:33.132140258 +0200
> @@ -5157,14 +5157,14 @@
>  		unsigned long flags;
>  
>  		/* FIXME: use a bounce buffer */
> -		local_irq_save(flags);
> +		local_irq_save_nort(flags);

eww.  If we're going to have to do stuff like this in mainline then
there might be a revolt.

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Alan Cox]

On Sun, 13 Jul 2008 01:27:36 +0200
"Jakub W. Jozwicki" <jozwicki@aster.pl> wrote:

> Change locking surrounding kmap_atomic from local_irqsave to 
> local_irqsave_nort. This fixes issues with PREEMPT_RT.
>  
> Signed-off-by:  Jakub Jozwicki <jozwicki@aster.pl>

The right way to fix this is to use a bounce buffer. I sent Jeff patches
to do that a couple of times. I'm not sure what has happened to them
since ? So NAK this, we should apply the patch to actually fix the bug
instead.

Jeff ?

Alan

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Jakub W. Jozwicki]

Sunday, 13 of July 2008 10:26:30 Andrew Morton napisał(a):
> On Sun, 13 Jul 2008 01:27:36 +0200 "Jakub W. Jozwicki" <jozwicki@aster.pl> 
wrote:
> > Change locking surrounding kmap_atomic from local_irqsave to
> > local_irqsave_nort. This fixes issues with PREEMPT_RT.
>
> I assume this is a -rt kernel patch.
>
> > --- linux-2.6.25.10/drivers/ata/libata-core.c	2008-07-03
> > 05:46:47.000000000 +0200
> > +++ linux-2.6.25.10-rt7/drivers/ata/libata-core.c	2008-07-12
> > 23:59:33.132140258 +0200
> > @@ -5157,14 +5157,14 @@
> >  		unsigned long flags;
> >
> >  		/* FIXME: use a bounce buffer */
> > -		local_irq_save(flags);
> > +		local_irq_save_nort(flags);
>
> eww.  If we're going to have to do stuff like this in mainline then
> there might be a revolt.

Sorry, this was for -rt only.

[   17.012011] BUG: sleeping function called from invalid context IRQ-14(5732) 
at arch/x86/mm/highmem_32.c:8
[   17.012011] in_atomic():0 [00000000], irqs_disabled():1
[   17.012011] Pid: 5732, comm: IRQ-14 Not tainted 2.6.25.10-rtXXX #11
[   17.012011]  [<c0120fc4>] __might_sleep+0xf1/0xf8
[   17.012011]  [<c011c035>] kmap+0x47/0x5a
[   17.012011]  [<c033e728>] ata_hsm_move+0x3d7/0x657
[   17.012011]  [<c0342fd6>] ata_interrupt+0x14e/0x1cb
[   17.012011]  [<c015c72c>] handle_IRQ_event+0x4e/0xd1
[   17.012011]  [<c015d266>] do_irqd+0x126/0x224
[   17.012011]  [<c015d140>] ? do_irqd+0x0/0x224
[   17.012011]  [<c013ae9d>] kthread+0x3b/0x62
[   17.012011]  [<c013ae62>] ? kthread+0x0/0x62
[   17.012011]  [<c0108287>] kernel_thread_helper+0x7/0x10
[   17.012011]  =======================

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Jakub W. Jozwicki]

Sunday, 13 of July 2008 12:25:43 Alan Cox napisał(a):
> On Sun, 13 Jul 2008 01:27:36 +0200
>
> "Jakub W. Jozwicki" <jozwicki@aster.pl> wrote:
> > Change locking surrounding kmap_atomic from local_irqsave to
> > local_irqsave_nort. This fixes issues with PREEMPT_RT.
> >
> > Signed-off-by:  Jakub Jozwicki <jozwicki@aster.pl>
>
> The right way to fix this is to use a bounce buffer. I sent Jeff patches
> to do that a couple of times. I'm not sure what has happened to them
> since ? So NAK this, we should apply the patch to actually fix the bug
> instead.
>
> Jeff ?
>
> Alan

Sent patch was for:

[   16.787021] BUG: sleeping function called from invalid context IRQ-14(5730) 
at arch/x86/mm/highmem_32.c:8
[   16.787021] in_atomic():0 [00000000], irqs_disabled():1
[   16.787021] Pid: 5730, comm: IRQ-14 Not tainted 2.6.25.10-rtXXX #12
[   16.787021]  [<c0120fc4>] __might_sleep+0xf1/0xf8
[   16.787021]  [<c011c035>] kmap+0x47/0x5a
[   16.787021]  [<c03442b2>] ata_scsi_rbuf_get+0x23/0x31
[   16.787021]  [<c0345975>] atapi_qc_complete+0x252/0x2c5
[   16.787021]  [<c012edee>] ? irq_exit+0x7b/0x7e
[   16.787021]  [<c0109727>] ? do_IRQ+0x6e/0x87
[   16.787021]  [<c033d041>] __ata_qc_complete+0x8e/0x93
[   16.787021]  [<c033ddfb>] ata_qc_complete+0x182/0x188
[   16.787021]  [<c033e188>] ata_hsm_qc_complete+0xa1/0xa5
[   16.787021]  [<c033e8fd>] ata_hsm_move+0x5dd/0x628
[   16.787021]  [<c0342f76>] ata_interrupt+0x14e/0x1cb
[   16.787021]  [<c015c72c>] handle_IRQ_event+0x4e/0xd1
[   16.787021]  [<c015d266>] do_irqd+0x126/0x224
[   16.787021]  [<c015d140>] ? do_irqd+0x0/0x224
[   16.787021]  [<c013ae9d>] kthread+0x3b/0x62
[   16.787021]  [<c013ae62>] ? kthread+0x0/0x62
[   16.787021]  [<c0108287>] kernel_thread_helper+0x7/0x10
[   16.787021]  =======================

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Alan Cox]

> Sent patch was for:
> 
> [   16.787021] BUG: sleeping function called from invalid context IRQ-14(5730) 
> at arch/x86/mm/highmem_32.c:8

Fine - but the right fix is still to apply the patches that use a bounce
buffer, otherwise you might paper over the warning but you'll trash rt
performance. It needs fixing anyway especially with Tejun's changes to
the ATAPI logic to use PIO more often

Alan

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Jeremy Fitzhardinge]

Jakub W. Jozwicki wrote:
>
> Sorry, this was for -rt only.
>
> [   17.012011] BUG: sleeping function called from invalid context IRQ-14(5732) 
> at arch/x86/mm/highmem_32.c:8
> [   17.012011] in_atomic():0 [00000000], irqs_disabled():1
> [   17.012011] Pid: 5732, comm: IRQ-14 Not tainted 2.6.25.10-rtXXX #11
> [   17.012011]  [<c0120fc4>] __might_sleep+0xf1/0xf8
> [   17.012011]  [<c011c035>] kmap+0x47/0x5a
>   

The subject says kmap_atomic, but this is kmap.  It definitely makes no 
sense to call kmap in an IRQ, regardless of the locking.  There seems to 
be a larger structural problem here.

> [   17.012011]  [<c033e728>] ata_hsm_move+0x3d7/0x657
> [   17.012011]  [<c0342fd6>] ata_interrupt+0x14e/0x1cb
> [   17.012011]  [<c015c72c>] handle_IRQ_event+0x4e/0xd1
> [   17.012011]  [<c015d266>] do_irqd+0x126/0x224
> [   17.012011]  [<c015d140>] ? do_irqd+0x0/0x224
> [   17.012011]  [<c013ae9d>] kthread+0x3b/0x62
> [   17.012011]  [<c013ae62>] ? kthread+0x0/0x62
> [   17.012011]  [<c0108287>] kernel_thread_helper+0x7/0x10
> [   17.012011]  =======================
>   

    J

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Jakub W. Jozwicki]

Sunday, 13 of July 2008 19:46:29 Jeremy Fitzhardinge napisał(a):
> Jakub W. Jozwicki wrote:
> > Sorry, this was for -rt only.
> >
> > [   17.012011] BUG: sleeping function called from invalid context
> > IRQ-14(5732) at arch/x86/mm/highmem_32.c:8
> > [   17.012011] in_atomic():0 [00000000], irqs_disabled():1
> > [   17.012011] Pid: 5732, comm: IRQ-14 Not tainted 2.6.25.10-rtXXX #11
> > [   17.012011]  [<c0120fc4>] __might_sleep+0xf1/0xf8
> > [   17.012011]  [<c011c035>] kmap+0x47/0x5a
>
> The subject says kmap_atomic, but this is kmap.  It definitely makes no
> sense to call kmap in an IRQ, regardless of the locking.  There seems to
> be a larger structural problem here.
>

--- linux-2.6.25.8-rt7.orig/include/asm-x86/highmem.h   2008-06-23 
19:12:47.000000000 -0400
+++ linux-2.6.25.8-rt7/include/asm-x86/highmem.h        2008-06-23 
19:13:58.000000000 -0400


+/*
+ * on PREEMPT_RT kmap_atomic() is a wrapper that uses kmap():
+ */
+#ifdef CONFIG_PREEMPT_RT
+# define kmap_atomic_prot(page, type, prot)    ({ pagefault_disable(); 
kmap(page); })
+# define kmap_atomic(page, type)       ({ pagefault_disable(); kmap(page); })
+# define kmap_atomic_pfn(pfn, type)    kmap(pfn_to_page(pfn))
+# define kunmap_atomic(kvaddr, type)   do { pagefault_enable(); 
kunmap_virt(kvaddr); } while(0)
+# define kmap_atomic_to_page(kvaddr)   kmap_to_page(kvaddr)
+#else
+# define kmap_atomic_prot(page, type, prot)    __kmap_atomic_prot(page, type, 
prot)
+# define kmap_atomic(page, type)       __kmap_atomic(page, type)
+# define kmap_atomic_pfn(pfn, type)    __kmap_atomic_pfn(pfn, type)
+# define kunmap_atomic(kvaddr, type)   __kunmap_atomic(kvaddr, type)
+# define kmap_atomic_to_page(kvaddr)   __kmap_atomic_to_page(kvaddr)
+#endif
+

Re: [PATCH 2.6.25.10 1/2] libata: fix locking for kmap_atomic

[Jeremy Fitzhardinge]

Jakub W. Jozwicki wrote:
> Sunday, 13 of July 2008 19:46:29 Jeremy Fitzhardinge napisał(a):
>   
>> Jakub W. Jozwicki wrote:
>>     
>>> Sorry, this was for -rt only.
>>>
>>> [   17.012011] BUG: sleeping function called from invalid context
>>> IRQ-14(5732) at arch/x86/mm/highmem_32.c:8
>>> [   17.012011] in_atomic():0 [00000000], irqs_disabled():1
>>> [   17.012011] Pid: 5732, comm: IRQ-14 Not tainted 2.6.25.10-rtXXX #11
>>> [   17.012011]  [<c0120fc4>] __might_sleep+0xf1/0xf8
>>> [   17.012011]  [<c011c035>] kmap+0x47/0x5a
>>>       
>> The subject says kmap_atomic, but this is kmap.  It definitely makes no
>> sense to call kmap in an IRQ, regardless of the locking.  There seems to
>> be a larger structural problem here.
>>
>>     
>
> --- linux-2.6.25.8-rt7.orig/include/asm-x86/highmem.h   2008-06-23 
> 19:12:47.000000000 -0400
> +++ linux-2.6.25.8-rt7/include/asm-x86/highmem.h        2008-06-23 
> 19:13:58.000000000 -0400
>
>
> +/*
> + * on PREEMPT_RT kmap_atomic() is a wrapper that uses kmap():
>   

OK, I guess the "structural problem" is RT ;)

    J