Re: 2.6.22-rc6 bad page error

Re: 2.6.22-rc6 bad page error

[Michal Piotrowski]

Hi Bob,

On 05/07/07, Bob Tracy <rct@gherkin.frus.com> wrote:
> This happened on an Alpha computer (433au) running 2.6.22-rc6.

Can you reproduce this bug on 2.6.22-rc7-git6?

>  The
> error was triggered when I started up a NX KDE session, which I've
> done successfully on all occasions prior to this one.  In other words,
> I wasn't doing anything out of the ordinary...  The machine is still up
> and running as I type this, and can't/won't be rebooted for a few hours
> if there's any other troubleshooting information I need to retrieve that
> might be helpful.
>
> >Bad page state in process 'gnome-terminal'
> >page:fffffc0000a407f0 flags:0x000000000000000c mapping:0000000000000000 mapcount:1 count:1
> >Trying to fix it up, but a reboot is needed
> >Backtrace:
> >fffffc000960bc00 0000000000000000 0000000000000001 0000000000000001
> >       fffffc0000356e40 fffffc0000a407f0 fffffc0000356ee8 fffffc002399b0a0
> >       fffffc000f784a68 00000000000280d2 0000000000000000 000000012029a3c8
> >       fffffc00006fb6c0 fffffc001b0667a0 0000000000000000 0000000000000000
> >       fffffc00006fb6c8 00000000000280d2 0000000000000000 0000000000000044
> >       fffffc00006fb1f0 0000000000000000 0000000000000000 0000000000000001
> >Trace:
> >[<fffffc0000356e40>] get_page_from_freelist+0x420/0x430
> >[<fffffc0000356ee8>] __alloc_pages+0x98/0x3b0
> >[<fffffc00003638fc>] __handle_mm_fault+0x65c/0xb50
> >[<fffffc0000353554>] filemap_nopage+0x1a4/0x3d0
> >[<fffffc0000363534>] __handle_mm_fault+0x294/0xb50
> >[<fffffc000031e904>] do_page_fault+0x2d4/0x490
> >[<fffffc0000310bdc>] entMM+0x9c/0xc0
> >[<fffffc000032b0c0>] __do_softirq+0x90/0x120
> >[<fffffc000032b1b4>] do_softirq+0x64/0x70
> >[<fffffc000032b1b4>] do_softirq+0x64/0x70
> >[<fffffc0000315d94>] handle_irq+0x74/0xc0
> >
> >Bad page state in process 'syslogd'
> >page:fffffc0000a40828 flags:0x000000000000000c mapping:0000000000000000 mapcount:1 count:1
> >Trying to fix it up, but a reboot is needed
> >Backtrace:
> >fffffc002242fa98 0000000000000000 0000000000000001 0000000000000001
> >       fffffc0000356e40 fffffc0000a40828 fffffc0000356ee8 fffffc0023f6e0c0
> >       0000000000000003 00000000000000d0 0000000000000000 00000000000000d0
> >       fffffc00006fb6c0 0000000000000020 0000000000000000 0000000000000000
> >       fffffc00006fb6c8 00000000000200d0 0000000000000000 0000000000000044
> >       fffffc00006fb1f0 0000000000000000 0000000000000000 0000000000000001
> >Trace:
> >[<fffffc0000356e40>] get_page_from_freelist+0x420/0x430
> >[<fffffc0000356ee8>] __alloc_pages+0x98/0x3b0
> >[<fffffc0000375b30>] cache_alloc_refill+0x390/0x690
> >[<fffffc0000387018>] __link_path_walk+0x8a8/0x10b0
> >[<fffffc0000387374>] __link_path_walk+0xc04/0x10b0
> >[<fffffc0000375798>] kmem_cache_alloc+0x78/0x80
> >[<fffffc0000322e04>] copy_process+0xb4/0x11a0
> >[<fffffc0000323f88>] do_fork+0x98/0x330
> >[<fffffc0000323f40>] do_fork+0x50/0x330
> >[<fffffc0000313340>] alpha_clone+0x40/0x60
> >[<fffffc0000388f5c>] open_namei+0xbc/0x820
> >[<fffffc0000311480>] sys_clone+0x10/0x20
> >[<fffffc0000310f84>] entSys+0xa4/0xc0
>
> --
> -----------------------------------------------------------------------
> Bob Tracy               | "Eagles may soar, but weasels don't get
> rct@frus.com            |  sucked into jet engines."       --Anon
> -----------------------------------------------------------------------
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Regards,
Michal

-- 
LOG
http://www.stardust.webpages.pl/log/

Re: 2.6.22-rc6 bad page error

[Bob Tracy]

Michal Piotrowski wrote:
> Hi Bob,
> 
> On 05/07/07, Bob Tracy <rct@gherkin.frus.com> wrote:
> > This happened on an Alpha computer (433au) running 2.6.22-rc6.
> 
> Can you reproduce this bug on 2.6.22-rc7-git6?
> 
> >  The
> > error was triggered when I started up a NX KDE session, which I've
> > done successfully on all occasions prior to this one.  In other words,
> > I wasn't doing anything out of the ordinary...  The machine is still up
> > and running as I type this, and can't/won't be rebooted for a few hours
> > if there's any other troubleshooting information I need to retrieve that
> > might be helpful.

ACK on the suggestion to try -rc7-git6, but I'll probably wait until
Monday to give it a try.  The client side of my NX "test fixture" is
at my day job location :-).

-- 
-----------------------------------------------------------------------
Bob Tracy               | "Eagles may soar, but weasels don't get
rct@frus.com            |  sucked into jet engines."       --Anon
-----------------------------------------------------------------------

Re: 2.6.22-rc6 bad page error

[Hugh Dickins]

On Sat, 7 Jul 2007, Bob Tracy wrote:
> Michal Piotrowski wrote:
> > On 05/07/07, Bob Tracy <rct@gherkin.frus.com> wrote:
> > > This happened on an Alpha computer (433au) running 2.6.22-rc6.
> > 
> > Can you reproduce this bug on 2.6.22-rc7-git6?

My guess is that you'll find yes.

> > 
> > >  The
> > > error was triggered when I started up a NX KDE session, which I've
> > > done successfully on all occasions prior to this one.  In other words,
> > > I wasn't doing anything out of the ordinary...  The machine is still up
> > > and running as I type this, and can't/won't be rebooted for a few hours
> > > if there's any other troubleshooting information I need to retrieve that
> > > might be helpful.
> 
> ACK on the suggestion to try -rc7-git6, but I'll probably wait until
> Monday to give it a try.  The client side of my NX "test fixture" is
> at my day job location :-).

Fair enough: and I'm expecting this to be a regression we introduced in
2.6.15, rather than recently in 2.6.22 (now, that's better isn't it ;-?)

Helpful bad page state messages from your original post:

>Bad page state in process 'gnome-terminal'
>page:fffffc0000a407f0 flags:0x000000000000000c mapping:0000000000000000 mapcount:1 count:1
>Bad page state in process 'syslogd'
>page:fffffc0000a40828 flags:0x000000000000000c mapping:0000000000000000 mapcount:1 count:1

And very helpful info from your followup:

> Same type of "stuff" happening with 2.6.22-rc7.  I've managed to narrow
> it down somewhat.  The page corruption happens only when I've got the sound
> driver loaded: might be a DMA kind of thing that's killing me.  Anyone got
> any ideas how I might narrow it down further?  The sound modules loaded
> include ALSA's "snd_es18xx" with all the various OSS-compatibility modules.

I think sound/isa/es18xx.c's
	snd_pcm_lib_preallocate_pages_for_all(pcm, SNDRV_DMA_TYPE_DEV,
will take it to sound/core/memalloc.c's
	res = dma_alloc_coherent(dev, PAGE_SIZE << pg, dma, gfp_flags);
where we've carefully included __GFP_COMP in gfp_flags to avoid this
kind of problem (replacing the pre-2.6.15 use of PageReserved).

> At the risk of making an inflammatory comment, I don't recall this kind of
> thing happening when I was using the equivalent OSS driver.  N.B.: the ALSA
> driver only recently (in the past few months) became usable on the Alpha,
> and I made the switch to ALSA as soon as it was feasible to do so.  I may
> try going back to the OSS driver (if it's still available) to see if the
> problem goes away.  That would at least confirm my suspicions as to where
> the problem lies.

No, don't blame ALSA for this.  Blame me or Nick for removing the
special PageReserved usage, or Alpha for ignoring our gfp_flags:

  #define dma_alloc_coherent(dev, size, addr, gfp)	\
		pci_alloc_consistent(alpha_gendev_to_pci(dev), size, addr)

When you get a chance, please would you try patch below?  It's not ideal:
trying to satisfy a high-order allocation with GFP_ATOMIC is difficult
once you get beyond system startup, but apparently that part worked for
you before; and I don't think it'll do any harm - there were reasons
we couldn't just add __GFP_COMP behaviour everywhere, but I don't
think those could apply to the Alpha pci_alloc_consistent.

It would be a lot better if an Alpha (CONFIG_PCI=y) dma_alloc_coherent
were written to respect the dma flags passed in (masking off __GFP_DMA
and __GFP_HIGHMEM? other arches seem to do that).  And I wonder if its
pci_alloc_consistent couldn't use GFP_KERNEL rather than GFP_ATOMIC,
other arches seem to manage that.  But I've no Alpha nor Alpha
experience, I wouldn't dare to offer more than this one-liner.

Thanks for reporting: please let us all know whether this
patch does fix your problem: I may be guessing wrong.

Hugh

--- 2.6.22-rc7/arch/alpha/kernel/pci_iommu.c	2007-06-05 06:19:19.000000000 +0100
+++ linux/arch/alpha/kernel/pci_iommu.c	2007-07-07 15:00:04.000000000 +0100
@@ -402,7 +402,7 @@ pci_alloc_consistent(struct pci_dev *pde
 {
 	void *cpu_addr;
 	long order = get_order(size);
-	gfp_t gfp = GFP_ATOMIC;
+	gfp_t gfp = GFP_ATOMIC|__GFP_COMP;
 
 try_again:
 	cpu_addr = (void *)__get_free_pages(gfp, order);

Re: 2.6.22-rc6 bad page error

[Bob Tracy]

Hugh Dickins wrote:
> (...) I'm expecting this to be a regression we introduced in
> 2.6.15, rather than recently in 2.6.22 (now, that's better isn't it ;-?)

I was reasonably certain this wasn't a recent regression.  I share a set
of speakers between two machines, and I hadn't used sound on the Alpha
in quite a while.

> Thanks for reporting: please let us all know whether this
> patch does fix your problem: I may be guessing wrong.

ACK.  I'll build the kernel today, and fire up the test fixture
tomorrow.

-- 
-----------------------------------------------------------------------
Bob Tracy               | "Eagles may soar, but weasels don't get
rct@frus.com            |  sucked into jet engines."       --Anon
-----------------------------------------------------------------------

Re: 2.6.22-rc6 bad page error

[Bob Tracy]

Hugh Dickins wrote:
> (...) I'm expecting this to be a regression we introduced in
> 2.6.15, rather than recently in 2.6.22 (now, that's better isn't it ;-?)
> (...)
> I think sound/isa/es18xx.c's
> 	snd_pcm_lib_preallocate_pages_for_all(pcm, SNDRV_DMA_TYPE_DEV,
> will take it to sound/core/memalloc.c's
> 	res = dma_alloc_coherent(dev, PAGE_SIZE << pg, dma, gfp_flags);
> where we've carefully included __GFP_COMP in gfp_flags to avoid this
> kind of problem (replacing the pre-2.6.15 use of PageReserved).
> (...)
> Blame me or Nick for removing the
> special PageReserved usage, or Alpha for ignoring our gfp_flags:
> 
>   #define dma_alloc_coherent(dev, size, addr, gfp)	\
> 		pci_alloc_consistent(alpha_gendev_to_pci(dev), size, addr)
> 
> When you get a chance, please would you try patch below?
> 
> --- 2.6.22-rc7/arch/alpha/kernel/pci_iommu.c	2007-06-05 06:19:19.000000000 +0100
> +++ linux/arch/alpha/kernel/pci_iommu.c	2007-07-07 15:00:04.000000000 +0100

That seems to have done the trick.  Normally, I get the "bad page"
errors on the second NX session, but I'm on the third session of the
day (thus far), and everything seems to be ok as far as I can tell.

-- 
-----------------------------------------------------------------------
Bob Tracy               | "Eagles may soar, but weasels don't get
rct@frus.com            |  sucked into jet engines."       --Anon
-----------------------------------------------------------------------

Re: 2.6.22-rc6 bad page error

[Hugh Dickins]

On Mon, 9 Jul 2007, Bob Tracy wrote:
> > 
> > --- 2.6.22-rc7/arch/alpha/kernel/pci_iommu.c	2007-06-05 06:19:19.000000000 +0100
> > +++ linux/arch/alpha/kernel/pci_iommu.c	2007-07-07 15:00:04.000000000 +0100
> 
> That seems to have done the trick.  Normally, I get the "bad page"
> errors on the second NX session, but I'm on the third session of the
> day (thus far), and everything seems to be ok as far as I can tell.

Thanks a lot for the feedback, that tells us we've made the right
guesses, and my __GFP_COMP patch should be a good bandaid for you;
but for the right patch, let's wait to hear from Ivan.

Hugh

Re: 2.6.22-rc6 bad page error

[Ivan Kokshaysky]

On Tue, Jul 10, 2007 at 09:25:50PM +0100, Hugh Dickins wrote:
> On Mon, 9 Jul 2007, Bob Tracy wrote:
> > That seems to have done the trick.  Normally, I get the "bad page"
> > errors on the second NX session, but I'm on the third session of the
> > day (thus far), and everything seems to be ok as far as I can tell.
> 
> Thanks a lot for the feedback, that tells us we've made the right
> guesses, and my __GFP_COMP patch should be a good bandaid for you;
> but for the right patch, let's wait to hear from Ivan.

Here it is. Bob, can you test this one?

---

Take care of DMA flags passed to dma_alloc_coherent().
Particularly ALSA needs that.

Signed-off-by: Ivan Kokshaysky <ink@jurassic.park.msu.ru>

Ivan.

--- 2.6.22/arch/alpha/kernel/pci_iommu.c	Mon Jul  9 03:32:17 2007
+++ linux/arch/alpha/kernel/pci_iommu.c	Wed Jul 11 14:40:37 2007
@@ -398,11 +398,13 @@ EXPORT_SYMBOL(pci_unmap_page);
    else DMA_ADDRP is undefined.  */
 
 void *
-pci_alloc_consistent(struct pci_dev *pdev, size_t size, dma_addr_t *dma_addrp)
+__pci_alloc_consistent(struct pci_dev *pdev, size_t size,
+		       dma_addr_t *dma_addrp, gfp_t gfp)
 {
 	void *cpu_addr;
 	long order = get_order(size);
-	gfp_t gfp = GFP_ATOMIC;
+
+	gfp &= ~GFP_DMA;
 
 try_again:
 	cpu_addr = (void *)__get_free_pages(gfp, order);
@@ -432,7 +434,7 @@ try_again:
 
 	return cpu_addr;
 }
-EXPORT_SYMBOL(pci_alloc_consistent);
+EXPORT_SYMBOL(__pci_alloc_consistent);
 
 /* Free and unmap a consistent DMA buffer.  CPU_ADDR and DMA_ADDR must
    be values that were returned from pci_alloc_consistent.  SIZE must
--- 2.6.22/include/asm-alpha/pci.h	Mon Jul  9 03:32:17 2007
+++ linux/include/asm-alpha/pci.h	Wed Jul 11 14:40:37 2007
@@ -75,7 +75,13 @@ extern inline void pcibios_penalize_isa_
    successful and sets *DMA_ADDRP to the pci side dma address as well,
    else DMA_ADDRP is undefined.  */
 
-extern void *pci_alloc_consistent(struct pci_dev *, size_t, dma_addr_t *);
+extern void *__pci_alloc_consistent(struct pci_dev *, size_t,
+				    dma_addr_t *, gfp_t);
+static inline void *
+pci_alloc_consistent(struct pci_dev *dev, size_t size, dma_addr_t *dma)
+{
+	return __pci_alloc_consistent(dev, size, dma, GFP_ATOMIC);
+}
 
 /* Free and unmap a consistent DMA buffer.  CPU_ADDR and DMA_ADDR must
    be values that were returned from pci_alloc_consistent.  SIZE must
--- 2.6.22/include/asm-alpha/dma-mapping.h	Mon Jul  9 03:32:17 2007
+++ linux/include/asm-alpha/dma-mapping.h	Wed Jul 11 14:40:37 2007
@@ -11,7 +11,7 @@
 #define dma_unmap_single(dev, addr, size, dir)		\
 		pci_unmap_single(alpha_gendev_to_pci(dev), addr, size, dir)
 #define dma_alloc_coherent(dev, size, addr, gfp)	\
-		pci_alloc_consistent(alpha_gendev_to_pci(dev), size, addr)
+	      __pci_alloc_consistent(alpha_gendev_to_pci(dev), size, addr, gfp)
 #define dma_free_coherent(dev, size, va, addr)		\
 		pci_free_consistent(alpha_gendev_to_pci(dev), size, va, addr)
 #define dma_map_page(dev, page, off, size, dir)		\

Re: 2.6.22-rc6 bad page error

[Hugh Dickins]

On Wed, 11 Jul 2007, Ivan Kokshaysky wrote:
> On Tue, Jul 10, 2007 at 09:25:50PM +0100, Hugh Dickins wrote:
> > On Mon, 9 Jul 2007, Bob Tracy wrote:
> > > That seems to have done the trick.  Normally, I get the "bad page"
> > > errors on the second NX session, but I'm on the third session of the
> > > day (thus far), and everything seems to be ok as far as I can tell.
> > 
> > Thanks a lot for the feedback, that tells us we've made the right
> > guesses, and my __GFP_COMP patch should be a good bandaid for you;
> > but for the right patch, let's wait to hear from Ivan.
> 
> Here it is. Bob, can you test this one?
> 
> ---
> 
> Take care of DMA flags passed to dma_alloc_coherent().
> Particularly ALSA needs that.
> 
> Signed-off-by: Ivan Kokshaysky <ink@jurassic.park.msu.ru>

Yes, I agree that's the right way to go.  Aside from fixing Bob's
crash, it also liberates ALSA from needing GFP_ATOMIC there on Alpha.
Couple of GFP_ notes below.

Hugh

> 
> Ivan.
> 
> --- 2.6.22/arch/alpha/kernel/pci_iommu.c	Mon Jul  9 03:32:17 2007
> +++ linux/arch/alpha/kernel/pci_iommu.c	Wed Jul 11 14:40:37 2007
> @@ -398,11 +398,13 @@ EXPORT_SYMBOL(pci_unmap_page);
>     else DMA_ADDRP is undefined.  */
>  
>  void *
> -pci_alloc_consistent(struct pci_dev *pdev, size_t size, dma_addr_t *dma_addrp)
> +__pci_alloc_consistent(struct pci_dev *pdev, size_t size,
> +		       dma_addr_t *dma_addrp, gfp_t gfp)
>  {
>  	void *cpu_addr;
>  	long order = get_order(size);
> -	gfp_t gfp = GFP_ATOMIC;
> +
> +	gfp &= ~GFP_DMA;

The canonical form for this masking, even on arches without HIGHMEM,
appears to be:
	gfp &= ~(__GFP_DMA | __GFP_HIGHMEM);
But I guess that won't make any actual difference.

>  
>  try_again:
>  	cpu_addr = (void *)__get_free_pages(gfp, order);
> @@ -432,7 +434,7 @@ try_again:
>  
>  	return cpu_addr;
>  }
> -EXPORT_SYMBOL(pci_alloc_consistent);
> +EXPORT_SYMBOL(__pci_alloc_consistent);
>  
>  /* Free and unmap a consistent DMA buffer.  CPU_ADDR and DMA_ADDR must
>     be values that were returned from pci_alloc_consistent.  SIZE must
> --- 2.6.22/include/asm-alpha/pci.h	Mon Jul  9 03:32:17 2007
> +++ linux/include/asm-alpha/pci.h	Wed Jul 11 14:40:37 2007
> @@ -75,7 +75,13 @@ extern inline void pcibios_penalize_isa_
>     successful and sets *DMA_ADDRP to the pci side dma address as well,
>     else DMA_ADDRP is undefined.  */
>  
> -extern void *pci_alloc_consistent(struct pci_dev *, size_t, dma_addr_t *);
> +extern void *__pci_alloc_consistent(struct pci_dev *, size_t,
> +				    dma_addr_t *, gfp_t);
> +static inline void *
> +pci_alloc_consistent(struct pci_dev *dev, size_t size, dma_addr_t *dma)
> +{
> +	return __pci_alloc_consistent(dev, size, dma, GFP_ATOMIC);

I was going to ask you why that needs to be GFP_ATOMIC on Alpha.
But find you're following the example of asm-generic and others.
So really should be asking Sparc how it gets away with GFP_KERNEL.

> +}
>  
>  /* Free and unmap a consistent DMA buffer.  CPU_ADDR and DMA_ADDR must
>     be values that were returned from pci_alloc_consistent.  SIZE must
> --- 2.6.22/include/asm-alpha/dma-mapping.h	Mon Jul  9 03:32:17 2007
> +++ linux/include/asm-alpha/dma-mapping.h	Wed Jul 11 14:40:37 2007
> @@ -11,7 +11,7 @@
>  #define dma_unmap_single(dev, addr, size, dir)		\
>  		pci_unmap_single(alpha_gendev_to_pci(dev), addr, size, dir)
>  #define dma_alloc_coherent(dev, size, addr, gfp)	\
> -		pci_alloc_consistent(alpha_gendev_to_pci(dev), size, addr)
> +	      __pci_alloc_consistent(alpha_gendev_to_pci(dev), size, addr, gfp)
>  #define dma_free_coherent(dev, size, va, addr)		\
>  		pci_free_consistent(alpha_gendev_to_pci(dev), size, va, addr)
>  #define dma_map_page(dev, page, off, size, dir)		\

Re: 2.6.22-rc6 bad page error

[Ivan Kokshaysky]

On Wed, Jul 11, 2007 at 06:07:59PM +0100, Hugh Dickins wrote:
> > +	gfp &= ~GFP_DMA;
> 
> The canonical form for this masking, even on arches without HIGHMEM,
> appears to be:
> 	gfp &= ~(__GFP_DMA | __GFP_HIGHMEM);
> But I guess that won't make any actual difference.

As far as I can see, none of 64-bit architectures masks the HIGHMEM
(or any other) bits. It's just not obvious because most of their
"alloc_coherent" routines are hidden in iommu_ops or something like that.

> > +	return __pci_alloc_consistent(dev, size, dma, GFP_ATOMIC);
> 
> I was going to ask you why that needs to be GFP_ATOMIC on Alpha.
> But find you're following the example of asm-generic and others.
> So really should be asking Sparc how it gets away with GFP_KERNEL.

According to DMA-mapping.txt GFP_ATOMIC is mandatory for
pci_alloc_consistent() since it can be called from interrupt context.
So sparc code is probably incorrect...

Ivan.

Re: 2.6.22-rc6 bad page error

[Hugh Dickins]

On Thu, 12 Jul 2007, Ivan Kokshaysky wrote:
> On Wed, Jul 11, 2007 at 06:07:59PM +0100, Hugh Dickins wrote:
> 
> > > +	return __pci_alloc_consistent(dev, size, dma, GFP_ATOMIC);
> > 
> > I was going to ask you why that needs to be GFP_ATOMIC on Alpha.
> > But find you're following the example of asm-generic and others.
> > So really should be asking Sparc how it gets away with GFP_KERNEL.
> 
> According to DMA-mapping.txt GFP_ATOMIC is mandatory for
> pci_alloc_consistent() since it can be called from interrupt context.
> So sparc code is probably incorrect...

Hi Bill, we've veered away from the original topic, and are wondering
how the sparc32 pci_alloc_consistent() gets away with using GFP_KERNEL,
when GFP_ATOMIC seems to be required?

Hugh