bad page when freeing preallocated pages

bad page when freeing preallocated pages

[Hans-Christian Egtvedt]

Hello

I'm currently developing an alsa driver for an audio DAC used on an
embedded platform. I have come upon an issue when I try to free my
preallocated pages.

Some code snips from the driver I assume is most relevant:

static void snd_at73c213_pcm_free(snd_pcm_t *pcm)
{
	struct snd_at73c213 *chip = snd_pcm_chip(pcm);
	if (chip->pcm != 0 ) {
		snd_pcm_lib_preallocate_free_for_all(chip->pcm);
		chip->pcm = NULL;
	}
}

static int __devinit snd_at73c213_new_pcm(struct snd_at73c213 *chip, int
device)
{
	snd_pcm_t *pcm;
	int retval;

	retval = snd_pcm_new(chip->card, "AT73C213 on STK1000", device, 1, 0,
&pcm);
	if (retval < 0)
		return retval;

	pcm->private_data = chip;
	pcm->private_free = snd_at73c213_pcm_free;
	pcm->info_flags = SNDRV_PCM_INFO_BLOCK_TRANSFER;
	strcpy(pcm->name, "AT73C213 on STK1000");
	chip->pcm = pcm;

	snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK, &at73c213_playback_ops);

	snd_pcm_lib_preallocate_pages_for_all(chip->pcm, SNDRV_DMA_TYPE_DEV,
snd_dma_continuous_data(GFP_KERNEL), 64 * 1024, 64 * 1024);

	return 0;
}

When removing the module from the kernel with rmmod I get alot of bad
pages, because the page is mapped when it's freed. The weird thing is
that the pages are mapped to address 0x00000000.

Example:
[  141.380000] Bad page state in process 'rmmod'
[17179650.992000] page:90200600 flags:0x00004000 mapping:00000000
mapcount:0 count:0

Sorry if my email is lacking information, I'll gladly provide more
information.

-- 
With kind regards,
Med vennlig hilsen,

Hans-Christian Egtvedt
Applications Engineer - AVR Applications Lab
Atmel Norway


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: bad page when freeing preallocated pages

[Takashi Iwai]

At Wed, 26 Apr 2006 10:30:30 +0200,
Hans-Christian Egtvedt wrote:
> 
> Hello
> 
> I'm currently developing an alsa driver for an audio DAC used on an
> embedded platform. I have come upon an issue when I try to free my
> preallocated pages.
> 
> Some code snips from the driver I assume is most relevant:
> 
> static void snd_at73c213_pcm_free(snd_pcm_t *pcm)
> {
> 	struct snd_at73c213 *chip = snd_pcm_chip(pcm);
> 	if (chip->pcm != 0 ) {
> 		snd_pcm_lib_preallocate_free_for_all(chip->pcm);
> 		chip->pcm = NULL;
> 	}
> }
> 
> static int __devinit snd_at73c213_new_pcm(struct snd_at73c213 *chip, int
> device)
> {
> 	snd_pcm_t *pcm;
> 	int retval;
> 
> 	retval = snd_pcm_new(chip->card, "AT73C213 on STK1000", device, 1, 0,
> &pcm);
> 	if (retval < 0)
> 		return retval;
> 
> 	pcm->private_data = chip;
> 	pcm->private_free = snd_at73c213_pcm_free;
> 	pcm->info_flags = SNDRV_PCM_INFO_BLOCK_TRANSFER;
> 	strcpy(pcm->name, "AT73C213 on STK1000");
> 	chip->pcm = pcm;
> 
> 	snd_pcm_set_ops(pcm, SNDRV_PCM_STREAM_PLAYBACK, &at73c213_playback_ops);
> 
> 	snd_pcm_lib_preallocate_pages_for_all(chip->pcm, SNDRV_DMA_TYPE_DEV,
> snd_dma_continuous_data(GFP_KERNEL), 64 * 1024, 64 * 1024);
> 
> 	return 0;
> }
> 
> When removing the module from the kernel with rmmod I get alot of bad
> pages, because the page is mapped when it's freed. The weird thing is
> that the pages are mapped to address 0x00000000.
> 
> Example:
> [  141.380000] Bad page state in process 'rmmod'
> [17179650.992000] page:90200600 flags:0x00004000 mapping:00000000
> mapcount:0 count:0
> 
> Sorry if my email is lacking information, I'll gladly provide more
> information.

It's a new bug I haven't seen yet.

Could you give more detail on your system - the architecture, the
linux kernel version, and the ALSA version?


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: bad page when freeing preallocated pages

[Hans-Christian Egtvedt]

Takashi Iwai wrote:
> At Wed, 26 Apr 2006 10:30:30 +0200,
> Hans-Christian Egtvedt wrote:

<snip>

>> Example:
>> [  141.380000] Bad page state in process 'rmmod'
>> [17179650.992000] page:90200600 flags:0x00004000 mapping:00000000
>> mapcount:0 count:0
>>
>> Sorry if my email is lacking information, I'll gladly provide more
>> information.
> 
> It's a new bug I haven't seen yet.
> 
> Could you give more detail on your system - the architecture, the
> linux kernel version, and the ALSA version?

~ # uname -a
Linux (none) 2.6.16.11 #3 Wed Apr 26 11:21:06 CEST 2006 avr32 unknown

~ # cat /proc/asound/version
Advanced Linux Sound Architecture Driver Version 1.0.11rc2 (Wed Jan 04
08:57:20 2006 UTC).

The architecture is the AVR32 processor family from Atmel running on the
ATSTK1000 development board.

The interessting part is that if I preallocate a small area (4kB), then
the bad pages go away on rmmod. It could maybe be that we have some
undiscovered bug in dma_alloc_coherent() or dma_free_coherent(), but we
use there function calls in other drivers, so i doubt it (but keeping
the option open;) ).

-- 
With kind regards,
Med vennlig hilsen,

Hans-Christian Egtvedt
Applications Engineer - AVR Applications Lab
Atmel Norway


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: bad page when freeing preallocated pages

[Takashi Iwai]

At Thu, 27 Apr 2006 08:18:20 +0200,
Hans-Christian Egtvedt wrote:
> 
> Takashi Iwai wrote:
> > At Wed, 26 Apr 2006 10:30:30 +0200,
> > Hans-Christian Egtvedt wrote:
> 
> <snip>
> 
> >> Example:
> >> [  141.380000] Bad page state in process 'rmmod'
> >> [17179650.992000] page:90200600 flags:0x00004000 mapping:00000000
> >> mapcount:0 count:0
> >>
> >> Sorry if my email is lacking information, I'll gladly provide more
> >> information.
> > 
> > It's a new bug I haven't seen yet.
> > 
> > Could you give more detail on your system - the architecture, the
> > linux kernel version, and the ALSA version?
> 
> ~ # uname -a
> Linux (none) 2.6.16.11 #3 Wed Apr 26 11:21:06 CEST 2006 avr32 unknown
> 
> ~ # cat /proc/asound/version
> Advanced Linux Sound Architecture Driver Version 1.0.11rc2 (Wed Jan 04
> 08:57:20 2006 UTC).
> 
> The architecture is the AVR32 processor family from Atmel running on the
> ATSTK1000 development board.
> 
> The interessting part is that if I preallocate a small area (4kB), then
> the bad pages go away on rmmod. It could maybe be that we have some
> undiscovered bug in dma_alloc_coherent() or dma_free_coherent(), but we
> use there function calls in other drivers, so i doubt it (but keeping
> the option open;) ).

Ah, that enlighted me.

According to your first post, snd_pcm_preallocate_pages*() is called
like below:

	snd_pcm_lib_preallocate_pages_for_all(chip->pcm,
		SNDRV_DMA_TYPE_DEV, 
		snd_dma_continuous_data(GFP_KERNEL),
		 64 * 1024, 64 * 1024);

The third argument is wrong in this case.  You have to pass struct
device pointer there instead of snd_dma_continuous_data().

(Yeah, the function is messy and the clean-up is on my test tree for
long time.)


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: bad page when freeing preallocated pages

[Hans-Christian Egtvedt]

Takashi Iwai wrote:
> At Thu, 27 Apr 2006 08:18:20 +0200,
> Hans-Christian Egtvedt wrote:
>> Takashi Iwai wrote:
>>> At Wed, 26 Apr 2006 10:30:30 +0200,
>>> Hans-Christian Egtvedt wrote:

<snip>

>> The interessting part is that if I preallocate a small area (4kB), then
>> the bad pages go away on rmmod. It could maybe be that we have some
>> undiscovered bug in dma_alloc_coherent() or dma_free_coherent(), but we
>> use there function calls in other drivers, so i doubt it (but keeping
>> the option open;) ).
> 
> Ah, that enlighted me.
> 
> According to your first post, snd_pcm_preallocate_pages*() is called
> like below:
> 
> 	snd_pcm_lib_preallocate_pages_for_all(chip->pcm,
> 		SNDRV_DMA_TYPE_DEV, 
> 		snd_dma_continuous_data(GFP_KERNEL),
> 		 64 * 1024, 64 * 1024);
> 
> The third argument is wrong in this case.  You have to pass struct
> device pointer there instead of snd_dma_continuous_data().

I tried that, but the result is the same. I have stored the platform
device in chip->pdev and passed this to preallocation of pages:

snd_pcm_lib_preallocate_pages_for_all(chip->pcm, SNDRV_DMA_TYPE_DEV,
			&chip->pdev->dev, 64 * 1024, 64 * 1024);

I've looked into the library and it all seems to be correct in the
deeper layers.

Thank you for you feedback. Any tips or hints to where start debugging
would be greatly appreciated.

> (Yeah, the function is messy and the clean-up is on my test tree for
> long time.)

:)

-- 
With kind regards,
Med vennlig hilsen,

Hans-Christian Egtvedt
Applications Engineer - AVR Applications Lab
Atmel Norway


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: bad page when freeing preallocated pages

[Takashi Iwai]

At Thu, 27 Apr 2006 13:02:07 +0200,
Hans-Christian Egtvedt wrote:
> 
> Takashi Iwai wrote:
> > At Thu, 27 Apr 2006 08:18:20 +0200,
> > Hans-Christian Egtvedt wrote:
> >> Takashi Iwai wrote:
> >>> At Wed, 26 Apr 2006 10:30:30 +0200,
> >>> Hans-Christian Egtvedt wrote:
> 
> <snip>
> 
> >> The interessting part is that if I preallocate a small area (4kB), then
> >> the bad pages go away on rmmod. It could maybe be that we have some
> >> undiscovered bug in dma_alloc_coherent() or dma_free_coherent(), but we
> >> use there function calls in other drivers, so i doubt it (but keeping
> >> the option open;) ).
> > 
> > Ah, that enlighted me.
> > 
> > According to your first post, snd_pcm_preallocate_pages*() is called
> > like below:
> > 
> > 	snd_pcm_lib_preallocate_pages_for_all(chip->pcm,
> > 		SNDRV_DMA_TYPE_DEV, 
> > 		snd_dma_continuous_data(GFP_KERNEL),
> > 		 64 * 1024, 64 * 1024);
> > 
> > The third argument is wrong in this case.  You have to pass struct
> > device pointer there instead of snd_dma_continuous_data().
> 
> I tried that, but the result is the same. I have stored the platform
> device in chip->pdev and passed this to preallocation of pages:
> 
> snd_pcm_lib_preallocate_pages_for_all(chip->pcm, SNDRV_DMA_TYPE_DEV,
> 			&chip->pdev->dev, 64 * 1024, 64 * 1024);
> 
> I've looked into the library and it all seems to be correct in the
> deeper layers.
> 
> Thank you for you feedback. Any tips or hints to where start debugging
> would be greatly appreciated.

Removing snd_pcm_lib_preallocate_pages_for_all() might be a tentative
option.  Instead of snd_pcm_lib_malloc_pages() and *_free_pages(),
call dma_alloc_coherent() and dma_free_coherent() in hw_params and
hw_free PCM callbacks.  Return -ENOMEM if the alloc fails.

One thing to care in this case is that hw_params callback can be
called multiple times at each open.  You should check the buffer size
at each time, and re-allocate if a buffer was already allocated but
the size changed.

Another possile buggy point is the mmap support in pcm_native.c.
It uses virt_to_page() in nopage vma op, and it's known that this
doesn't work on some architectures.  Try to not to set
SNDRV_PCM_INFO_MMAP bit in snd_pcm_hardware.info bitflags to avoid
mmap.


Takashi


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

Re: bad page when freeing preallocated pages

[Hans-Christian Egtvedt]

Takashi Iwai wrote:
> At Thu, 27 Apr 2006 13:02:07 +0200,
> Hans-Christian Egtvedt wrote:
>> Takashi Iwai wrote:
>>> At Thu, 27 Apr 2006 08:18:20 +0200,
>>> Hans-Christian Egtvedt wrote:
>>>> Takashi Iwai wrote:
>>>>> At Wed, 26 Apr 2006 10:30:30 +0200,
>>>>> Hans-Christian Egtvedt wrote:

<snipp>

> Removing snd_pcm_lib_preallocate_pages_for_all() might be a tentative
> option.  Instead of snd_pcm_lib_malloc_pages() and *_free_pages(),
> call dma_alloc_coherent() and dma_free_coherent() in hw_params and
> hw_free PCM callbacks.  Return -ENOMEM if the alloc fails.
> 
> One thing to care in this case is that hw_params callback can be
> called multiple times at each open.  You should check the buffer size
> at each time, and re-allocate if a buffer was already allocated but
> the size changed.

Thank you, using this workaround solved my problem for now.

I have rewritten my _hw_params callback (pretty much what the library does):
+#if AT73C213_WORKAROUND_FOR_PREALLOC
+	int pg;
+	size_t size = params_buffer_bytes(hw_params);
+	struct snd_pcm_runtime *runtime;
+	struct snd_dma_buffer *dmab = NULL;
+
+	substream->dma_buffer.dev.type = SNDRV_DMA_TYPE_DEV;
+	snd_assert(substream != NULL, return -EINVAL);
+	runtime = substream->runtime;
+	snd_assert(runtime != NULL, return -EINVAL);
+
+	/* check if buffer is already allocated */
+	if (runtime->dma_buffer_p) {
+		size_t size_previouse;
+		int pg_previouse;
+
+		/* new buffer is smaler than previouse allocated buffer */
+		if (runtime->dma_buffer_p->bytes >= size) {
+			runtime->dma_bytes = size;
+			return 0; /* don't change buffer size */
+		}
+
+		size_previouse = runtime->dma_buffer_p->bytes;
+		pg_previouse = get_order(size_previouse);
+
+		dma_free_coherent(runtime->dma_buffer_p->dev.dev,
+				PAGE_SIZE << pg_previouse,
+			       	runtime->dma_buffer_p->area,
+				runtime->dma_buffer_p->addr);
+
+		kfree(runtime->dma_buffer_p);
+	}
+
+	dmab = kzalloc(sizeof(*dmab), GFP_KERNEL);
+	if (!dmab)
+		return -ENOMEM;
+	dmab->dev = substream->dma_buffer.dev;
+
+	pg = get_order(size);
+
+	dmab->area = dma_alloc_coherent(
+			substream->dma_buffer.dev.dev,
+		       	PAGE_SIZE << pg,
+		       	(dma_addr_t *)&dmab->addr,
+		       	GFP_KERNEL);
+
+	if (!dmab->area) {
+		kfree(dmab);
+		return -ENOMEM;
+	}
+
+	dmab->bytes = size;
+	snd_pcm_set_runtime_buffer(substream, dmab);
+	runtime->dma_bytes = size;
+	return 1;
+#else
 	return snd_pcm_lib_malloc_pages(substream,
 					params_buffer_bytes(hw_params));
+#endif


And hw_free callback:
+#if AT73C213_WORKAROUND_FOR_PREALLOC
+	int pg;
+	struct snd_pcm_runtime *runtime;
+	struct snd_dma_buffer *dmab = NULL;
+
+	snd_assert(substream != NULL, return -EINVAL);
+	runtime = substream->runtime;
+	snd_assert(runtime != NULL, return -EINVAL);
+	dmab = runtime->dma_buffer_p;
+
+	if (!dmab->area)
+		return 0;
+
+	pg = get_order(dmab->bytes);
+	dma_free_coherent(dmab->dev.dev, PAGE_SIZE << pg, dmab->area, dmab->addr);
+	kfree(runtime->dma_buffer_p);
+	snd_pcm_set_runtime_buffer(substream, NULL);
+	return 0;
+#else
 	return snd_pcm_lib_free_pages(substream);
+#endif


> Another possile buggy point is the mmap support in pcm_native.c.
> It uses virt_to_page() in nopage vma op, and it's known that this
> doesn't work on some architectures.  Try to not to set
> SNDRV_PCM_INFO_MMAP bit in snd_pcm_hardware.info bitflags to avoid
> mmap.

I've already tried this setting, and it didn't effect the result.

-- 
With kind regards,
Med vennlig hilsen,

Hans-Christian Egtvedt
Applications Engineer - AVR Applications Lab
Atmel Norway


-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642