dma_unmap causing issues with __get_free_pages

dma_unmap causing issues with __get_free_pages

[Joel Fernandes]

Hi,

I'm having some trouble with using the dma_map/unmap API.

On unmapping a particular page using dma_unmap, it seems that the
PG_dcache_clean flag is set in the page->flags. This is set by the
following statement in __dma_page_dev_to_cpu function in
arch/arm/mm/dma-mapping.c
                set_bit(PG_dcache_clean, &page->flags);

Due to this, on any subsequent page allocations using __get_free_pages,
the following BUG gets triggered.

[   15.267913] BUG: Bad page state in process insmod  pfn:acb26
[   15.274017] page:c151e4c0 count:0 mapcount:0 mapping:  (null) index:0x0
[   15.281097] page flags: 0x200(arch_1)
[   15.285003] Modules linked in: tcrypt(+)
[   15.289062] CPU: 0 PID: 1616 Comm: insmod Tainted: G    B
3.10.0-00038-gbafd29d-dirty #89
[   15.298706] [<c001c720>] (unwind_backtrace+0x0/0xfc) from
[<c0018794>] (show_stack+0x20/0x24)
[   15.307861] [<c0018794>] (show_stack+0x20/0x24) from [<c05a7284>]
(dump_stack+0x20/0x28)
[   15.316497] [<c05a7284>] (dump_stack+0x20/0x28) from [<c00fa220>]
(bad_page+0xbc/0x11c)
[   15.325195] [<c00fa220>] (bad_page+0xbc/0x11c) from [<c00fa8c8>]
(get_page_from_freelist+0x4dc/0x620)
[   15.335083] [<c00fa8c8>] (get_page_from_freelist+0x4dc/0x620) from
[<c00fb57c>] (__alloc_pages_nodemask+0x114/0x8b4)
[   15.346343] [<c00fb57c>] (__alloc_pages_nodemask+0x114/0x8b4) from
[<c00fbd3c>] (__get_free_pages+0x20/0x3c)
[   15.356872] [<c00fbd3c>] (__get_free_pages+0x20/0x3c) from
[<c0486bd0>] (omap_aes_copy_sgs+0x48/0x1bc)


If I don't do the unmap and leave the page alone, everything works just
fine.

What is correct way to fix this? Why does the page allocator think its a
BAD page descriptor after the unmap?

Thanks,

-Joel

Re: dma_unmap causing issues with __get_free_pages

[Russell King - ARM Linux]

On Thu, Aug 15, 2013 at 02:35:59AM -0500, Joel Fernandes wrote:
> Hi,
> 
> I'm having some trouble with using the dma_map/unmap API.
> 
> On unmapping a particular page using dma_unmap, it seems that the
> PG_dcache_clean flag is set in the page->flags. This is set by the
> following statement in __dma_page_dev_to_cpu function in
> arch/arm/mm/dma-mapping.c
>                 set_bit(PG_dcache_clean, &page->flags);
> 
> Due to this, on any subsequent page allocations using __get_free_pages,
> the following BUG gets triggered.

Are you calling dma_unmap() after the page has been freed?

> What is correct way to fix this? Why does the page allocator think its a
> BAD page descriptor after the unmap?

Well, on free, this is done:

        if (page->flags & PAGE_FLAGS_CHECK_AT_PREP)
                page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;

which clears PG_arch_1.  On allocation:

        if (unlikely(page_mapcount(page) |
                (page->mapping != NULL)  |
                (atomic_read(&page->_count) != 0)  |
                (page->flags & PAGE_FLAGS_CHECK_AT_PREP) |
                (mem_cgroup_bad_page_check(page)))) {
                bad_page(page);
                return 1;
        }

As PG_arch_1 is part of the PAGE_FLAGS_CHECK_AT_PREP mask, this means that
when a page is freed, it has PG_arch_1 cleared.  Therefore, if on allocation
the page now has this bit set, it means that something touched the page
after it was freed.  Quite simply, the page was freed while still being
in use.  That's very bad and needs fixing.

Re: dma_unmap causing issues with __get_free_pages

[Joel Fernandes]

Hi Russell,

On 08/15/2013 06:55 AM, Russell King - ARM Linux wrote:
> On Thu, Aug 15, 2013 at 02:35:59AM -0500, Joel Fernandes wrote:
>> Hi,
>>
>> I'm having some trouble with using the dma_map/unmap API.
>>
>> On unmapping a particular page using dma_unmap, it seems that the
>> PG_dcache_clean flag is set in the page->flags. This is set by the
>> following statement in __dma_page_dev_to_cpu function in
>> arch/arm/mm/dma-mapping.c
>>                 set_bit(PG_dcache_clean, &page->flags);
>>
>> Due to this, on any subsequent page allocations using __get_free_pages,
>> the following BUG gets triggered.
> 
> Are you calling dma_unmap() after the page has been freed?
> 
>> What is correct way to fix this? Why does the page allocator think its a
>> BAD page descriptor after the unmap?
> 
> Well, on free, this is done:
> 
>         if (page->flags & PAGE_FLAGS_CHECK_AT_PREP)
>                 page->flags &= ~PAGE_FLAGS_CHECK_AT_PREP;
> 
> which clears PG_arch_1.  On allocation:
> 
>         if (unlikely(page_mapcount(page) |
>                 (page->mapping != NULL)  |
>                 (atomic_read(&page->_count) != 0)  |
>                 (page->flags & PAGE_FLAGS_CHECK_AT_PREP) |
>                 (mem_cgroup_bad_page_check(page)))) {
>                 bad_page(page);
>                 return 1;
>         }
> 
> As PG_arch_1 is part of the PAGE_FLAGS_CHECK_AT_PREP mask, this means that
> when a page is freed, it has PG_arch_1 cleared.  Therefore, if on allocation
> the page now has this bit set, it means that something touched the page
> after it was freed.  Quite simply, the page was freed while still being
> in use.  That's very bad and needs fixing.
> 

Absolutely you nailed it! I was doing alloc, map, free, unmap, changed
this to alloc, map, unmap, free and its working fine now and I learnt a
thing or 2 about page->flags. Thanks!

-Joel