Re: [PATCH] lazy freeing of memory through MADV_FREE

Rik van Riel
2007-04-23T10:45:04Z

[-- Attachment #1: Type: text/plain, Size: 1847 bytes --]

Use TLB batching for MADV_FREE. Adds another 10-15% extra performance
to the MySQL sysbench results on my quad core system.

Signed-off-by: Rik van Riel <riel@redhat.com>
---

Nick Piggin wrote:

>> 3) because of this, we can treat any such accesses as
>> happening simultaneously with the MADV_FREE and
>> as illegal, aka undefined behaviour territory and
>> we do not need to worry about them
>
> Yes, but I'm wondering if it is legal in all architectures.

It's similar to trying to access memory during an munmap.

You may be able to for a short time, but it'll come back to
haunt you.

>> 4) because we flush the tlb before releasing the page
>> table lock, other CPUs cannot remove this page from
>> the address space - they will block on the page
>> table lock before looking at this pte
>
> We don't when the ptl is split.

Even then we do. Each invocation of zap_pte_range() only touches
one page table page, and it flushes the TLB before releasing the
page table lock.

> What the tlb flush used to be able to assume is that the page
> has been removed from the pagetables when they are put in the
> tlb flush batch.

All the tlb flush code seems to assume is that the tlb entries
should be invalidated.

> I'm not saying there is any bugs, but just suggesting there
> might be.

Jakub found a potential bug, in that I did not use an atomic
operation to clear the page table entries. I've attached a
new patch which simply uses ptep_test_and_clear_dirty/young
to get rid of the dirty and accessed bits.

It uses the same atomic accesses we use elsewhere in the VM
and the code is a line shorter than before.

Andrew, please use this one.

--
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is. Each group
calls the other unpatriotic.

[-- Attachment #2: linux-2.6-madv_free-lazytlb.patch --]
[-- Type: text/x-patch, Size: 697 bytes --]

--- linux-2.6.20.x86_64/mm/memory.c.orig 2007-04-23 02:48:36.000000000 -0400
+++ linux-2.6.20.x86_64/mm/memory.c 2007-04-23 02:54:42.000000000 -0400
@@ -677,11 +677,14 @@ static unsigned long zap_pte_range(struc
remove_exclusive_swap_page(page);
unlock_page(page);
}
- ptep_clear_flush_dirty(vma, addr, pte);
- ptep_clear_flush_young(vma, addr, pte);
+ ptep_test_and_clear_dirty(vma, addr, pte);
+ ptep_test_and_clear_young(vma, addr, pte);
SetPageLazyFree(page);
if (PageActive(page))
deactivate_tail_page(page);
+ /* tlb_remove_page frees it again */
+ get_page(page);
+ tlb_remove_page(tlb, page);
continue;
}
}