[PATCH] ksm: fix conflict between mmput and scan_get_next_rmap_item

Zhou Chengming
2016-05-05T12:47:33Z

A concurrency issue about KSM in the function scan_get_next_rmap_item.

task A (ksmd): |task B (the mm's task):
|
mm = slot->mm; |
down_read(&mm->mmap_sem); |
|
... |
|
spin_lock(&ksm_mmlist_lock); |
|
ksm_scan.mm_slot go to the next slot; |
|
spin_unlock(&ksm_mmlist_lock); |
|mmput() ->
| ksm_exit():
|
|spin_lock(&ksm_mmlist_lock);
|if (mm_slot && ksm_scan.mm_slot != mm_slot) {
| if (!mm_slot->rmap_list) {
| easy_to_free = 1;
| ...
|
|if (easy_to_free) {
| mmdrop(mm);
| ...
|
|So this mm_struct will be freed successfully.
|
up_read(&mm->mmap_sem); |

As we can see above, the ksmd thread may access a mm_struct that already
been freed to the kmem_cache.
Suppose a fork will get this mm_struct from the kmem_cache, the ksmd thread
then call up_read(&mm->mmap_sem), will cause mmap_sem.count to become -1.
I changed the scan_get_next_rmap_item function refered to the khugepaged
scan function.

Signed-off-by: Zhou Chengming <zhouchengming1@huawei.com>
---
mm/ksm.c | 7 ++-----
1 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/mm/ksm.c b/mm/ksm.c
index 7ee101e..6e4324d 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -1650,6 +1650,7 @@ next_mm:
* because there were no VM_MERGEABLE vmas with such addresses.
*/
remove_trailing_rmap_items(slot, ksm_scan.rmap_list);
+ up_read(&mm->mmap_sem);

spin_lock(&ksm_mmlist_lock);
ksm_scan.mm_slot = list_entry(slot->mm_list.next,
@@ -1666,16 +1667,12 @@ next_mm:
*/
hash_del(&slot->link);
list_del(&slot->mm_list);
- spin_unlock(&ksm_mmlist_lock);

free_mm_slot(slot);
clear_bit(MMF_VM_MERGEABLE, &mm->flags);
- up_read(&mm->mmap_sem);
mmdrop(mm);
- } else {
- spin_unlock(&ksm_mmlist_lock);
- up_read(&mm->mmap_sem);
}
+ spin_unlock(&ksm_mmlist_lock);

/* Repeat until we've completed scanning the whole list */
slot = ksm_scan.mm_slot;
--
1.7.7