* Re: understand KSM [not found] <395413139.534301288167215758.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> @ 2010-10-27 8:16 ` caiqian 2010-11-09 6:48 ` Hugh Dickins 0 siblings, 1 reply; 7+ messages in thread From: caiqian @ 2010-10-27 8:16 UTC (permalink / raw) To: Hugh Dickins; +Cc: linux-mm > Since your 1MB malloc'ed buffers may not fall on page boundaries, > and there might occasionally be other malloc'ed areas interspersed > amongst them, I'm not surprised that pages_sharing falls a little > short of 98302. But I am surprised that pages_unshared does not > make up the difference; probably pages_volatile does, but I don't > see why some should remain volatile indefinitely. The test program (http://people.redhat.com/qcai/ksm01.c) was changed to use mmap instead of malloc, and pages_sharing was short of the expected value and pages_volatile was indeed non-zero. Those makes it is difficult to predict pages_sharing and pages_volatile although it might be fine to check pages_sharing + pages_volatile with an expected value. Any suggestion to alter the test code to check the stable numbers? Thanks. ksm01 0 TINFO : child 0 allocates 128 MB filled with 'c'. ksm01 0 TINFO : child 1 allocates 128 MB filled with 'a'. ksm01 0 TINFO : child 2 allocates 128 MB filled with 'a'. ksm01 0 TINFO : pages_shared is 2. ksm01 0 TINFO : pages_sharing is 98300. ksm01 0 TINFO : pages_unshared is 0. ksm01 0 TINFO : pages_volatile is 2. ksm01 0 TINFO : child 1 changes memory content to 'b'. ksm01 0 TINFO : pages_shared is 3. ksm01 0 TINFO : pages_sharing is 98291. ksm01 0 TINFO : pages_unshared is 0. ksm01 0 TINFO : pages_volatile is 10. ksm01 0 TINFO : child 0 changes memory content to 'd'. ksm01 0 TINFO : child 1 changes memory content to 'd' ksm01 0 TINFO : child 2 changes memory content to 'd' ksm01 0 TINFO : pages_shared is 1. ksm01 0 TINFO : pages_sharing is 98299. ksm01 0 TINFO : pages_unshared is 0. ksm01 0 TINFO : pages_volatile is 4. ksm01 0 TINFO : child 1 changes one page to 'e'. ksm01 0 TINFO : pages_shared is 1. ksm01 0 TINFO : pages_sharing is 98299. ksm01 0 TINFO : pages_unshared is 1. ksm01 0 TINFO : pages_volatile is 3. CAI Qian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: understand KSM 2010-10-27 8:16 ` understand KSM caiqian @ 2010-11-09 6:48 ` Hugh Dickins 2010-11-10 9:25 ` CAI Qian 0 siblings, 1 reply; 7+ messages in thread From: Hugh Dickins @ 2010-11-09 6:48 UTC (permalink / raw) To: caiqian; +Cc: Andrea Arcangeli, linux-mm On Wed, 27 Oct 2010, caiqian@redhat.com wrote: > > Since your 1MB malloc'ed buffers may not fall on page boundaries, > > and there might occasionally be other malloc'ed areas interspersed > > amongst them, I'm not surprised that pages_sharing falls a little > > short of 98302. But I am surprised that pages_unshared does not > > make up the difference; probably pages_volatile does, but I don't > > see why some should remain volatile indefinitely. > The test program (http://people.redhat.com/qcai/ksm01.c) was changed to use mmap instead of malloc, and pages_sharing was short of the expected value and pages_volatile was indeed non-zero. Those makes it is difficult to predict pages_sharing and pages_volatile although it might be fine to check pages_sharing + pages_volatile with an expected value. Any suggestion to alter the test code to check the stable numbers? Thanks. > > ksm01 0 TINFO : child 0 allocates 128 MB filled with 'c'. > ksm01 0 TINFO : child 1 allocates 128 MB filled with 'a'. > ksm01 0 TINFO : child 2 allocates 128 MB filled with 'a'. > ksm01 0 TINFO : pages_shared is 2. > ksm01 0 TINFO : pages_sharing is 98300. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 0 TINFO : pages_volatile is 2. > > ksm01 0 TINFO : child 1 changes memory content to 'b'. > ksm01 0 TINFO : pages_shared is 3. > ksm01 0 TINFO : pages_sharing is 98291. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 0 TINFO : pages_volatile is 10. > > ksm01 0 TINFO : child 0 changes memory content to 'd'. > ksm01 0 TINFO : child 1 changes memory content to 'd' > ksm01 0 TINFO : child 2 changes memory content to 'd' > ksm01 0 TINFO : pages_shared is 1. > ksm01 0 TINFO : pages_sharing is 98299. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 0 TINFO : pages_volatile is 4. > > ksm01 0 TINFO : child 1 changes one page to 'e'. > ksm01 0 TINFO : pages_shared is 1. > ksm01 0 TINFO : pages_sharing is 98299. > ksm01 0 TINFO : pages_unshared is 1. > ksm01 0 TINFO : pages_volatile is 3. Thank you for persisting, I was surprised by that, but didn't find time to try for myself until yesterday: yes, running your ksm01, pages_volatile stayed non-0 for erratic periods of time, say 10 or 20 seconds. I had to insert more debugging to find out what and why was failing, but in the end it was rather obvious. Fix below, but I haven't yet signed off the patch - we usually prefer to avoid lru_add_drain_all() (all those inter-cpu interrupts): I think this is the natural place to call it, but I haven't quite decided yet whether it's worth adding a few lines to limit how often we call it there. Why did my own testing never see this? Largely because I was using system() to run a shell script to show the /sys/kernel/mm/ksm numbers (whereas your ksm01.c opens and reads the files directly): there's more than enough overhead in doing it my way to flush those pagevecs (on one cpu, but I'm still surprised I didn't see it from other cpus). Hugh --- mm/ksm.c | 9 +++++++++ 1 file changed, 9 insertions(+) --- 2.6.37-rc1/mm/ksm.c 2010-10-20 13:30:22.000000000 -0700 +++ linux/mm/ksm.c 2010-11-07 23:49:26.000000000 -0800 @@ -1352,6 +1352,15 @@ static void ksm_do_scan(unsigned int sca struct rmap_item *rmap_item; struct page *uninitialized_var(page); + /* + * A number of pages can hang around indefinitely on per-cpu pagevecs, + * with raised page count preventing write_protect_page() from merging + * them: though it doesn't really matter much, it is disturbing to see + * them stuck in pages_volatile until other activity jostles them out, + * and it prevents deterministic LTP success; so drain them here. + */ + lru_add_drain_all(); + while (scan_npages--) { cond_resched(); rmap_item = scan_get_next_rmap_item(&page); -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: understand KSM 2010-11-09 6:48 ` Hugh Dickins @ 2010-11-10 9:25 ` CAI Qian 0 siblings, 0 replies; 7+ messages in thread From: CAI Qian @ 2010-11-10 9:25 UTC (permalink / raw) To: Hugh Dickins; +Cc: Andrea Arcangeli, linux-mm > mm/ksm.c | 9 +++++++++ > 1 file changed, 9 insertions(+) This fixed the problem for me. Thanks. CAI Qian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <356409918.247361287997619777.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com>]
* understand KSM [not found] <356409918.247361287997619777.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com> @ 2010-10-25 9:11 ` caiqian 2010-10-26 8:03 ` Hugh Dickins 0 siblings, 1 reply; 7+ messages in thread From: caiqian @ 2010-10-25 9:11 UTC (permalink / raw) To: linux-mm Hi everyone, while developing some tests for KSM in LTP - http://marc.info/?l=ltp-list&m=128754077917739&w=2 , noticed that pages_shared, pages_sharing and pages_unshared have different values than the expected values in the tests after read the doc. I am not sure if I misunderstood those values or there were bugs somewhere. There are 3 programs (A, B ,C) to allocate 128M memory each using KSM. A has memory content equal 'c'. B has memory content equal 'a'. C has memory content equal 'a'. Then (using the latest mmotm tree), pages_shared = 2 pages_sharing = 98292 pages_unshared = 0 Later, A has memory content = 'c' B has memory content = 'b' C has memory content = 'a'. Then, pages_shared = 4 pages_sharing = 98282 pages_unshared = 0 Finally, A has memory content = 'd' B has memory content = 'd' C has memory content = 'd' Then, pages_shared = 0 pages_sharing = 0 pages_unshared = 0 The following was the failed LTP output, # ./ksm01 ksm01 0 TINFO : KSM merging... ksm01 0 TINFO : child 0 allocates 128 MB filled with 'c'. ksm01 0 TINFO : child 1 allocates 128 MB filled with 'a'. ksm01 0 TINFO : child 2 allocates 128 MB filled with 'a'. ksm01 0 TINFO : check! ksm01 0 TINFO : run is 1. ksm01 0 TINFO : pages_shared is 2. ksm01 1 TFAIL : pages_shared is not 32768. ksm01 0 TINFO : pages_sharing is 98292. ksm01 2 TFAIL : pages_sharing is not 32768. ksm01 0 TINFO : pages_unshared is 0. ksm01 3 TFAIL : pages_unshared is not 32768. ksm01 0 TINFO : child 1 continues... ksm01 0 TINFO : child 1 changes memory content to 'b'. ksm01 0 TINFO : check! ksm01 0 TINFO : run is 1. ksm01 0 TINFO : pages_shared is 4. ksm01 4 TFAIL : pages_shared is not 0. ksm01 0 TINFO : pages_sharing is 98282. ksm01 5 TFAIL : pages_sharing is not 0. ksm01 0 TINFO : pages_unshared is 0. ksm01 6 TFAIL : pages_unshared is not 98304. ksm01 0 TINFO : child 0 continues... ksm01 0 TINFO : child 0 changes memory content to 'd'. ksm01 0 TINFO : child 1 continues... ksm01 0 TINFO : child 1 changes memory content to 'd' ksm01 0 TINFO : child 2 continues... ksm01 0 TINFO : child 2 changes memory content to 'd' ksm01 0 TINFO : check! ksm01 0 TINFO : run is 1. ksm01 0 TINFO : pages_shared is 0. ksm01 7 TFAIL : pages_shared is not 32768. ksm01 0 TINFO : pages_sharing is 0. ksm01 8 TFAIL : pages_sharing is not 65536. ksm01 0 TINFO : pages_unshared is 0. ksm01 0 TINFO : KSM unmerging... ksm01 0 TINFO : check! ksm01 0 TINFO : run is 2. ksm01 0 TINFO : pages_shared is 0. ksm01 0 TINFO : pages_sharing is 0. ksm01 0 TINFO : pages_unshared is 0. ksm01 0 TINFO : stop KSM. ksm01 0 TINFO : check! ksm01 0 TINFO : run is 0. ksm01 0 TINFO : pages_shared is 0. ksm01 0 TINFO : pages_sharing is 0. ksm01 0 TINFO : pages_unshared is 0. ksm01 9 TFAIL : ksmtest() failed with 1. CAI Qian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: understand KSM 2010-10-25 9:11 ` caiqian @ 2010-10-26 8:03 ` Hugh Dickins 2010-10-26 11:14 ` CAI Qian 0 siblings, 1 reply; 7+ messages in thread From: Hugh Dickins @ 2010-10-26 8:03 UTC (permalink / raw) To: caiqian; +Cc: linux-mm On Mon, 25 Oct 2010, caiqian@redhat.com wrote: > Hi everyone, while developing some tests for KSM in LTP Thank you! > - http://marc.info/?l=ltp-list&m=128754077917739&w=2 , noticed that pages_shared, pages_sharing and pages_unshared have different values than the expected values in the tests after read the doc. I am not sure if I misunderstood those values or there were bugs somewhere. You were expecting KSM to share pages between processes, but you were not expecting it to share pages within a process, which it does also. To check the exact numbers, it would be easier if you use page-aligned mmap() rather than byte-aligned malloc() for your MADV_MERGEABLE buffers: some numbers are a little "off" because of part-pages at start and end. > > There are 3 programs (A, B ,C) to allocate 128M memory each using KSM. > > A has memory content equal 'c'. > B has memory content equal 'a'. > C has memory content equal 'a'. > > Then (using the latest mmotm tree), > pages_shared = 2 > pages_sharing = 98292 > pages_unshared = 0 So, after KSM has done its best, it all reduces to 1 page full of 'a's and another 1 page full of 'c's. > > Later, > A has memory content = 'c' > B has memory content = 'b' > C has memory content = 'a'. > > Then, > pages_shared = 4 > pages_sharing = 98282 > pages_unshared = 0 pages_shared 3 would be the obvious: I expect the extra 1 is an artifact of part-pages at start and end of your buffers, a page shared there too. > > Finally, > A has memory content = 'd' > B has memory content = 'd' > C has memory content = 'd' > > Then, > pages_shared = 0 > pages_sharing = 0 > pages_unshared = 0 The children appear to exit(1) as soon as they have filled their buffers with 'd's, so there's nothing left to share. Hugh > > The following was the failed LTP output, > > # ./ksm01 > ksm01 0 TINFO : KSM merging... > ksm01 0 TINFO : child 0 allocates 128 MB filled with 'c'. > ksm01 0 TINFO : child 1 allocates 128 MB filled with 'a'. > ksm01 0 TINFO : child 2 allocates 128 MB filled with 'a'. > ksm01 0 TINFO : check! > ksm01 0 TINFO : run is 1. > ksm01 0 TINFO : pages_shared is 2. > ksm01 1 TFAIL : pages_shared is not 32768. > ksm01 0 TINFO : pages_sharing is 98292. > ksm01 2 TFAIL : pages_sharing is not 32768. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 3 TFAIL : pages_unshared is not 32768. > ksm01 0 TINFO : child 1 continues... > ksm01 0 TINFO : child 1 changes memory content to 'b'. > ksm01 0 TINFO : check! > ksm01 0 TINFO : run is 1. > ksm01 0 TINFO : pages_shared is 4. > ksm01 4 TFAIL : pages_shared is not 0. > ksm01 0 TINFO : pages_sharing is 98282. > ksm01 5 TFAIL : pages_sharing is not 0. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 6 TFAIL : pages_unshared is not 98304. > ksm01 0 TINFO : child 0 continues... > ksm01 0 TINFO : child 0 changes memory content to 'd'. > ksm01 0 TINFO : child 1 continues... > ksm01 0 TINFO : child 1 changes memory content to 'd' > ksm01 0 TINFO : child 2 continues... > ksm01 0 TINFO : child 2 changes memory content to 'd' > ksm01 0 TINFO : check! > ksm01 0 TINFO : run is 1. > ksm01 0 TINFO : pages_shared is 0. > ksm01 7 TFAIL : pages_shared is not 32768. > ksm01 0 TINFO : pages_sharing is 0. > ksm01 8 TFAIL : pages_sharing is not 65536. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 0 TINFO : KSM unmerging... > ksm01 0 TINFO : check! > ksm01 0 TINFO : run is 2. > ksm01 0 TINFO : pages_shared is 0. > ksm01 0 TINFO : pages_sharing is 0. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 0 TINFO : stop KSM. > ksm01 0 TINFO : check! > ksm01 0 TINFO : run is 0. > ksm01 0 TINFO : pages_shared is 0. > ksm01 0 TINFO : pages_sharing is 0. > ksm01 0 TINFO : pages_unshared is 0. > ksm01 9 TFAIL : ksmtest() failed with 1. > > CAI Qian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: understand KSM 2010-10-26 8:03 ` Hugh Dickins @ 2010-10-26 11:14 ` CAI Qian 2010-10-27 4:00 ` Hugh Dickins 0 siblings, 1 reply; 7+ messages in thread From: CAI Qian @ 2010-10-26 11:14 UTC (permalink / raw) To: Hugh Dickins; +Cc: linux-mm > > There are 3 programs (A, B ,C) to allocate 128M memory each using > KSM. > > > > A has memory content equal 'c'. > > B has memory content equal 'a'. > > C has memory content equal 'a'. > > > > Then (using the latest mmotm tree), > > pages_shared = 2 > > pages_sharing = 98292 > > pages_unshared = 0 > > So, after KSM has done its best, it all reduces to 1 page full of > 'a's and another 1 page full of 'c's. I would expect pages_sharing to be 98302 (128 * 256 - 2), but this one looks unstable. Increased pages_to_scan to 2 * 98304 did not help either. Thanks for the other suggestions! After modified the test accordingly, it looks like work as expected. CAI Qian -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: understand KSM 2010-10-26 11:14 ` CAI Qian @ 2010-10-27 4:00 ` Hugh Dickins 0 siblings, 0 replies; 7+ messages in thread From: Hugh Dickins @ 2010-10-27 4:00 UTC (permalink / raw) To: CAI Qian; +Cc: linux-mm On Tue, 26 Oct 2010, CAI Qian wrote: > > > > There are 3 programs (A, B ,C) to allocate 128M memory each using > > KSM. > > > > > > A has memory content equal 'c'. > > > B has memory content equal 'a'. > > > C has memory content equal 'a'. > > > > > > Then (using the latest mmotm tree), > > > pages_shared = 2 > > > pages_sharing = 98292 > > > pages_unshared = 0 > > > > So, after KSM has done its best, it all reduces to 1 page full of > > 'a's and another 1 page full of 'c's. > I would expect pages_sharing to be 98302 (128 * 256 - 2), but this one looks unstable. Increased pages_to_scan to 2 * 98304 did not help either. Since your 1MB malloc'ed buffers may not fall on page boundaries, and there might occasionally be other malloc'ed areas interspersed amongst them, I'm not surprised that pages_sharing falls a little short of 98302. But I am surprised that pages_unshared does not make up the difference; probably pages_volatile does, but I don't see why some should remain volatile indefinitely. > > Thanks for the other suggestions! After modified the test accordingly, it looks like work as expected. Oh good, that's a relief. Hugh -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a> ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2010-11-10 9:25 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <395413139.534301288167215758.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com>
2010-10-27 8:16 ` understand KSM caiqian
2010-11-09 6:48 ` Hugh Dickins
2010-11-10 9:25 ` CAI Qian
[not found] <356409918.247361287997619777.JavaMail.root@zmail06.collab.prod.int.phx2.redhat.com>
2010-10-25 9:11 ` caiqian
2010-10-26 8:03 ` Hugh Dickins
2010-10-26 11:14 ` CAI Qian
2010-10-27 4:00 ` Hugh Dickins
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).