ksm questions

ksm questions

[Ni zhan Chen]

Hi all,

I have some questions about ksm.

1) khugepaged default nice value is 19, but ksmd default nice value is 
5, why this big different?
2) why ksm doesn't support pagecache and tmpfs now? What's the bottleneck?
3) ksm kernel doc said that "KSM only merges anonymous(private) pages, 
never pagecache(file) pages". But where judege it should be private?
4) ksm kernel doc said that "To avoid the instability and the resulting 
false negatives to be permanent, KSM re-initializes the unstable tree 
root node to an empty tree, at every KSM pass." But I can't find where 
re-initializes the unstable tree, could you explain me?

Thanks in advance. :-)

Regards,
Chen

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: ksm questions

[Izik Eidus]

On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
> Hi all,
>
> I have some questions about ksm.
>
> 1) khugepaged default nice value is 19, but ksmd default nice value is 
> 5, why this big different?
> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
> bottleneck?
> 3) ksm kernel doc said that "KSM only merges anonymous(private) pages, 
> never pagecache(file) pages". But where judege it should be private?
> 4) ksm kernel doc said that "To avoid the instability and the 
> resulting false negatives to be permanent, KSM re-initializes the 
> unstable tree root node to an empty tree, at every KSM pass." But I 
> can't find where re-initializes the unstable tree, could you explain me?


in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
root_unstable_tree = RB_ROOT; this will result in root_unstable_tree 
being empty.

>
> Thanks in advance. :-)
>
> Regards,
> Chen
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: ksm questions

[Ni zhan Chen]

On 10/24/2012 07:46 PM, Izik Eidus wrote:
> On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
>> Hi all,
>>
>> I have some questions about ksm.
>>
>> 1) khugepaged default nice value is 19, but ksmd default nice value 
>> is 5, why this big different?
>> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
>> bottleneck?
>> 3) ksm kernel doc said that "KSM only merges anonymous(private) 
>> pages, never pagecache(file) pages". But where judege it should be 
>> private?
>> 4) ksm kernel doc said that "To avoid the instability and the 
>> resulting false negatives to be permanent, KSM re-initializes the 
>> unstable tree root node to an empty tree, at every KSM pass." But I 
>> can't find where re-initializes the unstable tree, could you explain me?
>
>
> in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
> root_unstable_tree = RB_ROOT; this will result in root_unstable_tree 
> being empty.

thanks Izik, what about the other three questions?

>
>>
>> Thanks in advance. :-)
>>
>> Regards,
>> Chen
>>
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: ksm questions

[Izik Eidus]

On 10/24/2012 01:59 PM, Ni zhan Chen wrote:
> On 10/24/2012 07:46 PM, Izik Eidus wrote:
>> On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
>>> Hi all,
>>>
>>> I have some questions about ksm.
>>>
>>> 1) khugepaged default nice value is 19, but ksmd default nice value 
>>> is 5, why this big different?
>>> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
>>> bottleneck?
>>> 3) ksm kernel doc said that "KSM only merges anonymous(private) 
>>> pages, never pagecache(file) pages". But where judege it should be 
>>> private?
>>> 4) ksm kernel doc said that "To avoid the instability and the 
>>> resulting false negatives to be permanent, KSM re-initializes the 
>>> unstable tree root node to an empty tree, at every KSM pass." But I 
>>> can't find where re-initializes the unstable tree, could you explain 
>>> me?
>>
>>
>> in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
>> root_unstable_tree = RB_ROOT; this will result in root_unstable_tree 
>> being empty.
>
> thanks Izik, what about the other three questions?

Question number 2 is beacuse it is forced to work with anonymous pages, 
about question 3 - I will have to remember why from the very begining I 
wrote it to support only anonymous pages (few years have been passed), 
maybe Andrea/Huge have it more hot in their heads?

>
>>
>>>
>>> Thanks in advance. :-)
>>>
>>> Regards,
>>> Chen
>>>
>>
>>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: ksm questions

[Ni zhan Chen]

On 10/24/2012 08:02 PM, Izik Eidus wrote:
> On 10/24/2012 01:59 PM, Ni zhan Chen wrote:
>> On 10/24/2012 07:46 PM, Izik Eidus wrote:
>>> On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
>>>> Hi all,
>>>>
>>>> I have some questions about ksm.
>>>>
>>>> 1) khugepaged default nice value is 19, but ksmd default nice value 
>>>> is 5, why this big different?
>>>> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
>>>> bottleneck?
>>>> 3) ksm kernel doc said that "KSM only merges anonymous(private) 
>>>> pages, never pagecache(file) pages". But where judege it should be 
>>>> private?
>>>> 4) ksm kernel doc said that "To avoid the instability and the 
>>>> resulting false negatives to be permanent, KSM re-initializes the 
>>>> unstable tree root node to an empty tree, at every KSM pass." But I 
>>>> can't find where re-initializes the unstable tree, could you 
>>>> explain me?
>>>
>>>
>>> in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
>>> root_unstable_tree = RB_ROOT; this will result in root_unstable_tree 
>>> being empty.
>>
>> thanks Izik, what about the other three questions?
>
> Question number 2 is beacuse it is forced to work with anonymous 
> pages, about question 3 - I will have to remember why from the very 
> begining I wrote it to support only anonymous pages (few years have 
> been passed), maybe Andrea/Huge have it more hot in their heads?

The initialize goal of design ksm is for virtualization, but now it also 
can be used for other apps. So can it also support tmpfs or pagecache in 
the future?

>
>>
>>>
>>>>
>>>> Thanks in advance. :-)
>>>>
>>>> Regards,
>>>> Chen
>>>>
>>>
>>>
>>
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: ksm questions

[Izik Eidus]

On 10/24/2012 02:16 PM, Ni zhan Chen wrote:
> On 10/24/2012 08:02 PM, Izik Eidus wrote:
>> On 10/24/2012 01:59 PM, Ni zhan Chen wrote:
>>> On 10/24/2012 07:46 PM, Izik Eidus wrote:
>>>> On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
>>>>> Hi all,
>>>>>
>>>>> I have some questions about ksm.
>>>>>
>>>>> 1) khugepaged default nice value is 19, but ksmd default nice 
>>>>> value is 5, why this big different?
>>>>> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
>>>>> bottleneck?
>>>>> 3) ksm kernel doc said that "KSM only merges anonymous(private) 
>>>>> pages, never pagecache(file) pages". But where judege it should be 
>>>>> private?
>>>>> 4) ksm kernel doc said that "To avoid the instability and the 
>>>>> resulting false negatives to be permanent, KSM re-initializes the 
>>>>> unstable tree root node to an empty tree, at every KSM pass." But 
>>>>> I can't find where re-initializes the unstable tree, could you 
>>>>> explain me?
>>>>
>>>>
>>>> in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
>>>> root_unstable_tree = RB_ROOT; this will result in 
>>>> root_unstable_tree being empty.
>>>
>>> thanks Izik, what about the other three questions?
>>
>> Question number 2 is beacuse it is forced to work with anonymous 
>> pages, about question 3 - I will have to remember why from the very 
>> begining I wrote it to support only anonymous pages (few years have 
>> been passed), maybe Andrea/Huge have it more hot in their heads?
>
> The initialize goal of design ksm is for virtualization, but now it 
> also can be used for other apps. So can it also support tmpfs or 
> pagecache in the future?

Talking from far memory, I remember it was easier/cleaner to only 
support anonymous pages, until Andrea/Huge won`t remind us why I can`t 
answer this
(Maybe later I will have some time to read a little bit the code again 
and to remember what is going on in linux VM...)

>
>>
>>>
>>>>
>>>>>
>>>>> Thanks in advance. :-)
>>>>>
>>>>> Regards,
>>>>> Chen
>>>>>
>>>>
>>>>
>>>
>>
>>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: ksm questions

[Ni zhan Chen]

[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]

On 10/24/2012 07:46 PM, Izik Eidus wrote:
> On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
>> Hi all,
>>
>> I have some questions about ksm.
>>
>> 1) khugepaged default nice value is 19, but ksmd default nice value 
>> is 5, why this big different?
>> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
>> bottleneck?
>> 3) ksm kernel doc said that "KSM only merges anonymous(private) 
>> pages, never pagecache(file) pages". But where judege it should be 
>> private?
>> 4) ksm kernel doc said that "To avoid the instability and the 
>> resulting false negatives to be permanent, KSM re-initializes the 
>> unstable tree root node to an empty tree, at every KSM pass." But I 
>> can't find where re-initializes the unstable tree, could you explain me?
>
>
> in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
> root_unstable_tree = RB_ROOT; this will result in root_unstable_tree 
> being empty.

Hi Izik,

Another four questions, thank for your patience and excellent codes. :-)

1) Why judge if(page->mapping != expected_mapping) in function 
get_ksm_page called twice? And it also call put_page(page) in the second 
time, when this put_page associated get_page(page) is called?
2)
in function scan_get_next_rmap_item,
if (PageAnon(*page)) ||
     page_trans_compound_anon(*page)) {
     flush_anon_page(vma, *page, ksm_scan.address);
     flush_dcache_page(*page);
     rmap_item = get_next_rmap_item(slot,
????????????????????
why call flush_dcache_page here? in kernel doc 
Documentation/cachetlb.txt, it said that "Any time the kernel writes to 
a page cache page, _OR_ the kernel is about to read from a page cache 
page and user space shared/writable mappings of this page potentially 
exist, this routine is called", it is used for flush page cache related 
cpu cache, but ksmd only scan anonymous page.
3) in function remove_rmap_item_from_tree, how to understand formula age 
= (unsigned char) (ksm_scan.seqr - rmap_item->address); why need aging?
4) in function page_volatile_show, how to understand ksm_pages_volatile 
= ksm_rmap_items - ksm_pages_shared - ksm_pages_sharing - 
ksm_pages_unshared; I mean that how this formula can figure out "how 
many pages changing too fast to be placed in a tree"?

Regards,
Chen

>
>>
>> Thanks in advance. :-)
>>
>> Regards,
>> Chen
>>
>
>


[-- Attachment #2: Type: text/html, Size: 4193 bytes --]

Re: ksm questions

[Ni zhan Chen]

[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]

On 10/24/2012 07:46 PM, Izik Eidus wrote:
> On 10/24/2012 01:19 PM, Ni zhan Chen wrote:
>> Hi all,
>>
>> I have some questions about ksm.
>>
>> 1) khugepaged default nice value is 19, but ksmd default nice value 
>> is 5, why this big different?
>> 2) why ksm doesn't support pagecache and tmpfs now? What's the 
>> bottleneck?
>> 3) ksm kernel doc said that "KSM only merges anonymous(private) 
>> pages, never pagecache(file) pages". But where judege it should be 
>> private?
>> 4) ksm kernel doc said that "To avoid the instability and the 
>> resulting false negatives to be permanent, KSM re-initializes the 
>> unstable tree root node to an empty tree, at every KSM pass." But I 
>> can't find where re-initializes the unstable tree, could you explain me?
>
>
> in scan_get_next_rmap_item(), if (slot == &ksm_mm_head) then we do 
> root_unstable_tree = RB_ROOT; this will result in root_unstable_tree 
> being empty.

Hi Izik,

Another four questions, thank for your patience and excellent codes. :-)

1) Why judge if(page->mapping != expected_mapping) in function 
get_ksm_page called twice? And it also call put_page(page) in the second 
time, when this put_page associated get_page(page) is called?
2)
in function scan_get_next_rmap_item,
if (PageAnon(*page)) ||
     page_trans_compound_anon(*page)) {
     flush_anon_page(vma, *page, ksm_scan.address);
     flush_dcache_page(*page);
     rmap_item = get_next_rmap_item(slot,
????????????????????
why call flush_dcache_page here? in kernel doc 
Documentation/cachetlb.txt, it said that "Any time the kernel writes to 
a page cache page, _OR_ the kernel is about to read from a page cache 
page and user space shared/writable mappings of this page potentially 
exist, this routine is called", it is used for flush page cache related 
cpu cache, but ksmd only scan anonymous page.
3) in function remove_rmap_item_from_tree, how to understand formula age 
= (unsigned char) (ksm_scan.seqr - rmap_item->address); why need aging?
4) in function page_volatile_show, how to understand ksm_pages_volatile 
= ksm_rmap_items - ksm_pages_shared - ksm_pages_sharing - 
ksm_pages_unshared; I mean that how this formula can figure out "how 
many pages changing too fast to be placed in a tree"?

Regards,
Chen

>
>>
>> Thanks in advance. :-)
>>
>> Regards,
>> Chen
>>
>
>


[-- Attachment #2: Type: text/html, Size: 4110 bytes --]