Host where KSM appears to save a negative amount of memory

Host where KSM appears to save a negative amount of memory

[Chris Webb]

We're running KSM on kernel 2.6.39.2 with hosts running a number qemu-kvm
virtual machines, and it has consistently been saving us a useful amount of
RAM.

To monitor the effective amount of memory saved, I've been looking at the
difference between /sys/kernel/mm/ksm/pages_sharing and pages_shared. On a
typical 32GB host, this has been coming out as at least a hundred thousand
or so, which is presumably half to one gigabyte worth of 4k pages.

However, this morning we've spotted something odd - a host where
pages_sharing is smaller than pages_shared, giving a negative saving by the
above calculation:

  # cat /sys/kernel/mm/ksm/pages_sharing
  1099994
  # cat /sys/kernel/mm/ksm/pages_shared
  1761313

I think this means my interpretation of these values must be wrong, as I
presumably can't have more pages being shared than instances of their use!
Can anyone shed any light on what might be going on here for me? Am I
misinterpreting these values, or does this look like it might be an
accounting bug? (If the latter, what useful debug info can I extract from
the system to help identify it?)

Best wishes,

Chris.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Host where KSM appears to save a negative amount of memory

[Hugh Dickins]

On Sun, 21 Aug 2011, Chris Webb wrote:

> We're running KSM on kernel 2.6.39.2 with hosts running a number qemu-kvm
> virtual machines, and it has consistently been saving us a useful amount of
> RAM.
> 
> To monitor the effective amount of memory saved, I've been looking at the
> difference between /sys/kernel/mm/ksm/pages_sharing and pages_shared. On a
> typical 32GB host, this has been coming out as at least a hundred thousand
> or so, which is presumably half to one gigabyte worth of 4k pages.
> 
> However, this morning we've spotted something odd - a host where
> pages_sharing is smaller than pages_shared, giving a negative saving by the
> above calculation:
> 
>   # cat /sys/kernel/mm/ksm/pages_sharing
>   1099994
>   # cat /sys/kernel/mm/ksm/pages_shared
>   1761313
> 
> I think this means my interpretation of these values must be wrong, as I
> presumably can't have more pages being shared than instances of their use!
> Can anyone shed any light on what might be going on here for me? Am I
> misinterpreting these values, or does this look like it might be an
> accounting bug? (If the latter, what useful debug info can I extract from
> the system to help identify it?)

Your interpretation happens to be wrong, it is expected behaviour,
but I agree it's a little odd.

KSM chooses to show the numbers pages_shared and pages_sharing as
exclusive counts: pages_sharing indicates the saving being made.  So it
would be perfectly reasonable to add those two numbers together to get
the "total" number of pages sharing, the number you expected it to show;
but it doesn't make sense to subtract shared from sharing.

(I think Documentation/vm/ksm.txt does make that clear.)

But you'd be right to question further, how come pages_sharing is less
than pages_shared: what is a shared page if it's not being shared with
anything else?  (And, at the extreme, it might be that all those 1099994
pages_sharing are actually sharing the same one of the pages_shared.)

It's a page that was shared with (at least one) others before, but all
but one of these instances have got freed since, and we've left this
page in the "shared tree", so that it can be more quickly matched up
with duplicates in future when they appear, as seems quite likely.

We don't actively do anything to move them out of the shared state:
some effort was needed to get them there, and no disadvantage in leaving
them like that; but yes, it is misleading to describe them as "shared".

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Host where KSM appears to save a negative amount of memory

[Chris Webb]

Hugh Dickins <hughd@google.com> writes:

> KSM chooses to show the numbers pages_shared and pages_sharing as
> exclusive counts: pages_sharing indicates the saving being made.  So it
> would be perfectly reasonable to add those two numbers together to get
> the "total" number of pages sharing, the number you expected it to show;
> but it doesn't make sense to subtract shared from sharing.

Hi. Many thanks for your helpful and detailed explanation. I've fixed our
monitoring to correctly use just pages_sharing to measure the savings. I
think I just assumed the meanings of pages_shared and pages_sharing from
their names. This means that ksm has been saving even more memory than we
thought on our hosts in the past!

Best wishes,

Chris.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>