[PATCH 0/2] ksm: take dirty bit as reference to avoid volatile pages

[PATCH 0/2] ksm: take dirty bit as reference to avoid volatile pages

[Nai Xia]

Currently, ksm uses page checksum to detect volatile pages. Izik Eidus 
suggested that we could use pte dirty bit to optimize. This patch series
adds this new logic.

Preliminary benchmarks show that the scan speed is improved by up to 16 
times on volatile transparent huge pages and up to 8 times on volatile 
regular pages.

Following is the test program to show this top speed up (you need to make 
ksmd takes about more than 90% of the cpu and watch the ksm/full_scans).

  #include <stdio.h>
  #include <stdlib.h>
  #include <errno.h>
  #include <string.h>
  #include <unistd.h>
  #include <sys/mman.h>
  
  #define MADV_MERGEABLE   12
  
  
  #define SIZE (2000*1024*1024)
  #define PAGE_SIZE 4096
  
  int main(int argc, char **argv)
  {
  	unsigned char *p;
  	int j;
  	int ret;
  
          p = mmap(NULL, SIZE, PROT_WRITE|PROT_READ,
                   MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
  	
  	if (p == MAP_FAILED) {
  		printf("mmap error\n");
  		return 0;
  	}
      
          ret = madvise(p, SIZE, MADV_MERGEABLE);
      
          if (ret==-1) {
                  printf("madvise failed \n");
                  return 0;
          }
  
  	
  	memset(p, 1, SIZE);
  
  	while (1) {
  		for (j=0; j<SIZE; j+=PAGE_SIZE) {
  			*((long*)(p+j+PAGE_SIZE-4)) = random();
  		}
  	}
  
  	return 0;
  }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] ksm: take dirty bit as reference to avoid volatile pages

[Izik Eidus]

On 03/28/2011 04:14 PM, Nai Xia wrote:
> Currently, ksm uses page checksum to detect volatile pages. Izik Eidus
> suggested that we could use pte dirty bit to optimize. This patch series
> adds this new logic.
>

Hi,

One small note:
When kvm will use ksm on intel cpu with extended page tables support, 
the cpu won`t track
dirty bit, therefore the calc_hash() logic should be used in such cases
(untill intel will fadd this support in their cpus)...

Moreover I think that even though that AMD nested page tables does 
update dirty bit, you still need
to sync it with the host page table using mmu notifiers ?

(Not that on regular application use case of ksm any of this should be 
an issue)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 0/2] ksm: take dirty bit as reference to avoid volatile pages

[Nai Xia]

On Monday 28 March 2011 22:59:02 Izik Eidus wrote:
> On 03/28/2011 04:14 PM, Nai Xia wrote:
> > Currently, ksm uses page checksum to detect volatile pages. Izik Eidus
> > suggested that we could use pte dirty bit to optimize. This patch series
> > adds this new logic.
> >
> 
> Hi,
> 
> One small note:
> When kvm will use ksm on intel cpu with extended page tables support, 
> the cpu won`t track
> dirty bit, therefore the calc_hash() logic should be used in such cases
> (untill intel will fadd this support in their cpus)...
> 
> Moreover I think that even though that AMD nested page tables does 
> update dirty bit, you still need
> to sync it with the host page table using mmu notifiers ?
> 
> (Not that on regular application use case of ksm any of this should be 
> an issue)
> 
> 

Hmm, I will consider these two issues in the next version. Thanks for
input!


Nai

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>