All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andy Lutomirski <luto@amacapital.net>
To: Pavel Emelyanov <xemul@parallels.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Glauber Costa <glommer@parallels.com>,
	KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
	Matt Mackall <mpm@selenic.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Linux MM <linux-mm@kvack.org>
Subject: Re: [PATCH 4/5] mm: soft-dirty bits for user memory changes tracking
Date: Thu, 11 Jul 2013 11:46:54 -0700	[thread overview]
Message-ID: <51DEFD9E.7010703@mit.edu> (raw)
In-Reply-To: <517FED64.4020400@parallels.com>

On 04/30/2013 09:12 AM, Pavel Emelyanov wrote:
> The soft-dirty is a bit on a PTE which helps to track which pages a task
> writes to. In order to do this tracking one should
> 
>   1. Clear soft-dirty bits from PTEs ("echo 4 > /proc/PID/clear_refs)
>   2. Wait some time.
>   3. Read soft-dirty bits (55'th in /proc/PID/pagemap entries)
> 
> To do this tracking, the writable bit is cleared from PTEs when the
> soft-dirty bit is. Thus, after this, when the task tries to modify a page
> at some virtual address the #PF occurs and the kernel sets the soft-dirty
> bit on the respective PTE.
> 
> Note, that although all the task's address space is marked as r/o after the
> soft-dirty bits clear, the #PF-s that occur after that are processed fast.
> This is so, since the pages are still mapped to physical memory, and thus
> all the kernel does is finds this fact out and puts back writable, dirty
> and soft-dirty bits on the PTE.
> 
> Another thing to note, is that when mremap moves PTEs they are marked with
> soft-dirty as well, since from the user perspective mremap modifies the
> virtual memory at mremap's new address.
> 
> 

Sorry I'm late to the party -- I didn't notice this until the lwn
article this week.

How does this get munmap + mmap right?  mremap marks things soft-dirty,
but unmapping and remapping seems like it will result in the soft-dirty
bit being cleared.  For that matter, won't this sequence also end up wrong:

 - clear_refs
 - Write to mapping
 - Page and pte evicted due to memory pressure
 - Read from mapping -- clean page faulted back in
 - pte soft-dirty is now clear ?!?

--Andy

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Andy Lutomirski <luto@amacapital.net>
To: Pavel Emelyanov <xemul@parallels.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Glauber Costa <glommer@parallels.com>,
	KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
	Matt Mackall <mpm@selenic.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Linux MM <linux-mm@kvack.org>
Subject: Re: [PATCH 4/5] mm: soft-dirty bits for user memory changes tracking
Date: Thu, 11 Jul 2013 11:46:54 -0700	[thread overview]
Message-ID: <51DEFD9E.7010703@mit.edu> (raw)
In-Reply-To: <517FED64.4020400@parallels.com>

On 04/30/2013 09:12 AM, Pavel Emelyanov wrote:
> The soft-dirty is a bit on a PTE which helps to track which pages a task
> writes to. In order to do this tracking one should
> 
>   1. Clear soft-dirty bits from PTEs ("echo 4 > /proc/PID/clear_refs)
>   2. Wait some time.
>   3. Read soft-dirty bits (55'th in /proc/PID/pagemap entries)
> 
> To do this tracking, the writable bit is cleared from PTEs when the
> soft-dirty bit is. Thus, after this, when the task tries to modify a page
> at some virtual address the #PF occurs and the kernel sets the soft-dirty
> bit on the respective PTE.
> 
> Note, that although all the task's address space is marked as r/o after the
> soft-dirty bits clear, the #PF-s that occur after that are processed fast.
> This is so, since the pages are still mapped to physical memory, and thus
> all the kernel does is finds this fact out and puts back writable, dirty
> and soft-dirty bits on the PTE.
> 
> Another thing to note, is that when mremap moves PTEs they are marked with
> soft-dirty as well, since from the user perspective mremap modifies the
> virtual memory at mremap's new address.
> 
> 

Sorry I'm late to the party -- I didn't notice this until the lwn
article this week.

How does this get munmap + mmap right?  mremap marks things soft-dirty,
but unmapping and remapping seems like it will result in the soft-dirty
bit being cleared.  For that matter, won't this sequence also end up wrong:

 - clear_refs
 - Write to mapping
 - Page and pte evicted due to memory pressure
 - Read from mapping -- clean page faulted back in
 - pte soft-dirty is now clear ?!?

--Andy

  parent reply	other threads:[~2013-07-11 18:46 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-04-30 16:10 [PATCH 0/5] mm: Ability to monitor task memory changes (v4) Pavel Emelyanov
2013-04-30 16:11 ` [PATCH 1/5] clear_refs: sanitize accepted commands declaration Pavel Emelyanov
2013-04-30 16:11   ` Pavel Emelyanov
2013-04-30 16:11 ` [PATCH 2/5] clear_refs: introduce private struct for mm_walk Pavel Emelyanov
2013-04-30 16:11   ` Pavel Emelyanov
2013-04-30 16:12 ` [PATCH 3/5] pagemap: introduce pagemap_entry_t without pmshift bits Pavel Emelyanov
2013-04-30 16:12   ` Pavel Emelyanov
2013-04-30 16:12 ` [PATCH 4/5] mm: soft-dirty bits for user memory changes tracking Pavel Emelyanov
2013-04-30 16:12   ` Pavel Emelyanov
2013-05-03 11:36   ` Xiao Guangrong
2013-05-03 11:36     ` Xiao Guangrong
2013-05-03 19:09     ` Pavel Emelyanov
2013-05-03 19:09       ` Pavel Emelyanov
2013-07-11 18:46   ` Andy Lutomirski [this message]
2013-07-11 18:46     ` Andy Lutomirski
2013-07-12 18:38     ` Pavel Emelyanov
2013-07-12 18:38       ` Pavel Emelyanov
2013-04-30 16:12 ` [PATCH 5/5] pagemap: Prepare to reuse constant bits with page-shitf Pavel Emelyanov
2013-04-30 16:12   ` Pavel Emelyanov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=51DEFD9E.7010703@mit.edu \
    --to=luto@amacapital.net \
    --cc=akpm@linux-foundation.org \
    --cc=glommer@parallels.com \
    --cc=kosaki.motohiro@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mpm@selenic.com \
    --cc=mtosatti@redhat.com \
    --cc=torvalds@linux-foundation.org \
    --cc=xemul@parallels.com \
    --cc=xiaoguangrong@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.