mmu_notifers, pte_dirty questions

mmu_notifers, pte_dirty questions

[Avi Kivity]

Why no notifer when testing and clearing the dirty bit?

(*clear_flush_dirty)(...).

> static int page_mkclean_one(struct page *page, struct vm_area_struct *vma,
>                 unsigned long address)
> {
>     struct mm_struct *mm = vma->vm_mm;
>     pte_t *pte;
>     spinlock_t *ptl;
>     int ret = 0;
>
>     pte = page_check_address(page, mm, address, &ptl, 1);
>     if (!pte)
>         goto out;
>
>     if (pte_dirty(*pte) || pte_write(*pte)) {
>         pte_t entry;
>
>         flush_cache_page(vma, address, pte_pfn(*pte));
>         entry = ptep_clear_flush_notify(vma, address, pte);
>         entry = pte_wrprotect(entry);
>         entry = pte_mkclean(entry);
>         set_pte_at(mm, address, pte, entry);

set_pte_at_notify()?  without this (or clear_flush_dirty) Linux will 
assume all ptes are now clean; if the guest writes to a page nothing 
will catch it.

-> with set_pte_at_notify(), we can drop the spte and mark the page as 
dirty, so the next write will re-instantiate the spte
-> with ->clear_flush_dirty(), we can track the dirty state without 
dropping the spte.

>         ret = 1;
>     }
>
>     pte_unmap_unlock(pte, ptl);
> out:
>     return ret;

I'm probably missing something big as I can't see how this works.

-- 
error compiling committee.c: too many arguments to function

Re: mmu_notifers, pte_dirty questions

[Andrea Arcangeli]

On Sun, Jun 06, 2010 at 03:07:27PM +0300, Avi Kivity wrote:
> Why no notifer when testing and clearing the dirty bit?
> 
> (*clear_flush_dirty)(...).
> 
> > static int page_mkclean_one(struct page *page, struct vm_area_struct *vma,
> >                 unsigned long address)
> > {
> >     struct mm_struct *mm = vma->vm_mm;
> >     pte_t *pte;
> >     spinlock_t *ptl;
> >     int ret = 0;
> >
> >     pte = page_check_address(page, mm, address, &ptl, 1);
> >     if (!pte)
> >         goto out;
> >
> >     if (pte_dirty(*pte) || pte_write(*pte)) {
> >         pte_t entry;
> >
> >         flush_cache_page(vma, address, pte_pfn(*pte));
> >         entry = ptep_clear_flush_notify(vma, address, pte);
> >         entry = pte_wrprotect(entry);
> >         entry = pte_mkclean(entry);
> >         set_pte_at(mm, address, pte, entry);
> 
> set_pte_at_notify()?  without this (or clear_flush_dirty) Linux will 
> assume all ptes are now clean; if the guest writes to a page nothing 
> will catch it.
> 
> -> with set_pte_at_notify(), we can drop the spte and mark the page as 
> dirty, so the next write will re-instantiate the spte
> -> with ->clear_flush_dirty(), we can track the dirty state without 
> dropping the spte.
> 
> >         ret = 1;
> >     }
> >
> >     pte_unmap_unlock(pte, ptl);
> > out:
> >     return ret;
> 
> I'm probably missing something big as I can't see how this works.

Under the PT lock it's safe to keep the PTE zero, just the pte must be
non zero again before pte_unmap_unlock.

The sptes are all gone by the time ptep_clear_flush_notify returns
(also gup-fast is stopped with the IPI of the flush) and no spte can
be established again before pte_unmap_unlock runs, so it's all safe as
far as I can tell.

set_pte_at_notify might prevent a minor fault though.

Re: mmu_notifers, pte_dirty questions

[Avi Kivity]

On 06/06/2010 09:36 PM, Andrea Arcangeli wrote:
> On Sun, Jun 06, 2010 at 03:07:27PM +0300, Avi Kivity wrote:
>    
>> Why no notifer when testing and clearing the dirty bit?
>>
>> (*clear_flush_dirty)(...).
>>
>>      
>>> static int page_mkclean_one(struct page *page, struct vm_area_struct *vma,
>>>                  unsigned long address)
>>> {
>>>      struct mm_struct *mm = vma->vm_mm;
>>>      pte_t *pte;
>>>      spinlock_t *ptl;
>>>      int ret = 0;
>>>
>>>      pte = page_check_address(page, mm, address,&ptl, 1);
>>>      if (!pte)
>>>          goto out;
>>>
>>>      if (pte_dirty(*pte) || pte_write(*pte)) {
>>>          pte_t entry;
>>>
>>>          flush_cache_page(vma, address, pte_pfn(*pte));
>>>          entry = ptep_clear_flush_notify(vma, address, pte);
>>>          entry = pte_wrprotect(entry);
>>>          entry = pte_mkclean(entry);
>>>          set_pte_at(mm, address, pte, entry);
>>>        
>> set_pte_at_notify()?  without this (or clear_flush_dirty) Linux will
>> assume all ptes are now clean; if the guest writes to a page nothing
>> will catch it.
>>
>> ->  with set_pte_at_notify(), we can drop the spte and mark the page as
>> dirty, so the next write will re-instantiate the spte
>> ->  with ->clear_flush_dirty(), we can track the dirty state without
>> dropping the spte.
>>
>>      
>>>          ret = 1;
>>>      }
>>>
>>>      pte_unmap_unlock(pte, ptl);
>>> out:
>>>      return ret;
>>>        
>> I'm probably missing something big as I can't see how this works.
>>      
> Under the PT lock it's safe to keep the PTE zero, just the pte must be
> non zero again before pte_unmap_unlock.
>
> The sptes are all gone by the time ptep_clear_flush_notify returns
> (also gup-fast is stopped with the IPI of the flush) and no spte can
> be established again before pte_unmap_unlock runs, so it's all safe as
> far as I can tell.
>
>    

Somehow  I missed the ptep_clear_flush_notify()...  so all should be fine.

> set_pte_at_notify might prevent a minor fault though.
>    

I'm thinking of how to implement speculative write access for kvm: 
consider a read fault for a writeable page.  We could install a 
writeable spte with the dirty bit clear, and examine the dirty bit at 
pte_clear_flush_notify() time and transfer it to the page flags.  
However I can't see where the mm code checks the pte dirty bit for 
anonymous pages?  Does it assume anonymous pages are always dirty? (they 
could have a clean copy in swap, no?)

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.