how do file-mapped (mmapped) pages become dirty?

how do file-mapped (mmapped) pages become dirty?

[Amir Hermelin]

Hi,
When a process mmaps a file, how does the kernel know the memory has been
written to (and hence the page is dirty)? Is this done by setting the
protected flag, and when the memory is first written to it's set to dirty?
What function is responsible for this setting? And when will the page be
written back to disk (i.e. where's the flusher located)?

Thanks for any help,
Amir.

Re: how do file-mapped (mmapped) pages become dirty?

[Helge Hafting]

Amir Hermelin wrote:
> Hi,
> When a process mmaps a file, how does the kernel know the memory has been
> written to (and hence the page is dirty)? Is this done by setting the
> protected flag, and when the memory is first written to it's set to dirty?

No need on a i386.  The cpu sets the page dirty in the pagetables
when it is written to.  It doesn't matter what kind of page
it is.

> What function is responsible for this setting? And when will the page be
> written back to disk (i.e. where's the flusher located)?
> 
When there's memory pressure, or a sync.

Helge Hafting

Re: how do file-mapped (mmapped) pages become dirty?

[Chris Friesen]

Helge Hafting wrote:
> Amir Hermelin wrote:

>> What function is responsible for this setting? And when will the page be
>> written back to disk (i.e. where's the flusher located)?
>>
> When there's memory pressure, or a sync.

Note however that you need an msync() -- fsync() and fdatasync() do not 
catch changes to mmapped pages.

Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how do file-mapped (mmapped) pages become dirty?

[Richard B. Johnson]

On Tue, 28 Oct 2003, Chris Friesen wrote:

> Helge Hafting wrote:
> > Amir Hermelin wrote:
>
> >> What function is responsible for this setting? And when will the page be
> >> written back to disk (i.e. where's the flusher located)?
> >>
> > When there's memory pressure, or a sync.
>
> Note however that you need an msync() -- fsync() and fdatasync() do not
> catch changes to mmapped pages.
>
> Chris
>

Sure they do. fsync() will sync the whole file, regardless of
whether or not it's been mapped. msync()  allows you to sync
only a  specific portion and control how that portion is
handled with some flags.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: how do file-mapped (mmapped) pages become dirty?

[Chris Friesen]

Richard B. Johnson wrote:
> On Tue, 28 Oct 2003, Chris Friesen wrote:

>>Note however that you need an msync() -- fsync() and fdatasync() do not
>>catch changes to mmapped pages.

> Sure they do. fsync() will sync the whole file, regardless of
> whether or not it's been mapped. msync()  allows you to sync
> only a  specific portion and control how that portion is
> handled with some flags.

According to Rik van Riel and HPA, fsync() does not check page tables 
for dirty bits.  This is confirmed by my testing, in which msync() takes 
significantly longer than fsync() for the same file (in a ramdisk). 
msync() on the 3 pages dirtied took 39 usec, while fsync() on the whole 
200KB file took 12 usec.

Also, some googling leads to to believe that if the file is mmapped and 
you close() it, it won't get synced.

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: how do file-mapped (mmapped) pages become dirty?

[Richard B. Johnson]

On Tue, 28 Oct 2003, Amir Hermelin wrote:

> Hi,
> When a process mmaps a file, how does the kernel know the memory has been
> written to (and hence the page is dirty)? Is this done by setting the

This is automatically done by the CPU (no overhead) with Intel
CPUs.

> protected flag, and when the memory is first written to it's set to dirty?
> What function is responsible for this setting? And when will the page be
> written back to disk (i.e. where's the flusher located)?
>

If you start to run low on memory, the disk buffer(s) may be flushed
to the hardware. Otherwise, you need an explicit fsync() or a close().

Any reads by others will always show updated changes. You don't
need to do anything special.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.22 on an i686 machine (797.90 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: how do file-mapped (mmapped) pages become dirty?

[bill davidsen]

In article <006901c39d50$0b1313d0$2501a8c0@CARTMAN>,
Amir Hermelin <amir@montilio.com> wrote:

| When a process mmaps a file, how does the kernel know the memory has been
| written to (and hence the page is dirty)? Is this done by setting the
| protected flag, and when the memory is first written to it's set to dirty?
| What function is responsible for this setting? And when will the page be
| written back to disk (i.e. where's the flusher located)?

At least on x86, the CPU sets the dirty bit on write, although once
upon a time less capable CPUs did it the way you suggest. That said, I
think copy on write is still done the way you suggest, but look at the
code if you really care. Or wait for someone to tell me I'm wrong ;-)

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.