mtime updates for mmapped files.

mtime updates for mmapped files.

[Rogier Wolff]

Hi,

I  wrote a small app yesterday that updates a file by mmapping the 
file (RW), changing the thing around, and then exiting. 

This did not trigger a change in the mtime of the file. Thus rsync
didn't pick up that the file had changed.

I understand that tracking every change to a RW mmapped file is
costly, and thus unfeasable, but shouldn't then the close cause a
mtime update?

The server where this happened is running 2.6.21, so my apologies if
this has already been corrected.

	Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ

Re: mtime updates for mmapped files.

[Dave Young]

On Jan 16, 2008 5:46 PM, Rogier Wolff <R.E.Wolff@bitwizard.nl> wrote:
>
> Hi,
>
> I  wrote a small app yesterday that updates a file by mmapping the
> file (RW), changing the thing around, and then exiting.
>
> This did not trigger a change in the mtime of the file. Thus rsync
> didn't pick up that the file had changed.
>
> I understand that tracking every change to a RW mmapped file is
> costly, and thus unfeasable, but shouldn't then the close cause a
> mtime update?
>
> The server where this happened is running 2.6.21, so my apologies if
> this has already been corrected.

This is an old bug, don't know if it was fixed or not. Please see
http://lkml.org/lkml/2006/5/17/138

>
>         Roger.
>
> --
> ** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
> **    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
> *-- BitWizard writes Linux device drivers for any device you may have! --*
> Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
> Does it sit on the couch all day? Is it unemployed? Please be specific!
> Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: mtime updates for mmapped files.

[Miklos Szeredi]

> >
> > Hi,
> >
> > I  wrote a small app yesterday that updates a file by mmapping the
> > file (RW), changing the thing around, and then exiting.
> >
> > This did not trigger a change in the mtime of the file. Thus rsync
> > didn't pick up that the file had changed.
> >
> > I understand that tracking every change to a RW mmapped file is
> > costly, and thus unfeasable, but shouldn't then the close cause a
> > mtime update?
> >
> > The server where this happened is running 2.6.21, so my apologies if
> > this has already been corrected.
> 
> This is an old bug, don't know if it was fixed or not. Please see
> http://lkml.org/lkml/2006/5/17/138

It's not fixed yet, but Anton Salikhmetov is currently working on it:

http://lkml.org/lkml/2008/1/15/202

Miklos

Re: mtime updates for mmapped files.

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1008 bytes --]

On Wed, 16 Jan 2008 18:03:17 +0800, Dave Young said:
> On Jan 16, 2008 5:46 PM, Rogier Wolff <R.E.Wolff@bitwizard.nl> wrote:
> >
> > Hi,
> >
> > I  wrote a small app yesterday that updates a file by mmapping the
> > file (RW), changing the thing around, and then exiting.
> >
> > This did not trigger a change in the mtime of the file. Thus rsync
> > didn't pick up that the file had changed.
> >
> > I understand that tracking every change to a RW mmapped file is
> > costly, and thus unfeasable, but shouldn't then the close cause a
> > mtime update?
> >
> > The server where this happened is running 2.6.21, so my apologies if
> > this has already been corrected.
> 
> This is an old bug, don't know if it was fixed or not. Please see
> http://lkml.org/lkml/2006/5/17/138

If it was fixed, we'd not be having the currently ongoing long thread about how
to fix it. :)

See also:
http://lkml.org/lkml/2007/2/21/242
http://lkml.org/lkml/2008/1/7/234

(Note the last is the start of the *current* thread...)


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: mtime updates for mmapped files.

[Anton Salikhmetov]

2008/1/16, Rogier Wolff <R.E.Wolff@bitwizard.nl>:
>
> Hi,
>
> I  wrote a small app yesterday that updates a file by mmapping the
> file (RW), changing the thing around, and then exiting.
>
> This did not trigger a change in the mtime of the file. Thus rsync
> didn't pick up that the file had changed.
>
> I understand that tracking every change to a RW mmapped file is
> costly, and thus unfeasable, but shouldn't then the close cause a
> mtime update?
>
> The server where this happened is running 2.6.21, so my apologies if
> this has already been corrected.

Unfortunately, this issue has not been fully fixed yet.

My last attempt (http://lkml.org/lkml/2008/1/15/202) to solve
this problem has a couple of drawbacks:

1) calling a possibly sleeping function from atomic context -
I've already corrected this;

2) there's a very special case with retouching the memory-mapped data
after a call to msync() with MS_ASYNC.

I'm still working on the latter case, but I guess that I have found a solution.

If you badly need a quick fix, I can send my working unreleased patch to you.
I reckon that your particular problem will be fixed by this patch.
Let me know if you want that.

However, if your application calls msync() with the MS_ASYNC flag,
it's better to wait a little bit more - I'll release the next version
of my solution
shortly.

>
>         Roger.
>
> --
> ** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
> **    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
> *-- BitWizard writes Linux device drivers for any device you may have! --*
> Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
> Does it sit on the couch all day? Is it unemployed? Please be specific!
> Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: mtime updates for mmapped files.

[Rogier Wolff]

On Wed, Jan 16, 2008 at 02:22:49PM +0300, Anton Salikhmetov wrote:
> Unfortunately, this issue has not been fully fixed yet.
> 
> My last attempt (http://lkml.org/lkml/2008/1/15/202) to solve
> this problem has a couple of drawbacks:
> 
> 1) calling a possibly sleeping function from atomic context -
> I've already corrected this;
> 
> 2) there's a very special case with retouching the memory-mapped data
> after a call to msync() with MS_ASYNC.
 
> I'm still working on the latter case, but I guess that I have found
> a solution.
 
> If you badly need a quick fix, I can send my working unreleased
> patch to you.  I reckon that your particular problem will be fixed
> by this patch.  Let me know if you want that.

No need. Thanks for the offer. 

> However, if your application calls msync() with the MS_ASYNC flag,
> it's better to wait a little bit more - I'll release the next version
> of my solution
> shortly.

My application calls "exit" (*). My workaround was to type

	touch <destfile>

to get things to work. In my situation, I was worried about the more
general case, and not really about my personal situation.

I could integrate the "touch <destfile>" into my application, but I
think it is the OS's duty to do this for me. This would be the easiest
non-manual fix for me.

	Roger.

(*) So please check that: 
	- mmap file
	- change mmapped area
	- exit
also works (modifies the timestamp). 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
**    Delftechpark 26 2628 XH  Delft, The Netherlands. KVK: 27239233    **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ