Time nonlinearity (gettimeofday vs. mtime)

Time nonlinearity (gettimeofday vs. mtime)

[Matthias Urlichs]

Lately I've seen this ugliness:

13:39:06.000313 clock_gettime(CLOCK_REALTIME, {1259325546, 341196}) = 0 <0.000010>
13:39:06.000685 mkdir("/var/tmp/CP_FileTest_TempFolder_d0AOiP/tempFolder1", 0777) = 0 <0.000043>
13:39:06.000973 stat64("/var/tmp/CP_FileTest_TempFolder_d0AOiP/tempFolder1", {st_dev=makedev(252, 2), st_ino=1919104, st_mode=S_IFDIR|0755, st_nlink=2, st_uid=501, st_gid=501, st_blksize=4096, st_blocks=8, st_size=4096, st_atime=2009/11/27-13:39:05, st_mtime=2009/11/27-13:39:05, st_ctime=2009/11/27-13:39:05}) = 0 <0.000015>

This strace says that st.st_mtime is smaller than time.tv_sec even though the time was acquired earlier.
Apparently, the problem is that ext3 uses a cached time value for performance.

Question: Is there a reason that the cached time is not updated every time somebody calls gettimeofday() or clock_gettime()?
Or did just that nobody notice this problem yet?

Re: Time nonlinearity (gettimeofday vs. mtime)

[Andi Kleen]

Matthias Urlichs <matthias@urlichs.de> writes:

> Lately I've seen this ugliness:

You would probably see it more pronounced on a file system which supports
sub seconds time stamps.

> 13:39:06.000313 clock_gettime(CLOCK_REALTIME, {1259325546, 341196}) = 0 <0.000010>
> 13:39:06.000685 mkdir("/var/tmp/CP_FileTest_TempFolder_d0AOiP/tempFolder1", 0777) = 0 <0.000043>
> 13:39:06.000973 stat64("/var/tmp/CP_FileTest_TempFolder_d0AOiP/tempFolder1", {st_dev=makedev(252, 2), st_ino=1919104, st_mode=S_IFDIR|0755, st_nlink=2, st_uid=501, st_gid=501, st_blksize=4096, st_blocks=8, st_size=4096, st_atime=2009/11/27-13:39:05, st_mtime=2009/11/27-13:39:05, st_ctime=2009/11/27-13:39:05}) = 0 <0.000015>
>
> This strace says that st.st_mtime is smaller than time.tv_sec even though the time was acquired earlier.
> Apparently, the problem is that ext3 uses a cached time value for performance.
>
> Question: Is there a reason that the cached time is not updated every time somebody calls gettimeofday() or clock_gettime()?

At least on x86-64 gtod() and clock_gettime() run in user space and 
are unable to update anything in the kernel.

Also in general both calls are extremly time critical and making them
slower for anything else would be a bad idea.

Internally the file systems use the time from last timer tick (=
jiffies), rounded to their granuality. This is needed to avoid non
monotonicity which can break programs. 

In theory the file system could always get the current time, but it
would need to be rounded down anyways for the same reason, so you
would still see the same effect. Also it would be slower of course,
and not really help.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: Time nonlinearity (gettimeofday vs. mtime)

[john stultz]

On Fri, Nov 27, 2009 at 5:31 AM, Matthias Urlichs <matthias@urlichs.de> wrote:
> Lately I've seen this ugliness:
>
> 13:39:06.000313 clock_gettime(CLOCK_REALTIME, {1259325546, 341196}) = 0 <0.000010>
> 13:39:06.000685 mkdir("/var/tmp/CP_FileTest_TempFolder_d0AOiP/tempFolder1", 0777) = 0 <0.000043>
> 13:39:06.000973 stat64("/var/tmp/CP_FileTest_TempFolder_d0AOiP/tempFolder1", {st_dev=makedev(252, 2), st_ino=1919104, st_mode=S_IFDIR|0755, st_nlink=2, st_uid=501, st_gid=501, st_blksize=4096, st_blocks=8, st_size=4096, st_atime=2009/11/27-13:39:05, st_mtime=2009/11/27-13:39:05, st_ctime=2009/11/27-13:39:05}) = 0 <0.000015>
>
> This strace says that st.st_mtime is smaller than time.tv_sec even though the time was acquired earlier.
> Apparently, the problem is that ext3 uses a cached time value for performance.
>
> Question: Is there a reason that the cached time is not updated every time somebody calls gettimeofday() or clock_gettime()?
> Or did just that nobody notice this problem yet?

This behavior is expected. For performance reasons (some clocksources
take 1.3us per read), most filesystem code uses current_kernel_time()
which provides the time at the last timer tick.

If you check mainline, there's a new CLOCK_REALTIME_COARSE clockid,
that provides the same behavior as the filesystem timestamps.

thanks
-john