[PATCH] UTC timestamp option for FAT filesystems

[PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

(This patch is based on one submitted last year by Paul Collins, but it
is updated to work with current source, and it uses a different option
name.)

Attached is a patch to provide a new mount option ("utc") for DOS
(vfat/msdos) filesystems, allowing timestamps to be in universal
coordinated time (UTC) rather than local time in applications where
doing this is advantageous.

Since DOS filesystems do not contain time zone information, and
timestamps are typically in local time, a file written in one time
zone will be wrong when read in a different time zone.  When interacting
with filesystems written or read in Windows, using local time is
often necessary, but there are other situations in which the
problems with using local time can be avoided.

Many digital cameras and other portable devices use vfat, and when
traveling across time zones it is inconvenient and confusing to
change the clock setting, but downloading files to a computer in
a different time zone will result in erroneous timestamps.  Also,
it is often necessary to remember to manually change the clock at
daylight saving time changes.  Forgetting to do this also results
in incorrect timestamps.

Setting the clock to UTC is an obvious solution, eliminating
these issues, but Linux is currently hard-coded to always offset the
timestamps by sys_tz.tz_minuteswest in order to interpret them as
local time only.  The new mount option removed this limitation.

						-Joe

[-- Attachment #2: utc-timestamp-option-for-fat-filesystems.patch --]
[-- Type: text/x-patch, Size: 7510 bytes --]

New mount option ("utc") for DOS (vfat/msdos) filesystems allowing
timestamps to be in universal coordinated time (UTC) rather than
local time in applications where doing this is advantageous (like
digital cameras, etc.)

Signed-off-by: Joe Peterson <joe@skyrush.com>
---

diff -ur linux-2.6.26-rc7-git1.old/fs/fat/dir.c linux-2.6.26-rc7-git1/fs/fat/dir.c
--- linux-2.6.26-rc7-git1.old/fs/fat/dir.c	2008-06-24 13:46:02.000000000 -0600
+++ linux-2.6.26-rc7-git1/fs/fat/dir.c	2008-06-24 14:12:56.000000000 -0600
@@ -1082,7 +1082,7 @@
 		goto error_free;
 	}
 
-	fat_date_unix2dos(ts->tv_sec, &time, &date);
+	fat_date_unix2dos(ts->tv_sec, &time, &date, sbi->options.utc);
 
 	de = (struct msdos_dir_entry *)bhs[0]->b_data;
 	/* filling the new directory slots ("." and ".." entries) */
diff -ur linux-2.6.26-rc7-git1.old/fs/fat/inode.c linux-2.6.26-rc7-git1/fs/fat/inode.c
--- linux-2.6.26-rc7-git1.old/fs/fat/inode.c	2008-06-24 13:46:02.000000000 -0600
+++ linux-2.6.26-rc7-git1/fs/fat/inode.c	2008-06-24 14:40:52.000000000 -0600
@@ -382,17 +382,20 @@
 	inode->i_blocks = ((inode->i_size + (sbi->cluster_size - 1))
 			   & ~((loff_t)sbi->cluster_size - 1)) >> 9;
 	inode->i_mtime.tv_sec =
-		date_dos2unix(le16_to_cpu(de->time), le16_to_cpu(de->date));
+		date_dos2unix(le16_to_cpu(de->time), le16_to_cpu(de->date),
+			      sbi->options.utc);
 	inode->i_mtime.tv_nsec = 0;
 	if (sbi->options.isvfat) {
 		int secs = de->ctime_cs / 100;
 		int csecs = de->ctime_cs % 100;
 		inode->i_ctime.tv_sec  =
 			date_dos2unix(le16_to_cpu(de->ctime),
-				      le16_to_cpu(de->cdate)) + secs;
+				      le16_to_cpu(de->cdate),
+				      sbi->options.utc) + secs;
 		inode->i_ctime.tv_nsec = csecs * 10000000;
 		inode->i_atime.tv_sec =
-			date_dos2unix(0, le16_to_cpu(de->adate));
+			date_dos2unix(0, le16_to_cpu(de->adate),
+				      sbi->options.utc);
 		inode->i_atime.tv_nsec = 0;
 	} else
 		inode->i_ctime = inode->i_atime = inode->i_mtime;
@@ -592,11 +595,11 @@
 	raw_entry->attr = fat_attr(inode);
 	raw_entry->start = cpu_to_le16(MSDOS_I(inode)->i_logstart);
 	raw_entry->starthi = cpu_to_le16(MSDOS_I(inode)->i_logstart >> 16);
-	fat_date_unix2dos(inode->i_mtime.tv_sec, &raw_entry->time, &raw_entry->date);
+	fat_date_unix2dos(inode->i_mtime.tv_sec, &raw_entry->time, &raw_entry->date, sbi->options.utc);
 	if (sbi->options.isvfat) {
 		__le16 atime;
-		fat_date_unix2dos(inode->i_ctime.tv_sec,&raw_entry->ctime,&raw_entry->cdate);
-		fat_date_unix2dos(inode->i_atime.tv_sec,&atime,&raw_entry->adate);
+		fat_date_unix2dos(inode->i_ctime.tv_sec,&raw_entry->ctime,&raw_entry->cdate, sbi->options.utc);
+		fat_date_unix2dos(inode->i_atime.tv_sec,&atime,&raw_entry->adate, sbi->options.utc);
 		raw_entry->ctime_cs = (inode->i_ctime.tv_sec & 1) * 100 +
 			inode->i_ctime.tv_nsec / 10000000;
 	}
@@ -848,7 +851,7 @@
 	Opt_charset, Opt_shortname_lower, Opt_shortname_win95,
 	Opt_shortname_winnt, Opt_shortname_mixed, Opt_utf8_no, Opt_utf8_yes,
 	Opt_uni_xl_no, Opt_uni_xl_yes, Opt_nonumtail_no, Opt_nonumtail_yes,
-	Opt_obsolate, Opt_flush, Opt_err,
+	Opt_obsolate, Opt_flush, Opt_utc, Opt_err,
 };
 
 static match_table_t fat_tokens = {
@@ -883,6 +886,7 @@
 	{Opt_obsolate, "cvf_options=%100s"},
 	{Opt_obsolate, "posix"},
 	{Opt_flush, "flush"},
+	{Opt_utc, "utc"},
 	{Opt_err, NULL},
 };
 static match_table_t msdos_tokens = {
@@ -947,6 +951,7 @@
 	opts->utf8 = opts->unicode_xlate = 0;
 	opts->numtail = 1;
 	opts->usefree = opts->nocase = 0;
+	opts->utc = 0;
 	*debug = 0;
 
 	if (!options)
@@ -1036,6 +1041,9 @@
 		case Opt_flush:
 			opts->flush = 1;
 			break;
+		case Opt_utc:
+			opts->utc = 1;
+			break;
 
 		/* msdos specific */
 		case Opt_dots:
diff -ur linux-2.6.26-rc7-git1.old/fs/fat/misc.c linux-2.6.26-rc7-git1/fs/fat/misc.c
--- linux-2.6.26-rc7-git1.old/fs/fat/misc.c	2008-04-16 20:49:44.000000000 -0600
+++ linux-2.6.26-rc7-git1/fs/fat/misc.c	2008-06-24 14:44:05.000000000 -0600
@@ -142,7 +142,7 @@
 };
 
 /* Convert a MS-DOS time/date pair to a UNIX date (seconds since 1 1 70). */
-int date_dos2unix(unsigned short time, unsigned short date)
+int date_dos2unix(unsigned short time, unsigned short date, int utc)
 {
 	int month, year, secs;
 
@@ -156,16 +156,18 @@
 	    ((date & 31)-1+day_n[month]+(year/4)+year*365-((year & 3) == 0 &&
 	    month < 2 ? 1 : 0)+3653);
 			/* days since 1.1.70 plus 80's leap day */
-	secs += sys_tz.tz_minuteswest*60;
+	if (!utc)
+		secs += sys_tz.tz_minuteswest*60;
 	return secs;
 }
 
 /* Convert linear UNIX date to a MS-DOS time/date pair. */
-void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date)
+void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date, int utc)
 {
 	int day, year, nl_day, month;
 
-	unix_date -= sys_tz.tz_minuteswest*60;
+	if (!utc)
+		unix_date -= sys_tz.tz_minuteswest*60;
 
 	/* Jan 1 GMT 00:00:00 1980. But what about another time zone? */
 	if (unix_date < 315532800)
diff -ur linux-2.6.26-rc7-git1.old/fs/msdos/namei.c linux-2.6.26-rc7-git1/fs/msdos/namei.c
--- linux-2.6.26-rc7-git1.old/fs/msdos/namei.c	2008-06-24 13:46:02.000000000 -0600
+++ linux-2.6.26-rc7-git1/fs/msdos/namei.c	2008-06-24 16:32:19.000000000 -0600
@@ -243,6 +243,7 @@
 			   int is_dir, int is_hid, int cluster,
 			   struct timespec *ts, struct fat_slot_info *sinfo)
 {
+	struct msdos_sb_info *sbi = MSDOS_SB(dir->i_sb);
 	struct msdos_dir_entry de;
 	__le16 time, date;
 	int err;
@@ -252,7 +253,7 @@
 	if (is_hid)
 		de.attr |= ATTR_HIDDEN;
 	de.lcase = 0;
-	fat_date_unix2dos(ts->tv_sec, &time, &date);
+	fat_date_unix2dos(ts->tv_sec, &time, &date, sbi->options.utc);
 	de.cdate = de.adate = 0;
 	de.ctime = 0;
 	de.ctime_cs = 0;
diff -ur linux-2.6.26-rc7-git1.old/fs/vfat/namei.c linux-2.6.26-rc7-git1/fs/vfat/namei.c
--- linux-2.6.26-rc7-git1.old/fs/vfat/namei.c	2008-06-24 13:46:03.000000000 -0600
+++ linux-2.6.26-rc7-git1/fs/vfat/namei.c	2008-06-24 14:39:34.000000000 -0600
@@ -621,7 +621,7 @@
 	memcpy(de->name, msdos_name, MSDOS_NAME);
 	de->attr = is_dir ? ATTR_DIR : ATTR_ARCH;
 	de->lcase = lcase;
-	fat_date_unix2dos(ts->tv_sec, &time, &date);
+	fat_date_unix2dos(ts->tv_sec, &time, &date, sbi->options.utc);
 	de->time = de->ctime = time;
 	de->date = de->cdate = de->adate = date;
 	de->ctime_cs = 0;
diff -ur linux-2.6.26-rc7-git1.old/include/linux/msdos_fs.h linux-2.6.26-rc7-git1/include/linux/msdos_fs.h
--- linux-2.6.26-rc7-git1.old/include/linux/msdos_fs.h	2008-06-24 13:46:04.000000000 -0600
+++ linux-2.6.26-rc7-git1/include/linux/msdos_fs.h	2008-06-24 15:03:58.000000000 -0600
@@ -205,7 +205,8 @@
 		 atari:1,         /* Use Atari GEMDOS variation of MS-DOS fs */
 		 flush:1,	  /* write things quickly */
 		 nocase:1,	  /* Does this need case conversion? 0=need case conversion*/
-		 usefree:1;	  /* Use free_clusters for FAT32 */
+		 usefree:1,	  /* Use free_clusters for FAT32 */
+		 utc:1;		  /* Filesystem timestamps are in UTC */
 };
 
 #define FAT_HASH_BITS	8
@@ -428,8 +429,8 @@
 extern void fat_fs_panic(struct super_block *s, const char *fmt, ...);
 extern void fat_clusters_flush(struct super_block *sb);
 extern int fat_chain_add(struct inode *inode, int new_dclus, int nr_cluster);
-extern int date_dos2unix(unsigned short time, unsigned short date);
-extern void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date);
+extern int date_dos2unix(unsigned short time, unsigned short date, int utc);
+extern void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date, int utc);
 extern int fat_sync_bhs(struct buffer_head **bhs, int nr_bhs);
 
 int fat_cache_init(void);

Re: [PATCH] UTC timestamp option for FAT filesystems

[Andi Kleen]

Joe Peterson <joe@skyrush.com> writes:

> Attached is a patch to provide a new mount option ("utc") for DOS
> (vfat/msdos) filesystems, allowing timestamps to be in universal
> coordinated time (UTC) rather than local time in applications where
> doing this is advantageous.

The time zone handling seems racy. e.g. consider the case DST 
changes on that day. You convert before the switch over 
and suddenly the time offset is different.

I'm actually not sure someone in user space is even updating the kernel
idea of the timezone for DST on a switch (generally it was assumed it's some
obsolete BSD concept and that all real programs only use the 
user space glibc implementation that knows all the rules). So 
DST might not be supported at all.

Also even if it worked it seems very limited. If you do that why not
have an option to set an arbitary time zone offset?

-Andi

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

Andi Kleen wrote:
> The time zone handling seems racy. e.g. consider the case DST 
> changes on that day. You convert before the switch over 
> and suddenly the time offset is different.

Are you saying that the tz code is racy in general (I saw the following thread
that you responded to as well):

	http://marc.info/?l=linux-kernel&m=115954057716441&w=2

Or are you saying that the proposed "utc" option makes things racy?

All the "utc" option does (with this patch) is disable application of the
kernel's "tz_minuteswest" offset, so using it actually should remove races with
a DST switch, right?

> I'm actually not sure someone in user space is even updating the kernel
> idea of the timezone for DST on a switch (generally it was assumed it's some
> obsolete BSD concept and that all real programs only use the 
> user space glibc implementation that knows all the rules). So 
> DST might not be supported at all.

I am not sure about that either (i.e. whether tz_minuteswest is altered on DST
changes or if it is only changed at boot).

And I think you are right that the kernel's idea of timezone (i.e.
"tz_minuteswest") is basically obsolete ("man gettimeofday" talks about this).

> Also even if it worked it seems very limited. If you do that why not
> have an option to set an arbitary time zone offset?

Yes, I thought about also providing an option that lets the user specify the
offset at mount - this may be useful for other scenarios (like moving Windows
disks between time zones), but the UTC option, itself, is very useful in that it
let's one use UTC on FAT devices, which is the least problematic way to do
things (e.g. for digital cameras, etc.).

						-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

Joe Peterson wrote:
>> Also even if it worked it seems very limited. If you do that why not
>> have an option to set an arbitary time zone offset?
> 
> Yes, I thought about also providing an option that lets the user specify the
> offset at mount - this may be useful for other scenarios (like moving Windows
> disks between time zones), but the UTC option, itself, is very useful in that it
> let's one use UTC on FAT devices, which is the least problematic way to do
> things (e.g. for digital cameras, etc.).

After thinking about this some more, I actually do not see much point in
allowing an arbitrary offset on FAT.  Only two modes make sense: local
time (current default) and UTC (new proposed option).  Here's the
reason: in "local time" mode (the default in Linux and Windows), a FAT
filesystem records just that to the disk: local time.  For example, on
Windows, it doesn't matter which time zone is in use; a file written at
12:00 will still show a time of 12:00 even if the time zone is changed
on the system.  If a user with a Windows laptop travels around, changing
the time zone, files saved at noon in one time zone will appear to have
the same time when in another.  This does not make sense in a "real
time" sense (it would be possible, for example, to have a file written
later but with an earlier timestamp if the user crosses a time zone
boundary to the west (and sets the time zone) just before saving the
second file).  Such a typical traveling Windows user will have files
saved in many different time zones, so there would be no meaning to
setting an arbitrary time zone offset when mounting such a volume.  The
files on the volume would just have to considered stamped in "local
time", wherever the user happened to be.

Linux's implementation of FAT "local time" mode does this as well.  It
simply adds to a timestamp exactly what the userland level will subtract
when displaying the time, making that timestamp appear as it is
numerically on the disk, in local time.

So FAT filesystem timestamps make sense in one of two modes for a given
volume: local time or UTC.  The former handles volumes used under
Windows well, and the latter would be extremely useful for other devices
which use FAT filesystems but need to have a more sensible "real" time
format (like, as I pointed out, digital cameras).

						-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[OGAWA Hirofumi]

Joe Peterson <joe@skyrush.com> writes:

> So FAT filesystem timestamps make sense in one of two modes for a given
> volume: local time or UTC.  The former handles volumes used under
> Windows well, and the latter would be extremely useful for other devices
> which use FAT filesystems but need to have a more sensible "real" time
> format (like, as I pointed out, digital cameras).

Um, so how does the camera show the timestamp of file? UTC seems to be
hack for changing FAT for Linux. Arbitrary value is more useful and
correct option, I think.

It will be specified the timezone of FAT on one disk. So, the timestamp
is right for specified timezone on Windows always, on Linux should be
always right. If you want to write, of course, you can specify UTC by it
to write timestamp as UTC to FAT.

No?
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

OGAWA Hirofumi wrote:
>> So FAT filesystem timestamps make sense in one of two modes for a given
>> volume: local time or UTC.  The former handles volumes used under
>> Windows well, and the latter would be extremely useful for other devices
>> which use FAT filesystems but need to have a more sensible "real" time
>> format (like, as I pointed out, digital cameras).
> 
> Um, so how does the camera show the timestamp of file? UTC seems to be
> hack for changing FAT for Linux. Arbitrary value is more useful and
> correct option, I think.

Since the camera does not have a concept of time zone, the camera's
clock, itself, would show UTC.  You are correct that one could, instead,
choose an arbitrary time offset when setting the camera's clock, and if
an option existed in Linux to always use this fixed offset on mount, the
Linux timestamp could be correct in this case as well.

However, there are some issues I see with choosing to do this:

1) It is more confusing than using UTC (the user, in essence, is
defining a new absolute reference time similar to UTC but not UTC (and
that matches his local time, at least for 1/2 the year).

2) If the user moves, either the camera and mount offset could be left
at the "wrong" setting that would now be less meaningful than UTC, or
the user could change both.

3) When the daylight saving time switch happens, the camera's time will
now be wrong, even though the Linux time will still be OK - they will
differ by 1 hour unless corrected.  If the camera could (and did) adjust
for DST automatically, then this would give a bad Linux time and
potentially go unnoticed until the fixed offset were corrected (note
that my camera does not ever auto-adjust; I'm not sure if any can).

So in cases 2 and 3, the user would end up needing to change the offset,
perhaps twice a year.  This is one thing I am trying to avoid by just
using UTC.  Using UTC as the "fixed offset" is the only one that makes
sense in that if one is to choose some arbitrary "universal time", it
might as well be UTC (and there are no DST issues with UTC).

> It will be specified the timezone of FAT on one disk. So, the timestamp
> is right for specified timezone on Windows always, on Linux should be
> always right...
> 
> No?

Not really.  Here's an example:

1) Create a folder on FAT in Windows in winter at local 12:00
2) Create a folder on FAT in Windows in summer at local 12:00
3) Notice that in Windows, they both will read "12:00"
4) Mount the volume under Linux with the default "local time" behavior,
   and you will notice the times are off by one hour (because Linux
   adjusts both by the same offset in the kernel, but userland
   correctly adjusts them differently due to different DST status)

This is one case in which Linux's method of FAT local time correction
breaks down, and the real root of the problem is that FAT is not using
UTC.  If, in Linux, the same FAT partition were looked at 6 months
later, the two times would still differ by one hour, but they would both
also be shifted by one hour due to the kernel's different DST correction
(i.e. the "other one" would now read "12:00".

As I mentioned, local time is not a realistic way to handle time anyway,
but the paradigm has existed in DOS/Windows FAT throughout its history
(note that NTFS, on the other hand, behaves more like a real fs).

So, if one were to, instead of UTC, use an arbitrary "fixed" offset when
mounting a FAT partition, the same issue would occur (Linux and Windows
would not agree always).

						-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[OGAWA Hirofumi]

Joe Peterson <joe@skyrush.com> writes:

> Since the camera does not have a concept of time zone, the camera's
> clock, itself, would show UTC.  You are correct that one could, instead,
> choose an arbitrary time offset when setting the camera's clock, and if
> an option existed in Linux to always use this fixed offset on mount, the
> Linux timestamp could be correct in this case as well.

It means the timestamp of FAT on the camera is just wrong.

> However, there are some issues I see with choosing to do this:
>
> 1) It is more confusing than using UTC (the user, in essence, is
> defining a new absolute reference time similar to UTC but not UTC (and
> that matches his local time, at least for 1/2 the year).
>
> 2) If the user moves, either the camera and mount offset could be left
> at the "wrong" setting that would now be less meaningful than UTC, or
> the user could change both.
>
> 3) When the daylight saving time switch happens, the camera's time will
> now be wrong, even though the Linux time will still be OK - they will
> differ by 1 hour unless corrected.  If the camera could (and did) adjust
> for DST automatically, then this would give a bad Linux time and
> potentially go unnoticed until the fixed offset were corrected (note
> that my camera does not ever auto-adjust; I'm not sure if any can).
>
> So in cases 2 and 3, the user would end up needing to change the offset,
> perhaps twice a year.  This is one thing I am trying to avoid by just
> using UTC.  Using UTC as the "fixed offset" is the only one that makes
> sense in that if one is to choose some arbitrary "universal time", it
> might as well be UTC (and there are no DST issues with UTC).

Of course, UTC is right design for on disk format. But, this is FAT, the
writing UTC means you modified the design of FAT. It is not a correct
option, it is just a hack like I said, right?

However, I can accept that hack for many broken devices on realworld,
but, the modifying design is not right option. Do you see what I want
to say?

>> It will be specified the timezone of FAT on one disk. So, the timestamp
>> is right for specified timezone on Windows always, on Linux should be
>> always right...
>> 
>> No?
>
> Not really.  Here's an example:
>
> 1) Create a folder on FAT in Windows in winter at local 12:00
> 2) Create a folder on FAT in Windows in summer at local 12:00
> 3) Notice that in Windows, they both will read "12:00"
> 4) Mount the volume under Linux with the default "local time" behavior,
>    and you will notice the times are off by one hour (because Linux
>    adjusts both by the same offset in the kernel, but userland
>    correctly adjusts them differently due to different DST status)

Yes. And sys_tz is wrong and needs to fix.

> So, if one were to, instead of UTC, use an arbitrary "fixed" offset when
> mounting a FAT partition, the same issue would occur

Yes. To store localtime, complex one is needed.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

OGAWA Hirofumi wrote:
>> Since the camera does not have a concept of time zone, the camera's
>> clock, itself, would show UTC.  You are correct that one could, instead,
>> choose an arbitrary time offset when setting the camera's clock, and if
>> an option existed in Linux to always use this fixed offset on mount, the
>> Linux timestamp could be correct in this case as well.
> 
> It means the timestamp of FAT on the camera is just wrong.

I am not sure what you mean.

> Of course, UTC is right design for on disk format. But, this is FAT, the
> writing UTC means you modified the design of FAT. It is not a correct
> option, it is just a hack like I said, right?

I do not consider this modifying the design of FAT.  FAT does not have
the concept of time zone, DST, or UTC.  It is just a date/time (stamped
on the volume with no info about what that means).  It is customary to
say these times are in local time, and Windows happens to use it as if
it's straight local time (Linux tries to emulate this).  But using FAT
to store UTC instead is not changing the design, it is just using it
differently.  I agree this would not be a good idea when sharing a
volume with Windows, but for cameras, e.g., why not?

I do see what you are saying, in that the semantics of using FAT imply
using local time, but I'm not sure this is reason enough to not allow
someone to adjust how they use the timestamps.  After all, if someone in
Iceland, Morocco, or Algeria writes a file to a FAT filesystem using
Windows, those timestamps *will* be equivalent to UTC.  There is an
inherent problem with using local time anyway (since someone receiving a
disk has no idea which time zone was used when writing the file), so I'm
not sure why it would be wrong to adjust how a particular volume is used.

> However, I can accept that hack for many broken devices on realworld,
> but, the modifying design is not right option. Do you see what I want
> to say?

If it is a hack (and I do not consider it to be so), I still do not see
why the utc option is a design change - how else would you get the
desired behavior?

> Yes. And sys_tz is wrong and needs to fix.

I agree it would be good to fix this, but it is probably not trivial,
and this is a completely independent issue (even if it were fixed,
things like dealing with DST or moving around with a camera are still
problematic).

Also, as I understand it, sys_tz is obsolete, but is still used for
these legacy things, meaning a new way to flag something as "local time"
should probably be used ultimately to fix the sys_tz issue.

>> So, if one were to, instead of UTC, use an arbitrary "fixed" offset when
>> mounting a FAT partition, the same issue would occur
> 
> Yes. To store localtime, complex one is needed.

Right, and I see that as a separate issue.  Back to your original point,
I do not see that letting a user specify a "fixed" arbitrary offset has
any advantage.

Summary: I believe it would be nice if the current local time mode were
fixed to match Windows behavior, but it would also be handy to have the
utc option for situations where that has advantages.

						-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[OGAWA Hirofumi]

Joe Peterson <joe@skyrush.com> writes:

> I do not consider this modifying the design of FAT.  FAT does not have
> the concept of time zone, DST, or UTC.  It is just a date/time (stamped
> on the volume with no info about what that means).  It is customary to
> say these times are in local time, and Windows happens to use it as if
> it's straight local time (Linux tries to emulate this).  But using FAT
> to store UTC instead is not changing the design, it is just using it
> differently.  I agree this would not be a good idea when sharing a
> volume with Windows, but for cameras, e.g., why not?

UTC itself is not wrong. But, I think *"utc" option* is not way to go.

>> However, I can accept that hack for many broken devices on realworld,
>> but, the modifying design is not right option. Do you see what I want
>> to say?
>
> If it is a hack (and I do not consider it to be so), I still do not see
> why the utc option is a design change - how else would you get the
> desired behavior?

I think we agreed to fix sys_tz is good thing.

If so, I'd like to see "tz=xxx" or something for the future, instead of
"utc", don't you think so? I really hate to add the random options.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

OGAWA Hirofumi wrote:
> I think we agreed to fix sys_tz is good thing.

Yep, but doesn't have to be fixed before adding a new option, since they
are separate issues and uses, I think.

> If so, I'd like to see "tz=xxx" or something for the future, instead of
> "utc", don't you think so? I really hate to add the random options.

Can you help be to understand what "tz=xxx" (if not =0) would be good
for?  It seems to me that either you want the "local time" functionality
(which deals, to a degree, with DST, in which case you use the existing
behavior) or you use tz=0 (which is the same as utc).

If someone were to, say, set "tz=-7", it would not work correctly with a
camera or with a Windows-written volume.

For a camera, since DST is not taken into account.  If you really want
to set your camera to local time and are willing to accept the inherent
issues, why not just use local time FAT behavior as it is now in Linux.

If you are talking about using a Windows FAT partition in a different
time zone, setting tz=xxx would not help either, since all files are
just written in local time whichever time zone the person was in (the
traveling laptop example).  Again, the only solution in that case is to
use the default local time behavior.

So, which cases benefit from tz=xxx that would not benefit from utc
(which is eseentially tz=0)?

					-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[James Cloos]

>>>>> "Joe" == Joe Peterson <joe@skyrush.com> writes:

Joe> For a camera, since DST is not taken into account.  If you really
Joe> want to set your camera to local time and are willing to accept the
Joe> inherent issues, why not just use local time FAT behavior as it is
Joe> now in Linux.

Taking the rôle of Devil’s Advocate, why presume that the camera and the
box will have the same idea of local time?  One might set the camera to
local time on a vacation, take some pics, return home and only them mount
the card on one's box.

Of course, that only works well if one uses different cards for each
timezone one visits.  And it may well be unusual for typical digital
camera owners to bother with setting the time.  But it can't be unknown.

It does seem likely that anyone who sets the camera's time at all will
probably prefer localtime.  And if they know that one card has photos
taken during a winter visit to Orlando, and another from a summer trip
to Alaska, they may want to copy over accurate timestamps.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

James Cloos wrote:
> Joe> For a camera, since DST is not taken into account.  If you really
> Joe> want to set your camera to local time and are willing to accept the
> Joe> inherent issues, why not just use local time FAT behavior as it is
> Joe> now in Linux.
> 
> Taking the rôle of Devil’s Advocate, why presume that the camera and the
> box will have the same idea of local time?  One might set the camera to
> local time on a vacation, take some pics, return home and only them mount
> the card on one's box.

Hi James, I think you have hit squarely on the issue.  One could do
this, and the result would be potentially confusing and incorrect.

Say someone lives in Denver, and it is summer.  The UTC offset here is
-6 hours.  That person travels to the east coast, sets his camera to
local time there (UTC - 4), and takes a picture at 9:00am.  He then
returns to Denver and transfers the pic to his PC.  Using the current
Linux FAT mount behavior, the picture would be given a timestamp in
Linux of 15:00 UTC (9:00am local time in Denver, since the code
preserves the local time stamp, which is 9:00).  Now, the picture was
actually taken at 13:00 UTC (7:00am local time in Denver), and since UTC
is what matters in Linux, the actual time is two hours off.

In order to make sense of the timestamp on the picture, he would have to
remember where the picture was taken and do a time zone correction,
remembering that the picture was downloaded in Denver preserving local
time.  It's starting to get a bit mind-bending already, but see the
example of moving to England below for how it can get worse.

> It does seem likely that anyone who sets the camera's time at all will
> probably prefer localtime.  And if they know that one card has photos
> taken during a winter visit to Orlando, and another from a summer trip
> to Alaska, they may want to copy over accurate timestamps.

Some people will perfer the timestamps be in localtime, but this is
really problematic, since times in Linux's filesystems (ext3, etc.), and
even Windows' NTFS, are stored in UTC internally (which is a good thing).

Now things potentially get interesting...

Say that same person who took the picture on the east coast at 9:00am
(but timestamped 9:00am Denver time now) moves to England with his PC.
He then sets his computer's time zone to his new local time zone, and it
now says the picture was taken at 4:00pm!  He can no longer think of the
timestamp as local time, but then again it is not even correct in local
time where the picture was taken, since that would have been 11:00 east
coast time.

Now, if the guy had kept his camera on Denver time during his travels
and no DST change happened between taking the pictures and downloading
them, the correct timestamps would have been preserved (if DST status
was different, however, the timestamp would be off by one hour).  But if
one is going to pick a "universal time" that is not necessarily local
time for the camera while traveling, why not use UTC and avoid all of
the issues?

So I guess my point is that the option I propose will allow people to
set their cameras to UTC and thereby always get unambiguously correct
timestamps.  Plus there is no need to ever change the camera's clock
when traveling or when DST changes happen.  Not all people will choose
to operate this way, so it would be nice to have the choice.

						-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[Bernd Eckenfels]

In article <4867178D.50106@skyrush.com> you wrote:
> So I guess my point is that the option I propose will allow people to
> set their cameras to UTC and thereby always get unambiguously correct
> timestamps.  Plus there is no need to ever change the camera's clock
> when traveling or when DST changes happen.  Not all people will choose
> to operate this way, so it would be nice to have the choice.

And just as a reminder, tools tend to use EXIF informations anyway.

Gruss
Bernd

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

OGAWA Hirofumi wrote:
> If so, I'd like to see "tz=xxx" or something for the future, instead of
> "utc", don't you think so? I really hate to add the random options.

One question: do you see the "tz=" option having "xxx" being a numerical
offset or a time zone.  If the latter, then the DST situation could be
solved (and tz=UTC could be used to get the behavior I like, but I still
wonder if a timezone feature on FAT is useful.  Also, would it not
require the kernel to look up timezone info and keep track of DST
changes?  I thought the trend was to push that kind of thing into
userland...

					Thanks, Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[OGAWA Hirofumi]

Joe Peterson <joe@skyrush.com> writes:

> OGAWA Hirofumi wrote:
>> If so, I'd like to see "tz=xxx" or something for the future, instead of
>> "utc", don't you think so? I really hate to add the random options.
>
> One question: do you see the "tz=" option having "xxx" being a numerical
> offset or a time zone.

Ah, sorry. I was imagining, e.g. POSIX TZ string like
"EST5EDT,M3.2.0/2,M11.1.0 " or something, it is not a simple numerical.

> If the latter, then the DST situation could be
> solved (and tz=UTC could be used to get the behavior I like, but I still
> wonder if a timezone feature on FAT is useful.  Also, would it not
> require the kernel to look up timezone info and keep track of DST
> changes? I thought the trend was to push that kind of thing into
> userland...

Yes. If it can do in userland (on embeded system too), it's very
good. (also, nls stuff is same situation.) But, I'm not sure yet,
need to think about it.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

[PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]

Hi Andrew,

Attached is a patch to provide a new mount option ("tz=UTC") for DOS
(vfat/msdos) filesystems, allowing timestamps to be in coordinated
universal time (UTC) rather than local time in applications where
doing this is advantageous.

In particular, portable devices that use fat/vfat (such as digital
cameras) can benefit from using UTC in their internal clocks, thus
avoiding daylight saving time errors and general time ambiguity issues.
 The user of the device does not have to worry about changing the time
when moving from place or when daylight saving changes.

The new mount option, when set, disables the counter-adjustment that
Linux currently makes to FAT timestamp info in anticipation of the
normal userspace time zone correction.  When used in this new mode, all
daylight saving time and time zone handling is done in userspace as is
normal for many other filesystems (like ext3).  The default mode, which
remains unchanged, is still appropriate when mounting volumes written in
Windows (because of its use of local time).

(I originally based this patch on one submitted last year by Paul
Collins, but I updated it to work with current source and changed
variable/option naming.  Ogawa Hirofumi (who maintains these
filesystems) and I discussed this patch at length on lkml, and he
suggested using the option name in the attached version of the patch.
Barry Bouwsma pointed out a good addition to the patch as well.)

						-Thanks, Joe

[-- Attachment #2: utc-timestamp-option-for-fat-filesystems.patch --]
[-- Type: text/x-patch, Size: 7771 bytes --]

New mount option ("tz=UTC") for FAT (vfat/msdos) filesystems allowing
timestamps to be in coordinated universal time (UTC) rather than
local time in applications where doing this is advantageous (like
digital cameras, etc.)

Signed-off-by: Joe Peterson <joe@skyrush.com>
Acked-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
---

diff -puNr a/fs/fat/dir.c b/fs/fat/dir.c
--- a/fs/fat/dir.c	2008-06-25 08:53:35.676901351 -0600
+++ b/fs/fat/dir.c	2008-06-26 12:58:56.096272279 -0600
@@ -1082,7 +1082,7 @@ int fat_alloc_new_dir(struct inode *dir,
 		goto error_free;
 	}
 
-	fat_date_unix2dos(ts->tv_sec, &time, &date);
+	fat_date_unix2dos(ts->tv_sec, &time, &date, sbi->options.tz_utc);
 
 	de = (struct msdos_dir_entry *)bhs[0]->b_data;
 	/* filling the new directory slots ("." and ".." entries) */
diff -puNr a/fs/fat/inode.c b/fs/fat/inode.c
--- a/fs/fat/inode.c	2008-06-25 08:53:35.676901351 -0600
+++ b/fs/fat/inode.c	2008-06-26 14:57:07.087942764 -0600
@@ -382,17 +382,20 @@ static int fat_fill_inode(struct inode *
 	inode->i_blocks = ((inode->i_size + (sbi->cluster_size - 1))
 			   & ~((loff_t)sbi->cluster_size - 1)) >> 9;
 	inode->i_mtime.tv_sec =
-		date_dos2unix(le16_to_cpu(de->time), le16_to_cpu(de->date));
+		date_dos2unix(le16_to_cpu(de->time), le16_to_cpu(de->date),
+			      sbi->options.tz_utc);
 	inode->i_mtime.tv_nsec = 0;
 	if (sbi->options.isvfat) {
 		int secs = de->ctime_cs / 100;
 		int csecs = de->ctime_cs % 100;
 		inode->i_ctime.tv_sec  =
 			date_dos2unix(le16_to_cpu(de->ctime),
-				      le16_to_cpu(de->cdate)) + secs;
+				      le16_to_cpu(de->cdate),
+				      sbi->options.tz_utc) + secs;
 		inode->i_ctime.tv_nsec = csecs * 10000000;
 		inode->i_atime.tv_sec =
-			date_dos2unix(0, le16_to_cpu(de->adate));
+			date_dos2unix(0, le16_to_cpu(de->adate),
+				      sbi->options.tz_utc);
 		inode->i_atime.tv_nsec = 0;
 	} else
 		inode->i_ctime = inode->i_atime = inode->i_mtime;
@@ -592,11 +595,14 @@ retry:
 	raw_entry->attr = fat_attr(inode);
 	raw_entry->start = cpu_to_le16(MSDOS_I(inode)->i_logstart);
 	raw_entry->starthi = cpu_to_le16(MSDOS_I(inode)->i_logstart >> 16);
-	fat_date_unix2dos(inode->i_mtime.tv_sec, &raw_entry->time, &raw_entry->date);
+	fat_date_unix2dos(inode->i_mtime.tv_sec, &raw_entry->time,
+			  &raw_entry->date, sbi->options.tz_utc);
 	if (sbi->options.isvfat) {
 		__le16 atime;
-		fat_date_unix2dos(inode->i_ctime.tv_sec,&raw_entry->ctime,&raw_entry->cdate);
-		fat_date_unix2dos(inode->i_atime.tv_sec,&atime,&raw_entry->adate);
+		fat_date_unix2dos(inode->i_ctime.tv_sec, &raw_entry->ctime,
+				  &raw_entry->cdate, sbi->options.tz_utc);
+		fat_date_unix2dos(inode->i_atime.tv_sec, &atime,
+				  &raw_entry->adate, sbi->options.tz_utc);
 		raw_entry->ctime_cs = (inode->i_ctime.tv_sec & 1) * 100 +
 			inode->i_ctime.tv_nsec / 10000000;
 	}
@@ -836,6 +842,8 @@ static int fat_show_options(struct seq_f
 	}
 	if (sbi->options.flush)
 		seq_puts(m, ",flush");
+	if (opts->tz_utc)
+		seq_puts(m, ",tz=UTC");
 
 	return 0;
 }
@@ -848,7 +856,7 @@ enum {
 	Opt_charset, Opt_shortname_lower, Opt_shortname_win95,
 	Opt_shortname_winnt, Opt_shortname_mixed, Opt_utf8_no, Opt_utf8_yes,
 	Opt_uni_xl_no, Opt_uni_xl_yes, Opt_nonumtail_no, Opt_nonumtail_yes,
-	Opt_obsolate, Opt_flush, Opt_err,
+	Opt_obsolate, Opt_flush, Opt_tz_utc, Opt_err,
 };
 
 static match_table_t fat_tokens = {
@@ -883,6 +891,7 @@ static match_table_t fat_tokens = {
 	{Opt_obsolate, "cvf_options=%100s"},
 	{Opt_obsolate, "posix"},
 	{Opt_flush, "flush"},
+	{Opt_tz_utc, "tz=UTC"},
 	{Opt_err, NULL},
 };
 static match_table_t msdos_tokens = {
@@ -947,6 +956,7 @@ static int parse_options(char *options, 
 	opts->utf8 = opts->unicode_xlate = 0;
 	opts->numtail = 1;
 	opts->usefree = opts->nocase = 0;
+	opts->tz_utc = 0;
 	*debug = 0;
 
 	if (!options)
@@ -1036,6 +1046,9 @@ static int parse_options(char *options, 
 		case Opt_flush:
 			opts->flush = 1;
 			break;
+		case Opt_tz_utc:
+			opts->tz_utc = 1;
+			break;
 
 		/* msdos specific */
 		case Opt_dots:
diff -puNr a/fs/fat/misc.c b/fs/fat/misc.c
--- a/fs/fat/misc.c	2008-06-25 08:53:35.676901351 -0600
+++ b/fs/fat/misc.c	2008-06-26 12:59:08.627376422 -0600
@@ -142,7 +142,7 @@ static int day_n[] = {
 };
 
 /* Convert a MS-DOS time/date pair to a UNIX date (seconds since 1 1 70). */
-int date_dos2unix(unsigned short time, unsigned short date)
+int date_dos2unix(unsigned short time, unsigned short date, int tz_utc)
 {
 	int month, year, secs;
 
@@ -156,16 +156,18 @@ int date_dos2unix(unsigned short time, u
 	    ((date & 31)-1+day_n[month]+(year/4)+year*365-((year & 3) == 0 &&
 	    month < 2 ? 1 : 0)+3653);
 			/* days since 1.1.70 plus 80's leap day */
-	secs += sys_tz.tz_minuteswest*60;
+	if (!tz_utc)
+		secs += sys_tz.tz_minuteswest*60;
 	return secs;
 }
 
 /* Convert linear UNIX date to a MS-DOS time/date pair. */
-void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date)
+void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date, int tz_utc)
 {
 	int day, year, nl_day, month;
 
-	unix_date -= sys_tz.tz_minuteswest*60;
+	if (!tz_utc)
+		unix_date -= sys_tz.tz_minuteswest*60;
 
 	/* Jan 1 GMT 00:00:00 1980. But what about another time zone? */
 	if (unix_date < 315532800)
diff -puNr a/fs/msdos/namei.c b/fs/msdos/namei.c
--- a/fs/msdos/namei.c	2008-06-25 08:53:35.696902410 -0600
+++ b/fs/msdos/namei.c	2008-06-26 12:58:42.255055811 -0600
@@ -243,6 +243,7 @@ static int msdos_add_entry(struct inode 
 			   int is_dir, int is_hid, int cluster,
 			   struct timespec *ts, struct fat_slot_info *sinfo)
 {
+	struct msdos_sb_info *sbi = MSDOS_SB(dir->i_sb);
 	struct msdos_dir_entry de;
 	__le16 time, date;
 	int err;
@@ -252,7 +253,7 @@ static int msdos_add_entry(struct inode 
 	if (is_hid)
 		de.attr |= ATTR_HIDDEN;
 	de.lcase = 0;
-	fat_date_unix2dos(ts->tv_sec, &time, &date);
+	fat_date_unix2dos(ts->tv_sec, &time, &date, sbi->options.tz_utc);
 	de.cdate = de.adate = 0;
 	de.ctime = 0;
 	de.ctime_cs = 0;
diff -puNr a/fs/vfat/namei.c b/fs/vfat/namei.c
--- a/fs/vfat/namei.c	2008-06-25 08:53:35.686902663 -0600
+++ b/fs/vfat/namei.c	2008-06-26 12:58:27.073724049 -0600
@@ -621,7 +621,7 @@ shortname:
 	memcpy(de->name, msdos_name, MSDOS_NAME);
 	de->attr = is_dir ? ATTR_DIR : ATTR_ARCH;
 	de->lcase = lcase;
-	fat_date_unix2dos(ts->tv_sec, &time, &date);
+	fat_date_unix2dos(ts->tv_sec, &time, &date, sbi->options.tz_utc);
 	de->time = de->ctime = time;
 	de->date = de->cdate = de->adate = date;
 	de->ctime_cs = 0;
diff -puNr a/include/linux/msdos_fs.h b/include/linux/msdos_fs.h
--- a/include/linux/msdos_fs.h	2008-06-25 08:53:17.435303045 -0600
+++ b/include/linux/msdos_fs.h	2008-06-26 14:58:40.626135522 -0600
@@ -205,7 +205,8 @@ struct fat_mount_options {
 		 atari:1,         /* Use Atari GEMDOS variation of MS-DOS fs */
 		 flush:1,	  /* write things quickly */
 		 nocase:1,	  /* Does this need case conversion? 0=need case conversion*/
-		 usefree:1;	  /* Use free_clusters for FAT32 */
+		 usefree:1,	  /* Use free_clusters for FAT32 */
+		 tz_utc:1;	  /* Filesystem timestamps are in UTC */
 };
 
 #define FAT_HASH_BITS	8
@@ -428,8 +429,9 @@ extern int fat_flush_inodes(struct super
 extern void fat_fs_panic(struct super_block *s, const char *fmt, ...);
 extern void fat_clusters_flush(struct super_block *sb);
 extern int fat_chain_add(struct inode *inode, int new_dclus, int nr_cluster);
-extern int date_dos2unix(unsigned short time, unsigned short date);
-extern void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date);
+extern int date_dos2unix(unsigned short time, unsigned short date, int tz_utc);
+extern void fat_date_unix2dos(int unix_date, __le16 *time, __le16 *date,
+			      int tz_utc);
 extern int fat_sync_bhs(struct buffer_head **bhs, int nr_bhs);
 
 int fat_cache_init(void);

Re: [PATCH] UTC timestamp option for FAT filesystems

[Andrew Morton]

On Thu, 26 Jun 2008 23:12:05 -0600
Joe Peterson <joe@skyrush.com> wrote:

> (I originally based this patch on one submitted last year by Paul
> Collins, but I updated it to work with current source and changed
> variable/option naming.  Ogawa Hirofumi (who maintains these
> filesystems) and I discussed this patch at length on lkml, and he
> suggested using the option name in the attached version of the patch.
> Barry Bouwsma pointed out a good addition to the patch as well.)

It'd be nice to gather Paul's signed-off-by:.  And to add a Cc to
Barry.  That way they get a mention in the kernel logs (pure chick
magnetry!) and they are presumably people who can help review the
change for us.

But I don't know their email addresses.

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

Sounds good!  And yes, you are most certainly right about the chick
magnet effect!  :)

I sent an email to Paul asking for a sign-off (copied here too - not
sure the address is still valid), and Barry is also on the cc list of
this email (he was in the thread, so he will likely reply).

				Thanks, Joe

Andrew Morton wrote:
> On Thu, 26 Jun 2008 23:12:05 -0600
> Joe Peterson <joe@skyrush.com> wrote:
> 
>> (I originally based this patch on one submitted last year by Paul
>> Collins, but I updated it to work with current source and changed
>> variable/option naming.  Ogawa Hirofumi (who maintains these
>> filesystems) and I discussed this patch at length on lkml, and he
>> suggested using the option name in the attached version of the patch.
>> Barry Bouwsma pointed out a good addition to the patch as well.)
> 
> It'd be nice to gather Paul's signed-off-by:.  And to add a Cc to
> Barry.  That way they get a mention in the kernel logs (pure chick
> magnetry!) and they are presumably people who can help review the
> change for us.
> 
> But I don't know their email addresses.
> 

Re: [PATCH] UTC timestamp option for FAT filesystems

[Andrew Morton]

On Thu, 26 Jun 2008 23:12:05 -0600
Joe Peterson <joe@skyrush.com> wrote:

> New mount option ("tz=UTC") for FAT (vfat/msdos) filesystems allowing
> timestamps to be in coordinated universal time (UTC) rather than
> local time in applications where doing this is advantageous (like
> digital cameras, etc.)

Can we update Documentation/filesystems/vfat.txt please?

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

[-- Attachment #1: Type: text/plain, Size: 121 bytes --]

Andrew Morton wrote:
> Can we update Documentation/filesystems/vfat.txt please?

Here's a patch to that doc file.

	-Joe

[-- Attachment #2: vfat_doc.patch --]
[-- Type: text/x-patch, Size: 811 bytes --]

Signed-off-by: Joe Peterson <joe@skyrush.com>

--

--- linux/Documentation/filesystems/vfat.txt.orig	2008-05-31 12:07:25.000000000 -0500
+++ linux/Documentation/filesystems/vfat.txt	2008-07-02 09:40:48.000000000 -0500
@@ -96,6 +96,14 @@
 			emulate the Windows 95 rule for create.
 		 Default setting is `lower'.
 
+tz=UTC        -- Interpret timestamps as UTC rather than local time.
+                 This option disables the conversion of timestamps
+                 between local time (as used by Windows on FAT) and UTC
+                 (which Linux uses internally).  This is particuluarly
+                 useful when mounting devices (like digital cameras)
+                 that are set to UTC in order to avoid the pitfalls of
+                 local time.
+
 <bool>: 0,1,yes,no,true,false
 
 TODO

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

OGAWA Hirofumi wrote:
> Of course, UTC is right design for on disk format. But, this is FAT, the
> writing UTC means you modified the design of FAT. It is not a correct
> option, it is just a hack like I said, right?

I just looked at the FAT32 specification from Microsoft, and nowhere
does it say that FAT timestamps are in local time.  It does not mention
local time, time standards, or time zones at all.  Here it is:

http://www.microsoft.com/whdc/system/platform/firmware/fatgen.mspx

So there is nothing inherent in FAT's design or specification that
requires dates and times to be in local time.  It appears it is up to
the OS or device to convey a meaning by how the timestamps are used.
For this reason, I do not consider it a hack or a design
change/violation to optionally use UTC for the timestamps on a given FAT
volume.

					-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[OGAWA Hirofumi]

Joe Peterson <joe@skyrush.com> writes:

> I just looked at the FAT32 specification from Microsoft, and nowhere
> does it say that FAT timestamps are in local time.  It does not mention
> local time, time standards, or time zones at all.  Here it is:
>
> http://www.microsoft.com/whdc/system/platform/firmware/fatgen.mspx
>
> So there is nothing inherent in FAT's design or specification that
> requires dates and times to be in local time.  It appears it is up to
> the OS or device to convey a meaning by how the timestamps are used.
> For this reason, I do not consider it a hack or a design
> change/violation to optionally use UTC for the timestamps on a given FAT
> volume.

This is not interesting argument. That mentions MSDOS/Windows
everywhere, you just ignored it.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [PATCH] UTC timestamp option for FAT filesystems

[barry bouwsma]

Moin moin!

May I just add a resounding YES-YES-YES to the message which was
posted earlier today as
Message-ID: <>
oh b*gger, that doesn't help when snarfing from a web archive
(voluntary use as a non-subscirber of a webmail interface prevents
me from including a References: header without more effort than I
care to do now, not that this would help)


I was actually just about to post an almost identical message today
to clean out my repository of hacks, with almost the same justifications.

In addition, may I request that not only the option `-o utc,...' be
accepted, but also as an alternate, the string `gmt', not because it's
more accurate (it isn't) but because it seems to be tradition to use
GMT+/-xxxx in lots of places.

Another justification which wasn't mentioned, but which I would put
forth, is that last time I checked, the BSDen were using UTC as their
MSDOS timestamp base, so such an option would make it easier to
exchange data between BSD and Linux machines with a normal USB stick
or flash card.  And I was doing that a year or three ago when I hacked
my hack.


I'll have to do a line-by-line comparison with my hacked code (last
applied to 2.6.24-ish) to make sure I don't have any differences, but
in order to allow users the option to specify the seeped-in-tradition-
especially-for-us-old-f*rts `gmt', here's the trivial-to-correct
section of code -- may well be different than in the patch since I
didn't save it before being kicked offline:

in  fs/fat/inode.c
    880         {Opt_debug, "debug"},
    881         {Opt_immutable, "sys_immutable"},
    882 /* XXX HACK */
    883         {Opt_utc, "utc"},
    884         {Opt_utc, "gmt"},
    885         {Opt_obsolate, "conv=binary"},
    886         {Opt_obsolate, "conv=text"},

(I use `-o utc' myself but would prefer to be flexible to alternate uses)
I notice this most when I fail to include the needed patches when
updating kernel

thanks
barry bouwsma


      

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

barry bouwsma wrote:
> May I just add a resounding YES-YES-YES...

Thanks for your feedback and support!

> In addition, may I request that not only the option `-o utc,...' be
> accepted, but also as an alternate, the string `gmt', not because it's
> more accurate (it isn't) but because it seems to be tradition to use
> GMT+/-xxxx in lots of places.

I do understand what you are saying, however I would suggest not making
"gmt" a synonym for the following reasons:

1) Better to keep the options simple and less cluttered
2) GMT is not strictly the same as UTC (according to Wikipedia, it
   usually is used to mean the same thing, but it can also mean
   "UT1" (or the less exact "UT"), which is not exactly equal to UTC).
   So UTC is the more precise term.

> Another justification which wasn't mentioned, but which I would put
> forth, is that last time I checked, the BSDen were using UTC as their
> MSDOS timestamp base, so such an option would make it easier to
> exchange data between BSD and Linux machines with a normal USB stick
> or flash card.  And I was doing that a year or three ago when I hacked
> my hack.

Ah, then that is, indeed, another good reason.

						-Thanks, Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[barry bouwsma]

--- On Wed, 6/25/08, Joe Peterson <joe@skyrush.com> wrote:

> > In addition, may I request that not only the option
> `-o utc,...' be
> > accepted, but also as an alternate, the string
> `gmt', not because it's
> 
> I do understand what you are saying, however I would
> suggest not making
> "gmt" a synonym for the following reasons:

Fair enough, I won't press the matter.  I'm just one of those old
f*rts who still remembers the american WWV talking clock and how
unnerving it was after years to be hearing ``Coordinated Universal
Time'' so I deserve to wither and drop off the vine.  Also, in case
of all my consumer equipment, I think 100% of it refers to GMT+2
(in my present time zone), so I wonder if in different languages,
there's a preference for casual use of the phrase GMT (a quick
look in my dictionary shows an entry for GMT, nothing for UTC,
but then, it has no entry for `operating system' or its translated
equivalent.

Wait, I said I wouldn't press the matter.

Anyway, I'm doing a quick diff against my code.  (Mine also
contains no documentation either, which is why I didn't post my
hacks a couple days earlier.)  I see some things that are likely
irrelevant; however, here's something missing -- WARNING, line
numbers are based on hacked 2.6.24-ish code; later kernels I've
built so far have been without these hacks -- PLEASE REVIEW AND
IF APPLICABLE, APPLY BY HAND


--- /mnt/usr/local/src/linux-2.6.24/fs/fat/inode.c-DIST 2008-01-30 10:59:19.000000000 +0100
+++ /mnt/usr/local/src/linux-2.6.24/fs/fat/inode.c      2008-04-13 07:09:10.000000000 +0200
@@ -826,6 +829,9 @@ static int fat_show_options(struct seq_f
                seq_puts(m, ",showexec");
        if (opts->sys_immutable)
                seq_puts(m, ",sys_immutable");
+/* XXX HACK */
+       if (opts->utc)
+               seq_puts(m, ",utc");
        if (!isvfat) {
                if (opts->dotsOK)
                        seq_puts(m, ",dotsOK=yes");


The above will result in something like
/dev/sdc1 on /opt type vfat (ro,utc,dmask=222,fmask=133,shortname=win95)
(sorry if your code has this and I overlooked it)

Apart from this, if your diffs compile and work, it looks like only
trivial coding style differences between ours (thus yours are better).


thanks,
barry bouwsma


      

Re: [PATCH] UTC timestamp option for FAT filesystems

[Joe Peterson]

barry bouwsma wrote:
> I think 100% of it refers to GMT+2
> (in my present time zone), so I wonder if in different languages,
> there's a preference for casual use of the phrase GMT (a quick
> look in my dictionary shows an entry for GMT, nothing for UTC,
> but then, it has no entry for `operating system' or its translated
> equivalent.

Well, your question certainly prompted me to go and read up on it again.
 In my work (unmanned space missions - solar system stuff), we deal with
many representations of time, and so I am used to being anal about it.
For example, there is GMT, UTC, TDT, ET, etc.  They all differ ever so
slightly.

> --- /mnt/usr/local/src/linux-2.6.24/fs/fat/inode.c-DIST 2008-01-30 10:59:19.000000000 +0100
> +++ /mnt/usr/local/src/linux-2.6.24/fs/fat/inode.c      2008-04-13 07:09:10.000000000 +0200
> @@ -826,6 +829,9 @@ static int fat_show_options(struct seq_f
>                 seq_puts(m, ",showexec");
>         if (opts->sys_immutable)
>                 seq_puts(m, ",sys_immutable");
> +/* XXX HACK */
> +       if (opts->utc)
> +               seq_puts(m, ",utc");
>         if (!isvfat) {
>                 if (opts->dotsOK)
>                         seq_puts(m, ",dotsOK=yes");
> 

Indeed!  Thanks, I'll put this in the patch.  The above is actually not
needed for the mount command or /etc/mtab, but it *is* used for, e.g.,
/proc/mounts.

						-Joe

Re: [PATCH] UTC timestamp option for FAT filesystems

[barry bouwsma]

--- On Thu, 6/26/08, Joe Peterson <joe@skyrush.com> wrote:

> > (in my present time zone), so I wonder if in different languages,
> > there's a preference for casual use of the phrase GMT (a quick
> > look in my dictionary shows an entry for GMT, nothing for UTC,

> Well, your question certainly prompted me to go and read up on it
> again.

Oops, didn't mean to...  Very sorry.

Thinking more about my hack (dates back to at least Aug.2005,
if not before -- can't find older code I ran), originally I
wanted to be able to allow the user to specify the timezone/
offset -- for example, if you were, hypothetically, to visit
me where I am now, and your camera had been set to your local
time, there's no way I'd easily be seeing the correct times.

Fortunately, I began to think better of it after realizing how
much work and bother it would be, and decided that if my friends
were to be giving me USB sticks and whatnot to mount that weren't
with UTC timestamps or in my local timezone, then they weren't
going to remain my friends for much longer.  Unless they meant
to give me those sticks without expecting them back, in which
case I'd compromise and let them remain my friends just to see
what else they might have up their sleeves to give me.


> In my work (unmanned space missions - solar system stuff), we deal with
> many representations of time, and so I am used to being anal about it.
> For example, there is GMT, UTC, TDT, ET, etc.  They all differ ever so
> slightly.

Agreed.  And technically UTC is correct.  And while for you, the
difference between UTC and GMT by now would be the difference
between a probe floating gently down to the surface of the sun
(oops, make that some planet) or embedding itself a metre or
two under the surface, for too many casual users, GMT and UTC
are interchangeable.  In the case of my camera, technically it's
drifted to something that's neither UTC or GMT.  If this notebook
doesn't get it's NTP sync from my other machine's radio clock,
it rapidly becomes far removed from either.

Unfortunately, even the Linux filesystem codes are either careless
about referring to GMT (I too am guilty, rereading the comment to
explain my hack which I wrote, before I decided to add `gmt' as an
inaccurate alternative), or are indeed accurate and some filesystems
are in fact based on GMT, not UTC -- I don't wish to know.  I see
some 35-odd references to GMT in the fs/*/* files, try a grep
just for, well, to see whether people are as careful with UTC/GMT
as they should be.

And it's even worse in the next timezone over from me -- I've seen
websites referring to broadcasts in GMT -- no, they are in *local*
*British* time, whether summer or winter time.  Arrgh.  NURSE!

As it's a new option, no need to cater to (or encourage) inaccurate
usage.  I'd rather advocate and see compatibility in userspace
utilities for my BSD-afflicted brain, like `mount -u' == `mount
-o remount -o rw' and `patch -C' == `patch --dry-run', but that's
off-topic, as is this whole discussion.  Blacklist this e-mail,
please?


Anyway, thanks for trying to get this patch (probably 3 1/2 years
old by me, if I find my oldest source) into the kernel, and I'll
shut up now.

barry bouwsma