* Wrong free clusters count on FAT32
@ 2007-04-19 8:57 DervishD
2007-04-19 10:27 ` Boaz Harrosh
2007-04-19 11:52 ` Juergen Beisert
0 siblings, 2 replies; 25+ messages in thread
From: DervishD @ 2007-04-19 8:57 UTC (permalink / raw)
To: Linux-kernel
Hi all :)
I have a portable device with a FAT32 formatted hard disk in it, and
everytime I delete a file in the device *using the device itself to do
it* the device increases its count of free space and if I plug the
device in a Windows system, Windows agrees on the free space. Linux
doesn't. Linux believes that the files are still there ocuppying space,
and I have to run fsck.vfat to fix the problem.
As far as I've seen, the device is probably not updating correctly
the list of free clusters, but it doesn't seem to worry about it,
neither does Windows. So my question is: is there any way of making
Linux "bug compatible" with Windows? If Windows itself don't worry about
the free cluster count and computes the free space in some other way,
then it can be done. I haven't seen anything in mount(8) neither
googling.
Apart from not using the device itself to delete files (and probably
not using Windows for that, either) and to run fsck.vfat now and then,
is anything I can do to avoid this problem?
Thanks a lot in advance :)
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-19 8:57 Wrong free clusters count on FAT32 DervishD
@ 2007-04-19 10:27 ` Boaz Harrosh
2007-04-19 14:23 ` DervishD
2007-04-19 11:52 ` Juergen Beisert
1 sibling, 1 reply; 25+ messages in thread
From: Boaz Harrosh @ 2007-04-19 10:27 UTC (permalink / raw)
To: Linux-kernel
DervishD wrote:
> Hi all :)
>
> I have a portable device with a FAT32 formatted hard disk in it, and
> everytime I delete a file in the device *using the device itself to do
> it* the device increases its count of free space and if I plug the
> device in a Windows system, Windows agrees on the free space. Linux
> doesn't. Linux believes that the files are still there ocuppying space,
> and I have to run fsck.vfat to fix the problem.
>
> As far as I've seen, the device is probably not updating correctly
> the list of free clusters, but it doesn't seem to worry about it,
> neither does Windows. So my question is: is there any way of making
> Linux "bug compatible" with Windows? If Windows itself don't worry about
> the free cluster count and computes the free space in some other way,
> then it can be done. I haven't seen anything in mount(8) neither
> googling.
>
> Apart from not using the device itself to delete files (and probably
> not using Windows for that, either) and to run fsck.vfat now and then,
> is anything I can do to avoid this problem?
>
> Thanks a lot in advance :)
>
> Raúl Núñez de Arenas Coronado
>
Not that I know how to fix it. But have you tried running chkdsk on a
device deleted files in windows. Just that I know, that windows would
silently ignore fat errors and will only report them in chkdsk.
Deleting files in windows works fine. There is no problems with it and
windows updates everything. I do that all the time.
(I would even theorize that if you delete files on the device and than farther
delete more files on windows, than windows will fix the problem on the fly)
So it is "bug compatible" with the device, and error recovery
compatible with windows.
Boaz
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-19 8:57 Wrong free clusters count on FAT32 DervishD
2007-04-19 10:27 ` Boaz Harrosh
@ 2007-04-19 11:52 ` Juergen Beisert
2007-04-19 14:19 ` DervishD
1 sibling, 1 reply; 25+ messages in thread
From: Juergen Beisert @ 2007-04-19 11:52 UTC (permalink / raw)
To: linux-kernel; +Cc: DervishD
On Thursday 19 April 2007 10:57, DervishD wrote:
> I have a portable device with a FAT32 formatted hard disk in it, and
> everytime I delete a file in the device *using the device itself to do
> it* the device increases its count of free space and if I plug the
> device in a Windows system, Windows agrees on the free space. Linux
> doesn't. Linux believes that the files are still there ocuppying space,
> and I have to run fsck.vfat to fix the problem.
As I remember: It needs a large amount of time to calculate the free space on
a big FAT32 system. So the last free sector count is also stored. When
mounting this filesystem you don't need to walk through the whole FAT to
calculate the available space, you can use this "cached" value instead. And
this cached value seems not to be updated in your portable device.
Maybe it helps, but maybe I'm wrong.
Juergen
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-19 11:52 ` Juergen Beisert
@ 2007-04-19 14:19 ` DervishD
2007-04-21 22:42 ` OGAWA Hirofumi
0 siblings, 1 reply; 25+ messages in thread
From: DervishD @ 2007-04-19 14:19 UTC (permalink / raw)
To: Juergen Beisert; +Cc: linux-kernel
Hi Juergen :)
* Juergen Beisert <juergen127@kreuzholzen.de> dixit:
> On Thursday 19 April 2007 10:57, DervishD wrote:
> > I have a portable device with a FAT32 formatted hard disk in it, and
> > everytime I delete a file in the device *using the device itself to
> > do it* the device increases its count of free space and if I plug
> > the device in a Windows system, Windows agrees on the free space.
> > Linux doesn't. Linux believes that the files are still there
> > ocuppying space, and I have to run fsck.vfat to fix the problem.
>
> As I remember: It needs a large amount of time to calculate the free
> space on a big FAT32 system.
Big fat truth, I'm afraid. The thing is that I thought that Linux
did that from time to time to update the count. Obviously, doing it for
every statfs call would be very expensive :((
> So the last free sector count is also stored. When mounting this
> filesystem you don't need to walk through the whole FAT to calculate
> the available space, you can use this "cached" value instead. And this
> cached value seems not to be updated in your portable device.
It doesn't, certainly, but Windows doesn't care. Moreover, the
device doesn't seem to recalculate the value on every run (unless it
does it lightning fast!), so maybe the number is stored elsewhere (the
count can be stored in many places as far as I've read, but I don't know
the details).
A mount option to force walking the FAT and getting the real info
could be interesting. That way, it will be only done for certain devices
(small disks, for example).
Thanks for your answer :)
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-19 10:27 ` Boaz Harrosh
@ 2007-04-19 14:23 ` DervishD
0 siblings, 0 replies; 25+ messages in thread
From: DervishD @ 2007-04-19 14:23 UTC (permalink / raw)
To: Boaz Harrosh; +Cc: Linux-kernel
Hi Boaz :)
* Boaz Harrosh <bharrosh@panasas.com> dixit:
> > Apart from not using the device itself to delete files (and probably
> > not using Windows for that, either) and to run fsck.vfat now and
> > then, is anything I can do to avoid this problem?
> >
> Not that I know how to fix it. But have you tried running chkdsk on a
> device deleted files in windows. Just that I know, that windows would
> silently ignore fat errors and will only report them in chkdsk.
I've done that and the problem is not fixed. I have to use fsck.vfat
instead.
> Deleting files in windows works fine. There is no problems with it and
> windows updates everything. I do that all the time.
True, I've tested it, too. I'm going to report the problem to the
device manufacturer (it's a Cowon iAUDIO), but probably doing it in the
device will be very slow :(
> (I would even theorize that if you delete files on the device and than
> farther delete more files on windows, than windows will fix the
> problem on the fly)
Probably, and it may be a good fix, but I seldom plug the device on
Windows.
Thanks for the answer :)
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-19 14:19 ` DervishD
@ 2007-04-21 22:42 ` OGAWA Hirofumi
2007-04-22 3:18 ` Andrew Morton
` (2 more replies)
0 siblings, 3 replies; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-21 22:42 UTC (permalink / raw)
To: Juergen Beisert; +Cc: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]
DervishD <lkml@dervishd.net> writes:
> * Juergen Beisert <juergen127@kreuzholzen.de> dixit:
>> On Thursday 19 April 2007 10:57, DervishD wrote:
>> > I have a portable device with a FAT32 formatted hard disk in it, and
>> > everytime I delete a file in the device *using the device itself to
>> > do it* the device increases its count of free space and if I plug
>> > the device in a Windows system, Windows agrees on the free space.
>> > Linux doesn't. Linux believes that the files are still there
>> > ocuppying space, and I have to run fsck.vfat to fix the problem.
>>
>> As I remember: It needs a large amount of time to calculate the free
>> space on a big FAT32 system.
>
> Big fat truth, I'm afraid. The thing is that I thought that Linux
> did that from time to time to update the count. Obviously, doing it for
> every statfs call would be very expensive :((
>
>> So the last free sector count is also stored. When mounting this
>> filesystem you don't need to walk through the whole FAT to calculate
>> the available space, you can use this "cached" value instead. And this
>> cached value seems not to be updated in your portable device.
>
> It doesn't, certainly, but Windows doesn't care. Moreover, the
> device doesn't seem to recalculate the value on every run (unless it
> does it lightning fast!), so maybe the number is stored elsewhere (the
> count can be stored in many places as far as I've read, but I don't know
> the details).
>
> A mount option to force walking the FAT and getting the real info
> could be interesting. That way, it will be only done for certain devices
> (small disks, for example).
Yes. It seems that Windows does not update the ->free_clusters correctly.
Probably, I think the option is good for now. What do you think about
an attached patch?
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fat_ignore_free_clusters.patch --]
[-- Type: text/x-diff, Size: 3186 bytes --]
Recent Windows doesn't update ->free_clusters correctly. The "nofree"
option ignores the ->free_clusters stored on FSINFO.
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
---
fs/fat/inode.c | 13 ++++++++++---
include/linux/msdos_fs.h | 3 ++-
2 files changed, 12 insertions(+), 4 deletions(-)
diff -puN fs/fat/inode.c~fat_ignore_free_clusters fs/fat/inode.c
--- linux-2.6/fs/fat/inode.c~fat_ignore_free_clusters 2007-04-22 07:30:13.000000000 +0900
+++ linux-2.6-hirofumi/fs/fat/inode.c 2007-04-22 07:30:13.000000000 +0900
@@ -825,6 +825,8 @@ static int fat_show_options(struct seq_f
}
if (opts->name_check != 'n')
seq_printf(m, ",check=%c", opts->name_check);
+ if (opts->nofree)
+ seq_puts(m, ",nofree");
if (opts->quiet)
seq_puts(m, ",quiet");
if (opts->showexec)
@@ -850,7 +852,7 @@ static int fat_show_options(struct seq_f
enum {
Opt_check_n, Opt_check_r, Opt_check_s, Opt_uid, Opt_gid,
- Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_nocase,
+ Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_nofree, Opt_nocase,
Opt_quiet, Opt_showexec, Opt_debug, Opt_immutable,
Opt_dots, Opt_nodots,
Opt_charset, Opt_shortname_lower, Opt_shortname_win95,
@@ -872,6 +874,7 @@ static match_table_t fat_tokens = {
{Opt_dmask, "dmask=%o"},
{Opt_fmask, "fmask=%o"},
{Opt_codepage, "codepage=%u"},
+ {Opt_nofree, "nofree"},
{Opt_nocase, "nocase"},
{Opt_quiet, "quiet"},
{Opt_showexec, "showexec"},
@@ -951,7 +954,7 @@ static int parse_options(char *options,
opts->quiet = opts->showexec = opts->sys_immutable = opts->dotsOK = 0;
opts->utf8 = opts->unicode_xlate = 0;
opts->numtail = 1;
- opts->nocase = 0;
+ opts->nofree = opts->nocase = 0;
*debug = 0;
if (!options)
@@ -979,6 +982,9 @@ static int parse_options(char *options,
case Opt_check_n:
opts->name_check = 'n';
break;
+ case Opt_nofree:
+ opts->nofree = 1;
+ break;
case Opt_nocase:
if (!is_vfat)
opts->nocase = 1;
@@ -1352,7 +1358,8 @@ int fat_fill_super(struct super_block *s
sbi->max_cluster = total_clusters + FAT_START_ENT;
/* check the free_clusters, it's not necessarily correct */
- if (sbi->free_clusters != -1 && sbi->free_clusters > total_clusters)
+ if ((sbi->free_clusters != -1 && sbi->free_clusters > total_clusters) ||
+ sbi->options.nofree)
sbi->free_clusters = -1;
/* check the prev_free, it's not necessarily correct */
sbi->prev_free %= sbi->max_cluster;
diff -puN include/linux/msdos_fs.h~fat_ignore_free_clusters include/linux/msdos_fs.h
--- linux-2.6/include/linux/msdos_fs.h~fat_ignore_free_clusters 2007-04-22 07:30:13.000000000 +0900
+++ linux-2.6-hirofumi/include/linux/msdos_fs.h 2007-04-22 07:30:13.000000000 +0900
@@ -205,7 +205,8 @@ struct fat_mount_options {
numtail:1, /* Does first alias have a numeric '~1' type tail? */
atari:1, /* Use Atari GEMDOS variation of MS-DOS fs */
flush:1, /* write things quickly */
- nocase:1; /* Does this need case conversion? 0=need case conversion*/
+ nocase:1, /* Does this need case conversion? 0=need case conversion*/
+ nofree:1; /* Does use free_clusters */
};
#define FAT_HASH_BITS 8
_
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-21 22:42 ` OGAWA Hirofumi
@ 2007-04-22 3:18 ` Andrew Morton
2007-04-22 4:26 ` OGAWA Hirofumi
2007-04-22 11:17 ` DervishD
2007-04-22 11:28 ` Juergen Beisert
2 siblings, 1 reply; 25+ messages in thread
From: Andrew Morton @ 2007-04-22 3:18 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: Juergen Beisert, linux-kernel
On Sun, 22 Apr 2007 07:42:22 +0900 OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> wrote:
> Recent Windows doesn't update ->free_clusters correctly. The "nofree"
> option ignores the ->free_clusters stored on FSINFO.
It'd be better to avoid the new mount option if possible: fatfs is used by
a *lot* of people who aren't particualrly expert, and we want to make things
easy for them.
Is there some way in which we can work out what's happened and fix it up?
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 3:18 ` Andrew Morton
@ 2007-04-22 4:26 ` OGAWA Hirofumi
2007-04-22 11:26 ` DervishD
0 siblings, 1 reply; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-22 4:26 UTC (permalink / raw)
To: Andrew Morton; +Cc: Juergen Beisert, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 947 bytes --]
Andrew Morton <akpm@linux-foundation.org> writes:
> On Sun, 22 Apr 2007 07:42:22 +0900 OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> wrote:
>
>> Recent Windows doesn't update ->free_clusters correctly. The "nofree"
>> option ignores the ->free_clusters stored on FSINFO.
>
> It'd be better to avoid the new mount option if possible: fatfs is used by
> a *lot* of people who aren't particualrly expert, and we want to make things
> easy for them.
Ok.
> Is there some way in which we can work out what's happened and fix it up?
It seems that the recent Windows changed specification, and it's
undocumented. Windows doesn't update ->free_clusters correctly.
Probably, what we can do is to throw away speed of statfs(2) and not
using ->free_clusters. (And if possible, recover it by optimization.)
This patch doesn't use ->free_clusters by default. (instead, add
"usefree" for forcing to use it)
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fat_dont-use_free_clusters-for-fat32.patch --]
[-- Type: text/x-diff, Size: 3250 bytes --]
It seems that the recent Windows changed specification, and it's
undocumented. Windows doesn't update ->free_clusters correctly.
This patch doesn't use ->free_clusters by default. (instead, add
"usefree" for forcing to use it)
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
---
fs/fat/inode.c | 14 +++++++++++---
include/linux/msdos_fs.h | 3 ++-
2 files changed, 13 insertions(+), 4 deletions(-)
diff -puN fs/fat/inode.c~fat_dont-use_free_clusters-for-fat32 fs/fat/inode.c
--- linux-2.6/fs/fat/inode.c~fat_dont-use_free_clusters-for-fat32 2007-04-22 13:07:05.000000000 +0900
+++ linux-2.6-hirofumi/fs/fat/inode.c 2007-04-22 13:18:07.000000000 +0900
@@ -825,6 +825,8 @@ static int fat_show_options(struct seq_f
}
if (opts->name_check != 'n')
seq_printf(m, ",check=%c", opts->name_check);
+ if (opts->usefree)
+ seq_puts(m, ",usefree");
if (opts->quiet)
seq_puts(m, ",quiet");
if (opts->showexec)
@@ -850,7 +852,7 @@ static int fat_show_options(struct seq_f
enum {
Opt_check_n, Opt_check_r, Opt_check_s, Opt_uid, Opt_gid,
- Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_nocase,
+ Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_usefree, Opt_nocase,
Opt_quiet, Opt_showexec, Opt_debug, Opt_immutable,
Opt_dots, Opt_nodots,
Opt_charset, Opt_shortname_lower, Opt_shortname_win95,
@@ -872,6 +874,7 @@ static match_table_t fat_tokens = {
{Opt_dmask, "dmask=%o"},
{Opt_fmask, "fmask=%o"},
{Opt_codepage, "codepage=%u"},
+ {Opt_usefree, "usefree"},
{Opt_nocase, "nocase"},
{Opt_quiet, "quiet"},
{Opt_showexec, "showexec"},
@@ -951,7 +954,7 @@ static int parse_options(char *options,
opts->quiet = opts->showexec = opts->sys_immutable = opts->dotsOK = 0;
opts->utf8 = opts->unicode_xlate = 0;
opts->numtail = 1;
- opts->nocase = 0;
+ opts->usefree = opts->nocase = 0;
*debug = 0;
if (!options)
@@ -979,6 +982,9 @@ static int parse_options(char *options,
case Opt_check_n:
opts->name_check = 'n';
break;
+ case Opt_usefree:
+ opts->usefree = 1;
+ break;
case Opt_nocase:
if (!is_vfat)
opts->nocase = 1;
@@ -1306,7 +1312,9 @@ int fat_fill_super(struct super_block *s
le32_to_cpu(fsinfo->signature2),
sbi->fsinfo_sector);
} else {
- sbi->free_clusters = le32_to_cpu(fsinfo->free_clusters);
+ if (sbi->options.usefree)
+ sbi->free_clusters =
+ le32_to_cpu(fsinfo->free_clusters);
sbi->prev_free = le32_to_cpu(fsinfo->next_cluster);
}
diff -puN include/linux/msdos_fs.h~fat_dont-use_free_clusters-for-fat32 include/linux/msdos_fs.h
--- linux-2.6/include/linux/msdos_fs.h~fat_dont-use_free_clusters-for-fat32 2007-04-22 13:07:05.000000000 +0900
+++ linux-2.6-hirofumi/include/linux/msdos_fs.h 2007-04-22 13:09:14.000000000 +0900
@@ -205,7 +205,8 @@ struct fat_mount_options {
numtail:1, /* Does first alias have a numeric '~1' type tail? */
atari:1, /* Use Atari GEMDOS variation of MS-DOS fs */
flush:1, /* write things quickly */
- nocase:1; /* Does this need case conversion? 0=need case conversion*/
+ nocase:1, /* Does this need case conversion? 0=need case conversion*/
+ usefree:1; /* Use free_clusters for FAT32 */
};
#define FAT_HASH_BITS 8
_
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-21 22:42 ` OGAWA Hirofumi
2007-04-22 3:18 ` Andrew Morton
@ 2007-04-22 11:17 ` DervishD
2007-04-22 11:28 ` Juergen Beisert
2 siblings, 0 replies; 25+ messages in thread
From: DervishD @ 2007-04-22 11:17 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: Juergen Beisert, linux-kernel
Hi Ogawa :)
* OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> dixit:
> DervishD <lkml@dervishd.net> writes:
> > A mount option to force walking the FAT and getting the real
> > info could be interesting. That way, it will be only done for
> > certain devices (small disks, for example).
>
> Yes. It seems that Windows does not update the ->free_clusters
> correctly. Probably, I think the option is good for now. What do you
> think about an attached patch?
The option ignores ->free_clusters by default, am I wrong? I'm not
sure about this: given that nobody cared before, chances are that this
problem didn't bite anyone, so I think that the default behaviour should
be the _old_ behaviour (that is, nofree=0, as if the option never
existed).
Moreover, I'm not sure if Windows updates correctly or not the
free_clustes FSINFO entry, this problem has happened (by now) in a
portable device only. What I can ensure is that both the portable device
and any Windows box I've tested ignores this value and properly
calculate the real free space. Linux doesn't because it relies on an
invalid value. Again, the fault is in the portable device, but the
workaround seems common practice out there...
So I will suggest just "nofree:0" instead of "nofree:1". And thanks
a lot for the patch :))) I'm not sure about this: given that nobody
cared before, chances are that this problem didn't bite anyone, so I
think that the default behaviour should be the _old_ behaviour (that is,
nofree=0, as if the option never existed).
Moreover, I'm not sure if Windows updates correctly or not the
free_clustes FSINFO entry, this problem has happened (by now) in a
portable device only. What I can ensure is that both the portable device
and any Windows box I've tested ignores this value and properly
calculate the real free space. Linux doesn't because it relies on an
invalid value. Again, the fault is in the portable device, but the
workaround seems common practice out there...
So I will suggest just "nofree:0" instead of "nofree:1". And thanks
a lot for the patch :)))
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 4:26 ` OGAWA Hirofumi
@ 2007-04-22 11:26 ` DervishD
2007-04-22 11:55 ` OGAWA Hirofumi
0 siblings, 1 reply; 25+ messages in thread
From: DervishD @ 2007-04-22 11:26 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: Andrew Morton, Juergen Beisert, linux-kernel
Hi Ogawa (and Andrew) :)
* OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> dixit:
> Andrew Morton <akpm@linux-foundation.org> writes:
> > Is there some way in which we can work out what's happened and fix
> > it up?
>
> It seems that the recent Windows changed specification, and it's
> undocumented. Windows doesn't update ->free_clusters correctly.
>
> Probably, what we can do is to throw away speed of statfs(2) and not
> using ->free_clusters. (And if possible, recover it by optimization.)
Even if Windows updates ->free_clusters correctly, it doesn't even
consider it to compute the number of free clusters, and IMHO Linux
should do the same. If speed is a concern, ->free_clusters can be
computed at mount time. That way statfs won't slow.
The problem is that if a program writes a file onto the filesystem
without using statfs first to check for free space, the free_clusters
entry won't have the real value and the driver may report "disk full" (I
haven't read the code for the vfat driver, sorry, so I'm not sure about
this) when really there are plenty of clusters to write the new file.
Probably it's stupid to update the free clusters count at mount time
(sorry if so...) but it looks like a good idea to me. And of course, I
don't mean to update the value _on disk_, but the kernel's idea of free
clusters (so even FAT filesystems mounted R/O will report correct
values).
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-21 22:42 ` OGAWA Hirofumi
2007-04-22 3:18 ` Andrew Morton
2007-04-22 11:17 ` DervishD
@ 2007-04-22 11:28 ` Juergen Beisert
2 siblings, 0 replies; 25+ messages in thread
From: Juergen Beisert @ 2007-04-22 11:28 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: linux-kernel
Hi,
On Sunday 22 April 2007 00:42, OGAWA Hirofumi wrote:
> Yes. It seems that Windows does not update the ->free_clusters correctly.
> Probably, I think the option is good for now. What do you think about
> an attached patch?
Please don't ask me, because I'm not a filesystem expert. Sorry. My suggestion
was only a tiny idea.
Juergen
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 11:26 ` DervishD
@ 2007-04-22 11:55 ` OGAWA Hirofumi
2007-04-22 20:08 ` DervishD
0 siblings, 1 reply; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-22 11:55 UTC (permalink / raw)
To: DervishD; +Cc: Andrew Morton, Juergen Beisert, linux-kernel
DervishD <lkml@dervishd.net> writes:
> The problem is that if a program writes a file onto the filesystem
> without using statfs first to check for free space, the free_clusters
> entry won't have the real value and the driver may report "disk full" (I
> haven't read the code for the vfat driver, sorry, so I'm not sure about
> this) when really there are plenty of clusters to write the new file.
No need to worry about it. If we ignored the ->free_clusters in
FSINFO, the fat drivers counts the current free clusters by scaning
FAT entries if needed.
> Probably it's stupid to update the free clusters count at mount time
> (sorry if so...) but it looks like a good idea to me. And of course, I
> don't mean to update the value _on disk_, but the kernel's idea of free
> clusters (so even FAT filesystems mounted R/O will report correct
> values).
It would add the limitation to following simple usage,
# mount -t vfat /dev/sda1 /mnt
# cp -a * /mnt
# umount
if /dev/sda1 was the large and slow device, "mount" will need several
minutes to counts free clusters. I think the user will be hard to
accept the several minutes at "mount".
Thanks.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
[not found] ` <8cwz8-2fE-13@gated-at.bofh.it>
@ 2007-04-22 13:28 ` Bodo Eggert
2007-04-22 14:04 ` OGAWA Hirofumi
2007-04-22 20:11 ` DervishD
[not found] ` <8cAMl-du-3@gated-at.bofh.it>
1 sibling, 2 replies; 25+ messages in thread
From: Bodo Eggert @ 2007-04-22 13:28 UTC (permalink / raw)
To: OGAWA Hirofumi, Juergen Beisert, linux-kernel, Andrew Morton
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> wrote:
>> * Juergen Beisert <juergen127@kreuzholzen.de> dixit:
>>> So the last free sector count is also stored. When mounting this
>>> filesystem you don't need to walk through the whole FAT to calculate
>>> the available space, you can use this "cached" value instead. And this
>>> cached value seems not to be updated in your portable device.
>>
>> It doesn't, certainly, but Windows doesn't care. Moreover, the
>> device doesn't seem to recalculate the value on every run (unless it
>> does it lightning fast!), so maybe the number is stored elsewhere (the
>> count can be stored in many places as far as I've read, but I don't know
>> the details).
AFAIR it's stored twice on FAT32, once in a backup sector and once in the
superblock or extended superblock (don't remember, I think it was the
extended ~). It's not stored on FAT{12,16}.
>> A mount option to force walking the FAT and getting the real info
>> could be interesting. That way, it will be only done for certain devices
>> (small disks, for example).
>
> Yes. It seems that Windows does not update the ->free_clusters correctly.
> Probably, I think the option is good for now. What do you think about
> an attached patch?
Windows _does_ care*, it will pretend the disk to be full. Therefore the
stored value *SHOULD* be updated. (I think your patch does this.)
Recalculating the free space is a nice idea, and modern hardware might be
fast enough to recalculate the value on mount by default. (I didn't try this
for years.) Maybe the default should depend on arch?
About this patch: (news:8cwz8-2fE-13@gated-at.bofh.it)
- usefree is a bad name (I'd suggest recalc_free instead), and your
description is too cryptic to be understood by a non-linux FAT expert.
- You forgot to update Documentation/
--
Never forget: 2 + 2 = 5 for extremely large values of 2.
Friß, Spammer: NyjnoQ@0S0gJr.7eggert.dyndns.org uh@e.7eggert.dyndns.org
VnmVge7Vj@3tlxued.7eggert.dyndns.org RggsNsl@al.7eggert.dyndns.org
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 13:28 ` Bodo Eggert
@ 2007-04-22 14:04 ` OGAWA Hirofumi
2007-04-22 14:29 ` OGAWA Hirofumi
2007-04-22 21:46 ` Bodo Eggert
2007-04-22 20:11 ` DervishD
1 sibling, 2 replies; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-22 14:04 UTC (permalink / raw)
To: 7eggert; +Cc: Juergen Beisert, linux-kernel, Andrew Morton
Bodo Eggert <7eggert@gmx.de> writes:
> Windows _does_ care*, it will pretend the disk to be full.
Did you test on 2000 or XP? (e.g. write 0 to free_clusters, then
create new file.)
> - usefree is a bad name (I'd suggest recalc_free instead),
Is it about nofree option?
> and your description is too cryptic to be understood by a non-linux
> FAT expert.
Um... why do we need to care about non-linux people in the patch?
Anyway, if you correct it, I'll update it gladly.
> - You forgot to update Documentation/
Sure. If you can post the patch, it'll be great.
Thanks.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 14:04 ` OGAWA Hirofumi
@ 2007-04-22 14:29 ` OGAWA Hirofumi
2007-04-22 14:53 ` Andreas Schwab
2007-04-22 21:46 ` Bodo Eggert
1 sibling, 1 reply; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-22 14:29 UTC (permalink / raw)
To: 7eggert; +Cc: Juergen Beisert, linux-kernel, Andrew Morton
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> writes:
>> - You forgot to update Documentation/
>
> Sure. If you can post the patch, it'll be great.
Updated patch is here.
Thanks.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
It seems that the recent Windows changed specification, and it's
undocumented. Windows doesn't update ->free_clusters correctly.
This patch doesn't use ->free_clusters by default. (instead, add
"usefree" for forcing to use it)
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
---
Documentation/filesystems/vfat.txt | 7 +++++++
fs/fat/inode.c | 14 +++++++++++---
include/linux/msdos_fs.h | 3 ++-
3 files changed, 20 insertions(+), 4 deletions(-)
diff -puN fs/fat/inode.c~fat_dont-use_free_clusters-for-fat32 fs/fat/inode.c
--- linux-2.6/fs/fat/inode.c~fat_dont-use_free_clusters-for-fat32 2007-04-22 13:07:05.000000000 +0900
+++ linux-2.6-hirofumi/fs/fat/inode.c 2007-04-22 13:18:07.000000000 +0900
@@ -825,6 +825,8 @@ static int fat_show_options(struct seq_f
}
if (opts->name_check != 'n')
seq_printf(m, ",check=%c", opts->name_check);
+ if (opts->usefree)
+ seq_puts(m, ",usefree");
if (opts->quiet)
seq_puts(m, ",quiet");
if (opts->showexec)
@@ -850,7 +852,7 @@ static int fat_show_options(struct seq_f
enum {
Opt_check_n, Opt_check_r, Opt_check_s, Opt_uid, Opt_gid,
- Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_nocase,
+ Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_usefree, Opt_nocase,
Opt_quiet, Opt_showexec, Opt_debug, Opt_immutable,
Opt_dots, Opt_nodots,
Opt_charset, Opt_shortname_lower, Opt_shortname_win95,
@@ -872,6 +874,7 @@ static match_table_t fat_tokens = {
{Opt_dmask, "dmask=%o"},
{Opt_fmask, "fmask=%o"},
{Opt_codepage, "codepage=%u"},
+ {Opt_usefree, "usefree"},
{Opt_nocase, "nocase"},
{Opt_quiet, "quiet"},
{Opt_showexec, "showexec"},
@@ -951,7 +954,7 @@ static int parse_options(char *options,
opts->quiet = opts->showexec = opts->sys_immutable = opts->dotsOK = 0;
opts->utf8 = opts->unicode_xlate = 0;
opts->numtail = 1;
- opts->nocase = 0;
+ opts->usefree = opts->nocase = 0;
*debug = 0;
if (!options)
@@ -979,6 +982,9 @@ static int parse_options(char *options,
case Opt_check_n:
opts->name_check = 'n';
break;
+ case Opt_usefree:
+ opts->usefree = 1;
+ break;
case Opt_nocase:
if (!is_vfat)
opts->nocase = 1;
@@ -1306,7 +1312,9 @@ int fat_fill_super(struct super_block *s
le32_to_cpu(fsinfo->signature2),
sbi->fsinfo_sector);
} else {
- sbi->free_clusters = le32_to_cpu(fsinfo->free_clusters);
+ if (sbi->options.usefree)
+ sbi->free_clusters =
+ le32_to_cpu(fsinfo->free_clusters);
sbi->prev_free = le32_to_cpu(fsinfo->next_cluster);
}
diff -puN include/linux/msdos_fs.h~fat_dont-use_free_clusters-for-fat32 include/linux/msdos_fs.h
--- linux-2.6/include/linux/msdos_fs.h~fat_dont-use_free_clusters-for-fat32 2007-04-22 13:07:05.000000000 +0900
+++ linux-2.6-hirofumi/include/linux/msdos_fs.h 2007-04-22 13:09:14.000000000 +0900
@@ -205,7 +205,8 @@ struct fat_mount_options {
numtail:1, /* Does first alias have a numeric '~1' type tail? */
atari:1, /* Use Atari GEMDOS variation of MS-DOS fs */
flush:1, /* write things quickly */
- nocase:1; /* Does this need case conversion? 0=need case conversion*/
+ nocase:1, /* Does this need case conversion? 0=need case conversion*/
+ usefree:1; /* Use free_clusters for FAT32 */
};
#define FAT_HASH_BITS 8
diff -puN Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 Documentation/filesystems/vfat.txt
--- linux-2.6/Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 2007-04-22 23:06:21.000000000 +0900
+++ linux-2.6-hirofumi/Documentation/filesystems/vfat.txt 2007-04-22 23:28:19.000000000 +0900
@@ -34,6 +34,13 @@ iocharset=name -- Character set to use f
NOTE: "iocharset=utf8" is not recommended. If unsure,
you should consider the following option instead.
+usefree -- Use the "free clusters" value stored on FSINFO. It'll
+ be used to determine number of free clusters without
+ scanning disk. But it's not used by default, because
+ recent Windows don't update it correctly in some
+ case. If you are sure the "free clusters" on FSINFO is
+ correct, by this option you can avoid scanning disk.
+
utf8=<bool> -- UTF-8 is the filesystem safe version of Unicode that
is used by the console. It can be enabled for the
filesystem with this option. If 'uni_xlate' gets set,
_
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 14:29 ` OGAWA Hirofumi
@ 2007-04-22 14:53 ` Andreas Schwab
2007-04-22 15:13 ` OGAWA Hirofumi
0 siblings, 1 reply; 25+ messages in thread
From: Andreas Schwab @ 2007-04-22 14:53 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: 7eggert, Juergen Beisert, linux-kernel, Andrew Morton
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> writes:
> diff -puN Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 Documentation/filesystems/vfat.txt
> --- linux-2.6/Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 2007-04-22 23:06:21.000000000 +0900
> +++ linux-2.6-hirofumi/Documentation/filesystems/vfat.txt 2007-04-22 23:28:19.000000000 +0900
> @@ -34,6 +34,13 @@ iocharset=name -- Character set to use f
> NOTE: "iocharset=utf8" is not recommended. If unsure,
> you should consider the following option instead.
"the following option" is no longer correct.
>
> +usefree -- Use the "free clusters" value stored on FSINFO. It'll
> + be used to determine number of free clusters without
> + scanning disk. But it's not used by default, because
> + recent Windows don't update it correctly in some
> + case. If you are sure the "free clusters" on FSINFO is
> + correct, by this option you can avoid scanning disk.
> +
> utf8=<bool> -- UTF-8 is the filesystem safe version of Unicode that
> is used by the console. It can be enabled for the
> filesystem with this option. If 'uni_xlate' gets set,
Andreas.
--
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 14:53 ` Andreas Schwab
@ 2007-04-22 15:13 ` OGAWA Hirofumi
0 siblings, 0 replies; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-22 15:13 UTC (permalink / raw)
To: Andreas Schwab; +Cc: 7eggert, Juergen Beisert, linux-kernel, Andrew Morton
Andreas Schwab <schwab@suse.de> writes:
> OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> writes:
>
>> diff -puN Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 Documentation/filesystems/vfat.txt
>> --- linux-2.6/Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 2007-04-22 23:06:21.000000000 +0900
>> +++ linux-2.6-hirofumi/Documentation/filesystems/vfat.txt 2007-04-22 23:28:19.000000000 +0900
>> @@ -34,6 +34,13 @@ iocharset=name -- Character set to use f
>> NOTE: "iocharset=utf8" is not recommended. If unsure,
>> you should consider the following option instead.
>
> "the following option" is no longer correct.
Whoops. Thanks for correcting it.
The patch was fixed.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
It seems that the recent Windows changed specification, and it's
undocumented. Windows doesn't update ->free_clusters correctly.
This patch doesn't use ->free_clusters by default. (instead, add
"usefree" for forcing to use it)
Signed-off-by: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
---
Documentation/filesystems/vfat.txt | 7 +++++++
fs/fat/inode.c | 14 +++++++++++---
include/linux/msdos_fs.h | 3 ++-
3 files changed, 20 insertions(+), 4 deletions(-)
diff -puN fs/fat/inode.c~fat_dont-use_free_clusters-for-fat32 fs/fat/inode.c
--- linux-2.6/fs/fat/inode.c~fat_dont-use_free_clusters-for-fat32 2007-04-22 13:07:05.000000000 +0900
+++ linux-2.6-hirofumi/fs/fat/inode.c 2007-04-22 13:18:07.000000000 +0900
@@ -825,6 +825,8 @@ static int fat_show_options(struct seq_f
}
if (opts->name_check != 'n')
seq_printf(m, ",check=%c", opts->name_check);
+ if (opts->usefree)
+ seq_puts(m, ",usefree");
if (opts->quiet)
seq_puts(m, ",quiet");
if (opts->showexec)
@@ -850,7 +852,7 @@ static int fat_show_options(struct seq_f
enum {
Opt_check_n, Opt_check_r, Opt_check_s, Opt_uid, Opt_gid,
- Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_nocase,
+ Opt_umask, Opt_dmask, Opt_fmask, Opt_codepage, Opt_usefree, Opt_nocase,
Opt_quiet, Opt_showexec, Opt_debug, Opt_immutable,
Opt_dots, Opt_nodots,
Opt_charset, Opt_shortname_lower, Opt_shortname_win95,
@@ -872,6 +874,7 @@ static match_table_t fat_tokens = {
{Opt_dmask, "dmask=%o"},
{Opt_fmask, "fmask=%o"},
{Opt_codepage, "codepage=%u"},
+ {Opt_usefree, "usefree"},
{Opt_nocase, "nocase"},
{Opt_quiet, "quiet"},
{Opt_showexec, "showexec"},
@@ -951,7 +954,7 @@ static int parse_options(char *options,
opts->quiet = opts->showexec = opts->sys_immutable = opts->dotsOK = 0;
opts->utf8 = opts->unicode_xlate = 0;
opts->numtail = 1;
- opts->nocase = 0;
+ opts->usefree = opts->nocase = 0;
*debug = 0;
if (!options)
@@ -979,6 +982,9 @@ static int parse_options(char *options,
case Opt_check_n:
opts->name_check = 'n';
break;
+ case Opt_usefree:
+ opts->usefree = 1;
+ break;
case Opt_nocase:
if (!is_vfat)
opts->nocase = 1;
@@ -1306,7 +1312,9 @@ int fat_fill_super(struct super_block *s
le32_to_cpu(fsinfo->signature2),
sbi->fsinfo_sector);
} else {
- sbi->free_clusters = le32_to_cpu(fsinfo->free_clusters);
+ if (sbi->options.usefree)
+ sbi->free_clusters =
+ le32_to_cpu(fsinfo->free_clusters);
sbi->prev_free = le32_to_cpu(fsinfo->next_cluster);
}
diff -puN include/linux/msdos_fs.h~fat_dont-use_free_clusters-for-fat32 include/linux/msdos_fs.h
--- linux-2.6/include/linux/msdos_fs.h~fat_dont-use_free_clusters-for-fat32 2007-04-22 13:07:05.000000000 +0900
+++ linux-2.6-hirofumi/include/linux/msdos_fs.h 2007-04-22 13:09:14.000000000 +0900
@@ -205,7 +205,8 @@ struct fat_mount_options {
numtail:1, /* Does first alias have a numeric '~1' type tail? */
atari:1, /* Use Atari GEMDOS variation of MS-DOS fs */
flush:1, /* write things quickly */
- nocase:1; /* Does this need case conversion? 0=need case conversion*/
+ nocase:1, /* Does this need case conversion? 0=need case conversion*/
+ usefree:1; /* Use free_clusters for FAT32 */
};
#define FAT_HASH_BITS 8
diff -puN Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 Documentation/filesystems/vfat.txt
--- linux-2.6/Documentation/filesystems/vfat.txt~fat_dont-use_free_clusters-for-fat32 2007-04-22 23:06:21.000000000 +0900
+++ linux-2.6-hirofumi/Documentation/filesystems/vfat.txt 2007-04-23 00:09:35.000000000 +0900
@@ -57,6 +57,13 @@ nonumtail=<bool> -- When creating 8.3 al
currently exist in the directory, 'longfile.txt' will
be the short alias instead of 'longfi~1.txt'.
+usefree -- Use the "free clusters" value stored on FSINFO. It'll
+ be used to determine number of free clusters without
+ scanning disk. But it's not used by default, because
+ recent Windows don't update it correctly in some
+ case. If you are sure the "free clusters" on FSINFO is
+ correct, by this option you can avoid scanning disk.
+
quiet -- Stops printing certain warning messages.
check=s|r|n -- Case sensitivity checking setting.
_
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
[not found] ` <8cITG-40H-5@gated-at.bofh.it>
@ 2007-04-22 17:21 ` Bodo Eggert
2007-04-22 17:44 ` OGAWA Hirofumi
0 siblings, 1 reply; 25+ messages in thread
From: Bodo Eggert @ 2007-04-22 17:21 UTC (permalink / raw)
To: OGAWA Hirofumi, DervishD, Andrew Morton, Juergen Beisert,
linux-kernel
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> wrote:
> DervishD <lkml@dervishd.net> writes:
>> Probably it's stupid to update the free clusters count at mount time
>> (sorry if so...) but it looks like a good idea to me. And of course, I
>> don't mean to update the value _on disk_, but the kernel's idea of free
>> clusters (so even FAT filesystems mounted R/O will report correct
>> values).
>
> It would add the limitation to following simple usage,
>
> # mount -t vfat /dev/sda1 /mnt
> # cp -a * /mnt
> # umount
>
> if /dev/sda1 was the large and slow device, "mount" will need several
> minutes to counts free clusters. I think the user will be hard to
> accept the several minutes at "mount".
I simulated recalculating the FAT using a suboptimal perl script on my
32 GB FAT32 partition on a 40 GB drive. The results should show the worst
case for my system.
The results were below 2 seconds (optimized perl at 0.5 s) on my AMD Athlon
XP 2400+. (BTW: I don't think the device speed itself will be relevant in
many cases even if you'd asume a slow link, since you'll need to read the
FAT anyway.)
I think this is acceptable, and on this system, I'd like to enable
recalculating by default. On slower systems, you'll need to chose another
default.
I think you'll want that option as a module parameter to make all users happy.
And while you're at it, fmask and dmask are good candidates for module
parameters, too.-)
# echo 3 > /proc/sys/vm/drop_caches
# /usr/bin/time sh -c "dd if=/dev/hda2 bs=$((77834925/64)) count=4 | perl -e
'"'use POSIX;while (sysread STDIN,$x,4) {$n++ if $x eq "\0\0\0\0"} print
$n*32,"\n"'\'
4+0 records in
4+0 records out
13954720
0.76user 0.60system 0:01.95elapsed 69%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (8major+1583minor)pagefaults 0swaps
# /usr/bin/time sh -c "dd if=/dev/hda2 bs=$((77834925/64)) count=4 | perl -e
'"'use POSIX;while (sysread STDIN,$x,4) {$n++ if $x eq "\0\0\0\0"} print
$n*32,"\n"'\'
4+0 records in
4+0 records out
13955392
0.74user 0.69system 0:01.50elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1591minor)pagefaults 0swaps
# /usr/bin/time sh -c "dd if=/dev/hda2 bs=$((77834925/64)) count=4 | perl -e
'"'use POSIX;while (sysread STDIN,$x,4096) {for (unpack "L*",$x){$n++ if !$_}}
print $n*32,"\n"'\'
4+0 records in
4+0 records out
13955392
0.43user 0.06system 0:00.50elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1599minor)pagefaults 0swaps
--
The worse the weather, the more you are required to be out in it.
Friß, Spammer: fv@upXs.7eggert.dyndns.org z@Z1rk.7eggert.dyndns.org
VRo@Gb2b4rxc.7eggert.dyndns.org LimmgYHv@smpZsGtB.7eggert.dyndns.org
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 17:21 ` Bodo Eggert
@ 2007-04-22 17:44 ` OGAWA Hirofumi
0 siblings, 0 replies; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-22 17:44 UTC (permalink / raw)
To: 7eggert; +Cc: DervishD, Andrew Morton, Juergen Beisert, linux-kernel
Bodo Eggert <7eggert@gmx.de> writes:
> # /usr/bin/time sh -c "dd if=/dev/hda2 bs=$((77834925/64)) count=4 | perl -e
> '"'use POSIX;while (sysread STDIN,$x,4) {$n++ if $x eq "\0\0\0\0"} print
> $n*32,"\n"'\'
> 4+0 records in
> 4+0 records out
> 13954720
> 0.76user 0.60system 0:01.95elapsed 69%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (8major+1583minor)pagefaults 0swaps
dd will use readahead, and I'm thinking about usb. Well, anyway,
recalc is default in new patch.
Thanks.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 11:55 ` OGAWA Hirofumi
@ 2007-04-22 20:08 ` DervishD
2007-04-23 2:27 ` OGAWA Hirofumi
0 siblings, 1 reply; 25+ messages in thread
From: DervishD @ 2007-04-22 20:08 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: Andrew Morton, Juergen Beisert, linux-kernel
Hi Ogawa :)
* OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> dixit:
> DervishD <lkml@dervishd.net> writes:
> > The problem is that if a program writes a file onto the filesystem
> > without using statfs first to check for free space, the free_clusters
> > entry won't have the real value and the driver may report "disk full" (I
> > haven't read the code for the vfat driver, sorry, so I'm not sure about
> > this) when really there are plenty of clusters to write the new file.
>
> No need to worry about it. If we ignored the ->free_clusters in
> FSINFO, the fat drivers counts the current free clusters by scaning
> FAT entries if needed.
Cool! :)
> > Probably it's stupid to update the free clusters count at mount time
> > (sorry if so...) but it looks like a good idea to me. And of course, I
> > don't mean to update the value _on disk_, but the kernel's idea of free
> > clusters (so even FAT filesystems mounted R/O will report correct
> > values).
>
> It would add the limitation to following simple usage,
>
> # mount -t vfat /dev/sda1 /mnt
> # cp -a * /mnt
> # umount
>
> if /dev/sda1 was the large and slow device, "mount" will need several
> minutes to counts free clusters. I think the user will be hard to
> accept the several minutes at "mount".
I can carry some tests, but if Windows does that tasks lightning
fast, Linux surely does it faster ;) I don't think, anyway, that having
a huge USB disk is a common practice when using "modest" machines.
If you want, I can perform a couple of tests. I have a 80GB disk
that I can connect using an USB adapter and my machine is AMD Athlon XP
1900+ with 1GB of RAM, which looks pretty slow nowadays O:)
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 13:28 ` Bodo Eggert
2007-04-22 14:04 ` OGAWA Hirofumi
@ 2007-04-22 20:11 ` DervishD
1 sibling, 0 replies; 25+ messages in thread
From: DervishD @ 2007-04-22 20:11 UTC (permalink / raw)
To: Bodo Eggert; +Cc: OGAWA Hirofumi, Juergen Beisert, linux-kernel, Andrew Morton
Hi Bodo :)
* Bodo Eggert <7eggert@gmx.de> dixit:
> OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> wrote:
> >> * Juergen Beisert <juergen127@kreuzholzen.de> dixit:
>
> >>> So the last free sector count is also stored. When mounting this
> >>> filesystem you don't need to walk through the whole FAT to calculate
> >>> the available space, you can use this "cached" value instead. And this
> >>> cached value seems not to be updated in your portable device.
> >>
> >> It doesn't, certainly, but Windows doesn't care. Moreover, the
> >> device doesn't seem to recalculate the value on every run (unless it
> >> does it lightning fast!), so maybe the number is stored elsewhere (the
> >> count can be stored in many places as far as I've read, but I don't know
> >> the details).
>
> AFAIR it's stored twice on FAT32, once in a backup sector and once in the
> superblock or extended superblock (don't remember, I think it was the
> extended ~). It's not stored on FAT{12,16}.
I hadn't noticed the same problem with FAT16 pendrives, so I
suspected that. Thanks for confirming :)
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 14:04 ` OGAWA Hirofumi
2007-04-22 14:29 ` OGAWA Hirofumi
@ 2007-04-22 21:46 ` Bodo Eggert
2007-04-23 2:20 ` OGAWA Hirofumi
1 sibling, 1 reply; 25+ messages in thread
From: Bodo Eggert @ 2007-04-22 21:46 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: 7eggert, Juergen Beisert, linux-kernel, Andrew Morton
On Sun, 22 Apr 2007, OGAWA Hirofumi wrote:
> Bodo Eggert <7eggert@gmx.de> writes:
> > Windows _does_ care*, it will pretend the disk to be full.
>
> Did you test on 2000 or XP? (e.g. write 0 to free_clusters, then
> create new file.)
That was back when I still used W98.
> > - usefree is a bad name (I'd suggest recalc_free instead),
>
> Is it about nofree option?
Yes. I think recalc_free is way more descriptive.
> > and your description is too cryptic to be understood by a non-linux
> > FAT expert.
>
> Um... why do we need to care about non-linux people in the patch?
Non-experts in Linux FAT implementation details. "recalc sounds like might
take long, let's try to disable it" should be all it takes to get a fast
mount.
--
Top 100 things you don't want the sysadmin to say:
35. Ummm... Didn't you say you turned it off?
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 21:46 ` Bodo Eggert
@ 2007-04-23 2:20 ` OGAWA Hirofumi
0 siblings, 0 replies; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-23 2:20 UTC (permalink / raw)
To: Bodo Eggert; +Cc: Juergen Beisert, linux-kernel, Andrew Morton
Bodo Eggert <7eggert@gmx.de> writes:
>> > - usefree is a bad name (I'd suggest recalc_free instead),
>>
>> Is it about nofree option?
>
> Yes. I think recalc_free is way more descriptive.
Recalc is already default on current patch.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-22 20:08 ` DervishD
@ 2007-04-23 2:27 ` OGAWA Hirofumi
2007-04-23 6:19 ` DervishD
0 siblings, 1 reply; 25+ messages in thread
From: OGAWA Hirofumi @ 2007-04-23 2:27 UTC (permalink / raw)
To: DervishD; +Cc: Andrew Morton, Juergen Beisert, linux-kernel
DervishD <lkml@dervishd.net> writes:
>> It would add the limitation to following simple usage,
>>
>> # mount -t vfat /dev/sda1 /mnt
>> # cp -a * /mnt
>> # umount
>>
>> if /dev/sda1 was the large and slow device, "mount" will need several
>> minutes to counts free clusters. I think the user will be hard to
>> accept the several minutes at "mount".
>
> I can carry some tests, but if Windows does that tasks lightning
> fast, Linux surely does it faster ;) I don't think, anyway, that having
> a huge USB disk is a common practice when using "modest" machines.
>
> If you want, I can perform a couple of tests. I have a 80GB disk
> that I can connect using an USB adapter and my machine is AMD Athlon XP
> 1900+ with 1GB of RAM, which looks pretty slow nowadays O:)
Yes, I think it's not common practice too. But I don't see why do you
want to scanning at the mount.
Thanks.
--
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Wrong free clusters count on FAT32
2007-04-23 2:27 ` OGAWA Hirofumi
@ 2007-04-23 6:19 ` DervishD
0 siblings, 0 replies; 25+ messages in thread
From: DervishD @ 2007-04-23 6:19 UTC (permalink / raw)
To: OGAWA Hirofumi; +Cc: Andrew Morton, Juergen Beisert, linux-kernel
Hi Ogawa :)
* OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> dixit:
> DervishD <lkml@dervishd.net> writes:
> >> It would add the limitation to following simple usage,
> >>
> >> # mount -t vfat /dev/sda1 /mnt
> >> # cp -a * /mnt
> >> # umount
> >>
> >> if /dev/sda1 was the large and slow device, "mount" will need several
> >> minutes to counts free clusters. I think the user will be hard to
> >> accept the several minutes at "mount".
> >
> > I can carry some tests, but if Windows does that tasks lightning
> > fast, Linux surely does it faster ;) I don't think, anyway, that having
> > a huge USB disk is a common practice when using "modest" machines.
> >
> > If you want, I can perform a couple of tests. I have a 80GB disk
> > that I can connect using an USB adapter and my machine is AMD Athlon XP
> > 1900+ with 1GB of RAM, which looks pretty slow nowadays O:)
>
> Yes, I think it's not common practice too. But I don't see why do you
> want to scanning at the mount.
Just because I was thinking that, otherwise, the scanning would need
to be done at each statfs call, but I was wrong: once the scanning is
done and the count is right, it is written at free_clusters and used
afterwards.
I thought the mount was the best point in time for doing this.
Raúl Núñez de Arenas Coronado
--
Linux Registered User 88736 | http://www.dervishd.net
It's my PC and I'll cry if I want to... RAmen!
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2007-04-23 6:17 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-04-19 8:57 Wrong free clusters count on FAT32 DervishD
2007-04-19 10:27 ` Boaz Harrosh
2007-04-19 14:23 ` DervishD
2007-04-19 11:52 ` Juergen Beisert
2007-04-19 14:19 ` DervishD
2007-04-21 22:42 ` OGAWA Hirofumi
2007-04-22 3:18 ` Andrew Morton
2007-04-22 4:26 ` OGAWA Hirofumi
2007-04-22 11:26 ` DervishD
2007-04-22 11:55 ` OGAWA Hirofumi
2007-04-22 20:08 ` DervishD
2007-04-23 2:27 ` OGAWA Hirofumi
2007-04-23 6:19 ` DervishD
2007-04-22 11:17 ` DervishD
2007-04-22 11:28 ` Juergen Beisert
[not found] <8bAF0-3Yj-29@gated-at.bofh.it>
[not found] ` <8bDta-8rc-31@gated-at.bofh.it>
[not found] ` <8bFEr-3B4-1@gated-at.bofh.it>
[not found] ` <8cwz8-2fE-13@gated-at.bofh.it>
2007-04-22 13:28 ` Bodo Eggert
2007-04-22 14:04 ` OGAWA Hirofumi
2007-04-22 14:29 ` OGAWA Hirofumi
2007-04-22 14:53 ` Andreas Schwab
2007-04-22 15:13 ` OGAWA Hirofumi
2007-04-22 21:46 ` Bodo Eggert
2007-04-23 2:20 ` OGAWA Hirofumi
2007-04-22 20:11 ` DervishD
[not found] ` <8cAMl-du-3@gated-at.bofh.it>
[not found] ` <8cBS7-1Qa-1@gated-at.bofh.it>
[not found] ` <8cIqE-3qZ-9@gated-at.bofh.it>
[not found] ` <8cITG-40H-5@gated-at.bofh.it>
2007-04-22 17:21 ` Bodo Eggert
2007-04-22 17:44 ` OGAWA Hirofumi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox