* End of FAT directories @ 2011-04-22 18:48 Michael Karcher 2011-04-22 18:51 ` End of FAT directories (added missing reference) Michael Karcher 2011-04-22 20:40 ` End of FAT directories OGAWA Hirofumi 0 siblings, 2 replies; 12+ messages in thread From: Michael Karcher @ 2011-04-22 18:48 UTC (permalink / raw) To: linux kernel FAT, dosfstools, mtools; +Cc: linux-kernel Hello Linux FAT developers and developers of related tools, there seem to be different interpretations around how to read a FAT directory that contains an entry starting with a NUL byte before further entries *NOT* starting with NUL bytes. I uploaded an example image of this type to [1]. It contains three zero-byte files "WINDOWS", "IS", "GREAT", an empty directory slot and a file with contents named "NOT". Listing a floppy with this directory contents in Windows gives the output "WINDOWS", "IS" and "GREAT". Mounting the image in Linux results in four visible files. Running dosfsck on that image outputs four files, everything OK, while Windows CHKDSK will found one lost cluster in one chain. mdir only lists the three files Windows will find too, but when you copy a file that needs a long name to the image using mcopy, the file will be appended where there are enough free slots in succession, i.e. it will not fill the single empty slot. That means mcopy allocates space for the file, writes a directory entry for it, but mdir does not see it afterwards. On the other hand, as soon as you add a file that does not need a long name entry that fills the hole, mdir shows all the files it previously didn't show. I'm afraid this is not a pure academic post, but I write it because I spent hours on trying to find out why I had problem to make some USB flash drive bootable with syslinux (which follows the MS model on loading files while booting) after copying files with the linux kernel (which follows the Linux model, obviously), additionally dosfsck on that stick showed quite strange results (it dived into a directory that does no longer exist). Another incarnation is mentioned in https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php?id=himddiskformat in the section "Filesystem Issues with some Models on Linux". Some of the Sony devices clear only the first half of the cluster containing that subdirectory to zeroes, which works fine if software aborts on the first entry being zero, but obviously makes problem if the second half of the cluster is accessed. I would really appreciate it if all FOSS working on FAT would agree to one behaviour, and I think we have no choice except being MS compatible (i.e. implement the behaviour as it has always been in DOS and Windows) Please comment. Regards, Michael Karcher ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories (added missing reference) 2011-04-22 18:48 End of FAT directories Michael Karcher @ 2011-04-22 18:51 ` Michael Karcher 2011-04-22 20:40 ` End of FAT directories OGAWA Hirofumi 1 sibling, 0 replies; 12+ messages in thread From: Michael Karcher @ 2011-04-22 18:51 UTC (permalink / raw) To: linux kernel FAT; +Cc: dosfstools, mtools, linux-kernel Am Freitag, den 22.04.2011, 20:48 +0200 schrieb Michael Karcher: > Hello Linux FAT developers and developers of related tools, > > there seem to be different interpretations around how to read a FAT > directory that contains an entry starting with a NUL byte before further > entries *NOT* starting with NUL bytes. I uploaded an example image of > this type to [1]. It contains three zero-byte files "WINDOWS", "IS", > "GREAT", an empty directory slot and a file with contents named "NOT". [1]: http://users.physik.fu-berlin.de/~mkarcher/fatimg.gz ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-22 18:48 End of FAT directories Michael Karcher 2011-04-22 18:51 ` End of FAT directories (added missing reference) Michael Karcher @ 2011-04-22 20:40 ` OGAWA Hirofumi 2011-04-22 21:56 ` Michael Karcher 1 sibling, 1 reply; 12+ messages in thread From: OGAWA Hirofumi @ 2011-04-22 20:40 UTC (permalink / raw) To: Michael Karcher; +Cc: dosfstools, mtools, linux-kernel Michael Karcher <kernel@mkarcher.dialup.fu-berlin.de> writes: > I'm afraid this is not a pure academic post, but I write it because I > spent hours on trying to find out why I had problem to make some USB > flash drive bootable with syslinux (which follows the MS model on > loading files while booting) after copying files with the linux kernel > (which follows the Linux model, obviously), additionally dosfsck on that > stick showed quite strange results (it dived into a directory that does > no longer exist). Another incarnation is mentioned in > https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php?id=himddiskformat > in the section "Filesystem Issues with some Models on Linux". Some of > the Sony devices clear only the first half of the cluster containing > that subdirectory to zeroes, which works fine if software aborts on the > first entry being zero, but obviously makes problem if the second half > of the cluster is accessed. > > I would really appreciate it if all FOSS working on FAT would agree to > one behaviour, and I think we have no choice except being MS compatible > (i.e. implement the behaviour as it has always been in DOS and Windows) It is simply the bug of Sony devices, and can't work on it for both of Windows and Linux (it have to be fixed by fsck). Yeah, I know Linux can stop on first zeroed directory entry (I will not complain about it), but it doesn't fix the problem at all. Assume 0-3 directory entires fine, 4 is zeroed entry, and 5- is crappy data. 1| ENT1 valid 2| ENT2 valid 3| ENT3 valid 4| zeroed 5| djafdjoi crappy data ... Windows will stop at 4, but if windows write new entry as 4, 5- will show those up as crap like linux. I.e. never adds new zeroed entry as end of directory. In short, in spec, directory must be initialized by zero (it is same assumption on linux and windows). Thanks. -- OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-22 20:40 ` End of FAT directories OGAWA Hirofumi @ 2011-04-22 21:56 ` Michael Karcher 2011-04-23 0:06 ` OGAWA Hirofumi 0 siblings, 1 reply; 12+ messages in thread From: Michael Karcher @ 2011-04-22 21:56 UTC (permalink / raw) To: OGAWA Hirofumi; +Cc: dosfstools, mtools, linux-kernel Am Samstag, den 23.04.2011, 05:40 +0900 schrieb OGAWA Hirofumi: > It is simply the bug of Sony devices, and can't work on it for both of > Windows and Linux (it have to be fixed by fsck). You are right. Although as the sony device *does* clear out the first 8K of the 16K cluster, that makes 256 correctly working directory entries. This directory is only used for storing a dozen of files with no long names, the existing problem you correctly describe is not relevant in practice. > Assume 0-3 directory entires fine, 4 is zeroed entry, and 5- is crappy data. > > 1| ENT1 valid > 2| ENT2 valid > 3| ENT3 valid > 4| zeroed > 5| djafdjoi crappy data > ... > > Windows will stop at 4, but if windows write new entry as 4, 5- will > show those up as crap like linux. You are right. Thanks for your input. This means Windows and mtools behave exactly the same way (and thus mtools can not really be considered buggy). I expected Windows to fix up the end, but it does not. I still would prefer greatly if Windows and Linux at least read media the same way (i.e. stop on the first zeroed entry), I even have a patch for that somewhere around I can dig out. I furthermore think it is quite important for dosfsck and the linux kernel to behave consistent (think of valid entries after the zeroed entry linking to "lost clusters"). If I prepare patches for the Linux kernel to - terminate reading at the first zeroed entry - making sure that if a zeroed entry gets populated, the next entry gets zeroed unless we hit the end of the directory (deviating from Windows behaviour, intentionally, to keep the filesystem consistent) And for dosfsck to - terminate reading at the first zeroed entry - offer filling the remainder of a directory cluster with zero bytes. Would this set of patches be acceptable? > In short, in spec, directory must be initialized by > zero (it is same assumption on linux and windows). Turns out you are right. Sorry for making the implicit assumption that windows handles the case better. Regards, Michael Karcher ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-22 21:56 ` Michael Karcher @ 2011-04-23 0:06 ` OGAWA Hirofumi 2011-04-23 11:38 ` Michael Karcher 0 siblings, 1 reply; 12+ messages in thread From: OGAWA Hirofumi @ 2011-04-23 0:06 UTC (permalink / raw) To: Michael Karcher; +Cc: dosfstools, mtools, linux-kernel Michael Karcher <kernel@mkarcher.dialup.fu-berlin.de> writes: >> Windows will stop at 4, but if windows write new entry as 4, 5- will >> show those up as crap like linux. > You are right. Thanks for your input. This means Windows and mtools > behave exactly the same way (and thus mtools can not really be > considered buggy). I expected Windows to fix up the end, but it does > not. > > I still would prefer greatly if Windows and Linux at least read media > the same way (i.e. stop on the first zeroed entry), I even have a patch > for that somewhere around I can dig out. > I furthermore think it is quite important for dosfsck and the linux > kernel to behave consistent (think of valid entries after the zeroed > entry linking to "lost clusters"). If I prepare patches for the Linux > kernel to > - terminate reading at the first zeroed entry This is acceptable (stop at zero is an simply optimization), and I'll just care about stability and cleanness about it. > - making sure that if a zeroed entry gets populated, the next entry > gets zeroed unless we hit the end of the directory (deviating from > Windows behaviour, intentionally, to keep the filesystem consistent) This would be unacceptable. There are several FAT implementations like this crap, I can't honor an one of those until to decide to try to support all of craps. I.e. This is the out of specs anymore. Why can you say those are invalid entries even if Windows honors those entries? And the above is simulate Windows implementation over specs, why isn't here? For overwriting the zeroed entry, why we have to check all entries until see next zeroed (yeah, now, we allowed the crappy data after zeroed)? My state is simple - the current implement should be fine. I.e. uninitialized directory is the simply bug. (However, I can honor to stop at zero, because it is mentioned in spec as optimization.) > And for dosfsck to > - terminate reading at the first zeroed entry > - offer filling the remainder of a directory cluster with zero bytes. > Would this set of patches be acceptable? Sorry. I can't say about dosfsck, I'm not maintainer of it. Ideas, dosfstools maintainer? -- OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-23 0:06 ` OGAWA Hirofumi @ 2011-04-23 11:38 ` Michael Karcher 2011-04-23 13:46 ` OGAWA Hirofumi 2011-04-28 13:25 ` Pavel Machek 0 siblings, 2 replies; 12+ messages in thread From: Michael Karcher @ 2011-04-23 11:38 UTC (permalink / raw) To: OGAWA Hirofumi; +Cc: dosfstools, mtools, linux-kernel Am Samstag, den 23.04.2011, 09:06 +0900 schrieb OGAWA Hirofumi: > > - terminate reading at the first zeroed entry > This is acceptable (stop at zero is an simply optimization), and I'll > just care about stability and cleanness about it. OK, then I will dig out my patch for that. > > - making sure that if a zeroed entry gets populated, the next entry > > gets zeroed unless we hit the end of the directory (deviating from > > Windows behaviour, intentionally, to keep the filesystem consistent)> > I.e. This is the out of specs anymore. Why can you say those are invalid > entries even if Windows honors those entries? My point is that creating a file on a consistent file system (i.e. one where all cluster chains are referenced by exactly one visible directory entry) should still result in a consistent file system. And by "consistent" I mean any state windows chkdsk accepts. > And the above is simulate Windows implementation over specs, why isn't > here? Wow, there actually *is* a formal specification for FAT? I am positively surprised by that. I used to think that the "spec" for FAT is "do as DOS does". On a quick google, I do find the MS EFI FAT32 specification, which is obviuosly non-free. Is there anything else? > For overwriting the > zeroed entry, why we have to check all entries until see next zeroed > (yeah, now, we allowed the crappy data after zeroed)? My point was not about checking all entries until the next zeroed entry, which is overcomplicating stuff. I meant that if we hit the end (i.e. the first zeroed entry) when searching for free slots, in that case we just clear the one entry directly following the inserted entries. This makes sure that no files magically appear. I *do* understand that they also appear on Windows, though. My point about "modifying a FAT system that passes MS chkdsk with Linux should result in a FAT system that passes MS chkdsk" is in my eyes more important than "Windows damages a file system it considers fine in this situation, so we should do the same." In the end, there are two valid interpretations on the problem at hand: - DOS/Windows is buggy in exposing the crap when filling the "hole", which is proven by chkdsk not complaining about the garbage in the directory, and we should not follow the bug in DOS/Windows - chkdsk is buggy in not complaining about trailing garbage in directories, which are proven to be invalid by DOS/Windows messing up on them, and we shouldn't complicate our FAT implementation by trying to work around a bug in chkdsk. Obviously, I tend to the first interpretation while you prefer the second interpretation. As you are the maintainer, it looks like I have to get away with that. > My state is simple - the current implement should be fine. > I.e. uninitialized directory is the simply bug. For the uninitialized entries on Hi-MD media, I agree with you that these are caused by a firmware bug of older Sony devices. And to support these devices "good enough", the stop-at-zero would be enough. Regretfully I don't know the history of the USB flash memory I recently got where the root directory was ended by a single entry starting with a NUL byte and containing valid entries past that. The experience with the inconsistent behaviour of different tools on that flash memory made me write the mail about the current state of the FAT implementation. > > And for dosfsck to > > - terminate reading at the first zeroed entry > > - offer filling the remainder of a directory cluster with zero bytes. > > Would this set of patches be acceptable? > Sorry. I can't say about dosfsck, I'm not maintainer of it. Ideas, > dosfstools maintainer? I think the *most* important fact is to have at least dosfsck and linux consistent about directories with trailing garbage. So I won't even try to submit a patch to the Linux kernel that stops reading at the first unused directory entry if dosfsck will not accept to stop reading at the first unused directory entry. Regards and thanks for discussing, Michael Karcher ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-23 11:38 ` Michael Karcher @ 2011-04-23 13:46 ` OGAWA Hirofumi 2011-04-28 13:25 ` Pavel Machek 1 sibling, 0 replies; 12+ messages in thread From: OGAWA Hirofumi @ 2011-04-23 13:46 UTC (permalink / raw) To: Michael Karcher; +Cc: dosfstools, mtools, linux-kernel Michael Karcher <kernel@mkarcher.dialup.fu-berlin.de> writes: >> And the above is simulate Windows implementation over specs, why isn't >> here? > Wow, there actually *is* a formal specification for FAT? I am positively > surprised by that. I used to think that the "spec" for FAT is "do as DOS > does". On a quick google, I do find the MS EFI FAT32 specification, > which is obviuosly non-free. Is there anything else? I guess it is called fatgen.doc in past, but I'm not sure. I can't help about what MS does. >> For overwriting the >> zeroed entry, why we have to check all entries until see next zeroed >> (yeah, now, we allowed the crappy data after zeroed)? > My point was not about checking all entries until the next zeroed entry, > which is overcomplicating stuff. I meant that if we hit the end (i.e. > the first zeroed entry) when searching for free slots, in that case we > just clear the one entry directly following the inserted entries. This > makes sure that no files magically appear. I *do* understand that they > also appear on Windows, though. My point about "modifying a FAT system > that passes MS chkdsk with Linux should result in a FAT system that > passes MS chkdsk" is in my eyes more important than "Windows damages a > file system it considers fine in this situation, so we should do the > same." I'll call it - first step of in-kernel fsck. > In the end, there are two valid interpretations on the problem at hand: > - DOS/Windows is buggy in exposing the crap when filling the "hole", > which is proven by chkdsk not complaining about the garbage in the > directory, and we should not follow the bug in DOS/Windows First of all, the bug is non-initialized directory. That's all. Yes, chkdsk should fix it. But, unfortunately chkdsk is not perfect, I remember another breakage of chkdsk can't fix. > - chkdsk is buggy in not complaining about trailing garbage in > directories, which are proven to be invalid by DOS/Windows messing up on > them, and we shouldn't complicate our FAT implementation by trying to > work around a bug in chkdsk. > > Obviously, I tend to the first interpretation while you prefer the > second interpretation. As you are the maintainer, it looks like I have > to get away with that. Because fsck is the job of fsck. Furthermore, without interactivity, fsck can't perfectly decide how to fix it actually (yes, it can guess, but that's guess). >> My state is simple - the current implement should be fine. >> I.e. uninitialized directory is the simply bug. > For the uninitialized entries on Hi-MD media, I agree with you that > these are caused by a firmware bug of older Sony devices. And to support > these devices "good enough", the stop-at-zero would be enough. > Regretfully I don't know the history of the USB flash memory I recently > got where the root directory was ended by a single entry starting with a > NUL byte and containing valid entries past that. The experience with the > inconsistent behaviour of different tools on that flash memory made me > write the mail about the current state of the FAT implementation. Yes, zero is the end of directory. It's right and said in the above fatgen. Guys might misread - fatgen says the rest of entries must be zero (i.e. it means initialized by zero). Or some spec (I know there is the FS spec in SD specs, I can't recall the detail though) may not mention about latter part, I don't know. > I think the *most* important fact is to have at least dosfsck and linux > consistent about directories with trailing garbage. So I won't even try > to submit a patch to the Linux kernel that stops reading at the first > unused directory entry if dosfsck will not accept to stop reading at the > first unused directory entry. OK. I think "stop at zero" is not effected by dosfsck though. With "stop at zero", linux will do like windows does, but crap data is already showed in current implementation. Whether dosfsck can fix it or not would be another story. Of course, I hope it does though. Thanks. -- OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-23 11:38 ` Michael Karcher 2011-04-23 13:46 ` OGAWA Hirofumi @ 2011-04-28 13:25 ` Pavel Machek 2011-04-28 13:43 ` Michael Karcher 1 sibling, 1 reply; 12+ messages in thread From: Pavel Machek @ 2011-04-28 13:25 UTC (permalink / raw) To: Michael Karcher; +Cc: OGAWA Hirofumi, dosfstools, mtools, linux-kernel Hi! > > For overwriting the > > zeroed entry, why we have to check all entries until see next zeroed > > (yeah, now, we allowed the crappy data after zeroed)? > My point was not about checking all entries until the next zeroed entry, > which is overcomplicating stuff. I meant that if we hit the end (i.e. > the first zeroed entry) when searching for free slots, in that case we > just clear the one entry directly following the inserted entries. This > makes sure that no files magically appear. I *do* understand that This sounds like a good idea, and costs nothing. I hope we can convince Ogawa... > In the end, there are two valid interpretations on the problem at hand: > - DOS/Windows is buggy in exposing the crap when filling the "hole", > which is proven by chkdsk not complaining about the garbage in the > directory, and we should not follow the bug in DOS/Windows > - chkdsk is buggy in not complaining about trailing garbage in > directories, which are proven to be invalid by DOS/Windows messing up on > them, and we shouldn't complicate our FAT implementation by trying to > work around a bug in chkdsk. I believe 2) is the case; but we should still work around that chkdsk bug. > I think the *most* important fact is to have at least dosfsck and linux > consistent about directories with trailing garbage. So I won't even try > to submit a patch to the Linux kernel that stops reading at the first > unused directory entry if dosfsck will not accept to stop reading at the > first unused directory entry. They should *not* be consistent. Kernel should stop at zero invalid entry. fsck should consider any garbage past zero entry as an error, and zero it out. (Complaining about duplicate blocks is unhelpful but better than nothing. It should really zero the garbage out.) -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-28 13:25 ` Pavel Machek @ 2011-04-28 13:43 ` Michael Karcher 2011-04-28 15:44 ` OGAWA Hirofumi 2011-04-28 20:51 ` Pavel Machek 0 siblings, 2 replies; 12+ messages in thread From: Michael Karcher @ 2011-04-28 13:43 UTC (permalink / raw) To: Pavel Machek; +Cc: OGAWA Hirofumi, dosfstools, mtools, linux-kernel Am Donnerstag, den 28.04.2011, 15:25 +0200 schrieb Pavel Machek: > > > For overwriting the > > > zeroed entry, why we have to check all entries until see next zeroed > > > (yeah, now, we allowed the crappy data after zeroed)? > > My point was not about checking all entries until the next zeroed entry, > > which is overcomplicating stuff. I meant that if we hit the end (i.e. > > the first zeroed entry) when searching for free slots, in that case we > > just clear the one entry directly following the inserted entries. This > > makes sure that no files magically appear. I *do* understand that > This sounds like a good idea, and costs nothing. I hope we can > convince Ogawa... I am going to implement that and publish a patch. If the patch is clean enough, maybe we can convince him. But the freedom of open source can't prevent me from using a patch like that, of course. And maybe it also helps when the patch gets some testing. > > I think the *most* important fact is to have at least dosfsck and linux > > consistent about directories with trailing garbage. So I won't even try > > to submit a patch to the Linux kernel that stops reading at the first > > unused directory entry if dosfsck will not accept to stop reading at the > > first unused directory entry. > They should *not* be consistent. > > Kernel should stop at zero invalid entry. > > fsck should consider any garbage past zero entry as an error, and zero > it out. (Complaining about duplicate blocks is unhelpful but better > than nothing. It should really zero the garbage out.) In fact, what you describe is something I would call "consistent". I was not implying that kernel and dosfsck should do exactly the same thing, but I was implying that the kernel and dosfsck should either both or none consider entries past the gap as existing file. Current state is: both consider past-gap entries valid. Your described state will be: none consider past-gap entries valid. The behaviour you describe is exactly what I would imagine as best way to go. Maybe fsck should ask for "shift post-gap entries/create a dummy deleted entry" vs. "clear post-gap entries". Regards, Michael Karcher ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-28 13:43 ` Michael Karcher @ 2011-04-28 15:44 ` OGAWA Hirofumi 2011-06-28 23:09 ` Alain Knaff 2011-04-28 20:51 ` Pavel Machek 1 sibling, 1 reply; 12+ messages in thread From: OGAWA Hirofumi @ 2011-04-28 15:44 UTC (permalink / raw) To: Michael Karcher; +Cc: Pavel Machek, dosfstools, mtools, linux-kernel Michael Karcher <kernel@mkarcher.dialup.fu-berlin.de> writes: > Am Donnerstag, den 28.04.2011, 15:25 +0200 schrieb Pavel Machek: >> > > For overwriting the >> > > zeroed entry, why we have to check all entries until see next zeroed >> > > (yeah, now, we allowed the crappy data after zeroed)? >> > My point was not about checking all entries until the next zeroed entry, >> > which is overcomplicating stuff. I meant that if we hit the end (i.e. >> > the first zeroed entry) when searching for free slots, in that case we >> > just clear the one entry directly following the inserted entries. This >> > makes sure that no files magically appear. I *do* understand that >> This sounds like a good idea, and costs nothing. I hope we can >> convince Ogawa... > > I am going to implement that and publish a patch. If the patch is clean > enough, maybe we can convince him. But the freedom of open source can't > prevent me from using a patch like that, of course. And maybe it also > helps when the patch gets some testing. Of course, it's good to publish. However if you can't detect buggy or corrupted directory, I wouldn't ack to help buggy stuff. So it can be the mount option, but I can't see at all the point it is in-kernel as FS. >> Kernel should stop at zero invalid entry. I'm not complaining to stop at zero. Rather I acked already if it didn't overwrite after zeroed entry. >> fsck should consider any garbage past zero entry as an error, and zero >> it out. (Complaining about duplicate blocks is unhelpful but better >> than nothing. It should really zero the garbage out.) > > In fact, what you describe is something I would call "consistent". I was > not implying that kernel and dosfsck should do exactly the same thing, > but I was implying that the kernel and dosfsck should either both or > none consider entries past the gap as existing file. > Current state is: both consider past-gap entries valid. Your described > state will be: none consider past-gap entries valid. The behaviour you > describe is exactly what I would imagine as best way to go. Maybe fsck > should ask for "shift post-gap entries/create a dummy deleted entry" > vs. "clear post-gap entries". Yes. Also my opinion is the fsck should get ack from user to overwrite data or salvage (has possibility to salvage) if interactive mode, then fixes it. Thanks. -- OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-28 15:44 ` OGAWA Hirofumi @ 2011-06-28 23:09 ` Alain Knaff 0 siblings, 0 replies; 12+ messages in thread From: Alain Knaff @ 2011-06-28 23:09 UTC (permalink / raw) To: OGAWA Hirofumi Cc: Michael Karcher, Pavel Machek, dosfstools, mtools, linux-kernel Hello, I've just released mtools 4.0.17 It fixes the issue by: 1. Considering all directory entry after an "endmark" (first byte of filename zero) to be free, even if the first byte is not zero (this avoids the problem of not finding any free space because of phantom entries after endmark) 2. Writing an endmark after having created a filename that's beyond the endmark (this avoids the problem of accidentally transforming phantom entries into real entries by "filling the gap") http://www.gnu.org/software/mtools/download.html Thanks for drawing my attention to this issue. Regards, Alain On 2011-04-28 17:44, OGAWA Hirofumi wrote: > Michael Karcher <kernel@mkarcher.dialup.fu-berlin.de> writes: > >> Am Donnerstag, den 28.04.2011, 15:25 +0200 schrieb Pavel Machek: >>>>> For overwriting the >>>>> zeroed entry, why we have to check all entries until see next zeroed >>>>> (yeah, now, we allowed the crappy data after zeroed)? >>>> My point was not about checking all entries until the next zeroed entry, >>>> which is overcomplicating stuff. I meant that if we hit the end (i.e. >>>> the first zeroed entry) when searching for free slots, in that case we >>>> just clear the one entry directly following the inserted entries. This >>>> makes sure that no files magically appear. I *do* understand that >>> This sounds like a good idea, and costs nothing. I hope we can >>> convince Ogawa... >> >> I am going to implement that and publish a patch. If the patch is clean >> enough, maybe we can convince him. But the freedom of open source can't >> prevent me from using a patch like that, of course. And maybe it also >> helps when the patch gets some testing. > > Of course, it's good to publish. However if you can't detect buggy or > corrupted directory, I wouldn't ack to help buggy stuff. > > So it can be the mount option, but I can't see at all the point it is > in-kernel as FS. > >>> Kernel should stop at zero invalid entry. > > I'm not complaining to stop at zero. Rather I acked already if it didn't > overwrite after zeroed entry. > >>> fsck should consider any garbage past zero entry as an error, and zero >>> it out. (Complaining about duplicate blocks is unhelpful but better >>> than nothing. It should really zero the garbage out.) >> >> In fact, what you describe is something I would call "consistent". I was >> not implying that kernel and dosfsck should do exactly the same thing, >> but I was implying that the kernel and dosfsck should either both or >> none consider entries past the gap as existing file. >> Current state is: both consider past-gap entries valid. Your described >> state will be: none consider past-gap entries valid. The behaviour you >> describe is exactly what I would imagine as best way to go. Maybe fsck >> should ask for "shift post-gap entries/create a dummy deleted entry" >> vs. "clear post-gap entries". > > Yes. Also my opinion is the fsck should get ack from user to overwrite > data or salvage (has possibility to salvage) if interactive mode, then > fixes it. > > Thanks. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: End of FAT directories 2011-04-28 13:43 ` Michael Karcher 2011-04-28 15:44 ` OGAWA Hirofumi @ 2011-04-28 20:51 ` Pavel Machek 1 sibling, 0 replies; 12+ messages in thread From: Pavel Machek @ 2011-04-28 20:51 UTC (permalink / raw) To: Michael Karcher; +Cc: OGAWA Hirofumi, dosfstools, mtools, linux-kernel Hi! > > Kernel should stop at zero invalid entry. > > > > fsck should consider any garbage past zero entry as an error, and zero > > it out. (Complaining about duplicate blocks is unhelpful but better > > than nothing. It should really zero the garbage out.) > > In fact, what you describe is something I would call "consistent". I was > not implying that kernel and dosfsck should do exactly the same thing, > but I was implying that the kernel and dosfsck should either both or Aha, violent agreement, then. Good :-). Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2011-06-28 23:21 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-04-22 18:48 End of FAT directories Michael Karcher 2011-04-22 18:51 ` End of FAT directories (added missing reference) Michael Karcher 2011-04-22 20:40 ` End of FAT directories OGAWA Hirofumi 2011-04-22 21:56 ` Michael Karcher 2011-04-23 0:06 ` OGAWA Hirofumi 2011-04-23 11:38 ` Michael Karcher 2011-04-23 13:46 ` OGAWA Hirofumi 2011-04-28 13:25 ` Pavel Machek 2011-04-28 13:43 ` Michael Karcher 2011-04-28 15:44 ` OGAWA Hirofumi 2011-06-28 23:09 ` Alain Knaff 2011-04-28 20:51 ` Pavel Machek
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).