[U-Boot] Fat write problem

[U-Boot] Fat write problem

[Ruud Commandeur]

Hello again,

This week I decided to do some further research and testing regarding
this problem.
With the image I had from the previous time, I could immediately
reproduce it and
by adding more and more debug prints, I tried to find the cause. Sofar,
I have not
succeeded in this yet.

However: later on I started testing with a freshly formatted drive (32
MB FAT partition)
and kept repeating the fatwrite command:

fatwrite mmc 0:1 42000000 test-x 200

where x runs from 1, 2,3 and further. And this way I could reproduce it
quite easily.
Writing always fails for the 32nd file. This is with the partition
formatted with a 512 byte
sector size and a cluster size of 4. If the cluster size is 1 (formatted
by Windows),
it already fails at the 8th file.

If I create a subdirectory (from Linux) with already 24 files in it, I
can still write 29 files
and it fails at number 30. Also, if earlier files were deleted from the
root-directory, they
still count in the total number of files here.

If I take out the card where u-boot fails to write new files, I can
still add new files from
my PC with Linux or Windows.

I tested with both long and short filenames (same result), VFAT is
enabled.

I hope this gives you all some more information about this problem and
perhaps it is even a
known problem (limited number of files in the root directory?). I know
it is voor FAT16, but
that was 512 entries if I am correct.

Regards,

Ruud


> -----Oorspronkelijk bericht-----
> Van: Ruud Commandeur 
> Verzonden: woensdag 23 oktober 2013 17:18
> Aan: U-Boot list
> Onderwerp: Fat write problem
> 
> Hi Everyone,
> 
> After about half a year without problems for fatwrite, I have 
> some new problem.
> The fatwrites for the u-boot are used to write the uimage and 
> dtb file to a 32
> MB FAT16 partition on an SD-card. Since this morning, the 
> writing of a new file
> keeps failing. I still can write files that are already present.
> But if try and write some file with a name that doe not exist 
> yet, it fails. I guess
> this just hasn't been tested for a while, since these 
> filenames don't change.
> Meanwhile some extra files have been copied to the partition 
> from the Linux
> platform. I did check the SD-card partition with fsck and 
> could not find any
> problems.
> 
> I did enable the debug for MMC and FAT to get the following output:
> 
> fatwrite mmc 0:1 42000000 test-file 200
> writing test-file
> MMC0: CMD16
> MMC0: CMD17
> MMC0: CMD16
> MMC0: CMD18
> MMC0: CMD12
> get_dentfromdir: test-file
> gc - clustnum: -6, startsect: 132
> MMC0: CMD16
> MMC0: CMD18
> MMC0: CMD12
> vfatname: |uimage|
> Mismatch: |uimage|uimage|
> vfatname: |imx28-evk.dtb|
> Mismatch: |imx28-~1.dtb|imx28-evk.dtb|
> vfatname: |linux-file1.txt|
> Mismatch: |linux-~1.txt|linux-file1.txt|
> Mismatch: |address.ini||
> vfatname: |imx28-clb.dtb|
> Mismatch: |imx28-~2.dtb|imx28-clb.dtb|
> vfatname: |img-copy|
> Mismatch: |img-copy|img-copy|
> vfatname: |tessie|
> Mismatch: |tessie|tessie|
> vfatname: |img-copy2|
> Mismatch: |img-co~1|img-copy2|
> vfatname: |imx-28-clb-37.dtb|
> Mismatch: |imx-28~1.dtb|imx-28-clb-37.dtb|
> vfatname: |hello8000.wav|
> Mismatch: |hello8~1.wav|hello8000.wav|
> vfatname: |versions.ini|
> Mismatch: |versions.ini|versions.ini|
> vfatname: |mx28-310.dtb|
> Mismatch: |mx28-310.dtb|mx28-310.dtb|
> FAT16: entry: 0xfffffffa = -6, offset: 0x03fa = 1018
> MMC: block number 0x1000806 exceeds max(0x762c00)
> FAT16: ret: 00000000, entry: fffffffa, offset: 03fa
> curclust: 0x0
> Invalid FAT entry
> name.ext : <NULL>.
> FAT16: entry: 0x0003 = 3, offset: 0x0003 = 3
> error: overflow occurs
> error: writing FAT blocks
> FAT16: entry: 0x0004 = 4, offset: 0x0004 = 4
> error: overflow occurs
> error: writing FAT blocks
> FAT16: entry: 0x0005 = 5, offset: 0x0005 = 5
> error: overflow occurs
> error: writing FAT blocks
> FAT16: entry: 0x0006 = 6, offset: 0x0006 = 6
> ... and this goes on for a long long time...
> 
> It points to get_fatent_value( ) where things go wrong. Or 
> perhaps the call of
> this function with the value 0xfffffffa (-6). I don't know if 
> that is supposed
> to be possible?
> 
> At the moment I am a bit stuck here, so if anyone would have 
> an idea what
> goes wrong here, please let me know.
> 
> Thanks,
> 
> Ruud
> 
> N.B. It might be that this can be solved by clearing the FAT 
> partition and starting
> with a clean sheet. But I prefer not to lose this error 
> situaion before I know
> what goes wrong here.

[U-Boot] Fat write problem

[Detlev Zundel]

Hi Ruud,

> This week I decided to do some further research and testing regarding
> this problem.
> With the image I had from the previous time, I could immediately
> reproduce it and
> by adding more and more debug prints, I tried to find the cause. Sofar,
> I have not
> succeeded in this yet.
>
> However: later on I started testing with a freshly formatted drive (32
> MB FAT partition)
> and kept repeating the fatwrite command:
>
> fatwrite mmc 0:1 42000000 test-x 200
>
> where x runs from 1, 2,3 and further. And this way I could reproduce it
> quite easily.
> Writing always fails for the 32nd file. This is with the partition
> formatted with a 512 byte
> sector size and a cluster size of 4. If the cluster size is 1 (formatted
> by Windows),
> it already fails at the 8th file.
>
> If I create a subdirectory (from Linux) with already 24 files in it, I
> can still write 29 files
> and it fails at number 30. Also, if earlier files were deleted from the
> root-directory, they
> still count in the total number of files here.
>
> If I take out the card where u-boot fails to write new files, I can
> still add new files from
> my PC with Linux or Windows.
>
> I tested with both long and short filenames (same result), VFAT is
> enabled.
>
> I hope this gives you all some more information about this problem and
> perhaps it is even a
> known problem (limited number of files in the root directory?). I know
> it is voor FAT16, but
> that was 512 entries if I am correct.

Thanks for the extensive research into this problem.  For people to
help, I think the barrier of reproducing the problem is "somewhat
high", so it occurred to me if you can help setup a very easy test for
people to work on.  Would you be able to generate a small image that one
can dd to a mmc card and then immediately provoke the error?  If you
don't have any hosting space, as a last resort I'd be fine for you to
put it on our wiki [1].

Cheers
  Detlev

[1] http://www.denx.de/wiki/view/U-Boot/TooBigPatches

-- 
Golden rule #12:   When the comments do not match the code, they probably are
both wrong. -- Steven Rostedt <1300126962.9910.128.camel@gandalf.stny.rr.com>
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de

[U-Boot] Fat write problem

[Ruud Commandeur]

 

> -----Oorspronkelijk bericht-----
> Van: Detlev Zundel [mailto:dzu at denx.de] 
> Verzonden: vrijdag 17 januari 2014 15:59
> Aan: Ruud Commandeur
> CC: U-Boot list
> Onderwerp: Re: [U-Boot] Fat write problem
> 
> Hi Ruud,
> 
> > This week I decided to do some further research and testing 
> regarding
> > this problem.
> > With the image I had from the previous time, I could immediately
> > reproduce it and
> > by adding more and more debug prints, I tried to find the 
> cause. Sofar,
> > I have not
> > succeeded in this yet.
> >
> > However: later on I started testing with a freshly 
> formatted drive (32
> > MB FAT partition)
> > and kept repeating the fatwrite command:
> >
> > fatwrite mmc 0:1 42000000 test-x 200
> >
> > where x runs from 1, 2,3 and further. And this way I could 
> reproduce it
> > quite easily.
> > Writing always fails for the 32nd file. This is with the partition
> > formatted with a 512 byte
> > sector size and a cluster size of 4. If the cluster size is 
> 1 (formatted
> > by Windows),
> > it already fails at the 8th file.
> >
> > If I create a subdirectory (from Linux) with already 24 
> files in it, I
> > can still write 29 files
> > and it fails at number 30. Also, if earlier files were 
> deleted from the
> > root-directory, they
> > still count in the total number of files here.
> >
> > If I take out the card where u-boot fails to write new files, I can
> > still add new files from
> > my PC with Linux or Windows.
> >
> > I tested with both long and short filenames (same result), VFAT is
> > enabled.
> >
> > I hope this gives you all some more information about this 
> problem and
> > perhaps it is even a
> > known problem (limited number of files in the root 
> directory?). I know
> > it is voor FAT16, but
> > that was 512 entries if I am correct.
> 
> Thanks for the extensive research into this problem.  For people to
> help, I think the barrier of reproducing the problem is "somewhat
> high", so it occurred to me if you can help setup a very easy test for
> people to work on.  Would you be able to generate a small 
> image that one
> can dd to a mmc card and then immediately provoke the error?  If you
> don't have any hosting space, as a last resort I'd be fine for you to
> put it on our wiki [1].
> 
Hi Detlev,

Thanks for your response. Finally I get some ;-)

I can look for creating some image, but I don't think it is that hard to
reproduce (now I think to know what causes it). Either use 32 times the
fatwrite command (using the cmd history and increment the x-nr each
time)
or just copy up to 31 files to the root of the FAT partition by using
the host
machine and place the SD-card back on the targe device running u-boot.

Hope this makes it easy enough, otherwise I will be glad to hear from
you.

N.B. All this was with a FAT16 partition. If I use FAT32, I have no such
problem! At least not for 45 files up till now...

Regards,

Ruud

> Cheers
>   Detlev
> 
> [1] http://www.denx.de/wiki/view/U-Boot/TooBigPatches
> 
> -- 
> Golden rule #12:   When the comments do not match the code, 
> they probably are
> both wrong. -- Steven Rostedt 
> <1300126962.9910.128.camel@gandalf.stny.rr.com>
> --
> DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
> 

[U-Boot] Fat write problem

[Ruud Commandeur]

 

> -----Oorspronkelijk bericht-----
> Van: Ruud Commandeur 
> Verzonden: vrijdag 17 januari 2014 16:50
> Aan: 'Detlev Zundel'
> CC: U-Boot list
> Onderwerp: RE: [U-Boot] Fat write problem
> 
>  
> 
> > -----Oorspronkelijk bericht-----
> > Van: Detlev Zundel [mailto:dzu at denx.de] 
> > Verzonden: vrijdag 17 januari 2014 15:59
> > Aan: Ruud Commandeur
> > CC: U-Boot list
> > Onderwerp: Re: [U-Boot] Fat write problem
> > 
> > Hi Ruud,
> > 
> > > This week I decided to do some further research and testing 
> > regarding
> > > this problem.
> > > With the image I had from the previous time, I could immediately
> > > reproduce it and
> > > by adding more and more debug prints, I tried to find the 
> > cause. Sofar,
> > > I have not
> > > succeeded in this yet.
> > >
> > > However: later on I started testing with a freshly 
> > formatted drive (32
> > > MB FAT partition)
> > > and kept repeating the fatwrite command:
> > >
> > > fatwrite mmc 0:1 42000000 test-x 200
> > >
> > > where x runs from 1, 2,3 and further. And this way I could 
> > reproduce it
> > > quite easily.
> > > Writing always fails for the 32nd file. This is with the partition
> > > formatted with a 512 byte
> > > sector size and a cluster size of 4. If the cluster size is 
> > 1 (formatted
> > > by Windows),
> > > it already fails at the 8th file.
> > >
> > > If I create a subdirectory (from Linux) with already 24 
> > files in it, I
> > > can still write 29 files
> > > and it fails at number 30. Also, if earlier files were 
> > deleted from the
> > > root-directory, they
> > > still count in the total number of files here.
> > >
> > > If I take out the card where u-boot fails to write new 
> files, I can
> > > still add new files from
> > > my PC with Linux or Windows.
> > >
> > > I tested with both long and short filenames (same result), VFAT is
> > > enabled.
> > >
> > > I hope this gives you all some more information about this 
> > problem and
> > > perhaps it is even a
> > > known problem (limited number of files in the root 
> > directory?). I know
> > > it is voor FAT16, but
> > > that was 512 entries if I am correct.
> > 
> > Thanks for the extensive research into this problem.  For people to
> > help, I think the barrier of reproducing the problem is "somewhat
> > high", so it occurred to me if you can help setup a very 
> easy test for
> > people to work on.  Would you be able to generate a small 
> > image that one
> > can dd to a mmc card and then immediately provoke the error?  If you
> > don't have any hosting space, as a last resort I'd be fine 
> for you to
> > put it on our wiki [1].
> > 
> Hi Detlev,
> 
> Thanks for your response. Finally I get some ;-)
> 
> I can look for creating some image, but I don't think it is 
> that hard to
> reproduce (now I think to know what causes it). Either use 32 
> times the
> fatwrite command (using the cmd history and increment the 
> x-nr each time)
> or just copy up to 31 files to the root of the FAT partition 
> by using the host
> machine and place the SD-card back on the targe device running u-boot.
> 
> Hope this makes it easy enough, otherwise I will be glad to 
> hear from you.
> 
> N.B. All this was with a FAT16 partition. If I use FAT32, I 
> have no such
> problem! At least not for 45 files up till now...
> 
> Regards,
> 
> Ruud

Hi Detlev (and others),

Just to be absolutely sure, I did a new attempt.

Created 32 MB FAT16 (type 06) partition with fdisk on
my SD-Card (1st parttion).
Formatted as FAT16, 4 sectors per cluster:

"sudo mkfs.vfat -v -n IPCAN -F 16 -s 4 /dev/sdf1"

Created 30 files on this mounted partition:

"for i in {1..30}; do touch /media/IPCAN/test-$i; done"

Plugged the SD-card into my EVK running u-boot and
started fatwrite testing:

fatwrite mmc 0:1 42000000 test-31
writing test-31
0 bytes written
CLB IP-CAN U-Boot > fatwrite mmc 0:1 42000000 test-32
writing test-32
MMC: block number 0x1002006 exceeds max(0x746000)
error: overflow occurs
error: overflow occurs
...
...

And the error is there and will remain there for any further attempts.
So it doesn't matter how and where the files were created, and what
size the files are, adding files >= 32 jut fails for FAT16.

Note that also "test-16" is missing when doing a "fatls mmc 0:1". The
file can be read though when doing a fatload for this file. This is
probably some other problem, but could be related.

For now, I will continue using FAT32. If any problems would arise there
I will post them.

Regards,

Ruud

> 
> > Cheers
> >   Detlev
> > 
> > [1] http://www.denx.de/wiki/view/U-Boot/TooBigPatches
> > 
> > -- 
> > Golden rule #12:   When the comments do not match the code, 
> > they probably are
> > both wrong. -- Steven Rostedt 
> > <1300126962.9910.128.camel@gandalf.stny.rr.com>
> > --
> > DENX Software Engineering GmbH,      MD: Wolfgang Denk & 
> Detlev Zundel
> > HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 
> Groebenzell, Germany
> > Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: 
> dzu at denx.de
> > 

[U-Boot] Fat write problem

[Ruud Commandeur]

Hi Everyone,

After about half a year without problems for fatwrite, I have some new
problem.
The fatwrites for the u-boot are used to write the uimage and dtb file
to a 32
MB FAT16 partition on an SD-card. Since this morning, the writing of a
new file
keeps failing. I still can write files that are already present.
But if try and write some file with a name that doe not exist yet, it
fails. I guess
this just hasn't been tested for a while, since these filenames don't
change.
Meanwhile some extra files have been copied to the partition from the
Linux
platform. I did check the SD-card partition with fsck and could not find
any
problems.

I did enable the debug for MMC and FAT to get the following output:

fatwrite mmc 0:1 42000000 test-file 200
writing test-file
MMC0: CMD16
MMC0: CMD17
MMC0: CMD16
MMC0: CMD18
MMC0: CMD12
get_dentfromdir: test-file
gc - clustnum: -6, startsect: 132
MMC0: CMD16
MMC0: CMD18
MMC0: CMD12
vfatname: |uimage|
Mismatch: |uimage|uimage|
vfatname: |imx28-evk.dtb|
Mismatch: |imx28-~1.dtb|imx28-evk.dtb|
vfatname: |linux-file1.txt|
Mismatch: |linux-~1.txt|linux-file1.txt|
Mismatch: |address.ini||
vfatname: |imx28-clb.dtb|
Mismatch: |imx28-~2.dtb|imx28-clb.dtb|
vfatname: |img-copy|
Mismatch: |img-copy|img-copy|
vfatname: |tessie|
Mismatch: |tessie|tessie|
vfatname: |img-copy2|
Mismatch: |img-co~1|img-copy2|
vfatname: |imx-28-clb-37.dtb|
Mismatch: |imx-28~1.dtb|imx-28-clb-37.dtb|
vfatname: |hello8000.wav|
Mismatch: |hello8~1.wav|hello8000.wav|
vfatname: |versions.ini|
Mismatch: |versions.ini|versions.ini|
vfatname: |mx28-310.dtb|
Mismatch: |mx28-310.dtb|mx28-310.dtb|
FAT16: entry: 0xfffffffa = -6, offset: 0x03fa = 1018
MMC: block number 0x1000806 exceeds max(0x762c00)
FAT16: ret: 00000000, entry: fffffffa, offset: 03fa
curclust: 0x0
Invalid FAT entry
name.ext : <NULL>.
FAT16: entry: 0x0003 = 3, offset: 0x0003 = 3
error: overflow occurs
error: writing FAT blocks
FAT16: entry: 0x0004 = 4, offset: 0x0004 = 4
error: overflow occurs
error: writing FAT blocks
FAT16: entry: 0x0005 = 5, offset: 0x0005 = 5
error: overflow occurs
error: writing FAT blocks
FAT16: entry: 0x0006 = 6, offset: 0x0006 = 6
... and this goes on for a long long time...

It points to get_fatent_value( ) where things go wrong. Or perhaps the
call of
this function with the value 0xfffffffa (-6). I don't know if that is
supposed
to be possible?

At the moment I am a bit stuck here, so if anyone would have an idea
what
goes wrong here, please let me know.

Thanks,

Ruud

N.B. It might be that this can be solved by clearing the FAT partition
and starting
with a clean sheet. But I prefer not to lose this error situaion before
I know
what goes wrong here.