Re: UBIFS Corruption

[Artem Bityutskiy <dedekind1@gmail.com>]

On Fri, 2011-07-08 at 06:12 -0700, Reginald Perrin wrote:
> Hi folks,
> 
> We're using ubifs in an embedded uclinux system (based on ADI Blackfin's BF524). 
>  Been working great for us for a while.  Kernel is 2.6.34.7 (uclinux).
> 
> However, we just saw 2 corruptions within the past 48h that we can't explain. 
>  We've been doing the same basic operation (in terms of flashing/reading/writing 
> images) for quite some time, and have reflashed our units many times (over 
> thousands of different hardware units).
> 
> Device #1 failure:
> * Device was running out of a partition mounted to /home (a 93MB partition from 
> a 128MB NAND device)
> * Our app was running normally and locked up (not sure why).  Our code may have 
> been updating a sqlite database located in that partition
> * When we power cycled, the partition had the corruption issue noted.
> 
> Device #1 boot log:
> UBI device number 1, total 750 LEBs (96768000 bytes, 92.3 MiB), available 0 LEBs 
> (0 bytes), LEB size 129024 bytes (126.0 KiB) 
> [ 5.228000] UBIFS: recovery needed 
> [ 5.320000] UBIFS error (pid 363): ubifs_scanned_corruption: corruption at LEB 
> 172:45056 
> [ 5.348000] UBIFS error (pid 363): ubifs_recover_leb: LEB 172 scanning failed 
> mount: mounting ubi1:home on /home failed: Structure needs cleaning 

While this is very typical symptom which might mean many things,

> Device #2 failure:
> * Device was running normally.
> * We upgraded our application (which involved updating executables on that 
> partition)
> * After the successful upgrade, we powered the unit down and stored 
> * Days later, powered up the device and the above invalid CRC as noted
> 
> Device #2 boot log:
> UBI device number 1, total 750 LEBs (96768000 bytes, 92.3 MiB), available 0 LEBs 
> (0 bytes), LEB size 129024 bytes (126.0 KiB) 
> [ 5.488000] UBIFS: recovery needed 
> [ 5.492000] UBIFS error (pid 365): check_lpt_crc: invalid crc in LPT node: crc 
> a0 calc 9013
> mount: mounting ubi1:home on /home failed: Invalid argument

I do not remember this one. This may be an UBIFS bug of course, but
quite possible this is just flash corruption or memory corruption (e.g.,
some driver corrupts memory and makes UBIFS write incorrect information
to the flash).

-- 
Best Regards,
Artem Bityutskiy