JFFS2 mount fail & crash on 2.4.26 kernel

JFFS2 mount fail & crash on 2.4.26 kernel

[Amol Lad]

Hi,

Our embedded device is successfully running JFFS2 (on NOR) for past
3-4 years now. Recently we are seeing mount fail & crashes. This is
observed when fs is around 90% full. The below dump shows the error we
get during mount in startup. Any ideas what could be going wrong. Is
there a way I can recover from this error ? (or any utility aka fsck).

(We've patched jffs2 in 2.4.26 from mtd cvs snapshot dated
11/20/2004). I'm in the process of decoding the oops and will post as
soon as I'm done...

Please CC me..

# d nlink for child "compare_oids" (ino #228)
<7>Increased nlink for child "crossconnect" (ino #137)
<7>Increased nlink for child "snmp_exclude" (ino #227)
<7>Increased nlink for child "alarm_history" (ino #490)
<7>Pass 1: ino #648
<7>Pass 1: ino #137
<7>Pass 1: ino #649
<7>Pass 1: ino #10
<7>Pass 1: ino #11
<7>Pass 1: ino #12
<7>Pass 1: ino #13
<7>Pass 1: ino #14
<7>Pass 1: ino #15
<7>jffs2_build_inode building directory inode #15
<7>Increased nlink for child "XE1400TXRX.ABS" (ino #247)
<7>Increased nlink for child "XE1500RX.ABS" (ino #245)
<7>Increased nlink for child "XE1500TX.ABS" (ino #246)
<7>Pass 1: ino #16
<7>Pass 1: ino #17
<7>Pass 1: ino #913
<7>Pass 1: ino #18
<7>Pass 1: ino #914
<7>Pass 1: ino #19
<7>Pass 1: ino #915
<7>jffs2_build_inode building directory inode #915
<7>Increased nlink for child "P_0M125_004_02_01.txt" (ino #936)
<7>Increased nlink for child "P_0M125_004_02_02.txt" (ino #937)
<7>Increased nlink for child "P_14M000_064_02_02.txt" (ino #979)
<7>Increased nlink for child "P_0M075_016_02_02.txt" (ino #927)
<7>Increased nlink for child "P_0M025_128_02_02.txt" (ino #920)
<7>Increased nlink for child "P_1M350_016_02_01.txt" (ino #992)
<7>Increased nlink for child "P_1M350_016_02_02.txt" (ino #993)
<7>Increased nlink for child "P_0M075_064_02_02.txt" (ino #929)
<7>Increased nlink for child "P_3M500_032_02_01.txt" (ino #1012)
<7>Increased nlink for child "P_3M500_032_02_02.txt" (ino #1013)
<7>Increased nlink for child "P_1M750_128_02_02.txt" (ino #1007)
<7>Increased nlink for child "P_1M350_064_02_01.txt" (ino #996)
<7>Increased nlink for child "P_1M350_064_02_02.txt" (ino #997)
<7>Increased nlink for child "P_7M000_016_02_02.txt" (ino #1018)
<7>Increased nlink for child "P_1M000_128_02_02.txt" (ino #989)
<7>Increased nlink for child "P_0M100_016_02_02.txt" (ino #932)
<7>Increased nlink for child "P_3M500_004_02_01.txt" (ino #1008)
<7>Increased nlink for child "P_3M500_004_02_02.txt" (ino #1009)
<7>Increased nlink for child "P_7M000_064_02_02.txt" (ino #1020)
<7>Increased nlink for child "P_0M150_032_02_02.txt" (ino #946)
<7>Increased nlink for child "P_0M500_128_02_02.txt" (ino #975)
<7>Increased nlink for child "P_0M250_016_02_01.txt" (ino #960)
<7>Increased nlink for child "P_0M250_016_02_02.txt" (ino #961)
<7>Increased nlink for child "P_0M100_064_02_02.txt" (ino #934)
eth0: config: auto-negotiation on, 10FDX.
<7>Increased nlink for child "P_0M200_128_02_02.txt" (ino #957)
<7>Increased nlink for child "P_0M250_064_02_01.txt" (ino #964)
<7>Increased nlink for child "P_0M250_064_02_02.txt" (ino #965)
<7>Increased nlink for child "P_0M150_004_02_02.txt" (ino #944)
<7>Increased nlink for child "P_0M025_032_02_02.txt" (ino #918)
<7>Increased nlink for child "P_0M050_128_02_02.txt" (ino #925)
<7>Increased nlink for child "P_0M125_016_02_01.txt" (ino #938)
<7>Increased nlink for child "P_0M125_016_02_02.txt" (ino #939)
<7>Increased nlink for child "P_1M750_032_02_01.txt" (ino #1003)
<7>Increased nlink for child "P_1M750_032_02_02.txt" (ino #1004)
<7>Increased nlink for child "P_0M125_064_02_02.txt" (ino #942)
<7>Increased nlink for child "P_0M025_004_02_02.txt" (ino #916)
<7>Increased nlink for child "P_1M000_032_02_01.txt" (ino #985)
<7>Increased nlink for child "P_1M000_032_02_02.txt" (ino #986)
<7>Increased nlink for child "P_14M000_128_02_02.txt" (ino #980)
<7>Increased nlink for child "P_1M750_004_02_01.txt" (ino #999)
<7>Increased nlink for child "P_1M750_004_02_02.txt" (ino #1000)
<7>Increased nlink for child "P_0M500_032_02_01.txt" (ino #971)
<7>Increased nlink for child "P_0M500_032_02_02.txt" (ino #972)
<7>Increased nlink for child "P_1M000_004_02_01.txt" (ino #981)
<7>Increased nlink for child "P_1M000_004_02_02.txt" (ino #982)
<7>Increased nlink for child "P_0M200_032_02_01.txt" (ino #953)
<7>Increased nlink for child "P_0M200_032_02_02.txt" (ino #954)
<7>Increased nlink for child "P_0M075_128_02_02.txt" (ino #930)
<7>Increased nlink for child "P_3M500_016_02_01.txt" (ino #1010)
<7>Increased nlink for child "P_3M500_016_02_02.txt" (ino #1011)
<7>Increased nlink for child "P_0M500_004_02_01.txt" (ino #967)
<7>Increased nlink for child "P_0M500_004_02_02.txt" (ino #968)
<7>Increased nlink for child "P_1M350_128_02_02.txt" (ino #998)
<7>Increased nlink for child "P_0M200_004_02_01.txt" (ino #949)
<7>Increased nlink for child "P_0M200_004_02_02.txt" (ino #950)
<7>Increased nlink for child "P_3M500_064_02_01.txt" (ino #1014)
<7>Increased nlink for child "P_3M500_064_02_02.txt" (ino #1015)
<7>Increased nlink for child "P_0M050_032_02_02.txt" (ino #923)
<7>Increased nlink for child "P_0M150_016_02_02.txt" (ino #945)
<7>Increased nlink for child "P_7M000_128_02_02.txt" (ino #1021)
<7>Increased nlink for child "P_0M100_128_02_02.txt" (ino #935)
<7>Increased nlink for child "P_0M150_064_02_02.txt" (ino #947)
<7>Increased nlink for child "P_0M050_004_02_02.txt" (ino #921)
<7>Increased nlink for child "P_14M000_032_02_02.txt" (ino #978)
<7>Increased nlink for child "P_0M250_128_02_02.txt" (ino #966)
<7>Increased nlink for child "P_0M025_016_02_02.txt" (ino #917)
<7>Increased nlink for child "P_0M075_032_02_02.txt" (ino #928)
<7>Increased nlink for child "P_14M000_004_02_02.txt" (ino #976)
<7>Increased nlink for child "P_0M025_064_02_02.txt" (ino #919)
<7>Increased nlink for child "P_1M750_016_02_01.txt" (ino #1001)
<7>Increased nlink for child "P_1M750_016_02_02.txt" (ino #1002)
<7>Increased nlink for child "P_1M350_032_02_01.txt" (ino #994)
<7>Increased nlink for child "P_1M350_032_02_02.txt" (ino #995)
<7>Increased nlink for child "P_0M125_128_02_02.txt" (ino #943)
<7>Increased nlink for child "P_1M000_016_02_01.txt" (ino #983)
<7>Increased nlink for child "P_1M000_016_02_02.txt" (ino #984)
<7>Increased nlink for child "P_0M075_004_02_02.txt" (ino #926)
<7>Increased nlink for child "P_1M750_064_02_01.txt" (ino #1005)
<7>Increased nlink for child "P_1M750_064_02_02.txt" (ino #1006)
<7>Increased nlink for child "P_7M000_032_02_02.txt" (ino #1019)
<7>Increased nlink for child "P_0M500_016_02_01.txt" (ino #969)
<7>Increased nlink for child "P_0M500_016_02_02.txt" (ino #970)
<7>Increased nlink for child "P_1M000_064_02_01.txt" (ino #987)
<7>Increased nlink for child "P_1M000_064_02_02.txt" (ino #988)
<7>Increased nlink for child "P_1M350_004_02_01.txt" (ino #990)
<7>Increased nlink for child "P_1M350_004_02_02.txt" (ino #991)
<7>Increased nlink for child "P_0M100_032_02_02.txt" (ino #933)
<7>Increased nlink for child "P_0M200_016_02_01.txt" (ino #951)
<7>Increased nlink for child "P_0M200_016_02_02.txt" (ino #952)
<7>Increased nlink for child "P_0M500_064_02_01.txt" (ino #973)
<7>Increased nlink for child "P_0M500_064_02_02.txt" (ino #974)
<7>Increased nlink for child "P_0M250_032_02_01.txt" (ino #962)
<7>Increased nlink for child "P_0M250_032_02_02.txt" (ino #963)
<7>Increased nlink for child "P_7M000_004_02_02.txt" (ino #1017)
<7>Increased nlink for child "P_0M200_064_02_01.txt" (ino #955)
<7>Increased nlink for child "P_0M200_064_02_02.txt" (ino #956)
<7>Increased nlink for child "P_0M100_004_02_02.txt" (ino #931)
<7>Increased nlink for child "P_0M050_016_02_02.txt" (ino #922)
<7>Increased nlink for child "P_3M500_128_02_02.txt" (ino #1016)
<7>Increased nlink for child "P_0M250_004_02_01.txt" (ino #958)
<7>Increased nlink for child "P_0M250_004_02_02.txt" (ino #959)
<7>Increased nlink for child "P_0M050_064_02_02.txt" (ino #924)
<7>Increased nlink for child "P_0M125_032_02_01.txt" (ino #940)
<7>Increased nlink for child "P_0M125_032_02_02.txt" (ino #941)
<7>Increased nlink for child "P_0M150_128_02_02.txt" (ino #948)
<7>Increased nlink for child "P_14M000_016_02_02.txt" (ino #977)
<7>Pass 1: ino #20
<7>Pass 1: ino #916
<7>Pass 1: ino #21
<7>Pass 1: ino #917
<7>Pass 1: ino #22
<7>Pass 1: ino #918
<7>Pass 1: ino #919
<7>Pass 1: ino #920
<7>Pass 1: ino #25
<7>Pass 1: ino #921
<7>Pass 1: ino #26
<7>jffs2_build_inode building directory inode #26
<7>Increased nlink for child "ee_00000_007ff.sav" (ino #28)
<7>Increased nlink for child "ee_00800_009ff.sav" (ino #31)
<7>Increased nlink for child "ee_10000_1402f.sav" (ino #434)
<7>Increased nlink for child "ee_02d10_02e6f.sav" (ino #37)
<7>Increased nlink for child "ee_00a40_00bff.sav" (ino #29)
<7>Increased nlink for child "ee_01000_0103f.sav" (ino #34)
<7>Increased nlink for child "ee_1ff00_1ffff.sav" (ino #27)
<7>Increased nlink for child "ee_16860_1a87b.sav" (ino #432)
<7>Increased nlink for child "ee_00ee0_00fbf.sav" (ino #32)
<7>Increased nlink for child "ee_00c00_00edf.sav" (ino #35)
<7>Increased nlink for child "ee_00fc0_00fdf.sav" (ino #130)
<7>Increased nlink for child "ee_14030_1685f.sav" (ino #433)
<7>Increased nlink for child "ee_00a00_00a2f.sav" (ino #39)
<7>Increased nlink for child "ee_00fe0_00fff.sav" (ino #131)
<7>Increased nlink for child "ee_03810_0384f.sav" (ino #38)
<7>Increased nlink for child "ee_024c0_0250f.sav" (ino #36)
<7>Pass 1: ino #922
<7>Pass 1: ino #27
<7>Pass 1: ino #923
<7>Pass 1: ino #28
<7>Pass 1: ino #924
<7>Pass 1: ino #29
<7>Pass 1: ino #925
<7>Pass 1: ino #926
<7>Pass 1: ino #31
<7>Pass 1: ino #927
<7>Pass 1: ino #32
<7>Pass 1: ino #928
<7>Pass 1: ino #801
<7>Pass 1: ino #929
<7>Pass 1: ino #34
<7>Pass 1: ino #930
<7>Pass 1: ino #35
<7>Pass 1: ino #931
<7>Pass 1: ino #36
<7>Pass 1: ino #932
<7>Pass 1: ino #37
<7>Pass 1: ino #933
<7>Pass 1: ino #38
<7>Pass 1: ino #934
<7>Pass 1: ino #39
<7>Pass 1: ino #935
<7>Pass 1: ino #40
<7>Pass 1: ino #936
<7>Pass 1: ino #937
<7>Pass 1: ino #938
<7>Pass 1: ino #939
<7>Pass 1: ino #44
<7>Pass 1: ino #940
<7>Pass 1: ino #941
<7>Pass 1: ino #942
<7>Pass 1: ino #943
<7>Pass 1: ino #432
<7>Pass 1: ino #944
<7>Pass 1: ino #433
<7>Pass 1: ino #945
<7>Pass 1: ino #434
<7>Pass 1: ino #946
<7>Pass 1: ino #947
<7>Pass 1: ino #948
<7>Pass 1: ino #949
<7>Pass 1: ino #950
<7>Pass 1: ino #951
<7>Pass 1: ino #952
<7>Pass 1: ino #953
<7>Pass 1: ino #954
<7>Pass 1: ino #443
<7>Pass 1: ino #955
<7>Pass 1: ino #956
<7>Pass 1: ino #957
<7>Pass 1: ino #958
<7>Pass 1: ino #959
<7>Pass 1: ino #960
<7>Pass 1: ino #961
<7>Pass 1: ino #962
<7>Pass 1: ino #963
<7>Pass 1: ino #964
<7>Pass 1: ino #965
<7>Pass 1: ino #966
<7>Pass 1: ino #967
<7>Pass 1: ino #968
<7>Pass 1: ino #969
<7>Pass 1: ino #970
<7>Pass 1: ino #971
<7>Pass 1: ino #972
<7>Pass 1: ino #973
<7>Pass 1: ino #974
<7>Pass 1: ino #975
<7>Pass 1: ino #976
<7>Pass 1: ino #977
<7>Pass 1: ino #978
<7>Pass 1: ino #979
<7>Pass 1: ino #980
<7>Pass 1: ino #981
<7>Pass 1: ino #982
<7>Pass 1: ino #983
<7>Pass 1: ino #984
<7>Pass 1: ino #985
<7>Pass 1: ino #986
<7>Pass 1: ino #987
<7>Pass 1: ino #988
<7>Pass 1: ino #989
<7>Pass 1: ino #990
<7>Pass 1: ino #991
<7>Pass 1: ino #992
<7>Pass 1: ino #993
<7>Pass 1: ino #994
<7>Pass 1: ino #227
<7>Pass 1: ino #611
<7>Pass 1: ino #995
<7>Pass 1: ino #228
<7>Pass 1: ino #612
<7>Pass 1: ino #996
<7>Pass 1: ino #229
<7>jffs2_build_inode building directory inode #229
<7>Pass 1: ino #997
<7>Pass 1: ino #998
<7>Pass 1: ino #487
<7>Pass 1: ino #999
<7>Pass 1: ino #1000
<7>Pass 1: ino #1001
<7>Pass 1: ino #490
<7>Pass 1: ino #1002
<7>Pass 1: ino #1003
<7>Pass 1: ino #1004
<7>Pass 1: ino #1005
<7>Pass 1: ino #1006
<7>Pass 1: ino #1007
<7>Pass 1: ino #1008
<7>Pass 1: ino #1009
<7>Pass 1: ino #1010
<7>Pass 1: ino #1011
<7>Pass 1: ino #1012
<7>Pass 1: ino #245
<7>Pass 1: ino #1013
<7>Pass 1: ino #246
<7>Pass 1: ino #1014
<7>Pass 1: ino #247
<7>Pass 1: ino #1015
<7>Pass 1: ino #1016
<7>Pass 1: ino #1017
<7>Pass 1: ino #1018
<7>Pass 1: ino #1019
<7>Pass 1: ino #1020
<7>Pass 1: ino #1021
<7>Pass 1: ino #1023
<7>Pass 1 complete
<7>Pass 2 starting
<7>Pass 2: ino #1, nlink 1, ic c14282c0, nodes c1407650
<7>Pass 2: ino #2, nlink 1, ic c1428398, nodes c1407620
<7>Pass 2: ino #130, nlink 1, ic c1409278, nodes c140ad30
<7>Pass 2: ino #1026, nlink 1, ic c1428b60, nodes c1423ad0
<7>Pass 2: ino #3, nlink 1, ic c1428a88, nodes c14238e0
<7>Pass 2: ino #131, nlink 1, ic c1428d40, nodes c1420550
<7>Pass 2: ino #5, nlink 1, ic c1428338, nodes c14269e0
<7>Pass 2: ino #6, nlink 1, ic c1428350, nodes c1421f30
<7>Pass 2: ino #7, nlink 1, ic c1428368, nodes c140eda0
<7>Pass 2: ino #8, nlink 1, ic c1428380, nodes c1426a40
<7>Pass 2: ino #136, nlink 1, ic c14282d8, nodes c140ede0
<7>Pass 2: ino #648, nlink 1, ic c1428ba8, nodes c1421ab0
<7>Pass 2: ino #137, nlink 1, ic c1428de8, nodes c140df60
<7>Pass 2: ino #649, nlink 1, ic c1428bc0, nodes c1421b50
<7>Pass 2: ino #10, nlink 1, ic c14289f8, nodes c1423870
<7>Pass 2: ino #11, nlink 1, ic c1428a58, nodes c14238b0
<7>Pass 2: ino #12, nlink 1, ic c1428a28, nodes c1423890
<7>Pass 2: ino #13, nlink 1, ic c1428a40, nodes c14238a0
<7>Pass 2: ino #14, nlink 1, ic c1428a10, nodes c1423880
<7>Pass 2: ino #15, nlink 1, ic c14282f0, nodes c1426ab0
<7>Pass 2: ino #16, nlink 1, ic c14283b0, nodes c1426ad0
<7>Pass 2: ino #17, nlink 1, ic c14283c8, nodes c1426b00
<7>Pass 2: ino #913, nlink 1, ic c1428290, nodes c14077e0
<7>Pass 2: ino #18, nlink 1, ic c14283e0, nodes c1426b20
<7>Pass 2: ino #914, nlink 1, ic c1428488, nodes c1421e00
<7>Pass 2: ino #19, nlink 1, ic c14283f8, nodes c1426b40
<7>Pass 2: ino #915, nlink 1, ic c14284b8, nodes c1407410
<7>Pass 2: ino #20, nlink 1, ic c1428410, nodes c1426b60
<7>Pass 2: ino #916, nlink 1, ic c1428bf0, nodes c1421f90
<7>Pass 2: ino #21, nlink 1, ic c1428428, nodes c1426b80
<7>Pass 2: ino #917, nlink 1, ic c1428c08, nodes c1421fc0
<7>Pass 2: ino #22, nlink 1, ic c1428440, nodes c1426ba0
<7>Pass 2: ino #918, nlink 1, ic c1428c20, nodes c1421ff0
<7>Pass 2: ino #919, nlink 1, ic c1428c38, nodes c1420370
<7>Pass 2: ino #920, nlink 1, ic c1428c50, nodes c14203a0
<7>Pass 2: ino #25, nlink 1, ic c1428b48, nodes c14239d0
<7>Pass 2: ino #921, nlink 1, ic c1428c68, nodes c14203d0
<7>Pass 2: ino #26, nlink 1, ic c14282a8, nodes c140a7b0
<7>Pass 2: ino #922, nlink 1, ic c1428c80, nodes c1420400
<7>Pass 2: ino #27, nlink 1, ic c1409458, nodes c1407640
<7>Pass 2: ino #923, nlink 1, ic c1428c98, nodes c1420430
<7>Pass 2: ino #28, nlink 1, ic c1428aa0, nodes c14238f0
<7>Pass 2: ino #924, nlink 1, ic c1428cb0, nodes c1420460
<7>Pass 2: ino #29, nlink 1, ic c1428b30, nodes c14239b0
<7>Pass 2: ino #925, nlink 1, ic c1428cc8, nodes c1420490
<7>Pass 2: ino #926, nlink 1, ic c1428ce0, nodes c14204c0
<7>Pass 2: ino #31, nlink 1, ic c1428b18, nodes c14239a0
<7>Pass 2: ino #927, nlink 1, ic c1428cf8, nodes c14204f0
<7>Pass 2: ino #32, nlink 1, ic c1428b90, nodes c1423b40
<7>Pass 2: ino #928, nlink 1, ic c1428d10, nodes c1420520
<7>Pass 2: ino #801, nlink 1, ic c1428ab8, nodes c1423900
<7>Pass 2: ino #929, nlink 1, ic c1428d28, nodes c140a960
<7>Pass 2: ino #34, nlink 1, ic c1428d58, nodes c1420560
<7>Pass 2: ino #930, nlink 1, ic c1428e48, nodes c140a990
<7>Pass 2: ino #35, nlink 1, ic c1428d70, nodes c140ed30
<7>Pass 2: ino #931, nlink 1, ic c1428e60, nodes c140a9c0
<7>Pass 2: ino #36, nlink 1, ic c1428d88, nodes c140ed50
<7>Pass 2: ino #932, nlink 1, ic c1428e78, nodes c140a9f0
<7>Pass 2: ino #37, nlink 1, ic c1428b78, nodes c1423b00
<7>Pass 2: ino #933, nlink 1, ic c1428e90, nodes c140aa20
<7>Pass 2: ino #38, nlink 1, ic c1428dd0, nodes c140ee10
<7>Pass 2: ino #934, nlink 1, ic c1428ea8, nodes c140aa50
<7>Pass 2: ino #39, nlink 1, ic c1428da0, nodes c140ed90
<7>Pass 2: ino #935, nlink 1, ic c1428ec0, nodes c140aa80
<7>Pass 2: ino #40, nlink 0, ic c1428260, nodes c1407600
<7>JFFS2: Removing ino #40 with nlink == zero.
<7>obsoleting node at 0x001c4308
<7>Obsoleting previously unchecked node at 0x001c4308 of len 80: Dirtying
<7>Eraseblock at 0x001c0000 not moved anywhere. (free 0x00000000,
dirty 0x000067c4, used 0x000000a8)
<7>obliterating obsoleted node at 0x001c4308
<4>Oops: kernel access of bad area, sig: 11
<4>NIP: C0072D88 XER: 00000000 LR: C0072DA4 SP: C1439D10 REGS:
c1439c60 TRAP: 0300    Not tainted
<4>MSR: 00009032 EE: 1 PR: 0 FP: 0 ME: 1 IR/DR: 11
<4>DAR: 00000008, DSISR: 0000009F
<4>TASK = c1438000[15] 'mount' Last syscall: 21
<4>last math 00000000 last altivec 00000000
<4>GPR00: C0072DA4 C1439D10 C1438000 00000000 00001032 0000000C
C1F709C0 C1439BE4
<4>GPR08: 0000000A C1428268 C3092000 C14075F0 42002022 00000000
30005140 30005000
<4>GPR16: 00000002 000000E4 300050A0 000000E4 00009032 01439F40
00000000 C0120000
<4>GPR24: C01D29B8 C01D29B0 00000000 C1439D4C C01D28E4 C0110000
C1428260 00000000
<4>Call backtrace:
<4>C0072DA4 C0072CF8 C00731EC C00754CC C0075A14 C003AC7C C003B078
<4>C004FA90 C004FE0C C0050344 C000433C 1003D830 1003DC4C 1003E06C
<4>10003AF8 10003728 30046314 10003588

--
Real men program in C

Sent from Wellington, Wgn, New Zealand

Re: JFFS2 mount fail & crash on 2.4.26 kernel

[David Woodhouse]

On Thu, 2010-02-11 at 18:57 +1300, Amol Lad wrote:
> (We've patched jffs2 in 2.4.26 from mtd cvs snapshot dated
> 11/20/2004).

Hahaha.

April 1st isn't for another few weeks.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: JFFS2 mount fail & crash on 2.4.26 kernel

[Amol Lad]

Sorry for non giving enough information. The devices are in the field
since last 5 years and never showed problems. (This shows the
robustness of jffs2 fs). Recently only some devices are approaching
90% usage and this problem showed up....

It will be completely unfair to ask for any fix on such an old kernel.
So any ideas on recovery will help before I push RED button (i.e. call
the devices back to factory and re-flash jffs2 image..)

Do we have any tool like fsck that can be used for recovery? (I tried
mounting the device after these errors and crash, but mount hangs and
never returns. "ps" shows mount in "D" state)

On Thu, Feb 11, 2010 at 8:22 PM, David Woodhouse <dwmw2@infradead.org> wrote:
> On Thu, 2010-02-11 at 18:57 +1300, Amol Lad wrote:
>> (We've patched jffs2 in 2.4.26 from mtd cvs snapshot dated
>> 11/20/2004).
>
> Hahaha.
>
> April 1st isn't for another few weeks.
>
> --
> David Woodhouse                            Open Source Technology Centre
> David.Woodhouse@intel.com                              Intel Corporation
>
>



-- 
Real men program in C

Sent from Wellington, Wgn, New Zealand

Re: JFFS2 mount fail & crash on 2.4.26 kernel

[David Woodhouse]

On Fri, 2010-02-12 at 09:22 +1300, Amol Lad wrote:
> Sorry for non giving enough information. The devices are in the field
> since last 5 years and never showed problems. (This shows the
> robustness of jffs2 fs). Recently only some devices are approaching
> 90% usage and this problem showed up....

OK, if it's really been in the field for 5 years then perhaps you're not
_quite_ as insane as I thought you were for using a 2.4 kernel -- so
maybe I was a little harsh in my initial response. But only a little --
even by 2005, deploying a 2.4 kernel was a strange thing to do.

Did you get a proper backtrace?

> It will be completely unfair to ask for any fix on such an old kernel.
> So any ideas on recovery will help before I push RED button (i.e. call
> the devices back to factory and re-flash jffs2 image..)
> 
> Do we have any tool like fsck that can be used for recovery? (I tried
> mounting the device after these errors and crash, but mount hangs and
> never returns. "ps" shows mount in "D" state)

Does this crash every time you mount the file system? That makes it nice
and simple, in some ways.

Can you take an image and try mounting it with a 2.6.3x kernel -- I
assume it'll be fine, but if it _does_ fail, then you magically get us
to care.

And if it does work, then you get to look through the commits between
then and now and work out how we fixed it...

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: JFFS2 mount fail & crash on 2.4.26 kernel

[Jamie Lokier]

Another amazing coincidence...

My work is using JFFS2 on 2.4.26-uc0 kernels too.  The devices have
been in the field for about 5 years and are still very much in use -
we're still shipping new ones, and I'm still doing some development
work on those devices - and still write patches occasionally for those
ancient kernels.  Binary patches sometimes to fix bugs without
reflashing the kernel.  No joke :-)

Sure, 2.4.26 was old even 5 years ago, but it wasn't old when the
upstream chip supplier produced their SDK, from which the shipped SDK
was a "stable" branch presumably, and it wasn't economic to follow the
2.6 cycle, as the CPU architecture wasn't well supported in 2.6 even
1 year ago (no-MMU ARM).

Back to JFFS2 and 90% full.

In our case, we found that on a 1MB JFFS2 filesytem, with 64k erase
blocks, 90% is simply too much.  All sorts of problems ensue, like the
GC thread running for 5 minutes or more occasionally, writes sometimes
being very slow (but sometimes quick), can't predict when "disk full"
will occur.  Sometimes they linger near full for a long time without
problems, and then suddenly it's a problem.  We've had many devices
crash (without a panic message) and go into weird bootup states where
they don't boot properly and their watchdogs get stuck.  I suspect a
kernel bug is responsible for triggering the weird bootup state.

Limiting fullness to 60% appears to have solved all those problems.

Amol, the amount you can get away with will depend on erase block size
and filesystem size.

-- Jamie