From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from gateway-1237.mvista.com ([63.81.120.158]) by bombadil.infradead.org with esmtp (Exim 4.68 #1 (Red Hat Linux)) id 1Jk0OI-0001kn-8v for linux-mtd@lists.infradead.org; Thu, 10 Apr 2008 17:16:55 +0000 From: Jean Pihet To: linux-mtd@lists.infradead.org Subject: Re: [BUG] JFFS2 usage of write_begin and write_end functions causes kernel panic Date: Thu, 10 Apr 2008 19:18:21 +0200 References: In-Reply-To: MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_dvk/HoZ99O+y2Uq" Message-Id: <200804101918.21362.jpihet@mvista.com> Cc: nickpiggin@yahoo.com.au, joern@logfs.org, dwmw2@infradead.org, Alexey Korolev , akpm@linux-foundation.org List-Id: Linux MTD discussion mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , --Boundary-00=_dvk/HoZ99O+y2Uq Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Alexey, Here is the patch (attached) I applied to get a stable JFFS2 filesystem. The changes have been cherry picked from http://sourceforge.net/projects/mtd-mods. I stressed it with intensive read/write, flash fill-up and data removal. A few GB's of data have been transferred without any problem. I didn't try the tool you mentionned though. Tested on OMAP3 platform. Now I am testing with PREEMPT_RT config. Regards, Jean. On Thursday 10 April 2008 18:53:55 Alexey Korolev wrote: > Hi all, > > We faced JFFS2 kernel panic issues on Linux 2.6.24. > Bug is easy reproducible if to perform random file system operations > (esepcially trunkate). We have made a trivial tool which reproduces the > problem on JFFS2 partiton. Usually less than 10 min is needed to catch a > panic. > > The problem is related to introduction of write_begin and write_end > functions instead of original prepare_write & commit_write. The kernel > panic has disappeared when we rolled back write_begin and write_end changes > in JFFS2. We tried to fix it - but it seems problem is bit tough for us. I > would be much appreciate if someone would fix the problem or suggest where > the problem is. We made several attempts to identify the source but still > have not idea about what exactly is the source of the bug. Each time panic > messages are different. > > Here is the log of one of kernel panic messages: > > 4/103: write f0 [915918,86035] 28 > 4/104: unlink d3/d6/df/f16 0 > 4/106: getdents d3 0 > 4/107: write d3/d6/fd [1352490,60815] 28 > 10/209: truncate d5/f9 1534232 28 > 10/210Unable to handle kernel NULL pointer dereference at virtual > address 000000 > 00 > > : chown d5/df/d12/d1e 156 28 > > 10/211: rename d5/df/d10/d15/f18 tpgd = c310c000 > o d5/d20/f3c 28 > 10/212: mkdir d5/df/d10/d15/d28/d3d 28 > 10/213:[00000000] *pgd=a3107031 creat d5/df/d10/d15/f3e x:0 28 0 > 10/214: fsync d5/df/d10/d15/d, *pte=000000001b/f22 0 > 10/215: rename d5/df/d12/d1e to d5/df/d12/d3f 28 > 10/2, *ppte=0000000016: readlink - no filename > 10/217: write d5/df/f1c [484960,1048 > 83] 28 > 10/218: readlink - no filename > 87/134: mkdir d3/d8/d32 Internal error: Oops: 817 [#7] > Modules linked in: > CPU: 0 Tainted: G D (2.6.24.2-pxa27x #18) > PC is at shrink_dcache_parent+0x6c/0x108 > LR is at __init_begin+0x3fff8000/0x34 > pc : [] lr : [<00000000>] psr: 00000013 > sp : c310becc ip : c3b00324 fp : c310bef4 > r10: c3b00358 r9 : 00000000 r8 : c021403c > r7 : c0214010 r6 : c310a000 r5 : c383781c r4 : 0000000b > r3 : 00000000 r2 : 00000000 r1 : c3b00348 r0 : c383781c > Flags: nzcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user > Control: 0000397f Table: a310c000 DAC: 00000015 > Process fsstress (pid: 825, stack limit = 0xc310a268) > Stack: (0xc310becc to 0xc310c000) > bec0: c383781c 00000000 c3838098 c310bf28 > c001e004 bee0: c310a000 0000f510 c310bf08 c310bef8 c0082d80 c008bd5c > c383781c c310bf24 bf00: c310bf0c c00830f4 c0082d3c c383781c 00000000 > c3fba000 c310bf94 c310bf28 bf20: c0084ff0 c00830b0 c3837e68 c3c0d120 > 00031837 00000002 c3fba000 00000010 bf40: 00000000 00000000 00000006 > c001e004 c310bf6c c310bf60 c007c030 c007bd38 bf60: c310bf88 c310bf70 > c0078f58 c007c008 00000000 c30847a0 0000000a bed68de4 bf80: 0000f484 > 00000028 c310bfa4 c310bf98 c0085080 c0084f50 00000000 c310bfa8 bfa0: > c001de60 c0085074 0000000a bed68de4 400006b0 bed689dc bed689d4 bed689dc > bfc0: 0000000a bed68de4 0000f484 00000000 000081cc 00000000 0000f510 > bed689c4 bfe0: bed689c8 bed685a4 0000aaa0 0001ffc4 20000010 400006b0 > 00000000 00000000 Backtrace: > [] (shrink_dcache_parent+0x0/0x108) from [] > (dentry_unhash+0 > x50/0x9c) > [] (dentry_unhash+0x0/0x9c) from [] > (vfs_rmdir+0x50/0xbc) > r4:c383781c > [] (vfs_rmdir+0x0/0xbc) from [] (do_rmdir+0xac/0xf0) > r6:c3fba000 r5:00000000 r4:c383781c > [] (do_rmdir+0x0/0xf0) from [] (sys_rmdir+0x18/0x1c) > r7:00000028 r6:0000f484 r5:bed68de4 r4:0000000a > [] (sys_rmdir+0x0/0x1c) from [] > (ret_fast_syscall+0x0/0x2c) > Code: e2433001 e5873004 e5912004 e59c3024 (e5823000) > 28 > > > Here is a code of trivial tool which reproduces the issue very fast: > > /*Code is based on fsstress GPL program*/ > #define _LARGEFILE64_SOURCE > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > #include > > #define MAX_FILE_NAME_LEN 256 > #define FILE_MASK "file" > #define MNT_POINT "/mnt/mtd8/" > #define BUF_SIZE (128*1024) > > char rnd_buf[BUF_SIZE]; > int files_created; > char mnt_pnt[256]; > > enum ops > { > OPS_CREAT, > OPS_WRITE, > OPS_TRUNC, > OPS_TOTAL > }; > > void creat_f() > { > int fd; > char fname[MAX_FILE_NAME_LEN]; > sprintf(fname,"%s%s.%d",mnt_pnt,FILE_MASK,files_created); > printf("creat fname=%s\n",fname); > fd=creat(fname,0666); > if(!fd) > return; > close(fd); > files_created++; > } > > void write_f(int file_num) > { > printf("write file_num:%d\n",file_num); > struct stat64 stb; > int fd; > __int64_t lr = ((__int64_t)random() << 32) + random(); > off64_t writeofft=0; > long writelen=0; > char fname[MAX_FILE_NAME_LEN]; > sprintf(fname,"%s%s.%d",mnt_pnt,FILE_MASK,file_num); > > fd=open(fname,O_WRONLY); > if(!fd) > return; > if(fstat64(fd,&stb)<0) > { > close(fd); > return; > } > printf("stb.st_size=%lld\n",stb.st_size); > if(stb.st_size!=0) > writeofft=(off64_t)(lr%(stb.st_size + BUF_SIZE*4)); > writelen=random()%BUF_SIZE; > if(writeofft!=lseek64(fd,writeofft,SEEK_SET)) > { > printf("lseek64 failed with errno %d > %s\n",errno,strerror(errno)); > return; > } > memset(rnd_buf,(file_num%127)&0xFF,BUF_SIZE); > printf("fname=%s writeofft=%lld > writelen=%ld\n",fname,writeofft,writelen); > write(fd,rnd_buf,writelen); > printf("after write\n"); > close(fd); > } > > void trunc_f(int file_num) > { > printf("trunc file_num:%d\n",file_num); > int fd; > struct stat64 stb; > stb.st_size=0; > __int64_t lr = ((__int64_t)random() << 32) + random(); > off64_t truncofft=0; > char fname[MAX_FILE_NAME_LEN]; > sprintf(fname,"%s%s.%d",mnt_pnt,FILE_MASK,file_num); > fd=open(fname,O_RDONLY); > if(!fd) > return; > if(fstat64(fd,&stb)<0) > { > close(fd); > return; > } > close(fd); > > truncofft=(off64_t)(lr%(stb.st_size + BUF_SIZE*4)); > printf("fname=%s truncofft=%lld\n",fname,truncofft); > truncate64(fname,truncofft); > } > > int main(int argc,char** argv) > { > int op_num; > int file_num; > files_created=0; > > if(argc != 2 || (argc==2 && (strcmp(argv[1],"--help")==0 || > strcmp(argv[1],"-h")==0))) > { > printf("Using: rndops dir\n"); > return 0; > } > else > { > strcpy(mnt_pnt,argv[1]); > } > > if(mnt_pnt[strlen(mnt_pnt)-1]!='/') > strcat(mnt_pnt,"/"); > > creat_f(); > > while(1) > { > op_num=random()%OPS_TOTAL; > file_num=random()%files_created; > switch(op_num) > { > case OPS_CREAT: > creat_f(); > break; > case OPS_WRITE: > write_f(file_num); > break; > case OPS_TRUNC: > trunc_f(file_num); > break; > } > } > > return 0; > } > > Thanks, > Alexey > > ______________________________________________________ > Linux MTD discussion mailing list > http://lists.infradead.org/mailman/listinfo/linux-mtd/ --Boundary-00=_dvk/HoZ99O+y2Uq Content-Type: text/x-diff; charset="iso-8859-1"; name="0001-mtd-jffs2-Fix-non-contiguous-write-BUG-warning-abo.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="0001-mtd-jffs2-Fix-non-contiguous-write-BUG-warning-abo.patch" =46rom 04574ba1c6233a09578988d565a4cc3f2e27f723 Mon Sep 17 00:00:00 2001 =46rom: Jean Pihet Date: Wed, 9 Apr 2008 11:04:32 +0200 Subject: [PATCH] mtd jffs2: Fix non contiguous write BUG, warning about 'le= n too short' Merged 2 fixes from mtd-jffs2-stable-linux-2.6.23 at http://sourceforge.net= /projects/mtd-mods: =2D non contiguous write BUG =2D false warning about 'len too short' Cf. http://sourceforge.net/project/showfiles.php?group_id=3D207166&package_= id=3D261645 for archive download Tested with intensive (2+ GB data) simultaneous read/write, flash fill-up, = data removal cycles Signed-off-by: Jean Pihet =2D-- fs/jffs2/readinode.c | 2 +- fs/jffs2/wbuf.c | 4 ++++ 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/fs/jffs2/readinode.c b/fs/jffs2/readinode.c index 2eae5d2..20efdda 100644 =2D-- a/fs/jffs2/readinode.c +++ b/fs/jffs2/readinode.c @@ -63,7 +63,7 @@ static int check_node_data(struct jffs2_sb_info *c, struc= t jffs2_tmp_dnode_info * adding and jffs2_flash_read_end() interface. */ if (c->mtd->point) { err =3D c->mtd->point(c->mtd, ofs, len, &retlen, &buffer); =2D if (!err && retlen < tn->csize) { + if (!err && retlen < len) { JFFS2_WARNING("MTD point returned len too short: %zu instead of %u.\n",= retlen, tn->csize); c->mtd->unpoint(c->mtd, buffer, ofs, retlen); } else if (err) diff --git a/fs/jffs2/wbuf.c b/fs/jffs2/wbuf.c index d1d4f27..e023511 100644 =2D-- a/fs/jffs2/wbuf.c +++ b/fs/jffs2/wbuf.c @@ -888,6 +888,10 @@ int jffs2_flash_writev(struct jffs2_sb_info *c, const = struct kvec *invecs, donelen +=3D wbuf_retlen; } =20 + /* adjust write buffer offset, else we get a non contiguous write bug */ + if (!(c->wbuf_ofs % c->sector_size) && !c->wbuf_len) + c->wbuf_ofs =3D 0xffffffff; + /* * If there's a remainder in the wbuf and it's a non-GC write, * remember that the wbuf affects this ino =2D-=20 1.5.4.1.159.gc4a6c --Boundary-00=_dvk/HoZ99O+y2Uq--