infinite loop in jffs2_flush_wbuf_gc()?

infinite loop in jffs2_flush_wbuf_gc()?

[Rick Bronson]

Hi,

  I seem to encounter an infinite loop in in jffs2_flush_wbuf_gc() in
wbuf.c starting at: 

	} else while (old_wbuf_len &&
		      old_wbuf_ofs == c->wbuf_ofs) {

  Once it gets in this loop, I don't see how it ever gets out if
c->wbuf_ofs doesn't change.

  Thanks so much for any help.

  Rick Bronson

Re: infinite loop in jffs2_flush_wbuf_gc()?

[Artem B. Bityuckiy]

Rick Bronson wrote:
> Hi,
> 
>   I seem to encounter an infinite loop in in jffs2_flush_wbuf_gc() in
> wbuf.c starting at: 
> 
Rick,

this is the second time you're reporting this and this is the second 
time I inquire the version of your MTD :-) Is it the last MTD?

http://lists.infradead.org/pipermail/linux-mtd/2005-July/012948.html

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: infinite loop in jffs2_flush_wbuf_gc()?

[Rick Bronson]

Dear Artem,

  Thanks very much for the response.

  Sorry, I never got your previous email answering my original post.
Nor did I see it in my saved archive of the mtd mailing list.  Hmmm.

  Anyway, I got the latest CVS a few weeks ago but then had to do:

cvs update -dP -D 2005-03-04

  because I'm using 2.4.27 on a ARM.

  I noticed that the code in wbuf.c is pretty much the same in the
latest CVS as in the 2005-03-04 version so I thought someone might
have stumbled upon this issue.

> you didn't write the version of Linux/MTD you're using. It's important 
> because I fixed one endless GC thread loop problem recently. Please, try 
> a fresh MTD snapshot (download it from CVS). If it won't help - come back.

  Maybe you could tell me roughly the fix, I'd really appreciate that.

  Thanks again.

  Rick
  

> Rick Bronson wrote:
> > Hi,
> > 
> >   I seem to encounter an infinite loop in in jffs2_flush_wbuf_gc() in
> > wbuf.c starting at: 
> > 
> Rick,
> 
> this is the second time you're reporting this and this is the second 
> time I inquire the version of your MTD :-) Is it the last MTD?
> 
> http://lists.infradead.org/pipermail/linux-mtd/2005-July/012948.html
> 
> -- 
> Best Regards,
> Artem B. Bityuckiy,
> St.-Petersburg, Russia.
> 

Re: infinite loop in jffs2_flush_wbuf_gc()?

[David Woodhouse]

On Fri, 2005-07-08 at 09:26 -0700, Rick Bronson wrote:
>   I seem to encounter an infinite loop in in jffs2_flush_wbuf_gc() in
> wbuf.c starting at: 
> 
>         } else while (old_wbuf_len &&
>                       old_wbuf_ofs == c->wbuf_ofs) {
> 
>   Once it gets in this loop, I don't see how it ever gets out if
> c->wbuf_ofs doesn't change.

If we call the garbage collector and it doesn't make progress, then it
will return an error. In the error case we break out of the loop.

If the garbage collector _does_ make progress, then (as soon as it's
finished CRC-checking inodes) c->wbuf_ofs will be changing.

-- 
dwmw2

Re: infinite loop in jffs2_flush_wbuf_gc()?

[Rick Bronson]

David,

  In a netshell, here is the test case:

  Using 20050712 cvs mtd on an Atmel AT91RM9200 running linux-2.6.9
with a NAND part and doing:

1. Erase a jffs2 NAND partition, then mount it.
2. Make 5000 170 byte files named test0000.bin through test4999.bin in
   the partition.
3. run "ls -Rl" on the partition mount point.
4. remove all files in the partition via "rm -rf"

  A few seconds after the command line prompt appears the gc code goes
  into a 12 minute temporary "hang" loop in jffs2_flush_wbuf_gc() in
wbuf.c starting at: 

	} else while (old_wbuf_len &&
		      old_wbuf_ofs == c->wbuf_ofs) {

  Rick Bronson

> Hm. 12 minutes is a fairly significant problem. Can you report your test
> case to the mailing list?
> 
> -- 
> dwmw2
> 
> 

Re: infinite loop in jffs2_flush_wbuf_gc()?

[David Woodhouse]

On Thu, 2005-07-14 at 07:11 -0700, Rick Bronson wrote:
>   A few seconds after the command line prompt appears the gc code goes
>   into a 12 minute temporary "hang" loop in jffs2_flush_wbuf_gc() in
> wbuf.c starting at: 

OK. So it _is_ making progress, but just very slowly. It _is_ garbage
collecting, but it's not having to write out new nodes, because it's
mostly just erasing.

Can you build with CONFIG_JFFS2_FS_DEBUG=1 and show the output while
it's doing this? It shouldn't be taking that long -- I wonder what else
is going on.

-- 
dwmw2

Re: infinite loop in jffs2_flush_wbuf_gc()?

[Artem B. Bityuckiy]

Rick,

sorry for long delay in my reply, I've forgotten about you :-(

'cvs log gc.c' has the following entry:
----------------------------
revision 1.146
date: 2005/03/20 17:45:25;  author: dedekind;  state: Exp;  lines: +7 -1
Make sure the erase_pending_wbuf_list's blocks are taken into account
when picking the block to GC.
----------------------------

This was the fix of the endless GC bug. Probably this is what you faced. 
May be there was one more modified file, don't remember, probably 
wbuf.c. You may find this out if you may use cvs. Try to make the same 
changes.

AFAIR, the bug was easily reproduced if you unlink a file then fsync it. 
I've even found the test for you (don't mind - it isn't nice):

#include <sys/vfs.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

#define JFFS2_SUPER_MAGIC 0x72b6

#define FNAME "tmp"

void print_jffs_stat(char *path, struct statfs *fsinfo){
     if (statfs(path, fsinfo) == -1){
	perror("statfs() failed"); exit(1);
     }
     if (fsinfo->f_type != JFFS2_SUPER_MAGIC){
	fprintf(stderr, "%s is not jffs2 file system\n", path); exit(1);
     }
     printf("--- JFFS2 stat ---\n");
     printf("f_bsize   = %d\n", fsinfo->f_bsize);
     printf("f_blocks  = %d\n", fsinfo->f_blocks);
     printf("f_bfree   = %d\n", fsinfo->f_bfree);
     printf("f_bavail  = %d\n", fsinfo->f_bavail);
     printf("f_files   = %d\n", fsinfo->f_files);
     printf("f_ffree   = %d\n", fsinfo->f_ffree);
     printf("f_fsid    = %d\n", fsinfo->f_fsid);
     printf("f_namelen = %d\n", fsinfo->f_namelen);
}

int main(int argc, char *argv[]){
     if (argc < 1){
	fprintf(stderr, "%s jffs_root\n", argv[0]); return 1;
     }
     if (chdir(argv[1]) == -1){
	perror("chdir to jffs2 root failed"); return 1;
     }

     struct statfs fsinfo;
     struct stat   fdinfo;

     /* get initial jffs2 stat */
     print_jffs_stat(argv[1], &fsinfo);

     int		i, fd, cnt;
     off_t	pos;
     char	buf[fsinfo.f_bsize];

     printf("\ncreate file filled by zero with size more then flash\n");
     printf("it should fit flash since it maybe compressed perfectly\n");
     bzero(buf, fsinfo.f_bsize);

     if ((fd = open(FNAME, O_WRONLY | O_CREAT, 0664)) == -1){
	perror("can't open temporary file"); return 1;
     }
     unlink(FNAME);

     for(i = 0; i < fsinfo.f_blocks ; i++){
	cnt = write(fd, buf, fsinfo.f_bsize);
	if (cnt != fsinfo.f_bsize){
	    fprintf(stderr, "write to tmp file failed\n"); return 1;
	}
     }

     printf("\nfill the file with random data. Now in should not fit on 
flash\n");
     printf("so the error 'No space left on device' should appear\n");
     srand(1); for(i = 0; i < fsinfo.f_bsize; i++) buf[i] = rand();

     pos = lseek(fd, 0, SEEK_SET);
     if (pos == (off_t)-1){
	perror("lseek failed"); return 1;
     }

     for(i = 0; i < fsinfo.f_blocks ; i++){
	cnt = write(fd, buf, fsinfo.f_bsize);
	if (cnt != fsinfo.f_bsize) break;
     }
     if ((cnt != -1)||(errno != ENOSPC)){
	perror("Warning!!! Expected ENOSPC:"); return 1;
     }
     fsync(fd);
     printf("Ok. ENOSPC occured. Thats fine!\n");

     close(fd);
}

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: infinite loop in jffs2_flush_wbuf_gc()?

[Artem B. Bityuckiy]

Oh, there was something nasty with indentation, here it is again.

/*
  * JFFS2 file expansion test.
  */

#include <sys/vfs.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>

#define JFFS2_SUPER_MAGIC 0x72b6

#define FNAME "tmp"

void print_jffs_stat(char *path, struct statfs *fsinfo){
     if (statfs(path, fsinfo) == -1){
	perror("statfs() failed"); exit(1);
     }
     if (fsinfo->f_type != JFFS2_SUPER_MAGIC){
	fprintf(stderr, "%s is not jffs2 file system\n", path); exit(1);
     }
     printf("--- JFFS2 stat ---\n");
     printf("f_bsize   = %d\n", fsinfo->f_bsize);
     printf("f_blocks  = %d\n", fsinfo->f_blocks);
     printf("f_bfree   = %d\n", fsinfo->f_bfree);
     printf("f_bavail  = %d\n", fsinfo->f_bavail);
     printf("f_files   = %d\n", fsinfo->f_files);
     printf("f_ffree   = %d\n", fsinfo->f_ffree);
     printf("f_fsid    = %d\n", fsinfo->f_fsid);
     printf("f_namelen = %d\n", fsinfo->f_namelen);
}

int main(int argc, char *argv[]){
     if (argc < 1){
	fprintf(stderr, "%s jffs_root\n", argv[0]); return 1;
     }
     if (chdir(argv[1]) == -1){
	perror("chdir to jffs2 root failed"); return 1;
     }

     struct statfs fsinfo;
     struct stat   fdinfo;

     /* get initial jffs2 stat */
     print_jffs_stat(argv[1], &fsinfo);

     int		i, fd, cnt;
     off_t	pos;
     char	buf[fsinfo.f_bsize];

     printf("\ncreate file filled by zero with size more then flash\n");
     printf("it should fit flash since it maybe compressed perfectly\n");
     bzero(buf, fsinfo.f_bsize);

     if ((fd = open(FNAME, O_WRONLY | O_CREAT, 0664)) == -1){
	perror("can't open temporary file"); return 1;
     }
     unlink(FNAME);

     for(i = 0; i < fsinfo.f_blocks ; i++){
	cnt = write(fd, buf, fsinfo.f_bsize);
	if (cnt != fsinfo.f_bsize){
	    fprintf(stderr, "write to tmp file failed\n"); return 1;
	}
     }

     printf("\nfill the file with random data. Now in should not fit on 
flash\n");
     printf("so the error 'No space left on device' should appear\n");
     srand(1); for(i = 0; i < fsinfo.f_bsize; i++) buf[i] = rand();

     pos = lseek(fd, 0, SEEK_SET);
     if (pos == (off_t)-1){
	perror("lseek failed"); return 1;
     }

     for(i = 0; i < fsinfo.f_blocks ; i++){
	cnt = write(fd, buf, fsinfo.f_bsize);
	if (cnt != fsinfo.f_bsize) break;
     }
     if ((cnt != -1)||(errno != ENOSPC)){
	perror("Warning!!! Expected ENOSPC:"); return 1;
     }
     fsync(fd);
     printf("Ok. ENOSPC occured. Thats fine!\n");

     close(fd);
}


-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.