Please Help, problem mounting jffs2

Please Help, problem mounting jffs2

[Frank R Callaghan]

Hi All,

I have a problem with mounting my new mtd/jffs2 system
after lots of experiment I have found the after doing
'eraseall /dev/mtd0' I can 'mount -t jffs2 /dev/mtdblock0 /mnt/flash'
with no problem :) but if I umount it and try to re-mount it
will sit there forever !
If I cp a jffs2.img to the device the mount will also lockup 
If I erase -> mount I can create files df with no problems, 

It seem like the mount does not like mounting with anything
in the filesystem !

What infomation should I gather to find out whats gonig on here ?

Any help greatly appreciated,

TIA,
        Frank.

RE: Please Help, problem mounting jffs2

[Sid Gluckman]

I am kind of a newbie to the jffs2 thing but are you sure that it is just
"sitting" there?  I found that when a jffs2 system is mounted for the first
time with something in the filesystem it goes through some kind of
cataloging function where I'm guessing it is characterizing the device for
bad flash bytes?  Anyways, if you do a 'df' in another shell while it is
sitting there you will see that the device is mounted and that the used
space first slowly cycles to fully used then slowly back down to the correct
used space.  I'm sure someone else could enlighten us both as to what it is
actually doing during this time.  Once it completes this cycle everything is
fine and it doesn't do it on subsequent mounts.  Depending on the flash size
this could take quite a while.

Hope this helps.

Sid

-----Original Message-----
From: linux-mtd-bounces@lists.infradead.org
[mailto:linux-mtd-bounces@lists.infradead.org]On Behalf Of Frank R
Callaghan
Sent: Friday, June 13, 2003 8:52 AM
To: linux-mtd@lists.infradead.org
Subject: Please Help, problem mounting jffs2


Hi All,

I have a problem with mounting my new mtd/jffs2 system
after lots of experiment I have found the after doing
'eraseall /dev/mtd0' I can 'mount -t jffs2 /dev/mtdblock0 /mnt/flash'
with no problem :) but if I umount it and try to re-mount it
will sit there forever !
If I cp a jffs2.img to the device the mount will also lockup
If I erase -> mount I can create files df with no problems,

It seem like the mount does not like mounting with anything
in the filesystem !

What infomation should I gather to find out whats gonig on here ?

Any help greatly appreciated,

TIA,
        Frank.







______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

Thanks Sid, but I don't think I'm getting that far :(

# mount -t jffs2 /dev/mtdblock0 /var/fixed_flash &
# mtdblock_open
ok

# ps ax
  PID TTY     Uid        Size State Command
    1         root        468   S   init
    2         root          0   S   [keventd]
    3         root          0   S   [ksoftirqd_CPU0]
    4         root          0   S   [kswapd]
    5         root          0   S   [bdflush]
    6         root          0   D   [kupdated]
    7         root          0   S   [mtdblockd]
   11         root          0   S   [kjournald]
   35         root        376   S   /sbin/udhcpc -n -s 
/etc/udhcpd/default.scri
   41         nobody      472   S   /sbin/boa
   46         root        492   S   /sbin/bftpd -d
   50         root          0   S   [rpciod]
   54         root          0   S   [kjournald]
   81         root       1528   R   /myapp/hiresprinthead
   83         root        656   R   /myapp/hires_controler
   85 ttyS1   root        488   S   -sh
   87 ttyS1   root        468   D   mount -t jffs2 /dev/mtdblock0 
/var/fixed_fl
   88 ttyS1   root        468   R   mount -t jffs2 /dev/mtdblock0 
/var/fixed_fl
   92 ttyS1   root        460   R   ps ax



On Friday 13 June 2003 11:47 am, Sid Gluckman wrote:
> I am kind of a newbie to the jffs2 thing but are you sure that it is just
> "sitting" there?  I found that when a jffs2 system is mounted for the first
> time with something in the filesystem it goes through some kind of
> cataloging function where I'm guessing it is characterizing the device for
> bad flash bytes?  Anyways, if you do a 'df' in another shell while it is
> sitting there you will see that the device is mounted and that the used
> space first slowly cycles to fully used then slowly back down to the
> correct used space.  I'm sure someone else could enlighten us both as to
> what it is actually doing during this time.  Once it completes this cycle
> everything is fine and it doesn't do it on subsequent mounts.  Depending on
> the flash size this could take quite a while.
>
> Hope this helps.
>
> Sid
>
> -----Original Message-----
> From: linux-mtd-bounces@lists.infradead.org
> [mailto:linux-mtd-bounces@lists.infradead.org]On Behalf Of Frank R
> Callaghan
> Sent: Friday, June 13, 2003 8:52 AM
> To: linux-mtd@lists.infradead.org
> Subject: Please Help, problem mounting jffs2
>
>
> Hi All,
>
> I have a problem with mounting my new mtd/jffs2 system
> after lots of experiment I have found the after doing
> 'eraseall /dev/mtd0' I can 'mount -t jffs2 /dev/mtdblock0 /mnt/flash'
> with no problem :) but if I umount it and try to re-mount it
> will sit there forever !
> If I cp a jffs2.img to the device the mount will also lockup
> If I erase -> mount I can create files df with no problems,
>
> It seem like the mount does not like mounting with anything
> in the filesystem !
>
> What infomation should I gather to find out whats gonig on here ?
>
> Any help greatly appreciated,
>
> TIA,
>         Frank.
>
>
>
>
>
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Please Help, problem mounting jffs2

[David Woodhouse]

On Fri, 2003-06-13 at 16:53, Frank R Callaghan wrote:
> Thanks Sid, but I don't think I'm getting that far :(
> 
> # mount -t jffs2 /dev/mtdblock0 /var/fixed_flash &
> # mtdblock_open
> ok
> 
> # ps ax

>     6         root          0   D   [kupdated]
>  
>    87 ttyS1   root        468   D   mount -t jffs2 /dev/mtdblock0 
> /var/fixed_fl
>    88 ttyS1   root        468   R   mount -t jffs2 /dev/mtdblock0 
> /var/fixed_fl

Why are there two mounts running simultaneously? What if you compile
with CONFIG_JFFS2_FS_DEBUG=1 and set the console loglevel to 9 before
mounting? Is it actually _doing_ anything or is it just deadlocked?

If it's just deadlocked (and you're not using 2.4.18) then what was the
last thing it said _before_ stopping?


-- 
dwmw2

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

On Friday 13 June 2003 12:31 pm, David Woodhouse wrote:
> On Fri, 2003-06-13 at 16:53, Frank R Callaghan wrote:
> > Thanks Sid, but I don't think I'm getting that far :(
> >
> > # mount -t jffs2 /dev/mtdblock0 /var/fixed_flash &
> > # mtdblock_open
> > ok
> >
> > # ps ax
> >
> >     6         root          0   D   [kupdated]
> >
> >    87 ttyS1   root        468   D   mount -t jffs2 /dev/mtdblock0
> > /var/fixed_fl
> >    88 ttyS1   root        468   R   mount -t jffs2 /dev/mtdblock0
> > /var/fixed_fl
>
> Why are there two mounts running simultaneously? What if you compile
> with CONFIG_JFFS2_FS_DEBUG=1 and set the console loglevel to 9 before
> mounting? Is it actually _doing_ anything or is it just deadlocked?
>
> If it's just deadlocked (and you're not using 2.4.18) then what was the
> last thing it said _before_ stopping?

jffs2_dump_block_lists:
flash_size: 00100000
used_size: 00000000
dirty_size: 00000000
wasted_size: 00000000
unchecked_size: 00000000
free_size: 00100000
erasing_size: 00000000
bad_size: 00000000
sector_size: 00010000
jffs2_reserved_blocks size: 00050000
nextblock: NULL
gcblock: NULL
clean_list: empty
very_dirty_list: empty
dirty_list: empty
erasable_list: empty
erasing_list: empty
erase_pending_list: 000f0000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 000e0000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 000d0000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 000c0000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 000b0000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 000a0000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00090000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00080000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00070000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00060000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00050000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00040000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00030000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00020000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00010000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erase_pending_list: 00000000 (used 00000000, dirty 00000000, wasted 00000000, 
un
checked 00000000, free 00010000)
erasable_pending_wbuf_list: empty
free_list: empty
bad_list: empty
bad_used_list: empty
Not rotating empty clean_list
Not rotating empty very_dirty_list
Not rotating empty dirty_list
Not rotating empty erasable_list
Rotating erase_pending_list by 0
Erase block at front of erase_pending_list is at 000e0000
Not rotating empty free_list
jffs2_do_fill_super(): Getting root inode
jffs2_read_inode(): inode->i_ino == 1
jffs2_do_read_inode(): getting inocache
jffs2_get_ino_cache(): ino 1
jffs2_get_ino_cache found 00000000 for ino 1
jffs2_do_read_inode(): Got inocache at 00000000
jffs2_do_read_inode(): Creating inocache for root inode
jffs2_add_ino_cache: Add c058d260 (ino #1)
jffs2_do_read_inode_internal(): ino #1 nlink is 1
jffs2_get_inode_nodes(): ino #1
jffs2_read_inode() returning
jffs2_do_fill_super(): d_alloc_root()
JFFS2: Garbage collect thread is pid 95

kernel 2.4.19 + RTAI + mtd<CVS> yesterday
Oops that was DEBUG=3 oh well the more info the better ;)
dmesg -n will only let me set level=8 max !

TIA,
	Frank.

Re: Please Help, problem mounting jffs2

[David Woodhouse]

Looks like it's waiting for the garbage collect thread. What happened to
it? Can you put a printk into the very beginning of
'jffs2_garbage_collect_thread()' in background.c, and another printk
after the daemonize() call. 

Also hit SysRq-T and see exactly where the mount process is waiting -- I
suspect it's in jffs2_start_garbage_collect_thread().

-- 
dwmw2

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

On Friday 13 June 2003 01:39 pm, David Woodhouse wrote:
> Looks like it's waiting for the garbage collect thread. What happened to
> it? Can you put a printk into the very beginning of
> 'jffs2_garbage_collect_thread()' in background.c, and another printk
> after the daemonize() call.

Like this:

static int jffs2_garbage_collect_thread(void *_c)
{
        struct jffs2_sb_info *c = _c;

        D1(printk(KERN_DEBUG "****jffs2_garbage_collect_thread start...\n"));

        daemonize("jffs2_gcd_mtd%d", c->mtd->index);

        D1(printk(KERN_DEBUG "****jffs2_garbage_collect_thread after 
daemonize()...\n"));

        c->gc_task = current;
        up(&c->gc_thread_start);

-----------------
No extra output !!!
-----------------

>
> Also hit SysRq-T and see exactly where the mount process is waiting -- I
> suspect it's in jffs2_start_garbage_collect_thread().

I'm running this on an embedded system and the console is via minicom (ttyS1), 
SysRq-T doesn't seem to work !

If I have to I can bodge up some H/W to allow a real console (only as a last 
resort) ;)

TIA,
	Frank.

Re: Please Help, problem mounting jffs2

[David Woodhouse]

On Fri, 2003-06-13 at 19:11, Frank R Callaghan wrote:
> On Friday 13 June 2003 01:39 pm, David Woodhouse wrote:
> > Looks like it's waiting for the garbage collect thread. What happened to
> > it? Can you put a printk into the very beginning of
> > 'jffs2_garbage_collect_thread()' in background.c, and another printk
> > after the daemonize() call.
> 
> Like this:
> 
> static int jffs2_garbage_collect_thread(void *_c)
> {
>         struct jffs2_sb_info *c = _c;
> 
>         D1(printk(KERN_DEBUG "****jffs2_garbage_collect_thread start...\n"));
> 
> -----------------
> No extra output !!!
> -----------------

Just like that... in which case for some reason kernel_thread() is not
working. What architecture is it?

> >
> > Also hit SysRq-T and see exactly where the mount process is waiting -- I
> > suspect it's in jffs2_start_garbage_collect_thread().
> 
> I'm running this on an embedded system and the console is via minicom (ttyS1), 
> SysRq-T doesn't seem to work !

Hmmm. It may require that there's something in userspace trying to read
from the port. There are also hacks around to enable sysrq by echoing
letters to a /proc file. Doesn't really matter -- it's fairly obvious
it's never actually making it into the kernel thread.

-- 
dwmw2

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

On Friday 13 June 2003 02:28 pm, David Woodhouse wrote:
> On Fri, 2003-06-13 at 19:11, Frank R Callaghan wrote:
> > On Friday 13 June 2003 01:39 pm, David Woodhouse wrote:
> > > Looks like it's waiting for the garbage collect thread. What happened
> > > to it? Can you put a printk into the very beginning of
> > > 'jffs2_garbage_collect_thread()' in background.c, and another printk
> > > after the daemonize() call.
> >
> > Like this:
> >
> > static int jffs2_garbage_collect_thread(void *_c)
> > {
> >         struct jffs2_sb_info *c = _c;
> >
> >         D1(printk(KERN_DEBUG "****jffs2_garbage_collect_thread
> > start...\n"));
> >
> > -----------------
> > No extra output !!!
> > -----------------
>
> Just like that... in which case for some reason kernel_thread() is not
> working. What architecture is it?
>

i486 ELAN(SC520)

> > > Also hit SysRq-T and see exactly where the mount process is waiting --
> > > I suspect it's in jffs2_start_garbage_collect_thread().
> >
> > I'm running this on an embedded system and the console is via minicom
> > (ttyS1), SysRq-T doesn't seem to work !
>
> Hmmm. It may require that there's something in userspace trying to read
> from the port. There are also hacks around to enable sysrq by echoing
> letters to a /proc file. Doesn't really matter -- it's fairly obvious
> it's never actually making it into the kernel thread.

OK, this sounds bad !  Where can I go from here ?
I really need this flash working for the product to be viable !
Is this a kernel bug ? uclibc ? RTAI ?
Is there anyway to hack the code to get it working ?

Help !!!

Re: Please Help, problem mounting jffs2

[David Woodhouse]

On Fri, 2003-06-13 at 19:47, Frank R Callaghan wrote:
> OK, this sounds bad !  Where can I go from here ?

Well, you can start by just commenting out the whole of both the
jffs2_{start,stop}_garbage_collect_thread() functions since the thread
is an optimisation anyway. But basically you need to find out why
kernel_thread() isn't working.

It's not a uclibc bug -- it's a kernel bug, and not one in the standard
kernel either -- I'm inclined to suspect RTAI at this stage, although
you blatantly have other kernel threads running so it's not a simple and
obvious breakage.


-- 
dwmw2

Re: Please Help, problem mounting jffs2

[Thomas Gleixner]

On Friday 13 June 2003 21:18, David Woodhouse wrote:
> On Fri, 2003-06-13 at 19:47, Frank R Callaghan wrote:
> > OK, this sounds bad !  Where can I go from here ?
>
> Well, you can start by just commenting out the whole of both the
> jffs2_{start,stop}_garbage_collect_thread() functions since the thread
> is an optimisation anyway. But basically you need to find out why
> kernel_thread() isn't working.
>
> It's not a uclibc bug -- it's a kernel bug, and not one in the standard
> kernel either -- I'm inclined to suspect RTAI at this stage, although
> you blatantly have other kernel threads running so it's not a simple and
> obvious breakage.

Maybe it's a RTAI problem. But I have JFFS2 / RTAI running on a couple of 
boards. 

Have you tried this with a vanilla kernel without RTAI ? If not, can you do 
that ?

If RTAI is the guilty one, then consider the following:
kernel_thread () is a syscall. RTAI captures syscalls. Maybe the syscall is 
lost anywhere in RTAI.

-- 
Thomas
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

On Friday 13 June 2003 18:52, you wrote:
> On Friday 13 June 2003 21:18, David Woodhouse wrote:
> > On Fri, 2003-06-13 at 19:47, Frank R Callaghan wrote:
> > > OK, this sounds bad !  Where can I go from here ?
> >
> > Well, you can start by just commenting out the whole of both the
> > jffs2_{start,stop}_garbage_collect_thread() functions since the thread
> > is an optimisation anyway. But basically you need to find out why
> > kernel_thread() isn't working.
> >
> > It's not a uclibc bug -- it's a kernel bug, and not one in the standard
> > kernel either -- I'm inclined to suspect RTAI at this stage, although
> > you blatantly have other kernel threads running so it's not a simple and
> > obvious breakage.
>
> Maybe it's a RTAI problem. But I have JFFS2 / RTAI running on a couple of
> boards.
>
> Have you tried this with a vanilla kernel without RTAI ? If not, can you do
> that ?
>
> If RTAI is the guilty one, then consider the following:
> kernel_thread () is a syscall. RTAI captures syscalls. Maybe the syscall is
> lost anywhere in RTAI.

I have two userland processes communicating via shared mem to an rtai_module
also using two rtf's.

what I have found is after loading rtai & starting the programs - attempting 
to mount the jffs2 filesystem stops. When I killed the two processes the mount 
resumed and all works perfectly (killing the proceses forces the unload of the
shared memory & forces the closing of the rtf's)
Starting them backup after mounting sees to work fine !

I guess I can try to isolate it futher by not starting the
 fifo's / shared memory,  unless anyone can suggest a better way to locate
the problem.

TIA,
	Frank.

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

Hi All,

I methodically removed code from my usrland programs until I'm left with

int main()
{
    while(1) {
    //  sleep(1);
    }
}

With no RTAI modules loaded, the gc thread still stalls.
With the sleep(1) works fine !

Does this mean that the gc thread has such a low priority that it never runs ?

I'm sorry if this is a stupid question, but it doesn't seem right that a mount
should suspend if a fg process is in a while loop !!!

What should I do/try next ?

TIA,
        Frank.
  

> attempting to mount the jffs2 filesystem stops. When I killed the two
> processes the mount resumed and all works perfectly (killing the processes
> forces the unload of the shared memory & forces the closing of the rtf's)
> Starting them backup after mounting sees to work fine !
>
> I guess I can try to isolate it futher by not starting the
>  fifo's / shared memory,  unless anyone can suggest a better way to locate
> the problem.
>

Re: Please Help, problem mounting jffs2

[David Woodhouse]

On Mon, 2003-06-16 at 17:06, Frank R Callaghan wrote:
> Hi All,
> 
> I methodically removed code from my usrland programs until I'm left with
> 
> int main()
> {
>     while(1) {
>     //  sleep(1);
>     }
> }
> 
> With no RTAI modules loaded, the gc thread still stalls.
> With the sleep(1) works fine !
> 
> Does this mean that the gc thread has such a low priority that it never runs ?

No, it looks like your userspace prograem has such a _high_ priority
that the GC thread never runs. A subtle difference :)

> What should I do/try next ?

This is an RTAI-specific issue. I suspect your test program above has
its priority set such that when it's eating 100% CPU time in a tight
loop, nothing else gets to run. Have you asked on the RTAI mailing
lists?

-- 
dwmw2

Re: Please Help, problem mounting jffs2

[Frank R Callaghan]

On Monday 16 June 2003 12:25 pm, David Woodhouse wrote:
> On Mon, 2003-06-16 at 17:06, Frank R Callaghan wrote:
> > Hi All,
> >
> > I methodically removed code from my usrland programs until I'm left with
> >
> > int main()
> > {
> >     while(1) {
> >     //  sleep(1);
> >     }
> > }
> >
> > With no RTAI modules loaded, the gc thread still stalls.
> > With the sleep(1) works fine !
> >
> > Does this mean that the gc thread has such a low priority that it never
> > runs ?
>
> No, it looks like your userspace program has such a _high_ priority
> that the GC thread never runs. A subtle difference :)
>

I have reniced the test program to +20 but the mount is still hung !

> > What should I do/try next ?
>
> This is an RTAI-specific issue. I suspect your test program above has
> its priority set such that when it's eating 100% CPU time in a tight
> loop, nothing else gets to run. Have you asked on the RTAI mailing
> lists?

yes - no response yet !

I'm not sure how rtai can be interfering if there are no-rtai modules
loaded - unless the kernel patches messed something up !

Re: Please Help, problem mounting jffs2

[Thomas Gleixner]

On Monday 16 June 2003 20:21, Frank R Callaghan wrote:
> > No, it looks like your userspace program has such a _high_ priority
> > that the GC thread never runs. A subtle difference :)
>
> I have reniced the test program to +20 but the mount is still hung !
>
> > > What should I do/try next ?
> >
> > This is an RTAI-specific issue. I suspect your test program above has
> > its priority set such that when it's eating 100% CPU time in a tight
> > loop, nothing else gets to run. Have you asked on the RTAI mailing
> > lists?
>
> yes - no response yet !
>
> I'm not sure how rtai can be interfering if there are no-rtai modules
> loaded - unless the kernel patches messed something up !
A RTAI patched kernel without modules loaded should behave like a unpatched 
kernel. 

Switch off CONFIG_RTHAL and try again. This should remove all RTAI tweaks from 
the kernel.

If that does not work try a (RTAI) unpatched kernel.

-- 
Thomas
________________________________________________________________________
linutronix - competence in embedded & realtime linux
http://www.linutronix.de
mail: tglx@linutronix.de