sync() takes minutes while erasing

sync() takes minutes while erasing

[Joakim Tjernlund]

Here we go again :)

Noticed one again large delays when rebooting and jffs2 is busy
erasing block via pdflush.

Can't see where sync() ends up in jffs2, anyone
know the jffs2(or pdflush) functions involved during a sync?

 Jocke 

Re: sync() takes minutes while erasing

[Joakim Tjernlund]

On Wed, 2007-06-20 at 16:21 +0200, Joakim Tjernlund wrote:
> Here we go again :)
> 
> Noticed one again large delays when rebooting and jffs2 is busy
> erasing block via pdflush.
> 
> Can't see where sync() ends up in jffs2, anyone
> know the jffs2(or pdflush) functions involved during a sync?
> 
>  Jocke 

Ok, found the cause:
 jffs2_erase_pending_blocks(c, 0) is called from
 jffs2_write_super() and it hangs until all
 pending erases are complete.

Why do jffs2_write_super() need to call 
jffs2_erase_pending_blocks(c, 0)? Seems like a bad idea to me.

 Jocke

Re: sync() takes minutes while erasing

[Josh Boyer]

On Wed, 2007-06-20 at 16:49 +0200, Joakim Tjernlund wrote:
> On Wed, 2007-06-20 at 16:21 +0200, Joakim Tjernlund wrote:
> > Here we go again :)
> > 
> > Noticed one again large delays when rebooting and jffs2 is busy
> > erasing block via pdflush.
> > 
> > Can't see where sync() ends up in jffs2, anyone
> > know the jffs2(or pdflush) functions involved during a sync?
> > 
> >  Jocke 
> 
> Ok, found the cause:
>  jffs2_erase_pending_blocks(c, 0) is called from
>  jffs2_write_super() and it hangs until all
>  pending erases are complete.
> 
> Why do jffs2_write_super() need to call 
> jffs2_erase_pending_blocks(c, 0)? Seems like a bad idea to me.

jffs2_write_super is called during unmount.  If there are blocks pending
on the erase list, it's better to erase those now before the unmount is
completed.  Saves some time on the next mount.

josh

Re: sync() takes minutes while erasing

[Joakim Tjernlund]

On Wed, 2007-06-20 at 11:09 -0500, Josh Boyer wrote:
> On Wed, 2007-06-20 at 16:49 +0200, Joakim Tjernlund wrote:
> > On Wed, 2007-06-20 at 16:21 +0200, Joakim Tjernlund wrote:
> > > Here we go again :)
> > > 
> > > Noticed one again large delays when rebooting and jffs2 is busy
> > > erasing block via pdflush.
> > > 
> > > Can't see where sync() ends up in jffs2, anyone
> > > know the jffs2(or pdflush) functions involved during a sync?
> > > 
> > >  Jocke 
> > 
> > Ok, found the cause:
> >  jffs2_erase_pending_blocks(c, 0) is called from
> >  jffs2_write_super() and it hangs until all
> >  pending erases are complete.
> > 
> > Why do jffs2_write_super() need to call 
> > jffs2_erase_pending_blocks(c, 0)? Seems like a bad idea to me.
> 
> jffs2_write_super is called during unmount.  If there are blocks pending
> on the erase list, it's better to erase those now before the unmount is
> completed.  Saves some time on the next mount.

No, not if the reboot/unmont is suspended for minutes just because jffs2
wants to finish erasing. It doesn't save you much, if anything, at
remount either.

The erasing in context is just something that needs to be done at some
point but it don't have to happen now.

  Jocke 

> 
> josh
> 
> 
> 

Re: sync() takes minutes while erasing

[Jörn Engel]

On Wed, 20 June 2007 18:26:36 +0200, Joakim Tjernlund wrote:
> On Wed, 2007-06-20 at 11:09 -0500, Josh Boyer wrote:
> > 
> > jffs2_write_super is called during unmount.  If there are blocks pending
> > on the erase list, it's better to erase those now before the unmount is
> > completed.  Saves some time on the next mount.
> 
> No, not if the reboot/unmont is suspended for minutes just because jffs2
> wants to finish erasing. It doesn't save you much, if anything, at
> remount either.
> 
> The erasing in context is just something that needs to be done at some
> point but it don't have to happen now.

In case you need an example:

$ flash_eraseall /dev/mtd0
$ time mount mtd0 /mnt -t jffs2
$ time umount /mnt

Depending on device size, this will take some seconds for mount and some
minutes for umount.  Spending minutes to save some seconds on next mount
hardly sounds sensible.

Jörn

-- 
"Security vulnerabilities are here to stay."
-- Scott Culp, Manager of the Microsoft Security Response Center, 2001

Re: sync() takes minutes while erasing

[Joakim Tjernlund]

On Wed, 2007-06-20 at 18:29 +0200, Jörn Engel wrote:
> On Wed, 20 June 2007 18:26:36 +0200, Joakim Tjernlund wrote:
> > On Wed, 2007-06-20 at 11:09 -0500, Josh Boyer wrote:
> > > 
> > > jffs2_write_super is called during unmount.  If there are blocks pending
> > > on the erase list, it's better to erase those now before the unmount is
> > > completed.  Saves some time on the next mount.
> > 
> > No, not if the reboot/unmont is suspended for minutes just because jffs2
> > wants to finish erasing. It doesn't save you much, if anything, at
> > remount either.
> > 
> > The erasing in context is just something that needs to be done at some
> > point but it don't have to happen now.
> 
> In case you need an example:
> 
> $ flash_eraseall /dev/mtd0
> $ time mount mtd0 /mnt -t jffs2
> $ time umount /mnt
> 
> Depending on device size, this will take some seconds for mount and some
> minutes for umount.  Spending minutes to save some seconds on next mount
> hardly sounds sensible.
> 
> Jörn

Exactly, 
David where are you? You havn't been active for a long time.
Is there any reason to keep as is?

 Jocke

Re: sync() takes minutes while erasing

[Josh Boyer]

On Wed, 2007-06-20 at 18:45 +0200, Joakim Tjernlund wrote:
> On Wed, 2007-06-20 at 18:29 +0200, Jörn Engel wrote:
> > On Wed, 20 June 2007 18:26:36 +0200, Joakim Tjernlund wrote:
> > > On Wed, 2007-06-20 at 11:09 -0500, Josh Boyer wrote:
> > > > 
> > > > jffs2_write_super is called during unmount.  If there are blocks pending
> > > > on the erase list, it's better to erase those now before the unmount is
> > > > completed.  Saves some time on the next mount.
> > > 
> > > No, not if the reboot/unmont is suspended for minutes just because jffs2
> > > wants to finish erasing. It doesn't save you much, if anything, at
> > > remount either.
> > > 
> > > The erasing in context is just something that needs to be done at some
> > > point but it don't have to happen now.
> > 
> > In case you need an example:
> > 
> > $ flash_eraseall /dev/mtd0
> > $ time mount mtd0 /mnt -t jffs2
> > $ time umount /mnt
> > 
> > Depending on device size, this will take some seconds for mount and some
> > minutes for umount.  Spending minutes to save some seconds on next mount
> > hardly sounds sensible.
> > 
> > Jörn
> 
> Exactly, 

Ok, you've both convinced me.

> David where are you? You havn't been active for a long time.

He's in China on business.

> Is there any reason to keep as is?

Maybe not.  We definitely want it to flush the write buffer and complete
any block in the _middle_ of an erase though.

josh

Re: sync() takes minutes while erasing

[David Woodhouse]

On Wed, 2007-06-20 at 16:49 +0200, Joakim Tjernlund wrote:
> Ok, found the cause:
>  jffs2_erase_pending_blocks(c, 0) is called from
>  jffs2_write_super() and it hangs until all
>  pending erases are complete.
> 
> Why do jffs2_write_super() need to call 
> jffs2_erase_pending_blocks(c, 0)? Seems like a bad idea to me. 

It's because pdflush is the best context I could find for actually doing
the erases we need. But now I think we can use pdflush_operation()
instead of having to do it from write_super(), so maybe we can fix the
thing that offends you.

-- 
dwmw2

RE: sync() takes minutes while erasing

[Joakim Tjernlund]

> -----Original Message-----
> From: David Woodhouse [mailto:dwmw2@infradead.org] 
> Sent: den 28 juni 2007 20:54
> To: joakim.tjernlund@transmode.se
> Cc: Linux MTD mailing list
> Subject: Re: sync() takes minutes while erasing
> 
> On Wed, 2007-06-20 at 16:49 +0200, Joakim Tjernlund wrote:
> > Ok, found the cause:
> >  jffs2_erase_pending_blocks(c, 0) is called from
> >  jffs2_write_super() and it hangs until all
> >  pending erases are complete.
> > 
> > Why do jffs2_write_super() need to call 
> > jffs2_erase_pending_blocks(c, 0)? Seems like a bad idea to me. 
> 
> It's because pdflush is the best context I could find for 
> actually doing
> the erases we need. But now I think we can use pdflush_operation()
> instead of having to do it from write_super(), so maybe we can fix the
> thing that offends you.

Thanks, this stop the complaints from the test team. They
don't like to wait a few mins every time they delete an old install
and reboot.

  Jocke

RE: sync() takes minutes while erasing

[David Woodhouse]

On Thu, 2007-06-28 at 20:59 +0200, Joakim Tjernlund wrote:
> Thanks, this stop the complaints from the test team. They
> don't like to wait a few mins every time they delete an old install
> and reboot.

This is what the rest of us use 'flash_eraseall -j' for, btw :)

But I'll look into it. If we can do the erases from some other context
it would be nice. Although pdflush_operation() doesn't seem to give us
any way to _stop_ a pending call if we unmount.

-- 
dwmw2

RE: sync() takes minutes while erasing

[Joakim Tjernlund]

On Thu, 2007-06-28 at 20:02 +0100, David Woodhouse wrote:
> On Thu, 2007-06-28 at 20:59 +0200, Joakim Tjernlund wrote:
> > Thanks, this stop the complaints from the test team. They
> > don't like to wait a few mins every time they delete an old install
> > and reboot.
> 
> This is what the rest of us use 'flash_eraseall -j' for, btw :)
> 
> But I'll look into it. If we can do the erases from some other context
> it would be nice. Although pdflush_operation() doesn't seem to give us
> any way to _stop_ a pending call if we unmount.
> 

hmm, can't you make so that pdflush_operation() just erases one
block at a time? Then the longest you have to wait is just
the time it takes to finish one EB.

 Jocke

RE: sync() takes minutes while erasing

[Joakim Tjernlund]

On Mon, 2007-07-02 at 14:58 +0200, Joakim Tjernlund wrote:
> On Thu, 2007-06-28 at 20:02 +0100, David Woodhouse wrote:
> > On Thu, 2007-06-28 at 20:59 +0200, Joakim Tjernlund wrote:
> > > Thanks, this stop the complaints from the test team. They
> > > don't like to wait a few mins every time they delete an old install
> > > and reboot.
> > 
> > This is what the rest of us use 'flash_eraseall -j' for, btw :)
> > 
> > But I'll look into it. If we can do the erases from some other context
> > it would be nice. Although pdflush_operation() doesn't seem to give us
> > any way to _stop_ a pending call if we unmount.
> > 
> 
> hmm, can't you make so that pdflush_operation() just erases one
> block at a time? Then the longest you have to wait is just
> the time it takes to finish one EB.
> 
>  Jocke

Hi David

Had any time to think on the above?

   Jocke

RE: sync() takes minutes while erasing

[Joakim Tjernlund]

On Mon, 2007-08-06 at 18:18 +0200, Joakim Tjernlund wrote:
> On Mon, 2007-07-02 at 14:58 +0200, Joakim Tjernlund wrote:
> > On Thu, 2007-06-28 at 20:02 +0100, David Woodhouse wrote:
> > > On Thu, 2007-06-28 at 20:59 +0200, Joakim Tjernlund wrote:
> > > > Thanks, this stop the complaints from the test team. They
> > > > don't like to wait a few mins every time they delete an old install
> > > > and reboot.
> > > 
> > > This is what the rest of us use 'flash_eraseall -j' for, btw :)
> > > 
> > > But I'll look into it. If we can do the erases from some other context
> > > it would be nice. Although pdflush_operation() doesn't seem to give us
> > > any way to _stop_ a pending call if we unmount.

hmm, can't jffs2_erase_pending_blocks() see if a unmount is in progress
and if so, stop erasing blocks? 

> > > 
> > 
> > hmm, can't you make so that pdflush_operation() just erases one
> > block at a time? Then the longest you have to wait is just
> > the time it takes to finish one EB.
> > 
> >  Jocke
> 
> Hi David
> 
> Had any time to think on the above?
> 
>    Jocke
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 
> 

RE: sync() takes minutes while erasing

[Joakim Tjernlund]

On Wed, 2007-08-08 at 15:06 +0200, Joakim Tjernlund wrote:
> On Mon, 2007-08-06 at 18:18 +0200, Joakim Tjernlund wrote:
> > On Mon, 2007-07-02 at 14:58 +0200, Joakim Tjernlund wrote:
> > > On Thu, 2007-06-28 at 20:02 +0100, David Woodhouse wrote:
> > > > On Thu, 2007-06-28 at 20:59 +0200, Joakim Tjernlund wrote:
> > > > > Thanks, this stop the complaints from the test team. They
> > > > > don't like to wait a few mins every time they delete an old install
> > > > > and reboot.
> > > > 
> > > > This is what the rest of us use 'flash_eraseall -j' for, btw :)
> > > > 
> > > > But I'll look into it. If we can do the erases from some other context
> > > > it would be nice. Although pdflush_operation() doesn't seem to give us
> > > > any way to _stop_ a pending call if we unmount.
> 
> hmm, can't jffs2_erase_pending_blocks() see if a unmount is in progress
> and if so, stop erasing blocks? 

Any hints on the "unmount is in progress" idea? I have a few
hours to spare for this ATM.

  Jocke

> 
> > > > 
> > > 
> > > hmm, can't you make so that pdflush_operation() just erases one
> > > block at a time? Then the longest you have to wait is just
> > > the time it takes to finish one EB.
> > > 
> > >  Jocke
> > 
> > Hi David
> > 
> > Had any time to think on the above?
> > 
> >    Jocke
> > 
> > ______________________________________________________
> > Linux MTD discussion mailing list
> > http://lists.infradead.org/mailman/listinfo/linux-mtd/
> > 
> > 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 
> 

RE: sync() takes minutes while erasing

[Joakim Tjernlund]

On Thu, 2007-11-08 at 17:37 +0100, Joakim Tjernlund wrote:
> On Wed, 2007-08-08 at 15:06 +0200, Joakim Tjernlund wrote:
> > On Mon, 2007-08-06 at 18:18 +0200, Joakim Tjernlund wrote:
> > > On Mon, 2007-07-02 at 14:58 +0200, Joakim Tjernlund wrote:
> > > > On Thu, 2007-06-28 at 20:02 +0100, David Woodhouse wrote:
> > > > > On Thu, 2007-06-28 at 20:59 +0200, Joakim Tjernlund wrote:
> > > > > > Thanks, this stop the complaints from the test team. They
> > > > > > don't like to wait a few mins every time they delete an old install
> > > > > > and reboot.
> > > > > 
> > > > > This is what the rest of us use 'flash_eraseall -j' for, btw :)
> > > > > 
> > > > > But I'll look into it. If we can do the erases from some other context
> > > > > it would be nice. Although pdflush_operation() doesn't seem to give us
> > > > > any way to _stop_ a pending call if we unmount.
> > 
> > hmm, can't jffs2_erase_pending_blocks() see if a unmount is in progress
> > and if so, stop erasing blocks? 
> 
> Any hints on the "unmount is in progress" idea? I have a few
> hours to spare for this ATM.

Tried to test for sb->s_flags & MS_ACTIVE but that didn't work.
Maybe it will work if jffs2_erase_pending_blocks() is moved to
pdflush_operation() context? Dunno how to do that though.

 Jocke