nilfs_cleanerd some questions

nilfs_cleanerd some questions

[tom]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi

I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686
and nilfs version 2.0.6. Currently I have a 250 Gb single partition
drive connected via USB2, which I have formatted for nilfs2. I have
noticed that when extracting a large archive onto the the file system
that it seems to "burst", that is, it will extract for 5 seconds then
halt for 1 and then repeat this cycle faithfully for the duration.

If I switch (kill) off the GC then the "bursting" effect disappears. I
can remove checkpoints more efficiently using rmcp, however am I right
in thinking that the GC does more than just remove checkpoints ?

Is the behaviour that I am witnessing typical of the nilfs GC or is
being highlighted because I am using usb instead of directly attached
ide / sata ?

I'm hoping to apply nilfs in production environment for our print unit
at work because they would like self service recovery and because they
have large amounts of graphical data that is updated daily but could
only be safely backed up using a snaphot technology.

However I am concerned that any large file / archive extracts onto a
nilfs file system with the GC switched on would impede efficiency.

It is safe to say that I wont be using USB attached storage in the
production environment, but I thought it might be worth mentioning my
findings as the behaviour may still be present in a very muted form on
IDE / SATA based storage devices.

I guess I could run GC at off peak times as a workaround, but would it
ever catch up ?

I would be most interested in your comments.

Thanks (for nilfs)

Tom.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkl7WbUACgkQBX2gJWUv0is23gCeJ4cdHUxnkKcOoffc5Ic1Qdco
8HYAn3Lt7MsFIbxX+ayZwbfaeTBTLVXY
=sytK
-----END PGP SIGNATURE-----

Re: nilfs_cleanerd some questions

[Ryusuke Konishi]

Hi,
On Sat, 24 Jan 2009 18:11:14 +0000, tom wrote:
> I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686
> and nilfs version 2.0.6. Currently I have a 250 Gb single partition
> drive connected via USB2, which I have formatted for nilfs2. I have
> noticed that when extracting a large archive onto the the file system
> that it seems to "burst", that is, it will extract for 5 seconds then
> halt for 1 and then repeat this cycle faithfully for the duration.
> 
> If I switch (kill) off the GC then the "bursting" effect disappears. I
> can remove checkpoints more efficiently using rmcp, however am I right
> in thinking that the GC does more than just remove checkpoints ?

Yes, only GC can reclaim disk space; rmcp does not make free space.
 
> Is the behaviour that I am witnessing typical of the nilfs GC or is
> being highlighted because I am using usb instead of directly attached
> ide / sata ?

I don't know why that happens, but a likely cause is bulky reads of
blocks during GC.

In that case, the following workarounds may make a difference.

1) Decrease value of `nsegments_per_clean' in /etc/nilfs_cleanerd.conf
  e.g.
  #nsegments_per_clean   2
  nsegments_per_clean   1

2) Re-create the partition using mkfs.nilfs2 with a smaller segment size.
  e.g.
  # mkfs -t nilfs2 -B 1024 /dev/sdxx
  (This halves the number of blocks per segment)

> I'm hoping to apply nilfs in production environment for our print unit
> at work because they would like self service recovery and because they
> have large amounts of graphical data that is updated daily but could
> only be safely backed up using a snaphot technology.
> 
> However I am concerned that any large file / archive extracts onto a
> nilfs file system with the GC switched on would impede efficiency.
>
> It is safe to say that I wont be using USB attached storage in the
> production environment, but I thought it might be worth mentioning my
> findings as the behaviour may still be present in a very muted form on
> IDE / SATA based storage devices.

Thanks for reporting.

I feel, the current GC is indeed immature and needs more love
especially for production use.
 
> I guess I could run GC at off peak times as a workaround, but would it
> ever catch up ?

Not guaranteed.  The speed of the GC is static unless you change
nilfs_cleanerd.conf and send a HUP signal to the GC daemon.
At least the speed should be adjusted adaptively, but not done yet.

Regards,
Ryusuke Konishi

Re: nilfs_cleanerd some questions

[Gergely Gábor]

Hello!

Sorry for my intrusion, but probably a way to pause the GC and resume
it, would be nice, maybe with SIGUSR1 and SIGUSR2, or even trough dbus
>:-), to let other cases be handled, like running on battery, and
similiar. IMHO this is much more elegant way, then killing, and
respawning the GC at every such change, and it can finish its work,
before getting killed aswell.

Greeings:
Gergely Gábor.

On Fri, Jan 30, 2009 at 10:40 AM, Ryusuke Konishi <ryusuke@osrg.net> wrote:
> Hi,
> On Sat, 24 Jan 2009 18:11:14 +0000, tom wrote:
>> I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686
>> and nilfs version 2.0.6. Currently I have a 250 Gb single partition
>> drive connected via USB2, which I have formatted for nilfs2. I have
>> noticed that when extracting a large archive onto the the file system
>> that it seems to "burst", that is, it will extract for 5 seconds then
>> halt for 1 and then repeat this cycle faithfully for the duration.
>>
>> If I switch (kill) off the GC then the "bursting" effect disappears. I
>> can remove checkpoints more efficiently using rmcp, however am I right
>> in thinking that the GC does more than just remove checkpoints ?
>
> Yes, only GC can reclaim disk space; rmcp does not make free space.
>
>> Is the behaviour that I am witnessing typical of the nilfs GC or is
>> being highlighted because I am using usb instead of directly attached
>> ide / sata ?
>
> I don't know why that happens, but a likely cause is bulky reads of
> blocks during GC.
>
> In that case, the following workarounds may make a difference.
>
> 1) Decrease value of `nsegments_per_clean' in /etc/nilfs_cleanerd.conf
>  e.g.
>  #nsegments_per_clean   2
>  nsegments_per_clean   1
>
> 2) Re-create the partition using mkfs.nilfs2 with a smaller segment size.
>  e.g.
>  # mkfs -t nilfs2 -B 1024 /dev/sdxx
>  (This halves the number of blocks per segment)
>
>> I'm hoping to apply nilfs in production environment for our print unit
>> at work because they would like self service recovery and because they
>> have large amounts of graphical data that is updated daily but could
>> only be safely backed up using a snaphot technology.
>>
>> However I am concerned that any large file / archive extracts onto a
>> nilfs file system with the GC switched on would impede efficiency.
>>
>> It is safe to say that I wont be using USB attached storage in the
>> production environment, but I thought it might be worth mentioning my
>> findings as the behaviour may still be present in a very muted form on
>> IDE / SATA based storage devices.
>
> Thanks for reporting.
>
> I feel, the current GC is indeed immature and needs more love
> especially for production use.
>
>> I guess I could run GC at off peak times as a workaround, but would it
>> ever catch up ?
>
> Not guaranteed.  The speed of the GC is static unless you change
> nilfs_cleanerd.conf and send a HUP signal to the GC daemon.
> At least the speed should be adjusted adaptively, but not done yet.
>
> Regards,
> Ryusuke Konishi
> _______________________________________________
> users mailing list
> users@nilfs.org
> https://www.nilfs.org/mailman/listinfo/users
>
_______________________________________________
users mailing list
users@nilfs.org
https://www.nilfs.org/mailman/listinfo/users

Re: nilfs_cleanerd some questions

[Ryusuke Konishi]

Hi,
On Fri, 30 Jan 2009 10:55:14 +0100, Gergely Gábor wrote:
> Hello!
> 
> Sorry for my intrusion, but probably a way to pause the GC and resume
> it, would be nice, maybe with SIGUSR1 and SIGUSR2, or even trough dbus
> >:-), to let other cases be handled, like running on battery, and
> similiar. IMHO this is much more elegant way, then killing, and
> respawning the GC at every such change, and it can finish its work,
> before getting killed aswell.

I don't disagree.
We sometimes do the simliar thing by raising `protection_period'
temporarily, but it's not elegant either.

Regards,
Ryusuke
 
> On Fri, Jan 30, 2009 at 10:40 AM, Ryusuke Konishi <ryusuke-sG5X7nlA6pw@public.gmane.org> wrote:
> > Hi,
> > On Sat, 24 Jan 2009 18:11:14 +0000, tom wrote:
> >> I'm using nilfs2 on a debian (lenny machine) using kernel 2.6.26-1-686
> >> and nilfs version 2.0.6. Currently I have a 250 Gb single partition
> >> drive connected via USB2, which I have formatted for nilfs2. I have
> >> noticed that when extracting a large archive onto the the file system
> >> that it seems to "burst", that is, it will extract for 5 seconds then
> >> halt for 1 and then repeat this cycle faithfully for the duration.
> >>
> >> If I switch (kill) off the GC then the "bursting" effect disappears. I
> >> can remove checkpoints more efficiently using rmcp, however am I right
> >> in thinking that the GC does more than just remove checkpoints ?
> >
> > Yes, only GC can reclaim disk space; rmcp does not make free space.
> >
> >> Is the behaviour that I am witnessing typical of the nilfs GC or is
> >> being highlighted because I am using usb instead of directly attached
> >> ide / sata ?
> >
> > I don't know why that happens, but a likely cause is bulky reads of
> > blocks during GC.
> >
> > In that case, the following workarounds may make a difference.
> >
> > 1) Decrease value of `nsegments_per_clean' in /etc/nilfs_cleanerd.conf
> >  e.g.
> >  #nsegments_per_clean   2
> >  nsegments_per_clean   1
> >
> > 2) Re-create the partition using mkfs.nilfs2 with a smaller segment size.
> >  e.g.
> >  # mkfs -t nilfs2 -B 1024 /dev/sdxx
> >  (This halves the number of blocks per segment)
> >
> >> I'm hoping to apply nilfs in production environment for our print unit
> >> at work because they would like self service recovery and because they
> >> have large amounts of graphical data that is updated daily but could
> >> only be safely backed up using a snaphot technology.
> >>
> >> However I am concerned that any large file / archive extracts onto a
> >> nilfs file system with the GC switched on would impede efficiency.
> >>
> >> It is safe to say that I wont be using USB attached storage in the
> >> production environment, but I thought it might be worth mentioning my
> >> findings as the behaviour may still be present in a very muted form on
> >> IDE / SATA based storage devices.
> >
> > Thanks for reporting.
> >
> > I feel, the current GC is indeed immature and needs more love
> > especially for production use.
> >
> >> I guess I could run GC at off peak times as a workaround, but would it
> >> ever catch up ?
> >
> > Not guaranteed.  The speed of the GC is static unless you change
> > nilfs_cleanerd.conf and send a HUP signal to the GC daemon.
> > At least the speed should be adjusted adaptively, but not done yet.
> >
> > Regards,
> > Ryusuke Konishi
> > _______________________________________________
> > users mailing list
> > users-JrjvKiOkagjYtjvyW6yDsg@public.gmane.org
> > https://www.nilfs.org/mailman/listinfo/users
> >
> _______________________________________________
> users mailing list
> users-JrjvKiOkagjYtjvyW6yDsg@public.gmane.org
> https://www.nilfs.org/mailman/listinfo/users