Questions regarding use of nilfs2 on SSDs

Questions regarding use of nilfs2 on SSDs

[Ingo Glöckner]

Hello,

first of all thank you very much for contributing
this great file system, nilfs2!
I would like to use nilfs2 on an SSD but there
are some open questions, hopefully you can help
me clearing these issues.

- I have read somewhere
> http://www.ocztechnologyforum.com/forum/showpost.php?p=361424&postcount=10

  that using nilfs2 as the rootfs generates enormous I/O traffic
  (up to 45 GB write in 1-2 days!) If this is true, then it will
  eventually translate into reduced lifetime for SSDs.
  Can you confirm that the segment management of nilfs2
  is so write-heavy on the drive?
  If so, are there ways to avoid this?

- The standard segment size of nilfs2 appears to be 8MB.
  This compares to an erase block size of the SSD of
  (say) 512 KB. I wonder if reducing the segment size
  of nilfs to a smaller value (i.e., closer to the
  erase block size of the SSD) will help to
  keep the number of erased blocks/day small
  (this will benefit the lifetime of the drive)
  or if there is no such connection.
  In any case, what is the useful range for segment
  sizes? If I reduce the segment size, will I get
  a performance penalty?

- nilfs_cleanerd.conf specifies the following default values,

# The maximum number of segments to be cleaned at a time.
nsegments_per_clean     2

# Cleaning interval in second.
cleaning_interval       5

Do I interpret this correctly as stating that only 2 segments
(i.e. 16MB) can be cleaned in 5 seconds?
What negative effects will I get if I increase the number
of segments that can be cleaned at a time?

Thanks in advance for your help!
Cheers, Ingo Glöckner

Re: Questions regarding use of nilfs2 on SSDs

[Ryusuke Konishi]

Hi!
On Sat, 28 Mar 2009 21:55:54 +0100, Ingo Glöckner wrote:
> Hello,
> 
> first of all thank you very much for contributing
> this great file system, nilfs2!
> I would like to use nilfs2 on an SSD but there
> are some open questions, hopefully you can help
> me clearing these issues.
> 
> - I have read somewhere
> > http://www.ocztechnologyforum.com/forum/showpost.php?p=361424&postcount=10
> 
>   that using nilfs2 as the rootfs generates enormous I/O traffic
>   (up to 45 GB write in 1-2 days!) If this is true, then it will
>   eventually translate into reduced lifetime for SSDs.
>   Can you confirm that the segment management of nilfs2
>   is so write-heavy on the drive?
>   If so, are there ways to avoid this?

Yes, this is true.  This is mainly because garbage collection creep
around the partition until no old updates are found.

The current GC algorithm is not intelligent.  Under the algorithm, a
filesystem holding large amount of data suffers long-run of GC.  That
is why the rootfs generated enourmous I/O traffic.

An easy way to reduce this is increasing ``protection_period'' in
/etc/nilfs_cleanerd.conf or just stopping GC while the partition has
enough free space.

Your requirement is clearly one of primary TODO items of nilfs2, but
unfortunately I have too many things to do.  One of my colleagues is
trying to improve the GC, so I'd like to expect his work.  Any
proposal to refine clearnerd is welcome.
 
> - The standard segment size of nilfs2 appears to be 8MB.
>   This compares to an erase block size of the SSD of
>   (say) 512 KB. I wonder if reducing the segment size
>   of nilfs to a smaller value (i.e., closer to the
>   erase block size of the SSD) will help to
>   keep the number of erased blocks/day small
>   (this will benefit the lifetime of the drive)
>   or if there is no such connection.
>   In any case, what is the useful range for segment
>   sizes? If I reduce the segment size, will I get
>   a performance penalty?

The segment size can be changed with -B option of mkfs.nilfs2.
It specifies the number of blocks per segment.

  # mkfs -t nilf2 -B 128  /dev/xxx

will make a partition with the segment size of 512 KB.
(the block size is 4KB unless you change it with -b option)

This will mitigate GC overhead because the number of segments per
cleaning is fixed to a value defined in /etc/nilfs_cleanerd.conf.

But, it's just effect of speed decreasing of the cleaner.  If the
purpose is slowing down the speed, just reducing
``nsegments_per_clean'' or increasing ``cleaning_interval'' in the
conf file can serve the purpose.

> - nilfs_cleanerd.conf specifies the following default values,
> 
> # The maximum number of segments to be cleaned at a time.
> nsegments_per_clean     2
> 
> # Cleaning interval in second.
> cleaning_interval       5
> 
> Do I interpret this correctly as stating that only 2 segments
> (i.e. 16MB) can be cleaned in 5 seconds?

Exactly.

> What negative effects will I get if I increase the number
> of segments that can be cleaned at a time?

It will increase memory pressure, and may unstabilize system.  And,
some trouble is reported in this mainling list.  We know filesystem
corrution will occasionally occur due to a nilfs2 bug which comes to
surface under such condition; I'm actually dig into the problem now.

I recommend you to decrease cleaning_interval instead of increasing
nsegemnts_per_clean if you try to speed up the cleaner.
 
Cheers,
Ryusuke Konishi

Re: Questions regarding use of nilfs2 on SSDs

[Ingo Glöckner]

Dear Ryusuke Konishi,

thanks for your immediate reply!

>> - I have read somewhere
>>> http://www.ocztechnologyforum.com/forum/showpost.php?p=361424&postcount=10
>>   that using nilfs2 as the rootfs generates enormous I/O traffic
>>   (up to 45 GB write in 1-2 days!) If this is true, then it will
>>   eventually translate into reduced lifetime for SSDs.
>>   Can you confirm that the segment management of nilfs2
>>   is so write-heavy on the drive?
>>   If so, are there ways to avoid this?
> 
> Yes, this is true.  This is mainly because garbage collection creep
> around the partition until no old updates are found.

I have made a small test in order to reproduce this
phenomenon. I report the results here just to
demonstrate how drastic the effect is, knowing that
an improved cleanerd will likely behave differently.

- First I copied 4 GB of data to a fresh nilfs2 partition.
  Once the protection period expired, the cleanerd obviously
  reorganized the segments, which resulted in another
  4 GB of writes. After reorganization, no further writes
  due to cleanerd were observed.

- Then I created a snapshot with chcp ss and changed it
  back to a checkpoint using chcp cp.
  This caused the cleanerd to become active after the
  protection period and write another 4 GB once again!

- Finally I created an empty file on the nilfs2 partition
  using touch emptyfile.
  This too resulted in the cleanerd generating 4 GB write
  traffic after the protection period, effectively
  reorganizing the complete content of the partition.

Considering the limited lifetime of SSDs with respect to
writes, reorganizing the data *once* (in order to fill
segments more densely) is okay for me, but reorganizing
the whole data every time there is the tiniest change to the
file system (setting/unsetting a snapshot, creating an
empty file...) troubles me when it comes to SSDs.
It would be better if the GC would create stable segments.
At the moment even those segments that are already
completely filled with data are copied again and again.

But as you write, your colleague is currently improving
the GC, so I am confident that he will solve this problem!

In the meantime I will stick to your advice and avoid
the use of the GC if possible.

Thanks again for your help,
- Ingo