Re: NILFS2 on an Intel X25-M SSD

Re: NILFS2 on an Intel X25-M SSD

[Ryusuke Konishi]

Hi,
On Tue, 29 Dec 2009 20:48:47 +0100, "Carlos R. Mafra" wrote:
> Hi,
> 
> Thanks for your work with NILFS2 on the kernel. 
> 
> I've just bought an Intel 160 GB SSD drive and while I am waiting
> it to arrive I am reading about which filesystem is better to use
> with it.
> 
> As I was reading about it I found out about NILFS2 and I am almost 
> certain that I will try it on the SSD.
> 
> I plan to use the 160 GB as 10 GB to / and 150 GB to /home. As I am not sure
> grub will handle the NILFS2 on /boot, / will be ext4 and /home will be
> NILFS2.

Nilfs is applicable to / by putting nilfs2.ko in initrd, but I think
using nilfs for /home is a convenient choice.

> Is there any specific tweak in NILFS2 to use with an SSD? Would you
> reccomend using NILFS2 on a SSD right now?
>
> Is the issue mentioned here
> 
> https://www.nilfs.org/pipermail/users/2009-March/000514.html
> 
> about excessive writing in the GC already fixed?
> That is the most pressing fear I have about NILFS2 for the SSD.
>
> Regards,
> Carlos

Not yet, sorry.  I'm working on some performance optimization for high
speed drives espacially for SSD.  Revising GC is another priority, but
it would need some time.

The massive I/O by the current garbage collector may shorten the life
of SSD.  So, I don't recommend yet.

If you are not interested in snapshots, I think nilfs is not always a
good choice because high-end SSD drives are already doing lfs like
optimization internally.  Intel SSD exactly looks like this sort of
drives though I'm using nilfs on X25-M and X25-E ;)

> PS: If you would like to answer me with a Cc: to some mailing list
> so that others benefit from the information, feel free to do so if
> you wish.

Thanks,
Ryusuke Konishi

Re: NILFS2 on an Intel X25-M SSD

[Carlos R. Mafra]

Am 30.12.2009 19:46, schrieb Ryusuke Konishi:
> On Tue, 29 Dec 2009 20:48:47 +0100, "Carlos R. Mafra" wrote:
>> Is there any specific tweak in NILFS2 to use with an SSD? Would you
>> reccomend using NILFS2 on a SSD right now?
>>
>> Is the issue mentioned here
>>
>> https://www.nilfs.org/pipermail/users/2009-March/000514.html
>>
>> about excessive writing in the GC already fixed?
>> That is the most pressing fear I have about NILFS2 for the SSD.
> 
> Not yet, sorry.  I'm working on some performance optimization for high
> speed drives espacially for SSD.  Revising GC is another priority, but
> it would need some time.
> 
> The massive I/O by the current garbage collector may shorten the life
> of SSD.  So, I don't recommend yet.

Hmm, then I won't use NILFS2 on the SSD for now.

I am testing it in an external HD for the last two days to get a feeling,
and so far everything looks good and stable. It is a pity that the 
massive I/O problem kind of prevents putting it on the SSD.

Naively I would think that the GC will always create some I/O that
other filesystems don't have, by its very definition. Can you
quantify by how much the current GC is writing in excess and what would
be the acceptable numbers?

> If you are not interested in snapshots, I think nilfs is not always a
> good choice because high-end SSD drives are already doing lfs like
> optimization internally.  Intel SSD exactly looks like this sort of
> drives though I'm using nilfs on X25-M and X25-E ;)

The log-structured nature of the file system seemed a good thing for
the health of the SSD, yes. But the snapshot feature is something
interesting to have too and I was willing to have it :-)

Re: NILFS2 on an Intel X25-M SSD

[Ryusuke Konishi]

On Thu, 31 Dec 2009 10:47:18 +0100, "Carlos R. Mafra" wrote:
> Am 30.12.2009 19:46, schrieb Ryusuke Konishi:
> > On Tue, 29 Dec 2009 20:48:47 +0100, "Carlos R. Mafra" wrote:
> >> Is there any specific tweak in NILFS2 to use with an SSD? Would you
> >> reccomend using NILFS2 on a SSD right now?
> >>
> >> Is the issue mentioned here
> >>
> >> https://www.nilfs.org/pipermail/users/2009-March/000514.html
> >>
> >> about excessive writing in the GC already fixed?
> >> That is the most pressing fear I have about NILFS2 for the SSD.
> > 
> > Not yet, sorry.  I'm working on some performance optimization for high
> > speed drives espacially for SSD.  Revising GC is another priority, but
> > it would need some time.
> > 
> > The massive I/O by the current garbage collector may shorten the life
> > of SSD.  So, I don't recommend yet.
> 
> Hmm, then I won't use NILFS2 on the SSD for now.
> 
> I am testing it in an external HD for the last two days to get a feeling,
> and so far everything looks good and stable. It is a pity that the 
> massive I/O problem kind of prevents putting it on the SSD.

Definitely yes.

NILFS2 is pretty stable, but this issue is spoiling the goodness
especially for netbook users.
 
> Naively I would think that the GC will always create some I/O that
> other filesystems don't have, by its very definition. Can you
> quantify by how much the current GC is writing in excess and what would
> be the acceptable numbers?

The overhead is roughly measured as the number of copied (moved)
blocks per reclaimed segment.  The following patch will print the
ratio (number of copied blocks / cleaned blocks) into syslog.

A NILFS2 root of my laptop constantly marks 70~99%.

This is because the current GC regularly frees segments from older
ones without any attempt to minimize the above ratio; it doesn't
select target segments that way, and it doesn't delay reclamation even
when the partition has enough free space.

I don't know how much is acceptable, but at least, I think the GC
frequency should be suppressed to 0 while the partition has enough
free space, and the selection algorithm should be improved to give
priority to free segments containing less in-use blocks.


Regards,
Ryusuke Konishi
--
diff --git a/sbin/cleanerd/cleanerd.c b/sbin/cleanerd/cleanerd.c
index f36dfce..6a35f15 100644
--- a/sbin/cleanerd/cleanerd.c
+++ b/sbin/cleanerd/cleanerd.c
@@ -921,6 +921,8 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 {
 	struct nilfs_vector *vdescv, *bdescv, *periodv, *vblocknrv;
 	ssize_t n, ret = -1;
+	size_t nr_vblocks, nr_pblocks, nr_live_vblocks, nr_live_pblocks;
+	size_t nr_blocks, nr_live_blocks;
 	int i;
 
 	if (nsegs == 0)
@@ -953,10 +955,14 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 	if (ret < 0)
 		goto out_lock;
 
+	nr_vblocks = nilfs_vector_get_size(vdescv);
+
 	ret = nilfs_cleanerd_toss_vdescs(cleanerd, vdescv, periodv, vblocknrv);
 	if (ret < 0)
 		goto out_lock;
 
+	nr_live_vblocks = nilfs_vector_get_size(vdescv);
+
 	nilfs_vector_sort(vdescv, nilfs_comp_vdesc_blocknr);
 	nilfs_cleanerd_unify_period(cleanerd, periodv);
 
@@ -964,10 +970,17 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 	if (ret < 0)
 		goto out_lock;
 
+	nr_pblocks = nilfs_vector_get_size(bdescv);
+
 	ret = nilfs_cleanerd_toss_bdescs(cleanerd, bdescv);
 	if (ret < 0)
 		goto out_lock;
 
+	nr_live_pblocks = nilfs_vector_get_size(bdescv);
+
+	nr_blocks = nr_vblocks + nr_pblocks;
+	nr_live_blocks = nr_live_vblocks + nr_live_pblocks;
+
 	ret = nilfs_clean_segments(cleanerd->c_nilfs,
 				   nilfs_vector_get_data(vdescv),
 				   nilfs_vector_get_size(vdescv),
@@ -983,8 +996,13 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 	} else {
 		if (n > 0) {
 			for (i = 0; i < n; i++)
-				syslog(LOG_DEBUG, "segment %llu cleaned",
+				syslog(LOG_INFO, "segment %llu cleaned",
 				       (unsigned long long)segnums[i]);
+			syslog(LOG_INFO,
+			       "copied blocks / total blocks : %zu / %zu "
+			       "(%3.1f%%)",
+			       nr_live_blocks, nr_blocks,
+			       nr_live_blocks * 100.0 / nr_blocks);
 		} else {
 			syslog(LOG_DEBUG, "no segments cleaned");
 		}

Re: NILFS2 on an Intel X25-M SSD

[admin-/LHdS3kC8BfYtjvyW6yDsg]

Well, I am actually mounting nilfs partitions without cleaner and starting it manually when I have not enough space, maybe this could be a temporary solution.

-original message-
Subject: Re: NILFS2 on an Intel X25-M SSD
From: Ryusuke Konishi <ryusuke-sG5X7nlA6pw@public.gmane.org>
Date: 02/01/2010 18:24

On Thu, 31 Dec 2009 10:47:18 +0100, "Carlos R. Mafra" wrote:
> Am 30.12.2009 19:46, schrieb Ryusuke Konishi:
> > On Tue, 29 Dec 2009 20:48:47 +0100, "Carlos R. Mafra" wrote:
> >> Is there any specific tweak in NILFS2 to use with an SSD? Would you
> >> reccomend using NILFS2 on a SSD right now?
> >>
> >> Is the issue mentioned here
> >>
> >> https://www.nilfs.org/pipermail/users/2009-March/000514.html
> >>
> >> about excessive writing in the GC already fixed?
> >> That is the most pressing fear I have about NILFS2 for the SSD.
> > 
> > Not yet, sorry.  I'm working on some performance optimization for high
> > speed drives espacially for SSD.  Revising GC is another priority, but
> > it would need some time.
> > 
> > The massive I/O by the current garbage collector may shorten the life
> > of SSD.  So, I don't recommend yet.
> 
> Hmm, then I won't use NILFS2 on the SSD for now.
> 
> I am testing it in an external HD for the last two days to get a feeling,
> and so far everything looks good and stable. It is a pity that the 
> massive I/O problem kind of prevents putting it on the SSD.

Definitely yes.

NILFS2 is pretty stable, but this issue is spoiling the goodness
especially for netbook users.
 
> Naively I would think that the GC will always create some I/O that
> other filesystems don't have, by its very definition. Can you
> quantify by how much the current GC is writing in excess and what would
> be the acceptable numbers?

The overhead is roughly measured as the number of copied (moved)
blocks per reclaimed segment.  The following patch will print the
ratio (number of copied blocks / cleaned blocks) into syslog.

A NILFS2 root of my laptop constantly marks 70~99%.

This is because the current GC regularly frees segments from older
ones without any attempt to minimize the above ratio; it doesn't
select target segments that way, and it doesn't delay reclamation even
when the partition has enough free space.

I don't know how much is acceptable, but at least, I think the GC
frequency should be suppressed to 0 while the partition has enough
free space, and the selection algorithm should be improved to give
priority to free segments containing less in-use blocks.


Regards,
Ryusuke Konishi
--
diff --git a/sbin/cleanerd/cleanerd.c b/sbin/cleanerd/cleanerd.c
index f36dfce..6a35f15 100644
--- a/sbin/cleanerd/cleanerd.c
+++ b/sbin/cleanerd/cleanerd.c
@@ -921,6 +921,8 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 {
 	struct nilfs_vector *vdescv, *bdescv, *periodv, *vblocknrv;
 	ssize_t n, ret = -1;
+	size_t nr_vblocks, nr_pblocks, nr_live_vblocks, nr_live_pblocks;
+	size_t nr_blocks, nr_live_blocks;
 	int i;
 
 	if (nsegs == 0)
@@ -953,10 +955,14 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 	if (ret < 0)
 		goto out_lock;
 
+	nr_vblocks = nilfs_vector_get_size(vdescv);
+
 	ret = nilfs_cleanerd_toss_vdescs(cleanerd, vdescv, periodv, vblocknrv);
 	if (ret < 0)
 		goto out_lock;
 
+	nr_live_vblocks = nilfs_vector_get_size(vdescv);
+
 	nilfs_vector_sort(vdescv, nilfs_comp_vdesc_blocknr);
 	nilfs_cleanerd_unify_period(cleanerd, periodv);
 
@@ -964,10 +970,17 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 	if (ret < 0)
 		goto out_lock;
 
+	nr_pblocks = nilfs_vector_get_size(bdescv);
+
 	ret = nilfs_cleanerd_toss_bdescs(cleanerd, bdescv);
 	if (ret < 0)
 		goto out_lock;
 
+	nr_live_pblocks = nilfs_vector_get_size(bdescv);
+
+	nr_blocks = nr_vblocks + nr_pblocks;
+	nr_live_blocks = nr_live_vblocks + nr_live_pblocks;
+
 	ret = nilfs_clean_segments(cleanerd->c_nilfs,
 				   nilfs_vector_get_data(vdescv),
 				   nilfs_vector_get_size(vdescv),
@@ -983,8 +996,13 @@ static ssize_t nilfs_cleanerd_clean_segments(struct nilfs_cleanerd *cleanerd,
 	} else {
 		if (n > 0) {
 			for (i = 0; i < n; i++)
-				syslog(LOG_DEBUG, "segment %llu cleaned",
+				syslog(LOG_INFO, "segment %llu cleaned",
 				       (unsigned long long)segnums[i]);
+			syslog(LOG_INFO,
+			       "copied blocks / total blocks : %zu / %zu "
+			       "(%3.1f%%)",
+			       nr_live_blocks, nr_blocks,
+			       nr_live_blocks * 100.0 / nr_blocks);
 		} else {
 			syslog(LOG_DEBUG, "no segments cleaned");
 		}
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: NILFS2 on an Intel X25-M SSD

[Carlos R. Mafra]

On So  3.Jan'10 at  2:24:25 +0900, Ryusuke Konishi wrote:
> On Thu, 31 Dec 2009 10:47:18 +0100, "Carlos R. Mafra" wrote:
>  
> > Naively I would think that the GC will always create some I/O that
> > other filesystems don't have, by its very definition. Can you
> > quantify by how much the current GC is writing in excess and what would
> > be the acceptable numbers?
> 
> The overhead is roughly measured as the number of copied (moved)
> blocks per reclaimed segment.  The following patch will print the
> ratio (number of copied blocks / cleaned blocks) into syslog.

Thanks for the reply and I will test your patch to see the numbers
in the following days.

> I don't know how much is acceptable, but at least, I think the GC
> frequency should be suppressed to 0 while the partition has enough
> free space, and the selection algorithm should be improved to give
> priority to free segments containing less in-use blocks.

Good that you mentioned it, because that is something I found a bit
unusual about NILFS2. 

Initially I copied 100 GB into the NILFS2 partition in the external
hard disk via a backup script using rsync. Then I deleted 10 GB from
my $HOME and reran the script to update the backup on the NILFS2
partition. The size reported by 'df' initially increased and I had
to wait around 10 hours to notice it decrease, but the decrease rate
was soo slow (around 100 MB/hour or so) that I did not wait anymore
and disconnected the hard disk. 

The partition has 150 GB, so perhaps NILFS2 thinks that there 
is "enough free space" and did not care too much about 
deleting the old data. But I thought that what would matter
most is the protection_period from /etc/nilfs_cleanerd.conf (3600)...

I must have understood it incorrectly, because I thought that after
3600 secs after deleting those 10 GB the partition would shrink to 90 GB
"quickly" (like in 1-2 hours). Than after those 1-2 hours my old
data would be forever gone.

So that was my odd feeling about NILFS2. I would not necessarily
say that it is a "problem" because if one is not using an external
hard disk then eventually there will be enough time to clean things
up, but it feels like it could be better. I have even read some
emails from the archives where people had problems with a full
partition even though they knew they had free space.

To test it a bit more, I increased nsegments_per_clean to 12, 
decreased protection_period to 60 and cleaning_interval to 2,
but now the cleanerd was using like 15% of CPU and it was not
cleaning the data much faster (I don't remember the numbers).

So what is more important to NILFS2, the notion of "free space
left" or the protection_period setting?

Re: NILFS2 on an Intel X25-M SSD

[Ryusuke Konishi]

On Sat, 2 Jan 2010 21:44:17 +0100, "Carlos R. Mafra" wrote:
> The partition has 150 GB, so perhaps NILFS2 thinks that there 
> is "enough free space" and did not care too much about 
> deleting the old data. But I thought that what would matter
> most is the protection_period from /etc/nilfs_cleanerd.conf (3600)...

No, no, I meant the current GC of NILFS2 does not see how much free
space is left.  It just monotonically works unless user changes GC
parameters with a HUP signal.
 
> I must have understood it incorrectly, because I thought that after
> 3600 secs after deleting those 10 GB the partition would shrink to 90 GB
> "quickly" (like in 1-2 hours). Than after those 1-2 hours my old
> data would be forever gone.
> 
> So that was my odd feeling about NILFS2. I would not necessarily
> say that it is a "problem" because if one is not using an external
> hard disk then eventually there will be enough time to clean things
> up, but it feels like it could be better. I have even read some
> emails from the archives where people had problems with a full
> partition even though they knew they had free space.
> 
> To test it a bit more, I increased nsegments_per_clean to 12, 
> decreased protection_period to 60 and cleaning_interval to 2,
> but now the cleanerd was using like 15% of CPU and it was not
> cleaning the data much faster (I don't remember the numbers).
> 
> So what is more important to NILFS2, the notion of "free space
> left" or the protection_period setting?

"free space left" should be cared, but the current GC does not as I
mentioned above.

So, the protection_period setting is the most important.  The next is
GC speed given by cleaning_interval and nsegments_per_clean (tweaking
cleaning_interval seems preferable because increasing
nsegments_per_clean causes a burst of memory allocation).

Cheers,
Ryusuke Konishi