Ongoing data reorganization/defragmentation?

Ongoing data reorganization/defragmentation?

[norritt]

Ongoing data reorganization/defragmentation?

In my efforts to optimize an ext4 for usage on SSDs, I have found 
information on the web about ext4 I've not been able to verify.

 From what I've found out it's a good idea to:
- mount with noatime, discard (if drive supports trim)
- put /tmp and maybe /var on a tempfs
- set vm.swappiness=1, vm.vfs_cache_pressure=50 (to utilize RAM rather 
than the disk)

Another proposal found is to disable journaling what I personally won't do.

I also know from the ext4 dcumentation, that ext4 is written to support 
online defragmentation. I've read several rumours on forums and blogs 
that indicate ext4 is doing some kind of "ongoing data reorganisations 
or defragmentation" in the background, in case the mechanisms to prevent 
fragmentation in the first place failed for some reason (i.e. when a fs 
is running out of free space). I did not find any credible information 
to back up these claims, that ext4 is actively reorganizing data, but I 
didn't find any information that prove these claims wrong either.

So my question is, does ext4 reorganize data when it encounters 
fragmented files? If so, is there a way to disable that feature which 
would not do any good on SSDs? Is there anything else to consider when 
using ext4 on SSDs, except the things I mentioned above?

Please cc me since I'm not subscribed, thanks!

Regards Nor

Re: Ongoing data reorganization/defragmentation?

[Andreas Dilger]

On 2012-09-12, at 10:36, norritt@t-online.de wrote:

> Ongoing data reorganization/defragmentation?
> 
> I also know from the ext4 dcumentation, that ext4 is written to support online defragmentation. I've read several rumours on forums and blogs that indicate ext4 is doing some kind of "ongoing data reorganisations or defragmentation" in the background, in case the mechanisms to prevent fragmentation in the first place failed for some reason (i.e. when a fs is running out of free space). I did not find any credible information to back up these claims, that ext4 is actively reorganizing data, but I didn't find any information that prove these claims wrong either.
> 
> So my question is, does ext4 reorganize data when it encounters fragmented files? If so, is there a way to disable that feature which would not do any good on SSDs? Is there anything else to consider when using ext4 on SSDs, except the things I mentioned above?

No, there is no such background reorganization happening of allocated files. With "delalloc" there is batching of block allocations to try and optimize the initial block selection, but it is static after this time. 

Cheers, Andreas

Re: Ongoing data reorganization/defragmentation?

[Lukáš Czerner]

On Wed, 12 Sep 2012, norritt@t-online.de wrote:

> Date: Wed, 12 Sep 2012 18:36:24 +0200
> From: norritt@t-online.de
> To: linux-ext4@vger.kernel.org
> Subject: Ongoing data reorganization/defragmentation?
> 
> Ongoing data reorganization/defragmentation?
> 
> In my efforts to optimize an ext4 for usage on SSDs, I have found information
> on the web about ext4 I've not been able to verify.
> 
> From what I've found out it's a good idea to:
> - mount with noatime, discard (if drive supports trim)

The -o discard option might be tricky since some devices still
handles it quite badly and you can experience big performance drops
(http://people.redhat.com/lczerner/discard/ although it is somewhat
outdated already). But you might to try to see yourself if it does
not hurt your performance.

The other option would be to use batched discard with fstrim (see
fstrim from util-linux) and run it once per (day, week, month).

Thanks!
-Lukas

Re: Ongoing data reorganization/defragmentation?

[norritt]

On 12.09.2012 19:35, Andreas Dilger wrote:
> On 2012-09-12, at 10:36, norritt@t-online.de wrote:
>
>> Ongoing data reorganization/defragmentation?
>>
>> So my question is, does ext4 reorganize data when it encounters fragmented files? If so, is there a way to disable that feature which would not do any good on SSDs? Is there anything else to consider when using ext4 on SSDs, except the things I mentioned above?
> No, there is no such background reorganization happening of allocated files. With "delalloc" there is batching of block allocations to try and optimize the initial block selection, but it is static after this time.
>
> Cheers, Andreas
Hi Andreas,

thank you for the quick reply and clearing up the rumours about ext4. 
Its good to know for sure, that ext4 doesn't reorganize data, once it 
has been written to the disk.

Regards Nor

Re: Ongoing data reorganization/defragmentation?

[norritt]

On 12.09.2012 20:15, Lukáš Czerner wrote:
> On Wed, 12 Sep 2012, norritt@t-online.de wrote:
>
>  From what I've found out it's a good idea to:
> - mount with noatime, discard (if drive supports trim)
> The -o discard option might be tricky since some devices still
> handles it quite badly and you can experience big performance drops
> (http://people.redhat.com/lczerner/discard/ although it is somewhat
> outdated already). But you might to try to see yourself if it does
> not hurt your performance.
>
> The other option would be to use batched discard with fstrim (see
> fstrim from util-linux) and run it once per (day, week, month).
>
> Thanks!
> -Lukas
>

Hi  Lukáš,
thank you for pointing out the performance issue with the discard mount 
option, I hadn't read about that yet. As far is I know I'm using a quite 
special SSD (SanDisk, with a SanDisk controller), where most other SSDs 
on the market seem to be based on Marvell, Sandforce or Indilinx 
controllers. I saw that you were using SSDs from various vendors for 
your benchmarking tests. Was a model with SanDisk controller among the 
drives you tested? Would postmark data about disks with SanDisk 
controller be of interest to you?

Regards Nor
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html