Re: Howto switch off ext4's delayed allocation?

Re: Howto switch off ext4's delayed allocation?

[Tomasz Chmielewski]

> Just happend again to me.
> After the intel driver crashed my system, the source-file I was
> working on was empty. Fourtunatly eclipse has a history-log.

Never had your /etc/passwd and /etc/shadow zeroed? Lucky you...


> Isn't there a way to switch off the more "dangerous" optimizations in ext4?
> I already searched howto do this, but from what I've seen there was
> only discussion about providing a switch for disabling delayed
> allocation, but it was never done?

Documentation/filesystems/ext4.txt?


-- 
Tomasz Chmielewski
http://wpkg.org

Howto switch off ext4's delayed allocation?

[Clemens Eisserer]

Hi,

Every since switching to ext4, I suffer fromt he empty files problem
after crashes.
Since GEM/KMS was included, as well as the changes to the usb-serial
subsystem I experience crashes from time to time.

As far as I understand this is caused by delayed allocation.
I really enjoy the performance benefits of ext4, and I know
applications should be fixed - but having this problem seen with so
many apps I doubt it will ever happen. (Kwrite, bash itself, umtsmon,
...).

Is there any way to disable delayed allocation? (running 2.6.31rc8)

Thank you in advance, Clemens

Re: Howto switch off ext4's delayed allocation?

[Clemens Eisserer]

Hi,

Just happend again to me.
After the intel driver crashed my system, the source-file I was
working on was empty. Fourtunatly eclipse has a history-log.

Isn't there a way to switch off the more "dangerous" optimizations in ext4?
I already searched howto do this, but from what I've seen there was
only discussion about providing a switch for disabling delayed
allocation, but it was never done?

Thanks, Clemens

2009/9/9 Clemens Eisserer <linuxhippy@gmail.com>:
> Hi,
>
> Every since switching to ext4, I suffer fromt he empty files problem
> after crashes.
> Since GEM/KMS was included, as well as the changes to the usb-serial
> subsystem I experience crashes from time to time.
>
> As far as I understand this is caused by delayed allocation.
> I really enjoy the performance benefits of ext4, and I know
> applications should be fixed - but having this problem seen with so
> many apps I doubt it will ever happen. (Kwrite, bash itself, umtsmon,
> ...).
>
> Is there any way to disable delayed allocation? (running 2.6.31rc8)
>
> Thank you in advance, Clemens
>

Re: Howto switch off ext4's delayed allocation?

[Theodore Tso]

On Wed, Sep 16, 2009 at 08:32:12AM -0400, Clemens Eisserer wrote:
> 
> Just happend again to me.
> After the intel driver crashed my system, the source-file I was
> working on was empty. Fourtunatly eclipse has a history-log.

Sounds like eclipse is broken; most Unix editors (emacs, vim, etc.) do
correctly use fsync() when writing precious files, such people's
source files.  In addition, it must be writing the data through a
decidedly non-standard way.  Is it out-and-out deleting the file
before writing the new version of the file, or something insane like
that?  Ext4's hueristics are designed so that for the most common ways
that applications update files, the data gets forced to disk.  It
sounds like Eclipse is doing something decidedly non-standard.

If you can run a strace on eclipse while and then arrange to edit an
existing file and save it, it would be interesting to see what the
heck it is doing.

> Isn't there a way to switch off the more "dangerous" optimizations in ext4?

The mount option nodelalloc will turn off delayed allocation.  This is
documented in Documentation/filesystems/ext4.txt.

					- Ted

Re: Howto switch off ext4's delayed allocation?

[Clemens Eisserer]

Hi Theodore,

Thanks a lot for your work on linux filesystems through the years.

> Sounds like eclipse is broken; most Unix editors (emacs, vim, etc.) do
> correctly use fsync() when writing precious files, such people's
> source files.  In addition, it must be writing the data through a
> decidedly non-standard way.  Is it out-and-out deleting the file
> before writing the new version of the file, or something insane like
> that?  Ext4's hueristics are designed so that for the most common ways
> that applications update files, the data gets forced to disk.  It
> sounds like Eclipse is doing something decidedly non-standard.

I've seen zero-length files after crashes (mostly caused by the intel
2D driver, just updated from 2.8.0 to 2.8.99, hope its better now) by
files edited with kwrite, eclipse and I've even had empty
.bash_history files.
I understand these are application's faults, but for now I had more
troubles with this feature than the additional performance is worth
for me.
My machine is a laptop, where is quite common to suddenly loose power
(my battery is dead).

> If you can run a strace on eclipse while and then arrange to edit an
> existing file and save it, it would be interesting to see what the
> heck it is doing.
I'll have a look.


>> Isn't there a way to switch off the more "dangerous" optimizations in ext4?
>
> The mount option nodelalloc will turn off delayed allocation.  This is
> documented in Documentation/filesystems/ext4.txt.
Thanks :)

- Clemens

Re: Howto switch off ext4's delayed allocation?

[Theodore Tso]

On Wed, Sep 16, 2009 at 02:41:09PM -0400, Clemens Eisserer wrote:
> 
> > If you can run a strace on eclipse while and then arrange to edit an
> > existing file and save it, it would be interesting to see what the
> > heck it is doing.
> I'll have a look.

Thanks, but no need.  I did some investigating, and when I found that
kwrite was doing a truncate-and-rewrite, I did some more digging, and
found that we had a bug in our truncate hueristics.  I'll get this
fixed for 2.6.32, and it will be a candidate for the 2.6.31 stable
tree.

(This needs to get backported for ext3 as well.)

						 - Ted

commit dbc380bc2268bed9faa687b2ec1f98a7706b5909
Author: Theodore Ts'o <tytso@mit.edu>
Date:   Thu Sep 17 09:33:32 2009 -0400

    ext4: Fix the alloc on close after a truncate hueristic
    
    In an attempt to avoid doing an unneeded flush after opening a
    (previously non-existent) file with O_CREAT|O_TRUNC, the code only
    triggered the hueristic if ei->disksize was non-zero.  Turns out that
    the VFS doesn't call ->truncate() if the file doesn't exist, and
    ei->disksize is always zero even if the file previously existed.  So
    remove the test, since it isn't necessary and in fact disabled the
    hueristic.
    
    Thanks to Clemens Eisserer that he was seeing problems with files
    written using kwrite and eclipse after sudden crashes caused by a
    buggy Intel video driver.
    
    Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 9887a0c..4abd683 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3973,8 +3973,7 @@ void ext4_truncate(struct inode *inode)
 	if (!ext4_can_truncate(inode))
 		return;
 
-	if (ei->i_disksize && inode->i_size == 0 &&
-	    !test_opt(inode->i_sb, NO_AUTO_DA_ALLOC))
+	if (inode->i_size == 0 && !test_opt(inode->i_sb, NO_AUTO_DA_ALLOC))
 		ei->i_state |= EXT4_STATE_DA_ALLOC_CLOSE;
 
 	if (EXT4_I(inode)->i_flags & EXT4_EXTENTS_FL) {

Re: Howto switch off ext4's delayed allocation?

[Clemens Eisserer]

> Thanks, but no need.  I did some investigating, and when I found that
> kwrite was doing a truncate-and-rewrite, I did some more digging, and
> found that we had a bug in our truncate hueristics.  I'll get this
> fixed for 2.6.32, and it will be a candidate for the 2.6.31 stable
> tree.

Cool, thanks a lot :)

- Clemens