* sparse file handling bug in XFS
@ 2011-06-16 14:49 Sean Noonan
2011-06-16 16:08 ` Eric Sandeen
0 siblings, 1 reply; 6+ messages in thread
From: Sean Noonan @ 2011-06-16 14:49 UTC (permalink / raw)
To: 'linux-xfs@oss.sgi.com'
Cc: Martin Bligh, Trammell Hudson, Ian Baum, Stephen Degler
Sparse files do not stay sparse.
Here's the simplest test case I've got so far. I don't think it can get much simpler than this.
This did not exist in 2.6.36. It appeared by 2.6.38-rc8. It continued into 2.6.38.2. It continues to exist on 3.0.0-rc3.
# for x in gogo xfs; do date | dd of=sparse-file bs=1k seek=4096; stat sparse-file; done
0+1 records in
0+1 records out
29 bytes (29 B) copied, 0.00151976 s, 19.1 kB/s
File: `sparse-file'
Size: 4194333 Blocks: 8 IO Block: 4096 regular file
Device: 6809h/26633d Inode: 12153368 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-06-14 13:56:43.826251881 +0000
Modify: 2011-06-14 13:56:43.826251881 +0000
Change: 2011-06-14 13:56:43.826251881 +0000
0+1 records in
0+1 records out
29 bytes (29 B) copied, 0.00152104 s, 19.1 kB/s
File: `sparse-file'
Size: 4194333 Blocks: 8192 IO Block: 4096 regular file
Device: 6809h/26633d Inode: 12153368 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-06-14 13:56:43.826251881 +0000
Modify: 2011-06-14 13:56:43.834252062 +0000
Change: 2011-06-14 13:56:43.834252062 +0000
For more fun, it even happens if we explicitly remove the file in between! (It looks like this is just because the recently freed inode is reused and not properly initialized.)
FILE=sparse-file-99; for x in gogo xfs; do rm -f $FILE; date | dd of=$FILE bs=1k seek=4096; stat $FILE; done
0+1 records in
0+1 records out
29 bytes (29 B) copied, 0.00151841 s, 19.1 kB/s
File: `sparse-file-99'
Size: 4194333 Blocks: 8 IO Block: 4096 regular file
Device: 6809h/26633d Inode: 12153371 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-06-14 14:03:37.933888145 +0000
Modify: 2011-06-14 14:03:37.937888543 +0000
Change: 2011-06-14 14:03:37.937888543 +0000
0+1 records in
0+1 records out
29 bytes (29 B) copied, 2.1681e-05 s, 1.3 MB/s
File: `sparse-file-99'
Size: 4194333 Blocks: 8192 IO Block: 4096 regular file
Device: 6809h/26633d Inode: 12153371 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-06-14 14:03:37.941888722 +0000
Modify: 2011-06-14 14:03:37.941888722 +0000
Change: 2011-06-14 14:03:37.941888722 +0000
Here is a run on ext3. Notice how blocks stays at 16.
# for x in gogo xfs; do date | dd of=sparse-file bs=1k seek=4096; stat sparse-file; done
0+1 records in
0+1 records out
29 bytes (29 B) copied, 0.00603038 s, 4.8 kB/s
File: `sparse-file'
Size: 4194333 Blocks: 16 IO Block: 4096 regular file
Device: 6805h/26629d Inode: 24 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-06-16 14:48:32.480750867 +0000
Modify: 2011-06-16 14:48:32.480750867 +0000
Change: 2011-06-16 14:48:32.480750867 +0000
0+1 records in
0+1 records out
29 bytes (29 B) copied, 0.00125449 s, 23.1 kB/s
File: `sparse-file'
Size: 4194333 Blocks: 16 IO Block: 4096 regular file
Device: 6805h/26629d Inode: 24 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-06-16 14:48:32.480750867 +0000
Modify: 2011-06-16 14:48:32.492751347 +0000
Change: 2011-06-16 14:48:32.492751347 +0000
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: sparse file handling bug in XFS
2011-06-16 14:49 sparse file handling bug in XFS Sean Noonan
@ 2011-06-16 16:08 ` Eric Sandeen
2011-06-16 17:33 ` Eric Sandeen
0 siblings, 1 reply; 6+ messages in thread
From: Eric Sandeen @ 2011-06-16 16:08 UTC (permalink / raw)
To: Sean Noonan
Cc: 'linux-xfs@oss.sgi.com', Martin Bligh, Stephen Degler,
Ian Baum, Trammell Hudson
On 6/16/11 9:49 AM, Sean Noonan wrote:
> Sparse files do not stay sparse.
> Here's the simplest test case I've got so far. I don't think it can get much simpler than this.
>
> This did not exist in 2.6.36. It appeared by 2.6.38-rc8. It continued into 2.6.38.2. It continues to exist on 3.0.0-rc3.
>
> # for x in gogo xfs; do date | dd of=sparse-file bs=1k seek=4096; stat sparse-file; done
Funky; if we do xfs_bmap, it shows the right nr of blocks allocated:
# for x in gogo xfs; do date | dd of=sparse-file bs=1k seek=4096; stat sparse-file | grep Blocks; xfs_bmap -v sparse-file; done
Size: 4194333 Blocks: 8 IO Block: 4096 regular file
sparse-file:
EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL
0: [0..8191]: hole 8192
1: [8192..8199]: 450475168..450475175 2 (18736320..18736327) 8
Size: 4194333 Blocks: 8192 IO Block: 4096 regular file
sparse-file:
EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL
0: [0..8191]: hole 8192
1: [8192..8199]: 459367952..459367959 2 (27629104..27629111) 8
And if we unmount & remount it's right again:
# stat sparse-file | grep Blocks
Size: 4194333 Blocks: 8 IO Block: 4096 regular file
so they do remain sparse, but stat tells us the wrong thing. I think it has
to do with the count of delayed blocks but I'll try to look into it.
-Eric
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: sparse file handling bug in XFS
2011-06-16 16:08 ` Eric Sandeen
@ 2011-06-16 17:33 ` Eric Sandeen
2011-06-16 17:57 ` Christoph Hellwig
0 siblings, 1 reply; 6+ messages in thread
From: Eric Sandeen @ 2011-06-16 17:33 UTC (permalink / raw)
To: Sean Noonan
Cc: 'linux-xfs@oss.sgi.com', Martin Bligh, Trammell Hudson,
Ian Baum, Stephen Degler
On 6/16/11 11:08 AM, Eric Sandeen wrote:
> On 6/16/11 9:49 AM, Sean Noonan wrote:
>> Sparse files do not stay sparse.
>> Here's the simplest test case I've got so far. I don't think it can get much simpler than this.
>>
>> This did not exist in 2.6.36. It appeared by 2.6.38-rc8. It continued into 2.6.38.2. It continues to exist on 3.0.0-rc3.
>>
>> # for x in gogo xfs; do date | dd of=sparse-file bs=1k seek=4096; stat sparse-file; done
>
> Funky; if we do xfs_bmap, it shows the right nr of blocks allocated:
>
> # for x in gogo xfs; do date | dd of=sparse-file bs=1k seek=4096; stat sparse-file | grep Blocks; xfs_bmap -v sparse-file; done
> Size: 4194333 Blocks: 8 IO Block: 4096 regular file
> sparse-file:
> EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL
> 0: [0..8191]: hole 8192
> 1: [8192..8199]: 450475168..450475175 2 (18736320..18736327) 8
>
> Size: 4194333 Blocks: 8192 IO Block: 4096 regular file
> sparse-file:
> EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL
> 0: [0..8191]: hole 8192
> 1: [8192..8199]: 459367952..459367959 2 (27629104..27629111) 8
>
>
> And if we unmount & remount it's right again:
>
> # stat sparse-file | grep Blocks
> Size: 4194333 Blocks: 8 IO Block: 4096 regular file
>
> so they do remain sparse, but stat tells us the wrong thing. I think it has
> to do with the count of delayed blocks but I'll try to look into it.
Actually this looks like it's a result of
6e857567dbbfe14dd6cc3f7414671b047b1ff5c7 xfs: don't truncate prealloc from frequently accessed inodes
I thought Dave's patch from the "Re: drastic changes to allocsize semantics in or around 2.6.38?"
thread would fix it, but it doesn't seem to. Here it is anyway ;)
xfs: clear inode dirty release flag when recycling it
From: Dave Chinner <dchinner@redhat.com>
The state used to track dirty inode release calls is not reset when
an inode is reallocated and reused from the reclaimable state. This
leads to specualtive preallocation not being truncated away in the
expected manner for local files until the inode is subsequently
truncated, freed or cycles out of the cache.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
fs/xfs/xfs_iget.c | 7 +++++++
1 files changed, 7 insertions(+), 0 deletions(-)
diff --git a/fs/xfs/xfs_iget.c b/fs/xfs/xfs_iget.c
index cb9b6d1..e75e757 100644
--- a/fs/xfs/xfs_iget.c
+++ b/fs/xfs/xfs_iget.c
@@ -241,6 +241,13 @@ xfs_iget_cache_hit(
*/
ip->i_flags |= XFS_IRECLAIM;
+ /*
+ * clear the dirty release state as we are now effectively a
+ * new inode and so we need to treat speculative preallocation
+ * accordingly.
+ */
+ ip->i_flags &= ~XFS_IDIRTY_RELEASE;
+
spin_unlock(&ip->i_flags_lock);
rcu_read_unlock();
-Eric
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: sparse file handling bug in XFS
2011-06-16 17:33 ` Eric Sandeen
@ 2011-06-16 17:57 ` Christoph Hellwig
2011-06-16 21:08 ` Eric Sandeen
0 siblings, 1 reply; 6+ messages in thread
From: Christoph Hellwig @ 2011-06-16 17:57 UTC (permalink / raw)
To: Eric Sandeen
Cc: Trammell Hudson, Sean Noonan, Martin Bligh, Ian Baum,
Stephen Degler, 'linux-xfs@oss.sgi.com'
On Thu, Jun 16, 2011 at 12:33:49PM -0500, Eric Sandeen wrote:
> Actually this looks like it's a result of
>
> 6e857567dbbfe14dd6cc3f7414671b047b1ff5c7 xfs: don't truncate prealloc from frequently accessed inodes
>
> I thought Dave's patch from the "Re: drastic changes to allocsize semantics in or around 2.6.38?"
> thread would fix it, but it doesn't seem to. Here it is anyway ;)
It should fix the thing about the preallocation staying when removing
and recreating the file. Keeping pre-allocate blocks around otherwise
is a considered a feature.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: sparse file handling bug in XFS
2011-06-16 17:57 ` Christoph Hellwig
@ 2011-06-16 21:08 ` Eric Sandeen
2011-06-19 22:27 ` Dave Chinner
0 siblings, 1 reply; 6+ messages in thread
From: Eric Sandeen @ 2011-06-16 21:08 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Trammell Hudson, Sean Noonan, Martin Bligh, Ian Baum,
Stephen Degler, 'linux-xfs@oss.sgi.com'
On 6/16/11 12:57 PM, Christoph Hellwig wrote:
> On Thu, Jun 16, 2011 at 12:33:49PM -0500, Eric Sandeen wrote:
>> Actually this looks like it's a result of
>>
>> 6e857567dbbfe14dd6cc3f7414671b047b1ff5c7 xfs: don't truncate prealloc from frequently accessed inodes
>>
>> I thought Dave's patch from the "Re: drastic changes to allocsize semantics in or around 2.6.38?"
>> thread would fix it, but it doesn't seem to. Here it is anyway ;)
>
> It should fix the thing about the preallocation staying when removing
> and recreating the file. Keeping pre-allocate blocks around otherwise
> is a considered a feature.
It strikes me as odd that truncating & rewriting would lead to
different preallocation though .. I'll let Dave chime in.
Thanks,
-Eric
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: sparse file handling bug in XFS
2011-06-16 21:08 ` Eric Sandeen
@ 2011-06-19 22:27 ` Dave Chinner
0 siblings, 0 replies; 6+ messages in thread
From: Dave Chinner @ 2011-06-19 22:27 UTC (permalink / raw)
To: Eric Sandeen
Cc: Trammell Hudson, Sean Noonan, Martin Bligh, Ian Baum,
Stephen Degler, Christoph Hellwig,
'linux-xfs@oss.sgi.com'
On Thu, Jun 16, 2011 at 04:08:07PM -0500, Eric Sandeen wrote:
> On 6/16/11 12:57 PM, Christoph Hellwig wrote:
> > On Thu, Jun 16, 2011 at 12:33:49PM -0500, Eric Sandeen wrote:
> >> Actually this looks like it's a result of
> >>
> >> 6e857567dbbfe14dd6cc3f7414671b047b1ff5c7 xfs: don't truncate prealloc from frequently accessed inodes
> >>
> >> I thought Dave's patch from the "Re: drastic changes to allocsize semantics in or around 2.6.38?"
> >> thread would fix it, but it doesn't seem to. Here it is anyway ;)
> >
> > It should fix the thing about the preallocation staying when removing
> > and recreating the file. Keeping pre-allocate blocks around otherwise
> > is a considered a feature.
>
> It strikes me as odd that truncating & rewriting would lead to
> different preallocation though .. I'll let Dave chime in.
Yeah, it's an oversight - we should reset the XFS_IDIRTY_RELEASE
flag when we set the XFS_ITRUNCATED flag. We remove the
preallocation beyond EOF when we truncate, so we should also trash
the state that tries to preserve it at the same time....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-06-19 22:27 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-16 14:49 sparse file handling bug in XFS Sean Noonan
2011-06-16 16:08 ` Eric Sandeen
2011-06-16 17:33 ` Eric Sandeen
2011-06-16 17:57 ` Christoph Hellwig
2011-06-16 21:08 ` Eric Sandeen
2011-06-19 22:27 ` Dave Chinner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox