* Strange hole creation behavior
@ 2014-04-11 17:13 Pádraig Brady
2014-04-11 20:43 ` Brian Foster
0 siblings, 1 reply; 4+ messages in thread
From: Pádraig Brady @ 2014-04-11 17:13 UTC (permalink / raw)
To: xfs-oss; +Cc: Ondřej Vašík
So this coreutils test is failing on XFS:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/dd/sparse.sh;h=06efc7017
Specifically the last hole check on line 66.
In summary what's happening is that a write(1MiB), lseek(1MiB), write(1MiB)
creates only a 64KiB hole. Is that expected?
Now a 1MiB hole is supported using truncate:
dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock
truncate -s+1M file.in
dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock conv=notrunc oflag=append
$ du -k file.in
2048 file.in
But when trying to create the 1MiB hole with dd (lseek) it fails?
# Create 3MiB input file file
$ dd if=/dev/urandom of=file.in bs=1M count=3 iflag=fullblock
$ dd if=/dev/zero of=file.in bs=1M count=1 seek=1 conv=notrunc
$ du -k file.in
3072 file.in
# Convert to 1MiB hole doesn't work :(
$ dd if=file.in of=file.out bs=1M conv=sparse
$ du -k file.out
3008 file.out
# Again with syscall details:
$ strace -e write,lseek dd if=file.in of=file.out bs=1M conv=sparse
write(1, "...", 1048576) = 1048576
lseek(1, 1048576, SEEK_CUR) = 2097152
write(1, "...", 1048576) = 1048576
So it seems that the lseeks are treated differently to the truncate
that was done in the first example, which is surprising.
If we look at the file layout we can see the hole is
only at the last 64KiB of the middle 1MiB of zeros,
rather than for the whole middle 1MiB as in the first example??
$ filefrag -v file.out
Filesystem type is: 58465342
File size of file.out is 3145728 (768 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 495: 31271.. 31766: 496:
1: 512.. 767: 31783.. 32038: 256: 31767: eof
thanks,
Pádraig.
Versions etc. in case useful
$ uname -a
Linux tp2 3.12.6-300.fc20.x86_64 #1 SMP Mon Dec 23 16:44:31 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
$ xfs_info .
meta-data=/dev/loop2 isize=256 agcount=4, agsize=65536 blks
= sectsz=512 attr=2
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: Strange hole creation behavior 2014-04-11 17:13 Strange hole creation behavior Pádraig Brady @ 2014-04-11 20:43 ` Brian Foster 2014-04-11 22:58 ` Pádraig Brady 0 siblings, 1 reply; 4+ messages in thread From: Brian Foster @ 2014-04-11 20:43 UTC (permalink / raw) To: Pádraig Brady; +Cc: Ondřej Vašík, xfs-oss On Fri, Apr 11, 2014 at 06:13:59PM +0100, Pádraig Brady wrote: > So this coreutils test is failing on XFS: > http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/dd/sparse.sh;h=06efc7017 > Specifically the last hole check on line 66. > > In summary what's happening is that a write(1MiB), lseek(1MiB), write(1MiB) > creates only a 64KiB hole. Is that expected? > This is expected behavior due to speculative preallocation. An FAQ with regard to this behavior is pending, but see here for reference: http://oss.sgi.com/archives/xfs/2014-04/msg00083.html In that particular write(1MB), lseek(+1MB), write(1MB) workload, each write is preallocating some extra space beyond the current EOF. The seek then moves past that space, but the space doesn't go away. The subsequent writes will extend EOF. The previously preallocated space now resides in the middle of the file and can't be trimmed away when the file is closed. > Now a 1MiB hole is supported using truncate: > dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock > truncate -s+1M file.in > dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock conv=notrunc oflag=append > $ du -k file.in > 2048 file.in > This works simply because it is broken into multiple commands. When the first dd exits, the excess space is trimmed off (the file descriptor is closed). The subsequent truncate extends the file size without any extra space getting caught between the old and new EOF. You can confirm this by using the 'allocsize=4k' mount option to the XFS mount. If you wanted something more generic for the purpose of testing the coreutils functionality, you could also set the size of file.out in advance. E.g., with preallocation in effect: # dd if=file.in of=file.out bs=1M conv=sparse # xfs_bmap -v file.out file.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..3967]: 9773944..9777911 1 (9080..13047) 3968 1: [3968..4095]: hole 128 2: [4096..6143]: 9778040..9780087 1 (13176..15223) 2048 ... and then prevent preallocation by ensuring writes do not extend the file: # rm -f file.out # truncate --size=3M file.out # dd if=file.in of=file.out bs=1M conv=sparse,notrunc # xfs_bmap -v file.out file.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..2047]: 9773944..9775991 1 (9080..11127) 2048 1: [2048..4095]: hole 2048 2: [4096..6143]: 9778040..9780087 1 (13176..15223) 2048 Hope that helps. Brian > But when trying to create the 1MiB hole with dd (lseek) it fails? > > # Create 3MiB input file file > $ dd if=/dev/urandom of=file.in bs=1M count=3 iflag=fullblock > $ dd if=/dev/zero of=file.in bs=1M count=1 seek=1 conv=notrunc > $ du -k file.in > 3072 file.in > > # Convert to 1MiB hole doesn't work :( > $ dd if=file.in of=file.out bs=1M conv=sparse > $ du -k file.out > 3008 file.out > > # Again with syscall details: > $ strace -e write,lseek dd if=file.in of=file.out bs=1M conv=sparse > write(1, "...", 1048576) = 1048576 > lseek(1, 1048576, SEEK_CUR) = 2097152 > write(1, "...", 1048576) = 1048576 > > So it seems that the lseeks are treated differently to the truncate > that was done in the first example, which is surprising. > If we look at the file layout we can see the hole is > only at the last 64KiB of the middle 1MiB of zeros, > rather than for the whole middle 1MiB as in the first example?? > > $ filefrag -v file.out > Filesystem type is: 58465342 > File size of file.out is 3145728 (768 blocks of 4096 bytes) > ext: logical_offset: physical_offset: length: expected: flags: > 0: 0.. 495: 31271.. 31766: 496: > 1: 512.. 767: 31783.. 32038: 256: 31767: eof > > thanks, > Pádraig. > > Versions etc. in case useful > > $ uname -a > Linux tp2 3.12.6-300.fc20.x86_64 #1 SMP Mon Dec 23 16:44:31 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux > > $ xfs_info . > meta-data=/dev/loop2 isize=256 agcount=4, agsize=65536 blks > = sectsz=512 attr=2 > data = bsize=4096 blocks=262144, imaxpct=25 > = sunit=0 swidth=0 blks > naming =version 2 bsize=4096 ascii-ci=0 > log =internal bsize=4096 blocks=2560, version=2 > = sectsz=512 sunit=0 blks, lazy-count=1 > realtime =none extsz=4096 blocks=0, rtextents=0 > > _______________________________________________ > xfs mailing list > xfs@oss.sgi.com > http://oss.sgi.com/mailman/listinfo/xfs _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Strange hole creation behavior 2014-04-11 20:43 ` Brian Foster @ 2014-04-11 22:58 ` Pádraig Brady 2014-04-11 23:05 ` Eric Sandeen 0 siblings, 1 reply; 4+ messages in thread From: Pádraig Brady @ 2014-04-11 22:58 UTC (permalink / raw) To: Brian Foster; +Cc: Coreutils, Ondřej Vašík, xfs-oss [-- Attachment #1: Type: text/plain, Size: 5311 bytes --] On 04/11/2014 09:43 PM, Brian Foster wrote: > On Fri, Apr 11, 2014 at 06:13:59PM +0100, Pádraig Brady wrote: >> So this coreutils test is failing on XFS: >> http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/dd/sparse.sh;h=06efc7017 >> Specifically the last hole check on line 66. >> >> In summary what's happening is that a write(1MiB), lseek(1MiB), write(1MiB) >> creates only a 64KiB hole. Is that expected? >> > > This is expected behavior due to speculative preallocation. An FAQ with > regard to this behavior is pending, but see here for reference: > > http://oss.sgi.com/archives/xfs/2014-04/msg00083.html > > In that particular write(1MB), lseek(+1MB), write(1MB) workload, each > write is preallocating some extra space beyond the current EOF. The seek > then moves past that space, but the space doesn't go away. The > subsequent writes will extend EOF. The previously preallocated space now > resides in the middle of the file and can't be trimmed away when the > file is closed. > >> Now a 1MiB hole is supported using truncate: >> dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock >> truncate -s+1M file.in >> dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock conv=notrunc oflag=append >> $ du -k file.in >> 2048 file.in >> > > This works simply because it is broken into multiple commands. When the > first dd exits, the excess space is trimmed off (the file descriptor is > closed). The subsequent truncate extends the file size without any > extra space getting caught between the old and new EOF. > > You can confirm this by using the 'allocsize=4k' mount option to the XFS > mount. If you wanted something more generic for the purpose of testing > the coreutils functionality, you could also set the size of file.out in > advance. E.g., with preallocation in effect: > > # dd if=file.in of=file.out bs=1M conv=sparse > # xfs_bmap -v file.out > file.out: > EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL > 0: [0..3967]: 9773944..9777911 1 (9080..13047) 3968 > 1: [3968..4095]: hole 128 > 2: [4096..6143]: 9778040..9780087 1 (13176..15223) 2048 > > ... and then prevent preallocation by ensuring writes do not extend the > file: > > # rm -f file.out > # truncate --size=3M file.out > # dd if=file.in of=file.out bs=1M conv=sparse,notrunc > # xfs_bmap -v file.out > file.out: > EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL > 0: [0..2047]: 9773944..9775991 1 (9080..11127) 2048 > 1: [2048..4095]: hole 2048 > 2: [4096..6143]: 9778040..9780087 1 (13176..15223) 2048 > > Hope that helps. Excellent info thanks. With that I can adjust the test so it passes (patch attached). So for reference this means that cp can no longer recreate holes <= 1MiB from source to dest (with the default XFS allocation size): $ cp --sparse=always file.in cp.out $ xfs_bmap -v !$ xfs_bmap -v cp.out cp.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..3967]: 219104..223071 0 (219104..223071) 3968 1: [3968..4095]: hole 128 2: [4096..6143]: 225720..227767 0 (225720..227767) 2048 $ xfs_bmap -v file.out file.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..2047]: 229816..231863 0 (229816..231863) 2048 1: [2048..4095]: hole 2048 2: [4096..6143]: 233912..235959 0 (233912..235959) 2048 $ cp file.out cp.out $ xfs_bmap -v cp.out cp.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..3967]: 250296..254263 0 (250296..254263) 3968 1: [3968..4095]: hole 128 2: [4096..6143]: 254392..256439 0 (254392..256439) 2048 Though if we bump up the hole size the representation is better: $ dd if=/dev/urandom of=bigfile.in bs=1M count=1 iflag=fullblock $ truncate -s+10M bigfile.in $ dd if=/dev/urandom of=bigfile.in bs=1M count=1 iflag=fullblock conv=notrunc oflag=append $ xfs_bmap -v bigfile.in bigfile.in: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..2047]: 231864..233911 0 (231864..233911) 2048 1: [2048..22527]: hole 20480 2: [22528..24575]: 256440..258487 0 (256440..258487) 2048 $ cp bigfile.in bigfile.out $ xfs_bmap -v bigfile.out bigfile.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..3967]: 260408..264375 0 (260408..264375) 3968 1: [3968..22527]: hole 18560 2: [22528..24575]: 264376..266423 0 (264376..266423) 2048 We could I suppose use FALLOC_FL_PUNCH_HOLE where available to cater for this case. I'll see whether this is worth adding. That can be used after the fact anyway: $ fallocate --dig-holes bigfile.out $ xfs_bmap -v bigfile.out bigfile.out: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL 0: [0..2047]: 260408..262455 0 (260408..262455) 2048 1: [2048..22527]: hole 20480 2: [22528..24575]: 264376..266423 0 (264376..266423) 2048 thanks, Pádraig. [-- Attachment #2: dd-sparse-xfs.patch --] [-- Type: text/x-patch, Size: 2042 bytes --] >From 7c03fe2c9f498bad7e40d29f2eb4573d23e102d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= <P@draigBrady.com> Date: Fri, 11 Apr 2014 23:44:13 +0100 Subject: [PATCH] tests: fix false dd conv=sparse failure on newer XFS * tests/dd/sparse.sh: When testing that a hole is created, use an existing sparse destination file, so that we're not write extending the file size, and thus avoiding speculative preallocation which can result in smaller holes than requested. Workaround suggested by Brian Foster --- THANKS.in | 1 + tests/dd/sparse.sh | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletions(-) diff --git a/THANKS.in b/THANKS.in index e7298ef..a92540a 100644 --- a/THANKS.in +++ b/THANKS.in @@ -95,6 +95,7 @@ Bjorn Helgaas helgaas@rsn.hp.com Bob McCracken kerouac@ravenet.com Branden Robinson branden@necrotic.deadbeast.net Brendan O'Dea bod@compusol.com.au +Brian Foster bfoster@redhat.com Brian Kimball bfk@footbag.org Brian M. Carlson sandals@crustytoothpaste.ath.cx Brian Silverman bsilverman@conceptxdesign.com diff --git a/tests/dd/sparse.sh b/tests/dd/sparse.sh index 06efc70..a7e90d2 100755 --- a/tests/dd/sparse.sh +++ b/tests/dd/sparse.sh @@ -61,8 +61,15 @@ if test $(kb_alloc file.in) -gt 3000; then dd if=file.in of=file.out bs=2M conv=sparse test 2500 -lt $(kb_alloc file.out) || fail=1 + # Note we recreate a sparse file first to avoid + # speculative preallocation seen in XFS, where a write() that + # extends a file can preallocate some extra space that + # a subsequent seek will not convert to a hole. + rm -f file.out + truncate --size=3M file.out + # Ensure that this 1MiB string of NULs *is* converted to a hole. - dd if=file.in of=file.out bs=1M conv=sparse + dd if=file.in of=file.out bs=1M conv=sparse,notrunc test $(kb_alloc file.out) -lt 2500 || fail=1 fi -- 1.7.7.6 [-- Attachment #3: Type: text/plain, Size: 121 bytes --] _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: Strange hole creation behavior 2014-04-11 22:58 ` Pádraig Brady @ 2014-04-11 23:05 ` Eric Sandeen 0 siblings, 0 replies; 4+ messages in thread From: Eric Sandeen @ 2014-04-11 23:05 UTC (permalink / raw) To: Pádraig Brady, Brian Foster Cc: xfs-oss, Coreutils, Ondřej Vašík On 4/11/14, 5:58 PM, Pádraig Brady wrote: > On 04/11/2014 09:43 PM, Brian Foster wrote: >> On Fri, Apr 11, 2014 at 06:13:59PM +0100, Pádraig Brady wrote: >>> So this coreutils test is failing on XFS: >>> http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=tests/dd/sparse.sh;h=06efc7017 >>> Specifically the last hole check on line 66. >>> >>> In summary what's happening is that a write(1MiB), lseek(1MiB), write(1MiB) >>> creates only a 64KiB hole. Is that expected? >>> >> >> This is expected behavior due to speculative preallocation. An FAQ with >> regard to this behavior is pending, but see here for reference: >> >> http://oss.sgi.com/archives/xfs/2014-04/msg00083.html >> >> In that particular write(1MB), lseek(+1MB), write(1MB) workload, each >> write is preallocating some extra space beyond the current EOF. The seek >> then moves past that space, but the space doesn't go away. The >> subsequent writes will extend EOF. The previously preallocated space now >> resides in the middle of the file and can't be trimmed away when the >> file is closed. >> >>> Now a 1MiB hole is supported using truncate: >>> dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock >>> truncate -s+1M file.in >>> dd if=/dev/urandom of=file.in bs=1M count=1 iflag=fullblock conv=notrunc oflag=append >>> $ du -k file.in >>> 2048 file.in >>> >> >> This works simply because it is broken into multiple commands. When the >> first dd exits, the excess space is trimmed off (the file descriptor is >> closed). The subsequent truncate extends the file size without any >> extra space getting caught between the old and new EOF. >> >> You can confirm this by using the 'allocsize=4k' mount option to the XFS >> mount. If you wanted something more generic for the purpose of testing >> the coreutils functionality, you could also set the size of file.out in >> advance. E.g., with preallocation in effect: >> >> # dd if=file.in of=file.out bs=1M conv=sparse >> # xfs_bmap -v file.out >> file.out: >> EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL >> 0: [0..3967]: 9773944..9777911 1 (9080..13047) 3968 >> 1: [3968..4095]: hole 128 >> 2: [4096..6143]: 9778040..9780087 1 (13176..15223) 2048 >> >> ... and then prevent preallocation by ensuring writes do not extend the >> file: >> >> # rm -f file.out >> # truncate --size=3M file.out >> # dd if=file.in of=file.out bs=1M conv=sparse,notrunc >> # xfs_bmap -v file.out >> file.out: >> EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL >> 0: [0..2047]: 9773944..9775991 1 (9080..11127) 2048 >> 1: [2048..4095]: hole 2048 >> 2: [4096..6143]: 9778040..9780087 1 (13176..15223) 2048 >> >> Hope that helps. > > Excellent info thanks. > With that I can adjust the test so it passes (patch attached). > > So for reference this means that cp can no longer recreate holes > <= 1MiB from source to dest (with the default XFS allocation size): Well, the allocation size changes based on the filesize; there's a heuristic involved. So I fear that if you hard-code it into your tests, you risk failing again in the future... > We could I suppose use FALLOC_FL_PUNCH_HOLE where available > to cater for this case. I'll see whether this is worth adding. That might make sense. But filesystems get to pick their layout; even ext4 may opportunistically fill in holes, etc - so I think you need to be pretty careful with these sorts of tests... -Eric _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-04-11 23:05 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-04-11 17:13 Strange hole creation behavior Pádraig Brady 2014-04-11 20:43 ` Brian Foster 2014-04-11 22:58 ` Pádraig Brady 2014-04-11 23:05 ` Eric Sandeen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).