* xfstests failures with xfs, dax and v4.4-rc3
@ 2015-12-02 18:34 Ross Zwisler
2015-12-02 20:10 ` Dan Williams
2015-12-02 20:29 ` Dave Chinner
0 siblings, 2 replies; 9+ messages in thread
From: Ross Zwisler @ 2015-12-02 18:34 UTC (permalink / raw)
To: Dave Chinner, xfs, Brian Foster, linux-fsdevel, linux-kernel; +Cc: Jeff Moyer
[-- Attachment #1: Type: text/plain, Size: 2906 bytes --]
I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
machine. Here are the failures:
# ./check generic/256 generic/274 xfs/041
FSTYP -- xfs (debug)
PLATFORM -- Linux/x86_64 alara 4.4.0-rc3
MKFS_OPTIONS -- -f -bsize=4096 /dev/pmem0p2
MOUNT_OPTIONS -- -o dax -o context=system_u:object_r:nfs_t:s0 /dev/pmem0p2 /mnt/xfstests_scratch
generic/256 43s ... - output mismatch (see /root/xfstests/results//generic/256.out.bad)
--- tests/generic/256.out 2015-10-02 10:19:36.807795900 -0600
+++ /root/xfstests/results//generic/256.out.bad 2015-12-02 11:22:34.838845475 -0700
@@ -1 +1,500 @@
QA output created by 256
+fallocate: No space left on device
+fallocate: No space left on device
+fallocate: No space left on device
+fallocate: No space left on device
+fallocate: No space left on device
+fallocate: No space left on device
...
(Run 'diff -u tests/generic/256.out /root/xfstests/results//generic/256.out.bad' to see the entire diff)
generic/274 7s ... [failed, exit status 1] - output mismatch (see /root/xfstests/results//generic/274.out.bad)
--- tests/generic/274.out 2015-10-02 10:19:36.808795907 -0600
+++ /root/xfstests/results//generic/274.out.bad 2015-12-02 11:22:37.656868990 -0700
@@ -2,4 +2,5 @@
------------------------------
preallocation test
------------------------------
-done
+failed to write to test file
+(see /root/xfstests/results//generic/274.full for details)
...
(Run 'diff -u tests/generic/274.out /root/xfstests/results//generic/274.out.bad' to see the entire diff)
xfs/041 14s ... [failed, exit status 1] - output mismatch (see /root/xfstests/results//xfs/041.out.bad)
--- tests/xfs/041.out 2015-10-02 10:19:36.818795975 -0600
+++ /root/xfstests/results//xfs/041.out.bad 2015-12-02 11:22:42.553909854 -0700
@@ -4,16 +4,5 @@
Grow filesystem to 33m... done
Flush filesystem... done
Check files... done
-Fill filesystem... done
-Grow filesystem to 67m... done
-Flush filesystem... done
-Check files... done
...
(Run 'diff -u tests/xfs/041.out /root/xfstests/results//xfs/041.out.bad' to see the entire diff)
Ran: generic/256 generic/274 xfs/041
Failures: generic/256 generic/274 xfs/041
Failed 3 of 3 tests
The .bad files are attached to this mail.
These three tests pass 100% of the time with and without DAX using v4.3, pass
100% of the time with v4.4-rc3 without DAX, and fail 100% of the time with
v4.4-rc3 with DAX enabled.
I did try doubling the size of my ramdisk partition to see if that made any
difference, and the failure was still present.
I'm using xfsprogs v4.3.0.
Please let me know if you have any other questions about how to reproduce the
failure. I'm also happy to test patches.
Thanks,
- Ross
[-- Attachment #2: 041.out.bad --]
[-- Type: text/plain, Size: 402 bytes --]
QA output created by 041
Make 32 megabyte filesystem on SCRATCH_DEV and mount... done
Fill filesystem... done
Grow filesystem to 33m... done
Flush filesystem... done
Check files... done
Fill filesystem... fail
src/fill2fs --verbose --dir=/mnt/xfstests_scratch/fill_67 --seed=0 --filesize=65536 --stddev=32768 --list=- >>/tmp/32491.manifest failed (returned 1): see /root/xfstests/results//xfs/041.full
[-- Attachment #3: 256.out.bad --]
[-- Type: text/plain, Size: 17490 bytes --]
QA output created by 256
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
fallocate: No space left on device
[-- Attachment #4: 274.out.bad --]
[-- Type: text/plain, Size: 194 bytes --]
QA output created by 274
------------------------------
preallocation test
------------------------------
failed to write to test file
(see /root/xfstests/results//generic/274.full for details)
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-02 18:34 xfstests failures with xfs, dax and v4.4-rc3 Ross Zwisler
@ 2015-12-02 20:10 ` Dan Williams
2015-12-02 20:29 ` Dave Chinner
1 sibling, 0 replies; 9+ messages in thread
From: Dan Williams @ 2015-12-02 20:10 UTC (permalink / raw)
To: Ross Zwisler, Dave Chinner, xfs, Brian Foster, linux-fsdevel,
Linux Kernel Mailing List, Jeff Moyer
On Wed, Dec 2, 2015 at 10:34 AM, Ross Zwisler
<ross.zwisler@linux.intel.com> wrote:
> I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
> and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
> machine. Here are the failures:
>
Is this a passing test with a v4.3 baseline? git bisect?
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-02 18:34 xfstests failures with xfs, dax and v4.4-rc3 Ross Zwisler
2015-12-02 20:10 ` Dan Williams
@ 2015-12-02 20:29 ` Dave Chinner
2015-12-02 20:45 ` Dave Chinner
1 sibling, 1 reply; 9+ messages in thread
From: Dave Chinner @ 2015-12-02 20:29 UTC (permalink / raw)
To: Ross Zwisler, xfs, Brian Foster, linux-fsdevel, linux-kernel,
Jeff Moyer
On Wed, Dec 02, 2015 at 11:34:38AM -0700, Ross Zwisler wrote:
> I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
> and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
> machine. Here are the failures:
Which are caused by commit 1ca1915 ("xfs: Don't use unwritten extents
for DAX") because of this code for unwritten extent conversion in
get_blocks:
tp->t_flags |= XFS_TRANS_RESERVE;
It's a minor problem compared to all the other issues DAX has right
now, so I ignored it to get the bigger problem solved first.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-02 20:29 ` Dave Chinner
@ 2015-12-02 20:45 ` Dave Chinner
2015-12-02 21:39 ` Ross Zwisler
0 siblings, 1 reply; 9+ messages in thread
From: Dave Chinner @ 2015-12-02 20:45 UTC (permalink / raw)
To: Ross Zwisler, xfs, Brian Foster, linux-fsdevel, linux-kernel,
Jeff Moyer
On Thu, Dec 03, 2015 at 07:29:10AM +1100, Dave Chinner wrote:
> On Wed, Dec 02, 2015 at 11:34:38AM -0700, Ross Zwisler wrote:
> > I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
> > and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
> > machine. Here are the failures:
>
> Which are caused by commit 1ca1915 ("xfs: Don't use unwritten extents
> for DAX") because of this code for unwritten extent conversion in
> get_blocks:
>
> tp->t_flags |= XFS_TRANS_RESERVE;
>
> It's a minor problem compared to all the other issues DAX has right
> now, so I ignored it to get the bigger problem solved first.
Patch to fix the problem below.
-Dave.
--
Dave Chinner
david@fromorbit.com
xfs: Don't use reserved blocks for data blocks with DAX
From: Dave Chinner <dchinner@redhat.com>
Commit 1ca1915 ("xfs: Don't use unwritten extents for DAX") enabled
the DAX allocation call to dip into the reserve pool in case it was
converting unwritten extents rather than allocating blocks. This was
a direct copy of the unwritten extent conversion code, but had an
unintended side effect of allowing normal data block allocation to
use the reserve pool. Hence normal block allocation could deplete
the reserve pool and prevent unwritten extent conversion at ENOSPC,
hence violating fallocate guarantees on preallocated space.
Fix it by checking whether the incoming map from __xfs_get_blocks()
spans an unwritten extent and only use the reserve pool if the
allocation covers an unwritten extent.
Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
fs/xfs/xfs_iomap.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
index f4f5b43..9ed146b 100644
--- a/fs/xfs/xfs_iomap.c
+++ b/fs/xfs/xfs_iomap.c
@@ -203,15 +203,20 @@ xfs_iomap_write_direct(
* this outside the transaction context, but if we commit and then crash
* we may not have zeroed the blocks and this will be exposed on
* recovery of the allocation. Hence we must zero before commit.
+ *
* Further, if we are mapping unwritten extents here, we need to zero
* and convert them to written so that we don't need an unwritten extent
* callback for DAX. This also means that we need to be able to dip into
- * the reserve block pool if there is no space left but we need to do
- * unwritten extent conversion.
+ * the reserve block pool for bmbt block allocation if there is no space
+ * left but we need to do unwritten extent conversion.
*/
+
if (IS_DAX(VFS_I(ip))) {
bmapi_flags = XFS_BMAPI_CONVERT | XFS_BMAPI_ZERO;
- tp->t_flags |= XFS_TRANS_RESERVE;
+ if (ISUNWRITTEN(imap)) {
+ tp->t_flags |= XFS_TRANS_RESERVE;
+ resblks = XFS_DIOSTRAT_SPACE_RES(mp, 0) << 1;
+ }
}
error = xfs_trans_reserve(tp, &M_RES(mp)->tr_write,
resblks, resrtextents);
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-02 20:45 ` Dave Chinner
@ 2015-12-02 21:39 ` Ross Zwisler
2015-12-10 16:54 ` Ross Zwisler
0 siblings, 1 reply; 9+ messages in thread
From: Ross Zwisler @ 2015-12-02 21:39 UTC (permalink / raw)
To: Dave Chinner
Cc: Ross Zwisler, xfs, Brian Foster, linux-fsdevel, linux-kernel,
Jeff Moyer
On Thu, Dec 03, 2015 at 07:45:02AM +1100, Dave Chinner wrote:
> On Thu, Dec 03, 2015 at 07:29:10AM +1100, Dave Chinner wrote:
> > On Wed, Dec 02, 2015 at 11:34:38AM -0700, Ross Zwisler wrote:
> > > I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
> > > and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
> > > machine. Here are the failures:
> >
> > Which are caused by commit 1ca1915 ("xfs: Don't use unwritten extents
> > for DAX") because of this code for unwritten extent conversion in
> > get_blocks:
> >
> > tp->t_flags |= XFS_TRANS_RESERVE;
> >
> > It's a minor problem compared to all the other issues DAX has right
> > now, so I ignored it to get the bigger problem solved first.
>
> Patch to fix the problem below.
>
> -Dave.
> --
> Dave Chinner
> david@fromorbit.com
>
> xfs: Don't use reserved blocks for data blocks with DAX
>
> From: Dave Chinner <dchinner@redhat.com>
>
> Commit 1ca1915 ("xfs: Don't use unwritten extents for DAX") enabled
> the DAX allocation call to dip into the reserve pool in case it was
> converting unwritten extents rather than allocating blocks. This was
> a direct copy of the unwritten extent conversion code, but had an
> unintended side effect of allowing normal data block allocation to
> use the reserve pool. Hence normal block allocation could deplete
> the reserve pool and prevent unwritten extent conversion at ENOSPC,
> hence violating fallocate guarantees on preallocated space.
>
> Fix it by checking whether the incoming map from __xfs_get_blocks()
> spans an unwritten extent and only use the reserve pool if the
> allocation covers an unwritten extent.
>
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
Tested-by: Ross Zwisler <ross.zwisler@linux.intel.com>
I've verified that this fixes all three failing xfstests reported in this mail.
Thanks!
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-02 21:39 ` Ross Zwisler
@ 2015-12-10 16:54 ` Ross Zwisler
2015-12-10 22:33 ` Dave Chinner
0 siblings, 1 reply; 9+ messages in thread
From: Ross Zwisler @ 2015-12-10 16:54 UTC (permalink / raw)
To: Ross Zwisler, Dave Chinner, xfs, Brian Foster, linux-fsdevel,
linux-kernel, Jeff Moyer
On Wed, Dec 02, 2015 at 02:39:32PM -0700, Ross Zwisler wrote:
> On Thu, Dec 03, 2015 at 07:45:02AM +1100, Dave Chinner wrote:
> > On Thu, Dec 03, 2015 at 07:29:10AM +1100, Dave Chinner wrote:
> > > On Wed, Dec 02, 2015 at 11:34:38AM -0700, Ross Zwisler wrote:
> > > > I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
> > > > and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
> > > > machine. Here are the failures:
> > >
> > > Which are caused by commit 1ca1915 ("xfs: Don't use unwritten extents
> > > for DAX") because of this code for unwritten extent conversion in
> > > get_blocks:
> > >
> > > tp->t_flags |= XFS_TRANS_RESERVE;
> > >
> > > It's a minor problem compared to all the other issues DAX has right
> > > now, so I ignored it to get the bigger problem solved first.
> >
> > Patch to fix the problem below.
> >
> > -Dave.
> > --
> > Dave Chinner
> > david@fromorbit.com
> >
> > xfs: Don't use reserved blocks for data blocks with DAX
> >
> > From: Dave Chinner <dchinner@redhat.com>
> >
> > Commit 1ca1915 ("xfs: Don't use unwritten extents for DAX") enabled
> > the DAX allocation call to dip into the reserve pool in case it was
> > converting unwritten extents rather than allocating blocks. This was
> > a direct copy of the unwritten extent conversion code, but had an
> > unintended side effect of allowing normal data block allocation to
> > use the reserve pool. Hence normal block allocation could deplete
> > the reserve pool and prevent unwritten extent conversion at ENOSPC,
> > hence violating fallocate guarantees on preallocated space.
> >
> > Fix it by checking whether the incoming map from __xfs_get_blocks()
> > spans an unwritten extent and only use the reserve pool if the
> > allocation covers an unwritten extent.
> >
> > Signed-off-by: Dave Chinner <dchinner@redhat.com>
>
> Tested-by: Ross Zwisler <ross.zwisler@linux.intel.com>
>
> I've verified that this fixes all three failing xfstests reported in this mail.
> Thanks!
Hey Dave,
Are you planning on pushing this fix for v4.4?
- Ross
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-10 16:54 ` Ross Zwisler
@ 2015-12-10 22:33 ` Dave Chinner
2015-12-11 4:26 ` Ross Zwisler
0 siblings, 1 reply; 9+ messages in thread
From: Dave Chinner @ 2015-12-10 22:33 UTC (permalink / raw)
To: Ross Zwisler, xfs, Brian Foster, linux-fsdevel, linux-kernel,
Jeff Moyer
On Thu, Dec 10, 2015 at 09:54:58AM -0700, Ross Zwisler wrote:
> On Wed, Dec 02, 2015 at 02:39:32PM -0700, Ross Zwisler wrote:
> > On Thu, Dec 03, 2015 at 07:45:02AM +1100, Dave Chinner wrote:
> > > On Thu, Dec 03, 2015 at 07:29:10AM +1100, Dave Chinner wrote:
> > > > On Wed, Dec 02, 2015 at 11:34:38AM -0700, Ross Zwisler wrote:
> > > > > I'm hitting a few more test failures in my testing setup with v4.4-rc3, xfs
> > > > > and DAX. My test setup is a pair of 4GiB PMEM partitions in a KVM virtual
> > > > > machine. Here are the failures:
> > > >
> > > > Which are caused by commit 1ca1915 ("xfs: Don't use unwritten extents
> > > > for DAX") because of this code for unwritten extent conversion in
> > > > get_blocks:
> > > >
> > > > tp->t_flags |= XFS_TRANS_RESERVE;
> > > >
> > > > It's a minor problem compared to all the other issues DAX has right
> > > > now, so I ignored it to get the bigger problem solved first.
> > >
> > > Patch to fix the problem below.
> > >
> > > -Dave.
> > > --
> > > Dave Chinner
> > > david@fromorbit.com
> > >
> > > xfs: Don't use reserved blocks for data blocks with DAX
> > >
> > > From: Dave Chinner <dchinner@redhat.com>
> > >
> > > Commit 1ca1915 ("xfs: Don't use unwritten extents for DAX") enabled
> > > the DAX allocation call to dip into the reserve pool in case it was
> > > converting unwritten extents rather than allocating blocks. This was
> > > a direct copy of the unwritten extent conversion code, but had an
> > > unintended side effect of allowing normal data block allocation to
> > > use the reserve pool. Hence normal block allocation could deplete
> > > the reserve pool and prevent unwritten extent conversion at ENOSPC,
> > > hence violating fallocate guarantees on preallocated space.
> > >
> > > Fix it by checking whether the incoming map from __xfs_get_blocks()
> > > spans an unwritten extent and only use the reserve pool if the
> > > allocation covers an unwritten extent.
> > >
> > > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> >
> > Tested-by: Ross Zwisler <ross.zwisler@linux.intel.com>
> >
> > I've verified that this fixes all three failing xfstests reported in this mail.
> > Thanks!
>
> Hey Dave,
>
> Are you planning on pushing this fix for v4.4?
No plans to right now - ENOSPC is a corner case that most users
won't be anywhere near, especially for experimental functionality on
hardware nobody actually has....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-10 22:33 ` Dave Chinner
@ 2015-12-11 4:26 ` Ross Zwisler
2015-12-14 0:20 ` Dave Chinner
0 siblings, 1 reply; 9+ messages in thread
From: Ross Zwisler @ 2015-12-11 4:26 UTC (permalink / raw)
To: Dave Chinner
Cc: Ross Zwisler, xfs, Brian Foster, linux-fsdevel, linux-kernel,
Jeff Moyer
On Fri, Dec 11, 2015 at 09:33:33AM +1100, Dave Chinner wrote:
> On Thu, Dec 10, 2015 at 09:54:58AM -0700, Ross Zwisler wrote:
> > On Wed, Dec 02, 2015 at 02:39:32PM -0700, Ross Zwisler wrote:
> > > I've verified that this fixes all three failing xfstests reported in this mail.
> > > Thanks!
> >
> > Hey Dave,
> >
> > Are you planning on pushing this fix for v4.4?
>
> No plans to right now - ENOSPC is a corner case that most users
> won't be anywhere near, especially for experimental functionality on
> hardware nobody actually has....
Really? I realize that it may be a case that most users won't actually hit,
but it is a 5 line change that fixes four xfstests regressions between v4.3 and
v4.4 for my DAX testing...
Is there a strong reason *not* to push it in the v4.4 cycle? I'm trying to
clear up all xfstests differences between DAX and non-DAX, and this would help
quite a bit.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: xfstests failures with xfs, dax and v4.4-rc3
2015-12-11 4:26 ` Ross Zwisler
@ 2015-12-14 0:20 ` Dave Chinner
0 siblings, 0 replies; 9+ messages in thread
From: Dave Chinner @ 2015-12-14 0:20 UTC (permalink / raw)
To: Ross Zwisler, xfs, Brian Foster, linux-fsdevel, linux-kernel,
Jeff Moyer
On Thu, Dec 10, 2015 at 09:26:09PM -0700, Ross Zwisler wrote:
> On Fri, Dec 11, 2015 at 09:33:33AM +1100, Dave Chinner wrote:
> > On Thu, Dec 10, 2015 at 09:54:58AM -0700, Ross Zwisler wrote:
> > > On Wed, Dec 02, 2015 at 02:39:32PM -0700, Ross Zwisler wrote:
> > > > I've verified that this fixes all three failing xfstests reported in this mail.
> > > > Thanks!
> > >
> > > Hey Dave,
> > >
> > > Are you planning on pushing this fix for v4.4?
> >
> > No plans to right now - ENOSPC is a corner case that most users
> > won't be anywhere near, especially for experimental functionality on
> > hardware nobody actually has....
>
> Really? I realize that it may be a case that most users won't actually hit,
> but it is a 5 line change that fixes four xfstests regressions between v4.3 and
> v4.4 for my DAX testing...
>
> Is there a strong reason *not* to push it in the v4.4 cycle? I'm trying to
> clear up all xfstests differences between DAX and non-DAX, and this would help
> quite a bit.
Against my better judgement, I committed a largely untested,
fundamental change to allocation policy for DAX-on-XFS *inside the
4.4 merge window* justifying it as "it's experimental code" and that
"we needed to get it out there". IOWs, I've already stretched the
rules and committed stuff I knew wasn't ready or worked 100%
correctly just to keep you guys happy, but I'm not going to continue
to do so. I'm reverting to usual policy of "soak in for-next, use
"cc: stable" tags to get it backported when upstreamed in the next
merge window.
Indeed, from my point of view it's pretty clear that lots of recent
DAX code has been committed prematurely and without sufficient
review and/or testing. This has lead to having to revert chunks of
code and completely rework algorithms, I really don't care if you're
being pushed to "make DAX shit happen fast" - my only concern here
is *don't screw up production filesystems*.
Hence when it comes to making fundamental changes to allocation
behaviour, I've already bent the policy/rules as far as I'm willing
to. Changes in this area carry an inherent risk of breaking stuff
unrelated to DAX and that is far more important right now than
whether DAX (and experimental feature) works correctly or not.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2015-12-14 0:20 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-02 18:34 xfstests failures with xfs, dax and v4.4-rc3 Ross Zwisler
2015-12-02 20:10 ` Dan Williams
2015-12-02 20:29 ` Dave Chinner
2015-12-02 20:45 ` Dave Chinner
2015-12-02 21:39 ` Ross Zwisler
2015-12-10 16:54 ` Ross Zwisler
2015-12-10 22:33 ` Dave Chinner
2015-12-11 4:26 ` Ross Zwisler
2015-12-14 0:20 ` Dave Chinner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).