[PATCH v2 0/4] fstests: sector size fixes and whiteouts...

[PATCH v2 0/4] fstests: sector size fixes and whiteouts...

[Dave Chinner]

Hi folks,

Version 2 of the patchset I sent yesterday here:

http://thread.gmane.org/gmane.comp.file-systems.fstests/293

V2:
- log size too small for other tests as well (e.g. xfs/297) so
  added them to patch 1/4
- realised I'd only updated the external log file output for xfs/096
  in patch 2/4, so fixed that and updated the filtering to handle
  differences in log version as crc enabled ffilesystems override
  the log version given on the CLI.

[PATCH 1/4] xfs/104: log size too small for 4k sector drives

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
recent change to the kernel ramdisk changed it's physical sector
size from 512B to 4kB, and this results in mkfs calculating a log
size larger than the fixed test size and hence the tests fail.

Change the log size to a larger size that works with 4k sectors, and
also increase the size of the filesystem being created so that the
amount of data space in the filesystem does not change and hence
does not perturb the rest of the test.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 tests/xfs/104     |  8 ++++----
 tests/xfs/104.out | 46 +++++++++++++++++++++++-----------------------
 tests/xfs/119     |  2 +-
 tests/xfs/291     |  2 +-
 tests/xfs/297     |  2 +-
 5 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/tests/xfs/104 b/tests/xfs/104
index 69fcc69..ca2ae21 100755
--- a/tests/xfs/104
+++ b/tests/xfs/104
@@ -81,10 +81,10 @@ modsize=`expr   4 \* $incsize`	# pause after this many increments
 [ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
 
 nags=4
-size=`expr 120 \* 1048576`	# 120 megabytes initially
+size=`expr 125 \* 1048576`	# 120 megabytes initially
 sizeb=`expr $size / $dbsize`	# in data blocks
 echo "*** creating scratch filesystem"
-_create_scratch -lsize=5m -dsize=${size} -dagcount=${nags}
+_create_scratch -lsize=10m -dsize=${size} -dagcount=${nags}
 
 fillsize=`expr 110 \* 1048576`	# 110 megabytes of filling
 echo "*** using some initial space on scratch filesystem"
@@ -95,13 +95,13 @@ _fill_scratch $fillsize
 # Kick off more stress threads on each iteration, grow; repeat.
 #
 while [ $size -le $endsize ]; do
-	echo "*** stressing a ${size} byte filesystem"
+	echo "*** stressing filesystem"
 	echo "*** stressing a ${sizeb} block filesystem" >> $seqres.full
 	_stress_scratch
 	sleep 1
 	size=`expr $size + $incsize`
 	sizeb=`expr $size / $dbsize`	# in data blocks
-	echo "*** growing to a ${size} byte filesystem"
+	echo "*** growing filesystem"
 	echo "*** growing to a ${sizeb} block filesystem" >> $seqres.full
 	xfs_growfs -D ${sizeb} $SCRATCH_MNT \
 		| tee -a $seqres.full | _filter_mkfs 2>$tmp.growfs
diff --git a/tests/xfs/104.out b/tests/xfs/104.out
index f237e5e..de6c7f2 100644
--- a/tests/xfs/104.out
+++ b/tests/xfs/104.out
@@ -15,8 +15,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 *** mount
 *** using some initial space on scratch filesystem
-*** stressing a 125829120 byte filesystem
-*** growing to a 169869312 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -25,8 +25,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=4
 
-*** stressing a 169869312 byte filesystem
-*** growing to a 213909504 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -35,8 +35,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=6
 
-*** stressing a 213909504 byte filesystem
-*** growing to a 257949696 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -45,8 +45,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=7
 
-*** stressing a 257949696 byte filesystem
-*** growing to a 301989888 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -55,8 +55,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=9
 
-*** stressing a 301989888 byte filesystem
-*** growing to a 346030080 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -65,8 +65,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=10
 
-*** stressing a 346030080 byte filesystem
-*** growing to a 390070272 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -75,8 +75,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=11
 
-*** stressing a 390070272 byte filesystem
-*** growing to a 434110464 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -85,8 +85,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=13
 
-*** stressing a 434110464 byte filesystem
-*** growing to a 478150656 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -95,18 +95,18 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=14
 
-*** stressing a 478150656 byte filesystem
-*** growing to a 522190848 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
 naming   =VERN bsize=XXX
 log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
-AGCOUNT=16
+AGCOUNT=15
 
-*** stressing a 522190848 byte filesystem
-*** growing to a 566231040 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
@@ -115,8 +115,8 @@ log      =LDEV bsize=XXX blocks=XXX
 realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
 AGCOUNT=17
 
-*** stressing a 566231040 byte filesystem
-*** growing to a 610271232 byte filesystem
+*** stressing filesystem
+*** growing filesystem
 meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
 data     = bsize=XXX blocks=XXX, imaxpct=PCT
          = sunit=XXX swidth=XXX, unwritten=X
diff --git a/tests/xfs/119 b/tests/xfs/119
index c7c46d9..490495b 100755
--- a/tests/xfs/119
+++ b/tests/xfs/119
@@ -54,7 +54,7 @@ _require_scratch
 # this may hang
 sync
 
-export MKFS_OPTIONS="-l version=2,size=1200b,su=64k" 
+export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 
 export MOUNT_OPTIONS="-o logbsize=64k"
 _scratch_mkfs_xfs >/dev/null
 
diff --git a/tests/xfs/291 b/tests/xfs/291
index fbf9c51..c226e65 100755
--- a/tests/xfs/291
+++ b/tests/xfs/291
@@ -46,7 +46,7 @@ _supported_os IRIX Linux
 # real QA test starts here
 rm -f $seqres.full
 _require_scratch
-_scratch_mkfs_xfs -n size=16k -l size=5m -d size=128m >> $seqres.full 2>&1
+_scratch_mkfs_xfs -n size=16k -l size=10m -d size=133m >> $seqres.full 2>&1
 _scratch_mount
 
 # First we cause very badly fragmented freespace, then
diff --git a/tests/xfs/297 b/tests/xfs/297
index 1cdbbb9..25b597e 100755
--- a/tests/xfs/297
+++ b/tests/xfs/297
@@ -50,7 +50,7 @@ _require_scratch
 _require_freeze
 
 rm -f $seqres.full
-_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b >/dev/null 2>&1
+_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=5120b >/dev/null 2>&1
 _scratch_mount >/dev/null 2>&1
 
 STRESS_DIR="$SCRATCH_MNT/testdir"
-- 
2.0.0

Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives

[Brian Foster]

On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
> recent change to the kernel ramdisk changed it's physical sector
> size from 512B to 4kB, and this results in mkfs calculating a log
> size larger than the fixed test size and hence the tests fail.
> 
> Change the log size to a larger size that works with 4k sectors, and
> also increase the size of the filesystem being created so that the
> amount of data space in the filesystem does not change and hence
> does not perturb the rest of the test.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---

Well for some reason I can't mount a ramdisk on the current tot to test
this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
The mount actually reports success too, but there's nothing there... :/

# modprobe brd
# mkfs.xfs -f /dev/ram0 
meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal log           bsize=4096   blocks=1605, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
# mount /dev/ram0 /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

... and then I can't even mount my normal scratch device until after a
reboot:

# mount /dev/test/scratch /mnt/
# mount | grep mnt
# umount  /mnt/
umount: /mnt/: not mounted

I see some general flakiness with loop devices as well (re: patch 3/4).
Anyways... until I get a chance to look at that, a couple nits to
follow. Otherwise this looks Ok to me.

>  tests/xfs/104     |  8 ++++----
>  tests/xfs/104.out | 46 +++++++++++++++++++++++-----------------------
>  tests/xfs/119     |  2 +-
>  tests/xfs/291     |  2 +-
>  tests/xfs/297     |  2 +-
>  5 files changed, 30 insertions(+), 30 deletions(-)
> 
> diff --git a/tests/xfs/104 b/tests/xfs/104
> index 69fcc69..ca2ae21 100755
> --- a/tests/xfs/104
> +++ b/tests/xfs/104
> @@ -81,10 +81,10 @@ modsize=`expr   4 \* $incsize`	# pause after this many increments
>  [ `expr $endsize / $dbsize` -lt $dblocks ] || _notrun "Scratch device too small"
>  
>  nags=4
> -size=`expr 120 \* 1048576`	# 120 megabytes initially
> +size=`expr 125 \* 1048576`	# 120 megabytes initially

The comment is wrong now.

>  sizeb=`expr $size / $dbsize`	# in data blocks
>  echo "*** creating scratch filesystem"
> -_create_scratch -lsize=5m -dsize=${size} -dagcount=${nags}
> +_create_scratch -lsize=10m -dsize=${size} -dagcount=${nags}
>  
>  fillsize=`expr 110 \* 1048576`	# 110 megabytes of filling
>  echo "*** using some initial space on scratch filesystem"
> @@ -95,13 +95,13 @@ _fill_scratch $fillsize
>  # Kick off more stress threads on each iteration, grow; repeat.
>  #
>  while [ $size -le $endsize ]; do
> -	echo "*** stressing a ${size} byte filesystem"
> +	echo "*** stressing filesystem"
>  	echo "*** stressing a ${sizeb} block filesystem" >> $seqres.full
>  	_stress_scratch
>  	sleep 1
>  	size=`expr $size + $incsize`
>  	sizeb=`expr $size / $dbsize`	# in data blocks
> -	echo "*** growing to a ${size} byte filesystem"
> +	echo "*** growing filesystem"
>  	echo "*** growing to a ${sizeb} block filesystem" >> $seqres.full
>  	xfs_growfs -D ${sizeb} $SCRATCH_MNT \
>  		| tee -a $seqres.full | _filter_mkfs 2>$tmp.growfs
> diff --git a/tests/xfs/104.out b/tests/xfs/104.out
> index f237e5e..de6c7f2 100644
> --- a/tests/xfs/104.out
> +++ b/tests/xfs/104.out
> @@ -15,8 +15,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  *** mount
>  *** using some initial space on scratch filesystem
> -*** stressing a 125829120 byte filesystem
> -*** growing to a 169869312 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -25,8 +25,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=4
>  
> -*** stressing a 169869312 byte filesystem
> -*** growing to a 213909504 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -35,8 +35,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=6
>  
> -*** stressing a 213909504 byte filesystem
> -*** growing to a 257949696 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -45,8 +45,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=7
>  
> -*** stressing a 257949696 byte filesystem
> -*** growing to a 301989888 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -55,8 +55,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=9
>  
> -*** stressing a 301989888 byte filesystem
> -*** growing to a 346030080 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -65,8 +65,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=10
>  
> -*** stressing a 346030080 byte filesystem
> -*** growing to a 390070272 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -75,8 +75,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=11
>  
> -*** stressing a 390070272 byte filesystem
> -*** growing to a 434110464 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -85,8 +85,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=13
>  
> -*** stressing a 434110464 byte filesystem
> -*** growing to a 478150656 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -95,18 +95,18 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=14
>  
> -*** stressing a 478150656 byte filesystem
> -*** growing to a 522190848 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
>  naming   =VERN bsize=XXX
>  log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
> -AGCOUNT=16
> +AGCOUNT=15
>  
> -*** stressing a 522190848 byte filesystem
> -*** growing to a 566231040 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> @@ -115,8 +115,8 @@ log      =LDEV bsize=XXX blocks=XXX
>  realtime =RDEV extsz=XXX blocks=XXX, rtextents=XXX
>  AGCOUNT=17
>  
> -*** stressing a 566231040 byte filesystem
> -*** growing to a 610271232 byte filesystem
> +*** stressing filesystem
> +*** growing filesystem
>  meta-data=DDEV isize=XXX agcount=N, agsize=XXX blks
>  data     = bsize=XXX blocks=XXX, imaxpct=PCT
>           = sunit=XXX swidth=XXX, unwritten=X
> diff --git a/tests/xfs/119 b/tests/xfs/119
> index c7c46d9..490495b 100755
> --- a/tests/xfs/119
> +++ b/tests/xfs/119
> @@ -54,7 +54,7 @@ _require_scratch
>  # this may hang
>  sync
>  
> -export MKFS_OPTIONS="-l version=2,size=1200b,su=64k" 
> +export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 

Trailing space here.

Brian

>  export MOUNT_OPTIONS="-o logbsize=64k"
>  _scratch_mkfs_xfs >/dev/null
>  
> diff --git a/tests/xfs/291 b/tests/xfs/291
> index fbf9c51..c226e65 100755
> --- a/tests/xfs/291
> +++ b/tests/xfs/291
> @@ -46,7 +46,7 @@ _supported_os IRIX Linux
>  # real QA test starts here
>  rm -f $seqres.full
>  _require_scratch
> -_scratch_mkfs_xfs -n size=16k -l size=5m -d size=128m >> $seqres.full 2>&1
> +_scratch_mkfs_xfs -n size=16k -l size=10m -d size=133m >> $seqres.full 2>&1
>  _scratch_mount
>  
>  # First we cause very badly fragmented freespace, then
> diff --git a/tests/xfs/297 b/tests/xfs/297
> index 1cdbbb9..25b597e 100755
> --- a/tests/xfs/297
> +++ b/tests/xfs/297
> @@ -50,7 +50,7 @@ _require_scratch
>  _require_freeze
>  
>  rm -f $seqres.full
> -_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b >/dev/null 2>&1
> +_scratch_mkfs_xfs -d agcount=16,su=256k,sw=12 -l su=256k,size=5120b >/dev/null 2>&1
>  _scratch_mount >/dev/null 2>&1
>  
>  STRESS_DIR="$SCRATCH_MNT/testdir"
> -- 
> 2.0.0
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Dave Chinner]

[cc linux-fsdevel, Boaz and others]

On Wed, Feb 25, 2015 at 11:11:51AM -0500, Brian Foster wrote:
> On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
> > From: Dave Chinner <dchinner@redhat.com>
> > 
> > xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
> > recent change to the kernel ramdisk changed it's physical sector
> > size from 512B to 4kB, and this results in mkfs calculating a log
> > size larger than the fixed test size and hence the tests fail.
> > 
> > Change the log size to a larger size that works with 4k sectors, and
> > also increase the size of the filesystem being created so that the
> > amount of data space in the filesystem does not change and hence
> > does not perturb the rest of the test.
> > 
> > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> > ---
> 
> Well for some reason I can't mount a ramdisk on the current tot to test
> this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
> The mount actually reports success too, but there's nothing there... :/
> 
> # modprobe brd
> # mkfs.xfs -f /dev/ram0 
> meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
> blks
>          =                       sectsz=4096  attr=2, projid32bit=1
>          =                       crc=0        finobt=0
> data     =                       bsize=4096   blocks=4096, imaxpct=25
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
> log      =internal log           bsize=4096   blocks=1605, version=2
>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> realtime =none                   extsz=4096   blocks=0, rtextents=0
> # mount /dev/ram0 /mnt/
> # mount | grep mnt
> # umount  /mnt/
> umount: /mnt/: not mounted
> 
> ... and then I can't even mount my normal scratch device until after a
> reboot:
> 
> # mount /dev/test/scratch /mnt/
> # mount | grep mnt
> # umount  /mnt/
> umount: /mnt/: not mounted

Ok, so that's just plain broken. What's in dmesg?

As it is, I'm seeing plenty of weirdness in 4.0-rc1 on ramdisks as
well. Apart from the change to 4k physical sector size causing all
sorts of chaos with xfstests results due to it changing mkfs.xfs
behaviour, I'm also seeing this happen randomly:

....
Feb 25 11:48:35 test4 dave: run xfstest generic/083
Feb 25 11:48:37 test4 kernel: [ 8732.316223] XFS (ram1): Mounting V5 Filesystem
Feb 25 11:48:37 test4 kernel: [ 8732.318904] XFS (ram1): Ending clean mount
Feb 25 11:48:40 test4 kernel: [ 8735.871968] XFS (ram1): Unmounting Filesystem
Feb 25 11:48:40 test4 kernel: [ 8735.930160]  ram1: [POWERTEC] p1 p2 p3 p4 p5 p6 p7
Feb 25 11:48:40 test4 kernel: [ 8735.932081] ram1: p2 start 3158599292 is beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.933983] ram1: p3 size 1627389952 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.936177] ram1: p4 size 1158021120 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.938269] ram1: p5 start 50924556 is beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.940103] ram1: p6 size 67108864 extends beyond EOD, truncated
Feb 25 11:48:40 test4 kernel: [ 8735.942101] ram1: p7 start 4294967295 is beyond EOD, truncated
Feb 25 11:48:40 test4 dave: run xfstest generic/088
....

Something is causing partition rescans on ram devices that don't
have partitions, and this is new behaviour. Boaz, your commit
937af5ecd05 ("brd: Fix all partitions BUGs") seems the likely cause
of this problem I'm seeing - looks likea behaviour regression to
me as no other block device I have on any machine running the
same kernel throws these strange warnings from partition probing...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Brian Foster]

On Thu, Feb 26, 2015 at 09:32:48AM +1100, Dave Chinner wrote:
> [cc linux-fsdevel, Boaz and others]
> 
> On Wed, Feb 25, 2015 at 11:11:51AM -0500, Brian Foster wrote:
> > On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
> > > From: Dave Chinner <dchinner@redhat.com>
> > > 
> > > xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
> > > recent change to the kernel ramdisk changed it's physical sector
> > > size from 512B to 4kB, and this results in mkfs calculating a log
> > > size larger than the fixed test size and hence the tests fail.
> > > 
> > > Change the log size to a larger size that works with 4k sectors, and
> > > also increase the size of the filesystem being created so that the
> > > amount of data space in the filesystem does not change and hence
> > > does not perturb the rest of the test.
> > > 
> > > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> > > ---
> > 
> > Well for some reason I can't mount a ramdisk on the current tot to test
> > this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
> > The mount actually reports success too, but there's nothing there... :/
> > 
> > # modprobe brd
> > # mkfs.xfs -f /dev/ram0 
> > meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
> > blks
> >          =                       sectsz=4096  attr=2, projid32bit=1
> >          =                       crc=0        finobt=0
> > data     =                       bsize=4096   blocks=4096, imaxpct=25
> >          =                       sunit=0      swidth=0 blks
> > naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
> > log      =internal log           bsize=4096   blocks=1605, version=2
> >          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> > realtime =none                   extsz=4096   blocks=0, rtextents=0
> > # mount /dev/ram0 /mnt/
> > # mount | grep mnt
> > # umount  /mnt/
> > umount: /mnt/: not mounted
> > 
> > ... and then I can't even mount my normal scratch device until after a
> > reboot:
> > 
> > # mount /dev/test/scratch /mnt/
> > # mount | grep mnt
> > # umount  /mnt/
> > umount: /mnt/: not mounted
> 
> Ok, so that's just plain broken. What's in dmesg?
> 

Once I got back to this I found that for some reason systemd is
immediately invoking a umount on the mount. :/ No idea why or how to
stop it, but if I do something like this:

mount /dev/ram0 /mnt; cd /mnt

... I can occasionally win the race and get systemd to spin in a
umount() cycle trying to undo the mount. I haven't gone back to confirm
it's the same behavior with the normal devices at that point, but I
suspect it is, perhaps due to getting into some kind of bad state.

So fyi that this particular problem doesn't appear to be directly kernel
related...

Brian

> As it is, I'm seeing plenty of weirdness in 4.0-rc1 on ramdisks as
> well. Apart from the change to 4k physical sector size causing all
> sorts of chaos with xfstests results due to it changing mkfs.xfs
> behaviour, I'm also seeing this happen randomly:
> 
> ....
> Feb 25 11:48:35 test4 dave: run xfstest generic/083
> Feb 25 11:48:37 test4 kernel: [ 8732.316223] XFS (ram1): Mounting V5 Filesystem
> Feb 25 11:48:37 test4 kernel: [ 8732.318904] XFS (ram1): Ending clean mount
> Feb 25 11:48:40 test4 kernel: [ 8735.871968] XFS (ram1): Unmounting Filesystem
> Feb 25 11:48:40 test4 kernel: [ 8735.930160]  ram1: [POWERTEC] p1 p2 p3 p4 p5 p6 p7
> Feb 25 11:48:40 test4 kernel: [ 8735.932081] ram1: p2 start 3158599292 is beyond EOD, truncated
> Feb 25 11:48:40 test4 kernel: [ 8735.933983] ram1: p3 size 1627389952 extends beyond EOD, truncated
> Feb 25 11:48:40 test4 kernel: [ 8735.936177] ram1: p4 size 1158021120 extends beyond EOD, truncated
> Feb 25 11:48:40 test4 kernel: [ 8735.938269] ram1: p5 start 50924556 is beyond EOD, truncated
> Feb 25 11:48:40 test4 kernel: [ 8735.940103] ram1: p6 size 67108864 extends beyond EOD, truncated
> Feb 25 11:48:40 test4 kernel: [ 8735.942101] ram1: p7 start 4294967295 is beyond EOD, truncated
> Feb 25 11:48:40 test4 dave: run xfstest generic/088
> ....
> 
> Something is causing partition rescans on ram devices that don't
> have partitions, and this is new behaviour. Boaz, your commit
> 937af5ecd05 ("brd: Fix all partitions BUGs") seems the likely cause
> of this problem I'm seeing - looks likea behaviour regression to
> me as no other block device I have on any machine running the
> same kernel throws these strange warnings from partition probing...
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Dave Chinner]

On Wed, Feb 25, 2015 at 06:31:15PM -0500, Brian Foster wrote:
> On Thu, Feb 26, 2015 at 09:32:48AM +1100, Dave Chinner wrote:
> > [cc linux-fsdevel, Boaz and others]
> > 
> > On Wed, Feb 25, 2015 at 11:11:51AM -0500, Brian Foster wrote:
> > > On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
> > > > From: Dave Chinner <dchinner@redhat.com>
> > > > 
> > > > xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
> > > > recent change to the kernel ramdisk changed it's physical sector
> > > > size from 512B to 4kB, and this results in mkfs calculating a log
> > > > size larger than the fixed test size and hence the tests fail.
> > > > 
> > > > Change the log size to a larger size that works with 4k sectors, and
> > > > also increase the size of the filesystem being created so that the
> > > > amount of data space in the filesystem does not change and hence
> > > > does not perturb the rest of the test.
> > > > 
> > > > Signed-off-by: Dave Chinner <dchinner@redhat.com>
> > > > ---
> > > 
> > > Well for some reason I can't mount a ramdisk on the current tot to test
> > > this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
> > > The mount actually reports success too, but there's nothing there... :/
> > > 
> > > # modprobe brd
> > > # mkfs.xfs -f /dev/ram0 
> > > meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
> > > blks
> > >          =                       sectsz=4096  attr=2, projid32bit=1
> > >          =                       crc=0        finobt=0
> > > data     =                       bsize=4096   blocks=4096, imaxpct=25
> > >          =                       sunit=0      swidth=0 blks
> > > naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
> > > log      =internal log           bsize=4096   blocks=1605, version=2
> > >          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> > > realtime =none                   extsz=4096   blocks=0, rtextents=0
> > > # mount /dev/ram0 /mnt/
> > > # mount | grep mnt
> > > # umount  /mnt/
> > > umount: /mnt/: not mounted
> > > 
> > > ... and then I can't even mount my normal scratch device until after a
> > > reboot:
> > > 
> > > # mount /dev/test/scratch /mnt/
> > > # mount | grep mnt
> > > # umount  /mnt/
> > > umount: /mnt/: not mounted
> > 
> > Ok, so that's just plain broken. What's in dmesg?
> > 
> 
> Once I got back to this I found that for some reason systemd is
> immediately invoking a umount on the mount. :/ No idea why or how to
> stop it, but if I do something like this:
> 
> mount /dev/ram0 /mnt; cd /mnt
> 
> ... I can occasionally win the race and get systemd to spin in a
> umount() cycle trying to undo the mount. I haven't gone back to confirm
> it's the same behavior with the normal devices at that point, but I
> suspect it is, perhaps due to getting into some kind of bad state.
> 
> So fyi that this particular problem doesn't appear to be directly kernel
> related...

It may still be related to the kernel changes  e.g. by triggering
udev events when they didn't previously. The only machine I have
that is triggering the partition probing is also the only test
machine that I have that runs systemd and it didn't have this
problem on 3.19.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Boaz Harrosh]

On 02/26/2015 01:43 AM, Dave Chinner wrote:
> On Wed, Feb 25, 2015 at 06:31:15PM -0500, Brian Foster wrote:
>> On Thu, Feb 26, 2015 at 09:32:48AM +1100, Dave Chinner wrote:
>>> [cc linux-fsdevel, Boaz and others]
>>>
>>> On Wed, Feb 25, 2015 at 11:11:51AM -0500, Brian Foster wrote:
>>>> On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
>>>>> From: Dave Chinner <dchinner@redhat.com>
>>>>>
>>>>> xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
>>>>> recent change to the kernel ramdisk changed it's physical sector
>>>>> size from 512B to 4kB, and this results in mkfs calculating a log
>>>>> size larger than the fixed test size and hence the tests fail.
>>>>>
>>>>> Change the log size to a larger size that works with 4k sectors, and
>>>>> also increase the size of the filesystem being created so that the
>>>>> amount of data space in the filesystem does not change and hence
>>>>> does not perturb the rest of the test.
>>>>>
>>>>> Signed-off-by: Dave Chinner <dchinner@redhat.com>
>>>>> ---
>>>>
>>>> Well for some reason I can't mount a ramdisk on the current tot to test
>>>> this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
>>>> The mount actually reports success too, but there's nothing there... :/
>>>>
>>>> # modprobe brd
>>>> # mkfs.xfs -f /dev/ram0 
>>>> meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
>>>> blks
>>>>          =                       sectsz=4096  attr=2, projid32bit=1
>>>>          =                       crc=0        finobt=0
>>>> data     =                       bsize=4096   blocks=4096, imaxpct=25
>>>>          =                       sunit=0      swidth=0 blks
>>>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
>>>> log      =internal log           bsize=4096   blocks=1605, version=2
>>>>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
>>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
>>>> # mount /dev/ram0 /mnt/
>>>> # mount | grep mnt
>>>> # umount  /mnt/
>>>> umount: /mnt/: not mounted
>>>>
>>>> ... and then I can't even mount my normal scratch device until after a
>>>> reboot:
>>>>
>>>> # mount /dev/test/scratch /mnt/
>>>> # mount | grep mnt
>>>> # umount  /mnt/
>>>> umount: /mnt/: not mounted
>>>
>>> Ok, so that's just plain broken. What's in dmesg?
>>>
>>
>> Once I got back to this I found that for some reason systemd is
>> immediately invoking a umount on the mount. :/ No idea why or how to
>> stop it, but if I do something like this:
>>
>> mount /dev/ram0 /mnt; cd /mnt
>>
>> ... I can occasionally win the race and get systemd to spin in a
>> umount() cycle trying to undo the mount. I haven't gone back to confirm
>> it's the same behavior with the normal devices at that point, but I
>> suspect it is, perhaps due to getting into some kind of bad state.
>>
>> So fyi that this particular problem doesn't appear to be directly kernel
>> related...
> 
> It may still be related to the kernel changes  e.g. by triggering
> udev events when they didn't previously. The only machine I have
> that is triggering the partition probing is also the only test
> machine that I have that runs systemd and it didn't have this
> problem on 3.19.
> 

Sigh, thanks Dave. Yes you are correct my patch enabled the
udev events, as part of fixing ramdisk with partitions.
This is because if you do not enable them then mount by UUID
and all sort of lsblk and friends do not work.

I did try to test this in all kind of ways, xfstest+ext4
as well, and ran with it on Fedora 20 for a while, sorry
about that.

It looks like the system anticipates that ramdisk "should
not have these events"

I will send a patch ASAP that re-instates the module_parameter
for enabling notification, and leaving the default off. It should
be easy to set the param if one intends to use these utilities.

That said, please do agree with me that there is brokenness in
systemd?

BTW: You also said something about the 4k sectors thing, It looks
like we are pulled in two different directions here. If you will
want to use DAX on ramdisk then you want it on, if you are not
using DAX, and wants to use smaller-then-page_size FS blocks than
you do not want it.

Please advise what we should do? Maybe only do 4k if BLK_DEV_RAM_DAX
is set in Kconfig ?


Sorry for the mess, I'll send a fix ASAP

> Cheers,
> Dave.
> 

Thanks
Boaz

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Brian Foster]

On Thu, Feb 26, 2015 at 09:46:27AM +0200, Boaz Harrosh wrote:
> On 02/26/2015 01:43 AM, Dave Chinner wrote:
> > On Wed, Feb 25, 2015 at 06:31:15PM -0500, Brian Foster wrote:
> >> On Thu, Feb 26, 2015 at 09:32:48AM +1100, Dave Chinner wrote:
> >>> [cc linux-fsdevel, Boaz and others]
> >>>
> >>> On Wed, Feb 25, 2015 at 11:11:51AM -0500, Brian Foster wrote:
> >>>> On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
> >>>>> From: Dave Chinner <dchinner@redhat.com>
> >>>>>
> >>>>> xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
> >>>>> recent change to the kernel ramdisk changed it's physical sector
> >>>>> size from 512B to 4kB, and this results in mkfs calculating a log
> >>>>> size larger than the fixed test size and hence the tests fail.
> >>>>>
> >>>>> Change the log size to a larger size that works with 4k sectors, and
> >>>>> also increase the size of the filesystem being created so that the
> >>>>> amount of data space in the filesystem does not change and hence
> >>>>> does not perturb the rest of the test.
> >>>>>
> >>>>> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> >>>>> ---
> >>>>
> >>>> Well for some reason I can't mount a ramdisk on the current tot to test
> >>>> this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
> >>>> The mount actually reports success too, but there's nothing there... :/
> >>>>
> >>>> # modprobe brd
> >>>> # mkfs.xfs -f /dev/ram0 
> >>>> meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
> >>>> blks
> >>>>          =                       sectsz=4096  attr=2, projid32bit=1
> >>>>          =                       crc=0        finobt=0
> >>>> data     =                       bsize=4096   blocks=4096, imaxpct=25
> >>>>          =                       sunit=0      swidth=0 blks
> >>>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
> >>>> log      =internal log           bsize=4096   blocks=1605, version=2
> >>>>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> >>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
> >>>> # mount /dev/ram0 /mnt/
> >>>> # mount | grep mnt
> >>>> # umount  /mnt/
> >>>> umount: /mnt/: not mounted
> >>>>
> >>>> ... and then I can't even mount my normal scratch device until after a
> >>>> reboot:
> >>>>
> >>>> # mount /dev/test/scratch /mnt/
> >>>> # mount | grep mnt
> >>>> # umount  /mnt/
> >>>> umount: /mnt/: not mounted
> >>>
> >>> Ok, so that's just plain broken. What's in dmesg?
> >>>
> >>
> >> Once I got back to this I found that for some reason systemd is
> >> immediately invoking a umount on the mount. :/ No idea why or how to
> >> stop it, but if I do something like this:
> >>
> >> mount /dev/ram0 /mnt; cd /mnt
> >>
> >> ... I can occasionally win the race and get systemd to spin in a
> >> umount() cycle trying to undo the mount. I haven't gone back to confirm
> >> it's the same behavior with the normal devices at that point, but I
> >> suspect it is, perhaps due to getting into some kind of bad state.
> >>
> >> So fyi that this particular problem doesn't appear to be directly kernel
> >> related...
> > 
> > It may still be related to the kernel changes  e.g. by triggering
> > udev events when they didn't previously. The only machine I have
> > that is triggering the partition probing is also the only test
> > machine that I have that runs systemd and it didn't have this
> > problem on 3.19.
> > 
> 
> Sigh, thanks Dave. Yes you are correct my patch enabled the
> udev events, as part of fixing ramdisk with partitions.
> This is because if you do not enable them then mount by UUID
> and all sort of lsblk and friends do not work.
> 
> I did try to test this in all kind of ways, xfstest+ext4
> as well, and ran with it on Fedora 20 for a while, sorry
> about that.
> 
> It looks like the system anticipates that ramdisk "should
> not have these events"
> 
> I will send a patch ASAP that re-instates the module_parameter
> for enabling notification, and leaving the default off. It should
> be easy to set the param if one intends to use these utilities.
> 

Thanks Boaz, but I still see the same behavior with the part_show patch.
It seems to be something that broke in systemd on Fedora between
versions systemd-218 and systemd-219. The latter is broken on a 3.19
kernel as well.

I've filed a systemd bug so we'll see what comes of it from that end:

https://bugzilla.redhat.com/show_bug.cgi?id=1196452

Brian

> That said, please do agree with me that there is brokenness in
> systemd?
> 
> BTW: You also said something about the 4k sectors thing, It looks
> like we are pulled in two different directions here. If you will
> want to use DAX on ramdisk then you want it on, if you are not
> using DAX, and wants to use smaller-then-page_size FS blocks than
> you do not want it.
> 
> Please advise what we should do? Maybe only do 4k if BLK_DEV_RAM_DAX
> is set in Kconfig ?
> 
> 
> Sorry for the mess, I'll send a fix ASAP
> 
> > Cheers,
> > Dave.
> > 
> 
> Thanks
> Boaz
> 

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Boaz Harrosh]

On 02/26/2015 07:23 PM, Brian Foster wrote:
<>
> 
> Thanks Boaz, but I still see the same behavior with the part_show patch.
> It seems to be something that broke in systemd on Fedora between
> versions systemd-218 and systemd-219. The latter is broken on a 3.19
> kernel as well.
> 
> I've filed a systemd bug so we'll see what comes of it from that end:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1196452
> 
> Brian
> 

Hi Brian

It says in bugzilla (link above) that this issue is "fixed in git" so
I guess we should be fine ?

Jens does *not* need to take
	[PATCH] brd: Re-instate ram disk visibility option (part_show)

Please confirm.

Please tell me if there is anything I can help with?

Thanks
Boaz

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Brian Foster]

On Sun, Mar 01, 2015 at 10:49:16AM +0200, Boaz Harrosh wrote:
> On 02/26/2015 07:23 PM, Brian Foster wrote:
> <>
> > 
> > Thanks Boaz, but I still see the same behavior with the part_show patch.
> > It seems to be something that broke in systemd on Fedora between
> > versions systemd-218 and systemd-219. The latter is broken on a 3.19
> > kernel as well.
> > 
> > I've filed a systemd bug so we'll see what comes of it from that end:
> > 
> > https://bugzilla.redhat.com/show_bug.cgi?id=1196452
> > 
> > Brian
> > 
> 
> Hi Brian
> 
> It says in bugzilla (link above) that this issue is "fixed in git" so
> I guess we should be fine ?
> 

Yes, I picked up a more recent systemd version and it seems to work fine
now without the patch referenced below.

Brian

> Jens does *not* need to take
> 	[PATCH] brd: Re-instate ram disk visibility option (part_show)
> 
> Please confirm.
> 
> Please tell me if there is anything I can help with?
> 
> Thanks
> Boaz
> 

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Dave Chinner]

On Thu, Feb 26, 2015 at 09:46:27AM +0200, Boaz Harrosh wrote:
> On 02/26/2015 01:43 AM, Dave Chinner wrote:
> > On Wed, Feb 25, 2015 at 06:31:15PM -0500, Brian Foster wrote:
> >> On Thu, Feb 26, 2015 at 09:32:48AM +1100, Dave Chinner wrote:
> >>> [cc linux-fsdevel, Boaz and others]
> >>>
> >>> On Wed, Feb 25, 2015 at 11:11:51AM -0500, Brian Foster wrote:
> >>>> On Wed, Feb 25, 2015 at 09:54:36AM +1100, Dave Chinner wrote:
> >>>>> From: Dave Chinner <dchinner@redhat.com>
> >>>>>
> >>>>> xfs/104, xfs/119, xfs/291 and xfs/297 have small fixed log sizes. A
> >>>>> recent change to the kernel ramdisk changed it's physical sector
> >>>>> size from 512B to 4kB, and this results in mkfs calculating a log
> >>>>> size larger than the fixed test size and hence the tests fail.
> >>>>>
> >>>>> Change the log size to a larger size that works with 4k sectors, and
> >>>>> also increase the size of the filesystem being created so that the
> >>>>> amount of data space in the filesystem does not change and hence
> >>>>> does not perturb the rest of the test.
> >>>>>
> >>>>> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> >>>>> ---
> >>>>
> >>>> Well for some reason I can't mount a ramdisk on the current tot to test
> >>>> this. In fact, I can't mount _anything_ after the ramdisk mount attempt.
> >>>> The mount actually reports success too, but there's nothing there... :/
> >>>>
> >>>> # modprobe brd
> >>>> # mkfs.xfs -f /dev/ram0 
> >>>> meta-data=/dev/ram0              isize=256    agcount=1, agsize=4096
> >>>> blks
> >>>>          =                       sectsz=4096  attr=2, projid32bit=1
> >>>>          =                       crc=0        finobt=0
> >>>> data     =                       bsize=4096   blocks=4096, imaxpct=25
> >>>>          =                       sunit=0      swidth=0 blks
> >>>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
> >>>> log      =internal log           bsize=4096   blocks=1605, version=2
> >>>>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> >>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
> >>>> # mount /dev/ram0 /mnt/
> >>>> # mount | grep mnt
> >>>> # umount  /mnt/
> >>>> umount: /mnt/: not mounted
> >>>>
> >>>> ... and then I can't even mount my normal scratch device until after a
> >>>> reboot:
> >>>>
> >>>> # mount /dev/test/scratch /mnt/
> >>>> # mount | grep mnt
> >>>> # umount  /mnt/
> >>>> umount: /mnt/: not mounted
> >>>
> >>> Ok, so that's just plain broken. What's in dmesg?
> >>>
> >>
> >> Once I got back to this I found that for some reason systemd is
> >> immediately invoking a umount on the mount. :/ No idea why or how to
> >> stop it, but if I do something like this:
> >>
> >> mount /dev/ram0 /mnt; cd /mnt
> >>
> >> ... I can occasionally win the race and get systemd to spin in a
> >> umount() cycle trying to undo the mount. I haven't gone back to confirm
> >> it's the same behavior with the normal devices at that point, but I
> >> suspect it is, perhaps due to getting into some kind of bad state.
> >>
> >> So fyi that this particular problem doesn't appear to be directly kernel
> >> related...
> > 
> > It may still be related to the kernel changes  e.g. by triggering
> > udev events when they didn't previously. The only machine I have
> > that is triggering the partition probing is also the only test
> > machine that I have that runs systemd and it didn't have this
> > problem on 3.19.
> > 
> 
> Sigh, thanks Dave. Yes you are correct my patch enabled the
> udev events, as part of fixing ramdisk with partitions.
> This is because if you do not enable them then mount by UUID
> and all sort of lsblk and friends do not work.

Sure, that's what the gendisk abstraction just you. But why am I
seeing random partition probes on a ramdisk that *isn't using
partitions*?

> I did try to test this in all kind of ways, xfstest+ext4
> as well, and ran with it on Fedora 20 for a while, sorry
> about that.
> 
> It looks like the system anticipates that ramdisk "should
> not have these events"

Right, but not because it's a ramdisk. Those events should not be
occurring because I'm not creating or destroying devices, I'm not
changing partition tables, I'm not resizing ramdisks or partitions,
and so on. I'm simply mkfs'ing, mounting and unmounting filesystems
on the ramdisks - nothing should be generating device based udev
events...

Finding the trigger that is causing these events will tell us what
the bug is - restricting the config won't help, especially as DAX
will *always* be enabled on my test machines as it's something
needed in my test matrix. I'm not sure how to go about finding that
trigger right now and as such I won't really have time to look at it
until after lsfmm/vault...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Boaz Harrosh]

On 02/27/2015 02:58 AM, Dave Chinner wrote:
<>
>>
>> Sigh, thanks Dave. Yes you are correct my patch enabled the
>> udev events, as part of fixing ramdisk with partitions.
>> This is because if you do not enable them then mount by UUID
>> and all sort of lsblk and friends do not work.
> 
> Sure, that's what the gendisk abstraction just you. But why am I
> seeing random partition probes on a ramdisk that *isn't using
> partitions*?
> 

Yes, There should be the one new event on create (modprobe or
mknod) which was not there before. Perhaps it triggers a systemd
process that never used to run before. (And is now sitting there
and making a mess)

<>
>> It looks like the system anticipates that ramdisk "should
>> not have these events"
> 
> Right, but not because it's a ramdisk. Those events should not be
> occurring because I'm not creating or destroying devices, I'm not
> changing partition tables, I'm not resizing ramdisks or partitions,
> and so on. I'm simply mkfs'ing, mounting and unmounting filesystems
> on the ramdisks - nothing should be generating device based udev
> events...
> 
> Finding the trigger that is causing these events will tell us what
> the bug is - 

> restricting the config won't help, especially as DAX
> will *always* be enabled on my test machines as it's something
> needed in my test matrix.

No the "if DAX" is for the 4k thing. The enablement of the uevents is
with a new "part_show" module parameter (See patch-1).

> I'm not sure how to go about finding that
> trigger right now and as such I won't really have time to look at it
> until after lsfmm/vault...
> 

I'll try to reproduce this here. What Fedora version do I need?

> Cheers,
> Dave.
> 

Thanks
Boaz

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Dave Chinner]

On Sun, Mar 01, 2015 at 10:27:49AM +0200, Boaz Harrosh wrote:
> On 02/27/2015 02:58 AM, Dave Chinner wrote:
> >> It looks like the system anticipates that ramdisk "should
> >> not have these events"
> > 
> > Right, but not because it's a ramdisk. Those events should not be
> > occurring because I'm not creating or destroying devices, I'm not
> > changing partition tables, I'm not resizing ramdisks or partitions,
> > and so on. I'm simply mkfs'ing, mounting and unmounting filesystems
> > on the ramdisks - nothing should be generating device based udev
> > events...
> > 
> > Finding the trigger that is causing these events will tell us what
> > the bug is - 
> 
> > restricting the config won't help, especially as DAX
> > will *always* be enabled on my test machines as it's something
> > needed in my test matrix.
> 
> No the "if DAX" is for the 4k thing. The enablement of the uevents is
> with a new "part_show" module parameter (See patch-1).

Sure, but that doesn't answer my question: what is generating device
level uevents when all I'm doing is mkfs/mount/umount on the device?

> > I'm not sure how to go about finding that
> > trigger right now and as such I won't really have time to look at it
> > until after lsfmm/vault...
> > 
> 
> I'll try to reproduce this here. What Fedora version do I need?

I'm running debian unstable w/ systemd-215 on the particular test
machine that is hitting this problem.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: ramdisk problems in 4.0-rc1? (was Re: [PATCH 1/4] xfs/104: log size too small for 4k sector drives)

[Boaz Harrosh]

On 03/02/2015 03:09 AM, Dave Chinner wrote:
> On Sun, Mar 01, 2015 at 10:27:49AM +0200, Boaz Harrosh wrote:
>> On 02/27/2015 02:58 AM, Dave Chinner wrote:
<>
>> No the "if DAX" is for the 4k thing. The enablement of the uevents is
>> with a new "part_show" module parameter (See patch-1).
> 
> Sure, but that doesn't answer my question: what is generating device
> level uevents when all I'm doing is mkfs/mount/umount on the device?
> 

I was suspecting it is this systemd bug which keeps trying to tier-down
the devices.

>>> I'm not sure how to go about finding that
>>> trigger right now and as such I won't really have time to look at it
>>> until after lsfmm/vault...
>>>
>>
>> I'll try to reproduce this here. What Fedora version do I need?
> 
> I'm running debian unstable w/ systemd-215 on the particular test
> machine that is hitting this problem.
> 

Oooff, On my fedora 20 I'm at systemd 208. I'll see if I have time to
install an fc21 vm or maybe upgrade from source. (Any easy way?)

I have setup my xfs rig and ran "./check -g auto" by now. I tried both
part_show=1/0 and both look working as expected.
(Do I need any special $MKFS_OPTIONS or anything else)

I'll probably be giving up soon, and will just wait for more reports.

With the patch-1 I sent I am reverting to old behavior so I need a
reproducer to try and run with patch-1 and part_show=1 should show
the problem and part_show=0 should not. Else this is something else

> Cheers,
> Dave.
> 

Thanks Dave, sorry for trapping you in this boring mess, life
at Kernel the one reproducing the problem needs to help fix it ;-)

Have an enjoyable and productive LSF
Thanks
Boaz

[PATCH] brd: Re-instate ram disk visibility option (part_show)

[Boaz Harrosh]

This re-enables part_show option, so we can keep it false by
default.

Here is what Dmitry said in the original patch:
  [aeac318] brd: add ram disk visibility option
	Currently ram disk is not visible inside /proc/partitions.
	This was done for compatibility reasons here: 53978d0a7a27.
	But some utilities expect disk presents in /proc/partitions.
	Let's add module's option and let's administrator chose
	visibility behavior. By default, old behavior preserved.

Dave Chinner and other have reported problems with current system
when udev events start firing for ramdisk:
	http://marc.info/?l=linux-fsdevel&m=142490357918672&w=2

It was me who enabled these notifications through this patch:
   [937af5e] brd: Fix all partitions BUGs
Sorry for the mess.

CC: Dave Chinner <david@fromorbit.com>
CC: Dmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: Boaz Harrosh <boaz@plexistor.com>
---
 drivers/block/brd.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 64ab495..6e0775b 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -450,6 +450,10 @@ static int max_part = 1;
 module_param(max_part, int, S_IRUGO);
 MODULE_PARM_DESC(max_part, "Num Minors to reserve between devices");
 
+static int part_show = 0;
+module_param(part_show, int, S_IRUGO);
+MODULE_PARM_DESC(part_show, "Control RAM disk visibility in /proc/partitions");
+
 MODULE_LICENSE("GPL");
 MODULE_ALIAS_BLOCKDEV_MAJOR(RAMDISK_MAJOR);
 MODULE_ALIAS("rd");
@@ -513,6 +517,8 @@ static struct brd_device *brd_alloc(int i)
 	disk->private_data	= brd;
 	disk->queue		= brd->brd_queue;
 	disk->flags		= GENHD_FL_EXT_DEVT;
+	if (!part_show)
+		disk->flags |= GENHD_FL_SUPPRESS_PARTITION_INFO;
 	sprintf(disk->disk_name, "ram%d", i);
 	set_capacity(disk, rd_size * 2);
 
-- 
1.9.3

[PATCH] brd: Only request 4K sectors if DAX is enabled

[Boaz Harrosh]

People systems have been using ramdisk with
smaller-than-page-size blocks in their mkfs.

The 4K sectors is only important if we will be
using brd with a DAX filesystem.

So only enable 4K sectors if DAX is configured

Signed-off-by: Boaz Harrosh <boaz@plexistor.com>
---
 drivers/block/brd.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 6e0775b..e875f12 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -495,6 +495,7 @@ static struct brd_device *brd_alloc(int i)
 	blk_queue_max_hw_sectors(brd->brd_queue, 1024);
 	blk_queue_bounce_limit(brd->brd_queue, BLK_BOUNCE_ANY);
 
+#ifdef CONFIG_BLK_DEV_RAM_DAX
 	/* This is so fdisk will align partitions on 4k, because of
 	 * direct_access API needing 4k alignment, returning a PFN
 	 * (This is only a problem on very small devices <= 4M,
@@ -502,6 +503,7 @@ static struct brd_device *brd_alloc(int i)
 	 *  is harmless)
 	 */
 	blk_queue_physical_block_size(brd->brd_queue, PAGE_SIZE);
+#endif
 
 	brd->brd_queue->limits.discard_granularity = PAGE_SIZE;
 	brd->brd_queue->limits.max_discard_sectors = UINT_MAX;
-- 
1.9.3

Re: [PATCH] brd: Only request 4K sectors if DAX is enabled

[Boaz Harrosh]

On 02/26/2015 10:41 AM, Boaz Harrosh wrote:
> 
> People systems have been using ramdisk with
> smaller-than-page-size blocks in their mkfs.
> 
> The 4K sectors is only important if we will be
> using brd with a DAX filesystem.
> 
> So only enable 4K sectors if DAX is configured
> 
> Signed-off-by: Boaz Harrosh <boaz@plexistor.com>
> ---
>  drivers/block/brd.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/block/brd.c b/drivers/block/brd.c
> index 6e0775b..e875f12 100644
> --- a/drivers/block/brd.c
> +++ b/drivers/block/brd.c
> @@ -495,6 +495,7 @@ static struct brd_device *brd_alloc(int i)
>  	blk_queue_max_hw_sectors(brd->brd_queue, 1024);
>  	blk_queue_bounce_limit(brd->brd_queue, BLK_BOUNCE_ANY);
>  
> +#ifdef CONFIG_BLK_DEV_RAM_DAX
>  	/* This is so fdisk will align partitions on 4k, because of
>  	 * direct_access API needing 4k alignment, returning a PFN
>  	 * (This is only a problem on very small devices <= 4M,
> @@ -502,6 +503,7 @@ static struct brd_device *brd_alloc(int i)
>  	 *  is harmless)
>  	 */

Hi Dave

I was just thinking should we also do

+	if (part_show)
>  		blk_queue_physical_block_size(brd->brd_queue, PAGE_SIZE);

The rational is that if part_show is off then we are not using partitions
at all and then there can be no problems?

I'll try to run your tests here. I never test xfs always ext4, maybe I should
start to ;-). Actually I do have xfsprogs so why not. But if you could run
a quick test as well it could be great.

Thanks
Boaz

<>

Re: [PATCH] brd: Only request 4K sectors if DAX is enabled

[Dave Chinner]

On Thu, Feb 26, 2015 at 10:41:46AM +0200, Boaz Harrosh wrote:
> 
> People systems have been using ramdisk with
> smaller-than-page-size blocks in their mkfs.
> 
> The 4K sectors is only important if we will be
> using brd with a DAX filesystem.
> 
> So only enable 4K sectors if DAX is configured

4k sectors are not a problem - we should be handling them fine and
because it makes the ramdisk look like a 512e drive, no applications
should fail, either.

The main "unexpected" part about it was how much of xfstests didn't
handle 4k sectors in mkfs output properly. This isn't a problem with
the kernel change and so doesn't need fixing.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] brd: Only request 4K sectors if DAX is enabled

[Boaz Harrosh]

On 02/27/2015 02:23 AM, Dave Chinner wrote:
> On Thu, Feb 26, 2015 at 10:41:46AM +0200, Boaz Harrosh wrote:
>>
>> People systems have been using ramdisk with
>> smaller-than-page-size blocks in their mkfs.
>>
>> The 4K sectors is only important if we will be
>> using brd with a DAX filesystem.
>>
>> So only enable 4K sectors if DAX is configured
> 
> 4k sectors are not a problem - we should be handling them fine and
> because it makes the ramdisk look like a 512e drive, no applications
> should fail, either.
> 
> The main "unexpected" part about it was how much of xfstests didn't
> handle 4k sectors in mkfs output properly. This isn't a problem with
> the kernel change and so doesn't need fixing.
> 

Glad to be of service ;-)

Please tell me if there is anything I can help with

> Cheers,
> Dave.
> 

Thanks
Boaz

[PATCH 2/4] xfs: don't output mkfs sector sizes into golden output

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

The mkfs sector sizes are dependent on the underlying device in use,
and so is not fixed. hence it needs to be filtered from any golden
output file, otherwise tests that just differ by sector size will
fail.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 tests/xfs/096          |  8 ++++++--
 tests/xfs/096.external | 15 +++++++--------
 tests/xfs/096.internal | 16 ++++++++--------
 tests/xfs/119          |  2 +-
 tests/xfs/206          | 33 +++++++++++++++------------------
 5 files changed, 37 insertions(+), 37 deletions(-)

diff --git a/tests/xfs/096 b/tests/xfs/096
index 0ecf88f..2da8fa1 100755
--- a/tests/xfs/096
+++ b/tests/xfs/096
@@ -47,6 +47,8 @@ _cleanup()
 #
 # filter out counts which will vary
 #   - extsz, blocks, agsize, agcount, device name, rtextents
+#   - log version varies for crc enabled fs
+#   - lsunit varies for 512/4k sector devices
 # filter out differences between linux and irix:
 #   - sectsz on Linux
 #   - mmr, mixed-case on IRIX
@@ -63,7 +65,7 @@ _cleanup()
 #           =                       sunit=0 blks
 #  realtime =none                   extsz=65536  blocks=N, rtextents=N
 #
-_mkfs_filter()
+mkfs_filter()
 {
    tee -a $seqres.full | \
    sed \
@@ -80,6 +82,8 @@ _mkfs_filter()
 	-e '/ *= *mmr=[0-9][0-9]* *$/d' \
 	-e 's/ *mixed-case=[YN]//' \
 	-e 's/ *ascii-ci=[01]//' \
+	-e 's/\(version=\)\([12]\)/\1N/' \
+	-e 's/\(sunit=\)\([018] blks\)/\1N blks/' \
 	-e 's/sectsz=[0-9][0-9]* *//' \
 	-e 's/, lazy-count.*//' \
 	-e '/inode-paths/d' \
@@ -145,7 +149,7 @@ do
     fi
     echo "--- mkfs=$mkfs ---"
     export MKFS_OPTIONS="$mkfs"
-    _scratch_mkfs_xfs 2>&1 | _mkfs_filter
+    _scratch_mkfs_xfs 2>&1 | mkfs_filter
     echo ""
     echo ""
 done
diff --git a/tests/xfs/096.external b/tests/xfs/096.external
index 95833c8..7923340 100644
--- a/tests/xfs/096.external
+++ b/tests/xfs/096.external
@@ -11,8 +11,7 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=0      swidth=0 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -22,8 +21,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -33,8 +32,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -44,8 +43,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
diff --git a/tests/xfs/096.internal b/tests/xfs/096.internal
index 7bf848c..eaba13f 100644
--- a/tests/xfs/096.internal
+++ b/tests/xfs/096.internal
@@ -11,8 +11,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=0      swidth=0 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -22,8 +22,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=2
-         =                       sunit=8 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -33,8 +33,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
@@ -44,8 +44,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
 data     =                       bsize=4096   blocks=N, imaxpct=N
          =                       sunit=65     swidth=65 blks, unwritten=1
 naming   =version 2              bsize=4096
-log      =LOG                    bsize=4096   blocks=N, version=1
-         =                       sunit=0 blks
+log      =LOG                    bsize=4096   blocks=N, version=N
+         =                       sunit=N blks
 realtime =REALTIME               extsz=N, blocks=N, rtextents=N
 
 
diff --git a/tests/xfs/119 b/tests/xfs/119
index 490495b..156d5e4 100755
--- a/tests/xfs/119
+++ b/tests/xfs/119
@@ -54,7 +54,7 @@ _require_scratch
 # this may hang
 sync
 
-export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 
+export MKFS_OPTIONS="-l version=2,size=2560b,su=64k" 
 export MOUNT_OPTIONS="-o logbsize=64k"
 _scratch_mkfs_xfs >/dev/null
 
diff --git a/tests/xfs/206 b/tests/xfs/206
index f6dcca6..198e413 100755
--- a/tests/xfs/206
+++ b/tests/xfs/206
@@ -73,34 +73,31 @@ echo "=== truncate file ==="
 dd if=/dev/zero of=$tmpfile bs=1 seek=19998630180864 count=1 >/dev/null 2>&1 \
 	|| _fail "!!! failed to truncate loopback file to correct size"
 
+mkfs_filter()
+{
+	sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
+	    -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
+	    -e "s/, projid32bit=[0-9]//" \
+	    -e "s/ ftype=[0-9]//" \
+	    -e "s/\(sectsz\)\(=[0-9]* *\)/\1=512   /" \
+	    -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \
+	    -e "/.*crc=/d"
+}
+
 # mkfs slightly smaller than that
 echo "=== mkfs.xfs ==="
-mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b -llazy-count=0 $tmpfile \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b \
+	 -llazy-count=0 $tmpfile  | mkfs_filter
 
 mount -o loop $tmpfile $tmpdir || _fail "!!! failed to loopback mount"
 
 # see what happens when we growfs it
 echo "=== xfs_growfs ==="
-xfs_growfs $tmpdir \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+xfs_growfs $tmpdir  | mkfs_filter
 
 # and double-check the new geometry
 echo "=== xfs_info ==="
-xfs_info $tmpdir \
-	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
-		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
-		-e "s/, projid32bit=[0-9]//" \
-		-e "s/ ftype=[0-9]//" \
-		-e "/.*crc=/d"
+xfs_info $tmpdir | mkfs_filter
 
 # _cleanup cleans up for us
 
-- 
2.0.0

Re: [PATCH 2/4] xfs: don't output mkfs sector sizes into golden output

[Brian Foster]

On Wed, Feb 25, 2015 at 09:54:37AM +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> The mkfs sector sizes are dependent on the underlying device in use,
> and so is not fixed. hence it needs to be filtered from any golden
> output file, otherwise tests that just differ by sector size will
> fail.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>  tests/xfs/096          |  8 ++++++--
>  tests/xfs/096.external | 15 +++++++--------
>  tests/xfs/096.internal | 16 ++++++++--------
>  tests/xfs/119          |  2 +-
>  tests/xfs/206          | 33 +++++++++++++++------------------
>  5 files changed, 37 insertions(+), 37 deletions(-)
> 
> diff --git a/tests/xfs/096 b/tests/xfs/096
> index 0ecf88f..2da8fa1 100755
> --- a/tests/xfs/096
> +++ b/tests/xfs/096
> @@ -47,6 +47,8 @@ _cleanup()
>  #
>  # filter out counts which will vary
>  #   - extsz, blocks, agsize, agcount, device name, rtextents
> +#   - log version varies for crc enabled fs
> +#   - lsunit varies for 512/4k sector devices
>  # filter out differences between linux and irix:
>  #   - sectsz on Linux
>  #   - mmr, mixed-case on IRIX
> @@ -63,7 +65,7 @@ _cleanup()
>  #           =                       sunit=0 blks
>  #  realtime =none                   extsz=65536  blocks=N, rtextents=N
>  #
> -_mkfs_filter()
> +mkfs_filter()
>  {
>     tee -a $seqres.full | \
>     sed \
> @@ -80,6 +82,8 @@ _mkfs_filter()
>  	-e '/ *= *mmr=[0-9][0-9]* *$/d' \
>  	-e 's/ *mixed-case=[YN]//' \
>  	-e 's/ *ascii-ci=[01]//' \
> +	-e 's/\(version=\)\([12]\)/\1N/' \
> +	-e 's/\(sunit=\)\([018] blks\)/\1N blks/' \
>  	-e 's/sectsz=[0-9][0-9]* *//' \
>  	-e 's/, lazy-count.*//' \
>  	-e '/inode-paths/d' \
> @@ -145,7 +149,7 @@ do
>      fi
>      echo "--- mkfs=$mkfs ---"
>      export MKFS_OPTIONS="$mkfs"
> -    _scratch_mkfs_xfs 2>&1 | _mkfs_filter
> +    _scratch_mkfs_xfs 2>&1 | mkfs_filter
>      echo ""
>      echo ""
>  done
> diff --git a/tests/xfs/096.external b/tests/xfs/096.external
> index 95833c8..7923340 100644
> --- a/tests/xfs/096.external
> +++ b/tests/xfs/096.external
> @@ -11,8 +11,7 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=0      swidth=0 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=2
> -         =                       sunit=8 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> @@ -22,8 +21,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=65     swidth=65 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=2
> -         =                       sunit=0 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> @@ -33,8 +32,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=65     swidth=65 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=1
> -         =                       sunit=0 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> @@ -44,8 +43,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=65     swidth=65 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=1
> -         =                       sunit=0 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> diff --git a/tests/xfs/096.internal b/tests/xfs/096.internal
> index 7bf848c..eaba13f 100644
> --- a/tests/xfs/096.internal
> +++ b/tests/xfs/096.internal
> @@ -11,8 +11,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=0      swidth=0 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=2
> -         =                       sunit=8 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> @@ -22,8 +22,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=65     swidth=65 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=2
> -         =                       sunit=8 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> @@ -33,8 +33,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=65     swidth=65 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=1
> -         =                       sunit=0 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> @@ -44,8 +44,8 @@ meta-data=DEV isize=256    agcount=N, agsize=N blks
>  data     =                       bsize=4096   blocks=N, imaxpct=N
>           =                       sunit=65     swidth=65 blks, unwritten=1
>  naming   =version 2              bsize=4096
> -log      =LOG                    bsize=4096   blocks=N, version=1
> -         =                       sunit=0 blks
> +log      =LOG                    bsize=4096   blocks=N, version=N
> +         =                       sunit=N blks
>  realtime =REALTIME               extsz=N, blocks=N, rtextents=N
>  
>  
> diff --git a/tests/xfs/119 b/tests/xfs/119
> index 490495b..156d5e4 100755
> --- a/tests/xfs/119
> +++ b/tests/xfs/119
> @@ -54,7 +54,7 @@ _require_scratch
>  # this may hang
>  sync
>  
> -export MKFS_OPTIONS="-l version=2,size=2500b,su=64k" 
> +export MKFS_OPTIONS="-l version=2,size=2560b,su=64k" 

Not clear if this was intended to be part of the previous patch?
Anyways, still a trailing space here. Otherwise this looks fine to me:

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  export MOUNT_OPTIONS="-o logbsize=64k"
>  _scratch_mkfs_xfs >/dev/null
>  
> diff --git a/tests/xfs/206 b/tests/xfs/206
> index f6dcca6..198e413 100755
> --- a/tests/xfs/206
> +++ b/tests/xfs/206
> @@ -73,34 +73,31 @@ echo "=== truncate file ==="
>  dd if=/dev/zero of=$tmpfile bs=1 seek=19998630180864 count=1 >/dev/null 2>&1 \
>  	|| _fail "!!! failed to truncate loopback file to correct size"
>  
> +mkfs_filter()
> +{
> +	sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
> +	    -e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
> +	    -e "s/, projid32bit=[0-9]//" \
> +	    -e "s/ ftype=[0-9]//" \
> +	    -e "s/\(sectsz\)\(=[0-9]* *\)/\1=512   /" \
> +	    -e "s/\(sunit=\)\([0-9]* blks,\)/\10 blks,/" \
> +	    -e "/.*crc=/d"
> +}
> +
>  # mkfs slightly smaller than that
>  echo "=== mkfs.xfs ==="
> -mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b -llazy-count=0 $tmpfile \
> -	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
> -		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
> -		-e "s/, projid32bit=[0-9]//" \
> -		-e "s/ ftype=[0-9]//" \
> -		-e "/.*crc=/d"
> +mkfs.xfs -f -bsize=4096 -dagsize=76288719b,size=3905982455b \
> +	 -llazy-count=0 $tmpfile  | mkfs_filter
>  
>  mount -o loop $tmpfile $tmpdir || _fail "!!! failed to loopback mount"
>  
>  # see what happens when we growfs it
>  echo "=== xfs_growfs ==="
> -xfs_growfs $tmpdir \
> -	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
> -		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
> -		-e "s/, projid32bit=[0-9]//" \
> -		-e "s/ ftype=[0-9]//" \
> -		-e "/.*crc=/d"
> +xfs_growfs $tmpdir  | mkfs_filter
>  
>  # and double-check the new geometry
>  echo "=== xfs_info ==="
> -xfs_info $tmpdir \
> -	| sed -e "s,^meta-data=.*isize,meta-data=FILE                   isize,g" \
> -		-e "s/\(^log.*blocks=\)\([0-9]*,\)/\1XXXXX,/" \
> -		-e "s/, projid32bit=[0-9]//" \
> -		-e "s/ ftype=[0-9]//" \
> -		-e "/.*crc=/d"
> +xfs_info $tmpdir | mkfs_filter
>  
>  # _cleanup cleans up for us
>  
> -- 
> 2.0.0
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/4] xfs/049: umount -d fails when kernel wins teardown race

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

When /etc/mtab is linked to /proc/mounts and we are using mount time
created loop devices (i.e. mount -o loop), the unmount can fail
with this amazingly informative error message:

umount: /mnt/scratch/test2: filesystem was unmounted, but mount(8) failed: Invalid argument

What it actually means in this case is that the kernel tore down the
loop device when the last reference went away, and it did it so fast
that mount was not able to find it in /etc/mtab after the unmount
syscall. Hence it could not find the loop device it was supposed to
tear down and has a hissy fit.

This is simple to fix: mount does not need to tear down the loop
device as the kernel does it automatically. Remove the "-d" from
the umount command, and the test passes again.

There's quite a few other tests that also use umount -d - fix them
as well.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 tests/xfs/049 | 8 ++++----
 tests/xfs/073 | 6 +++---
 tests/xfs/078 | 4 ++--
 tests/xfs/216 | 2 +-
 tests/xfs/217 | 2 +-
 tests/xfs/250 | 4 ++--
 6 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/tests/xfs/049 b/tests/xfs/049
index 04c2c75..8d4e074 100755
--- a/tests/xfs/049
+++ b/tests/xfs/049
@@ -29,8 +29,8 @@ echo "QA output created by $seq"
 _cleanup()
 {
     cd /
-    umount -d $SCRATCH_MNT/test2 > /dev/null 2>&1
-    umount -d $SCRATCH_MNT/test > /dev/null 2>&1
+    umount $SCRATCH_MNT/test2 > /dev/null 2>&1
+    umount $SCRATCH_MNT/test > /dev/null 2>&1
     rm -f $tmp.*
 
     if [ -w $seqres.full ]
@@ -123,11 +123,11 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \
     || _fail "!!! clean failed"
 
 _log "umount ext2 on xfs"
-umount -d $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
+umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
     || _fail "!!! umount ext2 failed"
 
 _log "umount xfs"
-umount -d $SCRATCH_MNT/test >> $seqres.full 2>&1 \
+umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \
     || _fail "!!! umount xfs failed"
 
 echo "--- mounts at end (before cleanup)" >> $seqres.full
diff --git a/tests/xfs/073 b/tests/xfs/073
index f955771..38ed2cb 100755
--- a/tests/xfs/073
+++ b/tests/xfs/073
@@ -41,9 +41,9 @@ _cleanup()
 {
 	cd /
 	umount $SCRATCH_MNT 2>/dev/null
-	umount -d $imgs.loop 2>/dev/null
+	umount $imgs.loop 2>/dev/null
 	[ -d $imgs.loop ] && rmdir $imgs.loop
-	umount -d $imgs.source_dir 2>/dev/null
+	umount $imgs.source_dir 2>/dev/null
 	[ -d $imgs.source_dir ] && rm -rf $imgs.source_dir
 	rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*
 }
@@ -119,7 +119,7 @@ _verify_copy()
 
 	echo unmounting and removing new image
 	umount $source_dir
-	umount -d $target_dir > /dev/null 2>&1
+	umount $target_dir > /dev/null 2>&1
 	rm -f $target
 }
 
diff --git a/tests/xfs/078 b/tests/xfs/078
index f859efc..d8cb919 100755
--- a/tests/xfs/078
+++ b/tests/xfs/078
@@ -36,7 +36,7 @@ _cleanup()
 {
     cd /
     rm -f $tmp.*
-    umount -d $LOOP_MNT 2>/dev/null
+    umount $LOOP_MNT 2>/dev/null
     rmdir $LOOP_MNT
 }
 
@@ -97,7 +97,7 @@ _grow_loop()
 	$XFS_GROWFS_PROG $LOOP_MNT 2>&1 |  _filter_growfs 2>&1
 
 	echo "*** unmount"
-	umount -d $LOOP_MNT > /dev/null 2>&1
+	umount $LOOP_MNT > /dev/null 2>&1
 
 	# Large grows takes forever to check..
 	if [ "$check" -gt "0" ]
diff --git a/tests/xfs/216 b/tests/xfs/216
index 8513479..76f79ca 100755
--- a/tests/xfs/216
+++ b/tests/xfs/216
@@ -60,7 +60,7 @@ _do_mkfs()
 			-d name=$LOOP_DEV,size=${i}g |grep log
 		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
 		echo "test write" > $LOOP_MNT/test
-		umount -d $LOOP_MNT > /dev/null 2>&1
+		umount $LOOP_MNT > /dev/null 2>&1
 	done
 }
 # make large holey file
diff --git a/tests/xfs/217 b/tests/xfs/217
index ab55a30..8aacdf9 100755
--- a/tests/xfs/217
+++ b/tests/xfs/217
@@ -62,7 +62,7 @@ _do_mkfs()
 			-d name=$LOOP_DEV,size=${i}g |grep log
 		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
 		echo "test write" > $LOOP_MNT/test
-		umount -d $LOOP_MNT > /dev/null 2>&1
+		umount $LOOP_MNT > /dev/null 2>&1
 
 		# punch out the previous blocks so that we keep the amount of
 		# disk space the test requires down to a minimum.
diff --git a/tests/xfs/250 b/tests/xfs/250
index c1622a4..0cdc382 100755
--- a/tests/xfs/250
+++ b/tests/xfs/250
@@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
 _cleanup()
 {
 	cd /
-	umount -d $LOOP_MNT 2>/dev/null
+	umount $LOOP_MNT 2>/dev/null
 	rm -f $LOOP_DEV
 	rmdir $LOOP_MNT
 }
@@ -84,7 +84,7 @@ _test_loop()
 	xfs_io -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io
 
 	echo "*** unmount loop filesystem"
-	umount -d $LOOP_MNT > /dev/null 2>&1
+	umount $LOOP_MNT > /dev/null 2>&1
 
 	echo "*** check loop filesystem"
 	 _check_xfs_filesystem $LOOP_DEV none none
-- 
2.0.0

Re: [PATCH 3/4] xfs/049: umount -d fails when kernel wins teardown race

[Brian Foster]

On Wed, Feb 25, 2015 at 09:54:38AM +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> When /etc/mtab is linked to /proc/mounts and we are using mount time
> created loop devices (i.e. mount -o loop), the unmount can fail
> with this amazingly informative error message:
> 
> umount: /mnt/scratch/test2: filesystem was unmounted, but mount(8) failed: Invalid argument
> 
> What it actually means in this case is that the kernel tore down the
> loop device when the last reference went away, and it did it so fast
> that mount was not able to find it in /etc/mtab after the unmount
> syscall. Hence it could not find the loop device it was supposed to
> tear down and has a hissy fit.
> 
> This is simple to fix: mount does not need to tear down the loop
> device as the kernel does it automatically. Remove the "-d" from
> the umount command, and the test passes again.
> 
> There's quite a few other tests that also use umount -d - fix them
> as well.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  tests/xfs/049 | 8 ++++----
>  tests/xfs/073 | 6 +++---
>  tests/xfs/078 | 4 ++--
>  tests/xfs/216 | 2 +-
>  tests/xfs/217 | 2 +-
>  tests/xfs/250 | 4 ++--
>  6 files changed, 13 insertions(+), 13 deletions(-)
> 
> diff --git a/tests/xfs/049 b/tests/xfs/049
> index 04c2c75..8d4e074 100755
> --- a/tests/xfs/049
> +++ b/tests/xfs/049
> @@ -29,8 +29,8 @@ echo "QA output created by $seq"
>  _cleanup()
>  {
>      cd /
> -    umount -d $SCRATCH_MNT/test2 > /dev/null 2>&1
> -    umount -d $SCRATCH_MNT/test > /dev/null 2>&1
> +    umount $SCRATCH_MNT/test2 > /dev/null 2>&1
> +    umount $SCRATCH_MNT/test > /dev/null 2>&1
>      rm -f $tmp.*
>  
>      if [ -w $seqres.full ]
> @@ -123,11 +123,11 @@ rm -rf $SCRATCH_MNT/test/* >> $seqres.full 2>&1 \
>      || _fail "!!! clean failed"
>  
>  _log "umount ext2 on xfs"
> -umount -d $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
> +umount $SCRATCH_MNT/test2 >> $seqres.full 2>&1 \
>      || _fail "!!! umount ext2 failed"
>  
>  _log "umount xfs"
> -umount -d $SCRATCH_MNT/test >> $seqres.full 2>&1 \
> +umount $SCRATCH_MNT/test >> $seqres.full 2>&1 \
>      || _fail "!!! umount xfs failed"
>  
>  echo "--- mounts at end (before cleanup)" >> $seqres.full
> diff --git a/tests/xfs/073 b/tests/xfs/073
> index f955771..38ed2cb 100755
> --- a/tests/xfs/073
> +++ b/tests/xfs/073
> @@ -41,9 +41,9 @@ _cleanup()
>  {
>  	cd /
>  	umount $SCRATCH_MNT 2>/dev/null
> -	umount -d $imgs.loop 2>/dev/null
> +	umount $imgs.loop 2>/dev/null
>  	[ -d $imgs.loop ] && rmdir $imgs.loop
> -	umount -d $imgs.source_dir 2>/dev/null
> +	umount $imgs.source_dir 2>/dev/null
>  	[ -d $imgs.source_dir ] && rm -rf $imgs.source_dir
>  	rm -f $imgs.* $tmp.* /var/tmp/xfs_copy.log.*
>  }
> @@ -119,7 +119,7 @@ _verify_copy()
>  
>  	echo unmounting and removing new image
>  	umount $source_dir
> -	umount -d $target_dir > /dev/null 2>&1
> +	umount $target_dir > /dev/null 2>&1
>  	rm -f $target
>  }
>  
> diff --git a/tests/xfs/078 b/tests/xfs/078
> index f859efc..d8cb919 100755
> --- a/tests/xfs/078
> +++ b/tests/xfs/078
> @@ -36,7 +36,7 @@ _cleanup()
>  {
>      cd /
>      rm -f $tmp.*
> -    umount -d $LOOP_MNT 2>/dev/null
> +    umount $LOOP_MNT 2>/dev/null
>      rmdir $LOOP_MNT
>  }
>  
> @@ -97,7 +97,7 @@ _grow_loop()
>  	$XFS_GROWFS_PROG $LOOP_MNT 2>&1 |  _filter_growfs 2>&1
>  
>  	echo "*** unmount"
> -	umount -d $LOOP_MNT > /dev/null 2>&1
> +	umount $LOOP_MNT > /dev/null 2>&1
>  
>  	# Large grows takes forever to check..
>  	if [ "$check" -gt "0" ]
> diff --git a/tests/xfs/216 b/tests/xfs/216
> index 8513479..76f79ca 100755
> --- a/tests/xfs/216
> +++ b/tests/xfs/216
> @@ -60,7 +60,7 @@ _do_mkfs()
>  			-d name=$LOOP_DEV,size=${i}g |grep log
>  		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
>  		echo "test write" > $LOOP_MNT/test
> -		umount -d $LOOP_MNT > /dev/null 2>&1
> +		umount $LOOP_MNT > /dev/null 2>&1
>  	done
>  }
>  # make large holey file
> diff --git a/tests/xfs/217 b/tests/xfs/217
> index ab55a30..8aacdf9 100755
> --- a/tests/xfs/217
> +++ b/tests/xfs/217
> @@ -62,7 +62,7 @@ _do_mkfs()
>  			-d name=$LOOP_DEV,size=${i}g |grep log
>  		mount -o loop -t xfs $LOOP_DEV $LOOP_MNT
>  		echo "test write" > $LOOP_MNT/test
> -		umount -d $LOOP_MNT > /dev/null 2>&1
> +		umount $LOOP_MNT > /dev/null 2>&1
>  
>  		# punch out the previous blocks so that we keep the amount of
>  		# disk space the test requires down to a minimum.
> diff --git a/tests/xfs/250 b/tests/xfs/250
> index c1622a4..0cdc382 100755
> --- a/tests/xfs/250
> +++ b/tests/xfs/250
> @@ -33,7 +33,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
>  _cleanup()
>  {
>  	cd /
> -	umount -d $LOOP_MNT 2>/dev/null
> +	umount $LOOP_MNT 2>/dev/null
>  	rm -f $LOOP_DEV
>  	rmdir $LOOP_MNT
>  }
> @@ -84,7 +84,7 @@ _test_loop()
>  	xfs_io -f -c "resvsp 0 $fsize" $LOOP_MNT/foo | _filter_io
>  
>  	echo "*** unmount loop filesystem"
> -	umount -d $LOOP_MNT > /dev/null 2>&1
> +	umount $LOOP_MNT > /dev/null 2>&1
>  
>  	echo "*** check loop filesystem"
>  	 _check_xfs_filesystem $LOOP_DEV none none
> -- 
> 2.0.0
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/4] generic: Add rudimetary RENAME_WHITEOUT test

[Dave Chinner]

From: Dave Chinner <dchinner@redhat.com>

There is no API documentation for RENAME_WHITEOUT. There is no
developer documentation for RENAME_WHITEOUT. There are not comments
in the overlayfs or ext4 implementation of RENAME_WHITEOUT.

Hence, this test simply tries to expose basic RENAME_WHITEOUT
behaviour from ext4 so we can reverse-engineer and verify
bug-for-bug renameat2(RENAME_WHITEOUT) ext4 compatibility.

Note: uses generic/078 just to keep out of the way of the 6-7 other
pending new tests.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
---
 src/renameat2.c       |  4 ++--
 tests/generic/078     | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/generic/078.out | 51 +++++++++++++++++++++++++++++++++++++++
 tests/generic/group   |  1 +
 4 files changed, 120 insertions(+), 2 deletions(-)
 create mode 100755 tests/generic/078
 create mode 100644 tests/generic/078.out

diff --git a/src/renameat2.c b/src/renameat2.c
index 5ac0936..c59ce65 100644
--- a/src/renameat2.c
+++ b/src/renameat2.c
@@ -96,9 +96,9 @@ int main(int argc, char *argv[])
 		 * Turn EEXIST into ENOTEMPTY.  E.g. XFS uses EEXIST, and that
 		 * is also accepted by the standards.
 		 *
-		 * This applies only to plain rename (flags == 0).
+		 * This applies only to plain rename and RENAME_WHITEOUT
 		 */
-		if (!flags && errno == EEXIST)
+		if (errno == EEXIST && (!flags || (flags & RENAME_WHITEOUT)))
 			errno = ENOTEMPTY;
 
 		perror("");
diff --git a/tests/generic/078 b/tests/generic/078
new file mode 100755
index 0000000..92ece0c
--- /dev/null
+++ b/tests/generic/078
@@ -0,0 +1,66 @@
+#! /bin/bash
+# FS QA Test No. generic/078
+#
+# Check renameat2 syscall with RENAME_WHITEOUT flag
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2014 Miklos Szeredi.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/renameat2
+
+_supported_fs generic
+_supported_os Linux
+
+_require_test
+_requires_renameat2
+_require_test_symlinks
+
+rename_dir=$TEST_DIR/$$
+mkdir $rename_dir
+touch $rename_dir/foo $rename_dir/bar
+if ! src/renameat2 -t -w $rename_dir/foo $rename_dir/bar; then
+    rm -f $rename_dir/foo $rename_dir/bar; rmdir $rename_dir
+    _notrun "fs doesn't support RENAME_WHITEOUT"
+fi
+rm -f $rename_dir/foo $rename_dir/bar
+
+# real QA test starts here
+
+_rename_tests $rename_dir -w
+rmdir $rename_dir
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/078.out b/tests/generic/078.out
new file mode 100644
index 0000000..5d5e3a0
--- /dev/null
+++ b/tests/generic/078.out
@@ -0,0 +1,51 @@
+QA output created by 078
+samedir  none/none -> No such file or directory
+samedir  none/regu -> No such file or directory
+samedir  none/symb -> No such file or directory
+samedir  none/dire -> No such file or directory
+samedir  none/tree -> No such file or directory
+samedir  regu/none -> char/regu.
+samedir  regu/regu -> char/regu.
+samedir  regu/symb -> char/regu.
+samedir  regu/dire -> Is a directory
+samedir  regu/tree -> Is a directory
+samedir  symb/none -> char/symb.
+samedir  symb/regu -> char/symb.
+samedir  symb/symb -> char/symb.
+samedir  symb/dire -> Is a directory
+samedir  symb/tree -> Is a directory
+samedir  dire/none -> char/dire.
+samedir  dire/regu -> Not a directory
+samedir  dire/symb -> Not a directory
+samedir  dire/dire -> char/dire.
+samedir  dire/tree -> Directory not empty
+samedir  tree/none -> char/tree.
+samedir  tree/regu -> Not a directory
+samedir  tree/symb -> Not a directory
+samedir  tree/dire -> char/tree.
+samedir  tree/tree -> Directory not empty
+crossdir none/none -> No such file or directory
+crossdir none/regu -> No such file or directory
+crossdir none/symb -> No such file or directory
+crossdir none/dire -> No such file or directory
+crossdir none/tree -> No such file or directory
+crossdir regu/none -> char/regu.
+crossdir regu/regu -> char/regu.
+crossdir regu/symb -> char/regu.
+crossdir regu/dire -> Is a directory
+crossdir regu/tree -> Is a directory
+crossdir symb/none -> char/symb.
+crossdir symb/regu -> char/symb.
+crossdir symb/symb -> char/symb.
+crossdir symb/dire -> Is a directory
+crossdir symb/tree -> Is a directory
+crossdir dire/none -> char/dire.
+crossdir dire/regu -> Not a directory
+crossdir dire/symb -> Not a directory
+crossdir dire/dire -> char/dire.
+crossdir dire/tree -> Directory not empty
+crossdir tree/none -> char/tree.
+crossdir tree/regu -> Not a directory
+crossdir tree/symb -> Not a directory
+crossdir tree/dire -> char/tree.
+crossdir tree/tree -> Directory not empty
diff --git a/tests/generic/group b/tests/generic/group
index f2eb87a..0f274a6 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -69,6 +69,7 @@
 075 rw udf auto quick
 076 metadata rw udf auto quick stress
 077 acl attr auto enospc
+078 auto quick metadata
 079 acl attr ioctl metadata auto quick
 083 rw auto enospc stress
 088 perms auto quick
-- 
2.0.0

Re: [PATCH 4/4] generic: Add rudimetary RENAME_WHITEOUT test

[Brian Foster]

On Wed, Feb 25, 2015 at 09:54:39AM +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> There is no API documentation for RENAME_WHITEOUT. There is no
> developer documentation for RENAME_WHITEOUT. There are not comments
> in the overlayfs or ext4 implementation of RENAME_WHITEOUT.
> 
> Hence, this test simply tries to expose basic RENAME_WHITEOUT
> behaviour from ext4 so we can reverse-engineer and verify
> bug-for-bug renameat2(RENAME_WHITEOUT) ext4 compatibility.
> 
> Note: uses generic/078 just to keep out of the way of the 6-7 other
> pending new tests.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---

Reviewed-by: Brian Foster <bfoster@redhat.com>

>  src/renameat2.c       |  4 ++--
>  tests/generic/078     | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/078.out | 51 +++++++++++++++++++++++++++++++++++++++
>  tests/generic/group   |  1 +
>  4 files changed, 120 insertions(+), 2 deletions(-)
>  create mode 100755 tests/generic/078
>  create mode 100644 tests/generic/078.out
> 
> diff --git a/src/renameat2.c b/src/renameat2.c
> index 5ac0936..c59ce65 100644
> --- a/src/renameat2.c
> +++ b/src/renameat2.c
> @@ -96,9 +96,9 @@ int main(int argc, char *argv[])
>  		 * Turn EEXIST into ENOTEMPTY.  E.g. XFS uses EEXIST, and that
>  		 * is also accepted by the standards.
>  		 *
> -		 * This applies only to plain rename (flags == 0).
> +		 * This applies only to plain rename and RENAME_WHITEOUT
>  		 */
> -		if (!flags && errno == EEXIST)
> +		if (errno == EEXIST && (!flags || (flags & RENAME_WHITEOUT)))
>  			errno = ENOTEMPTY;
>  
>  		perror("");
> diff --git a/tests/generic/078 b/tests/generic/078
> new file mode 100755
> index 0000000..92ece0c
> --- /dev/null
> +++ b/tests/generic/078
> @@ -0,0 +1,66 @@
> +#! /bin/bash
> +# FS QA Test No. generic/078
> +#
> +# Check renameat2 syscall with RENAME_WHITEOUT flag
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2014 Miklos Szeredi.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +    cd /
> +    rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/renameat2
> +
> +_supported_fs generic
> +_supported_os Linux
> +
> +_require_test
> +_requires_renameat2
> +_require_test_symlinks
> +
> +rename_dir=$TEST_DIR/$$
> +mkdir $rename_dir
> +touch $rename_dir/foo $rename_dir/bar
> +if ! src/renameat2 -t -w $rename_dir/foo $rename_dir/bar; then
> +    rm -f $rename_dir/foo $rename_dir/bar; rmdir $rename_dir
> +    _notrun "fs doesn't support RENAME_WHITEOUT"
> +fi
> +rm -f $rename_dir/foo $rename_dir/bar
> +
> +# real QA test starts here
> +
> +_rename_tests $rename_dir -w
> +rmdir $rename_dir
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/078.out b/tests/generic/078.out
> new file mode 100644
> index 0000000..5d5e3a0
> --- /dev/null
> +++ b/tests/generic/078.out
> @@ -0,0 +1,51 @@
> +QA output created by 078
> +samedir  none/none -> No such file or directory
> +samedir  none/regu -> No such file or directory
> +samedir  none/symb -> No such file or directory
> +samedir  none/dire -> No such file or directory
> +samedir  none/tree -> No such file or directory
> +samedir  regu/none -> char/regu.
> +samedir  regu/regu -> char/regu.
> +samedir  regu/symb -> char/regu.
> +samedir  regu/dire -> Is a directory
> +samedir  regu/tree -> Is a directory
> +samedir  symb/none -> char/symb.
> +samedir  symb/regu -> char/symb.
> +samedir  symb/symb -> char/symb.
> +samedir  symb/dire -> Is a directory
> +samedir  symb/tree -> Is a directory
> +samedir  dire/none -> char/dire.
> +samedir  dire/regu -> Not a directory
> +samedir  dire/symb -> Not a directory
> +samedir  dire/dire -> char/dire.
> +samedir  dire/tree -> Directory not empty
> +samedir  tree/none -> char/tree.
> +samedir  tree/regu -> Not a directory
> +samedir  tree/symb -> Not a directory
> +samedir  tree/dire -> char/tree.
> +samedir  tree/tree -> Directory not empty
> +crossdir none/none -> No such file or directory
> +crossdir none/regu -> No such file or directory
> +crossdir none/symb -> No such file or directory
> +crossdir none/dire -> No such file or directory
> +crossdir none/tree -> No such file or directory
> +crossdir regu/none -> char/regu.
> +crossdir regu/regu -> char/regu.
> +crossdir regu/symb -> char/regu.
> +crossdir regu/dire -> Is a directory
> +crossdir regu/tree -> Is a directory
> +crossdir symb/none -> char/symb.
> +crossdir symb/regu -> char/symb.
> +crossdir symb/symb -> char/symb.
> +crossdir symb/dire -> Is a directory
> +crossdir symb/tree -> Is a directory
> +crossdir dire/none -> char/dire.
> +crossdir dire/regu -> Not a directory
> +crossdir dire/symb -> Not a directory
> +crossdir dire/dire -> char/dire.
> +crossdir dire/tree -> Directory not empty
> +crossdir tree/none -> char/tree.
> +crossdir tree/regu -> Not a directory
> +crossdir tree/symb -> Not a directory
> +crossdir tree/dire -> char/tree.
> +crossdir tree/tree -> Directory not empty
> diff --git a/tests/generic/group b/tests/generic/group
> index f2eb87a..0f274a6 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -69,6 +69,7 @@
>  075 rw udf auto quick
>  076 metadata rw udf auto quick stress
>  077 acl attr auto enospc
> +078 auto quick metadata
>  079 acl attr ioctl metadata auto quick
>  083 rw auto enospc stress
>  088 perms auto quick
> -- 
> 2.0.0
> 
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html