* [PATCH RFC 0/2] fsx: make fsx perceptive to cluster size @ 2020-04-24 9:33 Jeffle Xu 2020-04-24 9:33 ` [PATCH RFC 1/2] xfstests: fsx: add support for " Jeffle Xu 2020-04-24 9:33 ` [PATCH RFC 2/2] xfstests: common/rc: add cluster size support for ext4 Jeffle Xu 0 siblings, 2 replies; 6+ messages in thread From: Jeffle Xu @ 2020-04-24 9:33 UTC (permalink / raw) To: fstests; +Cc: linux-ext4, joseph.qi, Jeffle Xu Offset and size should be aligned with cluster_size when inserting or collapsing range on ext4 with 'bigalloc' enabled. Currently fsx only align offset/size with block size. In fact I have no idea which is the best way to fix this isue. On one hand, fsx should be general and has no knowledge of the underlying filesystem. Besides the cluster size seems to be stored on ext4_super_block and there's no easy way to get it. But on the oter hand, quite many tests call fsx directly, e.g., generic/127, rather than the generic routine run_fsx() defined in common/rc. Jeffle Xu (2): xfstests: fsx: add support for cluster size xfstests: common/rc: add cluster size support for ext4 common/rc | 9 +++++++++ ltp/fsx.c | 20 ++++++++++++++------ 2 files changed, 23 insertions(+), 6 deletions(-) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH RFC 1/2] xfstests: fsx: add support for cluster size 2020-04-24 9:33 [PATCH RFC 0/2] fsx: make fsx perceptive to cluster size Jeffle Xu @ 2020-04-24 9:33 ` Jeffle Xu 2020-04-27 17:29 ` Darrick J. Wong 2020-04-24 9:33 ` [PATCH RFC 2/2] xfstests: common/rc: add cluster size support for ext4 Jeffle Xu 1 sibling, 1 reply; 6+ messages in thread From: Jeffle Xu @ 2020-04-24 9:33 UTC (permalink / raw) To: fstests; +Cc: linux-ext4, joseph.qi, Jeffle Xu The offset and size should be aligned with cluster size when inserting or collapsing range on ext4 with 'bigalloc' feature enabled. Currently I can find only ext4 with this limitation. Since fsx should have no assumption of the underlying filesystem, and thus add the '-u cluster_size' option. Tests can set this option when the underlying filesystem is ext4 with bigalloc enabled. Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> --- ltp/fsx.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/ltp/fsx.c b/ltp/fsx.c index 9d598a4..5fe5738 100644 --- a/ltp/fsx.c +++ b/ltp/fsx.c @@ -133,6 +133,7 @@ int dirpath = 0; /* -P flag */ int fd; /* fd for our test file */ blksize_t block_size = 0; +blksize_t cluster_size = 0; off_t file_size = 0; off_t biggest = 0; long long testcalls = 0; /* calls to function "test" */ @@ -2146,8 +2147,8 @@ have_op: break; case OP_COLLAPSE_RANGE: TRIM_OFF_LEN(offset, size, file_size - 1); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); + offset = offset & ~(cluster_size - 1); + size = size & ~(cluster_size - 1); if (size == 0) { log4(OP_COLLAPSE_RANGE, offset, size, FL_SKIPPED); goto out; @@ -2157,8 +2158,8 @@ have_op: case OP_INSERT_RANGE: TRIM_OFF(offset, file_size); TRIM_LEN(file_size, size, maxfilelen); - offset = offset & ~(block_size - 1); - size = size & ~(block_size - 1); + offset = offset & ~(cluster_size - 1); + size = size & ~(cluster_size - 1); if (size == 0) { log4(OP_INSERT_RANGE, offset, size, FL_SKIPPED); goto out; @@ -2231,7 +2232,7 @@ void usage(void) { fprintf(stdout, "usage: %s", - "fsx [-dknqxABEFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ + "fsx [-dknqxABEFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-u csize] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ -b opnum: beginning operation number (default 1)\n\ -c P: 1 in P chance of file close+open at each op (default infinity)\n\ -d: debug output for all operations\n\ @@ -2249,6 +2250,7 @@ usage(void) -r readbdy: 4096 would make reads page aligned (default 1)\n\ -s style: 1 gives smaller truncates (default 0)\n\ -t truncbdy: 4096 would make truncates page aligned (default 1)\n\ + -u csize: filesystem specific cluster size that may be used for ops like insert/collapse range\n\ -w writebdy: 4096 would make writes page aligned (default 1)\n\ -x: preallocate file space before starting, XFS only (default 0)\n\ -y synchronize changes to a file\n" @@ -2485,7 +2487,7 @@ main(int argc, char **argv) setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ while ((ch = getopt_long(argc, argv, - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:EFJKHzCILN:OP:RS:WXZ", + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:u:w:xyABD:EFJKHzCILN:OP:RS:WXZ", longopts, NULL)) != EOF) switch (ch) { case 'b': @@ -2579,6 +2581,11 @@ main(int argc, char **argv) if (truncbdy <= 0) usage(); break; + case 'u': + cluster_size = getnum(optarg, &endp); + if (cluster_size <= 0) + usage(); + break; case 'w': writebdy = getnum(optarg, &endp); if (writebdy <= 0) @@ -2720,6 +2727,7 @@ main(int argc, char **argv) exit(91); } block_size = statbuf.st_blksize; + cluster_size = cluster_size ? : block_size; #ifdef XFS if (prealloc) { xfs_flock64_t resv = { 0 }; -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH RFC 1/2] xfstests: fsx: add support for cluster size 2020-04-24 9:33 ` [PATCH RFC 1/2] xfstests: fsx: add support for " Jeffle Xu @ 2020-04-27 17:29 ` Darrick J. Wong 2020-04-29 7:22 ` JeffleXu 0 siblings, 1 reply; 6+ messages in thread From: Darrick J. Wong @ 2020-04-27 17:29 UTC (permalink / raw) To: Jeffle Xu; +Cc: fstests, linux-ext4, joseph.qi On Fri, Apr 24, 2020 at 05:33:49PM +0800, Jeffle Xu wrote: > The offset and size should be aligned with cluster size when inserting > or collapsing range on ext4 with 'bigalloc' feature enabled. Currently > I can find only ext4 with this limitation. ocfs2 also has this magic, um, ability. As does xfs under certain circumstance (realtime volumes). > Since fsx should have no assumption of the underlying filesystem, and > thus add the '-u cluster_size' option. Tests can set this option when > the underlying filesystem is ext4 with bigalloc enabled. Do copyrange, clonerange, or deduperange have this problem? ;) > Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> > --- > ltp/fsx.c | 20 ++++++++++++++------ > 1 file changed, 14 insertions(+), 6 deletions(-) > > diff --git a/ltp/fsx.c b/ltp/fsx.c > index 9d598a4..5fe5738 100644 > --- a/ltp/fsx.c > +++ b/ltp/fsx.c > @@ -133,6 +133,7 @@ int dirpath = 0; /* -P flag */ > int fd; /* fd for our test file */ > > blksize_t block_size = 0; > +blksize_t cluster_size = 0; > off_t file_size = 0; > off_t biggest = 0; > long long testcalls = 0; /* calls to function "test" */ > @@ -2146,8 +2147,8 @@ have_op: > break; > case OP_COLLAPSE_RANGE: > TRIM_OFF_LEN(offset, size, file_size - 1); > - offset = offset & ~(block_size - 1); > - size = size & ~(block_size - 1); > + offset = offset & ~(cluster_size - 1); > + size = size & ~(cluster_size - 1); > if (size == 0) { > log4(OP_COLLAPSE_RANGE, offset, size, FL_SKIPPED); > goto out; > @@ -2157,8 +2158,8 @@ have_op: > case OP_INSERT_RANGE: > TRIM_OFF(offset, file_size); > TRIM_LEN(file_size, size, maxfilelen); > - offset = offset & ~(block_size - 1); > - size = size & ~(block_size - 1); > + offset = offset & ~(cluster_size - 1); > + size = size & ~(cluster_size - 1); > if (size == 0) { > log4(OP_INSERT_RANGE, offset, size, FL_SKIPPED); > goto out; > @@ -2231,7 +2232,7 @@ void > usage(void) > { > fprintf(stdout, "usage: %s", > - "fsx [-dknqxABEFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ > + "fsx [-dknqxABEFJLOWZ] [-b opnum] [-c Prob] [-g filldata] [-i logdev] [-j logid] [-l flen] [-m start:end] [-o oplen] [-p progressinterval] [-r readbdy] [-s style] [-t truncbdy] [-u csize] [-w writebdy] [-D startingop] [-N numops] [-P dirpath] [-S seed] fname\n\ > -b opnum: beginning operation number (default 1)\n\ > -c P: 1 in P chance of file close+open at each op (default infinity)\n\ > -d: debug output for all operations\n\ > @@ -2249,6 +2250,7 @@ usage(void) > -r readbdy: 4096 would make reads page aligned (default 1)\n\ > -s style: 1 gives smaller truncates (default 0)\n\ > -t truncbdy: 4096 would make truncates page aligned (default 1)\n\ > + -u csize: filesystem specific cluster size that may be used for ops like insert/collapse range\n\ > -w writebdy: 4096 would make writes page aligned (default 1)\n\ > -x: preallocate file space before starting, XFS only (default 0)\n\ > -y synchronize changes to a file\n" > @@ -2485,7 +2487,7 @@ main(int argc, char **argv) > setvbuf(stdout, (char *)0, _IOLBF, 0); /* line buffered stdout */ > > while ((ch = getopt_long(argc, argv, > - "b:c:dfg:i:j:kl:m:no:p:qr:s:t:w:xyABD:EFJKHzCILN:OP:RS:WXZ", > + "b:c:dfg:i:j:kl:m:no:p:qr:s:t:u:w:xyABD:EFJKHzCILN:OP:RS:WXZ", > longopts, NULL)) != EOF) > switch (ch) { > case 'b': > @@ -2579,6 +2581,11 @@ main(int argc, char **argv) > if (truncbdy <= 0) > usage(); > break; > + case 'u': > + cluster_size = getnum(optarg, &endp); > + if (cluster_size <= 0) > + usage(); > + break; > case 'w': > writebdy = getnum(optarg, &endp); > if (writebdy <= 0) > @@ -2720,6 +2727,7 @@ main(int argc, char **argv) > exit(91); > } > block_size = statbuf.st_blksize; > + cluster_size = cluster_size ? : block_size; > #ifdef XFS > if (prealloc) { > xfs_flock64_t resv = { 0 }; > -- > 1.8.3.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH RFC 1/2] xfstests: fsx: add support for cluster size 2020-04-27 17:29 ` Darrick J. Wong @ 2020-04-29 7:22 ` JeffleXu 0 siblings, 0 replies; 6+ messages in thread From: JeffleXu @ 2020-04-29 7:22 UTC (permalink / raw) To: Darrick J. Wong; +Cc: fstests, linux-ext4, joseph.qi On 4/28/20 1:29 AM, Darrick J. Wong wrote: > On Fri, Apr 24, 2020 at 05:33:49PM +0800, Jeffle Xu wrote: >> The offset and size should be aligned with cluster size when inserting >> or collapsing range on ext4 with 'bigalloc' feature enabled. Currently >> I can find only ext4 with this limitation. > ocfs2 also has this magic, um, ability. > > As does xfs under certain circumstance (realtime volumes). > >> Since fsx should have no assumption of the underlying filesystem, and >> thus add the '-u cluster_size' option. Tests can set this option when >> the underlying filesystem is ext4 with bigalloc enabled. > Do copyrange, clonerange, or deduperange have this problem? ;) clonerange and deduperange are not supported in ext4, while copyrange and zerorange work as the range has no need to be aligned with cluster size in these two situations. ^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH RFC 2/2] xfstests: common/rc: add cluster size support for ext4 2020-04-24 9:33 [PATCH RFC 0/2] fsx: make fsx perceptive to cluster size Jeffle Xu 2020-04-24 9:33 ` [PATCH RFC 1/2] xfstests: fsx: add support for " Jeffle Xu @ 2020-04-24 9:33 ` Jeffle Xu 2020-04-27 17:33 ` Darrick J. Wong 1 sibling, 1 reply; 6+ messages in thread From: Jeffle Xu @ 2020-04-24 9:33 UTC (permalink / raw) To: fstests; +Cc: linux-ext4, joseph.qi, Jeffle Xu Inserting and collapsing range on ext4 with 'bigalloc' feature will fail due to the offset and size should be alligned with the cluster size. The previous patch has add support for cluster size in fsx. Detect and pass the cluster size parameter to fsx if the underlying filesystem is ext4 with bigalloc. Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> --- common/rc | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/common/rc b/common/rc index 2000bd9..71dde5f 100644 --- a/common/rc +++ b/common/rc @@ -3908,6 +3908,15 @@ run_fsx() { echo fsx $@ local args=`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psize /g"` + + if [ "$FSTYP" == "ext4" ]; then + local cluster_size=$(tune2fs -l $TEST_DEV | grep 'Cluster size' | awk '{print $3}') + if [ -n $cluster_size ]; then + echo "cluster size: $cluster_size" + args="$args -u $cluster_size" + fi + fi + set -- $here/ltp/fsx $args $FSX_AVOID $TEST_DIR/junk echo "$@" >>$seqres.full rm -f $TEST_DIR/junk -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH RFC 2/2] xfstests: common/rc: add cluster size support for ext4 2020-04-24 9:33 ` [PATCH RFC 2/2] xfstests: common/rc: add cluster size support for ext4 Jeffle Xu @ 2020-04-27 17:33 ` Darrick J. Wong 0 siblings, 0 replies; 6+ messages in thread From: Darrick J. Wong @ 2020-04-27 17:33 UTC (permalink / raw) To: Jeffle Xu; +Cc: fstests, linux-ext4, joseph.qi On Fri, Apr 24, 2020 at 05:33:50PM +0800, Jeffle Xu wrote: > Inserting and collapsing range on ext4 with 'bigalloc' feature will > fail due to the offset and size should be alligned with the cluster > size. > > The previous patch has add support for cluster size in fsx. Detect and > pass the cluster size parameter to fsx if the underlying filesystem > is ext4 with bigalloc. > > Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> > --- > common/rc | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/common/rc b/common/rc > index 2000bd9..71dde5f 100644 > --- a/common/rc > +++ b/common/rc > @@ -3908,6 +3908,15 @@ run_fsx() > { > echo fsx $@ > local args=`echo $@ | sed -e "s/ BSIZE / $bsize /g" -e "s/ PSIZE / $psize /g"` > + > + if [ "$FSTYP" == "ext4" ]; then > + local cluster_size=$(tune2fs -l $TEST_DEV | grep 'Cluster size' | awk '{print $3}') > + if [ -n $cluster_size ]; then > + echo "cluster size: $cluster_size" > + args="$args -u $cluster_size" > + fi > + fi Computing the file allocation block size ought to be a separate helper. I wonder if there's a standard way to report cluster sizes, seeing as fat, ext4, ocfs2, and xfs can all have minimum space allocation units that are larger than the base fs block size. --D > + > set -- $here/ltp/fsx $args $FSX_AVOID $TEST_DIR/junk > echo "$@" >>$seqres.full > rm -f $TEST_DIR/junk > -- > 1.8.3.1 > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-04-29 7:22 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-04-24 9:33 [PATCH RFC 0/2] fsx: make fsx perceptive to cluster size Jeffle Xu 2020-04-24 9:33 ` [PATCH RFC 1/2] xfstests: fsx: add support for " Jeffle Xu 2020-04-27 17:29 ` Darrick J. Wong 2020-04-29 7:22 ` JeffleXu 2020-04-24 9:33 ` [PATCH RFC 2/2] xfstests: common/rc: add cluster size support for ext4 Jeffle Xu 2020-04-27 17:33 ` Darrick J. Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).