From: Brian Foster <bfoster@redhat.com>
To: linux-xfs@vger.kernel.org
Subject: [PATCH v2 00/11] xfs: rework extent allocation
Date: Wed, 22 May 2019 14:05:35 -0400 [thread overview]
Message-ID: <20190522180546.17063-1-bfoster@redhat.com> (raw)
Hi all,
This is v2 of the extent allocation rework series. The changes in this
version are mostly associated with code factoring, based on feedback to
v1. The small mode helper refactoring has been isolated and pulled to
the start of the series. The active flag that necessitated the btree
cursor container structure has been pushed into the xfs_btree_cur
private area. The resulting high level allocation code in
xfs_ag_alloc_vextent() has been cleaned up to remove an unnecessary
level of abstraction. Finally, there are various minor cleanups and
fixes.
On the testing front, I've run a couple more filebench oriented tests
since v1. The first is a high load, large filesystem, parallel file
write+fsync test to try and determine whether the modified near mode
allocation algorithm resulted in larger latencies in the common
(non-fragmented) case. The results show comparable latencies, though the
updated algorithm has a slightly faster overall runtime for whatever
reason.
The second is another filebench test (but with a smaller fileset against
a smaller filesystem), but with the purpose of measuring "locality
effectiveness" of the updated algorithm via post-test analysis of the
resulting/populated filesystem. I've been thinking a bit about how to
test for locality since starting on this series and ultimately came up
with the following, fairly crude heuristic: track and compare the worst
locality allocation for each regular file inode in the fs. This
essentially locates the most distant extent for each inode, tracks the
delta from that extent to the inode location on disk and calculates the
average worst case delta across the entire set of regular files. The
results show that the updated algorithm provides a comparable level of
locality to the existing algorithm. Note that this test also involves a
control run using a kernel modified to replace all near mode allocations
with by-size allocations. This is to show the effectiveness of near mode
allocation in general as compared to essentially random allocations.
Further details and results from both of these tests are appended below
the changelog. Thoughts, reviews, flames appreciated.
Brian
v2:
- Lift small mode refactoring into separate patch (retained review
tag(s).
- Various logic cleanups and refactors.
- Push active flag down into btree cursor private area; eliminate cursor
container struct.
- Refactor final allocation code. Fold xfs_alloc_ag_vextent_type() into
caller and factor out accounting.
- Fix up tracepoints.
v1: https://marc.info/?l=linux-xfs&m=155742169729590&w=2
- Continued development (various fixes, refinements) on generic bits and
near mode implementation.
- Added patches 4-6 to refactor exact, by-size and small allocation
modes.
rfcv2: https://marc.info/?l=linux-xfs&m=155197946630582&w=2
- Dropped spurious initial refactoring.
- Added minlen functionality.
- Properly tied into near alloc path.
- General refactoring and cleanups.
rfcv1: https://marc.info/?l=linux-xfs&m=154479089914351&w=2
--- filebench 200 thread file create test, 20k files, 7TB 32AG fs:
- baseline summary results (3 runs):
3541: 34498.540: Run took 34495 seconds...
3541: 34498.547: Per-Operation Breakdown
closefile1 19801ops 1ops/s 0.0mb/s 0.0ms/op 318us/op-cpu [0ms - 5ms]
fsync1 19801ops 1ops/s 0.0mb/s 883.0ms/op 408529us/op-cpu [0ms - 106670ms]
writefile1 19801ops 1ops/s 152.5mb/s 322687.8ms/op 80040329us/op-cpu [0ms - 10344864ms]
createfile1 20000ops 1ops/s 0.0mb/s 1.0ms/op 8460us/op-cpu [0ms - 593ms]
3541: 34498.548: IO Summary: 79403 ops, 2.302 ops/s, (0/1 r/w), 152.5mb/s, 174211us cpu/op, 323571.9ms latency
94799: 34457.516: Run took 34454 seconds...
94799: 34457.524: Per-Operation Breakdown
closefile1 19801ops 1ops/s 0.0mb/s 0.1ms/op 266us/op-cpu [0ms - 329ms]
fsync1 19801ops 1ops/s 0.0mb/s 1014.5ms/op 394296us/op-cpu [0ms - 188565ms]
writefile1 19801ops 1ops/s 152.7mb/s 322228.1ms/op 80961984us/op-cpu [0ms - 10354419ms]
createfile1 20000ops 1ops/s 0.0mb/s 0.9ms/op 10845us/op-cpu [0ms - 944ms]
94799: 34457.524: IO Summary: 79403 ops, 2.304 ops/s, (0/1 r/w), 152.7mb/s, 174494us cpu/op, 323243.6ms latency
70863: 34440.504: Run took 34437 seconds...
70863: 34440.513: Per-Operation Breakdown
closefile1 19801ops 1ops/s 0.0mb/s 0.0ms/op 322us/op-cpu [0ms - 43ms]
fsync1 19801ops 1ops/s 0.0mb/s 909.0ms/op 412324us/op-cpu [0ms - 135906ms]
writefile1 19801ops 1ops/s 152.8mb/s 322198.4ms/op 82080900us/op-cpu [0ms - 10368105ms]
createfile1 20000ops 1ops/s 0.0mb/s 1.0ms/op 9544us/op-cpu [0ms - 605ms]
70863: 34440.513: IO Summary: 79403 ops, 2.306 ops/s, (0/1 r/w), 152.8mb/s, 174420us cpu/op, 323108.4ms latency
- test summary results:
22975: 34215.483: Run took 34212 seconds...
22975: 34215.491: Per-Operation Breakdown
closefile1 19801ops 1ops/s 0.0mb/s 0.0ms/op 272us/op-cpu [0ms - 7ms]
fsync1 19801ops 1ops/s 0.0mb/s 914.5ms/op 383151us/op-cpu [0ms - 176244ms]
writefile1 19801ops 1ops/s 153.8mb/s 319896.9ms/op 75523888us/op-cpu [0ms - 10337439ms]
createfile1 20000ops 1ops/s 0.0mb/s 1.1ms/op 12534us/op-cpu [0ms - 628ms]
22975: 34215.491: IO Summary: 79403 ops, 2.321 ops/s, (0/1 r/w), 153.8mb/s, 322858us cpu/op, 320812.6ms latency
3954: 34197.494: Run took 34194 seconds...
3954: 34197.503: Per-Operation Breakdown
closefile1 19801ops 1ops/s 0.0mb/s 0.1ms/op 261us/op-cpu [0ms - 549ms]
fsync1 19801ops 1ops/s 0.0mb/s 808.8ms/op 376216us/op-cpu [0ms - 107815ms]
writefile1 19801ops 1ops/s 153.8mb/s 319904.9ms/op 86184221us/op-cpu [0ms - 10315909ms]
createfile1 20000ops 1ops/s 0.0mb/s 1.0ms/op 5954us/op-cpu [0ms - 591ms]
3954: 34197.503: IO Summary: 79403 ops, 2.322 ops/s, (0/1 r/w), 153.8mb/s, 172877us cpu/op, 320714.8ms latency
180845: 34295.481: Run took 34292 seconds...
180845: 34295.489: Per-Operation Breakdown
closefile1 19801ops 1ops/s 0.0mb/s 0.1ms/op 306us/op-cpu [0ms - 330ms]
fsync1 19801ops 1ops/s 0.0mb/s 868.4ms/op 387699us/op-cpu [0ms - 159923ms]
writefile1 19801ops 1ops/s 153.4mb/s 320814.7ms/op 84496457us/op-cpu [0ms - 10314107ms]
createfile1 20000ops 1ops/s 0.0mb/s 1.4ms/op 6386us/op-cpu [0ms - 425ms]
180845: 34295.489: IO Summary: 79403 ops, 2.315 ops/s, (0/1 r/w), 153.4mb/s, 174443us cpu/op, 321684.6ms latency
--- filebench 200 thread file create test, 5k files, 2TB 64AG fs
(agsize: 8388608 4k blocks)
Locality measured in 512b sectors.
- baseline - min: 8 max: 568752250 median: 434794.5 mean: 11446328
- test - min: 33 max: 568402234 median: 437405.5 mean: 11752963
- by-size only - min: 33 max: 568593146 median: 784805 mean: 11912300
Brian Foster (11):
xfs: clean up small allocation helper
xfs: move small allocation helper
xfs: skip small alloc cntbt logic on NULL cursor
xfs: always update params on small allocation
xfs: track active state of allocation btree cursors
xfs: use locality optimized cntbt lookups for near mode allocations
xfs: refactor exact extent allocation mode
xfs: refactor by-size extent allocation mode
xfs: replace small allocation logic with agfl only logic
xfs: refactor successful AG allocation accounting code
xfs: condense high level AG allocation functions
fs/xfs/libxfs/xfs_alloc.c | 1485 +++++++++++++------------------
fs/xfs/libxfs/xfs_alloc_btree.c | 1 +
fs/xfs/libxfs/xfs_btree.h | 3 +
fs/xfs/xfs_trace.h | 44 +-
4 files changed, 643 insertions(+), 890 deletions(-)
--
2.17.2
next reply other threads:[~2019-05-22 18:05 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-22 18:05 Brian Foster [this message]
2019-05-22 18:05 ` [PATCH v2 01/11] xfs: clean up small allocation helper Brian Foster
2019-06-21 23:57 ` Darrick J. Wong
2019-05-22 18:05 ` [PATCH v2 02/11] xfs: move " Brian Foster
2019-06-21 23:58 ` Darrick J. Wong
2019-05-22 18:05 ` [PATCH v2 03/11] xfs: skip small alloc cntbt logic on NULL cursor Brian Foster
2019-06-21 23:58 ` Darrick J. Wong
2019-05-22 18:05 ` [PATCH v2 04/11] xfs: always update params on small allocation Brian Foster
2019-06-21 23:59 ` Darrick J. Wong
2019-05-22 18:05 ` [PATCH v2 05/11] xfs: track active state of allocation btree cursors Brian Foster
2019-05-22 18:05 ` [PATCH v2 06/11] xfs: use locality optimized cntbt lookups for near mode allocations Brian Foster
2019-05-22 18:05 ` [PATCH v2 07/11] xfs: refactor exact extent allocation mode Brian Foster
2019-05-22 18:05 ` [PATCH v2 08/11] xfs: refactor by-size " Brian Foster
2019-05-22 18:05 ` [PATCH v2 09/11] xfs: replace small allocation logic with agfl only logic Brian Foster
2019-05-22 18:05 ` [PATCH v2 10/11] xfs: refactor successful AG allocation accounting code Brian Foster
2019-05-22 18:05 ` [PATCH v2 11/11] xfs: condense high level AG allocation functions Brian Foster
2019-05-23 1:56 ` [PATCH v2 00/11] xfs: rework extent allocation Dave Chinner
2019-05-23 12:55 ` Brian Foster
2019-05-23 22:15 ` Dave Chinner
2019-05-24 12:00 ` Brian Foster
2019-05-25 22:43 ` Dave Chinner
2019-05-31 17:11 ` Brian Foster
2019-06-06 15:21 ` Brian Foster
2019-06-06 22:13 ` Dave Chinner
2019-06-07 12:57 ` Brian Foster
2019-06-06 22:05 ` Dave Chinner
2019-06-07 12:56 ` Brian Foster
2019-06-21 15:18 ` Darrick J. Wong
2019-07-01 19:12 ` Brian Foster
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190522180546.17063-1-bfoster@redhat.com \
--to=bfoster@redhat.com \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).