From: Thomas Lynema <lyz27@yahoo.com>
To: Dave Chinner <david@fromorbit.com>
Cc: xfs@oss.sgi.com
Subject: Re: Poor performance using discard
Date: Fri, 02 Mar 2012 10:41:50 -0500 [thread overview]
Message-ID: <1330702910.6438.40.camel@core24> (raw)
In-Reply-To: <20120229040819.GZ3592@dastard>
[-- Attachment #1.1.1: Type: text/plain, Size: 14684 bytes --]
[....]
> Any errors in dmesg?
None
> Also, I think you need to provide a block trace (output of
> blktrace/blkparse for the rm -rf workloads) for both the XFS and
> ext4 cases
Here's the output from running similar
commands as Christoph:
blkparse xfs.trace.blktrace.* | grep -w D | wc -l
1134996
blkparse ext4.trace.blktrace.* | grep -w D | wc -l
27240
Attached archives of half of the ext4 results.
Summary:
CPU0 (ext4.trace):
Reads Queued: 372, 1,944KiB Writes Queued: 1,932, 7,728KiB
Read Dispatches: 372, 1,944KiB Write Dispatches: 19, 7,756KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 419, 2,104KiB Writes Completed: 13, 5,123KiB
Read Merges: 0, 0KiB Write Merges: 1,914, 7,656KiB
Read depth: 29 Write depth: 32
IO unplugs: 12 Timer unplugs: 0
CPU1 (ext4.trace):
Reads Queued: 3,119, 12,484KiB Writes Queued: 2,591, 420,624KiB
Read Dispatches: 3,168, 12,484KiB Write Dispatches: 356, 423,100KiB
Reads Requeued: 49 Writes Requeued: 115
Reads Completed: 3,271, 13,104KiB Writes Completed: 251, 11,317KiB
Read Merges: 0, 0KiB Write Merges: 2,358, 9,504KiB
Read depth: 29 Write depth: 32
IO unplugs: 10 Timer unplugs: 2
CPU2 (ext4.trace):
Reads Queued: 191, 844KiB Writes Queued: 2, 3KiB
Read Dispatches: 191, 844KiB Write Dispatches: 1, 3KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 144, 684KiB Writes Completed: 10, 2,608KiB
Read Merges: 0, 0KiB Write Merges: 0, 0KiB
Read depth: 29 Write depth: 32
IO unplugs: 6 Timer unplugs: 0
CPU3 (ext4.trace):
Reads Queued: 1,273, 5,116KiB Writes Queued: 852, 112,049KiB
Read Dispatches: 1,315, 5,116KiB Write Dispatches: 72, 109,545KiB
Reads Requeued: 42 Writes Requeued: 11
Reads Completed: 1,121, 4,496KiB Writes Completed: 56, 4,024KiB
Read Merges: 0, 0KiB Write Merges: 779, 3,392KiB
Read depth: 29 Write depth: 32
IO unplugs: 15 Timer unplugs: 11
Total (ext4.trace):
Reads Queued: 4,955, 20,388KiB Writes Queued: 5,377, 540,404KiB
Read Dispatches: 5,046, 20,388KiB Write Dispatches: 448, 540,404KiB
Reads Requeued: 91 Writes Requeued: 126
Reads Completed: 4,955, 20,388KiB Writes Completed: 330, 23,072KiB
Read Merges: 0, 0KiB Write Merges: 5,051, 20,552KiB
IO unplugs: 43 Timer unplugs: 13
Throughput (R/W): 5,011KiB/s / 5,671KiB/s
Events (ext4.trace): 57,930 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (ext4.trace):
Reads Queued: 372, 1,944KiB Writes Queued: 1,932, 7,728KiB
Read Dispatches: 372, 1,944KiB Write Dispatches: 19, 7,756KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 419, 2,104KiB Writes Completed: 13, 5,123KiB
Read Merges: 0, 0KiB Write Merges: 1,914, 7,656KiB
Read depth: 29 Write depth: 32
IO unplugs: 12 Timer unplugs: 0
CPU1 (ext4.trace):
Reads Queued: 3,119, 12,484KiB Writes Queued: 2,591, 420,624KiB
Read Dispatches: 3,168, 12,484KiB Write Dispatches: 356, 423,100KiB
Reads Requeued: 49 Writes Requeued: 115
Reads Completed: 3,271, 13,104KiB Writes Completed: 251, 11,317KiB
Read Merges: 0, 0KiB Write Merges: 2,358, 9,504KiB
Read depth: 29 Write depth: 32
IO unplugs: 10 Timer unplugs: 2
CPU2 (ext4.trace):
Reads Queued: 191, 844KiB Writes Queued: 2, 3KiB
Read Dispatches: 191, 844KiB Write Dispatches: 1, 3KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 144, 684KiB Writes Completed: 10, 2,608KiB
Read Merges: 0, 0KiB Write Merges: 0, 0KiB
Read depth: 29 Write depth: 32
IO unplugs: 6 Timer unplugs: 0
CPU3 (ext4.trace):
Reads Queued: 1,273, 5,116KiB Writes Queued: 852, 112,049KiB
Read Dispatches: 1,315, 5,116KiB Write Dispatches: 72, 109,545KiB
Reads Requeued: 42 Writes Requeued: 11
Reads Completed: 1,121, 4,496KiB Writes Completed: 56, 4,024KiB
Read Merges: 0, 0KiB Write Merges: 779, 3,392KiB
Read depth: 29 Write depth: 32
IO unplugs: 15 Timer unplugs: 11
Total (ext4.trace):
Reads Queued: 4,955, 20,388KiB Writes Queued: 5,377, 540,404KiB
Read Dispatches: 5,046, 20,388KiB Write Dispatches: 448, 540,404KiB
Reads Requeued: 91 Writes Requeued: 126
Reads Completed: 4,955, 20,388KiB Writes Completed: 330, 23,072KiB
Read Merges: 0, 0KiB Write Merges: 5,051, 20,552KiB
IO unplugs: 43 Timer unplugs: 13
Throughput (R/W): 5,011KiB/s / 5,671KiB/s
Events (ext4.trace): 57,930 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (ext4.trace):
Reads Queued: 372, 1,944KiB Writes Queued: 1,932, 7,728KiB
Read Dispatches: 372, 1,944KiB Write Dispatches: 19, 7,756KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 419, 2,104KiB Writes Completed: 13, 5,123KiB
Read Merges: 0, 0KiB Write Merges: 1,914, 7,656KiB
Read depth: 29 Write depth: 32
IO unplugs: 12 Timer unplugs: 0
CPU1 (ext4.trace):
Reads Queued: 3,119, 12,484KiB Writes Queued: 2,591, 420,624KiB
Read Dispatches: 3,168, 12,484KiB Write Dispatches: 356, 423,100KiB
Reads Requeued: 49 Writes Requeued: 115
Reads Completed: 3,271, 13,104KiB Writes Completed: 251, 11,317KiB
Read Merges: 0, 0KiB Write Merges: 2,358, 9,504KiB
Read depth: 29 Write depth: 32
IO unplugs: 10 Timer unplugs: 2
CPU2 (ext4.trace):
Reads Queued: 191, 844KiB Writes Queued: 2, 3KiB
Read Dispatches: 191, 844KiB Write Dispatches: 1, 3KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 144, 684KiB Writes Completed: 10, 2,608KiB
Read Merges: 0, 0KiB Write Merges: 0, 0KiB
Read depth: 29 Write depth: 32
IO unplugs: 6 Timer unplugs: 0
CPU3 (ext4.trace):
Reads Queued: 1,273, 5,116KiB Writes Queued: 852, 112,049KiB
Read Dispatches: 1,315, 5,116KiB Write Dispatches: 72, 109,545KiB
Reads Requeued: 42 Writes Requeued: 11
Reads Completed: 1,121, 4,496KiB Writes Completed: 56, 4,024KiB
Read Merges: 0, 0KiB Write Merges: 779, 3,392KiB
Read depth: 29 Write depth: 32
IO unplugs: 15 Timer unplugs: 11
Total (ext4.trace):
Reads Queued: 4,955, 20,388KiB Writes Queued: 5,377, 540,404KiB
Read Dispatches: 5,046, 20,388KiB Write Dispatches: 448, 540,404KiB
Reads Requeued: 91 Writes Requeued: 126
Reads Completed: 4,955, 20,388KiB Writes Completed: 330, 23,072KiB
Read Merges: 0, 0KiB Write Merges: 5,051, 20,552KiB
IO unplugs: 43 Timer unplugs: 13
Throughput (R/W): 5,011KiB/s / 5,671KiB/s
Events (ext4.trace): 57,930 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (ext4.trace):
Reads Queued: 372, 1,944KiB Writes Queued: 1,932, 7,728KiB
Read Dispatches: 372, 1,944KiB Write Dispatches: 19, 7,756KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 419, 2,104KiB Writes Completed: 13, 5,123KiB
Read Merges: 0, 0KiB Write Merges: 1,914, 7,656KiB
Read depth: 29 Write depth: 32
IO unplugs: 12 Timer unplugs: 0
CPU1 (ext4.trace):
Reads Queued: 3,119, 12,484KiB Writes Queued: 2,591, 420,624KiB
Read Dispatches: 3,168, 12,484KiB Write Dispatches: 356, 423,100KiB
Reads Requeued: 49 Writes Requeued: 115
Reads Completed: 3,271, 13,104KiB Writes Completed: 251, 11,317KiB
Read Merges: 0, 0KiB Write Merges: 2,358, 9,504KiB
Read depth: 29 Write depth: 32
IO unplugs: 10 Timer unplugs: 2
CPU2 (ext4.trace):
Reads Queued: 191, 844KiB Writes Queued: 2, 3KiB
Read Dispatches: 191, 844KiB Write Dispatches: 1, 3KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 144, 684KiB Writes Completed: 10, 2,608KiB
Read Merges: 0, 0KiB Write Merges: 0, 0KiB
Read depth: 29 Write depth: 32
IO unplugs: 6 Timer unplugs: 0
CPU3 (ext4.trace):
Reads Queued: 1,273, 5,116KiB Writes Queued: 852, 112,049KiB
Read Dispatches: 1,315, 5,116KiB Write Dispatches: 72, 109,545KiB
Reads Requeued: 42 Writes Requeued: 11
Reads Completed: 1,121, 4,496KiB Writes Completed: 56, 4,024KiB
Read Merges: 0, 0KiB Write Merges: 779, 3,392KiB
Read depth: 29 Write depth: 32
IO unplugs: 15 Timer unplugs: 11
Total (ext4.trace):
Reads Queued: 4,955, 20,388KiB Writes Queued: 5,377, 540,404KiB
Read Dispatches: 5,046, 20,388KiB Write Dispatches: 448, 540,404KiB
Reads Requeued: 91 Writes Requeued: 126
Reads Completed: 4,955, 20,388KiB Writes Completed: 330, 23,072KiB
Read Merges: 0, 0KiB Write Merges: 5,051, 20,552KiB
IO unplugs: 43 Timer unplugs: 13
Throughput (R/W): 5,011KiB/s / 5,671KiB/s
Events (ext4.trace): 57,930 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (ext4.trace):
Reads Queued: 372, 1,944KiB Writes Queued: 1,932, 7,728KiB
Read Dispatches: 372, 1,944KiB Write Dispatches: 19, 7,756KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 419, 2,104KiB Writes Completed: 13, 5,123KiB
Read Merges: 0, 0KiB Write Merges: 1,914, 7,656KiB
Read depth: 29 Write depth: 32
IO unplugs: 12 Timer unplugs: 0
CPU1 (ext4.trace):
Reads Queued: 3,119, 12,484KiB Writes Queued: 2,591, 420,624KiB
Read Dispatches: 3,168, 12,484KiB Write Dispatches: 356, 423,100KiB
Reads Requeued: 49 Writes Requeued: 115
Reads Completed: 3,271, 13,104KiB Writes Completed: 251, 11,317KiB
Read Merges: 0, 0KiB Write Merges: 2,358, 9,504KiB
Read depth: 29 Write depth: 32
IO unplugs: 10 Timer unplugs: 2
CPU2 (ext4.trace):
Reads Queued: 191, 844KiB Writes Queued: 2, 3KiB
Read Dispatches: 191, 844KiB Write Dispatches: 1, 3KiB
Reads Requeued: 0 Writes Requeued: 0
Reads Completed: 144, 684KiB Writes Completed: 10, 2,608KiB
Read Merges: 0, 0KiB Write Merges: 0, 0KiB
Read depth: 29 Write depth: 32
IO unplugs: 6 Timer unplugs: 0
CPU3 (ext4.trace):
Reads Queued: 1,273, 5,116KiB Writes Queued: 852, 112,049KiB
Read Dispatches: 1,315, 5,116KiB Write Dispatches: 72, 109,545KiB
Reads Requeued: 42 Writes Requeued: 11
Reads Completed: 1,121, 4,496KiB Writes Completed: 56, 4,024KiB
Read Merges: 0, 0KiB Write Merges: 779, 3,392KiB
Read depth: 29 Write depth: 32
IO unplugs: 15 Timer unplugs: 11
Total (ext4.trace):
Reads Queued: 4,955, 20,388KiB Writes Queued: 5,377, 540,404KiB
Read Dispatches: 5,046, 20,388KiB Write Dispatches: 448, 540,404KiB
Reads Requeued: 91 Writes Requeued: 126
Reads Completed: 4,955, 20,388KiB Writes Completed: 330, 23,072KiB
Read Merges: 0, 0KiB Write Merges: 5,051, 20,552KiB
IO unplugs: 43 Timer unplugs: 13
Throughput (R/W): 5,011KiB/s / 5,671KiB/s
Events (ext4.trace): 57,930 entries
Skips: 0 forward (0 - 0.0%)
Peter was right then. Per multiple recommendations
I'm switching to fstrim, it is quicker than the deletes.
fstrim consistently takes about a minute to run on a 40GB
volume.
time fstrim -v /usr
/usr: 9047117824 bytes were trimmed
real 0m56.121s
user 0m0.090s
sys 0m0.000s
I see that the running of fstrim was already discussed
http://oss.sgi.com/archives/xfs/2011-05/msg00338.html.
Please add something to the FAQ about using SSDs. It would be great if
people could see the recommended mount options and fstrim crontab entry
(or other option) for running xfs on a SSD.
Thanks for the input and patience.
~tom
[-- Attachment #1.1.2: ext4.trace.blktrace.tar.bz --]
[-- Type: application/x-bzip-compressed-tar, Size: 364717 bytes --]
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
[-- Attachment #2: Type: text/plain, Size: 121 bytes --]
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2012-03-02 15:42 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-02-28 22:56 Poor performance using discard Thomas Lynema
2012-02-28 23:58 ` Peter Grandi
2012-02-29 1:22 ` Dave Chinner
2012-02-29 2:00 ` Thomas Lynema
2012-02-29 4:08 ` Dave Chinner
2012-02-29 10:38 ` Peter Grandi
2012-02-29 19:46 ` Eric Sandeen
2012-03-01 5:59 ` Christoph Hellwig
2012-03-01 6:27 ` Dave Chinner
2012-03-01 6:31 ` Christoph Hellwig
[not found] ` <1330658311.6438.24.camel@core24>
2012-03-02 14:57 ` Thomas Lynema
2012-03-02 15:41 ` Thomas Lynema [this message]
2012-03-05 3:02 ` Dave Chinner
2012-03-05 6:41 ` Jeffrey Hundstad
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1330702910.6438.40.camel@core24 \
--to=lyz27@yahoo.com \
--cc=david@fromorbit.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox