From: Dave Chinner <david@fromorbit.com>
To: Speedy Milan <speedy.milan@gmail.com>
Cc: xfs@oss.sgi.com, linux-kernel@vger.kernel.org,
Ivan Pantovic <gyro.ivan@gmail.com>
Subject: Re: rm -f * on large files very slow on XFS + MD RAID 6 volume of 15x 4TB of HDDs (52TB)
Date: Wed, 23 Apr 2014 12:18:35 +1000 [thread overview]
Message-ID: <20140423021835.GI15995@dastard> (raw)
In-Reply-To: <CAHuzUScfp19c_th_pfsZs05+yDz34MuEH-P1f+FF1dcivfH=5Q@mail.gmail.com>
[cc xfs@oss.sgi.com]
On Mon, Apr 21, 2014 at 10:58:53PM +0200, Speedy Milan wrote:
> I want to report very slow deletion of 24 50GB files (in total 12 TB),
> all present in the same folder.
total = 1.2TB?
> OS is CentOS 6.4, with upgraded kernel 3.13.1.
>
> The hardware is a Supermicro server with 15x 4TB WD Se drives in MD
> RAID 6, totalling 52TB of free space.
>
> XFS is formated directly on the RAID volume, without LVM layers.
>
> Deletion was done with rm -f * command, and it took upwards of 1 hour
> to delete the files.
>
> File system was filled completely prior to deletion.
Oh, that's bad. it's likely you fragmented the files into
millions of extents?
> rm was mostly waiting (D state), probably for kworker threads, and
No, waiting for IO.
> iostat was showing big HDD utilization numbers and very low throughput
> so it looked like a random HDD workload was in effect.
Yup, smells like file fragmentation. Non-fragmented 50GB files
should be removed in a few milliseconds. but if you've badly
fragmented the files, there could be 10 million extents in a 50GB
file. A few milliseconds per extent removal gives you....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next parent reply other threads:[~2014-04-23 2:18 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAHuzUScfp19c_th_pfsZs05+yDz34MuEH-P1f+FF1dcivfH=5Q@mail.gmail.com>
2014-04-23 2:18 ` Dave Chinner [this message]
2014-04-23 7:23 ` rm -f * on large files very slow on XFS + MD RAID 6 volume of 15x 4TB of HDDs (52TB) Ivan Pantovic
2014-04-23 8:25 ` Dave Chinner
2014-04-23 9:21 ` Ivan Pantovic
2014-04-23 22:12 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140423021835.GI15995@dastard \
--to=david@fromorbit.com \
--cc=gyro.ivan@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=speedy.milan@gmail.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).