From: Stefan Priebe - Profihost AG <s.priebe@profihost.ag>
To: "xfs@oss.sgi.com" <xfs@oss.sgi.com>
Subject: Is XFS suitable for 350 million files on 20TB storage?
Date: Fri, 05 Sep 2014 11:47:29 +0200 [thread overview]
Message-ID: <540986B1.4080306@profihost.ag> (raw)
Hi,
i have a backup system running 20TB of storage having 350 million files.
This was working fine for month.
But now the free space is so heavily fragmented that i only see the
kworker with 4x 100% CPU and write speed beeing very slow. 15TB of the
20TB are in use.
Overall files are 350 Million - all in different directories. Max 5000
per dir.
Kernel is 3.10.53 and mount options are:
noatime,nodiratime,attr2,inode64,logbufs=8,logbsize=256k,noquota
# xfs_db -r -c freesp /dev/sda1
from to extents blocks pct
1 1 29484138 29484138 2,16
2 3 16930134 39834672 2,92
4 7 16169985 87877159 6,45
8 15 78202543 999838327 73,41
16 31 3562456 83746085 6,15
32 63 2370812 102124143 7,50
64 127 280885 18929867 1,39
256 511 2 827 0,00
512 1023 65 35092 0,00
2048 4095 2 6561 0,00
16384 32767 1 23951 0,00
Is there anything i can optimize? Or is it just a bad idea to do this
with XFS? Any other options? Maybe rsync options like --inplace /
--no-whole-file?
Greets,
Stefan
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next reply other threads:[~2014-09-05 9:47 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-05 9:47 Stefan Priebe - Profihost AG [this message]
2014-09-05 12:30 ` Is XFS suitable for 350 million files on 20TB storage? Brian Foster
2014-09-05 12:40 ` Stefan Priebe - Profihost AG
2014-09-05 13:48 ` Brian Foster
2014-09-05 18:07 ` Stefan Priebe
2014-09-05 19:18 ` Brian Foster
2014-09-05 20:14 ` Stefan Priebe
2014-09-05 21:24 ` Brian Foster
2014-09-05 22:39 ` Sean Caron
2014-09-05 23:05 ` Dave Chinner
2014-09-06 7:35 ` Stefan Priebe
2014-09-06 15:04 ` Brian Foster
2014-09-06 22:56 ` Dave Chinner
2014-09-08 8:35 ` Stefan Priebe - Profihost AG
2014-09-08 9:46 ` Dave Chinner
2014-09-08 9:49 ` Stefan Priebe - Profihost AG
2014-09-06 14:51 ` Brian Foster
2014-09-06 22:54 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=540986B1.4080306@profihost.ag \
--to=s.priebe@profihost.ag \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox