All of lore.kernel.org
 help / color / mirror / Atom feed
From: Stefan Priebe - Profihost AG <s.priebe@profihost.ag>
To: "xfs@oss.sgi.com" <xfs@oss.sgi.com>
Subject: Is XFS suitable for 350 million files on 20TB storage?
Date: Fri, 05 Sep 2014 11:47:29 +0200	[thread overview]
Message-ID: <540986B1.4080306@profihost.ag> (raw)

Hi,

i have a backup system running 20TB of storage having 350 million files.
This was working fine for month.

But now the free space is so heavily fragmented that i only see the
kworker with 4x 100% CPU and write speed beeing very slow. 15TB of the
20TB are in use.

Overall files are 350 Million - all in different directories. Max 5000
per dir.

Kernel is 3.10.53 and mount options are:
noatime,nodiratime,attr2,inode64,logbufs=8,logbsize=256k,noquota

# xfs_db -r -c freesp /dev/sda1
   from      to extents  blocks    pct
      1       1 29484138 29484138   2,16
      2       3 16930134 39834672   2,92
      4       7 16169985 87877159   6,45
      8      15 78202543 999838327  73,41
     16      31 3562456 83746085   6,15
     32      63 2370812 102124143   7,50
     64     127  280885 18929867   1,39
    256     511       2     827   0,00
    512    1023      65   35092   0,00
   2048    4095       2    6561   0,00
  16384   32767       1   23951   0,00

Is there anything i can optimize? Or is it just a bad idea to do this
with XFS? Any other options? Maybe rsync options like --inplace /
--no-whole-file?

Greets,
Stefan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

             reply	other threads:[~2014-09-05  9:47 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-05  9:47 Stefan Priebe - Profihost AG [this message]
2014-09-05 12:30 ` Is XFS suitable for 350 million files on 20TB storage? Brian Foster
2014-09-05 12:40   ` Stefan Priebe - Profihost AG
2014-09-05 13:48     ` Brian Foster
2014-09-05 18:07       ` Stefan Priebe
2014-09-05 19:18         ` Brian Foster
2014-09-05 20:14           ` Stefan Priebe
2014-09-05 21:24             ` Brian Foster
2014-09-05 22:39               ` Sean Caron
2014-09-05 23:05     ` Dave Chinner
2014-09-06  7:35       ` Stefan Priebe
2014-09-06 15:04         ` Brian Foster
2014-09-06 22:56           ` Dave Chinner
2014-09-08  8:35             ` Stefan Priebe - Profihost AG
2014-09-08  9:46               ` Dave Chinner
2014-09-08  9:49                 ` Stefan Priebe - Profihost AG
2014-09-06 14:51       ` Brian Foster
2014-09-06 22:54         ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=540986B1.4080306@profihost.ag \
    --to=s.priebe@profihost.ag \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.