Looking for advice on creating XFS on a large (to me) RAID

Looking for advice on creating XFS on a large (to me) RAID

[Krzysztof Adamski]

Hi All,

I'm about to create a XFS file system on a MD RAID6 made up of 12 3T
drives.

This is the command line that I was planning to use:

# mkfs.xfs  -l lazy-count=1 -s size=4096 -N /dev/md7
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=7320776704,imaxpct=5
         =                       sunit=128    swidth=1280 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=521728, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0


This is the MD examine for one of the drives:
Nitrogen:~# mdadm -E /dev/sda2
/dev/sda2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : ee8cb70e:2f40f34f:79d46ade:4b12c78b
  Creation Time : Tue Nov  1 17:03:38 2011
     Raid Level : raid6
   Raid Devices : 12

 Avail Dev Size : 5856624640 (2792.66 GiB 2998.59 GB)
     Array Size : 58566236160 (27926.56 GiB 29985.91 GB)
  Used Dev Size : 5856623616 (2792.66 GiB 2998.59 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
    Device UUID : 72fa45ca:775f244d:15b030fe:f569fe8c

Internal Bitmap : 8 sectors from superblock
    Update Time : Thu Nov  3 11:07:39 2011
       Checksum : 4c72bfd4 - correct
         Events : 11029

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAAAAAAAAAAA ('A' == active, '.' == missing)

# mkfs.xfs -V
mkfs.xfs version 3.1.4

System is Debian Squeeze with self compiled 3.0.8 kernel (64bit)

Should I change any default parameters to mkfs.xfs

If I didn't provide any important info let me know.

Thanks in advance,
K

Re: Looking for advice on creating XFS on a large (to me) RAID

[Christoph Hellwig]

On Thu, Nov 03, 2011 at 12:29:58PM -0400, Krzysztof Adamski wrote:
> Hi All,
> 
> I'm about to create a XFS file system on a MD RAID6 made up of 12 3T
> drives.
> 
> This is the command line that I was planning to use:
> 
> # mkfs.xfs

> -l lazy-count=1

This is the default in any semi-recent xfsprogs version.

> -s size=4096

using a sector size larger than the actual disk sector size might risk
data loss, as the log code expects sector sized writes to be atomic.

If you have a real 4k logical sector disk it should be picked up
automatically with the kernel topology support.  We don't quite do the
right thing for 512 byte logical / 4k physical disks, in which case
you might need this line.

> -N

That won't actually create you a filesystem :)

>          =                       sunit=128    swidth=1280 blks

In general you're better off with a 32k stripe unit / chunk size,
than the extremly large default of 512k.

> Should I change any default parameters to mkfs.xfs

Usually not.

Re: Looking for advice on creating XFS on a large (to me) RAID

[Krzysztof Adamski]

On Thu, 2011-11-03 at 13:14 -0400, Christoph Hellwig wrote:
> On Thu, Nov 03, 2011 at 12:29:58PM -0400, Krzysztof Adamski wrote:
> > Hi All,
> > 
> > I'm about to create a XFS file system on a MD RAID6 made up of 12 3T
> > drives.
> > 
> > This is the command line that I was planning to use:
> > 
> > -s size=4096
> 
> using a sector size larger than the actual disk sector size might risk
> data loss, as the log code expects sector sized writes to be atomic.
> 
> If you have a real 4k logical sector disk it should be picked up
> automatically with the kernel topology support.  We don't quite do the
> right thing for 512 byte logical / 4k physical disks, in which case
> you might need this line.

The drives I'm using are Hitachi H3IK30003272SW (0S03208) 3TB, I belive
they are the 512 byte logical / 4k physical disks.


> >          =                       sunit=128    swidth=1280 blks
> 
> In general you're better off with a 32k stripe unit / chunk size,
> than the extremly large default of 512k.

I'll compare a simple dd benchmark of the raw md device with 32k and
512k and report back. The resync takes over 23 hours :-(

Thanks for the info.
K