Re: Suggestions for building new 44TB Raid5 array

[Marc MERLIN <marc@merlins.org>]

On Sat, Jun 11, 2022 at 10:54:16PM +0500, Roman Mamedov wrote:
> On Sat, 11 Jun 2022 07:52:59 -0700
> Marc MERLIN <marc@merlins.org> wrote:
> 
> > 1) mdadm --create /dev/md7 --level=5 --consistency-policy=ppl
> > --raid-devices=5 /dev/sd[abdef]1 --chunk=256 --bitmap=internal
> 
> One more thing I wanted to mention, did you have PPL on your previous array?
> Or it was not implemented yet back then? I know it is supposed to protect
> against the write hole, which could have caused your previous FS corruption.
 
Looks like I had internal bitmap instead
gargamel:~# mdadm --query --detail  /dev/md7
/dev/md7:
           Version : 1.2
     Creation Time : Sun Feb 11 20:38:30 2018
        Raid Level : raid5
        Array Size : 23441561600 (22355.62 GiB 24004.16 GB)
     Used Dev Size : 5860390400 (5588.90 GiB 6001.04 GB)
      Raid Devices : 5
     Total Devices : 5
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Fri Jun 10 12:09:08 2022
             State : clean 
    Active Devices : 5
   Working Devices : 5
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : bitmap

I'll switch PPL instead, thanks for that. Actually I need to migrate
my other raid5 arrays to that too. It looks like it can be done at runtime.

> Yes, but in MergerFS each file is stored entirely within a single disk,
> there's no striping. So only files which happened to be on the failed disk are
> lost and need to be restored from backups. For this it helps to keep track of
> what was where, with something like "find /mnt/ > `date`.lst" in crontab.

Right, I figured. It's not bad, but I do want no data loss if I lose a
drive, so I'll take raid5.
I realize that filesystem aware raid5, like the raid5 in btrfs which I'm
not sure I can really trust, still? , could lay out the files to be one per disk
without striping.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/                       | PGP 7F55D5F27AAF9D08