Linux RAID subsystem development
 help / color / mirror / Atom feed
From: David Niklas <simd@vfemail.net>
To: Wol <antlists@youngman.org.uk>
Cc: Linux RAID <linux-raid@vger.kernel.org>, linux-kernel@vger.kernel.org
Subject: Re: Need help increasing raid scan efficiency.
Date: Tue, 3 Jun 2025 16:04:15 -0400	[thread overview]
Message-ID: <20250603160415.61c9ca7c@Zen-II-x12.niklas.com> (raw)
In-Reply-To: <24815d81-2e4f-4ddf-b194-b03ea3232b91@youngman.org.uk>

On Tue, 3 Jun 2025 17:46:01 +0100
Wol <antlists@youngman.org.uk> wrote:
> On 03/06/2025 02:05, David Niklas wrote:
> > So I setup the array into read-only mode and started the array with
> > only two of the drives. Drives 0 and 1. Then I proceeded to try and
> > start a second pair, drives 2 and 3, so that I could scan them
> > simultaneously. With the intent of then switching it over to 0 and 2
> > and 1 and 3, then 0 and 3 and 1 and 2.  
> 
> BACKUP! BACKUP!! BACKUP!!!

It's when I was trying to make my yearly backup that I found out it was
corrupting. I have HDDs I backup to. When I backup, I erase the
previous year (as I don't have enough room otherwise), then backup the new
year. As a system, it worked up until now.

> Is your array that messed up that it won't assemble? If you can just
> get it to assemble normally that's your best bet by far. Trying to
> assemble it as two pairs is throwing away the whole point of a raid 6!

It assembles fine with all the disks, the problem is the data corruption
that has occurred across the members.

> And make sure you know the order of the drives in the array! I hope you
> haven't lost that infof.

Everything is written down on paper.

> If your event counts are all similar, then you'll hopefully recover
> most of your data. Your biggest worry will be the mobo and ram having
> trashing an in-flight write that corrupts the disk.

Yes, that's my problem. I wanted to try and isolate the disk pairs so
that I could try and figure out if there is any pattern or differing
copies that would allow me to restore the corrupted data.

> Then once you've got the array assembled, I can't remember the command,
> but there is a command that will read the entire stripe, check the
> paritIES - both of them, and recreate the data. If that fails, your
> data is probably toast, and nothing you can do will be able to retrieve
> much :-(
> 
> Cheers,
> Wol
> 

Searching online turned up raid6check.
https://unix.stackexchange.com/questions/137384/raid6-scrubbing-mismatch-repair

But the people there also pointed out that Linux's raid repair operation
only recalculates the parity. I would have thought that it did a best of
3 option. I mean, that's a big part of why we have RAID6 instead of RAID5,
right?

I think you misunderstood my original question, how do I assemble the
RAID6 pairs (RO mode) into two different arrays such that I can read from
them simultaneously?

If I have to do some coding with respect to the mdadm utility, I'm
willing. But for all I know, the Linux Kernel might destroy all of my data
if I try something like that.

Thanks,
David

WARNING: multiple messages have this Message-ID (diff)
From: David Niklas <simd@vfemail.net>
To: Linux RAID <linux-raid@vger.kernel.org>
Cc: linux-kernel@vger.kernel.org
Subject: Re: Need help increasing raid scan efficiency.
Date: Tue, 3 Jun 2025 16:04:40 -0400	[thread overview]
Message-ID: <20250603160415.61c9ca7c@Zen-II-x12.niklas.com> (raw)
Message-ID: <20250603200440.H9NzN4tlI-91FzsXaXnLrZ3IJQrIa35hE8Ax3NfI3t4@z> (raw)
In-Reply-To: <24815d81-2e4f-4ddf-b194-b03ea3232b91@youngman.org.uk>

On Tue, 3 Jun 2025 17:46:01 +0100
Wol <antlists@youngman.org.uk> wrote:
> On 03/06/2025 02:05, David Niklas wrote:
> > So I setup the array into read-only mode and started the array with
> > only two of the drives. Drives 0 and 1. Then I proceeded to try and
> > start a second pair, drives 2 and 3, so that I could scan them
> > simultaneously. With the intent of then switching it over to 0 and 2
> > and 1 and 3, then 0 and 3 and 1 and 2.  
> 
> BACKUP! BACKUP!! BACKUP!!!

It's when I was trying to make my yearly backup that I found out it was
corrupting. I have HDDs I backup to. When I backup, I erase the
previous year (as I don't have enough room otherwise), then backup the new
year. As a system, it worked up until now.

> Is your array that messed up that it won't assemble? If you can just
> get it to assemble normally that's your best bet by far. Trying to
> assemble it as two pairs is throwing away the whole point of a raid 6!

It assembles fine with all the disks, the problem is the data corruption
that has occurred across the members.

> And make sure you know the order of the drives in the array! I hope you
> haven't lost that infof.

Everything is written down on paper.

> If your event counts are all similar, then you'll hopefully recover
> most of your data. Your biggest worry will be the mobo and ram having
> trashing an in-flight write that corrupts the disk.

Yes, that's my problem. I wanted to try and isolate the disk pairs so
that I could try and figure out if there is any pattern or differing
copies that would allow me to restore the corrupted data.

> Then once you've got the array assembled, I can't remember the command,
> but there is a command that will read the entire stripe, check the
> paritIES - both of them, and recreate the data. If that fails, your
> data is probably toast, and nothing you can do will be able to retrieve
> much :-(
> 
> Cheers,
> Wol
> 

Searching online turned up raid6check.
https://unix.stackexchange.com/questions/137384/raid6-scrubbing-mismatch-repair

But the people there also pointed out that Linux's raid repair operation
only recalculates the parity. I would have thought that it did a best of
3 option. I mean, that's a big part of why we have RAID6 instead of RAID5,
right?

I think you misunderstood my original question, how do I assemble the
RAID6 pairs (RO mode) into two different arrays such that I can read from
them simultaneously?

If I have to do some coding with respect to the mdadm utility, I'm
willing. But for all I know, the Linux Kernel might destroy all of my data
if I try something like that.

Thanks,
David

  reply	other threads:[~2025-06-03 20:04 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-06-03  1:05 Need help increasing raid scan efficiency David Niklas
2025-06-03 16:46 ` Wol
2025-06-03 20:04   ` David Niklas [this message]
2025-06-03 20:04     ` David Niklas
2025-06-03 20:27     ` anthony
2025-06-04  4:22     ` Jani Partanen
2025-06-04  8:59     ` Reindl Harald
2025-06-04  8:46 ` Reindl Harald
2025-06-04 12:24   ` John Stoffel
2025-06-04 12:34 ` John Stoffel
2025-06-04 12:52   ` Reindl Harald
2025-06-04 15:45   ` David Niklas

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250603160415.61c9ca7c@Zen-II-x12.niklas.com \
    --to=simd@vfemail.net \
    --cc=antlists@youngman.org.uk \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox