public inbox for linux-raid@vger.kernel.org
 help / color / mirror / Atom feed
From: Johannes Truschnigg <johannes@truschnigg.info>
To: Roger Heflin <rogerheflin@gmail.com>
Cc: eyal@eyal.emu.id.au, linux-raid@vger.kernel.org
Subject: Re: problem with recovered array
Date: Thu, 2 Nov 2023 09:34:51 +0100	[thread overview]
Message-ID: <ZUNfK1jqBNsm97Q-@vault.lan> (raw)
In-Reply-To: <CAAMCDefUcuz2Nzh7AvP9m50uq86ZBK3AhEAEynVG_mmmY_f0jQ@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

Hi list,

for the record, I do not think that any of the observations the OP made can be
explained by non-pathological phenomena/patterns of behavior. Something is
very clearly wrong with how this system behaves (the reported figures do not
at all match the expected performance of even a degraded RAID6 array in my
experience) and how data written to the filesystem apparently fails to make it
into the backing devices in acceptable time.

The whole affair reeks either of "subtle kernel bug", or maybe "subtle
hardware failure", I think.

Still, it'd be interesting to know what happens when writes to the array thru
the file system are performed with O_DIRECT in effect, i.e., using `dd
oflag=direct status=progress ...` - does this yield any observable (via
`iostat` et al.) thruput to the disks beneath? What transfer speeds does `dd`
report this way with varying block sizes? Are there no concerning messages in
the debug ringbuffer (`dmesg`) at this time?

I'm not sure how we'd best learn more about what the participating busy kernel
threads (Fedora 38 might have more convenient devices up its sleeve than
ftrace?) are spending their time on in particular, but I think that's what's
needed to be known to pin down the underlying cause of the problem.

-- 
with best regards:
- Johannes Truschnigg ( johannes@truschnigg.info )

www:   https://johannes.truschnigg.info/
phone: +436502133337
xmpp:  johannes@truschnigg.info

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

  reply	other threads:[~2023-11-02  8:40 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-30 13:35 problem with recovered array eyal
2023-10-30 16:14 ` Roger Heflin
2023-10-31  2:35   ` eyal
2023-10-31  2:50     ` eyal
2023-10-31  3:21   ` Carlos Carvalho
2023-10-31  6:25     ` eyal
2023-10-31  9:29     ` eyal
2023-10-31 10:24       ` Roger Heflin
2023-10-31 21:40         ` eyal
2023-11-01 10:30           ` Roger Heflin
2023-11-01 13:08             ` eyal
2023-11-01 14:29               ` Roger Heflin
2023-11-02  8:34                 ` Johannes Truschnigg [this message]
2023-11-02 11:27                   ` eyal
2023-11-02 11:57                     ` Roger Heflin
2023-11-02 13:05                       ` eyal
2023-11-02 17:05                         ` Roger Heflin
2023-11-02 23:23                           ` eyal
2023-11-03 12:08                             ` Roger Heflin
2023-11-02 12:29                     ` eyal
2023-11-02 12:46                       ` Reindl Harald
2023-11-03 14:16                   ` Carlos Carvalho
2023-11-03 14:32                     ` Dirty page flushing regression in 6.5.x vs 6.1.x Roman Mamedov
2023-11-03 15:57                     ` problem with recovered array Roger Heflin
2023-11-03 22:38                       ` eyal
2023-11-04  0:48                         ` eyal
2023-11-04  1:01                           ` Roger Heflin
2023-11-04 10:04                             ` eyal
2023-10-31 12:39       ` Carlos Carvalho
2023-10-31 14:19         ` Roger Heflin
2023-10-31 19:20           ` Carlos Carvalho
2023-10-31 21:44 ` problem with recovered array [more details] eyal
2023-10-31 22:00   ` eyal
2023-11-01  4:31   ` [now urgent] problem with recovered array eyal
2023-11-01  6:44     ` eyal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZUNfK1jqBNsm97Q-@vault.lan \
    --to=johannes@truschnigg.info \
    --cc=eyal@eyal.emu.id.au \
    --cc=linux-raid@vger.kernel.org \
    --cc=rogerheflin@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox