Re: known issue with specific kernel releases and snapshots on LVM on md RAID integrity devices

[Marc SCHAEFER <schaefer@alphanet.ch>]

Hello and thank you for your ideas,

On Sat, Oct 04, 2025 at 08:34:26PM +0200, Milan Broz wrote:
> > Do you see anything peculiar with this?
> I do not see anything special, just checksum errors that can be caused by many things.

Indeed.

> I would start with checking hardware, specifically reseat RAM modules and cables,
> check and run some extensive memory and other system tests etc.

I did extensive memory checks before, and I did them again, found nothing
(memtest86, long kernel compilation loops). What I find peculiar is that in
most cases, both RAID1 mirrors had an integrity error at the same blocks:
especially knowing that each RAID1 mirror is, in this case, over its own
dm-integrity device.

But maybe I am mistaken, and there is some optimisation which makes both
dm-integrity devices recognize the original block to be written is shared
between the two device and it is optimized by dm-integrity to be calculated
only once?  If yes, then a simple RAM corruption would explain this issue.

> Check system logs, there could be something that precedes this issue.

No, there was nothing, I get logcheck summaries for this system and there
was no I/O error or anything before this.  I also monitor temperatures,
and there were quite ok just before and during the issues.

Locating the issue more closely I determined which LV contained errors (the
errors, except the correcte one, were always at the same addresses), then
checked all the files from the LV (*) (as errors were on both R1 copies, it
should have shown something), and the error is only seen when reading the raw
device LV, but not in the files theselves.  This also explain why the
initial error was on a snapshot (that is temporary, quickly deleted).

I thus wrote a big new file filling the LV fs, no error, then re-read the raw
LV volume without errors. I am now running a md raidcheck again, that should
tell me if the dm-integrity CRCs are now written correctly on both
devices.

Another question: does --enable-discard really work?  Looking at the
documentation, I am not sure it really works with crc32 and bitmap
integrity mode, do you have a final answer?  If you have no idea,
I will do some tests on a spare system to see what combination
of options really does discard (I am not sure how to test this,
presumably NVMs have a namespace-full percentage, not sure
it's really reliable). Do you have any suggestion?

Thank you and have a nice week.

(*) simple read, then backup comparison for the files that don't
    change.