* LVM snapshot and log record CRC mismatch
@ 2016-05-24 12:58 Gionatan Danti
2016-05-24 22:12 ` Dave Chinner
0 siblings, 1 reply; 8+ messages in thread
From: Gionatan Danti @ 2016-05-24 12:58 UTC (permalink / raw)
To: xfs; +Cc: g.danti
Hi list,
I have a question about LVM snapshot and XFS.
Some preliminary information: CentOS Linux release 7.0.1406 (Core), with
kernel version 3.10.0-123.20.1.el7.x86_64
Basically, each time I mount a LVM snapshot (with the "nouuid" option),
I have the following dmesg entries:
[824501.391058] XFS (dm-7): Mounting Filesystem
[824502.085459] XFS (dm-7): Starting recovery (logdev: internal)
[824502.120435] XFS (dm-7): log record CRC mismatch: found 0x324293f8,
expected 0x43a1b3b5.
[824502.121035] ffffc90018271000: 00 00 00 10 00 00 00 00 69 01 00 00 6d
a0 9d a7 ........i...m...
[824502.121649] ffffc90018271010: 00 00 00 10 69 00 00 00 4e 41 52 54 2a
00 00 00 ....i...NART*...
[824502.122403] XFS (dm-7): log record CRC mismatch: found 0x873f6b2,
expected 0x49045c16.
[824502.122952] ffffc90018271000: 00 00 00 10 00 00 00 08 aa 20 00 00 6e
55 00 00 ......... ..nU..
[824502.123508] ffffc90018271010: 00 00 00 00 69 00 00 00 4e 41 52 54 2a
00 00 00 ....i...NART*...
[824502.124239] XFS (dm-7): log record CRC mismatch: found 0x2f6e47af,
expected 0x7efb222.
[824502.124792] ffffc90018271000: 00 00 00 10 00 00 00 00 69 01 00 00 a7
66 d7 2e ........i....f..
[824502.125359] ffffc90018271010: 00 00 00 10 69 00 00 00 4e 41 52 54 2a
00 00 00 ....i...NART*...
[824502.126173] XFS (dm-7): log record CRC mismatch: found 0x324293f8,
expected 0x43a1b3b5.
[824502.126751] ffffc900182b2000: 00 00 00 10 00 00 00 00 69 01 00 00 6d
a0 9d a7 ........i...m...
[824502.127332] ffffc900182b2010: 00 00 00 10 69 00 00 00 4e 41 52 54 2a
00 00 00 ....i...NART*...
[824502.153510] XFS (dm-7): log record CRC mismatch: found 0x873f6b2,
expected 0x49045c16.
[824502.154158] ffffc900182b2000: 00 00 00 10 00 00 00 08 aa 20 00 00 6e
55 00 00 ......... ..nU..
[824502.154818] ffffc900182b2010: 00 00 00 00 69 00 00 00 4e 41 52 54 2a
00 00 00 ....i...NART*...
[824502.155654] XFS (dm-7): log record CRC mismatch: found 0x2f6e47af,
expected 0x7efb222.
[824502.156258] ffffc900182b2000: 00 00 00 10 00 00 00 00 69 01 00 00 a7
66 d7 2e ........i....f..
[824502.156869] ffffc900182b2010: 00 00 00 10 69 00 00 00 4e 41 52 54 2a
00 00 00 ....i...NART*...
[824502.906702] XFS (dm-7): Ending recovery (logdev: internal)
[824502.906714] SELinux: initialized (dev dm-7, type xfs), uses xattr
From my understanding, this should be more or less the expected
behavior: the snapshot is a "crash consistent" backup point, and
mounting the snapshotted filesystem can led to mismatched CRC. Anyway, I
can read from the snapshot without problems.
However, issuing a xfs_info on the original volume shows that crc
options is 0 (crc=0), so I am somewhat puzzled about the warning above.
Moreover, on a more recent CentOS 7.2 server (kernel version
3.10.0-327.13.1.el7.x86_64), I can not reproduce this warning.
So I ask:
1) it really is the expected behavior, or should I be worried by the
dmesg entries?
2) this is kernel-related, should I update the kernel to newer version?
Thanks.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: LVM snapshot and log record CRC mismatch
2016-05-24 12:58 LVM snapshot and log record CRC mismatch Gionatan Danti
@ 2016-05-24 22:12 ` Dave Chinner
2016-05-25 7:37 ` Gionatan Danti
0 siblings, 1 reply; 8+ messages in thread
From: Dave Chinner @ 2016-05-24 22:12 UTC (permalink / raw)
To: Gionatan Danti; +Cc: xfs
On Tue, May 24, 2016 at 02:58:51PM +0200, Gionatan Danti wrote:
> Hi list,
> I have a question about LVM snapshot and XFS.
>
> Some preliminary information: CentOS Linux release 7.0.1406 (Core),
> with kernel version 3.10.0-123.20.1.el7.x86_64
>
> Basically, each time I mount a LVM snapshot (with the "nouuid"
> option), I have the following dmesg entries:
>
> [824501.391058] XFS (dm-7): Mounting Filesystem
> [824502.085459] XFS (dm-7): Starting recovery (logdev: internal)
> [824502.120435] XFS (dm-7): log record CRC mismatch: found
> 0x324293f8, expected 0x43a1b3b5.
> [824502.121035] ffffc90018271000: 00 00 00 10 00 00 00 00 69 01 00
> 00 6d a0 9d a7 ........i...m...
> [824502.121649] ffffc90018271010: 00 00 00 10 69 00 00 00 4e 41 52
> 54 2a 00 00 00 ....i...NART*...
> [824502.122403] XFS (dm-7): log record CRC mismatch: found
> 0x873f6b2, expected 0x49045c16.
....
>
> From my understanding, this should be more or less the expected
> behavior: the snapshot is a "crash consistent" backup point, and
> mounting the snapshotted filesystem can led to mismatched CRC.
No, a snapshot should not have mismatched CRC errors in the log.
However, log recovery should be seeing only an unmount record during
recovery, so seeing multiple CRC errors indicates that the
filesystem was not quiesced correctly when the snapshot was taken.
How did you take the snapshot? (full command lines, please)
> Anyway, I can read from the snapshot without problems.
Well, until you hit whatever was not replayed out of the log....
> However, issuing a xfs_info on the original volume shows that crc
> options is 0 (crc=0), so I am somewhat puzzled about the warning
> above.
CRCs are now always enabled on the journal. The difference is that
for crc=1 filesystems this is a fatal error and the fs won't mount,
while for crc=0 it is simply a canary for developers when bugs are
reported that something might have gone wrong in log recovery....
> Moreover, on a more recent CentOS 7.2 server (kernel version
> 3.10.0-327.13.1.el7.x86_64), I can not reproduce this warning.
>
> So I ask:
> 1) it really is the expected behavior, or should I be worried by the
> dmesg entries?
Not expected. indicates a problem with the snapshot image.
> 2) this is kernel-related
Maybe.
> should I update the kernel to newer version?
Depends if it is kernel related...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: LVM snapshot and log record CRC mismatch
2016-05-24 22:12 ` Dave Chinner
@ 2016-05-25 7:37 ` Gionatan Danti
2016-05-26 10:23 ` Emmanuel Florac
0 siblings, 1 reply; 8+ messages in thread
From: Gionatan Danti @ 2016-05-25 7:37 UTC (permalink / raw)
To: Dave Chinner; +Cc: Gionatan Danti, xfs
On 25/05/2016 00:12, Dave Chinner wrote:
>
> How did you take the snapshot? (full command lines, please)
>
The snapshot is accomplished using the following commands:
lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
lvchange -ay -K hypervisor/Backup_SNAP
mount -o nouuid /dev/hypervisor/Backup_SNAP /mnt/lvm/
As additional information, this is my logical volume setup (I am using
thin provisioning):
[root@hypervisor ~]# lvs
LV VG Attr LSize Pool Origin Data%
Move Log Cpy%Sync Convert
000-ThinPool hypervisor twi-a-t--- 1.75t 42.42
KVM_Storage hypervisor Vwi-aot--- 1.70t 000-ThinPool 43.61
root hypervisor -wi-ao---- 64.00g
swap hypervisor -wi-ao---- 7.73g
Thanks.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: LVM snapshot and log record CRC mismatch
2016-05-25 7:37 ` Gionatan Danti
@ 2016-05-26 10:23 ` Emmanuel Florac
2016-05-26 11:17 ` Gionatan Danti
0 siblings, 1 reply; 8+ messages in thread
From: Emmanuel Florac @ 2016-05-26 10:23 UTC (permalink / raw)
To: Gionatan Danti; +Cc: xfs
Le Wed, 25 May 2016 09:37:41 +0200
Gionatan Danti <g.danti@assyoma.it> écrivait:
> The snapshot is accomplished using the following commands:
>
> lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
> lvchange -ay -K hypervisor/Backup_SNAP
> mount -o nouuid /dev/hypervisor/Backup_SNAP /mnt/lvm/
>
You need to use xfs_freeze to make the filesystem coherent before
snapshot:
xfs_freeze /dev/hypervisor/KVM_Storage
lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
xfs_freeze -u /dev/hypervisor/KVM_Storage
lvchange -ay -K hypervisor/Backup_SNAP
--
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <eflorac@intellique.com>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: LVM snapshot and log record CRC mismatch
2016-05-26 10:23 ` Emmanuel Florac
@ 2016-05-26 11:17 ` Gionatan Danti
2016-05-26 11:34 ` Emmanuel Florac
2016-05-26 17:39 ` Eric Sandeen
0 siblings, 2 replies; 8+ messages in thread
From: Gionatan Danti @ 2016-05-26 11:17 UTC (permalink / raw)
To: Emmanuel Florac; +Cc: Gionatan Danti, xfs
On 26/05/2016 12:23, Emmanuel Florac wrote:
> Le Wed, 25 May 2016 09:37:41 +0200
> Gionatan Danti <g.danti@assyoma.it> écrivait:
>
>> The snapshot is accomplished using the following commands:
>>
>> lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
>> lvchange -ay -K hypervisor/Backup_SNAP
>> mount -o nouuid /dev/hypervisor/Backup_SNAP /mnt/lvm/
>>
>
> You need to use xfs_freeze to make the filesystem coherent before
> snapshot:
>
> xfs_freeze /dev/hypervisor/KVM_Storage
> lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
> xfs_freeze -u /dev/hypervisor/KVM_Storage
> lvchange -ay -K hypervisor/Backup_SNAP
>
It was my understanding that filesystem freezing was automatically
managed by LVM, and integrated into VFS long ago.
I am missing something?
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: LVM snapshot and log record CRC mismatch
2016-05-26 11:17 ` Gionatan Danti
@ 2016-05-26 11:34 ` Emmanuel Florac
2016-05-30 6:33 ` Gionatan Danti
2016-05-26 17:39 ` Eric Sandeen
1 sibling, 1 reply; 8+ messages in thread
From: Emmanuel Florac @ 2016-05-26 11:34 UTC (permalink / raw)
To: Gionatan Danti; +Cc: xfs
Le Thu, 26 May 2016 13:17:04 +0200
Gionatan Danti <g.danti@assyoma.it> écrivait:
> >
> > You need to use xfs_freeze to make the filesystem coherent before
> > snapshot:
> >
> > xfs_freeze /dev/hypervisor/KVM_Storage
> > lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
> > xfs_freeze -u /dev/hypervisor/KVM_Storage
> > lvchange -ay -K hypervisor/Backup_SNAP
> >
>
> It was my understanding that filesystem freezing was automatically
> managed by LVM, and integrated into VFS long ago.
>
> I am missing something?
Yes, I thought that too, however it doesn't seem to be working in your
case, and freezing twice can't do any harm anyway :)
--
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <eflorac@intellique.com>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: LVM snapshot and log record CRC mismatch
2016-05-26 11:34 ` Emmanuel Florac
@ 2016-05-30 6:33 ` Gionatan Danti
0 siblings, 0 replies; 8+ messages in thread
From: Gionatan Danti @ 2016-05-30 6:33 UTC (permalink / raw)
To: Emmanuel Florac; +Cc: g.danti, xfs
Il 26-05-2016 13:34 Emmanuel Florac ha scritto:
>
> Yes, I thought that too, however it doesn't seem to be working in your
> case, and freezing twice can't do any harm anyway :)
Hi all,
after upgrading to CentOS 7.2, kernel version
3.10.0-327.18.2.el7.x86_64, the problem was solved: mounting a snapshot
shows no more CRC error (ie: the filesystem is quiesced before taking
the snapshot).
I am not sure of what component upgrade solved the problem, if kernel or
lvm tools. I are more inclined to the latter, anyway.
Thanks.
--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: LVM snapshot and log record CRC mismatch
2016-05-26 11:17 ` Gionatan Danti
2016-05-26 11:34 ` Emmanuel Florac
@ 2016-05-26 17:39 ` Eric Sandeen
1 sibling, 0 replies; 8+ messages in thread
From: Eric Sandeen @ 2016-05-26 17:39 UTC (permalink / raw)
To: xfs
On 5/26/16 6:17 AM, Gionatan Danti wrote:
> On 26/05/2016 12:23, Emmanuel Florac wrote:
>> Le Wed, 25 May 2016 09:37:41 +0200
>> Gionatan Danti <g.danti@assyoma.it> écrivait:
>>
>>> The snapshot is accomplished using the following commands:
>>>
>>> lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
>>> lvchange -ay -K hypervisor/Backup_SNAP
>>> mount -o nouuid /dev/hypervisor/Backup_SNAP /mnt/lvm/
>>>
>>
>> You need to use xfs_freeze to make the filesystem coherent before
>> snapshot:
>>
>> xfs_freeze /dev/hypervisor/KVM_Storage
>> lvcreate -s hypervisor/KVM_Storage --name Backup_SNAP
>> xfs_freeze -u /dev/hypervisor/KVM_Storage
>> lvchange -ay -K hypervisor/Backup_SNAP
>>
>
> It was my understanding that filesystem freezing was automatically managed by LVM, and integrated into VFS long ago.
>
> I am missing something?
Nope, you are correct. lvm snapshots handle fs quiesce.
-Eric
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-05-30 6:33 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-05-24 12:58 LVM snapshot and log record CRC mismatch Gionatan Danti
2016-05-24 22:12 ` Dave Chinner
2016-05-25 7:37 ` Gionatan Danti
2016-05-26 10:23 ` Emmanuel Florac
2016-05-26 11:17 ` Gionatan Danti
2016-05-26 11:34 ` Emmanuel Florac
2016-05-30 6:33 ` Gionatan Danti
2016-05-26 17:39 ` Eric Sandeen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox