log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f

log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f

[Christian Kujau]

Hi,

this PowerBook G4 crashed (power supply failed) and now I have to wait a 
few days until the new PSU arrives. In the meantime I attached its 
external disks to an Intel Atom machine (i686) and wanted to mount the 
disks:

# mount -t xfs /dev/mapper/wdc1 /mnt/media
mount: /dev/mapper/wdc1: can't read superblock

In dmesg:

XFS (dm-3): Mounting Filesystem
XFS (dm-3): Starting recovery (logdev: internal)
XFS (dm-3): log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f.
f8808000: 00 00 00 01 00 00 00 00 69 01 00 00 76 63 5a 10  ........i...vcZ.
f8808010: 00 00 00 10 69 00 00 00 54 52 41 4e 00 00 00 2a  ....i...TRAN...*
XFS (dm-3): dirty log written in incompatible format - can't recover
XFS (dm-3): log mount/recovery failed: error 5
XFS (dm-3): log mount failed

Note: /dev/mapper/wdc1 is a dm-crypt device, not sure if this matters 
though.

Running "xfs_repair -n /dev/mapper/wdc1" came back with an exit code of 
"0" and did not report anything unusual (attached below).

Running xfs_repair (v3.2.0 from Fedora 20) *without* "-n" warns that the 
filesystem has valuable metadata changes and advises to mount it first. Or 
use "xfs_repair -L" to clear the log, but I have not done this just yet.

I *think* I have mounted the same filesystem before on this i686 system 
(some weeks ago) but then the filesystem got umounted cleanly from the 
PowerPC system, where this disk is attached to usually.

The PowerPC box is running the latest vanilla kernel (sometimes a few -rc 
versions behind) and was running 3.15.0 when the power supply failed. This 
i686 system (an IdeaPad S10 Laptop) is running a semi-current Fedora 20 
installation (with 3.14.5-200.fc20.i686 right now).

Could it be that endianess has something to do with it? The PowerPC is 
big-endian I think, and the i686 Laptop is little-endian. Even though I'm 
almost certain that I've mounted the disk before, now that the filesystem 
needs to be recovered first, maybe endianess matters now?

Ideas welcome :)

Thanks,
Christian.

# xfs_repair -n /dev/mapper/wdc1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
        - scan filesystem freespace and inode maps...
        - found root inode chunk
Phase 3 - for each AG...
        - scan (but don't clear) agi unlinked lists...
        - process known inodes and perform inode discovery...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
        - process newly discovered inodes...
Phase 4 - check for duplicate blocks...
        - setting up duplicate extent list...
        - check for inodes claiming duplicate blocks...
        - agno = 0
        - agno = 1
        - agno = 2
        - agno = 3
No modify flag set, skipping phase 5
Phase 6 - check inode connectivity...
        - traversing filesystem ...
        - traversal finished ...
        - moving disconnected inodes to lost+found ...
Phase 7 - verify link counts...
No modify flag set, skipping filesystem flush and exiting.
# echo $?
0

# file -Ls /dev/mapper/wdc1 
/dev/mapper/wdc1: SGI XFS filesystem data (blksz 4096, inosz 256, v2 dirs)

# blockdev --getsize64 /dev/mapper/wdc1 
1000204323328

# cryptsetup status wdc1
/dev/mapper/wdc1 is active.
  type:    LUKS1
  cipher:  aes-cbc-essiv:sha256
  keysize: 128 bits
  device:  /dev/sdc1
  offset:  1032 sectors
  size:    1953524069 sectors
  mode:    read/write

-- 
BOFH excuse #121:

halon system went off and killed the operators.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f

[Dave Chinner]

On Wed, Jun 18, 2014 at 02:44:27AM -0700, Christian Kujau wrote:
> Hi,
> 
> this PowerBook G4 crashed (power supply failed) and now I have to wait a 
> few days until the new PSU arrives. In the meantime I attached its 
> external disks to an Intel Atom machine (i686) and wanted to mount the 
> disks:

The XFS transactions in the log are not endian neutral - they are
written in the endian format of the current CPU. PowerBook G4 is a
ppc chip which is big endian, intel atom is little endian.

The log headers are always big endian, and they have a field in
the header to determine the endianness of the contents of the log
records...

> # mount -t xfs /dev/mapper/wdc1 /mnt/media
> mount: /dev/mapper/wdc1: can't read superblock
> 
> In dmesg:
> 
> XFS (dm-3): Mounting Filesystem
> XFS (dm-3): Starting recovery (logdev: internal)
> XFS (dm-3): log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f.
> f8808000: 00 00 00 01 00 00 00 00 69 01 00 00 76 63 5a 10  ........i...vcZ.
> f8808010: 00 00 00 10 69 00 00 00 54 52 41 4e 00 00 00 2a  ....i...TRAN...*
> XFS (dm-3): dirty log written in incompatible format - can't recover
> XFS (dm-3): log mount/recovery failed: error 5
> XFS (dm-3): log mount failed

	if (unlikely(head->h_fmt != cpu_to_be32(XLOG_FMT))) {
                xfs_warn(mp,
        "dirty log written in incompatible format - can't recover");

#ifdef XFS_NATIVE_HOST
#define XLOG_FMT XLOG_FMT_LINUX_BE
#else
#define XLOG_FMT XLOG_FMT_LINUX_LE
#endif

So that's exactly what I'd expect to see from filesystem with the
wrong endianness in the log.

> Note: /dev/mapper/wdc1 is a dm-crypt device, not sure if this matters 
> though.
> 
> Running "xfs_repair -n /dev/mapper/wdc1" came back with an exit code of 
> "0" and did not report anything unusual (attached below).
> 
> Running xfs_repair (v3.2.0 from Fedora 20) *without* "-n" warns that the 
> filesystem has valuable metadata changes and advises to mount it first. Or 
> use "xfs_repair -L" to clear the log, but I have not done this just yet.

"-n" simply ignores what is in the log.

> I *think* I have mounted the same filesystem before on this i686 system 
> (some weeks ago) but then the filesystem got umounted cleanly from the 
> PowerPC system, where this disk is attached to usually.

Yes, that's the only way to safely move a filesystem between
machines of different endianness - a clean log has nothing to
replay, so no endian problems and all new records will be written
according to the endianness of the host the filesystem is currently
mounted on.

> Could it be that endianess has something to do with it? The PowerPC is 
> big-endian I think, and the i686 Laptop is little-endian. Even though I'm 
> almost certain that I've mounted the disk before, now that the filesystem 
> needs to be recovered first, maybe endianess matters now?

Always has mattered for the log. We're slowly moving the structure
of the internal code to be able to fix the endianess of the
transaction contents, but we're not really in any hurry given most
archs are moving to be little endian anyway....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f

[Christian Kujau]

On June 18, 2014 5:01:24 AM PDT, Dave Chinner <david@fromorbit.com> wrote:
>Yes, that's the only way to safely move a filesystem between
>machines of different endianness - a clean log has nothing to
>replay, so no endian problems and all new records will be written
>according to the endianness of the host the filesystem is currently
>mounted on.

OK, thanks for explaining that.

When the PowerPC box crashed there should not have been any i/o on the file system - so, if there was nothing to commit, clearing the log with "xfs_repair -L" should not lose any data, right?

Christian.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f

[Dave Chinner]

On Wed, Jun 18, 2014 at 10:13:23AM -0700, Christian Kujau wrote:
> 
> 
> On June 18, 2014 5:01:24 AM PDT, Dave Chinner
> <david@fromorbit.com> wrote:
> >Yes, that's the only way to safely move a filesystem between
> >machines of different endianness - a clean log has nothing to
> >replay, so no endian problems and all new records will be written
> >according to the endianness of the host the filesystem is
> >currently mounted on.
> 
> OK, thanks for explaining that.
> 
> When the PowerPC box crashed there should not have been any i/o on
> the file system - so, if there was nothing to commit, clearing the
> log with "xfs_repair -L" should not lose any data, right?

In theory. use xfs_logprint to check the log is empty...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: log record CRC mismatch: found 0x10a71f1d, expected 0xe012d25f

[Christian Kujau]

On Wed, 18 Jun 2014 at 17:22, Christian Kujau wrote:
> On Thu, 19 Jun 2014 at 09:07, Dave Chinner wrote:
> > > When the PowerPC box crashed there should not have been any i/o on
> > > the file system - so, if there was nothing to commit, clearing the
> > > log with "xfs_repair -L" should not lose any data, right?
> > 
> > In theory. use xfs_logprint to check the log is empty...
> 
> Ah - next time I'll use that. I was too impatient and went with 
> "-L" and now the filesystem could be mounted. All is good.

My PowerBook G4 is offline again (rats!) and again I connected this disk 
to an i686 laptop. Trying to mount fails again, because the XFS file system 
was not unmounted properly and the i686 laptop cannot read the big-endian 
XFS log. OK, no biggie, this had been explained earlier. I just wanted to 
share the xfs_logprint(8) output of this filesystem and I haven't cleared 
the log yet - just in case anyone wants to play around with that some 
more:

# xfs_logprint /dev/mapper/wdc1 
xfs_logprint:
    data device: 0xfd03
    log device: 0xfd03 daddr: 976762064 length: 262144

cycle: 1        version: 2              lsn: 1,0        tail_lsn: 1,0
length of Log Record: 20        prev offset: -1         num ops: 1
uuid: 3f121b9a-19e9-4cc8-97ba-86ec62240026   format: little endian linux
h_size: 32768
----------------------------------------------------------------------------
Oper (0): tid: b0c0d0d0  len: 8  clientid: LOG  flags: UNMOUNT 
Unmount filesystem

============================================================================
cycle: 1        version: 2              lsn: 1,2        tail_lsn: 1,2
length of Log Record: 512       prev offset: 0          num ops: 5
uuid: 3f121b9a-19e9-4cc8-97ba-86ec62240026   format: big endian linux
h_size: 32768
----------------------------------------------------------------------------
Oper (0): tid: 4029c0bb  len: 0  clientid: TRANS  flags: START 
----------------------------------------------------------------------------
Oper (1): tid: 4029c0bb  len: 16  clientid: TRANS  flags: none
xfs_logprint: unknown log operation type (5254)
**********************************************************************
* ERROR: data block=2                                                 *
**********************************************************************
Bad data in log



When the PowerPC crashed, the XFS file system was mounte read-write but 
no writes should have been active, only read operations at best.

I'll probably clear the log tomorrow. And yes, this might be a more exotic
scenario so don't bother with code fixes, I just wanted this to be 
somewhat documented in the archives.

Cheers,
Christian.

# mount -t xfs -o ro /dev/mapper/wdc1 /mnt
# dmesg
[...]
 XFS (dm-3): Mounting V4 Filesystem
 XFS (dm-3): Starting recovery (logdev: internal)
 XFS (dm-3): log record CRC mismatch: found 0xbb3995f5, expected 0x6e17c591.
 fa400000: 00 00 00 01 00 00 00 00 69 01 00 00 8e 6c 64 e0  ........i....ld.
 fa400010: 00 00 00 10 69 00 00 00 54 52 41 4e 00 00 00 2a  ....i...TRAN...*
 XFS (dm-3): dirty log written in incompatible format - can't recover
 XFS (dm-3): log mount/recovery failed: error 5
 XFS (dm-3): log mount failed

> Now I only have to make sure to cleanly unmount from the i686 box 
> once my PowerPC system is back online :-)
> 
> Thanks you!
> Christian.
> -- 
> BOFH excuse #25:
> 
> Decreasing electron flux
-- 
BOFH excuse #334:

50% of the manual is in .pdf readme files

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs