Re: 5.3-rc1 regression with XFS log recovery

hch
2019-08-19T04:40:17Z

On Mon, Aug 19, 2019 at 06:29:05AM +0200, hch@lst.de wrote:
> On Mon, Aug 19, 2019 at 02:22:59PM +1000, Dave Chinner wrote:
> > That implies a kmalloc heap issue.
> >
> > Oh, is memory poisoning or something that modifies the alignment of
> > slabs turned on?
> >
> > i.e. 4k/8k allocations from the kmalloc heap slabs might not be
> > appropriately aligned for IO, similar to the problems we have with
> > the xen blk driver?
>
> That is what I suspect, and as you can see in the attached config I
> usually run with slab debuggig on.

Yep, looks like an unaligned allocation:

root@testvm:~# mount /dev/pmem1 /mnt/
[ 62.346660] XFS (pmem1): Mounting V5 Filesystem
[ 62.347960] unaligned allocation, offset = 680
[ 62.349019] unaligned allocation, offset = 680
[ 62.349872] unaligned allocation, offset = 680
[ 62.350703] XFS (pmem1): totally zeroed log
[ 62.351443] unaligned allocation, offset = 680
[ 62.452203] unaligned allocation, offset = 344
[ 62.528964] XFS: Assertion failed: head_blk != tail_blk, file:
fs/xfs/xfs_lo6
[ 62.529879] ------------[ cut here ]------------
[ 62.530334] kernel BUG at fs/xfs/xfs_message.c:102!
[ 62.530824] invalid opcode: 0000 [#1] SMP PTI

With the following debug patch. Based on that I think I'll just
formally submit the vmalloc switch as we're at -rc5, and then we
can restart the unaligned slub allocation drama..

diff --git a/fs/xfs/xfs_log_recover.c b/fs/xfs/xfs_log_recover.c
index 13d1d3e95b88..6a098d35931a 100644
--- a/fs/xfs/xfs_log_recover.c
+++ b/fs/xfs/xfs_log_recover.c
@@ -97,6 +97,9 @@ xlog_alloc_buffer(
struct xlog *log,
int nbblks)
{
+ void *ret;
+ unsigned long offset;
+
/*
* Pass log block 0 since we don't have an addr yet, buffer will be
* verified on read.
@@ -125,7 +128,14 @@ xlog_alloc_buffer(
if (nbblks > 1 && log->l_sectBBsize > 1)
nbblks += log->l_sectBBsize;
nbblks = round_up(nbblks, log->l_sectBBsize);
- return kmem_alloc_large(BBTOB(nbblks), KM_MAYFAIL);
+ ret = kmem_alloc_large(BBTOB(nbblks), KM_MAYFAIL);
+ if (!ret)
+ return NULL;
+ offset = offset_in_page(ret);
+ if (offset % 512)
+ printk("unaligned allocation, offset = %lu\n",
+ offset);
+ return ret;
}

/*