Re: Dcache oops

Oleg Drokin
2016-06-03T21:17:15Z

On Jun 3, 2016, at 4:07 PM, Al Viro wrote:

> On Fri, Jun 03, 2016 at 02:35:41PM -0400, Oleg Drokin wrote:
>
>>>> [ 2642.364383] BUG: unable to handle kernel paging request at ffff880113f82000
>>>> [ 2642.365014] IP: [<ffffffff817f87d4>] bad_gs+0xd1d/0x1ba9
>>>
>>> *ow*
>>> Could you dump your vmlinux (and System.map) somewhere on anonftp?
>>> This 'bad_gs' is there simply because it's one of the few labels in
>>> .fixup - to say anything useful we'll need to find out where we'd
>>> really come from.
>>
>> I see.
>> vmlinux with debug symbols: http://knox.linuxhacker.ru/tmp/dcache/vmlinux.gz
>> System.map: http://knox.linuxhacker.ru/tmp/dcache/System.map.gz
>
> OK...
> ffffffff817f87cd: 48 8d 0a lea (%rdx),%rcx
> ffffffff817f87d0: 48 83 e1 f8 and $0xfffffffffffffff8,%rcx
> ffffffff817f87d4: 4c 8b 01 mov (%rcx),%r8
> ffffffff817f87d7: 8d 0a lea (%rdx),%ecx
> ffffffff817f87d9: 83 e1 07 and $0x7,%ecx
> ffffffff817f87dc: c1 e1 03 shl $0x3,%ecx
> ffffffff817f87df: 49 d3 e8 shr %cl,%r8
> ffffffff817f87e2: e9 9b b3 a4 ff jmpq ffffffff81243b82 <__d_lookup+0x132>
>
> Aha... It's load_unaligned_zeropad() from dentry_string_cmp(), hitting
> a genuinely unmapped address. That sends it into fixup, where it tries to
> load an aligned word containing the address in question, in hope that
> fault was on attempt to cross into the next page. No such luck, address
> was aligned in the first place (it's in %rdx - 0xffff880113f82000), so
> we still oops.
>
> The unexpected part is that unmapped address did *NOT* come from a dentry;
> it's .name of qstr we were looking for. And your call chain was
> __d_lookup() <- d_lookup() <- lookup_open(), so in lookup_open() it was
> nd->last.name...
>
> Can the same thing be reproduced (with NFS fix) on v4.6, ede4090, 7f427d3,
> 4e8440b?

Well, that was faster than I expected. 4e8440b triggers right away, so I guess
there's no point in trying the later ones?
BTW, just to confirm you are noticing - this is a DEBUG_PAGEALLOC build,
so all freed memory is unmapped which is likely causing this oops - as a sign
of use after free.

[ 54.990119] BUG: unable to handle kernel paging request at ffff8800d2b7f000
[ 54.990423] IP: [<ffffffff817f91b6>] bad_gs+0xcff/0x1b89
[ 54.990598] PGD 2dca067 PUD 11f900067 PMD 11f86a067 PTE 80000000d2b7f060
[ 54.990942] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 54.991044] Modules linked in: loop rpcsec_gss_krb5 joydev pcspkr i2c_piix4 acpi_cpufreq tpm_tis tpm nfsd drm_kms_helper ttm drm serio_raw virtio_blk
[ 54.992301] CPU: 7 PID: 5550 Comm: file_concat.sh Not tainted 4.6.0-4e8440b+ #1
[ 54.993019] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 54.993407] task: ffff8800d6ec6200 ti: ffff880110f44000 task.ti: ffff880110f44000
[ 54.994112] RIP: 0010:[<ffffffff817f91b6>] [<ffffffff817f91b6>] bad_gs+0xcff/0x1b89
[ 54.994870] RSP: 0018:ffff880110f47ba0 EFLAGS: 00010286
[ 54.995288] RAX: ffff8800d2760f30 RBX: ffff8800d2760f00 RCX: ffff8800d2b7f000
[ 54.995736] RDX: ffff8800d2b7f000 RSI: 0000000000000000 RDI: 0000000000000001
[ 54.996154] RBP: ffff880110f47be8 R08: 0000000000000000 R09: ffff8800d2760ed0
[ 54.996628] R10: 0000000000000059 R11: ffff8800d6ec6d78 R12: ffff8800d2694ed0
[ 54.997044] R13: ffff880110f47df0 R14: ffff8800d2760f50 R15: 00000000b761b37e
[ 54.997450] FS: 00007fe8804d2700(0000) GS:ffff88011f5c0000(0000) knlGS:0000000000000000
[ 54.998164] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 54.998624] CR2: ffff8800d2b7f000 CR3: 000000011931f000 CR4: 00000000000006e0
[ 54.999225] Stack:
[ 54.999596] ffffffff81243965 0000000000000096 ffff8800d2b7f000 ffffffff00000001
[ 55.000632] 000000000000042c ffff880110f47df0 ffff8800d2694ed0 ffff880110f47de0
[ 55.001497] ffff880110f47df0 ffff880110f47c18 ffffffff81243b94 ffffffff81234b3e
[ 55.002353] Call Trace:
[ 55.002709] [<ffffffff81243965>] ? __d_lookup+0x5/0x1b0
[ 55.003104] [<ffffffff81243b94>] d_lookup+0x84/0xb0
[ 55.003487] [<ffffffff81234b3e>] ? lookup_open+0xfe/0x7a0
[ 55.003878] [<ffffffff81234b3e>] lookup_open+0xfe/0x7a0
[ 55.004272] [<ffffffff8123717e>] path_openat+0x4de/0xfc0
[ 55.004665] [<ffffffff8123925e>] do_filp_open+0x7e/0xe0
[ 55.005061] [<ffffffff812330a0>] ? lock_rename+0x100/0x100
[ 55.005452] [<ffffffff817f5367>] ? _raw_spin_unlock+0x27/0x40
[ 55.005846] [<ffffffff8124868c>] ? __alloc_fd+0xbc/0x170
[ 55.006246] [<ffffffff81226896>] do_sys_open+0x116/0x1f0
[ 55.006635] [<ffffffff8122698e>] SyS_open+0x1e/0x20
[ 55.007036] [<ffffffff817f5b36>] entry_SYSCALL_64_fastpath+0x1e/0xad
[ 55.007434] Code: e1 03 49 d3 e8 e9 8f a3 a4 ff 48 8d 0a 48 83 e1 f8 4c 8b 01 8d 0a 83 e1 07 c1 e1 03 49 d3 e8 e9 4e a4 a4 ff 48 8d 0a 48 83 e1 f8 <4c> 8b 01 8d 0a 83 e1 07 c1 e1 03 49 d3 e8 e9 c9 a8 a4 ff b9 f2
[ 55.011146] RIP [<ffffffff817f91b6>] bad_gs+0xcff/0x1b89
[ 55.011584] RSP <ffff880110f47ba0>
[ 55.011943] CR2: ffff8800d2b7f000