Re: BUG_ON() in pfn_swap_entry_to_page()

David Hildenbrand
2024-04-26T08:49:11Z

On 25.04.24 16:33, Felix Kuehling wrote:
>
>
> On 2024-04-25 5:32, David Hildenbrand wrote:
>> On 24.04.24 21:45, Felix Kuehling wrote:
>>> Sorry for top-posting. I'm resurrecting an old thread here because I
>>> think I ran into the same problem with this assertion failing on Linux
>>> 6.7:
>>>
>>> static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry)
>>> {
>>>          struct page *p = pfn_to_page(swp_offset_pfn(entry));
>>>
>>>          /*
>>>           * Any use of migration entries may only occur while the
>>>           * corresponding page is locked
>>>           */
>>> -->     BUG_ON(is_migration_entry(entry) && !PageLocked(p));
>>>
>>>          return p;
>>> }
>>>
>>> It looks like this thread just fizzled two years ago. Did anything
>>> ever come of this?
>>>
>>> Maybe I should add that I saw this in a pre-silicon test environment.
>>> I've never seen this on real hardware. Maybe something timing-sensitive.
>>
>> In the past, it indicated a swp pte corruption, that would e.g., mess up
>> the stored PFN ot the swap entry type.
>>
>> On which call chain do you see that?
>>
>
> This is the backtrace, it's coming from hmm_range_fault. Looks like the
> swap entries are from migrated DEVICE_PRIVATE pages.

Thanks, on which kernel version can you reproduce this?

>
> [Apr 3 20:11] ------------[ cut here ]------------
> [ +0.000041] kernel BUG at include/linux/swapops.h:466!
> [ +0.000691] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> [ +0.000342] CPU: 2 PID: 49 Comm: kworker/2:1 Not tainted
> 6.7.0-kfd-compute-rocm-npi-186 #1
> [ +0.000556] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
> rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
> [ +0.000703] Workqueue: events amdgpu_irq_handle_ih_soft [amdgpu]
> [ +0.000501] RIP: 0010:migration_entry_wait_on_locked+0x26b/0x2b0
> [ +0.000389] Code: fe ff ff 48 8d 7c 24 07 e8 02 7e f0 ff e9 58 fe ff
> ff 48 8b 43 08 a8 01 75 3f 66 90 48 89 d8 48 8b 00 a8 01 0f 85 f1 fd ff
> ff <0f> 0b 48 8d 58 ff e9 f7 fd ff ff 48 89 d8 f7 c3 ff 0f 00 00 75 df
> [ +0.001161] RSP: 0018:ffffb211c01bb788 EFLAGS: 00010246
> [ +0.000339] RAX: 017fff8000080018 RBX: fffff682c40ce8c0 RCX:
> 0000000000000001
> [ +0.000463] RDX: 0000000000000000 RSI: ffff977a45034840 RDI:
> 000000000000001a
> [ +0.000454] RBP: ffff977a45034840 R08: 68000000001033a3 R09:
> 0000000000000030
> [ +0.000451] R10: ffffb211c01bb6a8 R11: 0000000000000001 R12:
> ffff977a46bd1318
> [ +0.000461] R13: 0000000000000003 R14: 4000000000000000 R15:
> ffffb211c01bb9b8
> [ +0.000454] FS: 0000000000000000(0000) GS:ffff977dafd00000(0000)
> knlGS:0000000000000000
> [ +0.000518] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ +0.000372] CR2: 00007fa2d1cba000 CR3: 00000001030d2004 CR4:
> 0000000000770ef0
> [ +0.000453] PKRU: 55555554
> [ +0.000182] Call Trace:
> [ +0.000171] <TASK>
> [ +0.000147] ? die+0x37/0x90
> [ +0.000211] ? do_trap+0xe0/0x110
> [ +0.000221] ? migration_entry_wait_on_locked+0x26b/0x2b0
> [ +0.000351] ? do_error_trap+0x98/0x120
> [ +0.000252] ? migration_entry_wait_on_locked+0x26b/0x2b0
> [ +0.000346] ? migration_entry_wait_on_locked+0x26b/0x2b0
> [ +0.000355] ? exc_invalid_op+0x52/0x70
> [ +0.000254] ? migration_entry_wait_on_locked+0x26b/0x2b0
> [ +0.000345] ? asm_exc_invalid_op+0x1a/0x20
> [ +0.000274] ? migration_entry_wait_on_locked+0x26b/0x2b0
> [ +0.000361] ? migration_entry_wait+0x4e/0x160
> [ +0.000293] ? lock_release+0x119/0x260
> [ +0.000255] migration_entry_wait+0x105/0x160
> [ +0.000290] hmm_vma_walk_pmd+0x822/0x8a0
> [ +0.000263] walk_pgd_range+0x40b/0x900
> [ +0.000268] __walk_page_range+0x205/0x220

I wonder if that is coming from pmd_migration_entry_wait() or
migration_entry_wait() -- the "?" above adds uncertainty :)

Likely it's from migration_entry_wait().

I was first concerned about the lack of PTL in this function, but
migration_entry_wait() will take the PTL and re-read the PTE.

So when we call into migration_entry_wait_on_locked(), we are holding
the PTL and we verified that we indeed have a migration entry.

So if we fail in
migration_entry_wait_on_locked()->pfn_swap_entry_folio(), we verified
under PTL and still have a migration entry.

The referenced folio is indeed not locked then.

--
Cheers,

David / dhildenb