Re: "virt_to_phys used for non-linear address" warnings

[Greg Ungerer <gerg@kernel.org>]

Hi Serge,

On 6/7/22 03:42, Serge Semin wrote:
> !To += Thomas
> !Cc += Jiaxun
> 
> Hi Greg,
> 
> On Fri, Jul 01, 2022 at 05:24:22PM +1000, Greg Ungerer wrote:
>> Hi,
>>
>> I am debugging a strange memory problem on a Mediatek MT7621 SoC based
>> hardware platform. That problem leads to rare and somewhat random
>> oops that are mostly within vma and generic memory functions (often in
>> kmem_cache_alloc(), but sometimes other places like unlink_anon_vmas() or
>> anon_vma_interval_tree_remove() or vma_interval_tree_remove()).
>>
>> To help track this down I enabled a few of the kernels hacking memory
>> debug config options. Now this immediately leads to getting this on
>> every process fork/exec:
>>
>>    ------------[ cut here ]------------
>>    WARNING: CPU: 0 PID: 1 at arch/mips/mm/physaddr.c:38 __virt_to_phys+0x50/0x84
>>    virt_to_phys used for non-linear address: c443e370 (0xbfbd0000)
>>    Modules linked in:
>>    CPU: 0 PID: 1 Comm: init Not tainted 5.17.0-ac0 #1
>>    Stack : 81c70000 7ffbd000 77de5000 81086784 00000000 00000004 00000000 d95b60c8
>>            80441c84 81c43654 81b70000 81b60000 804583d8 00000001 80441c28 8045cd00
>>            00000000 00000000 81aaf3b4 80441a70 00000187 80441adc 00000000 20306361
>>            203a6d6d 81c4d2f5 81c4d31c 74696e69 81b60000 00000001 80441d3c 81b6dbf0
>>            82070040 8066f240 81c70000 7ffbd000 00000000 814ff0f4 00000000 81c40000
>>            ...
>>    Call Trace:
>>    [<81008ed0>] show_stack+0x38/0x118
>>    [<8198f9e4>] dump_stack_lvl+0x5c/0x7c
>>    [<81989300>] __warn+0xc0/0xf4
>>    [<819893c0>] warn_slowpath_fmt+0x8c/0xb8
>>    [<81025480>] __virt_to_phys+0x50/0x84
>>    [<8100bb30>] arch_setup_additional_pages+0x120/0x230
>>    [<81239280>] load_elf_binary+0xacc/0x14e0
>>    [<811d6788>] bprm_execve+0x288/0x5dc
>>    [<811d7240>] kernel_execve+0x130/0x1b4
>>    [<81988154>] try_to_run_init_process+0x14/0x4c
>>    [<81995e40>] kernel_init+0xe4/0x118
>>    [<81003398>] ret_from_kernel_thread+0x14/0x1c
>>    ---[ end trace 0000000000000000 ]---
>>
>> This is caused by this code in arch/mips/kernel/vdso.c, function
>> arch_setup_additional_pages():
>>
>>          /* Map GIC user page. */
>>          if (gic_size) {
>>                  gic_base = (unsigned long)mips_gic_base + MIPS_GIC_USER_OFS;
>>                  gic_pfn = virt_to_phys((void *)gic_base) >> PAGE_SHIFT;
>>
>>                  ret = io_remap_pfn_range(vma, base, gic_pfn, gic_size,
>>                                           pgprot_noncached(vma->vm_page_prot));
>>                  if (ret)
>>                          goto out;
>>          }
>>
> 
>> That virt_to_phys() is being passed a value assigned from an ioremap().
>> According to the comments in io.h, this is not a correct usage of it:
>>
>>   *     The returned physical address is the physical (CPU) mapping for
>>   *     the memory address given. It is only valid to use this function on
>>   *     addresses directly mapped or allocated via kmalloc.
> 
> Physical address you are trying to retrieve is directly mapped. It's
> 0xbfbd0000, which belong to the UCAC kseg1 MIPS space:
> https://johnloomis.org/microchip/pic32/memory/memory.html
> So virt_to_phys() shall work for it with no problem.

Yeah, I can see that in this case it always ended up with the same pfn.


> Anyway IIUC from the __debug_virt_addr_valid() implementation you've
> got the "high_memory" variable initialized with inaccurate value. At
> very least it causes your directly mapped IO-address to cause the
> warning printed. In some circumstance it may lead to more complex
> problems. I've got a patch created some time ago, which fixes that
> misconfiguration. Could you try it out and see whether it solves your
> problems?

Tried the patch. Does not fix the issue. Still get the warning and dump
on every process startup.


> Could you also send out a log with "Zone ranges:" info retrieved on
> the kernel without my patch applied?

Before patch (original code):

     Zone ranges:
       Normal   [mem 0x0000000000000000-0x000000000fffffff]
     Movable zone start for each node
     Early memory node ranges
       node   0: [mem 0x0000000000000000-0x000000000fffffff]
     Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]

Regards
Greg



> -Sergey
> 
>>
>> So the debug warning is consistent with this comment.
>>
>> It is trivial to fix this by using __pa() directly here instead of
>> virt_to_phys(). So simply doing this fixes it:
>>
>> --- a/arch/mips/kernel/vdso.c
>> +++ b/arch/mips/kernel/vdso.c
>> @@ -159,7 +159,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
>>          /* Map GIC user page. */
>>          if (gic_size) {
>>                  gic_base = (unsigned long)mips_gic_base + MIPS_GIC_USER_OFS;
>> -               gic_pfn = virt_to_phys((void *)gic_base) >> PAGE_SHIFT;
>> +               gic_pfn = __pa(gic_base) >> PAGE_SHIFT;
>>                  ret = io_remap_pfn_range(vma, base, gic_pfn, gic_size,
>>                                           pgprot_noncached(vma->vm_page_prot));
>>
>> I am wondering if that is the right approach or if there is a more correct way?
>>
>> Regards
>> Greg
>>