Kernel Oops when reading kernel_page_tables debugfs file

Kernel Oops when reading kernel_page_tables debugfs file

[Vivien Bernet-Rollande]

[-- Attachment #1.1: Type: text/plain, Size: 5574 bytes --]

Hi list.
I'm  currently trying to get a device driver to work on Xen dom0. The driver
maps PCI space to userland, and for some reason I have yet to figure, it
doesn't work.

Trying to debug this, I came  across the kernel_page_tables debugfs file,
that basically displays the content of the pagetable. It doesn't allow me to
read the page entries for a process, but I was thinking of hacking it up a
bit. For now, on a non Xen system, it just shows :
---[ User Space ]---
0x0000000000000000-0xffff800000000000 16777088T pgd
---[ Kernel Space ]---
 ...
 ...

The thing is, it doesn't work on dom0 :

[root@x-dev-4 ~]# mount -t debugfs none /sys/kernel/debug/
mount: none already mounted or /sys/kernel/debug/ busy
mount: according to mtab, none is already mounted on /sys/kernel/debug
[root@x-dev-4 ~]# cat /sys/kernel/debug/kernel_page_tables

Message from syslogd@x-dev-4 at Nov 24 19:02:55 ...
 kernel:[  210.164008] Oops: 0000 [#2] SMP DEBUG_PAGEALLOC

Message from syslogd@x-dev-4 at Nov 24 19:02:55 ...
 kernel:[  210.164008] last sysfs file: /sys/kernel/mm/ksm/run

Message from syslogd@x-dev-4 at Nov 24 19:02:55 ...
 kernel:[  210.164008] Stack:

Message from syslogd@x-dev-4 at Nov 24 19:02:55 ...
 kernel:[  210.164008] Call Trace:

Message from syslogd@x-dev-4 at Nov 24 19:02:55 ...
 kernel:[  210.164008] Code: 0f 00 00 00 88 ff ff 48 8d 14 10 4e 8d 24 20 48
8b 45 98 48 89 55 80 48 89 45 88 48 8b 45 88 48 c1 e0 10 48 c1 f8 10 48 89
45 b8 <49> 8b 3c 24 48 85 ff 0f 84 aa 01 00 00 ff 14 25 70 9c 6a 81 48

Message from syslogd@x-dev-4 at Nov 24 19:02:55 ...
 kernel:[  210.164008] CR2: ffff9d5555555000
Killed


[root@x-dev-4 ~]# dmesg
[  210.163806] BUG: unable to handle kernel paging request at
ffff9d5555555000
[  210.163940] IP: [<ffffffff8103cc2c>] ptdump_show+0xcb/0x30f
[  210.164008] PGD 0
[  210.164008] Oops: 0000 [#2] SMP DEBUG_PAGEALLOC
[  210.164008] last sysfs file: /sys/kernel/mm/ksm/run
[  210.164008] CPU 0
[  210.164008] Modules linked in: bridge stp llc sunrpc rdma_ucm rdma_cm
iw_cm ib_addr ib_ipoib ib_cm ib_sa ipv6 ib_uverbs ib_umad iw_nes libcrc32c
iw_cxgb3 cxgb3 mlx4_en mlx4_ib mlx4_core xen_netback xen_blkback
blkback_pagemap xen_gntdev xen_evtchn xenfs ib_mthca snd_hda_codec_realtek
snd_hda_intel snd_hda_codec snd_hwdep snd_seq ib_mad snd_seq_device snd_pcm
hp_wmi ib_core snd_timer rfkill tg3 snd edac_core ppdev k8temp soundcore
parport_pc edac_mce_amd i2c_piix4 snd_page_alloc wmi parport shpchp
serio_raw xfs exportfs pata_acpi ata_generic dm_multipath pata_atiixp radeon
ttm drm_kms_helper drm i2c_algo_bit i2c_core [last unloaded: scsi_wait_scan]
[  210.164008] Pid: 1729, comm: cat Tainted: G      D W
2.6.32.25-172.xendom0.debug.fc12.x86_64 #1 HP Compaq dc5750 Microtower
[  210.164008] RIP: e030:[<ffffffff8103cc2c>]  [<ffffffff8103cc2c>]
ptdump_show+0xcb/0x30f
[  210.164008] RSP: e02b:ffff88005083bdd8  EFLAGS: 00010296
[  210.164008] RAX: ffff800000000000 RBX: ffff88005050ca80 RCX:
0000000000000000
[  210.164008] RDX: ffff9d5555555ff8 RSI: 0000000000000000 RDI:
0000000077db4067
[  210.164008] RBP: ffff88005083be78 R08: 0000000000000006 R09:
0000000000000073
[  210.164008] R10: 0000000000008fff R11: 0000000000000246 R12:
ffff9d5555555000
[  210.164008] R13: ffff88005083bf58 R14: ffff88004fa3c9c0 R15:
ffffffff81001800
[  210.164008] FS:  00007effbc512700(0000) GS:ffff880005d45000(0000)
knlGS:0000000000000000
[  210.164008] CS:  e033 DS: 0000 ES: 0000 CR0: 000000008005003b
[  210.164008] CR2: ffff9d5555555000 CR3: 000000004f89e000 CR4:
0000000000000660
[  210.164008] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  210.164008] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  210.164008] Process cat (pid: 1729, threadinfo ffff88005083a000, task
ffff88005044aea0)
[  210.164008] Stack:
[  210.164008]  ffffffff8100f001 ffff88005083be78 ffffffff8100f742
0000000000000246
[  210.164008] <0> ffff9d5555555ff8 0000800000000000 ffff880005d57040
0000800000000000
[  210.164008] <0> 0000000000000001 0000000000000000 0000000000000000
ffff800000000000
[  210.164008] Call Trace:
[  210.164008]  [<ffffffff8100f001>] ? xen_force_evtchn_callback+0xd/0xf
[  210.164008]  [<ffffffff8100f742>] ? check_events+0x12/0x20
[  210.164008]  [<ffffffff81139d32>] ? seq_read+0x93/0x354
[  210.164008]  [<ffffffff81139e25>] seq_read+0x186/0x354
[  210.164008]  [<ffffffff8111ed4c>] vfs_read+0xab/0x108
[  210.164008]  [<ffffffff8111ee69>] sys_read+0x4a/0x6e
[  210.164008]  [<ffffffff81012d32>] system_call_fastpath+0x16/0x1b
[  210.164008] Code: 0f 00 00 00 88 ff ff 48 8d 14 10 4e 8d 24 20 48 8b 45
98 48 89 55 80 48 89 45 88 48 8b 45 88 48 c1 e0 10 48 c1 f8 10 48 89 45 b8
<49> 8b 3c 24 48 85 ff 0f 84 aa 01 00 00 ff 14 25 70 9c 6a 81 48
[  210.164008] RIP  [<ffffffff8103cc2c>] ptdump_show+0xcb/0x30f
[  210.164008]  RSP <ffff88005083bdd8>
[  210.164008] CR2: ffff9d5555555000
[  210.164008] ---[ end trace a7919e7f17c0a729 ]---



How to reproduce :
 - compile kernel with CONFIG_X86_PTDUMP=y
 # mount -t debugfs none /sys/kernel/debug/
 # cat /sys/kernel/debug/kernel_page_tables

This is possibly caused by the driver I'm trying to fix (ib_mthca, see
http://lists.xensource.com/archives/html/xen-devel/2010-11/msg00462.html for
symptoms). However, since the memory management is quite different between
Xen and bare x86_64, I figured the file might just not work on Xen.

So, basically, it comes down to this question : is this feature actually
supposed to work in Xen ?

By the way, kernel is 2.6.32 with pvops patch, Xen is 4.0.1.

Thanks for any input you may have.

-- 
Vivien

[-- Attachment #1.2: Type: text/html, Size: 6158 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Kernel Oops when reading kernel_page_tables debugfs file

[Jan Beulich]

>>> On 24.11.10 at 17:07, Vivien Bernet-Rollande <vbernetr@gmail.com> wrote:
> So, basically, it comes down to this question : is this feature actually
> supposed to work in Xen ?

I think it is supposed to work, but for it to actually do so it is pretty
clear that the Xen region must be excluded from getting its page
tables dumped.

Jan

Re: Kernel Oops when reading kernel_page_tables debugfs file

[Konrad Rzeszutek Wilk]

On Wed, Nov 24, 2010 at 05:07:49PM +0100, Vivien Bernet-Rollande wrote:
> Hi list.
> I'm  currently trying to get a device driver to work on Xen dom0. The driver
> maps PCI space to userland, and for some reason I have yet to figure, it
> doesn't work.

Did you set VM_IO on your mapping?

Re: Kernel Oops when reading kernel_page_tables debugfs file

[Vivien Bernet-Rollande]

On Mon, Nov 29, 2010 at 4:20 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Wed, Nov 24, 2010 at 05:07:49PM +0100, Vivien Bernet-Rollande wrote:
>> Hi list.
>> I'm  currently trying to get a device driver to work on Xen dom0. The driver
>> maps PCI space to userland, and for some reason I have yet to figure, it
>> doesn't work.
>
> Did you set VM_IO on your mapping?
>

The driver doesn't set the VM_IO flag itself. However, it calls
io_remap_pfn_range(), which is a macro wrapper around
remap_pfn_range(). The later does :
vma->vm_flags |= VM_IO | VM_RESERVED | VM_PFNMAP;

So the VM_IO flag is set. I actually corrected the bug by adding the
_PAGE_IOMAP flag in the vma->vm_pgprot.

Shouldn't remap_pfn_range() set that flag if VM_IO is set ?

Re: Kernel Oops when reading kernel_page_tables debugfs file

[Konrad Rzeszutek Wilk]

On Tue, Nov 30, 2010 at 10:34:07AM +0100, Vivien Bernet-Rollande wrote:
> On Mon, Nov 29, 2010 at 4:20 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > On Wed, Nov 24, 2010 at 05:07:49PM +0100, Vivien Bernet-Rollande wrote:
> >> Hi list.
> >> I'm  currently trying to get a device driver to work on Xen dom0. The driver
> >> maps PCI space to userland, and for some reason I have yet to figure, it
> >> doesn't work.
> >
> > Did you set VM_IO on your mapping?
> >
> 
> The driver doesn't set the VM_IO flag itself. However, it calls
> io_remap_pfn_range(), which is a macro wrapper around
> remap_pfn_range(). The later does :
> vma->vm_flags |= VM_IO | VM_RESERVED | VM_PFNMAP;
> 
> So the VM_IO flag is set. I actually corrected the bug by adding the
> _PAGE_IOMAP flag in the vma->vm_pgprot.

Ah, there is a macro for that: vm_get_page_prot, as so:

vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);

And under Xen the vm_get_page_prot will set _PAGE_IOMAP if it detects VM_IO flag.
> 
> Shouldn't remap_pfn_range() set that flag if VM_IO is set ?

Nope. You need to use the above mentioned macro to do it.