Re: Problem with gntdev : Bad page map

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, Mar 26, 2013 at 05:32:00PM +0100, Sébastien Frémal wrote:
> Hello,
> 
> I'm using xen 4.1.2 and I try to shared memory pages between the dom0 and
> domU's. For that purpose, I looked at gntalloc and gntdev. To test these
> modules, I created two programs :

You are using an older kernel - and there wer some updates in the gntalloc/gntdev
driver. Do you see this when you use an updated kernel?
> 
>    - gntalloc_test : a program running in dom0 and granting some pages to a
>    domU and putting random values in it.
>    - gntdev_test : a program running in the domU, mapping the granted pages
>    and reading the values contained in it.
> 
> I succeeded in transferring values from dom0 to domU, but I have a Bad page
> map error when the program in domU exits :
> 
> [  123.903755] BUG: Bad page map in process test  pte:80000000107ff167
> pmd:0c7e3067
> [  123.903765] page:ffffea000041ffc0 count:0 mapcount:-1 mapping:
>  (null) index:0xffff88000e56b500
> [  123.903899] page flags:
> 0x100000000000c14(referenced|dirty|reserved|private)
> [  123.903910] addr:00007f9845c3d000 vm_flags:000e00fb anon_vma:
>  (null) mapping:ffff88000d637668 index:0
> [  123.903920] vma->vm_ops->fault: 0x0
> [  123.903924] vma->vm_file->f_op->mmap: gntdev_mmap+0x0/0x290 [xen_gntdev]
> 
> If I run several times gntdev_test, the count and the mapcount keeps
> decreasing :
> 
> [  125.144693] page:ffffea000041ffc0 count:-1 mapcount:-2 mapping:
>  (null) index:0xffff88000e56b720
> [  126.072717] page:ffffea000041ffc0 count:-2 mapcount:-3 mapping:
>  (null) index:0xffff88000e56b7c0
> 
> The error message coming from dmesg is :
> 
> [  126.072703] BUG: Bad page map in process test  pte:80000000107ff167
> pmd:0e6a2067
> [  126.072717] page:ffffea000041ffc0 count:-2 mapcount:-3 mapping:
>  (null) index:0xffff88000e56b7c0
> [  126.072724] page flags:
> 0x100000000000c14(referenced|dirty|reserved|private)
> [  126.072739] addr:00007ff931aec000 vm_flags:000e00fb anon_vma:
>  (null) mapping:ffff88000d637668 index:0
> [  126.072749] vma->vm_ops->fault: 0x0
> [  126.072754] vma->vm_file->f_op->mmap: gntdev_mmap+0x0/0x290 [xen_gntdev]
> [  126.072761] Pid: 1268, comm: test Tainted: G    B   W
>  3.2.0-34-generic #53-Ubuntu
> [  126.072764] Call Trace:
> [  126.072769]  [<ffffffff81139c99>] print_bad_pte+0x1d9/0x270
> [  126.072772]  [<ffffffff8113c799>] zap_pte_range+0x369/0x3d0
> [  126.072775]  [<ffffffff81005339>] ?
> __raw_callee_save_xen_pmd_val+0x11/0x1e
> [  126.072779]  [<ffffffff8113c9aa>] unmap_page_range+0x1aa/0x300
> [  126.072782]  [<ffffffff8113d08a>] unmap_vmas+0xca/0x1a0
> [  126.072785]  [<ffffffff81144607>] exit_mmap+0x97/0x140
> [  126.072788]  [<ffffffff8113dde8>] ? handle_mm_fault+0x1f8/0x350
> [  126.072792]  [<ffffffff8165bfde>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
> [  126.072795]  [<ffffffff81064b92>] mmput.part.16+0x42/0x130
> [  126.072798]  [<ffffffff81064ca9>] mmput+0x29/0x30
> [  126.072802]  [<ffffffff8106b603>] exit_mm+0x113/0x130
> [  126.072806]  [<ffffffff810e41c5>] ? taskstats_exit+0x45/0x240
> [  126.072809]  [<ffffffff8165c0b5>] ? _raw_spin_lock_irq+0x15/0x20
> [  126.072812]  [<ffffffff8106b78e>] do_exit+0x16e/0x450
> [  126.072817]  [<ffffffff811783e0>] ? vfs_write+0x110/0x180
> [  126.072820]  [<ffffffff8106bc14>] do_group_exit+0x44/0xa0
> [  126.072823]  [<ffffffff8106bc87>] sys_exit_group+0x17/0x20
> [  126.072826]  [<ffffffff816643c2>] system_call_fastpath+0x16/0x1b
> 
> Here are the codes of the two programs :
> 
> gntalloc_test.c :
> 
> #include <stdio.h>
> #include <stdint.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <sys/mman.h>
> #include <fcntl.h>
> #include <xen/gntalloc.h>
> #include <linux/ioctl.h>
> 
> int main(){
>         int fd = open("/dev/xen/gntalloc", O_RDWR);
>         if(fd == -1){
>                 perror("Open : ");
>                 return -1;
>         }
>         struct ioctl_gntalloc_alloc_gref arg;
>         arg.domid = 5;
>         arg.flags = GNTALLOC_FLAG_WRITABLE;
>         arg.count = 4;
>         ioctl(fd, IOCTL_GNTALLOC_ALLOC_GREF, &arg);
>         int i;
>         for(i=0; i<4;++i)
>             printf("Gref : %d\n", arg.gref_ids[i]);
>         struct ioctl_gntalloc_dealloc_gref dearg;
>         dearg.index = arg.index;
>         dearg.count = arg.count;
>         int * vadr = mmap(0, getpagesize(), PROT_READ | PROT_WRITE,
> MAP_SHARED, fd, 0);
>         vadr[0]=1337;
>         vadr[1]=20057;
>         vadr[2]=9000;
>         int wait;
>         printf("vadr : %d %d %d\nEn attente ...\n", vadr[0], vadr[1],
> vadr[2]);
>         scanf("%d", &wait);
>         ioctl(fd, IOCTL_GNTALLOC_DEALLOC_GREF, &dearg);
> }
> 
> gntdev_test.c :
> 
> #include <stdio.h>
> #include <stdint.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <sys/mman.h>
> #include <fcntl.h>
> #include <xen/gntdev.h>
> #include <linux/ioctl.h>
> 
> int main(){
>         int fd;
>         if((fd = open("/dev/xen/gntdev2", O_RDWR))==-1){
>                 perror("open");
>                 return;
>         }
>         struct ioctl_gntdev_map_grant_ref arg;
>         struct ioctl_gntdev_unmap_grant_ref dearg[2];
>         arg.count = 1;
>         arg.refs[0].ref=32;
>         arg.refs[0].domid=0;
>         ioctl(fd, IOCTL_GNTDEV_MAP_GRANT_REF, &arg);
>         dearg[0].index = arg.index;
>         dearg[0].count = arg.count;
>         arg.refs[0].ref=33;
>         ioctl(fd, IOCTL_GNTDEV_MAP_GRANT_REF, &arg);
>         printf("Index in the file : %ld\n", arg.index);
>         int * vaddr;
>         if((vaddr = mmap(0, getpagesize(), PROT_READ | PROT_WRITE,
> MAP_SHARED, f
> d, 0))==NULL)
>                 return;
>         printf("Vaddr : %d %d %d\n", vaddr[0], vaddr[1], vaddr[2]);
> 
>         dearg[1].index = arg.index;
>         dearg[1].count = arg.count;
>         printf("First grant\n");
>         ioctl(fd, IOCTL_GNTDEV_UNMAP_GRANT_REF, &dearg[0]);
>         printf("Second grant\n");
>         ioctl(fd, IOCTL_GNTDEV_UNMAP_GRANT_REF, &dearg[1]);
>         printf("The end\n");
> }
> 
> I tried to track where the error comes from. When I execute gntdev_test,
> the error message is displayed after the final message (The end) ; if I
> look dmesg I can see than :
> - The first page is "unmapped"
> - The second page is "unmapped"
> - The second page is freed
> - The error happens
> - gntdev_vma_close is called
> - The first page is freed
> I used a modified version of the module to track these steps, but the
> original module
> (/lib/modules/3.2.0-34-generic/kernel/drivers/xen/xen-gntdev.ko) gives the
> same error.
> 
> I suppose that something is missing in gntdev_test.c to manage the mapping
> of pages but I don't know what. I tried to call the unmap function, but
> it's worst as it adds other errors.
> 
> Does someone know what's missing please ?
> 
> Best regards,
> 
> Frémal Sébastien

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel