From mboxrd@z Thu Jan 1 00:00:00 1970 From: Shriram Rajagopalan Subject: Re: blktap2 and CONFIG_XEN_BLKBACK_PAGEMAP Date: Thu, 15 Jul 2010 15:06:22 -0700 Message-ID: References: <2E49F371-C04A-4DE2-9945-66ABADD0042C@rice.edu> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1482587546==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Kaushik Kumar Ram Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org --===============1482587546== Content-Type: multipart/alternative; boundary=000e0cd28d3816b56f048b745028 --000e0cd28d3816b56f048b745028 Content-Type: text/plain; charset=ISO-8859-1 On Thu, Jul 15, 2010 at 12:02 PM, Kaushik Kumar Ram wrote: > > On Jul 15, 2010, at 1:19 PM, Shriram Rajagopalan wrote: > > > IIRC during my early experiments with blkback & blktap2, I hit a similar > error.. tracing through the code, I gathered that the pagemap stuff is used > to manage page grants to blktap2 kernel driver . So, the #else (ie > !BLKBK_PAGEMAP) code is not going to work. > > I suggest, you try to look at the blkback_pagemap.c and the > blktap2/device.c or something like that to get a better picture. > > Thanks Shriram.0 I have been looking at the code over the past few days. > Since I am not familiar with the Linux block I/O layers its taking a lot of > time! > > It seems like on enabling CONFIG_BLKBACK_PAGEMAP the grant mechanism is > used to map guest pages into user space too. This means the guest pages are > mapped twice using the grant mechanism, first into dom0 kernel space (in > blkback/blback.c) and then into tapdisk process's address space (in > blktap2/device.c). This is the new implementation of blkback. > > yep.. > On disabling CONFIG_BLKBACK_PAGEMAP, the code falls back on the old > implementation. Here, the guest pages are mapped into user space by directly > manipulating the page tables without going through the grant mechanism. > (Things seem slightly different when XENFEAT_auto_translated_physmap is set > but I will ignore that for now). IIRC, that XENFEAT_auto_translated_physmap is kinda deprecated.. it was used in xen 3.1 or so I guess.. (basically, it makes pfn = mfn, instead of the current style : p2m & m2p tables) > First, does the old way still work? AFAIK, nope. I am not sure if some other config needs to be set to get that old code to work. It looks like dead code to me. I cannot figure out the "backward compatibility" angle to it either. > The problem seems to arise when the page table entry is set in > blktap_umap_uaddr_fn() (in blktap2/device.c) which leads to a page fault and > Xen does not seem to like this page fault to handle it correctly and this > results in a panic. Why is the page table entry set directly without using a > hypercall here? > > Any further explanation will be much appreciated. > > Thanks. > -Kaushik > > > On Wed, Jul 14, 2010 at 4:59 PM, Kaushik Kumar Ram > wrote: > > Is it necessary to use blkback_pagemap with blktap2? Since the use of > blkback_pagemap is configurable I tried without it and my system crashed > (crash dump attached below). Or is it a bug? > > > > I am using about a month old xen-unstable.hg with linux-2.6.18-xen.hg > (both 64 bit). > > > > Thanks. > > -Kaushik > > > > (XEN) mm.c:889:d0 Error getting mfn 80765 (pfn 3fba6) from L1 entry > 8000000080765027 for l1e_owner=0, pg_owner=0 > > (XEN) mm.c:5046:d0 ptwr_emulate: could not get_page_from_l1e() > > Unable to handle kernel paging request at ffff8800388f6688 RIP: > > [] blktap_map_uaddr_fn+0xa6/0xc0 > > PGD 1140067 PUD 1141067 PMD 1306067 PTE 80100000388f6065 > > Oops: 0003 [1] SMP > > CPU 0 > > Modules linked in: e1000e sd_mod ata_piix libata thermal fan > > Pid: 4183, comm: blkback.1.sda1 Not tainted 2.6.18.8-xen0 #40 > > RIP: e030:[] [] > blktap_map_uaddr_fn+0xa6/0xc0 > > RSP: e02b:ffff880039d01840 EFLAGS: 00010297 > > RAX: 8000000080765027 RBX: ffff8800388f6688 RCX: ffff880039d01908 > > RDX: 00002b218a8d1000 RSI: ffff880001fb15d0 RDI: ffff8800388f6688 > > RBP: ffff880039d01850 R08: 00000000000388f6 R09: 0000000000000000 > > R10: 0000000000000000 R11: 00000000000002c8 R12: ffff8800388f6688 > > R13: 00002b218a8d1000 R14: 00002b218a8d2000 R15: ffff88003890e2a0 > > FS: 00002af9674c06e0(0000) GS:ffffffff8058c000(0000) > knlGS:0000000000000000 > > CS: e033 DS: 0000 ES: 0000 > > Process blkback.1.sda1 (pid: 4183, threadinfo ffff880039d00000, task > ffff88003e8cf080) > > Stack: ffff8800388f6688 ffff880001fb15d0 ffff880039d018f0 > ffffffff80270033 > > 0000001a00000039 ffff880039d01908 ffffffff803dc730 ffff88003a714080 > > ffff8800389802b0 00002b218a8d2000 00002b218a8d2000 ffff88003c03b430 > > Call Trace: > > [] apply_to_page_range+0x4e3/0x590 > > [] blktap_map_uaddr_fn+0x0/0xc0 > > [] blktap_map_uaddr+0x21/0x30 > > [] blktap_device_do_request+0x67c/0xfe0 > > [] __mod_timer+0xbc/0xe0 > > [] __switch_to+0x370/0x5b0 > > [] lock_timer_base+0x2c/0x60 > > [] del_timer+0x56/0x70 > > [] __generic_unplug_device+0x25/0x30 > > [] generic_unplug_device+0x20/0x60 > > [] unplug_queue+0x26/0x50 > > [] blkif_schedule+0x55a/0x690 > > [] blkif_schedule+0x0/0x690 > > [] kthread+0xda/0x110 > > [] child_rip+0xa/0x12 > > [] kthread+0x0/0x110 > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > > > > > -- > > perception is but an offspring of its own self > > -- perception is but an offspring of its own self --000e0cd28d3816b56f048b745028 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Thu, Jul 15, 2010 at 12:02 PM, Kaushi= k Kumar Ram <kaushik@rice.edu> wrote:

On Jul 15, 2010, at 1:19 PM, Shriram Rajagopalan wrote:

> IIRC during my early experiments with blkback & blktap2, I hit a s= imilar error.. tracing through the code, I gathered that the pagemap stuff = is used to manage page grants to blktap2 kernel driver . So, the #else (ie = !BLKBK_PAGEMAP) code is not going to work.
> I suggest, you try to look at the blkback_pagemap.c and the blktap2/de= vice.c or something like that to get a better picture.

Thanks Shriram.0 I have been looking at the code over the past few da= ys. Since I am not familiar with the Linux block I/O layers its taking a lo= t of time!

It seems like on enabling CONFIG_BLKBACK_PAGEMAP the grant mechanism is use= d to map guest pages into user space too. This means the guest pages are ma= pped twice using the grant mechanism, first into dom0 kernel space (in blkb= ack/blback.c) and then into tapdisk process's address space (in blktap2= /device.c). =A0This is the new implementation of blkback.

yep..


=A0
On disabling CONFIG_BLKBACK_PAGEMAP, the code falls back on the old impleme= ntation. Here, the guest pages are mapped into user space by directly manip= ulating the page tables without going through the grant mechanism. (Things = seem slightly different when XENFEAT_auto_translated_physmap is set but I w= ill ignore that for now).
IIRC, that XENFEAT_auto_translated_physmap is kinda deprecated..=A0 it= was used in xen 3.1 or so=A0 I guess.. (basically, it makes pfn =3D mfn, i= nstead of the current style : p2m & m2p tables)
=A0
=A0
First, does the old way still work?
AFAIK, nope. I am not= sure if some other config needs to be set to get that old code to work. It= looks like dead code to me. I cannot figure out the "backward compati= bility" angle to it either.
The problem= seems to arise when the page table entry is set in blktap_umap_uaddr_fn() = (in blktap2/device.c) which leads to a page fault and Xen does not seem to = like this page fault to handle it correctly and this results in a panic. Wh= y is the page table entry set directly without using a hypercall here?

Any further explanation will be much appreciated.

Thanks.
-Kaushik

> On Wed, Jul 14, 2010 at 4:59 PM, Kaushik Kumar Ram <kaushik@rice.edu> wrote:
> Is it necessary to use blkback_pagemap with blktap2? Since the use of = blkback_pagemap is configurable I tried without it and my system crashed (c= rash dump attached below). Or is it a bug?
>
> I am using about a month old xen-unstable.hg with linux-2.6.18-xen.hg = (both 64 bit).
>
> Thanks.
> -Kaushik
>
> (XEN) mm.c:889:d0 Error getting mfn 80765 (pfn 3fba6) from L1 entry 80= 00000080765027 for l1e_owner=3D0, pg_owner=3D0
> (XEN) mm.c:5046:d0 ptwr_emulate: could not get_page_from_l1e()
> Unable to handle kernel paging request at ffff8800388f6688 RIP:
> =A0[<ffffffff803dc7d6>] blktap_map_uaddr_fn+0xa6/0xc0
> PGD 1140067 PUD 1141067 PMD 1306067 PTE 80100000388f6065
> Oops: 0003 [1] SMP
> CPU 0
> Modules linked in: e1000e sd_mod ata_piix libata thermal fan
> Pid: 4183, comm: blkback.1.sda1 Not tainted 2.6.18.8-xen0 #40
> RIP: e030:[<ffffffff803dc7d6>] =A0[<ffffffff803dc7d6>] blk= tap_map_uaddr_fn+0xa6/0xc0
> RSP: e02b:ffff880039d01840 =A0EFLAGS: 00010297
> RAX: 8000000080765027 RBX: ffff8800388f6688 RCX: ffff880039d01908
> RDX: 00002b218a8d1000 RSI: ffff880001fb15d0 RDI: ffff8800388f6688
> RBP: ffff880039d01850 R08: 00000000000388f6 R09: 0000000000000000
> R10: 0000000000000000 R11: 00000000000002c8 R12: ffff8800388f6688
> R13: 00002b218a8d1000 R14: 00002b218a8d2000 R15: ffff88003890e2a0
> FS: =A000002af9674c06e0(0000) GS:ffffffff8058c000(0000) knlGS:00000000= 00000000
> CS: =A0e033 DS: 0000 ES: 0000
> Process blkback.1.sda1 (pid: 4183, threadinfo ffff880039d00000, task f= fff88003e8cf080)
> Stack: =A0ffff8800388f6688 ffff880001fb15d0 ffff880039d018f0 ffffffff8= 0270033
> =A00000001a00000039 ffff880039d01908 ffffffff803dc730 ffff88003a714080=
> =A0ffff8800389802b0 00002b218a8d2000 00002b218a8d2000 ffff88003c03b430=
> Call Trace:
> =A0[<ffffffff80270033>] apply_to_page_range+0x4e3/0x590
> =A0[<ffffffff803dc730>] blktap_map_uaddr_fn+0x0/0xc0
> =A0[<ffffffff803dac01>] blktap_map_uaddr+0x21/0x30
> =A0[<ffffffff803db70c>] blktap_device_do_request+0x67c/0xfe0
> =A0[<ffffffff8023f36c>] __mod_timer+0xbc/0xe0
> =A0[<ffffffff802088b0>] __switch_to+0x370/0x5b0
> =A0[<ffffffff8023f1dc>] lock_timer_base+0x2c/0x60
> =A0[<ffffffff8023f9c6>] del_timer+0x56/0x70
> =A0[<ffffffff80344715>] __generic_unplug_device+0x25/0x30
> =A0[<ffffffff803459d0>] generic_unplug_device+0x20/0x60
> =A0[<ffffffff803d3196>] unplug_queue+0x26/0x50
> =A0[<ffffffff803d3dea>] blkif_schedule+0x55a/0x690
> =A0[<ffffffff803d3890>] blkif_schedule+0x0/0x690
> =A0[<ffffffff8024b12a>] kthread+0xda/0x110
> =A0[<ffffffff8020a428>] child_rip+0xa/0x12
> =A0[<ffffffff8024b050>] kthread+0x0/0x110
> _______________________________________________
> Xen-devel mailing list
> Xen= -devel@lists.xensource.com
> htt= p://lists.xensource.com/xen-devel
>
>
>
> --
> perception is but an offspring of its own self




--
perception = is but an offspring of its own self
--000e0cd28d3816b56f048b745028-- --===============1482587546== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1482587546==--