From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jiageng Yu Subject: Re: Linux Stubdom Problem Date: Mon, 15 Aug 2011 20:46:36 +0800 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Stefano Stabellini Cc: Ian Campbell , Anthony PERARD , "Xen-devel@lists.xensource.com" , Samuel Thibault List-Id: xen-devel@lists.xenproject.org 2011/8/13 Jiageng Yu : > 2011/7/29 Stefano Stabellini : >> On Fri, 29 Jul 2011, Jiageng Yu wrote: >>> 2011/7/29 Stefano Stabellini >>> On Fri, 29 Jul 2011, Jiageng Yu wrote: >>> > 2011/7/29 Stefano Stabellini >>> > On Fri, 29 Jul 2011, Jiageng Yu wrote: >>> > > I have noticed the mistake. In fact, we shall stop the map_foreign_= pages of xen_console and xenfb devices in >>> qemu. >>> > Because >>> > > the front drivers in stubdom has already map the memories. This is = my new patch. But it is not stable, I am >>> > testing it. >>> > > >>> > > We use xc_handle to map foreign pages in xenfb and xen_console devi= ces. If qemu running on stubdom, the >>> xc_handle >>> > is >>> > > invalid. >>> > > >>> > > >>> > > diff --git a/hw/xen_backend.c b/hw/xen_backend.c >>> > > index cfb53c8..11c53fe 100644 >>> > > --- a/hw/xen_backend.c >>> > > +++ b/hw/xen_backend.c >>> > > @@ -47,6 +47,7 @@ XenXC xen_xc =3D XC_HANDLER_INITIAL_VALUE; >>> > > =C2=A0XenGnttab xen_xcg =3D XC_HANDLER_INITIAL_VALUE; >>> > > =C2=A0struct xs_handle *xenstore =3D NULL; >>> > > =C2=A0const char *xen_protocol; >>> > > +XenXC xc_handle =3D XC_HANDLER_INITIAL_VALUE; >>> > > >>> > > =C2=A0/* private */ >>> > > =C2=A0static QTAILQ_HEAD(XenDeviceHead, XenDevice) xendevs =3D QTAI= LQ_HEAD_INITIALIZER(xendevs); >>> > > @@ -655,6 +656,7 @@ static void xen_be_evtchn_event(void *opaque) >>> > > >>> > > =C2=A0int xen_be_init(void) >>> > > =C2=A0{ >>> > > +#ifndef CONFIG_STUBDOM >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 xenstore =3D xs_daemon_open(); >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (!xenstore) { >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xen_be_printf(NULL= , 0, "can't connect to xenstored\n"); >>> > > @@ -665,10 +667,16 @@ int xen_be_init(void) >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 goto err; >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 } >>> > > >>> > > +=C2=A0=C2=A0=C2=A0 if (xc_handle =3D=3D XC_HANDLER_INITIAL_VALUE) = { >>> > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 goto err; >>> > > +=C2=A0=C2=A0=C2=A0 } >>> > > +#endif >>> > > + >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (xen_xc =3D=3D XC_HANDLER_INITIAL_VALUE= ) { >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /* Check if xen_in= it() have been called */ >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 goto err; >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 } >>> > > + >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 return 0; >>> > > >>> > > =C2=A0err: >>> > > diff --git a/hw/xen_backend.h b/hw/xen_backend.h >>> > > index 9d36df3..bc5a157 100644 >>> > > --- a/hw/xen_backend.h >>> > > +++ b/hw/xen_backend.h >>> > > @@ -59,6 +59,9 @@ extern XenXC xen_xc; >>> > > =C2=A0extern struct xs_handle *xenstore; >>> > > =C2=A0extern const char *xen_protocol; >>> > > >>> > > +/* invalid in linux stubdom */ >>> > > +extern XenXC xc_handle; >>> > > + >>> > > =C2=A0/* xenstore helper functions */ >>> > > =C2=A0int xenstore_write_str(const char *base, const char *node, co= nst char *val); >>> > > =C2=A0int xenstore_write_int(const char *base, const char *node, in= t ival); >>> > > diff --git a/hw/xen_console.c b/hw/xen_console.c >>> > > index c6c8163..66b6dd7 100644 >>> > > --- a/hw/xen_console.c >>> > > +++ b/hw/xen_console.c >>> > > @@ -213,7 +213,7 @@ static int con_connect(struct XenDevice *xendev= ) >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (xenstore_read_int(con->console, "limit= ", &limit) =3D=3D 0) >>> > > =C2=A0=C2=A0con->buffer.max_capacity =3D limit; >>> > > >>> > > -=C2=A0=C2=A0=C2=A0 con->sring =3D xc_map_foreign_range(xen_xc, con= ->xendev.dom, >>> > > +=C2=A0=C2=A0=C2=A0con->sring =3D xc_map_foreign_range(xc_handle, c= on->xendev.dom, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 XC_PAG= E_SIZE, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PROT_R= EAD|PROT_WRITE, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 con->r= ing_ref); >>> > > diff --git a/hw/xenfb.c b/hw/xenfb.c >>> > > index 039076a..278fa60 100644 >>> > > --- a/hw/xenfb.c >>> > > +++ b/hw/xenfb.c >>> > > @@ -104,7 +104,7 @@ static int common_bind(struct common *c) >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (xenstore_read_fe_int(&c->xendev, "even= t-channel", &c->xendev.remote_port) =3D=3D -1) >>> > > =C2=A0=C2=A0return -1; >>> > > >>> > > -=C2=A0=C2=A0=C2=A0 c->page =3D xc_map_foreign_range(xen_xc, c->xen= dev.dom, >>> > > +=C2=A0=C2=A0=C2=A0c->page =3D xc_map_foreign_range(xc_handle, c->x= endev.dom, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 XC_PAGE_SIZE, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PROT_READ | PROT_WRITE, = mfn); >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (c->page =3D=3D NULL) >>> > > @@ -482,14 +482,14 @@ static int xenfb_map_fb(struct XenFB *xenfb) >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 fbmfns =3D qemu_mallocz(sizeof(unsigned lo= ng) * xenfb->fbpages); >>> > > >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 xenfb_copy_mfns(mode, n_fbdirs, pgmfns, pd= ); >>> > > -=C2=A0=C2=A0=C2=A0 map =3D xc_map_foreign_pages(xen_xc, xenfb->c.x= endev.dom, >>> > > +=C2=A0=C2=A0=C2=A0map =3D xc_map_foreign_pages(xc_handle, xenfb->c= .xendev.dom, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PROT_R= EAD, pgmfns, n_fbdirs); >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (map =3D=3D NULL) >>> > > =C2=A0=C2=A0goto out; >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 xenfb_copy_mfns(mode, xenfb->fbpages, fbmf= ns, map); >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 munmap(map, n_fbdirs * XC_PAGE_SIZE); >>> > > >>> > > -=C2=A0=C2=A0=C2=A0 xenfb->pixels =3D xc_map_foreign_pages(xen_xc, = xenfb->c.xendev.dom, >>> > > +=C2=A0=C2=A0=C2=A0xenfb->pixels =3D xc_map_foreign_pages(xc_handle= , xenfb->c.xendev.dom, >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 PROT_READ | PROT_WRITE, fbmfns= , xenfb->fbpages); >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 if (xenfb->pixels =3D=3D NULL) >>> > > =C2=A0=C2=A0goto out; >>> > > diff --git a/xen-all.c b/xen-all.c >>> > > index b73fc43..04dfb51 100644 >>> > > --- a/xen-all.c >>> > > +++ b/xen-all.c >>> > > @@ -527,12 +534,22 @@ int xen_init(void) >>> > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -1; >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 } >>> > > >>> > > +#ifdef CONFIG_STUBDOM >>> > > +=C2=A0=C2=A0=C2=A0 return 0; >>> > > +#endif >>> > > + >>> > > +=C2=A0=C2=A0=C2=A0 xc_handle =3D xen_xc_interface_open(0, 0, 0); >>> > > +=C2=A0=C2=A0=C2=A0 if (xc_handle =3D=3D XC_HANDLER_INITIAL_VALUE) = { >>> > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 xen_be_printf(NULL, 0, = "can't open xen interface\n"); >>> > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return -1; >>> > > +=C2=A0=C2=A0=C2=A0 } >>> > > + >>> > > =C2=A0=C2=A0=C2=A0=C2=A0 return 0; >>> > > =C2=A0} >>> > > >>> > > >>> > >>> > I think that the backends shouldn't be initialized at all when runnin= g >>> > in the stubdom, so I would do something like this instead: >>> > >>> > diff --git a/xen-all.c b/xen-all.c >>> > index 167bed6..e3f630b 100644 >>> > --- a/xen-all.c >>> > +++ b/xen-all.c >>> > @@ -922,6 +922,7 @@ int xen_hvm_init(void) >>> > =C2=A0 =C2=A0 cpu_register_phys_memory_client(&state->client); >>> > =C2=A0 =C2=A0 state->log_for_dirtybit =3D NULL; >>> > >>> > +#ifndef CONFIG_STUBDOM >>> > =C2=A0 =C2=A0 /* Initialize backend core & drivers */ >>> > =C2=A0 =C2=A0 if (xen_be_init() !=3D 0) { >>> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 fprintf(stderr, "%s: xen backend core set= up failed\n", __FUNCTION__); >>> > @@ -930,7 +931,7 @@ int xen_hvm_init(void) >>> > =C2=A0 =C2=A0 xen_be_register("console", &xen_console_ops); >>> > =C2=A0 =C2=A0 xen_be_register("vkbd", &xen_kbdmouse_ops); >>> > =C2=A0 =C2=A0 xen_be_register("qdisk", &xen_blkdev_ops); >>> > - >>> > +#endif >>> > =C2=A0 =C2=A0 return 0; >>> > =C2=A0} >>> > >>> > >>> > >>> > >>> > Yes. this implementation of stubdom=C2=A0is more closer to the old qe= mu's implementation. Which branch this patch works >>> on? The >>> > qemu-dm-v15 would not so lucky. >>> >>> That branch is old now, checkout this one: >>> >>> git://xenbits.xen.org/people/sstabellini/qemu-dm.git xen-stable-0.15 >>> >>> >>> >>> Do you mean I migrate my work=C2=A0to this qemu branch? >> >> Yes, if it doesn't cost too much time for you to rebase your patches. > > > Hi Stefano, > > =C2=A0 =C2=A0 These days I updated my patches to support =C2=A0xen-stable= -0.15 qemu > and latest xen-unstable. The fbdev is still not work. However, I have > found three vram mapping areas in linux based stubdom. I think maybe > our problem is caused by the mismatch of these areas. I list them with > the order of initialization. > > 1. =C2=A0vga_common_init(s,8M) > =C2=A0 =C2=A0 This vram area is mapped by xc_map_foreign_bulk() > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ->linux_privcmd_map_fore= ign_bulk() > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0->mmap(NULL, (unsigned long)num << > XC_PAGE_SHIFT, prot, MAP_SHARED,fd, 0); > =C2=A0 =C2=A0 The fd is descriptor of privcmd device. The > entry->vaddr_base=3D0xb6bdc000. Therefore, the mapping area is > 0xb6bdc000 ~ 0xb73dc000, 8M. > > =C2=A0 =C2=A0 The block->offset allocated by qemu_get_ram_ptr() is ram_si= ze. > The ram_size is the declared memory of HVM guest. In my case, the hvm > guest takes 384M memory. So block->offset=3D0x18000000. The guest > physical memory of vram is 0x18000000 ~ 0x18800000, 8M. > > 2. fbdev_init() > =C2=A0 =C2=A0 This vram area is mapped by fbdev_init() > > ->mmap(NULL,fb_fix.smem_len+fb_mem_offset,PROT_READ|PROT_WRITE,MAP_SHARED= ,fb,0); > =C2=A0 =C2=A0 The fd is descriptor of fb0 device. The mapping area is > 0xb680a000 ~ 0xb6a0a000, 2M. > > 3. xen_add_to_physmap() > =C2=A0 =C2=A0 =C2=A0This function is mapping the vram to 0xf0000000 ~ 0xf= 080000, 8M. > > =C2=A0 =C2=A0Referring to minios-based stubdom, I think the stubdom will = work > well if fbdev maps vram according to 0x18000000 ~ 0x18800000. I look > forward to your comments and suggestions. > > =C2=A0 =C2=A0Thanks, > > > Jiageng Yu. > Hi Stefano, In the linux-pv xenfbfront driver, the vram is allocated by: xenfb_probe() ->info->fb =3D vmalloc(fb_size); In the linux-stubdom, the memory areas: (info->fb, info->fb+fb_size) in linux-pv kernel, (s->vram_offset, s->vram_offset+VGA_RAM_SIZE) in vga_common_init function, (s->vram_ptr, s->vram_ptr+VGA_RAM_SIZE) in stubdom. These memory areas should be mapped into the same machine memory region. But the (info->fb, info->fb+fb_size) in linux-pv kernel is allocated independently. I have two optional plans to slove the problem. 1. Modify linux-pv kernel. This plan is more closer to the minios stubdom. First, I delay the initial process of xenfbfront driver until qemu allocates s->vram_ptr. Then, I set info->fb =3D s->vram_ptr. 2. Modify libxc. The s->vram_ptr is generated by mmap() function in linux_privcmd_map_foreign_bulk(). Maybe I could replace the return value of mmap() with the info->fb, which is obtained from xenstore. Can these plans solve the problem? Or there is the better one? Regards! Jiageng Yu.