Re: Linux Stubdom Problem

[Jiageng Yu <yujiageng734@gmail.com>]

2011/8/13 Jiageng Yu <yujiageng734@gmail.com>:
> 2011/7/29 Stefano Stabellini <stefano.stabellini@eu.citrix.com>:
>> On Fri, 29 Jul 2011, Jiageng Yu wrote:
>>> 2011/7/29 Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>> On Fri, 29 Jul 2011, Jiageng Yu wrote:
>>> > 2011/7/29 Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>> > On Fri, 29 Jul 2011, Jiageng Yu wrote:
>>> > > I have noticed the mistake. In fact, we shall stop the map_foreign_pages of xen_console and xenfb devices in
>>> qemu.
>>> > Because
>>> > > the front drivers in stubdom has already map the memories. This is my new patch. But it is not stable, I am
>>> > testing it.
>>> > >
>>> > > We use xc_handle to map foreign pages in xenfb and xen_console devices. If qemu running on stubdom, the
>>> xc_handle
>>> > is
>>> > > invalid.
>>> > >
>>> > >
>>> > > diff --git a/hw/xen_backend.c b/hw/xen_backend.c
>>> > > index cfb53c8..11c53fe 100644
>>> > > --- a/hw/xen_backend.c
>>> > > +++ b/hw/xen_backend.c
>>> > > @@ -47,6 +47,7 @@ XenXC xen_xc = XC_HANDLER_INITIAL_VALUE;
>>> > >  XenGnttab xen_xcg = XC_HANDLER_INITIAL_VALUE;
>>> > >  struct xs_handle *xenstore = NULL;
>>> > >  const char *xen_protocol;
>>> > > +XenXC xc_handle = XC_HANDLER_INITIAL_VALUE;
>>> > >
>>> > >  /* private */
>>> > >  static QTAILQ_HEAD(XenDeviceHead, XenDevice) xendevs = QTAILQ_HEAD_INITIALIZER(xendevs);
>>> > > @@ -655,6 +656,7 @@ static void xen_be_evtchn_event(void *opaque)
>>> > >
>>> > >  int xen_be_init(void)
>>> > >  {
>>> > > +#ifndef CONFIG_STUBDOM
>>> > >      xenstore = xs_daemon_open();
>>> > >      if (!xenstore) {
>>> > >          xen_be_printf(NULL, 0, "can't connect to xenstored\n");
>>> > > @@ -665,10 +667,16 @@ int xen_be_init(void)
>>> > >          goto err;
>>> > >      }
>>> > >
>>> > > +    if (xc_handle == XC_HANDLER_INITIAL_VALUE) {
>>> > > +        goto err;
>>> > > +    }
>>> > > +#endif
>>> > > +
>>> > >      if (xen_xc == XC_HANDLER_INITIAL_VALUE) {
>>> > >          /* Check if xen_init() have been called */
>>> > >          goto err;
>>> > >      }
>>> > > +
>>> > >      return 0;
>>> > >
>>> > >  err:
>>> > > diff --git a/hw/xen_backend.h b/hw/xen_backend.h
>>> > > index 9d36df3..bc5a157 100644
>>> > > --- a/hw/xen_backend.h
>>> > > +++ b/hw/xen_backend.h
>>> > > @@ -59,6 +59,9 @@ extern XenXC xen_xc;
>>> > >  extern struct xs_handle *xenstore;
>>> > >  extern const char *xen_protocol;
>>> > >
>>> > > +/* invalid in linux stubdom */
>>> > > +extern XenXC xc_handle;
>>> > > +
>>> > >  /* xenstore helper functions */
>>> > >  int xenstore_write_str(const char *base, const char *node, const char *val);
>>> > >  int xenstore_write_int(const char *base, const char *node, int ival);
>>> > > diff --git a/hw/xen_console.c b/hw/xen_console.c
>>> > > index c6c8163..66b6dd7 100644
>>> > > --- a/hw/xen_console.c
>>> > > +++ b/hw/xen_console.c
>>> > > @@ -213,7 +213,7 @@ static int con_connect(struct XenDevice *xendev)
>>> > >      if (xenstore_read_int(con->console, "limit", &limit) == 0)
>>> > >   con->buffer.max_capacity = limit;
>>> > >
>>> > > -    con->sring = xc_map_foreign_range(xen_xc, con->xendev.dom,
>>> > > +   con->sring = xc_map_foreign_range(xc_handle, con->xendev.dom,
>>> > >            XC_PAGE_SIZE,
>>> > >            PROT_READ|PROT_WRITE,
>>> > >            con->ring_ref);
>>> > > diff --git a/hw/xenfb.c b/hw/xenfb.c
>>> > > index 039076a..278fa60 100644
>>> > > --- a/hw/xenfb.c
>>> > > +++ b/hw/xenfb.c
>>> > > @@ -104,7 +104,7 @@ static int common_bind(struct common *c)
>>> > >      if (xenstore_read_fe_int(&c->xendev, "event-channel", &c->xendev.remote_port) == -1)
>>> > >   return -1;
>>> > >
>>> > > -    c->page = xc_map_foreign_range(xen_xc, c->xendev.dom,
>>> > > +   c->page = xc_map_foreign_range(xc_handle, c->xendev.dom,
>>> > >         XC_PAGE_SIZE,
>>> > >         PROT_READ | PROT_WRITE, mfn);
>>> > >      if (c->page == NULL)
>>> > > @@ -482,14 +482,14 @@ static int xenfb_map_fb(struct XenFB *xenfb)
>>> > >      fbmfns = qemu_mallocz(sizeof(unsigned long) * xenfb->fbpages);
>>> > >
>>> > >      xenfb_copy_mfns(mode, n_fbdirs, pgmfns, pd);
>>> > > -    map = xc_map_foreign_pages(xen_xc, xenfb->c.xendev.dom,
>>> > > +   map = xc_map_foreign_pages(xc_handle, xenfb->c.xendev.dom,
>>> > >            PROT_READ, pgmfns, n_fbdirs);
>>> > >      if (map == NULL)
>>> > >   goto out;
>>> > >      xenfb_copy_mfns(mode, xenfb->fbpages, fbmfns, map);
>>> > >      munmap(map, n_fbdirs * XC_PAGE_SIZE);
>>> > >
>>> > > -    xenfb->pixels = xc_map_foreign_pages(xen_xc, xenfb->c.xendev.dom,
>>> > > +   xenfb->pixels = xc_map_foreign_pages(xc_handle, xenfb->c.xendev.dom,
>>> > >        PROT_READ | PROT_WRITE, fbmfns, xenfb->fbpages);
>>> > >      if (xenfb->pixels == NULL)
>>> > >   goto out;
>>> > > diff --git a/xen-all.c b/xen-all.c
>>> > > index b73fc43..04dfb51 100644
>>> > > --- a/xen-all.c
>>> > > +++ b/xen-all.c
>>> > > @@ -527,12 +534,22 @@ int xen_init(void)
>>> > >          return -1;
>>> > >      }
>>> > >
>>> > > +#ifdef CONFIG_STUBDOM
>>> > > +    return 0;
>>> > > +#endif
>>> > > +
>>> > > +    xc_handle = xen_xc_interface_open(0, 0, 0);
>>> > > +    if (xc_handle == XC_HANDLER_INITIAL_VALUE) {
>>> > > +        xen_be_printf(NULL, 0, "can't open xen interface\n");
>>> > > +        return -1;
>>> > > +    }
>>> > > +
>>> > >      return 0;
>>> > >  }
>>> > >
>>> > >
>>> >
>>> > I think that the backends shouldn't be initialized at all when running
>>> > in the stubdom, so I would do something like this instead:
>>> >
>>> > diff --git a/xen-all.c b/xen-all.c
>>> > index 167bed6..e3f630b 100644
>>> > --- a/xen-all.c
>>> > +++ b/xen-all.c
>>> > @@ -922,6 +922,7 @@ int xen_hvm_init(void)
>>> >     cpu_register_phys_memory_client(&state->client);
>>> >     state->log_for_dirtybit = NULL;
>>> >
>>> > +#ifndef CONFIG_STUBDOM
>>> >     /* Initialize backend core & drivers */
>>> >     if (xen_be_init() != 0) {
>>> >         fprintf(stderr, "%s: xen backend core setup failed\n", __FUNCTION__);
>>> > @@ -930,7 +931,7 @@ int xen_hvm_init(void)
>>> >     xen_be_register("console", &xen_console_ops);
>>> >     xen_be_register("vkbd", &xen_kbdmouse_ops);
>>> >     xen_be_register("qdisk", &xen_blkdev_ops);
>>> > -
>>> > +#endif
>>> >     return 0;
>>> >  }
>>> >
>>> >
>>> >
>>> >
>>> > Yes. this implementation of stubdom is more closer to the old qemu's implementation. Which branch this patch works
>>> on? The
>>> > qemu-dm-v15 would not so lucky.
>>>
>>> That branch is old now, checkout this one:
>>>
>>> git://xenbits.xen.org/people/sstabellini/qemu-dm.git xen-stable-0.15
>>>
>>>
>>>
>>> Do you mean I migrate my work to this qemu branch?
>>
>> Yes, if it doesn't cost too much time for you to rebase your patches.
>
>
> Hi Stefano,
>
>     These days I updated my patches to support  xen-stable-0.15 qemu
> and latest xen-unstable. The fbdev is still not work. However, I have
> found three vram mapping areas in linux based stubdom. I think maybe
> our problem is caused by the mismatch of these areas. I list them with
> the order of initialization.
>
> 1.  vga_common_init(s,8M)
>     This vram area is mapped by xc_map_foreign_bulk()
>               ->linux_privcmd_map_foreign_bulk()
>                        ->mmap(NULL, (unsigned long)num <<
> XC_PAGE_SHIFT, prot, MAP_SHARED,fd, 0);
>     The fd is descriptor of privcmd device. The
> entry->vaddr_base=0xb6bdc000. Therefore, the mapping area is
> 0xb6bdc000 ~ 0xb73dc000, 8M.
>
>     The block->offset allocated by qemu_get_ram_ptr() is ram_size.
> The ram_size is the declared memory of HVM guest. In my case, the hvm
> guest takes 384M memory. So block->offset=0x18000000. The guest
> physical memory of vram is 0x18000000 ~ 0x18800000, 8M.
>
> 2. fbdev_init()
>     This vram area is mapped by fbdev_init()
>
> ->mmap(NULL,fb_fix.smem_len+fb_mem_offset,PROT_READ|PROT_WRITE,MAP_SHARED,fb,0);
>     The fd is descriptor of fb0 device. The mapping area is
> 0xb680a000 ~ 0xb6a0a000, 2M.
>
> 3. xen_add_to_physmap()
>      This function is mapping the vram to 0xf0000000 ~ 0xf080000, 8M.
>
>    Referring to minios-based stubdom, I think the stubdom will work
> well if fbdev maps vram according to 0x18000000 ~ 0x18800000. I look
> forward to your comments and suggestions.
>
>    Thanks,
>
>
> Jiageng Yu.
>

Hi Stefano,

    In the linux-pv xenfbfront driver, the vram is allocated by:
    xenfb_probe()
        ->info->fb = vmalloc(fb_size);

    In the linux-stubdom, the memory areas: (info->fb,
info->fb+fb_size) in linux-pv kernel, (s->vram_offset,
s->vram_offset+VGA_RAM_SIZE) in vga_common_init function,
(s->vram_ptr, s->vram_ptr+VGA_RAM_SIZE) in stubdom. These memory areas
should be mapped into the same machine memory region.

    But the (info->fb, info->fb+fb_size) in linux-pv kernel is
allocated independently. I have two optional plans to slove the
problem.

1. Modify linux-pv kernel.
    This plan is more closer to the minios stubdom. First, I delay the
initial process of xenfbfront driver until qemu allocates s->vram_ptr.
Then, I set info->fb = s->vram_ptr.

2. Modify libxc.
    The s->vram_ptr is generated by mmap() function in
linux_privcmd_map_foreign_bulk(). Maybe I could replace the return
value of mmap() with the info->fb, which is obtained from xenstore.

    Can these plans solve the problem? Or there is the better one?

    Regards!

Jiageng Yu.