some questions of IO ring in xenpaging

some questions of IO ring in xenpaging

[zhen shi]

[-- Attachment #1: Type: text/plain, Size: 2033 bytes --]

Hi Olaf --

I have two questions to ask you about xenpaging.
1) When guest os causes page_fault for the accessed page is paging_out
or paged,it will execute p2m_mem_paging_populate() .
and in p2m_mem_paging_populate() it will first check if the ring is full.
when I ran with domU  suse11 4G memory and 8vcpus,I found there will
be a corruption in checking the ring.
For example,if 4vcpus are met with page faults when they access
different pages,and there is only four free-requests for the ring.
and then they call p2m_mem_paging_populate(),and execute
mem_event_check_ring(d) at the same time.All will find ring is not
full,and will fill the requests.It will cause the latter request to
cover the front request.
and I think there should a lock before the mem_event_check_ring(d)
,and normally it unlock after mem_event_put_request(d, &req).
You can review the attached doc of xenpaging_IO_ring.txt to see if my
opnion is right.

2)mem_sharing and xenpaging are shared with one IO ring for domU.In
the function of mem_sharing_alloc_page(),if alloc_domheap_page(d, 0)
returns NULL,then it will pause VCPU ,check if the ring is full,and
fill the request at last.
I think there is also a corruption of mem_event_check_ring(d) with it
in p2m_mem_paging_populate().We should assure
exclusively in reading the free_request and puting requests.
What's more,although it hardly fails in alloc_domheap_page(d, 0) from
mem_sharing_alloc_page() ,it will fill the requests in IO ring.
But  in xenpaging when handling the page_in requests,we have not
distinguished the requests with flag "MEM_EVENT_FLAG_VCPU_PAUSED" from
paging or sharing.It will cause if the request is from
mem_sharing_alloc_page(),it will
go to p2m_mem_paging_resume() at last,and the page's p2mt is
p2m_ram_rw.I think this is wrong.Maybe we should add the req.type when
page in .

I'm so sorry to have a poor English.But I look forward to your early reply.


                                                              Thank
you！

[-- Attachment #2: xenpaging_IO_ring.txt --]
[-- Type: text/plain, Size: 1406 bytes --]

void p2m_mem_paging_populate(struct p2m_domain *p2m, unsigned long gfn)
{
    struct vcpu *v = current;
    mem_event_request_t req;
    p2m_type_t p2mt;
    struct domain *d = p2m->domain;

 +  p2m_lock(p2m);
    /* Check that there's space on the ring for this request */
    if ( mem_event_check_ring(d) )
+  {
+     p2m_unlock(p2m);
        return;
+ }

    memset(&req, 0, sizeof(req));
    req.type = MEM_EVENT_TYPE_PAGING;

    /* Fix p2m mapping */
    /* XXX: It seems inefficient to have this here, as it's only needed
     *      in one case (ept guest accessing paging out page) */
    gfn_to_mfn(p2m, gfn, &p2mt);
    if ( p2mt == p2m_ram_paged )
    {
-      p2m_lock(p2m);
        set_p2m_entry(p2m, gfn, _mfn(PAGING_MFN), 0, p2m_ram_paging_in_start, p2m->default_access);
        audit_p2m(p2m, 1);
-     p2m_unlock(p2m);
    }

    /* Pause domain */
    if ( v->domain->domain_id == d->domain_id )
    {
        vcpu_pause_nosync(v);
        req.flags |= MEM_EVENT_FLAG_VCPU_PAUSED;
    }
    else if ( p2mt != p2m_ram_paging_out && p2mt != p2m_ram_paged )
    {
        /* gfn is already on its way back and vcpu is not paused */
+     p2m_unlock(p2m);
        return;
    }

    /* Send request to pager */
    req.gfn = gfn;
    req.p2mt = p2mt;
    req.vcpu_id = v->vcpu_id;

    mem_event_put_request(d, &req);
+ p2m_unlock(p2m);
}

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: some questions of IO ring in xenpaging

[Olaf Hering]

On Thu, Sep 01, zhen shi wrote:

> Hi Olaf --
> 
> I have two questions to ask you about xenpaging.
> 1) When guest os causes page_fault for the accessed page is paging_out
> or paged, it will execute p2m_mem_paging_populate(). and in
> p2m_mem_paging_populate() it will first check if the ring is full.
> when I ran with domU suse11 4G memory and 8vcpus, I found there will
> be a corruption in checking the ring.
> For example, if 4vcpus are met with page faults when they access
> different pages, and there is only four free-requests for the ring.
> and then they call p2m_mem_paging_populate(),and execute
> mem_event_check_ring(d) at the same time.All will find ring is not
> full,and will fill the requests. It will cause the latter request to
> cover the front request.
> and I think there should a lock before the mem_event_check_ring(d),
> and normally it unlock after mem_event_put_request(d, &req).
> You can review the attached doc of xenpaging_IO_ring.txt to see if my
> opnion is right.

Yes, you are right.
I think mem_event_check_ring() should reserve a reference, and
mem_event_put_request() should use that reference.
mem_sharing_alloc_page() even has a comment that this should be done.


> 2) mem_sharing and xenpaging are shared with one IO ring for domU. In
> the function of mem_sharing_alloc_page(), if alloc_domheap_page(d, 0)
> returns NULL, then it will pause VCPU, check if the ring is full, and
> fill the request at last.
> I think there is also a corruption of mem_event_check_ring(d) with it
> in p2m_mem_paging_populate(). We should assure exclusively in reading
> the free_request and puting requests.  What's more, although it hardly
> fails in alloc_domheap_page(d, 0) from mem_sharing_alloc_page(), it
> will fill the requests in IO ring.  But in xenpaging when handling the
> page_in requests, we have not distinguished the requests with flag
> "MEM_EVENT_FLAG_VCPU_PAUSED" from paging or sharing. It will cause if
> the request is from mem_sharing_alloc_page(), it will go to
> p2m_mem_paging_resume() at last, and the page's p2mt is p2m_ram_rw. I
> think this is wrong. Maybe we should add the req.type when page in.

Yes, get_request() in xenpaging should check the type before popping the
request from the ring. Perhaps memsharing and xenpaging should use its
own rings.

Olaf

Re: some questions of IO ring in xenpaging

[Olaf Hering]

On Thu, Sep 01, Olaf Hering wrote:

> On Thu, Sep 01, zhen shi wrote:
> 
> > Hi Olaf --
> > 
> > I have two questions to ask you about xenpaging.
> > 1) When guest os causes page_fault for the accessed page is paging_out
> > or paged, it will execute p2m_mem_paging_populate(). and in
> > p2m_mem_paging_populate() it will first check if the ring is full.
> > when I ran with domU suse11 4G memory and 8vcpus, I found there will
> > be a corruption in checking the ring.
> > For example, if 4vcpus are met with page faults when they access
> > different pages, and there is only four free-requests for the ring.
> > and then they call p2m_mem_paging_populate(),and execute
> > mem_event_check_ring(d) at the same time.All will find ring is not
> > full,and will fill the requests. It will cause the latter request to
> > cover the front request.
> > and I think there should a lock before the mem_event_check_ring(d),
> > and normally it unlock after mem_event_put_request(d, &req).
> > You can review the attached doc of xenpaging_IO_ring.txt to see if my
> > opnion is right.
> 
> Yes, you are right.
> I think mem_event_check_ring() should reserve a reference, and
> mem_event_put_request() should use that reference.
> mem_sharing_alloc_page() even has a comment that this should be done.

Try this patch. It implements some ref counting.

http://lists.xensource.com/archives/html/xen-devel/2011-09/msg00189.html

Olaf