event channel in xenpaging

event channel in xenpaging

[zhen shi]

Hi,Olaf,

I have some questions about event channel in Xenpaging to ask you.
1) In xenpaging it uses Inter-Domainain Commnication (IDC) between
dom0 and domU to build bidirectional connection,but I found there is
only an event channel notification from xen to dom0 when page faults
happens.It seems that xenpaging_resume_page()->xc_evtchn_notify()
doesn't make any difference.So why don't use vIRQ between dom0 and xen
instead of IDC between dom0 and domU?

2)In your latest patch,[PATCH 9 of 9] xenpaging: watch the domains
/xenpaging/num_pages xenstore value.I found some problems.
  a、In main(),you put the page_out process into while(1) and make the
following change.
     if ( interrupted )
    victims[i].gfn = INVALID_MFN;
    - else
    - evict_victim(paging, &victims[i], fd, i);
I think the" if ( interrupted )" should remove here.Because once
handling page_in requests,the related victims slot should clear,
then in evict_pages we can populate new page in this slot because in
evict_page() the condition as followings:
 /* Slot is allocated */
+ if ( victims[slot].gfn != INVALID_MFN )
+ continue;

b、In xenpaging_init (),when it goes to err,you add
"free(dom_path);free(watch_targetpages)" in PATCH 9,but I think we
should firstly judge if dom_path and watch_targetpages variables are
NULL.what's more ,if it initializes paging successfully，before "return
paging",we should free (dom_path),and free(watch_targetpages) in
xenpaging_teardown() lastly.
c、when initializing xenpaging fails,and goes to err,it returns NULL to
main() then exit main().
If it has already opened event channel,bind event notification,opened
connections to xen successfully when goes to err in
xenpaging_init()，how about dealing with these resources such as
xenpaging_teardown.

3)We have tested on Win7-32bit about 40vms to start xenpaging at the
same time.The vm is 1G 2VCPUS,and we page out 180M.Then we put 80-90%
memory pressure on each vm.About one hour later,there happens many
BSOD on vms.and most blue screen code is 0x... 7F or 0x...19. We guess
some special pages which systems need use to run may be paged out and
cause BSOD.and sometimes print out mmio information in messages.Do you
have any ideas about the BSOD on win7-32 bit vms?

I am looking forward to hearing from you. Thank you very much!  :)

Re: event channel in xenpaging

[Olaf Hering]

On Wed, Sep 28, zhen shi wrote:

> Hi,Olaf,
> 
> I have some questions about event channel in Xenpaging to ask you.
> 1) In xenpaging it uses Inter-Domainain Commnication (IDC) between
> dom0 and domU to build bidirectional connection,but I found there is
> only an event channel notification from xen to dom0 when page faults
> happens.It seems that xenpaging_resume_page()->xc_evtchn_notify()
> doesn't make any difference.So why don't use vIRQ between dom0 and xen
> instead of IDC between dom0 and domU?

I talked with Adin Scannel about that, he has changes to use an event
channel instead of domctrl for communication. The current event channel
usage in xenpaging is a noop.

> 2)In your latest patch,[PATCH 9 of 9] xenpaging: watch the domains
> /xenpaging/num_pages xenstore value.I found some problems.
>   a、In main(),you put the page_out process into while(1) and make the
> following change.
>      if ( interrupted )
>     victims[i].gfn = INVALID_MFN;
>     - else
>     - evict_victim(paging, &victims[i], fd, i);
> I think the" if ( interrupted )" should remove here.Because once
> handling page_in requests,the related victims slot should clear,
> then in evict_pages we can populate new page in this slot because in
> evict_page() the condition as followings:
>  /* Slot is allocated */
> + if ( victims[slot].gfn != INVALID_MFN )
> + continue;

Thanks for spotting this, that part is not correct and the comment is
now bogus as well.

> b、In xenpaging_init (),when it goes to err,you add
> "free(dom_path);free(watch_targetpages)" in PATCH 9,but I think we
> should firstly judge if dom_path and watch_targetpages variables are
> NULL.what's more ,if it initializes paging successfully，before "return
> paging",we should free (dom_path),and free(watch_targetpages) in
> xenpaging_teardown() lastly.
> c、when initializing xenpaging fails,and goes to err,it returns NULL to
> main() then exit main().
> If it has already opened event channel,bind event notification,opened
> connections to xen successfully when goes to err in
> xenpaging_init()，how about dealing with these resources such as
> xenpaging_teardown.

watch_targetpages is used in xenpaging_wait_for_event_or_timeout(), and
dom_path is used elsewhere in upcoming changes.
You are right about xenpaging_teardown(), perhaps the error path in
xenpaging_init() could make use of that function.

> 3)We have tested on Win7-32bit about 40vms to start xenpaging at the
> same time.The vm is 1G 2VCPUS,and we page out 180M.Then we put 80-90%
> memory pressure on each vm.About one hour later,there happens many
> BSOD on vms.and most blue screen code is 0x... 7F or 0x...19. We guess
> some special pages which systems need use to run may be paged out and
> cause BSOD.and sometimes print out mmio information in messages.Do you
> have any ideas about the BSOD on win7-32 bit vms?

That sounds like incorrect handling of paged-out pages in Xen, many code
paths are supposed to retry the gfn_to_mfn* calls currently, but some do
not. 
I will start to work on that bug soon.
Do these guests use the PV drivers?

Olaf

Re: Re: event channel in xenpaging

[Adin Scannell]

>> I have some questions about event channel in Xenpaging to ask you.
>> 1) In xenpaging it uses Inter-Domainain Commnication (IDC) between
>> dom0 and domU to build bidirectional connection,but I found there is
>> only an event channel notification from xen to dom0 when page faults
>> happens.It seems that xenpaging_resume_page()->xc_evtchn_notify()
>> doesn't make any difference.So why don't use vIRQ between dom0 and xen
>> instead of IDC between dom0 and domU?
>
> I talked with Adin Scannel about that, he has changes to use an event
> channel instead of domctrl for communication. The current event channel
> usage in xenpaging is a noop.

I just sent these patches to xen-devel today if you'd like to test
them.  Basically, you need only do the xc_evtchn_notify() -- although
the domctl will also work if you'd like to do a synchronous resume.