domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

[manish jaggi]

In xenstore-ls I dont see this node (control node is not there as well)
What could be missing here.

Re: domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

[Wei Liu]

On Wed, Aug 20, 2014 at 06:54:26PM +0530, manish jaggi wrote:
> In xenstore-ls I dont see this node (control node is not there as well)
> What could be missing here.
> 

It's libxl's resposibility to write that node. However libxl should fail
if it fails to create that node. See libxl_create.c.

You can try to increase xl's verbosity to have an idea what went wrong.

Wei.

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

[Ian Campbell]

On Thu, 2014-08-21 at 12:54 +0100, Wei Liu wrote:
> On Wed, Aug 20, 2014 at 06:54:26PM +0530, manish jaggi wrote:
> > In xenstore-ls I dont see this node (control node is not there as well)
> > What could be missing here.
> > 
> 
> It's libxl's resposibility to write that node. However libxl should fail
> if it fails to create that node. See libxl_create.c.
> 

But also the guest ought to be robust to that node not being present,
afterall that is the point of a feature-* node...

Ian.

> You can try to increase xl's verbosity to have an idea what went wrong.
> 
> Wei.
> 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

[manish jaggi]

On 21 August 2014 22:53, Ian Campbell <ian.campbell@citrix.com> wrote:
> On Thu, 2014-08-21 at 12:54 +0100, Wei Liu wrote:
>> On Wed, Aug 20, 2014 at 06:54:26PM +0530, manish jaggi wrote:
>> > In xenstore-ls I dont see this node (control node is not there as well)
>> > What could be missing here.
>> >
>>
>> It's libxl's resposibility to write that node. However libxl should fail
>> if it fails to create that node. See libxl_create.c.
>>
>
> But also the guest ought to be robust to that node not being present,
> afterall that is the point of a feature-* node...
>
> Ian.
>
>> You can try to increase xl's verbosity to have an idea what went wrong.
>>
>> Wei.
>>
>> > _______________________________________________
>> > Xen-devel mailing list
>> > Xen-devel@lists.xen.org
>> > http://lists.xen.org/xen-devel
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>
>
I debugged a bit more to put xen_raw_printk in xs_talkv, xb_write and
notify_remote_via_evtchn. I found that the notify_remote_via_evtchn
for port 1 did not complete domU didnt return from that function. (I
added two printk before and after the HYPERVISOR_event_channel_op).

__func__, __LINE__

(d1) xs_talkv 249
(d1) xb_write 104
(d1) xb_write 137
(d1) notify_remote_via_evtchn 62 PORT=1
notify_remote_via_evtchn 62 PORT=5
notify_remote_via_evtchn 64 PORT=5
notify_remote_via_evtchn 62 PORT=5
notify_remote_via_evtchn 64 PORT=5

Dumping event channel on Xen Console gave this

(XEN) 'e' pressed -> dumping event-channel info
(XEN) Event channel information for domain 0:
(XEN) Polling vCPUs: {}
(XEN)     port [p/m/s]
(XEN)        1 [0/0/0]: s=3 n=0 x=0 d=0 p=3
(XEN)        2 [0/0/0]: s=5 n=0 x=0 v=2
(XEN)        3 [0/0/0]: s=3 n=0 x=0 d=0 p=1
(XEN)        4 [0/0/0]: s=5 n=0 x=0 v=3
(XEN)        5 [0/0/0]: s=3 n=0 x=0 d=1 p=1
(XEN)        6 [0/0/0]: s=3 n=0 x=0 d=1 p=2
(XEN) Event channel information for domain 1:
(XEN) Polling vCPUs: {}
(XEN)     port [p/m/s]
(XEN)        1 [1/0/  -   ]: s=3 n=0 x=0 d=0 p=5
(XEN)        2 [0/1/  -   ]: s=3 n=0 x=0 d=0 p=6

What does p/m/s/x/n/d mean ? Any pointers on where I should start looking ?

Re: domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

[Ian Campbell]

On Fri, 2014-08-22 at 19:15 +0530, manish jaggi wrote:

> What does p/m/s/x/n/d mean ? Any pointers on where I should start looking ?

I don't know off hand. The code which prints them presumably contains
some clues to what they mean.

Ian.

Re: domU hang in xenbus_scanf for /control/platform-feature-xs_reset_watches

[Wei Liu]

On Fri, Aug 22, 2014 at 07:15:58PM +0530, manish jaggi wrote:
> On 21 August 2014 22:53, Ian Campbell <ian.campbell@citrix.com> wrote:
> > On Thu, 2014-08-21 at 12:54 +0100, Wei Liu wrote:
> >> On Wed, Aug 20, 2014 at 06:54:26PM +0530, manish jaggi wrote:
> >> > In xenstore-ls I dont see this node (control node is not there as well)
> >> > What could be missing here.
> >> >
> >>
> >> It's libxl's resposibility to write that node. However libxl should fail
> >> if it fails to create that node. See libxl_create.c.
> >>
> >
> > But also the guest ought to be robust to that node not being present,
> > afterall that is the point of a feature-* node...
> >
> > Ian.
> >
> >> You can try to increase xl's verbosity to have an idea what went wrong.
> >>
> >> Wei.
> >>
> >> > _______________________________________________
> >> > Xen-devel mailing list
> >> > Xen-devel@lists.xen.org
> >> > http://lists.xen.org/xen-devel
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xen.org
> >> http://lists.xen.org/xen-devel
> >
> >
> I debugged a bit more to put xen_raw_printk in xs_talkv, xb_write and
> notify_remote_via_evtchn. I found that the notify_remote_via_evtchn
> for port 1 did not complete domU didnt return from that function. (I
> added two printk before and after the HYPERVISOR_event_channel_op).
> 

How odd. notify_remote_via_evtchn only issues one hypercall and does
nothing else. So you're saying it's stuck in hypervisor? Setting up a
series console and get hypervisor log might help.

> __func__, __LINE__
> 
> (d1) xs_talkv 249
> (d1) xb_write 104
> (d1) xb_write 137
> (d1) notify_remote_via_evtchn 62 PORT=1
> notify_remote_via_evtchn 62 PORT=5
> notify_remote_via_evtchn 64 PORT=5
> notify_remote_via_evtchn 62 PORT=5
> notify_remote_via_evtchn 64 PORT=5
> 
> Dumping event channel on Xen Console gave this
> 
> (XEN) 'e' pressed -> dumping event-channel info
> (XEN) Event channel information for domain 0:
> (XEN) Polling vCPUs: {}
> (XEN)     port [p/m/s]
> (XEN)        1 [0/0/0]: s=3 n=0 x=0 d=0 p=3
> (XEN)        2 [0/0/0]: s=5 n=0 x=0 v=2
> (XEN)        3 [0/0/0]: s=3 n=0 x=0 d=0 p=1
> (XEN)        4 [0/0/0]: s=5 n=0 x=0 v=3
> (XEN)        5 [0/0/0]: s=3 n=0 x=0 d=1 p=1
> (XEN)        6 [0/0/0]: s=3 n=0 x=0 d=1 p=2
> (XEN) Event channel information for domain 1:
> (XEN) Polling vCPUs: {}
> (XEN)     port [p/m/s]
> (XEN)        1 [1/0/  -   ]: s=3 n=0 x=0 d=0 p=5
> (XEN)        2 [0/1/  -   ]: s=3 n=0 x=0 d=0 p=6
> 
> What does p/m/s/x/n/d mean ? Any pointers on where I should start looking ?

Look at Xen source code common/event_channel.c.

Wei.