How console data travel in Xen?

How console data travel in Xen?

[NAHieu]

Hello,

I am trying to investigate in detail how console data go thru the Xen
system. Suppose that I am running a domainU, when I type at the
domainU console (I connect to the console with xencons or ssh), how
data (from keyboard in this case) travel from low level (hardware) to
Xen, then to dom0, then to domU?

I am not afraid of reading and hacking the source to understand this,
but I dont know where to start in the jungle of code. (Which files
should I pay attention to?)

Looks like xen/drivers/char/console.c is not related to this problem?

All the helps are highly appreciated ;-)

Thank you a lot,
Hieu

Re: How console data travel in Xen?

[Anthony Liguori]

NAHieu wrote:

>Hello,
>
>I am trying to investigate in detail how console data go thru the Xen
>system. Suppose that I am running a domainU, when I type at the
>domainU console (I connect to the console with xencons or ssh), how
>data (from keyboard in this case) travel from low level (hardware) to
>Xen, then to dom0, then to domU?
>  
>
There exists a well-known event channel and shared memory page between 
each dom0 and domU which the control tools are responsible for setting 
up (in Xend, that's xcs--in VM-Tools, that's xenctld).  The shared 
memory page is defined by the shared_info_frame which is queriable as 
part of getdomaininfo().  The event channel is chosen by the control 
tools and is almost always 1.

The event channel and shared memory page make up the control channel.  
The shared memory page is used as a ring queue for messages (see 
/usr/include/xen/io/domain_controller.h) and the event channel is a 
notification message to say when a new message has arrived.

One of the message types is CMSG_CONSOLE.  It has one message subtype 
CMSG_CONSOLE_DATA.  Console data (input and output) is passed in 60 byte 
chunks using this message type between dom0 and domU.  This is handled 
at drivers/xen/console/console.c in domU (in the kernel tree) and in 
tools/xcs/*.c in dom0 (or xenctld/console.c in VM-Tools).  In VM-Tools, 
the console messages are just put into a queue until a file (usually a 
pty) is associated with the device in which case everything is 
dumped/read from the file.

The Xend/xcs architecture is a bit more complex.  xcs forwards all 
messages over two sockets to Xend.  Xend then dispatches these messages 
internally.  The messages are then read/written to a socket.  The stuff 
that's unique to the console is mostly handled in 
tools/python/xen/xend/server/console.py

Hope this helps.

Regards,

Anthony Liguori

>I am not afraid of reading and hacking the source to understand this,
>but I dont know where to start in the jungle of code. (Which files
>should I pay attention to?)
>
>Looks like xen/drivers/char/console.c is not related to this problem?
>
>All the helps are highly appreciated ;-)
>
>Thank you a lot,
>Hieu
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
>  
>

Re: How console data travel in Xen?

[NAHieu]

On 7/1/05, Anthony Liguori <aliguori@us.ibm.com> wrote:
> NAHieu wrote:
> 
> >Hello,
> >
> >I am trying to investigate in detail how console data go thru the Xen
> >system. Suppose that I am running a domainU, when I type at the
> >domainU console (I connect to the console with xencons or ssh), how
> >data (from keyboard in this case) travel from low level (hardware) to
> >Xen, then to dom0, then to domU?
> >
> >
> There exists a well-known event channel and shared memory page between
> each dom0 and domU which the control tools are responsible for setting
> up (in Xend, that's xcs--in VM-Tools, that's xenctld).  The shared
> memory page is defined by the shared_info_frame which is queriable as
> part of getdomaininfo().  The event channel is chosen by the control
> tools and is almost always 1.
> 
> The event channel and shared memory page make up the control channel.
> The shared memory page is used as a ring queue for messages (see
> /usr/include/xen/io/domain_controller.h) and the event channel is a
> notification message to say when a new message has arrived.
> 
> One of the message types is CMSG_CONSOLE.  It has one message subtype
> CMSG_CONSOLE_DATA.  Console data (input and output) is passed in 60 byte
> chunks using this message type between dom0 and domU.  This is handled
> at drivers/xen/console/console.c in domU (in the kernel tree) and in
> tools/xcs/*.c in dom0 (or xenctld/console.c in VM-Tools).  In VM-Tools,
> the console messages are just put into a queue until a file (usually a
> pty) is associated with the device in which case everything is
> dumped/read from the file.
> 
> The Xend/xcs architecture is a bit more complex.  xcs forwards all
> messages over two sockets to Xend.  Xend then dispatches these messages
> internally.  The messages are then read/written to a socket.  The stuff
> that's unique to the console is mostly handled in
> tools/python/xen/xend/server/console.py

Anthony, thank you for a great detail explaination. These information
is really helpful to me.

But still few questions stand: how do console data go from HW to Xen,
then to dom0? (the above explaination is only about sessions between
dom0 and domU)

Thank you a lot,
Hieu

Re: How console data travel in Xen?

[Anthony Liguori]

NAHieu wrote:

>Anthony, thank you for a great detail explaination. These information
>is really helpful to me.
>  
>
no problem :-)

>But still few questions stand: how do console data go from HW to Xen,
>then to dom0? (the above explaination is only about sessions between
>dom0 and domU)
>  
>
Ok, so let's say you enter some keyboard data.  Those keyboard actions 
are going to go to dom0.  If you're typing at a console it's getting 
intercepted by some program that's push that keyboard data to Xend let's 
say.  Xend takes that data and splits it up into CMSG_CONSOLE messages 
which it then passes through the previously described mechanism to domU.

Then domU sees that data as input on a virtual serial device (the domU 
kernel's been hacked to use the serial device by default instead of the 
typical virtual terminals).  Then when domU decides it wants to display 
that data back to the user, it sends it back to dom0 again through the 
previously described channel.

Xend ends up getting those messages and assembles it into a string which 
it then displays back on the console.

This is probably a bit clearer if you look at the VM-Tools source tree.  
Everything is pretty much confined to tools/vm-console.c and 
xenctld/console.c.  Xend does pretty much the same thing but there's a 
bit more indirection involved.

Regards,

Anthony Liguori

>Thank you a lot,
>Hieu
>
>  
>

Re: How console data travel in Xen?

[Anthony Liguori]

NAHieu wrote:

>Oops, probably you missed my point: I want to know how the data go
>from low level (HW) to Xen (hypervisor), then to dom0.
>  
>
I was avoiding answering this because I'm sure that someone who's much 
smarter than me is going to jump in and correct me on this :-)

dom0 takes the keyboard interrupts just like it normally would.  I 
believe (and I may be wrong here) that in x86 the IDT is setup such that 
keyboard interrupts go directly to the dom0 kernel (since it's in ring 1).

I would imagine that in x86-64 that those interrupts have to be 
reflected back down to the guest OS.  This is almost pure speculation 
though so hopefully someone else will jump in and correct me.

>I guess it works this way: dom0 has privileged right to access to HW,
>so it can take keyboard code directly (I guess it goes thru Xen, but
>dont know exactly how?). Then from dom to domU, it works like you
>explained. Is that correct?
>
>Many thanks, Anthony.
>Hieu.
>
>  
>

RE: How console data travel in Xen?

[Li, Xin B]

NAHieu wrote:
> Hello,
> 
> I am trying to investigate in detail how console data go
> thru the Xen system. Suppose that I am running a domainU,
> when I type at the domainU console (I connect to the
> console with xencons or ssh), how data (from keyboard in
> this case) travel from low level (hardware) to Xen, then
> to dom0, then to domU? 
> 
> I am not afraid of reading and hacking the source to
> understand this, but I dont know where to start in the
> jungle of code. (Which files should I pay attention to?)
> 
> Looks like xen/drivers/char/console.c is not related to
> this problem? 

Xcs.c should help!

> 
> All the helps are highly appreciated ;-)
> 
> Thank you a lot,
> Hieu
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

How console data travel in Xen?

[NAHieu]

Hello,

I am trying to investigate in detail how console data go thru the Xen
system. Suppose that I am running a domainU, when I type at the
domainU console (I connect to the console with xencons or ssh), how
data (from keyboard in this case) travel from low level (hardware) to
Xen, then to dom0, then to domU?

I am not afraid of reading and hacking the source to understand this,
but I dont know where to start in the jungle of code. (Which files
should I pay attention to?)

Looks like xen/drivers/char/console.c is not related to this problem?

All the helps are highly appreciated ;-)

Thank you a lot,
Hieu