[RFC] Lightweight socket communication between domains in a single machine

[RFC] Lightweight socket communication between domains in a single machine

[김강호]

[-- Attachment #1.1: Type: text/plain, Size: 1723 bytes --]

 Hi, all

I'd like to introduce a socket communication facility optimized for sending
and receiving data between domains in a single machine, named Xway.
Xway is a network module that enables socket programs running on virtual
machines in a single machine to communicate each other efficiently though
socket interface. Xway shows better socket communication latency and
bandwidth than the native socket of Xen. Xway also provides binary
compatiblity for socket interface, so that the socket programs don't have to
be recompiled to exploit the advantages of Xway.
We have tested the binary compatiblity for a couple of programs such as ssh,
vsftp, proftp, apache, and mysql.
Those programs works well without any recompilication or modification to the
source codes as we expected.

The main schemes of Xway are as follows:
1) shared memory between domains
2) bypassing TCP/IP and Domain 0
3) dual channel approach dividing data send/receive and others on a socket

The implementation of Xway can be found at
http://sourceforge.net/projects/xway.
It consists of  kernel patch, a kernel module, and a user daemon.

Please evaluate Xway and comments on it .
Feel free to contact me if you meet problems when installing and evaluating
it.


Thank you  in advance.


----------------------------------------------------

Kang-ho Kim

Senior Member of Engineering Staff

System Software Research Team/Internet Server Group
Digital Home Research Division/Electronics and Telecommunications Research
Institute

161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
Phone : 82-42-860-6226

Fax : 82-42-860-6699
Mobile : 82-10-4433-8538

E-mail : khk@etri.re.kr, sunnykuz@gmail.com
----------------------------------------------------

[-- Attachment #1.2: Type: text/html, Size: 2806 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: [RFC] Lightweight socket communication between domainsin a single machine

[Ian Pratt]

> I'd like to introduce a socket communication facility optimized for
> sending and receiving data between domains in a single machine, named
> Xway.

Cool.
Please can you write a few notes on the implementation, comparing and
contrasting to XenSocket:
http://www.xensource.com/files/xensummit_4/SuzanneMcIntosh_XenSummit_200
7.pdf

I'd like to get a reliable inter-domain byte stream protocol in to
mainline Xen. Ideally, there'd be a story on remoting it if the VM's got
migrated to different nodes e.g. tunnelled over TCP or SDP. 

Ian

> Xway is a network module that enables socket programs running on
> virtual machines in a single machine to communicate each other
> efficiently though socket interface. Xway shows better socket
> communication latency and bandwidth than the native socket of Xen.
Xway
> also provides binary compatiblity for socket interface, so that the
> socket programs don't have to be recompiled to exploit the advantages
> of Xway.
> We have tested the binary compatiblity for a couple of programs such
as
> ssh, vsftp, proftp, apache, and mysql.
> Those programs works well without any recompilication or modification
> to the source codes as we expected.
> 
> The main schemes of Xway are as follows:
> 1) shared memory between domains
> 2) bypassing TCP/IP and Domain 0
> 3) dual channel approach dividing data send/receive and others on a
> socket
> 
> The implementation of Xway can be found at
> http://sourceforge.net/projects/xway.
> It consists of  kernel patch, a kernel module, and a user daemon.
> 
> Please evaluate Xway and comments on it .
> Feel free to contact me if you meet problems when installing and
> evaluating it.
> 
> 
> Thank you  in advance.
> 
> 
> ----------------------------------------------------
> 
> Kang-ho Kim
> 
> Senior Member of Engineering Staff
> 
> System Software Research Team/Internet Server Group
> Digital Home Research Division/Electronics and Telecommunications
> Research Institute
> 
> 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
> Phone : 82-42-860-6226
> 
> Fax : 82-42-860-6699
> Mobile : 82-10-4433-8538
> 
> E-mail : khk@etri.re.kr, sunnykuz@gmail.com
> ----------------------------------------------------

Re: [RFC] Lightweight socket communication between domainsin a single machine

[김강호]

[-- Attachment #1.1: Type: text/plain, Size: 2849 bytes --]

Dear Ian Pratt.

Thanks for your attention.

I attached a PPT file that has some note on the implementation, comparing to
XenSocket.
In each slide of the PPT, I wrote a few lines of slide note. Please read the
note for more details.
Since I don't know much about XenSocket I comapred only a few features that
I'm interested in.

I know that this summary slide is not enough to understand the whole things
of Xway.
However, let's start discussion on Xway as well as XenSocket.


2007/5/1, Ian Pratt <Ian.Pratt@cl.cam.ac.uk>:
>
> > I'd like to introduce a socket communication facility optimized for
> > sending and receiving data between domains in a single machine, named
> > Xway.
>
> Cool.
> Please can you write a few notes on the implementation, comparing and
> contrasting to XenSocket:
> http://www.xensource.com/files/xensummit_4/SuzanneMcIntosh_XenSummit_200
> 7.pdf
>
> I'd like to get a reliable inter-domain byte stream protocol in to
> mainline Xen. Ideally, there'd be a story on remoting it if the VM's got
> migrated to different nodes e.g. tunnelled over TCP or SDP.
>
> Ian
>
> > Xway is a network module that enables socket programs running on
> > virtual machines in a single machine to communicate each other
> > efficiently though socket interface. Xway shows better socket
> > communication latency and bandwidth than the native socket of Xen.
> Xway
> > also provides binary compatiblity for socket interface, so that the
> > socket programs don't have to be recompiled to exploit the advantages
> > of Xway.
> > We have tested the binary compatiblity for a couple of programs such
> as
> > ssh, vsftp, proftp, apache, and mysql.
> > Those programs works well without any recompilication or modification
> > to the source codes as we expected.
> >
> > The main schemes of Xway are as follows:
> > 1) shared memory between domains
> > 2) bypassing TCP/IP and Domain 0
> > 3) dual channel approach dividing data send/receive and others on a
> > socket
> >
> > The implementation of Xway can be found at
> > http://sourceforge.net/projects/xway.
> > It consists of  kernel patch, a kernel module, and a user daemon.
> >
> > Please evaluate Xway and comments on it .
> > Feel free to contact me if you meet problems when installing and
> > evaluating it.
> >
> >
> > Thank you  in advance.
> >
> >
> > ----------------------------------------------------
> >
> > Kang-ho Kim
> >
> > Senior Member of Engineering Staff
> >
> > System Software Research Team/Internet Server Group
> > Digital Home Research Division/Electronics and Telecommunications
> > Research Institute
> >
> > 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
> > Phone : 82-42-860-6226
> >
> > Fax : 82-42-860-6699
> > Mobile : 82-10-4433-8538
> >
> > E-mail : khk@etri.re.kr, sunnykuz@gmail.com
> > ----------------------------------------------------
>
>

[-- Attachment #1.2: Type: text/html, Size: 3707 bytes --]

[-- Attachment #2: xway-intro-summary2.pdf --]
[-- Type: application/pdf, Size: 253574 bytes --]

[-- Attachment #3: xway-intro-summary2.ppt --]
[-- Type: application/vnd.ms-powerpoint, Size: 229376 bytes --]

[-- Attachment #4: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] Lightweight socket communication between domainsin a single machine

[김강호]

[-- Attachment #1.1: Type: text/plain, Size: 2849 bytes --]

Dear Ian Pratt.

Thanks for your attention.

I attached a PPT file that has some note on the implementation, comparing to
XenSocket.
In each slide of the PPT, I wrote a few lines of slide note. Please read the
note for more details.
Since I don't know much about XenSocket I comapred only a few features that
I'm interested in.

I know that this summary slide is not enough to understand the whole things
of Xway.
However, let's start discussion on Xway as well as XenSocket.


2007/5/1, Ian Pratt <Ian.Pratt@cl.cam.ac.uk>:
>
> > I'd like to introduce a socket communication facility optimized for
> > sending and receiving data between domains in a single machine, named
> > Xway.
>
> Cool.
> Please can you write a few notes on the implementation, comparing and
> contrasting to XenSocket:
> http://www.xensource.com/files/xensummit_4/SuzanneMcIntosh_XenSummit_200
> 7.pdf
>
> I'd like to get a reliable inter-domain byte stream protocol in to
> mainline Xen. Ideally, there'd be a story on remoting it if the VM's got
> migrated to different nodes e.g. tunnelled over TCP or SDP.
>
> Ian
>
> > Xway is a network module that enables socket programs running on
> > virtual machines in a single machine to communicate each other
> > efficiently though socket interface. Xway shows better socket
> > communication latency and bandwidth than the native socket of Xen.
> Xway
> > also provides binary compatiblity for socket interface, so that the
> > socket programs don't have to be recompiled to exploit the advantages
> > of Xway.
> > We have tested the binary compatiblity for a couple of programs such
> as
> > ssh, vsftp, proftp, apache, and mysql.
> > Those programs works well without any recompilication or modification
> > to the source codes as we expected.
> >
> > The main schemes of Xway are as follows:
> > 1) shared memory between domains
> > 2) bypassing TCP/IP and Domain 0
> > 3) dual channel approach dividing data send/receive and others on a
> > socket
> >
> > The implementation of Xway can be found at
> > http://sourceforge.net/projects/xway.
> > It consists of  kernel patch, a kernel module, and a user daemon.
> >
> > Please evaluate Xway and comments on it .
> > Feel free to contact me if you meet problems when installing and
> > evaluating it.
> >
> >
> > Thank you  in advance.
> >
> >
> > ----------------------------------------------------
> >
> > Kang-ho Kim
> >
> > Senior Member of Engineering Staff
> >
> > System Software Research Team/Internet Server Group
> > Digital Home Research Division/Electronics and Telecommunications
> > Research Institute
> >
> > 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
> > Phone : 82-42-860-6226
> >
> > Fax : 82-42-860-6699
> > Mobile : 82-10-4433-8538
> >
> > E-mail : khk@etri.re.kr, sunnykuz@gmail.com
> > ----------------------------------------------------
>
>

[-- Attachment #1.2: Type: text/html, Size: 3707 bytes --]

[-- Attachment #2: xway-intro-summary2.pdf --]
[-- Type: application/pdf, Size: 253574 bytes --]

[-- Attachment #3: xway-intro-summary2.ppt --]
[-- Type: application/vnd.ms-powerpoint, Size: 229376 bytes --]

[-- Attachment #4: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] Lightweight socket communication between domainsin a single machine

[Mark Williamson]

Hi there,

> I attached a PPT file that has some note on the implementation, comparing
> to XenSocket.
> In each slide of the PPT, I wrote a few lines of slide note. Please read
> the note for more details.

So, to summarise, the main points of your approach are:

1) shared memory data transfers for streams of data - effectively a 
Xen "device" channel per stream
2) use existing TCP syscall interface and protocol code to handle connection 
setup and teardown (+ calls into additional setup / teardown code for Xway 
itself, of course)
3) allow TCP-oriented applications to leverage Xway without any modification 
whatsoever, using the same API as usual

It looks like a nice side effect of your strategy of leveraging TCP is that 
you could perhaps modify your "Xway switch" to plumb the data stream either 
through the TCP stack, or the Xway shared memory at will - allowing you to 
fall back to TCP when migrating off a host, and allowing you to transparently 
switch to Xway when migrating to be colocated with the server you're talking 
to.

You seem to get some impressive throughput numbers when working on shared 
memory ;-)  Do you think there are any further optimisations that you could 
do here?

I'm curious as to how you handle addressing...  Is an IP-address->local domain 
id lookup process happening somewhere?  How does this work?  Can any TCP 
connection take advantage of Xway, or is some advance configuration required?

> Since I don't know much about XenSocket I comapred only a few features that
> I'm interested in.

Perhaps the XenSocket folks can comment more on this.  It would be good to 
identify how much commonality in code / purpose there are between the two 
projects.

> I know that this summary slide is not enough to understand the whole things
> of Xway.
> However, let's start discussion on Xway as well as XenSocket.

-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: [RFC] Lightweight socket communication between domainsin a single machine

[Zhu Han]

[-- Attachment #1: Type: text/plain, Size: 3383 bytes --]

HI,

I'm quite curious how Xway find out whether the peer is on the same
host or on another host? Does the switch provide such capability?

On 5/3/07, 김강호 <sunnykuz@gmail.com> wrote:
> Dear Ian Pratt.
>
> Thanks for your attention.
>
> I attached a PPT file that has some note on the implementation, comparing to
> XenSocket.
> In each slide of the PPT, I wrote a few lines of slide note. Please read the
> note for more details.
> Since I don't know much about XenSocket I comapred only a few features that
> I'm interested in.
>
> I know that this summary slide is not enough to understand the whole things
> of Xway.
> However, let's start discussion on Xway as well as XenSocket.
>
>
> 2007/5/1, Ian Pratt <Ian.Pratt@cl.cam.ac.uk>:
> > > I'd like to introduce a socket communication facility optimized for
> > > sending and receiving data between domains in a single machine, named
> > > Xway.
> >
> > Cool.
> > Please can you write a few notes on the implementation, comparing and
> > contrasting to XenSocket:
> >
> http://www.xensource.com/files/xensummit_4/SuzanneMcIntosh_XenSummit_200
> > 7.pdf
> >
> > I'd like to get a reliable inter-domain byte stream protocol in to
> > mainline Xen. Ideally, there'd be a story on remoting it if the VM's got
> > migrated to different nodes e.g. tunnelled over TCP or SDP.
> >
> > Ian
> >
> > > Xway is a network module that enables socket programs running on
> > > virtual machines in a single machine to communicate each other
> > > efficiently though socket interface. Xway shows better socket
> > > communication latency and bandwidth than the native socket of Xen.
> > Xway
> > > also provides binary compatiblity for socket interface, so that the
> > > socket programs don't have to be recompiled to exploit the advantages
> > > of Xway.
> > > We have tested the binary compatiblity for a couple of programs such
> > as
> > > ssh, vsftp, proftp, apache, and mysql.
> > > Those programs works well without any recompilication or modification
> > > to the source codes as we expected.
> > >
> > > The main schemes of Xway are as follows:
> > > 1) shared memory between domains
> > > 2) bypassing TCP/IP and Domain 0
> > > 3) dual channel approach dividing data send/receive and others on a
> > > socket
> > >
> > > The implementation of Xway can be found at
> > > http://sourceforge.net/projects/xway.
> > > It consists of  kernel patch, a kernel module, and a user daemon.
> > >
> > > Please evaluate Xway and comments on it .
> > > Feel free to contact me if you meet problems when installing and
> > > evaluating it.
> > >
> > >
> > > Thank you  in advance.
> > >
> > >
> > > ----------------------------------------------------
> > >
> > > Kang-ho Kim
> > >
> > > Senior Member of Engineering Staff
> > >
> > > System Software Research Team/Internet Server Group
> > > Digital Home Research Division/Electronics and Telecommunications
> > > Research Institute
> > >
> > > 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
> > > Phone : 82-42-860-6226
> > >
> > > Fax : 82-42-860-6699
> > > Mobile : 82-10-4433-8538
> > >
> > > E-mail : khk@etri.re.kr, sunnykuz@gmail.com
> > > ----------------------------------------------------
> >
> >
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
>
>


-- 
best regards,
hanzhu

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] Lightweight socket communication between domainsin a single machine

[kangho kim]

[-- Attachment #1.1: Type: text/plain, Size: 4078 bytes --]

Yes. the switch layer has to know whether the peer is in the same host or
not.
In the current implementation, version 0.6, we assume that Xen administrator
knows the IP address of
each domain in the host, and he/she gives the information manually to the
swith layers residing every domain that wishes to use Xway. However, that
manual chore could be replaced with automatic way without difficulty.

best regards,
kangho kim.

2007/5/9, Zhu Han <schumi.han@gmail.com>:
>
> HI,
>
> I'm quite curious how Xway find out whether the peer is on the same
> host or on another host? Does the switch provide such capability?
>
> On 5/3/07, 김강호 <sunnykuz@gmail.com> wrote:
> > Dear Ian Pratt.
> >
> > Thanks for your attention.
> >
> > I attached a PPT file that has some note on the implementation,
> comparing to
> > XenSocket.
> > In each slide of the PPT, I wrote a few lines of slide note. Please read
> the
> > note for more details.
> > Since I don't know much about XenSocket I comapred only a few features
> that
> > I'm interested in.
> >
> > I know that this summary slide is not enough to understand the whole
> things
> > of Xway.
> > However, let's start discussion on Xway as well as XenSocket.
> >
> >
> > 2007/5/1, Ian Pratt <Ian.Pratt@cl.cam.ac.uk>:
> > > > I'd like to introduce a socket communication facility optimized for
> > > > sending and receiving data between domains in a single machine,
> named
> > > > Xway.
> > >
> > > Cool.
> > > Please can you write a few notes on the implementation, comparing and
> > > contrasting to XenSocket:
> > >
> > http://www.xensource.com/files/xensummit_4/SuzanneMcIntosh_XenSummit_200
> > > 7.pdf
> > >
> > > I'd like to get a reliable inter-domain byte stream protocol in to
> > > mainline Xen. Ideally, there'd be a story on remoting it if the VM's
> got
> > > migrated to different nodes e.g. tunnelled over TCP or SDP.
> > >
> > > Ian
> > >
> > > > Xway is a network module that enables socket programs running on
> > > > virtual machines in a single machine to communicate each other
> > > > efficiently though socket interface. Xway shows better socket
> > > > communication latency and bandwidth than the native socket of Xen.
> > > Xway
> > > > also provides binary compatiblity for socket interface, so that the
> > > > socket programs don't have to be recompiled to exploit the
> advantages
> > > > of Xway.
> > > > We have tested the binary compatiblity for a couple of programs such
> > > as
> > > > ssh, vsftp, proftp, apache, and mysql.
> > > > Those programs works well without any recompilication or
> modification
> > > > to the source codes as we expected.
> > > >
> > > > The main schemes of Xway are as follows:
> > > > 1) shared memory between domains
> > > > 2) bypassing TCP/IP and Domain 0
> > > > 3) dual channel approach dividing data send/receive and others on a
> > > > socket
> > > >
> > > > The implementation of Xway can be found at
> > > > http://sourceforge.net/projects/xway.
> > > > It consists of  kernel patch, a kernel module, and a user daemon.
> > > >
> > > > Please evaluate Xway and comments on it .
> > > > Feel free to contact me if you meet problems when installing and
> > > > evaluating it.
> > > >
> > > >
> > > > Thank you  in advance.
> > > >
> > > >
> > > > ----------------------------------------------------
> > > >
> > > > Kang-ho Kim
> > > >
> > > > Senior Member of Engineering Staff
> > > >
> > > > System Software Research Team/Internet Server Group
> > > > Digital Home Research Division/Electronics and Telecommunications
> > > > Research Institute
> > > >
> > > > 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
> > > > Phone : 82-42-860-6226
> > > >
> > > > Fax : 82-42-860-6699
> > > > Mobile : 82-10-4433-8538
> > > >
> > > > E-mail : khk@etri.re.kr, sunnykuz@gmail.com
> > > > ----------------------------------------------------
> > >
> > >
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> >
> >
>
>
> --
> best regards,
> hanzhu
>

[-- Attachment #1.2: Type: text/html, Size: 5609 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] Lightweight socket communication between domainsin a single machine

[Mark Williamson]

> Yes. the switch layer has to know whether the peer is in the same host or
> not.
> In the current implementation, version 0.6, we assume that Xen
> administrator knows the IP address of
> each domain in the host, and he/she gives the information manually to the
> swith layers residing every domain that wishes to use Xway. However, that
> manual chore could be replaced with automatic way without difficulty.

Via Xenstore, presumably.  This shouldn't be difficult...

You'd need a way to figure out what IP addresses the domains were using in the 
case of bridging but you can handle this in dom0 I expect.

Cheers,
Mark

> best regards,
> kangho kim.
>
> 2007/5/9, Zhu Han <schumi.han@gmail.com>:
> > HI,
> >
> > I'm quite curious how Xway find out whether the peer is on the same
> > host or on another host? Does the switch provide such capability?
> >
> > On 5/3/07, 김강호 <sunnykuz@gmail.com> wrote:
> > > Dear Ian Pratt.
> > >
> > > Thanks for your attention.
> > >
> > > I attached a PPT file that has some note on the implementation,
> >
> > comparing to
> >
> > > XenSocket.
> > > In each slide of the PPT, I wrote a few lines of slide note. Please
> > > read
> >
> > the
> >
> > > note for more details.
> > > Since I don't know much about XenSocket I comapred only a few features
> >
> > that
> >
> > > I'm interested in.
> > >
> > > I know that this summary slide is not enough to understand the whole
> >
> > things
> >
> > > of Xway.
> > > However, let's start discussion on Xway as well as XenSocket.
> > >
> > > 2007/5/1, Ian Pratt <Ian.Pratt@cl.cam.ac.uk>:
> > > > > I'd like to introduce a socket communication facility optimized for
> > > > > sending and receiving data between domains in a single machine,
> >
> > named
> >
> > > > > Xway.
> > > >
> > > > Cool.
> > > > Please can you write a few notes on the implementation, comparing and
> > > > contrasting to XenSocket:
> > >
> > > http://www.xensource.com/files/xensummit_4/SuzanneMcIntosh_XenSummit_20
> > >0
> > >
> > > > 7.pdf
> > > >
> > > > I'd like to get a reliable inter-domain byte stream protocol in to
> > > > mainline Xen. Ideally, there'd be a story on remoting it if the VM's
> >
> > got
> >
> > > > migrated to different nodes e.g. tunnelled over TCP or SDP.
> > > >
> > > > Ian
> > > >
> > > > > Xway is a network module that enables socket programs running on
> > > > > virtual machines in a single machine to communicate each other
> > > > > efficiently though socket interface. Xway shows better socket
> > > > > communication latency and bandwidth than the native socket of Xen.
> > > >
> > > > Xway
> > > >
> > > > > also provides binary compatiblity for socket interface, so that the
> > > > > socket programs don't have to be recompiled to exploit the
> >
> > advantages
> >
> > > > > of Xway.
> > > > > We have tested the binary compatiblity for a couple of programs
> > > > > such
> > > >
> > > > as
> > > >
> > > > > ssh, vsftp, proftp, apache, and mysql.
> > > > > Those programs works well without any recompilication or
> >
> > modification
> >
> > > > > to the source codes as we expected.
> > > > >
> > > > > The main schemes of Xway are as follows:
> > > > > 1) shared memory between domains
> > > > > 2) bypassing TCP/IP and Domain 0
> > > > > 3) dual channel approach dividing data send/receive and others on a
> > > > > socket
> > > > >
> > > > > The implementation of Xway can be found at
> > > > > http://sourceforge.net/projects/xway.
> > > > > It consists of  kernel patch, a kernel module, and a user daemon.
> > > > >
> > > > > Please evaluate Xway and comments on it .
> > > > > Feel free to contact me if you meet problems when installing and
> > > > > evaluating it.
> > > > >
> > > > >
> > > > > Thank you  in advance.
> > > > >
> > > > >
> > > > > ----------------------------------------------------
> > > > >
> > > > > Kang-ho Kim
> > > > >
> > > > > Senior Member of Engineering Staff
> > > > >
> > > > > System Software Research Team/Internet Server Group
> > > > > Digital Home Research Division/Electronics and Telecommunications
> > > > > Research Institute
> > > > >
> > > > > 161 Gajeong-dong, Yuseong-gu, Daejeon, 305-350 KOREA
> > > > > Phone : 82-42-860-6226
> > > > >
> > > > > Fax : 82-42-860-6699
> > > > > Mobile : 82-10-4433-8538
> > > > >
> > > > > E-mail : khk@etri.re.kr, sunnykuz@gmail.com
> > > > > ----------------------------------------------------
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> >
> > --
> > best regards,
> > hanzhu



-- 
Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
Mark: To answer a question with a question: What use is a skateboard?
Dave: Skateboards have wheels.
Mark: My wheel has a wheel!

Re: [RFC] Lightweight socket communication between domainsin a single machine

[Suzanne McIntosh]

[-- Attachment #1.1: Type: text/plain, Size: 475 bytes --]

We are working on making our XenSocket code publicly available 
to enable the Xen community to test XenSockets in their own
environments. Once that is done, it would be easier to compare 
and contrast the XenSocket and Xway approaches.

Thanks, 
Sue McIntosh

_______________________________________________________________
Suzanne K. McIntosh
Security, Privacy, and Extensible Technologies - Internet Security Group
IBM T.J. Watson Research Center
Email: skranjac@us.ibm.com

[-- Attachment #1.2: Type: text/html, Size: 776 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] Lightweight socket communication between domainsin a single machine

[kangho kim]

[-- Attachment #1.1: Type: text/plain, Size: 4190 bytes --]

Hi, there.

2007/5/8, Mark Williamson <mark.williamson@cl.cam.ac.uk>:
>
> Hi there,
>
> > I attached a PPT file that has some note on the implementation,
> comparing
> > to XenSocket.
> > In each slide of the PPT, I wrote a few lines of slide note. Please read
> > the note for more details.
>
> So, to summarise, the main points of your approach are:
>
> 1) shared memory data transfers for streams of data - effectively a
> Xen "device" channel per stream
> 2) use existing TCP syscall interface and protocol code to handle
> connection
> setup and teardown (+ calls into additional setup / teardown code for Xway
> itself, of course)

3) allow TCP-oriented applications to leverage Xway without any modification
> whatsoever, using the same API as usual


It looks like a nice side effect of your strategy of leveraging TCP is that
> you could perhaps modify your "Xway switch" to plumb the data stream
> either
> through the TCP stack, or the Xway shared memory at will - allowing you to
> fall back to TCP when migrating off a host, and allowing you to
> transparently
> switch to Xway when migrating to be colocated with the server you're
> talking
> to.


I agree with you that fortunately, it's easy for Xway to adjust the
migration though we didn't consider the migration when designing it. The key
point that Xway need to deal with for the migration is how Xway in each
domain in a communication channel become aware that  your domain or the peer
domain is migrated. Once Xway detects the migration, the rest is simple as
you commented.


> You seem to get some impressive throughput numbers when working on shared
> memory ;-)  Do you think there are any further optimisations that you
> could
> do here?


In the current design & implementation, Xway sends user data to the peer
whenever user calls send request. The naive send machanism can be improved,
so that we could expect higher bandwidth than the current. We have no idea
above this improvement by now.

I'm curious as to how you handle addressing...  Is an IP-address->local
> domain
> id lookup process happening somewhere?  How does this work?  Can any TCP
> connection take advantage of Xway, or is some advance configuration
> required?


Xway maintains only IP addresses of domains that are in the same host, but
does not IP-address->local domain id mapping. To know how Xway handles
addressing, you need to know what is going on when a connection is
established. I'll show you the detail steps concerning the connection
establishment.

Suppose domains, D1 and D2 are in the same host, and socket application A1
in D1 attempts to connect socket appliation A2 in D2. Xway executes
following tasks:
1) creates TCP session, C1, as if there were no Xway.
2) detects that the peer in the connection is in the same host by
referencing IP address table in the current implementation.
3) creates another TCP session, C2,  between Xway in D1 and the helper
daemon living in D2. C2 is supposed to bypass Xway module.
4) sends to D2 through C2, domain id of D1, grant references of shared
memory and port number of event channel that D1 created.
4) receives from D2 through C2, domain id of D2, grant references of shared
memory that D2 ceated.
5) creates Xway session which will be bound to C1, with the initial setup
information.
6) destroys C2.

As you see in the steps above, Xway in D1 sends its domain id to D2, and
become to know domain id of peer by querying the helper daemon in D2. This
is IP-address->domain id lookup process that you want to know I guess.

> Since I don't know much about XenSocket I comapred only a few features
> that
> > I'm interested in.
>
> Perhaps the XenSocket folks can comment more on this.  It would be good to
> identify how much commonality in code / purpose there are between the two
> projects.
>
> > I know that this summary slide is not enough to understand the whole
> things
> > of Xway.
> > However, let's start discussion on Xway as well as XenSocket.
>
> --
> Dave: Just a question. What use is a unicyle with no seat?  And no pedals!
>
> Mark: To answer a question with a question: What use is a skateboard?
> Dave: Skateboards have wheels.
> Mark: My wheel has a wheel!
>

[-- Attachment #1.2: Type: text/html, Size: 5676 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] Lightweight socket communication between domainsin a single machine

[tgh]

hi
xensocket and xway, what are the differences between them ,in the 
mechanism ,functionality ,or performance ,etc..?
and what about the xen interdomain communication mechanism ?and are xway 
or xensocket or some mixed thing or something else adopted into the xen 
or not ?

Thanks in advance


kangho kim 写道:
> Hi, there.
> 2007/5/8, Mark Williamson <mark.williamson@cl.cam.ac.uk 
> <mailto:mark.williamson@cl.cam.ac.uk>>:
>
>     Hi there,
>
>     > I attached a PPT file that has some note on the implementation,
>     comparing
>     > to XenSocket.
>     > In each slide of the PPT, I wrote a few lines of slide note.
>     Please read
>     > the note for more details.
>
>     So, to summarise, the main points of your approach are:
>
>     1) shared memory data transfers for streams of data - effectively a
>     Xen "device" channel per stream
>     2) use existing TCP syscall interface and protocol code to handle
>     connection
>     setup and teardown (+ calls into additional setup / teardown code
>     for Xway
>     itself, of course) 
>
>     3) allow TCP-oriented applications to leverage Xway without any
>     modification
>     whatsoever, using the same API as usual 
>
>
>     It looks like a nice side effect of your strategy of leveraging
>     TCP is that
>     you could perhaps modify your "Xway switch" to plumb the data
>     stream either
>     through the TCP stack, or the Xway shared memory at will -
>     allowing you to
>     fall back to TCP when migrating off a host, and allowing you to
>     transparently
>     switch to Xway when migrating to be colocated with the server
>     you're talking
>     to.
>
> I agree with you that fortunately, it's easy for Xway to adjust the 
> migration though we didn't consider the migration when designing it. 
> The key point that Xway need to deal with for the migration is how 
> Xway in each domain in a communication channel become aware that your 
> domain or the peer domain is migrated. Once Xway detects the 
> migration, the rest is simple as you commented.
>
>     You seem to get some impressive throughput numbers when working on
>     shared
>     memory ;-) Do you think there are any further optimisations that
>     you could
>     do here?
>
> In the current design & implementation, Xway sends user data to the 
> peer whenever user calls send request. The naive send machanism can be 
> improved, so that we could expect higher bandwidth than the current. 
> We have no idea above this improvement by now.
>
>     I'm curious as to how you handle addressing... Is an
>     IP-address->local domain
>     id lookup process happening somewhere? How does this work? Can any
>     TCP
>     connection take advantage of Xway, or is some advance
>     configuration required? 
>
> Xway maintains only IP addresses of domains that are in the same host, 
> but does not IP-address->local domain id mapping. To know how Xway 
> handles addressing, you need to know what is going on when a 
> connection is established. I'll show you the detail steps concerning 
> the connection establishment.
> Suppose domains, D1 and D2 are in the same host, and socket 
> application A1 in D1 attempts to connect socket appliation A2 in D2. 
> Xway executes following tasks:
> 1) creates TCP session, C1, as if there were no Xway.
> 2) detects that the peer in the connection is in the same host by 
> referencing IP address table in the current implementation.
> 3) creates another TCP session, C2, between Xway in D1 and the helper 
> daemon living in D2. C2 is supposed to bypass Xway module.
> 4) sends to D2 through C2, domain id of D1, grant references of shared 
> memory and port number of event channel that D1 created.
> 4) receives from D2 through C2, domain id of D2, grant references of 
> shared memory that D2 ceated.
> 5) creates Xway session which will be bound to C1, with the initial 
> setup information.
> 6) destroys C2.
> As you see in the steps above, Xway in D1 sends its domain id to D2, 
> and become to know domain id of peer by querying the helper daemon in 
> D2. This is IP-address->domain id lookup process that you want to know 
> I guess.
>
>     > Since I don't know much about XenSocket I comapred only a few
>     features that
>     > I'm interested in.
>
>     Perhaps the XenSocket folks can comment more on this. It would be
>     good to
>     identify how much commonality in code / purpose there are between
>     the two
>     projects.
>
>     > I know that this summary slide is not enough to understand the
>     whole things
>     > of Xway.
>     > However, let's start discussion on Xway as well as XenSocket.
>
>     --
>     Dave: Just a question. What use is a unicyle with no seat? And no
>     pedals!
>     Mark: To answer a question with a question: What use is a skateboard?
>     Dave: Skateboards have wheels.
>     Mark: My wheel has a wheel!
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>