Improve performance between Network driver domain to guest OS

Improve performance between Network driver domain to guest OS

[shikha thakur]

[-- Attachment #1.1.1: Type: text/plain, Size: 1211 bytes --]

Hi all,
 I am Shikha Thakur pursuing post graduation from Pune University in
Computer Engineering.Recently, I started my research in Xen hypervisor for
my curriculum project.I am enjoying working with Xen. Being newbie i want
to get more familiar with Xen.

So, I request all please tell me that my idea for this research is valid or
not?

Research Topic::
I am working on "Improving Network I/O virtualization of Xen for Cloud
Computing."



Xen Hypervisor ::
 The driver domain based model of Xen for network I/O virtualization
exhibits poor networking performance.Two Experiments are carried out and
they are:-

1)Network Performance analysis has been carried out to analyze the
virtualization over- head and analyze the baseline reception, transmission
and forwarding performance of the VMs(shown in fig 1).

`[image: Inline image 4]


2)Experiment has carried out to find the hardware bottlenecks and it is
observe that the memory latency is the problem for costly driver domain -
virtual machine communication (shown in fig 2).



[image: Inline image 3]Solution::
  As Memory latency is the bottleneck,Can Packet aggregation be applied to
overcome this?

I would be awaiting for the reply.

Many Thanks.

[-- Attachment #1.1.2: Type: text/html, Size: 2650 bytes --]

[-- Attachment #1.2: image.png --]
[-- Type: image/png, Size: 25004 bytes --]

[-- Attachment #1.3: image.png --]
[-- Type: image/png, Size: 24379 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Improve performance between Network driver domain to guest OS

[Wei Liu]

Hello, Shikha

On Sun, Oct 06, 2013 at 11:49:13AM +0530, shikha thakur wrote:
> Hi all,
>  I am Shikha Thakur pursuing post graduation from Pune University in
> Computer Engineering.Recently, I started my research in Xen hypervisor for
> my curriculum project.I am enjoying working with Xen. Being newbie i want
> to get more familiar with Xen.
> 
> So, I request all please tell me that my idea for this research is valid or
> not?
> 
> Research Topic::
> I am working on "Improving Network I/O virtualization of Xen for Cloud
> Computing."
> 
> 

What's your goal? Higher throughput? Lower latency? Do you have any
merit in mind?

> 
> Xen Hypervisor ::
>  The driver domain based model of Xen for network I/O virtualization
> exhibits poor networking performance.Two Experiments are carried out and
> they are:-
> 
> 1)Network Performance analysis has been carried out to analyze the
> virtualization over- head and analyze the baseline reception, transmission
> and forwarding performance of the VMs(shown in fig 1).
> 
> `[image: Inline image 4]
> 
> 
> 2)Experiment has carried out to find the hardware bottlenecks and it is
> observe that the memory latency is the problem for costly driver domain -
> virtual machine communication (shown in fig 2).
> 

Sorry the image quality is too low I am not able parse those graphs. But
if you have any specific questions regarding Xen network drivers in
Linux I will be happy to answer.

> 
> 
> [image: Inline image 3]Solution::
>   As Memory latency is the bottleneck,Can Packet aggregation be applied to
> overcome this?
> 

It depends. AIUI packet aggregation can introduce latency too. The only
way to prove whether it works is to implement it. :-)

Wei.

Re: Improve performance between Network driver domain to guest OS

[George Dunlap]

On Mon, Oct 7, 2013 at 4:57 PM, Wei Liu <wei.liu2@citrix.com> wrote:
> Hello, Shikha
>
> On Sun, Oct 06, 2013 at 11:49:13AM +0530, shikha thakur wrote:
>> Hi all,
>>  I am Shikha Thakur pursuing post graduation from Pune University in
>> Computer Engineering.Recently, I started my research in Xen hypervisor for
>> my curriculum project.I am enjoying working with Xen. Being newbie i want
>> to get more familiar with Xen.
>>
>> So, I request all please tell me that my idea for this research is valid or
>> not?
>>
>> Research Topic::
>> I am working on "Improving Network I/O virtualization of Xen for Cloud
>> Computing."
>>
>>
>
> What's your goal? Higher throughput? Lower latency? Do you have any
> merit in mind?
>
>>
>> Xen Hypervisor ::
>>  The driver domain based model of Xen for network I/O virtualization
>> exhibits poor networking performance.Two Experiments are carried out and
>> they are:-
>>
>> 1)Network Performance analysis has been carried out to analyze the
>> virtualization over- head and analyze the baseline reception, transmission
>> and forwarding performance of the VMs(shown in fig 1).
>>
>> `[image: Inline image 4]
>>
>>
>> 2)Experiment has carried out to find the hardware bottlenecks and it is
>> observe that the memory latency is the problem for costly driver domain -
>> virtual machine communication (shown in fig 2).
>>
>
> Sorry the image quality is too low I am not able parse those graphs.

+1.

 -George

Re: Improve performance between Network driver domain to guest OS

[Wei Liu]

Please don't drop xen-devel.

On Sun, Oct 13, 2013 at 12:12:26PM +0530, shikha thakur wrote:
> Hello,
> I apologized for the late reply.
> 
> >>What's your goal? Higher throughput? Lower latency? Do you have any
> merit in mind?
> 
> *Me: Basically higher throughput is my Aim.
>        **To increase the throughput, the paper i am referencing  analyzed
> the Xen hypervisor and got the memory latency problem.It has been overcome
> by packet aggregation.Thus the higher throughput is achieved.With the
> aggregation mechanism ,the throughput almost increases linearly  with the
> number of VM.(As Shown in Figure below).However, It introduced delay &
> jitter , hence i am trying to either eliminate delay & jitter or find new
> way of increasing throughput.
> 
> *
> 

I skimmed the paper. "Memory latency" seems to refer to the latency
introduced by memory copy. And the solution in that paper might not work
quite well in the scenario when a VM is expecting low latency with small
packets.

> 
> >>It depends. AIUI packet aggregation can introduce latency too. The only
> way to prove whether it works is to implement it.
> 
>  *Me:: It has been experimented and it is introducing delay and jitter.
> *
> *First thing i wanna know, Whether the  maximize delay and jitter is really
> a problem for higher throughput?

Sorry I don't know. you'd better think of the system as a whole.
Eliminating one single bottleneck might actually decrease performance.

> *
> *
> If yes, there are two method :-
> 1) Packet dimensioning tool which dynamically tune the packet
> aggregation.(Refer attached pdf)**
> 2) What i am planning to do is to imply adaptive jitter buffering algorithm
> to dynamically tune the packet aggregation*.
> 
> *If not of much trouble,Could anyone guide me which one of the above 2
> method is better for minimizing delay and jitter.*
> *
> 

Sorry I cannot give you a definitive answer, that's beyond my expertise.
I think the fun of doing research lies in the uncertainty and the thrill
of solving problems. If there's definitive answer to this question,
somebody would have done it already, right?

The only viable way is to hack a prototype IMHO.

Wei.

Re: Improve performance between Network driver domain to guest OS

[Wei Liu]

Please don't drop Xen-devel.

On Sat, Oct 26, 2013 at 11:45:45AM +0530, shikha thakur wrote:
> Hello,
> I am not getting where to do coding for network I/O virtualization. Can you
> please give me some Documentation or any tutorial which can guide me for
> the same?
> 

A starting point would be Linux's netback / netfront drivers. You can
find them in drivers/net/xen-netback/ and drivers/net/xen-netfront.c of
Linux source tree.

Wei.

> Many thanks,
> Shikha thakur
> 
> 
> On Thu, Oct 17, 2013 at 7:07 PM, shikha thakur <srthakur26@gmail.com> wrote:
> 
> >
> >
> > ---------- Forwarded message ----------
> > From: Wei Liu <wei.liu2@citrix.com>
> > Date: Mon, Oct 14, 2013 at 3:59 PM
> > Subject: Re: [Xen-devel] Improve performance between Network driver domain
> > to guest OS
> > To: shikha thakur <srthakur26@gmail.com>
> > Cc: Wei Liu <wei.liu2@citrix.com>, George Dunlap <
> > george.dunlap@eu.citrix.com>, Ritesh Malaiya <ritesh.opensource@gmail.com>,
> > xen-devel@lists.xen.org
> >
> >
> > Please don't drop xen-devel.
> >
> > On Sun, Oct 13, 2013 at 12:12:26PM +0530, shikha thakur wrote:
> > > Hello,
> > > I apologized for the late reply.
> > >
> > > >>What's your goal? Higher throughput? Lower latency? Do you have any
> > > merit in mind?
> > >
> > > *Me: Basically higher throughput is my Aim.
> > >        **To increase the throughput, the paper i am referencing  analyzed
> > > the Xen hypervisor and got the memory latency problem.It has been
> > overcome
> > > by packet aggregation.Thus the higher throughput is achieved.With the
> > > aggregation mechanism ,the throughput almost increases linearly  with the
> > > number of VM.(As Shown in Figure below).However, It introduced delay &
> > > jitter , hence i am trying to either eliminate delay & jitter or find new
> > > way of increasing throughput.
> > >
> > > *
> > >
> >
> > I skimmed the paper. "Memory latency" seems to refer to the latency
> > introduced by memory copy. And the solution in that paper might not work
> > quite well in the scenario when a VM is expecting low latency with small
> > packets.
> >
> > >
> > > >>It depends. AIUI packet aggregation can introduce latency too. The only
> > > way to prove whether it works is to implement it.
> > >
> > >  *Me:: It has been experimented and it is introducing delay and jitter.
> > > *
> > > *First thing i wanna know, Whether the  maximize delay and jitter is
> > really
> > > a problem for higher throughput?
> >
> > Sorry I don't know. you'd better think of the system as a whole.
> > Eliminating one single bottleneck might actually decrease performance.
> >
> > > *
> > > *
> > > If yes, there are two method :-
> > > 1) Packet dimensioning tool which dynamically tune the packet
> > > aggregation.(Refer attached pdf)**
> > > 2) What i am planning to do is to imply adaptive jitter buffering
> > algorithm
> > > to dynamically tune the packet aggregation*.
> > >
> > > *If not of much trouble,Could anyone guide me which one of the above 2
> > > method is better for minimizing delay and jitter.*
> > > *
> > >
> >
> > Sorry I cannot give you a definitive answer, that's beyond my expertise.
> > I think the fun of doing research lies in the uncertainty and the thrill
> > of solving problems. If there's definitive answer to this question,
> > somebody would have done it already, right?
> >
> > The only viable way is to hack a prototype IMHO.
> >
> > Wei.
> >
> >