Request for help, implementing a new network scheduler

Request for help, implementing a new network scheduler

[Ronald PIna]

Hi

I am working for the msc thesis to improve the performance on network 
for guest domains that uses real-time services like voip or video 
streaming servers , i have an idea to implement a network scheduler on 
network backend, the schedulers may be weighted fair queuing or weighted 
round robin,  the idea is to schedule first the packet coming from 
real-time guest services, one of those schedulers could make the job and 
can prioritize the network traffic.

As far as i have studied from previous works explain that the outgoing 
network traffic in xen is scheduled in round robin manner inside the 
function net_tx_action(). Later on the last version of xen this function 
has changed xenvif_tx_action() and have changed his primarily structure. 
My primarily goal is to modify the round robin to more advanced 
scheduler  which introduce priorities. The clear question is that where 
is located the function which schedule in round robin and if possible 
are there any concern about using another network scheduling method in 
netback ?

Thanks in advance

Re: Request for help, implementing a new network scheduler

[Wei Liu]

On Tue, Feb 03, 2015 at 01:42:09AM +0100, Ronald PIna wrote:
> Hi
> 

Hello

> I am working for the msc thesis to improve the performance on network for
> guest domains that uses real-time services like voip or video streaming
> servers , i have an idea to implement a network scheduler on network
> backend, the schedulers may be weighted fair queuing or weighted round
> robin,  the idea is to schedule first the packet coming from real-time guest
> services, one of those schedulers could make the job and can prioritize the
> network traffic.
> 
> As far as i have studied from previous works explain that the outgoing
> network traffic in xen is scheduled in round robin manner inside the
> function net_tx_action(). Later on the last version of xen this function has
> changed xenvif_tx_action() and have changed his primarily structure. My

I take it you're looking at Linux source code because xenvif_tx_action
is in Linux. Keep in mind that there is indeed other implementations of
netback, say, the FreeBSD netback and the ClickOS guys own
implementation.

I think net_tx_action to xenvif_tx_action is mostly a name change. But
the internal model of netback did change drastically in the last few
years. If you want to work on netback you'd better work on upstream
Linux netback.

> primarily goal is to modify the round robin to more advanced scheduler
> which introduce priorities. The clear question is that where is located the
> function which schedule in round robin and if possible are there any concern
> about using another network scheduling method in netback ?
> 

Upstream netback has a 1:1 model, which means every frontend stream has
one serving routine in the backend. And we rely on Linux's scheduler to
make sensible decision.

So if you want to change scheduler you're heading into instrumenting
Linux's scheduler to meet your need. YMMW.

Wei.

> Thanks in advance
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Request for help, implementing a new network scheduler

[Ian Campbell]

On Tue, 2015-02-03 at 11:05 +0000, Wei Liu wrote:
> So if you want to change scheduler you're heading into instrumenting
> Linux's scheduler to meet your need. YMMW.

Right, I'd expect this sort of network traffic scheduling/shaping to be
achievable with the normal Linux facilities for doing such things (i.e.
tc and friends) and if not then it is those which should be enhanced
rather than adding Xen specific stuff to netback (or else a convincing
reason for it being Xen specific should be given).

The same goes for the other possible backend OSes, e.g. *BSDs
shaping/filtering facility, whatever it is called, should be used there.

Ian.

Re: Request for help, implementing a new network scheduler

[ronald pina]

[-- Attachment #1.1: Type: text/plain, Size: 1756 bytes --]

Thanks Ian for your helpful answer, you are right that is on Linux side but
to clear my idea , the goal  is to prioritize traffic that comes from
different guest, for example if guest_1 is used for a voip server than it
would be reasonable to schedule first the vif guest_1 before other guests.
The diagram atached below explains better my idea. The goal is to improve
the latency and jitter for real-time traffic which have critical
requirements to QoS especially on heavy network i/o hosts, and if the vifs
of different guest are scheduled on round robin manner in dom0 backend than
we can improve this scheduling algorithm like using weighted round robin
which can give more scheduling time to a vif that was used by a
 voip-server guest.

As a case considering an i/o heavy load network traffic host, i think it
would be more efficient using a simple scheduler like weighted round robin
or maybe others rather than using tools that may increase overhead .  What
do you think Ian ?

Thanks
Ronald

On Tue, Feb 3, 2015 at 12:12 PM, Ian Campbell <Ian.Campbell@citrix.com>
wrote:

> On Tue, 2015-02-03 at 11:05 +0000, Wei Liu wrote:
> > So if you want to change scheduler you're heading into instrumenting
> > Linux's scheduler to meet your need. YMMW.
>
> Right, I'd expect this sort of network traffic scheduling/shaping to be
> achievable with the normal Linux facilities for doing such things (i.e.
> tc and friends) and if not then it is those which should be enhanced
> rather than adding Xen specific stuff to netback (or else a convincing
> reason for it being Xen specific should be given).
>
> The same goes for the other possible backend OSes, e.g. *BSDs
> shaping/filtering facility, whatever it is called, should be used there.
>
> Ian.
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 2751 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Request for help, implementing a new network scheduler

[Wei Liu]

Add xen-devel back on Cc list.

And please don't top-post.

On Tue, Feb 03, 2015 at 09:26:20PM +0100, ronald pina wrote:
> Thanks Wei, it was useful, my primary goal is to contribute on improving
> the latency and jitter on xen, that is  my purpose not to implement a
> general QoS linux scheduler, the basic idea is  prioritize real-time
> traffic which have critical requirements to QoS especially on heavy network
> i/o hosts, and if the vifs of different guest are scheduled on round robin
> manner in dom0 backend than we can improve this scheduling algorithm like
> using weighted round robin which can give more scheduling time to a vif
> that was used by a  voip-server guest.
> 

As Ian and Zoli suggested, there might be already tools that can achieve
your goal. Your idea to improve latency and jitter  is certainly
valuable, but I would suggest you do through research on existing tools.
That would help you understand the strength and weakness of existing
tools, as well as the strength and weakness of your own idea.

If you have any problem with regard to netback driver, feel free to post
on xen-devel. However I don't have much knowledge on Linux scheduler so
I cannot comment on the round robin and QoS part.  :-)

Wei.

Re: Request for help, implementing a new network scheduler

[Zoltan Kiss]

On 03/02/15 00:42, Ronald PIna wrote:
> Hi
>
> I am working for the msc thesis to improve the performance on network
> for guest domains that uses real-time services like voip or video
> streaming servers , i have an idea to implement a network scheduler on
> network backend, the schedulers may be weighted fair queuing or weighted
> round robin,  the idea is to schedule first the packet coming from
> real-time guest services, one of those schedulers could make the job and
> can prioritize the network traffic.
>
> As far as i have studied from previous works explain that the outgoing
> network traffic in xen is scheduled in round robin manner inside the
> function net_tx_action(). Later on the last version of xen this function
> has changed xenvif_tx_action() and have changed his primarily structure.
> My primarily goal is to modify the round robin to more advanced
> scheduler  which introduce priorities. The clear question is that where
> is located the function which schedule in round robin and if possible
> are there any concern about using another network scheduling method in
> netback ?
xenvif_tx_action is called from NAPI polling context, per queue. It 
copies & maps the packets from the guest to Dom0 (at which point it 
doesn't have the slightest idea about the packet content), formats them 
in xenvif_tx_submit to be a well formed skb, and calls netif_receive_skb 
to hand it over to the stack.
You could try doing some reordering in xenvif_tx_submit or introducing 
priorities in the ring buffer, but I would strongly discourage you to do 
that, for similar reasons mentioned by Ian. You would be better off 
using standard Linux tools for doing that, see 'man tc', and I would try 
on the sending side, in the guest, before netfront gets the packet.
Or, if your idea is to prioritize between traffic coming from separate 
guests, then xenvif_tx_action is definitely the wrong place to do that: 
as said above, it deals only with the traffic of one particular queue of 
a netback device. You might want to look into the entity 
switching/routing between the guests and NICs: Linux bridge module, 
openvswitch, the IP routing engine of the backend etc. I think the first 
two could be the reasonable choices, bridge might lacks such 
functionality though.

Zoli


>
> Thanks in advance
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Request for help, implementing a new network scheduler

[ronald pina]

[-- Attachment #1.1: Type: text/plain, Size: 3404 bytes --]

Thanks Zoli for your clear and very helpfull answer, my idea is to
prioritize traffic that comes from different guest, for example if guest_1
is used for a voip server than it would be reasonable to schedule first the
vif guest_1 before other guests. The diagram atached below explains better
my idea. The goal is to improve the latency and jitter for real-time
traffic which have critical requirements to QoS especially on heavy network
i/o hosts, and if the vifs of different guest are scheduled on round robin
manner in dom0 backend than we can improve this scheduling algorithm like
using weighted round robin which can give more scheduling time to a vif
that was used by a  voip-server guest. I would like to know that which
module or function is responsible for scheduling vif on dom0 ?

Thanks
Ronald

On Tue, Feb 3, 2015 at 12:54 PM, Zoltan Kiss <zoltan.kiss@linaro.org> wrote:

>
>
> On 03/02/15 00:42, Ronald PIna wrote:
>
>> Hi
>>
>> I am working for the msc thesis to improve the performance on network
>> for guest domains that uses real-time services like voip or video
>> streaming servers , i have an idea to implement a network scheduler on
>> network backend, the schedulers may be weighted fair queuing or weighted
>> round robin,  the idea is to schedule first the packet coming from
>> real-time guest services, one of those schedulers could make the job and
>> can prioritize the network traffic.
>>
>> As far as i have studied from previous works explain that the outgoing
>> network traffic in xen is scheduled in round robin manner inside the
>> function net_tx_action(). Later on the last version of xen this function
>> has changed xenvif_tx_action() and have changed his primarily structure.
>> My primarily goal is to modify the round robin to more advanced
>> scheduler  which introduce priorities. The clear question is that where
>> is located the function which schedule in round robin and if possible
>> are there any concern about using another network scheduling method in
>> netback ?
>>
> xenvif_tx_action is called from NAPI polling context, per queue. It copies
> & maps the packets from the guest to Dom0 (at which point it doesn't have
> the slightest idea about the packet content), formats them in
> xenvif_tx_submit to be a well formed skb, and calls netif_receive_skb to
> hand it over to the stack.
> You could try doing some reordering in xenvif_tx_submit or introducing
> priorities in the ring buffer, but I would strongly discourage you to do
> that, for similar reasons mentioned by Ian. You would be better off using
> standard Linux tools for doing that, see 'man tc', and I would try on the
> sending side, in the guest, before netfront gets the packet.
> Or, if your idea is to prioritize between traffic coming from separate
> guests, then xenvif_tx_action is definitely the wrong place to do that: as
> said above, it deals only with the traffic of one particular queue of a
> netback device. You might want to look into the entity switching/routing
> between the guests and NICs: Linux bridge module, openvswitch, the IP
> routing engine of the backend etc. I think the first two could be the
> reasonable choices, bridge might lacks such functionality though.
>
> Zoli
>
>
>
>
>> Thanks in advance
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>
>

[-- Attachment #1.2: Type: text/html, Size: 4284 bytes --]

[-- Attachment #2: xen-net-arch.jpg --]
[-- Type: image/jpeg, Size: 27732 bytes --]

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Request for help, implementing a new network scheduler

[Zoltan Kiss]

Hi,

On 03/02/15 19:55, ronald pina wrote:
> Thanks Zoli for your clear and very helpfull answer, my idea is to
> prioritize traffic that comes from different guest, for example if
> guest_1 is used for a voip server than it would be reasonable to
> schedule first the vif guest_1 before other guests. The diagram atached
> below explains better my idea. The goal is to improve the latency and
> jitter for real-time traffic which have critical requirements to QoS
> especially on heavy network i/o hosts, and if the vifs of different
> guest are scheduled on round robin manner in dom0 backend than we can
> improve this scheduling algorithm like using weighted round robin which
> can give more scheduling time to a vif that was used by a  voip-server
> guest. I would like to know that which module or function is responsible
> for scheduling vif on dom0 ?

As I said, it is a NAPI polling function:

http://www.linuxfoundation.org/collaborate/workgroups/networking/napi

So it is scheduled by NAPI, which doesn't have a clue about what kind of 
packets the device dealing with, therefore can't make a decision based 
on that. You can try manually configure the weight of a device through 
netif_napi_add(), but I strongly recommend to look for other ways I 
mentioned in my previous letter.

>
> Thanks
> Ronald
>
> On Tue, Feb 3, 2015 at 12:54 PM, Zoltan Kiss <zoltan.kiss@linaro.org
> <mailto:zoltan.kiss@linaro.org>> wrote:
>
>
>
>     On 03/02/15 00:42, Ronald PIna wrote:
>
>         Hi
>
>         I am working for the msc thesis to improve the performance on
>         network
>         for guest domains that uses real-time services like voip or video
>         streaming servers , i have an idea to implement a network
>         scheduler on
>         network backend, the schedulers may be weighted fair queuing or
>         weighted
>         round robin,  the idea is to schedule first the packet coming from
>         real-time guest services, one of those schedulers could make the
>         job and
>         can prioritize the network traffic.
>
>         As far as i have studied from previous works explain that the
>         outgoing
>         network traffic in xen is scheduled in round robin manner inside the
>         function net_tx_action(). Later on the last version of xen this
>         function
>         has changed xenvif_tx_action() and have changed his primarily
>         structure.
>         My primarily goal is to modify the round robin to more advanced
>         scheduler  which introduce priorities. The clear question is
>         that where
>         is located the function which schedule in round robin and if
>         possible
>         are there any concern about using another network scheduling
>         method in
>         netback ?
>
>     xenvif_tx_action is called from NAPI polling context, per queue. It
>     copies & maps the packets from the guest to Dom0 (at which point it
>     doesn't have the slightest idea about the packet content), formats
>     them in xenvif_tx_submit to be a well formed skb, and calls
>     netif_receive_skb to hand it over to the stack.
>     You could try doing some reordering in xenvif_tx_submit or
>     introducing priorities in the ring buffer, but I would strongly
>     discourage you to do that, for similar reasons mentioned by Ian. You
>     would be better off using standard Linux tools for doing that, see
>     'man tc', and I would try on the sending side, in the guest, before
>     netfront gets the packet.
>     Or, if your idea is to prioritize between traffic coming from
>     separate guests, then xenvif_tx_action is definitely the wrong place
>     to do that: as said above, it deals only with the traffic of one
>     particular queue of a netback device. You might want to look into
>     the entity switching/routing between the guests and NICs: Linux
>     bridge module, openvswitch, the IP routing engine of the backend
>     etc. I think the first two could be the reasonable choices, bridge
>     might lacks such functionality though.
>
>     Zoli
>
>
>
>
>         Thanks in advance
>
>         _________________________________________________
>         Xen-devel mailing list
>         Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>         http://lists.xen.org/xen-devel
>
>