[virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Dor Laor]

commit 763769621d271d92204ed27552d75448587c1ac0
Author: Dor Laor <dor.laor@qumranet.com>
Date:   Wed Dec 12 14:52:00 2007 +0200

    [virtio-net][PATCH] Don't arm tx hrtimer with a constant 50us each 
transmit
   
    The current start_xmit sets 500us hrtimer to kick the host.
    The problem is that if another xmit happens before the timer was 
fired then
    the first xmit will have to wait additional 500us.
    This patch does not re-arm the timer if there is existing one.
    This will shorten the latency for tx.
   
    Signed-off-by: Dor Laor <dor.laor@qumranet.com>

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 7b051d5..8bb17d1
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -406,10 +405,10 @@ again:
         virtio_debug(vdebug, "%s: before calling kick %d\n", 
__FUNCTION__, __LINE__);
         vi->svq->vq_ops->kick(vi->svq);
         vi->out_num = 0;
-    } else {
-        vi->stats.hrtimer_starts++;
-        hrtimer_start(&vi->tx_timer, ktime_set(0,500000),
-                  HRTIMER_MODE_REL);
+    } else if (!hrtimer_is_queued(&vi->tx_timer)) {
+            vi->stats.hrtimer_starts++;
+            hrtimer_start(&vi->tx_timer, ktime_set(0,500000),
+                      HRTIMER_MODE_REL);
     }
     return 0;
 }

Re: [kvm-devel] [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Christian Borntraeger]

Am Mittwoch, 12. Dezember 2007 schrieb Dor Laor:
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -406,10 +405,10 @@ again:

Hmm, while I agree in general with the patch, I fail to find the proper 
version of virtio_net where this patch applies. I tried kvm.git and 
linux-2.6.git from kernel.org. Can you give me a pointer to the repository 
where you work on virtio?

Christian

Re: [kvm-devel] [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Dor Laor]

Christian Borntraeger wrote:
>
> Am Mittwoch, 12. Dezember 2007 schrieb Dor Laor:
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -406,10 +405,10 @@ again:
>
> Hmm, while I agree in general with the patch, I fail to find the proper
> version of virtio_net where this patch applies. I tried kvm.git and
> linux-2.6.git from kernel.org. Can you give me a pointer to the repository
> where you work on virtio?
>
Sorry for that, I added some debug prints of my one.
Here it is: *git clone 
git*://kvm.*qumranet*.com/home/*dor*/src/linux-2.6-nv use branch 'virtio'.
BTW: what git repository do you use?

This patch improves my tx performance from 720Mbps to 900Mbps.
Dor
>
> Christian
>

Re: [kvm-devel] [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Christian Borntraeger]

Am Mittwoch, 12. Dezember 2007 schrieb Dor Laor:
> Christian Borntraeger wrote:
> >
> > Am Mittwoch, 12. Dezember 2007 schrieb Dor Laor:
> > > --- a/drivers/net/virtio_net.c
> > > +++ b/drivers/net/virtio_net.c
> > > @@ -406,10 +405,10 @@ again:
> >
> > Hmm, while I agree in general with the patch, I fail to find the proper
> > version of virtio_net where this patch applies. I tried kvm.git and
> > linux-2.6.git from kernel.org. Can you give me a pointer to the 
repository
> > where you work on virtio?
> >
> Sorry for that, I added some debug prints of my one.
> Here it is: *git clone 
> git*://kvm.*qumranet*.com/home/*dor*/src/linux-2.6-nv use branch 'virtio'.

Ah, ok. I will look into that branch.

> BTW: what git repository do you use?

I use Avis git from kernel.org:
git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm

Christian

Re: [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Dor Laor]

Christian Borntraeger wrote:
>
> Am Mittwoch, 12. Dezember 2007 schrieb Dor Laor:
> > Christian Borntraeger wrote:
> > >
> > > Am Mittwoch, 12. Dezember 2007 schrieb Dor Laor:
> > > > --- a/drivers/net/virtio_net.c
> > > > +++ b/drivers/net/virtio_net.c
> > > > @@ -406,10 +405,10 @@ again:
> > >
> > > Hmm, while I agree in general with the patch, I fail to find the 
> proper
> > > version of virtio_net where this patch applies. I tried kvm.git and
> > > linux-2.6.git from kernel.org. Can you give me a pointer to the
> repository
> > > where you work on virtio?
> > >
> > Sorry for that, I added some debug prints of my one.
> > Here it is: *git clone
> > git*://kvm.*qumranet*.com/home/*dor*/src/linux-2.6-nv use branch 
> 'virtio'.
>
> Ah, ok. I will look into that branch.
>
> > BTW: what git repository do you use?
>
> I use Avis git from kernel.org:
> git://git.kernel.org/pub/scm/linux/kernel/git/avi/kvm
>
I patch it with Anthony's http://hg.codemonkey.ws/linux-virtio/
Over that I send the patches. One can use my repository directly.
>
> Christian
>


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Rusty Russell]

On Wednesday 12 December 2007 23:54:00 Dor Laor wrote:
> commit 763769621d271d92204ed27552d75448587c1ac0
> Author: Dor Laor <dor.laor@qumranet.com>
> Date:   Wed Dec 12 14:52:00 2007 +0200
>
>     [virtio-net][PATCH] Don't arm tx hrtimer with a constant 50us each
> transmit
>
>     The current start_xmit sets 500us hrtimer to kick the host.
>     The problem is that if another xmit happens before the timer was
> fired then
>     the first xmit will have to wait additional 500us.
>     This patch does not re-arm the timer if there is existing one.
>     This will shorten the latency for tx.

Hi Dor!

    Yes, I pondered this when I wrote the code.  On the one hand, it's a 
low-probability pathological corner case, on the other, your patch reduces 
the number of timer reprograms in the normal case.

So I've applied it, thanks!
Rusty.





>
>     Signed-off-by: Dor Laor <dor.laor@qumranet.com>
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 7b051d5..8bb17d1
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -406,10 +405,10 @@ again:
>          virtio_debug(vdebug, "%s: before calling kick %d\n",
> __FUNCTION__, __LINE__);
>          vi->svq->vq_ops->kick(vi->svq);
>          vi->out_num = 0;
> -    } else {
> -        vi->stats.hrtimer_starts++;
> -        hrtimer_start(&vi->tx_timer, ktime_set(0,500000),
> -                  HRTIMER_MODE_REL);
> +    } else if (!hrtimer_is_queued(&vi->tx_timer)) {
> +            vi->stats.hrtimer_starts++;
> +            hrtimer_start(&vi->tx_timer, ktime_set(0,500000),
> +                      HRTIMER_MODE_REL);
>      }
>      return 0;
>  }

Re: [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Avi Kivity]

Rusty Russell wrote:
> On Wednesday 12 December 2007 23:54:00 Dor Laor wrote:
>   
>> commit 763769621d271d92204ed27552d75448587c1ac0
>> Author: Dor Laor <dor.laor-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
>> Date:   Wed Dec 12 14:52:00 2007 +0200
>>
>>     [virtio-net][PATCH] Don't arm tx hrtimer with a constant 50us each
>> transmit
>>
>>     The current start_xmit sets 500us hrtimer to kick the host.
>>     The problem is that if another xmit happens before the timer was
>> fired then
>>     the first xmit will have to wait additional 500us.
>>     This patch does not re-arm the timer if there is existing one.
>>     This will shorten the latency for tx.
>>     
>
> Hi Dor!
>
>     Yes, I pondered this when I wrote the code.  On the one hand, it's a 
> low-probability pathological corner case, on the other, your patch reduces 
> the number of timer reprograms in the normal case.
>   

One thing that came up in our discussions is to let the host do the 
timer processing instead of the guest.  When tx exit mitigation is 
enabled, the guest bumps the queue pointer, but carefully refrains from 
kicking the host.  The host polls the tx pointer using a timer, kicking 
itself periodically; if polling yields no packets it disables tx exit 
mitigation.  This saves the guest the bother of programming the timer, 
which presumably requires an exit if the timer is the closest one to 
expiration.

[btw, this can be implemented in virtqueue rather than virtio-net, no?]

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Rusty Russell]

On Tuesday 18 December 2007 16:30:08 Avi Kivity wrote:
> Rusty Russell wrote:
> >     Yes, I pondered this when I wrote the code.  On the one hand, it's a
> > low-probability pathological corner case, on the other, your patch
> > reduces the number of timer reprograms in the normal case.
>
> One thing that came up in our discussions is to let the host do the
> timer processing instead of the guest.  When tx exit mitigation is
> enabled, the guest bumps the queue pointer, but carefully refrains from
> kicking the host.  The host polls the tx pointer using a timer, kicking
> itself periodically; if polling yields no packets it disables tx exit
> mitigation.  This saves the guest the bother of programming the timer,
> which presumably requires an exit if the timer is the closest one to
> expiration.
>
> [btw, this can be implemented in virtqueue rather than virtio-net, no?]

Yes, the current patch is a hack (look at the hardcoded constant); wanted to 
see how much it helps, if any.

More sophisticated timer management would be a definite win... funny, I have a 
patch here which helps that....

Rusty.

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

Re: [virtio-net][PATCH] Don't arm tx hrtimer with a constant 500us each transmit

[Dor Laor]

[-- Attachment #1.1: Type: text/plain, Size: 1942 bytes --]

Rusty Russell wrote:
> On Wednesday 12 December 2007 23:54:00 Dor Laor wrote:
>   
>> commit 763769621d271d92204ed27552d75448587c1ac0
>> Author: Dor Laor <dor.laor-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
>> Date:   Wed Dec 12 14:52:00 2007 +0200
>>
>>     [virtio-net][PATCH] Don't arm tx hrtimer with a constant 50us each
>> transmit
>>
>>     The current start_xmit sets 500us hrtimer to kick the host.
>>     The problem is that if another xmit happens before the timer was
>> fired then
>>     the first xmit will have to wait additional 500us.
>>     This patch does not re-arm the timer if there is existing one.
>>     This will shorten the latency for tx.
>>     
>
> Hi Dor!
>
>     Yes, I pondered this when I wrote the code.  On the one hand, it's a 
> low-probability pathological corner case, on the other, your patch reduces 
> the number of timer reprograms in the normal case.
>
> So I've applied it, thanks!
> Rusty.
>
>
>   
Thanks, it actually improved my tx performance in 20%!
Expecting the host side timer with your new patch.
Cheers,
Dor.
>
>
>   
>>     Signed-off-by: Dor Laor <dor.laor-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
>>
>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>> index 7b051d5..8bb17d1
>> --- a/drivers/net/virtio_net.c
>> +++ b/drivers/net/virtio_net.c
>> @@ -406,10 +405,10 @@ again:
>>          virtio_debug(vdebug, "%s: before calling kick %d\n",
>> __FUNCTION__, __LINE__);
>>          vi->svq->vq_ops->kick(vi->svq);
>>          vi->out_num = 0;
>> -    } else {
>> -        vi->stats.hrtimer_starts++;
>> -        hrtimer_start(&vi->tx_timer, ktime_set(0,500000),
>> -                  HRTIMER_MODE_REL);
>> +    } else if (!hrtimer_is_queued(&vi->tx_timer)) {
>> +            vi->stats.hrtimer_starts++;
>> +            hrtimer_start(&vi->tx_timer, ktime_set(0,500000),
>> +                      HRTIMER_MODE_REL);
>>      }
>>      return 0;
>>  }
>>     
>
>
>
>   


[-- Attachment #1.2: Type: text/html, Size: 2755 bytes --]

[-- Attachment #2: Type: text/plain, Size: 308 bytes --]

-------------------------------------------------------------------------
SF.Net email is sponsored by:
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services
for just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel