[Qemu-devel] High speed polling

[Qemu-devel] High speed polling

[Clay Andreasen]

I have a network device simulation that I am connecting to multiple
instances of Qemu (nodes) via a shared memory queue.  It works pretty 
well as
long as all of the nodes are initiating communication but when one node is
passive, it must poll to get packets.  So far the fastest I have been 
able to
get it to poll is about every 2M emulated clocks.
This is with CONFIG_HIGH_RES_TIMERS and CONFIG_NO_HZ on the host.
I also set MIN_TIMER_REARM_NS in qemu-timer.c to 100000.
Is there some way to increase the polling rate by about an order of 
magnitude?

Re: [Qemu-devel] High speed polling

[Mulyadi Santosa]

Hi Clay.....

On Wed, Jun 15, 2011 at 05:32, Clay Andreasen <cda@cray.com> wrote:
> I have a network device simulation that I am connecting to multiple
> instances of Qemu (nodes) via a shared memory queue.  It works pretty well
> as
> long as all of the nodes are initiating communication but when one node is
> passive, it must poll to get packets.  So far the fastest I have been able
> to
> get it to poll is about every 2M emulated clocks.

that means, once every ... 1 mili second? (I doubt it, but
theoritically possible)

> This is with CONFIG_HIGH_RES_TIMERS and CONFIG_NO_HZ on the host.
> I also set MIN_TIMER_REARM_NS in qemu-timer.c to 100000.
> Is there some way to increase the polling rate by about an order of
> magnitude?

with the way current kernel did fair scheduling, your problem is
timing handling latency actually i.e timer is fast, but the handler
needs some time to catch up.

what I can suggest is experimenting with rt- kernel and put yourself
into such high priority process with realtime scheduling, keep
reducing MIN_TIMER_REARM_NS..

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Re: [Qemu-devel] High speed polling

[Stefan Hajnoczi]

On Tue, Jun 14, 2011 at 11:32 PM, Clay Andreasen <cda@cray.com> wrote:
> I have a network device simulation that I am connecting to multiple
> instances of Qemu (nodes) via a shared memory queue.  It works pretty well
> as
> long as all of the nodes are initiating communication but when one node is
> passive, it must poll to get packets.  So far the fastest I have been able
> to
> get it to poll is about every 2M emulated clocks.
> This is with CONFIG_HIGH_RES_TIMERS and CONFIG_NO_HZ on the host.
> I also set MIN_TIMER_REARM_NS in qemu-timer.c to 100000.
> Is there some way to increase the polling rate by about an order of
> magnitude?

Without more details it's hard to say what is going on:

Running an x86 guest?  Are you using ./configure --enable-io-thread?
It sounds like you may not be using KVM?  How many vcpus are running
on the host in total compared to the number of logical CPUs on the
host?

You haven't given details on how you are polling in the guest.  Are
you running a polling loop in ring 0 or is the guest running a
full-blown OS and polling from userspace?

Why are you polling in the first place - to minimize latency?

Stefan

Re: [Qemu-devel] High speed polling

[Clay Andreasen]

Thank you for your reply.
I am still a novice with Qemu so pardon me if I don't make any sense.

I tried --enable-io-thread.  I get the error:
cpus.o: In function `qemu_kvm_eat_signal':
cpus.c:(.text+0x111a): undefined reference to `kvm_on_sigbus_vcpu'

so I assume it requires KVM.  I'm not using KVM because I don't have 
full control
over the host I am running on.
I have 8 host processors running 4 Qemu copies (1 vcpu each) plus my 
network simulator.
I have tried polling via a call in vl.c:mainloop and via qemu_mod_timer().
There doesn't appear to be much difference.
The guest is a full-blown x86_64 OS.
I am polling to minimize latency.
I am looking at other ways to tolerate the current latency in case I can't
do much better.

Clay


On 06/15/11 01:22, Stefan Hajnoczi wrote:
> On Tue, Jun 14, 2011 at 11:32 PM, Clay Andreasen<cda@cray.com>  wrote:
>> I have a network device simulation that I am connecting to multiple
>> instances of Qemu (nodes) via a shared memory queue.  It works pretty well
>> as
>> long as all of the nodes are initiating communication but when one node is
>> passive, it must poll to get packets.  So far the fastest I have been able
>> to
>> get it to poll is about every 2M emulated clocks.
>> This is with CONFIG_HIGH_RES_TIMERS and CONFIG_NO_HZ on the host.
>> I also set MIN_TIMER_REARM_NS in qemu-timer.c to 100000.
>> Is there some way to increase the polling rate by about an order of
>> magnitude?
> Without more details it's hard to say what is going on:
>
> Running an x86 guest?  Are you using ./configure --enable-io-thread?
> It sounds like you may not be using KVM?  How many vcpus are running
> on the host in total compared to the number of logical CPUs on the
> host?
>
> You haven't given details on how you are polling in the guest.  Are
> you running a polling loop in ring 0 or is the guest running a
> full-blown OS and polling from userspace?
>
> Why are you polling in the first place - to minimize latency?
>
> Stefan