[Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Luiz Capitulino]

Hi there,

Summary:

 - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
   the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
   as fast as qemu-kvm.git)

 - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
   with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)

I tried with qemu.git v0.13.0 in order to check if this was a regression, but
I got the same problem...

Then I inspected qemu-kvm.git under the assumption that it could have a fix
that wasn't commited to qemu.git. Found this:

 - commit 0836b77f0f65d56d08bdeffbac25cd6d78267dc9 which is merge, works

 - commit cc015e9a5dde2f03f123357fa060acbdfcd570a4 does not work (it's slow)

I tried a bisect, but it brakes due to gcc4 vs. gcc3 changes. Then I inspected
commits manually, and found out that commit 64d7e9a4 doesn't work, which makes
me think that the fix could be in the conflict resolution of 0836b77f, which
makes me remember that I'm late for diner, so my conclusions at this point are
not reliable :)

Ideas?

[Qemu-devel] Re: Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Anthony Liguori]

On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
> Hi there,
>
> Summary:
>
>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>     as fast as qemu-kvm.git)
>
>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>
> I tried with qemu.git v0.13.0 in order to check if this was a regression, but
> I got the same problem...
>
> Then I inspected qemu-kvm.git under the assumption that it could have a fix
> that wasn't commited to qemu.git. Found this:
>
>   - commit 0836b77f0f65d56d08bdeffbac25cd6d78267dc9 which is merge, works
>
>   - commit cc015e9a5dde2f03f123357fa060acbdfcd570a4 does not work (it's slow)
>
> I tried a bisect, but it brakes due to gcc4 vs. gcc3 changes. Then I inspected
> commits manually, and found out that commit 64d7e9a4 doesn't work, which makes
> me think that the fix could be in the conflict resolution of 0836b77f, which
> makes me remember that I'm late for diner, so my conclusions at this point are
> not reliable :)

Can you run kvm_stat to see what the exit rates are?

Maybe we're missing a coalesced io in qemu.git?  It's also possible that 
gpxe is hitting the apic or pit quite a lot.

Regards,

Anthony Liguori

> Ideas?
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Qemu-devel] Re: Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Stefan Hajnoczi]

On Sat, Apr 9, 2011 at 1:50 AM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
>>
>> Hi there,
>>
>> Summary:
>>
>>  - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes.
>> Got
>>    the problem with e1000, virtio and rtl8139. However, pcnet *works*
>> (it's
>>    as fast as qemu-kvm.git)
>>
>>  - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute.
>> Tried
>>    with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>>
>> I tried with qemu.git v0.13.0 in order to check if this was a regression,
>> but
>> I got the same problem...
>>
>> Then I inspected qemu-kvm.git under the assumption that it could have a
>> fix
>> that wasn't commited to qemu.git. Found this:
>>
>>  - commit 0836b77f0f65d56d08bdeffbac25cd6d78267dc9 which is merge, works
>>
>>  - commit cc015e9a5dde2f03f123357fa060acbdfcd570a4 does not work (it's
>> slow)
>>
>> I tried a bisect, but it brakes due to gcc4 vs. gcc3 changes. Then I
>> inspected
>> commits manually, and found out that commit 64d7e9a4 doesn't work, which
>> makes
>> me think that the fix could be in the conflict resolution of 0836b77f,
>> which
>> makes me remember that I'm late for diner, so my conclusions at this point
>> are
>> not reliable :)
>
> Can you run kvm_stat to see what the exit rates are?
>
> Maybe we're missing a coalesced io in qemu.git?  It's also possible that
> gpxe is hitting the apic or pit quite a lot.

In gPXE's main loop it will do real <-> protected mode switches and
poll hardware.  It doesn't handle interrupts itself but sets up the
8254 timer chip.

I once found that polling the keyboard only every couple of gPXE main
loop iterations significantly speeds up network throughput under KVM.
I never got around to auditing the entire main loop and implementing a
clean patch.

Anyway, kvm_stat is a good idea.  It may be tickling qemu in a way
that qemu-kvm is immune to.

Stefan

[Qemu-devel] Re: Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Avi Kivity]

On 04/09/2011 03:50 AM, Anthony Liguori wrote:
>
> Maybe we're missing a coalesced io in qemu.git?


e1000 coalesced mmio is in qemu.git (always the chance it's broken, of 
course).  I don't think it would make a 5X difference in any case.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Luiz Capitulino]

On Fri, 08 Apr 2011 19:50:57 -0500
Anthony Liguori <anthony@codemonkey.ws> wrote:

> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
> > Hi there,
> >
> > Summary:
> >
> >   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
> >     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
> >     as fast as qemu-kvm.git)
> >
> >   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
> >     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
> >
> > I tried with qemu.git v0.13.0 in order to check if this was a regression, but
> > I got the same problem...
> >
> > Then I inspected qemu-kvm.git under the assumption that it could have a fix
> > that wasn't commited to qemu.git. Found this:
> >
> >   - commit 0836b77f0f65d56d08bdeffbac25cd6d78267dc9 which is merge, works
> >
> >   - commit cc015e9a5dde2f03f123357fa060acbdfcd570a4 does not work (it's slow)
> >
> > I tried a bisect, but it brakes due to gcc4 vs. gcc3 changes. Then I inspected
> > commits manually, and found out that commit 64d7e9a4 doesn't work, which makes
> > me think that the fix could be in the conflict resolution of 0836b77f, which
> > makes me remember that I'm late for diner, so my conclusions at this point are
> > not reliable :)
> 
> Can you run kvm_stat to see what the exit rates are?

Here you go, both collected after the VM is fully booted:

qemu.git:

efer_reload                    0         0
exits                  159767195        99
fpu_reload                   203         0
halt_exits                   544        27
halt_wakeup                    0         0
host_state_reload         299851        70
hypercalls                     0         0
insn_emulation         134495973        41
insn_emulation_fail            0         0
invlpg                      9687         0
io_exits                   85979         0
irq_exits                 162179         4
irq_injections             11582        27
irq_window                 20712        27
largepages                     0         0
mmio_exits                  9545        41
mmu_cache_miss              5307         0
mmu_flooded                 2493         0
mmu_pde_zapped              1188         0
mmu_pte_updated             5355         0
mmu_pte_write             181550         0
mmu_recycled                   0         0
mmu_shadow_zapped           6437         0
mmu_unsync                    15         0
nmi_injections                 0         0
nmi_window                     0         0
pf_fixed                   73983         0
pf_guest                    4027         0
remote_tlb_flush               1         0
request_irq                    6         0
signal_exits              135731         2
tlb_flush                  26760         0

qemu-kvm.git:

efer_reload                    0         0
exits                    8697244        33
fpu_reload                    46         0
halt_exits                   206         8
halt_wakeup                    7         0
host_state_reload         105173         8
hypercalls                     0         0
insn_emulation           6984118        21
insn_emulation_fail            0         0
invlpg                      9682         0
io_exits                  626201         0
irq_exits                  22930         4
irq_injections              2815         8
irq_window                  1029         0
largepages                     0         0
mmio_exits                  3657         0
mmu_cache_miss              5271         0
mmu_flooded                 2466         0
mmu_pde_zapped              1146         0
mmu_pte_updated             5294         0
mmu_pte_write             191173         0
mmu_recycled                   0         0
mmu_shadow_zapped           6405         0
mmu_unsync                    17         0
nmi_injections                 0         0
nmi_window                     0         0
pf_fixed                   73580         0
pf_guest                    4169         0
remote_tlb_flush               1         0
request_irq                    0         0
signal_exits               24873         0
tlb_flush                  26628         0

> 
> Maybe we're missing a coalesced io in qemu.git?  It's also possible that 
> gpxe is hitting the apic or pit quite a lot.
> 
> Regards,
> 
> Anthony Liguori
> 
> > Ideas?
> > --
> > To unsubscribe from this list: send the line "unsubscribe kvm" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Alex Williamson]

On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
> On Fri, 08 Apr 2011 19:50:57 -0500
> Anthony Liguori <anthony@codemonkey.ws> wrote:
> 
> > On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
> > > Hi there,
> > >
> > > Summary:
> > >
> > >   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
> > >     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
> > >     as fast as qemu-kvm.git)
> > >
> > >   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
> > >     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
> > >

I was having this problem too, but I think it's because I forgot to
build qemu with --enable-io-thread, which is the default for qemu-kvm.
Can you re-configure and build with that and see if it's fast?  Thanks,

Alex

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Luiz Capitulino]

On Mon, 11 Apr 2011 13:00:32 -0600
Alex Williamson <alex.williamson@redhat.com> wrote:

> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
> > On Fri, 08 Apr 2011 19:50:57 -0500
> > Anthony Liguori <anthony@codemonkey.ws> wrote:
> > 
> > > On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
> > > > Hi there,
> > > >
> > > > Summary:
> > > >
> > > >   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
> > > >     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
> > > >     as fast as qemu-kvm.git)
> > > >
> > > >   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
> > > >     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
> > > >
> 
> I was having this problem too, but I think it's because I forgot to
> build qemu with --enable-io-thread, which is the default for qemu-kvm.
> Can you re-configure and build with that and see if it's fast?  Thanks,

Yes, nice catch, it's faster with I/O thread enabled, even seem faster
than qemu-kvm.git.

So, does this have to be fixed w/o I/O thread?

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1441 bytes --]

On 2011-04-11 21:15, Luiz Capitulino wrote:
> On Mon, 11 Apr 2011 13:00:32 -0600
> Alex Williamson <alex.williamson@redhat.com> wrote:
> 
>> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
>>> On Fri, 08 Apr 2011 19:50:57 -0500
>>> Anthony Liguori <anthony@codemonkey.ws> wrote:
>>>
>>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
>>>>> Hi there,
>>>>>
>>>>> Summary:
>>>>>
>>>>>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>>>>>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>>>>>     as fast as qemu-kvm.git)
>>>>>
>>>>>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>>>>>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>>>>>
>>
>> I was having this problem too, but I think it's because I forgot to
>> build qemu with --enable-io-thread, which is the default for qemu-kvm.
>> Can you re-configure and build with that and see if it's fast?  Thanks,
> 
> Yes, nice catch, it's faster with I/O thread enabled, even seem faster
> than qemu-kvm.git.

What's the performance under qemu-kvm with -no-kvm-irqchip?

> 
> So, does this have to be fixed w/o I/O thread?

If it's most probably an architectural deficit of non-io-thread mode, I
would say let it rest in peace. But maybe it points to a generic issues
that is just magnified by non-threaded mode.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Alex Williamson]

On Mon, 2011-04-11 at 22:04 +0200, Jan Kiszka wrote:
> On 2011-04-11 21:15, Luiz Capitulino wrote:
> > On Mon, 11 Apr 2011 13:00:32 -0600
> > Alex Williamson <alex.williamson@redhat.com> wrote:
> > 
> >> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
> >>> On Fri, 08 Apr 2011 19:50:57 -0500
> >>> Anthony Liguori <anthony@codemonkey.ws> wrote:
> >>>
> >>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
> >>>>> Hi there,
> >>>>>
> >>>>> Summary:
> >>>>>
> >>>>>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
> >>>>>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
> >>>>>     as fast as qemu-kvm.git)
> >>>>>
> >>>>>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
> >>>>>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
> >>>>>
> >>
> >> I was having this problem too, but I think it's because I forgot to
> >> build qemu with --enable-io-thread, which is the default for qemu-kvm.
> >> Can you re-configure and build with that and see if it's fast?  Thanks,
> > 
> > Yes, nice catch, it's faster with I/O thread enabled, even seem faster
> > than qemu-kvm.git.
> 
> What's the performance under qemu-kvm with -no-kvm-irqchip?
> 
> > 
> > So, does this have to be fixed w/o I/O thread?
> 
> If it's most probably an architectural deficit of non-io-thread mode, I
> would say let it rest in peace. But maybe it points to a generic issues
> that is just magnified by non-threaded mode.

I've probably been told, but forget.  Why isn't io-thread enabled by
default?  Thanks,

Alex

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2032 bytes --]

On 2011-04-11 22:14, Alex Williamson wrote:
> On Mon, 2011-04-11 at 22:04 +0200, Jan Kiszka wrote:
>> On 2011-04-11 21:15, Luiz Capitulino wrote:
>>> On Mon, 11 Apr 2011 13:00:32 -0600
>>> Alex Williamson <alex.williamson@redhat.com> wrote:
>>>
>>>> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
>>>>> On Fri, 08 Apr 2011 19:50:57 -0500
>>>>> Anthony Liguori <anthony@codemonkey.ws> wrote:
>>>>>
>>>>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
>>>>>>> Hi there,
>>>>>>>
>>>>>>> Summary:
>>>>>>>
>>>>>>>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>>>>>>>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>>>>>>>     as fast as qemu-kvm.git)
>>>>>>>
>>>>>>>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>>>>>>>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>>>>>>>
>>>>
>>>> I was having this problem too, but I think it's because I forgot to
>>>> build qemu with --enable-io-thread, which is the default for qemu-kvm.
>>>> Can you re-configure and build with that and see if it's fast?  Thanks,
>>>
>>> Yes, nice catch, it's faster with I/O thread enabled, even seem faster
>>> than qemu-kvm.git.
>>
>> What's the performance under qemu-kvm with -no-kvm-irqchip?
>>
>>>
>>> So, does this have to be fixed w/o I/O thread?
>>
>> If it's most probably an architectural deficit of non-io-thread mode, I
>> would say let it rest in peace. But maybe it points to a generic issues
>> that is just magnified by non-threaded mode.
> 
> I've probably been told, but forget.  Why isn't io-thread enabled by
> default?  Thanks,

TCG performance still sucks in io-threaded mode. I've three patches in
my queue that reduces the overhead a bit further - for me to a
reasonable level (will post them the next days). But, still, YMMV
depending on the workload.

At least Windows should no longer we a functional blocker thanks to
Paolo's work.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2105 bytes --]

On 2011-04-11 22:18, Jan Kiszka wrote:
> On 2011-04-11 22:14, Alex Williamson wrote:
>> On Mon, 2011-04-11 at 22:04 +0200, Jan Kiszka wrote:
>>> On 2011-04-11 21:15, Luiz Capitulino wrote:
>>>> On Mon, 11 Apr 2011 13:00:32 -0600
>>>> Alex Williamson <alex.williamson@redhat.com> wrote:
>>>>
>>>>> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
>>>>>> On Fri, 08 Apr 2011 19:50:57 -0500
>>>>>> Anthony Liguori <anthony@codemonkey.ws> wrote:
>>>>>>
>>>>>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
>>>>>>>> Hi there,
>>>>>>>>
>>>>>>>> Summary:
>>>>>>>>
>>>>>>>>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>>>>>>>>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>>>>>>>>     as fast as qemu-kvm.git)
>>>>>>>>
>>>>>>>>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>>>>>>>>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>>>>>>>>
>>>>>
>>>>> I was having this problem too, but I think it's because I forgot to
>>>>> build qemu with --enable-io-thread, which is the default for qemu-kvm.
>>>>> Can you re-configure and build with that and see if it's fast?  Thanks,
>>>>
>>>> Yes, nice catch, it's faster with I/O thread enabled, even seem faster
>>>> than qemu-kvm.git.
>>>
>>> What's the performance under qemu-kvm with -no-kvm-irqchip?
>>>
>>>>
>>>> So, does this have to be fixed w/o I/O thread?
>>>
>>> If it's most probably an architectural deficit of non-io-thread mode, I
>>> would say let it rest in peace. But maybe it points to a generic issues
>>> that is just magnified by non-threaded mode.
>>
>> I've probably been told, but forget.  Why isn't io-thread enabled by
>> default?  Thanks,
> 
> TCG performance still sucks in io-threaded mode. I've three patches in
> my queue that reduces the overhead a bit further - for me to a
> reasonable level (will post them the next days). But, still, YMMV
> depending on the workload.

In fact, they were already prepared. So I've just sent them out.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Paolo Bonzini]

On 04/11/2011 10:18 PM, Jan Kiszka wrote:
> At least Windows should no longer we a functional blocker thanks to
> Paolo's work.

What's left now is icount.  I also have had patches ready for a while, 
I'll send them out.

Paolo

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Luiz Capitulino]

On Mon, 11 Apr 2011 22:04:52 +0200
Jan Kiszka <jan.kiszka@web.de> wrote:

> On 2011-04-11 21:15, Luiz Capitulino wrote:
> > On Mon, 11 Apr 2011 13:00:32 -0600
> > Alex Williamson <alex.williamson@redhat.com> wrote:
> > 
> >> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
> >>> On Fri, 08 Apr 2011 19:50:57 -0500
> >>> Anthony Liguori <anthony@codemonkey.ws> wrote:
> >>>
> >>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
> >>>>> Hi there,
> >>>>>
> >>>>> Summary:
> >>>>>
> >>>>>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
> >>>>>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
> >>>>>     as fast as qemu-kvm.git)
> >>>>>
> >>>>>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
> >>>>>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
> >>>>>
> >>
> >> I was having this problem too, but I think it's because I forgot to
> >> build qemu with --enable-io-thread, which is the default for qemu-kvm.
> >> Can you re-configure and build with that and see if it's fast?  Thanks,
> > 
> > Yes, nice catch, it's faster with I/O thread enabled, even seem faster
> > than qemu-kvm.git.
> 
> What's the performance under qemu-kvm with -no-kvm-irqchip?

Still fast, but just realized that qemu-kvm's configure says that I/O thread
is disabled:

 IO thread         no

And it's fast..

> 
> > 
> > So, does this have to be fixed w/o I/O thread?
> 
> If it's most probably an architectural deficit of non-io-thread mode, I
> would say let it rest in peace. But maybe it points to a generic issues
> that is just magnified by non-threaded mode.
> 
> Jan
> 

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1936 bytes --]

On 2011-04-11 23:05, Luiz Capitulino wrote:
> On Mon, 11 Apr 2011 22:04:52 +0200
> Jan Kiszka <jan.kiszka@web.de> wrote:
> 
>> On 2011-04-11 21:15, Luiz Capitulino wrote:
>>> On Mon, 11 Apr 2011 13:00:32 -0600
>>> Alex Williamson <alex.williamson@redhat.com> wrote:
>>>
>>>> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
>>>>> On Fri, 08 Apr 2011 19:50:57 -0500
>>>>> Anthony Liguori <anthony@codemonkey.ws> wrote:
>>>>>
>>>>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
>>>>>>> Hi there,
>>>>>>>
>>>>>>> Summary:
>>>>>>>
>>>>>>>   - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>>>>>>>     the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>>>>>>>     as fast as qemu-kvm.git)
>>>>>>>
>>>>>>>   - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>>>>>>>     with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>>>>>>>
>>>>
>>>> I was having this problem too, but I think it's because I forgot to
>>>> build qemu with --enable-io-thread, which is the default for qemu-kvm.
>>>> Can you re-configure and build with that and see if it's fast?  Thanks,
>>>
>>> Yes, nice catch, it's faster with I/O thread enabled, even seem faster
>>> than qemu-kvm.git.
>>
>> What's the performance under qemu-kvm with -no-kvm-irqchip?
> 
> Still fast, 

I meant: is it even faster with unaccelerated userspace irqchip? I've
seen such effects with emulated NICs before.

> but just realized that qemu-kvm's configure says that I/O thread
> is disabled:
> 
>  IO thread         no
> 
> And it's fast..

That only means (so far) that the upstream io-thread code is disabled.
Qemu-kvm's own solution is enabled all the time, and you can't switch to
upstream anyway as both are incompatible. That's going to change soon
(hopefully) when we migrate qemu-kvm to the upstream version.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Anthony Liguori]

On 04/11/2011 03:04 PM, Jan Kiszka wrote:
> On 2011-04-11 21:15, Luiz Capitulino wrote:
>> On Mon, 11 Apr 2011 13:00:32 -0600
>> Alex Williamson<alex.williamson@redhat.com>  wrote:
>>
>>> On Mon, 2011-04-11 at 15:35 -0300, Luiz Capitulino wrote:
>>>> On Fri, 08 Apr 2011 19:50:57 -0500
>>>> Anthony Liguori<anthony@codemonkey.ws>  wrote:
>>>>
>>>>> On 04/08/2011 06:25 PM, Luiz Capitulino wrote:
>>>>>> Hi there,
>>>>>>
>>>>>> Summary:
>>>>>>
>>>>>>    - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>>>>>>      the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>>>>>>      as fast as qemu-kvm.git)
>>>>>>
>>>>>>    - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>>>>>>      with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>>>>>>
>>> I was having this problem too, but I think it's because I forgot to
>>> build qemu with --enable-io-thread, which is the default for qemu-kvm.
>>> Can you re-configure and build with that and see if it's fast?  Thanks,
>> Yes, nice catch, it's faster with I/O thread enabled, even seem faster
>> than qemu-kvm.git.
> What's the performance under qemu-kvm with -no-kvm-irqchip?
>
>> So, does this have to be fixed w/o I/O thread?
> If it's most probably an architectural deficit of non-io-thread mode, I
> would say let it rest in peace. But maybe it points to a generic issues
> that is just magnified by non-threaded mode.

If gpxe is spinning waiting for I/O to complete, that's going to prevent 
select from running until the next signal (timer event).

Regards,

Anthony Liguori

> Jan
>

Re: [Qemu-devel] Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Avi Kivity]

On 04/12/2011 03:39 AM, Anthony Liguori wrote:
>> would say let it rest in peace. But maybe it points to a generic issues
>> that is just magnified by non-threaded mode.
> If it's most probably an architectural deficit of non-io-thread mode, I
>
> If gpxe is spinning waiting for I/O to complete, that's going to 
> prevent select from running until the next signal (timer event).

Or the next signal (packet received on fd).

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Blue Swirl]

On Sat, Apr 9, 2011 at 2:25 AM, Luiz Capitulino <lcapitulino@redhat.com> wrote:
> Hi there,
>
> Summary:
>
>  - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
>   the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
>   as fast as qemu-kvm.git)
>
>  - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
>   with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
>
> I tried with qemu.git v0.13.0 in order to check if this was a regression, but
> I got the same problem...
>
> Then I inspected qemu-kvm.git under the assumption that it could have a fix
> that wasn't commited to qemu.git. Found this:
>
>  - commit 0836b77f0f65d56d08bdeffbac25cd6d78267dc9 which is merge, works
>
>  - commit cc015e9a5dde2f03f123357fa060acbdfcd570a4 does not work (it's slow)
>
> I tried a bisect, but it brakes due to gcc4 vs. gcc3 changes. Then I inspected
> commits manually, and found out that commit 64d7e9a4 doesn't work, which makes
> me think that the fix could be in the conflict resolution of 0836b77f, which
> makes me remember that I'm late for diner, so my conclusions at this point are
> not reliable :)
>
> Ideas?

What is the test case? I tried PXE booting a 10M file with and without
KVM and the results are pretty much the same with pcnet and e1000.
time qemu -monitor stdio -boot n -net nic,model=e1000 -net
user,tftp=.,bootfile=10M -net dump,file=foo -enable-kvm
time qemu -monitor stdio -boot n -net nic,model=pcnet -net
user,tftp=.,bootfile=10M -net dump,file=foo -enable-kvm
time qemu -monitor stdio -boot n -net nic,model=e1000 -net
user,tftp=.,bootfile=10M -net dump,file=foo
time qemu -monitor stdio -boot n -net nic,model=pcnet -net
user,tftp=.,bootfile=10M -net dump,file=foo

All times are ~10s.

[Qemu-devel] Re: Slow PXE boot in qemu.git (fast in qemu-kvm.git)

[Luiz Capitulino]

On Sat, 9 Apr 2011 13:34:43 +0300
Blue Swirl <blauwirbel@gmail.com> wrote:

> On Sat, Apr 9, 2011 at 2:25 AM, Luiz Capitulino <lcapitulino@redhat.com> wrote:
> > Hi there,
> >
> > Summary:
> >
> >  - PXE boot in qemu.git (HEAD f124a41) is quite slow, more than 5 minutes. Got
> >   the problem with e1000, virtio and rtl8139. However, pcnet *works* (it's
> >   as fast as qemu-kvm.git)
> >
> >  - PXE boot in qemu-kvm.git (HEAD df85c051) is fast, less than a minute. Tried
> >   with e1000, virtio and rtl8139 (I don't remember if I tried with pcnet)
> >
> > I tried with qemu.git v0.13.0 in order to check if this was a regression, but
> > I got the same problem...
> >
> > Then I inspected qemu-kvm.git under the assumption that it could have a fix
> > that wasn't commited to qemu.git. Found this:
> >
> >  - commit 0836b77f0f65d56d08bdeffbac25cd6d78267dc9 which is merge, works
> >
> >  - commit cc015e9a5dde2f03f123357fa060acbdfcd570a4 does not work (it's slow)
> >
> > I tried a bisect, but it brakes due to gcc4 vs. gcc3 changes. Then I inspected
> > commits manually, and found out that commit 64d7e9a4 doesn't work, which makes
> > me think that the fix could be in the conflict resolution of 0836b77f, which
> > makes me remember that I'm late for diner, so my conclusions at this point are
> > not reliable :)
> >
> > Ideas?
> 
> What is the test case?

It's an external PXE server, command-line is:

 qemu -boot n -enable-kvm -net nic,model=virtio -net tap,ifname=vnet0,script=

> I tried PXE booting a 10M file with and without
> KVM and the results are pretty much the same with pcnet and e1000.
> time qemu -monitor stdio -boot n -net nic,model=e1000 -net
> user,tftp=.,bootfile=10M -net dump,file=foo -enable-kvm
> time qemu -monitor stdio -boot n -net nic,model=pcnet -net
> user,tftp=.,bootfile=10M -net dump,file=foo -enable-kvm
> time qemu -monitor stdio -boot n -net nic,model=e1000 -net
> user,tftp=.,bootfile=10M -net dump,file=foo
> time qemu -monitor stdio -boot n -net nic,model=pcnet -net
> user,tftp=.,bootfile=10M -net dump,file=foo
> 
> All times are ~10s.

Yeah, you're using the internal tftp server.