[Qemu-devel] Qemu 2D performance plunges below acceptable levels

[Qemu-devel] Qemu 2D performance plunges below acceptable levels

[Alexey Eremenko]

Hi All !

In KVM-84 I found that 2D performance simply plunges below acceptable
levels. It turned out to be Qemu bug (or "feature")

How To reproduce:
It is *very* visible if you work with remote X forwarding, using SDL
rendering. (where your Qemu server and local desktop are different
machines)
A. Qemu -boot n
This tried to initialize network boot from PXE. All the text rendered
moves very slowly on screen...
-or-
B. run Windows guest + Passmark Performance Test -> 2D Performance -> Fonts Test
-or-
C. run Windows guest -> Start -> Run -> cmd -> "tree C:\" command

Performance decrease is over tenfold -- so it is very visible with naked eye.

Now I have found the cause:
It is Qemu-git commit: 7d957bd8cbcbf56f7916d375e65042d767f544b5
"-DisplayState interface change (Stefano Stabellini) from 2009-01-15"

At last ! It took 2 days to figure it out !
First I had to bisect KVM-userspace git,
then I had to map KVM-userspace git commits to Qemu-git commits, as
they are not the same.
Finally I had to bisect Qemu git.

-- 
-Alexey Eromenko, 15.2.2009

[Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Alexey Eremenko]

Hello... Has anyone looked into this?

-- 
-Alexey Eromenko "Technologov"

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

Alexey Eremenko wrote:

> Hello... Has anyone looked into this?
> 


Does kvm have this fix "Fix SDL slowness.":

git-svn-id: c046a42c-6fe2-441c-8c8c-71466251a162

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Avi Kivity]

Stefano Stabellini wrote:
> Alexey Eremenko wrote:
>
>   
>> Hello... Has anyone looked into this?
>>
>>     
>
>
> Does kvm have this fix "Fix SDL slowness.":
>
> git-svn-id: c046a42c-6fe2-441c-8c8c-71466251a162
>
>   

Yes.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

Avi Kivity wrote:

> Stefano Stabellini wrote:
>> Alexey Eremenko wrote:
>>
>>   
>>> Hello... Has anyone looked into this?
>>>
>>>     
>>
>> Does kvm have this fix "Fix SDL slowness.":
>>
>> git-svn-id: c046a42c-6fe2-441c-8c8c-71466251a162
>>
>>   
> 
> Yes.
> 


Do you see the problem only on X11 remote?
How bad is on vnc?

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Avi Kivity]

Stefano Stabellini wrote:
>>>>     
>>>>         
>>> Does kvm have this fix "Fix SDL slowness.":
>>>
>>> git-svn-id: c046a42c-6fe2-441c-8c8c-71466251a162
>>>
>>>   
>>>       
>> Yes.
>>
>>     
>
>
> Do you see the problem only on X11 remote?
>   

Only tested X11 local so far.

> How bad is on vnc?
>   

Will test.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Avi Kivity]

Stefano Stabellini wrote:
> Avi Kivity wrote:
>
>   
>> Stefano Stabellini wrote:
>>     
>>> Alexey Eremenko wrote:
>>>
>>>   
>>>       
>>>> Hello... Has anyone looked into this?
>>>>
>>>>     
>>>>         
>>> Does kvm have this fix "Fix SDL slowness.":
>>>
>>> git-svn-id: c046a42c-6fe2-441c-8c8c-71466251a162
>>>
>>>   
>>>       
>> Yes.
>>
>>     
>
>
> Do you see the problem only on X11 remote?
> How bad is on vnc?
>   

vnc regresses in exactly the same way.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

Avi Kivity wrote:

> Stefano Stabellini wrote:
>> Do you see the problem only on X11 remote?
>> How bad is on vnc?
>>   
> 
> vnc regresses in exactly the same way.
> 


This is very strange since the conversions in place for the vnc case are
exactly the same as before.

In your particular case, with a Windows 24bpp, vga.c takes care of
converting the 24bpp format from the emulated videoram to the
DisplaySurface into 32bpp, then vnc reads directly from there.
Before, vnc.c used to set 32bpp in the DisplayState struct and vga.c did
the conversion from whatever format the guest used to 32bpp.
In both case vga.c does the conversion using the same routines in
hw/vga_template.h.

The sdl case is more complicated, but if you are sure that in the vnc
case the slow down is more or less the same, then we can rule out the
color conversions.
Just to be sure, if the guest is in 32bpp do you still see the slow
down? Try in particular with stdvga.

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Alexey Eremenko]

On Tue, Feb 17, 2009 at 7:00 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> Avi Kivity wrote:
>
>> Stefano Stabellini wrote:
>>> Do you see the problem only on X11 remote?
>>> How bad is on vnc?
>>>
>>
>> vnc regresses in exactly the same way.
>>

Well, in my experience both local X11 and VNC are fast in KVM-84.

But remote X11/SDL performance plunged below usable levels.

-- 
-Alexey Eromenko "Technologov"

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Wed, 18 Feb 2009, Alexey Eremenko wrote:

> On Tue, Feb 17, 2009 at 7:00 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > Avi Kivity wrote:
> >
> >> Stefano Stabellini wrote:
> >>> Do you see the problem only on X11 remote?
> >>> How bad is on vnc?
> >>>
> >>
> >> vnc regresses in exactly the same way.
> >>
> 
> Well, in my experience both local X11 and VNC are fast in KVM-84.
> 
> But remote X11/SDL performance plunged below usable levels.

FWIW same thing happened to Mac OS X + SDL.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

malc wrote:

> On Wed, 18 Feb 2009, Alexey Eremenko wrote:
> 
>> On Tue, Feb 17, 2009 at 7:00 PM, Stefano Stabellini
>> <stefano.stabellini@eu.citrix.com> wrote:
>>> Avi Kivity wrote:
>>>
>>>> Stefano Stabellini wrote:
>>>>> Do you see the problem only on X11 remote?
>>>>> How bad is on vnc?
>>>>>
>>>> vnc regresses in exactly the same way.
>>>>
>> Well, in my experience both local X11 and VNC are fast in KVM-84.
>>
>> But remote X11/SDL performance plunged below usable levels.
> 
> FWIW same thing happened to Mac OS X + SDL.
> 


Are you talking about SDL local?
If so, is your host a big endian machine and the guest a little endian?
What is the color depth and bpp of the guest (I assume that the host is
32bpp)?
As usual two good tests are to check the vnc case and a guest in 32bpp too.

I tried to repeat the "tree C:\" test on qemu i386 but I couldn't see
any significant difference.

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Wed, 18 Feb 2009, Stefano Stabellini wrote:

> malc wrote:
> 
> > On Wed, 18 Feb 2009, Alexey Eremenko wrote:
> > 
> >> On Tue, Feb 17, 2009 at 7:00 PM, Stefano Stabellini
> >> <stefano.stabellini@eu.citrix.com> wrote:
> >>> Avi Kivity wrote:
> >>>
> >>>> Stefano Stabellini wrote:
> >>>>> Do you see the problem only on X11 remote?
> >>>>> How bad is on vnc?
> >>>>>
> >>>> vnc regresses in exactly the same way.
> >>>>
> >> Well, in my experience both local X11 and VNC are fast in KVM-84.
> >>
> >> But remote X11/SDL performance plunged below usable levels.
> > 
> > FWIW same thing happened to Mac OS X + SDL.
> > 
> 
> 
> Are you talking about SDL local?

Yes.

> If so, is your host a big endian machine and the guest a little endian?

Yes.

> What is the color depth and bpp of the guest (I assume that the host is
> 32bpp)?

Guest is in VGA mode3 (i.e. POST screen and DOS prompt)

> As usual two good tests are to check the vnc case and a guest in 32bpp too.

This doesn't happen on the very same machine running very same guest but
host OS being Linux.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Alexey Eremenko]

>> As usual two good tests are to check the vnc case and a guest in 32bpp too.
>
> This doesn't happen on the very same machine running very same guest but
> host OS being Linux.

Then it seems that your problem is a different one - it seems to be Mac related.

-- 
-Alexey Eromenko "Technologov"

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Wed, 18 Feb 2009, Alexey Eremenko wrote:

> >> As usual two good tests are to check the vnc case and a guest in 32bpp too.
> >
> > This doesn't happen on the very same machine running very same guest but
> > host OS being Linux.
> 
> Then it seems that your problem is a different one - it seems to be Mac 
> related.

It's directly related to the display changes though.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

malc wrote:

>> As usual two good tests are to check the vnc case and a guest in 32bpp too.
> 
> This doesn't happen on the very same machine running very same guest but
> host OS being Linux.
> 



Uhm.. are you using one of those infamous 20-inch iMac with a resolution
of 6 bits per pixel?
That would explain the slow down.
Please try this patch and let me know if it makes a difference:

diff --git a/sdl.c b/sdl.c
index cfdf852..a5cc6d1 100644
--- a/sdl.c
+++ b/sdl.c
@@ -95,7 +95,7 @@ static void sdl_resize(DisplayState *ds)
 
     width = ds_get_width(ds);
     height = ds_get_height(ds);
-    real_screen = SDL_SetVideoMode(width, height, 0, flags);
+    real_screen = SDL_SetVideoMode(width, height, 32, flags);
     if (!real_screen) {
         fprintf(stderr, "Could not open SDL display\n");
         exit(1);

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Wed, 18 Feb 2009, Stefano Stabellini wrote:

> malc wrote:
> 
> >> As usual two good tests are to check the vnc case and a guest in 32bpp too.
> > 
> > This doesn't happen on the very same machine running very same guest but
> > host OS being Linux.
> > 
> 
> 
> 
> Uhm.. are you using one of those infamous 20-inch iMac with a resolution
> of 6 bits per pixel?

No.

> That would explain the slow down.
> Please try this patch and let me know if it makes a difference:

Will do next time i'm in OS X.

[..snip..]

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Wed, 18 Feb 2009, Stefano Stabellini wrote:

> malc wrote:
> 
> >> As usual two good tests are to check the vnc case and a guest in 32bpp too.
> > 
> > This doesn't happen on the very same machine running very same guest but
> > host OS being Linux.
> > 
> 
> 
> 
> Uhm.. are you using one of those infamous 20-inch iMac with a resolution
> of 6 bits per pixel?
> That would explain the slow down.
> Please try this patch and let me know if it makes a difference:
> 

It doesn't. FWIW not quite "value preserving" operation of removing call
to SDL_UpdateRect from sdl_update and putting SDL_Flip into sdl_refresh
does.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

malc wrote:

> 
> It doesn't. FWIW not quite "value preserving" operation of removing call
> to SDL_UpdateRect from sdl_update and putting SDL_Flip into sdl_refresh
> does.
> 



Going through the SDL documentation I found this:

http://www.libsdl.org/faq.php?action=listentries&category=7
Frequently Asked Questions: Mac OS

You may be able to make things faster by calling SDL_UpdateRects()
instead of SDL_UpdateRect() or SDL_Flip(). With SDL_UpdateRects()
you can tell SDL exactly what rectangles of the surface you painted,
and they will all be handed to the window server at once. That way,
the window server does the least amount of work possible.

So in theory this should make sdl faster on MacOSX:

diff --git a/sdl.c b/sdl.c
index 266fbcc..2ce20d2 100644
--- a/sdl.c
+++ b/sdl.c
@@ -62,7 +62,7 @@ static void sdl_update(DisplayState *ds, int x, int y,
int w, int h)
     //    printf("updating x=%d y=%d w=%d h=%d\n", x, y, w, h);

     SDL_BlitSurface(guest_screen, &rec, real_screen, &rec);
-    SDL_UpdateRect(real_screen, x, y, w, h);
+    SDL_UpdateRects(real_screen, 1, &rec);
 }

 static void sdl_setdata(DisplayState *ds)

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Paul Brook]

>      SDL_BlitSurface(guest_screen, &rec, real_screen, &rec);
> -    SDL_UpdateRect(real_screen, x, y, w, h);
> +    SDL_UpdateRects(real_screen, 1, &rec);

Have you actually tried this? I'd be amazed and dismayed if it made any 
difference. I can believe that SDL_UpdateRects is better that *multiple* 
calls to SDL_UpdateRect. However I see absolutely no justification for this 
change.

Paul

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

Paul Brook wrote:

>>      SDL_BlitSurface(guest_screen, &rec, real_screen, &rec);
>> -    SDL_UpdateRect(real_screen, x, y, w, h);
>> +    SDL_UpdateRects(real_screen, 1, &rec);
> 
> Have you actually tried this? I'd be amazed and dismayed if it made any 
> difference. I can believe that SDL_UpdateRects is better that *multiple* 
> calls to SDL_UpdateRect. However I see absolutely no justification for this 
> change.
> 

Of course I didn't: I don't have any MacOSX available and this is the
main reason I am asking other people to test this patch and doing it myself.

>From the documentation I thought they were implemented differently
somehow but now that you make me think about it I went through the
actual code and SDL_UpdateRect is implemented using SDL_UpdateRects :(

Also I don't how to explain the fact that SDL_Flip is faster than
SDL_UpdateRect on MacOSX according to malc: the same libsdl macosx faq I
 linked before claims that SDL_DOUBLEBUF is not even supported on MacOsX.

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Paul Brook]

> Also I don't how to explain the fact that SDL_Flip is faster than
> SDL_UpdateRect on MacOSX according to malc: the same libsdl macosx faq I
>  linked before claims that SDL_DOUBLEBUF is not even supported on MacOsX.

I guess it's possible that under a sufficiently lame implementation a partial 
update could be slower than an update of the whole screen. e.g. if your 
accelerated blit routines isn't capable of copying rectangular regions you 
have to do the partial update a line at a time, whereas a full update is a 
single blit operation. Seems a fairly unlikely corner case though, especially 
as we tend to update a whole number of lines.

Paul

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Fri, 20 Feb 2009, Stefano Stabellini wrote:

> Paul Brook wrote:
> 
> >>      SDL_BlitSurface(guest_screen, &rec, real_screen, &rec);
> >> -    SDL_UpdateRect(real_screen, x, y, w, h);
> >> +    SDL_UpdateRects(real_screen, 1, &rec);
> > 
> > Have you actually tried this? I'd be amazed and dismayed if it made any 
> > difference. I can believe that SDL_UpdateRects is better that *multiple* 
> > calls to SDL_UpdateRect. However I see absolutely no justification for this 
> > change.
> > 
> 
> Of course I didn't: I don't have any MacOSX available and this is the
> main reason I am asking other people to test this patch and doing it myself.
> 
> >>From the documentation I thought they were implemented differently
> somehow but now that you make me think about it I went through the
> actual code and SDL_UpdateRect is implemented using SDL_UpdateRects :(
> 
> Also I don't how to explain the fact that SDL_Flip is faster than
> SDL_UpdateRect on MacOSX according to malc: the same libsdl macosx faq I
>  linked before claims that SDL_DOUBLEBUF is not even supported on MacOsX.

Let me reiterate - SDL_UpdateRect was REMOVED from sdl_update, SDL_Flip 
was added to _sdl_refresh_. So, no, SDL_Flip is NOT faster than
SDL_UpdateRect.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

malc wrote:

> Let me reiterate - SDL_UpdateRect was REMOVED from sdl_update, SDL_Flip 
> was added to _sdl_refresh_. So, no, SDL_Flip is NOT faster than
> SDL_UpdateRect.
> 


This change also shouldn't have any effect since sdl_refresh is the one
that calls sdl_update  in the end.
The code path is the following:

sdl_refresh -> vga_hw_update -> vga_draw_graphic -> dpy_update -> sdl_update

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[malc]

On Mon, 23 Feb 2009, Stefano Stabellini wrote:

> malc wrote:
> 
> > Let me reiterate - SDL_UpdateRect was REMOVED from sdl_update, SDL_Flip 
> > was added to _sdl_refresh_. So, no, SDL_Flip is NOT faster than
> > SDL_UpdateRect.
> > 
> 
> 
> This change also shouldn't have any effect since sdl_refresh is the one
> that calls sdl_update  in the end.
> The code path is the following:
> 
> sdl_refresh -> vga_hw_update -> vga_draw_graphic -> dpy_update -> sdl_update

That would mean that the vga_hw_update chain calls sdl_update multiple
times.

-- 
mailto:av1474@comtv.ru

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Laurent Desnogues]

On Fri, Feb 20, 2009 at 5:52 PM, Paul Brook <paul@codesourcery.com> wrote:
>>      SDL_BlitSurface(guest_screen, &rec, real_screen, &rec);
>> -    SDL_UpdateRect(real_screen, x, y, w, h);
>> +    SDL_UpdateRects(real_screen, 1, &rec);
>
> Have you actually tried this? I'd be amazed and dismayed if it made any
> difference. I can believe that SDL_UpdateRects is better that *multiple*
> calls to SDL_UpdateRect. However I see absolutely no justification for this
> change.

UpdateRect is a thin wrapper around UpdateRects.  So thin I don't believe
it makes any difference (unless you're doing dozens of UpdateRect which
is not the case here as you pointed).


Laurent

void SDL_UpdateRect(SDL_Surface *screen, Sint32 x, Sint32 y, Uint32 w, Uint32 h)
{
	if ( screen ) {
		SDL_Rect rect;

		/* Perform some checking */
		if ( w == 0 )
			w = screen->w;
		if ( h == 0 )
			h = screen->h;
		if ( (int)(x+w) > screen->w )
			return;
		if ( (int)(y+h) > screen->h )
			return;

		/* Fill the rectangle */
		rect.x = (Sint16)x;
		rect.y = (Sint16)y;
		rect.w = (Uint16)w;
		rect.h = (Uint16)h;
		SDL_UpdateRects(screen, 1, &rect);
	}
}

Re: [Qemu-devel] Re: Qemu 2D performance plunges below acceptable levels

[Stefano Stabellini]

Alexey Eremenko wrote:

> On Tue, Feb 17, 2009 at 7:00 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
>> Avi Kivity wrote:
>>
>>> Stefano Stabellini wrote:
>>>> Do you see the problem only on X11 remote?
>>>> How bad is on vnc?
>>>>
>>> vnc regresses in exactly the same way.
>>>
> 
> Well, in my experience both local X11 and VNC are fast in KVM-84.
> 
> But remote X11/SDL performance plunged below usable levels.
> 


the performances of remote X11/SDL depend on the performances of SDL:
SDL blitting functions are optimized to be run locally, not remotely, so
is possible to see a slow down.