Huge console switching lags

Huge console switching lags

[Lorenzo Allegrucci]

I've experienced huge (4/5 seconds) console switching lags with
2.4.10 running this [1], never seen before with any kernel.
2.4.10-ac2 is even worse, it can take up to 10/20 seconds and longer
to switch from a console to another (CTRL+F1,F2 etc) while running
the beast below:

[1]
#!/bin/sh
bomb(){bomb|bomb&};bomb

Swap is not an issue, you can swapoff -a and still have lags.
I've never seen any console switch lags with any kernel on any load.



-- 
Lorenzo

Re: Huge console switching lags

[Andrew Morton]

Lorenzo Allegrucci wrote:
> 
> I've experienced huge (4/5 seconds) console switching lags with
> 2.4.10 running this [1], never seen before with any kernel.

In 2.4.10, the console switching code moved from interrupt context
into process context.  So if your system is taking a long time to
schedule processes (in this case, keventd) then yes, console
switching will take a long time.

> 2.4.10-ac2 is even worse, it can take up to 10/20 seconds and longer
> to switch from a console to another (CTRL+F1,F2 etc) while running
> the beast below:
> 
> [1]
> #!/bin/sh
> bomb(){bomb|bomb&};bomb
> 

The simple ones are always the best ones, aren't they?

-

Re: Huge console switching lags

[James Simmons]

> > [1]
> > #!/bin/sh
> > bomb(){bomb|bomb&};bomb
> > 
> 
> The simple ones are always the best ones, aren't they?

Nice fork bomb :-/

Re: Huge console switching lags

[Ricky Beam]

On Tue, 2 Oct 2001, Andrew Morton wrote:
>In 2.4.10, the console switching code moved from interrupt context
>into process context.  So if your system is taking a long time to
>schedule processes (in this case, keventd) then yes, console
>switching will take a long time.

And what's the brilliant reason for this?  And don't give any BS about it
taking too long inside an interrupt context -- we're switching consoles not
start netscrape.

--Ricky

Re: Huge console switching lags

[Andrew Morton]

Ricky Beam wrote:
> 
> On Tue, 2 Oct 2001, Andrew Morton wrote:
> >In 2.4.10, the console switching code moved from interrupt context
> >into process context.  So if your system is taking a long time to
> >schedule processes (in this case, keventd) then yes, console
> >switching will take a long time.
> 
> And what's the brilliant reason for this?  And don't give any BS about it
> taking too long inside an interrupt context -- we're switching consoles not
> start netscrape.
> 

It takes too long in interrupt context :)  Tens of milliseconds
or more.  More importantly, it sorts the locking out - you can't
acquire a semaphore in interrupt context.

Re: Huge console switching lags

[Alan Cox]

> And what's the brilliant reason for this?  And don't give any BS about it
> taking too long inside an interrupt context -- we're switching consoles not
> start netscrape.

A console switch has to wait until queued I/O to that console is complete,
which in turn could take a measurable amount of time - so it has to block.
Also a console switch on a frame buffer with no hardware banking can take
a lot of time.


Alan

Re: Huge console switching lags

[Ricky Beam]

On Tue, 2 Oct 2001, Alan Cox wrote:
>A console switch has to wait until queued I/O to that console is complete,

Ok, so fix that. (assuming that's not "waiting on the hardware" queued IO)

>Also a console switch on a frame buffer with no hardware banking can take
>a lot of time.

Oh, *grin*, forgot about those evil framebuffer consoles. (never use them
myself, they really are freakin' slow.)  Arguablly, all access to fbdev's
should be from a process context (it's like having X in the kernel.)

In that case, keventd needs to be a high priority real-time task.  If it
takes SECONDS to change consoles, I'm very likely to assume the machine is
locked and push the reset button (and I'm sure many others will do the same.)

--Ricky

Re: Huge console switching lags

[Alan Cox]

> In that case, keventd needs to be a high priority real-time task.  If it
> takes SECONDS to change consoles, I'm very likely to assume the machine is
> locked and push the reset button (and I'm sure many others will do the same.)

Well one way to test that would be to renice it and see

Re: Huge console switching lags

[James Simmons]

> >A console switch has to wait until queued I/O to that console is complete,
> 
> Ok, so fix that. (assuming that's not "waiting on the hardware" queued IO)

Strange you are having this problem. I have been using Morton's patch for
months now with the framebuffer layer and have no problems. Well I have
done some modifications which I will post shortly.

> >Also a console switch on a frame buffer with no hardware banking can take
> >a lot of time.
> 
> Oh, *grin*, forgot about those evil framebuffer consoles. (never use them
> myself, they really are freakin' slow.)  Arguablly, all access to fbdev's
> should be from a process context (it's like having X in the kernel.)

Well the reason the framebuffer suck is because the current api sucks for
them. It draws pixel by pixel. Slow slow slow!!! I have developed a new
api that takes advantage of the accel engine of graphics hardware. It is
much simpler and more flexiable and most important much much faster. Yes
there exist graphics hardware that are pure dummy framebuffers. Here I
have written soft accels (fillrect, draw image, copy area). They are much
faster than the current implemenation. I haven't even assemblty optimized
them or used MMX type things yet. Once that is done it will be very fast.

The point of the console locking change was because their exist graphics
hardware that is IRQ/DMA based only. Using a spinlock turns off printing
to the screen. Oops, big problem. With the new console locking mechanism
we can use the DMA/irq engine of any graphics card. This is much much
faster than using the mmio region like some of the fbdev drivers do. It is
faster than drawing pixel by pixel but it could be much faster.

Re: Huge console switching lags

[Alan Cox]

> Well the reason the framebuffer suck is because the current api sucks for
> them. It draws pixel by pixel. Slow slow slow!!! I have developed a new
> api that takes advantage of the accel engine of graphics hardware. It is

Great. VESAfb doesnt have one. Lots of older machines dont have one.

Alan

Re: Huge console switching lags

[James Simmons]

> > Well the reason the framebuffer suck is because the current api sucks for
> > them. It draws pixel by pixel. Slow slow slow!!! I have developed a new
> > api that takes advantage of the accel engine of graphics hardware. It is
> 
> Great. VESAfb doesnt have one. Lots of older machines dont have one.

   True. Of course VESAfb exist because we lack so fbdev drivers. In time
that problem should go away. Also many embedded devices, which I do for a
living, lack hardware acceleration. Well okay alot of modern PDA's are
staring to have accel engines. This doesn't mean you can't write really
good optimized software code for devices that lack hardware acceleration.
   The software accel functions needed by the console layer (copyarea,
fillrect, and drawimage) have been already written. Okay the drawimage one
needs alot of work. I haven't benchmarked the new code versus the current
code but you can see the difference. One of the big changes I have have
made is that on write data to the framebuffer word aligned and a long at
a time. For 8bpp you have 4 pixels written at a time. This makes for a
much tigher loop. On ix86 you can see a huge difference in performance due
to the word alignment. I knwo because at first I had a bug that wasn't
doing it right. After I fix that bug you could see the difference. 
    We use the same techniques at work for embedded devices where the cpus 
don't have the horse power like desktops. Every single line of code counts. 
I haven't ported the assembly versions for different platforms yet but I
plan to. I know from experience writing proper assembly on ARM or using
MMX will increase preformance many fold. 

Re: Huge console switching lags

[Alan Cox]

>    The software accel functions needed by the console layer (copyarea,
> fillrect, and drawimage) have been already written. Okay the drawimage one
> needs alot of work. I haven't benchmarked the new code versus the current

On x86 they'll probably make no difference at all, unless the old code
is really really crap. Your bottleneck is the PCI bus. All you can do is
avoid reads.

Alan

Re: Huge console switching lags

[James Simmons]

> On x86 they'll probably make no difference at all, unless the old code
> is really really crap. Your bottleneck is the PCI bus. All you can do is
> avoid reads.

True. We have discussed the idea of placing the fonts into video memory
instead of system memory if the graphics card has room. At first I didn't
like the idea since handling scrolling would become more difficult. It can
be done tho with enough "tricks". I think it should be up to the driver
write where he/she can place the font image. This case drawimage becomes
copyarea except you grabbing off screen data. I have some thinking about
how to handle that.  

Re: Huge console switching lags

[Benjamin Herrenschmidt]

>>    The software accel functions needed by the console layer (copyarea,
>> fillrect, and drawimage) have been already written. Okay the drawimage one
>> needs alot of work. I haven't benchmarked the new code versus the current
>
>On x86 they'll probably make no difference at all, unless the old code
>is really really crap. Your bottleneck is the PCI bus. All you can do is
>avoid reads.

Well, there are indeed a few improvements to get with machine specific
optimisations on unaccelerated framebuffer.

One example is, on PPC, the use of a floating point register to do the
blits 64 bits at a time. This allow the PCI host controller to generate
bursts of 2 32 bits transactions (for machines with controllers unable
to write combine). Of course, having such optimisations in the kernel
is tricky because of the lazy FPU switching (well, at least on PPC),
but the point is that improvement _is_ possible.

Regards,
Ben.

Re: Huge console switching lags

[James Simmons]

> Well, there are indeed a few improvements to get with machine specific
> optimisations on unaccelerated framebuffer.
[snip]...

Neat trick. Please note also that no read operations to the framebuffer
are done by the fbcon layer. Such reads should be to the shadow buffers
(vc_screenbuffer) instead. Reading the framebuffer is a userland operation
and as such you really only tricks for reading in userland. 

Re: Huge console switching lags

[Paul Mundt]

[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]

On Wed, Oct 03, 2001 at 09:58:30AM -0700, James Simmons wrote:
> > Well, there are indeed a few improvements to get with machine specific
> > optimisations on unaccelerated framebuffer.
> [snip]...
> 
> Neat trick. Please note also that no read operations to the framebuffer
> are done by the fbcon layer. Such reads should be to the shadow buffers
> (vc_screenbuffer) instead. Reading the framebuffer is a userland operation
> and as such you really only tricks for reading in userland. 
> 
And while we're on the subject of architecture specific optimizations for
unaccelerated framebuffers (or framebuffers in general for that matter),
on SH4 you can remap the video memory area through a store queue and perform
all writes through the remapped store queue area (there are two store queues,
each are 32bytes, and are flushed to the memory they were mapped to on a
prefetch instruction). This allows for very high speed writes to external
memory, as it was designed for.

Regards,

-- 
Paul Mundt <pmundt@mvista.com>
MontaVista Software, Inc.


[-- Attachment #2: Type: application/pgp-signature, Size: 240 bytes --]

Re: Huge console switching lags

[Geert Uytterhoeven]

On Tue, 2 Oct 2001, James Simmons wrote:
> > > Well the reason the framebuffer suck is because the current api sucks for
> > > them. It draws pixel by pixel. Slow slow slow!!! I have developed a new
            ^^^^^^^^^^^^^^^^^^^^^^^
Where does it draw pixel by pixel?

> > > api that takes advantage of the accel engine of graphics hardware. It is
> > 
> > Great. VESAfb doesnt have one. Lots of older machines dont have one.
> 
>    True. Of course VESAfb exist because we lack so fbdev drivers. In time

Yep. Vesafb started as a nice gimmick to show that it's possible, and turned
out to be a solution for yet another
we-don't-release-specs-to-OS/FS-people company.

>    The software accel functions needed by the console layer (copyarea,
> fillrect, and drawimage) have been already written. Okay the drawimage one
> needs alot of work. I haven't benchmarked the new code versus the current
> code but you can see the difference. One of the big changes I have have
> made is that on write data to the framebuffer word aligned and a long at
> a time. For 8bpp you have 4 pixels written at a time. This makes for a
> much tigher loop. On ix86 you can see a huge difference in performance due
> to the word alignment. I knwo because at first I had a bug that wasn't
> doing it right. After I fix that bug you could see the difference. 

Euh, most fbcon-* drivers already do this. Grep for fb_write in e.g.
drivers/video/fbcon-cfb8.c and count the byte accesses (=> 0).

Gr{oetje,eeting}s,

						Geert

P.S. Not to criticize the development in the Ruby tree of the linux-console
     project, but I don't like facts that aren't true.
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Huge console switching lags

[James Simmons]

> > > > them. It draws pixel by pixel. Slow slow slow!!! I have developed a new
>             ^^^^^^^^^^^^^^^^^^^^^^^
> Where does it draw pixel by pixel?

Okay. Let me say most drivers don't take advantage of the graphics hardware 
to perform console operations. Instead they just draw directly to the
framebuffer which can be slow. 

> Yep. Vesafb started as a nice gimmick to show that it's possible, and turned
> out to be a solution for yet another
> we-don't-release-specs-to-OS/FS-people company.

I know. Same with OFfb.

> Euh, most fbcon-* drivers already do this. Grep for fb_write in e.g.
> drivers/video/fbcon-cfb8.c and count the byte accesses (=> 0).

Yep. The new code I developed came out the merging of all the fbcon-cfb*
drivers. 

Re: Huge console switching lags

[Chris Wedgwood]

On Tue, Oct 02, 2001 at 02:10:41PM -0400, Ricky Beam wrote:

    And what's the brilliant reason for this?  And don't give any BS
    about it taking too long inside an interrupt context -- we're
    switching consoles not start netscrape.

Having console switching in interrupt context was horrible... I would
much rather have the 'odd' slightly delayed console switch than drop
stuff or have sound jitter every time I switch console (which sucks
badly).

Under even fairly heavy load here it's not a problem, what problems
does the new bahaviour create (sorry, a fork bomb is a silly
example).  With a load-average of over 20 its still responsive.





  --cw