console font limits

console font limits

[Albert Cahalan]

I'm having problems with a font I just created. It's a rather big one,
intended for a framebuffer console in UTF-8 mode. The strace program
reports that /bin/setfont fails on a KDFONTOP ioctl with EINVAL.
In reading the kernel code, I find this:

vt.c:static int con_font_set(struct vc_data *vc, struct console_font_op *op)
vt.c-{
vt.c-   struct console_font font;
vt.c-   int rc = -EINVAL;
vt.c-   int size;
vt.c-
vt.c-   if (vc->vc_mode != KD_TEXT)
vt.c-           return -EINVAL;
vt.c-   if (!op->data)
vt.c-           return -EINVAL;
vt.c-   if (op->charcount > 512)
vt.c-           return -EINVAL;

Ouch. Why is the old VGA limit being applied to the framebuffer console?
Could this just get removed? I dearly hope we aren't still storing the
framebuffer data as two bytes per character+attribute pair.

I nearly hit the 32-pixel height limit as well, yet another relic from
the VGA hardware. I also nearly hit the 64 KB font size limit.

Currently I'm doing a 15x30 font with 870 glyphs to represent 978
different Unicode code points. This is for a 200 DPI display with
an anti-aliasing filter, so fonts need to be big. I'm considering 15x36
so that I'll have more room for double-accented letters, but clearly
the kernel would block that too.

BTW, the PSF font format documentation seems to suggest that
there is a way to make the kernel handle combining accents:
http://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html
Does anybody know if that really works? I could sure use that.

Re: console font limits

[Geert Uytterhoeven]

On Tue, 1 May 2007, Albert Cahalan wrote:
> I'm having problems with a font I just created. It's a rather big one,
> intended for a framebuffer console in UTF-8 mode. The strace program
> reports that /bin/setfont fails on a KDFONTOP ioctl with EINVAL.
> In reading the kernel code, I find this:
> 
> vt.c:static int con_font_set(struct vc_data *vc, struct console_font_op *op)
> vt.c-{
> vt.c-   struct console_font font;
> vt.c-   int rc = -EINVAL;
> vt.c-   int size;
> vt.c-
> vt.c-   if (vc->vc_mode != KD_TEXT)
> vt.c-           return -EINVAL;
> vt.c-   if (!op->data)
> vt.c-           return -EINVAL;
> vt.c-   if (op->charcount > 512)
> vt.c-           return -EINVAL;
> 
> Ouch. Why is the old VGA limit being applied to the framebuffer console?
> Could this just get removed? I dearly hope we aren't still storing the
> framebuffer data as two bytes per character+attribute pair.

The shadow screen (accessed using scr_*()) still uses the old VGA
format.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: console font limits

[Antonino A. Daplas]

On Tue, 2007-05-01 at 13:49 +0200, Geert Uytterhoeven wrote:
> On Tue, 1 May 2007, Albert Cahalan wrote:
> > I'm having problems with a font I just created. It's a rather big one,
> > intended for a framebuffer console in UTF-8 mode. The strace program
> > reports that /bin/setfont fails on a KDFONTOP ioctl with EINVAL.
> > In reading the kernel code, I find this:
> > 
> > vt.c:static int con_font_set(struct vc_data *vc, struct console_font_op *op)
> > vt.c-{
> > vt.c-   struct console_font font;
> > vt.c-   int rc = -EINVAL;
> > vt.c-   int size;
> > vt.c-
> > vt.c-   if (vc->vc_mode != KD_TEXT)
> > vt.c-           return -EINVAL;
> > vt.c-   if (!op->data)
> > vt.c-           return -EINVAL;
> > vt.c-   if (op->charcount > 512)
> > vt.c-           return -EINVAL;
> > 
> > Ouch. Why is the old VGA limit being applied to the framebuffer console?
> > Could this just get removed? I dearly hope we aren't still storing the
> > framebuffer data as two bytes per character+attribute pair.
> 
> The shadow screen (accessed using scr_*()) still uses the old VGA
> format.

And this will entail a lot of work to change (Is it worth it to rework
the code and remove the limitation?). The linux-console project
(http://linuxconsole.sourceforge.net/) might have , but I don't know its
current status.

Tony

Re: console font limits

[H. Peter Anvin]

Antonino A. Daplas wrote:
> 
> And this will entail a lot of work to change (Is it worth it to rework
> the code and remove the limitation?). The linux-console project
> (http://linuxconsole.sourceforge.net/) might have , but I don't know its
> current status.
> 

Well, I think the consensus is that anything beyond that should be done
in userspace; the main such console daemon was Kon2 last I checked.

	-hpa

Re: console font limits

[Albert Cahalan]

On 5/1/07, H. Peter Anvin <hpa@zytor.com> wrote:
> Antonino A. Daplas wrote:
> >
> > And this will entail a lot of work to change (Is it worth it to rework
> > the code and remove the limitation?). The linux-console project
> > (http://linuxconsole.sourceforge.net/) might have , but I don't know its
> > current status.
>
> Well, I think the consensus is that anything beyond that should be done
> in userspace; the main such console daemon was Kon2 last I checked.

Font size is not a sane place to draw the line. Features are.
The levels of support go something like this:

0. 7-bit ASCII
1. Simple direct-to-font VGA characters.
2. UTF-8 and large fonts, but no compositing or wide characters.
3. Simple compositing and double-wide characters. (like xterm)
4. Right-to-left. (like Kermit95)
5. Complex shaping, glyph substitution, and vertical text.

Without large fonts, UTF-8 is 90% pointless bloat.

Userspace console daemons are rotten to the core. There is no safe and
reliable way to make kernel messages pass through the userspace console.
You'd either be in graphics mode or you'd still be subject to the limit
of 256 simultaneous glyphs while normal VGA attributes are in use. This
is so defective that one might as well just run X with a fullscreen xterm.
If userspace is your answer, then let's rip out the UTF-8 code.

Personally I don't even need #1, but I think anything less than #3 is
really rude toward people outside of Europe+Americas. I especially hate
to hear Europeans argue against this when they have 100% precomposed
characters for themselves and appear to have played a role (via ISO votes)
in denying stuff like the mere 12 precomposed characters needed to use
the Yoruba language with simple renderers.

Re: console font limits

[Jan Engelhardt]

On May 1 2007 11:49, Albert Cahalan wrote:
>>
>> Well, I think the consensus is that anything beyond that should be done
>> in userspace; the main such console daemon was Kon2 last I checked.
>
> Font size is not a sane place to draw the line. Features are.
> The levels of support go something like this:
>
> 0. 7-bit ASCII
> 1. Simple direct-to-font VGA characters.
> 2. UTF-8 and large fonts, but no compositing or wide characters.
> 3. Simple compositing and double-wide characters. (like xterm)
> 4. Right-to-left. (like Kermit95)
> 5. Complex shaping, glyph substitution, and vertical text.
>
> Without large fonts, UTF-8 is 90% pointless bloat.

> Personally I don't even need #1, but I think anything less than #3 is
> really rude toward people outside of Europe+Americas. I especially hate
> to hear Europeans argue against this when they have 100% precomposed
> characters for themselves and appear to have played a role (via ISO votes)
> in denying stuff like the mere 12 precomposed characters needed to use
> the Yoruba language with simple renderers.

0. yes we want that

1. can't tell

2. utf8 yes, many text files are in that encoding.
   large fonts - can't tell, I am fine with the regular vga
   font infrastructure (8x16, 8x8)

3. compositing - no, don't need that,
   wide characters - does not even work in vga. just display a '??'
   and everything is fine.

4. I do not really think this has a future on VC.
   You would also 'need' kerning and that serif combiner thing (complex
   shaping?) for Arabic. At best, Arabic would look as horrible on VC
   as it does in xterm today (no RTL, no serif combiner)

5. Vertical text - who else supports this please? Webpages in languages
   that want to do TTB(top-to-bottom) scripting use html workarounds -
   probably because TTB availability it's not even guaranteed in a
   webbrowser.

In short, the current console is very much OK.


Jan
-- 

Re: console font limits

[Albert Cahalan]

On 5/2/07, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> On May 1 2007 11:49, Albert Cahalan wrote:
> >>
> >> Well, I think the consensus is that anything beyond that should be done
> >> in userspace; the main such console daemon was Kon2 last I checked.
> >
> > Font size is not a sane place to draw the line. Features are.
> > The levels of support go something like this:
> >
> > 0. 7-bit ASCII
> > 1. Simple direct-to-font VGA characters.
> > 2. UTF-8 and large fonts, but no compositing or wide characters.
> > 3. Simple compositing and double-wide characters. (like xterm)
> > 4. Right-to-left. (like Kermit95)
> > 5. Complex shaping, glyph substitution, and vertical text.
> >
> > Without large fonts, UTF-8 is 90% pointless bloat.
>
> > Personally I don't even need #1, but I think anything less than #3 is
> > really rude toward people outside of Europe+Americas. I especially hate
> > to hear Europeans argue against this when they have 100% precomposed
> > characters for themselves and appear to have played a role (via ISO votes)
> > in denying stuff like the mere 12 precomposed characters needed to use
> > the Yoruba language with simple renderers.

Note: I never suggested going beyond #3.

> 0. yes we want that
>
> 1. can't tell
>
> 2. utf8 yes, many text files are in that encoding.
>    large fonts - can't tell, I am fine with the regular vga
>    font infrastructure (8x16, 8x8)

Those sizes are unreadable on the 200 dpi OLPC XO screen,
and kind of icky on some of the really big desktop displays
when in native (framebuffer) mode. 200 dpi may be in your future.
Even the 32-pixel height limit is starting to be a problem.

> 3. compositing - no, don't need that,
>    wide characters - does not even work in vga. just display a '??'
>    and everything is fine.

It's been shown to be workable, and it allows support for
some additional languages.

> 4. I do not really think this has a future on VC.
>    You would also 'need' kerning and that serif combiner thing (complex
>    shaping?) for Arabic. At best, Arabic would look as horrible on VC
>    as it does in xterm today (no RTL, no serif combiner)

I agree. Hebrew is more doable, but probably not worth the effort
because of the rarity and because of the general lack of support
in text mode apps for such odd behavior. Very few emulators
support this; kermit95 is one of the few.

> 5. Vertical text - who else supports this please? Webpages in languages
>    that want to do TTB(top-to-bottom) scripting use html workarounds -
>    probably because TTB availability it's not even guaranteed in a
>    webbrowser.

I hope you didn't think I was suggesting this. It's quite absurd.
"Complex shaping, glyph substitution, and vertical text." was the
full item listed. Vertical is the least troublesome of those issues,
and as far as I know has never been implemented.

> In short, the current console is very much OK.

I wouldn't say that. We suffer the bloat of all this UTF-8 stuff
without being able to load a decent-sized font to go with it.
We're stuck at 256 characters really, with the very lame option
of trading foreground color intensity control for an extra 256.

I think one could make a reasonable argument that all the
internationalization is bloat, and that thus UTF-8 should go.
Given that we do support UTF-8 though, allowing a font with
more than 256 characters (with foreground intensity control)
is obviously sensible.

Re: console font limits

[Jan Engelhardt]

On May 3 2007 02:17, Albert Cahalan wrote:
> Note: I never suggested going beyond #3.
>
>> 0. yes we want that
>> 
>> 1. can't tell
>> 
>> 2. utf8 yes, many text files are in that encoding.
>> large fonts - can't tell, I am fine with the regular vga
>> font infrastructure (8x16, 8x8)
>
> Those sizes are unreadable on the 200 dpi OLPC XO screen,

Hm that should have read, for you:
I don't object implementing support for larger sizes.
(But I wonder how that should work without FB/CVIDIX/SVGA/VESA extensions.)

Note that I was assuming that no FB is used:

>> 3. compositing - no, don't need that,
>> wide characters - does not even work in vga. just display a '??'
>> and everything is fine.
>
> It's been shown to be workable, and it allows support for
> some additional languages.

What benefits does it actually bring? Your standard VGA knows 256/512
glyphs, and trying to display combined characters needs a
precomposed glyph.

>> 4. I do not really think this has a future on VC.
>> You would also 'need' kerning and that serif combiner thing (complex
>> shaping?) for Arabic. At best, Arabic would look as horrible on VC
>> as it does in xterm today (no RTL, no serif combiner)
>
> I agree. Hebrew is more doable, but probably not worth the effort
> because of the rarity and because of the general lack of support
> in text mode apps for such odd behavior. Very few emulators
> support this; kermit95 is one of the few.

For everything beyond Latin, fbiterm should work a lot better.

>> In short, the current console is very much OK.
>
> I wouldn't say that. We suffer the bloat of all this UTF-8 stuff
> without being able to load a decent-sized font to go with it.
> We're stuck at 256 characters really, with the very lame option
> of trading foreground color intensity control for an extra 256.

Making VC use a 16x32 font may be possible, but then you will also
need 4 glyph positions in a 4096 byte font to create a character.
Or the hardware needs to support 'non-standard' (that would be
8x8,12,14,16) fonts.


Jan
-- 

Re: console font limits

[Albert Cahalan]

On 5/3/07, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> On May 3 2007 02:17, Albert Cahalan wrote:

> > Those sizes are unreadable on the 200 dpi OLPC XO screen,
>
> Hm that should have read, for you:
> I don't object implementing support for larger sizes.
> (But I wonder how that should work without FB/CVIDIX/SVGA/VESA extensions.)
>
> Note that I was assuming that no FB is used:

I'm assuming that the FB is used. Neither of my two
computers can do VGA text mode. Even for computers
which can do VGA text mode, if you want large fonts
(either by number of characters or by character width)
you need to use FB. That's just a requirement; anything
else would be insane.

> For everything beyond Latin, fbiterm should work a lot better.

Then, as with X, you have problems with kernel messages.
Reliably sending printk through a userspace console is not
even possible. (consider a panic, OOM, or runaway RT task)

Re: console font limits

[Jan Engelhardt]

On May 3 2007 10:14, Albert Cahalan wrote:
> On 5/3/07, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
>> On May 3 2007 02:17, Albert Cahalan wrote:
>
>> > Those sizes are unreadable on the 200 dpi OLPC XO screen,
>> 
>> Hm that should have read, for you:
>> I don't object implementing support for larger sizes.
>> (But I wonder how that should work without FB/CVIDIX/SVGA/VESA
>> extensions.)
>> 
>> Note that I was assuming that no FB is used:
>
> I'm assuming that the FB is used. Neither of my two
> computers can do VGA text mode.

Some non-x86 boxes like sparc?

> Even for computers
> which can do VGA text mode, if you want large fonts
> (either by number of characters or by character width)
> you need to use FB. That's just a requirement; anything
> else would be insane.
>
>> For everything beyond Latin, fbiterm should work a lot better.
>
> Then, as with X, you have problems with kernel messages.
> Reliably sending printk through a userspace console is not
> even possible. (consider a panic, OOM, or runaway RT task)

Actually, the kernel could just "jump in". That is, when you start
in fb mode, you get the regular 8x16 vga font - on your choice of
resolution, and can also set it with setfont. So far so nice.
Now, when you start fbiterm -- for example to read CJK --, fbiterm
presents you with what looks like a 12x24 font. Also good.
It _looks_ to me like fbiterm does the drawing itself; ok too.
Now when the kernel is oopsing, it could just spew out the oops
trace in the regular font (vga8x16 or in case you use CONFIG_FONTS,
then that), possibly obstructing some output generated by fbiterm,
but in case of oops, I think this is ok.



Jan
-- 

Re: console font limits

[Geert Uytterhoeven]

On Thu, 3 May 2007, Jan Engelhardt wrote:
> On May 3 2007 10:14, Albert Cahalan wrote:
> > On 5/3/07, Jan Engelhardt <jengelh@linux01.gwdg.de> wrote:
> >> On May 3 2007 02:17, Albert Cahalan wrote:
> >
> >> > Those sizes are unreadable on the 200 dpi OLPC XO screen,
                                                 ^^^^
> >> 
> >> Hm that should have read, for you:
> >> I don't object implementing support for larger sizes.
> >> (But I wonder how that should work without FB/CVIDIX/SVGA/VESA
> >> extensions.)
> >> 
> >> Note that I was assuming that no FB is used:
> >
> > I'm assuming that the FB is used. Neither of my two
> > computers can do VGA text mode.
> 
> Some non-x86 boxes like sparc?

OLPC is x86 ;-)

> > Then, as with X, you have problems with kernel messages.
> > Reliably sending printk through a userspace console is not
> > even possible. (consider a panic, OOM, or runaway RT task)
> 
> Actually, the kernel could just "jump in". That is, when you start
> in fb mode, you get the regular 8x16 vga font - on your choice of
> resolution, and can also set it with setfont. So far so nice.
> Now, when you start fbiterm -- for example to read CJK --, fbiterm
> presents you with what looks like a 12x24 font. Also good.
> It _looks_ to me like fbiterm does the drawing itself; ok too.
> Now when the kernel is oopsing, it could just spew out the oops
> trace in the regular font (vga8x16 or in case you use CONFIG_FONTS,
> then that), possibly obstructing some output generated by fbiterm,
> but in case of oops, I think this is ok.

Funny, that's exactly how kernel messages where handled in the early
fbdev days: just draw them to the frame buffer, whatever mode it is
in...
Just like on the SunOS boxes ;-)

Put people didn't like that, and disabled text output when the console
is in KD_GRAPHICS mode...

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: console font limits

[Jan Engelhardt]

On May 3 2007 17:56, Geert Uytterhoeven wrote:
>> 
>> Actually, the kernel could just "jump in". That is, when you start
>> in fb mode, you get the regular 8x16 vga font - on your choice of
>> resolution, and can also set it with setfont. So far so nice.
>> Now, when you start fbiterm -- for example to read CJK --, fbiterm
>> presents you with what looks like a 12x24 font. Also good.
>> It _looks_ to me like fbiterm does the drawing itself; ok too.
>> Now when the kernel is oopsing, it could just spew out the oops
>> trace in the regular font (vga8x16 or in case you use CONFIG_FONTS,
>> then that), possibly obstructing some output generated by fbiterm,
>> but in case of oops, I think this is ok.
>
>Funny, that's exactly how kernel messages where handled in the early
>fbdev days: just draw them to the frame buffer, whatever mode it is
>in...
>Just like on the SunOS boxes ;-)

Right, I remember! But not only SunOS, _SPARCs_ 'themselves' also do it.
OBP is always sun-style, i.e. black-on-white and a big 12x22 font,
even if you jump to it from Linux using Stop-A.

>Put people didn't like that, and disabled text output when the console
>is in KD_GRAPHICS mode...

at the cost of not getting the kernel oops, heh.



Jan
-- 

Re: console font limits

[H. Peter Anvin]

Jan Engelhardt wrote:
> 
>> Put people didn't like that, and disabled text output when the console
>> is in KD_GRAPHICS mode...
> 
> at the cost of not getting the kernel oops, heh.
> 

I thought the reason we didn't display text in KD_GRAPHICS mode was that
KD_GRAPHICS might mean "in a completely different mode that only
userspace knows about."

	-hpa

Re: console font limits

[Jan Engelhardt]

On May 3 2007 13:15, H. Peter Anvin wrote:
>Jan Engelhardt wrote:
>> 
>>> Put people didn't like that, and disabled text output when the console
>>> is in KD_GRAPHICS mode...
>> 
>> at the cost of not getting the kernel oops, heh.
>
>I thought the reason we didn't display text in KD_GRAPHICS mode was that
>KD_GRAPHICS might mean "in a completely different mode that only
>userspace knows about."

Hrm. Maybe we need a distinction into KD_KGRAPHICS and KD_UGRAPHICS then.


Jan
-- 

Re: console font limits

[Kyle Moffett]

On May 03, 2007, at 16:16:51, Jan Engelhardt wrote:
> On May 3 2007 13:15, H. Peter Anvin wrote:
>> Jan Engelhardt wrote:
>>>
>>>> Put people didn't like that, and disabled text output when the  
>>>> console is in KD_GRAPHICS mode...
>>>
>>> at the cost of not getting the kernel oops, heh.
>>
>> I thought the reason we didn't display text in KD_GRAPHICS mode  
>> was that KD_GRAPHICS might mean "in a completely different mode  
>> that only userspace knows about."
>
> Hrm. Maybe we need a distinction into KD_KGRAPHICS and KD_UGRAPHICS  
> then.

Actually I think the real problem was that "KD_GRAPHICS" got  
overloaded to mean "some userspace program is probably poking at the  
GPU in very direct ways possibly including /dev/mem".  As such it  
really isn't safe at all for the kernel to write stuff to the screen  
in that situation; you could turn a panic()+reboot-after-30-secs into  
an unrecoverable hard PCI bus lockup.  IIRC there were at least a  
couple chipsets which had that problem with X.  If we can implement  
enough APIs for X to do all of its stuff from userspace without iopl 
() or /dev/mem then we could probably bring back the option for  
dumping oopses to screen in KD_GRAPHICS mode, but otherwise it'll  
just cause more headaches.

Cheers,
Kyle Moffett

Re: console font limits

[H. Peter Anvin]

Kyle Moffett wrote:
> 
> Actually I think the real problem was that "KD_GRAPHICS" got overloaded
> to mean "some userspace program is probably poking at the GPU in very
> direct ways possibly including /dev/mem".  As such it really isn't safe
> at all for the kernel to write stuff to the screen in that situation;
> you could turn a panic()+reboot-after-30-secs into an unrecoverable hard
> PCI bus lockup.  IIRC there were at least a couple chipsets which had
> that problem with X.  If we can implement enough APIs for X to do all of
> its stuff from userspace without iopl() or /dev/mem then we could
> probably bring back the option for dumping oopses to screen in
> KD_GRAPHICS mode, but otherwise it'll just cause more headaches.
> 

It never meant anything *BUT* that, to the best of my knowledge.  That
was certainly the original meaning of KD_GRAPHICS.

	-hpa

Re: console font limits

[Daniel Hazelton]

On Thursday 03 May 2007 20:39:05 H. Peter Anvin wrote:
> Kyle Moffett wrote:
> > Actually I think the real problem was that "KD_GRAPHICS" got overloaded
> > to mean "some userspace program is probably poking at the GPU in very
> > direct ways possibly including /dev/mem".  As such it really isn't safe
> > at all for the kernel to write stuff to the screen in that situation;
> > you could turn a panic()+reboot-after-30-secs into an unrecoverable hard
> > PCI bus lockup.  IIRC there were at least a couple chipsets which had
> > that problem with X.  If we can implement enough APIs for X to do all of
> > its stuff from userspace without iopl() or /dev/mem then we could
> > probably bring back the option for dumping oopses to screen in
> > KD_GRAPHICS mode, but otherwise it'll just cause more headaches.
>
> It never meant anything *BUT* that, to the best of my knowledge.  That
> was certainly the original meaning of KD_GRAPHICS.

I started work last year on making the framebuffer layer use the DRM internals 
for all controls, providing a unified kernel and userspace system for 
accessing the graphics devices. It never got anywhere because I couldn't 
figure out a simple system for figuring out which driver (out of the numerous 
ones that could potentially be compiled into the kernel) to actually give 
control to. (I know I could have just looped over them all and figured it out 
that way, but that is far from elegant)

I guess I could start on that work again - shouldn't take me all that long to 
recover the stuff I lost when a blackout caused my hard drive to get 
corrupted beyond recovery (and the automated journal replay didn't do a 
damned thing - I think it actually *added* to the corruption, but I don't 
think any filesystem would have survived that)

DRH

Re: console font limits

[Antonino A. Daplas]

On Thu, 2007-05-03 at 23:58 -0400, Daniel Hazelton wrote:
> On Thursday 03 May 2007 20:39:05 H. Peter Anvin wrote:
> > Kyle Moffett wrote:

> I guess I could start on that work again - shouldn't take me all that long to 
> recover the stuff I lost when a blackout caused my hard drive to get 
> corrupted beyond recovery (and the automated journal replay didn't do a 
> damned thing - I think it actually *added* to the corruption, but I don't 
> think any filesystem would have survived that)

You might want to look at the modesetting-101 branch of DRM.  It's goal
is similar to yours.  They even have a drm framebuffer.  I don't know
how far they are with their goal, but I can see some progress.

Here's their git tree:

git://git.freedesktop.org/git/mesa/drm#modesetting-101

Re: console font limits

[Jesse Barnes]

On Thursday, May 03, 2007, Antonino A. Daplas wrote:
> On Thu, 2007-05-03 at 23:58 -0400, Daniel Hazelton wrote:
> > On Thursday 03 May 2007 20:39:05 H. Peter Anvin wrote:
> > > Kyle Moffett wrote:
> >
> > I guess I could start on that work again - shouldn't take me all that
> > long to recover the stuff I lost when a blackout caused my hard drive
> > to get corrupted beyond recovery (and the automated journal replay
> > didn't do a damned thing - I think it actually *added* to the
> > corruption, but I don't think any filesystem would have survived that)
>
> You might want to look at the modesetting-101 branch of DRM.  It's goal
> is similar to yours.  They even have a drm framebuffer.  I don't know
> how far they are with their goal, but I can see some progress.
>
> Here's their git tree:
>
> git://git.freedesktop.org/git/mesa/drm#modesetting-101

It sounds like we'll need a new KD_* flag for this work too, something like 
KD_TEXT_BUT_ONLY_ON_PANIC, sine with the above tree, the kernel will have 
intimite knowledge of the current graphics mode, framebuffer location, 
etc.  It should be enough to clear the scanout buffer and output the 
printk, though if there's a lot of rendering going on, the DRM driver 
might have to be pretty smart about it.

Jesse

Re: console font limits

[Daniel Hazelton]

On Friday 04 May 2007 11:34:40 Jesse Barnes wrote:
> On Thursday, May 03, 2007, Antonino A. Daplas wrote:
> > On Thu, 2007-05-03 at 23:58 -0400, Daniel Hazelton wrote:
> > > On Thursday 03 May 2007 20:39:05 H. Peter Anvin wrote:
> > > > Kyle Moffett wrote:
> > >
> > > I guess I could start on that work again - shouldn't take me all that
> > > long to recover the stuff I lost when a blackout caused my hard drive
> > > to get corrupted beyond recovery (and the automated journal replay
> > > didn't do a damned thing - I think it actually *added* to the
> > > corruption, but I don't think any filesystem would have survived that)
> >
> > You might want to look at the modesetting-101 branch of DRM.  It's goal
> > is similar to yours.  They even have a drm framebuffer.  I don't know
> > how far they are with their goal, but I can see some progress.
> >
> > Here's their git tree:
> >
> > git://git.freedesktop.org/git/mesa/drm#modesetting-101

Nope. Address doesn't work for me. 

> It sounds like we'll need a new KD_* flag for this work too, something like
> KD_TEXT_BUT_ONLY_ON_PANIC, sine with the above tree, the kernel will have
> intimite knowledge of the current graphics mode, framebuffer location,
> etc.  It should be enough to clear the scanout buffer and output the
> printk, though if there's a lot of rendering going on, the DRM driver
> might have to be pretty smart about it.

Yeah, I had thought about this. What I was thinking was that, on panic, the 
kernel could either kill the graphics mode entirely or do something like what 
the Amiga used to do - ie: scroll the screeen *down* and insert the error 
output at the top. (well, the opposite, really - scroll up and insert, just 
to keep the code simple and elegant)

Other than that, well... I figure that if the kernel is using DRM internally 
it can easily tell DRM to cancel out all extended rendering contexts - 
OpenGL, overlays and such - when the panic happens.

DRH

Re: console font limits

[H. Peter Anvin]

Albert Cahalan wrote:
> Font size is not a sane place to draw the line. Features are.

Font sizes are an eminently sane place to draw the line, because the
crossover is largely determined by the point at which nailing down too
much memory (kernel memory = unswappable memory) gets too expensive.

	-hpa

Re: console font limits

[Helge Hafting]

H. Peter Anvin wrote:
> Albert Cahalan wrote:
>   
>> Font size is not a sane place to draw the line. Features are.
>>     
>
> Font sizes are an eminently sane place to draw the line, because the
> crossover is largely determined by the point at which nailing down too
> much memory (kernel memory = unswappable memory) gets too expensive.
>   
But then the sane limit is a number of pages, not "512 glyphs".

And there is no problem if the memory is allocated dynamically
when loading the font.  Americans may then load a 127-glyph
font without wasting memory.  And those who wish to lock down
5MB unswappable memory for a hi-res chinese console can do that too.
Doing so isn't really a problem on a desktop pc.

Helge Hafting

Re: console font limits

[Alan Cox]

On Thu, 03 May 2007 00:19:59 -0700
"H. Peter Anvin" <hpa@zytor.com> wrote:

> Albert Cahalan wrote:
> > Font size is not a sane place to draw the line. Features are.
> 
> Font sizes are an eminently sane place to draw the line, because the
> crossover is largely determined by the point at which nailing down too
> much memory (kernel memory = unswappable memory) gets too expensive.

Which is a value judgement made based upon the cost of memory, the
memory limit of the system (and the fixed kmalloc limits).

Obvious fix Albert would be to post patches which 

	-	Ignore the font size limits (up to sanity point) for 
		CAP_SYS_something
	-	Put the limit in sysfs where it belongs
	-	Use vmalloc not kmalloc if neccessary

Alan

Re: console font limits

[Ken Moffat]

On Tue, May 01, 2007 at 12:09:46AM -0400, Albert Cahalan wrote:
> 
> BTW, the PSF font format documentation seems to suggest that
> there is a way to make the kernel handle combining accents:
> http://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html
> Does anybody know if that really works? I could sure use that.

 My reading of it suggests you need a precomposed version to which
the combining combination can be mapped (the 00c5 for Å in the
example).  Possibly, the fffe and ffff are used as delimiters for
the combination).  Maybe, it could be used for yoruba (yeah, I was
looking at that an hour before I read your mails) _if_ the font
has room to fit in a "private use" character which can hold the
precomposed value.

 For any letter where a precomposed version is in the unicode
standards, the only use for combining characters in a console font
seems to be as a way of displaying text where somebody has already
used them.

 For latin languages which need additional composed letters, the
bigger problem will probably be the input - I've no idea if the
console keymap can use combining keys (direct input of the unicode
hex is available, of course), and I can't manage to get the direct
input of the codes for combining diacriticals to work in combination
in my preferred graphical terminal.

Ken
-- 
das eine Mal als Tragödie, das andere Mal als Farce

Re: console font limits

[Andries Brouwer]

On Tue, May 01, 2007 at 12:09:46AM -0400, Albert Cahalan wrote:
> I'm having problems with a font I just created. It's a rather big one,
> intended for a framebuffer console in UTF-8 mode. The strace program
> reports that /bin/setfont fails on a KDFONTOP ioctl with EINVAL.
> In reading the kernel code, I find this:
> 
> vt.c:static int con_font_set(struct vc_data *vc, struct console_font_op *op)
> vt.c-{
> vt.c-   struct console_font font;
> vt.c-   int rc = -EINVAL;
> vt.c-   int size;
> vt.c-
> vt.c-   if (vc->vc_mode != KD_TEXT)
> vt.c-           return -EINVAL;
> vt.c-   if (!op->data)
> vt.c-           return -EINVAL;
> vt.c-   if (op->charcount > 512)
> vt.c-           return -EINVAL;
> 
> Ouch. Why is the old VGA limit being applied to the framebuffer console?

This is common code, for both framebuffer and vga.
If you want you can push this check into the individual foo->con_font_set
routines. Have some check here that the total size is not ridiculous.

(There is a loop
	for (i = 0; i < op->charcount; i++)
that must have some bounded length. Easiest is to test against the total
font size some lines later:
	size = (op->width+7)/8 * 32 * op->charcount;
        if (size > max_font_size)
                return -ENOSPC;
E.g., move this up so that it becomes
	if (op->width <= 0 || op->width > 32)
		return -EINVAL;
	if (op->charcount > max_font_size / 32 / ((op->width+7)/8))
		return -ENOSPC;
.)

> I nearly hit the 32-pixel height limit as well, yet another relic from
> the VGA hardware. I also nearly hit the 64 KB font size limit.

Of course you can make everything more general if you want to and need to.
Some of this code was written when average machines had 4 or 8 MB of memory.
On the other hand, for VGA some of the limits are hardware restrictions.

> Currently I'm doing a 15x30 font with 870 glyphs to represent 978
> different Unicode code points. This is for a 200 DPI display with
> an anti-aliasing filter, so fonts need to be big. I'm considering 15x36
> so that I'll have more room for double-accented letters, but clearly
> the kernel would block that too.

In case you generalize stuff in such a way that also the kbd package
needs an update, write.

> BTW, the PSF font format documentation seems to suggest that
> there is a way to make the kernel handle combining accents:
> http://www.win.tue.nl/~aeb/linux/kbd/font-formats-1.html
> Does anybody know if that really works? I could sure use that.

Well, the PSF font format documentation shows that there is a way
to document in a font that some font position represents a
sequence of Unicode values. Whether the kernel is able to
use such information depends on the kernel version - maybe vanilla
kernels cannot.

[Very long ago I wrote Yiddish songs on the console, German on the left,
Yiddish on the right, and made a bidirectional console for this purpose.
Long ago I wrote Tibetan on the console and added enough input- and
output stuff to make that work. Only a small part of this has ended up
in the vanilla kernel. Adding features to the console is not a popular idea.]

Get the kbd package and read kdmapop.c for the docs on the kbd ioctls.
Tell me if these docs are outdated. I read

/*
 * Linux pre-0.96 defined GIO_SCRNMAP, PIO_SCRNMAP:
        typedef char scrnmap_t;
        #define E_TABSZ         256
        #define GIO_SCRNMAP     0x4B40
        #define PIO_SCRNMAP     0x4B41
 * and Linux 0.99.14y first implemented them.
 * Usage:
        scrnmap_t map[E_TABSZ];
        ioctl(fd,GIO_SCRNMAP,map);
        ioctl(fd,PIO_SCRNMAP,map);
 * to read or write the kernel translation table that is
 * applied to user application output before displaying.
 *
 * Before 1.3.1, the character set was completely undetermined,
 * and if the font was in an order different from the character
 * set in use, the user screen map was used to map application
 * codes to font indices. (To be more precise: there were four
 * translation tables, and this ioctl would get/set the fourth
 * table, while the other three tables are built-in and constant.)
 */

/*
 * Linux 1.3.1 introduces GIO_UNISCRNMAP, PIO_UNISCRNMAP:
        #define GIO_UNISCRNMAP  0x4B69
        #define PIO_UNISCRNMAP  0x4B6A
 * Usage:
        unsigned short umap[E_TABSZ];
        ioctl(fd,GIO_UNISCRNMAP,umap);
        ioctl(fd,PIO_UNISCRNMAP,umap);
 * to read or write the kernel translation table that is
 * applied to user application output before displaying.
 * (When the console is not in utf8 mode.)
 *
 * The idea is that the umap values are 16-bit unicode (ucs2)
 * values, and that the fonts will have an index giving the
 * unicode value for each glyph, so that the kernel can match up
 * application codes to font positions.
        #define UNI_DIRECT_BASE 0xF000
        #define UNI_DIRECT_MASK 0x01FF
 * For compatibility, and for fonts without table, the unicode
 * values 0xF000+n, 0 <= n <= 0x01FF, acts as direct font indices.
 * In the new scheme, the old PIO_SCRNMAP fills the kernel umap
 * table with such direct-to-font values.
 */

/*
 * Linux 1.1.63 introduces GIO_UNIMAP, PIO_UNIMAP, PIO_UNIMAPCLR:
        #define GIO_UNIMAP     0x4B66
        #define PIO_UNIMAP     0x4B67
        #define PIO_UNIMAPCLR  0x4B68
 * And Linux 1.1.92 implements them.
 * Usage:
        struct unimapinit {
                unsigned short advised_hashsize;
                unsigned short advised_hashstep;
                unsigned short advised_hashlevel;
        } ui;
        ioctl(fd, PIO_UNIMAPCLR, &ui);
 * to clear the unimap table and advise about the kind of
 * hash setup appropriate to what one is going to load
 * (with 0 for "don't care").
        struct unipair {
                unsigned short unicode;
                unsigned short fontpos;
        };
        struct unimapdesc {
                unsigned short entry_ct;
                struct unipair *entries;
        } ud;
        ioctl(fd, PIO_UNIMAP, &ud);
        ioctl(fd, GIO_UNIMAP, &ud);
 * to add the indicated pairs to the kernel unimap table
 * or to read the kernel unimap table, where in the latter case
 * ud.entry_ct indicated the room available.
 *
 * In Linux 1.3.28 the hash table was replaced by a 3-level
 * paged table, so the contents of a struct unimapinit are
 * no longer meaningful.
 *
 * Linux 2.6.1 makes GIO_UNIMAP, PIO_UNIMAP, PIO_UNIMAPCLR per-vt
 * so that fd no longer is random.
 */

Andries

[There is also a file with kbd docs I once wrote, under kernel/people/aeb.
Should probably be moved to Documentation, but it will be outdated today.]