VT console need rewrite

VT console need rewrite

[Microcai]

Hi, there

	I'm implementing the UNICODE font of the framebuffer console, (see
http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
current vt code is too bugy, too many direct assumes about vt buffer,
This makes me so hard to hack.  There is TODO telling me to add UNICODE
support, but no room for such code, that's why my patch is so tricky.

	And the code itself, if you'll excuse me, it isn't as beautiful as rest
of the kernel.
	So, it really really need a clean rewrite.I'm ganna take is hard job.	
	And, please tell me if is worth to do so.

Re: VT console need rewrite

[Theodore Tso]

On Nov 28, 2010, at 5:57 AM, Microcai wrote:

> Hi, there
> 
> 	I'm implementing the UNICODE font of the framebuffer console, (see
> http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> current vt code is too bugy, too many direct assumes about vt buffer,
> This makes me so hard to hack.  There is TODO telling me to add UNICODE
> support, but no room for such code, that's why my patch is so tricky.
> 
> 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> of the kernel.
> 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> 	And, please tell me if is worth to do so.

Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break backwards compatibility is hard.  i.e., it is hard to hack for a reason.

I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...

-- Ted

Re: VT console need rewrite

[Microcai]

在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
> 
> > Hi, there
> > 
> > 	I'm implementing the UNICODE font of the framebuffer console, (see
> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> > current vt code is too bugy, too many direct assumes about vt buffer,
> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
> > support, but no room for such code, that's why my patch is so tricky.
> > 
> > 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> > of the kernel.
> > 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> > 	And, please tell me if is worth to do so.
> 
> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break backwards compatibility is hard.  i.e., it is hard to hack for a reason.
> 
> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...
> 
> -- Ted
> 

Yeah, I'd also like to rewrite it incrementally. But... who will accept
that incrementally patch ? It just seems that incremental patch will be
horrible at the beginning...... It will be discard without a
reason .....


--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[jonsmirl]

On Sun, Nov 28, 2010 at 8:42 AM, Microcai <microcai@fedoraproject.org> wrote:
> 在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
>> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
>>
>> > Hi, there
>> >
>> >     I'm implementing the UNICODE font of the framebuffer console, (see
>> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
>> > current vt code is too bugy, too many direct assumes about vt buffer,
>> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
>> > support, but no room for such code, that's why my patch is so tricky.
>> >
>> >     And the code itself, if you'll excuse me, it isn't as beautiful as rest
>> > of the kernel.
>> >     So, it really really need a clean rewrite.I'm ganna take is hard job.
>> >     And, please tell me if is worth to do so.
>>
>> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break backwards compatibility is hard.  i.e., it is hard to hack for a reason.
>>
>> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...
>>
>> -- Ted
>>
>
> Yeah, I'd also like to rewrite it incrementally. But... who will accept
> that incrementally patch ? It just seems that incremental patch will be
> horrible at the beginning...... It will be discard without a
> reason .....

You can use CONFIG_VT to remove the entire VT subsystem. It might be
easier for you to write an alternative VT system that could be enabled
with a different flag.

The VT system is very old code from the earliest days of Linux.
Thousands of things depend on it both in the kernel and user space. It
will be very hard to make significant changes to it that don't break
lots of dependent code.

Another model to consider... Remove the VT subsystem. Replace it will
a Unicode VT system built in user space. Using the existing kernel
code, leave a single user console in the kernel that would only be
used for system maintenance. Normal users would never see this console
unless their system was really messed up.



>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>



-- 
Jon Smirl
jonsmirl@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Microcai]

在 2010-11-28日的 09:05 -0500，jonsmirl@gmail.com写道：
> On Sun, Nov 28, 2010 at 8:42 AM, Microcai <microcai@fedoraproject.org> wrote:
> > 在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
> >> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
> >>
> >> > Hi, there
> >> >
> >> >     I'm implementing the UNICODE font of the framebuffer console, (see
> >> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> >> > current vt code is too bugy, too many direct assumes about vt buffer,
> >> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
> >> > support, but no room for such code, that's why my patch is so tricky.
> >> >
> >> >     And the code itself, if you'll excuse me, it isn't as beautiful as rest
> >> > of the kernel.
> >> >     So, it really really need a clean rewrite.I'm ganna take is hard job.
> >> >     And, please tell me if is worth to do so.
> >>
> >> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break backwards compatibility is hard.  i.e., it is hard to hack for a reason.
> >>
> >> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...
> >>
> >> -- Ted
> >>
> >
> > Yeah, I'd also like to rewrite it incrementally. But... who will accept
> > that incrementally patch ? It just seems that incremental patch will be
> > horrible at the beginning...... It will be discard without a
> > reason .....
> 
> You can use CONFIG_VT to remove the entire VT subsystem. It might be
> easier for you to write an alternative VT system that could be enabled
> with a different flag.
> 
> The VT system is very old code from the earliest days of Linux.
> Thousands of things depend on it both in the kernel and user space. It
> will be very hard to make significant changes to it that don't break
> lots of dependent code.
> 
> Another model to consider... Remove the VT subsystem. Replace it will
> a Unicode VT system built in user space. Using the existing kernel
> code, leave a single user console in the kernel that would only be
> used for system maintenance. Normal users would never see this console
> unless their system was really messed up.
> 
> 

So, here is the design according to your description  .. 

JUST export /dev/fb and /dev/pts and /dev/console
The kernel use /dev/console, but invisible. 
The rest of the world uses /dev/pts

Make init using /dev/fb to display boot message, and agetty runs
on /dev/pts

Another user-space program use /dev/fb to display all /dev/pts , you can
use alt+Fx to switch between them (kernel no longer handles console
switch).

So, there is no virtual console ... . Just as windows  (who uses cmd.exe
to display console )

kernel just deal with BYTE stream, no need for font handling .....

Will you agree with that kind of system ? 





--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[jonsmirl]

On Sun, Nov 28, 2010 at 9:29 AM, Microcai <microcai@fedoraproject.org> wrote:
> 在 2010-11-28日的 09:05 -0500，jonsmirl@gmail.com写道：
>> On Sun, Nov 28, 2010 at 8:42 AM, Microcai <microcai@fedoraproject.org> wrote:
>> > 在 2010-11-28日的 08:24 -0500，Theodore Tso写道：
>> >> On Nov 28, 2010, at 5:57 AM, Microcai wrote:
>> >>
>> >> > Hi, there
>> >> >
>> >> >     I'm implementing the UNICODE font of the framebuffer console, (see
>> >> > http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
>> >> > current vt code is too bugy, too many direct assumes about vt buffer,
>> >> > This makes me so hard to hack.  There is TODO telling me to add UNICODE
>> >> > support, but no room for such code, that's why my patch is so tricky.
>> >> >
>> >> >     And the code itself, if you'll excuse me, it isn't as beautiful as rest
>> >> > of the kernel.
>> >> >     So, it really really need a clean rewrite.I'm ganna take is hard job.
>> >> >     And, please tell me if is worth to do so.
>> >>
>> >> Yes, the console is code is very old.   But please be aware that lots of code (both in the kernel and in userspace) has dependencies upon how the code behaves.   So changing it in a way that does not break backwards compatibility is hard.  i.e., it is hard to hack for a reason.
>> >>
>> >> I would recommend an incremental rewrite (i.e., one patch at a time), as opposed to a rewrite from scratch.   Because people will want to be assured that things haven't broken in a horrible way as a result of a complete rewrite...
>> >>
>> >> -- Ted
>> >>
>> >
>> > Yeah, I'd also like to rewrite it incrementally. But... who will accept
>> > that incrementally patch ? It just seems that incremental patch will be
>> > horrible at the beginning...... It will be discard without a
>> > reason .....
>>
>> You can use CONFIG_VT to remove the entire VT subsystem. It might be
>> easier for you to write an alternative VT system that could be enabled
>> with a different flag.
>>
>> The VT system is very old code from the earliest days of Linux.
>> Thousands of things depend on it both in the kernel and user space. It
>> will be very hard to make significant changes to it that don't break
>> lots of dependent code.
>>
>> Another model to consider... Remove the VT subsystem. Replace it will
>> a Unicode VT system built in user space. Using the existing kernel
>> code, leave a single user console in the kernel that would only be
>> used for system maintenance. Normal users would never see this console
>> unless their system was really messed up.
>>
>>
>
> So, here is the design according to your description  ..
>
> JUST export /dev/fb and /dev/pts and /dev/console
> The kernel use /dev/console, but invisible.
> The rest of the world uses /dev/pts
>
> Make init using /dev/fb to display boot message, and agetty runs
> on /dev/pts
>
> Another user-space program use /dev/fb to display all /dev/pts , you can
> use alt+Fx to switch between them (kernel no longer handles console
> switch).
>
> So, there is no virtual console ... . Just as windows  (who uses cmd.exe
> to display console )
>
> kernel just deal with BYTE stream, no need for font handling .....
>
> Will you agree with that kind of system ?

It is a lot of work to get a system like that running, but it should
work when finished. It also lets you avoid messing with the VT layer
which will be a very painful process.

Now that we have KMS you can even hide the boot display if you want.
My Ubuntu system jumps straight to a graphical boot display and you
have to hit a key to see the boot console.



>
>
>
>
>
>



-- 
Jon Smirl
jonsmirl@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> So, here is the design according to your description  .. 
> 
> JUST export /dev/fb and /dev/pts and /dev/console
> The kernel use /dev/console, but invisible. 
> The rest of the world uses /dev/pts
> 
> Make init using /dev/fb to display boot message, and agetty runs
> on /dev/pts

Errr, but that already exists, that's fbterm...

Samuel

Re: VT console need rewrite

[Microcai]

在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
> Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> > So, here is the design according to your description  .. 
> > 
> > JUST export /dev/fb and /dev/pts and /dev/console
> > The kernel use /dev/console, but invisible. 
> > The rest of the world uses /dev/pts
> > 
> > Make init using /dev/fb to display boot message, and agetty runs
> > on /dev/pts
> 
> Errr, but that already exists, that's fbterm...
> 
> Samuel

Every time someone try to implement something , some other guy will jump
out and say, that already exist, in user-space ..... blablablabla

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Américo Wang]

On Mon, Nov 29, 2010 at 09:10:06AM +0800, Microcai wrote:
>在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
>> Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
>> > So, here is the design according to your description  .. 
>> > 
>> > JUST export /dev/fb and /dev/pts and /dev/console
>> > The kernel use /dev/console, but invisible. 
>> > The rest of the world uses /dev/pts
>> > 
>> > Make init using /dev/fb to display boot message, and agetty runs
>> > on /dev/pts
>> 
>> Errr, but that already exists, that's fbterm...
>> 
>> Samuel
>
>Every time someone try to implement something , some other guy will jump
>out and say, that already exist, in user-space ..... blablablabla
>

Remember, remember, kernel provides a mechanism, not a policy.
(From this point, auto-groups *does* break it.)
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 10:49 +0800，Américo Wang写道：
> On Mon, Nov 29, 2010 at 09:10:06AM +0800, Microcai wrote:
> >在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
> >> Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> >> > So, here is the design according to your description  .. 
> >> > 
> >> > JUST export /dev/fb and /dev/pts and /dev/console
> >> > The kernel use /dev/console, but invisible. 
> >> > The rest of the world uses /dev/pts
> >> > 
> >> > Make init using /dev/fb to display boot message, and agetty runs
> >> > on /dev/pts
> >> 
> >> Errr, but that already exists, that's fbterm...
> >> 
> >> Samuel
> >
> >Every time someone try to implement something , some other guy will jump
> >out and say, that already exist, in user-space ..... blablablabla
> >
> 
> Remember, remember, kernel provides a mechanism, not a policy.
> (From this point, auto-groups *does* break it.)

YES, but handle UNICODE font is a policy????????




--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Mon 29 Nov 2010 09:10:06 +0800, a écrit :
> 在 2010-11-28日的 20:11 +0100，Samuel Thibault写道：
> > Microcai, le Sun 28 Nov 2010 22:29:05 +0800, a écrit :
> > > So, here is the design according to your description  .. 
> > > 
> > > JUST export /dev/fb and /dev/pts and /dev/console
> > > The kernel use /dev/console, but invisible. 
> > > The rest of the world uses /dev/pts
> > > 
> > > Make init using /dev/fb to display boot message, and agetty runs
> > > on /dev/pts
> > 
> > Errr, but that already exists, that's fbterm...
> 
> Every time someone try to implement something , some other guy will jump
> out and say, that already exist, in user-space ..... blablablabla

But isn't that basically what you are describing?

(i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
terminal to actually do the work?)

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[microcai]

> But isn't that basically what you are describing?
>
> (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> terminal to actually do the work?)
>

ok, try run zhcon on pts.  can you do that ?

Re: VT console need rewrite

[Samuel Thibault]

microcai, le Mon 29 Nov 2010 16:20:11 +0800, a écrit :
> > But isn't that basically what you are describing?
> >
> > (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> > terminal to actually do the work?)
> >
> 
> ok, try run zhcon on pts.  can you do that ?

zhcon _provides_ a pts, that's what I mean.

Samuel

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 09:27 +0100，Samuel Thibault写道：
> microcai, le Mon 29 Nov 2010 16:20:11 +0800, a écrit :
> > > But isn't that basically what you are describing?
> > >
> > > (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> > > terminal to actually do the work?)
> > >
> > 
> > ok, try run zhcon on pts.  can you do that ?
> 
> zhcon _provides_ a pts, that's what I mean.

So you agree zhcon can't run on pts.
Then there's dozens of zhcon.


--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Mon 29 Nov 2010 16:53:25 +0800, a écrit :
> 在 2010-11-29一的 09:27 +0100，Samuel Thibault写道：
> > microcai, le Mon 29 Nov 2010 16:20:11 +0800, a écrit :
> > > > But isn't that basically what you are describing?
> > > >
> > > > (i.e. what's the use of using a pts and /dev/fb, if not for a userlevel
> > > > terminal to actually do the work?)
> > > >
> > > 
> > > ok, try run zhcon on pts.  can you do that ?
> > 
> > zhcon _provides_ a pts, that's what I mean.
> 
> So you agree zhcon can't run on pts.

Sure. It doesn't make sense. But what zhcon does should be done in
fbterm.

> Then there's dozens of zhcon.

What do you mean by this?

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Lisa Milne]

On Sun, 28 Nov 2010 09:05:44 -0500
"jonsmirl@gmail.com" <jonsmirl@gmail.com> wrote:

> On Sun, Nov 28, 2010 at 8:42 AM, Microcai
> <microcai@fedoraproject.org> wrote:
> >
> > Yeah, I'd also like to rewrite it incrementally. But... who will
> > accept that incrementally patch ? It just seems that incremental
> > patch will be horrible at the beginning...... It will be discard
> > without a reason .....
> 
> You can use CONFIG_VT to remove the entire VT subsystem. It might be
> easier for you to write an alternative VT system that could be enabled
> with a different flag.
> 
> The VT system is very old code from the earliest days of Linux.
> Thousands of things depend on it both in the kernel and user space. It
> will be very hard to make significant changes to it that don't break
> lots of dependent code.
> 
> Another model to consider... Remove the VT subsystem. Replace it will
> a Unicode VT system built in user space. Using the existing kernel
> code, leave a single user console in the kernel that would only be
> used for system maintenance. Normal users would never see this console
> unless their system was really messed up.

Another possible model: split the current system in 2, so there's a
bytestream handler, and a vt-legacy module. Then use the interface
between bytestream/legacy as an interface for future vt-kernel and
vt-user modules.

This may make it possible to create an initial patch to do the split,
then work on the new system independently of the current vt system.
Hopefully reducing any problems with api/subsystem inconsistencies
breaking existing code elsewhere, by giving it time to adapt.

This is guesswork on my part as I haven't actually looked at the code,
so while it sounds good in theory, you'd have to check if it's actually
doable.

-- 
Lisa Milne <lisa@ltmnet.com>

Re: VT console need rewrite

[Microcai]

> 
> Another possible model: split the current system in 2, so there's a
> bytestream handler, and a vt-legacy module. Then use the interface
> between bytestream/legacy as an interface for future vt-kernel and
> vt-user modules.

this may cause early printk stop working. 

> 
> This may make it possible to create an initial patch to do the split,
> then work on the new system independently of the current vt system.
> Hopefully reducing any problems with api/subsystem inconsistencies
> breaking existing code elsewhere, by giving it time to adapt.
> 
> This is guesswork on my part as I haven't actually looked at the code,
> so while it sounds good in theory, you'd have to check if it's actually
> doable.
> 

Sounds like a good idea. Who is in charge of VT system ? Seems no
one ....

Re: VT console need rewrite

[Ted Ts'o]

On Mon, Nov 29, 2010 at 12:20:07AM +0800, Microcai wrote:
> 
> > Another possible model: split the current system in 2, so there's a
> > bytestream handler, and a vt-legacy module. Then use the interface
> > between bytestream/legacy as an interface for future vt-kernel and
> > vt-user modules.
> 
> this may cause early printk stop working. 

Let's start by asking a much more fundamental question; what the heck
are your goals?

If the main goal of the console is emergency debugging when the system
is in a very bad state (i.e., trashed initrd, etc.) do we really even
need Unicode support?

How many people do regular login, development, reading e-mail, etc.,
on the console?  Very few!  If the answer is because you hate X, as
you've already pointed out, we already have fbterm.  Where is it
written that we need to have a full unicode-capable console system?
Why is this so important; especially if doing this is going to be very
difficult, and risks breaking lots of stuff if we try to mess with it?

	       	     	      	      - Ted

Re: VT console need rewrite

[Microcai]

在 2010-11-28日的 14:46 -0500，Ted Ts'o写道：
> On Mon, Nov 29, 2010 at 12:20:07AM +0800, Microcai wrote:
> > 
> > > Another possible model: split the current system in 2, so there's a
> > > bytestream handler, and a vt-legacy module. Then use the interface
> > > between bytestream/legacy as an interface for future vt-kernel and
> > > vt-user modules.
> > 
> > this may cause early printk stop working. 
> 
> Let's start by asking a much more fundamental question; what the heck
> are your goals?
> 
> If the main goal of the console is emergency debugging when the system
> is in a very bad state (i.e., trashed initrd, etc.) do we really even
> need Unicode support?
> 
> How many people do regular login, development, reading e-mail, etc.,
> on the console?  Very few!  If the answer is because you hate X, as
> you've already pointed out, we already have fbterm.  Where is it
> written that we need to have a full unicode-capable console system?
> Why is this so important; especially if doing this is going to be very
> difficult, and risks breaking lots of stuff if we try to mess with it?
> 
> 	       	     	      	      - Ted

Hey, my old patch already did it , and do not break any old stuff.
Question is , the VT code *is really very old*. 
Just want to simplify the code, remove old stuff, make it future
compatible. Forward compatibility is more important than backward one  



--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Mon 29 Nov 2010 09:14:04 +0800, a écrit :
> Hey, my old patch already did it , and do not break any old stuff.

To your knowledge.  IIRC while reading your patch I found a couple of
things that would most probably break some corner cases, which could
go unnoticed, that's why I proposed to first extend the internal vc
content matrix into unicode values, so as to clearly have to upgrade all
drivers.

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 09:58 +0100，Samuel Thibault写道：
> Microcai, le Mon 29 Nov 2010 09:14:04 +0800, a écrit :
> > Hey, my old patch already did it , and do not break any old stuff.
> 
> To your knowledge.  IIRC while reading your patch I found a couple of
> things that would most probably break some corner cases, which could
> go unnoticed, that's why I proposed to first extend the internal vc
> content matrix into unicode values, so as to clearly have to upgrade all
> drivers.
> 
> Samuel

I doesn't matter *how to do* . I have many different kinds of patchs
that did it in different ways.

What realy matters is that, do we agree that add unicode font support is
acceptable ?



--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Mon 29 Nov 2010 17:32:37 +0800, a écrit :
> 在 2010-11-29一的 09:58 +0100，Samuel Thibault写道：
> > Microcai, le Mon 29 Nov 2010 09:14:04 +0800, a écrit :
> > > Hey, my old patch already did it , and do not break any old stuff.
> > 
> > To your knowledge.  IIRC while reading your patch I found a couple of
> > things that would most probably break some corner cases, which could
> > go unnoticed, that's why I proposed to first extend the internal vc
> > content matrix into unicode values, so as to clearly have to upgrade all
> > drivers.
> 
> I doesn't matter *how to do* .

Concerning breaking existing stuff, yes, it does.

> I have many different kinds of patchs that did it in different ways.
> 
> What realy matters is that, do we agree that add unicode font support is
> acceptable ?

Simple unicode font (i.e. just go beyond the arbitrary historical-vga
512 glyphs limit for fbcon) might go in.  Double-width is questionable,
but it can probably simple enough to be able to get in (there is already
some code for it anyway).

But complete unicode support (with arabic ligatures, tibetan
combinations, etc.) won't ever be accepted in the kernel as that's far
too involved in terms of font rendering.  So the userland way needs to
be fixed for these anyway, and then things like chinese will go along...

Samuel

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 10:58 +0100，Samuel Thibault写道：
> 
> Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> 512 glyphs limit for fbcon) might go in.  Double-width is
> questionable,
> but it can probably simple enough to be able to get in (there is
> already
> some code for it anyway).
> 
> But complete unicode support (with arabic ligatures, tibetan
> combinations, etc.) won't ever be accepted in the kernel as that's far
> too involved in terms of font rendering.  So the userland way needs to
> be fixed for these anyway, and then things like chinese will go
> along...
> 
> Samuel 

YES, my patch just include BMP(mostly CJK), not full UNICODE. IF it can
be included, then I can do more work to make sure it doesn't break old
stuff. And it's configurable. users don't have to be forced to use.





--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Mon 29 Nov 2010 18:11:16 +0800, a écrit :
> 在 2010-11-29一的 10:58 +0100，Samuel Thibault写道：
> > Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> > 512 glyphs limit for fbcon) might go in.  Double-width is
> > questionable,
> > but it can probably simple enough to be able to get in (there is
> > already
> > some code for it anyway).
> > 
> > But complete unicode support (with arabic ligatures, tibetan
> > combinations, etc.) won't ever be accepted in the kernel as that's far
> > too involved in terms of font rendering.  So the userland way needs to
> > be fixed for these anyway, and then things like chinese will go
> > along...
> > 
> > Samuel 
> 
> YES, my patch just include BMP(mostly CJK), not full UNICODE. IF it can
> be included, then I can do more work to make sure it doesn't break old
> stuff. And it's configurable. users don't have to be forced to use.

It seems you didn't get my point: what is the use of including CJK into
kernel when that won't solve the issue for e.g. arabic, while fixing
fbterm will solve the issue for both arabic and CJK?

Samuel

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 11:17 +0100，Samuel Thibault写道：
> It seems you didn't get my point: what is the use of including CJK
> into
> kernel when that won't solve the issue for e.g. arabic, while fixing
> fbterm will solve the issue for both arabic and CJK?
> 

You don't get my point.  fbterm of zhcon won't fit every thing.


--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Samuel Thibault]

Microcai, le Mon 29 Nov 2010 18:40:00 +0800, a écrit :
> 在 2010-11-29一的 11:17 +0100，Samuel Thibault写道：
> > It seems you didn't get my point: what is the use of including CJK
> > into
> > kernel when that won't solve the issue for e.g. arabic, while fixing
> > fbterm will solve the issue for both arabic and CJK?
> 
> You don't get my point.  fbterm of zhcon won't fit every thing.

Why?  Can't what zhcon does be integrated into fbterm?

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Pavel Machek]

Hi!

> > I have many different kinds of patchs that did it in different ways.
> > 
> > What realy matters is that, do we agree that add unicode font support is
> > acceptable ?
> 
> Simple unicode font (i.e. just go beyond the arbitrary historical-vga
> 512 glyphs limit for fbcon) might go in.  Double-width is questionable,

That would be very nice, yes.

> But complete unicode support (with arabic ligatures, tibetan
> combinations, etc.) won't ever be accepted in the kernel as that's far
> too involved in terms of font rendering.  So the userland way needs to
> be fixed for these anyway, and then things like chinese will go along...

Well... english has ligatures, too (fi); console does not support them
nicely and we still use it. Support for eastern european characters by
default would be very very nice... and yes, that's just breaking the
256 char limit, nothing more complex.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: VT console need rewrite

[Samuel Thibault]

Pavel Machek, le Wed 15 Dec 2010 15:45:02 +0100, a écrit :
> > But complete unicode support (with arabic ligatures, tibetan
> > combinations, etc.) won't ever be accepted in the kernel as that's far
> > too involved in terms of font rendering.  So the userland way needs to
> > be fixed for these anyway, and then things like chinese will go along...
> 
> Well... english has ligatures, too (fi);

English ligatures are extremely far from what arabic needs. People
don't even actually know that there are ligatures in English, while for
arabic, people would even have a hard time reading something that isn't
using ligatures.

> console does not support them nicely and we still use it.

Err, I've very rarely seen the fi/ff ligatures on the console, it was
mostly from copy/pastes from pdf documents.

> Support for eastern european characters by default would be very very
> nice... and yes, that's just breaking the 256 char limit, nothing more
> complex.

The console can do 512 glyphs, not only 256, so eastern europe support
is a matter of combining a latin1 font with parts of the other latin
fonts. That already covers a lot of european languages.

Samuel
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Alexey Gladkov]

28.11.2010 22:46, Ted Ts'o wrote:
> How many people do regular login, development, reading e-mail, etc.,
> on the console?  Very few!

Once I broke the euro symbol in the kbd keymaps and got a kick in the
ass very quickly :). It means that users continue to use the console
quite active.

Another example: I keep getting a new keymaps and console fonts for
each new kbd release.

Maybe not a lot of console users, but more of them than you think.

> If the answer is because you hate X, as
> you've already pointed out, we already have fbterm.  Where is it
> written that we need to have a full unicode-capable console system?

fbterm ‎can't help in all cases. For example: we have a limitation
NR_KEYS, which makes it impossible to use keycode > 256 in keymaps.
I've already received several complaints about it.

-- 
Rgrds, legion

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 04:55 +0300，Alexey Gladkov写道：
> 28.11.2010 22:46, Ted Ts'o wrote:
> > How many people do regular login, development, reading e-mail, etc.,
> > on the console?  Very few!
> 
> Once I broke the euro symbol in the kbd keymaps and got a kick in the
> ass very quickly :). It means that users continue to use the console
> quite active.
> 
> Another example: I keep getting a new keymaps and console fonts for
> each new kbd release.
> 
> Maybe not a lot of console users, but more of them than you think.
> 
> > If the answer is because you hate X, as
> > you've already pointed out, we already have fbterm.  Where is it
> > written that we need to have a full unicode-capable console system?
> 
> fbterm ‎can't help in all cases. For example: we have a limitation
> NR_KEYS, which makes it impossible to use keycode > 256 in keymaps.
> I've already received several complaints about it.
> 

YES, user-space can't always fit! Some function need to be implemented
in kernel. And that's why we have KMS, we have auto cgroup ....

Don't say, hey , hard to implement, all you have to to is apply the
path .. No need to do for you.

If the patch isn't as good as you expected, reject, I'll try better one.
The thing is, you really should agree that, implementing UNICODE font in
kernel isn't a bad idea.

Re: VT console need rewrite

[Alan Cox]

> If the patch isn't as good as you expected, reject, I'll try better one.
> The thing is, you really should agree that, implementing UNICODE font in
> kernel isn't a bad idea.

Unicode is a good deal more than a font.

Alan

Re: VT console need rewrite

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

On Mon, 29 Nov 2010 10:14:06 +0800, Microcai said:

> The thing is, you really should agree that, implementing UNICODE font in
> kernel isn't a bad idea.

Let us know when you have implemented support for bidirectional
scripts that require compositing or half-spacing characters.  Without
blowing out the 8K kernel stack.

Doing it *right* is important:

http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two-people-puts-three-more-in-jail

Two dead, three in jail.  All because of broken font support.  That's a good
reason to not even try unless you can do it *right*.


[-- Attachment #2: Type: application/pgp-signature, Size: 227 bytes --]

Re: VT console need rewrite

[Jerome Glisse]

On Mon, Nov 29, 2010 at 12:50 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Mon, 29 Nov 2010 10:14:06 +0800, Microcai said:
>
>> The thing is, you really should agree that, implementing UNICODE font in
>> kernel isn't a bad idea.
>
> Let us know when you have implemented support for bidirectional
> scripts that require compositing or half-spacing characters.  Without
> blowing out the 8K kernel stack.
>
> Doing it *right* is important:
>
> http://gizmodo.com/382026/a-cellphones-missing-dot-kills-two-people-puts-three-more-in-jail
>
> Two dead, three in jail.  All because of broken font support.  That's a good
> reason to not even try unless you can do it *right*.
>
>

Side point, KMS people would like to see the direct memory access to
fb buffer vanish, right now we are wasting vram to keep a front buffer
around for fb. So somethings like always using some callback to access
(write/read) to fb buffer.

Cheers,
Jerome Glisse
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Alan Cox]

> Sounds like a good idea. Who is in charge of VT system ? Seems no
> one ....

Which bit - lots of people own different parts of it, because there is a
lot of it, and people own different sections of currently ongoing
projects. Plus the X DRM folks own the KMS framebuffers, various authors
own their own low level drivers for each card and so on.

I kind of own the drivers/tty/vt* bits.

I simply don't see how you are going to address the majority of the
worlds languages in kernel space, all the major ones are too complex and
pango is too big and complex to put in kernel as are the needed font
renderers. Most of the needed compositing accelerations are also not
directly exposed in KMS to the console driver as would be needed

Your other problem is mode setting, VESA is on its way out, a lot of
devices today, especially non X86 ones pretty much need X to do any kind
of video management beyond "here is how the firmware console was
configured", and indeed many embedded devices don't have an VT code
enabled at all, just a printk console and possibly ttyprintk, both of
which may well only be talking to the debug ports

Alan

Re: VT console need rewrite

[Microcai]

在 2010-11-28日的 23:46 +0000，Alan Cox写道：
> Which bit - lots of people own different parts of it, because there is a
> lot of it, and people own different sections of currently ongoing
> projects. Plus the X DRM folks own the KMS framebuffers, various authors
> own their own low level drivers for each card and so on.
> 
> I kind of own the drivers/tty/vt* bits.

Don't you see the cosole code ? There is TODO that say , "hey , I need
UNICODE support..."

I'm not breaking old one to introduce new feature, but fix a little BUG
in the TODO list. 

> I simply don't see how you are going to address the majority of the
> worlds languages in kernel space, all the major ones are too complex and
> pango is too big and complex to put in kernel as are the needed font
> renderers. Most of the needed compositing accelerations are also not
> directly exposed in KMS to the console driver as would be needed

Simply drawing characters won't need any hw acceleration. It doesn't
matter if it is slow. KMS and Wayland will make more people work under
console, trust me.

> 
> Your other problem is mode setting, VESA is on its way out, a lot of
No, VESA will still be here for another decades. Some fb driver conflict
with the X one, even there is a fb driver for you card, you simply won't
use it unless it does not conflict with X one. My PC , for example,
nvidiafb conflict with X one, so I have to use VESA.

> devices today, especially non X86 ones pretty much need X to do any kind
> of video management beyond "here is how the firmware console was
> configured", and indeed many embedded devices don't have an VT code
> enabled at all, just a printk console and possibly ttyprintk, both of
> which may well only be talking to the debug ports
> 
> Alan

printk console , if redirect to a ttyS? , then the outer Linux world
terminal will do UNICODE handling. 



--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Alan Cox]

> Don't you see the cosole code ? There is TODO that say , "hey , I need
> UNICODE support..."

And what has happened over time is that we've learned

- that real unicode is hard
- that some languages need very complex character handling, and this
  happens to be most of the major non western european languages in the
  world today

and someone put all the needed logic into pango, and people wrote fbterm
and friends. The world has moved on

> Simply drawing characters won't need any hw acceleration. It doesn't
> matter if it is slow. KMS and Wayland will make more people work under
> console, trust me.

No I don't trust you, sorry. You need compositing acceleration to do some
of the character composing efficiently and to do anti-aliasing which for
small symbols in many non western fonts is rather essential.

> > Your other problem is mode setting, VESA is on its way out, a lot of
> No, VESA will still be here for another decades. Some fb driver conflict

VESA is X86 specific, PC class hardware specific and subject to various
assumptions, when it works.

No VESA on Intel MID platforms, no VESA on PPC, no VESA on Android phones

> printk console , if redirect to a ttyS? , then the outer Linux world
> terminal will do UNICODE handling. 

In a window in a debug/development environment under X11 with Pango..

Re: VT console need rewrite

[microcai]

OK, let me vanish in a few months.

hopefully I'll be back with a patch fix that stupid 512 glyph limitation.

Re: VT console need rewrite

[Pavel Machek]

On Tue 2010-11-30 13:05:51, microcai wrote:
> OK, let me vanish in a few months.
> 
> hopefully I'll be back with a patch fix that stupid 512 glyph limitation.

Looking forward.
								Pavel
(Who'd rather like to see reasonable support for eastern *and* western
europe.)

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: VT console need rewrite

[Andi Kleen]

Microcai <microcai@fedoraproject.org> writes:

> 	I'm implementing the UNICODE font of the framebuffer console, (see
> http://lkml.org/lkml/2010/11/26/50 in case you do not got my email). But
> current vt code is too bugy, too many direct assumes about vt buffer,
> This makes me so hard to hack.  There is TODO telling me to add UNICODE
> support, but no room for such code, that's why my patch is so tricky.
>
> 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> of the kernel.
> 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> 	And, please tell me if is worth to do so.

To be honest I have doubts its worth it. This was discussed years and
years ago and the conclusion back then that people who need unicode
fonts should use them in X or another graphical console environment
(there are plenty to chose from these days). Has this changed?

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: VT console need rewrite

[Microcai]

在 2010-11-29一的 11:01 +0100，Andi Kleen写道：

> To be honest I have doubts its worth it. This was discussed years and
> years ago and the conclusion back then that people who need unicode
> fonts should use them in X or another graphical console environment
> (there are plenty to chose from these days). Has this changed?
> 
> -Andi
> 

Yes, it change. Linux graphical stuck have changed a lot. With KMS,
console got a native resolution, and looks very nice.



--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: VT console need rewrite

[Jesse Barnes]

On Mon, 29 Nov 2010 18:12:57 +0800
Microcai <microcai@fedoraproject.org> wrote:

> 在 2010-11-29一的 11:01 +0100，Andi Kleen写道：
> 
> > To be honest I have doubts its worth it. This was discussed years and
> > years ago and the conclusion back then that people who need unicode
> > fonts should use them in X or another graphical console environment
> > (there are plenty to chose from these days). Has this changed?
> > 
> > -Andi
> > 
> 
> Yes, it change. Linux graphical stuck have changed a lot. With KMS,
> console got a native resolution, and looks very nice.

But really, keep this stuff out of the kernel.  Sure, the kernel can
provide memory and a framebuffer to draw in (subject to the constraints
Jerome already mentioned), but why try to deal with fancy input or font
handling there?  IMO the kernel console should be limited to printing
debug and error information; a separate userspace daemon should handle
input and console support.  It would be great to avoid the need for
CONFIG_VT entirely for the typical Linux system.

-- 
Jesse Barnes, Intel Open Source Technology Center

Re: VT console need rewrite

[Pavel Machek]

Hi!

> > 	And the code itself, if you'll excuse me, it isn't as beautiful as rest
> > of the kernel.
> > 	So, it really really need a clean rewrite.I'm ganna take is hard job.	
> > 	And, please tell me if is worth to do so.
> 
> To be honest I have doubts its worth it. This was discussed years and
> years ago and the conclusion back then that people who need unicode
> fonts should use them in X or another graphical console environment
> (there are plenty to chose from these days). Has this changed?

Yes. These days kernel sources and kernel messages both contain
non-ASCII characters... as well as many filesystems.
								Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: VT console need rewrite

[Alan Cox]

> Yes. These days kernel sources and kernel messages both contain
> non-ASCII characters... as well as many filesystems.

But not in kernel printed messages.

Alan

Re: VT console need rewrite

[Pavel Machek]

Hi!

> > Yes. These days kernel sources and kernel messages both contain
> > non-ASCII characters... as well as many filesystems.
> 
> But not in kernel printed messages.

Yes, even in those (but something eats my mail).

CAFE driver has non-ascii characters in pci device name, and that
leaks to dmesg.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html