Little console problem in 2.5.30

Little console problem in 2.5.30

[Pete Zaitcev]

Hi, all:

I would appreciate if someone would explain me if the attached patch
does the right thing. The problem is that I do not use the framebuffer,
and use a serial console. Whenever a legacy /sbin/init tries to
open /dev/tty0, the system oopses dereferencing conswitchp in
visual_init().

-- Pete

diff -ur -X dontdiff linux-2.5.30-sp_pbk/drivers/char/console.c linux-2.5.30-sparc/drivers/char/console.c
--- linux-2.5.30-sp_pbk/drivers/char/console.c	Thu Aug  1 14:16:34 2002
+++ linux-2.5.30-sparc/drivers/char/console.c	Sun Aug 18 23:14:20 2002
@@ -652,7 +652,7 @@
 
 int vc_allocate(unsigned int currcons)	/* return 0 on success */
 {
-	if (currcons >= MAX_NR_CONSOLES)
+	if (currcons >= MAX_NR_CONSOLES || conswitchp == NULL)
 		return -ENXIO;
 	if (!vc_cons[currcons].d) {
 	    long p, q;

Re: Little console problem in 2.5.30

[Geert Uytterhoeven]

On Mon, 19 Aug 2002, Pete Zaitcev wrote:
> I would appreciate if someone would explain me if the attached patch
> does the right thing. The problem is that I do not use the framebuffer,
> and use a serial console. Whenever a legacy /sbin/init tries to
> open /dev/tty0, the system oopses dereferencing conswitchp in
> visual_init().

And this worked before?

conswitchp must never be NULL, say `conswitchp = &dummy_con;' in your setup.c
if you have a serial console.

>From looking at arch/sparc/kernel/setup.c, perhaps you have
CONFIG_DUMMY_CONSOLE=n?

> -- Pete
> 
> diff -ur -X dontdiff linux-2.5.30-sp_pbk/drivers/char/console.c linux-2.5.30-sparc/drivers/char/console.c
> --- linux-2.5.30-sp_pbk/drivers/char/console.c	Thu Aug  1 14:16:34 2002
> +++ linux-2.5.30-sparc/drivers/char/console.c	Sun Aug 18 23:14:20 2002
> @@ -652,7 +652,7 @@
>  
>  int vc_allocate(unsigned int currcons)	/* return 0 on success */
>  {
> -	if (currcons >= MAX_NR_CONSOLES)
> +	if (currcons >= MAX_NR_CONSOLES || conswitchp == NULL)
>  		return -ENXIO;
>  	if (!vc_cons[currcons].d) {
>  	    long p, q;

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Little console problem in 2.5.30

[Petr Vandrovec]

On Mon, Aug 19, 2002 at 02:36:25PM +0200, Geert Uytterhoeven wrote:
> On Mon, 19 Aug 2002, Pete Zaitcev wrote:
> > I would appreciate if someone would explain me if the attached patch
> > does the right thing. The problem is that I do not use the framebuffer,
> > and use a serial console. Whenever a legacy /sbin/init tries to
> > open /dev/tty0, the system oopses dereferencing conswitchp in
> > visual_init().
> 
> And this worked before?

I was simillary surprised when it happened (between 2.5.25 and 2.5.26 if
my memory serves correctly).
 
> conswitchp must never be NULL, say `conswitchp = &dummy_con;' in your setup.c
> if you have a serial console.

No, it does not work that way for very loooong... Just remove VGA device from
your box, VGA con_startup will fail and conswitchp will become NULL... And
in 2.5.26 more than 50% of archs (including i386) does not use dummy_con, it
leaves conswitchp uninitialized (== NULL).
 
> > diff -ur -X dontdiff linux-2.5.30-sp_pbk/drivers/char/console.c linux-2.5.30-sparc/drivers/char/console.c
> > --- linux-2.5.30-sp_pbk/drivers/char/console.c	Thu Aug  1 14:16:34 2002
> > +++ linux-2.5.30-sparc/drivers/char/console.c	Sun Aug 18 23:14:20 2002
> > @@ -652,7 +652,7 @@
> >  
> >  int vc_allocate(unsigned int currcons)	/* return 0 on success */
> >  {
> > -	if (currcons >= MAX_NR_CONSOLES)
> > +	if (currcons >= MAX_NR_CONSOLES || conswitchp == NULL)
> >  		return -ENXIO;
> >  	if (!vc_cons[currcons].d) {
> >  	    long p, q;

In 2.5.25 con_init and vty_init was one function, which checked conswitchp == NULL
at beginning. In 2.5.26 it was spilt down, and vty_init does no conswitchp checking,
it blindly registers console tty driver. Proper fix is putting

if (!conswitchp) return;

at the beginning of vty_init(), unless we support hotplug tty. If we support hotplug tty,
then your fix is probably correct, but it needs deeper inspection, as no tty code
ever expected conswitchp == NULL.
								Petr Vandrovec
								vandrove@vc.cvut.cz

Re: Little console problem in 2.5.30

[David S. Miller]

   From: Pete Zaitcev <zaitcev@redhat.com>
   Date: Sat, 24 Aug 2002 00:57:56 -0400

   > CONFIG_DUMMY_CONSOLE=n?
   
   This only works if CONFIG_FB is present

Not true, only video/Config.in has this (false) dependency.
If you say define_bool CONFIG_DUMMY_CONSOLE=y in Sparc configuration
then dummycon.o will be built by itself and it will work.

Re: Little console problem in 2.5.30

[Pete Zaitcev]

> Date: Mon, 19 Aug 2002 14:36:25 +0200 (MEST)
> From: Geert Uytterhoeven <geert@linux-m68k.org>

> > I would appreciate if someone would explain me if the attached patch
> > does the right thing. The problem is that I do not use the framebuffer,
> > and use a serial console. Whenever a legacy /sbin/init tries to
> > open /dev/tty0, the system oopses dereferencing conswitchp in
> > visual_init().

> >  int vc_allocate(unsigned int currcons)	/* return 0 on success */
> >  {
> > -	if (currcons >= MAX_NR_CONSOLES)
> > +	if (currcons >= MAX_NR_CONSOLES || conswitchp == NULL)
> 
> And this worked before?
> 
> conswitchp must never be NULL, say `conswitchp = &dummy_con;' in your setup.c
> if you have a serial console.
> 
> >From looking at arch/sparc/kernel/setup.c, perhaps you have
> CONFIG_DUMMY_CONSOLE=n?

This only works if CONFIG_FB is present, and I do not want to add
one more useless chunk of code to the build. All my boxes have
serial consoles (like I said, I would throw CONFIG_VT away if
only it was not welded into the rest of the code so well).

-- Pete