2.6.3-rc3 serial console woes

2.6.3-rc3 serial console woes

[Keith Owens]

Between 2.6.3-rc2 and 2.6.3-rc3, the serial console initialisation
changed, due to this patch :-

  http://linux.bkbits.net:8080/linux-2.5/cset@1.1653?nav=index.html|ChangeSet@-7d

Now the serial console is not initialised until a long way into the
boot, just after the disks are probed.  This makes it impossible to use
a kernel debugger such as kdb or kgdb over a serial console during
device initialisation.

You get no serial console output until after devices have been probed.
So if something goes wrong during boot, you have no indication of the
problem.  Even without device problems, the delay on serial console
output can make it look like the machine has hung, with no output on
the serial console until a couple of minutes after boot.

.config extract

X86=y
MMU=y
UID16=y
GENERIC_ISA_DMA=y
EXPERIMENTAL=y
CLEAN_COMPILE=y
STANDALONE=y
SWAP=y
SYSVIPC=y
SYSCTL=y
LOG_BUF_SHIFT=16
KALLSYMS=y
FUTEX=y
EPOLL=y
IOSCHED_NOOP=y
IOSCHED_AS=y
IOSCHED_DEADLINE=y
MODULES=y
OBSOLETE_MODPARM=y
KMOD=y
X86_PC=y
M686=y
X86_CMPXCHG=y
X86_XADD=y
X86_L1_CACHE_SHIFT=5
RWSEM_XCHGADD_ALGORITHM=y
X86_PPRO_FENCE=y
X86_WP_WORKS_OK=y
X86_INVLPG=y
X86_BSWAP=y
X86_POPAD_OK=y
X86_GOOD_APIC=y
X86_USE_PPRO_CHECKSUM=y
SMP=y
NR_CPUS=2
X86_LOCAL_APIC=y
X86_IO_APIC=y
X86_TSC=y
X86_MCE=y
X86_MSR=m
X86_CPUID=m
NOHIGHMEM=y
MTRR=y
HAVE_DEC_LOCK=y
ACPI_BOOT=y
PCI=y
PCI_GOANY=y
PCI_BIOS=y
PCI_DIRECT=y
PCI_NAMES=y
ISA=y
BINFMT_ELF=y
BINFMT_AOUT=m
BINFMT_MISC=m
PARPORT=m
PARPORT_PC=m
PARPORT_PC_CML1=m
BLK_DEV_FD=y
BLK_DEV_LOOP=m
BLK_DEV_RAM=y
BLK_DEV_RAM_SIZE=4096
IDE=y
BLK_DEV_IDE=y
BLK_DEV_IDEDISK=y
BLK_DEV_IDECD=y
BLK_DEV_IDESCSI=y
IDE_GENERIC=y
BLK_DEV_IDEPCI=y
IDEPCI_SHARE_IRQ=y
BLK_DEV_IDEDMA_PCI=y
IDEDMA_PCI_AUTO=y
BLK_DEV_ADMA=y
BLK_DEV_HPT366=y
BLK_DEV_PIIX=y
BLK_DEV_IDEDMA=y
IDEDMA_AUTO=y
SCSI=y
SCSI_PROC_FS=y
BLK_DEV_SD=y
BLK_DEV_SR=m
BLK_DEV_SR_VENDOR=y
CHR_DEV_SG=m
SCSI_REPORT_LUNS=y
SCSI_CONSTANTS=y
SCSI_LOGGING=y
SCSI_AHA1542=m
SCSI_SYM53C8XX_2=y
SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
SCSI_SYM53C8XX_DEFAULT_TAGS=16
SCSI_SYM53C8XX_MAX_TAGS=64
SCSI_QLA2XXX=y
NET=y
PACKET=m
NETLINK_DEV=m
UNIX=y
INET=y
INET_ECN=y
SYN_COOKIES=y
NETFILTER=y
IP_NF_CONNTRACK=m
IP_NF_FTP=m
IP_NF_IRC=m
IP_NF_QUEUE=m
IP_NF_IPTABLES=m
IP_NF_MATCH_LIMIT=m
IP_NF_MATCH_MAC=m
IP_NF_MATCH_PKTTYPE=m
IP_NF_MATCH_MARK=m
IP_NF_MATCH_MULTIPORT=m
IP_NF_MATCH_TOS=m
IP_NF_MATCH_ECN=m
IP_NF_MATCH_DSCP=m
IP_NF_MATCH_AH_ESP=m
IP_NF_MATCH_LENGTH=m
IP_NF_MATCH_TTL=m
IP_NF_MATCH_TCPMSS=m
IP_NF_MATCH_HELPER=m
IP_NF_MATCH_STATE=m
IP_NF_MATCH_CONNTRACK=m
IP_NF_MATCH_OWNER=m
IP_NF_FILTER=m
IP_NF_TARGET_REJECT=m
IP_NF_NAT=m
IP_NF_NAT_NEEDED=y
IP_NF_TARGET_MASQUERADE=m
IP_NF_TARGET_REDIRECT=m
IP_NF_NAT_LOCAL=y
IP_NF_NAT_IRC=m
IP_NF_NAT_FTP=m
IP_NF_MANGLE=m
IP_NF_TARGET_TOS=m
IP_NF_TARGET_ECN=m
IP_NF_TARGET_DSCP=m
IP_NF_TARGET_MARK=m
IP_NF_TARGET_LOG=m
IP_NF_TARGET_ULOG=m
IP_NF_TARGET_TCPMSS=m
IP_NF_ARPTABLES=m
IP_NF_ARPFILTER=m
IP_NF_COMPAT_IPCHAINS=m
IP_NF_COMPAT_IPFWADM=m
IPV6_SCTP__=y
NETDEVICES=y
DUMMY=m
ETHERTAP=m
NET_ETHERNET=y
NET_TULIP=y
TULIP=m
NET_ISA=y
NE2000=m
NET_PCI=y
PLIP=m
PPP=m
PPP_ASYNC=m
PPP_DEFLATE=m
PPP_BSDCOMP=m
SLIP=m
SLIP_COMPRESSED=y
INPUT=y
INPUT_MOUSEDEV=y
INPUT_MOUSEDEV_PSAUX=y
INPUT_MOUSEDEV_SCREEN_X=1024
INPUT_MOUSEDEV_SCREEN_Y=768
SOUND_GAMEPORT=y
SERIO=y
SERIO_I8042=y
SERIO_SERPORT=y
INPUT_KEYBOARD=y
KEYBOARD_ATKBD=y
INPUT_MOUSE=y
MOUSE_PS2=y
VT=y
VT_CONSOLE=y
HW_CONSOLE=y
SERIAL_8250=y
SERIAL_8250_CONSOLE=y
SERIAL_8250_NR_UARTS=4
SERIAL_CORE=y
SERIAL_CORE_CONSOLE=y
UNIX98_PTYS=y
UNIX98_PTY_COUNT=256
PRINTER=m
LP_CONSOLE=y
NVRAM=m
RTC=y
VGA_CONSOLE=y
DUMMY_CONSOLE=y
SOUND=m
EXT2_FS=y
XFS_FS=y
XFS_QUOTA=y
MINIX_FS=m
ROMFS_FS=m
QUOTACTL=y
AUTOFS_FS=m
AUTOFS4_FS=m
ISO9660_FS=m
JOLIET=y
ZISOFS=y
ZISOFS_FS=m
FAT_FS=m
MSDOS_FS=m
VFAT_FS=m
NTFS_FS=m
PROC_FS=y
PROC_KCORE=y
DEVPTS_FS=y
TMPFS=y
RAMFS=y
HPFS_FS=m
NFS_FS=m
NFS_V3=y
NFSD=m
NFSD_V3=y
LOCKD=m
LOCKD_V4=y
EXPORTFS=m
SUNRPC=m
SMB_FS=m
MSDOS_PARTITION=y
NLS=y
NLS_DEFAULT="iso8859-1"
DEBUG_KERNEL=y
DEBUG_SLAB=y
MAGIC_SYSRQ=y
KDB=y
KDB_MODULES=m
KDB_CONTINUE_CATASTROPHIC=0
X86_FIND_SMP_CONFIG=y
X86_MPPARSE=y
CRC32=m
ZLIB_INFLATE=m
ZLIB_DEFLATE=m
X86_SMP=y
X86_HT=y
X86_BIOS_REBOOT=y
X86_TRAMPOLINE=y
PC=y

Re: 2.6.3-rc3 serial console woes

[Bjorn Helgaas]

On Monday 16 February 2004 6:14 am, Keith Owens wrote:
> Between 2.6.3-rc2 and 2.6.3-rc3, the serial console initialisation
> changed, due to this patch :-
> 
>   http://linux.bkbits.net:8080/linux-2.5/cset@1.1653?nav=index.html|ChangeSet@-7d
> 
> Now the serial console is not initialised until a long way into the
> boot, just after the disks are probed.  This makes it impossible to use
> a kernel debugger such as kdb or kgdb over a serial console during
> device initialisation.

Hmm....  I suspect the problem is that serial8250_isa_init_ports()
doesn't initialize port->type for the ports in SERIAL_PORT_DFNS,
so we fail the console setup.

Does the attached patch make it work like it used to?  ISTR that
Russell didn't really like testing port->ops, but I can't remember
why, and I don't see anything better.

===== drivers/serial/8250.c 1.44 vs edited =====
--- 1.44/drivers/serial/8250.c	Fri Feb 13 08:19:33 2004
+++ edited/drivers/serial/8250.c	Mon Feb 16 12:03:06 2004
@@ -1976,7 +1976,7 @@
 	if (co->index >= UART_NR)
 		co->index = 0;
 	port = &serial8250_ports[co->index].port;
-	if (port->type == PORT_UNKNOWN)
+	if (!port->ops)
 		return -ENODEV;
 
 	/*

Re: 2.6.3-rc3 serial console woes

[Keith Owens]

On Mon, 16 Feb 2004 12:07:56 -0700, 
Bjorn Helgaas <bjorn.helgaas@hp.com> wrote:
>Hmm....  I suspect the problem is that serial8250_isa_init_ports()
>doesn't initialize port->type for the ports in SERIAL_PORT_DFNS,
>so we fail the console setup.
>
>Does the attached patch make it work like it used to?  ISTR that
>Russell didn't really like testing port->ops, but I can't remember
>why, and I don't see anything better.
>
>===== drivers/serial/8250.c 1.44 vs edited =====
>--- 1.44/drivers/serial/8250.c	Fri Feb 13 08:19:33 2004
>+++ edited/drivers/serial/8250.c	Mon Feb 16 12:03:06 2004
>@@ -1976,7 +1976,7 @@
> 	if (co->index >= UART_NR)
> 		co->index = 0;
> 	port = &serial8250_ports[co->index].port;
>-	if (port->type == PORT_UNKNOWN)
>+	if (!port->ops)
> 		return -ENODEV;
> 
> 	/*

Works for me on i386.

Re: 2.6.3-rc3 serial console woes

[Keith Owens]

On Tue, 17 Feb 2004 08:05:54 +1100, 
Keith Owens <kaos@sgi.com> wrote:
>On Mon, 16 Feb 2004 12:07:56 -0700, 
>Bjorn Helgaas <bjorn.helgaas@hp.com> wrote:
>>Hmm....  I suspect the problem is that serial8250_isa_init_ports()
>>doesn't initialize port->type for the ports in SERIAL_PORT_DFNS,
>>so we fail the console setup.
>>===== drivers/serial/8250.c 1.44 vs edited =====
>>--- 1.44/drivers/serial/8250.c	Fri Feb 13 08:19:33 2004
>>+++ edited/drivers/serial/8250.c	Mon Feb 16 12:03:06 2004
>>@@ -1976,7 +1976,7 @@
>> 	if (co->index >= UART_NR)
>> 		co->index = 0;
>> 	port = &serial8250_ports[co->index].port;
>>-	if (port->type == PORT_UNKNOWN)
>>+	if (!port->ops)
>> 		return -ENODEV;
>> 
>> 	/*
>
>Works for me on i386.

Spoke too soon.  That one line patch makes the serial console available
early in boot.  But as soon as /sbin/init runs, the console is
gibberish due to a speed mismatch.  Completely reverting

  http://linux.bkbits.net:8080/linux-2.5/cset@1.1653?nav=index.html|ChangeSet@-7d

works fine.  Since this is a -rc kernel, can we revert the cset until
it is fixed?

Re: 2.6.3-rc3 serial console woes

[Bjorn Helgaas]

On Monday 16 February 2004 4:25 pm, Keith Owens wrote:
> Spoke too soon.  That one line patch makes the serial console available
> early in boot.  But as soon as /sbin/init runs, the console is
> gibberish due to a speed mismatch.  Completely reverting
> 
>   http://linux.bkbits.net:8080/linux-2.5/cset@1.1653?nav=index.html|ChangeSet@-7d
> 
> works fine.  Since this is a -rc kernel, can we revert the cset until
> it is fixed?

Reverting for now sounds like the right thing to me.  I would like to
understand what's going on, though.

For the serial console to work early, serial8250_console_setup() must
be returning zero.  So we can't be taking this return:

	if (!port->ops)
		return -ENODEV;

and therefore, the hunk in serial_core.c shouldn't have any effect either.
So is it merely the fact that we call serial8250_late_console_init()?

I wouldn't expect that to make any difference, because the setup()
call from serial8250_console_init() should have succeeded, so
CON_ENABLED should have been set, and we don't do anything in
that case.

Keith, can you tell me how to reproduce this?

Bjorn

Re: 2.6.3-rc3 serial console woes

[Keith Owens]

On Mon, 16 Feb 2004 16:45:14 -0700, 
Bjorn Helgaas <bjorn.helgaas@hp.com> wrote:
>Reverting for now sounds like the right thing to me.  I would like to
>understand what's going on, though.

My bad, patch merge error against kdb.  Your one line patch is all that
is required.  I have included it in kdb-v4.3-2.6.3-rc3-common-1 to
ensure that kdb works.

===== drivers/serial/8250.c 1.44 vs edited =====
--- 1.44/drivers/serial/8250.c	Fri Feb 13 08:19:33 2004
+++ edited/drivers/serial/8250.c	Mon Feb 16 12:03:06 2004
@@ -1976,7 +1976,7 @@
 	if (co->index >= UART_NR)
 		co->index = 0;
 	port = &serial8250_ports[co->index].port;
-	if (port->type == PORT_UNKNOWN)
+	if (!port->ops)
 		return -ENODEV;
 
 	/*