[APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

Hardware summary: http://tinyurl.com/ap79ra
APIC details: http://intertwingly.net/stories/2009/01/22/
Note acpidump.err: Wrong checksum for OEMB!

Messages on boot using Intrepid, Jaunty Alpha 3, or Fedora 10:

[    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
  Boot with apic=debug and send a report.  Then try booting with the 
'noapic' option.
[    0.296001]

Able to get past this issue using "noapic", at which point things mostly 
work, but rsync of large iso images result in corrupt files.  Able to 
copy those same files using Vista on the same machine, or using Hardy on 
another machine.  This problem may not be related to the above, but it 
seems plausible to me that this might be an interrupt issue.

memtest86+ runs clean.

Quite willing to invest time in installing kernels or distributions on 
fresh hard drives, run tests, obtain debug information, and report back.

More background here: http://intertwingly.net/blog/2009/01/20/noAPIC

Not subscribed, but will actively monitor the web archives for this 
mailing list for the next several days.

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Andrew Morton]

On Thu, 22 Jan 2009 08:37:01 -0500 Sam Ruby <rubys@intertwingly.net> wrote:

> Hardware summary: http://tinyurl.com/ap79ra
> APIC details: http://intertwingly.net/stories/2009/01/22/
> Note acpidump.err: Wrong checksum for OEMB!
> 
> Messages on boot using Intrepid, Jaunty Alpha 3, or Fedora 10:
> 
> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
>   Boot with apic=debug and send a report.  Then try booting with the 
> 'noapic' option.
> [    0.296001]
> 
> Able to get past this issue using "noapic", at which point things mostly 
> work,

Join the ever-growing noapic club :(

I assume this is an ACPI problem.  Or at least, a BIOS problem which
ACPI can solve for us.

> but rsync of large iso images result in corrupt files.  Able to 
> copy those same files using Vista on the same machine, or using Hardy on 
> another machine.  This problem may not be related to the above, but it 
> seems plausible to me that this might be an interrupt issue.

Yes, it might be unrelated.  There are no kernel messages when it happens?

> memtest86+ runs clean.
> 
> Quite willing to invest time in installing kernels or distributions on 
> fresh hard drives, run tests, obtain debug information, and report back.
> 
> More background here: http://intertwingly.net/blog/2009/01/20/noAPIC
> 
> Not subscribed, but will actively monitor the web archives for this 
> mailing list for the next several days.

It'd be best to raise a report against ACPI?BIOS (I think) at
bugzilla.kernel.org, please.

If any previous kernel version worked OK, please be sure to note that.

Thanks.

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

Andrew Morton wrote:
> On Thu, 22 Jan 2009 08:37:01 -0500 Sam Ruby <rubys@intertwingly.net> wrote:
> 
>> Hardware summary: http://tinyurl.com/ap79ra
>> APIC details: http://intertwingly.net/stories/2009/01/22/
>> Note acpidump.err: Wrong checksum for OEMB!
>>
>> Messages on boot using Intrepid, Jaunty Alpha 3, or Fedora 10:
>>
>> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
>>   Boot with apic=debug and send a report.  Then try booting with the 
>> 'noapic' option.
>> [    0.296001]
>>
>> Able to get past this issue using "noapic", at which point things mostly 
>> work,
> 
> Join the ever-growing noapic club :(
> 
> I assume this is an ACPI problem.  Or at least, a BIOS problem which
> ACPI can solve for us.

My understanding is that APCI and APIC are two different things.

>> but rsync of large iso images result in corrupt files.  Able to 
>> copy those same files using Vista on the same machine, or using Hardy on 
>> another machine.  This problem may not be related to the above, but it 
>> seems plausible to me that this might be an interrupt issue.
> 
> Yes, it might be unrelated.  There are no kernel messages when it happens?

I see no messages to /var/log/messages while doing the following (which 
involves rsync'ing a 2618793984 byte file from an NTFS to ext3 drive on 
the same machine:

rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
MD5(windows7_7000.iso)= 953b9ac92d58f5edef525004bcce048d
rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
MD5(windows7_7000.iso)= 695328ef1280708eb73303656f6ef0b2

>> memtest86+ runs clean.
>>
>> Quite willing to invest time in installing kernels or distributions on 
>> fresh hard drives, run tests, obtain debug information, and report back.
>>
>> More background here: http://intertwingly.net/blog/2009/01/20/noAPIC
>>
>> Not subscribed, but will actively monitor the web archives for this 
>> mailing list for the next several days.
> 
> It'd be best to raise a report against ACPI?BIOS (I think) at
> bugzilla.kernel.org, please.

Once again, I'm talking about apic not acpi... does this advice still hold?

> If any previous kernel version worked OK, please be sure to note that.

I don't believe that's the case.

> Thanks.

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

Andrew Morton wrote:
> On Thu, 22 Jan 2009 08:37:01 -0500 Sam Ruby <rubys@intertwingly.net> wrote:
> 
>> Hardware summary: http://tinyurl.com/ap79ra
>> APIC details: http://intertwingly.net/stories/2009/01/22/
>> Note acpidump.err: Wrong checksum for OEMB!
>>
>> Messages on boot using Intrepid, Jaunty Alpha 3, or Fedora 10:
>>
>> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
>>   Boot with apic=debug and send a report.  Then try booting with the 
>> 'noapic' option.
>> [    0.296001]
>>
>> Able to get past this issue using "noapic", at which point things mostly 
>> work,
> 
> Join the ever-growing noapic club :(
> 
> I assume this is an ACPI problem.  Or at least, a BIOS problem which
> ACPI can solve for us.

My understanding is that APCI and APIC are two different things.

>> but rsync of large iso images result in corrupt files.  Able to 
>> copy those same files using Vista on the same machine, or using Hardy on 
>> another machine.  This problem may not be related to the above, but it 
>> seems plausible to me that this might be an interrupt issue.
> 
> Yes, it might be unrelated.  There are no kernel messages when it happens?

I see no messages to /var/log/messages while doing the following (which 
involves rsync'ing a 2618793984 byte file from an NTFS to ext3 drive on 
the same machine:

rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
MD5(windows7_7000.iso)= 953b9ac92d58f5edef525004bcce048d
rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
MD5(windows7_7000.iso)= 695328ef1280708eb73303656f6ef0b2

>> memtest86+ runs clean.
>>
>> Quite willing to invest time in installing kernels or distributions on 
>> fresh hard drives, run tests, obtain debug information, and report back.
>>
>> More background here: http://intertwingly.net/blog/2009/01/20/noAPIC
>>
>> Not subscribed, but will actively monitor the web archives for this 
>> mailing list for the next several days.
> 
> It'd be best to raise a report against ACPI?BIOS (I think) at
> bugzilla.kernel.org, please.

Once again, I'm talking about apic not acpi... does this advice still hold?

> If any previous kernel version worked OK, please be sure to note that.

I don't believe that's the case.  Put another way, I've yet to find a 
Linux kernel that can boot without the noapic option on this machine.

> Thanks.

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Andrew Morton]

On Tue, 03 Feb 2009 12:58:36 -0500
Sam Ruby <rubys@intertwingly.net> wrote:

> Andrew Morton wrote:
> > On Thu, 22 Jan 2009 08:37:01 -0500 Sam Ruby <rubys@intertwingly.net> wrote:
> > 
> >> Hardware summary: http://tinyurl.com/ap79ra
> >> APIC details: http://intertwingly.net/stories/2009/01/22/
> >> Note acpidump.err: Wrong checksum for OEMB!
> >>
> >> Messages on boot using Intrepid, Jaunty Alpha 3, or Fedora 10:
> >>
> >> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> >> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
> >>   Boot with apic=debug and send a report.  Then try booting with the 
> >> 'noapic' option.
> >> [    0.296001]
> >>
> >> Able to get past this issue using "noapic", at which point things mostly 
> >> work,
> > 
> > Join the ever-growing noapic club :(
> > 
> > I assume this is an ACPI problem.  Or at least, a BIOS problem which
> > ACPI can solve for us.
> 
> My understanding is that APCI and APIC are two different things.

They sure are.  But it is ACPI which communicates with the BIOS to tell
the kernel about the APIC, hwo it's wired up, etc.

Although in this case it looks like the problem might be with the
mp-bios tables, which ACPI does not handle.  In which case it would be
core x86 code which will have to fix this up, not ACPI.

Did you try updating the BIOS?

> >> but rsync of large iso images result in corrupt files.  Able to 
> >> copy those same files using Vista on the same machine, or using Hardy on 
> >> another machine.  This problem may not be related to the above, but it 
> >> seems plausible to me that this might be an interrupt issue.
> > 
> > Yes, it might be unrelated.  There are no kernel messages when it happens?
> 
> I see no messages to /var/log/messages while doing the following (which 
> involves rsync'ing a 2618793984 byte file from an NTFS to ext3 drive on 
> the same machine:
> 
> rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
> rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
> MD5(windows7_7000.iso)= 953b9ac92d58f5edef525004bcce048d
> rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
> rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
> MD5(windows7_7000.iso)= 695328ef1280708eb73303656f6ef0b2
> 
> >> memtest86+ runs clean.
> >>
> >> Quite willing to invest time in installing kernels or distributions on 
> >> fresh hard drives, run tests, obtain debug information, and report back.
> >>
> >> More background here: http://intertwingly.net/blog/2009/01/20/noAPIC
> >>
> >> Not subscribed, but will actively monitor the web archives for this 
> >> mailing list for the next several days.
> > 
> > It'd be best to raise a report against ACPI?BIOS (I think) at
> > bugzilla.kernel.org, please.
> 
> Once again, I'm talking about apic not acpi... does this advice still hold?

Not sure.  Perhaps platform_i386/platform_x86_64 would be correct.
Can the x86 maintainers please advise?

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

Andrew Morton wrote:
> On Tue, 03 Feb 2009 12:58:36 -0500
> Sam Ruby <rubys@intertwingly.net> wrote:
> 
>> Andrew Morton wrote:
>>> On Thu, 22 Jan 2009 08:37:01 -0500 Sam Ruby <rubys@intertwingly.net> wrote:
>>>
>>>> Hardware summary: http://tinyurl.com/ap79ra
>>>> APIC details: http://intertwingly.net/stories/2009/01/22/
>>>> Note acpidump.err: Wrong checksum for OEMB!
>>>>
>>>> Messages on boot using Intrepid, Jaunty Alpha 3, or Fedora 10:
>>>>
>>>> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>>>> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
>>>>   Boot with apic=debug and send a report.  Then try booting with the 
>>>> 'noapic' option.
>>>> [    0.296001]
>>>>
>>>> Able to get past this issue using "noapic", at which point things mostly 
>>>> work,
>>> Join the ever-growing noapic club :(
>>>
>>> I assume this is an ACPI problem.  Or at least, a BIOS problem which
>>> ACPI can solve for us.
>> My understanding is that APCI and APIC are two different things.
> 
> They sure are.  But it is ACPI which communicates with the BIOS to tell
> the kernel about the APIC, hwo it's wired up, etc.
> 
> Although in this case it looks like the problem might be with the
> mp-bios tables, which ACPI does not handle.  In which case it would be
> core x86 code which will have to fix this up, not ACPI.
> 
> Did you try updating the BIOS?

My understanding is that a BIOS may be specific to the machine.
In the past, I've been able to find BIOS updates at manufacturer web 
sites.  This one is difficult to navigate, but the closest I have been 
able to find is:

http://www.acerpanam.com/synapse/forms/portal20.cfm?website=AcerPanAm.com&siteid=7117&areaid=2&formid=3394#results

Product Line => Desktop
Aspire M3640
Search

In the list that results, I see "INT15 Driver for AMI BIOS Core" for 
Vista, but no BIOS.  It could be that I just don't know where to look.

Note: my actual machine is designated AM3641-EQ8200A

>>>> but rsync of large iso images result in corrupt files.  Able to 
>>>> copy those same files using Vista on the same machine, or using Hardy on 
>>>> another machine.  This problem may not be related to the above, but it 
>>>> seems plausible to me that this might be an interrupt issue.
>>> Yes, it might be unrelated.  There are no kernel messages when it happens?
>> I see no messages to /var/log/messages while doing the following (which 
>> involves rsync'ing a 2618793984 byte file from an NTFS to ext3 drive on 
>> the same machine:
>>
>> rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
>> rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
>> MD5(windows7_7000.iso)= 953b9ac92d58f5edef525004bcce048d
>> rubys@rubix4:~/tmp$ rsync /mnt/shared/windows7_7000.iso .
>> rubys@rubix4:~/tmp$ openssl md5 windows7_7000.iso
>> MD5(windows7_7000.iso)= 695328ef1280708eb73303656f6ef0b2
>>
>>>> memtest86+ runs clean.
>>>>
>>>> Quite willing to invest time in installing kernels or distributions on 
>>>> fresh hard drives, run tests, obtain debug information, and report back.
>>>>
>>>> More background here: http://intertwingly.net/blog/2009/01/20/noAPIC
>>>>
>>>> Not subscribed, but will actively monitor the web archives for this 
>>>> mailing list for the next several days.
>>> It'd be best to raise a report against ACPI?BIOS (I think) at
>>> bugzilla.kernel.org, please.
>> Once again, I'm talking about apic not acpi... does this advice still hold?
> 
> Not sure.  Perhaps platform_i386/platform_x86_64 would be correct.
> Can the x86 maintainers please advise?

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Thomas Gleixner]

On Tue, 3 Feb 2009, Andrew Morton wrote:
> > >> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> > >> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
Sigh.

> Not sure.  Perhaps platform_i386/platform_x86_64 would be correct.
> Can the x86 maintainers please advise?

Yup. That's correct

@Sam: I just checked, that your machine has a serial port. That's a
good start :)

Do you have a second computer around with a serial port ? If yes, then
please add the following to the kernel command line:

       earlyprintk=serial,ttyS0,115200 apic=debug

and connect the serial ports with a null modem cable. Fire up a
terminal program on the second machine and capture the output.

Thanks,

	tglx

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

Thomas Gleixner wrote:
> On Tue, 3 Feb 2009, Andrew Morton wrote:
>>>>> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
>>>>> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
> Sigh.
> 
>> Not sure.  Perhaps platform_i386/platform_x86_64 would be correct.
>> Can the x86 maintainers please advise?
> 
> Yup. That's correct
> 
> @Sam: I just checked, that your machine has a serial port. That's a
> good start :)
> 
> Do you have a second computer around with a serial port ? If yes, then
> please add the following to the kernel command line:
> 
>        earlyprintk=serial,ttyS0,115200 apic=debug
> 
> and connect the serial ports with a null modem cable. Fire up a
> terminal program on the second machine and capture the output.

I do have a second computer, and went out and bought a null modem 
adapter for my serial cable and connected the two machines.  I've tried 
installing minicom and also connecting it to ttyS0 at 115200 baud on the 
second machine, but when I boot the first machine I don't see any output 
on the terminal.

What am I missing?

> Thanks,
> 
> 	tglx

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Len Brown]

> > Do you have a second computer around with a serial port ? If yes, then
> > please add the following to the kernel command line:
> > 
> >        earlyprintk=serial,ttyS0,115200 apic=debug
> > 
> > and connect the serial ports with a null modem cable. Fire up a
> > terminal program on the second machine and capture the output.
> 
> I do have a second computer, and went out and bought a null modem adapter for
> my serial cable and connected the two machines.  I've tried installing minicom
> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
> I boot the first machine I don't see any output on the terminal.

/boot/grub/menu.lst:

serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=300 serial console

is what I use.

this will give you a prompt from grub even before the kernel boots
so you can select (and edit) your kernel via menu over the serial line
if you wish.

If this doesn't work, then the kernel earlyprintk is unlikely to work 
also.

Note that there may be some BIOS SETUP options related to the serial port 
-- worth checking.

Also, in minicom, be sure to turn off HW flow control

there is a fancy serial console document someplace on this,
probably at http://tldp.org/

good luck,
-Len

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

Len Brown wrote:
>>> Do you have a second computer around with a serial port ? If yes, then
>>> please add the following to the kernel command line:
>>>
>>>        earlyprintk=serial,ttyS0,115200 apic=debug
>>>
>>> and connect the serial ports with a null modem cable. Fire up a
>>> terminal program on the second machine and capture the output.
>> I do have a second computer, and went out and bought a null modem adapter for
>> my serial cable and connected the two machines.  I've tried installing minicom
>> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
>> I boot the first machine I don't see any output on the terminal.
> 
> /boot/grub/menu.lst:
> 
> serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
> terminal --timeout=300 serial console
> 
> is what I use.
> 
> this will give you a prompt from grub even before the kernel boots
> so you can select (and edit) your kernel via menu over the serial line
> if you wish.

I've got that working now.  I used minicom on the remote machine, 
capturing the output.  You can see me fumbling around, editing the 
kernel command and booting here:

http://intertwingly.net/stories/2009/02/09/minicom.out

While it is difficult to make out what I did given line wrapping, etc, 
what I started with was:

/boot/vmlinuz-2.6.27-11-generic 
root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro noapic quiet splash

After editing, what I had was

/boot/vmlinuz-2.6.27-11-generic 
root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro quiet splash 
earlyprintk=serial,ttyS0,115200 apic=debug

As you can see, the last line I saw was "Starting up ...", after which 
point the "MP-BIOS bug: 8254 timer not connected to IO-APIC" etc output 
appeared on the monitor that is directly connected to the machine being 
booted (i.e., this text did not appear on the minicom session).

> If this doesn't work, then the kernel earlyprintk is unlikely to work 
> also.

At the moment, it looks like it works, but earlyprintk does not work for 
me, at least not on Ubuntu 8.10, kernel 2.6.27-11-generic.

> Note that there may be some BIOS SETUP options related to the serial port 
> -- worth checking.
> 
> Also, in minicom, be sure to turn off HW flow control
> 
> there is a fancy serial console document someplace on this,
> probably at http://tldp.org/

As I have managed to get grub to talk to the serial console, I did not 
explore these options further.  Please let me know if there is something 
in particular I should explore.

> good luck,
> -Len

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[yakui_zhao]

On Tue, 2009-02-10 at 07:50 +0800, Sam Ruby wrote:
> Len Brown wrote:
> >>> Do you have a second computer around with a serial port ? If yes, then
> >>> please add the following to the kernel command line:
> >>>
> >>>        earlyprintk=serial,ttyS0,115200 apic=debug
> >>>
> >>> and connect the serial ports with a null modem cable. Fire up a
> >>> terminal program on the second machine and capture the output.
> >> I do have a second computer, and went out and bought a null modem adapter for
> >> my serial cable and connected the two machines.  I've tried installing minicom
> >> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
> >> I boot the first machine I don't see any output on the terminal.
> > 
> > /boot/grub/menu.lst:
> > 
> > serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
> > terminal --timeout=300 serial console
Do you have an opportunity to try the boot option of "acpi=off" as
suggested by Lenb?
     From the log it seems that there exists the following warning
message:
    >MP-BIOS bug: 8254 timer not connected to IO-APIC

    Can you try the following boot options?
    a. acpi_use_timer_override
    b. acpi_skip_timer_override
    c. noapic

    Thanks.
> > 
> > is what I use.
> > 
> > this will give you a prompt from grub even before the kernel boots
> > so you can select (and edit) your kernel via menu over the serial line
> > if you wish.
> 
> I've got that working now.  I used minicom on the remote machine, 
> capturing the output.  You can see me fumbling around, editing the 
> kernel command and booting here:
> 
> http://intertwingly.net/stories/2009/02/09/minicom.out
> 
> While it is difficult to make out what I did given line wrapping, etc, 
> what I started with was:
> 
> /boot/vmlinuz-2.6.27-11-generic 
> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro noapic quiet splash
> 
> After editing, what I had was
> 
> /boot/vmlinuz-2.6.27-11-generic 
> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro quiet splash 
> earlyprintk=serial,ttyS0,115200 apic=debug
> 
> As you can see, the last line I saw was "Starting up ...", after which 
> point the "MP-BIOS bug: 8254 timer not connected to IO-APIC" etc output 
> appeared on the monitor that is directly connected to the machine being 
> booted (i.e., this text did not appear on the minicom session).
> 
> > If this doesn't work, then the kernel earlyprintk is unlikely to work 
> > also.
> 
> At the moment, it looks like it works, but earlyprintk does not work for 
> me, at least not on Ubuntu 8.10, kernel 2.6.27-11-generic.
> 
> > Note that there may be some BIOS SETUP options related to the serial port 
> > -- worth checking.
> > 
> > Also, in minicom, be sure to turn off HW flow control
> > 
> > there is a fancy serial console document someplace on this,
> > probably at http://tldp.org/
> 
> As I have managed to get grub to talk to the serial console, I did not 
> explore these options further.  Please let me know if there is something 
> in particular I should explore.
> 
> > good luck,
> > -Len
> 
> - Sam Ruby
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

yakui_zhao wrote:
> On Tue, 2009-02-10 at 07:50 +0800, Sam Ruby wrote:
>> Len Brown wrote:
>>>>> Do you have a second computer around with a serial port ? If yes, then
>>>>> please add the following to the kernel command line:
>>>>>
>>>>>        earlyprintk=serial,ttyS0,115200 apic=debug
>>>>>
>>>>> and connect the serial ports with a null modem cable. Fire up a
>>>>> terminal program on the second machine and capture the output.
>>>> I do have a second computer, and went out and bought a null modem adapter for
>>>> my serial cable and connected the two machines.  I've tried installing minicom
>>>> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
>>>> I boot the first machine I don't see any output on the terminal.
>>> /boot/grub/menu.lst:
>>>
>>> serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
>>> terminal --timeout=300 serial console
> Do you have an opportunity to try the boot option of "acpi=off" as
> suggested by Lenb?

That also produces the "MP-BIOS bug" message and halts.

>      From the log it seems that there exists the following warning
> message:
>     >MP-BIOS bug: 8254 timer not connected to IO-APIC
> 
>     Can you try the following boot options?
>     a. acpi_use_timer_override

That also produces the "MP-BIOS bug" message and halts.

>     b. acpi_skip_timer_override

That also produces the "MP-BIOS bug" message and halts.

>     c. noapic

With that, I can boot, but I get data corruption problems.  Data 
corruption problems I don't see when using the same hardware but with 
Microsoft Vista.  More details can be found here:

http://intertwingly.net/blog/2009/01/20/noAPIC
http://intertwingly.net/stories/2009/01/22/

I am willing to install new kernels on fresh hard drives, run diagnostic 
programs and report the output, including capturing serial output.  Is 
there any data I can gather to help diagnose this problem?

>     Thanks.
>>> is what I use.
>>>
>>> this will give you a prompt from grub even before the kernel boots
>>> so you can select (and edit) your kernel via menu over the serial line
>>> if you wish.
>> I've got that working now.  I used minicom on the remote machine, 
>> capturing the output.  You can see me fumbling around, editing the 
>> kernel command and booting here:
>>
>> http://intertwingly.net/stories/2009/02/09/minicom.out
>>
>> While it is difficult to make out what I did given line wrapping, etc, 
>> what I started with was:
>>
>> /boot/vmlinuz-2.6.27-11-generic 
>> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro noapic quiet splash
>>
>> After editing, what I had was
>>
>> /boot/vmlinuz-2.6.27-11-generic 
>> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro quiet splash 
>> earlyprintk=serial,ttyS0,115200 apic=debug
>>
>> As you can see, the last line I saw was "Starting up ...", after which 
>> point the "MP-BIOS bug: 8254 timer not connected to IO-APIC" etc output 
>> appeared on the monitor that is directly connected to the machine being 
>> booted (i.e., this text did not appear on the minicom session).
>>
>>> If this doesn't work, then the kernel earlyprintk is unlikely to work 
>>> also.
>> At the moment, it looks like it works, but earlyprintk does not work for 
>> me, at least not on Ubuntu 8.10, kernel 2.6.27-11-generic.
>>
>>> Note that there may be some BIOS SETUP options related to the serial port 
>>> -- worth checking.
>>>
>>> Also, in minicom, be sure to turn off HW flow control
>>>
>>> there is a fancy serial console document someplace on this,
>>> probably at http://tldp.org/
>> As I have managed to get grub to talk to the serial console, I did not 
>> explore these options further.  Please let me know if there is something 
>> in particular I should explore.
>>
>>> good luck,
>>> -Len
>> - Sam Ruby
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[yakui_zhao]

On Tue, 2009-02-10 at 19:46 +0800, Sam Ruby wrote:
> yakui_zhao wrote:
> > On Tue, 2009-02-10 at 07:50 +0800, Sam Ruby wrote:
> >> Len Brown wrote:
> >>>>> Do you have a second computer around with a serial port ? If yes, then
> >>>>> please add the following to the kernel command line:
> >>>>>
> >>>>>        earlyprintk=serial,ttyS0,115200 apic=debug
> >>>>>
> >>>>> and connect the serial ports with a null modem cable. Fire up a
> >>>>> terminal program on the second machine and capture the output.
> >>>> I do have a second computer, and went out and bought a null modem adapter for
> >>>> my serial cable and connected the two machines.  I've tried installing minicom
> >>>> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
> >>>> I boot the first machine I don't see any output on the terminal.
> >>> /boot/grub/menu.lst:
> >>>
> >>> serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
> >>> terminal --timeout=300 serial console
> > Do you have an opportunity to try the boot option of "acpi=off" as
> > suggested by Lenb?
> 
> That also produces the "MP-BIOS bug" message and halts.
Do you mean that the box still can't be booted with ACPI disabled? If
so, it should be a BIOS bug. It had better be fixed by bios upgrading.
   From the following test it seems that there exists the MPS table,
which describes how the timer is connected with I/O APIC. But
unfortunately it is still incorrect. 
   And the timer still can't work w/o the timer override.
   
   Only when I/O apic is skipped can the box be booted. The timer is
connected with 8259.  And the timer can work as expected. But there
still exist other issue.
    
> 
> >      From the log it seems that there exists the following warning
> > message:
> >     >MP-BIOS bug: 8254 timer not connected to IO-APIC
> > 
> >     Can you try the following boot options?
> >     a. acpi_use_timer_override
> 
> That also produces the "MP-BIOS bug" message and halts.
> 
> >     b. acpi_skip_timer_override
> 
> That also produces the "MP-BIOS bug" message and halts.
> 
> >     c. noapic
> 
> With that, I can boot, but I get data corruption problems.  Data 
> corruption problems I don't see when using the same hardware but with 
> Microsoft Vista.  More details can be found here:
> 
> http://intertwingly.net/blog/2009/01/20/noAPIC
> http://intertwingly.net/stories/2009/01/22/
> 
> I am willing to install new kernels on fresh hard drives, run diagnostic 
> programs and report the output, including capturing serial output.  Is 
> there any data I can gather to help diagnose this problem?
> 
> >     Thanks.
> >>> is what I use.
> >>>
> >>> this will give you a prompt from grub even before the kernel boots
> >>> so you can select (and edit) your kernel via menu over the serial line
> >>> if you wish.
> >> I've got that working now.  I used minicom on the remote machine, 
> >> capturing the output.  You can see me fumbling around, editing the 
> >> kernel command and booting here:
> >>
> >> http://intertwingly.net/stories/2009/02/09/minicom.out
> >>
> >> While it is difficult to make out what I did given line wrapping, etc, 
> >> what I started with was:
> >>
> >> /boot/vmlinuz-2.6.27-11-generic 
> >> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro noapic quiet splash
> >>
> >> After editing, what I had was
> >>
> >> /boot/vmlinuz-2.6.27-11-generic 
> >> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro quiet splash 
> >> earlyprintk=serial,ttyS0,115200 apic=debug
> >>
> >> As you can see, the last line I saw was "Starting up ...", after which 
> >> point the "MP-BIOS bug: 8254 timer not connected to IO-APIC" etc output 
> >> appeared on the monitor that is directly connected to the machine being 
> >> booted (i.e., this text did not appear on the minicom session).
> >>
> >>> If this doesn't work, then the kernel earlyprintk is unlikely to work 
> >>> also.
> >> At the moment, it looks like it works, but earlyprintk does not work for 
> >> me, at least not on Ubuntu 8.10, kernel 2.6.27-11-generic.
> >>
> >>> Note that there may be some BIOS SETUP options related to the serial port 
> >>> -- worth checking.
> >>>
> >>> Also, in minicom, be sure to turn off HW flow control
> >>>
> >>> there is a fancy serial console document someplace on this,
> >>> probably at http://tldp.org/
> >> As I have managed to get grub to talk to the serial console, I did not 
> >> explore these options further.  Please let me know if there is something 
> >> in particular I should explore.
> >>
> >>> good luck,
> >>> -Len
> >> - Sam Ruby
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> - Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

yakui_zhao wrote:
> On Tue, 2009-02-10 at 19:46 +0800, Sam Ruby wrote:
>> yakui_zhao wrote:
>>> On Tue, 2009-02-10 at 07:50 +0800, Sam Ruby wrote:
>>>> Len Brown wrote:
>>>>>>> Do you have a second computer around with a serial port ? If yes, then
>>>>>>> please add the following to the kernel command line:
>>>>>>>
>>>>>>>        earlyprintk=serial,ttyS0,115200 apic=debug
>>>>>>>
>>>>>>> and connect the serial ports with a null modem cable. Fire up a
>>>>>>> terminal program on the second machine and capture the output.
>>>>>> I do have a second computer, and went out and bought a null modem adapter for
>>>>>> my serial cable and connected the two machines.  I've tried installing minicom
>>>>>> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
>>>>>> I boot the first machine I don't see any output on the terminal.
>>>>> /boot/grub/menu.lst:
>>>>>
>>>>> serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
>>>>> terminal --timeout=300 serial console
>>> Do you have an opportunity to try the boot option of "acpi=off" as
>>> suggested by Lenb?
>> That also produces the "MP-BIOS bug" message and halts.
> Do you mean that the box still can't be booted with ACPI disabled? If
> so, it should be a BIOS bug. It had better be fixed by bios upgrading.

Yes, that's what I mean.  Linux won't boot with acpi=off, but will boot 
with noapic.

This is a fairly new machine, and I have yet to see where ACER has 
posted a more recent BIOS for this model.

>    From the following test it seems that there exists the MPS table,
> which describes how the timer is connected with I/O APIC. But
> unfortunately it is still incorrect. 
>    And the timer still can't work w/o the timer override.
>    
>    Only when I/O apic is skipped can the box be booted. The timer is
> connected with 8259.  And the timer can work as expected. But there
> still exist other issue.
>     
>>>      From the log it seems that there exists the following warning
>>> message:
>>>     >MP-BIOS bug: 8254 timer not connected to IO-APIC
>>>
>>>     Can you try the following boot options?
>>>     a. acpi_use_timer_override
>> That also produces the "MP-BIOS bug" message and halts.
>>
>>>     b. acpi_skip_timer_override
>> That also produces the "MP-BIOS bug" message and halts.
>>
>>>     c. noapic
>> With that, I can boot, but I get data corruption problems.  Data 
>> corruption problems I don't see when using the same hardware but with 
>> Microsoft Vista.  More details can be found here:
>>
>> http://intertwingly.net/blog/2009/01/20/noAPIC
>> http://intertwingly.net/stories/2009/01/22/
>>
>> I am willing to install new kernels on fresh hard drives, run diagnostic 
>> programs and report the output, including capturing serial output.  Is 
>> there any data I can gather to help diagnose this problem?
>>
>>>     Thanks.
>>>>> is what I use.
>>>>>
>>>>> this will give you a prompt from grub even before the kernel boots
>>>>> so you can select (and edit) your kernel via menu over the serial line
>>>>> if you wish.
>>>> I've got that working now.  I used minicom on the remote machine, 
>>>> capturing the output.  You can see me fumbling around, editing the 
>>>> kernel command and booting here:
>>>>
>>>> http://intertwingly.net/stories/2009/02/09/minicom.out
>>>>
>>>> While it is difficult to make out what I did given line wrapping, etc, 
>>>> what I started with was:
>>>>
>>>> /boot/vmlinuz-2.6.27-11-generic 
>>>> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro noapic quiet splash
>>>>
>>>> After editing, what I had was
>>>>
>>>> /boot/vmlinuz-2.6.27-11-generic 
>>>> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro quiet splash 
>>>> earlyprintk=serial,ttyS0,115200 apic=debug
>>>>
>>>> As you can see, the last line I saw was "Starting up ...", after which 
>>>> point the "MP-BIOS bug: 8254 timer not connected to IO-APIC" etc output 
>>>> appeared on the monitor that is directly connected to the machine being 
>>>> booted (i.e., this text did not appear on the minicom session).
>>>>
>>>>> If this doesn't work, then the kernel earlyprintk is unlikely to work 
>>>>> also.
>>>> At the moment, it looks like it works, but earlyprintk does not work for 
>>>> me, at least not on Ubuntu 8.10, kernel 2.6.27-11-generic.
>>>>
>>>>> Note that there may be some BIOS SETUP options related to the serial port 
>>>>> -- worth checking.
>>>>>
>>>>> Also, in minicom, be sure to turn off HW flow control
>>>>>
>>>>> there is a fancy serial console document someplace on this,
>>>>> probably at http://tldp.org/
>>>> As I have managed to get grub to talk to the serial console, I did not 
>>>> explore these options further.  Please let me know if there is something 
>>>> in particular I should explore.
>>>>
>>>>> good luck,
>>>>> -Len
>>>> - Sam Ruby
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> - Sam Ruby

- Sam Ruby

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Sam Ruby]

yakui_zhao wrote:
> On Tue, 2009-02-10 at 19:46 +0800, Sam Ruby wrote:
>> yakui_zhao wrote:
>>> On Tue, 2009-02-10 at 07:50 +0800, Sam Ruby wrote:
>>>> Len Brown wrote:
>>>>>>> Do you have a second computer around with a serial port ? If yes, then
>>>>>>> please add the following to the kernel command line:
>>>>>>>
>>>>>>>        earlyprintk=serial,ttyS0,115200 apic=debug
>>>>>>>
>>>>>>> and connect the serial ports with a null modem cable. Fire up a
>>>>>>> terminal program on the second machine and capture the output.
>>>>>> I do have a second computer, and went out and bought a null modem adapter for
>>>>>> my serial cable and connected the two machines.  I've tried installing minicom
>>>>>> and also connecting it to ttyS0 at 115200 baud on the second machine, but when
>>>>>> I boot the first machine I don't see any output on the terminal.
>>>>> /boot/grub/menu.lst:
>>>>>
>>>>> serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
>>>>> terminal --timeout=300 serial console
>>> Do you have an opportunity to try the boot option of "acpi=off" as
>>> suggested by Lenb?
>> That also produces the "MP-BIOS bug" message and halts.
> Do you mean that the box still can't be booted with ACPI disabled? If
> so, it should be a BIOS bug. It had better be fixed by bios upgrading.

BIOS information, in case it is helpful to anyone:

http://intertwingly.net/stories/2009/02/11/dmidecode.out

>    From the following test it seems that there exists the MPS table,
> which describes how the timer is connected with I/O APIC. But
> unfortunately it is still incorrect. 
>    And the timer still can't work w/o the timer override.
>    
>    Only when I/O apic is skipped can the box be booted. The timer is
> connected with 8259.  And the timer can work as expected. But there
> still exist other issue.
>     
>>>      From the log it seems that there exists the following warning
>>> message:
>>>     >MP-BIOS bug: 8254 timer not connected to IO-APIC
>>>
>>>     Can you try the following boot options?
>>>     a. acpi_use_timer_override
>> That also produces the "MP-BIOS bug" message and halts.
>>
>>>     b. acpi_skip_timer_override
>> That also produces the "MP-BIOS bug" message and halts.
>>
>>>     c. noapic
>> With that, I can boot, but I get data corruption problems.  Data 
>> corruption problems I don't see when using the same hardware but with 
>> Microsoft Vista.  More details can be found here:
>>
>> http://intertwingly.net/blog/2009/01/20/noAPIC
>> http://intertwingly.net/stories/2009/01/22/
>>
>> I am willing to install new kernels on fresh hard drives, run diagnostic 
>> programs and report the output, including capturing serial output.  Is 
>> there any data I can gather to help diagnose this problem?
>>
>>>     Thanks.
>>>>> is what I use.
>>>>>
>>>>> this will give you a prompt from grub even before the kernel boots
>>>>> so you can select (and edit) your kernel via menu over the serial line
>>>>> if you wish.
>>>> I've got that working now.  I used minicom on the remote machine, 
>>>> capturing the output.  You can see me fumbling around, editing the 
>>>> kernel command and booting here:
>>>>
>>>> http://intertwingly.net/stories/2009/02/09/minicom.out
>>>>
>>>> While it is difficult to make out what I did given line wrapping, etc, 
>>>> what I started with was:
>>>>
>>>> /boot/vmlinuz-2.6.27-11-generic 
>>>> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro noapic quiet splash
>>>>
>>>> After editing, what I had was
>>>>
>>>> /boot/vmlinuz-2.6.27-11-generic 
>>>> root=UUID=4fce230e-fe72-4685-aab0-294ef1c20efa ro quiet splash 
>>>> earlyprintk=serial,ttyS0,115200 apic=debug
>>>>
>>>> As you can see, the last line I saw was "Starting up ...", after which 
>>>> point the "MP-BIOS bug: 8254 timer not connected to IO-APIC" etc output 
>>>> appeared on the monitor that is directly connected to the machine being 
>>>> booted (i.e., this text did not appear on the minicom session).
>>>>
>>>>> If this doesn't work, then the kernel earlyprintk is unlikely to work 
>>>>> also.
>>>> At the moment, it looks like it works, but earlyprintk does not work for 
>>>> me, at least not on Ubuntu 8.10, kernel 2.6.27-11-generic.
>>>>
>>>>> Note that there may be some BIOS SETUP options related to the serial port 
>>>>> -- worth checking.
>>>>>
>>>>> Also, in minicom, be sure to turn off HW flow control
>>>>>
>>>>> there is a fancy serial console document someplace on this,
>>>>> probably at http://tldp.org/
>>>> As I have managed to get grub to talk to the serial console, I did not 
>>>> explore these options further.  Please let me know if there is something 
>>>> in particular I should explore.
>>>>
>>>>> good luck,
>>>>> -Len
>>>> - Sam Ruby
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> - Sam Ruby
> 

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Len Brown]

> > > >> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> > > >> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 

just for the record, check_timer() is x86 IO-APIC code, not ACPI code.
So if the machine actually has MPS support, you'll probably get the 
same result with "acpi=off".  (of course if it doesn't have MPS,
then acpi=off would boot in uniprocessor legacy PIC mode)

cheers,
-Len

Re: [APIC] Kernel panic, rsync corruption, intel q8200, 2.6.28-rc8

[Maciej W. Rozycki]

On Tue, 3 Feb 2009, Andrew Morton wrote:

> > >> [    0.296001] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> > >> [    0.296001] Kernel panic - not syncing: IO-APIC + timer doesn't work! 
> > >>   Boot with apic=debug and send a report.  Then try booting with the 
> > >> 'noapic' option.
> > >> [    0.296001]
> > >>
> > >> Able to get past this issue using "noapic", at which point things mostly 
> > >> work,
> > > 
> > > Join the ever-growing noapic club :(
> > > 
> > > I assume this is an ACPI problem.  Or at least, a BIOS problem which
> > > ACPI can solve for us.
> > 
> > My understanding is that APCI and APIC are two different things.
> 
> They sure are.  But it is ACPI which communicates with the BIOS to tell
> the kernel about the APIC, hwo it's wired up, etc.
> 
> Although in this case it looks like the problem might be with the
> mp-bios tables, which ACPI does not handle.  In which case it would be
> core x86 code which will have to fix this up, not ACPI.

 The message is misleading.  It predates the ACPI and has never been 
updated.  It applies equally to both MPS and ACPI systems.  Earlier 
bootstrap messages can be used to determine which configuration method has 
been used by the kernel on this system (many systems provide both).

  Maciej