Stable regression: usb-storage is stuck in 2.6.26.5

Stable regression: usb-storage is stuck in 2.6.26.5

[Michael Tokarev]

Just updated my machine from 2.6.26.1 to 2.6.26.5, and it does not
wan to boot anymore.  It says "Initializing USB Mass Storage driver.."
(when trying to load usb-storage module) and sits here forever, every
2 minutes showing warnings about stuck processes (traces below).
With 2.6.26.1 or any previous version I never experienced this
problem.  Both 32- and 64-bits kernels behave the same way.
While waiting for usb-storage to complete, no other USB devices
works (incl. mouse and keyboard).

The machine is an AMD X2-64 (BE-2400) processor with an nVidia MCP51
chipset (lspci is below).  Motherboard is Asus M2NPV-VM, latest BIOS
(1401, dated 08/07/2008)

I'm rebuilding 2.6.26.3 for now -- trying to bisect the issue...

Thanks.

/mjt

Example of dmesg output:

Sep 13 16:04:51 gandalf kernel: INFO: task insmod:2662 blocked for more than 120 seconds.
Sep 13 16:04:51 gandalf kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 13 16:04:51 gandalf kernel: insmod        D c01d7662     0  2662   2580
Sep 13 16:04:51 gandalf kernel:        f7241340 00000082 00000000 c01d7662 00000000 f7228d38 c03dddac f7c0b6c0 
Sep 13 16:04:51 gandalf kernel:        00000001 f7f07dc0 c0399080 f7fe4584 00000008 00001f00 f77925d0 f7792734 
Sep 13 16:04:51 gandalf kernel:        00000001 f7228d70 c03dddac f7792734 c205e7ac c01c24e0 c205e7ac f7c28000 
Sep 13 16:04:51 gandalf kernel: Call Trace:
Sep 13 16:04:51 gandalf kernel:  [<c01d7662>] idr_get_empty_slot+0xe2/0x260
Sep 13 16:04:51 gandalf kernel:  [<c01c24e0>] sysfs_ilookup_test+0x0/0x10
Sep 13 16:04:51 gandalf kernel:  [<c02c642d>] schedule_timeout+0x7d/0xb0
Sep 13 16:04:51 gandalf kernel:  [<c01c27b1>] sysfs_find_dirent+0x21/0x30
Sep 13 16:04:51 gandalf kernel:  [<c02c6da5>] __down+0x55/0x90
Sep 13 16:04:51 gandalf kernel:  [<c0140b2a>] down+0x3a/0x40
Sep 13 16:04:51 gandalf kernel:  [<c0233863>] __driver_attach+0x23/0x80
Sep 13 16:04:51 gandalf kernel:  [<c0233064>] bus_for_each_dev+0x44/0x70
Sep 13 16:04:51 gandalf kernel:  [<c02335b6>] driver_attach+0x16/0x20
Sep 13 16:04:51 gandalf kernel:  [<c0233840>] __driver_attach+0x0/0x80
Sep 13 16:04:51 gandalf kernel:  [<c0232a27>] bus_add_driver+0x1a7/0x220
Sep 13 16:04:51 gandalf kernel:  [<c0233a4c>] driver_register+0x5c/0x130
Sep 13 16:04:51 gandalf kernel:  [<f8896961>] usb_register_driver+0x81/0x100 [usbcore]
Sep 13 16:04:51 gandalf kernel:  [<f8b88024>] usb_stor_init+0x24/0x3f [usb_storage]
Sep 13 16:04:51 gandalf kernel:  [<c014be88>] sys_init_module+0x148/0x1bb0
Sep 13 16:04:51 gandalf kernel:  [<c01032f3>] sysenter_past_esp+0x78/0xb1
Sep 13 16:04:51 gandalf kernel:  =======================
Sep 13 16:05:16 gandalf kernel: INFO: task khubd:371 blocked for more than 120 seconds.
Sep 13 16:05:16 gandalf kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Sep 13 16:05:16 gandalf kernel: khubd         D f7fbbd10     0   371      2
Sep 13 16:05:16 gandalf kernel:        f7d2ce00 00000046 00000002 f7fbbd10 f7fbbd18 00000000 c01322f7 f7fbbd04 
Sep 13 16:05:16 gandalf kernel:        ffffffff 00000000 c0399080 f7fbbd18 f7fbbd14 f7fbbd10 f7d2a5d0 f7d2a734 
Sep 13 16:05:16 gandalf kernel:        00000000 ffff8dd5 00000000 f7d2a734 f7c5ec00 fffffffe 00000000 00000000 
Sep 13 16:05:16 gandalf kernel: Call Trace:
Sep 13 16:05:16 gandalf kernel:  [<c01322f7>] lock_timer_base+0x27/0x60
Sep 13 16:05:16 gandalf kernel:  [<f889393d>] usb_kill_urb+0x7d/0xc0 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<c013c4a0>] autoremove_wake_function+0x0/0x50
Sep 13 16:05:16 gandalf kernel:  [<f88949d3>] usb_start_wait_urb+0xa3/0xc0 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f8894c10>] usb_control_msg+0xd0/0x100 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f8895be2>] usb_get_descriptor+0xb2/0xe0 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<c0131ef0>] process_timeout+0x0/0x10
Sep 13 16:05:16 gandalf kernel:  [<f8896dd3>] usb_get_configuration+0xe3/0x1300 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<c01d8f5a>] kref_put+0x2a/0x60
Sep 13 16:05:16 gandalf kernel:  [<f88949a1>] usb_start_wait_urb+0x71/0xc0 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f8895be2>] usb_get_descriptor+0xb2/0xe0 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<c0132391>] del_timer_sync+0x11/0x20
Sep 13 16:05:16 gandalf kernel:  [<f8895cac>] usb_get_device_descriptor+0x9c/0xa0 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f888ecf7>] hub_port_init+0xa7/0x730 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f888fab5>] usb_configure_device+0xc5/0x100 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f889cd51>] usb_detect_quirks+0x41/0x60 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f888fc81>] usb_new_device+0x21/0x180 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<f8890a97>] hub_thread+0x3e7/0x1090 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<c013c4a0>] autoremove_wake_function+0x0/0x50
Sep 13 16:05:16 gandalf kernel:  [<f88906b0>] hub_thread+0x0/0x1090 [usbcore]
Sep 13 16:05:16 gandalf kernel:  [<c013c1b9>] kthread+0x39/0x70
Sep 13 16:05:16 gandalf kernel:  [<c013c180>] kthread+0x0/0x70
Sep 13 16:05:16 gandalf kernel:  [<c0103f93>] kernel_thread_helper+0x7/0x14
Sep 13 16:05:16 gandalf kernel:  =======================

lspci:
00:00.0 RAM memory: nVidia Corporation C51 Host Bridge (rev a2)
00:00.1 RAM memory: nVidia Corporation C51 Memory Controller 0 (rev a2)
00:00.2 RAM memory: nVidia Corporation C51 Memory Controller 1 (rev a2)
00:00.3 RAM memory: nVidia Corporation C51 Memory Controller 5 (rev a2)
00:00.4 RAM memory: nVidia Corporation C51 Memory Controller 4 (rev a2)
00:00.5 RAM memory: nVidia Corporation C51 Host Bridge (rev a2)
00:00.6 RAM memory: nVidia Corporation C51 Memory Controller 3 (rev a2)
00:00.7 RAM memory: nVidia Corporation C51 Memory Controller 2 (rev a2)
00:05.0 VGA compatible controller: nVidia Corporation C51PV [GeForce 6150] (rev a2)
00:09.0 RAM memory: nVidia Corporation MCP51 Host Bridge (rev a2)
00:0a.0 ISA bridge: nVidia Corporation MCP51 LPC Bridge (rev a2)
00:0a.1 SMBus: nVidia Corporation MCP51 SMBus (rev a2)
00:0a.2 RAM memory: nVidia Corporation MCP51 Memory Controller 0 (rev a2)
00:0b.0 USB Controller: nVidia Corporation MCP51 USB Controller (rev a2)
00:0b.1 USB Controller: nVidia Corporation MCP51 USB Controller (rev a2)
00:0d.0 IDE interface: nVidia Corporation MCP51 IDE (rev a1)
00:0e.0 IDE interface: nVidia Corporation MCP51 Serial ATA Controller (rev a1)
00:10.0 PCI bridge: nVidia Corporation MCP51 PCI Bridge (rev a2)
00:10.1 Audio device: nVidia Corporation MCP51 High Definition Audio (rev a2)
00:14.0 Bridge: nVidia Corporation MCP51 Ethernet Controller (rev a1)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
01:08.0 Serial controller: 3Com Corp, Modem Division 56K FaxModem Model 5610 (rev 01)
01:09.0 Multimedia controller: Philips Semiconductors SAA7134/SAA7135HL Video Broadcast Decoder (rev 01)

Re: Stable regression: usb-storage is stuck in 2.6.26.5

[Michael Tokarev]

Michael Tokarev wrote:
> Just updated my machine from 2.6.26.1 to 2.6.26.5, and it does not
> wan to boot anymore.  It says "Initializing USB Mass Storage driver.."
> (when trying to load usb-storage module) and sits here forever, every
> 2 minutes showing warnings about stuck processes (traces below).
> With 2.6.26.1 or any previous version I never experienced this
> problem.  Both 32- and 64-bits kernels behave the same way.
> While waiting for usb-storage to complete, no other USB devices
> works (incl. mouse and keyboard).
> 
> The machine is an AMD X2-64 (BE-2400) processor with an nVidia MCP51
> chipset (lspci is below).  Motherboard is Asus M2NPV-VM, latest BIOS
> (1401, dated 08/07/2008)
> 
> I'm rebuilding 2.6.26.3 for now -- trying to bisect the issue...
[...]

And it's more: even without usb-storage, other USB devices does
not work, waiting in the same usb_start_wait_urb.

Ok, it turned out to be pretty.. interesting.

It's not a regression within -stable kernel series.  The problem
happens when BOTH are true:

   - using latest BIOS for this mobo (1401), AND
   - enabling KVM_GUEST in kernel.

The only difference in my kernel config between .1 and .5 was
to enable KVM_GUEST and KVM_CLOCK, and also I2C_HELPER_AUTO
which was added by kernel update.

Going with previous BIOS (1201) OR disabling PARAVIRT_GUEST
fixes the problem.  I'll try to figure out which config
option is at problem here.

> Thanks.
> 
> /mjt
> 

PARAVIRT_GUEST bug? [was: Stable regression: usb-storage is stuck in 2.6.26.5]

[Michael Tokarev]

Michael Tokarev wrote:

[usb-storage hangs on asus M2NPV-VM (nVidia GeForce 6150/nForce 430)
  motherboard with latest bios]

> Ok, it turned out to be pretty.. interesting.
> 
> It's not a regression within -stable kernel series.  The problem
> happens when BOTH are true:
> 
>   - using latest BIOS for this mobo (1401), AND
>   - enabling KVM_GUEST in kernel.
> 
> The only difference in my kernel config between .1 and .5 was
> to enable KVM_GUEST and KVM_CLOCK, and also I2C_HELPER_AUTO
> which was added by kernel update.
> 
> Going with previous BIOS (1201) OR disabling PARAVIRT_GUEST
> fixes the problem.  I'll try to figure out which config
> option is at problem here.

After seeing similar situation on another motherboard, I
become curious.

Here's the result:  M3A78-EM motherboard (also from Asus),
AMD780G/SB700 etc.  Updating to one of the latest bios
makes the system unbootable.  Kernel hangs on boot right
after displaying

  SLUB: Genslabs=12, HWalign=64, Order=0-1, MinObjects=4, CPUs=2, Nodes=1

Reverting to the older BIOS or disabling PARAVIRT_GUEST in kernel
fixes this (turning off KVM_GUEST isn't enough), and kernel happily
boots further, with next line being

  hpet clockevent registered
  ...

Reportedly, another Asus motherboard (M2N-SLI DELUXE) also hangs
with new BIOS and KVM_GUEST optimizations turned on, but not on
older BIOS.

So it's quite.. funny.  At least SOME latest BIOSes from Asustec,
together with PARAVIRT_GUEST, -- a problem.  Don't do either of
those, and it's all ok.

The question is if Asus introduced the same bug in quite.. some
of their recent BIOSes, or PARAVIRT_GUEST code is somehow buggy
and the bug is exposed by new BIOS code.

Any hints on this? ;)

Thanks!

/mjt

Re: PARAVIRT_GUEST bug? [was: Stable regression: usb-storage is stuck in 2.6.26.5]

[Jeremy Fitzhardinge]

Michael Tokarev wrote:
> Michael Tokarev wrote:
>
> [usb-storage hangs on asus M2NPV-VM (nVidia GeForce 6150/nForce 430)
>  motherboard with latest bios]
>
>> Ok, it turned out to be pretty.. interesting.
>>
>> It's not a regression within -stable kernel series.  The problem
>> happens when BOTH are true:
>>
>>   - using latest BIOS for this mobo (1401), AND
>>   - enabling KVM_GUEST in kernel.
>>
>> The only difference in my kernel config between .1 and .5 was
>> to enable KVM_GUEST and KVM_CLOCK, and also I2C_HELPER_AUTO
>> which was added by kernel update.
>>
>> Going with previous BIOS (1201) OR disabling PARAVIRT_GUEST
>> fixes the problem.  I'll try to figure out which config
>> option is at problem here.
>
> After seeing similar situation on another motherboard, I
> become curious.
>
> Here's the result:  M3A78-EM motherboard (also from Asus),
> AMD780G/SB700 etc.  Updating to one of the latest bios
> makes the system unbootable.  Kernel hangs on boot right
> after displaying
>
>  SLUB: Genslabs=12, HWalign=64, Order=0-1, MinObjects=4, CPUs=2, Nodes=1
>
> Reverting to the older BIOS or disabling PARAVIRT_GUEST in kernel
> fixes this (turning off KVM_GUEST isn't enough), and kernel happily
> boots further, with next line being
>
>  hpet clockevent registered
>  ...
>
> Reportedly, another Asus motherboard (M2N-SLI DELUXE) also hangs
> with new BIOS and KVM_GUEST optimizations turned on, but not on
> older BIOS.
>
> So it's quite.. funny.  At least SOME latest BIOSes from Asustec,
> together with PARAVIRT_GUEST, -- a problem.  Don't do either of
> those, and it's all ok.
>
> The question is if Asus introduced the same bug in quite.. some
> of their recent BIOSes, or PARAVIRT_GUEST code is somehow buggy
> and the bug is exposed by new BIOS code.
>
> Any hints on this? ;)

PARAVIRT_GUEST doesn't enable anything by itself; it just makes other
config options appear.  What other config options are you enabling?

That is, what's the diff between a working and a non-working .config?

    J

Re: PARAVIRT_GUEST bug? [was: Stable regression: usb-storage is stuck in 2.6.26.5]

[Michael Tokarev]

Jeremy Fitzhardinge wrote:
[...]
> Michael Tokarev wrote:
>> The question is if Asus introduced the same bug in quite.. some
>> of their recent BIOSes, or PARAVIRT_GUEST code is somehow buggy
>> and the bug is exposed by new BIOS code.
>>
>> Any hints on this? ;)
> 
> PARAVIRT_GUEST doesn't enable anything by itself; it just makes other
> config options appear.  What other config options are you enabling?
> 
> That is, what's the diff between a working and a non-working .config?

I stand corrected.  I meant to say CONFIG_PARAVIRT, not CONFIG_PARAVIRT_GUEST:

# diff -U0 config.working config.bad
--- config.working	2008-09-16 22:09:14.844324435 +0400
+++ .config.bad	2008-09-16 22:10:04.914320258 +0400
@@ -4 +4 @@
-# Tue Sep 16 21:47:31 2008
+# Tue Sep 16 22:10:04 2008
@@ -181 +181 @@
-# CONFIG_PARAVIRT is not set
+CONFIG_PARAVIRT=y

/mjt

Re: PARAVIRT_GUEST bug? [was: Stable regression: usb-storage is stuck in 2.6.26.5]

[Michael Tokarev]

Michael Tokarev wrote:
> Michael Tokarev wrote:
> 
> [usb-storage hangs on asus M2NPV-VM (nVidia GeForce 6150/nForce 430)
>  motherboard with latest bios]
> 
>> Ok, it turned out to be pretty.. interesting.
>>
>> It's not a regression within -stable kernel series.  The problem
>> happens when BOTH are true:
>>
>>   - using latest BIOS for this mobo (1401), AND
>>   - enabling KVM_GUEST in kernel.
>>
>> The only difference in my kernel config between .1 and .5 was
>> to enable KVM_GUEST and KVM_CLOCK, and also I2C_HELPER_AUTO
>> which was added by kernel update.
>>
>> Going with previous BIOS (1201) OR disabling PARAVIRT_GUEST
>> fixes the problem.  I'll try to figure out which config
>> option is at problem here.
> 
> After seeing similar situation on another motherboard, I
> become curious.
> 
> Here's the result:  M3A78-EM motherboard (also from Asus),
> AMD780G/SB700 etc.  Updating to one of the latest bios
> makes the system unbootable.  Kernel hangs on boot right
> after displaying
> 
>  SLUB: Genslabs=12, HWalign=64, Order=0-1, MinObjects=4, CPUs=2, Nodes=1
> 
> Reverting to the older BIOS or disabling PARAVIRT_GUEST in kernel

PARAVIRT it is, not PARAVIRT_GUEST.

> fixes this (turning off KVM_GUEST isn't enough), and kernel happily
> boots further, with next line being
> 
>  hpet clockevent registered
>  ...
> 
> Reportedly, another Asus motherboard (M2N-SLI DELUXE) also hangs
> with new BIOS and KVM_GUEST optimizations turned on, but not on
> older BIOS.
> 
> So it's quite.. funny.  At least SOME latest BIOSes from Asustec,
> together with PARAVIRT_GUEST, -- a problem.  Don't do either of
> those, and it's all ok.
> 
> The question is if Asus introduced the same bug in quite.. some
> of their recent BIOSes, or PARAVIRT_GUEST code is somehow buggy
> and the bug is exposed by new BIOS code.

Two more Asus motherboards expose exactly the same problem.

M3A-H/HDMI - AMD 780G/SB700, and
M2N-VM - nVidia MCP67 /  GeForce 7050

And from the above, 3 more:

M2NPV-VM - nVidia GeForce 6150/nForce 430
M3A78-EM - AMD780G/SB700
M2N-SLI DELUXE - nVidia MCP55 (unconfirmed)

With new BIOSes and with CONFIG_PARAVIRT enabled in kernel,
system does not boot.  It seems the stalls happens at more
or less random places, but the place is always the same on
the same motherboard and different kernels -- f.e. my
M3A78-EM always stops after "SLUB: " line as shown above,
be it 32- or 64bits kernel, 2.6.26 or 2.6.26.5.

> Any hints on this? ;)
> 
> Thanks!
> 
> /mjt

Re: PARAVIRT_GUEST bug? [was: Stable regression: usb-storage is stuck in 2.6.26.5]

[Jeremy Fitzhardinge]

Michael Tokarev wrote:
> Michael Tokarev wrote:
>   
>> Michael Tokarev wrote:
>>
>> [usb-storage hangs on asus M2NPV-VM (nVidia GeForce 6150/nForce 430)
>>  motherboard with latest bios]
>>
>>     
>>> Ok, it turned out to be pretty.. interesting.
>>>
>>> It's not a regression within -stable kernel series.  The problem
>>> happens when BOTH are true:
>>>
>>>   - using latest BIOS for this mobo (1401), AND
>>>   - enabling KVM_GUEST in kernel.
>>>
>>> The only difference in my kernel config between .1 and .5 was
>>> to enable KVM_GUEST and KVM_CLOCK, and also I2C_HELPER_AUTO
>>> which was added by kernel update.
>>>
>>> Going with previous BIOS (1201) OR disabling PARAVIRT_GUEST
>>> fixes the problem.  I'll try to figure out which config
>>> option is at problem here.
>>>       
>> After seeing similar situation on another motherboard, I
>> become curious.
>>
>> Here's the result:  M3A78-EM motherboard (also from Asus),
>> AMD780G/SB700 etc.  Updating to one of the latest bios
>> makes the system unbootable.  Kernel hangs on boot right
>> after displaying
>>
>>  SLUB: Genslabs=12, HWalign=64, Order=0-1, MinObjects=4, CPUs=2, Nodes=1
>>
>> Reverting to the older BIOS or disabling PARAVIRT_GUEST in kernel
>>     
>
> PARAVIRT it is, not PARAVIRT_GUEST.
>   

Hm, mysterious.  CONFIG_PARAVIRT should have no functional effect on a
native boot.

Could you post the complete dmesg output (with "initcall_debug" on the
kernel command line), with both CONFIG_PARAVIRT on and off, and ideally,
comparing working vs non-working BIOS?

As a separate experiment (which is really grasping at straws), does
putting memmap=64k$0 on the kernel command line help?

>> fixes this (turning off KVM_GUEST isn't enough), and kernel happily
>> boots further, with next line being
>>
>>  hpet clockevent registered
>>  ...
>>
>> Reportedly, another Asus motherboard (M2N-SLI DELUXE) also hangs
>> with new BIOS and KVM_GUEST optimizations turned on, but not on
>> older BIOS.
>>
>> So it's quite.. funny.  At least SOME latest BIOSes from Asustec,
>> together with PARAVIRT_GUEST, -- a problem.  Don't do either of
>> those, and it's all ok.
>>
>> The question is if Asus introduced the same bug in quite.. some
>> of their recent BIOSes, or PARAVIRT_GUEST code is somehow buggy
>> and the bug is exposed by new BIOS code.
>>     
>
> Two more Asus motherboards expose exactly the same problem.
>
> M3A-H/HDMI - AMD 780G/SB700, and
> M2N-VM - nVidia MCP67 /  GeForce 7050
>
> And from the above, 3 more:
>
> M2NPV-VM - nVidia GeForce 6150/nForce 430
> M3A78-EM - AMD780G/SB700
> M2N-SLI DELUXE - nVidia MCP55 (unconfirmed)
>
> With new BIOSes and with CONFIG_PARAVIRT enabled in kernel,
> system does not boot.  It seems the stalls happens at more
> or less random places, but the place is always the same on
> the same motherboard and different kernels -- f.e. my
> M3A78-EM always stops after "SLUB: " line as shown above,
> be it 32- or 64bits kernel, 2.6.26 or 2.6.26.5.
>   

Thanks,
    J

Re: PARAVIRT_GUEST bug? [was: Stable regression: usb-storage is stuck in 2.6.26.5]

[Michael Tokarev]

Jeremy Fitzhardinge wrote:
> Michael Tokarev wrote:
>> Michael Tokarev wrote:
>>   
>>> Michael Tokarev wrote:
>>>
>>> [usb-storage hangs on asus M2NPV-VM (nVidia GeForce 6150/nForce 430)
>>>  motherboard with latest bios]
>>>
>>>     
>>>> Ok, it turned out to be pretty.. interesting.
>>>>
>>>> It's not a regression within -stable kernel series.  The problem
>>>> happens when BOTH are true:
>>>>
>>>>   - using latest BIOS for this mobo (1401), AND
>>>>   - enabling KVM_GUEST in kernel.
>>>>
>>>> The only difference in my kernel config between .1 and .5 was
>>>> to enable KVM_GUEST and KVM_CLOCK, and also I2C_HELPER_AUTO
>>>> which was added by kernel update.
>>>>
>>>> Going with previous BIOS (1201) OR disabling PARAVIRT_GUEST
>>>> fixes the problem.  I'll try to figure out which config
>>>> option is at problem here.
>>>>       
>>> After seeing similar situation on another motherboard, I
>>> become curious.
>>>
>>> Here's the result:  M3A78-EM motherboard (also from Asus),
>>> AMD780G/SB700 etc.  Updating to one of the latest bios
>>> makes the system unbootable.  Kernel hangs on boot right
>>> after displaying
>>>
>>>  SLUB: Genslabs=12, HWalign=64, Order=0-1, MinObjects=4, CPUs=2, Nodes=1
>>>
>>> Reverting to the older BIOS or disabling PARAVIRT_GUEST in kernel
>>>     
>> PARAVIRT it is, not PARAVIRT_GUEST.
>>   
> 
> Hm, mysterious.  CONFIG_PARAVIRT should have no functional effect on a
> native boot.
> 
> Could you post the complete dmesg output (with "initcall_debug" on the
> kernel command line), with both CONFIG_PARAVIRT on and off, and ideally,
> comparing working vs non-working BIOS?

It turned out to be more difficult than I thought.  There were other
problems with the motherboard I experimented with (my new home machine),
unrelated.  Also, it wasn't possible to re-flash bios back as it said
that the old version is incompatible with currently running bios so it
can't re-flash it.

Also, initcall_debug does not make any difference for my mobo.  All the
debugging shown by this option are AFTER the place where it hangs.
And for netconsole I'd need another machine at home, which I don't
have currently.

Instead, while debugging other issues, I tried 2.6.27-rc7 kernel -- this
one works on 2 motherboards with new BIOSes, no matter which PARAVIRT
options I turn on.  I'll try this kernel on other mobos where possible,
and if no issue will be found, I'd declare the issue to be solved.

If not, I'll try to find some time to debug it later.  After all, this
very issue is already worked around -- I enabled those options in order
to add (optimized) support for KVM guests, but for those I just built
another kernel flavour with only that config change.

Oh well.

And thank you for your support!

/mjt