How to determine which device crashes udev in boot?

How to determine which device crashes udev in boot?

[iwillallways forget1]

I'm trying to narrow which device in an NEC VF-6 causes Linux boots to
hang, permanently, in udevadm trigger and udevadm settle.  Many Google
searches found many pages but no tutorials on how to debug udev this
way.

 I downloaded fresh copies of Porteus 3.1 32-bit and 64-bit.

 Porteus 3.1 64-bit boots fine, udev finds everything, x Windows works
(I told you Windows rulez), etc.  Even on this NEC VF-6.

 Porteus 3.1 32-bit boots fine on everything except this NEC VF-6, x
Windows works, etc.

 Porteus 3.1 32-bit hangs when starting udev on this NEC VF-6.  It's
not a 120-second timeout in udevadm settle.  It never wakes up.  The
Caps Lock key stops toggling the Caps Lock light.  The Num Lock key
stops toggling the Num Lock light.  Ctrl+Alt+Delete is ignored.  It
does not appear to be a kernel panic because two of the keyboard
lights don't flash.  It is hanged, frozen, dead.

 Booting with "3 nohotplug" works (3 tells Porteus what run level to
use, and nohotplug is observed by both the kernel and Porteus).  In
text mode I can do some amount of experiments.  I don't know how to do
meaningful experiments, to try to track down which device causes the
hang.

 The kernel in Porteus 3.1 32-bit is configured with EISA enabled but
not in Porteus 3.1 64-bit.  I think I could figure out how to try
udevadm trigger excluding eisa, but it still hanged.  The PC doesn't
actually contain any EISA devices, so there's an eisa.0 bus but no
actual devices, so this experiment didn't change anything.

 Obviously for myself I could just run 64-bit Linux on this PC, but
this doesn't solve the problem.  I have to provide something that
customers can run on any PC made since around 1999, and a lot of those
don't have 64-bit CPUs.  I don't know if NEC VF-6 is the only model
that has problems, but I have to be able to patch something to work
around this kind of problem when I learn about them.

 Someone found that pci=use_crs solved udev hangs on some PCs, but it
didn't help on NEC VF-6.

 (By the way, my name is Norman Diamond.  vger.kernel.org bounces mail
from my real e-mail address n0diamond atsign yahoo period co period jp
because I've been a big bad spammer, but I don't know what spams I
seent.)

Re: How to determine which device crashes udev in boot?

[Greg KH]

On Mon, Mar 14, 2016 at 01:48:39PM +0900, iwillallways forget1 wrote:
> I'm trying to narrow which device in an NEC VF-6 causes Linux boots to
> hang, permanently, in udevadm trigger and udevadm settle.  Many Google
> searches found many pages but no tutorials on how to debug udev this
> way.
> 
>  I downloaded fresh copies of Porteus 3.1 32-bit and 64-bit.

What kernel versions are these?

>  Porteus 3.1 64-bit boots fine, udev finds everything, x Windows works
> (I told you Windows rulez), etc.  Even on this NEC VF-6.
> 
>  Porteus 3.1 32-bit boots fine on everything except this NEC VF-6, x
> Windows works, etc.
> 
>  Porteus 3.1 32-bit hangs when starting udev on this NEC VF-6.  It's
> not a 120-second timeout in udevadm settle.  It never wakes up.  The
> Caps Lock key stops toggling the Caps Lock light.  The Num Lock key
> stops toggling the Num Lock light.  Ctrl+Alt+Delete is ignored.  It
> does not appear to be a kernel panic because two of the keyboard
> lights don't flash.  It is hanged, frozen, dead.
> 
>  Booting with "3 nohotplug" works (3 tells Porteus what run level to
> use, and nohotplug is observed by both the kernel and Porteus).  In
> text mode I can do some amount of experiments.  I don't know how to do
> meaningful experiments, to try to track down which device causes the
> hang.

Try looking at the documentation of udevadm, you can manually enable the
'coldplug' options there to narrow down what hardware is causing the
problem.  Odds are, you have a kernel driver that doesn't like the
hardware and it is getting loaded automatically by udev.

good luck!

greg k-h