Regression since 4.1

Regression since 4.1

[Tobias Geiger]

Hello!

in short:
No Kernel since 4.1 boots on my machine.

I can make newer kernels boot, but ONLY when i remove "iommu=on 
intel_iommu=on" from the cmdline.
If i leave it in, as with all the kernels before and including 4.1, newer 
kernels dont boot, and in a really not very informative way:
They dont even print a single message to the console, i only see the grub-
messages (like "loading kernel" and "loading initial ramdisk") - after these 
messages NO more output of any kind - also no SysRQ keys working - nothing.

Because of this complete lack of information, i guess its hard to debug for 
you the cause of this.
Perhaps i can help - but how?

i'm writing from a pure IOMMU-User perspective - i can compile kernels, but 
never git-bisected, so maybe a hint to a tutorial for that would be great!

Greetings and thanks for your insight!
Tobias

Re: Regression since 4.1

[Alex Williamson]

On Wed, 2015-12-09 at 18:01 +0100, Tobias Geiger wrote:
> Hello!
> 
> in short:
> No Kernel since 4.1 boots on my machine.
> 
> I can make newer kernels boot, but ONLY when i remove "iommu=on 
> intel_iommu=on" from the cmdline.
> If i leave it in, as with all the kernels before and including 4.1, newer 
> kernels dont boot, and in a really not very informative way:
> They dont even print a single message to the console, i only see the grub-
> messages (like "loading kernel" and "loading initial ramdisk") - after these 
> messages NO more output of any kind - also no SysRQ keys working - nothing.
> 
> Because of this complete lack of information, i guess its hard to debug for 
> you the cause of this.
> Perhaps i can help - but how?
> 
> i'm writing from a pure IOMMU-User perspective - i can compile kernels, but 
> never git-bisected, so maybe a hint to a tutorial for that would be great!


Try simply removing the "quiet" and "splash" or "rhgb" options if there
are any from the kernel command line in grub first.  You can temporarily
just editing the commandline manually as you boot.  If you don't have a
serial console available, at least have a camera handy to take a picture
of the screen.  You can also add "ignore_loglevel" to the kernel
commandline to get more logging.  Thanks,

Alex

Re: Regression since 4.1

[Tobias Geiger]

Hello Alex,

thanks for looking into this.

Nothing helps - not adding "ignore_loglevel" , not removing everything but the 
most necessary (="BOOT_IMAGE=/boot/vmlinuz-4.3.0-trunk-amd64 ro 
root=LABEL=dom0root  panic=15 iommu=on intel_iommu=on ignore_loglevel") - grub 
loads the kernel, loads initrd, execs the kernel - and then nothing. no output 
of any kind from the kernel. no sysrq, no panic-reboot despite panic=15 ... 
nothing. not even a blinking cursor, just a steady cursor which is still from 
grub i guess...

It all comes down to kernel > 4.1 && iommu=on = not working here

Hardware is:
Intel DX58SO, Intel CPU i7 920... all pretty solid running for years now, 
always with iommo=on.

I'd love to debug this further, just dont know how...

Greetings!
Tobias

Am Mittwoch, 9. Dezember 2015, 12:07:48 schrieb Alex Williamson:
> Try simply removing the "quiet" and "splash" or "rhgb" options if there
> are any from the kernel command line in grub first.  You can temporarily
> just editing the commandline manually as you boot.  If you don't have a
> serial console available, at least have a camera handy to take a picture
> of the screen.  You can also add "ignore_loglevel" to the kernel
> commandline to get more logging.  Thanks,
> 
> Alex

Re: Regression since 4.1

[Joerg Roedel]

Hi Tobias,

On Thu, Dec 10, 2015 at 12:07:57AM +0100, Tobias Geiger wrote:
> It all comes down to kernel > 4.1 && iommu=on = not working here
> 
> Hardware is:
> Intel DX58SO, Intel CPU i7 920... all pretty solid running for years now, 
> always with iommo=on.
> 
> I'd love to debug this further, just dont know how...

Sounds like only bisecting the issue will help here. You can find an
introduction on bisecting here:

	https://git-scm.com/book/en/v2/Git-Tools-Debugging-with-Git

Would be great if you can find the time to bisect the issue.



	Joerg

Re: Regression since 4.1

[Tom Wimmenhove]

Joerg Roedel <joro@...> writes:

> 
> Hi Tobias,
> 
> On Thu, Dec 10, 2015 at 12:07:57AM +0100, Tobias Geiger wrote:
> > It all comes down to kernel > 4.1 && iommu=on = not working here
> > 
> > Hardware is:
> > Intel DX58SO, Intel CPU i7 920... all pretty solid running for years 
now, 
> > always with iommo=on.
> > 
> > I'd love to debug this further, just dont know how...
> 
> Sounds like only bisecting the issue will help here. You can find an
> introduction on bisecting here:
> 
> 	https://git-scm.com/book/en/v2/Git-Tools-Debugging-with-Git
> 
> Would be great if you can find the time to bisect the issue.
> 
> 	Joerg
> 
> 

I can confirm this bug with kernel "4.2.0-16-generic #19-Ubuntu" on an 
Intel DX58SO board with a W3520 Xeon CPU.
Before I was running 3.16.0 which did not lock up at boot but did have 
occasional DMAR issues with GPU passthrough while running a Windows 7 gues 
in Qemu. I don't know if the two are related.