IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Proctor, Frederick M. (Fed)]

I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
18-based x86/64 desktop system. I have these versions:

Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71

I have compiled the kernel three different ways: the stock
configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
CPU_IDLE options in all cases. The first two boot fine (stock kernel,
PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
phase.

I've tailored the kernel configuration somewhat based on the
hardware I have, but nothing has worked to get a booting IPIPE kernel.

Does anyone have guidance on setting up the IPIPE patch on an x86/64
kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
trim down the kernel config options, but before I do this, I want to
ask this group if anyone has had a similar problem.

Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Jan Kiszka]

On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
> I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
> 18-based x86/64 desktop system. I have these versions:
>
> Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71

Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.

>
> I have compiled the kernel three different ways: the stock
> configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
> PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
> CPU_IDLE options in all cases. The first two boot fine (stock kernel,
> PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
> phase.
>
> I've tailored the kernel configuration somewhat based on the
> hardware I have, but nothing has worked to get a booting IPIPE kernel.
>
> Does anyone have guidance on setting up the IPIPE patch on an x86/64
> kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
> trim down the kernel config options, but before I do this, I want to
> ask this group if anyone has had a similar problem.
>

Does yous system start to boot again when the I-pipe kernel is built without
CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare the config to a working one,
step-wise aligning to it.

Jan

RE: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Proctor, Frederick M. (Fed)]

> On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
> > I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
> > 18-based x86/64 desktop system. I have these versions:
> >
> > Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71
> 
> Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.
> 

I was using 4.14.71 since it has patches for both IPIPE and PREEMPT_RT. I had used 4.14.111 also, but had the same kernel hang problem at boot time. 

> >
> > I have compiled the kernel three different ways: the stock
> > configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
> > PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
> > CPU_IDLE options in all cases. The first two boot fine (stock kernel,
> > PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
> > phase.
> >
> > I've tailored the kernel configuration somewhat based on the hardware
> > I have, but nothing has worked to get a booting IPIPE kernel.
> >
> > Does anyone have guidance on setting up the IPIPE patch on an x86/64
> > kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
> > trim down the kernel config options, but before I do this, I want to
> > ask this group if anyone has had a similar problem.
> >
> 
> Does yous system start to boot again when the I-pipe kernel is built without
> CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare the config to a
> working one, step-wise aligning to it.
> 
> Jan

Yes it does - with the kernel patched with IPIPE and Xenomai, but with IPIPE and Xenomai *not selected* in the kernel config, the kernel boots fine. With IPIPE and Xenomai *selected* in a subsequent kernel config, that kernel hangs. Here's the diff on the config files: 

56c56
< CONFIG_LOCALVERSION="-noipipe"
---
> CONFIG_LOCALVERSION="-xeno"
118d117
< # CONFIG_NO_HZ_FULL is not set
126d124
< # CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set
272d269
< CONFIG_KPROBES_ON_FTRACE=y
321d317
< CONFIG_HAVE_CONTEXT_TRACKING=y
457a454,550
> CONFIG_XENOMAI=y
> CONFIG_XENO_ARCH_FPU=y
> CONFIG_XENO_ARCH_SYS3264=y
> 
> #
> # Core features
> #
> # CONFIG_XENO_OPT_SCHED_CLASSES is not set
> CONFIG_XENO_OPT_STATS=y
> # CONFIG_XENO_OPT_SHIRQ is not set
> CONFIG_XENO_OPT_RR_QUANTUM=1000
> CONFIG_XENO_OPT_AUTOTUNE=y
> # CONFIG_XENO_OPT_SCALABLE_SCHED is not set
> # CONFIG_XENO_OPT_TIMER_LIST is not set
> CONFIG_XENO_OPT_TIMER_RBTREE=y
> CONFIG_XENO_OPT_VFILE=y
> 
> #
> # Sizes and static limits
> #
> CONFIG_XENO_OPT_REGISTRY_NRSLOTS=512
> CONFIG_XENO_OPT_SYS_HEAPSZ=4096
> CONFIG_XENO_OPT_PRIVATE_HEAPSZ=256
> CONFIG_XENO_OPT_SHARED_HEAPSZ=256
> CONFIG_XENO_OPT_NRTIMERS=256
> 
> #
> # Latency settings
> #
> CONFIG_XENO_OPT_TIMING_SCHEDLAT=0
> CONFIG_XENO_OPT_TIMING_KSCHEDLAT=0
> CONFIG_XENO_OPT_TIMING_IRQLAT=0
> # CONFIG_XENO_OPT_DEBUG is not set
> 
> #
> # Drivers
> #
> CONFIG_XENO_OPT_RTDM_COMPAT_DEVNODE=y
> CONFIG_XENO_DRIVERS_AUTOTUNE=y
> 
> #
> # Serial drivers
> #
> # CONFIG_XENO_DRIVERS_16550A is not set
> 
> #
> # Testing drivers
> #
> CONFIG_XENO_DRIVERS_TIMERBENCH=y
> CONFIG_XENO_DRIVERS_SWITCHTEST=y
> # CONFIG_XENO_DRIVERS_RTDMTEST is not set
> 
> #
> # CAN drivers
> #
> # CONFIG_XENO_DRIVERS_CAN is not set
> 
> #
> # RTnet
> #
> # CONFIG_XENO_DRIVERS_NET is not set
> 
> #
> # ANALOGY drivers
> #
> # CONFIG_XENO_DRIVERS_ANALOGY is not set
> 
> #
> # Real-time IPC drivers
> #
> # CONFIG_XENO_DRIVERS_RTIPC is not set
> 
> #
> # UDD support
> #
> # CONFIG_XENO_DRIVERS_UDD is not set
> 
> #
> # Real-time GPIO drivers
> #
> # CONFIG_XENO_DRIVERS_GPIO is not set
> 
> #
> # Real-time SPI master drivers
> #
> 
> #
> # WARNING! Page migration (CONFIG_MIGRATION) may increase
> #
> 
> #
> # latency.
> #
> CONFIG_XENO_VERSION_MAJOR=3
> CONFIG_XENO_VERSION_MINOR=0
> CONFIG_XENO_REVISION_LEVEL=8
> CONFIG_XENO_VERSION_STRING="3.0.8"
519c612,617
< # CONFIG_IPIPE is not set
---
> CONFIG_IPIPE=y
> CONFIG_IPIPE_CORE=y
> CONFIG_IPIPE_WANT_PTE_PINNING=y
> CONFIG_IPIPE_CORE_APIREV=2
> CONFIG_IPIPE_WANT_APIREV_2=y
> CONFIG_IPIPE_TARGET_APIREV=2
689d786
< # CONFIG_LIVEPATCH is not set
8697a8795
> # CONFIG_IPIPE_DEBUG is not set
8833,8834d8930
< CONFIG_DYNAMIC_FTRACE=y
< CONFIG_DYNAMIC_FTRACE_WITH_REGS=y
8836d8931
< CONFIG_FTRACE_MCOUNT_RECORD=y

Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Jan Kiszka]

On 12.06.19 14:37, Proctor, Frederick M. (Fed) via Xenomai wrote:
>> On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
>>> I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
>>> 18-based x86/64 desktop system. I have these versions:
>>>
>>> Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71
>>
>> Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.
>>
> 
> I was using 4.14.71 since it has patches for both IPIPE and PREEMPT_RT. I had used 4.14.111 also, but had the same kernel hang problem at boot time.
> 

OK.

>>>
>>> I have compiled the kernel three different ways: the stock
>>> configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
>>> PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
>>> CPU_IDLE options in all cases. The first two boot fine (stock kernel,
>>> PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
>>> phase.
>>>
>>> I've tailored the kernel configuration somewhat based on the hardware
>>> I have, but nothing has worked to get a booting IPIPE kernel.
>>>
>>> Does anyone have guidance on setting up the IPIPE patch on an x86/64
>>> kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
>>> trim down the kernel config options, but before I do this, I want to
>>> ask this group if anyone has had a similar problem.
>>>
>>
>> Does yous system start to boot again when the I-pipe kernel is built without
>> CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare the config to a
>> working one, step-wise aligning to it.
>>
>> Jan
> 
> Yes it does - with the kernel patched with IPIPE and Xenomai, but with IPIPE and Xenomai *not selected* in the kernel config, the kernel boots fine. With IPIPE and Xenomai *selected* in a subsequent kernel config, that kernel hangs. Here's the diff on the config files:
> 

Can you share the full config? Maybe something subtly incompatible with I-pipe 
is enabled, maybe I can reproduce in KVM (ie. with a chance to debug).

Jan

-- 
-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Philippe Gerum]

On 6/12/19 2:37 PM, Proctor, Frederick M. (Fed) via Xenomai wrote:
>> On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
>>> I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
>>> 18-based x86/64 desktop system. I have these versions:
>>>
>>> Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71
>>
>> Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.
>>
> 
> I was using 4.14.71 since it has patches for both IPIPE and PREEMPT_RT. I had used 4.14.111 also, but had the same kernel hang problem at boot time. 
> 
>>>
>>> I have compiled the kernel three different ways: the stock
>>> configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
>>> PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
>>> CPU_IDLE options in all cases. The first two boot fine (stock kernel,
>>> PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
>>> phase.
>>>
>>> I've tailored the kernel configuration somewhat based on the hardware
>>> I have, but nothing has worked to get a booting IPIPE kernel.
>>>
>>> Does anyone have guidance on setting up the IPIPE patch on an x86/64
>>> kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
>>> trim down the kernel config options, but before I do this, I want to
>>> ask this group if anyone has had a similar problem.
>>>
>>
>> Does yous system start to boot again when the I-pipe kernel is built without
>> CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare the config to a
>> working one, step-wise aligning to it.
>>
>> Jan
> 
> Yes it does - with the kernel patched with IPIPE and Xenomai, but with IPIPE and Xenomai *not selected* in the kernel config, the kernel boots fine. With IPIPE and Xenomai *selected* in a subsequent kernel config, that kernel hangs. Here's the diff on the config files: 

This may be a timer issue. You could try booting with
xenomai.state=stopped on the kernel command line. If that boots as I
suspect, then you can try issuing the following command:

$ corectl --start

If that starts the core properly and no lockup, then this may have to do
with the timer grabbing sequence proper. If that hangs, then this may be
a general issue with the timing device when handling ticks for real-time
delivery.

-- 
Philippe.

RE: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Proctor, Frederick M. (Fed)]

> From: Philippe Gerum <rpm@xenomai.org>
> Sent: Wednesday, June 12, 2019 9:19 AM
> To: Proctor, Frederick M. (Fed) <frederick.proctor@nist.gov>; Jan Kiszka
> <jan.kiszka@web.de>; xenomai@xenomai.org
> Subject: Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop
> system
> 
> 
> This may be a timer issue. You could try booting with xenomai.state=stopped
> on the kernel command line. If that boots as I suspect, then you can try
> issuing the following command:
> 
> $ corectl --start
> 

I gave the kernel command line option "xenomai.state=stopped" but had the same symptom, a hang immediately on the loading of the initrd. 

> If that starts the core properly and no lockup, then this may have to do with
> the timer grabbing sequence proper. If that hangs, then this may be a
> general issue with the timing device when handling ticks for real-time
> delivery.
> 
> --
> Philippe.

Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Philippe Gerum]

On 6/12/19 3:55 PM, Proctor, Frederick M. (Fed) wrote:
>> From: Philippe Gerum <rpm@xenomai.org>
>> Sent: Wednesday, June 12, 2019 9:19 AM
>> To: Proctor, Frederick M. (Fed) <frederick.proctor@nist.gov>; Jan Kiszka
>> <jan.kiszka@web.de>; xenomai@xenomai.org
>> Subject: Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop
>> system
>>
>>
>> This may be a timer issue. You could try booting with xenomai.state=stopped
>> on the kernel command line. If that boots as I suspect, then you can try
>> issuing the following command:
>>
>> $ corectl --start
>>
> 
> I gave the kernel command line option "xenomai.state=stopped" but had the same symptom, a hang immediately on the loading of the initrd. 
> 

Ok. The output of /proc/timer_list and /proc/interrupts in the bootable
case (CONFIG_IPIPE alone) may help in debugging this.

-- 
Philippe.

Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Jan Kiszka]

On 12.06.19 15:05, Jan Kiszka via Xenomai wrote:
> On 12.06.19 14:37, Proctor, Frederick M. (Fed) via Xenomai wrote:
>>> On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
>>>> I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
>>>> 18-based x86/64 desktop system. I have these versions:
>>>>
>>>> Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71
>>>
>>> Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.
>>>
>>
>> I was using 4.14.71 since it has patches for both IPIPE and PREEMPT_RT. I had 
>> used 4.14.111 also, but had the same kernel hang problem at boot time.
>>
> 
> OK.
> 
>>>>
>>>> I have compiled the kernel three different ways: the stock
>>>> configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
>>>> PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
>>>> CPU_IDLE options in all cases. The first two boot fine (stock kernel,
>>>> PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
>>>> phase.
>>>>
>>>> I've tailored the kernel configuration somewhat based on the hardware
>>>> I have, but nothing has worked to get a booting IPIPE kernel.
>>>>
>>>> Does anyone have guidance on setting up the IPIPE patch on an x86/64
>>>> kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
>>>> trim down the kernel config options, but before I do this, I want to
>>>> ask this group if anyone has had a similar problem.
>>>>
>>>
>>> Does yous system start to boot again when the I-pipe kernel is built without
>>> CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare the config to a
>>> working one, step-wise aligning to it.
>>>
>>> Jan
>>
>> Yes it does - with the kernel patched with IPIPE and Xenomai, but with IPIPE 
>> and Xenomai *not selected* in the kernel config, the kernel boots fine. With 
>> IPIPE and Xenomai *selected* in a subsequent kernel config, that kernel hangs. 
>> Here's the diff on the config files:
>>
> 
> Can you share the full config? Maybe something subtly incompatible with I-pipe 
> is enabled, maybe I can reproduce in KVM (ie. with a chance to debug).
> 

With the configuration you sent offlist, I can confirm now that the system (in 
my case a KVM target) does not come up, rather gets stuck during secondary CPU 
boot. I'm not yet seeing the key difference, but we will find out.

I'm attaching the config I'm currently using for 4.14, maybe we can search in 
parallel for the relevant delta.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config-amd64.xz
Type: application/x-xz
Size: 22960 bytes
Desc: not available
URL: <http://xenomai.org/pipermail/xenomai/attachments/20190613/277c1b9e/attachment.bin>

Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Jan Kiszka]

On 13.06.19 18:36, Jan Kiszka via Xenomai wrote:
> On 12.06.19 15:05, Jan Kiszka via Xenomai wrote:
>> On 12.06.19 14:37, Proctor, Frederick M. (Fed) via Xenomai wrote:
>>>> On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
>>>>> I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
>>>>> 18-based x86/64 desktop system. I have these versions:
>>>>>
>>>>> Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71
>>>>
>>>> Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.
>>>>
>>>
>>> I was using 4.14.71 since it has patches for both IPIPE and PREEMPT_RT. I had 
>>> used 4.14.111 also, but had the same kernel hang problem at boot time.
>>>
>>
>> OK.
>>
>>>>>
>>>>> I have compiled the kernel three different ways: the stock
>>>>> configuration, with the PREEMPT_RT patch, and with the IPIPE patch (no
>>>>> PREEMPT_RT). I have configured out the ACPI processor, CPU_FREQ, and
>>>>> CPU_IDLE options in all cases. The first two boot fine (stock kernel,
>>>>> PREEMPT_RT). The IPIPE-patched kernel hangs during the initrd loading
>>>>> phase.
>>>>>
>>>>> I've tailored the kernel configuration somewhat based on the hardware
>>>>> I have, but nothing has worked to get a booting IPIPE kernel.
>>>>>
>>>>> Does anyone have guidance on setting up the IPIPE patch on an x86/64
>>>>> kernel on a desktop Ubuntu 18 machine? I am planning on continuing to
>>>>> trim down the kernel config options, but before I do this, I want to
>>>>> ask this group if anyone has had a similar problem.
>>>>>
>>>>
>>>> Does yous system start to boot again when the I-pipe kernel is built without
>>>> CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare the config to a
>>>> working one, step-wise aligning to it.
>>>>
>>>> Jan
>>>
>>> Yes it does - with the kernel patched with IPIPE and Xenomai, but with IPIPE 
>>> and Xenomai *not selected* in the kernel config, the kernel boots fine. With 
>>> IPIPE and Xenomai *selected* in a subsequent kernel config, that kernel 
>>> hangs. Here's the diff on the config files:
>>>
>>
>> Can you share the full config? Maybe something subtly incompatible with I-pipe 
>> is enabled, maybe I can reproduce in KVM (ie. with a chance to debug).
>>
> 
> With the configuration you sent offlist, I can confirm now that the system (in 
> my case a KVM target) does not come up, rather gets stuck during secondary CPU 
> boot. I'm not yet seeing the key difference, but we will find out.
> 
> I'm attaching the config I'm currently using for 4.14, maybe we can search in 
> parallel for the relevant delta.
> 

Found: CONFIG_MAXSMP. Needs to be off, NR_CPUS at 64 eg.

I need to check where our exact limit is, but this should likely be caught at 
kconfig level.

At the same time, you will likely want to tune down your config for your needs. 
CONFIG_MIGRATION (with CMA, transparent hugh pages etc.) is one thing that you 
do not want in an RT kernel, but there is likely more ballast.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

RE: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop system

[Proctor, Frederick M. (Fed)]

> From: Jan Kiszka <jan.kiszka@siemens.com>
> Sent: Thursday, June 13, 2019 3:12 PM
> To: Proctor, Frederick M. (Fed) <frederick.proctor@nist.gov>;
> xenomai@xenomai.org
> Subject: Re: IPIPE patch causes hang during initrd boot; Ubuntu 18 desktop
> system
> 
> On 13.06.19 18:36, Jan Kiszka via Xenomai wrote:
> > On 12.06.19 15:05, Jan Kiszka via Xenomai wrote:
> >> On 12.06.19 14:37, Proctor, Frederick M. (Fed) via Xenomai wrote:
> >>>> On 07.06.19 15:02, Proctor, Frederick M. (Fed) via Xenomai wrote:
> >>>>> I am installing Xenomai + Cobalt with the IPIPE patch on a Ubuntu
> >>>>> 18-based x86/64 desktop system. I have these versions:
> >>>>>
> >>>>> Linux kernel 4.14.71 PREEMPT_RT patch 4.14.71-rt44 IPIPE 4.14.71
> >>>>
> >>>> Latest stable I-pipe version is ipipe-core-4.14.111-x86-3.
> >>>>
> >>>
> >>> I was using 4.14.71 since it has patches for both IPIPE and
> >>> PREEMPT_RT. I had used 4.14.111 also, but had the same kernel hang
> problem at boot time.
> >>>
> >>
> >> OK.
> >>
> >>>>>
> >>>>> I have compiled the kernel three different ways: the stock
> >>>>> configuration, with the PREEMPT_RT patch, and with the IPIPE patch
> >>>>> (no PREEMPT_RT). I have configured out the ACPI processor,
> >>>>> CPU_FREQ, and CPU_IDLE options in all cases. The first two boot
> >>>>> fine (stock kernel, PREEMPT_RT). The IPIPE-patched kernel hangs
> >>>>> during the initrd loading phase.
> >>>>>
> >>>>> I've tailored the kernel configuration somewhat based on the
> >>>>> hardware I have, but nothing has worked to get a booting IPIPE
> kernel.
> >>>>>
> >>>>> Does anyone have guidance on setting up the IPIPE patch on an
> >>>>> x86/64 kernel on a desktop Ubuntu 18 machine? I am planning on
> >>>>> continuing to trim down the kernel config options, but before I do
> >>>>> this, I want to ask this group if anyone has had a similar problem.
> >>>>>
> >>>>
> >>>> Does yous system start to boot again when the I-pipe kernel is
> >>>> built without CONFIG_IPIPE and CONFIG_XENOMAI? If no, compare
> the
> >>>> config to a working one, step-wise aligning to it.
> >>>>
> >>>> Jan
> >>>
> >>> Yes it does - with the kernel patched with IPIPE and Xenomai, but
> >>> with IPIPE and Xenomai *not selected* in the kernel config, the
> >>> kernel boots fine. With IPIPE and Xenomai *selected* in a subsequent
> >>> kernel config, that kernel hangs. Here's the diff on the config files:
> >>>
> >>
> >> Can you share the full config? Maybe something subtly incompatible
> >> with I-pipe is enabled, maybe I can reproduce in KVM (ie. with a chance to
> debug).
> >>
> >
> > With the configuration you sent offlist, I can confirm now that the
> > system (in my case a KVM target) does not come up, rather gets stuck
> > during secondary CPU boot. I'm not yet seeing the key difference, but we
> will find out.
> >
> > I'm attaching the config I'm currently using for 4.14, maybe we can
> > search in parallel for the relevant delta.
> >
> 
> Found: CONFIG_MAXSMP. Needs to be off, NR_CPUS at 64 eg.
> 
> I need to check where our exact limit is, but this should likely be caught at
> kconfig level.
> 
> At the same time, you will likely want to tune down your config for your
> needs.
> CONFIG_MIGRATION (with CMA, transparent hugh pages etc.) is one thing
> that you do not want in an RT kernel, but there is likely more ballast.
> 
> Jan

Thanks - this worked, I disabled CONFIG_MAXSMP, rebuilt and it booted fine. I'll go in and tune down things once I run the Xenomai tests. 

--Fred