Re: next/master boot bisection: next-20190430 on beagle-xm

Re: next/master boot bisection: next-20190430 on beagle-xm

[Tony Lindgren]

Hi,

* Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 15:37]:
> 
> On 2019-04-30 13:51:40 [-0700], kernelci.org bot wrote:
> > next/master boot bisection: next-20190430 on beagle-xm
> > 
> > Summary:
> >   Start:      f43b05fd4c17 Add linux-next specific files for 20190430
> >   Details:    https://kernelci.org/boot/id/5cc84d7359b514b7ab55847b
> >   Plain log:  https://storage.kernelci.org//next/master/next-20190430/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-7/lab-baylibre/boot-omap3-beagle-xm.txt
> >   HTML log:   https://storage.kernelci.org//next/master/next-20190430/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-7/lab-baylibre/boot-omap3-beagle-xm.html
> >   Result:     6d25be5782e4 sched/core, workqueues: Distangle worker accounting from rq lock
> > 
> > Checks:
> >   revert:     PASS
> >   verify:     PASS
> > 
> > Parameters:
> >   Tree:       next
> >   URL:        git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> >   Branch:     master
> >   Target:     beagle-xm
> >   CPU arch:   arm
> >   Lab:        lab-baylibre
> >   Compiler:   gcc-7
> >   Config:     multi_v7_defconfig+CONFIG_SMP=n
> >   Test suite: boot
> > 
> > Breaking commit found:
> > 
> > -------------------------------------------------------------------------------
> > commit 6d25be5782e482eb93e3de0c94d0a517879377d0
> > Author: Thomas Gleixner <tglx@linutronix.de>
> > Date:   Wed Mar 13 17:55:48 2019 +0100
> > 
> >     sched/core, workqueues: Distangle worker accounting from rq lock
> 
> According to the bootlog it just stopped its output. This commit is in
> next since a week or two so I don't understand why this pops up now.

Adding Kevin to Cc, he just confirmed on #armlinux irc that he is able to
reproduce this with CONFIG_SMP=n and root=/dev/ram0. I could not reproduce
this issue so far on omap3 with NFSroot at least.

> I just revived my BBB and I can boot that commit in question. Currently
> that as close as I get to a beagle-xm. 
> Looking at
> 	https://kernelci.org/boot/id/5cc9a64359b514a77f5584af/
> it seems that the very same board managed to boot linux-next for
> next-20190501.
> 
> Side note: I can't boot next-20190501 on my BBB, bisect points to commit
>   1a5cd7c23cc52 ("bus: ti-sysc: Enable all clocks directly during init to read revision")
> 
> any idea?

Oh interesting thanks for letting me know. Next boots fine for me here
with NFSroot on BBB.

Do you have some output on what happens so I can investigate?

Regards,

Tony

Re: next/master boot bisection: next-20190430 on beagle-xm

[Sebastian Andrzej Siewior]

On 2019-05-01 09:29:44 [-0700], Tony Lindgren wrote:
> Hi,
> 
> * Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 15:37]:
> > 
> > On 2019-04-30 13:51:40 [-0700], kernelci.org bot wrote:
> > > next/master boot bisection: next-20190430 on beagle-xm
> > > 
> > > Summary:
> > >   Start:      f43b05fd4c17 Add linux-next specific files for 20190430
> > >   Details:    https://kernelci.org/boot/id/5cc84d7359b514b7ab55847b
> > >   Plain log:  https://storage.kernelci.org//next/master/next-20190430/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-7/lab-baylibre/boot-omap3-beagle-xm.txt
> > >   HTML log:   https://storage.kernelci.org//next/master/next-20190430/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-7/lab-baylibre/boot-omap3-beagle-xm.html
> > >   Result:     6d25be5782e4 sched/core, workqueues: Distangle worker accounting from rq lock
> > > 
> > > Checks:
> > >   revert:     PASS
> > >   verify:     PASS
> > > 
> > > Parameters:
> > >   Tree:       next
> > >   URL:        git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > >   Branch:     master
> > >   Target:     beagle-xm
> > >   CPU arch:   arm
> > >   Lab:        lab-baylibre
> > >   Compiler:   gcc-7
> > >   Config:     multi_v7_defconfig+CONFIG_SMP=n
> > >   Test suite: boot
> > > 
> > > Breaking commit found:
> > > 
> > > -------------------------------------------------------------------------------
> > > commit 6d25be5782e482eb93e3de0c94d0a517879377d0
> > > Author: Thomas Gleixner <tglx@linutronix.de>
> > > Date:   Wed Mar 13 17:55:48 2019 +0100
> > > 
> > >     sched/core, workqueues: Distangle worker accounting from rq lock
> > 
> > According to the bootlog it just stopped its output. This commit is in
> > next since a week or two so I don't understand why this pops up now.
> 
> Adding Kevin to Cc, he just confirmed on #armlinux irc that he is able to
> reproduce this with CONFIG_SMP=n and root=/dev/ram0. I could not reproduce
> this issue so far on omap3 with NFSroot at least.

So that problem remains even that the job for today passed?
 
> > I just revived my BBB and I can boot that commit in question. Currently
> > that as close as I get to a beagle-xm. 
> > Looking at
> > 	https://kernelci.org/boot/id/5cc9a64359b514a77f5584af/
> > it seems that the very same board managed to boot linux-next for
> > next-20190501.
> > 
> > Side note: I can't boot next-20190501 on my BBB, bisect points to commit
> >   1a5cd7c23cc52 ("bus: ti-sysc: Enable all clocks directly during init to read revision")
> > 
> > any idea?
> 
> Oh interesting thanks for letting me know. Next boots fine for me here
> with NFSroot on BBB.
> 
> Do you have some output on what happens so I can investigate?

Nope, the console remains dark.

> Regards,
> 
> Tony

Sebastian

Re: next/master boot bisection: next-20190430 on beagle-xm

[Tony Lindgren]

* Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 16:45]:
> On 2019-05-01 09:29:44 [-0700], Tony Lindgren wrote:
> > Hi,
> > 
> > * Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 15:37]:
> > > 
> > > On 2019-04-30 13:51:40 [-0700], kernelci.org bot wrote:
> > > > next/master boot bisection: next-20190430 on beagle-xm
> > > > 
> > > > Summary:
> > > >   Start:      f43b05fd4c17 Add linux-next specific files for 20190430
> > > >   Details:    https://kernelci.org/boot/id/5cc84d7359b514b7ab55847b
> > > >   Plain log:  https://storage.kernelci.org//next/master/next-20190430/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-7/lab-baylibre/boot-omap3-beagle-xm.txt
> > > >   HTML log:   https://storage.kernelci.org//next/master/next-20190430/arm/multi_v7_defconfig+CONFIG_SMP=n/gcc-7/lab-baylibre/boot-omap3-beagle-xm.html
> > > >   Result:     6d25be5782e4 sched/core, workqueues: Distangle worker accounting from rq lock
> > > > 
> > > > Checks:
> > > >   revert:     PASS
> > > >   verify:     PASS
> > > > 
> > > > Parameters:
> > > >   Tree:       next
> > > >   URL:        git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
> > > >   Branch:     master
> > > >   Target:     beagle-xm
> > > >   CPU arch:   arm
> > > >   Lab:        lab-baylibre
> > > >   Compiler:   gcc-7
> > > >   Config:     multi_v7_defconfig+CONFIG_SMP=n
> > > >   Test suite: boot
> > > > 
> > > > Breaking commit found:
> > > > 
> > > > -------------------------------------------------------------------------------
> > > > commit 6d25be5782e482eb93e3de0c94d0a517879377d0
> > > > Author: Thomas Gleixner <tglx@linutronix.de>
> > > > Date:   Wed Mar 13 17:55:48 2019 +0100
> > > > 
> > > >     sched/core, workqueues: Distangle worker accounting from rq lock
> > > 
> > > According to the bootlog it just stopped its output. This commit is in
> > > next since a week or two so I don't understand why this pops up now.
> > 
> > Adding Kevin to Cc, he just confirmed on #armlinux irc that he is able to
> > reproduce this with CONFIG_SMP=n and root=/dev/ram0. I could not reproduce
> > this issue so far on omap3 with NFSroot at least.
> 
> So that problem remains even that the job for today passed?

So it seems, let's wait and see what Kevin comes up with.

> > > I just revived my BBB and I can boot that commit in question. Currently
> > > that as close as I get to a beagle-xm. 
> > > Looking at
> > > 	https://kernelci.org/boot/id/5cc9a64359b514a77f5584af/
> > > it seems that the very same board managed to boot linux-next for
> > > next-20190501.
> > > 
> > > Side note: I can't boot next-20190501 on my BBB, bisect points to commit
> > >   1a5cd7c23cc52 ("bus: ti-sysc: Enable all clocks directly during init to read revision")
> > > 
> > > any idea?
> > 
> > Oh interesting thanks for letting me know. Next boots fine for me here
> > with NFSroot on BBB.
> > 
> > Do you have some output on what happens so I can investigate?
> 
> Nope, the console remains dark.

OK. Can you please email me your .config and the kernel cmdline you're
using? I'll try to reproduce that one here.

Regards,

Tony

Re: next/master boot bisection: next-20190430 on beagle-xm

[Sebastian Andrzej Siewior]

On 2019-05-01 09:52:24 [-0700], Tony Lindgren wrote:
> > > Oh interesting thanks for letting me know. Next boots fine for me here
> > > with NFSroot on BBB.
> > > 
> > > Do you have some output on what happens so I can investigate?
> > 
> > Nope, the console remains dark.
> 
> OK. Can you please email me your .config and the kernel cmdline you're
> using? I'll try to reproduce that one here.

This is "multi_v7_defconfig+CONFIG_SMP=n" and my earlyprintk vanished.
So with this added:
|[    0.000000] Booting Linux on physical CPU 0x0
|[    0.000000] Linux version 5.1.0-rc7-next-20190501 (bigeasy@flow) (gcc version 8.3.0 (Debian 8.3.0-7)) #29 Wed May 1 18:55:24 CEST 2019
|[    0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
|[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
|[    0.000000] OF: fdt: Machine model: TI AM335x BeagleBone Black
|[    0.000000] printk: bootconsole [earlycon0] enabled
|[    0.000000] Memory policy: Data cache writeback
|[    0.000000] efi: Getting EFI parameters from FDT:
|[    0.000000] efi: UEFI not found.
|[    0.000000] cma: Reserved 64 MiB at 0x9b800000
|[    0.000000] CPU: All CPU(s) started in SVC mode.
|[    0.000000] AM335X ES2.0 (neon)
|[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 129540
|[    0.000000] Kernel command line: console=ttyO0,115200n8 root=/dev/mmcblk1p2 rootwait coherent_pool=1M net.ifnames=0 earlyprintk
|[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
|[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
|[    0.000000] Memory: 430304K/522240K available (11264K kernel code, 1659K rwdata, 4924K rodata, 2048K init, 383K bss, 26400K reserved, 65536K cma-reserved, 0K highmem)
|[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
|[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
|[    0.000000] IRQ: Found an INTC at 0x(ptrval) (revision 5.0) with 128 interrupts
|[    0.000000] random: get_random_bytes called from start_kernel+0x2ec/0x478 with crng_init=0
|[    0.000000] OMAP clockevent source: timer2 at 24000000 Hz
|[    0.000015] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
|[    0.008050] clocksource: timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635851949 ns
|[    0.017499] OMAP clocksource: timer1 at 24000000 Hz
|[    0.028449] timer_probe: no matching timers found
|[    0.033509] Console: colour dummy device 80x30
|[    0.038089] WARNING: Your 'console=ttyO0' has been replaced by 'ttyS0'
|[    0.044781] This ensures that you still see kernel messages. Please
|[    0.051197] update your kernel commandline.
|[    0.055513] Calibrating delay loop... 996.14 BogoMIPS (lpj=4980736)
|[    0.100098] pid_max: default: 32768 minimum: 301
|[    0.104992] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
|[    0.111774] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
|[    0.119499] *** VALIDATE proc ***
|[    0.123004] *** VALIDATE cgroup1 ***
|[    0.126693] *** VALIDATE cgroup2 ***
|[    0.130358] CPU: Testing write buffer coherency: ok
|[    0.135408] CPU0: Spectre v2: using BPIALL workaround
|[    0.141140] Setting up static identity map for 0x80300000 - 0x803000ac
|[    0.153146] EFI services will not be available.
|[    0.159067] devtmpfs: initialized
|[    0.171139] VFP support v0.3: implementor 41 architecture 3 part 30 variant c rev 3
|[    0.179275] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
|[    0.189383] futex hash table entries: 256 (order: -1, 3072 bytes)
|[    0.199562] pinctrl core: initialized pinctrl subsystem
|[    0.207278] DMI not present or invalid.
|[    0.211649] NET: Registered protocol family 16
|[    0.219070] DMA: preallocated 1024 KiB pool for atomic coherent allocations
|[    0.244016] l3-aon-clkctrl:0000:0: failed to disable
|[    0.277247] cpuidle: using governor menu
|[    0.302433] No ATAGs?
|[    0.302443] hw-breakpoint: debug architecture 0x4 unsupported.
|[    0.311790] omap4_sram_init:Unable to allocate sram needed to handle errata I688
|[    0.319401] omap4_sram_init:Unable to get sram pool needed to handle errata I688
|[    0.332438] Serial: AMBA PL011 UART driver
|[    0.355726] edma 49000000.edma: TI EDMA DMA engine driver
|[    0.362309] AT91: Could not find identification node
|[    0.365980] vgaarb: loaded
|[    0.374698] SCSI subsystem initialized
|[    0.379005] usbcore: registered new interface driver usbfs
|[    0.384723] usbcore: registered new interface driver hub
|[    0.390205] usbcore: registered new device driver usb
|[    0.396826] pps_core: LinuxPPS API ver. 1 registered
|[    0.401912] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
|[    0.411320] PTP clock support registered
|[    0.415536] EDAC MC: Ver: 3.0.0
|[    0.421533] clocksource: Switched to clocksource timer1
|[    0.823224] NET: Registered protocol family 2
|[    0.828261] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes)
|[    0.836192] TCP established hash table entries: 4096 (order: 2, 16384 bytes)
|[    0.843460] TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
|[    0.850087] TCP: Hash tables configured (established 4096 bind 4096)
|[    0.856698] UDP hash table entries: 256 (order: 0, 4096 bytes)
|[    0.862700] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
|[    0.869257] NET: Registered protocol family 1
|[    0.874257] RPC: Registered named UNIX socket transport module.
|[    0.880325] RPC: Registered udp transport module.
|[    0.885174] RPC: Registered tcp transport module.
|[    0.889989] RPC: Registered tcp NFSv4.1 backchannel transport module.
|[    0.897690] hw perfevents: enabled with armv7_cortex_a8 PMU driver, 5 counters available
|[    0.907566] Initialise system trusted keyrings
|[    0.912417] workingset: timestamp_bits=30 max_order=17 bucket_order=0
|[    0.924685] squashfs: version 4.0 (2009/01/31) Phillip Lougher
|[    0.931401] NFS: Registering the id_resolver key type
|[    0.936677] Key type id_resolver registered
|[    0.940961] Key type id_legacy registered
|[    0.945116] ntfs: driver 2.1.32 [Flags: R/O].
|[    0.950130] Key type asymmetric registered
|[    0.954380] Asymmetric key parser 'x509' registered
|[    0.959421] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
|[    0.967018] io scheduler mq-deadline registered
|[    0.971667] io scheduler kyber registered
|[    0.983052] OMAP GPIO hardware version 0.1
|[    1.010470] ti-sysc 48042000.target-module: sysc_flags 00000222 != 00000022
|[    1.018713] ti-sysc 48044000.target-module: sysc_flags 00000222 != 00000022
|[    1.026947] ti-sysc 48046000.target-module: sysc_flags 00000222 != 00000022
|[    1.035182] ti-sysc 48048000.target-module: sysc_flags 00000222 != 00000022
|[    1.043393] ti-sysc 4804a000.target-module: sysc_flags 00000222 != 00000022
|[    1.073661] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa1cc000
|[    1.081518] pgd = (ptrval)
|[    1.084285] [fa1cc000] *pgd=48011452(bad)
|[    1.088398] Internal error: : 1028 [#1] ARM
|[    1.092681] Modules linked in:
|[    1.095814] CPU: 0 PID: 1 Comm: swapper Not tainted 5.1.0-rc7-next-20190501 #29
|[    1.103300] Hardware name: Generic AM33XX (Flattened Device Tree)
|[    1.109560] PC is at sysc_probe+0x958/0x10a4
|[    1.113932] LR is at sysc_probe+0x928/0x10a4
|[    1.118302] pc : [<c0644e38>]    lr : [<c0644e08>]    psr: 60000013
|[    1.124720] sp : db0b1db8  ip : 00000013  fp : c162ac60
|[    1.130069] r10: 00000000  r9 : 00000028  r8 : 00000001
|[    1.135418] r7 : 00000000  r6 : db191210  r5 : c1604048  r4 : db345940
|[    1.142103] r3 : fa1cc000  r2 : 00000000  r1 : 00000000  r0 : 00000000
|[    1.148793] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
|[    1.156103] Control: 10c5387d  Table: 80204019  DAC: 00000051
|[    1.161987] Process swapper (pid: 1, stack limit = 0x(ptrval))
|…
|[    1.331712] [<c0644e38>] (sysc_probe) from [<c08f027c>] (platform_drv_probe+0x48/0x98)
|[    1.339831] [<c08f027c>] (platform_drv_probe) from [<c08ee41c>] (really_probe+0xf0/0x2c8)
|[    1.348216] [<c08ee41c>] (really_probe) from [<c08ee754>] (driver_probe_device+0x60/0x16c)
|[    1.356688] [<c08ee754>] (driver_probe_device) from [<c08eea00>] (device_driver_attach+0x58/0x60)
|[    1.365782] [<c08eea00>] (device_driver_attach) from [<c08eea60>] (__driver_attach+0x58/0xcc)
|[    1.374521] [<c08eea60>] (__driver_attach) from [<c08ec8d8>] (bus_for_each_dev+0x74/0xb4)
|[    1.382903] [<c08ec8d8>] (bus_for_each_dev) from [<c08ed944>] (bus_add_driver+0x1b8/0x1d8)
|[    1.391374] [<c08ed944>] (bus_add_driver) from [<c08ef394>] (driver_register+0x74/0x108)
|[    1.399672] [<c08ef394>] (driver_register) from [<c0302d88>] (do_one_initcall+0x50/0x1a4)
|[    1.408064] [<c0302d88>] (do_one_initcall) from [<c1401064>] (kernel_init_freeable+0x1c4/0x25c)
|[    1.416989] [<c1401064>] (kernel_init_freeable) from [<c0dedba4>] (kernel_init+0x8/0x10c)
|[    1.425373] [<c0dedba4>] (kernel_init) from [<c03010e8>] (ret_from_fork+0x14/0x2c)
|[    1.433127] Exception stack(0xdb0b1fb0 to 0xdb0b1ff8)
|[    1.438301] 1fa0:                                     00000000 00000000 00000000 00000000
|[    1.446683] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
|[    1.455063] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000
|[    1.461845] Code: e3130004 1a000126 e5943014 e0833001 (e5930000) 
|[    1.468105] ---[ end trace 5481d6c45bd9fae0 ]---
|[    1.472934] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
|[    1.480784] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---

with  arm-linux-gnueabihf-addr2line -i c0644e38 -e vmlinux
| arch/arm/include/asm/io.h:117
| drivers/bus/ti-sysc.c:117
| drivers/bus/ti-sysc.c:132
| drivers/bus/ti-sysc.c:1361
| drivers/bus/ti-sysc.c:2117

Does this help?

> Regards,
> 
> Tony

Sebastian

Re: next/master boot bisection: next-20190430 on beagle-xm

[Tony Lindgren]

* Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 17:01]:
> On 2019-05-01 09:52:24 [-0700], Tony Lindgren wrote:
> > > > Oh interesting thanks for letting me know. Next boots fine for me here
> > > > with NFSroot on BBB.
> > > > 
> > > > Do you have some output on what happens so I can investigate?
> > > 
> > > Nope, the console remains dark.
> > 
> > OK. Can you please email me your .config and the kernel cmdline you're
> > using? I'll try to reproduce that one here.
> 
> This is "multi_v7_defconfig+CONFIG_SMP=n" and my earlyprintk vanished.
> So with this added:
> |[    0.000000] Booting Linux on physical CPU 0x0
> |[    0.000000] Linux version 5.1.0-rc7-next-20190501 (bigeasy@flow) (gcc version 8.3.0 (Debian 8.3.0-7)) #29 Wed May 1 18:55:24 CEST 2019
> |[    0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=10c5387d
> |[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
> |[    0.000000] OF: fdt: Machine model: TI AM335x BeagleBone Black
> |[    0.000000] printk: bootconsole [earlycon0] enabled
> |[    0.000000] Memory policy: Data cache writeback
> |[    0.000000] efi: Getting EFI parameters from FDT:
> |[    0.000000] efi: UEFI not found.
> |[    0.000000] cma: Reserved 64 MiB at 0x9b800000
> |[    0.000000] CPU: All CPU(s) started in SVC mode.
> |[    0.000000] AM335X ES2.0 (neon)
> |[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 129540
> |[    0.000000] Kernel command line: console=ttyO0,115200n8 root=/dev/mmcblk1p2 rootwait coherent_pool=1M net.ifnames=0 earlyprintk

Hmm so I tried without "earlycon" in command line thinking it might be
happening with just "earlyprintk" but still no luck.

BTW, in general you might want to update your kernel command line
options to:

debug earlyprintk earlycon

As that way you get early output without CONFIG_DEBUG_LL=y
with earlycon that should be enabled by default already.

> |[    1.073661] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa1cc000
> |[    1.081518] pgd = (ptrval)
> |[    1.084285] [fa1cc000] *pgd=48011452(bad)
> |[    1.088398] Internal error: : 1028 [#1] ARM
> |[    1.092681] Modules linked in:
> |[    1.095814] CPU: 0 PID: 1 Comm: swapper Not tainted 5.1.0-rc7-next-20190501 #29
> |[    1.103300] Hardware name: Generic AM33XX (Flattened Device Tree)
> |[    1.109560] PC is at sysc_probe+0x958/0x10a4
> |[    1.113932] LR is at sysc_probe+0x928/0x10a4
> |[    1.118302] pc : [<c0644e38>]    lr : [<c0644e08>]    psr: 60000013
> |[    1.124720] sp : db0b1db8  ip : 00000013  fp : c162ac60
> |[    1.130069] r10: 00000000  r9 : 00000028  r8 : 00000001
> |[    1.135418] r7 : 00000000  r6 : db191210  r5 : c1604048  r4 : db345940
> |[    1.142103] r3 : fa1cc000  r2 : 00000000  r1 : 00000000  r0 : 00000000
> |[    1.148793] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> |[    1.156103] Control: 10c5387d  Table: 80204019  DAC: 00000051
> |[    1.161987] Process swapper (pid: 1, stack limit = 0x(ptrval))
> |…
> |[    1.331712] [<c0644e38>] (sysc_probe) from [<c08f027c>] (platform_drv_probe+0x48/0x98)
> |[    1.339831] [<c08f027c>] (platform_drv_probe) from [<c08ee41c>] (really_probe+0xf0/0x2c8)
> |[    1.348216] [<c08ee41c>] (really_probe) from [<c08ee754>] (driver_probe_device+0x60/0x16c)
> |[    1.356688] [<c08ee754>] (driver_probe_device) from [<c08eea00>] (device_driver_attach+0x58/0x60)
> |[    1.365782] [<c08eea00>] (device_driver_attach) from [<c08eea60>] (__driver_attach+0x58/0xcc)
> |[    1.374521] [<c08eea60>] (__driver_attach) from [<c08ec8d8>] (bus_for_each_dev+0x74/0xb4)
> |[    1.382903] [<c08ec8d8>] (bus_for_each_dev) from [<c08ed944>] (bus_add_driver+0x1b8/0x1d8)
> |[    1.391374] [<c08ed944>] (bus_add_driver) from [<c08ef394>] (driver_register+0x74/0x108)
> |[    1.399672] [<c08ef394>] (driver_register) from [<c0302d88>] (do_one_initcall+0x50/0x1a4)
> |[    1.408064] [<c0302d88>] (do_one_initcall) from [<c1401064>] (kernel_init_freeable+0x1c4/0x25c)
> |[    1.416989] [<c1401064>] (kernel_init_freeable) from [<c0dedba4>] (kernel_init+0x8/0x10c)
> |[    1.425373] [<c0dedba4>] (kernel_init) from [<c03010e8>] (ret_from_fork+0x14/0x2c)
> |[    1.433127] Exception stack(0xdb0b1fb0 to 0xdb0b1ff8)
> |[    1.438301] 1fa0:                                     00000000 00000000 00000000 00000000
> |[    1.446683] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> |[    1.455063] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000
> |[    1.461845] Code: e3130004 1a000126 e5943014 e0833001 (e5930000) 
> |[    1.468105] ---[ end trace 5481d6c45bd9fae0 ]---
> |[    1.472934] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> |[    1.480784] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---
> 
> with  arm-linux-gnueabihf-addr2line -i c0644e38 -e vmlinux
> | arch/arm/include/asm/io.h:117
> | drivers/bus/ti-sysc.c:117
> | drivers/bus/ti-sysc.c:132
> | drivers/bus/ti-sysc.c:1361
> | drivers/bus/ti-sysc.c:2117
> 
> Does this help?

Yes getting closer thanks. Can you please boot one more time with
the following debug patch that sould confirm which target module
during probing triggers the abort?

Looking at the oops 0xfa1cc000, so 0x481cc000 I guess which is d_can0?

Regards,

Tony

8< ------------------
diff --git a/drivers/bus/ti-sysc.c b/drivers/bus/ti-sysc.c
--- a/drivers/bus/ti-sysc.c
+++ b/drivers/bus/ti-sysc.c
@@ -2069,6 +2069,8 @@ static int sysc_probe(struct platform_device *pdev)
 	struct sysc *ddata;
 	int error;
 
+	dev_info(&pdev->dev, "probing\n");
+
 	ddata = devm_kzalloc(&pdev->dev, sizeof(*ddata), GFP_KERNEL);
 	if (!ddata)
 		return -ENOMEM;

Re: next/master boot bisection: next-20190430 on beagle-xm

[Sebastian Andrzej Siewior]

On 2019-05-01 10:44:31 [-0700], Tony Lindgren wrote:
> Hmm so I tried without "earlycon" in command line thinking it might be
> happening with just "earlyprintk" but still no luck.
> 
> BTW, in general you might want to update your kernel command line
> options to:
> 
> debug earlyprintk earlycon

debug. Let me look if I manage to hide that `debug' from systemd…

> As that way you get early output without CONFIG_DEBUG_LL=y
> with earlycon that should be enabled by default already.

yup, thx.

> Yes getting closer thanks. Can you please boot one more time with
> the following debug patch that sould confirm which target module
> during probing triggers the abort?

|[    1.091575] ti-sysc 48042000.target-module: probing
|[    1.096694] ti-sysc 48042000.target-module: sysc_flags 00000222 != 00000022
|[    1.104784] ti-sysc 48044000.target-module: probing
|[    1.109974] ti-sysc 48044000.target-module: sysc_flags 00000222 != 00000022
|[    1.118079] ti-sysc 48046000.target-module: probing
|[    1.123206] ti-sysc 48046000.target-module: sysc_flags 00000222 != 00000022
|[    1.131331] ti-sysc 48048000.target-module: probing
|[    1.136505] ti-sysc 48048000.target-module: sysc_flags 00000222 != 00000022
|[    1.144571] ti-sysc 4804a000.target-module: probing
|[    1.149755] ti-sysc 4804a000.target-module: sysc_flags 00000222 != 00000022
|[    1.157846] ti-sysc 4804c000.target-module: probing
|[    1.164537] ti-sysc 480602fc.target-module: probing
|[    1.170858] ti-sysc 48080000.target-module: probing
|[    1.176168] ti-sysc 480c8000.target-module: probing
|[    1.182309] ti-sysc 480ca000.target-module: probing
|[    1.188357] ti-sysc 4819c000.target-module: probing
|[    1.206713] ti-sysc 481a0000.target-module: probing
|[    1.212003] ti-sysc 481a6050.target-module: probing
|[    1.217306] ti-sysc 481a8050.target-module: probing
|[    1.222546] ti-sysc 481aa050.target-module: probing
|[    1.227806] ti-sysc 481ac000.target-module: probing
|[    1.234355] ti-sysc 481ae000.target-module: probing
|[    1.240877] ti-sysc 481cc000.target-module: probing
|[    1.245976] Unhandled fault: external abort on non-linefetch (0x1028) at 0xfa1cc000
|[    1.253847] pgd = (ptrval)
|[    1.256623] [fa1cc000] *pgd=48011452(bad)
|[    1.260749] Internal error: : 1028 [#1] ARM
|[    1.265044] Modules linked in:
|[    1.268187] CPU: 0 PID: 1 Comm: swapper Not tainted 5.1.0-rc7-next-20190501-dirty #31
|[    1.276234] Hardware name: Generic AM33XX (Flattened Device Tree)
|[    1.282510] PC is at sysc_probe+0xb80/0xfc0
|[    1.286805] LR is at sysc_probe+0xb40/0xfc0
|[    1.291100] pc : [<c0644f90>]    lr : [<c0644f50>]    psr: 60000013
|[    1.297538] sp : db0b1dc0  ip : 00000013  fp : c1131648
|[    1.302904] r10: c116c9a4  r9 : c116c984  r8 : 00000028
|[    1.308270] r7 : 00000001  r6 : c0e373ac  r5 : 00000000  r4 : db345940
|[    1.314975] r3 : fa1cc000  r2 : db349700  r1 : 00000000  r0 : 00000000
|[    1.321684] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
|[    1.329017] Control: 10c5387d  Table: 80204019  DAC: 00000051
|[    1.334921] Process swapper (pid: 1, stack limit = 0x(ptrval))
…
|[    1.496788] [<c0644f90>] (sysc_probe) from [<c08f00c4>] (platform_drv_probe+0x48/0x98)
|[    1.504933] [<c08f00c4>] (platform_drv_probe) from [<c08ee264>] (really_probe+0xf0/0x2c8)
|[    1.513346] [<c08ee264>] (really_probe) from [<c08ee59c>] (driver_probe_device+0x60/0x16c)
|[    1.521843] [<c08ee59c>] (driver_probe_device) from [<c08ee848>] (device_driver_attach+0x58/0x60)
|[    1.530967] [<c08ee848>] (device_driver_attach) from [<c08ee8a8>] (__driver_attach+0x58/0xcc)
|[    1.539734] [<c08ee8a8>] (__driver_attach) from [<c08ec720>] (bus_for_each_dev+0x74/0xb4)
|[    1.548142] [<c08ec720>] (bus_for_each_dev) from [<c08ed78c>] (bus_add_driver+0x1b8/0x1d8)
|[    1.556640] [<c08ed78c>] (bus_add_driver) from [<c08ef1dc>] (driver_register+0x74/0x108)
|[    1.564963] [<c08ef1dc>] (driver_register) from [<c0302ce4>] (do_one_initcall+0x50/0x1a4)
|[    1.573384] [<c0302ce4>] (do_one_initcall) from [<c1401064>] (kernel_init_freeable+0x1c4/0x25c)
|[    1.582339] [<c1401064>] (kernel_init_freeable) from [<c0ded9e4>] (kernel_init+0x8/0x10c)
|[    1.590750] [<c0ded9e4>] (kernel_init) from [<c03010e8>] (ret_from_fork+0x14/0x2c)
|[    1.598531] Exception stack(0xdb0b1fb0 to 0xdb0b1ff8)
|[    1.603722] 1fa0:                                     00000000 00000000 00000000 00000000
|[    1.612129] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
|[    1.620535] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000
|[    1.627337] Code: ebfffa8a ea000002 e5943014 e0833001 (e5930000) 
|[    1.633616] ---[ end trace d02e59e9267a59cb ]---
|[    1.638458] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b

> Looking at the oops 0xfa1cc000, so 0x481cc000 I guess which is d_can0?

That node around it I guess.

> Regards,
> 
> Tony

Sebastian

Re: next/master boot bisection: next-20190430 on beagle-xm

[Tony Lindgren]

Hi,

* Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 19:03]:
> On 2019-05-01 10:44:31 [-0700], Tony Lindgren wrote:
> > Hmm so I tried without "earlycon" in command line thinking it might be
> > happening with just "earlyprintk" but still no luck.
> > 
> > BTW, in general you might want to update your kernel command line
> > options to:
> > 
> > debug earlyprintk earlycon
> 
> debug. Let me look if I manage to hide that `debug' from systemd…

Oh that.. I've been quite happy with openrc now for years :)

> > Looking at the oops 0xfa1cc000, so 0x481cc000 I guess which is d_can0?
> 
> That node around it I guess.

OK I found two issues. It seems that d_can also needs osc clock
on am335x. And there's no revision register for d_can.. We're now
reading the CTL register unnecessarily.

Below is what I hope fixes the boot issue for you, care to boot
test?

If this helps I'll send out proper patches for for both issues.

Regards,

Tony

8< ----------------------
diff --git a/arch/arm/boot/dts/am33xx-l4.dtsi b/arch/arm/boot/dts/am33xx-l4.dtsi
--- a/arch/arm/boot/dts/am33xx-l4.dtsi
+++ b/arch/arm/boot/dts/am33xx-l4.dtsi
@@ -1762,8 +1762,9 @@
 			reg = <0xcc000 0x4>;
 			reg-names = "rev";
 			/* Domains (P, C): per_pwrdm, l4ls_clkdm */
-			clocks = <&l4ls_clkctrl AM3_L4LS_D_CAN0_CLKCTRL 0>;
-			clock-names = "fck";
+			clocks = <&l4ls_clkctrl AM3_L4LS_D_CAN0_CLKCTRL 0>,
+				 <&dcan0_fck>;
+			clock-names = "fck", "osc";
 			#address-cells = <1>;
 			#size-cells = <1>;
 			ranges = <0x0 0xcc000 0x2000>;
@@ -1785,8 +1786,9 @@
 			reg = <0xd0000 0x4>;
 			reg-names = "rev";
 			/* Domains (P, C): per_pwrdm, l4ls_clkdm */
-			clocks = <&l4ls_clkctrl AM3_L4LS_D_CAN1_CLKCTRL 0>;
-			clock-names = "fck";
+			clocks = <&l4ls_clkctrl AM3_L4LS_D_CAN1_CLKCTRL 0>,
+				 <&dcan1_fck>;
+			clock-names = "fck", "osc";
 			#address-cells = <1>;
 			#size-cells = <1>;
 			ranges = <0x0 0xd0000 0x2000>;

Re: next/master boot bisection: next-20190430 on beagle-xm

[Sebastian Andrzej Siewior]

On 2019-05-01 13:21:49 [-0700], Tony Lindgren wrote:
> Hi,
Hi,

> OK I found two issues. It seems that d_can also needs osc clock
> on am335x. And there's no revision register for d_can.. We're now
> reading the CTL register unnecessarily.
> 
> Below is what I hope fixes the boot issue for you, care to boot
> test?

yup, that boots.
Thanks.

> If this helps I'll send out proper patches for for both issues.
> 
> Regards,
> 
> Tony

Sebastian

Re: next/master boot bisection: next-20190430 on beagle-xm

[Tony Lindgren]

* Sebastian Andrzej Siewior <bigeasy@linutronix.de> [190501 21:14]:
> On 2019-05-01 13:21:49 [-0700], Tony Lindgren wrote:
> > Hi,
> Hi,
> 
> > OK I found two issues. It seems that d_can also needs osc clock
> > on am335x. And there's no revision register for d_can.. We're now
> > reading the CTL register unnecessarily.
> > 
> > Below is what I hope fixes the boot issue for you, care to boot
> > test?
> 
> yup, that boots.

OK good to hear and thanks a lot for testing it. I'll post two
patches shortly.

Regards,

Tony