Deadlock under load with Linux 5.9 and other recent kernels

[Christian Hewitt <christianshewitt@gmail.com>]

I am using an ARM SBC device with Amlogic S922X chip (Beelink GS-King-X, an Android STB) to boot the Kodi mediacentre distro LibreELEC (which I work on) although the issue is also reproducible with Manjaro and Armbian on the same hardware, and with the GT-King and GT-King Pro devices from the same vendor - all three devices are using a common dtsi:

https://github.com/chewitt/linux/blob/amlogic-5.9-integ/arch/arm64/boot/dts/amlogic/meson-g12b-gsking-x.dts
https://github.com/chewitt/linux/blob/amlogic-5.9-integ/arch/arm64/boot/dts/amlogic/meson-g12b-gtking-pro.dts
https://github.com/chewitt/linux/blob/amlogic-5.9-integ/arch/arm64/boot/dts/amlogic/meson-g12b-gtking.dts
https://github.com/chewitt/linux/blob/amlogic-5.9-integ/arch/arm64/boot/dts/amlogic/meson-g12b-w400.dtsi

I have schematics for the devices, but can only share those privately on request.

For testing I am booting LibreELEC from SD card. The box has a 4TB SATA drive internally connected with a USB > SATA bridge, see dmesg: http://ix.io/2yLh and I connect a USB stick with a 4GB ISO file that I copy to the internal SATA drive. Within 10-20 seconds of starting the copy the box deadlocks needing a hard power cycle to recover. The timing of the deadlock is variable but the device _always_ deadlocks. Although I am using a simple copy use-case, there are similar reports in Armbian forums performing tasks like installs/updates that involve I/O loads.

Following advice in the #linux-amlogic IRC channel I added CONFIG_SOFTLOCKUP_DETECTOR and CONFIG_DETECT_HUNG_TASK and was able to get output on the HDMI screen (it is not possible to connect to UART pins without destroying the box case). If you advance the following video frame by frame in VLC you can see the output:

https://www.dropbox.com/s/klvcizim8cs5lze/lockup_clip.mov?dl=0

I am not a coding developer so the output doesn’t mean much to me, but I am happy to follow guidance or install patches to get more output or test things.

Christian