Linux (RHEL 7.6 with OSP 14) Bugs

Linux (RHEL 7.6 with OSP 14) Bugs

[Amer Hwitat]

[-- Attachment #1.1.1: Type: text/plain, Size: 4104 bytes --]

Dears,

I have the following Bugs that crashed my VM, I reported it to RH, they
didn't answer, and banned my developer account, the Bug is when you disable
the network on RHEL with OSP 14 installed all in one, it crashes the
system, I had a 12GB RAM, with 8 CPUs on the VM, and I found out that this
crash report pissed off someone in RH, because they called me, and said
what do you want from me!!, what I need is a Simple reply, is this a bug or
not.

here is the problem:



[root@localhost network-scripts]# systemctl status network -l
? network.service - LSB: Bring up/down networking
   Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 2019-01-19 03:47:01 EST;
21s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 86319 ExecStop=/etc/rc.d/init.d/network stop (code=exited,
status=0/SUCCESS)
  Process: 86591 ExecStart=/etc/rc.d/init.d/network start (code=exited,
status=1/FAILURE)
    Tasks: 0

Jan 19 03:47:01 localhost.localdomain dhclient[86963]: Please report for
this software via the Red Hat Bugzilla site:
Jan 19 03:47:01 localhost.localdomain dhclient[86963]:
http://bugzilla.redhat.com
Jan 19 03:47:01 localhost.localdomain dhclient[86963]: ution.
Jan 19 03:47:01 localhost.localdomain dhclient[86963]: exiting.
Jan 19 03:47:01 localhost.localdomain network[86591]: failed.
Jan 19 03:47:01 localhost.localdomain network[86591]: [FAILED]
Jan 19 03:47:01 localhost.localdomain systemd[1]: network.service: control
process exited, code=exited status=1
Jan 19 03:47:01 localhost.localdomain systemd[1]: Failed to start LSB:
Bring up/down networking.
Jan 19 03:47:01 localhost.localdomain systemd[1]: Unit network.service
entered failed state.
Jan 19 03:47:01 localhost.localdomain systemd[1]: network.service failed.
[root@localhost network-scripts]#

[root@localhost log]#
Message from syslogd@localhost at Jan 23 02:23:31 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s!
[ovsdb-server:10088]

[root@amer network-scripts]#
Message from syslogd@amer at Jan 27 12:46:38 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s!
[nova-api:102738]

Message from syslogd@amer at Jan 27 19:26:19 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#5 stuck for 26s! [swapper/5:0]

Message from syslogd@amer at Jan 27 19:26:19 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#1 stuck for 27s!
[dmeventd:71548]

Message from syslogd@amer at Jan 27 19:27:30 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s!
[6_scheduler:64928]

Message from syslogd@amer at Jan 27 19:31:25 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s!
[ksoftirqd/5:34]

Message from syslogd@amer at Jan 27 19:32:42 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#3 stuck for 33s!
[swift-object-up:11358]

Message from syslogd@amer at Jan 27 19:33:55 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s!
[dmeventd:71548]

Message from syslogd@amer at Jan 27 19:34:25 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#2 stuck for 65s!
[kworker/2:0:59993]

Message from syslogd@amer at Jan 27 19:37:50 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#2 stuck for 24s!
[kworker/u256:3:8447]

Message from syslogd@amer at Jan 27 19:37:50 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s!
[ksoftirqd/5:34]

Message from syslogd@amer at Jan 27 19:37:51 ...
 kernel:NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s!
[systemd:11968]

The CPU has been disabled by the guest operating system. Power off or reset
the virtual machine.

snapshots attached
[image: Red Hat Enterprise Linux 7 64-bit (2)-2019-01-28-03-57-27.png]
[image: Red Hat Enterprise Linux 7 64-bit (2)-2019-01-28-04-26-41.png]
[image: working solution.JPG]
the last snapshot is from a successful installation of OSP 14 that
specifically says that Kernel is not compatible with Firmware (Bios).

I didn't test on Debian flavors but I think it's the same, the problem is
with RabbitMQ heart beats, when the server is disconnected it times out
causing this problem of kernel loop.

Thanks and Best regards
Amer

[-- Attachment #1.1.2: Type: text/html, Size: 4745 bytes --]

[-- Attachment #1.2: Red Hat Enterprise Linux 7 64-bit (2)-2019-01-28-03-57-27.png --]
[-- Type: image/png, Size: 26978 bytes --]

[-- Attachment #1.3: Red Hat Enterprise Linux 7 64-bit (2)-2019-01-28-04-26-41.png --]
[-- Type: image/png, Size: 386086 bytes --]

[-- Attachment #1.4: working solution.JPG --]
[-- Type: image/jpeg, Size: 139627 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Linux (RHEL 7.6 with OSP 14) Bugs

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 2027 bytes --]

On Sat, 18 May 2019 19:50:27 +0300, Amer Hwitat said:

I have no idea why you sent this to the linux-audit list, as it doesn't seem to
involve the kernel audit subsystem in any way.

> I have the following Bugs that crashed my VM, I reported it to RH, they
> didn't answer, and banned my developer account, the Bug is when you disable
> the network on RHEL with OSP 14 installed all in one, it crashes the
> system, I had a 12GB RAM, with 8 CPUs on the VM, and I found out that this
> crash report pissed off someone in RH, because they called me, and said
> what do you want from me!!, what I need is a Simple reply, is this a bug or
> not.

It's probably a bug. Given that you mention 'RabbitMQ heartbeats'
and the problem happens when you yank the network out from under
RabbitMQ, it's probably a RabbitMQ bug failing to deal gracefully with
the removal of a network device from the configuration.

And no, you probably won't be able to get help from RedHat. Having said that,
it's unclear why your developer account got closed, unless there's more to the
story than you're telling.  Were you insisting that RedHat fix a bug in (probably)
RabbitMQ, which isn't a RedHat product?

Note in your screenshot:

"A kernel problem occurred, but your kernel has been tainetd (flags:GL)
Explanation: Kernel maintainers are unable to diagnose tained reports"

The L flag is saying that there was a kernel lockup detected before the panic
happened, which means that when the NMI caused the panic the kernel was already
screwed up. There's a good chance that kcrash was unable to capture useful
data. You need to be lookng at the cause of the softlockup, not the panic.

More troublesome is the G flag, which indicates that proprietary modules have
been loaded. Since source is unavailable, there's no way to debug it without
the assistance of whoever produced the proprietary module.

If you don't already know what module did it, 'dmesg | grep taint' should
reveal the culprit.

Take it up with whoever produced the module Good luck.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]