Re: [PATCH] ipmi:kcs: Rate limit the invalid state warning

[Breno Leitao <leitao@debian.org>]

On Fri, Oct 30, 2026 at 08:00:06AM -0500, Corey Minyard wrote:
> On Fri, Oct 30, 2026 at 03:29:45AM -0700, Breno Leitao wrote:
> > start_kcs_transaction() warns each time it is asked to start a
> > transaction while the KCS state machine is busy. The warning is not
> > rate limited, so if the SI layer believes the interface is idle while
> > the state machine is still running, every message sent to the BMC logs
> > another line.
> > 
> > Kernels carrying commit c608966f3f9c ("ipmi: fix msg stack when IPMI is
> > disconnected") hit this. When the interface is declared hosed, the
> > state machine is reset to KCS_ERROR0 rather than KCS_IDLE, and keeps
> > walking the error states while the SI layer, which treats it as idle,
> > tries to start new messages.
> > 
> > One host logged 29K of these lines in four days, several per second:
> > 
> >   [Thu Oct 29 14:55:54 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 9
> >   [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 7
> >   [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 8
> >   [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 9
> >   [Thu Oct 29 14:55:55 2026] ipmi_si dmi-ipmi-si.0: KCS in invalid state 7
> 
> The rate limiting is a good idea.  I've included the patch.
> 
> However, it also sounds like you don't have working IPMI hardware on
> these systems.  The change you mention just exacerbates the problem
> here, but I think it would be a good idea to track down why this is
> happening.
> 
> If you really have working IPMI hardware, then it would be a good
> idea to trace down why this is happening.

Right. This is the Meta fleet, where we see a bunch of IPMI problems,
given the absurd number of hosts.

My main interest here is keeping these errors from flooding our
monitoring systems, but I am fairly sure other folks are looking into
the IPMI issues themselves.

> What specific kernel version are you running?

It happened on a mix of kernels (6.13, 6.16 and possibly 7.1).

> BTW, I need a link to the discussion about this so I can add a Closes:
> tag to the git change.

Sorry, I don't have a public link to Dan's report.

Thanks for the review and picking the patch.
--breno