linuxppc-dev.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
From: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@linux.intel.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: linux-pci@vger.kernel.org, "Jon Pan-Doh" <pandoh@google.com>,
	"Karolina Stolarek" <karolina.stolarek@oracle.com>,
	"Martin Petersen" <martin.petersen@oracle.com>,
	"Ben Fuller" <ben.fuller@oracle.com>,
	"Drew Walton" <drewwalton@microsoft.com>,
	"Anil Agrawal" <anilagrawal@meta.com>,
	"Tony Luck" <tony.luck@intel.com>,
	"Ilpo Järvinen" <ilpo.jarvinen@linux.intel.com>,
	"Lukas Wunner" <lukas@wunner.de>,
	"Jonathan Cameron" <Jonathan.Cameron@huawei.com>,
	"Sargun Dhillon" <sargun@meta.com>,
	"Paul E . McKenney" <paulmck@kernel.org>,
	"Mahesh J Salgaonkar" <mahesh@linux.ibm.com>,
	"Oliver O'Halloran" <oohall@gmail.com>,
	"Kai-Heng Feng" <kaihengf@nvidia.com>,
	"Keith Busch" <kbusch@kernel.org>,
	"Robert Richter" <rrichter@amd.com>,
	"Terry Bowman" <terry.bowman@amd.com>,
	"Shiju Jose" <shiju.jose@huawei.com>,
	"Dave Jiang" <dave.jiang@intel.com>,
	linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
	"Bjorn Helgaas" <bhelgaas@google.com>
Subject: Re: [PATCH v6 14/16] PCI/AER: Introduce ratelimit for error logs
Date: Tue, 20 May 2025 11:42:10 -0700	[thread overview]
Message-ID: <c771e3de-b945-49cd-b078-762164d6ac5d@linux.intel.com> (raw)
In-Reply-To: <20250520183153.GA1316070@bhelgaas>


On 5/20/25 11:31 AM, Bjorn Helgaas wrote:
> On Mon, May 19, 2025 at 09:59:29PM -0700, Sathyanarayanan Kuppuswamy wrote:
>> On 5/19/25 2:35 PM, Bjorn Helgaas wrote:
>>> From: Jon Pan-Doh <pandoh@google.com>
>>>
>>> Spammy devices can flood kernel logs with AER errors and slow/stall
>>> execution. Add per-device ratelimits for AER correctable and uncorrectable
>>> errors that use the kernel defaults (10 per 5s).
>>>
>>> There are two AER logging entry points:
>>>
>>>     - aer_print_error() is used by DPC and native AER
>>>
>>>     - pci_print_aer() is used by GHES and CXL
>>>
>>> The native AER aer_print_error() case includes a loop that may log details
>>> from multiple devices.  This is ratelimited by the union of ratelimits for
>>> these devices, set by add_error_device(), which collects the devices.  If
>>> no such device is found, the Error Source message is ratelimited by the
>>> Root Port or RCEC that received the ERR_* message.
>>>
>>> The DPC aer_print_error() case is currently not ratelimited.
>> Can we also not rate limit fatal errors in AER driver?
> In other words, only rate limit AER_CORRECTABLE and AER_NONFATAL for
> AER?  Seems plausible to me.
Yes, we might lose important information by rate-limiting FATAL errors. I
believe FATAL errors should be infrequent, so it's reasonable to allow them
through without rate limiting. Once you make this change, please also
update the related SysFS documentation and update code accordingly.

-- 
Sathyanarayanan Kuppuswamy
Linux Kernel Developer



  reply	other threads:[~2025-05-20 18:42 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-05-19 21:35 [PATCH v6 00/16] Rate limit AER logs Bjorn Helgaas
2025-05-19 21:35 ` [PATCH v6 01/16] PCI/DPC: Initialize aer_err_info before using it Bjorn Helgaas
2025-05-19 22:41   ` Sathyanarayanan Kuppuswamy
2025-05-20 13:53     ` Bjorn Helgaas
2025-05-20  9:39   ` Ilpo Järvinen
2025-05-20 13:54     ` Bjorn Helgaas
2025-05-19 21:35 ` [PATCH v6 02/16] PCI/DPC: Log Error Source ID only when valid Bjorn Helgaas
2025-05-19 23:15   ` Sathyanarayanan Kuppuswamy
2025-05-20 14:00     ` Bjorn Helgaas
2025-05-20 14:20       ` Ilpo Järvinen
2025-05-20 10:28   ` Ilpo Järvinen
2025-05-20 14:05     ` Bjorn Helgaas
2025-05-19 21:35 ` [PATCH v6 03/16] PCI/AER: Consolidate Error Source ID logging in aer_print_port_info() Bjorn Helgaas
2025-05-19 23:39   ` Sathyanarayanan Kuppuswamy
2025-05-20 14:21     ` Bjorn Helgaas
2025-05-20 10:31   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 04/16] PCI/AER: Extract bus/dev/fn in aer_print_port_info() with PCI_BUS_NUM(), etc Bjorn Helgaas
2025-05-19 23:47   ` Sathyanarayanan Kuppuswamy
2025-05-20 10:32   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 05/16] PCI/AER: Rename aer_print_port_info() to aer_print_source() Bjorn Helgaas
2025-05-19 23:48   ` Sathyanarayanan Kuppuswamy
2025-05-20 10:33   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 06/16] PCI/AER: Move aer_print_source() earlier in file Bjorn Helgaas
2025-05-19 23:49   ` Sathyanarayanan Kuppuswamy
2025-05-20 10:34   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 07/16] PCI/AER: Initialize aer_err_info before using it Bjorn Helgaas
2025-05-19 23:50   ` Sathyanarayanan Kuppuswamy
2025-05-20 10:39   ` Ilpo Järvinen
2025-05-20 14:27     ` Bjorn Helgaas
2025-05-19 21:35 ` [PATCH v6 08/16] PCI/AER: Simplify pci_print_aer() Bjorn Helgaas
2025-05-20  0:02   ` Sathyanarayanan Kuppuswamy
2025-05-20 14:38     ` Bjorn Helgaas
2025-05-20 10:42   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 09/16] PCI/AER: Update statistics early in logging Bjorn Helgaas
2025-05-20  1:32   ` Sathyanarayanan Kuppuswamy
2025-05-20 11:04   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 10/16] PCI/AER: Combine trace_aer_event() with statistics updates Bjorn Helgaas
2025-05-20  1:49   ` Sathyanarayanan Kuppuswamy
2025-05-20 11:08   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 11/16] PCI/AER: Check log level once and remember it Bjorn Helgaas
2025-05-19 23:17   ` Weinan Liu
2025-05-20 14:46     ` Bjorn Helgaas
2025-05-20  2:49   ` Sathyanarayanan Kuppuswamy
2025-05-20 11:26   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 12/16] PCI/AER: Make all pci_print_aer() log levels depend on error type Bjorn Helgaas
2025-05-20  3:23   ` Sathyanarayanan Kuppuswamy
2025-05-20 11:37   ` Ilpo Järvinen
2025-05-20 15:04     ` Bjorn Helgaas
2025-05-19 21:35 ` [PATCH v6 13/16] PCI/AER: Rename struct aer_stats to aer_report Bjorn Helgaas
2025-05-20  3:30   ` Sathyanarayanan Kuppuswamy
2025-05-20 21:25     ` Bjorn Helgaas
2025-05-20 11:38   ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 14/16] PCI/AER: Introduce ratelimit for error logs Bjorn Helgaas
2025-05-20  4:59   ` Sathyanarayanan Kuppuswamy
2025-05-20 18:31     ` Bjorn Helgaas
2025-05-20 18:42       ` Sathyanarayanan Kuppuswamy [this message]
2025-05-20 11:55   ` Ilpo Järvinen
2025-05-20 19:38     ` Bjorn Helgaas
2025-05-21  9:57       ` Ilpo Järvinen
2025-05-19 21:35 ` [PATCH v6 15/16] PCI/AER: Add ratelimits to PCI AER Documentation Bjorn Helgaas
2025-05-20  5:01   ` Sathyanarayanan Kuppuswamy
2025-05-20 19:48     ` Bjorn Helgaas
2025-05-20 20:36       ` Sathyanarayanan Kuppuswamy
2025-05-19 21:35 ` [PATCH v6 16/16] PCI/AER: Add sysfs attributes for log ratelimits Bjorn Helgaas
2025-05-20  5:05   ` Sathyanarayanan Kuppuswamy
2025-05-20 12:02   ` Ilpo Järvinen
2025-05-20 16:31     ` Bjorn Helgaas
2025-05-20  9:05 ` [PATCH v6 00/16] Rate limit AER logs Krzysztof Wilczyński

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=c771e3de-b945-49cd-b078-762164d6ac5d@linux.intel.com \
    --to=sathyanarayanan.kuppuswamy@linux.intel.com \
    --cc=Jonathan.Cameron@huawei.com \
    --cc=anilagrawal@meta.com \
    --cc=ben.fuller@oracle.com \
    --cc=bhelgaas@google.com \
    --cc=dave.jiang@intel.com \
    --cc=drewwalton@microsoft.com \
    --cc=helgaas@kernel.org \
    --cc=ilpo.jarvinen@linux.intel.com \
    --cc=kaihengf@nvidia.com \
    --cc=karolina.stolarek@oracle.com \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=lukas@wunner.de \
    --cc=mahesh@linux.ibm.com \
    --cc=martin.petersen@oracle.com \
    --cc=oohall@gmail.com \
    --cc=pandoh@google.com \
    --cc=paulmck@kernel.org \
    --cc=rrichter@amd.com \
    --cc=sargun@meta.com \
    --cc=shiju.jose@huawei.com \
    --cc=terry.bowman@amd.com \
    --cc=tony.luck@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).