From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59CBBC433F5 for ; Fri, 18 Mar 2022 20:28:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233752AbiCRU3p (ORCPT ); Fri, 18 Mar 2022 16:29:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238537AbiCRU3n (ORCPT ); Fri, 18 Mar 2022 16:29:43 -0400 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CF6128E34A; Fri, 18 Mar 2022 13:28:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647635303; x=1679171303; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=bYWaejF1aAGMIe6zGaNynSygYXsQ9N7J6Zt5R6d87uw=; b=JZaJgVyeAsZJaMHs6y+DmuL9ofccna6Lp/0Wyms43oLEROF+DwKK6CkA jpblwA1Dl/DX0jmaZra6LIInQh+sU/MQLaDzW32kKut4C9AZrlzu0SsU6 Ut4Aqe4PUJjWDxr1JBCtOWMNsCN7AEHMAuue9XRjEKj2TJt1VBluHt36W GvGKBnu0zZ7YNtz0YZsuBvK6YmwPzpqqUft/j2KqUHrjhuhOgM8LnjXR5 WlyJJJDXZSLK6h57MRqCsdjiKD/N1nDuvehtNXu+9NnVfJ+xIy0akgGys KqGCkt6qAzcwroF7CjOMnPnyRMUu2gnmcjxD813Dm734FMGEf8Gia2BDg g==; X-IronPort-AV: E=McAfee;i="6200,9189,10290"; a="320425631" X-IronPort-AV: E=Sophos;i="5.90,192,1643702400"; d="scan'208";a="320425631" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2022 13:28:17 -0700 X-IronPort-AV: E=Sophos;i="5.90,192,1643702400"; d="scan'208";a="558633118" Received: from reaganlo-mobl.amr.corp.intel.com (HELO [10.212.159.210]) ([10.212.159.210]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2022 13:28:17 -0700 Message-ID: Date: Fri, 18 Mar 2022 13:28:16 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.5.0 Subject: Re: [PATCH v1] PCI/AER: Handle Multi UnCorrectable/Correctable errors properly Content-Language: en-US To: Bjorn Helgaas , Eric Badger Cc: "Raj, Ashok" , Bjorn Helgaas , Russell Currey , Oliver OHalloran , Kuppuswamy Sathyanarayanan , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org References: <20220317225944.GA765564@bhelgaas> From: Sathyanarayanan Kuppuswamy In-Reply-To: <20220317225944.GA765564@bhelgaas> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On 3/17/22 3:59 PM, Bjorn Helgaas wrote: > Thanks for the additional details! > > After this patch, I guess aer_irq() still reads 0x2 > (PCI_ERR_ROOT_MULTI_COR_RCV), but now it writes 0x2 back which clears > PCI_ERR_ROOT_MULTI_COR_RCV. > > In addition, aer_irq() will continue on to read PCI_ERR_ROOT_ERR_SRC, > which probably contains either 0 or junk left over from being captured > when PCI_ERR_ROOT_COR_RCV was set. > > And aer_irq() will queue an e_src record with status == > PCI_ERR_ROOT_MULTI_COR_RCV. But since PCI_ERR_ROOT_COR_RCV is not set > in status, aer_isr_one_error() will do nothing, right? > > That might not be*terrible* and is definitely better than not being > able to handle future interrupts. But we basically threw away the > information that multiple errors occurred, and we queued an e_src > record that occupies space without being used for anything. Yes, you are correct. One other way to minimize this race window is to clear the Root status register as soon as possible. Maybe we can move it before source ID read as below. --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1204,8 +1204,8 @@ static irqreturn_t aer_irq(int irq, void *context) if (!(e_src.status & AER_ERR_STATUS_MASK)) return IRQ_NONE; - pci_read_config_dword(rp, aer + PCI_ERR_ROOT_ERR_SRC, &e_src.id); pci_write_config_dword(rp, aer + PCI_ERR_ROOT_STATUS, e_src.status); + pci_read_config_dword(rp, aer + PCI_ERR_ROOT_ERR_SRC, &e_src.id) -- Sathyanarayanan Kuppuswamy Linux Kernel Developer From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC57DC433F5 for ; Fri, 18 Mar 2022 20:30:19 +0000 (UTC) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4KKwcV0sbgz3bdf for ; Sat, 19 Mar 2022 07:30:18 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=eSxagLLR; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=linux.intel.com (client-ip=134.134.136.65; helo=mga03.intel.com; envelope-from=sathyanarayanan.kuppuswamy@linux.intel.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=intel.com header.i=@intel.com header.a=rsa-sha256 header.s=Intel header.b=eSxagLLR; dkim-atps=neutral Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4KKwbQ1FdVz30NC for ; Sat, 19 Mar 2022 07:29:21 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1647635362; x=1679171362; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=bYWaejF1aAGMIe6zGaNynSygYXsQ9N7J6Zt5R6d87uw=; b=eSxagLLR4giQ7M6jrpW+0b7P/2sB4hHTCi/GohufgZ28wd9PE8sEm8aq qVg+IGc73oE7JHtC7sWSvOZhBZXz6Ls7/uEoZxLmiWSrfGb7/tq/NaKdZ qx9JpsmWaJLKmESuDDwi1D9/PFug7kubJefyRuUUMYVZ1Zd+A+QcPDR48 TkFrJWUQ5KZdOjIJQ7QK1tHjrE4vy6ZZAyengzY510Ww3gn1jhqJD7Ek7 TmVGQf7lV1GsgGtOaYnXusrLDFZX6K7Y1OjTm8XEP57frAckNWqxpay+q W+fC8EKKHf30N7iS1gxB7adnCJeRPWsXhgeQ0cbeD21GsdpdfO4g0myFV Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10290"; a="257167957" X-IronPort-AV: E=Sophos;i="5.90,192,1643702400"; d="scan'208";a="257167957" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2022 13:28:17 -0700 X-IronPort-AV: E=Sophos;i="5.90,192,1643702400"; d="scan'208";a="558633118" Received: from reaganlo-mobl.amr.corp.intel.com (HELO [10.212.159.210]) ([10.212.159.210]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Mar 2022 13:28:17 -0700 Message-ID: Date: Fri, 18 Mar 2022 13:28:16 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.5.0 Subject: Re: [PATCH v1] PCI/AER: Handle Multi UnCorrectable/Correctable errors properly Content-Language: en-US To: Bjorn Helgaas , Eric Badger References: <20220317225944.GA765564@bhelgaas> From: Sathyanarayanan Kuppuswamy In-Reply-To: <20220317225944.GA765564@bhelgaas> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kuppuswamy Sathyanarayanan , "Raj, Ashok" , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Oliver OHalloran , Bjorn Helgaas , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On 3/17/22 3:59 PM, Bjorn Helgaas wrote: > Thanks for the additional details! > > After this patch, I guess aer_irq() still reads 0x2 > (PCI_ERR_ROOT_MULTI_COR_RCV), but now it writes 0x2 back which clears > PCI_ERR_ROOT_MULTI_COR_RCV. > > In addition, aer_irq() will continue on to read PCI_ERR_ROOT_ERR_SRC, > which probably contains either 0 or junk left over from being captured > when PCI_ERR_ROOT_COR_RCV was set. > > And aer_irq() will queue an e_src record with status == > PCI_ERR_ROOT_MULTI_COR_RCV. But since PCI_ERR_ROOT_COR_RCV is not set > in status, aer_isr_one_error() will do nothing, right? > > That might not be*terrible* and is definitely better than not being > able to handle future interrupts. But we basically threw away the > information that multiple errors occurred, and we queued an e_src > record that occupies space without being used for anything. Yes, you are correct. One other way to minimize this race window is to clear the Root status register as soon as possible. Maybe we can move it before source ID read as below. --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1204,8 +1204,8 @@ static irqreturn_t aer_irq(int irq, void *context) if (!(e_src.status & AER_ERR_STATUS_MASK)) return IRQ_NONE; - pci_read_config_dword(rp, aer + PCI_ERR_ROOT_ERR_SRC, &e_src.id); pci_write_config_dword(rp, aer + PCI_ERR_ROOT_STATUS, e_src.status); + pci_read_config_dword(rp, aer + PCI_ERR_ROOT_ERR_SRC, &e_src.id) -- Sathyanarayanan Kuppuswamy Linux Kernel Developer