From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C305C433B4 for ; Wed, 28 Apr 2021 00:39:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 04FE161407 for ; Wed, 28 Apr 2021 00:39:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235454AbhD1Akb (ORCPT ); Tue, 27 Apr 2021 20:40:31 -0400 Received: from mga02.intel.com ([134.134.136.20]:29024 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230368AbhD1Aka (ORCPT ); Tue, 27 Apr 2021 20:40:30 -0400 IronPort-SDR: YjweTBve18neMbpX6+ZGO5WBJ6s2UnllxkbWHJEs1M8zOndw7+n3MTa0LAFhD4f9dpgZYLOmbV XRkEyDZIxGjQ== X-IronPort-AV: E=McAfee;i="6200,9189,9967"; a="183763213" X-IronPort-AV: E=Sophos;i="5.82,256,1613462400"; d="scan'208";a="183763213" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2021 17:39:46 -0700 IronPort-SDR: 0RqJwd+vV3gBvayyZ0K16jmwkWyPVJmz6/rcFgRvMTEQKoI8xvcFrjxqloggiU7vrsgUy99wSJ 0yR3HIQSCrcg== X-IronPort-AV: E=Sophos;i="5.82,256,1613462400"; d="scan'208";a="386326646" Received: from mchintha-mobl.amr.corp.intel.com (HELO skuppusw-mobl5.amr.corp.intel.com) ([10.254.5.143]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Apr 2021 17:39:45 -0700 Subject: Re: [PATCH] PCI: pciehp: Ignore Link Down/Up caused by DPC From: "Kuppuswamy, Sathyanarayanan" To: Lukas Wunner , Bjorn Helgaas , Dan Williams Cc: Ethan Zhao , Sinan Kaya , Ashok Raj , Keith Busch , linux-pci@vger.kernel.org, Russell Currey , Oliver O'Halloran , Stuart Hayes , Mika Westerberg References: <13bbd4f9-dff4-be79-d80a-342399961939@linux.intel.com> Message-ID: Date: Tue, 27 Apr 2021 17:39:43 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <13bbd4f9-dff4-be79-d80a-342399961939@linux.intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Hi Bjorn, On 3/30/21 1:53 PM, Kuppuswamy, Sathyanarayanan wrote: >> Downstream Port Containment (PCIe Base Spec, sec. 6.2.10) disables the >> link upon an error and attempts to re-enable it when instructed by the >> DPC driver. >> >> A slot which is both DPC- and hotplug-capable is currently brought down >> by pciehp once DPC is triggered (due to the link change) and brought up >> on successful recovery.  That's undesirable, the slot should remain up >> so that the hotplugged device remains bound to its driver.  DPC notifies >> the driver of the error and of successful recovery in pcie_do_recovery() >> and the driver may then restore the device to working state. >> >> Moreover, Sinan points out that turning off slot power by pciehp may >> foil recovery by DPC:  Power off/on is a cold reset concurrently to >> DPC's warm reset.  Sathyanarayanan reports extended delays or failure >> in link retraining by DPC if pciehp brings down the slot. >> >> Fix by detecting whether a Link Down event is caused by DPC and awaiting >> recovery if so.  On successful recovery, ignore both the Link Down and >> the subsequent Link Up event. >> >> Afterwards, check whether the link is down to detect surprise-removal or >> another DPC event immediately after DPC recovery.  Ensure that the >> corresponding DLLSC event is not ignored by synthesizing it and >> invoking irq_wake_thread() to trigger a re-run of pciehp_ist(). >> >> The IRQ threads of the hotplug and DPC drivers, pciehp_ist() and >> dpc_handler(), race against each other.  If pciehp is faster than DPC, >> it will wait until DPC recovery completes. >> >> Recovery consists of two steps:  The first step (waiting for link >> disablement) is recognizable by pciehp through a set DPC Trigger Status >> bit.  The second step (waiting for link retraining) is recognizable >> through a newly introduced PCI_DPC_RECOVERING flag. >> >> If DPC is faster than pciehp, neither of the two flags will be set and >> pciehp may glean the recovery status from the new PCI_DPC_RECOVERED flag. >> The flag is zero if DPC didn't occur at all, hence DLLSC events are not >> ignored by default. >> >> This commit draws inspiration from previous attempts to synchronize DPC >> with pciehp: >> >> By Sinan Kaya, August 2018: >> https://lore.kernel.org/linux-pci/20180818065126.77912-1-okaya@kernel.org/ >> >> By Ethan Zhao, October 2020: >> https://lore.kernel.org/linux-pci/20201007113158.48933-1-haifeng.zhao@intel.com/ >> >> By Sathyanarayanan Kuppuswamy, March 2021: >> https://lore.kernel.org/linux-pci/59cb30f5e5ac6d65427ceaadf1012b2ba8dbf66c.1615606143.git.sathyanarayanan.kuppuswamy@linux.intel.com/ >> > Looks good to me. This patch fixes the reported issue in our environment. > > Reviewed-by: Kuppuswamy Sathyanarayanan > Tested-by: Kuppuswamy Sathyanarayanan Any update on this patch? is this queued for merge? One of our customers is looking for this fix. So wondering about the status. -- Sathyanarayanan Kuppuswamy Linux Kernel Developer