From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB9F2C433EF for ; Wed, 18 May 2022 04:03:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229560AbiERED4 (ORCPT ); Wed, 18 May 2022 00:03:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51116 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbiEREDu (ORCPT ); Wed, 18 May 2022 00:03:50 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2ECE742A18 for ; Tue, 17 May 2022 21:02:10 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id ev18so784704pjb.4 for ; Tue, 17 May 2022 21:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=oMTtZkU4nCZ91NEzU4LZyarDEe9CrBJP6c+5gci4GhU=; b=hF2SJNQuyAPX27K3O9IKX/L7RDe9giGyYcZXLKfxFWF8u2tDkNwctNMV0RTCXeqRtY Z/h56M6pzUGxi0REj9rAvd+5EJwbbTuTDWrXHJHFAHMuLiUAg2edghW6NutRy2mfL/Jm GMPvVsvejGx11NLjXPDbv0od0DV06IaVgfKqzMldePlFVBPRgeTw+s3gy6CS6Q6xrJze 6igJeBRbf8skF17n1cjJu1pjR5o3rBESjG0GklU0cXmf8msxavMpTLRNixhQLL59apY2 lP+0Sw+pVuMPOZDXlWhb4y+h+vBrYz5EEnJNZwavhIqx6Af8rBeeMJdmBg6IXNl3Q/9S PC5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=oMTtZkU4nCZ91NEzU4LZyarDEe9CrBJP6c+5gci4GhU=; b=imdj9yf+6X8Ca13RguQIA3lP+XK16p3Rn5buub1Lrm/js+CI/Ryo8zTLUzPse5eS6P u+jKa0QQtSQfrf26Qrp3T+B9wlpLWA/x1vkTnXHehZxwh13IPSFIKX4OyFXF8CRHjN61 2rw2evmDpNHG+nQZEA8Y1Rwq1X3WQsBXRt0x5YIY1NZ3j9nW8H9br4TWk4YjPho1Ks1t 10s5e19iHrDmM1+tnscye4il8SYuxE3H9v1HH2yc+7+QRkI3DIrMPumS5FshQqlnMN+q w48jWiymRFDcio1uFcWxmCcQLXnZzIy7/gO/MoMeKudc8u2Bv3JB6uwXv5qHdA+gT5/t kVaw== X-Gm-Message-State: AOAM531VuTC0AV6ccn/nuWmlPWSToE8hBKvZAK3UpdWpsjXyLUsyZB43 Y6Md+M/Py/DqCz6TqbU8GG8s X-Google-Smtp-Source: ABdhPJzXyPrirmGjfB9wAO7hUPPrBdR7ojzbTs2rrQl2LhF5QrddfhsS+8ZM5ow+o6gPFsXwn0NTUA== X-Received: by 2002:a17:90a:f3d5:b0:1df:2191:5ad2 with SMTP id ha21-20020a17090af3d500b001df21915ad2mr20245819pjb.136.1652846365631; Tue, 17 May 2022 20:59:25 -0700 (PDT) Received: from thinkpad ([117.207.31.8]) by smtp.gmail.com with ESMTPSA id g8-20020a170902d5c800b0015e8d4eb225sm413568plh.111.2022.05.17.20.59.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 May 2022 20:59:25 -0700 (PDT) Date: Wed, 18 May 2022 09:29:15 +0530 From: Manivannan Sadhasivam To: Bjorn Helgaas Cc: Bjorn Helgaas , Lorenzo Pieralisi , Keith Busch , Christoph Hellwig , linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-kernel@vger.kernel.org, Stanimir Varbanov , Bjorn Andersson , Jens Axboe , Veerabhadrarao Badiganti , quic_krichai@quicinc.com, Nitin Rawat , Vidya Sagar , Sagi Grimberg Subject: Re: [PATCH 1/3] PCI: Add a flag to notify PCI drivers about powerdown during suspend Message-ID: <20220518035915.GB4791@thinkpad> References: <20220517150908.GA4528@thinkpad> <20220517172423.GA1083672@bhelgaas> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20220517172423.GA1083672@bhelgaas> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, May 17, 2022 at 12:24:23PM -0500, Bjorn Helgaas wrote: > On Tue, May 17, 2022 at 08:39:08PM +0530, Manivannan Sadhasivam wrote: > > On Mon, May 16, 2022 at 03:18:17PM -0500, Bjorn Helgaas wrote: > > > On Fri, May 13, 2022 at 04:30:25PM +0530, Manivannan Sadhasivam wrote: > > > > On some systems like Chromebooks based on Qcom chipsets, the OS may > > > > powerdown all PCIe devices during system suspend for aggressive > > > > powersaving. In that case, the PCI host controller drivers need to notify > > > > the PCI device drivers that the power will be taken off during system > > > > suspend so that the drivers can prepare the devices accordingly. > > > > > > "The OS may powerdown all PCIe devices ..." makes it sound like this > > > is an OS policy decision. Where exactly (what function) is that? > > > > > > Or if it's not an OS policy decision, but rather some property of the > > > hardware, say that specifically. > > > > On SC7280, it is the Resource Power Manager(RPMh) that's powering > > the devices down by cutting off the PCIe voltage domain. But the > > SC7280 RC driver itself may put the PCIe devices into D3cold state > > during system suspend. > > https://lore.kernel.org/lkml/CAE-0n53ho2DX2rqQMvvKAuDCfsWW62TceTaNPzv5Mn_NQ-U6dA@mail.gmail.com/T/ > > > > So to cover both cases (one is a hardware independent of SoC and > > another one is the device driver), and to be generic, I've used the > > term "OS" after looking at the previous flags. > > This sort of device-specific behavior definitely needs a pointer to an > example. Otherwise it seems like it could be generic PCIe behavior > that should be documented in the PCIe base spec. > Okay. > > > > One prime example is the PCI NVMe driver. This flag can be used by the > > > > driver to shutdown the NVMe device during suspend and recover it during > > > > resume. > > Apparently nvme is broken, or at least sub-optimal, without this flag. Yes, broken on SC7280 or any other SoCs that turn off power. > What other drivers will be similarly affected? > I don't have a list but the drivers that don't expect the device to be turned off or reset during suspend may experience this issue. Right now, we have only identified the issue with NVMe because that's what used on Chromebooks. But in the coming days, we may need to fix some of the drivers also. Thanks, Mani > > > > Signed-off-by: Manivannan Sadhasivam > > > > --- > > > > include/linux/pci.h | 1 + > > > > 1 file changed, 1 insertion(+) > > > > > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > > > index 60adf42460ab..069caf1fe88d 100644 > > > > --- a/include/linux/pci.h > > > > +++ b/include/linux/pci.h > > > > @@ -578,6 +578,7 @@ struct pci_host_bridge { > > > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > > > unsigned int size_windows:1; /* Enable root bus sizing */ > > > > unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > > > + unsigned int suspend_poweroff:1; /* OS may poweroff devices during system suspend */ > > > > > > > > /* Resource alignment requirements */ > > > > resource_size_t (*align_resource)(struct pci_dev *dev, -- மணிவண்ணன் சதாசிவம்