From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keith Busch Subject: Re: [PATCH] nvme-pci: Use non-operational power state instead of D3 on Suspend-to-Idle Date: Fri, 10 May 2019 08:02:09 -0600 Message-ID: <20190510140209.GG9675@localhost.localdomain> References: <20190509095601.GA19041@lst.de> <225CF4F7-C8E1-4C66-B362-97E84596A54E@canonical.com> <20190509103142.GA19550@lst.de> <31b7d7959bf94c15a04bab0ced518444@AUSX13MPC101.AMER.DELL.COM> <20190509192807.GB9675@localhost.localdomain> <7a002851c435481593f8629ec9193e40@AUSX13MPC101.AMER.DELL.COM> <20190509215409.GD9675@localhost.localdomain> <495d76c66aec41a8bfbbf527820f8eb9@AUSX13MPC101.AMER.DELL.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: Content-Disposition: inline In-Reply-To: Sender: linux-kernel-owner@vger.kernel.org To: Kai-Heng Feng Cc: "Mario.Limonciello@dell.com" , "hch@lst.de" , "axboe@fb.com" , "sagi@grimberg.me" , "rafael@kernel.org" , "linux-pm@vger.kernel.org" , "Wysocki, Rafael J" , "linux-kernel@vger.kernel.org" , "linux-nvme@lists.infradead.org" , "Busch, Keith" List-Id: linux-pm@vger.kernel.org On Thu, May 09, 2019 at 11:05:42PM -0700, Kai-Heng Feng wrote: > Yes, that’ what I was told by the NVMe vendor, so all I know is to impose a > memory barrier. > If mb() shouldn’t be used here, what’s the correct variant to use in this > context? I'm afraid the requirement is still not clear to me. AFAIK, all our barriers routines ensure data is visible either between CPUs, or between CPU and devices. The CPU never accesses HMB memory, so there must be some other reasoning if this barrier is a real requirement for this device. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_MUTT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0052C04A6B for ; Fri, 10 May 2019 14:07:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 713E12175B for ; Fri, 10 May 2019 14:07:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1557497259; bh=5p097pVLcDjCzTKIh2UDmbz2TvM3KsyTCxiFEEUGYX8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=Wcs47FrsBx3wx3gYJRo8fVRsBTUU/BHnptq82lwZ8xRKAk94Cn681Z0ltzOp0wqYu V9OcWJAIhyflZM3QT3Ugm/iNPZfPN78Mc/I3DkFLARaQNhgaLyCCRKf0UIEeKYlHjB HCyXTYIrJoyDfoWQy8yc/UPhpb1VVnat7zNQLdpY= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727708AbfEJOHi (ORCPT ); Fri, 10 May 2019 10:07:38 -0400 Received: from mga03.intel.com ([134.134.136.65]:32011 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727572AbfEJOHi (ORCPT ); Fri, 10 May 2019 10:07:38 -0400 X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 May 2019 07:07:38 -0700 X-ExtLoop1: 1 Received: from unknown (HELO localhost.localdomain) ([10.232.112.69]) by orsmga007.jf.intel.com with ESMTP; 10 May 2019 07:07:37 -0700 Date: Fri, 10 May 2019 08:02:09 -0600 From: Keith Busch To: Kai-Heng Feng Cc: "Mario.Limonciello@dell.com" , "hch@lst.de" , "axboe@fb.com" , "sagi@grimberg.me" , "rafael@kernel.org" , "linux-pm@vger.kernel.org" , "Wysocki, Rafael J" , "linux-kernel@vger.kernel.org" , "linux-nvme@lists.infradead.org" , "Busch, Keith" Subject: Re: [PATCH] nvme-pci: Use non-operational power state instead of D3 on Suspend-to-Idle Message-ID: <20190510140209.GG9675@localhost.localdomain> References: <20190509095601.GA19041@lst.de> <225CF4F7-C8E1-4C66-B362-97E84596A54E@canonical.com> <20190509103142.GA19550@lst.de> <31b7d7959bf94c15a04bab0ced518444@AUSX13MPC101.AMER.DELL.COM> <20190509192807.GB9675@localhost.localdomain> <7a002851c435481593f8629ec9193e40@AUSX13MPC101.AMER.DELL.COM> <20190509215409.GD9675@localhost.localdomain> <495d76c66aec41a8bfbbf527820f8eb9@AUSX13MPC101.AMER.DELL.COM> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.9.1 (2017-09-22) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Message-ID: <20190510140209.xLSWRH4ylb6_CD_Nt-K4NQSuip4gGENvcvjY3Ke46Xw@z> On Thu, May 09, 2019 at 11:05:42PM -0700, Kai-Heng Feng wrote: > Yes, that’ what I was told by the NVMe vendor, so all I know is to impose a > memory barrier. > If mb() shouldn’t be used here, what’s the correct variant to use in this > context? I'm afraid the requirement is still not clear to me. AFAIK, all our barriers routines ensure data is visible either between CPUs, or between CPU and devices. The CPU never accesses HMB memory, so there must be some other reasoning if this barrier is a real requirement for this device.