From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC49AC43387 for ; Thu, 10 Jan 2019 17:31:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 971FB21783 for ; Thu, 10 Jan 2019 17:31:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730187AbfAJRbq (ORCPT ); Thu, 10 Jan 2019 12:31:46 -0500 Received: from mga01.intel.com ([192.55.52.88]:27139 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729799AbfAJRbp (ORCPT ); Thu, 10 Jan 2019 12:31:45 -0500 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Jan 2019 09:31:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,462,1539673200"; d="scan'208";a="117118339" Received: from unknown (HELO localhost.localdomain) ([10.232.112.69]) by orsmga003.jf.intel.com with ESMTP; 10 Jan 2019 09:31:43 -0800 Date: Thu, 10 Jan 2019 10:30:17 -0700 From: Keith Busch To: "Aneesh Kumar K.V" Cc: linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-mm@kvack.org, Greg Kroah-Hartman , Rafael Wysocki , Dave Hansen , Dan Williams Subject: Re: [PATCHv3 07/13] node: Add heterogenous memory access attributes Message-ID: <20190110173016.GC21095@localhost.localdomain> References: <20190109174341.19818-1-keith.busch@intel.com> <20190109174341.19818-8-keith.busch@intel.com> <87y37sit8x.fsf@linux.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87y37sit8x.fsf@linux.ibm.com> User-Agent: Mutt/1.9.1 (2017-09-22) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jan 10, 2019 at 06:07:02PM +0530, Aneesh Kumar K.V wrote: > Keith Busch writes: > > > Heterogeneous memory systems provide memory nodes with different latency > > and bandwidth performance attributes. Provide a new kernel interface for > > subsystems to register the attributes under the memory target node's > > initiator access class. If the system provides this information, applications > > may query these attributes when deciding which node to request memory. > > > > The following example shows the new sysfs hierarchy for a node exporting > > performance attributes: > > > > # tree -P "read*|write*" /sys/devices/system/node/nodeY/classZ/ > > /sys/devices/system/node/nodeY/classZ/ > > |-- read_bandwidth > > |-- read_latency > > |-- write_bandwidth > > `-- write_latency > > > > The bandwidth is exported as MB/s and latency is reported in nanoseconds. > > Memory accesses from an initiator node that is not one of the memory's > > class "Z" initiator nodes may encounter different performance than > > reported here. When a subsystem makes use of this interface, initiators > > of a lower class number, "Z", have better performance relative to higher > > class numbers. When provided, class 0 is the highest performing access > > class. > > How does the definition of performance relate to bandwidth and latency here?. The > initiator in this class has the least latency and high bandwidth? Can there > be a scenario where both are not best for the same node? ie, for a > target Node Y, initiator Node A gives the highest bandwidth but initiator > Node B gets the least latency. How such a config can be represented? Or is > that not possible? I am not aware of a real platform that has an initiator-target pair with better latency but worse bandwidth than any different initiator paired to the same target. If such a thing exists and a subsystem wants to report that, you can register any arbitrary number of groups or classes and rank them according to how you want them presented.