From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63165C433F5 for ; Wed, 25 May 2022 11:37:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A874E8D0003; Wed, 25 May 2022 07:37:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A36768D0002; Wed, 25 May 2022 07:37:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91FC58D0003; Wed, 25 May 2022 07:37:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 82E898D0002 for ; Wed, 25 May 2022 07:37:06 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4B8511083 for ; Wed, 25 May 2022 11:37:06 +0000 (UTC) X-FDA: 79504064052.26.83A4049 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id AC8914002B for ; Wed, 25 May 2022 11:36:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1653478625; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M3hAOXoo6D5ZjzfUZf6OBWb/9SBl/lq9MIa99qLJlp4=; b=Pk/CqoIy9U0AF1OdoeeGOVs0Pz3OogEFvzMUNEQl/V4ZIVK1tY7KqSBtoIngQkOhIDbZEb xzctnh0Gq4/G1QPWzukJh7Q8C/J9RzdP3TOG12yJsD/TSJ/qGJ+uteoeGP4Rj1Pgd6GIVt A+ms64vv2vDKolaZ/BtlJJzh4ZKTt+o= Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-255-E37w-lU-PNGn2Kz_tfuEYQ-1; Wed, 25 May 2022 07:37:03 -0400 X-MC-Unique: E37w-lU-PNGn2Kz_tfuEYQ-1 Received: by mail-lj1-f199.google.com with SMTP id m11-20020a2ea88b000000b0024db6246908so4419572ljq.22 for ; Wed, 25 May 2022 04:37:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=M3hAOXoo6D5ZjzfUZf6OBWb/9SBl/lq9MIa99qLJlp4=; b=zNMCr19XoOfQLpaXG52RDp8zQ7cJgl6l8kVACSkrydRQuAt/6Bmg5tku17ooZsdDRO tMi6PLae2YpRUkTYZ0KbsI0ST5etTSp5B6oubca6E6dRnsONfLLuIqQsSdSpOZMm1+Zw kaSsKUB/hc8NgcQghNwGLArIm2K9XMvrKsGgPJw6C3+Pyzfp0l5yNlPYXOmPq7olap61 9xF1GhaFOi6eUrnSKGZcky0AEWG7AITG9gK+opj/Kl90P5uFGcng+Zy/BxZupNDYVA4Q 18zs9Y/N7N2/A5IJl0gxov5Na8s6vsDdpg5jwl4zLesInygojoup2vYPCeVrRgVG4a8x ug7A== X-Gm-Message-State: AOAM5302YyJT99iv1ST/1eeBaiq2KPlBe1biE0/xbHavxzB///Zfv+xH xvH6ib2lXe2dLTR8AQE7brAlZDwxDFx+A5Hmaxoh1D3bM1IGzYCx6yZ6qSJQBmvZcvmqDLbWJWp W6AdTEw2DxA== X-Received: by 2002:a05:6512:b96:b0:478:5d7b:2e21 with SMTP id b22-20020a0565120b9600b004785d7b2e21mr15160533lfv.305.1653478621408; Wed, 25 May 2022 04:37:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwriGoreU0E3V6nOEG0nIMHCpIDDL6uaEAL1ptQBVTwiAybYB4y/6yMJsflVcf0aO7Cn3SYrg== X-Received: by 2002:a05:6512:b96:b0:478:5d7b:2e21 with SMTP id b22-20020a0565120b9600b004785d7b2e21mr15160495lfv.305.1653478621052; Wed, 25 May 2022 04:37:01 -0700 (PDT) Received: from [192.168.1.121] (91-145-109-188.bb.dnainternet.fi. [91.145.109.188]) by smtp.gmail.com with ESMTPSA id m8-20020ac24288000000b0047255d211ecsm3057974lfh.283.2022.05.25.04.36.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 25 May 2022 04:37:00 -0700 (PDT) Message-ID: <2d06932c-7d80-94c7-89ab-78375e58d3cc@redhat.com> Date: Wed, 25 May 2022 14:36:59 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: RFC: Memory Tiering Kernel Interfaces (v2) To: Aneesh Kumar K V , Ying Huang , Wei Xu Cc: Jonathan Cameron , Andrew Morton , Greg Thelen , Yang Shi , Linux Kernel Mailing List , Jagdish Gediya , Michal Hocko , Tim C Chen , Dave Hansen , Alistair Popple , Baolin Wang , Feng Tang , Davidlohr Bueso , Dan Williams , David Rientjes , Linux MM , Brice Goglin , Hesham Almatary References: <20220512160010.00005bc4@Huawei.com> <6b7c472b50049592cde912f04ca47c696caa2227.camel@intel.com> <6ce724e5c67d4f7530457897fa08d0a8ba5dd6d0.camel@intel.com> <594046f8-9ab3-786a-fc48-8a61f1238f52@linux.ibm.com> From: =?UTF-8?Q?Mika_Penttil=c3=a4?= In-Reply-To: <594046f8-9ab3-786a-fc48-8a61f1238f52@linux.ibm.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: AC8914002B X-Stat-Signature: wbxia7snofnpoy851qgoubcttecrbuz6 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="Pk/CqoIy"; spf=none (imf04.hostedemail.com: domain of mpenttil@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=mpenttil@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Rspamd-Server: rspam03 X-HE-Tag: 1653478609-894958 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 25.5.2022 13.01, Aneesh Kumar K V wrote: > On 5/25/22 2:33 PM, Ying Huang wrote: >> On Tue, 2022-05-24 at 22:32 -0700, Wei Xu wrote: >>> On Tue, May 24, 2022 at 1:24 AM Ying Huang wrote: >>>> >>>> On Tue, 2022-05-24 at 00:04 -0700, Wei Xu wrote: >>>>> On Thu, May 19, 2022 at 8:06 PM Ying Huang >>>>> wrote: >>>>>> > > ... > >> >> OK.  Just to confirm.  Does this mean that we will have fixed device ID, >> for example, >> >> GPU            memtier255 >> DRAM (with CPU)        memtier0 >> PMEM            memtier1 >> >> When we add a new memtier, it can be memtier254, or memter2?  The rank >> value will determine the real demotion order. >> >> I think you may need to send v3 to make sure everyone is at the same >> page. >> > > What we have implemented which we will send as RFC shortly is below. > > cd /sys/dekvaneesh@ubuntu-guest:~$ cd /sys/devices/system/ > kvaneesh@ubuntu-guest:/sys/devices/system$ pwd > /sys/devices/system > kvaneesh@ubuntu-guest:/sys/devices/system$ ls > clockevents  clocksource  container  cpu  edac  memory  memtier  mpic > node  power > kvaneesh@ubuntu-guest:/sys/devices/system$ cd memtier/ > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ pwd > /sys/devices/system/memtier > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ ls > default_rank  max_rank  memtier1  power  uevent > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat default_rank > 1 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cat max_rank > 3 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier$ cd memtier1/ > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ ls > nodelist  power  rank  subsystem  uevent > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat nodelist > 0-3 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cat rank > 1 > kvaneesh@ubuntu-guest:/sys/devices/system/memtier/memtier1$ cd > ../../node/node1/ > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ cat memtier > 1 > kvaneesh@ubuntu-guest:/sys/devices/system/node/node1$ > root@ubuntu-guest:/sys/devices/system/node/node1# echo 0 > memtier > root@ubuntu-guest:/sys/devices/system/node/node1# cat memtier > 0 > root@ubuntu-guest:/sys/devices/system/node/node1# cd ../../memtier/ > root@ubuntu-guest:/sys/devices/system/memtier# ls > default_rank  max_rank  memtier0  memtier1  power  uevent > root@ubuntu-guest:/sys/devices/system/memtier# cd memtier0/ > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat nodelist > 1 > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# cat rank > 0 > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# echo 4 > rank > bash: rank: Permission denied > root@ubuntu-guest:/sys/devices/system/memtier/memtier0# > Just to confirm, unlike today's demotion code, the demotion target allocation is planned to honor mempolicies? --Mika