From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EF97C433E1 for ; Fri, 3 Jul 2020 22:22:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 38E1121473 for ; Fri, 3 Jul 2020 22:22:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="e5KnVuYr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 38E1121473 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 76B698D00A0; Fri, 3 Jul 2020 18:22:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 71B248D0010; Fri, 3 Jul 2020 18:22:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60B468D00A0; Fri, 3 Jul 2020 18:22:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id C532D8D0010 for ; Fri, 3 Jul 2020 18:22:31 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 481062839 for ; Fri, 3 Jul 2020 22:22:31 +0000 (UTC) X-FDA: 76998189702.13.quilt33_140624e26e95 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 1F98C18140B70 for ; Fri, 3 Jul 2020 22:22:31 +0000 (UTC) X-HE-Tag: quilt33_140624e26e95 X-Filterd-Recvd-Size: 4690 Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Fri, 3 Jul 2020 22:22:30 +0000 (UTC) Received: by mail-lj1-f172.google.com with SMTP id s9so38574301ljm.11 for ; Fri, 03 Jul 2020 15:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=RyL/27fhtr142h3y889Pn+A0n7ftUy7d4PNyw21qW/g=; b=e5KnVuYrtd9lFOZ6UyFiavj+At9RqXDaxTtIpWXIVj9J6+nChcPwDXZvM3iF+/0CxC WUtPRy++h69cTspgVxkptjYcSoLAUaIsrbindlPVv3qKBop6nPs3U4xEJKOZh9g4lE05 +/I4vC8wKlg4hK+TdAG6B5WW90Lrpj/1MsFh1xjsL76iF6kvPLUtZEDj2exbJZVXPaZX EWkFWRoAX312YN/0QVWfiJgt6/kvppJ5qYz4tWx4imDbqdr0LnLRpRhaCAXXsJ8Zw8nB fsAwGw6xw2W8EWq7olnYAfWzunOKEZfxiKfo7gns21sU142ovD5ujMA57hzxJwYnwDx6 Visw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=RyL/27fhtr142h3y889Pn+A0n7ftUy7d4PNyw21qW/g=; b=trY/7OZOoN8Cg7Et8YYoxbcTRG5FZRReJBd86f4K1l2IAGxqaNbinBPeTNepnPzGdo AOBWf/ESwsYBfyL2MtCgXxKLPZ1GnbodqA2wQJ2RbgPxNdfUMfQLc+MZ8wkBY5EU9L65 TkqxCJwLWoxko4m11k6J4h2QqR+EOqgv1I6Bip9bkG6EdtAW0KjHcA4L6q8CjC3PKOmz D+hVXlaNtDPHO8qN4+2sIK8rcAqCA4TcaPC6xafFeHZyohcDLOohC8H9jQLElG7eKiKK G/ujUw5RlmYbm1iJIYhFqwsyHQwd/gRswfN5fpUIoV3aXv/gztSRijrGx9htxHFvunA1 AI8g== X-Gm-Message-State: AOAM530w3oQG3oZwf57uW8b/EFFJ39mzsVCFdchbVHzhkj248e0mrQOi eAfti1ffDpgQOleyJogc87M= X-Google-Smtp-Source: ABdhPJz3DsPbDv8YQ80+y27DTaE0MqIpaJc2MR+AZNcfvQ0LRivXpbkYmgUfBhXF8foyZ9pwrcO9Yw== X-Received: by 2002:a2e:858e:: with SMTP id b14mr21823946lji.301.1593814949243; Fri, 03 Jul 2020 15:22:29 -0700 (PDT) Received: from pc636 (h5ef52e31.seluork.dyn.perspektivbredband.net. [94.245.46.49]) by smtp.gmail.com with ESMTPSA id r19sm2882312ljm.32.2020.07.03.15.22.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Jul 2020 15:22:28 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Sat, 4 Jul 2020 00:22:26 +0200 To: Matthew Wilcox Cc: Uladzislau Rezki , Linus Torvalds , peter enderborg , Linux Kernel Mailing List , Linux-MM , Andrew Morton , GregKroah-Hartmangregkh@linuxfoundation.org, Peter Zijlstra Subject: Re: nr_cpu_ids vs AMD 3970x(32 physical CPUs) Message-ID: <20200703222226.GA7472@pc636> References: <20200703155749.GA6255@pc636> <8a2a55e6-6087-e4bf-3d35-ed4b4c216369@sony.com> <20200703192807.GB5207@pc636> <20200703212047.GA6856@pc636> <20200703215157.GI25523@casper.infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200703215157.GI25523@casper.infradead.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 1F98C18140B70 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jul 03, 2020 at 10:51:57PM +0100, Matthew Wilcox wrote: > On Fri, Jul 03, 2020 at 11:20:47PM +0200, Uladzislau Rezki wrote: > > Some background: > > Actually i have been thinking about making vmalloc address space to > > be per-CPU, i.e. divide it to per-CPU address space making an allocation > > lock-less. It will eliminate a high lock contention. When i have done > > a prototype i noticed and realized that there is a silly issue with > > nr_cpu_ids on some systems. > > vfree() may happen on a different CPU from the one which called vmalloc(), > so I'm not sure you're going to get as large a win as you think you will. > Hmm.. According to my tests the difference is approximately 7x/10x but i also need to say as of now those tests are draft. Indeed vfree() can be done on different CPU, but i do not think it is a big issue. The main goal is to make the vmalloc() to be scaled to number of CPUs in a system. Because as number of CPUs increase as tight an allocation becomes. Doing vfree() on another CPU would be kind of noise(critical section is short), whereas other ones will be able to do progress because of their own locks. -- Vlad Rezki