From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC3AAC7EE2D for ; Tue, 23 May 2023 15:13:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59ECB6B0074; Tue, 23 May 2023 11:13:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 54EC66B0075; Tue, 23 May 2023 11:13:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4159C6B0078; Tue, 23 May 2023 11:13:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3107A6B0074 for ; Tue, 23 May 2023 11:13:24 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EA1B180674 for ; Tue, 23 May 2023 15:13:23 +0000 (UTC) X-FDA: 80821863486.08.60AFA26 Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by imf28.hostedemail.com (Postfix) with ESMTP id 44C65C0084 for ; Tue, 23 May 2023 15:12:34 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=qrvITLE4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684854755; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pq6A8hjAXuW6QduZxpJjlroh6Zzvo1r22L0X16yWMNU=; b=oh8aHPcrEUih3dHz0yHUhNO8h3HBSPEGU1C1bIceZ5Cknr7eaZz+AzLN7ExCmRUSyg8GMT DqZH2M73Qg8IHKTL81yK5qqVEEhbAORxURQXE75Tg+hUwk0sH/9Ct21cDSDV+Ux0XZaFF2 h6dTmGUq8zvbSKxwcI29Ma+qI9z/Szg= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=qrvITLE4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684854755; a=rsa-sha256; cv=none; b=GkyIGkAJjagNfPX3yy71Ec39sQFhHILQwBDhUKKvNlTis4llmwtH6VxPZrB2a/u6FQ7a8y wmAxmmveLv8QdSDnRCVhtYx+JR2NrAv9gatsOLBiDaILLpF139Fn3p15rV2ec4jaCbasPm Ck/VpoaLrR3S0becdahbAocV+lqMjYo= Received: by mail-lf1-f50.google.com with SMTP id 2adb3069b0e04-4f3b4ed6fdeso4171518e87.3 for ; Tue, 23 May 2023 08:12:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1684854753; x=1687446753; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=pq6A8hjAXuW6QduZxpJjlroh6Zzvo1r22L0X16yWMNU=; b=qrvITLE41FYdRADXj5t2RiPbwfhl/E/Fpj1Ve/+vGVELXDWQW+xXkesUiRGFoBZK7K WHj9XXEwJZ0htvvv7F7gsWmh4Tz8UVtsmGFZp0WtftmwKstEcKmUOKRlPj1cGh39F3PO Zd5cKfsGqWhokLLC7LF3+CvVEjKRUQw43hrk72/wkrzC0GGupeWd+aHRr1ZtnM+a8xmJ SeIOVHy10BPycFAtu4P46FItP2nyQAsGHHtAjKEVuIz1oCXtpj+bwct+90ARazmvl4sm FL4PSaZ2V8UEiI3wkc2HL44zLIs+VYeVjgN9SE0YGtWmKCk6AonEauR7t2phSowt3pXV iZmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684854753; x=1687446753; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=pq6A8hjAXuW6QduZxpJjlroh6Zzvo1r22L0X16yWMNU=; b=RE9KuvAWFRGYuxOqGDnT0738gmbuVzDsxegsbp5AnA5O8fPdlD5dwKXDHxyQk741Rh r+dRfsczxydMkwuFYHZcdWUxClhWPUYlyNM7XicPDwH0Zv2J1nHQvwirZNMPSrj5ps9I YSUf8JABV2SK72OyurqF2ZDhqYBMIfSk8w2VjvPJAPMgjZSI9+f5tp8igtYe+uRcD4JP Vq1ttIumLhsJfvNYh6qDSrX87mwDnBzK7dYMrXMfcY+4FVrXigfdck3LyVHGKH5ymZVR 8SNRM6Y3TJ6uWLS3Vcl5p3CetVLMyu/Ynjg83T1BysiNvWrYqcYpBC1t4+tbnWEMsvyD ZAMA== X-Gm-Message-State: AC+VfDxVuwCHO8duiMWponEWEBqrg0Xr2DyJVIPxiIK6LpM8xmghBcym X6jtkZiKFmym8cQ5w194EG0= X-Google-Smtp-Source: ACHHUZ66zIV8ytIKAe460yHt/3Wcypv69k5r3Y0Fc5gQE5tdr5oMlGTwVtmBN98sNrb43tfMilDzAQ== X-Received: by 2002:a19:f519:0:b0:4f3:af46:1941 with SMTP id j25-20020a19f519000000b004f3af461941mr3963289lfb.34.1684854753092; Tue, 23 May 2023 08:12:33 -0700 (PDT) Received: from pc636 (host-90-235-19-70.mobileonline.telia.com. [90.235.19.70]) by smtp.gmail.com with ESMTPSA id c18-20020a197612000000b004f3b520e0adsm1370711lff.107.2023.05.23.08.12.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 08:12:32 -0700 (PDT) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 23 May 2023 17:12:30 +0200 To: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: "Uladzislau Rezki (Sony)" , linux-mm@kvack.org, Andrew Morton , LKML , Baoquan He , Lorenzo Stoakes , Christoph Hellwig , Matthew Wilcox , "Liam R . Howlett" , Dave Chinner , "Paul E . McKenney" , Joel Fernandes , Oleksiy Avramchenko Subject: Re: [PATCH 0/9] Mitigate a vmap lock contention Message-ID: References: <20230522110849.2921-1-urezki@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 44C65C0084 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: oss7tbq9k1eoqb3a753uwbht7hiennjs X-HE-Tag: 1684854754-954258 X-HE-Meta: U2FsdGVkX19pXAR2VND2v1Edq14XV067veKdzeGQBdqaOhEvrnKumNSCFVc5fZw3FwJ0xt9qk9iS0ny3QapoOlAwUltgKSIQntxFPPg8qMBm9lHxnMJqvB8Nmz/bi/8EQc04HXKsHMr1Q8EKHkRbc9SfyqgE/s0mDiQq1UWmPpYlga1ROzdlWv1RAOQuJoBUllKpWDHVgDE0lt8Nyl/M2fz+2JFFBejG07pJX+GiSYsPxOwXP4WTd98C7Cfe20satSQiK3eTgf2FLlPn9QUimbnFzM3kuQkwl33MEBsOU2jLeiDwARKEb9hnmPq8Kj5sQjXACC/wKpMeo8O6T3ZO2BnSZgCrtKQEUr3brD5E3GF5DaSgYd2IB9VdKZmb+2/tzRMZRBTGtSnEUHMFVBxNV3hQqJfeHJrnMse1ijz6YsH1U6mhygNQkHMOszdBct6lpFPXwQut6PJof2OTI6bOKDSKL3BgNdpmknA3AmkO3JnQvufKzBZwZEgZxRLdz3XBhLDW+gMu9V4Th9pGMkAW0j+viAKV4LyDnUjGO390NwtWcWGda9UMFq2f+ZZRXMi+FlDI6hRmlgyoZ8+UoIDDCe6ibWm05cngwl1PhQR8CMbD2S/oxRjYTV85VGojbjYpVxx+G7RgMMlALHkA7qznWi3izL9/sNNsdG/P3hVxgN3F4Y2BNTgrCT1BsangqtfCb8z1kweMnEL0jJCS0yWSo31zqUmI/QKyw/ObT3JShXumG3bgxwdwiB/fIJK6p+nMcOKfTX41jKI/kQRMxF8u+cboYuvuTgvg0XXTCOEHOIwL4DR2YyqXSWBJnJY67tKQei+URsNKxDJDViCW4JFYCxOPeJ8xUtX8mnxi5v4q9u2VyjkGzbh5p4BGplllztokYFP56opJXIW2kiXAWAbQsRQsymxDojQM9JefncrMh93ModOyXLQeB7dFoFCOIQeX0frCuuplXFWcG2DGjTh tsm2Rm9n H29ESMzZPNr7U4qKw0mIElWfLyhiSfxavhExzXXuG7/ojGxSzelsdodbshCGcZ4/9OnvYuIfohNb+7E5ugFO3BsdBwY8To4yYd2IVaa1SPewU/VVLgteYoGKEg87H2OQ4umXjddqnxF2+xQzy8dPBfvMQ638iIGkgm8szR/oFys0ZKO3Z7tWBci1p84aZbcY8zc1URS25r8uqfAi9iHuzufhEVpVhNPzXFl2I5Gh0HaZS+PGWqQ9LDxLdVL+CLn3cOVxJj4x6xKGnbSWz9n7NL54U5mMgnc8WaIUBLUw8Uz2uDQCZ9z5rxYDQ5qtvZ0I4TaW6qyak+cHwwRuO7X4nvw8/Rm9hl5CI0aYr25qkcW7Gnv9CuGQF6JHjfgvGXZ1E9fCILo0EDrRm3ZQMCHoAUE0U8LO4wU+lReVzX7iFWpSgbQB1233k2zzK6mu/b5+KyC7sRLlKZMtZnavztSEPI5X2M9ecuCd0geUE7aKmXDGbmmXKYT2JqUL1nNsP5BB6c+itiCUh2+GkRv5lnoNWPfyx0F3hPufZ55uaoBg4SlTnMOc5aYMKuICvdWsArMLq7vkc1sMtcCzLIbl6Z53sUHK2esPMly3O/CCsjsMOXu+JXi8Z3lQyALq7k2Zus86ZCUmo4IW7cbuP1XEL7fB/mzpnQNJsRtBF+h014W1DSu/tBZ4y3P8D/F0/VoAnvaE6W0HzZY4kqJq1sU2tFNikeDsZNUJ9BPI7S9RP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 23, 2023 at 08:59:05PM +0900, Hyeonggon Yoo wrote: > On Mon, May 22, 2023 at 01:08:40PM +0200, Uladzislau Rezki (Sony) wrote: > > Hello, folk. > > > > 1. This is a followup of the vmap topic that was highlighted at the LSFMMBPF-2023 > > conference. This small serial attempts to mitigate the contention across the > > vmap/vmalloc code. The problem is described here: > > > > Hello Uladzislau, thank you for the work! > > > wget ftp://vps418301.ovh.net/incoming/Fix_a_vmalloc_lock_contention_in_SMP_env_v2.pdf > > I ran the exactly same command but couldn't download the file, did I > miss something? > wget ftp://vps418301.ovh.net/incoming/Mitigate_a_vmalloc_lock_contention_in_SMP_env_v2.pdf Sorry, i renamed the file name :) > $ wget ftp://vps418301.ovh.net/incoming/Fix_a_vmalloc_lock_contention_in_SMP_env_v2.pdf > [...] > ==> PASV ... done. ==> RETR Fix_a_vmalloc_lock_contention_in_SMP_env_v2.pdf ... > No such file `Fix_a_vmalloc_lock_contention_in_SMP_env_v2.pdf'. > > > The material is tagged as a v2 version. It contains extra slides about testing > > the throughput, steps and comparison with a current approach. > > > > 2. Motivation. > > > > - The vmap code is not scalled to number of CPUs and this should be fixed; > > - XFS folk has complained several times that vmalloc might be contented on > > their workloads: > > > > > > commit 8dc9384b7d75012856b02ff44c37566a55fc2abf > > Author: Dave Chinner > > Date: Tue Jan 4 17:22:18 2022 -0800 > > > > xfs: reduce kvmalloc overhead for CIL shadow buffers > > > > Oh, let me count the ways that the kvmalloc API sucks dog eggs. > > > > The problem is when we are logging lots of large objects, we hit > > kvmalloc really damn hard with costly order allocations, and > > behaviour utterly sucks: > > based on the commit I guess xfs should use vmalloc/kvmalloc is because > it allocates large buffers, how large could it be? > They use kvmalloc(). When the page allocator is not able to serve a request they fallback to vmalloc. At least what i see, the sizes are: from 73728 up to 1048576, i.e. 18 pages up to 256 pages. > > 3. Test > > > > On my: AMD Ryzen Threadripper 3970X 32-Core Processor, i have below figures: > > > > 1-page 1-page-this-patch > > 1 0.576131 vs 0.555889 > > 2 2.68376 vs 1.07895 > > 3 4.26502 vs 1.01739 > > 4 6.04306 vs 1.28924 > > 5 8.04786 vs 1.57616 > > 6 9.38844 vs 1.78142 > > > > > 29 20.06 vs 3.59869 > > 30 20.4353 vs 3.6991 > > 31 20.9082 vs 3.73028 > > 32 21.0865 vs 3.82904 > > > > 1..32 - is a number of jobs. The results are in usec and is a vmallco()/vfree() > > pair throughput. > > I would be more interested in real numbers than synthetic benchmarks, > Maybe XFS folks could help performing profiling similar to commit 8dc9384b7d750 > with and without this patchset? > I added Dave Chinner to this thread. But. The contention exists. Apart of that per-cpu-KVA allocator can go away if we make it generic instead. > By the way looking at the commit, teaching __p?d_alloc() about gfp > context (that I'm _slowly_ working on...) could be nice for allowing > non-GFP_KERNEL kvmalloc allocations, as Matthew mentioned. [1] > > Thanks! > > [1] https://lore.kernel.org/linux-mm/Y%2FOHC33YLedMXTlD@casper.infradead.org > Thanks! -- Uladzisdlau Rezki