From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42254C4828F for ; Thu, 8 Feb 2024 13:57:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B098F6B0075; Thu, 8 Feb 2024 08:57:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB9A96B0078; Thu, 8 Feb 2024 08:57:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95A0C6B007D; Thu, 8 Feb 2024 08:57:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 854E26B0075 for ; Thu, 8 Feb 2024 08:57:13 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4638080EEC for ; Thu, 8 Feb 2024 13:57:13 +0000 (UTC) X-FDA: 81768788346.24.8E7AF6D Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by imf12.hostedemail.com (Postfix) with ESMTP id 2735040011 for ; Thu, 8 Feb 2024 13:57:10 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RpJO+lyy; spf=pass (imf12.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707400631; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+/hAwBT47MJRRxh8alPQwYNx6sh+Hi8aT47CmZgK+yI=; b=wEIRzLgromJsJpQqov3lDAqAusi096Ys1xKuKNpQQ3GZw9SfMMFR3HT549t5LT83K5J2/J veZxt8tUQoyVWsvO3ZlzX8w/qr9AsHx79R8EHeGa0EkNKmILnwSFT25nAfFVJIfo99sCDu GvjnbixSMDXAFzkJCoeADJ3KlgThp5E= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RpJO+lyy; spf=pass (imf12.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.42 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707400631; a=rsa-sha256; cv=none; b=MKmSlk6jZlcci4Fh1rYkn6cVfTjirkdJATS+Ff+slYwdjDDgUSpckjWc4lpPZeSP6A2cBA DcYN7nFdd+ktVj4YXr0zD/In8JjoW/SZ91qpIxa3e+WjOHNJ70q0L19PaDurP1qLr9Ybax 4AD8/s9djbwOvobtK0PXuaeygyNnpEk= Received: by mail-lf1-f42.google.com with SMTP id 2adb3069b0e04-5116dfe1d9dso919367e87.3 for ; Thu, 08 Feb 2024 05:57:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1707400629; x=1708005429; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=+/hAwBT47MJRRxh8alPQwYNx6sh+Hi8aT47CmZgK+yI=; b=RpJO+lyyS5WbAn74jCSnw5qmcn480FiNdQXu4iN7bM1aJJE5r24APKwPQiyFxm6LUb ThU3/14kxVAGfNwG5mr6+QnejJoC6TTJJigFyIkhpgCjySV+B91GcqpkPtrT+gCFcMo4 0K5hVsGj9nSO6Sg01316KxKLW/09a8qt6jsV/8eK+epTxPrZv7wLvVZ+K+KcptR/bCMB TqmSNYaCJ1upyVXQm0G/bgF0Ki3/q3i+o1YvHXrvY0n8oEzeGnsEG4XXikjt03T4XZQy +cuZROFTxzn3NaHJtIUaJXhamQ3ZP9PXPG8yrQPnBgnszHDxqySfheINLaZnwn0Ua/Qi LzBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707400629; x=1708005429; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=+/hAwBT47MJRRxh8alPQwYNx6sh+Hi8aT47CmZgK+yI=; b=RBS0Fov/FR6CpMBqEuRcQ+fcTq+y6NSIbKh3n+iC/hRKmisLLrnWjH5ZiVfgw1rNd2 1pfXRr7eknfmtm/POz5lntjnG5kE31k+R4YQ+xCDJUS/vpmNgs/Xvi6I4HdSU7LdPKKZ RjcGmzQK85XbicjfnVFqD+EEwWZKEm45/WV9sk9r4edj4/JudqNyaEL26Q/ytvEzWVzj qDdDnWH4ZkTjjAGF5p+UJtGFwltvdY6SD2wBl4p/7+PY4daRIwXVb3Zvm3D8E2J3fPxG p1Nd61uukR+G+/gCSXRdHLwSMoYfdzxFf5nohhBtirFDdu4gx6+mt0WXt7pYDS8CpeQq UXHQ== X-Forwarded-Encrypted: i=1; AJvYcCWZ9MoOTkSnzvoKKHSt866KXtoCLvJ5Kc6fY7UTks8rlhf4GVkKuSHMBr40pmzBiB1HJ8Mw9yUrISc8dlUdAnCvoNI= X-Gm-Message-State: AOJu0YxaILl9t1HakvpyLArc4Ra5QbjLgn8swlpxSHO0tjlQGzR7Th0/ ux+15L5ddrsPOe/y64o8+LO1D9qvXrxkZCunYZ0jNi1IUNOEizUV X-Google-Smtp-Source: AGHT+IHjLb4jfqTrLqYuTbhBCjWANTP+gaLcCURf8aXIVTCkPV1j7zNN86Uc5xLsLXcUGT5Qz1ekgw== X-Received: by 2002:ac2:59c1:0:b0:511:5537:fb26 with SMTP id x1-20020ac259c1000000b005115537fb26mr7359213lfn.39.1707400629233; Thu, 08 Feb 2024 05:57:09 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWVATMuk3Idv+QnmvbT0ZEO29k05njN8wNzUl02jAYclIB1fcsOpUpe3kJvWF9R3LI7IlamaqseXdQF4Un6lCi5DjbUtMp065NaRE8xxRCUOeHLG6A7QgWzIRmfEN8zKuUxCTYdWWY/sE3IvQ42TOhRKZw18WKcZMcZpAox/S+s41YowJu9Vx/0RpCgRQ2x1oc6kS4vYROZx2EpRR6WY9rCdBcCUgFAsRK6WJnccjjQe72IcAjzn76bNJynw0XuHAARVI21DOVYwNacjxSiorbwZ1YPNA3Lun6bkTz32WZzl5FGdtbjIM0LGFOE/4/uc2LY/2HqT4vFsLmFkA64mU2F7XE7ivns07NQ+nZRqDC4jhUetAZ8AWE9qqK146xByXRJHwt9hg6wIgF1VtcFuTg= Received: from pc636 (host-90-233-221-0.mobileonline.telia.com. [90.233.221.0]) by smtp.gmail.com with ESMTPSA id k26-20020ac2457a000000b0051151b47c58sm7528lfm.97.2024.02.08.05.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Feb 2024 05:57:08 -0800 (PST) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Thu, 8 Feb 2024 14:57:06 +0100 To: Baoquan He Cc: "Uladzislau Rezki (Sony)" , linux-mm@kvack.org, Andrew Morton , LKML , Lorenzo Stoakes , Christoph Hellwig , Matthew Wilcox , "Liam R . Howlett" , Dave Chinner , "Paul E . McKenney" , Joel Fernandes , Oleksiy Avramchenko Subject: Re: [PATCH v3 07/11] mm: vmalloc: Offload free_vmap_area_lock lock Message-ID: References: <20240102184633.748113-1-urezki@gmail.com> <20240102184633.748113-8-urezki@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 2735040011 X-Rspam-User: X-Stat-Signature: pne8adt7d1bsoetpe9xt7wscsdmcdto7 X-Rspamd-Server: rspam01 X-HE-Tag: 1707400630-778551 X-HE-Meta: U2FsdGVkX1/2PobHDSwflwezKzfHMTcRhVZeATsKAKUgmMiL6d3jS7ryemd0iRg3Oe8Ka2N3manrHfE9/v8tpkmIuy8XB8Ej6R/xlt6R/Q/2QROmykuK56O1grcXIvFiJ4nEmQUwd+jjIiLHNsKFQVdQaEmY5W5bwScaX12WGZdyeywWvl59dbZdmh+eVYRoIL/AHOH2hu6b24p9+smuzt2FXHtU8OSBIX/QnyS4JjdIhYsN13KaIy+4Y6KCsZLOy5axwcjZuBWNRD8xUfuRkPPyRBFG9cjiVtXFemO3m6CdJsCm+bj5Ca7jIcRrqogdgNtyap9H0YXrPbIiJftgGkdNTFc4DFSw+7hYEL6wU5OA5c5OChSPZByOtGliKh4ZDMNLEA97GbBxd/+E/HclnRxaL6qJdzIkgAqYlvlJnwjNPob/ULYV4G+JAnIe+K25Y0IRhWmYehE05+YlmOPaZJHJ2XcsE+yuapFIXemViV50YDTkxODcS3rb/50dZQeqNqTIgx6tUBVO3vKmxXLU97VriFm/sp6x+6S/cSsOss9PF/9iv+i3pW+Hsrs5xHjcuT5hgc+ulZvleiLOP3zQywy2YbCiwLNNeUhb19N2iR4i9WOTXwjJLiFcnKjQdjW+o4O2f2p8Gm0jWAv4kwqQH/HGJSx2aQy0nTwTJFkCUt4Z/ZWedArJ9PtYZWRYewelmC+1AdiWhd33abQp788YU0qXAOB83hT4DYQZAjRtZ6MJh1jPnkwOxfyU58S69h1Kba8eM78HWmA9KdjlVZvf5mqMc6rP0xX7oFTlBYLvnVorhK5/XK7LMo931XZIFuV3M/uSy99w09jeLITDX0WbFcwtJzdby5mAjN8F4NA3um0AWpulAQ1kUEIPx9WZIdmqFXnfiLQE6b0F2ovUZADvqnXWYBtlkDG9LXl58VVP4Wv2c4spMmkldfdfMMmj3lzpR8ErRJ01nqOzouA2s9E 2fSElUBB i4WMeRNmBwBfvNoTBBYhfqCWhN345mRHLepDIPKkTUU70R8eX5QFGlX429UHoC2yIgqmacKjrnArJkIJ6UQpDFmlT3/EU82smsthkhYdTRwNkQZNaGVa+xaGjy5iceTOJLWEGd8Jmh8P4cAeTh5AdB3zsP571CUC4pefBo3Q0/bKwZFxprNbdgdQ9hSlSSfRq3ESpCHI/IlP0Ee7Hlcq8Anjl7Ljx1Y2TSY9bitrBQ/il32nLSmFgduu9TaKmNzed7KGsva0tN6q6mDSg++ji4h4HA4HLrv/zF1MBkywSKHvFtLt4Z0w8rlBWcmffT7NPiYUwwCp4AQZoXm1b1UGQHPMOHCSzGyKX2wlBWLAp8g48CDuLohvf3vh4nkuadRnCk9o52PBQ4bw8/jI9TxfTjiTl2jtorqOMrivRDb3upnxCckS0SB8sW4KfoYT+p7gfJWsaZQmkLLF7RtA4AtddDLVFr35OQnlXtd6d X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 08, 2024 at 08:25:23AM +0800, Baoquan He wrote: > On 01/02/24 at 07:46pm, Uladzislau Rezki (Sony) wrote: > ...... > > +static struct vmap_area * > > +node_alloc(unsigned long size, unsigned long align, > > + unsigned long vstart, unsigned long vend, > > + unsigned long *addr, unsigned int *vn_id) > > +{ > > + struct vmap_area *va; > > + > > + *vn_id = 0; > > + *addr = vend; > > + > > + /* > > + * Fallback to a global heap if not vmalloc or there > > + * is only one node. > > + */ > > + if (vstart != VMALLOC_START || vend != VMALLOC_END || > > + nr_vmap_nodes == 1) > > + return NULL; > > + > > + *vn_id = raw_smp_processor_id() % nr_vmap_nodes; > > + va = node_pool_del_va(id_to_node(*vn_id), size, align, vstart, vend); > > + *vn_id = encode_vn_id(*vn_id); > > + > > + if (va) > > + *addr = va->va_start; > > + > > + return va; > > +} > > + > > /* > > * Allocate a region of KVA of the specified size and alignment, within the > > * vstart and vend. > > @@ -1637,6 +1807,7 @@ static struct vmap_area *alloc_vmap_area(unsigned long size, > > struct vmap_area *va; > > unsigned long freed; > > unsigned long addr; > > + unsigned int vn_id; > > int purged = 0; > > int ret; > > > > @@ -1647,11 +1818,23 @@ static struct vmap_area *alloc_vmap_area(unsigned long size, > > return ERR_PTR(-EBUSY); > > > > might_sleep(); > > - gfp_mask = gfp_mask & GFP_RECLAIM_MASK; > > > > - va = kmem_cache_alloc_node(vmap_area_cachep, gfp_mask, node); > > - if (unlikely(!va)) > > - return ERR_PTR(-ENOMEM); > > + /* > > + * If a VA is obtained from a global heap(if it fails here) > > + * it is anyway marked with this "vn_id" so it is returned > > + * to this pool's node later. Such way gives a possibility > > + * to populate pools based on users demand. > > + * > > + * On success a ready to go VA is returned. > > + */ > > + va = node_alloc(size, align, vstart, vend, &addr, &vn_id); > > Sorry for late checking. > No problem :) > Here, if no available va got, e.g a empty vp, still we will get an > effective vn_id with the current cpu_id for VMALLOC region allocation > request. > > > + if (!va) { > > + gfp_mask = gfp_mask & GFP_RECLAIM_MASK; > > + > > + va = kmem_cache_alloc_node(vmap_area_cachep, gfp_mask, node); > > + if (unlikely(!va)) > > + return ERR_PTR(-ENOMEM); > > + } > > > > /* > > * Only scan the relevant parts containing pointers to other objects > > @@ -1660,10 +1843,12 @@ static struct vmap_area *alloc_vmap_area(unsigned long size, > > kmemleak_scan_area(&va->rb_node, SIZE_MAX, gfp_mask); > > > > retry: > > - preload_this_cpu_lock(&free_vmap_area_lock, gfp_mask, node); > > - addr = __alloc_vmap_area(&free_vmap_area_root, &free_vmap_area_list, > > - size, align, vstart, vend); > > - spin_unlock(&free_vmap_area_lock); > > + if (addr == vend) { > > + preload_this_cpu_lock(&free_vmap_area_lock, gfp_mask, node); > > + addr = __alloc_vmap_area(&free_vmap_area_root, &free_vmap_area_list, > > + size, align, vstart, vend); > > Then, here, we will get an available va from random location, but its > vn_id is from the current cpu. > > Then in purge_vmap_node(), we will decode the vn_id stored in va->flags, > and add the relevant va into vn->pool[] according to the vn_id. The > worst case could be most of va in vn->pool[] are not corresponding to > the vmap_nodes they belongs to. It doesn't matter? > We do not do any "in-front" population, instead it behaves as a cache miss when you need to access a main memmory to do a load and then keep the data in a cache. Same here. As a first step, for a CPU it always a miss, thus a VA is obtained from the global heap and is marked with a current CPU that makes an attempt to alloc. Later on that CPU/node is populated by that marked VA. So second alloc on same CPU goes via fast path. VAs are populated based on demand and those nodes which do allocations. > Should we adjust the code of vn_id assigning in node_alloc(), or I missed anything? Now it is open-coded. Some further refactoring should be done. Agree. -- Uladzislau Rezki