From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB362C4332F for ; Tue, 20 Dec 2022 18:46:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5AC278E0002; Tue, 20 Dec 2022 13:46:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 55B8F8E0001; Tue, 20 Dec 2022 13:46:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4248C8E0002; Tue, 20 Dec 2022 13:46:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 331F38E0001 for ; Tue, 20 Dec 2022 13:46:42 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E82AA80C7D for ; Tue, 20 Dec 2022 18:46:41 +0000 (UTC) X-FDA: 80263565802.02.8417916 Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) by imf06.hostedemail.com (Postfix) with ESMTP id F3C34180012 for ; Tue, 20 Dec 2022 18:46:39 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AgLFddmK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of urezki@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671562000; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=j9wQjMhTPfA51Xzi4rhcgp0eEQkDkG0Es94rv3RfqG8=; b=sWcw584lcInUkDML6MGH1gGlL9M+m1+1wmlpmfM3Sku3u5YgjkKWNDfxvA+WsjSIscH1ti zVoDZC7uVXBX88YuPMz6bPPuMkuS114rpsvpykc1QKwZH2tS3VN5FnMjeiMYpdyRCHrKKj ukWg6qOckcDu3V89AHoEdp7xkCeqmCg= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=AgLFddmK; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf06.hostedemail.com: domain of urezki@gmail.com designates 209.85.208.178 as permitted sender) smtp.mailfrom=urezki@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671562000; a=rsa-sha256; cv=none; b=0V0zJTYfYLTOwWXXUos7zfnl9Q/e3ecxtUi7Um8OH2Q41Wh3UqoZzQER03dYiuY63VXrft G+ItnLVwEbyILnVHkKOKJfTKETEkd4qEUfxI2C8KiBXoBnoQ54MrSpEvKNtgRuf3Nul5JB ZnY28YYT4ZVoqM7EcJGsvY4eDOyRn/o= Received: by mail-lj1-f178.google.com with SMTP id y4so13316051ljc.9 for ; Tue, 20 Dec 2022 10:46:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=j9wQjMhTPfA51Xzi4rhcgp0eEQkDkG0Es94rv3RfqG8=; b=AgLFddmKipaGcp9ZSzKmco925TMpUpCJfwlE60D5oEogokYyztcul/PLny/Cv8ebCi xXS3GYbwK6Iw8+vHQZmRPcSO01ACqhH7iHx+EKwjyqcrMoIcFRXi7aly9jS28P8gvAny gPS0e2QIwKjZOV+1Rm08EAKsss1j0dioRmhCMusspySKRRg3aKsUcN6+HhjGb/+IpWVR LzlPgzW0VSpACA/V2iTcBUIUaQ4EVDMRdWLRT0LSLYVVT8WjN9K/9Vq6fCkLe6kqLAfA bTbl46ofcnuIt8I5wqA6ODxsusIDZI0xnWNNqrQk4uXkreeAlGunoadLeKZo9BAVxqit StYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=j9wQjMhTPfA51Xzi4rhcgp0eEQkDkG0Es94rv3RfqG8=; b=SPqJCd6b7x6cO2nvER6UgVRD4kpxORqUYZmR5ByQN/B+VWKKRVJPH77UOVvYn40tVe 6wmiz0QrzPIO5W0nWmezfEabNI8FIYeG+dy0mGKp14+92/HtFDMjXEb39iwZvgbeyMqe KDYOoosSTzAXkQR1ij9U6mbnCwVed71AvSIod8TtH0A+Nfq2a1wY83uiRWgPNXF/AxAR 3ePHtpBzeoT99Hru4EzRvDUc6BTjilFP9JHRSxLEr0lvY4kwW2PPzYDijQ2uGYyOx7zP fzAohQ6k9FOOX4w9QhBWf4lMpA5xn7EhdEZwH+qbDNerPK0gkE4MSZdVS0NoW3LNCAUa p/TA== X-Gm-Message-State: ANoB5pnYiWdIzZsn+ugd87GAYwBlCL8Q/eYjNS3VFcKvO9NtfhsfBDRi 7SanRP8dZ1nhQkp1+3k7VZU= X-Google-Smtp-Source: AA0mqf7iFWtfPjL5WRmxy0Hgv3y/2Lm+9KRUsvsOyNxddWr8mFHVmH80woIXRa1Yb+l0VJKa+9MkGA== X-Received: by 2002:a05:651c:238a:b0:27b:4c80:d07b with SMTP id bk10-20020a05651c238a00b0027b4c80d07bmr12275528ljb.52.1671561998471; Tue, 20 Dec 2022 10:46:38 -0800 (PST) Received: from pc636 (host-90-233-218-120.mobileonline.telia.com. [90.233.218.120]) by smtp.gmail.com with ESMTPSA id e14-20020a05651c038e00b00277025ddc9esm1104057ljp.54.2022.12.20.10.46.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Dec 2022 10:46:38 -0800 (PST) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Tue, 20 Dec 2022 19:46:36 +0100 To: Andrew Morton Cc: Andrew Morton , linux-mm@kvack.org, LKML , Baoquan He , Christoph Hellwig , Matthew Wilcox , Nicholas Piggin , Oleksiy Avramchenko Subject: Re: [PATCH 1/2] mm: vmalloc: Avoid a double lookup of freed VA in a tree Message-ID: References: <20221220182704.181657-1-urezki@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: F3C34180012 X-Stat-Signature: 1eq35kbtw4na1c1igp99ze6an53j5tq5 X-HE-Tag: 1671561999-798883 X-HE-Meta: U2FsdGVkX1/rofm5us11G1sbZHmUBmd+Ex7FtZnVsgVvfKcTBW5Lro34IdjkkMAbxjF8PEwdLQOX8dd5ysftGzV7iSyLefvjG3NYdB0e+9pHSmG94vTuwxctZ9B8MeWl9HbXoToOMoXENc/eM3WqPn7VS3kc2/zQegIpAgTKnBLPrUSgKA7hF3BCat4s6eZHD7JYxHekSkBoJGw/D1EfZLcEisUQXZ0ihqb5ALZgSFvJK+X0RqajP3HZ2NuvaZPGof4/etPRDvdZ6z0/RJ+qyYKMsyWc/Zsa4cxJ/KwbygdgMxEd1w2ddIpSP0fJ4CFF+eisiVaQt/DdJaJEeI+U3PM3V3kJsty/y9A5bnm/h+bP851+T+Mlsoo/apO929O6KbFiiqO4JvRNpJ5w4IiKZZSwe774HWRK9FbeREizP4OUMuX/r4++hgwKV3DPw7/wOOSUfLHDNQLjraLrMZWw4sHt8PQGfQyfWEtLQeL0tj+nUOvD1Ej9xCV+pl0V6loa1+SSCsRNmtjyDU5hs7tlvRhznjzogvPRYh/vMSoLjFr0LcT+U3AYX08LZfR1Yr8ioHHI59s9kcInPbP3RT0HMwhaMnqb3XUogZj8bF1Nr1HTZlrIYOvI2AyQXod1NGg8leuFN4yr4iiYQyL3++Vi89VARtbhN2RYDkdvhtzNNtAhjfXoQxnM6DAJCivVQDZic24kspCiqlz06zk9NLWWhJK6xFRNI6Kw1RldXU5odH9zjLBwfAVGoKIX9/4UFCcch4bxelorVv0n6U+VaNvaUmD1rYBg9c5UmnnWgpCjf/Y1NNeZnuDFqvBXSGixYBfF8Q1a7NH5IYzn3tQ7iYbNI7muwUTDyzUDbk8ajj7vIDZYJhAvk41P3Dor4vUGXMXCIh72LGfTr7VVOJVPjbn22kz3NMn47UW/H/z7aFlOHcIBQ1hwbZzROZbcBK/NbRRKiUg/no31yx71hKviE2g C6s5goGN XSZMJdEbZCu3t+jIPVuo78DyE2S/2kEQIkU+neb9uEtbnNFcvy6EggLkOVF/bGntY4tfKOexVIw8Ok7EVAWreHobWpg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Dec 20, 2022 at 07:45:20PM +0100, Uladzislau Rezki wrote: > On Tue, Dec 20, 2022 at 07:27:03PM +0100, Uladzislau Rezki (Sony) wrote: > > When a VA is freed over a main path, for example by invoking > > the vfree() function, a tree is accessed two times what is odd: > > > > vfree(): > > __vunmap() > > __find_vmap_area() > > vm_remove_mappings() > > remove_vm_area() > > __find_vmap_area() > > > > __find_vmap_area() are called two times. Fix it by introducing > > a find_unlink_vmap_area() helper that finds and un-links a VA > > from a tree. > > > > Performance test results on a single CPU: > > > > - fix_size_alloc_test loops: 1000000 avg: 476847 usec > > - full_fit_alloc_test loops: 1000000 avg: 806746 usec > > - long_busy_list_alloc_test loops: 1000000 avg: 13552093 usec > > - random_size_alloc_test loops: 1000000 avg: 7441322 usec > > - fix_align_alloc_test loops: 1000000 avg: 1411132 usec > > All test took worker0=87650866284 cycles > > > > - fix_size_alloc_test loops: 1000000 avg: 490713 usec > > - full_fit_alloc_test loops: 1000000 avg: 579162 usec > > - long_busy_list_alloc_test loops: 1000000 avg: 10485448 usec > > - random_size_alloc_test loops: 1000000 avg: 5824449 usec > > - fix_align_alloc_test loops: 1000000 avg: 984735 usec > > All test took worker0=67952362802 cycles > > > > Signed-off-by: Uladzislau Rezki (Sony) > > --- > > mm/vmalloc.c | 40 ++++++++++++++++++++++++++++------------ > > 1 file changed, 28 insertions(+), 12 deletions(-) > > > > diff --git a/mm/vmalloc.c b/mm/vmalloc.c > > index 9e30f0b39203..0fc38c36e0df 100644 > > --- a/mm/vmalloc.c > > +++ b/mm/vmalloc.c > > @@ -1825,9 +1825,14 @@ static void free_vmap_area_noflush(struct vmap_area *va) > > unsigned long va_start = va->va_start; > > unsigned long nr_lazy; > > > > - spin_lock(&vmap_area_lock); > > - unlink_va(va, &vmap_area_root); > > - spin_unlock(&vmap_area_lock); > > + /* > > + * A free_vmap_block() is left. It is NOT a main free path. > > + */ > > + if (!list_empty(&va->list)) { > > + spin_lock(&vmap_area_lock); > > + unlink_va(va, &vmap_area_root); > > + spin_unlock(&vmap_area_lock); > > + } > > > > nr_lazy = atomic_long_add_return((va->va_end - va->va_start) >> > > PAGE_SHIFT, &vmap_lazy_nr); > > @@ -1871,6 +1876,19 @@ struct vmap_area *find_vmap_area(unsigned long addr) > > return va; > > } > > > > +static struct vmap_area *find_unlink_vmap_area(unsigned long addr) > > +{ > > + struct vmap_area *va; > > + > > + spin_lock(&vmap_area_lock); > > + va = __find_vmap_area(addr, &vmap_area_root); > > + if (va) > > + unlink_va(va, &vmap_area_root); > > + spin_unlock(&vmap_area_lock); > > + > > + return va; > > +} > > + > > /*** Per cpu kva allocator ***/ > > > > /* > > @@ -2236,7 +2254,7 @@ void vm_unmap_ram(const void *mem, unsigned int count) > > return; > > } > > > > - va = find_vmap_area(addr); > > + va = find_unlink_vmap_area(addr); > > BUG_ON(!va); > > debug_check_no_locks_freed((void *)va->va_start, > > (va->va_end - va->va_start)); > > @@ -2607,21 +2625,16 @@ struct vm_struct *remove_vm_area(const void *addr) > > > > might_sleep(); > > > > - spin_lock(&vmap_area_lock); > > - va = __find_vmap_area((unsigned long)addr, &vmap_area_root); > > - if (va && va->vm) { > > + va = find_unlink_vmap_area((unsigned long) addr); > > + if (va) { > > struct vm_struct *vm = va->vm; > > > > - va->vm = NULL; > > - spin_unlock(&vmap_area_lock); > > - > > kasan_free_module_shadow(vm); > > free_unmap_vmap_area(va); > > > > return vm; > > } > > > > - spin_unlock(&vmap_area_lock); > > return NULL; > > } > > > > @@ -2690,6 +2703,7 @@ static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) > > static void __vunmap(const void *addr, int deallocate_pages) > > { > > struct vm_struct *area; > > + struct vmap_area *va; > > > > if (!addr) > > return; > > @@ -2698,7 +2712,9 @@ static void __vunmap(const void *addr, int deallocate_pages) > > addr)) > > return; > > > > - area = find_vm_area(addr); > > + va = find_unlink_vmap_area((unsigned long)addr); > > + area = va->vm; > > + > > if (unlikely(!area)) { > > WARN(1, KERN_ERR "Trying to vfree() nonexistent vm area (%p)\n", > > addr); > > -- > > 2.30.2 > > Will send a v2. -- Uladzislau Rezki