From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A98C5C4363C for ; Mon, 10 May 2021 15:05:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 286A561482 for ; Mon, 10 May 2021 15:05:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 286A561482 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AA3B26B007B; Mon, 10 May 2021 11:05:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A2B926B007D; Mon, 10 May 2021 11:05:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CCAF6B007E; Mon, 10 May 2021 11:05:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 6A1AA6B007B for ; Mon, 10 May 2021 11:05:53 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 24CB19407 for ; Mon, 10 May 2021 15:05:53 +0000 (UTC) X-FDA: 78125646186.04.3B7C92B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf16.hostedemail.com (Postfix) with ESMTP id 8893480192E1 for ; Mon, 10 May 2021 15:05:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=U4gqJ7bNS9rSV6MmB2v7EvyoozJl8SDgfqOt8NDZWgg=; b=ZGsYavF1P0etL0/OcpiHxn8bpC cWk4y1c32RMRR3EZ+JjysG9SjS1+HvJtBj2CRB6eW1ibIduFgy4riLAGgppvMQu9VqULHAOSb2dur ekYB3o4Fnf/Pk9JOSWhN76Ij/1Zpl3OIySJhfdRkE7Chk6rtcH0AqekWVFt2qEnvQq88uLHnAflrS C4dIxZHJ+kJdCMycC5UqEPXMtDlYHuUONOOXOKI6If+19/ygUeXLRniG6I2F8ZM8kzesfSNKMRIKd YFFdotrZXOgNWzfQVo7/yp0GhJAsXuzC3ESn7WQhSVprxxGQDHHlQgSIUhA61m6JcInUcsHXDr5P9 6HD24veQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lg7R6-006HRE-Cb; Mon, 10 May 2021 15:03:49 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , David Rientjes Subject: [PATCH 2/3] mm/vmalloc: Use kvmalloc to allocate the table of pages Date: Mon, 10 May 2021 16:02:32 +0100 Message-Id: <20210510150233.1496689-2-willy@infradead.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210510150233.1496689-1-willy@infradead.org> References: <20210510150233.1496689-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ZGsYavF1; dmarc=none; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Stat-Signature: jnnnjja6nufptcpg1uc86pggs4qqporj X-Rspamd-Queue-Id: 8893480192E1 X-Rspamd-Server: rspam05 Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620659141-899510 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we're trying to allocate 4MB of memory, the table will be 8KiB in size (1024 pointers * 8 bytes per pointer), which can usually be satisfied by a kmalloc (which is significantly faster). Instead of changing this open-coded implementation, just use kvmalloc(). This improves the allocation speed of vmalloc(4MB) by approximately 5% in our benchmark. It's still dominated by the 1024 calls to alloc_pages_node(), which will be the subject of a later patch. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: David Rientjes --- mm/vmalloc.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index a13ac524f6ff..867c155c07e0 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2774,13 +2774,8 @@ static void *__vmalloc_area_node(struct vm_struct = *area, gfp_t gfp_mask, gfp_mask |=3D __GFP_HIGHMEM; =20 /* Please note that the recursion is strictly bounded. */ - if (array_size > PAGE_SIZE) { - pages =3D __vmalloc_node(array_size, 1, nested_gfp, node, - area->caller); - } else { - pages =3D kmalloc_node(array_size, nested_gfp, node); - } - + pages =3D kvmalloc_node_caller(array_size, nested_gfp, node, + (unsigned long)area->caller); if (!pages) { free_vm_area(area); warn_alloc(gfp_mask, NULL, --=20 2.30.2