From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36587C3279B for ; Tue, 10 Jul 2018 14:28:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EB5DB208E7 for ; Tue, 10 Jul 2018 14:28:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EB5DB208E7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=surriel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933705AbeGJO2l (ORCPT ); Tue, 10 Jul 2018 10:28:41 -0400 Received: from shelob.surriel.com ([96.67.55.147]:37850 "EHLO shelob.surriel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933229AbeGJO2k (ORCPT ); Tue, 10 Jul 2018 10:28:40 -0400 Received: from imladris.surriel.com ([96.67.55.152]) by shelob.surriel.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.90_1) (envelope-from ) id 1fctd5-0003mn-E8; Tue, 10 Jul 2018 10:28:35 -0400 From: Rik van Riel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, luto@kernel.org, dave.hansen@intel.com, mingo@kernel.org, kernel-team@fb.com, efault@gmx.de, tglx@linutronix.de, songliubraving@fb.com, hpa@zytor.com Subject: [PATCH v5 0/7] x86,tlb,mm: make lazy TLB mode even lazier Date: Tue, 10 Jul 2018 10:28:26 -0400 Message-Id: <20180710142833.26231-1-riel@surriel.com> X-Mailer: git-send-email 2.14.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Song noticed switch_mm_irqs_off taking a lot of CPU time in recent kernels, using 1.9% of a 48 CPU system during a netperf run. Digging into the profile, the atomic operations in cpumask_clear_cpu and cpumask_set_cpu are responsible for about half of that CPU use. However, the CPUs running netperf are simply switching back and forth between netperf and the idle task, which does not require any changes to the mm_cpumask if lazy TLB mode were used. Additionally, the init_mm cpumask ends up being the most heavily contended one in the system, for no reason at all. Making really lazy TLB mode work again on modern kernels, by sending a shootdown IPI only when page table pages are being unmapped, we get back some performance. v5 of the series fixes the preempt bug and string overflow compiler warnings pointed out by Mike Galbraith. On memcache workloads on 2 socket systems, this patch series seems to reduce total system CPU use by 1-2%. On Song's netbench tests, CPU use in the context switch time is about cut in half. These patches also provide a little memory savings by shrinking the size of mm_struct, especially on distro kernels compiled with a gigantically large NR_CPUS.