From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3F49C5519F for ; Sun, 22 Nov 2020 23:16:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3670220776 for ; Sun, 22 Nov 2020 23:16:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="w8K+TGnS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3670220776 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A0D896B005D; Sun, 22 Nov 2020 18:16:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BCD96B0070; Sun, 22 Nov 2020 18:16:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D26D8D0001; Sun, 22 Nov 2020 18:16:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0085.hostedemail.com [216.40.44.85]) by kanga.kvack.org (Postfix) with ESMTP id 60F626B005D for ; Sun, 22 Nov 2020 18:16:28 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E7797363E for ; Sun, 22 Nov 2020 23:16:27 +0000 (UTC) X-FDA: 77513615214.05.wave35_0f0be2627360 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id C9B8C1801E21A for ; Sun, 22 Nov 2020 23:16:27 +0000 (UTC) X-HE-Tag: wave35_0f0be2627360 X-Filterd-Recvd-Size: 5127 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Sun, 22 Nov 2020 23:16:27 +0000 (UTC) Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1985C20789 for ; Sun, 22 Nov 2020 23:16:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1606086986; bh=hDV+6vxRY4eADdd/pIRKC6Rd7kdzU/zz7SC7dpNa+uM=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=w8K+TGnSTqkQJ1qBb/OBagUufRHfBMX+SbL4pa0hGQq+xClrowduurW8vapUkLG0u mZkiwYd0sl4jQTEhxiH+o4kI5lyfz47u6/ptdzW5OP1EtgfohsHhPrzbF0Yev0xUQc d/syrtm5X/q7bh+Jx74OzciKLOCaTxfrRz/jnVEU= Received: by mail-wm1-f50.google.com with SMTP id a3so15710264wmb.5 for ; Sun, 22 Nov 2020 15:16:26 -0800 (PST) X-Gm-Message-State: AOAM533XcstDwudfk/NqgsjhGI7n7ZFZ0G9IKHIRjgWDeWyjpzkApcfJ 0BTWhY6biD+Vs0JYdvwXX5aBCtl5+8R/ytAgZUuKUw== X-Google-Smtp-Source: ABdhPJzY5ZafJcEvAWHLlKqjLDdYKnT78P2PX32Ynzq62mz1lBcZ/cIjgS1QAhjuqlJ5G1M0qhIJGSIiqEjP/XdzziI= X-Received: by 2002:a7b:c92a:: with SMTP id h10mr20937078wml.138.1606086984696; Sun, 22 Nov 2020 15:16:24 -0800 (PST) MIME-Version: 1.0 References: <20201118194838.753436396@linutronix.de> <20201118204007.269943012@linutronix.de> <20201119093834.GH3306@suse.de> <20201119111411.GL3121378@hirez.programming.kicks-ass.net> <20201119182843.GA2414@hirez.programming.kicks-ass.net> <87tutkolq1.fsf@nanos.tec.linutronix.de> <20201120092928.GA3021@hirez.programming.kicks-ass.net> In-Reply-To: <20201120092928.GA3021@hirez.programming.kicks-ass.net> From: Andy Lutomirski Date: Sun, 22 Nov 2020 15:16:12 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [patch V4 4/8] sched: Make migrate_disable/enable() independent of RT To: Peter Zijlstra Cc: Thomas Gleixner , Linus Torvalds , Mel Gorman , LKML , "the arch/x86 maintainers" , Christoph Hellwig , Matthew Wilcox , Daniel Vetter , Andrew Morton , Linux-MM , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Daniel Bristot de Oliveira Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Nov 20, 2020 at 1:29 AM Peter Zijlstra wrote: > > As already stated, per-cpu page-tables would allow for a much saner kmap > approach, but alas, x86 really can't sanely do that (the archs that have > separate kernel and user page-tables could do this, and how we cursed > x86 didn't have that when meltdown happened). > > [ and using fixmaps in the per-cpu memory space _could_ work, but is a > giant pain because then all accesses need GS prefix and blah... ] > > And I'm sure there's creative ways for other problems too, but yes, it's > hard. > > Anyway, clearly I'm the only one that cares, so I'll just crawl back > under my rock... I'll poke my head out of the rock for a moment, though... Several years ago, we discussed (in person at some conference IIRC) having percpu pagetables to get sane kmaps, percpu memory, etc. The conclusion was that Linus thought the performance would suck and we shouldn't do it. Since then, though, we added really fancy infrastructure for keeping track of a per-CPU list of recently used mms and efficiently tracking when they need to be invalidated. We called these "ASIDs". It would be fairly straightforward to have an entire pgd for each (cpu, asid) pair. Newly added second-level (p4d/pud/whatever -- have I ever mentioned how much I dislike the Linux pagetable naming conventions and folding tricks?) tables could be lazily faulted in, and copies of the full 2kB mess would only be neeced when a new (cpu,asid) is allocated because either a flush happened while the mm was inactive on the CPU in question or because the mm fell off the percpu cache. The total overhead would be a bit more cache usage, 4kB * num cpus * num ASIDs per CPU (or 8k for PTI), and a few extra page faults (max num cpus * 256 per mm over the entire lifetime of that mm). The common case of a CPU switching back and forth between a small number of mms would have no significant overhead. On an unrelated note, what happens if you migrate_disable(), sleep for a looooong time, and someone tries to offline your CPU?