From mboxrd@z Thu Jan 1 00:00:00 1970 Reply-To: kernel-hardening@lists.openwall.com Sender: Ingo Molnar Date: Sat, 7 Jan 2017 08:45:37 +0100 From: Ingo Molnar Message-ID: <20170107074537.GB13565@gmail.com> References: <20170106064900.GC28091@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: [kernel-hardening] Re: [RFC] x86/mm/KASLR: Remap GDTs at fixed location To: Andy Lutomirski Cc: Thomas Garnier , Arjan van de Ven , Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , Kees Cook , Borislav Petkov , Dave Hansen , Chen Yucong , Paul Gortmaker , Andrew Morton , Masahiro Yamada , Sebastian Andrzej Siewior , Anna-Maria Gleixner , Boris Ostrovsky , Rasmus Villemoes , Michael Ellerman , Juergen Gross , Richard Weinberger , X86 ML , "linux-kernel@vger.kernel.org" , "kernel-hardening@lists.openwall.com" List-ID: * Andy Lutomirski wrote: > > I looked back at the fixmap, and I can see a way it could be done (using > > NR_CPUS) like the other fixmap ranges. It would limit the number of cpus to > > 512 (there is 2M memory left on fixmap on the default configuration). That's > > if we never add any other fixmap on x64. I don't know if it is an acceptable > > number and if the fixmap region could be increased. (128 if we do your kvm > > trick, of course). > > IIRC we need 4096 CPUs. On 64-bit the limit is 8192 CPUs, and the SGI guys are relying on that up to the tune of 6144 cores already, and I'd say the 64-bit CPU count is likely to go up further with 5-level paging. On 32-bit the reasonable CPU limit is the number that the Intel 32-bit cluster computing nodes use. The latest public numbers are I think 36 'tiles' with each tile being a 2-CPU SMT core - i.e. a limit of 72 CPUs has to be maintained. (They'll obviously go to 64-bit as well so this problem will go away in a hardware generation or two.) So I'd say 128 CPUs on 32-bit should be a reasonable practical limit going forward. Right now our 32-bit limit is 512 CPUs IIRC, but I don't think any real hardware in production is reaching that. > P.S. Let's do the move to the fixmap, read/write as a separate patch. That will > make bisecting much easier. Absolutely, but this has to be within the same series, as the interim fixmap-only step is less secure in some circumstances: we are moving the writable GDT from a previously randomized location to a fixed location. Thanks, Ingo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S940938AbdAGHpu (ORCPT ); Sat, 7 Jan 2017 02:45:50 -0500 Received: from mail-wm0-f66.google.com ([74.125.82.66]:35220 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751218AbdAGHpn (ORCPT ); Sat, 7 Jan 2017 02:45:43 -0500 Date: Sat, 7 Jan 2017 08:45:37 +0100 From: Ingo Molnar To: Andy Lutomirski Cc: Thomas Garnier , Arjan van de Ven , Thomas Gleixner , Ingo Molnar , "H . Peter Anvin" , Kees Cook , Borislav Petkov , Dave Hansen , Chen Yucong , Paul Gortmaker , Andrew Morton , Masahiro Yamada , Sebastian Andrzej Siewior , Anna-Maria Gleixner , Boris Ostrovsky , Rasmus Villemoes , Michael Ellerman , Juergen Gross , Richard Weinberger , X86 ML , "linux-kernel@vger.kernel.org" , "kernel-hardening@lists.openwall.com" Subject: Re: [RFC] x86/mm/KASLR: Remap GDTs at fixed location Message-ID: <20170107074537.GB13565@gmail.com> References: <20170106064900.GC28091@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Andy Lutomirski wrote: > > I looked back at the fixmap, and I can see a way it could be done (using > > NR_CPUS) like the other fixmap ranges. It would limit the number of cpus to > > 512 (there is 2M memory left on fixmap on the default configuration). That's > > if we never add any other fixmap on x64. I don't know if it is an acceptable > > number and if the fixmap region could be increased. (128 if we do your kvm > > trick, of course). > > IIRC we need 4096 CPUs. On 64-bit the limit is 8192 CPUs, and the SGI guys are relying on that up to the tune of 6144 cores already, and I'd say the 64-bit CPU count is likely to go up further with 5-level paging. On 32-bit the reasonable CPU limit is the number that the Intel 32-bit cluster computing nodes use. The latest public numbers are I think 36 'tiles' with each tile being a 2-CPU SMT core - i.e. a limit of 72 CPUs has to be maintained. (They'll obviously go to 64-bit as well so this problem will go away in a hardware generation or two.) So I'd say 128 CPUs on 32-bit should be a reasonable practical limit going forward. Right now our 32-bit limit is 512 CPUs IIRC, but I don't think any real hardware in production is reaching that. > P.S. Let's do the move to the fixmap, read/write as a separate patch. That will > make bisecting much easier. Absolutely, but this has to be within the same series, as the interim fixmap-only step is less secure in some circumstances: we are moving the writable GDT from a previously randomized location to a fixed location. Thanks, Ingo