From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christian Borntraeger Subject: Re: [PATCH v6 02/13] array_index_nospec: sanitize speculative array de-references Date: Fri, 16 Feb 2018 09:55:05 +0100 Message-ID: <366bb116-e375-458c-2b63-1b26b4c8bdac@de.ibm.com> References: <151727412964.33451.17213780323040673404.stgit@dwillia2-desk3.amr.corp.intel.com> <151727414229.33451.18411580953862676575.stgit@dwillia2-desk3.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <151727414229.33451.18411580953862676575.stgit@dwillia2-desk3.amr.corp.intel.com> Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org To: Dan Williams , tglx@linutronix.de, mingo@kernel.org Cc: linux-arch@vger.kernel.org, Cyril Novikov , kernel-hardening@lists.openwall.com, Peter Zijlstra , Catalin Marinas , x86@kernel.org, Will Deacon , Russell King , linux-kernel@vger.kernel.org, Ingo Molnar , gregkh@linuxfoundation.org, "H. Peter Anvin" , torvalds@linux-foundation.org, alan@linux.intel.com List-Id: linux-arch.vger.kernel.org On 01/30/2018 02:02 AM, Dan Williams wrote: > array_index_nospec() is proposed as a generic mechanism to mitigate > against Spectre-variant-1 attacks, i.e. an attack that bypasses boundary > checks via speculative execution. The array_index_nospec() > implementation is expected to be safe for current generation CPUs across > multiple architectures (ARM, x86). > > Based on an original implementation by Linus Torvalds, tweaked to remove > speculative flows by Alexei Starovoitov, and tweaked again by Linus to > introduce an x86 assembly implementation for the mask generation. > > Co-developed-by: Linus Torvalds > Co-developed-by: Alexei Starovoitov > Suggested-by: Cyril Novikov > Cc: Russell King > Cc: Peter Zijlstra > Cc: Catalin Marinas > Cc: Will Deacon > Cc: Thomas Gleixner > Cc: Ingo Molnar > Cc: "H. Peter Anvin" > Cc: x86@kernel.org > Signed-off-by: Dan Williams > --- > include/linux/nospec.h | 72 ++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 72 insertions(+) > create mode 100644 include/linux/nospec.h > > diff --git a/include/linux/nospec.h b/include/linux/nospec.h > new file mode 100644 > index 000000000000..cf7be4bbcf17 > --- /dev/null > +++ b/include/linux/nospec.h > @@ -0,0 +1,72 @@ > +// SPDX-License-Identifier: GPL-2.0 > +// Copyright(c) 2018 Linus Torvalds. All rights reserved. > +// Copyright(c) 2018 Alexei Starovoitov. All rights reserved. > +// Copyright(c) 2018 Intel Corporation. All rights reserved. > + > +#ifndef _LINUX_NOSPEC_H > +#define _LINUX_NOSPEC_H Hmmm, shouldn't we include asm/barrier.h here? Otherwise users might or might not use the optimized variant depending on which headers are included by the users of array_index_nospec. > + > +/** > + * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise > + * @index: array element index > + * @size: number of elements in array > + * > + * When @index is out of bounds (@index >= @size), the sign bit will be > + * set. Extend the sign bit to all bits and invert, giving a result of > + * zero for an out of bounds index, or ~0 if within bounds [0, @size). > + */ > +#ifndef array_index_mask_nospec > +static inline unsigned long array_index_mask_nospec(unsigned long index, > + unsigned long size) > +{ > + /* > + * Warn developers about inappropriate array_index_nospec() usage. > + * > + * Even if the CPU speculates past the WARN_ONCE branch, the > + * sign bit of @index is taken into account when generating the > + * mask. > + * > + * This warning is compiled out when the compiler can infer that > + * @index and @size are less than LONG_MAX. > + */ > + if (WARN_ONCE(index > LONG_MAX || size > LONG_MAX, > + "array_index_nospec() limited to range of [0, LONG_MAX]\n")) > + return 0; > + > + /* > + * Always calculate and emit the mask even if the compiler > + * thinks the mask is not needed. The compiler does not take > + * into account the value of @index under speculation. > + */ > + OPTIMIZER_HIDE_VAR(index); > + return ~(long)(index | (size - 1UL - index)) >> (BITS_PER_LONG - 1); > +} > +#endif > + > +/* > + * array_index_nospec - sanitize an array index after a bounds check > + * > + * For a code sequence like: > + * > + * if (index < size) { > + * index = array_index_nospec(index, size); > + * val = array[index]; > + * } > + * > + * ...if the CPU speculates past the bounds check then > + * array_index_nospec() will clamp the index within the range of [0, > + * size). > + */ > +#define array_index_nospec(index, size) \ > +({ \ > + typeof(index) _i = (index); \ > + typeof(size) _s = (size); \ > + unsigned long _mask = array_index_mask_nospec(_i, _s); \ > + \ > + BUILD_BUG_ON(sizeof(_i) > sizeof(long)); \ > + BUILD_BUG_ON(sizeof(_s) > sizeof(long)); \ > + \ > + _i &= _mask; \ > + _i; \ > +}) > +#endif /* _LINUX_NOSPEC_H */ > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:49088 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1755229AbeBPIzP (ORCPT ); Fri, 16 Feb 2018 03:55:15 -0500 Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w1G8rpUq069660 for ; Fri, 16 Feb 2018 03:55:15 -0500 Received: from e06smtp12.uk.ibm.com (e06smtp12.uk.ibm.com [195.75.94.108]) by mx0b-001b2d01.pphosted.com with ESMTP id 2g5t55me1j-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Fri, 16 Feb 2018 03:55:14 -0500 Received: from localhost by e06smtp12.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 16 Feb 2018 08:55:12 -0000 Subject: Re: [PATCH v6 02/13] array_index_nospec: sanitize speculative array de-references References: <151727412964.33451.17213780323040673404.stgit@dwillia2-desk3.amr.corp.intel.com> <151727414229.33451.18411580953862676575.stgit@dwillia2-desk3.amr.corp.intel.com> From: Christian Borntraeger Date: Fri, 16 Feb 2018 09:55:05 +0100 MIME-Version: 1.0 In-Reply-To: <151727414229.33451.18411580953862676575.stgit@dwillia2-desk3.amr.corp.intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Message-ID: <366bb116-e375-458c-2b63-1b26b4c8bdac@de.ibm.com> Sender: linux-arch-owner@vger.kernel.org List-ID: To: Dan Williams , tglx@linutronix.de, mingo@kernel.org Cc: linux-arch@vger.kernel.org, Cyril Novikov , kernel-hardening@lists.openwall.com, Peter Zijlstra , Catalin Marinas , x86@kernel.org, Will Deacon , Russell King , linux-kernel@vger.kernel.org, Ingo Molnar , gregkh@linuxfoundation.org, "H. Peter Anvin" , torvalds@linux-foundation.org, alan@linux.intel.com Message-ID: <20180216085505.lMDaS6Dttw_X4EVEyIVDYZcQfsspXryRLKzjmo_YYdA@z> On 01/30/2018 02:02 AM, Dan Williams wrote: > array_index_nospec() is proposed as a generic mechanism to mitigate > against Spectre-variant-1 attacks, i.e. an attack that bypasses boundary > checks via speculative execution. The array_index_nospec() > implementation is expected to be safe for current generation CPUs across > multiple architectures (ARM, x86). > > Based on an original implementation by Linus Torvalds, tweaked to remove > speculative flows by Alexei Starovoitov, and tweaked again by Linus to > introduce an x86 assembly implementation for the mask generation. > > Co-developed-by: Linus Torvalds > Co-developed-by: Alexei Starovoitov > Suggested-by: Cyril Novikov > Cc: Russell King > Cc: Peter Zijlstra > Cc: Catalin Marinas > Cc: Will Deacon > Cc: Thomas Gleixner > Cc: Ingo Molnar > Cc: "H. Peter Anvin" > Cc: x86@kernel.org > Signed-off-by: Dan Williams > --- > include/linux/nospec.h | 72 ++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 72 insertions(+) > create mode 100644 include/linux/nospec.h > > diff --git a/include/linux/nospec.h b/include/linux/nospec.h > new file mode 100644 > index 000000000000..cf7be4bbcf17 > --- /dev/null > +++ b/include/linux/nospec.h > @@ -0,0 +1,72 @@ > +// SPDX-License-Identifier: GPL-2.0 > +// Copyright(c) 2018 Linus Torvalds. All rights reserved. > +// Copyright(c) 2018 Alexei Starovoitov. All rights reserved. > +// Copyright(c) 2018 Intel Corporation. All rights reserved. > + > +#ifndef _LINUX_NOSPEC_H > +#define _LINUX_NOSPEC_H Hmmm, shouldn't we include asm/barrier.h here? Otherwise users might or might not use the optimized variant depending on which headers are included by the users of array_index_nospec. > + > +/** > + * array_index_mask_nospec() - generate a ~0 mask when index < size, 0 otherwise > + * @index: array element index > + * @size: number of elements in array > + * > + * When @index is out of bounds (@index >= @size), the sign bit will be > + * set. Extend the sign bit to all bits and invert, giving a result of > + * zero for an out of bounds index, or ~0 if within bounds [0, @size). > + */ > +#ifndef array_index_mask_nospec > +static inline unsigned long array_index_mask_nospec(unsigned long index, > + unsigned long size) > +{ > + /* > + * Warn developers about inappropriate array_index_nospec() usage. > + * > + * Even if the CPU speculates past the WARN_ONCE branch, the > + * sign bit of @index is taken into account when generating the > + * mask. > + * > + * This warning is compiled out when the compiler can infer that > + * @index and @size are less than LONG_MAX. > + */ > + if (WARN_ONCE(index > LONG_MAX || size > LONG_MAX, > + "array_index_nospec() limited to range of [0, LONG_MAX]\n")) > + return 0; > + > + /* > + * Always calculate and emit the mask even if the compiler > + * thinks the mask is not needed. The compiler does not take > + * into account the value of @index under speculation. > + */ > + OPTIMIZER_HIDE_VAR(index); > + return ~(long)(index | (size - 1UL - index)) >> (BITS_PER_LONG - 1); > +} > +#endif > + > +/* > + * array_index_nospec - sanitize an array index after a bounds check > + * > + * For a code sequence like: > + * > + * if (index < size) { > + * index = array_index_nospec(index, size); > + * val = array[index]; > + * } > + * > + * ...if the CPU speculates past the bounds check then > + * array_index_nospec() will clamp the index within the range of [0, > + * size). > + */ > +#define array_index_nospec(index, size) \ > +({ \ > + typeof(index) _i = (index); \ > + typeof(size) _s = (size); \ > + unsigned long _mask = array_index_mask_nospec(_i, _s); \ > + \ > + BUILD_BUG_ON(sizeof(_i) > sizeof(long)); \ > + BUILD_BUG_ON(sizeof(_s) > sizeof(long)); \ > + \ > + _i &= _mask; \ > + _i; \ > +}) > +#endif /* _LINUX_NOSPEC_H */ >