From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3696C20CCCF for ; Thu, 23 Jan 2025 19:08:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737659289; cv=none; b=Iugh4kSWA8eJGQNmh4d5SS0b+x0Li0hF4LsitvnQj/X291FVsAIFlYHw+L+CKmWShA9qJ4H8wJa5FskEp89xRHsnHGzwb110gi0GvDna1gNHCeCnKWuB2rXMgxM1fRJa4j2UwcNOo+pItnxxxGEfrDG2VEg8bUVGPQORAgxHp4s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737659289; c=relaxed/simple; bh=1iuccXO6FFsp5UwzDf43h54JAyW3RNqTAAUFOfKKvcw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fTAQ/NW0WxQx+/BJO/xV2O00+8MYFe08fw2pLhNpw31mGlzdsDJF8mstj3RLZRduCWVmIqBdW+NJ5/hVaYFjSdC9mbXXFegaMxH6sPa0ZENxdxUDQycecIjIjsBcSSyKE28S6LBEDQJ5WXfVaL8tq9bbMH/vin2iRb8l5E37Hao= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B7QmJd5M; arc=none smtp.client-ip=209.85.160.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B7QmJd5M" Received: by mail-qt1-f172.google.com with SMTP id d75a77b69052e-4678afeb133so21730611cf.0 for ; Thu, 23 Jan 2025 11:08:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737659286; x=1738264086; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wBNgO0CHxquZoEGCsy2nYv6ostqVYQH8nnC57bRQXgY=; b=B7QmJd5MBVhGjbkGjKKHtBbTJjLOSpCqAh1+aj8qGL+cBG6XL7lfzWY11wSdnoXwe1 tONwnte74vjceMgONDTZs80kkFTYEglSN1EqbgJmHihspbgWguf1RvlBah6kwqDo0VNR cKfO8xwKKl6SONYaiQkBdDNuV9hTCnJkJblmQSAkRR7krF++J6F861/tW6tB92NMTVsW /PWtLkeueoFH/erzIVRPzS28PoNJuG4o6oQBtzTUC++2WpRWGrQDAchUfRXVwTTWAw0p iThmDuUFeUXqVGRs0KiBuL6ls6jKJ8yHxTZpX+6x42fxIK43xhLtNABa26fc5+hecY1u nqIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737659286; x=1738264086; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wBNgO0CHxquZoEGCsy2nYv6ostqVYQH8nnC57bRQXgY=; b=cUWxhGGUYd/NvDsx69HcWxKJ3+WevsfbBAcJwaTnEJdktq7AalPoVI1C8Ii66Ej1Rv XCFmqDFTRJJNtzW7eZSjKe3e2CLeFuVr94f7vTTnYu/W3ZRMBu7kfyVp9hLzV3cNMxx/ VtscgfjtR6XZvDVMqHHAj6ghV2YQ89F4PdjU7um3Wxd8jQxigTI8zHHu059lH6txbiEF VjinIdAuPeD/eAJk7yyqa64bH7RDhlEPf+jwSrNY0H3M6J8BNP24OMAA0825Ahue6D0T UjOxxW1oi6/rAeJV8gemPSfc6RdaTDQ40/QrVXqGJmkI8nJt/JCezCaw6xDk05UUPkoq BNWw== X-Gm-Message-State: AOJu0YyO9IbUhePVsSBiETzURXiGBaxuoYEzzyh2GjueFRSFeky/D63b V+/67/kBk1s90ATFE/5zHy+iAwqwR8XWg+VS1WCgGeE+r+VR2V36ZUda X-Gm-Gg: ASbGncsE8M6F18A8Nm2bxLwwhJuvr69TyMWzO//Q4BO2LRVelj0C2f8kSahpIo/TvxU QvE5YNlcoTEPHWAjIgVnUng5isIsz7uIU7ie36oAPC2FqnlFk5vJFAnapRq28u45mslLhV65hdJ oK8f5EChjpmGfVxO/4Q9PHra0WpBj147M82/Xjy85q/KRGpCb5A6diFfyA6KPZSiAEneHeCJuYD RJ7IBOYb7aPatCB3MJ6LrvxN8Vb1+gDiHaIz95NjWJAJXO42HMXCDwG6WOgqg== X-Google-Smtp-Source: AGHT+IE8y+PV+Eu7SO4RBrM4Vl2/zvHyiry1OucZ40KcGU33ofoFIhLkhCfWwX/yVbIqrl8deY0/2w== X-Received: by 2002:a05:622a:91:b0:46c:7737:c509 with SMTP id d75a77b69052e-46e5b760c32mr57490331cf.4.1737659285622; Thu, 23 Jan 2025 11:08:05 -0800 (PST) Received: from citadel.lan ([2600:6c4a:4d3f:6d5c::1019]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-46e66b880b6sm1768021cf.69.2025.01.23.11.08.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jan 2025 11:08:04 -0800 (PST) From: Brian Gerst To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Ingo Molnar , "H . Peter Anvin" , Thomas Gleixner , Borislav Petkov , Ard Biesheuvel , Uros Bizjak , Brian Gerst Subject: [PATCH v6 07/15] x86/stackprotector/64: Convert to normal percpu variable Date: Thu, 23 Jan 2025 14:07:39 -0500 Message-ID: <20250123190747.745588-8-brgerst@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250123190747.745588-1-brgerst@gmail.com> References: <20250123190747.745588-1-brgerst@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Older versions of GCC fixed the location of the stack protector canary at %gs:40. This constraint forced the percpu section to be linked at absolute address 0 so that the canary could be the first data object in the percpu section. Supporting the zero-based percpu section requires additional code to handle relocations for RIP-relative references to percpu data, extra complexity to kallsyms, and workarounds for linker bugs due to the use of absolute symbols. GCC 8.1 supports redefining where the canary is located, allowng it to become a normal percpu variable instead of at a fixed location. This removes the contraint that the percpu section must be zero-based. Signed-off-by: Brian Gerst Reviewed-by: Ard Biesheuvel Reviewed-by: Uros Bizjak --- arch/x86/Makefile | 20 +++++++++------ arch/x86/entry/entry.S | 2 -- arch/x86/entry/entry_64.S | 2 +- arch/x86/include/asm/processor.h | 16 ++---------- arch/x86/include/asm/stackprotector.h | 36 ++++----------------------- arch/x86/kernel/asm-offsets_64.c | 6 ----- arch/x86/kernel/cpu/common.c | 5 +--- arch/x86/kernel/head_64.S | 3 +-- arch/x86/xen/xen-head.S | 3 +-- 9 files changed, 23 insertions(+), 70 deletions(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 5b773b34768d..88a1705366f9 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -140,14 +140,7 @@ ifeq ($(CONFIG_X86_32),y) # temporary until string.h is fixed KBUILD_CFLAGS += -ffreestanding - ifeq ($(CONFIG_STACKPROTECTOR),y) - ifeq ($(CONFIG_SMP),y) - KBUILD_CFLAGS += -mstack-protector-guard-reg=fs \ - -mstack-protector-guard-symbol=__ref_stack_chk_guard - else - KBUILD_CFLAGS += -mstack-protector-guard=global - endif - endif + percpu_seg := fs else BITS := 64 UTS_MACHINE := x86_64 @@ -197,6 +190,17 @@ else KBUILD_CFLAGS += -mcmodel=kernel KBUILD_RUSTFLAGS += -Cno-redzone=y KBUILD_RUSTFLAGS += -Ccode-model=kernel + + percpu_seg := gs +endif + +ifeq ($(CONFIG_STACKPROTECTOR),y) + ifeq ($(CONFIG_SMP),y) + KBUILD_CFLAGS += -mstack-protector-guard-reg=$(percpu_seg) + KBUILD_CFLAGS += -mstack-protector-guard-symbol=__ref_stack_chk_guard + else + KBUILD_CFLAGS += -mstack-protector-guard=global + endif endif # diff --git a/arch/x86/entry/entry.S b/arch/x86/entry/entry.S index b7ea3e8e9ecc..fe5344a249a1 100644 --- a/arch/x86/entry/entry.S +++ b/arch/x86/entry/entry.S @@ -52,7 +52,6 @@ EXPORT_SYMBOL_GPL(mds_verw_sel); THUNK warn_thunk_thunk, __warn_thunk -#ifndef CONFIG_X86_64 /* * Clang's implementation of TLS stack cookies requires the variable in * question to be a TLS variable. If the variable happens to be defined as an @@ -66,4 +65,3 @@ THUNK warn_thunk_thunk, __warn_thunk #ifdef CONFIG_STACKPROTECTOR EXPORT_SYMBOL(__ref_stack_chk_guard); #endif -#endif diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index f52dbe0ad93c..33a955aa01d8 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -192,7 +192,7 @@ SYM_FUNC_START(__switch_to_asm) #ifdef CONFIG_STACKPROTECTOR movq TASK_stack_canary(%rsi), %rbx - movq %rbx, PER_CPU_VAR(fixed_percpu_data + FIXED_stack_canary) + movq %rbx, PER_CPU_VAR(__stack_chk_guard) #endif /* diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index c0cd10182e90..a4687122951f 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -422,16 +422,8 @@ struct irq_stack { #ifdef CONFIG_X86_64 struct fixed_percpu_data { - /* - * GCC hardcodes the stack canary as %gs:40. Since the - * irq_stack is the object at %gs:0, we reserve the bottom - * 48 bytes of the irq stack for the canary. - * - * Once we are willing to require -mstack-protector-guard-symbol= - * support for x86_64 stackprotector, we can get rid of this. - */ char gs_base[40]; - unsigned long stack_canary; + unsigned long reserved; }; DECLARE_PER_CPU_FIRST(struct fixed_percpu_data, fixed_percpu_data) __visible; @@ -446,11 +438,7 @@ extern asmlinkage void entry_SYSCALL32_ignore(void); /* Save actual FS/GS selectors and bases to current->thread */ void current_save_fsgs(void); -#else /* X86_64 */ -#ifdef CONFIG_STACKPROTECTOR -DECLARE_PER_CPU(unsigned long, __stack_chk_guard); -#endif -#endif /* !X86_64 */ +#endif /* X86_64 */ struct perf_event; diff --git a/arch/x86/include/asm/stackprotector.h b/arch/x86/include/asm/stackprotector.h index 00473a650f51..d43fb589fcf6 100644 --- a/arch/x86/include/asm/stackprotector.h +++ b/arch/x86/include/asm/stackprotector.h @@ -2,26 +2,10 @@ /* * GCC stack protector support. * - * Stack protector works by putting predefined pattern at the start of + * Stack protector works by putting a predefined pattern at the start of * the stack frame and verifying that it hasn't been overwritten when - * returning from the function. The pattern is called stack canary - * and unfortunately gcc historically required it to be at a fixed offset - * from the percpu segment base. On x86_64, the offset is 40 bytes. - * - * The same segment is shared by percpu area and stack canary. On - * x86_64, percpu symbols are zero based and %gs (64-bit) points to the - * base of percpu area. The first occupant of the percpu area is always - * fixed_percpu_data which contains stack_canary at the appropriate - * offset. On x86_32, the stack canary is just a regular percpu - * variable. - * - * Putting percpu data in %fs on 32-bit is a minor optimization compared to - * using %gs. Since 32-bit userspace normally has %fs == 0, we are likely - * to load 0 into %fs on exit to usermode, whereas with percpu data in - * %gs, we are likely to load a non-null %gs on return to user mode. - * - * Once we are willing to require GCC 8.1 or better for 64-bit stackprotector - * support, we can remove some of this complexity. + * returning from the function. The pattern is called the stack canary + * and is a unique value for each task. */ #ifndef _ASM_STACKPROTECTOR_H @@ -36,6 +20,8 @@ #include +DECLARE_PER_CPU(unsigned long, __stack_chk_guard); + /* * Initialize the stackprotector canary value. * @@ -51,25 +37,13 @@ static __always_inline void boot_init_stack_canary(void) { unsigned long canary = get_random_canary(); -#ifdef CONFIG_X86_64 - BUILD_BUG_ON(offsetof(struct fixed_percpu_data, stack_canary) != 40); -#endif - current->stack_canary = canary; -#ifdef CONFIG_X86_64 - this_cpu_write(fixed_percpu_data.stack_canary, canary); -#else this_cpu_write(__stack_chk_guard, canary); -#endif } static inline void cpu_init_stack_canary(int cpu, struct task_struct *idle) { -#ifdef CONFIG_X86_64 - per_cpu(fixed_percpu_data.stack_canary, cpu) = idle->stack_canary; -#else per_cpu(__stack_chk_guard, cpu) = idle->stack_canary; -#endif } #else /* STACKPROTECTOR */ diff --git a/arch/x86/kernel/asm-offsets_64.c b/arch/x86/kernel/asm-offsets_64.c index bb65371ea9df..590b6cd0eac0 100644 --- a/arch/x86/kernel/asm-offsets_64.c +++ b/arch/x86/kernel/asm-offsets_64.c @@ -54,11 +54,5 @@ int main(void) BLANK(); #undef ENTRY - BLANK(); - -#ifdef CONFIG_STACKPROTECTOR - OFFSET(FIXED_stack_canary, fixed_percpu_data, stack_canary); - BLANK(); -#endif return 0; } diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 7cce91b19fb2..b71178f0ed6c 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2089,8 +2089,7 @@ void syscall_init(void) if (!cpu_feature_enabled(X86_FEATURE_FRED)) idt_syscall_init(); } - -#else /* CONFIG_X86_64 */ +#endif /* CONFIG_X86_64 */ #ifdef CONFIG_STACKPROTECTOR DEFINE_PER_CPU(unsigned long, __stack_chk_guard); @@ -2099,8 +2098,6 @@ EXPORT_PER_CPU_SYMBOL(__stack_chk_guard); #endif #endif -#endif /* CONFIG_X86_64 */ - /* * Clear all 6 debug registers: */ diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 31345e0ba006..c3d73c04603f 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -361,8 +361,7 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) /* Set up %gs. * - * The base of %gs always points to fixed_percpu_data. If the - * stack protector canary is enabled, it is located at %gs:40. + * The base of %gs always points to fixed_percpu_data. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ diff --git a/arch/x86/xen/xen-head.S b/arch/x86/xen/xen-head.S index 9252652afe59..5d3866ec3100 100644 --- a/arch/x86/xen/xen-head.S +++ b/arch/x86/xen/xen-head.S @@ -33,8 +33,7 @@ SYM_CODE_START(startup_xen) /* Set up %gs. * - * The base of %gs always points to fixed_percpu_data. If the - * stack protector canary is enabled, it is located at %gs:40. + * The base of %gs always points to fixed_percpu_data. * Note that, on SMP, the boot cpu uses init data section until * the per cpu areas are set up. */ -- 2.47.1