* [PATCH 0/7] Ksplice: Rebootless kernel updates @ 2008-12-06 0:03 Jeff Arnold 2008-12-06 0:03 ` [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections Jeff Arnold 2008-12-17 2:48 ` [PATCH 0/7] Ksplice: Rebootless kernel updates Tim Abbott 0 siblings, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan Hello Andrew, We're interested in having Ksplice included in -mm. If you could consider this code for inclusion (or let us know what you'd like us to do next), we'd appreciate it. == These patches add support for Ksplice [1], a rebootless update system, to the kernel. [1] http://www.ksplice.com/download I previously mentioned Ksplice to the LKML in April [2], and I submitted RFC patches to the LKML in September [3] and November [4]. The basic idea is that Ksplice can apply an update to a running Linux kernel based only on a source code patch and the original kernel source. The Ksplice technical overview document [5] describes the design of Ksplice and one of the ways that Ksplice has been evaluated. [2] http://lkml.org/lkml/2008/4/23/330 [3] http://lkml.org/lkml/2008/9/13/6 [4] http://lkml.org/lkml/2008/11/21/484 [5] http://www.ksplice.com/doc/ksplice.pdf Specifically, Ksplice has been used to correct every significant Linux kernel CVE from May 2005 to May 2008 [6]. 88% of these CVEs can be fixed without writing any new code at all; the remaining 12% of CVEs require a minimal amount of new code (on average, roughly 20 new semicolon-terminated lines per CVE). [6] http://www.ksplice.com/cve-evaluation The interface between the Ksplice kernel component and the userspace utilities is documented in the last patch of this series. The current implementation of the userspace utilities is available from the Ksplice website and the Ksplice git repository [7]. The Ksplice userspace utilities transparently support both "integrated" operation (updating a kernel patched with this patch series) and "standalone" operation (updating a completely unmodified kernel). [7] http://www.ksplice.com/git/ksplice.git Just to let you know what direction we are going in: We're currently working on the problem of making it feasible to apply the entire stable tree using Ksplice. Although Ksplice's original evaluation focused on patches for CVEs, we understand the idea that "security bugs are just 'normal bugs'" [8] (i.e., tracking security bugs separately from normal bugs can be difficult and isn't necessarily advisable). We ultimately want to provide to long-running machines hot updates for all of the bug fixes that go into the corresponding stable tree. We think that long-running machines will be more secure and reliable if they can regularly be updated to the latest stable kernel code (along with the usual user space updates provided by distributions). [8] http://lkml.org/lkml/2008/7/14/465 Even though work on Ksplice is ongoing, I think that merging Ksplice makes sense at this point because (1) development on the Ksplice kernel code has stabilized, and (2) the system, as it currently stands, can be useful in a variety of situations. Jeff Arnold jbarnold@mit.edu Changes since v3: - removed use of init_mm, as requested by Christoph Hellwig Changes since v2: - added taint flag, as requested by Alexey Dobriyan - various speed optimizations - support for replacing functions mapped read-only (CONFIG_DEBUG_RODATA) - run-pre matching extended to bug table, ex_table, exported symbol table - added hooks for calling custom code during the update process - struct ksplice_export eliminated (struct ksplice_patch used instead) - minor sysfs API changes - added support for ksplice-apply's --partial option - improved -ffunction-sections patches and added them to this patch series - dropped ARM support for now, for simplicity No changes between v1 and v2. == Documentation/ksplice.txt | 282 ++ Documentation/mutex-design.txt | 4 +- MAINTAINERS | 10 + Makefile | 4 + arch/Kconfig | 14 + arch/alpha/kernel/head.S | 2 +- arch/alpha/kernel/init_task.c | 2 +- arch/alpha/kernel/vmlinux.lds.S | 14 +- arch/arm/kernel/head-nommu.S | 3 +- arch/arm/kernel/head.S | 3 +- arch/arm/kernel/init_task.c | 2 +- arch/arm/kernel/vmlinux.lds.S | 14 +- arch/arm/mm/proc-v6.S | 2 +- arch/arm/mm/proc-v7.S | 2 +- arch/arm/mm/tlb-v6.S | 2 +- arch/arm/mm/tlb-v7.S | 2 +- arch/avr32/kernel/init_task.c | 2 +- arch/avr32/kernel/vmlinux.lds.S | 6 +- arch/avr32/mm/init.c | 2 +- arch/blackfin/kernel/vmlinux.lds.S | 2 +- arch/cris/kernel/process.c | 2 +- arch/frv/kernel/break.S | 4 +- arch/frv/kernel/entry.S | 2 +- arch/frv/kernel/head-mmu-fr451.S | 2 +- arch/frv/kernel/head-uc-fr401.S | 2 +- arch/frv/kernel/head-uc-fr451.S | 2 +- arch/frv/kernel/head-uc-fr555.S | 2 +- arch/frv/kernel/head.S | 4 +- arch/frv/kernel/init_task.c | 2 +- arch/frv/kernel/vmlinux.lds.S | 18 +- arch/frv/mm/tlb-miss.S | 2 +- arch/h8300/boot/compressed/head.S | 2 +- arch/h8300/boot/compressed/vmlinux.lds | 2 +- arch/h8300/kernel/init_task.c | 2 +- arch/h8300/kernel/vmlinux.lds.S | 2 +- arch/ia64/include/asm/asmmacro.h | 12 +- arch/ia64/include/asm/cache.h | 2 +- arch/ia64/include/asm/percpu.h | 2 +- arch/ia64/kernel/Makefile | 2 +- arch/ia64/kernel/gate-data.S | 2 +- arch/ia64/kernel/gate.S | 8 +- arch/ia64/kernel/gate.lds.S | 10 +- arch/ia64/kernel/head.S | 2 +- arch/ia64/kernel/init_task.c | 4 +- arch/ia64/kernel/ivt.S | 2 +- arch/ia64/kernel/minstate.h | 4 +- arch/ia64/kernel/paravirtentry.S | 12 +- arch/ia64/kernel/vmlinux.lds.S | 48 +- arch/ia64/kvm/vmm_ivt.S | 2 +- arch/ia64/xen/xensetup.S | 2 +- arch/m32r/kernel/init_task.c | 2 +- arch/m32r/kernel/vmlinux.lds.S | 6 +- arch/m68k/kernel/head.S | 2 +- arch/m68k/kernel/process.c | 2 +- arch/m68k/kernel/sun3-head.S | 2 +- arch/m68k/kernel/vmlinux-std.lds | 6 +- arch/m68k/kernel/vmlinux-sun3.lds | 4 +- arch/m68knommu/kernel/init_task.c | 2 +- arch/m68knommu/kernel/vmlinux.lds.S | 6 +- arch/m68knommu/platform/68360/head-ram.S | 2 +- arch/m68knommu/platform/68360/head-rom.S | 2 +- arch/mips/kernel/init_task.c | 2 +- arch/mips/kernel/vmlinux.lds.S | 8 +- arch/mips/lasat/image/head.S | 2 +- arch/mips/lasat/image/romscript.normal | 2 +- arch/mn10300/kernel/head.S | 2 +- arch/mn10300/kernel/init_task.c | 2 +- arch/mn10300/kernel/vmlinux.lds.S | 18 +- arch/parisc/include/asm/cache.h | 2 +- arch/parisc/include/asm/system.h | 2 +- arch/parisc/kernel/head.S | 2 +- arch/parisc/kernel/init_task.c | 8 +- arch/parisc/kernel/vmlinux.lds.S | 26 +- arch/powerpc/include/asm/cache.h | 2 +- arch/powerpc/include/asm/page_64.h | 2 +- arch/powerpc/include/asm/ppc_asm.h | 4 +- arch/powerpc/kernel/head_32.S | 2 +- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_44x.S | 2 +- arch/powerpc/kernel/head_8xx.S | 2 +- arch/powerpc/kernel/head_fsl_booke.S | 2 +- arch/powerpc/kernel/init_task.c | 2 +- arch/powerpc/kernel/machine_kexec_64.c | 2 +- arch/powerpc/kernel/vdso.c | 2 +- arch/powerpc/kernel/vdso32/vdso32_wrapper.S | 2 +- arch/powerpc/kernel/vdso64/vdso64_wrapper.S | 2 +- arch/powerpc/kernel/vmlinux.lds.S | 28 +- arch/s390/include/asm/cache.h | 2 +- arch/s390/kernel/head.S | 2 +- arch/s390/kernel/init_task.c | 2 +- arch/s390/kernel/vmlinux.lds.S | 20 +- arch/sh/include/asm/cache.h | 2 +- arch/sh/kernel/cpu/sh5/entry.S | 4 +- arch/sh/kernel/head_32.S | 2 +- arch/sh/kernel/head_64.S | 2 +- arch/sh/kernel/init_task.c | 2 +- arch/sh/kernel/irq.c | 4 +- arch/sh/kernel/vmlinux_32.lds.S | 14 +- arch/sh/kernel/vmlinux_64.lds.S | 14 +- arch/sparc/boot/btfixupprep.c | 4 +- arch/sparc/include/asm/cache.h | 2 +- arch/sparc/kernel/head.S | 2 +- arch/sparc/kernel/vmlinux.lds.S | 8 +- arch/sparc64/kernel/head.S | 2 +- arch/sparc64/kernel/vmlinux.lds.S | 8 +- arch/um/include/asm/common.lds.S | 4 +- arch/um/kernel/dyn.lds.S | 4 +- arch/um/kernel/init_task.c | 4 +- arch/um/kernel/uml.lds.S | 4 +- arch/x86/Kconfig | 1 + arch/x86/boot/compressed/head_32.S | 2 +- arch/x86/boot/compressed/head_64.S | 2 +- arch/x86/boot/compressed/vmlinux.scr | 2 +- arch/x86/boot/compressed/vmlinux_32.lds | 14 +- arch/x86/boot/compressed/vmlinux_64.lds | 10 +- arch/x86/include/asm/cache.h | 4 +- arch/x86/kernel/acpi/wakeup_32.S | 2 +- arch/x86/kernel/head_32.S | 6 +- arch/x86/kernel/head_64.S | 4 +- arch/x86/kernel/init_task.c | 4 +- arch/x86/kernel/ksplice-arch.c | 125 + arch/x86/kernel/traps.c | 2 +- arch/x86/kernel/vmlinux_32.lds.S | 37 +- arch/x86/kernel/vmlinux_64.lds.S | 27 +- arch/xtensa/kernel/head.S | 2 +- arch/xtensa/kernel/init_task.c | 2 +- arch/xtensa/kernel/vmlinux.lds.S | 6 +- include/asm-frv/init.h | 8 +- include/asm-generic/vmlinux.lds.h | 16 +- include/linux/cache.h | 2 +- include/linux/init.h | 8 +- include/linux/kallsyms.h | 13 + include/linux/kernel.h | 1 + include/linux/ksplice.h | 212 ++ include/linux/linkage.h | 4 +- include/linux/module.h | 42 + include/linux/percpu.h | 12 +- include/linux/spinlock.h | 2 +- kernel/Makefile | 3 + kernel/extable.c | 3 + kernel/kallsyms.c | 19 + kernel/ksplice.c | 2715 +++++++++++++++++++ kernel/module.c | 164 +- kernel/panic.c | 1 + kernel/sched.c | 1 + lib/Kconfig.debug | 9 + lib/bug.c | 1 + scripts/Makefile.modpost | 1 + scripts/mod/modpost.c | 22 +- scripts/recordmcount.pl | 6 +- 150 files changed, 3927 insertions(+), 408 deletions(-) Jeff Arnold jbarnold@mit.edu ^ permalink raw reply [flat|nested] 43+ messages in thread
* [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-06 0:03 [PATCH 0/7] Ksplice: Rebootless kernel updates Jeff Arnold @ 2008-12-06 0:03 ` Jeff Arnold 2008-12-06 0:03 ` [PATCH 2/7] x86: Add an option to compile " Jeff Arnold ` (2 more replies) 2008-12-17 2:48 ` [PATCH 0/7] Ksplice: Rebootless kernel updates Tim Abbott 1 sibling, 3 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan From: Denys Vlasenko <vda.linux@googlemail.com> This patch was originally written by Denys Vlasenko, but it has been substantially modified by Anders Kaseorg and Tim Abbott to update it to apply to the current kernel and fix bugs. Any bugs were added by Anders Kaseorg and Tim Abbott. The purpose of this patch is to make it possible to build the kernel with "gcc -ffunction-sections -fdata-sections". The problem is that with -ffunction-sections -fdata-sections, gcc creates sections like .text.head and .data.nosave whenever someone has innocuous code like this: static void head(...) {...} or this: static int nosave = 1; somewhere in the kernel. The kernel linker scripts are confused by such names, and thus put these sections in the wrong places. This patch renames all "magic" section names used by the kernel to not have this format, eliminating the possibility of such collisions. Ksplice's 'run-pre matching' process is much simpler if the original kernel was compiled with -ffunction-sections and -fdata-sections. Changelog since Denys Vlasenko's latest patch: * Renamed .kernel.exit.text.refok to .kernel.text.exit.refok, for better analogy with .kernel.*.init.refok. * Updated from 2.6.27-rc4-git3 to 2.6.28-rc5. * Added some missing section flags for some of the .kernel.* sections (gcc assumes "ax" for sections with names starting with ".text", but does not for sections with names starting with ".kernel.text"). Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com> [andersk@mit.edu] Add missing section flags for .kernel.* sections. Signed-off-by: Anders Kaseorg <andersk@mit.edu> [tabbott@mit.edu] Update from 2.6.27-rc4-git3 to 2.6.28-rc5. [tabbott@mit.edu] Rename .kernel.exit.text.refok. Signed-off-by: Tim Abbott <tabbott@mit.edu> Tested-by: Waseem Daher <wdaher@mit.edu> --- Documentation/mutex-design.txt | 4 +- arch/alpha/kernel/head.S | 2 +- arch/alpha/kernel/init_task.c | 2 +- arch/alpha/kernel/vmlinux.lds.S | 14 ++++---- arch/arm/kernel/head-nommu.S | 3 +- arch/arm/kernel/head.S | 3 +- arch/arm/kernel/init_task.c | 2 +- arch/arm/kernel/vmlinux.lds.S | 14 ++++---- arch/arm/mm/proc-v6.S | 2 +- arch/arm/mm/proc-v7.S | 2 +- arch/arm/mm/tlb-v6.S | 2 +- arch/arm/mm/tlb-v7.S | 2 +- arch/avr32/kernel/init_task.c | 2 +- arch/avr32/kernel/vmlinux.lds.S | 6 ++-- arch/avr32/mm/init.c | 2 +- arch/blackfin/kernel/vmlinux.lds.S | 2 +- arch/cris/kernel/process.c | 2 +- arch/frv/kernel/break.S | 4 +- arch/frv/kernel/entry.S | 2 +- arch/frv/kernel/head-mmu-fr451.S | 2 +- arch/frv/kernel/head-uc-fr401.S | 2 +- arch/frv/kernel/head-uc-fr451.S | 2 +- arch/frv/kernel/head-uc-fr555.S | 2 +- arch/frv/kernel/head.S | 4 +- arch/frv/kernel/init_task.c | 2 +- arch/frv/kernel/vmlinux.lds.S | 18 +++++----- arch/frv/mm/tlb-miss.S | 2 +- arch/h8300/boot/compressed/head.S | 2 +- arch/h8300/boot/compressed/vmlinux.lds | 2 +- arch/h8300/kernel/init_task.c | 2 +- arch/h8300/kernel/vmlinux.lds.S | 2 +- arch/ia64/include/asm/asmmacro.h | 12 +++--- arch/ia64/include/asm/cache.h | 2 +- arch/ia64/include/asm/percpu.h | 2 +- arch/ia64/kernel/Makefile | 2 +- arch/ia64/kernel/gate-data.S | 2 +- arch/ia64/kernel/gate.S | 8 ++-- arch/ia64/kernel/gate.lds.S | 10 +++--- arch/ia64/kernel/head.S | 2 +- arch/ia64/kernel/init_task.c | 4 +- arch/ia64/kernel/ivt.S | 2 +- arch/ia64/kernel/minstate.h | 4 +- arch/ia64/kernel/paravirtentry.S | 12 +++--- arch/ia64/kernel/vmlinux.lds.S | 48 +++++++++++++------------- arch/ia64/kvm/vmm_ivt.S | 2 +- arch/ia64/xen/xensetup.S | 2 +- arch/m32r/kernel/init_task.c | 2 +- arch/m32r/kernel/vmlinux.lds.S | 6 ++-- arch/m68k/kernel/head.S | 2 +- arch/m68k/kernel/process.c | 2 +- arch/m68k/kernel/sun3-head.S | 2 +- arch/m68k/kernel/vmlinux-std.lds | 6 ++-- arch/m68k/kernel/vmlinux-sun3.lds | 4 +- arch/m68knommu/kernel/init_task.c | 2 +- arch/m68knommu/kernel/vmlinux.lds.S | 6 ++-- arch/m68knommu/platform/68360/head-ram.S | 2 +- arch/m68knommu/platform/68360/head-rom.S | 2 +- arch/mips/kernel/init_task.c | 2 +- arch/mips/kernel/vmlinux.lds.S | 8 ++-- arch/mips/lasat/image/head.S | 2 +- arch/mips/lasat/image/romscript.normal | 2 +- arch/mn10300/kernel/head.S | 2 +- arch/mn10300/kernel/init_task.c | 2 +- arch/mn10300/kernel/vmlinux.lds.S | 18 +++++----- arch/parisc/include/asm/cache.h | 2 +- arch/parisc/include/asm/system.h | 2 +- arch/parisc/kernel/head.S | 2 +- arch/parisc/kernel/init_task.c | 8 ++-- arch/parisc/kernel/vmlinux.lds.S | 26 +++++++------- arch/powerpc/include/asm/cache.h | 2 +- arch/powerpc/include/asm/page_64.h | 2 +- arch/powerpc/include/asm/ppc_asm.h | 4 +- arch/powerpc/kernel/head_32.S | 2 +- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_44x.S | 2 +- arch/powerpc/kernel/head_8xx.S | 2 +- arch/powerpc/kernel/head_fsl_booke.S | 2 +- arch/powerpc/kernel/init_task.c | 2 +- arch/powerpc/kernel/machine_kexec_64.c | 2 +- arch/powerpc/kernel/vdso.c | 2 +- arch/powerpc/kernel/vdso32/vdso32_wrapper.S | 2 +- arch/powerpc/kernel/vdso64/vdso64_wrapper.S | 2 +- arch/powerpc/kernel/vmlinux.lds.S | 28 ++++++++-------- arch/s390/include/asm/cache.h | 2 +- arch/s390/kernel/head.S | 2 +- arch/s390/kernel/init_task.c | 2 +- arch/s390/kernel/vmlinux.lds.S | 20 +++++----- arch/sh/include/asm/cache.h | 2 +- arch/sh/kernel/cpu/sh5/entry.S | 4 +- arch/sh/kernel/head_32.S | 2 +- arch/sh/kernel/head_64.S | 2 +- arch/sh/kernel/init_task.c | 2 +- arch/sh/kernel/irq.c | 4 +- arch/sh/kernel/vmlinux_32.lds.S | 14 ++++---- arch/sh/kernel/vmlinux_64.lds.S | 14 ++++---- arch/sparc/boot/btfixupprep.c | 4 +- arch/sparc/include/asm/cache.h | 2 +- arch/sparc/kernel/head.S | 2 +- arch/sparc/kernel/vmlinux.lds.S | 8 ++-- arch/sparc64/kernel/head.S | 2 +- arch/sparc64/kernel/vmlinux.lds.S | 8 ++-- arch/um/include/asm/common.lds.S | 4 +- arch/um/kernel/dyn.lds.S | 4 +- arch/um/kernel/init_task.c | 4 +- arch/um/kernel/uml.lds.S | 4 +- arch/x86/boot/compressed/head_32.S | 2 +- arch/x86/boot/compressed/head_64.S | 2 +- arch/x86/boot/compressed/vmlinux.scr | 2 +- arch/x86/boot/compressed/vmlinux_32.lds | 14 +++++-- arch/x86/boot/compressed/vmlinux_64.lds | 10 +++-- arch/x86/include/asm/cache.h | 4 +- arch/x86/kernel/acpi/wakeup_32.S | 2 +- arch/x86/kernel/head_32.S | 6 ++-- arch/x86/kernel/head_64.S | 4 +- arch/x86/kernel/init_task.c | 4 +- arch/x86/kernel/traps.c | 2 +- arch/x86/kernel/vmlinux_32.lds.S | 36 ++++++++++---------- arch/x86/kernel/vmlinux_64.lds.S | 26 +++++++------- arch/xtensa/kernel/head.S | 2 +- arch/xtensa/kernel/init_task.c | 2 +- arch/xtensa/kernel/vmlinux.lds.S | 6 ++-- include/asm-frv/init.h | 8 ++-- include/asm-generic/vmlinux.lds.h | 14 ++++---- include/linux/cache.h | 2 +- include/linux/init.h | 8 ++-- include/linux/linkage.h | 4 +- include/linux/percpu.h | 12 +++--- include/linux/spinlock.h | 2 +- kernel/module.c | 2 +- scripts/mod/modpost.c | 12 +++--- scripts/recordmcount.pl | 6 ++-- 131 files changed, 356 insertions(+), 346 deletions(-) diff --git a/Documentation/mutex-design.txt b/Documentation/mutex-design.txt index aa60d1f..b5f800b 100644 --- a/Documentation/mutex-design.txt +++ b/Documentation/mutex-design.txt @@ -66,14 +66,14 @@ of advantages of mutexes: c0377ccb <mutex_lock>: c0377ccb: f0 ff 08 lock decl (%eax) - c0377cce: 78 0e js c0377cde <.text.lock.mutex> + c0377cce: 78 0e js c0377cde <.kernel.text.lock.mutex> c0377cd0: c3 ret the unlocking fastpath is equally tight: c0377cd1 <mutex_unlock>: c0377cd1: f0 ff 00 lock incl (%eax) - c0377cd4: 7e 0f jle c0377ce5 <.text.lock.mutex+0x7> + c0377cd4: 7e 0f jle c0377ce5 <.kernel.text.lock.mutex+0x7> c0377cd6: c3 ret - 'struct mutex' semantics are well-defined and are enforced if diff --git a/arch/alpha/kernel/head.S b/arch/alpha/kernel/head.S index 7ac1f13..3726d71 100644 --- a/arch/alpha/kernel/head.S +++ b/arch/alpha/kernel/head.S @@ -10,7 +10,7 @@ #include <asm/system.h> #include <asm/asm-offsets.h> -.section .text.head, "ax" +.section .kernel.text.head, "ax" .globl swapper_pg_dir .globl _stext swapper_pg_dir=SWAPPER_PGD diff --git a/arch/alpha/kernel/init_task.c b/arch/alpha/kernel/init_task.c index 1f76218..4dfa998 100644 --- a/arch/alpha/kernel/init_task.c +++ b/arch/alpha/kernel/init_task.c @@ -18,5 +18,5 @@ EXPORT_SYMBOL(init_mm); EXPORT_SYMBOL(init_task); union thread_union init_thread_union - __attribute__((section(".data.init_thread"))) + __attribute__((section(".kernel.data.init_thread"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/alpha/kernel/vmlinux.lds.S b/arch/alpha/kernel/vmlinux.lds.S index ef37fc1..e3c637e 100644 --- a/arch/alpha/kernel/vmlinux.lds.S +++ b/arch/alpha/kernel/vmlinux.lds.S @@ -16,7 +16,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -93,18 +93,18 @@ SECTIONS /* Freed after init ends here */ /* Note 2 page alignment above. */ - .data.init_thread : { - *(.data.init_thread) + .kernel.data.init_thread : { + *(.kernel.data.init_thread) } . = ALIGN(PAGE_SIZE); - .data.page_aligned : { - *(.data.page_aligned) + .kernel.data.page_aligned : { + *(.kernel.data.page_aligned) } . = ALIGN(64); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } _data = .; diff --git a/arch/arm/kernel/head-nommu.S b/arch/arm/kernel/head-nommu.S index cc87e17..8163b2e 100644 --- a/arch/arm/kernel/head-nommu.S +++ b/arch/arm/kernel/head-nommu.S @@ -32,7 +32,8 @@ * numbers for r1. * */ - .section ".text.head", "ax" + .section ".kernel.text.head", "ax" + .type stext, %function ENTRY(stext) msr cpsr_c, #PSR_F_BIT | PSR_I_BIT | SVC_MODE @ ensure svc mode @ and irqs disabled diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S index 21e17dc..42f2d17 100644 --- a/arch/arm/kernel/head.S +++ b/arch/arm/kernel/head.S @@ -74,7 +74,8 @@ * crap here - that's what the boot loader (or in extreme, well justified * circumstances, zImage) is for. */ - .section ".text.head", "ax" + .section ".kernel.text.head", "ax" + .type stext, %function ENTRY(stext) msr cpsr_c, #PSR_F_BIT | PSR_I_BIT | SVC_MODE @ ensure svc mode @ and irqs disabled diff --git a/arch/arm/kernel/init_task.c b/arch/arm/kernel/init_task.c index 0bbf806..d64e3ed 100644 --- a/arch/arm/kernel/init_task.c +++ b/arch/arm/kernel/init_task.c @@ -30,7 +30,7 @@ EXPORT_SYMBOL(init_mm); * The things we do for performance.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S index 4898bdc..7ec0ff5 100644 --- a/arch/arm/kernel/vmlinux.lds.S +++ b/arch/arm/kernel/vmlinux.lds.S @@ -23,10 +23,10 @@ SECTIONS #else . = PAGE_OFFSET + TEXT_OFFSET; #endif - .text.head : { + .kernel.text.head : { _stext = .; _sinittext = .; - *(.text.head) + *(.kernel.text.head) } .init : { /* Init code and data */ @@ -65,8 +65,8 @@ SECTIONS #endif . = ALIGN(4096); __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; #ifndef CONFIG_XIP_KERNEL __init_begin = _stext; @@ -125,7 +125,7 @@ SECTIONS * first, the init task union, aligned * to an 8192 byte boundary. */ - *(.data.init_task) + *(.kernel.data.init_task) #ifdef CONFIG_XIP_KERNEL . = ALIGN(4096); @@ -137,7 +137,7 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(4096); __nosave_end = .; @@ -145,7 +145,7 @@ SECTIONS * then the cacheline aligned data */ . = ALIGN(32); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) /* * The exception fixup table (might need resorting at runtime) diff --git a/arch/arm/mm/proc-v6.S b/arch/arm/mm/proc-v6.S index 294943b..90c6d6b 100644 --- a/arch/arm/mm/proc-v6.S +++ b/arch/arm/mm/proc-v6.S @@ -130,7 +130,7 @@ cpu_v6_name: .asciz "ARMv6-compatible processor" .align - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr /* * __v6_setup diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index 4d3c0a7..06a74ff 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -144,7 +144,7 @@ cpu_v7_name: .ascii "ARMv7 Processor" .align - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr /* * __v7_setup diff --git a/arch/arm/mm/tlb-v6.S b/arch/arm/mm/tlb-v6.S index 20f84bb..f033268 100644 --- a/arch/arm/mm/tlb-v6.S +++ b/arch/arm/mm/tlb-v6.S @@ -87,7 +87,7 @@ ENTRY(v6wbi_flush_kern_tlb_range) mcr p15, 0, r2, c7, c5, 4 @ prefetch flush mov pc, lr - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr .type v6wbi_tlb_fns, #object ENTRY(v6wbi_tlb_fns) diff --git a/arch/arm/mm/tlb-v7.S b/arch/arm/mm/tlb-v7.S index 24ba510..1b6d40a 100644 --- a/arch/arm/mm/tlb-v7.S +++ b/arch/arm/mm/tlb-v7.S @@ -80,7 +80,7 @@ ENTRY(v7wbi_flush_kern_tlb_range) mov pc, lr ENDPROC(v7wbi_flush_kern_tlb_range) - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr .type v7wbi_tlb_fns, #object ENTRY(v7wbi_tlb_fns) diff --git a/arch/avr32/kernel/init_task.c b/arch/avr32/kernel/init_task.c index 4405846..79954ab 100644 --- a/arch/avr32/kernel/init_task.c +++ b/arch/avr32/kernel/init_task.c @@ -24,7 +24,7 @@ EXPORT_SYMBOL(init_mm); * Initial thread structure. Must be aligned on an 8192-byte boundary. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/avr32/kernel/vmlinux.lds.S b/arch/avr32/kernel/vmlinux.lds.S index 7910d41..1cb0345 100644 --- a/arch/avr32/kernel/vmlinux.lds.S +++ b/arch/avr32/kernel/vmlinux.lds.S @@ -95,15 +95,15 @@ SECTIONS /* * First, the init task union, aligned to an 8K boundary. */ - *(.data.init_task) + *(.kernel.data.init_task) /* Then, the page-aligned data */ . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) /* Then, the cacheline aligned data */ . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) /* And the rest... */ *(.data.rel*) diff --git a/arch/avr32/mm/init.c b/arch/avr32/mm/init.c index fa92ff6..fa5089c 100644 --- a/arch/avr32/mm/init.c +++ b/arch/avr32/mm/init.c @@ -24,7 +24,7 @@ #include <asm/setup.h> #include <asm/sections.h> -#define __page_aligned __attribute__((section(".data.page_aligned"))) +#define __page_aligned __attribute__((section(".kernel.data.page_aligned"))) DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); diff --git a/arch/blackfin/kernel/vmlinux.lds.S b/arch/blackfin/kernel/vmlinux.lds.S index 7d12c66..88e320d 100644 --- a/arch/blackfin/kernel/vmlinux.lds.S +++ b/arch/blackfin/kernel/vmlinux.lds.S @@ -92,7 +92,7 @@ SECTIONS __sdata = .; /* This gets done first, so the glob doesn't suck it in */ . = ALIGN(32); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) #if !L1_DATA_A_LENGTH . = ALIGN(32); diff --git a/arch/cris/kernel/process.c b/arch/cris/kernel/process.c index 5933656..d98f18e 100644 --- a/arch/cris/kernel/process.c +++ b/arch/cris/kernel/process.c @@ -52,7 +52,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/frv/kernel/break.S b/arch/frv/kernel/break.S index bd0bdf9..1b7148a 100644 --- a/arch/frv/kernel/break.S +++ b/arch/frv/kernel/break.S @@ -21,7 +21,7 @@ # # the break handler has its own stack # - .section .bss.stack + .section .bss.kernel.stack .globl __break_user_context .balign THREAD_SIZE __break_stack: @@ -63,7 +63,7 @@ __break_trace_through_exceptions: # entry point for Break Exceptions/Interrupts # ############################################################################### - .section .text.break + .section .kernel.text.break,"ax",@progbits .balign 4 .globl __entry_break __entry_break: diff --git a/arch/frv/kernel/entry.S b/arch/frv/kernel/entry.S index 99060ab..8d54945 100644 --- a/arch/frv/kernel/entry.S +++ b/arch/frv/kernel/entry.S @@ -38,7 +38,7 @@ #define nr_syscalls ((syscall_table_size)/4) - .section .text.entry + .section .kernel.text.entry,"ax",@progbits .balign 4 .macro LEDS val diff --git a/arch/frv/kernel/head-mmu-fr451.S b/arch/frv/kernel/head-mmu-fr451.S index c8f210d..248e557 100644 --- a/arch/frv/kernel/head-mmu-fr451.S +++ b/arch/frv/kernel/head-mmu-fr451.S @@ -31,7 +31,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr401.S b/arch/frv/kernel/head-uc-fr401.S index ee282be..282ab8d 100644 --- a/arch/frv/kernel/head-uc-fr401.S +++ b/arch/frv/kernel/head-uc-fr401.S @@ -30,7 +30,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr451.S b/arch/frv/kernel/head-uc-fr451.S index b10d9c8..2fcb045 100644 --- a/arch/frv/kernel/head-uc-fr451.S +++ b/arch/frv/kernel/head-uc-fr451.S @@ -30,7 +30,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr555.S b/arch/frv/kernel/head-uc-fr555.S index 39937c1..b5758bf 100644 --- a/arch/frv/kernel/head-uc-fr555.S +++ b/arch/frv/kernel/head-uc-fr555.S @@ -29,7 +29,7 @@ #define __551_LCR 0xfeff1100 #define __551_LSBR 0xfeff1c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head.S b/arch/frv/kernel/head.S index fecf751..8be8976 100644 --- a/arch/frv/kernel/head.S +++ b/arch/frv/kernel/head.S @@ -27,7 +27,7 @@ # command line string # ############################################################################### - .section .text.head,"ax" + .section .kernel.text.head,"ax" .balign 4 .globl _boot, __head_reference @@ -541,7 +541,7 @@ __head_end: .size _boot, .-_boot # provide a point for GDB to place a break - .section .text.start,"ax" + .section .kernel.text.start,"ax" .globl _start .balign 4 _start: diff --git a/arch/frv/kernel/init_task.c b/arch/frv/kernel/init_task.c index e219881..1cad6d0 100644 --- a/arch/frv/kernel/init_task.c +++ b/arch/frv/kernel/init_task.c @@ -25,7 +25,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/frv/kernel/vmlinux.lds.S b/arch/frv/kernel/vmlinux.lds.S index b95c4ea..219d286 100644 --- a/arch/frv/kernel/vmlinux.lds.S +++ b/arch/frv/kernel/vmlinux.lds.S @@ -26,7 +26,7 @@ SECTIONS _sinittext = .; .init.text : { - *(.text.head) + *(.kernel.text.head) #ifndef CONFIG_DEBUG_INFO INIT_TEXT EXIT_TEXT @@ -71,13 +71,13 @@ SECTIONS /* put sections together that have massive alignment issues */ . = ALIGN(THREAD_SIZE); - .data.init_task : { + .kernel.data.init_task : { /* init task record & stack */ - *(.data.init_task) + *(.kernel.data.init_task) } . = ALIGN(L1_CACHE_BYTES); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } .trap : { /* trap table management - read entry-table.S before modifying */ @@ -94,10 +94,10 @@ SECTIONS _text = .; _stext = .; .text : { - *(.text.start) - *(.text.entry) - *(.text.break) - *(.text.tlbmiss) + *(.kernel.text.start) + *(.kernel.text.entry) + *(.kernel.text.break) + *(.kernel.text.tlbmiss) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -152,7 +152,7 @@ SECTIONS .sbss : { *(.sbss .sbss.*) } .bss : { *(.bss .bss.*) } - .bss.stack : { *(.bss) } + .bss.kernel.stack : { *(.bss) } __bss_stop = .; _end = . ; diff --git a/arch/frv/mm/tlb-miss.S b/arch/frv/mm/tlb-miss.S index 0764348..e361931 100644 --- a/arch/frv/mm/tlb-miss.S +++ b/arch/frv/mm/tlb-miss.S @@ -16,7 +16,7 @@ #include <asm/highmem.h> #include <asm/spr-regs.h> - .section .text.tlbmiss + .section .kernel.text.tlbmiss,"ax",@progbits .balign 4 .globl __entry_insn_mmu_miss diff --git a/arch/h8300/boot/compressed/head.S b/arch/h8300/boot/compressed/head.S index 985a81a..c4964f1 100644 --- a/arch/h8300/boot/compressed/head.S +++ b/arch/h8300/boot/compressed/head.S @@ -9,7 +9,7 @@ #define SRAM_START 0xff4000 - .section .text.startup + .section .kernel.text.startup,"ax",@progbits .global startup startup: mov.l #SRAM_START+0x8000, sp diff --git a/arch/h8300/boot/compressed/vmlinux.lds b/arch/h8300/boot/compressed/vmlinux.lds index 65e2a0d..5965521 100644 --- a/arch/h8300/boot/compressed/vmlinux.lds +++ b/arch/h8300/boot/compressed/vmlinux.lds @@ -4,7 +4,7 @@ SECTIONS { __stext = . ; __text = .; - *(.text.startup) + *(.kernel.text.startup) *(.text) __etext = . ; } diff --git a/arch/h8300/kernel/init_task.c b/arch/h8300/kernel/init_task.c index 93a4899..5e76793 100644 --- a/arch/h8300/kernel/init_task.c +++ b/arch/h8300/kernel/init_task.c @@ -37,6 +37,6 @@ EXPORT_SYMBOL(init_task); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/h8300/kernel/vmlinux.lds.S b/arch/h8300/kernel/vmlinux.lds.S index 43a87b9..473e17d 100644 --- a/arch/h8300/kernel/vmlinux.lds.S +++ b/arch/h8300/kernel/vmlinux.lds.S @@ -101,7 +101,7 @@ SECTIONS ___data_start = . ; . = ALIGN(0x2000) ; - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(0x4) ; DATA_DATA . = ALIGN(0x4) ; diff --git a/arch/ia64/include/asm/asmmacro.h b/arch/ia64/include/asm/asmmacro.h index c1642fd..30c8715 100644 --- a/arch/ia64/include/asm/asmmacro.h +++ b/arch/ia64/include/asm/asmmacro.h @@ -70,12 +70,12 @@ name: * path (ivt.S - TLB miss processing) or in places where it might not be * safe to use a "tpa" instruction (mca_asm.S - error recovery). */ - .section ".data.patch.vtop", "a" // declare section & section attributes + .section ".kernel.data.patch.vtop", "a" // declare section & section attributes .previous #define LOAD_PHYSICAL(pr, reg, obj) \ [1:](pr)movl reg = obj; \ - .xdata4 ".data.patch.vtop", 1b-. + .xdata4 ".kernel.data.patch.vtop", 1b-. /* * For now, we always put in the McKinley E9 workaround. On CPUs that don't need it, @@ -84,11 +84,11 @@ name: #define DO_MCKINLEY_E9_WORKAROUND #ifdef DO_MCKINLEY_E9_WORKAROUND - .section ".data.patch.mckinley_e9", "a" + .section ".kernel.data.patch.mckinley_e9", "a" .previous /* workaround for Itanium 2 Errata 9: */ # define FSYS_RETURN \ - .xdata4 ".data.patch.mckinley_e9", 1f-.; \ + .xdata4 ".kernel.data.patch.mckinley_e9", 1f-.; \ 1:{ .mib; \ nop.m 0; \ mov r16=ar.pfs; \ @@ -107,11 +107,11 @@ name: * If physical stack register size is different from DEF_NUM_STACK_REG, * dynamically patch the kernel for correct size. */ - .section ".data.patch.phys_stack_reg", "a" + .section ".kernel.data.patch.phys_stack_reg", "a" .previous #define LOAD_PHYS_STACK_REG_SIZE(reg) \ [1:] adds reg=IA64_NUM_PHYS_STACK_REG*8+8,r0; \ - .xdata4 ".data.patch.phys_stack_reg", 1b-. + .xdata4 ".kernel.data.patch.phys_stack_reg", 1b-. /* * Up until early 2004, use of .align within a function caused bad unwind info. diff --git a/arch/ia64/include/asm/cache.h b/arch/ia64/include/asm/cache.h index e7482bd..7c8d940 100644 --- a/arch/ia64/include/asm/cache.h +++ b/arch/ia64/include/asm/cache.h @@ -24,6 +24,6 @@ # define SMP_CACHE_BYTES (1 << 3) #endif -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif /* _ASM_IA64_CACHE_H */ diff --git a/arch/ia64/include/asm/percpu.h b/arch/ia64/include/asm/percpu.h index 77f30b6..b4a5a2e 100644 --- a/arch/ia64/include/asm/percpu.h +++ b/arch/ia64/include/asm/percpu.h @@ -27,7 +27,7 @@ extern void *per_cpu_init(void); #else /* ! SMP */ -#define PER_CPU_ATTRIBUTES __attribute__((__section__(".data.percpu"))) +#define PER_CPU_ATTRIBUTES __attribute__((__section__(".kernel.data.percpu"))) #define per_cpu_init() (__phys_per_cpu_start) diff --git a/arch/ia64/kernel/Makefile b/arch/ia64/kernel/Makefile index c381ea9..610572a 100644 --- a/arch/ia64/kernel/Makefile +++ b/arch/ia64/kernel/Makefile @@ -72,7 +72,7 @@ GATECFLAGS_gate-syms.o = -r $(obj)/gate-syms.o: $(obj)/gate.lds $(obj)/gate.o FORCE $(call if_changed,gate) -# gate-data.o contains the gate DSO image as data in section .data.gate. +# gate-data.o contains the gate DSO image as data in section .kernel.data.gate. # We must build gate.so before we can assemble it. # Note: kbuild does not track this dependency due to usage of .incbin $(obj)/gate-data.o: $(obj)/gate.so diff --git a/arch/ia64/kernel/gate-data.S b/arch/ia64/kernel/gate-data.S index 258c0a3..7cfa6a2 100644 --- a/arch/ia64/kernel/gate-data.S +++ b/arch/ia64/kernel/gate-data.S @@ -1,3 +1,3 @@ - .section .data.gate, "aw" + .section .kernel.data.gate, "aw" .incbin "arch/ia64/kernel/gate.so" diff --git a/arch/ia64/kernel/gate.S b/arch/ia64/kernel/gate.S index 74b1ccc..eddece2 100644 --- a/arch/ia64/kernel/gate.S +++ b/arch/ia64/kernel/gate.S @@ -20,18 +20,18 @@ * to targets outside the shared object) and to avoid multi-phase kernel builds, we * simply create minimalistic "patch lists" in special ELF sections. */ - .section ".data.patch.fsyscall_table", "a" + .section ".kernel.data.patch.fsyscall_table", "a" .previous #define LOAD_FSYSCALL_TABLE(reg) \ [1:] movl reg=0; \ - .xdata4 ".data.patch.fsyscall_table", 1b-. + .xdata4 ".kernel.data.patch.fsyscall_table", 1b-. - .section ".data.patch.brl_fsys_bubble_down", "a" + .section ".kernel.data.patch.brl_fsys_bubble_down", "a" .previous #define BRL_COND_FSYS_BUBBLE_DOWN(pr) \ [1:](pr)brl.cond.sptk 0; \ ;; \ - .xdata4 ".data.patch.brl_fsys_bubble_down", 1b-. + .xdata4 ".kernel.data.patch.brl_fsys_bubble_down", 1b-. GLOBAL_ENTRY(__kernel_syscall_via_break) .prologue diff --git a/arch/ia64/kernel/gate.lds.S b/arch/ia64/kernel/gate.lds.S index 3cb1abc..9a41f12 100644 --- a/arch/ia64/kernel/gate.lds.S +++ b/arch/ia64/kernel/gate.lds.S @@ -32,21 +32,21 @@ SECTIONS */ . = GATE_ADDR + 0x600; - .data.patch : { + .kernel.data.patch : { __start_gate_mckinley_e9_patchlist = .; - *(.data.patch.mckinley_e9) + *(.kernel.data.patch.mckinley_e9) __end_gate_mckinley_e9_patchlist = .; __start_gate_vtop_patchlist = .; - *(.data.patch.vtop) + *(.kernel.data.patch.vtop) __end_gate_vtop_patchlist = .; __start_gate_fsyscall_patchlist = .; - *(.data.patch.fsyscall_table) + *(.kernel.data.patch.fsyscall_table) __end_gate_fsyscall_patchlist = .; __start_gate_brl_fsys_bubble_down_patchlist = .; - *(.data.patch.brl_fsys_bubble_down) + *(.kernel.data.patch.brl_fsys_bubble_down) __end_gate_brl_fsys_bubble_down_patchlist = .; } :readable diff --git a/arch/ia64/kernel/head.S b/arch/ia64/kernel/head.S index 59301c4..a88dc24 100644 --- a/arch/ia64/kernel/head.S +++ b/arch/ia64/kernel/head.S @@ -181,7 +181,7 @@ swapper_pg_dir: halt_msg: stringz "Halting kernel\n" - .section .text.head,"ax" + .section .kernel.text.head,"ax" .global start_ap diff --git a/arch/ia64/kernel/init_task.c b/arch/ia64/kernel/init_task.c index 9d7e1c6..16d7e24 100644 --- a/arch/ia64/kernel/init_task.c +++ b/arch/ia64/kernel/init_task.c @@ -28,7 +28,7 @@ EXPORT_SYMBOL(init_mm); * Initial task structure. * * We need to make sure that this is properly aligned due to the way process stacks are - * handled. This is done by having a special ".data.init_task" section... + * handled. This is done by having a special ".kernel.data.init_task" section... */ #define init_thread_info init_task_mem.s.thread_info @@ -38,7 +38,7 @@ union { struct thread_info thread_info; } s; unsigned long stack[KERNEL_STACK_SIZE/sizeof (unsigned long)]; -} init_task_mem asm ("init_task") __attribute__((section(".data.init_task"))) = {{ +} init_task_mem asm ("init_task") __attribute__((section(".kernel.data.init_task"))) = {{ .task = INIT_TASK(init_task_mem.s.task), .thread_info = INIT_THREAD_INFO(init_task_mem.s.task) }}; diff --git a/arch/ia64/kernel/ivt.S b/arch/ia64/kernel/ivt.S index f675d8e..5bd4502 100644 --- a/arch/ia64/kernel/ivt.S +++ b/arch/ia64/kernel/ivt.S @@ -83,7 +83,7 @@ mov r19=n;; /* prepare to save predicates */ \ br.sptk.many dispatch_to_fault_handler - .section .text.ivt,"ax" + .section .kernel.text.ivt,"ax" .align 32768 // align on 32KB boundary .global ia64_ivt diff --git a/arch/ia64/kernel/minstate.h b/arch/ia64/kernel/minstate.h index 292e214..9bcff80 100644 --- a/arch/ia64/kernel/minstate.h +++ b/arch/ia64/kernel/minstate.h @@ -16,7 +16,7 @@ #define ACCOUNT_SYS_ENTER #endif -.section ".data.patch.rse", "a" +.section ".kernel.data.patch.rse", "a" .previous /* @@ -215,7 +215,7 @@ (pUStk) extr.u r17=r18,3,6; \ (pUStk) sub r16=r18,r22; \ [1:](pKStk) br.cond.sptk.many 1f; \ - .xdata4 ".data.patch.rse",1b-. \ + .xdata4 ".kernel.data.patch.rse",1b-. \ ;; \ cmp.ge p6,p7 = 33,r17; \ ;; \ diff --git a/arch/ia64/kernel/paravirtentry.S b/arch/ia64/kernel/paravirtentry.S index 2f42fcb..418a5f4 100644 --- a/arch/ia64/kernel/paravirtentry.S +++ b/arch/ia64/kernel/paravirtentry.S @@ -24,12 +24,12 @@ #include <asm/asm-offsets.h> #include "entry.h" -#define DATA8(sym, init_value) \ - .pushsection .data.read_mostly ; \ - .align 8 ; \ - .global sym ; \ - sym: ; \ - data8 init_value ; \ +#define DATA8(sym, init_value) \ + .pushsection .kernel.data.read_mostly,"aw",@progbits ; \ + .align 8 ; \ + .global sym ; \ + sym: ; \ + data8 init_value ; \ .popsection #define BRANCH(targ, reg, breg) \ diff --git a/arch/ia64/kernel/vmlinux.lds.S b/arch/ia64/kernel/vmlinux.lds.S index 10a7d47..6e4e8a2 100644 --- a/arch/ia64/kernel/vmlinux.lds.S +++ b/arch/ia64/kernel/vmlinux.lds.S @@ -8,7 +8,7 @@ #define IVT_TEXT \ VMLINUX_SYMBOL(__start_ivt_text) = .; \ - *(.text.ivt) \ + *(.kernel.text.ivt) \ VMLINUX_SYMBOL(__end_ivt_text) = .; OUTPUT_FORMAT("elf64-ia64-little") @@ -51,13 +51,13 @@ SECTIONS KPROBES_TEXT *(.gnu.linkonce.t*) } - .text.head : AT(ADDR(.text.head) - LOAD_OFFSET) - { *(.text.head) } + .kernel.text.head : AT(ADDR(.kernel.text.head) - LOAD_OFFSET) + { *(.kernel.text.head) } .text2 : AT(ADDR(.text2) - LOAD_OFFSET) { *(.text2) } #ifdef CONFIG_SMP - .text.lock : AT(ADDR(.text.lock) - LOAD_OFFSET) - { *(.text.lock) } + .kernel.text.lock : AT(ADDR(.kernel.text.lock) - LOAD_OFFSET) + { *(.kernel.text.lock) } #endif _etext = .; @@ -84,10 +84,10 @@ SECTIONS __stop___mca_table = .; } - .data.patch.phys_stack_reg : AT(ADDR(.data.patch.phys_stack_reg) - LOAD_OFFSET) + .kernel.data.patch.phys_stack_reg : AT(ADDR(.kernel.data.patch.phys_stack_reg) - LOAD_OFFSET) { __start___phys_stack_reg_patchlist = .; - *(.data.patch.phys_stack_reg) + *(.kernel.data.patch.phys_stack_reg) __end___phys_stack_reg_patchlist = .; } @@ -148,24 +148,24 @@ SECTIONS __initcall_end = .; } - .data.patch.vtop : AT(ADDR(.data.patch.vtop) - LOAD_OFFSET) + .kernel.data.patch.vtop : AT(ADDR(.kernel.data.patch.vtop) - LOAD_OFFSET) { __start___vtop_patchlist = .; - *(.data.patch.vtop) + *(.kernel.data.patch.vtop) __end___vtop_patchlist = .; } - .data.patch.rse : AT(ADDR(.data.patch.rse) - LOAD_OFFSET) + .kernel.data.patch.rse : AT(ADDR(.kernel.data.patch.rse) - LOAD_OFFSET) { __start___rse_patchlist = .; - *(.data.patch.rse) + *(.kernel.data.patch.rse) __end___rse_patchlist = .; } - .data.patch.mckinley_e9 : AT(ADDR(.data.patch.mckinley_e9) - LOAD_OFFSET) + .kernel.data.patch.mckinley_e9 : AT(ADDR(.kernel.data.patch.mckinley_e9) - LOAD_OFFSET) { __start___mckinley_e9_bundles = .; - *(.data.patch.mckinley_e9) + *(.kernel.data.patch.mckinley_e9) __end___mckinley_e9_bundles = .; } @@ -193,34 +193,34 @@ SECTIONS __init_end = .; /* The initial task and kernel stack */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) - { *(.data.init_task) } + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) + { *(.kernel.data.init_task) } - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { *(__special_page_section) __start_gate_section = .; - *(.data.gate) + *(.kernel.data.gate) __stop_gate_section = .; } . = ALIGN(PAGE_SIZE); /* make sure the gate page doesn't expose * kernel data */ - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) - { *(.data.read_mostly) } + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) + { *(.kernel.data.read_mostly) } - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) - { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) + { *(.kernel.data.cacheline_aligned) } /* Per-cpu data: */ percpu : { } :percpu . = ALIGN(PERCPU_PAGE_SIZE); __phys_per_cpu_start = .; - .data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) + .kernel.data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } . = __phys_per_cpu_start + PERCPU_PAGE_SIZE; /* ensure percpu data fits diff --git a/arch/ia64/kvm/vmm_ivt.S b/arch/ia64/kvm/vmm_ivt.S index c1d7251..935295e 100644 --- a/arch/ia64/kvm/vmm_ivt.S +++ b/arch/ia64/kvm/vmm_ivt.S @@ -97,7 +97,7 @@ END(kvm_panic) - .section .text.ivt,"ax" + .section .kernel.text.ivt,"ax" .align 32768 // align on 32KB boundary .global kvm_ia64_ivt diff --git a/arch/ia64/xen/xensetup.S b/arch/ia64/xen/xensetup.S index 28fed1f..ebeadf4 100644 --- a/arch/ia64/xen/xensetup.S +++ b/arch/ia64/xen/xensetup.S @@ -14,7 +14,7 @@ #include <linux/init.h> #include <xen/interface/elfnote.h> - .section .data.read_mostly + .section .kernel.data.read_mostly, "a" .align 8 .global xen_domain_type xen_domain_type: diff --git a/arch/m32r/kernel/init_task.c b/arch/m32r/kernel/init_task.c index 0d658db..3a97818 100644 --- a/arch/m32r/kernel/init_task.c +++ b/arch/m32r/kernel/init_task.c @@ -26,7 +26,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/m32r/kernel/vmlinux.lds.S b/arch/m32r/kernel/vmlinux.lds.S index 9db05df..97e3cec 100644 --- a/arch/m32r/kernel/vmlinux.lds.S +++ b/arch/m32r/kernel/vmlinux.lds.S @@ -57,17 +57,17 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : { *(.kernel.data.nosave) } . = ALIGN(4096); __nosave_end = .; . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } _edata = .; /* End of data section */ . = ALIGN(8192); /* init_task */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* will be freed after init */ . = ALIGN(4096); /* Init code and data */ diff --git a/arch/m68k/kernel/head.S b/arch/m68k/kernel/head.S index f513f53..eb51357 100644 --- a/arch/m68k/kernel/head.S +++ b/arch/m68k/kernel/head.S @@ -577,7 +577,7 @@ func_define putn,1 #endif .endm -.section ".text.head","ax" +.section ".kernel.text.head","ax" ENTRY(_stext) /* * Version numbers of the bootinfo interface diff --git a/arch/m68k/kernel/process.c b/arch/m68k/kernel/process.c index 3042c2b..85b8a14 100644 --- a/arch/m68k/kernel/process.c +++ b/arch/m68k/kernel/process.c @@ -48,7 +48,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); EXPORT_SYMBOL(init_mm); union thread_union init_thread_union -__attribute__((section(".data.init_task"), aligned(THREAD_SIZE))) +__attribute__((section(".kernel.data.init_task"), aligned(THREAD_SIZE))) = { INIT_THREAD_INFO(init_task) }; /* initial task structure */ diff --git a/arch/m68k/kernel/sun3-head.S b/arch/m68k/kernel/sun3-head.S index aad0159..28d3a41 100644 --- a/arch/m68k/kernel/sun3-head.S +++ b/arch/m68k/kernel/sun3-head.S @@ -29,7 +29,7 @@ kernel_pmd_table: .skip 0x2000 .globl kernel_pg_dir .equ kernel_pg_dir,kernel_pmd_table - .section .text.head + .section .kernel.text.head, "ax" ENTRY(_stext) ENTRY(_start) diff --git a/arch/m68k/kernel/vmlinux-std.lds b/arch/m68k/kernel/vmlinux-std.lds index f846d4e..dc1e630 100644 --- a/arch/m68k/kernel/vmlinux-std.lds +++ b/arch/m68k/kernel/vmlinux-std.lds @@ -12,7 +12,7 @@ SECTIONS . = 0x1000; _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -35,7 +35,7 @@ SECTIONS } . = ALIGN(16); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } :data .bss : { *(.bss) } /* BSS */ @@ -78,7 +78,7 @@ SECTIONS . = ALIGN(8192); __init_end = .; - .data.init_task : { *(.data.init_task) } /* The initial task and kernel stack */ + .kernel.data.init_task : { *(.kernel.data.init_task) } /* The initial task and kernel stack */ _end = . ; diff --git a/arch/m68k/kernel/vmlinux-sun3.lds b/arch/m68k/kernel/vmlinux-sun3.lds index 8a4919e..2c55ffc 100644 --- a/arch/m68k/kernel/vmlinux-sun3.lds +++ b/arch/m68k/kernel/vmlinux-sun3.lds @@ -12,7 +12,7 @@ SECTIONS . = 0xE002000; _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -69,7 +69,7 @@ __init_begin = .; #endif . = ALIGN(PAGE_SIZE); __init_end = .; - .data.init.task : { *(.data.init_task) } + .kernel.data.init.task : { *(.kernel.data.init_task) } .bss : { *(.bss) } /* BSS */ diff --git a/arch/m68knommu/kernel/init_task.c b/arch/m68knommu/kernel/init_task.c index 344c01a..42ffc74 100644 --- a/arch/m68knommu/kernel/init_task.c +++ b/arch/m68knommu/kernel/init_task.c @@ -37,6 +37,6 @@ EXPORT_SYMBOL(init_task); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/m68knommu/kernel/vmlinux.lds.S b/arch/m68knommu/kernel/vmlinux.lds.S index 69ba9b1..d5d617c 100644 --- a/arch/m68knommu/kernel/vmlinux.lds.S +++ b/arch/m68knommu/kernel/vmlinux.lds.S @@ -55,7 +55,7 @@ SECTIONS { .romvec : { __rom_start = . ; _romvec = .; - *(.data.initvect) + *(.kernel.data.initvect) } > romvec #endif @@ -66,7 +66,7 @@ SECTIONS { TEXT_TEXT SCHED_TEXT LOCK_TEXT - *(.text.lock) + *(.kernel.text.lock) . = ALIGN(16); /* Exception table */ __start___ex_table = .; @@ -148,7 +148,7 @@ SECTIONS { _sdata = . ; DATA_DATA . = ALIGN(8192) ; - *(.data.init_task) + *(.kernel.data.init_task) _edata = . ; } > DATA diff --git a/arch/m68knommu/platform/68360/head-ram.S b/arch/m68knommu/platform/68360/head-ram.S index 2ef0624..88a02ed 100644 --- a/arch/m68knommu/platform/68360/head-ram.S +++ b/arch/m68knommu/platform/68360/head-ram.S @@ -280,7 +280,7 @@ _dprbase: * and then overwritten as needed. */ -.section ".data.initvect","awx" +.section ".kernel.data.initvect","awx" .long RAMEND /* Reset: Initial Stack Pointer - 0. */ .long _start /* Reset: Initial Program Counter - 1. */ .long buserr /* Bus Error - 2. */ diff --git a/arch/m68knommu/platform/68360/head-rom.S b/arch/m68knommu/platform/68360/head-rom.S index 62ecf41..b4da6af 100644 --- a/arch/m68knommu/platform/68360/head-rom.S +++ b/arch/m68knommu/platform/68360/head-rom.S @@ -291,7 +291,7 @@ _dprbase: * and then overwritten as needed. */ -.section ".data.initvect","awx" +.section ".kernel.data.initvect","awx" .long RAMEND /* Reset: Initial Stack Pointer - 0. */ .long _start /* Reset: Initial Program Counter - 1. */ .long buserr /* Bus Error - 2. */ diff --git a/arch/mips/kernel/init_task.c b/arch/mips/kernel/init_task.c index d72487a..55ef3f3 100644 --- a/arch/mips/kernel/init_task.c +++ b/arch/mips/kernel/init_task.c @@ -27,7 +27,7 @@ EXPORT_SYMBOL(init_mm); * The things we do for performance.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"), + __attribute__((__section__(".kernel.data.init_task"), __aligned__(THREAD_SIZE))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/mips/kernel/vmlinux.lds.S b/arch/mips/kernel/vmlinux.lds.S index 58738c8..c9a89d8 100644 --- a/arch/mips/kernel/vmlinux.lds.S +++ b/arch/mips/kernel/vmlinux.lds.S @@ -77,7 +77,7 @@ SECTIONS * object file alignment. Using 32768 */ . = ALIGN(_PAGE_SIZE); - *(.data.init_task) + *(.kernel.data.init_task) DATA_DATA CONSTRUCTORS @@ -99,14 +99,14 @@ SECTIONS . = ALIGN(_PAGE_SIZE); .data_nosave : { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(_PAGE_SIZE); __nosave_end = .; . = ALIGN(1 << CONFIG_MIPS_L1_CACHE_SHIFT); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } _edata = .; /* End of data section */ diff --git a/arch/mips/lasat/image/head.S b/arch/mips/lasat/image/head.S index efb95f2..d2e08b3 100644 --- a/arch/mips/lasat/image/head.S +++ b/arch/mips/lasat/image/head.S @@ -1,7 +1,7 @@ #include <asm/lasat/head.h> .text - .section .text.start, "ax" + .section .kernel.text.start, "ax" .set noreorder .set mips3 diff --git a/arch/mips/lasat/image/romscript.normal b/arch/mips/lasat/image/romscript.normal index 988f8ad..9429398 100644 --- a/arch/mips/lasat/image/romscript.normal +++ b/arch/mips/lasat/image/romscript.normal @@ -4,7 +4,7 @@ SECTIONS { .text : { - *(.text.start) + *(.kernel.text.start) } /* Data in ROM */ diff --git a/arch/mn10300/kernel/head.S b/arch/mn10300/kernel/head.S index 606bd8c..1b4b275 100644 --- a/arch/mn10300/kernel/head.S +++ b/arch/mn10300/kernel/head.S @@ -19,7 +19,7 @@ #include <asm/param.h> #include <asm/unit/serial.h> - .section .text.head,"ax" + .section .kernel.text.head,"ax" ############################################################################### # diff --git a/arch/mn10300/kernel/init_task.c b/arch/mn10300/kernel/init_task.c index af16f6e..7c7962d 100644 --- a/arch/mn10300/kernel/init_task.c +++ b/arch/mn10300/kernel/init_task.c @@ -32,7 +32,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/mn10300/kernel/vmlinux.lds.S b/arch/mn10300/kernel/vmlinux.lds.S index a3e80f4..4fd3d40 100644 --- a/arch/mn10300/kernel/vmlinux.lds.S +++ b/arch/mn10300/kernel/vmlinux.lds.S @@ -27,7 +27,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : { *( - .text.head + .kernel.text.head .text ) TEXT_TEXT @@ -57,25 +57,25 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : { *(.kernel.data.nosave) } . = ALIGN(4096); __nosave_end = .; . = ALIGN(4096); - .data.page_aligned : { *(.data.idt) } + .kernel.data.page_aligned : { *(.kernel.data.idt) } . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } /* rarely changed data like cpu maps */ . = ALIGN(32); - .data.read_mostly : AT(ADDR(.data.read_mostly)) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly)) { + *(.kernel.data.read_mostly) _edata = .; /* End of data section */ } . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* might get freed after init */ . = ALIGN(4096); @@ -128,7 +128,7 @@ SECTIONS . = ALIGN(32); __per_cpu_start = .; - .data.percpu : { *(.data.percpu) } + .kernel.data.percpu : { *(.kernel.data.percpu) } __per_cpu_end = .; . = ALIGN(4096); __init_end = .; @@ -136,7 +136,7 @@ SECTIONS __bss_start = .; /* BSS */ .bss : { - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) } . = ALIGN(4); diff --git a/arch/parisc/include/asm/cache.h b/arch/parisc/include/asm/cache.h index 32c2cca..d1e16aa 100644 --- a/arch/parisc/include/asm/cache.h +++ b/arch/parisc/include/asm/cache.h @@ -28,7 +28,7 @@ #define SMP_CACHE_BYTES L1_CACHE_BYTES -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) void parisc_cache_init(void); /* initializes cache-flushing */ void disable_sr_hashing_asm(int); /* low level support for above */ diff --git a/arch/parisc/include/asm/system.h b/arch/parisc/include/asm/system.h index ee80c92..7a6ac0c 100644 --- a/arch/parisc/include/asm/system.h +++ b/arch/parisc/include/asm/system.h @@ -174,7 +174,7 @@ static inline void set_eiem(unsigned long val) }) #ifdef CONFIG_SMP -# define __lock_aligned __attribute__((__section__(".data.lock_aligned"))) +# define __lock_aligned __attribute__((__section__(".kernel.data.lock_aligned"))) #endif #define arch_align_stack(x) (x) diff --git a/arch/parisc/kernel/head.S b/arch/parisc/kernel/head.S index 0e3d9f9..9112df2 100644 --- a/arch/parisc/kernel/head.S +++ b/arch/parisc/kernel/head.S @@ -345,7 +345,7 @@ smp_slave_stext: ENDPROC(stext) #ifndef CONFIG_64BIT - .section .data.read_mostly + .section .kernel.data.read_mostly,"aw",@progbits .align 4 .export $global$,data diff --git a/arch/parisc/kernel/init_task.c b/arch/parisc/kernel/init_task.c index f5941c0..dd741b2 100644 --- a/arch/parisc/kernel/init_task.c +++ b/arch/parisc/kernel/init_task.c @@ -49,7 +49,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((aligned(128))) __attribute__((__section__(".data.init_task"))) = + __attribute__((aligned(128))) __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; #if PT_NLEVELS == 3 @@ -58,11 +58,11 @@ union thread_union init_thread_union * guarantee that global objects will be laid out in memory in the same order * as the order of declaration, so put these in different sections and use * the linker script to order them. */ -pmd_t pmd0[PTRS_PER_PMD] __attribute__ ((__section__ (".data.vm0.pmd"), aligned(PAGE_SIZE))); +pmd_t pmd0[PTRS_PER_PMD] __attribute__ ((__section__ (".kernel.data.vm0.pmd"), aligned(PAGE_SIZE))); #endif -pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__ ((__section__ (".data.vm0.pgd"), aligned(PAGE_SIZE))); -pte_t pg0[PT_INITIAL * PTRS_PER_PTE] __attribute__ ((__section__ (".data.vm0.pte"), aligned(PAGE_SIZE))); +pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__ ((__section__ (".kernel.data.vm0.pgd"), aligned(PAGE_SIZE))); +pte_t pg0[PT_INITIAL * PTRS_PER_PTE] __attribute__ ((__section__ (".kernel.data.vm0.pte"), aligned(PAGE_SIZE))); /* * Initial task structure. diff --git a/arch/parisc/kernel/vmlinux.lds.S b/arch/parisc/kernel/vmlinux.lds.S index 1a3b6cc..861f15b 100644 --- a/arch/parisc/kernel/vmlinux.lds.S +++ b/arch/parisc/kernel/vmlinux.lds.S @@ -94,8 +94,8 @@ SECTIONS /* rarely changed data like cpu maps */ . = ALIGN(16); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } . = ALIGN(L1_CACHE_BYTES); @@ -106,14 +106,14 @@ SECTIONS } . = ALIGN(L1_CACHE_BYTES); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } /* PA-RISC locks requires 16-byte alignment */ . = ALIGN(16); - .data.lock_aligned : { - *(.data.lock_aligned) + .kernel.data.lock_aligned : { + *(.kernel.data.lock_aligned) } /* nosave data is really only used for software suspend...it's here @@ -122,7 +122,7 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; .data_nosave : { - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -134,10 +134,10 @@ SECTIONS __bss_start = .; /* page table entries need to be PAGE_SIZE aligned */ . = ALIGN(PAGE_SIZE); - .data.vmpages : { - *(.data.vm0.pmd) - *(.data.vm0.pgd) - *(.data.vm0.pte) + .kernel.data.vmpages : { + *(.kernel.data.vm0.pmd) + *(.kernel.data.vm0.pgd) + *(.kernel.data.vm0.pte) } .bss : { *(.bss) @@ -149,8 +149,8 @@ SECTIONS /* assembler code expects init_task to be 16k aligned */ . = ALIGN(16384); /* init_task */ - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } #ifdef CONFIG_64BIT diff --git a/arch/powerpc/include/asm/cache.h b/arch/powerpc/include/asm/cache.h index 81de6eb..69ead95 100644 --- a/arch/powerpc/include/asm/cache.h +++ b/arch/powerpc/include/asm/cache.h @@ -38,7 +38,7 @@ extern struct ppc64_caches ppc64_caches; #endif /* __powerpc64__ && ! __ASSEMBLY__ */ #if !defined(__ASSEMBLY__) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif #endif /* __KERNEL__ */ diff --git a/arch/powerpc/include/asm/page_64.h b/arch/powerpc/include/asm/page_64.h index 043bfdf..2e5f76b 100644 --- a/arch/powerpc/include/asm/page_64.h +++ b/arch/powerpc/include/asm/page_64.h @@ -157,7 +157,7 @@ do { \ #else #define __page_aligned \ __attribute__((__aligned__(PAGE_SIZE), \ - __section__(".data.page_aligned"))) + __section__(".kernel.data.page_aligned"))) #endif #define VM_DATA_DEFAULT_FLAGS \ diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h index c4a029c..1d1ab01 100644 --- a/arch/powerpc/include/asm/ppc_asm.h +++ b/arch/powerpc/include/asm/ppc_asm.h @@ -193,7 +193,7 @@ name: \ GLUE(.,name): #define _INIT_GLOBAL(name) \ - .section ".text.init.refok"; \ + .section ".kernel.text.init.refok","ax",@progbits; \ .align 2 ; \ .globl name; \ .globl GLUE(.,name); \ @@ -233,7 +233,7 @@ name: \ GLUE(.,name): #define _INIT_STATIC(name) \ - .section ".text.init.refok"; \ + .section ".kernel.text.init.refok","ax",@progbits; \ .align 2 ; \ .section ".opd","aw"; \ name: \ diff --git a/arch/powerpc/kernel/head_32.S b/arch/powerpc/kernel/head_32.S index 0c32682..87299a0 100644 --- a/arch/powerpc/kernel/head_32.S +++ b/arch/powerpc/kernel/head_32.S @@ -49,7 +49,7 @@ mtspr SPRN_DBAT##n##L,RB; \ 1: - .section .text.head, "ax" + .section .kernel.text.head, "ax" .stabs "arch/powerpc/kernel/",N_SO,0,0,0f .stabs "head_32.S",N_SO,0,0,0f 0: diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 56d8e5d..d18a199 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -52,7 +52,7 @@ * * This is all going to change RSN when we add bi_recs....... -- Dan */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); diff --git a/arch/powerpc/kernel/head_44x.S b/arch/powerpc/kernel/head_44x.S index f3a1ea9..d843c4a 100644 --- a/arch/powerpc/kernel/head_44x.S +++ b/arch/powerpc/kernel/head_44x.S @@ -50,7 +50,7 @@ * r7 - End of kernel command line string * */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); /* diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index 3c9452d..549a236 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -38,7 +38,7 @@ #else #define DO_8xx_CPU6(val, reg) #endif - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S index 590304c..aace79e 100644 --- a/arch/powerpc/kernel/head_fsl_booke.S +++ b/arch/powerpc/kernel/head_fsl_booke.S @@ -53,7 +53,7 @@ * r7 - End of kernel command line string * */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); /* diff --git a/arch/powerpc/kernel/init_task.c b/arch/powerpc/kernel/init_task.c index 4c85b8d..d331c22 100644 --- a/arch/powerpc/kernel/init_task.c +++ b/arch/powerpc/kernel/init_task.c @@ -22,7 +22,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/powerpc/kernel/machine_kexec_64.c b/arch/powerpc/kernel/machine_kexec_64.c index 3c4ca04..5140895 100644 --- a/arch/powerpc/kernel/machine_kexec_64.c +++ b/arch/powerpc/kernel/machine_kexec_64.c @@ -250,7 +250,7 @@ static void kexec_prepare_cpus(void) * current, but that audit has not been performed. */ static union thread_union kexec_stack - __attribute__((__section__(".data.init_task"))) = { }; + __attribute__((__section__(".kernel.data.init_task"))) = { }; /* Our assembly helper, in kexec_stub.S */ extern NORET_TYPE void kexec_sequence(void *newstack, unsigned long start, diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c index 65639a4..2737099 100644 --- a/arch/powerpc/kernel/vdso.c +++ b/arch/powerpc/kernel/vdso.c @@ -74,7 +74,7 @@ static int vdso_ready; static union { struct vdso_data data; u8 page[PAGE_SIZE]; -} vdso_data_store __attribute__((__section__(".data.page_aligned"))); +} vdso_data_store __attribute__((__section__(".kernel.data.page_aligned"))); struct vdso_data *vdso_data = &vdso_data_store.data; /* Format of the patch table */ diff --git a/arch/powerpc/kernel/vdso32/vdso32_wrapper.S b/arch/powerpc/kernel/vdso32/vdso32_wrapper.S index 556f0ca..e1b3e41 100644 --- a/arch/powerpc/kernel/vdso32/vdso32_wrapper.S +++ b/arch/powerpc/kernel/vdso32/vdso32_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned","aw",@progbits .globl vdso32_start, vdso32_end .balign PAGE_SIZE diff --git a/arch/powerpc/kernel/vdso64/vdso64_wrapper.S b/arch/powerpc/kernel/vdso64/vdso64_wrapper.S index 0529cb9..1d96b15 100644 --- a/arch/powerpc/kernel/vdso64/vdso64_wrapper.S +++ b/arch/powerpc/kernel/vdso64/vdso64_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned","aw",@progbits .globl vdso64_start, vdso64_end .balign PAGE_SIZE diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S index 2412c05..6cf1aad 100644 --- a/arch/powerpc/kernel/vmlinux.lds.S +++ b/arch/powerpc/kernel/vmlinux.lds.S @@ -52,9 +52,9 @@ SECTIONS /* Text and gots */ .text : AT(ADDR(.text) - LOAD_OFFSET) { ALIGN_FUNCTION(); - *(.text.head) + *(.kernel.text.head) _text = .; - *(.text .fixup .text.init.refok .exit.text.refok __ftr_alt_*) + *(.text .fixup .kernel.text.init.refok .kernel.text.exit.refok __ftr_alt_*) SCHED_TEXT LOCK_TEXT KPROBES_TEXT @@ -174,10 +174,10 @@ SECTIONS } #endif . = ALIGN(PAGE_SIZE); - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } @@ -251,28 +251,28 @@ SECTIONS #else . = ALIGN(16384); #endif - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) } . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) } - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } . = ALIGN(L1_CACHE_BYTES); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) } . = ALIGN(PAGE_SIZE); .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; } diff --git a/arch/s390/include/asm/cache.h b/arch/s390/include/asm/cache.h index 9b86681..2890a0f 100644 --- a/arch/s390/include/asm/cache.h +++ b/arch/s390/include/asm/cache.h @@ -14,6 +14,6 @@ #define L1_CACHE_BYTES 256 #define L1_CACHE_SHIFT 8 -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif diff --git a/arch/s390/kernel/head.S b/arch/s390/kernel/head.S index 83477c7..b6e58bf 100644 --- a/arch/s390/kernel/head.S +++ b/arch/s390/kernel/head.S @@ -35,7 +35,7 @@ #define ARCH_OFFSET 0 #endif -.section ".text.head","ax" +.section ".kernel.text.head","ax" #ifndef CONFIG_IPL .org 0 .long 0x00080000,0x80000000+startup # Just a restart PSW diff --git a/arch/s390/kernel/init_task.c b/arch/s390/kernel/init_task.c index e807168..91e8014 100644 --- a/arch/s390/kernel/init_task.c +++ b/arch/s390/kernel/init_task.c @@ -31,7 +31,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S index d796d05..873cac2 100644 --- a/arch/s390/kernel/vmlinux.lds.S +++ b/arch/s390/kernel/vmlinux.lds.S @@ -29,7 +29,7 @@ SECTIONS . = 0x00000000; .text : { _text = .; /* Text and read-only data */ - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -66,30 +66,30 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data_nosave : { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; . = ALIGN(PAGE_SIZE); - .data.page_aligned : { - *(.data.idt) + .kernel.data.page_aligned : { + *(.kernel.data.idt) } . = ALIGN(0x100); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } . = ALIGN(0x100); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } _edata = .; /* End of data section */ . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } /* will be freed after init */ diff --git a/arch/sh/include/asm/cache.h b/arch/sh/include/asm/cache.h index 02df18e..f44fed5 100644 --- a/arch/sh/include/asm/cache.h +++ b/arch/sh/include/asm/cache.h @@ -14,7 +14,7 @@ #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifndef __ASSEMBLY__ struct cache_info { diff --git a/arch/sh/kernel/cpu/sh5/entry.S b/arch/sh/kernel/cpu/sh5/entry.S index e640c63..43d031a 100644 --- a/arch/sh/kernel/cpu/sh5/entry.S +++ b/arch/sh/kernel/cpu/sh5/entry.S @@ -2058,10 +2058,10 @@ asm_uaccess_end: /* - * --- .text.init Section + * --- .kernel.text.init Section */ - .section .text.init, "ax" + .section .kernel.text.init, "ax" /* * void trap_init (void) diff --git a/arch/sh/kernel/head_32.S b/arch/sh/kernel/head_32.S index ae0a382..c99328e 100644 --- a/arch/sh/kernel/head_32.S +++ b/arch/sh/kernel/head_32.S @@ -40,7 +40,7 @@ ENTRY(empty_zero_page) 1: .skip PAGE_SIZE - empty_zero_page - 1b - .section .text.head, "ax" + .section .kernel.text.head, "ax" /* * Condition at the entry of _stext: diff --git a/arch/sh/kernel/head_64.S b/arch/sh/kernel/head_64.S index 7ccfb99..cf6ca73 100644 --- a/arch/sh/kernel/head_64.S +++ b/arch/sh/kernel/head_64.S @@ -110,7 +110,7 @@ empty_bad_pte_table: fpu_in_use: .quad 0 - .section .text.head, "ax" + .section .kernel.text.head, "ax" .balign L1_CACHE_BYTES /* * Condition at the entry of __stext: diff --git a/arch/sh/kernel/init_task.c b/arch/sh/kernel/init_task.c index b151a25..4993b2c 100644 --- a/arch/sh/kernel/init_task.c +++ b/arch/sh/kernel/init_task.c @@ -22,7 +22,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/sh/kernel/irq.c b/arch/sh/kernel/irq.c index 64b7690..9df37f6 100644 --- a/arch/sh/kernel/irq.c +++ b/arch/sh/kernel/irq.c @@ -158,10 +158,10 @@ asmlinkage int do_IRQ(unsigned int irq, struct pt_regs *regs) #ifdef CONFIG_IRQSTACKS static char softirq_stack[NR_CPUS * THREAD_SIZE] - __attribute__((__section__(".bss.page_aligned"))); + __attribute__((__section__(".bss.kernel.page_aligned"))); static char hardirq_stack[NR_CPUS * THREAD_SIZE] - __attribute__((__section__(".bss.page_aligned"))); + __attribute__((__section__(".bss.kernel.page_aligned"))); /* * allocate per-cpu stacks for hardirq and for softirq processing diff --git a/arch/sh/kernel/vmlinux_32.lds.S b/arch/sh/kernel/vmlinux_32.lds.S index 7b4b82b..04dd19f 100644 --- a/arch/sh/kernel/vmlinux_32.lds.S +++ b/arch/sh/kernel/vmlinux_32.lds.S @@ -28,7 +28,7 @@ SECTIONS } = 0 .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -58,19 +58,19 @@ SECTIONS . = ALIGN(THREAD_SIZE); .data : { /* Data */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) . = ALIGN(L1_CACHE_BYTES); - *(.data.read_mostly) + *(.kernel.data.read_mostly) . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -128,7 +128,7 @@ SECTIONS .bss : { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) *(COMMON) . = ALIGN(4); diff --git a/arch/sh/kernel/vmlinux_64.lds.S b/arch/sh/kernel/vmlinux_64.lds.S index 33fa464..6059a49 100644 --- a/arch/sh/kernel/vmlinux_64.lds.S +++ b/arch/sh/kernel/vmlinux_64.lds.S @@ -42,7 +42,7 @@ SECTIONS } = 0 .text : C_PHYS(.text) { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT *(.text64) *(.text..SHmedia32) @@ -70,19 +70,19 @@ SECTIONS . = ALIGN(THREAD_SIZE); .data : C_PHYS(.data) { /* Data */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) . = ALIGN(L1_CACHE_BYTES); - *(.data.read_mostly) + *(.kernel.data.read_mostly) . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -140,7 +140,7 @@ SECTIONS .bss : C_PHYS(.bss) { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) *(COMMON) . = ALIGN(4); diff --git a/arch/sparc/boot/btfixupprep.c b/arch/sparc/boot/btfixupprep.c index 52a4208..5f8472b 100644 --- a/arch/sparc/boot/btfixupprep.c +++ b/arch/sparc/boot/btfixupprep.c @@ -171,7 +171,7 @@ main1: } } else if (buffer[nbase+4] != '_') continue; - if (!strcmp (sect, ".text.exit")) + if (!strcmp (sect, ".kernel.text.exit")) continue; if (strcmp (sect, ".text") && strcmp (sect, ".init.text") && @@ -325,7 +325,7 @@ main1: (*rr)->next = NULL; } printf("! Generated by btfixupprep. Do not edit.\n\n"); - printf("\t.section\t\".data.init\",#alloc,#write\n\t.align\t4\n\n"); + printf("\t.section\t\".kernel.data.init\",#alloc,#write\n\t.align\t4\n\n"); printf("\t.global\t___btfixup_start\n___btfixup_start:\n\n"); for (i = 0; i < last; i++) { f = array + i; diff --git a/arch/sparc/include/asm/cache.h b/arch/sparc/include/asm/cache.h index 41f85ae..19f1a48 100644 --- a/arch/sparc/include/asm/cache.h +++ b/arch/sparc/include/asm/cache.h @@ -19,7 +19,7 @@ #define SMP_CACHE_BYTES (1 << SMP_CACHE_BYTES_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifdef CONFIG_SPARC32 #include <asm/asi.h> diff --git a/arch/sparc/kernel/head.S b/arch/sparc/kernel/head.S index 51b4042..8e7d300 100644 --- a/arch/sparc/kernel/head.S +++ b/arch/sparc/kernel/head.S @@ -735,7 +735,7 @@ go_to_highmem: nop /* The code above should be at beginning and we have to take care about - * short jumps, as branching to .text.init section from .text is usually + * short jumps, as branching to .kernel.text.init section from .text is usually * impossible */ __INIT /* Acquire boot time privileged register values, this will help debugging. diff --git a/arch/sparc/kernel/vmlinux.lds.S b/arch/sparc/kernel/vmlinux.lds.S index 5b7e69a..c1b13b0 100644 --- a/arch/sparc/kernel/vmlinux.lds.S +++ b/arch/sparc/kernel/vmlinux.lds.S @@ -87,12 +87,12 @@ SECTIONS . = ALIGN(PAGE_SIZE); __init_end = .; . = ALIGN(32); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } . = ALIGN(32); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } __bss_start = .; diff --git a/arch/sparc64/kernel/head.S b/arch/sparc64/kernel/head.S index 353226f..a5fb164 100644 --- a/arch/sparc64/kernel/head.S +++ b/arch/sparc64/kernel/head.S @@ -467,7 +467,7 @@ jump_to_sun4u_init: jmpl %g2 + %g0, %g0 nop - .section .text.init.refok + .section .kernel.text.init.refok,"ax",@progbits sun4u_init: BRANCH_IF_SUN4V(g1, sun4v_init) diff --git a/arch/sparc64/kernel/vmlinux.lds.S b/arch/sparc64/kernel/vmlinux.lds.S index 01f8096..863dd19 100644 --- a/arch/sparc64/kernel/vmlinux.lds.S +++ b/arch/sparc64/kernel/vmlinux.lds.S @@ -32,12 +32,12 @@ SECTIONS *(.data1) } . = ALIGN(64); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } . = ALIGN(64); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } _edata = .; PROVIDE (edata = .); diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S index cb02486..9ac4b36 100644 --- a/arch/um/include/asm/common.lds.S +++ b/arch/um/include/asm/common.lds.S @@ -49,9 +49,9 @@ } . = ALIGN(32); - .data.percpu : { + .kernel.data.percpu : { __per_cpu_start = . ; - *(.data.percpu) + *(.kernel.data.percpu) __per_cpu_end = . ; } diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S index 9975e1a..1ee6934 100644 --- a/arch/um/kernel/dyn.lds.S +++ b/arch/um/kernel/dyn.lds.S @@ -97,9 +97,9 @@ SECTIONS .fini_array : { *(.fini_array) } .data : { . = ALIGN(KERNEL_STACK_SIZE); /* init_task */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(KERNEL_STACK_SIZE); - *(.data.init_irqstack) + *(.kernel.data.init_irqstack) DATA_DATA *(.data.* .gnu.linkonce.d.*) SORT(CONSTRUCTORS) diff --git a/arch/um/kernel/init_task.c b/arch/um/kernel/init_task.c index 910eda8..6dd34e6 100644 --- a/arch/um/kernel/init_task.c +++ b/arch/um/kernel/init_task.c @@ -35,9 +35,9 @@ EXPORT_SYMBOL(init_task); */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; union thread_union cpu0_irqstack - __attribute__((__section__(".data.init_irqstack"))) = + __attribute__((__section__(".kernel.data.init_irqstack"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S index 11b8352..a0b6650 100644 --- a/arch/um/kernel/uml.lds.S +++ b/arch/um/kernel/uml.lds.S @@ -53,9 +53,9 @@ SECTIONS .data : { . = ALIGN(KERNEL_STACK_SIZE); /* init_task */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(KERNEL_STACK_SIZE); - *(.data.init_irqstack) + *(.kernel.data.init_irqstack) DATA_DATA *(.gnu.linkonce.d*) CONSTRUCTORS diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 29c5fbf..2c89079 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -29,7 +29,7 @@ #include <asm/boot.h> #include <asm/asm-offsets.h> -.section ".text.head","ax",@progbits +.section ".kernel.text.head","ax",@progbits .globl startup_32 startup_32: diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 1d5dff4..87bfcbe 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -33,7 +33,7 @@ #include <asm/processor-flags.h> #include <asm/asm-offsets.h> -.section ".text.head" +.section ".kernel.text.head","ax",@progbits .code32 .globl startup_32 diff --git a/arch/x86/boot/compressed/vmlinux.scr b/arch/x86/boot/compressed/vmlinux.scr index f02382a..e4dc231 100644 --- a/arch/x86/boot/compressed/vmlinux.scr +++ b/arch/x86/boot/compressed/vmlinux.scr @@ -1,6 +1,6 @@ SECTIONS { - .rodata.compressed : { + .kernel.rodata.compressed : { input_len = .; LONG(input_data_end - input_data) input_data = .; *(.data) diff --git a/arch/x86/boot/compressed/vmlinux_32.lds b/arch/x86/boot/compressed/vmlinux_32.lds index bb3c483..a24e272 100644 --- a/arch/x86/boot/compressed/vmlinux_32.lds +++ b/arch/x86/boot/compressed/vmlinux_32.lds @@ -7,13 +7,13 @@ SECTIONS * address 0. */ . = 0; - .text.head : { + .kernel.text.head : { _head = . ; - *(.text.head) + *(.kernel.text.head) _ehead = . ; } - .rodata.compressed : { - *(.rodata.compressed) + .kernel.rodata.compressed : { + *(.kernel.rodata.compressed) } .text : { _text = .; /* Text */ @@ -21,6 +21,10 @@ SECTIONS *(.text.*) _etext = . ; } + .got : { + *(.got) + *(.got.*) + } .rodata : { _rodata = . ; *(.rodata) /* read-only data */ @@ -40,4 +44,6 @@ SECTIONS *(COMMON) _end = . ; } + /* Be bold, and discard everything not explicitly mentioned */ + /DISCARD/ : { *(*) } } diff --git a/arch/x86/boot/compressed/vmlinux_64.lds b/arch/x86/boot/compressed/vmlinux_64.lds index bef1ac8..9414e60 100644 --- a/arch/x86/boot/compressed/vmlinux_64.lds +++ b/arch/x86/boot/compressed/vmlinux_64.lds @@ -7,13 +7,13 @@ SECTIONS * address 0. */ . = 0; - .text.head : { + .kernel.text.head : { _head = . ; - *(.text.head) + *(.kernel.text.head) _ehead = . ; } - .rodata.compressed : { - *(.rodata.compressed) + .kernel.rodata.compressed : { + *(.kernel.rodata.compressed) } .text : { _text = .; /* Text */ @@ -45,4 +45,6 @@ SECTIONS . = . + 4096 * 6; _ebss = .; } + /* Be bold, and discard everything not explicitly mentioned */ + /DISCARD/ : { *(*) } } diff --git a/arch/x86/include/asm/cache.h b/arch/x86/include/asm/cache.h index 5d367ca..980a79f 100644 --- a/arch/x86/include/asm/cache.h +++ b/arch/x86/include/asm/cache.h @@ -5,7 +5,7 @@ #define L1_CACHE_SHIFT (CONFIG_X86_L1_CACHE_SHIFT) #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifdef CONFIG_X86_VSMP /* vSMP Internode cacheline shift */ @@ -13,7 +13,7 @@ #ifdef CONFIG_SMP #define __cacheline_aligned_in_smp \ __attribute__((__aligned__(1 << (INTERNODE_CACHE_SHIFT)))) \ - __attribute__((__section__(".data.page_aligned"))) + __attribute__((__section__(".kernel.data.page_aligned"))) #endif #endif diff --git a/arch/x86/kernel/acpi/wakeup_32.S b/arch/x86/kernel/acpi/wakeup_32.S index a12e6a9..1b1beb9 100644 --- a/arch/x86/kernel/acpi/wakeup_32.S +++ b/arch/x86/kernel/acpi/wakeup_32.S @@ -1,4 +1,4 @@ - .section .text.page_aligned + .section .kernel.text.page_aligned,"ax",@progbits #include <linux/linkage.h> #include <asm/segment.h> #include <asm/page.h> diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S index e835b4e..ae011eb 100644 --- a/arch/x86/kernel/head_32.S +++ b/arch/x86/kernel/head_32.S @@ -81,7 +81,7 @@ INIT_MAP_BEYOND_END = BOOTBITMAP_SIZE + (PAGE_TABLE_SIZE + ALLOCATOR_SLOP)*PAGE_ * any particular GDT layout, because we load our own as soon as we * can. */ -.section .text.head,"ax",@progbits +.section .kernel.text.head,"ax",@progbits ENTRY(startup_32) /* test KEEP_SEGMENTS flag to see if the bootloader is asking us to not reload segments */ @@ -609,7 +609,7 @@ ENTRY(_stext) /* * BSS section */ -.section ".bss.page_aligned","wa" +.section ".bss.kernel.page_aligned","wa" .align PAGE_SIZE_asm #ifdef CONFIG_X86_PAE swapper_pg_pmd: @@ -626,7 +626,7 @@ ENTRY(empty_zero_page) * This starts the data section. */ #ifdef CONFIG_X86_PAE -.section ".data.page_aligned","wa" +.section ".kernel.data.page_aligned","wa" /* Page-aligned for the benefit of paravirt? */ .align PAGE_SIZE_asm ENTRY(swapper_pg_dir) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 26cfdc1..2a9dc1e 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -40,7 +40,7 @@ L4_START_KERNEL = pgd_index(__START_KERNEL_map) L3_START_KERNEL = pud_index(__START_KERNEL_map) .text - .section .text.head + .section .kernel.text.head,"ax",@progbits .code64 .globl startup_64 startup_64: @@ -414,7 +414,7 @@ ENTRY(phys_base) ENTRY(idt_table) .skip 256 * 16 - .section .bss.page_aligned, "aw", @nobits + .section .bss.kernel.page_aligned, "aw", @nobits .align PAGE_SIZE ENTRY(empty_zero_page) .skip PAGE_SIZE diff --git a/arch/x86/kernel/init_task.c b/arch/x86/kernel/init_task.c index a4f93b4..ec4a2fe 100644 --- a/arch/x86/kernel/init_task.c +++ b/arch/x86/kernel/init_task.c @@ -24,7 +24,7 @@ EXPORT_UNUSED_SYMBOL(init_mm); /* will be removed in 2.6.26 */ * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* @@ -38,7 +38,7 @@ EXPORT_SYMBOL(init_task); /* * per-CPU TSS segments. Threads are completely 'soft' on Linux, * no more per-task TSS's. The TSS size is kept cacheline-aligned - * so they are allowed to end up in the .data.cacheline_aligned + * so they are allowed to end up in the .kernel.data.cacheline_aligned * section. Since TSS's are completely CPU-local, we want them * on exact cacheline boundaries, to eliminate cacheline ping-pong. */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 04d242a..73b634a 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -86,7 +86,7 @@ char ignore_fpu_irq; * for this. */ gate_desc idt_table[256] - __attribute__((__section__(".data.idt"))) = { { { { 0, 0 } } }, }; + __attribute__((__section__(".kernel.data.idt"))) = { { { { 0, 0 } } }, }; #endif static int ignore_nmis; diff --git a/arch/x86/kernel/vmlinux_32.lds.S b/arch/x86/kernel/vmlinux_32.lds.S index a9b8560..454ab4c 100644 --- a/arch/x86/kernel/vmlinux_32.lds.S +++ b/arch/x86/kernel/vmlinux_32.lds.S @@ -31,15 +31,15 @@ SECTIONS . = LOAD_OFFSET + LOAD_PHYSICAL_ADDR; phys_startup_32 = startup_32 - LOAD_OFFSET; - .text.head : AT(ADDR(.text.head) - LOAD_OFFSET) { + .kernel.text.head : AT(ADDR(.kernel.text.head) - LOAD_OFFSET) { _text = .; /* Text and read-only data */ - *(.text.head) + *(.kernel.text.head) } :text = 0x9090 /* read-only */ .text : AT(ADDR(.text) - LOAD_OFFSET) { . = ALIGN(PAGE_SIZE); /* not really needed, already page aligned */ - *(.text.page_aligned) + *(.kernel.text.page_aligned) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -70,32 +70,32 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; } . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) - *(.data.idt) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) + *(.kernel.data.idt) } . = ALIGN(32); - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } /* rarely changed data like cpu maps */ . = ALIGN(32); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) _edata = .; /* End of data section */ } . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) } /* might get freed after init */ @@ -178,11 +178,11 @@ SECTIONS } #endif . = ALIGN(PAGE_SIZE); - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu.page_aligned) - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu.page_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } . = ALIGN(PAGE_SIZE); @@ -191,7 +191,7 @@ SECTIONS .bss : AT(ADDR(.bss) - LOAD_OFFSET) { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) . = ALIGN(4); __bss_stop = .; diff --git a/arch/x86/kernel/vmlinux_64.lds.S b/arch/x86/kernel/vmlinux_64.lds.S index 46e0544..6fd4547 100644 --- a/arch/x86/kernel/vmlinux_64.lds.S +++ b/arch/x86/kernel/vmlinux_64.lds.S @@ -28,7 +28,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : AT(ADDR(.text) - LOAD_OFFSET) { /* First the code that has to be first for bootstrapping */ - *(.text.head) + *(.kernel.text.head) _stext = .; /* Then the rest */ TEXT_TEXT @@ -62,17 +62,17 @@ SECTIONS . = ALIGN(PAGE_SIZE); . = ALIGN(CONFIG_X86_L1_CACHE_BYTES); - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } . = ALIGN(CONFIG_X86_INTERNODE_CACHE_BYTES); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) } #define VSYSCALL_ADDR (-10*1024*1024) -#define VSYSCALL_PHYS_ADDR ((LOADADDR(.data.read_mostly) + SIZEOF(.data.read_mostly) + 4095) & ~(4095)) -#define VSYSCALL_VIRT_ADDR ((ADDR(.data.read_mostly) + SIZEOF(.data.read_mostly) + 4095) & ~(4095)) +#define VSYSCALL_PHYS_ADDR ((LOADADDR(.kernel.data.read_mostly) + SIZEOF(.kernel.data.read_mostly) + 4095) & ~(4095)) +#define VSYSCALL_VIRT_ADDR ((ADDR(.kernel.data.read_mostly) + SIZEOF(.kernel.data.read_mostly) + 4095) & ~(4095)) #define VLOAD_OFFSET (VSYSCALL_ADDR - VSYSCALL_PHYS_ADDR) #define VLOAD(x) (ADDR(x) - VLOAD_OFFSET) @@ -121,13 +121,13 @@ SECTIONS #undef VVIRT . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) }:data.init . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) } /* might get freed after init */ @@ -214,13 +214,13 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; - .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.data.nosave) } + .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; __bss_start = .; /* BSS */ .bss : AT(ADDR(.bss) - LOAD_OFFSET) { - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) } __bss_stop = .; diff --git a/arch/xtensa/kernel/head.S b/arch/xtensa/kernel/head.S index 67e6913..581c735 100644 --- a/arch/xtensa/kernel/head.S +++ b/arch/xtensa/kernel/head.S @@ -234,7 +234,7 @@ should_never_return: * BSS section */ -.section ".bss.page_aligned", "w" +.section ".bss.kernel.page_aligned", "w" ENTRY(swapper_pg_dir) .fill PAGE_SIZE, 1, 0 ENTRY(empty_zero_page) diff --git a/arch/xtensa/kernel/init_task.c b/arch/xtensa/kernel/init_task.c index 3df469d..f24808c 100644 --- a/arch/xtensa/kernel/init_task.c +++ b/arch/xtensa/kernel/init_task.c @@ -29,7 +29,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); EXPORT_SYMBOL(init_mm); union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; struct task_struct init_task = INIT_TASK(init_task); diff --git a/arch/xtensa/kernel/vmlinux.lds.S b/arch/xtensa/kernel/vmlinux.lds.S index 51f4fb6..f43f7f5 100644 --- a/arch/xtensa/kernel/vmlinux.lds.S +++ b/arch/xtensa/kernel/vmlinux.lds.S @@ -121,14 +121,14 @@ SECTIONS DATA_DATA CONSTRUCTORS . = ALIGN(XCHAL_ICACHE_LINESIZE); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) } _edata = .; /* The initial task */ . = ALIGN(8192); - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* Initialization code and data: */ @@ -259,7 +259,7 @@ SECTIONS /* BSS section */ _bss_start = .; - .bss : { *(.bss.page_aligned) *(.bss) } + .bss : { *(.bss.kernel.page_aligned) *(.bss) } _bss_end = .; _end = .; diff --git a/include/asm-frv/init.h b/include/asm-frv/init.h index 8b15838..16bea47 100644 --- a/include/asm-frv/init.h +++ b/include/asm-frv/init.h @@ -1,12 +1,12 @@ #ifndef _ASM_INIT_H #define _ASM_INIT_H -#define __init __attribute__ ((__section__ (".text.init"))) -#define __initdata __attribute__ ((__section__ (".data.init"))) +#define __init __attribute__ ((__section__ (".kernel.text.init"))) +#define __initdata __attribute__ ((__section__ (".kernel.data.init"))) /* For assembly routines */ -#define __INIT .section ".text.init",#alloc,#execinstr +#define __INIT .section ".kernel.text.init",#alloc,#execinstr #define __FINIT .previous -#define __INITDATA .section ".data.init",#alloc,#write +#define __INITDATA .section ".kernel.data.init",#alloc,#write #endif diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 8074460..e8d1bdd 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -48,7 +48,7 @@ /* .data section */ #define DATA_DATA \ *(.data) \ - *(.data.init.refok) \ + *(.kernel.data.init.refok) \ *(.ref.data) \ DEV_KEEP(init.data) \ DEV_KEEP(exit.data) \ @@ -236,8 +236,8 @@ *(.text.hot) \ *(.text) \ *(.ref.text) \ - *(.text.init.refok) \ - *(.exit.text.refok) \ + *(.kernel.text.init.refok) \ + *(.kernel.text.exit.refok) \ DEV_KEEP(init.text) \ DEV_KEEP(exit.text) \ CPU_KEEP(init.text) \ @@ -404,9 +404,9 @@ #define PERCPU(align) \ . = ALIGN(align); \ VMLINUX_SYMBOL(__per_cpu_start) = .; \ - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { \ - *(.data.percpu.page_aligned) \ - *(.data.percpu) \ - *(.data.percpu.shared_aligned) \ + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { \ + *(.kernel.data.percpu.page_aligned) \ + *(.kernel.data.percpu) \ + *(.kernel.data.percpu.shared_aligned) \ } \ VMLINUX_SYMBOL(__per_cpu_end) = .; diff --git a/include/linux/cache.h b/include/linux/cache.h index 97e2488..12fb84d 100644 --- a/include/linux/cache.h +++ b/include/linux/cache.h @@ -31,7 +31,7 @@ #ifndef __cacheline_aligned #define __cacheline_aligned \ __attribute__((__aligned__(SMP_CACHE_BYTES), \ - __section__(".data.cacheline_aligned"))) + __section__(".kernel.data.cacheline_aligned"))) #endif /* __cacheline_aligned */ #ifndef __cacheline_aligned_in_smp diff --git a/include/linux/init.h b/include/linux/init.h index 68cb026..10d256a 100644 --- a/include/linux/init.h +++ b/include/linux/init.h @@ -62,9 +62,9 @@ /* backward compatibility note * A few places hardcode the old section names: - * .text.init.refok - * .data.init.refok - * .exit.text.refok + * .kernel.text.init.refok + * .kernel.data.init.refok + * .kernel.text.exit.refok * They should be converted to use the defines from this file */ @@ -301,7 +301,7 @@ void __init parse_early_param(void); #endif /* Data marked not to be saved by software suspend */ -#define __nosavedata __section(.data.nosave) +#define __nosavedata __section(.kernel.data.nosave) /* This means "can be init if no module support, otherwise module load may call it." */ diff --git a/include/linux/linkage.h b/include/linux/linkage.h index 9fd1f85..914e63c 100644 --- a/include/linux/linkage.h +++ b/include/linux/linkage.h @@ -18,8 +18,8 @@ # define asmregparm #endif -#define __page_aligned_data __section(.data.page_aligned) __aligned(PAGE_SIZE) -#define __page_aligned_bss __section(.bss.page_aligned) __aligned(PAGE_SIZE) +#define __page_aligned_data __section(.kernel.data.page_aligned) __aligned(PAGE_SIZE) +#define __page_aligned_bss __section(.bss.kernel.page_aligned) __aligned(PAGE_SIZE) /* * This is used by architectures to keep arguments on the stack diff --git a/include/linux/percpu.h b/include/linux/percpu.h index 9f2a375..55058fe 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -10,13 +10,13 @@ #ifdef CONFIG_SMP #define DEFINE_PER_CPU(type, name) \ - __attribute__((__section__(".data.percpu"))) \ + __attribute__((__section__(".kernel.data.percpu"))) \ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #ifdef MODULE -#define SHARED_ALIGNED_SECTION ".data.percpu" +#define SHARED_ALIGNED_SECTION ".kernel.data.percpu" #else -#define SHARED_ALIGNED_SECTION ".data.percpu.shared_aligned" +#define SHARED_ALIGNED_SECTION ".kernel.data.percpu.shared_aligned" #endif #define DEFINE_PER_CPU_SHARED_ALIGNED(type, name) \ @@ -24,14 +24,14 @@ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name \ ____cacheline_aligned_in_smp -#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ - __attribute__((__section__(".data.percpu.page_aligned"))) \ +#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ + __attribute__((__section__(".kernel.data.percpu.page_aligned")))\ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #else #define DEFINE_PER_CPU(type, name) \ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name -#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name) \ +#define DEFINE_PER_CPU_SHARED_ALIGNED(type, name) \ DEFINE_PER_CPU(type, name) #define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ diff --git a/include/linux/spinlock.h b/include/linux/spinlock.h index e0c0fcc..b041f4d 100644 --- a/include/linux/spinlock.h +++ b/include/linux/spinlock.h @@ -60,7 +60,7 @@ /* * Must define these before including other files, inline functions need them */ -#define LOCK_SECTION_NAME ".text.lock."KBUILD_BASENAME +#define LOCK_SECTION_NAME ".kernel.text.lock."KBUILD_BASENAME #define LOCK_SECTION_START(extra) \ ".subsection 1\n\t" \ diff --git a/kernel/module.c b/kernel/module.c index 1f4cc00..5275d86 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -484,7 +484,7 @@ static unsigned int find_pcpusec(Elf_Ehdr *hdr, Elf_Shdr *sechdrs, const char *secstrings) { - return find_sec(hdr, sechdrs, secstrings, ".data.percpu"); + return find_sec(hdr, sechdrs, secstrings, ".kernel.data.percpu"); } static void percpu_modcopy(void *pcpudest, const void *from, unsigned long size) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 8892161..afe9408 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -794,9 +794,9 @@ static const char *data_sections[] = { DATA_SECTIONS, NULL }; /* sections that may refer to an init/exit section with no warning */ static const char *initref_sections[] = { - ".text.init.refok*", - ".exit.text.refok*", - ".data.init.refok*", + ".kernel.text.init.refok*", + ".kernel.text.exit.refok*", + ".kernel.data.init.refok*", NULL }; @@ -915,7 +915,7 @@ static int section_mismatch(const char *fromsec, const char *tosec) * Pattern 0: * Do not warn if funtion/data are marked with __init_refok/__initdata_refok. * The pattern is identified by: - * fromsec = .text.init.refok* | .data.init.refok* + * fromsec = .kernel.text.init.refok* | .kernel.data.init.refok* * * Pattern 1: * If a module parameter is declared __initdata and permissions=0 @@ -939,8 +939,8 @@ static int section_mismatch(const char *fromsec, const char *tosec) * *probe_one, *_console, *_timer * * Pattern 3: - * Whitelist all refereces from .text.head to .init.data - * Whitelist all refereces from .text.head to .init.text + * Whitelist all refereces from .kernel.text.head to .init.data + * Whitelist all refereces from .kernel.text.head to .init.text * * Pattern 4: * Some symbols belong to init section but still it is ok to reference diff --git a/scripts/recordmcount.pl b/scripts/recordmcount.pl index 6b9fe3e..41f733b 100755 --- a/scripts/recordmcount.pl +++ b/scripts/recordmcount.pl @@ -26,7 +26,7 @@ # which will also be the location of that section after final link. # e.g. # -# .section ".text.sched" +# .section ".sched.text" # .globl my_func # my_func: # [...] @@ -39,7 +39,7 @@ # [...] # # Both relocation offsets for the mcounts in the above example will be -# offset from .text.sched. If we make another file called tmp.s with: +# offset from .sched.text. If we make another file called tmp.s with: # # .section __mcount_loc # .quad my_func + 0x5 @@ -51,7 +51,7 @@ # But this gets hard if my_func is not globl (a static function). # In such a case we have: # -# .section ".text.sched" +# .section ".sched.text" # my_func: # [...] # call mcount (offset: 0x5) -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections 2008-12-06 0:03 ` [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections Jeff Arnold @ 2008-12-06 0:03 ` Jeff Arnold 2008-12-06 0:03 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Jeff Arnold 2009-02-04 8:38 ` [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections Rusty Russell 2008-12-06 8:46 ` [PATCH 1/7] Make section names compatible " Andi Kleen 2009-02-04 8:15 ` Rusty Russell 2 siblings, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan From: Waseem Daher <wdaher@mit.edu> This patch makes it possible to link and boot an x86 kernel with -ffunction-sections and -fdata-sections enabled. Modpost currently warns whenever it sees a section with a name matching [.][0-9]+$ because they are often caused by section flag mismatch errors. When compiling with -ffunction-sections -fdata-sections, gcc places various classes of local symbols in sections with names such as .rodata.__func__.12345, causing these warnings to be printed spuriously. The simplest fix is to disable the warning when CONFIG_FUNCTION_DATA_SECTIONS is enabled. Signed-off-by: Waseem Daher <wdaher@mit.edu> [tabbott@mit.edu: modpost support] Signed-off-by: Tim Abbott <tabbott@mit.edu> --- Makefile | 4 ++++ arch/x86/kernel/vmlinux_32.lds.S | 1 + arch/x86/kernel/vmlinux_64.lds.S | 1 + include/asm-generic/vmlinux.lds.h | 2 ++ lib/Kconfig.debug | 9 +++++++++ scripts/Makefile.modpost | 1 + scripts/mod/modpost.c | 10 ++++++++-- 7 files changed, 26 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 9a49960..d3d6fa4 100644 --- a/Makefile +++ b/Makefile @@ -540,6 +540,10 @@ ifdef CONFIG_FUNCTION_TRACER KBUILD_CFLAGS += -pg endif +ifdef CONFIG_FUNCTION_DATA_SECTIONS +KBUILD_CFLAGS += -ffunction-sections -fdata-sections +endif + # We trigger additional mismatches with less inlining ifdef CONFIG_DEBUG_SECTION_MISMATCH KBUILD_CFLAGS += $(call cc-option, -fno-inline-functions-called-once) diff --git a/arch/x86/kernel/vmlinux_32.lds.S b/arch/x86/kernel/vmlinux_32.lds.S index 454ab4c..c0c4323 100644 --- a/arch/x86/kernel/vmlinux_32.lds.S +++ b/arch/x86/kernel/vmlinux_32.lds.S @@ -193,6 +193,7 @@ SECTIONS __bss_start = .; /* BSS */ *(.bss.kernel.page_aligned) *(.bss) + *(.bss.*) . = ALIGN(4); __bss_stop = .; _end = . ; diff --git a/arch/x86/kernel/vmlinux_64.lds.S b/arch/x86/kernel/vmlinux_64.lds.S index 6fd4547..ccb74b0 100644 --- a/arch/x86/kernel/vmlinux_64.lds.S +++ b/arch/x86/kernel/vmlinux_64.lds.S @@ -222,6 +222,7 @@ SECTIONS .bss : AT(ADDR(.bss) - LOAD_OFFSET) { *(.bss.kernel.page_aligned) *(.bss) + *(.bss.*) } __bss_stop = .; diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index e8d1bdd..4b5aea8 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -48,6 +48,7 @@ /* .data section */ #define DATA_DATA \ *(.data) \ + *(.data.*) \ *(.kernel.data.init.refok) \ *(.ref.data) \ DEV_KEEP(init.data) \ @@ -235,6 +236,7 @@ ALIGN_FUNCTION(); \ *(.text.hot) \ *(.text) \ + *(.text.*) \ *(.ref.text) \ *(.kernel.text.init.refok) \ *(.kernel.text.exit.refok) \ diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index b0f239e..e121ab7 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -557,6 +557,15 @@ config FRAME_POINTER some architectures or if you use external debuggers. If you don't debug the kernel, you can say N. +config FUNCTION_DATA_SECTIONS + bool "Compile with -ffunction-sections -fdata-sections" + depends on !FTRACE + help + If you say Y here the compiler will give each function + and data structure its own ELF section. + + If unsure, say N. + config BOOT_PRINTK_DELAY bool "Delay each boot printk message by N milliseconds" depends on DEBUG_KERNEL && PRINTK && GENERIC_CALIBRATE_DELAY diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index f4053dc..a712bb2 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -87,6 +87,7 @@ modpost = scripts/mod/modpost \ $(if $(CONFIG_DEBUG_SECTION_MISMATCH),,-S) \ $(if $(CONFIG_MARKERS),-K $(kernelmarkersfile)) \ $(if $(CONFIG_MARKERS),-M $(markersfile)) \ + $(if $(filter -ffunction-sections -fdata-sections,$(KBUILD_CFLAGS) $(CFLAGS_KERNEL)),-F) \ $(if $(KBUILD_EXTMOD)$(KBUILD_MODPOST_WARN),-w) \ $(if $(cross_build),-c) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index afe9408..68d9e95 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -32,6 +32,8 @@ static int warn_unresolved = 0; /* How a symbol is exported */ static int sec_mismatch_count = 0; static int sec_mismatch_verbose = 1; +/* Are we using CONFIG_FUNCTION_DATA_SECTIONS? */ +static int function_data_sections = 0; enum export { export_plain, export_unused, export_gpl, @@ -736,7 +738,8 @@ static int check_section(const char *modname, const char *sec) /* consume all digits */ while (*e && e != sec && isdigit(*e)) e--; - if (*e == '.' && !strstr(sec, ".linkonce")) { + if (*e == '.' && !strstr(sec, ".linkonce") && + !function_data_sections) { warn("%s (%s): unexpected section name.\n" "The (.[number]+) following section name are " "ld generated and not expected.\n" @@ -2063,7 +2066,7 @@ int main(int argc, char **argv) struct ext_sym_list *extsym_iter; struct ext_sym_list *extsym_start = NULL; - while ((opt = getopt(argc, argv, "i:I:e:cmsSo:awM:K:")) != -1) { + while ((opt = getopt(argc, argv, "i:I:e:cFmsSo:awM:K:")) != -1) { switch (opt) { case 'i': kernel_read = optarg; @@ -2083,6 +2086,9 @@ int main(int argc, char **argv) extsym_iter->file = optarg; extsym_start = extsym_iter; break; + case 'F': + function_data_sections = 1; + break; case 'm': modversions = 1; break; -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol 2008-12-06 0:03 ` [PATCH 2/7] x86: Add an option to compile " Jeff Arnold @ 2008-12-06 0:03 ` Jeff Arnold 2008-12-06 0:03 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Jeff Arnold 2009-02-04 9:26 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Rusty Russell 2009-02-04 8:38 ` [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections Rusty Russell 1 sibling, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan From: Tim Abbott <tabbott@mit.edu> Ksplice needs access to the kernel_symbol structure in order to support modifications to the exported symbol table. Signed-off-by: Tim Abbott <tabbott@mit.edu> --- kernel/module.c | 80 ++++++++++++++++++++++++++---------------------------- 1 files changed, 39 insertions(+), 41 deletions(-) diff --git a/kernel/module.c b/kernel/module.c index 5275d86..0d361f0 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -282,7 +282,7 @@ struct find_symbol_arg { /* Output */ struct module *owner; const unsigned long *crc; - unsigned long value; + const struct kernel_symbol *sym; }; static bool find_symbol_in_section(const struct symsearch *syms, @@ -323,17 +323,17 @@ static bool find_symbol_in_section(const struct symsearch *syms, fsa->owner = owner; fsa->crc = symversion(syms->crcs, symnum); - fsa->value = syms->start[symnum].value; + fsa->sym = &syms->start[symnum]; return true; } -/* Find a symbol, return value, (optional) crc and (optional) module - * which owns it */ -static unsigned long find_symbol(const char *name, - struct module **owner, - const unsigned long **crc, - bool gplok, - bool warn) +/* Find a symbol and return it, along with, (optional) crc and + * (optional) module which owns it */ +static const struct kernel_symbol *find_symbol(const char *name, + struct module **owner, + const unsigned long **crc, + bool gplok, + bool warn) { struct find_symbol_arg fsa; @@ -346,11 +346,11 @@ static unsigned long find_symbol(const char *name, *owner = fsa.owner; if (crc) *crc = fsa.crc; - return fsa.value; + return fsa.sym; } DEBUGP("Failed to find symbol %s\n", name); - return -ENOENT; + return NULL; } /* Search for module by name: must hold module_mutex. */ @@ -848,7 +848,7 @@ void __symbol_put(const char *symbol) struct module *owner; preempt_disable(); - if (IS_ERR_VALUE(find_symbol(symbol, &owner, NULL, true, false))) + if (find_symbol(symbol, &owner, NULL, true, false) == NULL) BUG(); module_put(owner); preempt_enable(); @@ -1011,7 +1011,7 @@ static inline int check_modstruct_version(Elf_Shdr *sechdrs, { const unsigned long *crc; - if (IS_ERR_VALUE(find_symbol("struct_module", NULL, &crc, true, false))) + if (find_symbol("struct_module", NULL, &crc, true, false) == NULL) BUG(); return check_version(sechdrs, versindex, "struct_module", mod, crc); } @@ -1052,25 +1052,23 @@ static inline int same_magic(const char *amagic, const char *bmagic, /* Resolve a symbol for this module. I.e. if we find one, record usage. Must be holding module_mutex. */ -static unsigned long resolve_symbol(Elf_Shdr *sechdrs, - unsigned int versindex, - const char *name, - struct module *mod) +static const struct kernel_symbol *resolve_symbol(Elf_Shdr *sechdrs, + unsigned int versindex, + const char *name, + struct module *mod) { struct module *owner; - unsigned long ret; + const struct kernel_symbol *sym; const unsigned long *crc; - ret = find_symbol(name, &owner, &crc, + sym = find_symbol(name, &owner, &crc, !(mod->taints & (1 << TAINT_PROPRIETARY_MODULE)), true); - if (!IS_ERR_VALUE(ret)) { - /* use_module can fail due to OOM, - or module initialization or unloading */ - if (!check_version(sechdrs, versindex, name, mod, crc) || - !use_module(mod, owner)) - ret = -EINVAL; - } - return ret; + /* use_module can fail due to OOM, + or module initialization or unloading */ + if (!check_version(sechdrs, versindex, name, mod, crc) || + !use_module(mod, owner)) + sym = NULL; + return sym; } /* @@ -1466,17 +1464,15 @@ static void free_module(struct module *mod) void *__symbol_get(const char *symbol) { struct module *owner; - unsigned long value; + const struct kernel_symbol *sym; preempt_disable(); - value = find_symbol(symbol, &owner, NULL, true, true); - if (IS_ERR_VALUE(value)) - value = 0; - else if (strong_try_module_get(owner)) - value = 0; + sym = find_symbol(symbol, &owner, NULL, true, true); + if (sym != NULL && strong_try_module_get(owner)) + sym = NULL; preempt_enable(); - return (void *)value; + return sym == NULL ? (void *)0 : (void *)sym->value; } EXPORT_SYMBOL_GPL(__symbol_get); @@ -1504,8 +1500,8 @@ static int verify_export_symbols(struct module *mod) for (i = 0; i < ARRAY_SIZE(arr); i++) { for (s = arr[i].sym; s < arr[i].sym + arr[i].num; s++) { - if (!IS_ERR_VALUE(find_symbol(s->name, &owner, - NULL, true, false))) { + if (find_symbol(s->name, &owner, NULL, true, false) + != NULL) { printk(KERN_ERR "%s: exports duplicate symbol %s" " (owned by %s)\n", @@ -1529,6 +1525,7 @@ static int simplify_symbols(Elf_Shdr *sechdrs, unsigned long secbase; unsigned int i, n = sechdrs[symindex].sh_size / sizeof(Elf_Sym); int ret = 0; + const struct kernel_symbol *ksym; for (i = 1; i < n; i++) { switch (sym[i].st_shndx) { @@ -1548,13 +1545,14 @@ static int simplify_symbols(Elf_Shdr *sechdrs, break; case SHN_UNDEF: - sym[i].st_value - = resolve_symbol(sechdrs, versindex, - strtab + sym[i].st_name, mod); - + ksym = resolve_symbol(sechdrs, versindex, + strtab + sym[i].st_name, mod); /* Ok if resolved. */ - if (!IS_ERR_VALUE(sym[i].st_value)) + if (ksym != NULL) { + sym[i].st_value = ksym->value; break; + } + /* Ok if weak. */ if (ELF_ST_BIND(sym[i].st_info) == STB_WEAK) break; -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) 2008-12-06 0:03 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Jeff Arnold @ 2008-12-06 0:03 ` Jeff Arnold 2008-12-06 0:03 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Jeff Arnold 2009-02-04 11:21 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Rusty Russell 2009-02-04 9:26 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Rusty Russell 1 sibling, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan From: Tim Abbott <tabbott@mit.edu> Ksplice's run-pre matching process needs to be able to determine the module that contains a particular text or data address. Signed-off-by: Tim Abbott <tabbott@mit.edu> --- include/linux/module.h | 2 ++ kernel/module.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 0 deletions(-) diff --git a/include/linux/module.h b/include/linux/module.h index 3bfed01..2268147 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -366,6 +366,8 @@ static inline int module_is_live(struct module *mod) /* Is this address in a module? (second is with no locks, for oops) */ struct module *module_text_address(unsigned long addr); struct module *__module_text_address(unsigned long addr); +struct module *module_data_address(unsigned long addr); +struct module *__module_data_address(unsigned long addr); int is_module_address(unsigned long addr); /* Returns 0 and fills in value, defined and namebuf, or -ERANGE if diff --git a/kernel/module.c b/kernel/module.c index 0d361f0..c3cf234 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -2724,6 +2724,7 @@ struct module *__module_text_address(unsigned long addr) return mod; return NULL; } +EXPORT_SYMBOL_GPL(__module_text_address); struct module *module_text_address(unsigned long addr) { @@ -2735,6 +2736,37 @@ struct module *module_text_address(unsigned long addr) return mod; } +EXPORT_SYMBOL_GPL(module_text_address); + +struct module *__module_data_address(unsigned long addr) +{ + struct module *mod; + + if (addr < module_addr_min || addr > module_addr_max) + return NULL; + + list_for_each_entry(mod, &modules, list) { + if (within(addr, mod->module_core + mod->core_text_size, + mod->core_size - mod->core_text_size) || + within(addr, mod->module_init + mod->init_text_size, + mod->init_size - mod->init_text_size)) + return mod; + } + return NULL; +} +EXPORT_SYMBOL_GPL(__module_data_address); + +struct module *module_data_address(unsigned long addr) +{ + struct module *mod; + + preempt_disable(); + mod = __module_data_address(addr); + preempt_enable(); + + return mod; +} +EXPORT_SYMBOL_GPL(module_data_address); /* Don't grab lock, we're oopsing. */ void print_modules(void) -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols 2008-12-06 0:03 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Jeff Arnold @ 2008-12-06 0:03 ` Jeff Arnold 2008-12-06 0:03 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Jeff Arnold 2009-02-04 11:30 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Rusty Russell 2009-02-04 11:21 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Rusty Russell 1 sibling, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan From: Anders Kaseorg <andersk@mit.edu> kallsyms_lookup_name only returns the first match that it finds. Ksplice needs information about all symbols with a given name in order to correctly resolve local symbols. kallsyms_on_each_symbol provides a generic mechanism for iterating over the kallsyms table. Signed-off-by: Anders Kaseorg <andersk@mit.edu> --- include/linux/kallsyms.h | 13 +++++++++++++ include/linux/module.h | 12 ++++++++++++ kernel/kallsyms.c | 19 +++++++++++++++++++ kernel/module.c | 19 +++++++++++++++++++ 4 files changed, 63 insertions(+), 0 deletions(-) diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index f3fe343..37746b2 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -17,6 +17,11 @@ /* Lookup the address for a symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name(const char *name); +/* Call a function on each kallsyms symbol in the core kernel */ +int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, + unsigned long), + void *data); + extern int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize, unsigned long *offset); @@ -43,6 +48,14 @@ static inline unsigned long kallsyms_lookup_name(const char *name) return 0; } +static inline int kallsyms_on_each_symbol(int (*fn)(void *, const char *, + struct module *, + unsigned long), + void *data) +{ + return 0; +} + static inline int kallsyms_lookup_size_offset(unsigned long addr, unsigned long *symbolsize, unsigned long *offset) diff --git a/include/linux/module.h b/include/linux/module.h index 2268147..29972a2 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -378,6 +378,10 @@ int module_get_kallsym(unsigned int symnum, unsigned long *value, char *type, /* Look for this name: can be of form module:name. */ unsigned long module_kallsyms_lookup_name(const char *name); +int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, + struct module *, unsigned long), + void *data); + extern void __module_put_and_exit(struct module *mod, long code) __attribute__((noreturn)); #define module_put_and_exit(code) __module_put_and_exit(THIS_MODULE, code); @@ -546,6 +550,14 @@ static inline unsigned long module_kallsyms_lookup_name(const char *name) return 0; } +static inline int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, + struct module *, + unsigned long), + void *data) +{ + return 0; +} + static inline int register_module_notifier(struct notifier_block * nb) { /* no events will happen anyway, so this can always succeed */ diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 7b8b0f2..374faf9 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -161,6 +161,25 @@ unsigned long kallsyms_lookup_name(const char *name) return module_kallsyms_lookup_name(name); } +int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, + unsigned long), + void *data) +{ + char namebuf[KSYM_NAME_LEN]; + unsigned long i; + unsigned int off; + int ret; + + for (i = 0, off = 0; i < kallsyms_num_syms; i++) { + off = kallsyms_expand_symbol(off, namebuf); + ret = fn(data, namebuf, NULL, kallsyms_addresses[i]); + if (ret != 0) + return ret; + } + return module_kallsyms_on_each_symbol(fn, data); +} +EXPORT_SYMBOL_GPL(kallsyms_on_each_symbol); + static unsigned long get_symbol_pos(unsigned long addr, unsigned long *symbolsize, unsigned long *offset) diff --git a/kernel/module.c b/kernel/module.c index c3cf234..8839d40 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -2554,6 +2554,25 @@ unsigned long module_kallsyms_lookup_name(const char *name) preempt_enable(); return ret; } + +int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, + struct module *, unsigned long), + void *data) +{ + struct module *mod; + unsigned int i; + int ret; + + list_for_each_entry(mod, &modules, list) { + for (i = 0; i < mod->num_symtab; i++) { + ret = fn(data, mod->strtab + mod->symtab[i].st_name, + mod, mod->symtab[i].st_value); + if (ret != 0) + return ret; + } + } + return 0; +} #endif /* CONFIG_KALLSYMS */ static char *module_flags(struct module *mod, char *buf) -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH 6/7] Ksplice: Export symbols needed for Ksplice 2008-12-06 0:03 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Jeff Arnold @ 2008-12-06 0:03 ` Jeff Arnold 2008-12-06 0:04 ` [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 Jeff Arnold 2009-02-04 11:35 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Rusty Russell 2009-02-04 11:30 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Rusty Russell 1 sibling, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:03 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan From: Tim Abbott <tabbott@mit.edu> Ksplice uses several functions from module.c in order to resolve symbols and implement dependency handling. Calling these functions requires holding module_mutex, so it is exported. Ksplice uses task_curr in order to confirm that no other tasks are executing during Ksplice's kernel stack check. Ksplice uses find_bug and search_exception_tables as part of its conservative check that the user provided the correct source code for the running kernel. Signed-off-by: Tim Abbott <tabbott@mit.edu> --- include/linux/module.h | 28 ++++++++++++++++++++++++++++ kernel/extable.c | 3 +++ kernel/module.c | 41 +++++++++++++++++------------------------ kernel/sched.c | 1 + lib/bug.c | 1 + 5 files changed, 50 insertions(+), 24 deletions(-) diff --git a/include/linux/module.h b/include/linux/module.h index 29972a2..cc613f6 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -355,6 +355,8 @@ struct module #define MODULE_ARCH_INIT {} #endif +extern struct mutex module_mutex; + /* FIXME: It'd be nice to isolate modules during init, too, so they aren't used before they (may) fail. But presently too much code (IDE & SCSI) require entry into the module during init.*/ @@ -370,6 +372,31 @@ struct module *module_data_address(unsigned long addr); struct module *__module_data_address(unsigned long addr); int is_module_address(unsigned long addr); +/* Search for module by name: must hold module_mutex. */ +struct module *find_module(const char *name); + +struct symsearch { + const struct kernel_symbol *start, *stop; + const unsigned long *crcs; + enum { + NOT_GPL_ONLY, + GPL_ONLY, + WILL_BE_GPL_ONLY, + } licence; + bool unused; +}; + +/* Search for an exported symbol by name. */ +const struct kernel_symbol *find_symbol(const char *name, + struct module **owner, + const unsigned long **crc, + bool gplok, + bool warn); + +/* Walk the exported symbol table */ +bool each_symbol(bool (*fn)(const struct symsearch *arr, struct module *owner, + unsigned int symnum, void *data), void *data); + /* Returns 0 and fills in value, defined and namebuf, or -ERANGE if symnum out of range. */ int module_get_kallsym(unsigned int symnum, unsigned long *value, char *type, @@ -435,6 +462,7 @@ static inline void __module_get(struct module *module) #define symbol_put_addr(p) do { } while(0) #endif /* CONFIG_MODULE_UNLOAD */ +int use_module(struct module *a, struct module *b); /* This is a #define so the string doesn't get put in every .o file */ #define module_name(mod) \ diff --git a/kernel/extable.c b/kernel/extable.c index a26cb2e..f7740b1 100644 --- a/kernel/extable.c +++ b/kernel/extable.c @@ -39,6 +39,7 @@ const struct exception_table_entry *search_exception_tables(unsigned long addr) e = search_module_extables(addr); return e; } +EXPORT_SYMBOL_GPL(search_exception_tables); int core_kernel_text(unsigned long addr) { @@ -59,6 +60,7 @@ int __kernel_text_address(unsigned long addr) return 1; return __module_text_address(addr) != NULL; } +EXPORT_SYMBOL_GPL(__kernel_text_address); int kernel_text_address(unsigned long addr) { @@ -66,3 +68,4 @@ int kernel_text_address(unsigned long addr) return 1; return module_text_address(addr) != NULL; } +EXPORT_SYMBOL_GPL(kernel_text_address); diff --git a/kernel/module.c b/kernel/module.c index 8839d40..b109aae 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -67,7 +67,8 @@ /* List of modules, protected by module_mutex or preempt_disable * (delete uses stop_machine/add uses RCU list operations). */ -static DEFINE_MUTEX(module_mutex); +DEFINE_MUTEX(module_mutex); +EXPORT_SYMBOL_GPL(module_mutex); static LIST_HEAD(modules); /* Waiting for a module to finish initializing? */ @@ -185,17 +186,6 @@ extern const unsigned long __start___kcrctab_unused_gpl[]; #define symversion(base, idx) ((base != NULL) ? ((base) + (idx)) : NULL) #endif -struct symsearch { - const struct kernel_symbol *start, *stop; - const unsigned long *crcs; - enum { - NOT_GPL_ONLY, - GPL_ONLY, - WILL_BE_GPL_ONLY, - } licence; - bool unused; -}; - static bool each_symbol_in_section(const struct symsearch *arr, unsigned int arrsize, struct module *owner, @@ -216,10 +206,8 @@ static bool each_symbol_in_section(const struct symsearch *arr, } /* Returns true as soon as fn returns true, otherwise false. */ -static bool each_symbol(bool (*fn)(const struct symsearch *arr, - struct module *owner, - unsigned int symnum, void *data), - void *data) +bool each_symbol(bool (*fn)(const struct symsearch *arr, struct module *owner, + unsigned int symnum, void *data), void *data) { struct module *mod; const struct symsearch arr[] = { @@ -272,6 +260,7 @@ static bool each_symbol(bool (*fn)(const struct symsearch *arr, } return false; } +EXPORT_SYMBOL_GPL(each_symbol); struct find_symbol_arg { /* Input */ @@ -329,11 +318,11 @@ static bool find_symbol_in_section(const struct symsearch *syms, /* Find a symbol and return it, along with, (optional) crc and * (optional) module which owns it */ -static const struct kernel_symbol *find_symbol(const char *name, - struct module **owner, - const unsigned long **crc, - bool gplok, - bool warn) +const struct kernel_symbol *find_symbol(const char *name, + struct module **owner, + const unsigned long **crc, + bool gplok, + bool warn) { struct find_symbol_arg fsa; @@ -352,9 +341,10 @@ static const struct kernel_symbol *find_symbol(const char *name, DEBUGP("Failed to find symbol %s\n", name); return NULL; } +EXPORT_SYMBOL_GPL(find_symbol); /* Search for module by name: must hold module_mutex. */ -static struct module *find_module(const char *name) +struct module *find_module(const char *name) { struct module *mod; @@ -364,6 +354,7 @@ static struct module *find_module(const char *name) } return NULL; } +EXPORT_SYMBOL_GPL(find_module); #ifdef CONFIG_SMP /* Number of blocks used and allocated. */ @@ -607,7 +598,7 @@ static int already_uses(struct module *a, struct module *b) } /* Module a uses b */ -static int use_module(struct module *a, struct module *b) +int use_module(struct module *a, struct module *b) { struct module_use *use; int no_warn, err; @@ -640,6 +631,7 @@ static int use_module(struct module *a, struct module *b) no_warn = sysfs_create_link(b->holders_dir, &a->mkobj.kobj, a->name); return 1; } +EXPORT_SYMBOL_GPL(use_module); /* Clear the unload stuff of the module. */ static void module_unload_free(struct module *mod) @@ -903,10 +895,11 @@ static inline void module_unload_free(struct module *mod) { } -static inline int use_module(struct module *a, struct module *b) +int use_module(struct module *a, struct module *b) { return strong_try_module_get(b) == 0; } +EXPORT_SYMBOL_GPL(use_module); static inline void module_unload_init(struct module *mod) { diff --git a/kernel/sched.c b/kernel/sched.c index b7480fb..970e034 100644 --- a/kernel/sched.c +++ b/kernel/sched.c @@ -1770,6 +1770,7 @@ inline int task_curr(const struct task_struct *p) { return cpu_curr(task_cpu(p)) == p; } +EXPORT_SYMBOL_GPL(task_curr); static inline void __set_task_cpu(struct task_struct *p, unsigned int cpu) { diff --git a/lib/bug.c b/lib/bug.c index bfeafd6..b5c0659 100644 --- a/lib/bug.c +++ b/lib/bug.c @@ -113,6 +113,7 @@ const struct bug_entry *find_bug(unsigned long bugaddr) return module_find_bug(bugaddr); } +EXPORT_SYMBOL_GPL(find_bug); enum bug_trap_type report_bug(unsigned long bugaddr, struct pt_regs *regs) { -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 2008-12-06 0:03 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Jeff Arnold @ 2008-12-06 0:04 ` Jeff Arnold 2008-12-17 5:41 ` Theodore Tso 2009-02-07 2:36 ` Rusty Russell 2009-02-04 11:35 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Rusty Russell 1 sibling, 2 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-06 0:04 UTC (permalink / raw) To: Andrew Morton Cc: linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Jeff Arnold Ksplice makes it possible to apply patches to the kernel without rebooting. Signed-off-by: Jeff Arnold <jbarnold@mit.edu> Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Tim Abbott <tabbott@mit.edu> Tested-by: Waseem Daher <wdaher@mit.edu> --- Documentation/ksplice.txt | 282 +++++ MAINTAINERS | 10 + arch/Kconfig | 14 + arch/x86/Kconfig | 1 + arch/x86/kernel/ksplice-arch.c | 125 ++ include/linux/kernel.h | 1 + include/linux/ksplice.h | 212 ++++ kernel/Makefile | 3 + kernel/ksplice.c | 2715 ++++++++++++++++++++++++++++++++++++++++ kernel/panic.c | 1 + 10 files changed, 3364 insertions(+), 0 deletions(-) create mode 100644 Documentation/ksplice.txt create mode 100644 arch/x86/kernel/ksplice-arch.c create mode 100644 include/linux/ksplice.h create mode 100644 kernel/ksplice.c diff --git a/Documentation/ksplice.txt b/Documentation/ksplice.txt new file mode 100644 index 0000000..971f03a --- /dev/null +++ b/Documentation/ksplice.txt @@ -0,0 +1,282 @@ +Ksplice +------- + +CONTENTS: + +1. Concepts: updates, packs, helper modules, primary modules +2. What changes can Ksplice handle? +3. Dependency model +4. Locking model +5. altinstructions, smplocks, and parainstructions +6. sysfs interface +7. debugfs interface +8. Hooks for running custom code during the update process + +0. Design Description +--------------------- + +For a description of the Ksplice design, please see the Ksplice technical +overview document: <http://www.ksplice.com/doc/ksplice.pdf>. For usage +examples and the Ksplice man pages, please see <http://www.ksplice.com>. + +The document below assumes familiarity with the Ksplice design and describes +notable implementation details and the interface between the Ksplice kernel +component and the Ksplice user space component. + +1. Concepts: Updates, packs, helper modules, primary modules +------------------------------------------------------------ + +A Ksplice update (struct update) contains one or more Ksplice packs, one for +each target kernel module that should be changed by the update. Ksplice packs +are grouped together into a Ksplice update in order to allow multiple +compilation units to be changed atomically. + +The contents of a Ksplice pack are documented via kernel-doc in +include/linux/ksplice.h. To construct a new Ksplice update to be performed +atomically, one needs to: + 1. Populate the fields of one or more ksplice_pack structures. + 2. Call the Ksplice function init_ksplice_pack() on each pack to register + the packs with the Ksplice kernel component. When init_ksplice_pack() + is called on a pack, that pack will be associated with the other packs + that share the same Ksplice identifier (KID) field. + 3. After all of the packs intended for a particular Ksplice update have + been loaded, that update can be applied via the sysfs interface + (described in Section 7 below). + +In order to save memory, each Ksplice pack has a "helper" module and a "primary" +module associated with it. + +The pack's "helper" module contains materials needed only for preparing for the +update. Specifically, the helper module contains a copy of the pre-patch +version of each of the compilation units changed by the Ksplice pack. The +helper module can be unloaded after the update has been applied. + +The pack's "primary" module contains the new sections to be inserted by the +update; it needs to remain loaded for as long as the update is applied. + +Here's an example: + +Let's say that the Ksplice user space component wants to update the core kernel +and the isdn module. The user space component will select a KID for this update +(let's say 123abc) and generate four modules: + +ksplice_123abc_vmlinux (the "primary" module for the vmlinux pack) +ksplice_123abc_vmlinux_helper (the "helper" module for the vmlinux pack) +ksplice_123abc_isdn (the "primary" module for the vmlinux pack) +ksplice_123abc_isdn_helper (the "helper" module for the vmlinux pack) + +Once both of the vmlinux modules have been loaded, one of the modules calls +init_ksplice_pack on a pack corresponding to the desired vmlinux changes. + +Similarly, once both of the isdn modules have been loaded, one of the modules +calls init_ksplice_pack on a pack corresponding to the desired isdn changes. + +Once all modules are loaded (in this example, four modules), the update can be +applied atomically using the Ksplice sysfs interface. Once the update has been +applied, the helper modules can be unloaded safely to save memory. + +2. What changes can Ksplice handle? +----------------------------------- + +The Ksplice user space component takes a source code patch and uses it to +construct appropriate Ksplice packs for an update. Ksplice can handle source +code patches that add new functions, modify the text or arguments of existing +functions, delete functions, move functions between compilation units, change +functions from local to global (or vice versa), add exported symbols, rename +exported symbols, and delete exported symbols. Ksplice can handle patches that +modify either C code or assembly code. + +As described in the Ksplice technical overview document, a programmer needs to +write some new code in order for Ksplice to apply a patch that makes semantic +changes to kernel data structures. Some other limitations also apply: + +Ksplice does not support changes to __init functions that been unloaded from +kernel memory. Ksplice also does not support changes to functions in +.exit.text sections since Ksplice currently requires that all Ksplice updates +affecting a module be reversed before that module can be unloaded. + +The Ksplice user space implementation does not currently support changes to +weak symbols and changes to global read-only data structures (changes to +read-only data structures that are local to a compilation unit are fine). + +Exported symbols: + +Ksplice can handle arbitrary changes to exported symbols in the source code +patch. + +Ksplice deletes exported symbols by looking up the relevant struct kernel_symbol +in the kernel's exported symbol table and replacing the name field with a +pointer to a string that begins with DISABLED. + +Ksplice adds new exported symbols through the same mechanism; the relevant +primary module will have a ksymtab entry containing a symbol with a name +beginning with DISABLED, and Ksplice will replace that with the name of the +symbol to be exported when the update is atomically applied. + +Because the struct kernel_symbol for a newly exported symbol is contained in the +Ksplice primary module, if a module using one of the newly exported symbols is +loaded, that module will correctly depend on the Ksplice primary module that +exported the symbol. + +3. Dependency model +------------------- + +Because Ksplice resolves symbols used in the post code using Ksplice +relocations, Ksplice must enforce additional dependencies. Ksplice uses the +use_module function to directly add dependencies on all the modules that the +post code references. + +4. Locking model +---------------- + +From a locking perspective, Ksplice treats applying or removing a Ksplice update +as analogous to loading or unloading a new version of the kernel modules patched +by the update. Ksplice uses module_mutex to protect against a variety of race +conditions related to modules being loaded or unloaded while Ksplice is applying +or reversing an update; this approach also protects against race conditions +involving multiple Ksplice updates being loaded or unloaded simultaneously as +well. + +5. altinstructions, smplocks, and parainstructions +-------------------------------------------------- + +There are currently several mechanisms through which the Linux kernel will +modify executable code at runtime. + +These mechanisms sometimes overwrite the storage unit of a relocation, which +would cause problems if not handled properly by Ksplice. + +Ksplice solves this problem by writing "canary" bytes (e.g., 0x77777777) in the +storage unit of the relocation in user space. Ksplice then checks whether the +canary has been overwritten before using a Ksplice relocation to detect symbol +values or to write a value to the storage unit of a Ksplice relocation. + +6. sysfs interface +------------------ + +Ksplice exports four sysfs files per Ksplice update in order to communicate with +user space. For each update, these four files are located in a directory of the +form /sys/kernel/ksplice/$kid, with $kid replaced by the KID of the Ksplice +update. + +A. /sys/kernel/ksplice/$kid/stage (mode 0600) + +This file contains one of three strings: +preparing: Indicates that this update has not yet been applied +applied: Indicates that this update has been applied and has not been reversed +reversed: Indicates that this update has been reversed + +When the stage is "preparing", the superuser can write "applied" to the stage +file in order to instruct Ksplice to apply the update. When the stage is +"applied", the superuser can write "reversed" to the stage file in order to +instruct Ksplice to reverse the update. Afterwards, the superuser can +write "cleanup" to the stage file in order to instruct Ksplice to +clean up the debugging information and sysfs directory associated with +the reversed update. Once an update is reversed, it cannot be re-applied without first +cleaning up the update. + +B. /sys/kernel/ksplice/$kid/debug (mode 0600) + +The file contains a single number: 1 if debugging is enabled for this Ksplice +update and 0 otherwise. + +The superuser can write a new value to this file to enable or disable debugging. + +C. /sys/kernel/ksplice/$kid/partial (mode 0600) + +The file contains a single number: 1 if the update should be applied even if +some of the target modules are not loaded and 0 otherwise. + +D. /sys/kernel/ksplice/$kid/abort_cause (mode 0400) + +This file contains a value indicating either 1) that Ksplice successfully +completed the most recently requested stage transition or 2) why Ksplice aborted +the most recently requested stage transition. + +Each abort_code string is described below, along with the stage transitions that +might potentially trigger each possible abort code. The stage transitions are +abbreviated as follows: preparing->applied (P->A), applied->reversed (A->R). + +ok (P->A, A->R): The most recent stage transition succeeded. + +no_match (P->A): Ksplice aborted the update because Ksplice was unable to match +the helper module's object code against the running kernel's object code. + +failed_to_find (P->A): Ksplice aborted the update because Ksplice was unable to +resolve some of the symbols used in the update. + +missing_export (P->A): Ksplice aborted the update because the symbols exported +by the kernel did not match Ksplice's expectations. + +already_reversed (P->A): Ksplice aborted the update because once an update has +been reversed, it cannot be applied again (without first being cleaned up and +reinitialized). + +module_busy (A->R): Ksplice aborted the undo operation because the target +Ksplice update is in use by another kernel module; specifically, either the +target Ksplice update exports a symbol that is in use by another module or +another Ksplice update depends on this Ksplice update. + +out_of_memory (P->A, A->R): Ksplice aborted the operation because a call to +kmalloc or vmalloc failed. + +code_busy (P->A, A->R): Ksplice aborted the operation because Ksplice was +unable to find a moment when one or more of the to-be-patched functions was not +a thread's kernel stack. + +target_not_loaded (P->A): Ksplice aborted the update because one of the target +modules is not loaded and the partial option (/sys/kernel/ksplice/$kid/partial) +is not enabled. + +call_failed (P->A, A->R): One of the calls included as part of this update +returned nonzero exit status. + +unexpected_running_task (P->A, A->R): Ksplice aborted the operation because +Ksplice observed a running task during the kernel stack check, at a time when +Ksplice expected all tasks to be stopped by stop_machine. + +unexpected (P->A, A->R): Ksplice aborted the operation because it encountered +an unspecified internal error. This condition can only be caused by an invalid +input to Ksplice or a bug in Ksplice. + +E. /sys/kernel/ksplice/$kid/conflicts (mode 0400) + +This file is empty until Ksplice aborts an operation because of a code_busy +condition (see "abort_code" above). This conflicts file then contains +information about the process(es) that caused the stack check failure. + +Specifically, each line of this file consists of three space-separated values, +describing a single conflict: + +$program_name $program_pid $conflict_label + +$program_name is the name of the program with the conflict. +$program_pid is the pid of the program with the conflict. +$conflict_label is the Ksplice label of the function with the conflict. + +7. debugfs interface +-------------------- + +Ksplice exports a single file to debugfs for each Ksplice update. The file has +a name of the form ksplice_KID, where KID is the unique identifier of the +Ksplice update. It contains debugging information in a human-readable format. + +8. Hooks for running custom code during the update process +---------------------------------------------------------- + +Ksplice allows a programmer to write custom code to be called from within the +kernel during the update process. The kernel component allows custom code to +be executed at the following times: + +pre_apply: Called before the update has been applied and before the machine has +been stopped. Allowed to fail. +check_apply: Called before the update has been applied but after the machine +has been stopped. Allowed to fail. +apply: Called when the update is definitely going to be applied and when the +machine is stopped. Not allowed to fail. +post_apply: Called when the update has been applied and the machine is no +longer stopped. Not allowed to fail. +fail_apply: Called when the update failed to apply and the machine is no longer +stopped. Not allowed to fail. + +Ksplice also provides six analagous xxx_reverse hooks. diff --git a/MAINTAINERS b/MAINTAINERS index 618c1ef..40227d4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2585,6 +2585,16 @@ W: http://miguelojeda.es/auxdisplay.htm W: http://jair.lab.fi.uva.es/~migojed/auxdisplay.htm S: Maintained +KSPLICE: +P: Jeff Arnold +M: jbarnold@mit.edu +P: Anders Kaseorg +M: andersk@mit.edu +P: Tim Abbott +M: tabbott@mit.edu +W: http://www.ksplice.com +S: Maintained + LAPB module L: linux-x25@vger.kernel.org S: Orphan diff --git a/arch/Kconfig b/arch/Kconfig index 471e72d..de4b6c8 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -41,6 +41,17 @@ config KPROBES for kernel debugging, non-intrusive instrumentation and testing. If in doubt, say "N". +config KSPLICE + tristate "Ksplice rebootless kernel updates" + depends on KALLSYMS_ALL && MODULE_UNLOAD && SYSFS && \ + FUNCTION_DATA_SECTIONS + depends on HAVE_KSPLICE + help + Say Y here if you want to be able to apply certain kinds of + patches to your running kernel, without rebooting. + + If unsure, say N. + config HAVE_EFFICIENT_UNALIGNED_ACCESS bool help @@ -70,6 +81,9 @@ config HAVE_IOREMAP_PROT config HAVE_KPROBES bool +config HAVE_KSPLICE + def_bool n + config HAVE_KRETPROBES bool diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index ac22bb7..0e2c276 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -24,6 +24,7 @@ config X86 select HAVE_OPROFILE select HAVE_IOREMAP_PROT select HAVE_KPROBES + select HAVE_KSPLICE select ARCH_WANT_OPTIONAL_GPIOLIB select HAVE_KRETPROBES select HAVE_FTRACE_MCOUNT_RECORD diff --git a/arch/x86/kernel/ksplice-arch.c b/arch/x86/kernel/ksplice-arch.c new file mode 100644 index 0000000..b04ae17 --- /dev/null +++ b/arch/x86/kernel/ksplice-arch.c @@ -0,0 +1,125 @@ +/* Copyright (C) 2007-2008 Jeff Arnold <jbarnold@mit.edu> + * Copyright (C) 2008 Anders Kaseorg <andersk@mit.edu>, + * Tim Abbott <tabbott@mit.edu> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA + * 02110-1301, USA. + */ + +#define KSPLICE_IP(x) ((x)->thread.ip) +#define KSPLICE_SP(x) ((x)->thread.sp) + +static struct ksplice_symbol trampoline_symbol = { + .name = NULL, + .label = "<trampoline>", +}; + +static const struct ksplice_reloc_howto trampoline_howto = { + .type = KSPLICE_HOWTO_RELOC, + .pcrel = 1, + .size = 4, + .dst_mask = 0xffffffffL, + .rightshift = 0, + .signed_addend = 1, +}; + +static const struct ksplice_reloc trampoline_reloc = { + .symbol = &trampoline_symbol, + .insn_addend = -4, + .target_addend = 0, + .howto = &trampoline_howto, +}; + +static abort_t trampoline_target(struct ksplice_pack *pack, unsigned long addr, + unsigned long *new_addr) +{ + abort_t ret; + unsigned char byte; + + if (probe_kernel_read(&byte, (void *)addr, sizeof(byte)) == -EFAULT) + return NO_MATCH; + + if (byte != 0xe9) + return NO_MATCH; + + ret = read_reloc_value(pack, &trampoline_reloc, addr + 1, new_addr); + if (ret != OK) + return ret; + + *new_addr += addr + 1; + return OK; +} + +static abort_t prepare_trampoline(struct ksplice_pack *pack, + struct ksplice_patch *p) +{ + p->size = 5; + ((unsigned char *)p->contents)[0] = 0xe9; + return write_reloc_value(pack, &trampoline_reloc, + (unsigned long)p->contents + 1, + p->repladdr - (p->oldaddr + 1)); +} + +static abort_t handle_bug(struct ksplice_pack *pack, + const struct ksplice_reloc *r, unsigned long run_addr) +{ + const struct bug_entry *run_bug = find_bug(run_addr); + struct ksplice_section *bug_sect = symbol_section(pack, r->symbol); + if (run_bug == NULL) + return NO_MATCH; + if (bug_sect == NULL) + return UNEXPECTED; + return create_labelval(pack, bug_sect->symbol, (unsigned long)run_bug, + TEMP); +} + +static abort_t handle_extable(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long run_addr) +{ + const struct exception_table_entry *run_ent = + search_exception_tables(run_addr); + struct ksplice_section *ex_sect = symbol_section(pack, r->symbol); + if (run_ent == NULL) + return NO_MATCH; + if (ex_sect == NULL) + return UNEXPECTED; + return create_labelval(pack, ex_sect->symbol, (unsigned long)run_ent, + TEMP); +} + +static abort_t handle_paravirt(struct ksplice_pack *pack, + unsigned long pre_addr, unsigned long run_addr, + int *matched) +{ + unsigned char run[5], pre[5]; + *matched = 0; + + if (probe_kernel_read(&run, (void *)run_addr, sizeof(run)) == -EFAULT || + probe_kernel_read(&pre, (void *)pre_addr, sizeof(pre)) == -EFAULT) + return OK; + + if ((run[0] == 0xe8 && pre[0] == 0xe8) || + (run[0] == 0xe9 && pre[0] == 0xe9)) + if (run_addr + 1 + *(int32_t *)&run[1] == + pre_addr + 1 + *(int32_t *)&pre[1]) + *matched = 5; + return OK; +} + +static bool valid_stack_ptr(const struct thread_info *tinfo, const void *p) +{ + return p > (const void *)tinfo + && p <= (const void *)tinfo + THREAD_SIZE - sizeof(long); +} + diff --git a/include/linux/kernel.h b/include/linux/kernel.h index dc7e0d0..15643a5 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -290,6 +290,7 @@ extern enum system_states { #define TAINT_OVERRIDDEN_ACPI_TABLE 8 #define TAINT_WARN 9 #define TAINT_CRAP 10 +#define TAINT_KSPLICE 11 extern void dump_stack(void) __cold; diff --git a/include/linux/ksplice.h b/include/linux/ksplice.h new file mode 100644 index 0000000..5f59cc5 --- /dev/null +++ b/include/linux/ksplice.h @@ -0,0 +1,212 @@ +#include <linux/types.h> + +/** + * struct ksplice_symbol - Ksplice's analogue of an ELF symbol + * @name: The ELF name of the symbol + * @label: A unique Ksplice name for the symbol + * @vals: A linked list of possible values for the symbol, or NULL + * @value: The value of the symbol (valid when vals is NULL) + **/ +struct ksplice_symbol { + const char *name; + const char *label; +/* private: */ + struct list_head *vals; + unsigned long value; +}; + +/** + * struct ksplice_reloc - Ksplice's analogue of an ELF relocation + * @blank_addr: The address of the relocation's storage unit + * @symbol: The ksplice_symbol associated with this relocation + * @howto: The information regarding the relocation type + * @addend: The ELF addend of the relocation + **/ +struct ksplice_reloc { + unsigned long blank_addr; + struct ksplice_symbol *symbol; + const struct ksplice_reloc_howto *howto; + long insn_addend; + long target_addend; +}; + +enum ksplice_reloc_howto_type { + KSPLICE_HOWTO_RELOC, + KSPLICE_HOWTO_RELOC_PATCH, + KSPLICE_HOWTO_DATE, + KSPLICE_HOWTO_TIME, + KSPLICE_HOWTO_BUG, + KSPLICE_HOWTO_EXTABLE, +}; + +/** + * struct ksplice_reloc_howto - Ksplice's relocation type information + * @type: The type of the relocation + * @pcrel: Is the relocation PC relative? + * @size: The size, in bytes, of the item to be relocated + * @dst_mask: Bitmask for which parts of the instruction or data are + * replaced with the relocated value + * (based on dst_mask from GNU BFD's reloc_howto_struct) + * @rightshift: The value the final relocation is shifted right by; + * used to drop unwanted data from the relocation + * (based on rightshift from GNU BFD's reloc_howto_struct) + * @signed_addend: Should the addend be interpreted as a signed value? + **/ +struct ksplice_reloc_howto { + enum ksplice_reloc_howto_type type; + int pcrel; + int size; + long dst_mask; + unsigned int rightshift; + int signed_addend; +}; + +#if BITS_PER_LONG == 32 +#define KSPLICE_CANARY 0x77777777UL +#elif BITS_PER_LONG == 64 +#define KSPLICE_CANARY 0x7777777777777777UL +#endif /* BITS_PER_LONG */ + +/** + * struct ksplice_section - Ksplice's analogue of an ELF section + * @symbol: The ksplice_symbol associated with this section + * @size: The length, in bytes, of this section + * @address: The address of the section + * @flags: Flags indicating the type of the section, whether or + * not it has been matched, etc. + **/ +struct ksplice_section { + struct ksplice_symbol *symbol; + unsigned long address; + unsigned long size; + unsigned int flags; + const unsigned char **match_map; +}; +#define KSPLICE_SECTION_TEXT 0x00000001 +#define KSPLICE_SECTION_RODATA 0x00000002 +#define KSPLICE_SECTION_DATA 0x00000004 +#define KSPLICE_SECTION_STRING 0x00000008 +#define KSPLICE_SECTION_MATCHED 0x10000000 + +#define MAX_TRAMPOLINE_SIZE 5 + +enum ksplice_patch_type { + KSPLICE_PATCH_TEXT, + KSPLICE_PATCH_BUGLINE, + KSPLICE_PATCH_DATA, + KSPLICE_PATCH_EXPORT, +}; + +/** + * struct ksplice_patch - A replacement that Ksplice should perform + * @oldaddr: The address of the obsolete function or structure + * @repladdr: The address of the replacement function + * @type: The type of the ksplice patch + * @size: The size of the patch + * @contents: The bytes to be installed at oldaddr + * @vaddr The address of the page mapping used to write at oldaddr + * @saved: The bytes originally at oldaddr which were + * overwritten by the patch + **/ +struct ksplice_patch { + unsigned long oldaddr; + unsigned long repladdr; + enum ksplice_patch_type type; + unsigned int size; + void *contents; +/* private: */ + void *vaddr; + void *saved; +}; + +#ifdef __KERNEL__ +#include <linux/module.h> +#include <linux/stringify.h> + +#define _PASTE(x, y) x##y +#define PASTE(x, y) _PASTE(x, y) +#define KSPLICE_UNIQ(s) PASTE(s##_, KSPLICE_MID) +#define KSPLICE_KID_UNIQ(s) PASTE(s##_, KSPLICE_KID) + +/** + * struct ksplice_module_list_entry - A record of a Ksplice pack's target + * @target: A module that is patched + * @primary: A Ksplice module that patches target + **/ +struct ksplice_module_list_entry { + struct module *target; + struct module *primary; +/* private: */ + struct list_head list; +}; + +/* List of all ksplice modules and the module they patch */ +extern struct list_head ksplice_module_list; + +/** + * struct ksplice_pack - Data for one module modified by a Ksplice update + * @name: The name of the primary module for the pack + * @kid: The Ksplice unique identifier for the pack + * @target_name: The name of the module modified by the pack + * @primary: The primary module associated with the pack + * @primary_relocs: The relocations for the primary module + * @primary_relocs_end: The end pointer for primary_relocs + * @primary_sections: The sections in the primary module + * @primary_sections_end: The end pointer for primary_sections array + * @helper_relocs: The relocations for the helper module + * @helper_relocs_end: The end pointer for helper_relocs array + * @helper_sections: The sections in the helper module + * @helper_sections_end: The end pointer for helper_sections array + * @patches: The function replacements in the pack + * @patches_end: The end pointer for patches array + * @update: The atomic update the pack is part of + * @target: The module modified by the pack + * @safety_records: The ranges of addresses that must not be on a + * kernel stack for the patch to apply safely + **/ +struct ksplice_pack { + const char *name; + const char *kid; + const char *target_name; + struct module *primary; + struct ksplice_reloc *primary_relocs, *primary_relocs_end; + const struct ksplice_section *primary_sections, *primary_sections_end; + struct ksplice_symbol *primary_symbols, *primary_symbols_end; + struct ksplice_reloc *helper_relocs, *helper_relocs_end; + struct ksplice_section *helper_sections, *helper_sections_end; + struct ksplice_symbol *helper_symbols, *helper_symbols_end; + struct ksplice_patch *patches, *patches_end; + const typeof(int (*)(void)) *pre_apply, *pre_apply_end, *check_apply, + *check_apply_end; + const typeof(void (*)(void)) *apply, *apply_end, *post_apply, + *post_apply_end, *fail_apply, *fail_apply_end; + const typeof(int (*)(void)) *pre_reverse, *pre_reverse_end, + *check_reverse, *check_reverse_end; + const typeof(void (*)(void)) *reverse, *reverse_end, *post_reverse, + *post_reverse_end, *fail_reverse, *fail_reverse_end; +/* private: */ + struct ksplice_module_list_entry module_list_entry; + struct update *update; + struct module *target; + struct list_head temp_labelvals; + struct list_head safety_records; + struct list_head list; +}; + + +/** + * init_ksplice_pack() - Initializes a pack + * @pack: The pack to be initialized. All of the public fields of the + * pack and its associated data structures should be populated + * before this function is called. The values of the private + * fields will be ignored. + **/ +int init_ksplice_pack(struct ksplice_pack *pack); + +/** + * cleanup_ksplice_pack() - Cleans up a pack + * @pack: The pack to be cleaned up + */ +void cleanup_ksplice_pack(struct ksplice_pack *pack); + +#endif /* __KERNEL__ */ diff --git a/kernel/Makefile b/kernel/Makefile index 19fad00..e8ecb64 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -11,6 +11,8 @@ obj-y = sched.o fork.o exec_domain.o panic.o printk.o \ hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \ notifier.o ksysfs.o pm_qos_params.o sched_clock.o +CFLAGS_ksplice.o += -Iarch/$(SRCARCH)/kernel + ifdef CONFIG_FUNCTION_TRACER # Do not trace debug files and internal ftrace files CFLAGS_REMOVE_lockdep.o = -pg @@ -68,6 +70,7 @@ obj-$(CONFIG_AUDIT) += audit.o auditfilter.o obj-$(CONFIG_AUDITSYSCALL) += auditsc.o obj-$(CONFIG_AUDIT_TREE) += audit_tree.o obj-$(CONFIG_KPROBES) += kprobes.o +obj-$(CONFIG_KSPLICE) += ksplice.o obj-$(CONFIG_KGDB) += kgdb.o obj-$(CONFIG_DETECT_SOFTLOCKUP) += softlockup.o obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ diff --git a/kernel/ksplice.c b/kernel/ksplice.c new file mode 100644 index 0000000..d12a6db --- /dev/null +++ b/kernel/ksplice.c @@ -0,0 +1,2715 @@ +/* Copyright (C) 2007-2008 Jeff Arnold <jbarnold@mit.edu> + * Copyright (C) 2008 Anders Kaseorg <andersk@mit.edu>, + * Tim Abbott <tabbott@mit.edu> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA + * 02110-1301, USA. + */ + +#include <linux/module.h> +#include <linux/bug.h> +#include <linux/ctype.h> +#include <linux/debugfs.h> +#include <linux/errno.h> +#include <linux/kallsyms.h> +#include <linux/kobject.h> +#include <linux/kthread.h> +#include <linux/pagemap.h> +#include <linux/sched.h> +#include <linux/sort.h> +#include <linux/stop_machine.h> +#include <linux/sysfs.h> +#include <linux/time.h> +#include <linux/uaccess.h> +#include <linux/vmalloc.h> +#include <linux/ksplice.h> + +enum stage { + STAGE_PREPARING, STAGE_APPLIED, STAGE_REVERSED +}; + +enum run_pre_mode { + RUN_PRE_INITIAL, RUN_PRE_DEBUG, RUN_PRE_FINAL, RUN_PRE_SILENT +}; + +enum { NOVAL, TEMP, VAL }; + +typedef int __bitwise__ abort_t; + +#define OK ((__force abort_t) 0) +#define NO_MATCH ((__force abort_t) 1) +#define CODE_BUSY ((__force abort_t) 2) +#define MODULE_BUSY ((__force abort_t) 3) +#define OUT_OF_MEMORY ((__force abort_t) 4) +#define FAILED_TO_FIND ((__force abort_t) 5) +#define ALREADY_REVERSED ((__force abort_t) 6) +#define MISSING_EXPORT ((__force abort_t) 7) +#define UNEXPECTED_RUNNING_TASK ((__force abort_t) 8) +#define UNEXPECTED ((__force abort_t) 9) +#define TARGET_NOT_LOADED ((__force abort_t) 10) +#define CALL_FAILED ((__force abort_t) 11) + +struct update { + const char *kid; + const char *name; + struct kobject kobj; + enum stage stage; + abort_t abort_cause; + int debug; +#ifdef CONFIG_DEBUG_FS + struct debugfs_blob_wrapper debug_blob; + struct dentry *debugfs_dentry; +#else /* !CONFIG_DEBUG_FS */ + bool debug_continue_line; +#endif /* CONFIG_DEBUG_FS */ + bool partial; + struct list_head packs; + struct list_head unused_packs; + struct list_head conflicts; + struct list_head list; +}; + +struct conflict { + const char *process_name; + pid_t pid; + struct list_head stack; + struct list_head list; +}; + +struct conflict_addr { + unsigned long addr; + bool has_conflict; + const char *label; + struct list_head list; +}; + +struct labelval { + struct list_head list; + struct ksplice_symbol *symbol; + struct list_head *saved_vals; +}; + +struct safety_record { + struct list_head list; + const char *label; + unsigned long addr; + unsigned long size; +}; + +struct candidate_val { + struct list_head list; + unsigned long val; +}; + +struct accumulate_struct { + struct ksplice_pack *pack; + const char *desired_name; + struct list_head *vals; +}; + +struct ksplice_lookup { +/* input */ + struct ksplice_pack *pack; + struct ksplice_symbol **arr; + size_t size; +/* output */ + abort_t ret; +}; + +static LIST_HEAD(updates); +LIST_HEAD(ksplice_module_list); +EXPORT_SYMBOL_GPL(ksplice_module_list); +static struct kobject *ksplice_kobj; + +static struct kobj_type ksplice_ktype; + +static struct update *init_ksplice_update(const char *kid); +static void cleanup_ksplice_update(struct update *update); +static void maybe_cleanup_ksplice_update(struct update *update); +static void add_to_update(struct ksplice_pack *pack, struct update *update); +static int ksplice_sysfs_init(struct update *update); + +/* Preparing the relocations and patches for application */ +static abort_t apply_update(struct update *update); +static abort_t prepare_pack(struct ksplice_pack *pack); +static abort_t finalize_pack(struct ksplice_pack *pack); +static abort_t finalize_patches(struct ksplice_pack *pack); +static abort_t add_dependency_on_address(struct ksplice_pack *pack, + unsigned long addr); +static abort_t map_trampoline_pages(struct update *update); +static void unmap_trampoline_pages(struct update *update); +static void *map_writable(void *addr, size_t len); +static abort_t apply_relocs(struct ksplice_pack *pack, + const struct ksplice_reloc *relocs, + const struct ksplice_reloc *relocs_end); +static abort_t apply_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r); +static abort_t apply_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r); +static abort_t apply_howto_date(struct ksplice_pack *pack, + const struct ksplice_reloc *r); +static abort_t read_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long *valp); +static abort_t write_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long sym_addr); +static void __attribute__((noreturn)) ksplice_deleted(void); + +/* run-pre matching */ +static abort_t match_pack_sections(struct ksplice_pack *pack, + bool consider_data_sections); +static abort_t find_section(struct ksplice_pack *pack, + struct ksplice_section *sect); +static abort_t try_addr(struct ksplice_pack *pack, + struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode); +static abort_t run_pre_cmp(struct ksplice_pack *pack, + const struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode); +static void print_bytes(struct ksplice_pack *pack, + const unsigned char *run, int runc, + const unsigned char *pre, int prec); +static const struct ksplice_reloc * +init_reloc_search(struct ksplice_pack *pack, + const struct ksplice_section *sect); +static const struct ksplice_reloc *find_reloc(const struct ksplice_reloc *start, + const struct ksplice_reloc *end, + unsigned long address, + unsigned long size); +static abort_t lookup_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc **fingerp, + unsigned long addr, + const struct ksplice_reloc **relocp); +static abort_t handle_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, enum run_pre_mode mode); +static abort_t handle_howto_date(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, + enum run_pre_mode mode); +static abort_t handle_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, + enum run_pre_mode mode); +static struct ksplice_section *symbol_section(struct ksplice_pack *pack, + const struct ksplice_symbol *sym); +static int compare_section_labels(const void *va, const void *vb); +static int symbol_section_bsearch_compare(const void *a, const void *b); +static const struct ksplice_reloc *patch_reloc(struct ksplice_pack *pack, + const struct ksplice_patch *p); + +/* Computing possible addresses for symbols */ +static abort_t lookup_symbol(struct ksplice_pack *pack, + const struct ksplice_symbol *ksym, + struct list_head *vals); +static void cleanup_symbol_arrays(struct ksplice_pack *pack); +static abort_t init_symbol_arrays(struct ksplice_pack *pack); +static abort_t init_symbol_array(struct ksplice_pack *pack, + struct ksplice_symbol *start, + struct ksplice_symbol *end); +static abort_t uniquify_symbols(struct ksplice_pack *pack); +static abort_t add_matching_values(struct ksplice_lookup *lookup, + const char *sym_name, unsigned long sym_val); +static bool add_export_values(const struct symsearch *syms, + struct module *owner, + unsigned int symnum, void *data); +static int symbolp_bsearch_compare(const void *key, const void *elt); +static int compare_symbolp_names(const void *a, const void *b); +static int compare_symbolp_labels(const void *a, const void *b); +static int add_kallsyms_values(void *data, const char *name, + struct module *owner, unsigned long val); +static abort_t new_export_lookup(struct ksplice_pack *ipack, const char *name, + struct list_head *vals); + +/* Atomic update insertion and removal */ +static abort_t apply_patches(struct update *update); +static abort_t reverse_patches(struct update *update); +static int __apply_patches(void *update); +static int __reverse_patches(void *update); +static abort_t check_each_task(struct update *update); +static abort_t check_task(struct update *update, + const struct task_struct *t, bool rerun); +static abort_t check_stack(struct update *update, struct conflict *conf, + const struct thread_info *tinfo, + const unsigned long *stack); +static abort_t check_address(struct update *update, + struct conflict *conf, unsigned long addr); +static abort_t check_record(struct conflict_addr *ca, + const struct safety_record *rec, + unsigned long addr); +static bool is_stop_machine(const struct task_struct *t); +static void cleanup_conflicts(struct update *update); +static void print_conflicts(struct update *update); +static void insert_trampoline(struct ksplice_patch *p); +static abort_t verify_trampoline(struct ksplice_pack *pack, + const struct ksplice_patch *p); +static void remove_trampoline(const struct ksplice_patch *p); + +static abort_t create_labelval(struct ksplice_pack *pack, + struct ksplice_symbol *ksym, + unsigned long val, int status); +static abort_t create_safety_record(struct ksplice_pack *pack, + const struct ksplice_section *sect, + struct list_head *record_list, + unsigned long run_addr, + unsigned long run_size); +static abort_t add_candidate_val(struct ksplice_pack *pack, + struct list_head *vals, unsigned long val); +static void release_vals(struct list_head *vals); +static void set_temp_labelvals(struct ksplice_pack *pack, int status_val); + +static int contains_canary(struct ksplice_pack *pack, unsigned long blank_addr, + const struct ksplice_reloc_howto *howto); +static unsigned long follow_trampolines(struct ksplice_pack *pack, + unsigned long addr); +static bool patches_module(const struct module *a, const struct module *b); +static bool starts_with(const char *str, const char *prefix); +static bool singular(struct list_head *list); +static void *bsearch(const void *key, const void *base, size_t n, + size_t size, int (*cmp)(const void *key, const void *elt)); +static int compare_relocs(const void *a, const void *b); +static int reloc_bsearch_compare(const void *key, const void *elt); + +/* Debugging */ +static abort_t init_debug_buf(struct update *update); +static void clear_debug_buf(struct update *update); +static int __attribute__((format(printf, 2, 3))) +_ksdebug(struct update *update, const char *fmt, ...); +#define ksdebug(pack, fmt, ...) \ + _ksdebug(pack->update, fmt, ## __VA_ARGS__) + +/* Architecture-specific functions defined in arch/ARCH/kernel/ksplice-arch.c */ +static abort_t prepare_trampoline(struct ksplice_pack *pack, + struct ksplice_patch *p); +static abort_t trampoline_target(struct ksplice_pack *pack, unsigned long addr, + unsigned long *new_addr); +static abort_t handle_paravirt(struct ksplice_pack *pack, unsigned long pre, + unsigned long run, int *matched); +static abort_t handle_bug(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long run_addr); +static abort_t handle_extable(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long run_addr); +static bool valid_stack_ptr(const struct thread_info *tinfo, const void *p); + +#include "ksplice-arch.c" + +#define clear_list(head, type, member) \ + do { \ + struct list_head *_pos, *_n; \ + list_for_each_safe(_pos, _n, head) { \ + list_del(_pos); \ + kfree(list_entry(_pos, type, member)); \ + } \ + } while (0) + +int init_ksplice_pack(struct ksplice_pack *pack) +{ + struct update *update; + struct ksplice_patch *p; + struct ksplice_section *s; + int ret = 0; + + INIT_LIST_HEAD(&pack->temp_labelvals); + INIT_LIST_HEAD(&pack->safety_records); + + sort(pack->helper_relocs, + pack->helper_relocs_end - pack->helper_relocs, + sizeof(*pack->helper_relocs), compare_relocs, NULL); + sort(pack->primary_relocs, + pack->primary_relocs_end - pack->primary_relocs, + sizeof(*pack->primary_relocs), compare_relocs, NULL); + sort(pack->helper_sections, + pack->helper_sections_end - pack->helper_sections, + sizeof(*pack->helper_sections), compare_section_labels, NULL); + + mutex_lock(&module_mutex); + for (p = pack->patches; p < pack->patches_end; p++) + p->vaddr = NULL; + for (s = pack->helper_sections; s < pack->helper_sections_end; s++) + s->match_map = NULL; + for (p = pack->patches; p < pack->patches_end; p++) { + const struct ksplice_reloc *r = patch_reloc(pack, p); + if (r == NULL) { + ret = -ENOENT; + goto out; + } + if (p->type == KSPLICE_PATCH_DATA) { + s = symbol_section(pack, r->symbol); + if (s == NULL) { + ret = -ENOENT; + goto out; + } + if (s->flags & KSPLICE_SECTION_DATA) + s->flags = (s->flags & ~KSPLICE_SECTION_DATA) | + KSPLICE_SECTION_RODATA; + } + } + + list_for_each_entry(update, &updates, list) { + if (strcmp(pack->kid, update->kid) == 0) { + if (update->stage != STAGE_PREPARING) { + ret = -EPERM; + goto out; + } + add_to_update(pack, update); + ret = 0; + goto out; + } + } + update = init_ksplice_update(pack->kid); + if (update == NULL) { + ret = -ENOMEM; + goto out; + } + ret = ksplice_sysfs_init(update); + if (ret != 0) { + cleanup_ksplice_update(update); + goto out; + } + add_to_update(pack, update); +out: + mutex_unlock(&module_mutex); + return ret; +} +EXPORT_SYMBOL_GPL(init_ksplice_pack); + +void cleanup_ksplice_pack(struct ksplice_pack *pack) +{ + if (pack->update == NULL) + return; + if (pack->update->stage == STAGE_APPLIED) { + /* If the pack wasn't actually applied (because we + only applied this update to loaded modules and this + target wasn't loaded), then unregister the pack + from the list of unused packs */ + struct ksplice_pack *p; + bool found = false; + + mutex_lock(&module_mutex); + list_for_each_entry(p, &pack->update->unused_packs, list) { + if (p == pack) + found = true; + } + if (found) + list_del(&pack->list); + mutex_unlock(&module_mutex); + return; + } + mutex_lock(&module_mutex); + list_del(&pack->list); + mutex_unlock(&module_mutex); + if (pack->update->stage == STAGE_PREPARING) + maybe_cleanup_ksplice_update(pack->update); + pack->update = NULL; +} +EXPORT_SYMBOL_GPL(cleanup_ksplice_pack); + +static struct update *init_ksplice_update(const char *kid) +{ + struct update *update; + update = kcalloc(1, sizeof(struct update), GFP_KERNEL); + if (update == NULL) + return NULL; + update->name = kasprintf(GFP_KERNEL, "ksplice_%s", kid); + if (update->name == NULL) { + kfree(update); + return NULL; + } + update->kid = kstrdup(kid, GFP_KERNEL); + if (update->kid == NULL) { + kfree(update->name); + kfree(update); + return NULL; + } + if (try_module_get(THIS_MODULE) != 1) { + kfree(update->kid); + kfree(update->name); + kfree(update); + return NULL; + } + INIT_LIST_HEAD(&update->packs); + INIT_LIST_HEAD(&update->unused_packs); + if (init_debug_buf(update) != OK) { + module_put(THIS_MODULE); + kfree(update->kid); + kfree(update->name); + kfree(update); + return NULL; + } + list_add(&update->list, &updates); + update->stage = STAGE_PREPARING; + update->abort_cause = OK; + update->partial = 0; + INIT_LIST_HEAD(&update->conflicts); + return update; +} + +static void cleanup_ksplice_update(struct update *update) +{ + mutex_lock(&module_mutex); + list_del(&update->list); + mutex_unlock(&module_mutex); + cleanup_conflicts(update); + clear_debug_buf(update); + kfree(update->kid); + kfree(update->name); + kfree(update); + module_put(THIS_MODULE); +} + +static void maybe_cleanup_ksplice_update(struct update *update) +{ + if (list_empty(&update->packs) && list_empty(&update->unused_packs)) + kobject_put(&update->kobj); +} + +static void add_to_update(struct ksplice_pack *pack, struct update *update) +{ + pack->update = update; + list_add(&pack->list, &update->unused_packs); + pack->module_list_entry.primary = pack->primary; +} + +static int ksplice_sysfs_init(struct update *update) +{ + int ret = 0; + memset(&update->kobj, 0, sizeof(update->kobj)); + ret = kobject_init_and_add(&update->kobj, &ksplice_ktype, + ksplice_kobj, "%s", update->kid); + if (ret != 0) + return ret; + kobject_uevent(&update->kobj, KOBJ_ADD); + return 0; +} + +static abort_t apply_update(struct update *update) +{ + struct ksplice_pack *pack, *n; + abort_t ret; + int retval; + + mutex_lock(&module_mutex); + list_for_each_entry_safe(pack, n, &update->unused_packs, list) { + if (strcmp(pack->target_name, "vmlinux") == 0) { + pack->target = NULL; + } else if (pack->target == NULL) { + pack->target = find_module(pack->target_name); + if (pack->target == NULL || + !module_is_live(pack->target)) { + if (update->partial) { + continue; + } else { + ret = TARGET_NOT_LOADED; + goto out; + } + } + retval = use_module(pack->primary, pack->target); + if (retval != 1) { + ret = UNEXPECTED; + goto out; + } + } + list_del(&pack->list); + list_add_tail(&pack->list, &update->packs); + pack->module_list_entry.target = pack->target; + + } + + list_for_each_entry(pack, &update->packs, list) { + const struct ksplice_section *sect; + for (sect = pack->primary_sections; + sect < pack->primary_sections_end; sect++) { + struct safety_record *rec = kmalloc(sizeof(*rec), + GFP_KERNEL); + if (rec == NULL) { + ret = OUT_OF_MEMORY; + goto out; + } + rec->addr = sect->address; + rec->size = sect->size; + rec->label = sect->symbol->label; + list_add(&rec->list, &pack->safety_records); + } + } + + list_for_each_entry(pack, &update->packs, list) { + ret = init_symbol_arrays(pack); + if (ret != OK) { + cleanup_symbol_arrays(pack); + goto out; + } + ret = prepare_pack(pack); + cleanup_symbol_arrays(pack); + if (ret != OK) + goto out; + } + ret = apply_patches(update); +out: + list_for_each_entry(pack, &update->packs, list) { + struct ksplice_section *s; + if (update->stage == STAGE_PREPARING) + clear_list(&pack->safety_records, struct safety_record, + list); + for (s = pack->helper_sections; s < pack->helper_sections_end; + s++) { + if (s->match_map != NULL) { + vfree(s->match_map); + s->match_map = NULL; + } + } + } + mutex_unlock(&module_mutex); + return ret; +} + +static int compare_symbolp_names(const void *a, const void *b) +{ + const struct ksplice_symbol *const *sympa = a, *const *sympb = b; + if ((*sympa)->name == NULL && (*sympb)->name == NULL) + return 0; + if ((*sympa)->name == NULL) + return -1; + if ((*sympb)->name == NULL) + return 1; + return strcmp((*sympa)->name, (*sympb)->name); +} + +static int compare_symbolp_labels(const void *a, const void *b) +{ + const struct ksplice_symbol *const *sympa = a, *const *sympb = b; + return strcmp((*sympa)->label, (*sympb)->label); +} + +static int symbolp_bsearch_compare(const void *key, const void *elt) +{ + const char *name = key; + const struct ksplice_symbol *const *symp = elt; + const struct ksplice_symbol *sym = *symp; + if (sym->name == NULL) + return 1; + return strcmp(name, sym->name); +} + +static abort_t add_matching_values(struct ksplice_lookup *lookup, + const char *sym_name, unsigned long sym_val) +{ + struct ksplice_symbol **symp; + abort_t ret; + + symp = bsearch(sym_name, lookup->arr, lookup->size, + sizeof(*lookup->arr), symbolp_bsearch_compare); + if (symp == NULL) + return OK; + + while (symp > lookup->arr && + symbolp_bsearch_compare(sym_name, symp - 1) == 0) + symp--; + + for (; symp < lookup->arr + lookup->size; symp++) { + struct ksplice_symbol *sym = *symp; + if (sym->name == NULL || strcmp(sym_name, sym->name) != 0) + break; + ret = add_candidate_val(lookup->pack, sym->vals, sym_val); + if (ret != OK) + return ret; + } + return OK; +} + +static int add_kallsyms_values(void *data, const char *name, + struct module *owner, unsigned long val) +{ + struct ksplice_lookup *lookup = data; + if (owner == lookup->pack->primary || + !patches_module(owner, lookup->pack->target)) + return (__force int)OK; + return (__force int)add_matching_values(lookup, name, val); +} + +static bool add_export_values(const struct symsearch *syms, + struct module *owner, + unsigned int symnum, void *data) +{ + struct ksplice_lookup *lookup = data; + abort_t ret; + + ret = add_matching_values(lookup, syms->start[symnum].name, + syms->start[symnum].value); + if (ret != OK) { + lookup->ret = ret; + return true; + } + return false; +} + +static void cleanup_symbol_arrays(struct ksplice_pack *pack) +{ + struct ksplice_symbol *sym; + for (sym = pack->primary_symbols; sym < pack->primary_symbols_end; + sym++) { + if (sym->vals != NULL) { + clear_list(sym->vals, struct candidate_val, list); + kfree(sym->vals); + sym->vals = NULL; + } + } + for (sym = pack->helper_symbols; sym < pack->helper_symbols_end; sym++) { + if (sym->vals != NULL) { + clear_list(sym->vals, struct candidate_val, list); + kfree(sym->vals); + sym->vals = NULL; + } + } +} + +static abort_t uniquify_symbols(struct ksplice_pack *pack) +{ + struct ksplice_reloc *r; + struct ksplice_section *s; + struct ksplice_symbol *sym, **sym_arr, **symp; + size_t size = pack->primary_symbols_end - pack->primary_symbols; + + if (size == 0) + return OK; + + sym_arr = vmalloc(sizeof(*sym_arr) * size); + if (sym_arr == NULL) + return OUT_OF_MEMORY; + + for (symp = sym_arr, sym = pack->primary_symbols; + symp < sym_arr + size && sym < pack->primary_symbols_end; + sym++, symp++) + *symp = sym; + + sort(sym_arr, size, sizeof(*sym_arr), compare_symbolp_labels, NULL); + + for (r = pack->helper_relocs; r < pack->helper_relocs_end; r++) { + symp = bsearch(&r->symbol, sym_arr, size, sizeof(*sym_arr), + compare_symbolp_labels); + if (symp != NULL) { + if ((*symp)->name == NULL) + (*symp)->name = r->symbol->name; + r->symbol = *symp; + } + } + + for (s = pack->helper_sections; s < pack->helper_sections_end; s++) { + symp = bsearch(&s->symbol, sym_arr, size, sizeof(*sym_arr), + compare_symbolp_labels); + if (symp != NULL) { + if ((*symp)->name == NULL) + (*symp)->name = s->symbol->name; + s->symbol = *symp; + } + } + + vfree(sym_arr); + return OK; +} + +static abort_t init_symbol_array(struct ksplice_pack *pack, + struct ksplice_symbol *start, + struct ksplice_symbol *end) +{ + struct ksplice_symbol *sym, **sym_arr, **symp; + struct ksplice_lookup lookup; + size_t size = end - start; + abort_t ret; + + if (size == 0) + return OK; + + for (sym = start; sym < end; sym++) { + if (starts_with(sym->label, "__ksymtab")) { + const struct kernel_symbol *ksym; + const char *colon = strchr(sym->label, ':'); + const char *name = colon + 1; + if (colon == NULL) + continue; + ksym = find_symbol(name, NULL, NULL, true, false); + if (ksym == NULL) { + ksdebug(pack, "Could not find kernel_symbol " + "structure for %s\n", name); + continue; + } + sym->value = (unsigned long)ksym; + sym->vals = NULL; + continue; + } + + sym->vals = kmalloc(sizeof(*sym->vals), GFP_KERNEL); + if (sym->vals == NULL) + return OUT_OF_MEMORY; + INIT_LIST_HEAD(sym->vals); + sym->value = 0; + } + + sym_arr = vmalloc(sizeof(*sym_arr) * size); + if (sym_arr == NULL) + return OUT_OF_MEMORY; + + for (symp = sym_arr, sym = start; symp < sym_arr + size && sym < end; + sym++, symp++) + *symp = sym; + + sort(sym_arr, size, sizeof(*sym_arr), compare_symbolp_names, NULL); + + lookup.pack = pack; + lookup.arr = sym_arr; + lookup.size = size; + lookup.ret = OK; + + each_symbol(add_export_values, &lookup); + ret = lookup.ret; + if (ret == OK) + ret = (__force abort_t) + kallsyms_on_each_symbol(add_kallsyms_values, &lookup); + vfree(sym_arr); + return ret; +} + +static abort_t init_symbol_arrays(struct ksplice_pack *pack) +{ + abort_t ret; + + ret = uniquify_symbols(pack); + if (ret != OK) + return ret; + + ret = init_symbol_array(pack, pack->helper_symbols, + pack->helper_symbols_end); + if (ret != OK) + return ret; + + ret = init_symbol_array(pack, pack->primary_symbols, + pack->primary_symbols_end); + if (ret != OK) + return ret; + + return OK; +} + +static abort_t prepare_pack(struct ksplice_pack *pack) +{ + abort_t ret; + + ksdebug(pack, "Preparing and checking %s\n", pack->name); + ret = match_pack_sections(pack, false); + if (ret == NO_MATCH) { + /* It is possible that by using relocations from .data sections + we can successfully run-pre match the rest of the sections. + To avoid using any symbols obtained from .data sections + (which may be unreliable) in the post code, we first prepare + the post code and then try to run-pre match the remaining + sections with the help of .data sections. + */ + ksdebug(pack, "Continuing without some sections; we might " + "find them later.\n"); + ret = finalize_pack(pack); + if (ret != OK) { + ksdebug(pack, "Aborted. Unable to continue without " + "the unmatched sections.\n"); + return ret; + } + + ksdebug(pack, "run-pre: Considering .data sections to find the " + "unmatched sections\n"); + ret = match_pack_sections(pack, true); + if (ret != OK) + return ret; + + ksdebug(pack, "run-pre: Found all previously unmatched " + "sections\n"); + return OK; + } else if (ret != OK) { + return ret; + } + + return finalize_pack(pack); +} + +static abort_t finalize_pack(struct ksplice_pack *pack) +{ + abort_t ret; + ret = apply_relocs(pack, pack->primary_relocs, + pack->primary_relocs_end); + if (ret != OK) + return ret; + + ret = finalize_patches(pack); + if (ret != OK) + return ret; + + return OK; +} + +static abort_t finalize_patches(struct ksplice_pack *pack) +{ + struct ksplice_patch *p; + struct safety_record *rec; + abort_t ret; + + for (p = pack->patches; p < pack->patches_end; p++) { + bool found = false; + list_for_each_entry(rec, &pack->safety_records, list) { + if (rec->addr <= p->oldaddr && + p->oldaddr < rec->addr + rec->size) { + found = true; + break; + } + } + if (!found && p->type != KSPLICE_PATCH_EXPORT) { + const struct ksplice_reloc *r = patch_reloc(pack, p); + if (r == NULL) { + ksdebug(pack, "A patch with no ksplice_reloc at" + " its oldaddr has no safety record\n"); + return NO_MATCH; + } + ksdebug(pack, "No safety record for patch with oldaddr " + "%s+%lx\n", r->symbol->label, r->target_addend); + return NO_MATCH; + } + + if (p->type == KSPLICE_PATCH_TEXT) { + ret = prepare_trampoline(pack, p); + if (ret != OK) + return ret; + } + + if (found && rec->addr + rec->size < p->oldaddr + p->size) { + ksdebug(pack, "Safety record %s is too short for " + "patch\n", rec->label); + return UNEXPECTED; + } + + if (p->type == KSPLICE_PATCH_TEXT) { + if (p->repladdr == 0) + p->repladdr = (unsigned long)ksplice_deleted; + } + + ret = add_dependency_on_address(pack, p->oldaddr); + if (ret != OK) + return ret; + } + return OK; +} + +static abort_t map_trampoline_pages(struct update *update) +{ + struct ksplice_pack *pack; + list_for_each_entry(pack, &update->packs, list) { + struct ksplice_patch *p; + for (p = pack->patches; p < pack->patches_end; p++) { + p->vaddr = map_writable((void *)p->oldaddr, p->size); + if (p->vaddr == NULL) { + ksdebug(pack, "Unable to map oldaddr read/write" + "\n"); + unmap_trampoline_pages(update); + return UNEXPECTED; + } + } + } + return OK; +} + +static void unmap_trampoline_pages(struct update *update) +{ + struct ksplice_pack *pack; + list_for_each_entry(pack, &update->packs, list) { + struct ksplice_patch *p; + for (p = pack->patches; p < pack->patches_end; p++) { + vunmap((void *)((unsigned long)p->vaddr & PAGE_MASK)); + p->vaddr = NULL; + } + } +} + +/* Based off of linux's text_poke. */ +static void *map_writable(void *addr, size_t len) +{ + void *vaddr; + int nr_pages = 2; + struct page *pages[2]; + + if (__module_text_address((unsigned long)addr) == NULL && + __module_data_address((unsigned long)addr) == NULL) { + pages[0] = virt_to_page(addr); + WARN_ON(!PageReserved(pages[0])); + pages[1] = virt_to_page(addr + PAGE_SIZE); + } else { + pages[0] = vmalloc_to_page(addr); + pages[1] = vmalloc_to_page(addr + PAGE_SIZE); + } + if (!pages[0]) + return NULL; + if (!pages[1]) + nr_pages = 1; + vaddr = vmap(pages, nr_pages, VM_MAP, PAGE_KERNEL); + if (vaddr == NULL) + return NULL; + return vaddr + offset_in_page(addr); +} + +static abort_t add_dependency_on_address(struct ksplice_pack *pack, + unsigned long addr) +{ + struct ksplice_pack *p; + struct module *m = + __module_text_address(follow_trampolines(pack, addr)); + if (m == NULL) + return OK; + list_for_each_entry(p, &pack->update->packs, list) { + if (m == p->primary) + return OK; + } + if (use_module(pack->primary, m) != 1) + return MODULE_BUSY; + return OK; +} + +static abort_t apply_relocs(struct ksplice_pack *pack, + const struct ksplice_reloc *relocs, + const struct ksplice_reloc *relocs_end) +{ + const struct ksplice_reloc *r; + for (r = relocs; r < relocs_end; r++) { + abort_t ret = apply_reloc(pack, r); + if (ret != OK) + return ret; + } + return OK; +} + +static abort_t apply_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r) +{ + switch (r->howto->type) { + case KSPLICE_HOWTO_RELOC: + case KSPLICE_HOWTO_RELOC_PATCH: + return apply_howto_reloc(pack, r); + case KSPLICE_HOWTO_DATE: + case KSPLICE_HOWTO_TIME: + return apply_howto_date(pack, r); + default: + ksdebug(pack, "Unexpected howto type %d\n", r->howto->type); + return UNEXPECTED; + } +} + +static abort_t apply_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r) +{ + abort_t ret; + int canary_ret; + unsigned long sym_addr; + LIST_HEAD(vals); + + canary_ret = contains_canary(pack, r->blank_addr, r->howto); + if (canary_ret < 0) + return UNEXPECTED; + if (canary_ret == 0) { + ksdebug(pack, "reloc: skipped %lx to %s+%lx (altinstr)\n", + r->blank_addr, r->symbol->label, r->target_addend); + return OK; + } + + ret = lookup_symbol(pack, r->symbol, &vals); + if (ret != OK) { + release_vals(&vals); + return ret; + } + if (!singular(&vals) || (r->symbol->vals != NULL && + r->howto->type == KSPLICE_HOWTO_RELOC_PATCH)) { + release_vals(&vals); + ksdebug(pack, "Failed to find %s for reloc\n", + r->symbol->label); + return FAILED_TO_FIND; + } + sym_addr = list_entry(vals.next, struct candidate_val, list)->val; + release_vals(&vals); + + ret = write_reloc_value(pack, r, r->blank_addr, + r->howto->pcrel ? sym_addr - r->blank_addr : + sym_addr); + if (ret != OK) + return ret; + + ksdebug(pack, "reloc: %lx to %s+%lx (S=%lx ", r->blank_addr, + r->symbol->label, r->target_addend, sym_addr); + switch (r->howto->size) { + case 1: + ksdebug(pack, "aft=%02x)\n", *(uint8_t *)r->blank_addr); + break; + case 2: + ksdebug(pack, "aft=%04x)\n", *(uint16_t *)r->blank_addr); + break; + case 4: + ksdebug(pack, "aft=%08x)\n", *(uint32_t *)r->blank_addr); + break; +#if BITS_PER_LONG >= 64 + case 8: + ksdebug(pack, "aft=%016llx)\n", *(uint64_t *)r->blank_addr); + break; +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + /* Create labelvals so that we can verify our choices in the second + round of run-pre matching that considers data sections. */ + ret = create_labelval(pack, r->symbol, sym_addr, VAL); + if (ret != OK) + return ret; + + return add_dependency_on_address(pack, sym_addr); +} + +static abort_t apply_howto_date(struct ksplice_pack *pack, + const struct ksplice_reloc *r) +{ + if (r->symbol->vals != NULL) { + ksdebug(pack, "Failed to find %s for date\n", r->symbol->label); + return FAILED_TO_FIND; + } + memcpy((unsigned char *)r->blank_addr, + (const unsigned char *)r->symbol->value, r->howto->size); + return OK; +} + +static abort_t read_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long *valp) +{ + unsigned char bytes[sizeof(long)]; + unsigned long val; + const struct ksplice_reloc_howto *howto = r->howto; + + if (howto->size <= 0 || howto->size > sizeof(long)) { + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + if (probe_kernel_read(bytes, (void *)addr, howto->size) == -EFAULT) + return NO_MATCH; + + switch (howto->size) { + case 1: + val = *(uint8_t *)bytes; + break; + case 2: + val = *(uint16_t *)bytes; + break; + case 4: + val = *(uint32_t *)bytes; + break; +#if BITS_PER_LONG >= 64 + case 8: + val = *(uint64_t *)bytes; + break; +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + val &= howto->dst_mask; + if (howto->signed_addend) + val |= -(val & (howto->dst_mask & ~(howto->dst_mask >> 1))); + val <<= howto->rightshift; + val -= r->insn_addend + r->target_addend; + *valp = val; + return OK; +} + +static abort_t write_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long sym_addr) +{ + unsigned long val = sym_addr + r->target_addend + r->insn_addend; + const struct ksplice_reloc_howto *howto = r->howto; + val >>= howto->rightshift; + switch (howto->size) { + case 1: + *(uint8_t *)addr = (*(uint8_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; + case 2: + *(uint16_t *)addr = (*(uint16_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; + case 4: + *(uint32_t *)addr = (*(uint32_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; +#if BITS_PER_LONG >= 64 + case 8: + *(uint64_t *)addr = (*(uint64_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + if (read_reloc_value(pack, r, addr, &val) != OK || val != sym_addr) { + ksdebug(pack, "Aborted. Relocation overflow.\n"); + return UNEXPECTED; + } + + return OK; +} + +static void __attribute__((noreturn)) ksplice_deleted(void) +{ + printk(KERN_CRIT "Called a kernel function deleted by Ksplice!\n"); + BUG(); +} + +static abort_t match_pack_sections(struct ksplice_pack *pack, + bool consider_data_sections) +{ + struct ksplice_section *sect; + abort_t ret; + int remaining = 0; + bool progress; + + for (sect = pack->helper_sections; sect < pack->helper_sections_end; + sect++) { + if ((sect->flags & KSPLICE_SECTION_DATA) == 0 && + (sect->flags & KSPLICE_SECTION_STRING) == 0 && + (sect->flags & KSPLICE_SECTION_MATCHED) == 0) + remaining++; + } + + while (remaining > 0) { + progress = false; + for (sect = pack->helper_sections; + sect < pack->helper_sections_end; sect++) { + if ((sect->flags & KSPLICE_SECTION_MATCHED) != 0) + continue; + if ((!consider_data_sections && + (sect->flags & KSPLICE_SECTION_DATA) != 0) || + (sect->flags & KSPLICE_SECTION_STRING) != 0) + continue; + ret = find_section(pack, sect); + if (ret == OK) { + sect->flags |= KSPLICE_SECTION_MATCHED; + if ((sect->flags & KSPLICE_SECTION_DATA) == 0) + remaining--; + progress = true; + } else if (ret != NO_MATCH) { + return ret; + } + } + + if (progress) + continue; + + for (sect = pack->helper_sections; + sect < pack->helper_sections_end; sect++) { + if ((sect->flags & KSPLICE_SECTION_MATCHED) != 0 || + (sect->flags & KSPLICE_SECTION_STRING) != 0) + continue; + ksdebug(pack, "run-pre: could not match %s " + "section %s\n", + (sect->flags & KSPLICE_SECTION_DATA) != 0 ? + "data" : + (sect->flags & KSPLICE_SECTION_RODATA) != 0 ? + "rodata" : "text", sect->symbol->label); + } + ksdebug(pack, "Aborted. run-pre: could not match some " + "sections.\n"); + return NO_MATCH; + } + return OK; +} + +static abort_t find_section(struct ksplice_pack *pack, + struct ksplice_section *sect) +{ + int i; + abort_t ret; + unsigned long run_addr; + LIST_HEAD(vals); + struct candidate_val *v, *n; + + ret = lookup_symbol(pack, sect->symbol, &vals); + if (ret != OK) { + release_vals(&vals); + return ret; + } + + ksdebug(pack, "run-pre: starting sect search for %s\n", + sect->symbol->label); + + list_for_each_entry_safe(v, n, &vals, list) { + run_addr = v->val; + + yield(); + ret = try_addr(pack, sect, run_addr, NULL, RUN_PRE_INITIAL); + if (ret == NO_MATCH) { + list_del(&v->list); + kfree(v); + } else if (ret != OK) { + release_vals(&vals); + return ret; + } + } + + if (singular(&vals)) { + LIST_HEAD(safety_records); + run_addr = list_entry(vals.next, struct candidate_val, + list)->val; + ret = try_addr(pack, sect, run_addr, &safety_records, + RUN_PRE_FINAL); + release_vals(&vals); + if (ret != OK) { + clear_list(&safety_records, struct safety_record, list); + ksdebug(pack, "run-pre: Final run failed for sect " + "%s:\n", sect->symbol->label); + } else { + list_splice(&safety_records, &pack->safety_records); + } + return ret; + } else if (!list_empty(&vals)) { + struct candidate_val *val; + ksdebug(pack, "run-pre: multiple candidates for sect %s:\n", + sect->symbol->label); + i = 0; + list_for_each_entry(val, &vals, list) { + i++; + ksdebug(pack, "%lx\n", val->val); + if (i > 5) { + ksdebug(pack, "...\n"); + break; + } + } + release_vals(&vals); + return NO_MATCH; + } + release_vals(&vals); + return NO_MATCH; +} + +static abort_t try_addr(struct ksplice_pack *pack, + struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode) +{ + abort_t ret; + const struct module *run_module; + + if ((sect->flags & KSPLICE_SECTION_RODATA) != 0 || + (sect->flags & KSPLICE_SECTION_DATA) != 0) + run_module = __module_data_address(run_addr); + else + run_module = __module_text_address(run_addr); + if (run_module == pack->primary) { + ksdebug(pack, "run-pre: unexpected address %lx in primary " + "module %s for sect %s\n", run_addr, run_module->name, + sect->symbol->label); + return UNEXPECTED; + } + if (!patches_module(run_module, pack->target)) { + ksdebug(pack, "run-pre: ignoring address %lx in other module " + "%s for sect %s\n", run_addr, run_module == NULL ? + "vmlinux" : run_module->name, sect->symbol->label); + return NO_MATCH; + } + + ret = create_labelval(pack, sect->symbol, run_addr, TEMP); + if (ret != OK) + return ret; + + ret = run_pre_cmp(pack, sect, run_addr, safety_records, mode); + if (ret == NO_MATCH && mode != RUN_PRE_FINAL) { + set_temp_labelvals(pack, NOVAL); + ksdebug(pack, "run-pre: %s sect %s does not match (r_a=%lx " + "p_a=%lx s=%lx)\n", + (sect->flags & KSPLICE_SECTION_RODATA) != 0 ? "rodata" : + (sect->flags & KSPLICE_SECTION_DATA) != 0 ? "data" : + "text", sect->symbol->label, run_addr, sect->address, + sect->size); + ksdebug(pack, "run-pre: "); + if (pack->update->debug >= 1) { + ret = run_pre_cmp(pack, sect, run_addr, safety_records, + RUN_PRE_DEBUG); + set_temp_labelvals(pack, NOVAL); + } + ksdebug(pack, "\n"); + return ret; + } else if (ret != OK) { + set_temp_labelvals(pack, NOVAL); + return ret; + } + + if (mode != RUN_PRE_FINAL) { + set_temp_labelvals(pack, NOVAL); + ksdebug(pack, "run-pre: candidate for sect %s=%lx\n", + sect->symbol->label, run_addr); + return OK; + } + + set_temp_labelvals(pack, VAL); + ksdebug(pack, "run-pre: found sect %s=%lx\n", sect->symbol->label, + run_addr); + return OK; +} + +static abort_t run_pre_cmp(struct ksplice_pack *pack, + const struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode) +{ + int matched = 0; + abort_t ret; + const struct ksplice_reloc *r, *finger; + const unsigned char *pre, *run, *pre_start, *run_start; + unsigned char runval; + + pre_start = (const unsigned char *)sect->address; + run_start = (const unsigned char *)run_addr; + + finger = init_reloc_search(pack, sect); + + pre = pre_start; + run = run_start; + while (pre < pre_start + sect->size) { + unsigned long offset = pre - pre_start; + ret = lookup_reloc(pack, &finger, (unsigned long)pre, &r); + if (ret == OK) { + ret = handle_reloc(pack, sect, r, (unsigned long)run, + mode); + if (ret != OK) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "reloc in sect does not " + "match after %lx/%lx bytes\n", + offset, sect->size); + return ret; + } + if (mode == RUN_PRE_DEBUG) + print_bytes(pack, run, r->howto->size, pre, + r->howto->size); + pre += r->howto->size; + run += r->howto->size; + finger++; + continue; + } else if (ret != NO_MATCH) { + return ret; + } + + if ((sect->flags & KSPLICE_SECTION_TEXT) != 0) { + ret = handle_paravirt(pack, (unsigned long)pre, + (unsigned long)run, &matched); + if (ret != OK) + return ret; + if (matched != 0) { + if (mode == RUN_PRE_DEBUG) + print_bytes(pack, run, matched, pre, + matched); + pre += matched; + run += matched; + continue; + } + } + + if (probe_kernel_read(&runval, (void *)run, 1) == -EFAULT) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "sect unmapped after %lx/%lx " + "bytes\n", offset, sect->size); + return NO_MATCH; + } + + if (runval != *pre && + (sect->flags & KSPLICE_SECTION_DATA) == 0) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "sect does not match after " + "%lx/%lx bytes\n", offset, sect->size); + if (mode == RUN_PRE_DEBUG) { + print_bytes(pack, run, 1, pre, 1); + ksdebug(pack, "[p_o=%lx] ! ", offset); + print_bytes(pack, run + 1, 2, pre + 1, 2); + } + return NO_MATCH; + } + if (mode == RUN_PRE_DEBUG) + print_bytes(pack, run, 1, pre, 1); + pre++; + run++; + } + return create_safety_record(pack, sect, safety_records, run_addr, + run - run_start); +} + +static void print_bytes(struct ksplice_pack *pack, + const unsigned char *run, int runc, + const unsigned char *pre, int prec) +{ + int o; + int matched = min(runc, prec); + for (o = 0; o < matched; o++) { + if (run[o] == pre[o]) + ksdebug(pack, "%02x ", run[o]); + else + ksdebug(pack, "%02x/%02x ", run[o], pre[o]); + } + for (o = matched; o < runc; o++) + ksdebug(pack, "%02x/ ", run[o]); + for (o = matched; o < prec; o++) + ksdebug(pack, "/%02x ", pre[o]); +} + +struct range { + unsigned long address; + unsigned long size; +}; + +static int reloc_bsearch_compare(const void *key, const void *elt) +{ + const struct range *range = key; + const struct ksplice_reloc *r = elt; + if (range->address + range->size <= r->blank_addr) + return -1; + if (range->address > r->blank_addr) + return 1; + return 0; +} + +static const struct ksplice_reloc *find_reloc(const struct ksplice_reloc *start, + const struct ksplice_reloc *end, + unsigned long address, + unsigned long size) +{ + const struct ksplice_reloc *r; + struct range range = { address, size }; + r = bsearch((void *)&range, start, end - start, sizeof(*r), + reloc_bsearch_compare); + if (r == NULL) + return NULL; + while (r > start && (r - 1)->blank_addr >= address) + r--; + return r; +} + +static const struct ksplice_reloc * +init_reloc_search(struct ksplice_pack *pack, const struct ksplice_section *sect) +{ + const struct ksplice_reloc *r; + r = find_reloc(pack->helper_relocs, pack->helper_relocs_end, + sect->address, sect->size); + if (r == NULL) + return pack->helper_relocs_end; + return r; +} + +static abort_t lookup_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc **fingerp, + unsigned long addr, + const struct ksplice_reloc **relocp) +{ + const struct ksplice_reloc *r = *fingerp; + int canary_ret; + + while (r < pack->helper_relocs_end && + addr >= r->blank_addr + r->howto->size && + !(addr == r->blank_addr && r->howto->size == 0)) + r++; + *fingerp = r; + if (r == pack->helper_relocs_end) + return NO_MATCH; + if (addr < r->blank_addr) + return NO_MATCH; + *relocp = r; + if (r->howto->type != KSPLICE_HOWTO_RELOC) + return OK; + + canary_ret = contains_canary(pack, r->blank_addr, r->howto); + if (canary_ret < 0) + return UNEXPECTED; + if (canary_ret == 0) { + ksdebug(pack, "run-pre: reloc skipped at p_a=%lx to %s+%lx " + "(altinstr)\n", r->blank_addr, r->symbol->label, + r->target_addend); + return NO_MATCH; + } + if (addr != r->blank_addr) { + ksdebug(pack, "Invalid nonzero relocation offset\n"); + return UNEXPECTED; + } + return OK; +} + +static abort_t handle_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, enum run_pre_mode mode) +{ + switch (r->howto->type) { + case KSPLICE_HOWTO_RELOC: + return handle_howto_reloc(pack, sect, r, run_addr, mode); + case KSPLICE_HOWTO_DATE: + case KSPLICE_HOWTO_TIME: + return handle_howto_date(pack, sect, r, run_addr, mode); + case KSPLICE_HOWTO_BUG: + return handle_bug(pack, r, run_addr); + case KSPLICE_HOWTO_EXTABLE: + return handle_extable(pack, r, run_addr); + default: + ksdebug(pack, "Unexpected howto type %d\n", r->howto->type); + return UNEXPECTED; + } +} + +static abort_t handle_howto_date(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, enum run_pre_mode mode) +{ + abort_t ret; + char *buf = kmalloc(r->howto->size, GFP_KERNEL); + + if (buf == NULL) + return OUT_OF_MEMORY; + if (probe_kernel_read(buf, (void *)run_addr, r->howto->size) == -EFAULT) { + ret = NO_MATCH; + goto out; + } + + switch (r->howto->type) { + case KSPLICE_HOWTO_TIME: + if (isdigit(buf[0]) && isdigit(buf[1]) && buf[2] == ':' && + isdigit(buf[3]) && isdigit(buf[4]) && buf[5] == ':' && + isdigit(buf[6]) && isdigit(buf[7])) + ret = OK; + else + ret = NO_MATCH; + break; + case KSPLICE_HOWTO_DATE: + if (isalpha(buf[0]) && isalpha(buf[1]) && isalpha(buf[2]) && + buf[3] == ' ' && (buf[4] == ' ' || isdigit(buf[4])) && + isdigit(buf[5]) && buf[6] == ' ' && isdigit(buf[7]) && + isdigit(buf[8]) && isdigit(buf[9]) && isdigit(buf[10])) + ret = OK; + else + ret = NO_MATCH; + break; + default: + ret = UNEXPECTED; + } + if (ret == NO_MATCH && mode == RUN_PRE_INITIAL) + ksdebug(pack, "%s string: \"%.*s\" does not match format\n", + r->howto->type == KSPLICE_HOWTO_DATE ? "date" : "time", + r->howto->size, buf); + + if (ret != OK) + goto out; + ret = create_labelval(pack, r->symbol, run_addr, TEMP); +out: + kfree(buf); + return ret; +} + +static abort_t handle_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, + enum run_pre_mode mode) +{ + struct ksplice_section *sym_sect = symbol_section(pack, r->symbol); + unsigned long offset = r->target_addend; + unsigned long val; + abort_t ret; + + ret = read_reloc_value(pack, r, run_addr, &val); + if (ret != OK) + return ret; + if (r->howto->pcrel) + val += run_addr; + + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "run-pre: reloc at r_a=%lx p_a=%lx to %s+%lx: " + "found %s = %lx\n", run_addr, r->blank_addr, + r->symbol->label, offset, r->symbol->label, val); + + if (contains_canary(pack, run_addr, r->howto) != 0) { + ksdebug(pack, "Aborted. Unexpected canary in run code at %lx" + "\n", run_addr); + return UNEXPECTED; + } + + if ((sect->flags & KSPLICE_SECTION_DATA) != 0 && + sect->symbol == r->symbol) + return OK; + ret = create_labelval(pack, r->symbol, val, TEMP); + if (ret == NO_MATCH && mode == RUN_PRE_INITIAL) + ksdebug(pack, "run-pre: reloc at r_a=%lx p_a=%lx: labelval %s " + "= %lx does not match expected %lx\n", run_addr, + r->blank_addr, r->symbol->label, r->symbol->value, val); + + if (ret != OK) + return ret; + if (sym_sect != NULL && (sym_sect->flags & KSPLICE_SECTION_MATCHED) == 0 + && (sym_sect->flags & KSPLICE_SECTION_STRING) != 0) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "Recursively comparing string section " + "%s\n", sym_sect->symbol->label); + else if (mode == RUN_PRE_DEBUG) + ksdebug(pack, "[str start] "); + ret = run_pre_cmp(pack, sym_sect, val, NULL, mode); + if (mode == RUN_PRE_DEBUG) + ksdebug(pack, "[str end] "); + if (ret == OK && mode == RUN_PRE_INITIAL) + ksdebug(pack, "Successfully matched string section %s" + "\n", sym_sect->symbol->label); + else if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "Failed to match string section %s\n", + sym_sect->symbol->label); + } + return ret; +} + +static int symbol_section_bsearch_compare(const void *a, const void *b) +{ + const struct ksplice_symbol *sym = a; + const struct ksplice_section *sect = b; + return strcmp(sym->label, sect->symbol->label); +} + +static int compare_section_labels(const void *va, const void *vb) +{ + const struct ksplice_section *a = va, *b = vb; + return strcmp(a->symbol->label, b->symbol->label); +} + +static struct ksplice_section *symbol_section(struct ksplice_pack *pack, + const struct ksplice_symbol *sym) +{ + return bsearch(sym, pack->helper_sections, pack->helper_sections_end - + pack->helper_sections, sizeof(struct ksplice_section), + symbol_section_bsearch_compare); +} + +static const struct ksplice_reloc *patch_reloc(struct ksplice_pack *pack, + const struct ksplice_patch *p) +{ + unsigned long addr = (unsigned long)&p->oldaddr; + const struct ksplice_reloc *r = + find_reloc(pack->primary_relocs, pack->primary_relocs_end, addr, + sizeof(addr)); + if (r == NULL || r->blank_addr < addr || + r->blank_addr >= addr + sizeof(addr)) + return NULL; + return r; +} + +static abort_t lookup_symbol(struct ksplice_pack *pack, + const struct ksplice_symbol *ksym, + struct list_head *vals) +{ + abort_t ret; + + if (ksym->vals == NULL) { + release_vals(vals); + ksdebug(pack, "using detected sym %s=%lx\n", ksym->label, + ksym->value); + return add_candidate_val(pack, vals, ksym->value); + } + + if (strcmp(ksym->label, "cleanup_module") == 0 && pack->target != NULL + && pack->target->exit != NULL) { + ret = add_candidate_val(pack, vals, + (unsigned long)pack->target->exit); + if (ret != OK) + return ret; + } + + if (ksym->name != NULL) { + struct candidate_val *val; + list_for_each_entry(val, ksym->vals, list) { + ret = add_candidate_val(pack, vals, val->val); + if (ret != OK) + return ret; + } + + ret = new_export_lookup(pack, ksym->name, vals); + if (ret != OK) + return ret; + } + + return OK; +} + +static abort_t new_export_lookup(struct ksplice_pack *ipack, const char *name, + struct list_head *vals) +{ + struct ksplice_pack *pack; + struct ksplice_patch *p; + list_for_each_entry(pack, &ipack->update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) { + const struct kernel_symbol *sym; + const struct ksplice_reloc *r; + if (p->type != KSPLICE_PATCH_EXPORT || + strcmp(name, *(const char **)p->contents) != 0) + continue; + + /* Check that the p->oldaddr reloc has been resolved */ + r = patch_reloc(pack, p); + if (r == NULL || + contains_canary(pack, r->blank_addr, r->howto) != 0) + continue; + sym = (const struct kernel_symbol *)r->symbol->value; + + /* Check that the sym->value reloc has been resolved, + if there is a ksplice relocation there */ + r = find_reloc(pack->primary_relocs, + pack->primary_relocs_end, + (unsigned long)&sym->value, + sizeof(&sym->value)); + if (r != NULL && + r->blank_addr == (unsigned long)&sym->value && + contains_canary(pack, r->blank_addr, r->howto) != 0) + continue; + return add_candidate_val(ipack, vals, sym->value); + } + } + return OK; +} + +static abort_t apply_patches(struct update *update) +{ + int i; + abort_t ret; + struct ksplice_pack *pack; + + ret = map_trampoline_pages(update); + if (ret != OK) + return ret; + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->pre_apply; f < pack->pre_apply_end; f++) { + if ((*f)() != 0) { + ret = CALL_FAILED; + goto out; + } + } + } + + for (i = 0; i < 5; i++) { + cleanup_conflicts(update); + ret = (__force abort_t)stop_machine(__apply_patches, update, + NULL); + if (ret != CODE_BUSY) + break; + set_current_state(TASK_INTERRUPTIBLE); + schedule_timeout(msecs_to_jiffies(1000)); + } +out: + unmap_trampoline_pages(update); + + if (ret == CODE_BUSY) { + print_conflicts(update); + _ksdebug(update, "Aborted %s. stack check: to-be-replaced " + "code is busy.\n", update->kid); + } else if (ret == ALREADY_REVERSED) { + _ksdebug(update, "Aborted %s. Ksplice update %s is already " + "reversed.\n", update->kid, update->kid); + } + + if (ret != OK) { + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->fail_apply; f < pack->fail_apply_end; + f++) + (*f)(); + } + + return ret; + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->post_apply; f < pack->post_apply_end; f++) + (*f)(); + } + + _ksdebug(update, "Atomic patch insertion for %s complete\n", + update->kid); + return OK; +} + +static abort_t reverse_patches(struct update *update) +{ + int i; + abort_t ret; + struct ksplice_pack *pack; + + clear_debug_buf(update); + ret = init_debug_buf(update); + if (ret != OK) + return ret; + + _ksdebug(update, "Preparing to reverse %s\n", update->kid); + + ret = map_trampoline_pages(update); + if (ret != OK) + return ret; + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->pre_reverse; f < pack->pre_reverse_end; f++) { + if ((*f)() != 0) { + ret = CALL_FAILED; + goto out; + } + } + } + + for (i = 0; i < 5; i++) { + cleanup_conflicts(update); + clear_list(&update->conflicts, struct conflict, list); + ret = (__force abort_t)stop_machine(__reverse_patches, update, + NULL); + if (ret != CODE_BUSY) + break; + set_current_state(TASK_INTERRUPTIBLE); + schedule_timeout(msecs_to_jiffies(1000)); + } +out: + unmap_trampoline_pages(update); + + if (ret == CODE_BUSY) { + print_conflicts(update); + _ksdebug(update, "Aborted %s. stack check: to-be-reversed " + "code is busy.\n", update->kid); + } else if (ret == MODULE_BUSY) { + _ksdebug(update, "Update %s is in use by another module\n", + update->kid); + } + + if (ret != OK) { + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->fail_reverse; f < pack->fail_reverse_end; + f++) + (*f)(); + } + + return ret; + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->post_reverse; f < pack->post_reverse_end; f++) + (*f)(); + } + + list_for_each_entry(pack, &update->packs, list) + clear_list(&pack->safety_records, struct safety_record, list); + + _ksdebug(update, "Atomic patch removal for %s complete\n", update->kid); + return OK; +} + +static int __apply_patches(void *updateptr) +{ + struct update *update = updateptr; + struct ksplice_pack *pack; + struct ksplice_patch *p; + abort_t ret; + + if (update->stage == STAGE_APPLIED) + return (__force int)OK; + + if (update->stage != STAGE_PREPARING) + return (__force int)UNEXPECTED; + + ret = check_each_task(update); + if (ret != OK) + return (__force int)ret; + + list_for_each_entry(pack, &update->packs, list) { + if (try_module_get(pack->primary) != 1) { + struct ksplice_pack *pack1; + list_for_each_entry(pack1, &update->packs, list) { + if (pack1 == pack) + break; + module_put(pack1->primary); + } + module_put(THIS_MODULE); + return (__force int)UNEXPECTED; + } + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->check_apply; f < pack->check_apply_end; f++) + if ((*f)() != 0) + return (__force int)CALL_FAILED; + } + + update->stage = STAGE_APPLIED; + add_taint(TAINT_KSPLICE); + + list_for_each_entry(pack, &update->packs, list) + list_add(&pack->module_list_entry.list, &ksplice_module_list); + + list_for_each_entry(pack, &update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) + insert_trampoline(p); + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->apply; f < pack->apply_end; f++) + (*f)(); + } + + return (__force int)OK; +} + +static int __reverse_patches(void *updateptr) +{ + struct update *update = updateptr; + struct ksplice_pack *pack; + const struct ksplice_patch *p; + abort_t ret; + + if (update->stage != STAGE_APPLIED) + return (__force int)OK; + + list_for_each_entry(pack, &update->packs, list) { + if (module_refcount(pack->primary) != 1) + return (__force int)MODULE_BUSY; + } + + ret = check_each_task(update); + if (ret != OK) + return (__force int)ret; + + list_for_each_entry(pack, &update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) { + ret = verify_trampoline(pack, p); + if (ret != OK) + return (__force int)ret; + } + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->check_reverse; f < pack->check_reverse_end; f++) + if ((*f)() != 0) + return (__force int)CALL_FAILED; + } + + update->stage = STAGE_REVERSED; + + list_for_each_entry(pack, &update->packs, list) + module_put(pack->primary); + + list_for_each_entry(pack, &update->packs, list) + list_del(&pack->module_list_entry.list); + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->reverse; f < pack->reverse_end; f++) + (*f)(); + } + + list_for_each_entry(pack, &update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) + remove_trampoline(p); + } + + return (__force int)OK; +} + +static abort_t check_each_task(struct update *update) +{ + const struct task_struct *g, *p; + abort_t status = OK, ret; + do_each_thread(g, p) { + /* do_each_thread is a double loop! */ + ret = check_task(update, p, false); + if (ret != OK) { + check_task(update, p, true); + status = ret; + } + if (ret != OK && ret != CODE_BUSY) + return ret; + } while_each_thread(g, p); + return status; +} + +static abort_t check_task(struct update *update, + const struct task_struct *t, bool rerun) +{ + abort_t status, ret; + struct conflict *conf = NULL; + + if (rerun) { + conf = kmalloc(sizeof(*conf), GFP_ATOMIC); + if (conf == NULL) + return OUT_OF_MEMORY; + conf->process_name = kstrdup(t->comm, GFP_ATOMIC); + if (conf->process_name == NULL) { + kfree(conf); + return OUT_OF_MEMORY; + } + conf->pid = t->pid; + INIT_LIST_HEAD(&conf->stack); + list_add(&conf->list, &update->conflicts); + } + + status = check_address(update, conf, KSPLICE_IP(t)); + if (t == current) { + ret = check_stack(update, conf, task_thread_info(t), + (unsigned long *)__builtin_frame_address(0)); + if (status == OK) + status = ret; + } else if (!task_curr(t)) { + ret = check_stack(update, conf, task_thread_info(t), + (unsigned long *)KSPLICE_SP(t)); + if (status == OK) + status = ret; + } else if (!is_stop_machine(t)) { + status = UNEXPECTED_RUNNING_TASK; + } + return status; +} + +static abort_t check_stack(struct update *update, struct conflict *conf, + const struct thread_info *tinfo, + const unsigned long *stack) +{ + abort_t status = OK, ret; + unsigned long addr; + + while (valid_stack_ptr(tinfo, stack)) { + addr = *stack++; + ret = check_address(update, conf, addr); + if (ret != OK) + status = ret; + } + return status; +} + +static abort_t check_address(struct update *update, + struct conflict *conf, unsigned long addr) +{ + abort_t status = OK, ret; + const struct safety_record *rec; + struct ksplice_pack *pack; + struct conflict_addr *ca = NULL; + + if (conf != NULL) { + ca = kmalloc(sizeof(*ca), GFP_ATOMIC); + if (ca == NULL) + return OUT_OF_MEMORY; + ca->addr = addr; + ca->has_conflict = false; + ca->label = NULL; + list_add(&ca->list, &conf->stack); + } + + list_for_each_entry(pack, &update->packs, list) { + unsigned long tramp_addr = follow_trampolines(pack, addr); + list_for_each_entry(rec, &pack->safety_records, list) { + ret = check_record(ca, rec, tramp_addr); + if (ret != OK) + status = ret; + } + } + return status; +} + +static abort_t check_record(struct conflict_addr *ca, + const struct safety_record *rec, unsigned long addr) +{ + if (addr >= rec->addr && addr < rec->addr + rec->size) { + if (ca != NULL) { + ca->label = rec->label; + ca->has_conflict = true; + } + return CODE_BUSY; + } + return OK; +} + +static bool is_stop_machine(const struct task_struct *t) +{ + const char *kstop_prefix = "kstop/"; + const char *num; + if (!starts_with(t->comm, kstop_prefix)) + return false; + num = t->comm + strlen(kstop_prefix); + return num[strspn(num, "0123456789")] == '\0'; +} + +static void cleanup_conflicts(struct update *update) +{ + struct conflict *conf; + list_for_each_entry(conf, &update->conflicts, list) { + clear_list(&conf->stack, struct conflict_addr, list); + kfree(conf->process_name); + } + clear_list(&update->conflicts, struct conflict, list); +} + +static void print_conflicts(struct update *update) +{ + const struct conflict *conf; + const struct conflict_addr *ca; + list_for_each_entry(conf, &update->conflicts, list) { + _ksdebug(update, "stack check: pid %d (%s):", conf->pid, + conf->process_name); + list_for_each_entry(ca, &conf->stack, list) { + _ksdebug(update, " %lx", ca->addr); + if (ca->has_conflict) + _ksdebug(update, " [<-CONFLICT]"); + } + _ksdebug(update, "\n"); + } +} + +static void insert_trampoline(struct ksplice_patch *p) +{ + mm_segment_t old_fs = get_fs(); + set_fs(KERNEL_DS); + memcpy(p->saved, p->vaddr, p->size); + memcpy(p->vaddr, p->contents, p->size); + flush_icache_range(p->oldaddr, p->oldaddr + p->size); + set_fs(old_fs); +} + +static abort_t verify_trampoline(struct ksplice_pack *pack, + const struct ksplice_patch *p) +{ + if (memcmp(p->vaddr, p->contents, p->size) != 0) { + ksdebug(pack, "Aborted. Trampoline at %lx has been " + "overwritten.\n", p->oldaddr); + return CODE_BUSY; + } + return OK; +} + +static void remove_trampoline(const struct ksplice_patch *p) +{ + mm_segment_t old_fs = get_fs(); + set_fs(KERNEL_DS); + memcpy(p->vaddr, p->saved, p->size); + flush_icache_range(p->oldaddr, p->oldaddr + p->size); + set_fs(old_fs); +} + +static abort_t create_labelval(struct ksplice_pack *pack, + struct ksplice_symbol *ksym, + unsigned long val, int status) +{ + val = follow_trampolines(pack, val); + if (ksym->vals == NULL) + return ksym->value == val ? OK : NO_MATCH; + + ksym->value = val; + if (status == TEMP) { + struct labelval *lv = kmalloc(sizeof(*lv), GFP_KERNEL); + if (lv == NULL) + return OUT_OF_MEMORY; + lv->symbol = ksym; + lv->saved_vals = ksym->vals; + list_add(&lv->list, &pack->temp_labelvals); + } + ksym->vals = NULL; + return OK; +} + +static abort_t create_safety_record(struct ksplice_pack *pack, + const struct ksplice_section *sect, + struct list_head *record_list, + unsigned long run_addr, + unsigned long run_size) +{ + struct safety_record *rec; + struct ksplice_patch *p; + + if (record_list == NULL) + return OK; + + for (p = pack->patches; p < pack->patches_end; p++) { + const struct ksplice_reloc *r = patch_reloc(pack, p); + if (strcmp(sect->symbol->label, r->symbol->label) == 0) + break; + } + if (p >= pack->patches_end) + return OK; + + rec = kmalloc(sizeof(*rec), GFP_KERNEL); + if (rec == NULL) + return OUT_OF_MEMORY; + rec->label = kstrdup(sect->symbol->label, GFP_KERNEL); + if (rec->label == NULL) { + kfree(rec); + return OUT_OF_MEMORY; + } + rec->addr = run_addr; + rec->size = run_size; + + list_add(&rec->list, record_list); + return OK; +} + +static abort_t add_candidate_val(struct ksplice_pack *pack, + struct list_head *vals, unsigned long val) +{ + struct candidate_val *tmp, *new; + val = follow_trampolines(pack, val); + + list_for_each_entry(tmp, vals, list) { + if (tmp->val == val) + return OK; + } + new = kmalloc(sizeof(*new), GFP_KERNEL); + if (new == NULL) + return OUT_OF_MEMORY; + new->val = val; + list_add(&new->list, vals); + return OK; +} + +static void release_vals(struct list_head *vals) +{ + clear_list(vals, struct candidate_val, list); +} + +static void set_temp_labelvals(struct ksplice_pack *pack, int status) +{ + struct labelval *lv, *n; + list_for_each_entry_safe(lv, n, &pack->temp_labelvals, list) { + if (status == NOVAL) { + lv->symbol->vals = lv->saved_vals; + } else { + release_vals(lv->saved_vals); + kfree(lv->saved_vals); + } + list_del(&lv->list); + kfree(lv); + } +} + +static int contains_canary(struct ksplice_pack *pack, unsigned long blank_addr, + const struct ksplice_reloc_howto *howto) +{ + switch (howto->size) { + case 1: + return (*(uint8_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); + case 2: + return (*(uint16_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); + case 4: + return (*(uint32_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); +#if BITS_PER_LONG >= 64 + case 8: + return (*(uint64_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return -1; + } +} + +static unsigned long follow_trampolines(struct ksplice_pack *pack, + unsigned long addr) +{ + unsigned long new_addr; + struct module *m; + + while (1) { + if (!__kernel_text_address(addr) || + trampoline_target(pack, addr, &new_addr) != OK) + return addr; + m = __module_text_address(new_addr); + if (m == NULL || m == pack->target || + !starts_with(m->name, "ksplice")) + return addr; + addr = new_addr; + } +} + +/* Does module a patch module b? */ +static bool patches_module(const struct module *a, const struct module *b) +{ + struct ksplice_module_list_entry *entry; + if (a == b) + return true; + list_for_each_entry(entry, &ksplice_module_list, list) { + if (entry->target == b && entry->primary == a) + return true; + } + return false; +} + +static bool starts_with(const char *str, const char *prefix) +{ + return strncmp(str, prefix, strlen(prefix)) == 0; +} + +static bool singular(struct list_head *list) +{ + return !list_empty(list) && list->next->next == list; +} + +static void *bsearch(const void *key, const void *base, size_t n, + size_t size, int (*cmp)(const void *key, const void *elt)) +{ + int start = 0, end = n - 1, mid, result; + if (n == 0) + return NULL; + while (start <= end) { + mid = (start + end) / 2; + result = cmp(key, base + mid * size); + if (result < 0) + end = mid - 1; + else if (result > 0) + start = mid + 1; + else + return (void *)base + mid * size; + } + return NULL; +} + +static int compare_relocs(const void *a, const void *b) +{ + const struct ksplice_reloc *ra = a, *rb = b; + if (ra->blank_addr > rb->blank_addr) + return 1; + else if (ra->blank_addr < rb->blank_addr) + return -1; + else + return ra->howto->size - rb->howto->size; +} + +#ifdef CONFIG_DEBUG_FS + +static abort_t init_debug_buf(struct update *update) +{ + update->debug_blob.size = 0; + update->debug_blob.data = NULL; + update->debugfs_dentry = + debugfs_create_blob(update->name, S_IFREG | S_IRUSR, NULL, + &update->debug_blob); + if (update->debugfs_dentry == NULL) + return OUT_OF_MEMORY; + return OK; +} + +static void clear_debug_buf(struct update *update) +{ + if (update->debugfs_dentry == NULL) + return; + debugfs_remove(update->debugfs_dentry); + update->debugfs_dentry = NULL; + update->debug_blob.size = 0; + vfree(update->debug_blob.data); + update->debug_blob.data = NULL; +} + +static int _ksdebug(struct update *update, const char *fmt, ...) +{ + va_list args; + unsigned long size, old_size, new_size; + + if (update->debug == 0) + return 0; + + /* size includes the trailing '\0' */ + va_start(args, fmt); + size = 1 + vsnprintf(update->debug_blob.data, 0, fmt, args); + va_end(args); + old_size = update->debug_blob.size == 0 ? 0 : + max(PAGE_SIZE, roundup_pow_of_two(update->debug_blob.size)); + new_size = update->debug_blob.size + size == 0 ? 0 : + max(PAGE_SIZE, roundup_pow_of_two(update->debug_blob.size + size)); + if (new_size > old_size) { + char *buf = vmalloc(new_size); + if (buf == NULL) + return -ENOMEM; + memcpy(buf, update->debug_blob.data, update->debug_blob.size); + vfree(update->debug_blob.data); + update->debug_blob.data = buf; + } + va_start(args, fmt); + update->debug_blob.size += vsnprintf(update->debug_blob.data + + update->debug_blob.size, + size, fmt, args); + va_end(args); + return 0; +} +#else /* CONFIG_DEBUG_FS */ +static abort_t init_debug_buf(struct update *update) +{ + return OK; +} + +static void clear_debug_buf(struct update *update) +{ + return; +} + +static int _ksdebug(struct update *update, const char *fmt, ...) +{ + va_list args; + + if (update->debug == 0) + return 0; + + if (!update->debug_continue_line) + printk(KERN_DEBUG "ksplice: "); + + va_start(args, fmt); + vprintk(fmt, args); + va_end(args); + + update->debug_continue_line = + fmt[0] == '\0' || fmt[strlen(fmt) - 1] != '\n'; + return 0; +} +#endif /* CONFIG_DEBUG_FS */ + +struct ksplice_attribute { + struct attribute attr; + ssize_t (*show)(struct update *update, char *buf); + ssize_t (*store)(struct update *update, const char *buf, size_t len); +}; + +static ssize_t ksplice_attr_show(struct kobject *kobj, struct attribute *attr, + char *buf) +{ + struct ksplice_attribute *attribute = + container_of(attr, struct ksplice_attribute, attr); + struct update *update = container_of(kobj, struct update, kobj); + if (attribute->show == NULL) + return -EIO; + return attribute->show(update, buf); +} + +static ssize_t ksplice_attr_store(struct kobject *kobj, struct attribute *attr, + const char *buf, size_t len) +{ + struct ksplice_attribute *attribute = + container_of(attr, struct ksplice_attribute, attr); + struct update *update = container_of(kobj, struct update, kobj); + if (attribute->store == NULL) + return -EIO; + return attribute->store(update, buf, len); +} + +static struct sysfs_ops ksplice_sysfs_ops = { + .show = ksplice_attr_show, + .store = ksplice_attr_store, +}; + +static void ksplice_release(struct kobject *kobj) +{ + struct update *update; + update = container_of(kobj, struct update, kobj); + cleanup_ksplice_update(update); +} + +static ssize_t stage_show(struct update *update, char *buf) +{ + switch (update->stage) { + case STAGE_PREPARING: + return snprintf(buf, PAGE_SIZE, "preparing\n"); + case STAGE_APPLIED: + return snprintf(buf, PAGE_SIZE, "applied\n"); + case STAGE_REVERSED: + return snprintf(buf, PAGE_SIZE, "reversed\n"); + } + return 0; +} + +static ssize_t abort_cause_show(struct update *update, char *buf) +{ + switch (update->abort_cause) { + case OK: + return snprintf(buf, PAGE_SIZE, "ok\n"); + case NO_MATCH: + return snprintf(buf, PAGE_SIZE, "no_match\n"); + case CODE_BUSY: + return snprintf(buf, PAGE_SIZE, "code_busy\n"); + case MODULE_BUSY: + return snprintf(buf, PAGE_SIZE, "module_busy\n"); + case OUT_OF_MEMORY: + return snprintf(buf, PAGE_SIZE, "out_of_memory\n"); + case FAILED_TO_FIND: + return snprintf(buf, PAGE_SIZE, "failed_to_find\n"); + case ALREADY_REVERSED: + return snprintf(buf, PAGE_SIZE, "already_reversed\n"); + case MISSING_EXPORT: + return snprintf(buf, PAGE_SIZE, "missing_export\n"); + case UNEXPECTED_RUNNING_TASK: + return snprintf(buf, PAGE_SIZE, "unexpected_running_task\n"); + case TARGET_NOT_LOADED: + return snprintf(buf, PAGE_SIZE, "target_not_loaded\n"); + case CALL_FAILED: + return snprintf(buf, PAGE_SIZE, "call_failed\n"); + case UNEXPECTED: + return snprintf(buf, PAGE_SIZE, "unexpected\n"); + default: + return snprintf(buf, PAGE_SIZE, "unknown\n"); + } + return 0; +} + +static ssize_t conflict_show(struct update *update, char *buf) +{ + const struct conflict *conf; + const struct conflict_addr *ca; + int used = 0; + list_for_each_entry(conf, &update->conflicts, list) { + used += snprintf(buf + used, PAGE_SIZE - used, "%s %d", + conf->process_name, conf->pid); + list_for_each_entry(ca, &conf->stack, list) { + if (!ca->has_conflict) + continue; + used += snprintf(buf + used, PAGE_SIZE - used, " %s", + ca->label); + } + used += snprintf(buf + used, PAGE_SIZE - used, "\n"); + } + return used; +} + +static int maybe_cleanup_ksplice_update_wrapper(void *updateptr) +{ + struct update *update = updateptr; + maybe_cleanup_ksplice_update(update); + return 0; +} + +static ssize_t stage_store(struct update *update, const char *buf, size_t len) +{ + enum stage old_stage = update->stage; + if ((strncmp(buf, "applied", len) == 0 || + strncmp(buf, "applied\n", len) == 0) && + update->stage == STAGE_PREPARING) + update->abort_cause = apply_update(update); + else if ((strncmp(buf, "reversed", len) == 0 || + strncmp(buf, "reversed\n", len) == 0) && + update->stage == STAGE_APPLIED) + update->abort_cause = reverse_patches(update); + else if ((strncmp(buf, "cleanup", len) == 0 || + strncmp(buf, "cleanup\n", len) == 0) && + update->stage == STAGE_REVERSED) + kthread_run(maybe_cleanup_ksplice_update_wrapper, update, + "ksplice_cleanup_%s", update->kid); + + if (old_stage != STAGE_REVERSED && update->abort_cause == OK) + printk(KERN_INFO "ksplice: Update %s %s successfully\n", + update->kid, + update->stage == STAGE_APPLIED ? "applied" : "reversed"); + return len; +} + +static ssize_t debug_show(struct update *update, char *buf) +{ + return snprintf(buf, PAGE_SIZE, "%d\n", update->debug); +} + +static ssize_t debug_store(struct update *update, const char *buf, size_t len) +{ + unsigned long l; + int ret = strict_strtoul(buf, 10, &l); + if (ret != 0) + return ret; + update->debug = l; + return len; +} + +static ssize_t partial_show(struct update *update, char *buf) +{ + return snprintf(buf, PAGE_SIZE, "%d\n", update->partial); +} + +static ssize_t partial_store(struct update *update, const char *buf, size_t len) +{ + unsigned long l; + int ret = strict_strtoul(buf, 10, &l); + if (ret != 0) + return ret; + update->partial = l; + return len; +} + +static struct ksplice_attribute stage_attribute = + __ATTR(stage, 0600, stage_show, stage_store); +static struct ksplice_attribute abort_cause_attribute = + __ATTR(abort_cause, 0400, abort_cause_show, NULL); +static struct ksplice_attribute debug_attribute = + __ATTR(debug, 0600, debug_show, debug_store); +static struct ksplice_attribute partial_attribute = + __ATTR(partial, 0600, partial_show, partial_store); +static struct ksplice_attribute conflict_attribute = + __ATTR(conflicts, 0400, conflict_show, NULL); + +static struct attribute *ksplice_attrs[] = { + &stage_attribute.attr, + &abort_cause_attribute.attr, + &debug_attribute.attr, + &partial_attribute.attr, + &conflict_attribute.attr, + NULL +}; + +static struct kobj_type ksplice_ktype = { + .sysfs_ops = &ksplice_sysfs_ops, + .release = ksplice_release, + .default_attrs = ksplice_attrs, +}; + +static int init_ksplice(void) +{ + ksplice_kobj = kobject_create_and_add("ksplice", kernel_kobj); + if (ksplice_kobj == NULL) + return -ENOMEM; + return 0; +} + +static void cleanup_ksplice(void) +{ + kobject_put(ksplice_kobj); +} + +module_init(init_ksplice); +module_exit(cleanup_ksplice); + +MODULE_AUTHOR("Jeff Arnold <jbarnold@mit.edu>"); +MODULE_DESCRIPTION("Ksplice rebootless update system"); +MODULE_LICENSE("GPL v2"); diff --git a/kernel/panic.c b/kernel/panic.c index 4d50883..f2ca7bf 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -155,6 +155,7 @@ static const struct tnt tnts[] = { { TAINT_OVERRIDDEN_ACPI_TABLE, 'A', ' ' }, { TAINT_WARN, 'W', ' ' }, { TAINT_CRAP, 'C', ' ' }, + { TAINT_KSPLICE, 'K', ' ' }, }; /** -- 1.5.6.3 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 2008-12-06 0:04 ` [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 Jeff Arnold @ 2008-12-17 5:41 ` Theodore Tso 2008-12-18 2:09 ` Tim Abbott 2009-02-07 2:36 ` Rusty Russell 1 sibling, 1 reply; 43+ messages in thread From: Theodore Tso @ 2008-12-17 5:41 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan One thing that would be *really* helpful is some function-level documentation. What various functions do, what locks they assume are held, what locks they take/release, etc. kernel/ksplice.c is *long* some signposts would definitely make it easier for folks to understand what is going on. Regards, - Ted ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 2008-12-17 5:41 ` Theodore Tso @ 2008-12-18 2:09 ` Tim Abbott 0 siblings, 0 replies; 43+ messages in thread From: Tim Abbott @ 2008-12-18 2:09 UTC (permalink / raw) To: Theodore Tso Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Wed, 17 Dec 2008, Theodore Tso wrote: > One thing that would be *really* helpful is some function-level > documentation. Hi Ted, Thanks for the feedback. Attached is a new version of PATCH 7/7 with more function-level documentation (I'm attaching a full patch rather than a diff against the original patch because I think this will be easier to read and comment on). > what locks they assume are held, what locks they take/release, etc. Almost all of the Ksplice code is run while holding module_mutex, to prevent the code in the running kernel from changing while Ksplice is preparing and applying updates. This locking consideration is discussed in Documentation/ksplice.txt. I hope this is helpful. If more documentation or other information would be helpful, please let us know. Thanks, -Tim Abbott --- From: Jeff Arnold <jbarnold@mit.edu> Subject: [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 Date: Wed, 17 Dec 2008 21:00:42 -0400 Ksplice makes it possible to apply patches to the kernel without rebooting. Changelog since Dec. 5 patch: - Added more function-level documentation - Bail our earlier (to give a better error message) when there are multiple writers to an update's stage file. Signed-off-by: Jeff Arnold <jbarnold@mit.edu> Signed-off-by: Anders Kaseorg <andersk@mit.edu> Signed-off-by: Tim Abbott <tabbott@mit.edu> Tested-by: Waseem Daher <wdaher@mit.edu> --- Documentation/ksplice.txt | 282 ++++ MAINTAINERS | 10 + arch/Kconfig | 14 + arch/x86/Kconfig | 1 + arch/x86/kernel/ksplice-arch.c | 125 ++ include/linux/kernel.h | 1 + include/linux/ksplice.h | 201 +++ kernel/Makefile | 3 + kernel/ksplice.c | 2910 ++++++++++++++++++++++++++++++++++++++++ kernel/panic.c | 1 + 10 files changed, 3548 insertions(+), 0 deletions(-) create mode 100644 Documentation/ksplice.txt create mode 100644 arch/x86/kernel/ksplice-arch.c create mode 100644 include/linux/ksplice.h create mode 100644 kernel/ksplice.c diff --git a/Documentation/ksplice.txt b/Documentation/ksplice.txt new file mode 100644 index 0000000..971f03a --- /dev/null +++ b/Documentation/ksplice.txt @@ -0,0 +1,282 @@ +Ksplice +------- + +CONTENTS: + +1. Concepts: updates, packs, helper modules, primary modules +2. What changes can Ksplice handle? +3. Dependency model +4. Locking model +5. altinstructions, smplocks, and parainstructions +6. sysfs interface +7. debugfs interface +8. Hooks for running custom code during the update process + +0. Design Description +--------------------- + +For a description of the Ksplice design, please see the Ksplice technical +overview document: <http://www.ksplice.com/doc/ksplice.pdf>. For usage +examples and the Ksplice man pages, please see <http://www.ksplice.com>. + +The document below assumes familiarity with the Ksplice design and describes +notable implementation details and the interface between the Ksplice kernel +component and the Ksplice user space component. + +1. Concepts: Updates, packs, helper modules, primary modules +------------------------------------------------------------ + +A Ksplice update (struct update) contains one or more Ksplice packs, one for +each target kernel module that should be changed by the update. Ksplice packs +are grouped together into a Ksplice update in order to allow multiple +compilation units to be changed atomically. + +The contents of a Ksplice pack are documented via kernel-doc in +include/linux/ksplice.h. To construct a new Ksplice update to be performed +atomically, one needs to: + 1. Populate the fields of one or more ksplice_pack structures. + 2. Call the Ksplice function init_ksplice_pack() on each pack to register + the packs with the Ksplice kernel component. When init_ksplice_pack() + is called on a pack, that pack will be associated with the other packs + that share the same Ksplice identifier (KID) field. + 3. After all of the packs intended for a particular Ksplice update have + been loaded, that update can be applied via the sysfs interface + (described in Section 7 below). + +In order to save memory, each Ksplice pack has a "helper" module and a "primary" +module associated with it. + +The pack's "helper" module contains materials needed only for preparing for the +update. Specifically, the helper module contains a copy of the pre-patch +version of each of the compilation units changed by the Ksplice pack. The +helper module can be unloaded after the update has been applied. + +The pack's "primary" module contains the new sections to be inserted by the +update; it needs to remain loaded for as long as the update is applied. + +Here's an example: + +Let's say that the Ksplice user space component wants to update the core kernel +and the isdn module. The user space component will select a KID for this update +(let's say 123abc) and generate four modules: + +ksplice_123abc_vmlinux (the "primary" module for the vmlinux pack) +ksplice_123abc_vmlinux_helper (the "helper" module for the vmlinux pack) +ksplice_123abc_isdn (the "primary" module for the vmlinux pack) +ksplice_123abc_isdn_helper (the "helper" module for the vmlinux pack) + +Once both of the vmlinux modules have been loaded, one of the modules calls +init_ksplice_pack on a pack corresponding to the desired vmlinux changes. + +Similarly, once both of the isdn modules have been loaded, one of the modules +calls init_ksplice_pack on a pack corresponding to the desired isdn changes. + +Once all modules are loaded (in this example, four modules), the update can be +applied atomically using the Ksplice sysfs interface. Once the update has been +applied, the helper modules can be unloaded safely to save memory. + +2. What changes can Ksplice handle? +----------------------------------- + +The Ksplice user space component takes a source code patch and uses it to +construct appropriate Ksplice packs for an update. Ksplice can handle source +code patches that add new functions, modify the text or arguments of existing +functions, delete functions, move functions between compilation units, change +functions from local to global (or vice versa), add exported symbols, rename +exported symbols, and delete exported symbols. Ksplice can handle patches that +modify either C code or assembly code. + +As described in the Ksplice technical overview document, a programmer needs to +write some new code in order for Ksplice to apply a patch that makes semantic +changes to kernel data structures. Some other limitations also apply: + +Ksplice does not support changes to __init functions that been unloaded from +kernel memory. Ksplice also does not support changes to functions in +.exit.text sections since Ksplice currently requires that all Ksplice updates +affecting a module be reversed before that module can be unloaded. + +The Ksplice user space implementation does not currently support changes to +weak symbols and changes to global read-only data structures (changes to +read-only data structures that are local to a compilation unit are fine). + +Exported symbols: + +Ksplice can handle arbitrary changes to exported symbols in the source code +patch. + +Ksplice deletes exported symbols by looking up the relevant struct kernel_symbol +in the kernel's exported symbol table and replacing the name field with a +pointer to a string that begins with DISABLED. + +Ksplice adds new exported symbols through the same mechanism; the relevant +primary module will have a ksymtab entry containing a symbol with a name +beginning with DISABLED, and Ksplice will replace that with the name of the +symbol to be exported when the update is atomically applied. + +Because the struct kernel_symbol for a newly exported symbol is contained in the +Ksplice primary module, if a module using one of the newly exported symbols is +loaded, that module will correctly depend on the Ksplice primary module that +exported the symbol. + +3. Dependency model +------------------- + +Because Ksplice resolves symbols used in the post code using Ksplice +relocations, Ksplice must enforce additional dependencies. Ksplice uses the +use_module function to directly add dependencies on all the modules that the +post code references. + +4. Locking model +---------------- + +From a locking perspective, Ksplice treats applying or removing a Ksplice update +as analogous to loading or unloading a new version of the kernel modules patched +by the update. Ksplice uses module_mutex to protect against a variety of race +conditions related to modules being loaded or unloaded while Ksplice is applying +or reversing an update; this approach also protects against race conditions +involving multiple Ksplice updates being loaded or unloaded simultaneously as +well. + +5. altinstructions, smplocks, and parainstructions +-------------------------------------------------- + +There are currently several mechanisms through which the Linux kernel will +modify executable code at runtime. + +These mechanisms sometimes overwrite the storage unit of a relocation, which +would cause problems if not handled properly by Ksplice. + +Ksplice solves this problem by writing "canary" bytes (e.g., 0x77777777) in the +storage unit of the relocation in user space. Ksplice then checks whether the +canary has been overwritten before using a Ksplice relocation to detect symbol +values or to write a value to the storage unit of a Ksplice relocation. + +6. sysfs interface +------------------ + +Ksplice exports four sysfs files per Ksplice update in order to communicate with +user space. For each update, these four files are located in a directory of the +form /sys/kernel/ksplice/$kid, with $kid replaced by the KID of the Ksplice +update. + +A. /sys/kernel/ksplice/$kid/stage (mode 0600) + +This file contains one of three strings: +preparing: Indicates that this update has not yet been applied +applied: Indicates that this update has been applied and has not been reversed +reversed: Indicates that this update has been reversed + +When the stage is "preparing", the superuser can write "applied" to the stage +file in order to instruct Ksplice to apply the update. When the stage is +"applied", the superuser can write "reversed" to the stage file in order to +instruct Ksplice to reverse the update. Afterwards, the superuser can +write "cleanup" to the stage file in order to instruct Ksplice to +clean up the debugging information and sysfs directory associated with +the reversed update. Once an update is reversed, it cannot be re-applied without first +cleaning up the update. + +B. /sys/kernel/ksplice/$kid/debug (mode 0600) + +The file contains a single number: 1 if debugging is enabled for this Ksplice +update and 0 otherwise. + +The superuser can write a new value to this file to enable or disable debugging. + +C. /sys/kernel/ksplice/$kid/partial (mode 0600) + +The file contains a single number: 1 if the update should be applied even if +some of the target modules are not loaded and 0 otherwise. + +D. /sys/kernel/ksplice/$kid/abort_cause (mode 0400) + +This file contains a value indicating either 1) that Ksplice successfully +completed the most recently requested stage transition or 2) why Ksplice aborted +the most recently requested stage transition. + +Each abort_code string is described below, along with the stage transitions that +might potentially trigger each possible abort code. The stage transitions are +abbreviated as follows: preparing->applied (P->A), applied->reversed (A->R). + +ok (P->A, A->R): The most recent stage transition succeeded. + +no_match (P->A): Ksplice aborted the update because Ksplice was unable to match +the helper module's object code against the running kernel's object code. + +failed_to_find (P->A): Ksplice aborted the update because Ksplice was unable to +resolve some of the symbols used in the update. + +missing_export (P->A): Ksplice aborted the update because the symbols exported +by the kernel did not match Ksplice's expectations. + +already_reversed (P->A): Ksplice aborted the update because once an update has +been reversed, it cannot be applied again (without first being cleaned up and +reinitialized). + +module_busy (A->R): Ksplice aborted the undo operation because the target +Ksplice update is in use by another kernel module; specifically, either the +target Ksplice update exports a symbol that is in use by another module or +another Ksplice update depends on this Ksplice update. + +out_of_memory (P->A, A->R): Ksplice aborted the operation because a call to +kmalloc or vmalloc failed. + +code_busy (P->A, A->R): Ksplice aborted the operation because Ksplice was +unable to find a moment when one or more of the to-be-patched functions was not +a thread's kernel stack. + +target_not_loaded (P->A): Ksplice aborted the update because one of the target +modules is not loaded and the partial option (/sys/kernel/ksplice/$kid/partial) +is not enabled. + +call_failed (P->A, A->R): One of the calls included as part of this update +returned nonzero exit status. + +unexpected_running_task (P->A, A->R): Ksplice aborted the operation because +Ksplice observed a running task during the kernel stack check, at a time when +Ksplice expected all tasks to be stopped by stop_machine. + +unexpected (P->A, A->R): Ksplice aborted the operation because it encountered +an unspecified internal error. This condition can only be caused by an invalid +input to Ksplice or a bug in Ksplice. + +E. /sys/kernel/ksplice/$kid/conflicts (mode 0400) + +This file is empty until Ksplice aborts an operation because of a code_busy +condition (see "abort_code" above). This conflicts file then contains +information about the process(es) that caused the stack check failure. + +Specifically, each line of this file consists of three space-separated values, +describing a single conflict: + +$program_name $program_pid $conflict_label + +$program_name is the name of the program with the conflict. +$program_pid is the pid of the program with the conflict. +$conflict_label is the Ksplice label of the function with the conflict. + +7. debugfs interface +-------------------- + +Ksplice exports a single file to debugfs for each Ksplice update. The file has +a name of the form ksplice_KID, where KID is the unique identifier of the +Ksplice update. It contains debugging information in a human-readable format. + +8. Hooks for running custom code during the update process +---------------------------------------------------------- + +Ksplice allows a programmer to write custom code to be called from within the +kernel during the update process. The kernel component allows custom code to +be executed at the following times: + +pre_apply: Called before the update has been applied and before the machine has +been stopped. Allowed to fail. +check_apply: Called before the update has been applied but after the machine +has been stopped. Allowed to fail. +apply: Called when the update is definitely going to be applied and when the +machine is stopped. Not allowed to fail. +post_apply: Called when the update has been applied and the machine is no +longer stopped. Not allowed to fail. +fail_apply: Called when the update failed to apply and the machine is no longer +stopped. Not allowed to fail. + +Ksplice also provides six analagous xxx_reverse hooks. diff --git a/MAINTAINERS b/MAINTAINERS index 618c1ef..40227d4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2585,6 +2585,16 @@ W: http://miguelojeda.es/auxdisplay.htm W: http://jair.lab.fi.uva.es/~migojed/auxdisplay.htm S: Maintained +KSPLICE: +P: Jeff Arnold +M: jbarnold@mit.edu +P: Anders Kaseorg +M: andersk@mit.edu +P: Tim Abbott +M: tabbott@mit.edu +W: http://www.ksplice.com +S: Maintained + LAPB module L: linux-x25@vger.kernel.org S: Orphan diff --git a/arch/Kconfig b/arch/Kconfig index 471e72d..de4b6c8 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -41,6 +41,17 @@ config KPROBES for kernel debugging, non-intrusive instrumentation and testing. If in doubt, say "N". +config KSPLICE + tristate "Ksplice rebootless kernel updates" + depends on KALLSYMS_ALL && MODULE_UNLOAD && SYSFS && \ + FUNCTION_DATA_SECTIONS + depends on HAVE_KSPLICE + help + Say Y here if you want to be able to apply certain kinds of + patches to your running kernel, without rebooting. + + If unsure, say N. + config HAVE_EFFICIENT_UNALIGNED_ACCESS bool help @@ -70,6 +81,9 @@ config HAVE_IOREMAP_PROT config HAVE_KPROBES bool +config HAVE_KSPLICE + def_bool n + config HAVE_KRETPROBES bool diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index ac22bb7..0e2c276 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -24,6 +24,7 @@ config X86 select HAVE_OPROFILE select HAVE_IOREMAP_PROT select HAVE_KPROBES + select HAVE_KSPLICE select ARCH_WANT_OPTIONAL_GPIOLIB select HAVE_KRETPROBES select HAVE_FTRACE_MCOUNT_RECORD diff --git a/arch/x86/kernel/ksplice-arch.c b/arch/x86/kernel/ksplice-arch.c new file mode 100644 index 0000000..b04ae17 --- /dev/null +++ b/arch/x86/kernel/ksplice-arch.c @@ -0,0 +1,125 @@ +/* Copyright (C) 2007-2008 Jeff Arnold <jbarnold@mit.edu> + * Copyright (C) 2008 Anders Kaseorg <andersk@mit.edu>, + * Tim Abbott <tabbott@mit.edu> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA + * 02110-1301, USA. + */ + +#define KSPLICE_IP(x) ((x)->thread.ip) +#define KSPLICE_SP(x) ((x)->thread.sp) + +static struct ksplice_symbol trampoline_symbol = { + .name = NULL, + .label = "<trampoline>", +}; + +static const struct ksplice_reloc_howto trampoline_howto = { + .type = KSPLICE_HOWTO_RELOC, + .pcrel = 1, + .size = 4, + .dst_mask = 0xffffffffL, + .rightshift = 0, + .signed_addend = 1, +}; + +static const struct ksplice_reloc trampoline_reloc = { + .symbol = &trampoline_symbol, + .insn_addend = -4, + .target_addend = 0, + .howto = &trampoline_howto, +}; + +static abort_t trampoline_target(struct ksplice_pack *pack, unsigned long addr, + unsigned long *new_addr) +{ + abort_t ret; + unsigned char byte; + + if (probe_kernel_read(&byte, (void *)addr, sizeof(byte)) == -EFAULT) + return NO_MATCH; + + if (byte != 0xe9) + return NO_MATCH; + + ret = read_reloc_value(pack, &trampoline_reloc, addr + 1, new_addr); + if (ret != OK) + return ret; + + *new_addr += addr + 1; + return OK; +} + +static abort_t prepare_trampoline(struct ksplice_pack *pack, + struct ksplice_patch *p) +{ + p->size = 5; + ((unsigned char *)p->contents)[0] = 0xe9; + return write_reloc_value(pack, &trampoline_reloc, + (unsigned long)p->contents + 1, + p->repladdr - (p->oldaddr + 1)); +} + +static abort_t handle_bug(struct ksplice_pack *pack, + const struct ksplice_reloc *r, unsigned long run_addr) +{ + const struct bug_entry *run_bug = find_bug(run_addr); + struct ksplice_section *bug_sect = symbol_section(pack, r->symbol); + if (run_bug == NULL) + return NO_MATCH; + if (bug_sect == NULL) + return UNEXPECTED; + return create_labelval(pack, bug_sect->symbol, (unsigned long)run_bug, + TEMP); +} + +static abort_t handle_extable(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long run_addr) +{ + const struct exception_table_entry *run_ent = + search_exception_tables(run_addr); + struct ksplice_section *ex_sect = symbol_section(pack, r->symbol); + if (run_ent == NULL) + return NO_MATCH; + if (ex_sect == NULL) + return UNEXPECTED; + return create_labelval(pack, ex_sect->symbol, (unsigned long)run_ent, + TEMP); +} + +static abort_t handle_paravirt(struct ksplice_pack *pack, + unsigned long pre_addr, unsigned long run_addr, + int *matched) +{ + unsigned char run[5], pre[5]; + *matched = 0; + + if (probe_kernel_read(&run, (void *)run_addr, sizeof(run)) == -EFAULT || + probe_kernel_read(&pre, (void *)pre_addr, sizeof(pre)) == -EFAULT) + return OK; + + if ((run[0] == 0xe8 && pre[0] == 0xe8) || + (run[0] == 0xe9 && pre[0] == 0xe9)) + if (run_addr + 1 + *(int32_t *)&run[1] == + pre_addr + 1 + *(int32_t *)&pre[1]) + *matched = 5; + return OK; +} + +static bool valid_stack_ptr(const struct thread_info *tinfo, const void *p) +{ + return p > (const void *)tinfo + && p <= (const void *)tinfo + THREAD_SIZE - sizeof(long); +} + diff --git a/include/linux/kernel.h b/include/linux/kernel.h index dc7e0d0..15643a5 100644 --- a/include/linux/kernel.h +++ b/include/linux/kernel.h @@ -290,6 +290,7 @@ extern enum system_states { #define TAINT_OVERRIDDEN_ACPI_TABLE 8 #define TAINT_WARN 9 #define TAINT_CRAP 10 +#define TAINT_KSPLICE 11 extern void dump_stack(void) __cold; diff --git a/include/linux/ksplice.h b/include/linux/ksplice.h new file mode 100644 index 0000000..4925971 --- /dev/null +++ b/include/linux/ksplice.h @@ -0,0 +1,201 @@ +#include <linux/types.h> + +/** + * struct ksplice_symbol - Ksplice's analogue of an ELF symbol + * @name: The ELF name of the symbol + * @label: A unique Ksplice name for the symbol + * @vals: A linked list of possible values for the symbol, or NULL + * @value: The value of the symbol (valid when vals is NULL) + **/ +struct ksplice_symbol { + const char *name; + const char *label; +/* private: */ + struct list_head *vals; + unsigned long value; +}; + +/** + * struct ksplice_reloc - Ksplice's analogue of an ELF relocation + * @blank_addr: The address of the relocation's storage unit + * @symbol: The ksplice_symbol associated with this relocation + * @howto: The information regarding the relocation type + * @addend: The ELF addend of the relocation + **/ +struct ksplice_reloc { + unsigned long blank_addr; + struct ksplice_symbol *symbol; + const struct ksplice_reloc_howto *howto; + long insn_addend; + long target_addend; +}; + +enum ksplice_reloc_howto_type { + KSPLICE_HOWTO_RELOC, + KSPLICE_HOWTO_RELOC_PATCH, + KSPLICE_HOWTO_DATE, + KSPLICE_HOWTO_TIME, + KSPLICE_HOWTO_BUG, + KSPLICE_HOWTO_EXTABLE, +}; + +/** + * struct ksplice_reloc_howto - Ksplice's relocation type information + * @type: The type of the relocation + * @pcrel: Is the relocation PC relative? + * @size: The size, in bytes, of the item to be relocated + * @dst_mask: Bitmask for which parts of the instruction or data are + * replaced with the relocated value + * (based on dst_mask from GNU BFD's reloc_howto_struct) + * @rightshift: The value the final relocation is shifted right by; + * used to drop unwanted data from the relocation + * (based on rightshift from GNU BFD's reloc_howto_struct) + * @signed_addend: Should the addend be interpreted as a signed value? + **/ +struct ksplice_reloc_howto { + enum ksplice_reloc_howto_type type; + int pcrel; + int size; + long dst_mask; + unsigned int rightshift; + int signed_addend; +}; + +#if BITS_PER_LONG == 32 +#define KSPLICE_CANARY 0x77777777UL +#elif BITS_PER_LONG == 64 +#define KSPLICE_CANARY 0x7777777777777777UL +#endif /* BITS_PER_LONG */ + +/** + * struct ksplice_section - Ksplice's analogue of an ELF section + * @symbol: The ksplice_symbol associated with this section + * @size: The length, in bytes, of this section + * @address: The address of the section + * @flags: Flags indicating the type of the section, whether or + * not it has been matched, etc. + **/ +struct ksplice_section { + struct ksplice_symbol *symbol; + unsigned long address; + unsigned long size; + unsigned int flags; + const unsigned char **match_map; +}; +#define KSPLICE_SECTION_TEXT 0x00000001 +#define KSPLICE_SECTION_RODATA 0x00000002 +#define KSPLICE_SECTION_DATA 0x00000004 +#define KSPLICE_SECTION_STRING 0x00000008 +#define KSPLICE_SECTION_MATCHED 0x10000000 + +#define MAX_TRAMPOLINE_SIZE 5 + +enum ksplice_patch_type { + KSPLICE_PATCH_TEXT, + KSPLICE_PATCH_BUGLINE, + KSPLICE_PATCH_DATA, + KSPLICE_PATCH_EXPORT, +}; + +/** + * struct ksplice_patch - A replacement that Ksplice should perform + * @oldaddr: The address of the obsolete function or structure + * @repladdr: The address of the replacement function + * @type: The type of the ksplice patch + * @size: The size of the patch + * @contents: The bytes to be installed at oldaddr + * @vaddr The address of the page mapping used to write at oldaddr + * @saved: The bytes originally at oldaddr which were + * overwritten by the patch + **/ +struct ksplice_patch { + unsigned long oldaddr; + unsigned long repladdr; + enum ksplice_patch_type type; + unsigned int size; + void *contents; +/* private: */ + void *vaddr; + void *saved; +}; + +#ifdef __KERNEL__ +#include <linux/module.h> +#include <linux/stringify.h> + +#define _PASTE(x, y) x##y +#define PASTE(x, y) _PASTE(x, y) +#define KSPLICE_UNIQ(s) PASTE(s##_, KSPLICE_MID) +#define KSPLICE_KID_UNIQ(s) PASTE(s##_, KSPLICE_KID) + +/** + * struct ksplice_module_list_entry - A record of a Ksplice pack's target + * @target: A module that is patched + * @primary: A Ksplice module that patches target + **/ +struct ksplice_module_list_entry { + struct module *target; + struct module *primary; +/* private: */ + struct list_head list; +}; + +/* List of all ksplice modules and the module they patch */ +extern struct list_head ksplice_module_list; + +/** + * struct ksplice_pack - Data for one module modified by a Ksplice update + * @name: The name of the primary module for the pack + * @kid: The Ksplice unique identifier for the pack + * @target_name: The name of the module modified by the pack + * @primary: The primary module associated with the pack + * @primary_relocs: The relocations for the primary module + * @primary_relocs_end: The end pointer for primary_relocs + * @primary_sections: The sections in the primary module + * @primary_sections_end: The end pointer for primary_sections array + * @helper_relocs: The relocations for the helper module + * @helper_relocs_end: The end pointer for helper_relocs array + * @helper_sections: The sections in the helper module + * @helper_sections_end: The end pointer for helper_sections array + * @patches: The function replacements in the pack + * @patches_end: The end pointer for patches array + * @update: The atomic update the pack is part of + * @target: The module modified by the pack + * @safety_records: The ranges of addresses that must not be on a + * kernel stack for the patch to apply safely + **/ +struct ksplice_pack { + const char *name; + const char *kid; + const char *target_name; + struct module *primary; + struct ksplice_reloc *primary_relocs, *primary_relocs_end; + const struct ksplice_section *primary_sections, *primary_sections_end; + struct ksplice_symbol *primary_symbols, *primary_symbols_end; + struct ksplice_reloc *helper_relocs, *helper_relocs_end; + struct ksplice_section *helper_sections, *helper_sections_end; + struct ksplice_symbol *helper_symbols, *helper_symbols_end; + struct ksplice_patch *patches, *patches_end; + const typeof(int (*)(void)) *pre_apply, *pre_apply_end, *check_apply, + *check_apply_end; + const typeof(void (*)(void)) *apply, *apply_end, *post_apply, + *post_apply_end, *fail_apply, *fail_apply_end; + const typeof(int (*)(void)) *pre_reverse, *pre_reverse_end, + *check_reverse, *check_reverse_end; + const typeof(void (*)(void)) *reverse, *reverse_end, *post_reverse, + *post_reverse_end, *fail_reverse, *fail_reverse_end; +/* private: */ + struct ksplice_module_list_entry module_list_entry; + struct update *update; + struct module *target; + struct list_head temp_labelvals; + struct list_head safety_records; + struct list_head list; +}; + + +int init_ksplice_pack(struct ksplice_pack *pack); + +void cleanup_ksplice_pack(struct ksplice_pack *pack); + +#endif /* __KERNEL__ */ diff --git a/kernel/Makefile b/kernel/Makefile index 19fad00..e8ecb64 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -11,6 +11,8 @@ obj-y = sched.o fork.o exec_domain.o panic.o printk.o \ hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \ notifier.o ksysfs.o pm_qos_params.o sched_clock.o +CFLAGS_ksplice.o += -Iarch/$(SRCARCH)/kernel + ifdef CONFIG_FUNCTION_TRACER # Do not trace debug files and internal ftrace files CFLAGS_REMOVE_lockdep.o = -pg @@ -68,6 +70,7 @@ obj-$(CONFIG_AUDIT) += audit.o auditfilter.o obj-$(CONFIG_AUDITSYSCALL) += auditsc.o obj-$(CONFIG_AUDIT_TREE) += audit_tree.o obj-$(CONFIG_KPROBES) += kprobes.o +obj-$(CONFIG_KSPLICE) += ksplice.o obj-$(CONFIG_KGDB) += kgdb.o obj-$(CONFIG_DETECT_SOFTLOCKUP) += softlockup.o obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ diff --git a/kernel/ksplice.c b/kernel/ksplice.c new file mode 100644 index 0000000..a3f21a6 --- /dev/null +++ b/kernel/ksplice.c @@ -0,0 +1,2910 @@ +/* Copyright (C) 2007-2008 Jeff Arnold <jbarnold@mit.edu> + * Copyright (C) 2008 Anders Kaseorg <andersk@mit.edu>, + * Tim Abbott <tabbott@mit.edu> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston, MA + * 02110-1301, USA. + */ + +#include <linux/module.h> +#include <linux/bug.h> +#include <linux/ctype.h> +#include <linux/debugfs.h> +#include <linux/errno.h> +#include <linux/kallsyms.h> +#include <linux/kobject.h> +#include <linux/kthread.h> +#include <linux/pagemap.h> +#include <linux/sched.h> +#include <linux/sort.h> +#include <linux/stop_machine.h> +#include <linux/sysfs.h> +#include <linux/time.h> +#include <linux/uaccess.h> +#include <linux/vmalloc.h> +#include <linux/ksplice.h> + +enum stage { + STAGE_PREPARING, /* the update is not yet applied */ + STAGE_APPLIED, /* the update is applied */ + STAGE_REVERSED, /* the update has been applied and reversed */ +}; + +/* parameter to modify run-pre matching */ +enum run_pre_mode { + RUN_PRE_INITIAL, /* dry run (only change temp_labelvals) */ + RUN_PRE_DEBUG, /* dry run with byte-by-byte debugging */ + RUN_PRE_FINAL, /* finalizes the matching */ +}; + +enum { NOVAL, TEMP, VAL }; + +typedef int __bitwise__ abort_t; + +#define OK ((__force abort_t) 0) +#define NO_MATCH ((__force abort_t) 1) +#define CODE_BUSY ((__force abort_t) 2) +#define MODULE_BUSY ((__force abort_t) 3) +#define OUT_OF_MEMORY ((__force abort_t) 4) +#define FAILED_TO_FIND ((__force abort_t) 5) +#define ALREADY_REVERSED ((__force abort_t) 6) +#define MISSING_EXPORT ((__force abort_t) 7) +#define UNEXPECTED_RUNNING_TASK ((__force abort_t) 8) +#define UNEXPECTED ((__force abort_t) 9) +#define TARGET_NOT_LOADED ((__force abort_t) 10) +#define CALL_FAILED ((__force abort_t) 11) + +struct update { + const char *kid; + const char *name; + struct kobject kobj; + enum stage stage; + abort_t abort_cause; + int debug; +#ifdef CONFIG_DEBUG_FS + struct debugfs_blob_wrapper debug_blob; + struct dentry *debugfs_dentry; +#else /* !CONFIG_DEBUG_FS */ + bool debug_continue_line; +#endif /* CONFIG_DEBUG_FS */ + bool partial; /* is it OK if some target mods aren't loaded */ + struct list_head packs; /* packs for loaded target mods */ + struct list_head unused_packs; /* packs for non-loaded target mods */ + struct list_head conflicts; + struct list_head list; +}; + +/* a process conflicting with an update */ +struct conflict { + const char *process_name; + pid_t pid; + struct list_head stack; + struct list_head list; +}; + +/* an address on the stack of a conflict */ +struct conflict_addr { + unsigned long addr; /* the address on the stack */ + bool has_conflict; /* does this address in particular conflict? */ + const char *label; /* the label of the conflicting safety_record */ + struct list_head list; +}; + +struct labelval { + struct list_head list; + struct ksplice_symbol *symbol; + struct list_head *saved_vals; +}; + +/* region to be checked for conflicts in the stack check */ +struct safety_record { + struct list_head list; + const char *label; + unsigned long addr; /* the address to be checked for conflicts + * (e.g. an obsolete function's starting addr) + */ + unsigned long size; /* the size of the region to be checked */ +}; + +/* possible value for a symbol */ +struct candidate_val { + struct list_head list; + unsigned long val; +}; + +/* private struct used by init_symbol_array */ +struct ksplice_lookup { +/* input */ + struct ksplice_pack *pack; + struct ksplice_symbol **arr; + size_t size; +/* output */ + abort_t ret; +}; + +static LIST_HEAD(updates); +LIST_HEAD(ksplice_module_list); +EXPORT_SYMBOL_GPL(ksplice_module_list); +static struct kobject *ksplice_kobj; + +static struct kobj_type ksplice_ktype; + +static struct update *init_ksplice_update(const char *kid); +static void cleanup_ksplice_update(struct update *update); +static void maybe_cleanup_ksplice_update(struct update *update); +static void add_to_update(struct ksplice_pack *pack, struct update *update); +static int ksplice_sysfs_init(struct update *update); + +/* Preparing the relocations and patches for application */ +static abort_t apply_update(struct update *update); +static abort_t prepare_pack(struct ksplice_pack *pack); +static abort_t finalize_pack(struct ksplice_pack *pack); +static abort_t finalize_patches(struct ksplice_pack *pack); +static abort_t add_dependency_on_address(struct ksplice_pack *pack, + unsigned long addr); +static abort_t map_trampoline_pages(struct update *update); +static void unmap_trampoline_pages(struct update *update); +static void *map_writable(void *addr, size_t len); +static abort_t apply_relocs(struct ksplice_pack *pack, + const struct ksplice_reloc *relocs, + const struct ksplice_reloc *relocs_end); +static abort_t apply_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r); +static abort_t apply_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r); +static abort_t apply_howto_date(struct ksplice_pack *pack, + const struct ksplice_reloc *r); +static abort_t read_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long *valp); +static abort_t write_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long sym_addr); +static void __attribute__((noreturn)) ksplice_deleted(void); + +/* run-pre matching */ +static abort_t match_pack_sections(struct ksplice_pack *pack, + bool consider_data_sections); +static abort_t find_section(struct ksplice_pack *pack, + struct ksplice_section *sect); +static abort_t try_addr(struct ksplice_pack *pack, + struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode); +static abort_t run_pre_cmp(struct ksplice_pack *pack, + const struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode); +static void print_bytes(struct ksplice_pack *pack, + const unsigned char *run, int runc, + const unsigned char *pre, int prec); +static const struct ksplice_reloc * +init_reloc_search(struct ksplice_pack *pack, + const struct ksplice_section *sect); +static const struct ksplice_reloc *find_reloc(const struct ksplice_reloc *start, + const struct ksplice_reloc *end, + unsigned long address, + unsigned long size); +static abort_t lookup_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc **fingerp, + unsigned long addr, + const struct ksplice_reloc **relocp); +static abort_t handle_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, enum run_pre_mode mode); +static abort_t handle_howto_date(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, + enum run_pre_mode mode); +static abort_t handle_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, + enum run_pre_mode mode); +static struct ksplice_section *symbol_section(struct ksplice_pack *pack, + const struct ksplice_symbol *sym); +static int compare_section_labels(const void *va, const void *vb); +static int symbol_section_bsearch_compare(const void *a, const void *b); +static const struct ksplice_reloc *patch_reloc(struct ksplice_pack *pack, + const struct ksplice_patch *p); + +/* Computing possible addresses for symbols */ +static abort_t lookup_symbol(struct ksplice_pack *pack, + const struct ksplice_symbol *ksym, + struct list_head *vals); +static void cleanup_symbol_arrays(struct ksplice_pack *pack); +static abort_t init_symbol_arrays(struct ksplice_pack *pack); +static abort_t init_symbol_array(struct ksplice_pack *pack, + struct ksplice_symbol *start, + struct ksplice_symbol *end); +static abort_t uniquify_symbols(struct ksplice_pack *pack); +static abort_t add_matching_values(struct ksplice_lookup *lookup, + const char *sym_name, unsigned long sym_val); +static bool add_export_values(const struct symsearch *syms, + struct module *owner, + unsigned int symnum, void *data); +static int symbolp_bsearch_compare(const void *key, const void *elt); +static int compare_symbolp_names(const void *a, const void *b); +static int compare_symbolp_labels(const void *a, const void *b); +static int add_kallsyms_values(void *data, const char *name, + struct module *owner, unsigned long val); +static abort_t new_export_lookup(struct ksplice_pack *ipack, const char *name, + struct list_head *vals); + +/* Atomic update trampoline insertion and removal */ +static abort_t apply_patches(struct update *update); +static abort_t reverse_patches(struct update *update); +static int __apply_patches(void *update); +static int __reverse_patches(void *update); +static abort_t check_each_task(struct update *update); +static abort_t check_task(struct update *update, + const struct task_struct *t, bool rerun); +static abort_t check_stack(struct update *update, struct conflict *conf, + const struct thread_info *tinfo, + const unsigned long *stack); +static abort_t check_address(struct update *update, + struct conflict *conf, unsigned long addr); +static abort_t check_record(struct conflict_addr *ca, + const struct safety_record *rec, + unsigned long addr); +static bool is_stop_machine(const struct task_struct *t); +static void cleanup_conflicts(struct update *update); +static void print_conflicts(struct update *update); +static void insert_trampoline(struct ksplice_patch *p); +static abort_t verify_trampoline(struct ksplice_pack *pack, + const struct ksplice_patch *p); +static void remove_trampoline(const struct ksplice_patch *p); + +static abort_t create_labelval(struct ksplice_pack *pack, + struct ksplice_symbol *ksym, + unsigned long val, int status); +static abort_t create_safety_record(struct ksplice_pack *pack, + const struct ksplice_section *sect, + struct list_head *record_list, + unsigned long run_addr, + unsigned long run_size); +static abort_t add_candidate_val(struct ksplice_pack *pack, + struct list_head *vals, unsigned long val); +static void release_vals(struct list_head *vals); +static void set_temp_labelvals(struct ksplice_pack *pack, int status_val); + +static int contains_canary(struct ksplice_pack *pack, unsigned long blank_addr, + const struct ksplice_reloc_howto *howto); +static unsigned long follow_trampolines(struct ksplice_pack *pack, + unsigned long addr); +static bool patches_module(const struct module *a, const struct module *b); +static bool starts_with(const char *str, const char *prefix); +static bool singular(struct list_head *list); +static void *bsearch(const void *key, const void *base, size_t n, + size_t size, int (*cmp)(const void *key, const void *elt)); +static int compare_relocs(const void *a, const void *b); +static int reloc_bsearch_compare(const void *key, const void *elt); + +/* Debugging */ +static abort_t init_debug_buf(struct update *update); +static void clear_debug_buf(struct update *update); +static int __attribute__((format(printf, 2, 3))) +_ksdebug(struct update *update, const char *fmt, ...); +#define ksdebug(pack, fmt, ...) \ + _ksdebug(pack->update, fmt, ## __VA_ARGS__) + +/* Architecture-specific functions defined in arch/ARCH/kernel/ksplice-arch.c */ + +/* Prepare a trampoline for the given patch */ +static abort_t prepare_trampoline(struct ksplice_pack *pack, + struct ksplice_patch *p); +/* What address does the trampoline at addr jump to? */ +static abort_t trampoline_target(struct ksplice_pack *pack, unsigned long addr, + unsigned long *new_addr); +/* Hook to handle pc-relative jumps inserted by parainstructions */ +static abort_t handle_paravirt(struct ksplice_pack *pack, unsigned long pre, + unsigned long run, int *matched); +/* Called for relocations of type KSPLICE_HOWTO_BUG */ +static abort_t handle_bug(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long run_addr); +/* Called for relocations of type KSPLICE_HOWTO_EXTABLE */ +static abort_t handle_extable(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long run_addr); +/* Is address p on the stack of the given thread? */ +static bool valid_stack_ptr(const struct thread_info *tinfo, const void *p); + +#include "ksplice-arch.c" + +#define clear_list(head, type, member) \ + do { \ + struct list_head *_pos, *_n; \ + list_for_each_safe(_pos, _n, head) { \ + list_del(_pos); \ + kfree(list_entry(_pos, type, member)); \ + } \ + } while (0) + +/** + * init_ksplice_pack() - Initializes a ksplice pack + * @pack: The pack to be initialized. All of the public fields of the + * pack and its associated data structures should be populated + * before this function is called. The values of the private + * fields will be ignored. + **/ +int init_ksplice_pack(struct ksplice_pack *pack) +{ + struct update *update; + struct ksplice_patch *p; + struct ksplice_section *s; + int ret = 0; + + INIT_LIST_HEAD(&pack->temp_labelvals); + INIT_LIST_HEAD(&pack->safety_records); + + sort(pack->helper_relocs, + pack->helper_relocs_end - pack->helper_relocs, + sizeof(*pack->helper_relocs), compare_relocs, NULL); + sort(pack->primary_relocs, + pack->primary_relocs_end - pack->primary_relocs, + sizeof(*pack->primary_relocs), compare_relocs, NULL); + sort(pack->helper_sections, + pack->helper_sections_end - pack->helper_sections, + sizeof(*pack->helper_sections), compare_section_labels, NULL); + + for (p = pack->patches; p < pack->patches_end; p++) + p->vaddr = NULL; + for (s = pack->helper_sections; s < pack->helper_sections_end; s++) + s->match_map = NULL; + for (p = pack->patches; p < pack->patches_end; p++) { + const struct ksplice_reloc *r = patch_reloc(pack, p); + if (r == NULL) + return -ENOENT; + if (p->type == KSPLICE_PATCH_DATA) { + s = symbol_section(pack, r->symbol); + if (s == NULL) + return -ENOENT; + /* Ksplice creates KSPLICE_PATCH_DATA patches in order + * to modify rodata sections that have been explicitly + * marked for patching using the ksplice-patch.h macro + * ksplice_assume_rodata. Here we modify the section + * flags appropriately. + */ + if (s->flags & KSPLICE_SECTION_DATA) + s->flags = (s->flags & ~KSPLICE_SECTION_DATA) | + KSPLICE_SECTION_RODATA; + } + } + + mutex_lock(&module_mutex); + list_for_each_entry(update, &updates, list) { + if (strcmp(pack->kid, update->kid) == 0) { + if (update->stage != STAGE_PREPARING) { + ret = -EPERM; + goto out; + } + add_to_update(pack, update); + ret = 0; + goto out; + } + } + update = init_ksplice_update(pack->kid); + if (update == NULL) { + ret = -ENOMEM; + goto out; + } + ret = ksplice_sysfs_init(update); + if (ret != 0) { + cleanup_ksplice_update(update); + goto out; + } + add_to_update(pack, update); +out: + mutex_unlock(&module_mutex); + return ret; +} +EXPORT_SYMBOL_GPL(init_ksplice_pack); + +/** + * cleanup_ksplice_pack() - Cleans up a pack + * @pack: The pack to be cleaned up + */ +void cleanup_ksplice_pack(struct ksplice_pack *pack) +{ + if (pack->update == NULL) + return; + + mutex_lock(&module_mutex); + if (pack->update->stage == STAGE_APPLIED) { + /* If the pack wasn't actually applied (because we + * only applied this update to loaded modules and this + * target was not loaded), then unregister the pack + * from the list of unused packs. + */ + struct ksplice_pack *p; + bool found = false; + + list_for_each_entry(p, &pack->update->unused_packs, list) { + if (p == pack) + found = true; + } + if (found) + list_del(&pack->list); + mutex_unlock(&module_mutex); + return; + } + list_del(&pack->list); + if (pack->update->stage == STAGE_PREPARING) + maybe_cleanup_ksplice_update(pack->update); + pack->update = NULL; + mutex_unlock(&module_mutex); +} +EXPORT_SYMBOL_GPL(cleanup_ksplice_pack); + +static struct update *init_ksplice_update(const char *kid) +{ + struct update *update; + update = kcalloc(1, sizeof(struct update), GFP_KERNEL); + if (update == NULL) + return NULL; + update->name = kasprintf(GFP_KERNEL, "ksplice_%s", kid); + if (update->name == NULL) { + kfree(update); + return NULL; + } + update->kid = kstrdup(kid, GFP_KERNEL); + if (update->kid == NULL) { + kfree(update->name); + kfree(update); + return NULL; + } + if (try_module_get(THIS_MODULE) != 1) { + kfree(update->kid); + kfree(update->name); + kfree(update); + return NULL; + } + INIT_LIST_HEAD(&update->packs); + INIT_LIST_HEAD(&update->unused_packs); + if (init_debug_buf(update) != OK) { + module_put(THIS_MODULE); + kfree(update->kid); + kfree(update->name); + kfree(update); + return NULL; + } + list_add(&update->list, &updates); + update->stage = STAGE_PREPARING; + update->abort_cause = OK; + update->partial = 0; + INIT_LIST_HEAD(&update->conflicts); + return update; +} + +static void cleanup_ksplice_update(struct update *update) +{ + list_del(&update->list); + cleanup_conflicts(update); + clear_debug_buf(update); + kfree(update->kid); + kfree(update->name); + kfree(update); + module_put(THIS_MODULE); +} + +/* Clean up the update if it no longer has any packs */ +static void maybe_cleanup_ksplice_update(struct update *update) +{ + if (list_empty(&update->packs) && list_empty(&update->unused_packs)) + kobject_put(&update->kobj); +} + +static void add_to_update(struct ksplice_pack *pack, struct update *update) +{ + pack->update = update; + list_add(&pack->list, &update->unused_packs); + pack->module_list_entry.primary = pack->primary; +} + +static int ksplice_sysfs_init(struct update *update) +{ + int ret = 0; + memset(&update->kobj, 0, sizeof(update->kobj)); + ret = kobject_init_and_add(&update->kobj, &ksplice_ktype, + ksplice_kobj, "%s", update->kid); + if (ret != 0) + return ret; + kobject_uevent(&update->kobj, KOBJ_ADD); + return 0; +} + +static abort_t apply_update(struct update *update) +{ + struct ksplice_pack *pack, *n; + abort_t ret; + int retval; + + list_for_each_entry_safe(pack, n, &update->unused_packs, list) { + if (strcmp(pack->target_name, "vmlinux") == 0) { + pack->target = NULL; + } else if (pack->target == NULL) { + pack->target = find_module(pack->target_name); + if (pack->target == NULL || + !module_is_live(pack->target)) { + if (update->partial) { + continue; + } else { + ret = TARGET_NOT_LOADED; + goto out; + } + } + retval = use_module(pack->primary, pack->target); + if (retval != 1) { + ret = UNEXPECTED; + goto out; + } + } + list_del(&pack->list); + list_add_tail(&pack->list, &update->packs); + pack->module_list_entry.target = pack->target; + + } + + list_for_each_entry(pack, &update->packs, list) { + const struct ksplice_section *sect; + for (sect = pack->primary_sections; + sect < pack->primary_sections_end; sect++) { + struct safety_record *rec = kmalloc(sizeof(*rec), + GFP_KERNEL); + if (rec == NULL) { + ret = OUT_OF_MEMORY; + goto out; + } + rec->addr = sect->address; + rec->size = sect->size; + rec->label = sect->symbol->label; + list_add(&rec->list, &pack->safety_records); + } + } + + list_for_each_entry(pack, &update->packs, list) { + ret = init_symbol_arrays(pack); + if (ret != OK) { + cleanup_symbol_arrays(pack); + goto out; + } + ret = prepare_pack(pack); + cleanup_symbol_arrays(pack); + if (ret != OK) + goto out; + } + ret = apply_patches(update); +out: + list_for_each_entry(pack, &update->packs, list) { + struct ksplice_section *s; + if (update->stage == STAGE_PREPARING) + clear_list(&pack->safety_records, struct safety_record, + list); + for (s = pack->helper_sections; s < pack->helper_sections_end; + s++) { + if (s->match_map != NULL) { + vfree(s->match_map); + s->match_map = NULL; + } + } + } + return ret; +} + +static int compare_symbolp_names(const void *a, const void *b) +{ + const struct ksplice_symbol *const *sympa = a, *const *sympb = b; + if ((*sympa)->name == NULL && (*sympb)->name == NULL) + return 0; + if ((*sympa)->name == NULL) + return -1; + if ((*sympb)->name == NULL) + return 1; + return strcmp((*sympa)->name, (*sympb)->name); +} + +static int compare_symbolp_labels(const void *a, const void *b) +{ + const struct ksplice_symbol *const *sympa = a, *const *sympb = b; + return strcmp((*sympa)->label, (*sympb)->label); +} + +static int symbolp_bsearch_compare(const void *key, const void *elt) +{ + const char *name = key; + const struct ksplice_symbol *const *symp = elt; + const struct ksplice_symbol *sym = *symp; + if (sym->name == NULL) + return 1; + return strcmp(name, sym->name); +} + +static abort_t add_matching_values(struct ksplice_lookup *lookup, + const char *sym_name, unsigned long sym_val) +{ + struct ksplice_symbol **symp; + abort_t ret; + + symp = bsearch(sym_name, lookup->arr, lookup->size, + sizeof(*lookup->arr), symbolp_bsearch_compare); + if (symp == NULL) + return OK; + + while (symp > lookup->arr && + symbolp_bsearch_compare(sym_name, symp - 1) == 0) + symp--; + + for (; symp < lookup->arr + lookup->size; symp++) { + struct ksplice_symbol *sym = *symp; + if (sym->name == NULL || strcmp(sym_name, sym->name) != 0) + break; + ret = add_candidate_val(lookup->pack, sym->vals, sym_val); + if (ret != OK) + return ret; + } + return OK; +} + +static int add_kallsyms_values(void *data, const char *name, + struct module *owner, unsigned long val) +{ + struct ksplice_lookup *lookup = data; + if (owner == lookup->pack->primary || + !patches_module(owner, lookup->pack->target)) + return (__force int)OK; + return (__force int)add_matching_values(lookup, name, val); +} + +static bool add_export_values(const struct symsearch *syms, + struct module *owner, + unsigned int symnum, void *data) +{ + struct ksplice_lookup *lookup = data; + abort_t ret; + + ret = add_matching_values(lookup, syms->start[symnum].name, + syms->start[symnum].value); + if (ret != OK) { + lookup->ret = ret; + return true; + } + return false; +} + +static void cleanup_symbol_arrays(struct ksplice_pack *pack) +{ + struct ksplice_symbol *sym; + for (sym = pack->primary_symbols; sym < pack->primary_symbols_end; + sym++) { + if (sym->vals != NULL) { + clear_list(sym->vals, struct candidate_val, list); + kfree(sym->vals); + sym->vals = NULL; + } + } + for (sym = pack->helper_symbols; sym < pack->helper_symbols_end; sym++) { + if (sym->vals != NULL) { + clear_list(sym->vals, struct candidate_val, list); + kfree(sym->vals); + sym->vals = NULL; + } + } +} + +/* + * The primary and helper modules each have their own independent + * ksplice_symbol structures. uniquify_symbols unifies these separate + * pieces of kernel symbol information by replacing all references to + * the helper copy of symbols with references to the primary copy. + */ +static abort_t uniquify_symbols(struct ksplice_pack *pack) +{ + struct ksplice_reloc *r; + struct ksplice_section *s; + struct ksplice_symbol *sym, **sym_arr, **symp; + size_t size = pack->primary_symbols_end - pack->primary_symbols; + + if (size == 0) + return OK; + + sym_arr = vmalloc(sizeof(*sym_arr) * size); + if (sym_arr == NULL) + return OUT_OF_MEMORY; + + for (symp = sym_arr, sym = pack->primary_symbols; + symp < sym_arr + size && sym < pack->primary_symbols_end; + sym++, symp++) + *symp = sym; + + sort(sym_arr, size, sizeof(*sym_arr), compare_symbolp_labels, NULL); + + for (r = pack->helper_relocs; r < pack->helper_relocs_end; r++) { + symp = bsearch(&r->symbol, sym_arr, size, sizeof(*sym_arr), + compare_symbolp_labels); + if (symp != NULL) { + if ((*symp)->name == NULL) + (*symp)->name = r->symbol->name; + r->symbol = *symp; + } + } + + for (s = pack->helper_sections; s < pack->helper_sections_end; s++) { + symp = bsearch(&s->symbol, sym_arr, size, sizeof(*sym_arr), + compare_symbolp_labels); + if (symp != NULL) { + if ((*symp)->name == NULL) + (*symp)->name = s->symbol->name; + s->symbol = *symp; + } + } + + vfree(sym_arr); + return OK; +} + +/* + * Initialize the ksplice_symbol structures in the given array using + * the kallsyms and exported symbol tables. + */ +static abort_t init_symbol_array(struct ksplice_pack *pack, + struct ksplice_symbol *start, + struct ksplice_symbol *end) +{ + struct ksplice_symbol *sym, **sym_arr, **symp; + struct ksplice_lookup lookup; + size_t size = end - start; + abort_t ret; + + if (size == 0) + return OK; + + for (sym = start; sym < end; sym++) { + if (starts_with(sym->label, "__ksymtab")) { + const struct kernel_symbol *ksym; + const char *colon = strchr(sym->label, ':'); + const char *name = colon + 1; + if (colon == NULL) + continue; + ksym = find_symbol(name, NULL, NULL, true, false); + if (ksym == NULL) { + ksdebug(pack, "Could not find kernel_symbol " + "structure for %s\n", name); + continue; + } + sym->value = (unsigned long)ksym; + sym->vals = NULL; + continue; + } + + sym->vals = kmalloc(sizeof(*sym->vals), GFP_KERNEL); + if (sym->vals == NULL) + return OUT_OF_MEMORY; + INIT_LIST_HEAD(sym->vals); + sym->value = 0; + } + + sym_arr = vmalloc(sizeof(*sym_arr) * size); + if (sym_arr == NULL) + return OUT_OF_MEMORY; + + for (symp = sym_arr, sym = start; symp < sym_arr + size && sym < end; + sym++, symp++) + *symp = sym; + + sort(sym_arr, size, sizeof(*sym_arr), compare_symbolp_names, NULL); + + lookup.pack = pack; + lookup.arr = sym_arr; + lookup.size = size; + lookup.ret = OK; + + each_symbol(add_export_values, &lookup); + ret = lookup.ret; + if (ret == OK) + ret = (__force abort_t) + kallsyms_on_each_symbol(add_kallsyms_values, &lookup); + vfree(sym_arr); + return ret; +} + +/* Prepare the pack's ksplice_symbol structures for run-pre matching */ +static abort_t init_symbol_arrays(struct ksplice_pack *pack) +{ + abort_t ret; + + ret = uniquify_symbols(pack); + if (ret != OK) + return ret; + + ret = init_symbol_array(pack, pack->helper_symbols, + pack->helper_symbols_end); + if (ret != OK) + return ret; + + ret = init_symbol_array(pack, pack->primary_symbols, + pack->primary_symbols_end); + if (ret != OK) + return ret; + + return OK; +} + +static abort_t prepare_pack(struct ksplice_pack *pack) +{ + abort_t ret; + + ksdebug(pack, "Preparing and checking %s\n", pack->name); + ret = match_pack_sections(pack, false); + if (ret == NO_MATCH) { + /* It is possible that by using relocations from .data sections + * we can successfully run-pre match the rest of the sections. + * To avoid using any symbols obtained from .data sections + * (which may be unreliable) in the post code, we first prepare + * the post code and then try to run-pre match the remaining + * sections with the help of .data sections. + */ + ksdebug(pack, "Continuing without some sections; we might " + "find them later.\n"); + ret = finalize_pack(pack); + if (ret != OK) { + ksdebug(pack, "Aborted. Unable to continue without " + "the unmatched sections.\n"); + return ret; + } + + ksdebug(pack, "run-pre: Considering .data sections to find the " + "unmatched sections\n"); + ret = match_pack_sections(pack, true); + if (ret != OK) + return ret; + + ksdebug(pack, "run-pre: Found all previously unmatched " + "sections\n"); + return OK; + } else if (ret != OK) { + return ret; + } + + return finalize_pack(pack); +} + +/* + * Finish preparing the pack for insertion into the kernel. + * Afterwards, the replacement code should be ready to run and the + * ksplice_patches should all be ready for trampoline insertion. + */ +static abort_t finalize_pack(struct ksplice_pack *pack) +{ + abort_t ret; + ret = apply_relocs(pack, pack->primary_relocs, + pack->primary_relocs_end); + if (ret != OK) + return ret; + + ret = finalize_patches(pack); + if (ret != OK) + return ret; + + return OK; +} + +static abort_t finalize_patches(struct ksplice_pack *pack) +{ + struct ksplice_patch *p; + struct safety_record *rec; + abort_t ret; + + for (p = pack->patches; p < pack->patches_end; p++) { + bool found = false; + list_for_each_entry(rec, &pack->safety_records, list) { + if (rec->addr <= p->oldaddr && + p->oldaddr < rec->addr + rec->size) { + found = true; + break; + } + } + if (!found && p->type != KSPLICE_PATCH_EXPORT) { + const struct ksplice_reloc *r = patch_reloc(pack, p); + if (r == NULL) { + ksdebug(pack, "A patch with no ksplice_reloc at" + " its oldaddr has no safety record\n"); + return NO_MATCH; + } + ksdebug(pack, "No safety record for patch with oldaddr " + "%s+%lx\n", r->symbol->label, r->target_addend); + return NO_MATCH; + } + + if (p->type == KSPLICE_PATCH_TEXT) { + ret = prepare_trampoline(pack, p); + if (ret != OK) + return ret; + } + + if (found && rec->addr + rec->size < p->oldaddr + p->size) { + ksdebug(pack, "Safety record %s is too short for " + "patch\n", rec->label); + return UNEXPECTED; + } + + if (p->type == KSPLICE_PATCH_TEXT) { + if (p->repladdr == 0) + p->repladdr = (unsigned long)ksplice_deleted; + } + + ret = add_dependency_on_address(pack, p->oldaddr); + if (ret != OK) + return ret; + } + return OK; +} + +static abort_t map_trampoline_pages(struct update *update) +{ + struct ksplice_pack *pack; + list_for_each_entry(pack, &update->packs, list) { + struct ksplice_patch *p; + for (p = pack->patches; p < pack->patches_end; p++) { + p->vaddr = map_writable((void *)p->oldaddr, p->size); + if (p->vaddr == NULL) { + ksdebug(pack, "Unable to map oldaddr read/write" + "\n"); + unmap_trampoline_pages(update); + return UNEXPECTED; + } + } + } + return OK; +} + +static void unmap_trampoline_pages(struct update *update) +{ + struct ksplice_pack *pack; + list_for_each_entry(pack, &update->packs, list) { + struct ksplice_patch *p; + for (p = pack->patches; p < pack->patches_end; p++) { + vunmap((void *)((unsigned long)p->vaddr & PAGE_MASK)); + p->vaddr = NULL; + } + } +} + +/* + * map_writable creates a shadow page mapping of the range + * [addr, addr + len) so that we can write to code mapped read-only. + * + * It is similar to a generalized version of x86's text_poke. But + * because one cannot use vmalloc/vfree() inside stop_machine, we use + * map_writable to map the pages before stop_machine, then use the + * mapping inside stop_machine, and unmap the pages afterwards. + */ +static void *map_writable(void *addr, size_t len) +{ + void *vaddr; + int nr_pages = 2; + struct page *pages[2]; + + if (__module_text_address((unsigned long)addr) == NULL && + __module_data_address((unsigned long)addr) == NULL) { + pages[0] = virt_to_page(addr); + WARN_ON(!PageReserved(pages[0])); + pages[1] = virt_to_page(addr + PAGE_SIZE); + } else { + pages[0] = vmalloc_to_page(addr); + pages[1] = vmalloc_to_page(addr + PAGE_SIZE); + } + if (!pages[0]) + return NULL; + if (!pages[1]) + nr_pages = 1; + vaddr = vmap(pages, nr_pages, VM_MAP, PAGE_KERNEL); + if (vaddr == NULL) + return NULL; + return vaddr + offset_in_page(addr); +} + +/* + * Ksplice adds a dependency on any symbol address used to resolve relocations + * in the primary module. + * + * Be careful to follow_trampolines so that we always depend on the + * latest version of the target function, since that's the code that + * will run if we call addr. + */ +static abort_t add_dependency_on_address(struct ksplice_pack *pack, + unsigned long addr) +{ + struct ksplice_pack *p; + struct module *m = + __module_text_address(follow_trampolines(pack, addr)); + if (m == NULL) + return OK; + list_for_each_entry(p, &pack->update->packs, list) { + if (m == p->primary) + return OK; + } + if (use_module(pack->primary, m) != 1) + return MODULE_BUSY; + return OK; +} + +static abort_t apply_relocs(struct ksplice_pack *pack, + const struct ksplice_reloc *relocs, + const struct ksplice_reloc *relocs_end) +{ + const struct ksplice_reloc *r; + for (r = relocs; r < relocs_end; r++) { + abort_t ret = apply_reloc(pack, r); + if (ret != OK) + return ret; + } + return OK; +} + +static abort_t apply_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r) +{ + switch (r->howto->type) { + case KSPLICE_HOWTO_RELOC: + case KSPLICE_HOWTO_RELOC_PATCH: + return apply_howto_reloc(pack, r); + case KSPLICE_HOWTO_DATE: + case KSPLICE_HOWTO_TIME: + return apply_howto_date(pack, r); + default: + ksdebug(pack, "Unexpected howto type %d\n", r->howto->type); + return UNEXPECTED; + } +} + +/* + * Applies a relocation. Aborts if the symbol referenced in it has + * not been uniquely resolved. + */ +static abort_t apply_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc *r) +{ + abort_t ret; + int canary_ret; + unsigned long sym_addr; + LIST_HEAD(vals); + + canary_ret = contains_canary(pack, r->blank_addr, r->howto); + if (canary_ret < 0) + return UNEXPECTED; + if (canary_ret == 0) { + ksdebug(pack, "reloc: skipped %lx to %s+%lx (altinstr)\n", + r->blank_addr, r->symbol->label, r->target_addend); + return OK; + } + + ret = lookup_symbol(pack, r->symbol, &vals); + if (ret != OK) { + release_vals(&vals); + return ret; + } + /* + * Relocations for the oldaddr fields of patches must have + * been resolved via run-pre matching. + */ + if (!singular(&vals) || (r->symbol->vals != NULL && + r->howto->type == KSPLICE_HOWTO_RELOC_PATCH)) { + release_vals(&vals); + ksdebug(pack, "Failed to find %s for reloc\n", + r->symbol->label); + return FAILED_TO_FIND; + } + sym_addr = list_entry(vals.next, struct candidate_val, list)->val; + release_vals(&vals); + + ret = write_reloc_value(pack, r, r->blank_addr, + r->howto->pcrel ? sym_addr - r->blank_addr : + sym_addr); + if (ret != OK) + return ret; + + ksdebug(pack, "reloc: %lx to %s+%lx (S=%lx ", r->blank_addr, + r->symbol->label, r->target_addend, sym_addr); + switch (r->howto->size) { + case 1: + ksdebug(pack, "aft=%02x)\n", *(uint8_t *)r->blank_addr); + break; + case 2: + ksdebug(pack, "aft=%04x)\n", *(uint16_t *)r->blank_addr); + break; + case 4: + ksdebug(pack, "aft=%08x)\n", *(uint32_t *)r->blank_addr); + break; +#if BITS_PER_LONG >= 64 + case 8: + ksdebug(pack, "aft=%016llx)\n", *(uint64_t *)r->blank_addr); + break; +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + /* + * Create labelvals so that we can verify our choices in the + * second round of run-pre matching that considers data sections. + */ + ret = create_labelval(pack, r->symbol, sym_addr, VAL); + if (ret != OK) + return ret; + + return add_dependency_on_address(pack, sym_addr); +} + +/* + * Date relocations are created wherever __DATE__ or __TIME__ is used + * in the kernel; we resolve them by simply copying in the date/time + * obtained from run-pre matching the relevant compilation unit. + */ +static abort_t apply_howto_date(struct ksplice_pack *pack, + const struct ksplice_reloc *r) +{ + if (r->symbol->vals != NULL) { + ksdebug(pack, "Failed to find %s for date\n", r->symbol->label); + return FAILED_TO_FIND; + } + memcpy((unsigned char *)r->blank_addr, + (const unsigned char *)r->symbol->value, r->howto->size); + return OK; +} + +/* + * Given a relocation and its run address, compute the address of the + * symbol the relocation referenced, and store it in *valp. + */ +static abort_t read_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long *valp) +{ + unsigned char bytes[sizeof(long)]; + unsigned long val; + const struct ksplice_reloc_howto *howto = r->howto; + + if (howto->size <= 0 || howto->size > sizeof(long)) { + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + if (probe_kernel_read(bytes, (void *)addr, howto->size) == -EFAULT) + return NO_MATCH; + + switch (howto->size) { + case 1: + val = *(uint8_t *)bytes; + break; + case 2: + val = *(uint16_t *)bytes; + break; + case 4: + val = *(uint32_t *)bytes; + break; +#if BITS_PER_LONG >= 64 + case 8: + val = *(uint64_t *)bytes; + break; +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + val &= howto->dst_mask; + if (howto->signed_addend) + val |= -(val & (howto->dst_mask & ~(howto->dst_mask >> 1))); + val <<= howto->rightshift; + val -= r->insn_addend + r->target_addend; + *valp = val; + return OK; +} + +/* + * Given a relocation, the address of its storage unit, and the + * address of the symbol the relocation references, write the + * relocation's final value into the storage unit. + */ +static abort_t write_reloc_value(struct ksplice_pack *pack, + const struct ksplice_reloc *r, + unsigned long addr, unsigned long sym_addr) +{ + unsigned long val = sym_addr + r->target_addend + r->insn_addend; + const struct ksplice_reloc_howto *howto = r->howto; + val >>= howto->rightshift; + switch (howto->size) { + case 1: + *(uint8_t *)addr = (*(uint8_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; + case 2: + *(uint16_t *)addr = (*(uint16_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; + case 4: + *(uint32_t *)addr = (*(uint32_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; +#if BITS_PER_LONG >= 64 + case 8: + *(uint64_t *)addr = (*(uint64_t *)addr & ~howto->dst_mask) | + (val & howto->dst_mask); + break; +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return UNEXPECTED; + } + + if (read_reloc_value(pack, r, addr, &val) != OK || val != sym_addr) { + ksdebug(pack, "Aborted. Relocation overflow.\n"); + return UNEXPECTED; + } + + return OK; +} + +/* Replacement address used for functions deleted by the patch */ +static void __attribute__((noreturn)) ksplice_deleted(void) +{ + printk(KERN_CRIT "Called a kernel function deleted by Ksplice!\n"); + BUG(); +} + +/* Floodfill to run-pre match the sections within a pack. */ +static abort_t match_pack_sections(struct ksplice_pack *pack, + bool consider_data_sections) +{ + struct ksplice_section *sect; + abort_t ret; + int remaining = 0; + bool progress; + + for (sect = pack->helper_sections; sect < pack->helper_sections_end; + sect++) { + if ((sect->flags & KSPLICE_SECTION_DATA) == 0 && + (sect->flags & KSPLICE_SECTION_STRING) == 0 && + (sect->flags & KSPLICE_SECTION_MATCHED) == 0) + remaining++; + } + + while (remaining > 0) { + progress = false; + for (sect = pack->helper_sections; + sect < pack->helper_sections_end; sect++) { + if ((sect->flags & KSPLICE_SECTION_MATCHED) != 0) + continue; + if ((!consider_data_sections && + (sect->flags & KSPLICE_SECTION_DATA) != 0) || + (sect->flags & KSPLICE_SECTION_STRING) != 0) + continue; + ret = find_section(pack, sect); + if (ret == OK) { + sect->flags |= KSPLICE_SECTION_MATCHED; + if ((sect->flags & KSPLICE_SECTION_DATA) == 0) + remaining--; + progress = true; + } else if (ret != NO_MATCH) { + return ret; + } + } + + if (progress) + continue; + + for (sect = pack->helper_sections; + sect < pack->helper_sections_end; sect++) { + if ((sect->flags & KSPLICE_SECTION_MATCHED) != 0 || + (sect->flags & KSPLICE_SECTION_STRING) != 0) + continue; + ksdebug(pack, "run-pre: could not match %s " + "section %s\n", + (sect->flags & KSPLICE_SECTION_DATA) != 0 ? + "data" : + (sect->flags & KSPLICE_SECTION_RODATA) != 0 ? + "rodata" : "text", sect->symbol->label); + } + ksdebug(pack, "Aborted. run-pre: could not match some " + "sections.\n"); + return NO_MATCH; + } + return OK; +} + +/* + * Search for the section in the running kernel. Returns OK if and + * only if it finds precisely one address in the kernel matching the + * section. + */ +static abort_t find_section(struct ksplice_pack *pack, + struct ksplice_section *sect) +{ + int i; + abort_t ret; + unsigned long run_addr; + LIST_HEAD(vals); + struct candidate_val *v, *n; + + ret = lookup_symbol(pack, sect->symbol, &vals); + if (ret != OK) { + release_vals(&vals); + return ret; + } + + ksdebug(pack, "run-pre: starting sect search for %s\n", + sect->symbol->label); + + list_for_each_entry_safe(v, n, &vals, list) { + run_addr = v->val; + + yield(); + ret = try_addr(pack, sect, run_addr, NULL, RUN_PRE_INITIAL); + if (ret == NO_MATCH) { + list_del(&v->list); + kfree(v); + } else if (ret != OK) { + release_vals(&vals); + return ret; + } + } + + if (singular(&vals)) { + LIST_HEAD(safety_records); + run_addr = list_entry(vals.next, struct candidate_val, + list)->val; + ret = try_addr(pack, sect, run_addr, &safety_records, + RUN_PRE_FINAL); + release_vals(&vals); + if (ret != OK) { + clear_list(&safety_records, struct safety_record, list); + ksdebug(pack, "run-pre: Final run failed for sect " + "%s:\n", sect->symbol->label); + } else { + list_splice(&safety_records, &pack->safety_records); + } + return ret; + } else if (!list_empty(&vals)) { + struct candidate_val *val; + ksdebug(pack, "run-pre: multiple candidates for sect %s:\n", + sect->symbol->label); + i = 0; + list_for_each_entry(val, &vals, list) { + i++; + ksdebug(pack, "%lx\n", val->val); + if (i > 5) { + ksdebug(pack, "...\n"); + break; + } + } + release_vals(&vals); + return NO_MATCH; + } + release_vals(&vals); + return NO_MATCH; +} + +/* + * try_addr is the the interface to run-pre matching. Its primary + * purpose is to manage debugging information for run-pre matching; + * all the hard work is in run_pre_cmp. + */ +static abort_t try_addr(struct ksplice_pack *pack, + struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode) +{ + abort_t ret; + const struct module *run_module; + + if ((sect->flags & KSPLICE_SECTION_RODATA) != 0 || + (sect->flags & KSPLICE_SECTION_DATA) != 0) + run_module = __module_data_address(run_addr); + else + run_module = __module_text_address(run_addr); + if (run_module == pack->primary) { + ksdebug(pack, "run-pre: unexpected address %lx in primary " + "module %s for sect %s\n", run_addr, run_module->name, + sect->symbol->label); + return UNEXPECTED; + } + if (!patches_module(run_module, pack->target)) { + ksdebug(pack, "run-pre: ignoring address %lx in other module " + "%s for sect %s\n", run_addr, run_module == NULL ? + "vmlinux" : run_module->name, sect->symbol->label); + return NO_MATCH; + } + + ret = create_labelval(pack, sect->symbol, run_addr, TEMP); + if (ret != OK) + return ret; + + ret = run_pre_cmp(pack, sect, run_addr, safety_records, mode); + if (ret == NO_MATCH && mode != RUN_PRE_FINAL) { + set_temp_labelvals(pack, NOVAL); + ksdebug(pack, "run-pre: %s sect %s does not match (r_a=%lx " + "p_a=%lx s=%lx)\n", + (sect->flags & KSPLICE_SECTION_RODATA) != 0 ? "rodata" : + (sect->flags & KSPLICE_SECTION_DATA) != 0 ? "data" : + "text", sect->symbol->label, run_addr, sect->address, + sect->size); + ksdebug(pack, "run-pre: "); + if (pack->update->debug >= 1) { + ret = run_pre_cmp(pack, sect, run_addr, safety_records, + RUN_PRE_DEBUG); + set_temp_labelvals(pack, NOVAL); + } + ksdebug(pack, "\n"); + return ret; + } else if (ret != OK) { + set_temp_labelvals(pack, NOVAL); + return ret; + } + + if (mode != RUN_PRE_FINAL) { + set_temp_labelvals(pack, NOVAL); + ksdebug(pack, "run-pre: candidate for sect %s=%lx\n", + sect->symbol->label, run_addr); + return OK; + } + + set_temp_labelvals(pack, VAL); + ksdebug(pack, "run-pre: found sect %s=%lx\n", sect->symbol->label, + run_addr); + return OK; +} + +/* + * run_pre_cmp is the primary run-pre matching function; it determines + * whether the given ksplice_section matches the code or data in the + * running kernel starting at run_addr. + * + * If run_pre_mode is RUN_PRE_FINAL, a safety record for the matched + * section is created. + * + * The run_pre_mode is also used to determine what debugging + * information to display. + */ +static abort_t run_pre_cmp(struct ksplice_pack *pack, + const struct ksplice_section *sect, + unsigned long run_addr, + struct list_head *safety_records, + enum run_pre_mode mode) +{ + int matched = 0; + abort_t ret; + const struct ksplice_reloc *r, *finger; + const unsigned char *pre, *run, *pre_start, *run_start; + unsigned char runval; + + pre_start = (const unsigned char *)sect->address; + run_start = (const unsigned char *)run_addr; + + finger = init_reloc_search(pack, sect); + + pre = pre_start; + run = run_start; + while (pre < pre_start + sect->size) { + unsigned long offset = pre - pre_start; + ret = lookup_reloc(pack, &finger, (unsigned long)pre, &r); + if (ret == OK) { + ret = handle_reloc(pack, sect, r, (unsigned long)run, + mode); + if (ret != OK) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "reloc in sect does not " + "match after %lx/%lx bytes\n", + offset, sect->size); + return ret; + } + if (mode == RUN_PRE_DEBUG) + print_bytes(pack, run, r->howto->size, pre, + r->howto->size); + pre += r->howto->size; + run += r->howto->size; + finger++; + continue; + } else if (ret != NO_MATCH) { + return ret; + } + + if ((sect->flags & KSPLICE_SECTION_TEXT) != 0) { + ret = handle_paravirt(pack, (unsigned long)pre, + (unsigned long)run, &matched); + if (ret != OK) + return ret; + if (matched != 0) { + if (mode == RUN_PRE_DEBUG) + print_bytes(pack, run, matched, pre, + matched); + pre += matched; + run += matched; + continue; + } + } + + if (probe_kernel_read(&runval, (void *)run, 1) == -EFAULT) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "sect unmapped after %lx/%lx " + "bytes\n", offset, sect->size); + return NO_MATCH; + } + + if (runval != *pre && + (sect->flags & KSPLICE_SECTION_DATA) == 0) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "sect does not match after " + "%lx/%lx bytes\n", offset, sect->size); + if (mode == RUN_PRE_DEBUG) { + print_bytes(pack, run, 1, pre, 1); + ksdebug(pack, "[p_o=%lx] ! ", offset); + print_bytes(pack, run + 1, 2, pre + 1, 2); + } + return NO_MATCH; + } + if (mode == RUN_PRE_DEBUG) + print_bytes(pack, run, 1, pre, 1); + pre++; + run++; + } + return create_safety_record(pack, sect, safety_records, run_addr, + run - run_start); +} + +static void print_bytes(struct ksplice_pack *pack, + const unsigned char *run, int runc, + const unsigned char *pre, int prec) +{ + int o; + int matched = min(runc, prec); + for (o = 0; o < matched; o++) { + if (run[o] == pre[o]) + ksdebug(pack, "%02x ", run[o]); + else + ksdebug(pack, "%02x/%02x ", run[o], pre[o]); + } + for (o = matched; o < runc; o++) + ksdebug(pack, "%02x/ ", run[o]); + for (o = matched; o < prec; o++) + ksdebug(pack, "/%02x ", pre[o]); +} + +struct range { + unsigned long address; + unsigned long size; +}; + +static int reloc_bsearch_compare(const void *key, const void *elt) +{ + const struct range *range = key; + const struct ksplice_reloc *r = elt; + if (range->address + range->size <= r->blank_addr) + return -1; + if (range->address > r->blank_addr) + return 1; + return 0; +} + +static const struct ksplice_reloc *find_reloc(const struct ksplice_reloc *start, + const struct ksplice_reloc *end, + unsigned long address, + unsigned long size) +{ + const struct ksplice_reloc *r; + struct range range = { address, size }; + r = bsearch((void *)&range, start, end - start, sizeof(*r), + reloc_bsearch_compare); + if (r == NULL) + return NULL; + while (r > start && (r - 1)->blank_addr >= address) + r--; + return r; +} + +static const struct ksplice_reloc * +init_reloc_search(struct ksplice_pack *pack, const struct ksplice_section *sect) +{ + const struct ksplice_reloc *r; + r = find_reloc(pack->helper_relocs, pack->helper_relocs_end, + sect->address, sect->size); + if (r == NULL) + return pack->helper_relocs_end; + return r; +} + +/* + * lookup_reloc implements an amortized O(1) lookup for the next + * helper relocation. It must be called with a strictly increasing + * sequence of addresses. + * + * The fingerp is private data for lookup_reloc, and needs to have + * been initialized as a pointer to the result of find_reloc (or + * init_reloc_search). + */ +static abort_t lookup_reloc(struct ksplice_pack *pack, + const struct ksplice_reloc **fingerp, + unsigned long addr, + const struct ksplice_reloc **relocp) +{ + const struct ksplice_reloc *r = *fingerp; + int canary_ret; + + while (r < pack->helper_relocs_end && + addr >= r->blank_addr + r->howto->size && + !(addr == r->blank_addr && r->howto->size == 0)) + r++; + *fingerp = r; + if (r == pack->helper_relocs_end) + return NO_MATCH; + if (addr < r->blank_addr) + return NO_MATCH; + *relocp = r; + if (r->howto->type != KSPLICE_HOWTO_RELOC) + return OK; + + canary_ret = contains_canary(pack, r->blank_addr, r->howto); + if (canary_ret < 0) + return UNEXPECTED; + if (canary_ret == 0) { + ksdebug(pack, "run-pre: reloc skipped at p_a=%lx to %s+%lx " + "(altinstr)\n", r->blank_addr, r->symbol->label, + r->target_addend); + return NO_MATCH; + } + if (addr != r->blank_addr) { + ksdebug(pack, "Invalid nonzero relocation offset\n"); + return UNEXPECTED; + } + return OK; +} + +static abort_t handle_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, enum run_pre_mode mode) +{ + switch (r->howto->type) { + case KSPLICE_HOWTO_RELOC: + return handle_howto_reloc(pack, sect, r, run_addr, mode); + case KSPLICE_HOWTO_DATE: + case KSPLICE_HOWTO_TIME: + return handle_howto_date(pack, sect, r, run_addr, mode); + case KSPLICE_HOWTO_BUG: + return handle_bug(pack, r, run_addr); + case KSPLICE_HOWTO_EXTABLE: + return handle_extable(pack, r, run_addr); + default: + ksdebug(pack, "Unexpected howto type %d\n", r->howto->type); + return UNEXPECTED; + } +} + +/* + * For date/time relocations, we check that the sequence of bytes + * matches the format of a date or time. + */ +static abort_t handle_howto_date(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, enum run_pre_mode mode) +{ + abort_t ret; + char *buf = kmalloc(r->howto->size, GFP_KERNEL); + + if (buf == NULL) + return OUT_OF_MEMORY; + if (probe_kernel_read(buf, (void *)run_addr, r->howto->size) == -EFAULT) { + ret = NO_MATCH; + goto out; + } + + switch (r->howto->type) { + case KSPLICE_HOWTO_TIME: + if (isdigit(buf[0]) && isdigit(buf[1]) && buf[2] == ':' && + isdigit(buf[3]) && isdigit(buf[4]) && buf[5] == ':' && + isdigit(buf[6]) && isdigit(buf[7])) + ret = OK; + else + ret = NO_MATCH; + break; + case KSPLICE_HOWTO_DATE: + if (isalpha(buf[0]) && isalpha(buf[1]) && isalpha(buf[2]) && + buf[3] == ' ' && (buf[4] == ' ' || isdigit(buf[4])) && + isdigit(buf[5]) && buf[6] == ' ' && isdigit(buf[7]) && + isdigit(buf[8]) && isdigit(buf[9]) && isdigit(buf[10])) + ret = OK; + else + ret = NO_MATCH; + break; + default: + ret = UNEXPECTED; + } + if (ret == NO_MATCH && mode == RUN_PRE_INITIAL) + ksdebug(pack, "%s string: \"%.*s\" does not match format\n", + r->howto->type == KSPLICE_HOWTO_DATE ? "date" : "time", + r->howto->size, buf); + + if (ret != OK) + goto out; + ret = create_labelval(pack, r->symbol, run_addr, TEMP); +out: + kfree(buf); + return ret; +} + +/* + * Extract the value of a symbol used in a relocation in the pre code + * during run-pre matching, giving an error if it conflicts with a + * previously found value of that symbol + */ +static abort_t handle_howto_reloc(struct ksplice_pack *pack, + const struct ksplice_section *sect, + const struct ksplice_reloc *r, + unsigned long run_addr, + enum run_pre_mode mode) +{ + struct ksplice_section *sym_sect = symbol_section(pack, r->symbol); + unsigned long offset = r->target_addend; + unsigned long val; + abort_t ret; + + ret = read_reloc_value(pack, r, run_addr, &val); + if (ret != OK) + return ret; + if (r->howto->pcrel) + val += run_addr; + + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "run-pre: reloc at r_a=%lx p_a=%lx to %s+%lx: " + "found %s = %lx\n", run_addr, r->blank_addr, + r->symbol->label, offset, r->symbol->label, val); + + if (contains_canary(pack, run_addr, r->howto) != 0) { + ksdebug(pack, "Aborted. Unexpected canary in run code at %lx" + "\n", run_addr); + return UNEXPECTED; + } + + if ((sect->flags & KSPLICE_SECTION_DATA) != 0 && + sect->symbol == r->symbol) + return OK; + ret = create_labelval(pack, r->symbol, val, TEMP); + if (ret == NO_MATCH && mode == RUN_PRE_INITIAL) + ksdebug(pack, "run-pre: reloc at r_a=%lx p_a=%lx: labelval %s " + "= %lx does not match expected %lx\n", run_addr, + r->blank_addr, r->symbol->label, r->symbol->value, val); + + if (ret != OK) + return ret; + if (sym_sect != NULL && (sym_sect->flags & KSPLICE_SECTION_MATCHED) == 0 + && (sym_sect->flags & KSPLICE_SECTION_STRING) != 0) { + if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "Recursively comparing string section " + "%s\n", sym_sect->symbol->label); + else if (mode == RUN_PRE_DEBUG) + ksdebug(pack, "[str start] "); + ret = run_pre_cmp(pack, sym_sect, val, NULL, mode); + if (mode == RUN_PRE_DEBUG) + ksdebug(pack, "[str end] "); + if (ret == OK && mode == RUN_PRE_INITIAL) + ksdebug(pack, "Successfully matched string section %s" + "\n", sym_sect->symbol->label); + else if (mode == RUN_PRE_INITIAL) + ksdebug(pack, "Failed to match string section %s\n", + sym_sect->symbol->label); + } + return ret; +} + +static int symbol_section_bsearch_compare(const void *a, const void *b) +{ + const struct ksplice_symbol *sym = a; + const struct ksplice_section *sect = b; + return strcmp(sym->label, sect->symbol->label); +} + +static int compare_section_labels(const void *va, const void *vb) +{ + const struct ksplice_section *a = va, *b = vb; + return strcmp(a->symbol->label, b->symbol->label); +} + +static struct ksplice_section *symbol_section(struct ksplice_pack *pack, + const struct ksplice_symbol *sym) +{ + return bsearch(sym, pack->helper_sections, pack->helper_sections_end - + pack->helper_sections, sizeof(struct ksplice_section), + symbol_section_bsearch_compare); +} + +/* Find the relocation for the oldaddr of a ksplice_patch */ +static const struct ksplice_reloc *patch_reloc(struct ksplice_pack *pack, + const struct ksplice_patch *p) +{ + unsigned long addr = (unsigned long)&p->oldaddr; + const struct ksplice_reloc *r = + find_reloc(pack->primary_relocs, pack->primary_relocs_end, addr, + sizeof(addr)); + if (r == NULL || r->blank_addr < addr || + r->blank_addr >= addr + sizeof(addr)) + return NULL; + return r; +} + +/* + * Populates vals with the possible values for ksym from the various + * sources Ksplice uses to resolve symbols + */ +static abort_t lookup_symbol(struct ksplice_pack *pack, + const struct ksplice_symbol *ksym, + struct list_head *vals) +{ + abort_t ret; + + if (ksym->vals == NULL) { + release_vals(vals); + ksdebug(pack, "using detected sym %s=%lx\n", ksym->label, + ksym->value); + return add_candidate_val(pack, vals, ksym->value); + } + + if (strcmp(ksym->label, "cleanup_module") == 0 && pack->target != NULL + && pack->target->exit != NULL) { + ret = add_candidate_val(pack, vals, + (unsigned long)pack->target->exit); + if (ret != OK) + return ret; + } + + if (ksym->name != NULL) { + struct candidate_val *val; + list_for_each_entry(val, ksym->vals, list) { + ret = add_candidate_val(pack, vals, val->val); + if (ret != OK) + return ret; + } + + ret = new_export_lookup(pack, ksym->name, vals); + if (ret != OK) + return ret; + } + + return OK; +} + +/* + * An update could one module to export a symbol and at the same time + * change another module to use that symbol. This violates the normal + * situation where the packs can be handled independently. + * + * new_export_lookup obtains symbol values from the changes to the + * exported symbol table made by other packs. + */ +static abort_t new_export_lookup(struct ksplice_pack *ipack, const char *name, + struct list_head *vals) +{ + struct ksplice_pack *pack; + struct ksplice_patch *p; + list_for_each_entry(pack, &ipack->update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) { + const struct kernel_symbol *sym; + const struct ksplice_reloc *r; + if (p->type != KSPLICE_PATCH_EXPORT || + strcmp(name, *(const char **)p->contents) != 0) + continue; + + /* Check that the p->oldaddr reloc has been resolved. */ + r = patch_reloc(pack, p); + if (r == NULL || + contains_canary(pack, r->blank_addr, r->howto) != 0) + continue; + sym = (const struct kernel_symbol *)r->symbol->value; + + /* + * Check that the sym->value reloc has been resolved, + * if there is a Ksplice relocation there. + */ + r = find_reloc(pack->primary_relocs, + pack->primary_relocs_end, + (unsigned long)&sym->value, + sizeof(&sym->value)); + if (r != NULL && + r->blank_addr == (unsigned long)&sym->value && + contains_canary(pack, r->blank_addr, r->howto) != 0) + continue; + return add_candidate_val(ipack, vals, sym->value); + } + } + return OK; +} + +/* + * When apply_patches is called, the update should be fully prepared. + * apply_patches will try to actually insert trampolines for the + * update. + */ +static abort_t apply_patches(struct update *update) +{ + int i; + abort_t ret; + struct ksplice_pack *pack; + + ret = map_trampoline_pages(update); + if (ret != OK) + return ret; + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->pre_apply; f < pack->pre_apply_end; f++) { + if ((*f)() != 0) { + ret = CALL_FAILED; + goto out; + } + } + } + + for (i = 0; i < 5; i++) { + cleanup_conflicts(update); + ret = (__force abort_t)stop_machine(__apply_patches, update, + NULL); + if (ret != CODE_BUSY) + break; + set_current_state(TASK_INTERRUPTIBLE); + schedule_timeout(msecs_to_jiffies(1000)); + } +out: + unmap_trampoline_pages(update); + + if (ret == CODE_BUSY) { + print_conflicts(update); + _ksdebug(update, "Aborted %s. stack check: to-be-replaced " + "code is busy.\n", update->kid); + } else if (ret == ALREADY_REVERSED) { + _ksdebug(update, "Aborted %s. Ksplice update %s is already " + "reversed.\n", update->kid, update->kid); + } + + if (ret != OK) { + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->fail_apply; f < pack->fail_apply_end; + f++) + (*f)(); + } + + return ret; + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->post_apply; f < pack->post_apply_end; f++) + (*f)(); + } + + _ksdebug(update, "Atomic patch insertion for %s complete\n", + update->kid); + return OK; +} + +static abort_t reverse_patches(struct update *update) +{ + int i; + abort_t ret; + struct ksplice_pack *pack; + + clear_debug_buf(update); + ret = init_debug_buf(update); + if (ret != OK) + return ret; + + _ksdebug(update, "Preparing to reverse %s\n", update->kid); + + ret = map_trampoline_pages(update); + if (ret != OK) + return ret; + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->pre_reverse; f < pack->pre_reverse_end; f++) { + if ((*f)() != 0) { + ret = CALL_FAILED; + goto out; + } + } + } + + for (i = 0; i < 5; i++) { + cleanup_conflicts(update); + clear_list(&update->conflicts, struct conflict, list); + ret = (__force abort_t)stop_machine(__reverse_patches, update, + NULL); + if (ret != CODE_BUSY) + break; + set_current_state(TASK_INTERRUPTIBLE); + schedule_timeout(msecs_to_jiffies(1000)); + } +out: + unmap_trampoline_pages(update); + + if (ret == CODE_BUSY) { + print_conflicts(update); + _ksdebug(update, "Aborted %s. stack check: to-be-reversed " + "code is busy.\n", update->kid); + } else if (ret == MODULE_BUSY) { + _ksdebug(update, "Update %s is in use by another module\n", + update->kid); + } + + if (ret != OK) { + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->fail_reverse; f < pack->fail_reverse_end; + f++) + (*f)(); + } + + return ret; + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->post_reverse; f < pack->post_reverse_end; f++) + (*f)(); + } + + list_for_each_entry(pack, &update->packs, list) + clear_list(&pack->safety_records, struct safety_record, list); + + _ksdebug(update, "Atomic patch removal for %s complete\n", update->kid); + return OK; +} + +/* Atomically insert the update; run from within stop_machine */ +static int __apply_patches(void *updateptr) +{ + struct update *update = updateptr; + struct ksplice_pack *pack; + struct ksplice_patch *p; + abort_t ret; + + if (update->stage == STAGE_APPLIED) + return (__force int)OK; + + if (update->stage != STAGE_PREPARING) + return (__force int)UNEXPECTED; + + ret = check_each_task(update); + if (ret != OK) + return (__force int)ret; + + list_for_each_entry(pack, &update->packs, list) { + if (try_module_get(pack->primary) != 1) { + struct ksplice_pack *pack1; + list_for_each_entry(pack1, &update->packs, list) { + if (pack1 == pack) + break; + module_put(pack1->primary); + } + module_put(THIS_MODULE); + return (__force int)UNEXPECTED; + } + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->check_apply; f < pack->check_apply_end; f++) + if ((*f)() != 0) + return (__force int)CALL_FAILED; + } + + /* Commit point: the update application will succeed. */ + + update->stage = STAGE_APPLIED; + add_taint(TAINT_KSPLICE); + + list_for_each_entry(pack, &update->packs, list) + list_add(&pack->module_list_entry.list, &ksplice_module_list); + + list_for_each_entry(pack, &update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) + insert_trampoline(p); + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->apply; f < pack->apply_end; f++) + (*f)(); + } + + return (__force int)OK; +} + +/* Atomically remove the update; run from within stop_machine */ +static int __reverse_patches(void *updateptr) +{ + struct update *update = updateptr; + struct ksplice_pack *pack; + const struct ksplice_patch *p; + abort_t ret; + + if (update->stage != STAGE_APPLIED) + return (__force int)OK; + + list_for_each_entry(pack, &update->packs, list) { + if (module_refcount(pack->primary) != 1) + return (__force int)MODULE_BUSY; + } + + ret = check_each_task(update); + if (ret != OK) + return (__force int)ret; + + list_for_each_entry(pack, &update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) { + ret = verify_trampoline(pack, p); + if (ret != OK) + return (__force int)ret; + } + } + + list_for_each_entry(pack, &update->packs, list) { + const typeof(int (*)(void)) *f; + for (f = pack->check_reverse; f < pack->check_reverse_end; f++) + if ((*f)() != 0) + return (__force int)CALL_FAILED; + } + + /* Commit point: the update reversal will succeed. */ + + update->stage = STAGE_REVERSED; + + list_for_each_entry(pack, &update->packs, list) + module_put(pack->primary); + + list_for_each_entry(pack, &update->packs, list) + list_del(&pack->module_list_entry.list); + + list_for_each_entry(pack, &update->packs, list) { + const typeof(void (*)(void)) *f; + for (f = pack->reverse; f < pack->reverse_end; f++) + (*f)(); + } + + list_for_each_entry(pack, &update->packs, list) { + for (p = pack->patches; p < pack->patches_end; p++) + remove_trampoline(p); + } + + return (__force int)OK; +} + +/* + * Check whether any thread's instruction pointer or any address of + * its stack is contained in one of the safety_records associated with + * the update. + * + * check_each_task must be called from inside stop_machine, because it + * does not take tasklist_lock (which cannot be held by anyone else + * during stop_machine). + */ +static abort_t check_each_task(struct update *update) +{ + const struct task_struct *g, *p; + abort_t status = OK, ret; + do_each_thread(g, p) { + /* do_each_thread is a double loop! */ + ret = check_task(update, p, false); + if (ret != OK) { + check_task(update, p, true); + status = ret; + } + if (ret != OK && ret != CODE_BUSY) + return ret; + } while_each_thread(g, p); + return status; +} + +static abort_t check_task(struct update *update, + const struct task_struct *t, bool rerun) +{ + abort_t status, ret; + struct conflict *conf = NULL; + + if (rerun) { + conf = kmalloc(sizeof(*conf), GFP_ATOMIC); + if (conf == NULL) + return OUT_OF_MEMORY; + conf->process_name = kstrdup(t->comm, GFP_ATOMIC); + if (conf->process_name == NULL) { + kfree(conf); + return OUT_OF_MEMORY; + } + conf->pid = t->pid; + INIT_LIST_HEAD(&conf->stack); + list_add(&conf->list, &update->conflicts); + } + + status = check_address(update, conf, KSPLICE_IP(t)); + if (t == current) { + ret = check_stack(update, conf, task_thread_info(t), + (unsigned long *)__builtin_frame_address(0)); + if (status == OK) + status = ret; + } else if (!task_curr(t)) { + ret = check_stack(update, conf, task_thread_info(t), + (unsigned long *)KSPLICE_SP(t)); + if (status == OK) + status = ret; + } else if (!is_stop_machine(t)) { + status = UNEXPECTED_RUNNING_TASK; + } + return status; +} + +static abort_t check_stack(struct update *update, struct conflict *conf, + const struct thread_info *tinfo, + const unsigned long *stack) +{ + abort_t status = OK, ret; + unsigned long addr; + + while (valid_stack_ptr(tinfo, stack)) { + addr = *stack++; + ret = check_address(update, conf, addr); + if (ret != OK) + status = ret; + } + return status; +} + +static abort_t check_address(struct update *update, + struct conflict *conf, unsigned long addr) +{ + abort_t status = OK, ret; + const struct safety_record *rec; + struct ksplice_pack *pack; + struct conflict_addr *ca = NULL; + + if (conf != NULL) { + ca = kmalloc(sizeof(*ca), GFP_ATOMIC); + if (ca == NULL) + return OUT_OF_MEMORY; + ca->addr = addr; + ca->has_conflict = false; + ca->label = NULL; + list_add(&ca->list, &conf->stack); + } + + list_for_each_entry(pack, &update->packs, list) { + unsigned long tramp_addr = follow_trampolines(pack, addr); + list_for_each_entry(rec, &pack->safety_records, list) { + ret = check_record(ca, rec, tramp_addr); + if (ret != OK) + status = ret; + } + } + return status; +} + +static abort_t check_record(struct conflict_addr *ca, + const struct safety_record *rec, unsigned long addr) +{ + if (addr >= rec->addr && addr < rec->addr + rec->size) { + if (ca != NULL) { + ca->label = rec->label; + ca->has_conflict = true; + } + return CODE_BUSY; + } + return OK; +} + +/* Is the task one of the stop_machine tasks? */ +static bool is_stop_machine(const struct task_struct *t) +{ + const char *kstop_prefix = "kstop/"; + const char *num; + if (!starts_with(t->comm, kstop_prefix)) + return false; + num = t->comm + strlen(kstop_prefix); + return num[strspn(num, "0123456789")] == '\0'; +} + +static void cleanup_conflicts(struct update *update) +{ + struct conflict *conf; + list_for_each_entry(conf, &update->conflicts, list) { + clear_list(&conf->stack, struct conflict_addr, list); + kfree(conf->process_name); + } + clear_list(&update->conflicts, struct conflict, list); +} + +static void print_conflicts(struct update *update) +{ + const struct conflict *conf; + const struct conflict_addr *ca; + list_for_each_entry(conf, &update->conflicts, list) { + _ksdebug(update, "stack check: pid %d (%s):", conf->pid, + conf->process_name); + list_for_each_entry(ca, &conf->stack, list) { + _ksdebug(update, " %lx", ca->addr); + if (ca->has_conflict) + _ksdebug(update, " [<-CONFLICT]"); + } + _ksdebug(update, "\n"); + } +} + +static void insert_trampoline(struct ksplice_patch *p) +{ + mm_segment_t old_fs = get_fs(); + set_fs(KERNEL_DS); + memcpy(p->saved, p->vaddr, p->size); + memcpy(p->vaddr, p->contents, p->size); + flush_icache_range(p->oldaddr, p->oldaddr + p->size); + set_fs(old_fs); +} + +static abort_t verify_trampoline(struct ksplice_pack *pack, + const struct ksplice_patch *p) +{ + if (memcmp(p->vaddr, p->contents, p->size) != 0) { + ksdebug(pack, "Aborted. Trampoline at %lx has been " + "overwritten.\n", p->oldaddr); + return CODE_BUSY; + } + return OK; +} + +static void remove_trampoline(const struct ksplice_patch *p) +{ + mm_segment_t old_fs = get_fs(); + set_fs(KERNEL_DS); + memcpy(p->vaddr, p->saved, p->size); + flush_icache_range(p->oldaddr, p->oldaddr + p->size); + set_fs(old_fs); +} + +/* Returns NO_MATCH if there's already a labelval with a different value */ +static abort_t create_labelval(struct ksplice_pack *pack, + struct ksplice_symbol *ksym, + unsigned long val, int status) +{ + val = follow_trampolines(pack, val); + if (ksym->vals == NULL) + return ksym->value == val ? OK : NO_MATCH; + + ksym->value = val; + if (status == TEMP) { + struct labelval *lv = kmalloc(sizeof(*lv), GFP_KERNEL); + if (lv == NULL) + return OUT_OF_MEMORY; + lv->symbol = ksym; + lv->saved_vals = ksym->vals; + list_add(&lv->list, &pack->temp_labelvals); + } + ksym->vals = NULL; + return OK; +} + +/* + * Creates a new safety_record for a helper section based on its + * ksplice_section and run-pre matching information. + */ +static abort_t create_safety_record(struct ksplice_pack *pack, + const struct ksplice_section *sect, + struct list_head *record_list, + unsigned long run_addr, + unsigned long run_size) +{ + struct safety_record *rec; + struct ksplice_patch *p; + + if (record_list == NULL) + return OK; + + for (p = pack->patches; p < pack->patches_end; p++) { + const struct ksplice_reloc *r = patch_reloc(pack, p); + if (strcmp(sect->symbol->label, r->symbol->label) == 0) + break; + } + if (p >= pack->patches_end) + return OK; + + rec = kmalloc(sizeof(*rec), GFP_KERNEL); + if (rec == NULL) + return OUT_OF_MEMORY; + /* + * The helper might be unloaded when checking reversing + * patches, so we need to kstrdup the label here. + */ + rec->label = kstrdup(sect->symbol->label, GFP_KERNEL); + if (rec->label == NULL) { + kfree(rec); + return OUT_OF_MEMORY; + } + rec->addr = run_addr; + rec->size = run_size; + + list_add(&rec->list, record_list); + return OK; +} + +static abort_t add_candidate_val(struct ksplice_pack *pack, + struct list_head *vals, unsigned long val) +{ + struct candidate_val *tmp, *new; + +/* + * Careful: follow trampolines before comparing values so that we do + * not mistake the obsolete function for another copy of the function. + */ + val = follow_trampolines(pack, val); + + list_for_each_entry(tmp, vals, list) { + if (tmp->val == val) + return OK; + } + new = kmalloc(sizeof(*new), GFP_KERNEL); + if (new == NULL) + return OUT_OF_MEMORY; + new->val = val; + list_add(&new->list, vals); + return OK; +} + +static void release_vals(struct list_head *vals) +{ + clear_list(vals, struct candidate_val, list); +} + +/* + * The temp_labelvals list is used to cache those temporary labelvals + * that have been created to cross-check the symbol values obtained + * from different relocations within a single section being matched. + * + * If status is VAL, commit the temp_labelvals as final values. + * + * If status is NOVAL, restore the list of possible values to the + * ksplice_symbol, so that it no longer has a known value. + */ +static void set_temp_labelvals(struct ksplice_pack *pack, int status) +{ + struct labelval *lv, *n; + list_for_each_entry_safe(lv, n, &pack->temp_labelvals, list) { + if (status == NOVAL) { + lv->symbol->vals = lv->saved_vals; + } else { + release_vals(lv->saved_vals); + kfree(lv->saved_vals); + } + list_del(&lv->list); + kfree(lv); + } +} + +/* Is there a Ksplice canary with given howto at blank_addr? */ +static int contains_canary(struct ksplice_pack *pack, unsigned long blank_addr, + const struct ksplice_reloc_howto *howto) +{ + switch (howto->size) { + case 1: + return (*(uint8_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); + case 2: + return (*(uint16_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); + case 4: + return (*(uint32_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); +#if BITS_PER_LONG >= 64 + case 8: + return (*(uint64_t *)blank_addr & howto->dst_mask) == + (KSPLICE_CANARY & howto->dst_mask); +#endif /* BITS_PER_LONG */ + default: + ksdebug(pack, "Aborted. Invalid relocation size.\n"); + return -1; + } +} + +/* + * Compute the address of the code you would actually run if you were + * to call the function at addr (i.e., follow the sequence of jumps + * starting at addr) + */ +static unsigned long follow_trampolines(struct ksplice_pack *pack, + unsigned long addr) +{ + unsigned long new_addr; + struct module *m; + + while (1) { + if (!__kernel_text_address(addr) || + trampoline_target(pack, addr, &new_addr) != OK) + return addr; + m = __module_text_address(new_addr); + if (m == NULL || m == pack->target || + !starts_with(m->name, "ksplice")) + return addr; + addr = new_addr; + } +} + +/* Does module a patch module b? */ +static bool patches_module(const struct module *a, const struct module *b) +{ + struct ksplice_module_list_entry *entry; + if (a == b) + return true; + list_for_each_entry(entry, &ksplice_module_list, list) { + if (entry->target == b && entry->primary == a) + return true; + } + return false; +} + +static bool starts_with(const char *str, const char *prefix) +{ + return strncmp(str, prefix, strlen(prefix)) == 0; +} + +static bool singular(struct list_head *list) +{ + return !list_empty(list) && list->next->next == list; +} + +static void *bsearch(const void *key, const void *base, size_t n, + size_t size, int (*cmp)(const void *key, const void *elt)) +{ + int start = 0, end = n - 1, mid, result; + if (n == 0) + return NULL; + while (start <= end) { + mid = (start + end) / 2; + result = cmp(key, base + mid * size); + if (result < 0) + end = mid - 1; + else if (result > 0) + start = mid + 1; + else + return (void *)base + mid * size; + } + return NULL; +} + +static int compare_relocs(const void *a, const void *b) +{ + const struct ksplice_reloc *ra = a, *rb = b; + if (ra->blank_addr > rb->blank_addr) + return 1; + else if (ra->blank_addr < rb->blank_addr) + return -1; + else + return ra->howto->size - rb->howto->size; +} + +#ifdef CONFIG_DEBUG_FS + +static abort_t init_debug_buf(struct update *update) +{ + update->debug_blob.size = 0; + update->debug_blob.data = NULL; + update->debugfs_dentry = + debugfs_create_blob(update->name, S_IFREG | S_IRUSR, NULL, + &update->debug_blob); + if (update->debugfs_dentry == NULL) + return OUT_OF_MEMORY; + return OK; +} + +static void clear_debug_buf(struct update *update) +{ + if (update->debugfs_dentry == NULL) + return; + debugfs_remove(update->debugfs_dentry); + update->debugfs_dentry = NULL; + update->debug_blob.size = 0; + vfree(update->debug_blob.data); + update->debug_blob.data = NULL; +} + +static int _ksdebug(struct update *update, const char *fmt, ...) +{ + va_list args; + unsigned long size, old_size, new_size; + + if (update->debug == 0) + return 0; + + /* size includes the trailing '\0' */ + va_start(args, fmt); + size = 1 + vsnprintf(update->debug_blob.data, 0, fmt, args); + va_end(args); + old_size = update->debug_blob.size == 0 ? 0 : + max(PAGE_SIZE, roundup_pow_of_two(update->debug_blob.size)); + new_size = update->debug_blob.size + size == 0 ? 0 : + max(PAGE_SIZE, roundup_pow_of_two(update->debug_blob.size + size)); + if (new_size > old_size) { + char *buf = vmalloc(new_size); + if (buf == NULL) + return -ENOMEM; + memcpy(buf, update->debug_blob.data, update->debug_blob.size); + vfree(update->debug_blob.data); + update->debug_blob.data = buf; + } + va_start(args, fmt); + update->debug_blob.size += vsnprintf(update->debug_blob.data + + update->debug_blob.size, + size, fmt, args); + va_end(args); + return 0; +} +#else /* CONFIG_DEBUG_FS */ +static abort_t init_debug_buf(struct update *update) +{ + return OK; +} + +static void clear_debug_buf(struct update *update) +{ + return; +} + +static int _ksdebug(struct update *update, const char *fmt, ...) +{ + va_list args; + + if (update->debug == 0) + return 0; + + if (!update->debug_continue_line) + printk(KERN_DEBUG "ksplice: "); + + va_start(args, fmt); + vprintk(fmt, args); + va_end(args); + + update->debug_continue_line = + fmt[0] == '\0' || fmt[strlen(fmt) - 1] != '\n'; + return 0; +} +#endif /* CONFIG_DEBUG_FS */ + +struct ksplice_attribute { + struct attribute attr; + ssize_t (*show)(struct update *update, char *buf); + ssize_t (*store)(struct update *update, const char *buf, size_t len); +}; + +static ssize_t ksplice_attr_show(struct kobject *kobj, struct attribute *attr, + char *buf) +{ + struct ksplice_attribute *attribute = + container_of(attr, struct ksplice_attribute, attr); + struct update *update = container_of(kobj, struct update, kobj); + if (attribute->show == NULL) + return -EIO; + return attribute->show(update, buf); +} + +static ssize_t ksplice_attr_store(struct kobject *kobj, struct attribute *attr, + const char *buf, size_t len) +{ + struct ksplice_attribute *attribute = + container_of(attr, struct ksplice_attribute, attr); + struct update *update = container_of(kobj, struct update, kobj); + if (attribute->store == NULL) + return -EIO; + return attribute->store(update, buf, len); +} + +static struct sysfs_ops ksplice_sysfs_ops = { + .show = ksplice_attr_show, + .store = ksplice_attr_store, +}; + +static void ksplice_release(struct kobject *kobj) +{ + struct update *update; + update = container_of(kobj, struct update, kobj); + cleanup_ksplice_update(update); +} + +static ssize_t stage_show(struct update *update, char *buf) +{ + switch (update->stage) { + case STAGE_PREPARING: + return snprintf(buf, PAGE_SIZE, "preparing\n"); + case STAGE_APPLIED: + return snprintf(buf, PAGE_SIZE, "applied\n"); + case STAGE_REVERSED: + return snprintf(buf, PAGE_SIZE, "reversed\n"); + } + return 0; +} + +static ssize_t abort_cause_show(struct update *update, char *buf) +{ + switch (update->abort_cause) { + case OK: + return snprintf(buf, PAGE_SIZE, "ok\n"); + case NO_MATCH: + return snprintf(buf, PAGE_SIZE, "no_match\n"); + case CODE_BUSY: + return snprintf(buf, PAGE_SIZE, "code_busy\n"); + case MODULE_BUSY: + return snprintf(buf, PAGE_SIZE, "module_busy\n"); + case OUT_OF_MEMORY: + return snprintf(buf, PAGE_SIZE, "out_of_memory\n"); + case FAILED_TO_FIND: + return snprintf(buf, PAGE_SIZE, "failed_to_find\n"); + case ALREADY_REVERSED: + return snprintf(buf, PAGE_SIZE, "already_reversed\n"); + case MISSING_EXPORT: + return snprintf(buf, PAGE_SIZE, "missing_export\n"); + case UNEXPECTED_RUNNING_TASK: + return snprintf(buf, PAGE_SIZE, "unexpected_running_task\n"); + case TARGET_NOT_LOADED: + return snprintf(buf, PAGE_SIZE, "target_not_loaded\n"); + case CALL_FAILED: + return snprintf(buf, PAGE_SIZE, "call_failed\n"); + case UNEXPECTED: + return snprintf(buf, PAGE_SIZE, "unexpected\n"); + default: + return snprintf(buf, PAGE_SIZE, "unknown\n"); + } + return 0; +} + +static ssize_t conflict_show(struct update *update, char *buf) +{ + const struct conflict *conf; + const struct conflict_addr *ca; + int used = 0; + mutex_lock(&module_mutex); + list_for_each_entry(conf, &update->conflicts, list) { + used += snprintf(buf + used, PAGE_SIZE - used, "%s %d", + conf->process_name, conf->pid); + list_for_each_entry(ca, &conf->stack, list) { + if (!ca->has_conflict) + continue; + used += snprintf(buf + used, PAGE_SIZE - used, " %s", + ca->label); + } + used += snprintf(buf + used, PAGE_SIZE - used, "\n"); + } + mutex_unlock(&module_mutex); + return used; +} + +/* Used to pass maybe_cleanup_ksplice_update to kthread_run */ +static int maybe_cleanup_ksplice_update_wrapper(void *updateptr) +{ + struct update *update = updateptr; + mutex_lock(&module_mutex); + maybe_cleanup_ksplice_update(update); + mutex_unlock(&module_mutex); + return 0; +} + +static ssize_t stage_store(struct update *update, const char *buf, size_t len) +{ + enum stage old_stage; + mutex_lock(&module_mutex); + old_stage = update->stage; + if ((strncmp(buf, "applied", len) == 0 || + strncmp(buf, "applied\n", len) == 0) && + update->stage == STAGE_PREPARING) + update->abort_cause = apply_update(update); + else if ((strncmp(buf, "reversed", len) == 0 || + strncmp(buf, "reversed\n", len) == 0) && + update->stage == STAGE_APPLIED) + update->abort_cause = reverse_patches(update); + else if ((strncmp(buf, "cleanup", len) == 0 || + strncmp(buf, "cleanup\n", len) == 0) && + update->stage == STAGE_REVERSED) + kthread_run(maybe_cleanup_ksplice_update_wrapper, update, + "ksplice_cleanup_%s", update->kid); + + if (old_stage != STAGE_REVERSED && update->abort_cause == OK) + printk(KERN_INFO "ksplice: Update %s %s successfully\n", + update->kid, + update->stage == STAGE_APPLIED ? "applied" : "reversed"); + mutex_unlock(&module_mutex); + return len; +} + +static ssize_t debug_show(struct update *update, char *buf) +{ + return snprintf(buf, PAGE_SIZE, "%d\n", update->debug); +} + +static ssize_t debug_store(struct update *update, const char *buf, size_t len) +{ + unsigned long l; + int ret = strict_strtoul(buf, 10, &l); + if (ret != 0) + return ret; + update->debug = l; + return len; +} + +static ssize_t partial_show(struct update *update, char *buf) +{ + return snprintf(buf, PAGE_SIZE, "%d\n", update->partial); +} + +static ssize_t partial_store(struct update *update, const char *buf, size_t len) +{ + unsigned long l; + int ret = strict_strtoul(buf, 10, &l); + if (ret != 0) + return ret; + update->partial = l; + return len; +} + +static struct ksplice_attribute stage_attribute = + __ATTR(stage, 0600, stage_show, stage_store); +static struct ksplice_attribute abort_cause_attribute = + __ATTR(abort_cause, 0400, abort_cause_show, NULL); +static struct ksplice_attribute debug_attribute = + __ATTR(debug, 0600, debug_show, debug_store); +static struct ksplice_attribute partial_attribute = + __ATTR(partial, 0600, partial_show, partial_store); +static struct ksplice_attribute conflict_attribute = + __ATTR(conflicts, 0400, conflict_show, NULL); + +static struct attribute *ksplice_attrs[] = { + &stage_attribute.attr, + &abort_cause_attribute.attr, + &debug_attribute.attr, + &partial_attribute.attr, + &conflict_attribute.attr, + NULL +}; + +static struct kobj_type ksplice_ktype = { + .sysfs_ops = &ksplice_sysfs_ops, + .release = ksplice_release, + .default_attrs = ksplice_attrs, +}; + +static int init_ksplice(void) +{ + ksplice_kobj = kobject_create_and_add("ksplice", kernel_kobj); + if (ksplice_kobj == NULL) + return -ENOMEM; + return 0; +} + +static void cleanup_ksplice(void) +{ + kobject_put(ksplice_kobj); +} + +module_init(init_ksplice); +module_exit(cleanup_ksplice); + +MODULE_AUTHOR("Jeff Arnold <jbarnold@mit.edu>"); +MODULE_DESCRIPTION("Ksplice rebootless update system"); +MODULE_LICENSE("GPL v2"); diff --git a/kernel/panic.c b/kernel/panic.c index 4d50883..f2ca7bf 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -155,6 +155,7 @@ static const struct tnt tnts[] = { { TAINT_OVERRIDDEN_ACPI_TABLE, 'A', ' ' }, { TAINT_WARN, 'W', ' ' }, { TAINT_CRAP, 'C', ' ' }, + { TAINT_KSPLICE, 'K', ' ' }, }; /** -- 1.5.6.5 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 2008-12-06 0:04 ` [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 Jeff Arnold 2008-12-17 5:41 ` Theodore Tso @ 2009-02-07 2:36 ` Rusty Russell 2009-02-10 1:01 ` Tim Abbott 1 sibling, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-07 2:36 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Saturday 06 December 2008 10:34:00 Jeff Arnold wrote: ... > +1. Concepts: Updates, packs, helper modules, primary modules > +------------------------------------------------------------ > + > +A Ksplice update (struct update) contains one or more Ksplice packs, one for > +each target kernel module that should be changed by the update. Ksplice packs > +are grouped together into a Ksplice update in order to allow multiple > +compilation units to be changed atomically. Hi Jeff et al. The temptation to invent new nomenclature is one we geeks constantly wrestle with. But when you have a *genuinely* good idea, it's not necessary. Unfortunately, changing names is a PITA, but it's not going to get easier. ksplice_update is a good name, but the rest of your nomenclature is horribly general. So before I become so experienced with it I can't see the flaws, let me suggest: pack -> ksplice_mod_change helper module -> old_code primary module -> new_code > +ksplice_123abc_vmlinux (the "primary" module for the vmlinux pack) > +ksplice_123abc_vmlinux_helper (the "helper" module for the vmlinux pack) > +ksplice_123abc_isdn (the "primary" module for the vmlinux pack) > +ksplice_123abc_isdn_helper (the "helper" module for the vmlinux pack) Or, ksplice_123abc_vmlinux_new, ksplice_123abc_vmlinux_old, etc. This has the advantage that it's pretty clear that _old can be removed afterwards, for example. > +unexpected_running_task (P->A, A->R): Ksplice aborted the operation because > +Ksplice observed a running task during the kernel stack check, at a time when > +Ksplice expected all tasks to be stopped by stop_machine. I'll have to check the code, but this should now just be a subset of "unexpected", no? > +++ b/arch/x86/kernel/ksplice-arch.c > @@ -0,0 +1,125 @@ > +/* Copyright (C) 2007-2008 Jeff Arnold <jbarnold@mit.edu> > + * Copyright (C) 2008 Anders Kaseorg <andersk@mit.edu>, > + * Tim Abbott <tabbott@mit.edu> > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License, version 2. Just in case you copied this boilerplate without thinking about it, I'll just note that my code is all GPLv2 or later. (Rest of ksplice-arch.c skipped... it's hard to review until I understand the core). > +/** > + * struct ksplice_symbol - Ksplice's analogue of an ELF symbol > + * @name: The ELF name of the symbol > + * @label: A unique Ksplice name for the symbol > + * @vals: A linked list of possible values for the symbol, or NULL > + * @value: The value of the symbol (valid when vals is NULL) > + **/ > +struct ksplice_symbol { > + const char *name; > + const char *label; > +/* private: */ > + struct list_head *vals; > + unsigned long value; > +}; I'd have to look at the usage, but might this be neater as a standard linked list, eg: struct ksplice_symval { struct list_head list; unsigned long val; }; struct ksplice_symbol { ... struct list_head vals; /* We always have one, so this saves dynamic alloc in common case? */ struct ksplice_symval first_val; }; (You could link off the first_val.list, but it we don't have iterators for such a "headless" list). > +/** > + * struct ksplice_reloc - Ksplice's analogue of an ELF relocation > + * @blank_addr: The address of the relocation's storage unit > + * @symbol: The ksplice_symbol associated with this relocation > + * @howto: The information regarding the relocation type > + * @addend: The ELF addend of the relocation > + **/ > +struct ksplice_reloc { > + unsigned long blank_addr; > + struct ksplice_symbol *symbol; > + const struct ksplice_reloc_howto *howto; howto is an interesting name. I *think* this is what we usually call "ops"? > +enum ksplice_reloc_howto_type { > + KSPLICE_HOWTO_RELOC, > + KSPLICE_HOWTO_RELOC_PATCH, > + KSPLICE_HOWTO_DATE, > + KSPLICE_HOWTO_TIME, > + KSPLICE_HOWTO_BUG, > + KSPLICE_HOWTO_EXTABLE, > +}; OK, there are two problems with really reviewing this code. (1) it does everything; it's not in stages. (2) it isn't complete unless you see a ksplice module which interacts with it. Could we start with something really simple and work up? I know it's a serious amount of work to split like this. How about a version which just patches time and date? Should also be easy to manually create a ksplice module to test that as well, no? Then say, substitute a new function. Finally work up to what you have now, with handling alternatives and everything. Otherwise the important review (design, not spelling and style) is not really possible in finite time. But trivial comments follow: > +/* List of all ksplice modules and the module they patch */ > +extern struct list_head ksplice_module_list; Just call it ksplice_modules. It's obvious from its usage that it's a list. > +/** > + * init_ksplice_pack() - Initializes a pack > + * @pack: The pack to be initialized. All of the public fields of the > + * pack and its associated data structures should be populated > + * before this function is called. The values of the private > + * fields will be ignored. > + **/ > +int init_ksplice_pack(struct ksplice_pack *pack); I think would more usually be called "register_ksplice_pack"? > + > +/** > + * cleanup_ksplice_pack() - Cleans up a pack unregister... ? > +typedef int __bitwise__ abort_t; > + > +#define OK ((__force abort_t) 0) > +#define NO_MATCH ((__force abort_t) 1) > +#define CODE_BUSY ((__force abort_t) 2) > +#define MODULE_BUSY ((__force abort_t) 3) > +#define OUT_OF_MEMORY ((__force abort_t) 4) > +#define FAILED_TO_FIND ((__force abort_t) 5) > +#define ALREADY_REVERSED ((__force abort_t) 6) > +#define MISSING_EXPORT ((__force abort_t) 7) > +#define UNEXPECTED_RUNNING_TASK ((__force abort_t) 8) > +#define UNEXPECTED ((__force abort_t) 9) > +#define TARGET_NOT_LOADED ((__force abort_t) 10) > +#define CALL_FAILED ((__force abort_t) 11) Yech... I'm not sold on abort_t; how about standard errno? Yes, you have to get a bit creative, say: -ENOENT, -EBUSY, -ETXTBSY, -ENOMEM, -ENOENT, -EEXIST, -ENOEXEC, *, *, -ENODEV, (whatever that call returned). > +static struct update *init_ksplice_update(const char *kid); > +static void cleanup_ksplice_update(struct update *update); > +static void maybe_cleanup_ksplice_update(struct update *update); > +static void add_to_update(struct ksplice_pack *pack, struct update *update); > +static int ksplice_sysfs_init(struct update *update); Please don't talk about what you're going to talk about: order properly and don't keep up in suspense :) [ cut 170 more lines of introduction ] > +#include "ksplice-arch.c" Hmm, that's a little unclean. include/linux/ksplice_arch.h and treat it as a normal .o I'd say. > +int init_ksplice_pack(struct ksplice_pack *pack) ... > + sort(pack->helper_relocs, > + pack->helper_relocs_end - pack->helper_relocs, > + sizeof(*pack->helper_relocs), compare_relocs, NULL); > + sort(pack->primary_relocs, > + pack->primary_relocs_end - pack->primary_relocs, > + sizeof(*pack->primary_relocs), compare_relocs, NULL); > + sort(pack->helper_sections, > + pack->helper_sections_end - pack->helper_sections, > + sizeof(*pack->helper_sections), compare_section_labels, NULL); Can't be sorted in userspace? > +static bool starts_with(const char *str, const char *prefix) > +{ > + return strncmp(str, prefix, strlen(prefix)) == 0; > +} Hmm, I call this strstarts() in CCAN. Turns out it's a fairly common need, I'll implement a common version as a separate patch. Cheers, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 2009-02-07 2:36 ` Rusty Russell @ 2009-02-10 1:01 ` Tim Abbott 0 siblings, 0 replies; 43+ messages in thread From: Tim Abbott @ 2009-02-10 1:01 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Sat, 7 Feb 2009, Rusty Russell wrote: > The temptation to invent new nomenclature is one we geeks constantly > wrestle with. But when you have a *genuinely* good idea, it's not necessary. > Unfortunately, changing names is a PITA, but it's not going to get easier. > > ksplice_update is a good name, but the rest of your nomenclature is > horribly general. So before I become so experienced with it I can't see the > flaws, let me suggest: > > pack -> ksplice_mod_change > helper module -> old_code > primary module -> new_code OK, I agree that these are improvements. We'll make the change for the next version of the patch series. > > +unexpected_running_task (P->A, A->R): Ksplice aborted the operation because > > +Ksplice observed a running task during the kernel stack check, at a time when > > +Ksplice expected all tasks to be stopped by stop_machine. > > I'll have to check the code, but this should now just be a subset of > "unexpected", no? The story here is that normally when an unexpected error occurs, we print something to the Ksplice debugfs file indicating what the actual error was. unexpected_running_task is special in that it occurs within stop_machine, where allocating memory with GFP_KERNEL to print to the debugfs file is not allowed. So, we use a different abort_cause for this particular unexpected error. > > +/** > > + * struct ksplice_symbol - Ksplice's analogue of an ELF symbol > > + * @name: The ELF name of the symbol > > + * @label: A unique Ksplice name for the symbol > > + * @vals: A linked list of possible values for the symbol, or NULL > > + * @value: The value of the symbol (valid when vals is NULL) > > + **/ > > +struct ksplice_symbol { > > + const char *name; > > + const char *label; > > +/* private: */ > > + struct list_head *vals; > > + unsigned long value; > > +}; > > I'd have to look at the usage, but might this be neater as a standard > linked list, eg: > > struct ksplice_symval { > struct list_head list; > unsigned long val; > }; > > struct ksplice_symbol { > ... > struct list_head vals; > /* We always have one, so this saves dynamic alloc in common case? */ > struct ksplice_symval first_val; > }; > > (You could link off the first_val.list, but it we don't have iterators for > such a "headless" list). ksplice_symbol structures are generated by userspace. Using a pointer to a list_head rather than a list_head means userspace doesn't need to know the structure of a list_head. The story with "vals" and "value" is the following. If we don't yet know the value of the symbol, vals is a pointer to a list of candidate values (obtained by lookups in kallsyms and the exported symbol table). Once we have resolved the symbol using run-pre matching, we store its address in the value field and set vals to NULL. One checks whether a symbol has been resolved by checking whether vals is NULL. The documentation of this struct is a bit terse and the names "vals" and "value" are confusing; would renaming vals to "candidate_values" and adding something like the above explanation in a comment improve the situation? > > +/** > > + * struct ksplice_reloc - Ksplice's analogue of an ELF relocation > > + * @blank_addr: The address of the relocation's storage unit > > + * @symbol: The ksplice_symbol associated with this relocation > > + * @howto: The information regarding the relocation type > > + * @addend: The ELF addend of the relocation > > + **/ > > +struct ksplice_reloc { > > + unsigned long blank_addr; > > + struct ksplice_symbol *symbol; > > + const struct ksplice_reloc_howto *howto; > > howto is an interesting name. I *think* this is what we usually call "ops"? ksplice_reloc_howto is named after libbfd's reloc_howto_type, which is a structure containing metadata for how to apply a relocation of that type (how many bytes long the relocation is, whether it is relative to the program counter, etc.). ksplice_reloc_howto has essentially the same purpose and contents as libbfd's reloc_howto_type. > > +enum ksplice_reloc_howto_type { > > + KSPLICE_HOWTO_RELOC, > > + KSPLICE_HOWTO_RELOC_PATCH, > > + KSPLICE_HOWTO_DATE, > > + KSPLICE_HOWTO_TIME, > > + KSPLICE_HOWTO_BUG, > > + KSPLICE_HOWTO_EXTABLE, > > +}; > > OK, there are two problems with really reviewing this code. (1) it does > everything; it's not in stages. The actual process is in stages, so I assume you're referring to the fact that we're submitting it as a single 2500-line patch; I comment on this below. > (2) it isn't complete unless you see a ksplice module which interacts > with it. Right. For this I recommend creating an update using the Ksplice "standalone" code available at [1] and a simple patch such as [2], and actually looking at the modules generated. I recommend the ksplice-inspect utility included in the Ksplice distribution for seeing the objects inside the helper and primary modules. This will show you the data that the update provides to the kernel; the primary and helper modules themselves just load the data into a ksplice_pack and then call init/cleanup_ksplice_pack as appropriate. [1] http://www.ksplice.com/dist/ksplice-0.9.6-src.tar.gz [2] http://www.ksplice.com/example-update > Could we start with something really simple and work up? I know it's a > serious amount of work to split like this. How about a version which > just patches time and date? Should also be easy to manually create a > ksplice module to test that as well, no? Then say, substitute a new > function. Finally work up to what you have now, with handling > alternatives and everything. Before I answer your question, I should explain that these are types of Ksplice relocations, not types of patches. Normal relocations have type KSPLICE_HOWTO_RELOC; if you want to start simple, just ignore the rest and look at what happens with that type. Most of the other Ksplice relocation types are classes of things that should be handled through the same mechanism as relocations. For example, KSPLICE_HOWTO_DATE and KSPLICE_HOWTO_TIME are Ksplice relocations that Ksplice generates in order to run-pre match strings containing __DATE__ and __TIME__. These are challenging for run-pre matching because the date and time when the pre code was compiled will not be the same as the date and time when those compilation units in the running kernel were compiled. Ksplice handles this by generating special Ksplice relocations that instruct Ksplice to do a date/time format check rather than a byte-by-byte comparison at a particular location. Anyway, your question was whether we can strip down Ksplice to just the "bare essentials" that would be easier to review. We could save about 200 lines of kernel code by removing support for altinstructions/parainstructions/smp_locks, KSPLICE_HOWTO_DATE, KSPLICE_HOWTO_TIME, KSPLICE_HOWTO_BUG and KSPLICE_HOWTO_EXTABLE. One could perhaps save another 150 lines of code by removing the optimizations we have in critical pathways, i.e. replacing binary searches and the O(1) amortized scan of relocations with linear searches, and another 70 or so if we make it not work on CONFIG_DEBUG_RODATA kernels. I could probably cut another 100 or so lines of code by removing code that handles some miscellaneous uncommon situations. The result would be slow, unable to run-pre match most compilation units, and have safety problems when patching compilation units containing altinstructions (etc.). So, not something suitable for merge, but perhaps a bit easier to review. Is this the sort of thing that you're looking for? I can get you a patch along those lines fairly quickly. > Otherwise the important review (design, not spelling and style) is not > really possible in finite time. My recommended path for understanding the code is to first read __apply_patches and the things it calls, which is the code run inside stop_machine, including the stack check. Then I'd look at finalize_pack, which is basically the consumer of all the information obtained in run-pre matching. To understand run-pre matching, I'd start with trying to understand run_pre_cmp, and then move up the chain of callers from there. Let us know if there's anything we can do to help with reviewing the code. We'd be happy to answer any questions about how the code works. > But trivial comments follow: > > > +/* List of all ksplice modules and the module they patch */ > > +extern struct list_head ksplice_module_list; > > Just call it ksplice_modules. It's obvious from its usage that it's a list. Agreed. > > +int init_ksplice_pack(struct ksplice_pack *pack); > > I think would more usually be called "register_ksplice_pack"? > > > + * cleanup_ksplice_pack() - Cleans up a pack > > unregister... ? We choose the name cleanup because unlike a typical unregister function, cleanup_ksplice_pack is currently called twice for each Ksplice update; once when the helper module is unloaded, and once when the primary module is unloaded. The extra call is necessary to avoid leaks if you unload the helper without applying the update. Perhaps the right answer is to keep the current name and add a comment documenting this behavior? > > +typedef int __bitwise__ abort_t; > > + > > +#define OK ((__force abort_t) 0) > > +#define NO_MATCH ((__force abort_t) 1) > > +#define CODE_BUSY ((__force abort_t) 2) > > +#define MODULE_BUSY ((__force abort_t) 3) > > +#define OUT_OF_MEMORY ((__force abort_t) 4) > > +#define FAILED_TO_FIND ((__force abort_t) 5) > > +#define ALREADY_REVERSED ((__force abort_t) 6) > > +#define MISSING_EXPORT ((__force abort_t) 7) > > +#define UNEXPECTED_RUNNING_TASK ((__force abort_t) 8) > > +#define UNEXPECTED ((__force abort_t) 9) > > +#define TARGET_NOT_LOADED ((__force abort_t) 10) > > +#define CALL_FAILED ((__force abort_t) 11) > > Yech... I'm not sold on abort_t; how about standard errno? Yes, you have > to get a bit creative, say: -ENOENT, -EBUSY, -ETXTBSY, -ENOMEM, -ENOENT, > -EEXIST, -ENOEXEC, *, *, -ENODEV, (whatever that call returned). The code actually used to make creative use of errno values as you suggest. It resulted in frequent confusion about which return value to use, and mistakes involving the wrong return value were common. After we switched to abort_t, return value errors essentially stopped happening. So I think that keeping abort_t might be worthwhile. We use a typedef so that sparse can check for mistakes involving returning a number rather than an abort_t; we could probably convert it into an enum at this point. > > +static struct update *init_ksplice_update(const char *kid); > > +static void cleanup_ksplice_update(struct update *update); > > +static void maybe_cleanup_ksplice_update(struct update *update); > > +static void add_to_update(struct ksplice_pack *pack, struct update *update); > > +static int ksplice_sysfs_init(struct update *update); > > Please don't talk about what you're going to talk about: order properly > and don't keep up in suspense :) > > [ cut 170 more lines of introduction ] Heh. We actually sorted the functions in ksplice.c to read well from top to bottom while preparing this patch series. I can re-sort our functions to eliminate the prototypes if that would be preferred. Perhaps there should be a note about use of prototypes in Documentation/CodingStyle? > > +#include "ksplice-arch.c" > > Hmm, that's a little unclean. include/linux/ksplice_arch.h > and treat it as a normal .o I'd say. Are you suggesting that include/linux/ksplice_arch.h would give prototypes for write_reloc_value and read_reloc_value along with the functions currently in ksplice_arch.c, and they'd be implemented in a ksplice_arch.c that would be linked in with the rest of Ksplice? I believe that this doesn't work because a module under kernel/ can't depend on objects under arch/ (or vice versa). I agree that this is a little unclean but don't know how to improve it; suggestions for how to clean this up are welcome. > > +int init_ksplice_pack(struct ksplice_pack *pack) > ... > > + sort(pack->helper_relocs, > > + pack->helper_relocs_end - pack->helper_relocs, > > + sizeof(*pack->helper_relocs), compare_relocs, NULL); > > + sort(pack->primary_relocs, > > + pack->primary_relocs_end - pack->primary_relocs, > > + sizeof(*pack->primary_relocs), compare_relocs, NULL); > > + sort(pack->helper_sections, > > + pack->helper_sections_end - pack->helper_sections, > > + sizeof(*pack->helper_sections), compare_section_labels, NULL); > > Can't be sorted in userspace? helper_sections could be sorted in userspace; the relocs arrays could not. They are sorted by a value which depends on where the ELF sections of the helper and primary modules are loaded. Also, these only need to be sorted because the kernel implementation uses a binary search; if we were to replace a binary search with a hashing algorithm, then userspace wouldn't need to sort these arrays. We might want to keep this implementation detail out of the userspace/kernelspace API. By the way, we use binary search a good deal, and I know we're not the only users of a simple binary search, for example search_extable in lib/extable.c. Grepping for "binary search" suggests there are a few dozen of them in the kernel. Should we split out bsearch into a lib/bsearch.c in the next patch series? > > +static bool starts_with(const char *str, const char *prefix) > > +{ > > + return strncmp(str, prefix, strlen(prefix)) == 0; > > +} > > Hmm, I call this strstarts() in CCAN. Turns out it's a fairly common need, > I'll implement a common version as a separate patch. strstarts is a good name for this function. I've made the change for the next version of the patch series. -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 6/7] Ksplice: Export symbols needed for Ksplice 2008-12-06 0:03 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Jeff Arnold 2008-12-06 0:04 ` [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 Jeff Arnold @ 2009-02-04 11:35 ` Rusty Russell 2009-02-13 1:46 ` Tim Abbott 1 sibling, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-04 11:35 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Saturday 06 December 2008 10:33:59 Jeff Arnold wrote: > From: Tim Abbott <tabbott@mit.edu> > > Ksplice uses several functions from module.c in order to resolve > symbols and implement dependency handling. Calling these functions > requires holding module_mutex, so it is exported. I have fought every attempt to export that. It was long a cause of misery and pain; people saw the lock and so started grabbing it. Any chance we can find another way? Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 6/7] Ksplice: Export symbols needed for Ksplice 2009-02-04 11:35 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Rusty Russell @ 2009-02-13 1:46 ` Tim Abbott 2009-02-16 7:11 ` Rusty Russell 0 siblings, 1 reply; 43+ messages in thread From: Tim Abbott @ 2009-02-13 1:46 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Wed, 4 Feb 2009, Rusty Russell wrote: > On Saturday 06 December 2008 10:33:59 Jeff Arnold wrote: > > From: Tim Abbott <tabbott@mit.edu> > > > > Ksplice uses several functions from module.c in order to resolve > > symbols and implement dependency handling. Calling these functions > > requires holding module_mutex, so it is exported. > > I have fought every attempt to export that. It was long a cause of misery > and pain; people saw the lock and so started grabbing it. > > Any chance we can find another way? It is definitely possible, but it wouldn't be pretty. There are a few things that we use module_mutex for: (1) Preventing races involving modules being unloaded during run-pre matching. Functions such as kallsyms_on_each_symbol, find_symbol, __module_text_address, __module_address, and find_module that are used to obtain a module or a symbol can only be used within preempt_disable or while holding module_mutex. Further, the values that they return can only be used in the same such context, unless one module_gets the relevant module in that context. So, we'd change/add wrappers to a bunch of these functions to do their own preempt_disable or grabbing of module_mutex, and to module_get the relevant module before they return. We'd also need to change the Ksplice code to module_put whenever we stop referencing a symbol. Since a lot of the Ksplice kernel code is related to resolving symbols, I suspect this latter part would be somewhat messy. (2) Calling use_module. use_module needs to be called while holding module_mutex because it changes the modules_which_use_me lists. We could change this by creating a modified version of use_module that grabs module_mutex itself; perhaps renaming the current use_module to __use_module. (3) Preventing races involving two different Ksplice updates being applied at the same time. This one is easy; we could just create a new ksplice_mutex. -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 6/7] Ksplice: Export symbols needed for Ksplice 2009-02-13 1:46 ` Tim Abbott @ 2009-02-16 7:11 ` Rusty Russell 0 siblings, 0 replies; 43+ messages in thread From: Rusty Russell @ 2009-02-16 7:11 UTC (permalink / raw) To: Tim Abbott Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Friday 13 February 2009 12:16:40 Tim Abbott wrote: > On Wed, 4 Feb 2009, Rusty Russell wrote: > > > On Saturday 06 December 2008 10:33:59 Jeff Arnold wrote: > > > From: Tim Abbott <tabbott@mit.edu> > > > > > > Ksplice uses several functions from module.c in order to resolve > > > symbols and implement dependency handling. Calling these functions > > > requires holding module_mutex, so it is exported. > > > > I have fought every attempt to export that. It was long a cause of misery > > and pain; people saw the lock and so started grabbing it. > > > > Any chance we can find another way? > > It is definitely possible, but it wouldn't be pretty. There are a few > things that we use module_mutex for: Yes, having looked through the code I don't think the export can be avoided. Thanks, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols 2008-12-06 0:03 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Jeff Arnold 2008-12-06 0:03 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Jeff Arnold @ 2009-02-04 11:30 ` Rusty Russell 2009-02-04 21:31 ` Anders Kaseorg 1 sibling, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-04 11:30 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Saturday 06 December 2008 10:33:58 Jeff Arnold wrote: > From: Anders Kaseorg <andersk@mit.edu> > > kallsyms_lookup_name only returns the first match that it finds. Ksplice > needs information about all symbols with a given name in order to correctly > resolve local symbols. > > kallsyms_on_each_symbol provides a generic mechanism for iterating over the > kallsyms table. Seems reasonable. Did you really not want a name-filtering version? That might be generally useful. > +/* Call a function on each kallsyms symbol in the core kernel */ > +int kallsyms_on_each_symbol(int (*fn)(void *, const char *, struct module *, > + unsigned long), > + void *data); But does no locking at all. It either needs a comment that it can only be called from inside stop_machine, or that it needs preempt disabled, or whatever. > +static inline int kallsyms_on_each_symbol(int (*fn)(void *, const char *, > + struct module *, > + unsigned long), > + void *data) > +{ > + return 0; > +} Is the !CONFIG_KALLSYMS version useful to you? If not, please don't implement the noop version. I want someone who *does* expect it to work to have to think about it if they use it... Thanks, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols 2009-02-04 11:30 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Rusty Russell @ 2009-02-04 21:31 ` Anders Kaseorg 0 siblings, 0 replies; 43+ messages in thread From: Anders Kaseorg @ 2009-02-04 21:31 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan On Wed, 4 Feb 2009, Rusty Russell wrote: > Seems reasonable. Did you really not want a name-filtering version? That > might be generally useful. We don’t actually want a name-filtering version for Ksplice. We process all the symbols we need in one pass, in order to avoid traversing the kallsyms list separately for each symbol. > But does no locking at all. It either needs a comment that it can only be > called from inside stop_machine, or that it needs preempt disabled, or > whatever. It might as well use preempt_disable() and RCU itself (patch below). > Is the !CONFIG_KALLSYMS version useful to you? If not, please don't > implement the noop version. I want someone who *does* expect it to work to > have to think about it if they use it... No, we don’t have a particular use for it; we added it for consistency with kallsyms_lookup_name(), etc. Anders diff --git a/kernel/module.c b/kernel/module.c index 556112c..6bb5814 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -2577,17 +2577,20 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, { struct module *mod; unsigned int i; - int ret; + int ret = 0; - list_for_each_entry(mod, &modules, list) { + preempt_disable(); + list_for_each_entry_rcu(mod, &modules, list) { for (i = 0; i < mod->num_symtab; i++) { ret = fn(data, mod->strtab + mod->symtab[i].st_name, mod, mod->symtab[i].st_value); if (ret != 0) - return ret; + goto out; } } - return 0; +out: + preempt_enable(); + return ret; } #endif /* CONFIG_KALLSYMS */ ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) 2008-12-06 0:03 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Jeff Arnold 2008-12-06 0:03 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Jeff Arnold @ 2009-02-04 11:21 ` Rusty Russell 2009-02-04 20:48 ` Anders Kaseorg 1 sibling, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-04 11:21 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Saturday 06 December 2008 10:33:57 Jeff Arnold wrote: > From: Tim Abbott <tabbott@mit.edu> > > Ksplice's run-pre matching process needs to be able to determine the module > that contains a particular text or data address. Does this mean you really want module_data_address, or you actually want a module_address()? Thanks, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) 2009-02-04 11:21 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Rusty Russell @ 2009-02-04 20:48 ` Anders Kaseorg 2009-02-07 12:45 ` Rusty Russell 0 siblings, 1 reply; 43+ messages in thread From: Anders Kaseorg @ 2009-02-04 20:48 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan On Wed, 4 Feb 2009, Rusty Russell wrote: > Does this mean you really want module_data_address, or you actually want > a module_address()? Yeah, that works better. diff --git a/include/linux/module.h b/include/linux/module.h index ccbcfa7..42e7c8a 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -363,8 +363,8 @@ static inline int module_is_live(struct module *mod) /* Is this address in a module? (second is with no locks, for oops) */ struct module *module_text_address(unsigned long addr); struct module *__module_text_address(unsigned long addr); -struct module *module_data_address(unsigned long addr); -struct module *__module_data_address(unsigned long addr); +struct module *module_address(unsigned long addr); +struct module *__module_address(unsigned long addr); int is_module_address(unsigned long addr); static inline int within_module_core(unsigned long addr, struct module *mod) diff --git a/kernel/module.c b/kernel/module.c index b47772d..556112c 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -2773,35 +2773,32 @@ struct module *module_text_address(unsigned long addr) } EXPORT_SYMBOL_GPL(module_text_address); -struct module *__module_data_address(unsigned long addr) +__notrace_funcgraph struct module *__module_address(unsigned long addr) { struct module *mod; if (addr < module_addr_min || addr > module_addr_max) return NULL; - list_for_each_entry(mod, &modules, list) { - if (within(addr, mod->module_core + mod->core_text_size, - mod->core_size - mod->core_text_size) || - within(addr, mod->module_init + mod->init_text_size, - mod->init_size - mod->init_text_size)) + list_for_each_entry_rcu(mod, &modules, list) + if (within_module_core(addr, mod) || + within_module_init(addr, mod)) return mod; - } return NULL; } -EXPORT_SYMBOL_GPL(__module_data_address); +EXPORT_SYMBOL_GPL(__module_address); -struct module *module_data_address(unsigned long addr) +struct module *module_address(unsigned long addr) { struct module *mod; preempt_disable(); - mod = __module_data_address(addr); + mod = __module_address(addr); preempt_enable(); return mod; } -EXPORT_SYMBOL_GPL(module_data_address); +EXPORT_SYMBOL_GPL(module_address); /* Don't grab lock, we're oopsing. */ void print_modules(void) diff --git a/kernel/ksplice.c b/kernel/ksplice.c index a3f21a6..0404e46 100644 --- a/kernel/ksplice.c +++ b/kernel/ksplice.c @@ -999,8 +999,7 @@ static void *map_writable(void *addr, size_t len) int nr_pages = 2; struct page *pages[2]; - if (__module_text_address((unsigned long)addr) == NULL && - __module_data_address((unsigned long)addr) == NULL) { + if (__module_address((unsigned long)addr) == NULL) { pages[0] = virt_to_page(addr); WARN_ON(!PageReserved(pages[0])); pages[1] = virt_to_page(addr + PAGE_SIZE); @@ -1412,13 +1411,8 @@ static abort_t try_addr(struct ksplice_pack *pack, enum run_pre_mode mode) { abort_t ret; - const struct module *run_module; + const struct module *run_module = __module_address(run_addr); - if ((sect->flags & KSPLICE_SECTION_RODATA) != 0 || - (sect->flags & KSPLICE_SECTION_DATA) != 0) - run_module = __module_data_address(run_addr); - else - run_module = __module_text_address(run_addr); if (run_module == pack->primary) { ksdebug(pack, "run-pre: unexpected address %lx in primary " "module %s for sect %s\n", run_addr, run_module->name, ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) 2009-02-04 20:48 ` Anders Kaseorg @ 2009-02-07 12:45 ` Rusty Russell 0 siblings, 0 replies; 43+ messages in thread From: Rusty Russell @ 2009-02-07 12:45 UTC (permalink / raw) To: Anders Kaseorg Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan On Thursday 05 February 2009 07:18:36 Anders Kaseorg wrote: > On Wed, 4 Feb 2009, Rusty Russell wrote: > > Does this mean you really want module_data_address, or you actually want > > a module_address()? > > Yeah, that works better. Ah, my point was that I can then implement module_text_address in terms of module_address too. I'm just about to dump out the module series I've got for 2.6.30; I've included your ksplice module.c patches or equiv. Cheers, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol 2008-12-06 0:03 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Jeff Arnold 2008-12-06 0:03 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Jeff Arnold @ 2009-02-04 9:26 ` Rusty Russell 1 sibling, 0 replies; 43+ messages in thread From: Rusty Russell @ 2009-02-04 9:26 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Saturday 06 December 2008 10:33:56 Jeff Arnold wrote: > From: Tim Abbott <tabbott@mit.edu> > > Ksplice needs access to the kernel_symbol structure in order to support > modifications to the exported symbol table. Applied, with some fixes: > preempt_disable(); > - if (IS_ERR_VALUE(find_symbol(symbol, &owner, NULL, true, false))) > + if (find_symbol(symbol, &owner, NULL, true, false) == NULL) > BUG(); I skipped the == NULL part; if (find_symbol... reads quite well. Same with three or so other places. > - ret = find_symbol(name, &owner, &crc, > + sym = find_symbol(name, &owner, &crc, > !(mod->taints & (1 << TAINT_PROPRIETARY_MODULE)), true); > - if (!IS_ERR_VALUE(ret)) { > - /* use_module can fail due to OOM, > - or module initialization or unloading */ > - if (!check_version(sechdrs, versindex, name, mod, crc) || > - !use_module(mod, owner)) > - ret = -EINVAL; > - } > - return ret; > + /* use_module can fail due to OOM, > + or module initialization or unloading */ > + if (!check_version(sechdrs, versindex, name, mod, crc) || > + !use_module(mod, owner)) > + sym = NULL; > + return sym; > } If sym is NULL those other values aren't valid! You can't take out the branch.... > - return (void *)value; > + return sym == NULL ? (void *)0 : (void *)sym->value; > } One NULL too many, and one too few: return sym ? (void *)sym->value : NULL; Here's the nett diff: diff --git a/kernel/module.c b/kernel/module.c --- a/kernel/module.c +++ b/kernel/module.c @@ -860,7 +860,7 @@ void __symbol_put(const char *symbol) struct module *owner; preempt_disable(); - if (find_symbol(symbol, &owner, NULL, true, false) == NULL) + if (!find_symbol(symbol, &owner, NULL, true, false)) BUG(); module_put(owner); preempt_enable(); @@ -1023,7 +1023,7 @@ static inline int check_modstruct_versio { const unsigned long *crc; - if (find_symbol("struct_module", NULL, &crc, true, false) == NULL) + if (!find_symbol("struct_module", NULL, &crc, true, false)) BUG(); return check_version(sechdrs, versindex, "struct_module", mod, crc); } @@ -1077,9 +1077,11 @@ static const struct kernel_symbol *resol !(mod->taints & (1 << TAINT_PROPRIETARY_MODULE)), true); /* use_module can fail due to OOM, or module initialization or unloading */ - if (!check_version(sechdrs, versindex, name, mod, crc) || - !use_module(mod, owner)) - sym = NULL; + if (sym) { + if (!check_version(sechdrs, versindex, name, mod, crc) || + !use_module(mod, owner)) + sym = NULL; + } return sym; } @@ -1481,11 +1483,11 @@ void *__symbol_get(const char *symbol) preempt_disable(); sym = find_symbol(symbol, &owner, NULL, true, true); - if (sym != NULL && strong_try_module_get(owner)) + if (sym && strong_try_module_get(owner)) sym = NULL; preempt_enable(); - return sym == NULL ? (void *)0 : (void *)sym->value; + return sym ? (void *)sym->value : NULL; } EXPORT_SYMBOL_GPL(__symbol_get); @@ -1513,8 +1515,7 @@ static int verify_export_symbols(struct for (i = 0; i < ARRAY_SIZE(arr); i++) { for (s = arr[i].sym; s < arr[i].sym + arr[i].num; s++) { - if (find_symbol(s->name, &owner, NULL, true, false) - != NULL) { + if (find_symbol(s->name, &owner, NULL, true, false)) { printk(KERN_ERR "%s: exports duplicate symbol %s" " (owned by %s)\n", @@ -1561,7 +1562,7 @@ static int simplify_symbols(Elf_Shdr *se ksym = resolve_symbol(sechdrs, versindex, strtab + sym[i].st_name, mod); /* Ok if resolved. */ - if (ksym != NULL) { + if (ksym) { sym[i].st_value = ksym->value; break; } ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections 2008-12-06 0:03 ` [PATCH 2/7] x86: Add an option to compile " Jeff Arnold 2008-12-06 0:03 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Jeff Arnold @ 2009-02-04 8:38 ` Rusty Russell 2009-02-04 10:26 ` Anders Kaseorg 1 sibling, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-04 8:38 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Saturday 06 December 2008 10:33:55 Jeff Arnold wrote: > From: Waseem Daher <wdaher@mit.edu> > > This patch makes it possible to link and boot an x86 kernel with > -ffunction-sections and -fdata-sections enabled. Nice: text data bss dec hex filename 3575210 361820 319488 4256518 40f306 vmlinux-with-sections 3579626 364892 323584 4268102 412046 vmlinux-no-sections > +config FUNCTION_DATA_SECTIONS > + bool "Compile with -ffunction-sections -fdata-sections" > + depends on !FTRACE > + help > + If you say Y here the compiler will give each function > + and data structure its own ELF section. You need to make this depend on X86, otherwise allmodconfig will turn it on and break other archs. What's the conflict with FTRACE? Thanks, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections 2009-02-04 8:38 ` [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections Rusty Russell @ 2009-02-04 10:26 ` Anders Kaseorg 2009-02-04 10:58 ` Rusty Russell 0 siblings, 1 reply; 43+ messages in thread From: Anders Kaseorg @ 2009-02-04 10:26 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan On Wed, 4 Feb 2009, Rusty Russell wrote: > You need to make this depend on X86, otherwise allmodconfig will turn it on > and break other archs. Okay. > What's the conflict with FTRACE? This actually should have changed from FTRACE to FUNCTION_TRACER as of commit 606576ce816603d9fe1fb453a88bc6eea16ca709 in v2.6.28-rc3. The conflict is that FUNCTION_TRACER enables KBUILD_CFLAGS += -pg, and passing -pg to current gcc causes it to emit a warning and disable -ffunction-sections: <http://gcc.gnu.org/ml/gcc-help/2008-11/msg00128.html> (If this gcc problem is resolved in the future, the code should work fine with FUNCTION_TRACER. A small amount of extra matching code is needed to handle DYNAMIC_FTRACE.) Patch for these two issues: diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 6e23e10..1d63b5f 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -29,6 +29,7 @@ config X86 select HAVE_KSPLICE select ARCH_WANT_OPTIONAL_GPIOLIB select ARCH_WANT_FRAME_POINTERS + select HAVE_FUNCTION_DATA_SECTIONS select HAVE_KRETPROBES select HAVE_FTRACE_MCOUNT_RECORD select HAVE_DYNAMIC_FTRACE diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 6d64083..c6a2674 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -591,9 +591,13 @@ config FRAME_POINTER larger and slower, but it gives very useful debugging information in case of kernel bugs. (precise oopses/stacktraces/warnings) +config HAVE_FUNCTION_DATA_SECTIONS + bool + config FUNCTION_DATA_SECTIONS bool "Compile with -ffunction-sections -fdata-sections" - depends on !FTRACE + depends on HAVE_FUNCTION_DATA_SECTIONS + depends on !FUNCTION_TRACER help If you say Y here the compiler will give each function and data structure its own ELF section. ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections 2009-02-04 10:26 ` Anders Kaseorg @ 2009-02-04 10:58 ` Rusty Russell 2009-02-04 20:50 ` Anders Kaseorg 0 siblings, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-04 10:58 UTC (permalink / raw) To: Anders Kaseorg Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan On Wednesday 04 February 2009 20:56:04 Anders Kaseorg wrote: > The conflict is that FUNCTION_TRACER enables KBUILD_CFLAGS += -pg, and > passing -pg to current gcc causes it to emit a warning and disable > -ffunction-sections: > <http://gcc.gnu.org/ml/gcc-help/2008-11/msg00128.html> Ah, OK, this explanation should probably be in a paragraph in the Kconfig help message, rather than hiding on lkml. Otherwise, updated looks good. Thanks, Rusty. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections 2009-02-04 10:58 ` Rusty Russell @ 2009-02-04 20:50 ` Anders Kaseorg 0 siblings, 0 replies; 43+ messages in thread From: Anders Kaseorg @ 2009-02-04 20:50 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan On Wed, 4 Feb 2009, Rusty Russell wrote: > Ah, OK, this explanation should probably be in a paragraph in the Kconfig > help message, rather than hiding on lkml. Sure. diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index c6a2674..3412706 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -602,6 +602,11 @@ config FUNCTION_DATA_SECTIONS If you say Y here the compiler will give each function and data structure its own ELF section. + This option conflicts with CONFIG_FUNCTION_TRACER, which + enables profiling code generation, because current GCC does + not support compiling with -ffunction-sections -pg (see + <http://gcc.gnu.org/ml/gcc-help/2008-11/msg00128.html>). + If unsure, say N. config BOOT_PRINTK_DELAY ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-06 0:03 ` [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections Jeff Arnold 2008-12-06 0:03 ` [PATCH 2/7] x86: Add an option to compile " Jeff Arnold @ 2008-12-06 8:46 ` Andi Kleen 2008-12-31 19:18 ` Tim Abbott 2009-02-04 8:15 ` Rusty Russell 2 siblings, 1 reply; 43+ messages in thread From: Andi Kleen @ 2008-12-06 8:46 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan Jeff Arnold <jbarnold@MIT.EDU> writes: > From: Denys Vlasenko <vda.linux@googlemail.com> > > This patch was originally written by Denys Vlasenko, but it has been > substantially modified by Anders Kaseorg and Tim Abbott to update it > to apply to the current kernel and fix bugs. Any bugs were added by > Anders Kaseorg and Tim Abbott. > > The purpose of this patch is to make it possible to build the kernel > with "gcc -ffunction-sections -fdata-sections". This patch seems to be a good thing regardless of ksplice or not. I would suggest to push it separately. It's quite large and touches a lot of code, so it probably would require special treatment during a merge window to avoid conflicts. -Andi -- ak@linux.intel.com ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-06 8:46 ` [PATCH 1/7] Make section names compatible " Andi Kleen @ 2008-12-31 19:18 ` Tim Abbott 2008-12-31 19:52 ` Andi Kleen 0 siblings, 1 reply; 43+ messages in thread From: Tim Abbott @ 2008-12-31 19:18 UTC (permalink / raw) To: Andi Kleen Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Sat, 6 Dec 2008, Andi Kleen wrote: > It's quite large and touches a lot of code, so it probably would require > special treatment during a merge window to avoid conflicts. OK. How does special treatment during the merge window work? Is it too late to get this patch scheduled during the present merge window? -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-31 19:18 ` Tim Abbott @ 2008-12-31 19:52 ` Andi Kleen 2008-12-31 21:59 ` Tim Abbott 0 siblings, 1 reply; 43+ messages in thread From: Andi Kleen @ 2008-12-31 19:52 UTC (permalink / raw) To: Tim Abbott Cc: Andi Kleen, Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Wed, Dec 31, 2008 at 02:18:43PM -0500, Tim Abbott wrote: > On Sat, 6 Dec 2008, Andi Kleen wrote: > > >It's quite large and touches a lot of code, so it probably would require > >special treatment during a merge window to avoid conflicts. > > OK. How does special treatment during the merge window work? You negotiate a suitable time with Andrew/Linus in advance. This is typically after all other trees got in. > Is it too late Probably not, as long as you can update the patch quickly enough. -Andi -- ak@linux.intel.com ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-31 19:52 ` Andi Kleen @ 2008-12-31 21:59 ` Tim Abbott 2009-01-01 16:32 ` Andrew Morton 0 siblings, 1 reply; 43+ messages in thread From: Tim Abbott @ 2008-12-31 21:59 UTC (permalink / raw) To: Andrew Morton, Andi Kleen Cc: Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan Hi Andrew, I'd like to get this -ffunction-sections patch merged sometime during the 2.6.29 merge window. I've updated it in my git repository to work on top of Linus' current master; how do you want to handle this? Thanks, -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-31 21:59 ` Tim Abbott @ 2009-01-01 16:32 ` Andrew Morton 2009-01-04 19:20 ` Tim Abbott 0 siblings, 1 reply; 43+ messages in thread From: Andrew Morton @ 2009-01-01 16:32 UTC (permalink / raw) To: Tim Abbott Cc: Andi Kleen, Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Wed, 31 Dec 2008 16:59:39 -0500 (EST) Tim Abbott <tabbott@MIT.EDU> wrote: > Hi Andrew, > > I'd like to get this -ffunction-sections patch merged sometime during the > 2.6.29 merge window. I've updated it in my git repository to work on top > of Linus' current master; how do you want to handle this? > <tries to remember what the patch does, and what the problem with it is> <finds it> OK, it hits a lot of files in minor ways. You could ask Linus to merge it late in the -rc1 merge window (please cc me on the email). Or you could just send it to me (with the usual cc's) and I'll toss it into my 2.6.29-rc1 pile. I doubt if it'll cause much trouble. But please do check its applicability to linux-next - people have been moving arch files into different directories lately. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2009-01-01 16:32 ` Andrew Morton @ 2009-01-04 19:20 ` Tim Abbott 2009-01-12 21:51 ` Andrew Morton 0 siblings, 1 reply; 43+ messages in thread From: Tim Abbott @ 2009-01-04 19:20 UTC (permalink / raw) To: Andrew Morton Cc: Andi Kleen, Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Thu, 1 Jan 2009, Andrew Morton wrote: > OK, it hits a lot of files in minor ways. > > You could ask Linus to merge it late in the -rc1 merge window (please > cc me on the email). Or you could just send it to me (with the usual > cc's) and I'll toss it into my 2.6.29-rc1 pile. I doubt if it'll cause > much trouble. But please do check its applicability to linux-next - people > have been moving arch files into different directories lately. Andrew, I've attached for you a version of the patch the applies to current linux-next (next-20090102). It does not apply to Linus' current master because d3794979a in linux-next adds use of the ".data.percpu.first" section name, which this patch updates to ".kernel.data.percpu.first". I think it'll be easiest to just merge this after d3794979a. If a version of d3794979a is not going to be merged first, there will be two merge conflicts that are easy to resolve as they are changes to code added by d3794979a, but we'll then need to fix d3794979a before merging d3794979a. Let me know if there are any problems with this plan. Thanks, -Tim Abbott --- From: Denys Vlasenko <vda.linux@googlemail.com> Date: Wed, 10 Sep 2008 01:07:56 -0400 Subject: [PATCH] Make section names compatible with -ffunction-sections -fdata-sections This patch was originally written by Denys Vlasenko, but it has been substantially modified by Anders Kaseorg and Tim Abbott to update it to apply to the current kernel and fix bugs. Any bugs were added by Anders Kaseorg and Tim Abbott. The purpose of this patch is to make it possible to build the kernel with "gcc -ffunction-sections -fdata-sections". The problem is that with -ffunction-sections -fdata-sections, gcc creates sections like .text.head and .data.nosave whenever someone has innocuous code like this: static void head(...) {...} or this: static int nosave = 1; somewhere in the kernel. The kernel linker scripts are confused by such names, and thus put these sections in the wrong places. This patch renames all "magic" section names used by the kernel to not have this format, eliminating the possibility of such collisions. Ksplice's 'run-pre matching' process is much simpler if the original kernel was compiled with -ffunction-sections and -fdata-sections. Changelog since Denys Vlasenko's latest patch: * Renamed .kernel.exit.text.refok to .kernel.text.exit.refok, for better analogy with .kernel.*.init.refok. * Updated from 2.6.27-rc4-git3 to next-20090102. * Added some missing section flags for some of the .kernel.* sections (gcc assumes "ax" for sections with names starting with ".text", but does not for sections with names starting with ".kernel.text"). Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com> [andersk@mit.edu] Add missing section flags for .kernel.* sections. Signed-off-by: Anders Kaseorg <andersk@mit.edu> [tabbott@mit.edu] Update from 2.6.27-rc4-git3 to next-20090102. [tabbott@mit.edu] Rename .kernel.exit.text.refok. Signed-off-by: Tim Abbott <tabbott@mit.edu> Tested-by: Waseem Daher <wdaher@mit.edu> --- Documentation/mutex-design.txt | 4 +- arch/alpha/kernel/head.S | 2 +- arch/alpha/kernel/init_task.c | 2 +- arch/alpha/kernel/vmlinux.lds.S | 14 ++++---- arch/arm/kernel/head-nommu.S | 3 +- arch/arm/kernel/head.S | 3 +- arch/arm/kernel/init_task.c | 2 +- arch/arm/kernel/vmlinux.lds.S | 14 ++++---- arch/arm/mm/proc-v6.S | 2 +- arch/arm/mm/proc-v7.S | 2 +- arch/arm/mm/tlb-v6.S | 2 +- arch/arm/mm/tlb-v7.S | 2 +- arch/avr32/kernel/init_task.c | 2 +- arch/avr32/kernel/vmlinux.lds.S | 6 ++-- arch/avr32/mm/init.c | 2 +- arch/blackfin/kernel/vmlinux.lds.S | 2 +- arch/cris/kernel/process.c | 2 +- arch/cris/kernel/vmlinux.lds.S | 2 +- arch/frv/kernel/break.S | 4 +- arch/frv/kernel/entry.S | 2 +- arch/frv/kernel/head-mmu-fr451.S | 2 +- arch/frv/kernel/head-uc-fr401.S | 2 +- arch/frv/kernel/head-uc-fr451.S | 2 +- arch/frv/kernel/head-uc-fr555.S | 2 +- arch/frv/kernel/head.S | 4 +- arch/frv/kernel/init_task.c | 2 +- arch/frv/kernel/vmlinux.lds.S | 18 +++++----- arch/frv/mm/tlb-miss.S | 2 +- arch/h8300/boot/compressed/head.S | 2 +- arch/h8300/boot/compressed/vmlinux.lds | 2 +- arch/h8300/kernel/init_task.c | 2 +- arch/h8300/kernel/vmlinux.lds.S | 2 +- arch/ia64/include/asm/asmmacro.h | 12 +++--- arch/ia64/include/asm/cache.h | 2 +- arch/ia64/include/asm/percpu.h | 2 +- arch/ia64/kernel/Makefile | 2 +- arch/ia64/kernel/gate-data.S | 2 +- arch/ia64/kernel/gate.S | 8 ++-- arch/ia64/kernel/gate.lds.S | 10 +++--- arch/ia64/kernel/head.S | 2 +- arch/ia64/kernel/init_task.c | 4 +- arch/ia64/kernel/ivt.S | 2 +- arch/ia64/kernel/minstate.h | 4 +- arch/ia64/kernel/paravirtentry.S | 12 +++--- arch/ia64/kernel/vmlinux.lds.S | 48 +++++++++++++------------- arch/ia64/kvm/vmm_ivt.S | 2 +- arch/ia64/xen/xensetup.S | 2 +- arch/m32r/kernel/head.S | 2 +- arch/m32r/kernel/init_task.c | 2 +- arch/m32r/kernel/vmlinux.lds.S | 8 ++-- arch/m68k/kernel/head.S | 2 +- arch/m68k/kernel/process.c | 2 +- arch/m68k/kernel/sun3-head.S | 2 +- arch/m68k/kernel/vmlinux-std.lds | 6 ++-- arch/m68k/kernel/vmlinux-sun3.lds | 4 +- arch/m68knommu/kernel/init_task.c | 2 +- arch/m68knommu/kernel/vmlinux.lds.S | 6 ++-- arch/m68knommu/platform/68360/head-ram.S | 2 +- arch/m68knommu/platform/68360/head-rom.S | 2 +- arch/mips/kernel/init_task.c | 2 +- arch/mips/kernel/vmlinux.lds.S | 8 ++-- arch/mips/lasat/image/head.S | 2 +- arch/mips/lasat/image/romscript.normal | 2 +- arch/mn10300/kernel/head.S | 2 +- arch/mn10300/kernel/init_task.c | 2 +- arch/mn10300/kernel/vmlinux.lds.S | 16 ++++---- arch/parisc/include/asm/cache.h | 2 +- arch/parisc/include/asm/system.h | 2 +- arch/parisc/kernel/head.S | 2 +- arch/parisc/kernel/init_task.c | 8 ++-- arch/parisc/kernel/vmlinux.lds.S | 26 +++++++------- arch/powerpc/include/asm/cache.h | 2 +- arch/powerpc/include/asm/page_64.h | 2 +- arch/powerpc/include/asm/ppc_asm.h | 4 +- arch/powerpc/kernel/head_32.S | 2 +- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_44x.S | 2 +- arch/powerpc/kernel/head_8xx.S | 2 +- arch/powerpc/kernel/head_fsl_booke.S | 2 +- arch/powerpc/kernel/init_task.c | 2 +- arch/powerpc/kernel/machine_kexec_64.c | 2 +- arch/powerpc/kernel/vdso.c | 2 +- arch/powerpc/kernel/vdso32/vdso32_wrapper.S | 2 +- arch/powerpc/kernel/vdso64/vdso64_wrapper.S | 2 +- arch/powerpc/kernel/vmlinux.lds.S | 28 ++++++++-------- arch/s390/include/asm/cache.h | 2 +- arch/s390/kernel/head.S | 2 +- arch/s390/kernel/init_task.c | 2 +- arch/s390/kernel/vdso32/vdso32_wrapper.S | 2 +- arch/s390/kernel/vdso64/vdso64_wrapper.S | 2 +- arch/s390/kernel/vmlinux.lds.S | 20 +++++----- arch/sh/include/asm/cache.h | 2 +- arch/sh/kernel/cpu/sh5/entry.S | 4 +- arch/sh/kernel/head_32.S | 2 +- arch/sh/kernel/head_64.S | 2 +- arch/sh/kernel/init_task.c | 2 +- arch/sh/kernel/irq.c | 4 +- arch/sh/kernel/vmlinux_32.lds.S | 14 ++++---- arch/sh/kernel/vmlinux_64.lds.S | 14 ++++---- arch/sparc/boot/btfixupprep.c | 4 +- arch/sparc/include/asm/cache.h | 2 +- arch/sparc/kernel/head_32.S | 2 +- arch/sparc/kernel/head_64.S | 2 +- arch/sparc/kernel/vmlinux.lds.S | 14 ++++---- arch/um/include/asm/common.lds.S | 4 +- arch/um/kernel/dyn.lds.S | 4 +- arch/um/kernel/init_task.c | 4 +- arch/um/kernel/uml.lds.S | 4 +- arch/x86/boot/compressed/head_32.S | 2 +- arch/x86/boot/compressed/head_64.S | 2 +- arch/x86/boot/compressed/vmlinux.scr | 2 +- arch/x86/boot/compressed/vmlinux_32.lds | 14 +++++-- arch/x86/boot/compressed/vmlinux_64.lds | 10 +++-- arch/x86/include/asm/cache.h | 4 +- arch/x86/kernel/acpi/wakeup_32.S | 2 +- arch/x86/kernel/head_32.S | 6 ++-- arch/x86/kernel/head_64.S | 4 +- arch/x86/kernel/init_task.c | 4 +- arch/x86/kernel/traps.c | 2 +- arch/x86/kernel/vmlinux_32.lds.S | 36 ++++++++++---------- arch/x86/kernel/vmlinux_64.lds.S | 26 +++++++------- arch/xtensa/kernel/head.S | 2 +- arch/xtensa/kernel/init_task.c | 2 +- arch/xtensa/kernel/vmlinux.lds.S | 6 ++-- include/asm-frv/init.h | 8 ++-- include/asm-generic/vmlinux.lds.h | 24 +++++++------- include/linux/cache.h | 2 +- include/linux/init.h | 8 ++-- include/linux/linkage.h | 4 +- include/linux/percpu.h | 12 +++--- include/linux/spinlock.h | 2 +- kernel/module.c | 2 +- scripts/mod/modpost.c | 12 +++--- scripts/recordmcount.pl | 6 ++-- 134 files changed, 364 insertions(+), 354 deletions(-) diff --git a/Documentation/mutex-design.txt b/Documentation/mutex-design.txt index aa60d1f..b5f800b 100644 --- a/Documentation/mutex-design.txt +++ b/Documentation/mutex-design.txt @@ -66,14 +66,14 @@ of advantages of mutexes: c0377ccb <mutex_lock>: c0377ccb: f0 ff 08 lock decl (%eax) - c0377cce: 78 0e js c0377cde <.text.lock.mutex> + c0377cce: 78 0e js c0377cde <.kernel.text.lock.mutex> c0377cd0: c3 ret the unlocking fastpath is equally tight: c0377cd1 <mutex_unlock>: c0377cd1: f0 ff 00 lock incl (%eax) - c0377cd4: 7e 0f jle c0377ce5 <.text.lock.mutex+0x7> + c0377cd4: 7e 0f jle c0377ce5 <.kernel.text.lock.mutex+0x7> c0377cd6: c3 ret - 'struct mutex' semantics are well-defined and are enforced if diff --git a/arch/alpha/kernel/head.S b/arch/alpha/kernel/head.S index 7ac1f13..3726d71 100644 --- a/arch/alpha/kernel/head.S +++ b/arch/alpha/kernel/head.S @@ -10,7 +10,7 @@ #include <asm/system.h> #include <asm/asm-offsets.h> -.section .text.head, "ax" +.section .kernel.text.head, "ax" .globl swapper_pg_dir .globl _stext swapper_pg_dir=SWAPPER_PGD diff --git a/arch/alpha/kernel/init_task.c b/arch/alpha/kernel/init_task.c index c2938e5..826c26a 100644 --- a/arch/alpha/kernel/init_task.c +++ b/arch/alpha/kernel/init_task.c @@ -17,5 +17,5 @@ EXPORT_SYMBOL(init_mm); EXPORT_SYMBOL(init_task); union thread_union init_thread_union - __attribute__((section(".data.init_thread"))) + __attribute__((section(".kernel.data.init_thread"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/alpha/kernel/vmlinux.lds.S b/arch/alpha/kernel/vmlinux.lds.S index ef37fc1..e3c637e 100644 --- a/arch/alpha/kernel/vmlinux.lds.S +++ b/arch/alpha/kernel/vmlinux.lds.S @@ -16,7 +16,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -93,18 +93,18 @@ SECTIONS /* Freed after init ends here */ /* Note 2 page alignment above. */ - .data.init_thread : { - *(.data.init_thread) + .kernel.data.init_thread : { + *(.kernel.data.init_thread) } . = ALIGN(PAGE_SIZE); - .data.page_aligned : { - *(.data.page_aligned) + .kernel.data.page_aligned : { + *(.kernel.data.page_aligned) } . = ALIGN(64); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } _data = .; diff --git a/arch/arm/kernel/head-nommu.S b/arch/arm/kernel/head-nommu.S index cc87e17..8163b2e 100644 --- a/arch/arm/kernel/head-nommu.S +++ b/arch/arm/kernel/head-nommu.S @@ -32,7 +32,8 @@ * numbers for r1. * */ - .section ".text.head", "ax" + .section ".kernel.text.head", "ax" + .type stext, %function ENTRY(stext) msr cpsr_c, #PSR_F_BIT | PSR_I_BIT | SVC_MODE @ ensure svc mode @ and irqs disabled diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S index 21e17dc..42f2d17 100644 --- a/arch/arm/kernel/head.S +++ b/arch/arm/kernel/head.S @@ -74,7 +74,8 @@ * crap here - that's what the boot loader (or in extreme, well justified * circumstances, zImage) is for. */ - .section ".text.head", "ax" + .section ".kernel.text.head", "ax" + .type stext, %function ENTRY(stext) msr cpsr_c, #PSR_F_BIT | PSR_I_BIT | SVC_MODE @ ensure svc mode @ and irqs disabled diff --git a/arch/arm/kernel/init_task.c b/arch/arm/kernel/init_task.c index e859af3..af0be90 100644 --- a/arch/arm/kernel/init_task.c +++ b/arch/arm/kernel/init_task.c @@ -29,7 +29,7 @@ EXPORT_SYMBOL(init_mm); * The things we do for performance.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S index 0021607..5f73ff8 100644 --- a/arch/arm/kernel/vmlinux.lds.S +++ b/arch/arm/kernel/vmlinux.lds.S @@ -23,10 +23,10 @@ SECTIONS #else . = PAGE_OFFSET + TEXT_OFFSET; #endif - .text.head : { + .kernel.text.head : { _stext = .; _sinittext = .; - *(.text.head) + *(.kernel.text.head) } .init : { /* Init code and data */ @@ -65,8 +65,8 @@ SECTIONS #endif . = ALIGN(4096); __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; #ifndef CONFIG_XIP_KERNEL __init_begin = _stext; @@ -125,7 +125,7 @@ SECTIONS * first, the init task union, aligned * to an 8192 byte boundary. */ - *(.data.init_task) + *(.kernel.data.init_task) #ifdef CONFIG_XIP_KERNEL . = ALIGN(4096); @@ -137,7 +137,7 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(4096); __nosave_end = .; @@ -145,7 +145,7 @@ SECTIONS * then the cacheline aligned data */ . = ALIGN(32); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) /* * The exception fixup table (might need resorting at runtime) diff --git a/arch/arm/mm/proc-v6.S b/arch/arm/mm/proc-v6.S index f0cc599..c2eb6a5 100644 --- a/arch/arm/mm/proc-v6.S +++ b/arch/arm/mm/proc-v6.S @@ -132,7 +132,7 @@ cpu_v6_name: .asciz "ARMv6-compatible processor" .align - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr /* * __v6_setup diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index d1ebec4..19b351b 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -153,7 +153,7 @@ cpu_v7_name: .ascii "ARMv7 Processor" .align - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr /* * __v7_setup diff --git a/arch/arm/mm/tlb-v6.S b/arch/arm/mm/tlb-v6.S index 20f84bb..f033268 100644 --- a/arch/arm/mm/tlb-v6.S +++ b/arch/arm/mm/tlb-v6.S @@ -87,7 +87,7 @@ ENTRY(v6wbi_flush_kern_tlb_range) mcr p15, 0, r2, c7, c5, 4 @ prefetch flush mov pc, lr - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr .type v6wbi_tlb_fns, #object ENTRY(v6wbi_tlb_fns) diff --git a/arch/arm/mm/tlb-v7.S b/arch/arm/mm/tlb-v7.S index 24ba510..1b6d40a 100644 --- a/arch/arm/mm/tlb-v7.S +++ b/arch/arm/mm/tlb-v7.S @@ -80,7 +80,7 @@ ENTRY(v7wbi_flush_kern_tlb_range) mov pc, lr ENDPROC(v7wbi_flush_kern_tlb_range) - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr .type v7wbi_tlb_fns, #object ENTRY(v7wbi_tlb_fns) diff --git a/arch/avr32/kernel/init_task.c b/arch/avr32/kernel/init_task.c index 993d56e..4002a5d 100644 --- a/arch/avr32/kernel/init_task.c +++ b/arch/avr32/kernel/init_task.c @@ -23,7 +23,7 @@ EXPORT_SYMBOL(init_mm); * Initial thread structure. Must be aligned on an 8192-byte boundary. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/avr32/kernel/vmlinux.lds.S b/arch/avr32/kernel/vmlinux.lds.S index 7910d41..1cb0345 100644 --- a/arch/avr32/kernel/vmlinux.lds.S +++ b/arch/avr32/kernel/vmlinux.lds.S @@ -95,15 +95,15 @@ SECTIONS /* * First, the init task union, aligned to an 8K boundary. */ - *(.data.init_task) + *(.kernel.data.init_task) /* Then, the page-aligned data */ . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) /* Then, the cacheline aligned data */ . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) /* And the rest... */ *(.data.rel*) diff --git a/arch/avr32/mm/init.c b/arch/avr32/mm/init.c index fa92ff6..fa5089c 100644 --- a/arch/avr32/mm/init.c +++ b/arch/avr32/mm/init.c @@ -24,7 +24,7 @@ #include <asm/setup.h> #include <asm/sections.h> -#define __page_aligned __attribute__((section(".data.page_aligned"))) +#define __page_aligned __attribute__((section(".kernel.data.page_aligned"))) DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); diff --git a/arch/blackfin/kernel/vmlinux.lds.S b/arch/blackfin/kernel/vmlinux.lds.S index 7d12c66..88e320d 100644 --- a/arch/blackfin/kernel/vmlinux.lds.S +++ b/arch/blackfin/kernel/vmlinux.lds.S @@ -92,7 +92,7 @@ SECTIONS __sdata = .; /* This gets done first, so the glob doesn't suck it in */ . = ALIGN(32); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) #if !L1_DATA_A_LENGTH . = ALIGN(32); diff --git a/arch/cris/kernel/process.c b/arch/cris/kernel/process.c index 60816e8..43c26bd 100644 --- a/arch/cris/kernel/process.c +++ b/arch/cris/kernel/process.c @@ -51,7 +51,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/cris/kernel/vmlinux.lds.S b/arch/cris/kernel/vmlinux.lds.S index 0d2adfc..526f853 100644 --- a/arch/cris/kernel/vmlinux.lds.S +++ b/arch/cris/kernel/vmlinux.lds.S @@ -68,7 +68,7 @@ SECTIONS _edata = . ; . = ALIGN(PAGE_SIZE); /* init_task and stack, must be aligned. */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } . = ALIGN(PAGE_SIZE); /* Init code and data. */ __init_begin = .; diff --git a/arch/frv/kernel/break.S b/arch/frv/kernel/break.S index bd0bdf9..1b7148a 100644 --- a/arch/frv/kernel/break.S +++ b/arch/frv/kernel/break.S @@ -21,7 +21,7 @@ # # the break handler has its own stack # - .section .bss.stack + .section .bss.kernel.stack .globl __break_user_context .balign THREAD_SIZE __break_stack: @@ -63,7 +63,7 @@ __break_trace_through_exceptions: # entry point for Break Exceptions/Interrupts # ############################################################################### - .section .text.break + .section .kernel.text.break,"ax",@progbits .balign 4 .globl __entry_break __entry_break: diff --git a/arch/frv/kernel/entry.S b/arch/frv/kernel/entry.S index 99060ab..8d54945 100644 --- a/arch/frv/kernel/entry.S +++ b/arch/frv/kernel/entry.S @@ -38,7 +38,7 @@ #define nr_syscalls ((syscall_table_size)/4) - .section .text.entry + .section .kernel.text.entry,"ax",@progbits .balign 4 .macro LEDS val diff --git a/arch/frv/kernel/head-mmu-fr451.S b/arch/frv/kernel/head-mmu-fr451.S index c8f210d..248e557 100644 --- a/arch/frv/kernel/head-mmu-fr451.S +++ b/arch/frv/kernel/head-mmu-fr451.S @@ -31,7 +31,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr401.S b/arch/frv/kernel/head-uc-fr401.S index ee282be..282ab8d 100644 --- a/arch/frv/kernel/head-uc-fr401.S +++ b/arch/frv/kernel/head-uc-fr401.S @@ -30,7 +30,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr451.S b/arch/frv/kernel/head-uc-fr451.S index b10d9c8..2fcb045 100644 --- a/arch/frv/kernel/head-uc-fr451.S +++ b/arch/frv/kernel/head-uc-fr451.S @@ -30,7 +30,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr555.S b/arch/frv/kernel/head-uc-fr555.S index 39937c1..b5758bf 100644 --- a/arch/frv/kernel/head-uc-fr555.S +++ b/arch/frv/kernel/head-uc-fr555.S @@ -29,7 +29,7 @@ #define __551_LCR 0xfeff1100 #define __551_LSBR 0xfeff1c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head.S b/arch/frv/kernel/head.S index fecf751..8be8976 100644 --- a/arch/frv/kernel/head.S +++ b/arch/frv/kernel/head.S @@ -27,7 +27,7 @@ # command line string # ############################################################################### - .section .text.head,"ax" + .section .kernel.text.head,"ax" .balign 4 .globl _boot, __head_reference @@ -541,7 +541,7 @@ __head_end: .size _boot, .-_boot # provide a point for GDB to place a break - .section .text.start,"ax" + .section .kernel.text.start,"ax" .globl _start .balign 4 _start: diff --git a/arch/frv/kernel/init_task.c b/arch/frv/kernel/init_task.c index 29429a8..7ac16e1 100644 --- a/arch/frv/kernel/init_task.c +++ b/arch/frv/kernel/init_task.c @@ -24,7 +24,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/frv/kernel/vmlinux.lds.S b/arch/frv/kernel/vmlinux.lds.S index b95c4ea..219d286 100644 --- a/arch/frv/kernel/vmlinux.lds.S +++ b/arch/frv/kernel/vmlinux.lds.S @@ -26,7 +26,7 @@ SECTIONS _sinittext = .; .init.text : { - *(.text.head) + *(.kernel.text.head) #ifndef CONFIG_DEBUG_INFO INIT_TEXT EXIT_TEXT @@ -71,13 +71,13 @@ SECTIONS /* put sections together that have massive alignment issues */ . = ALIGN(THREAD_SIZE); - .data.init_task : { + .kernel.data.init_task : { /* init task record & stack */ - *(.data.init_task) + *(.kernel.data.init_task) } . = ALIGN(L1_CACHE_BYTES); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } .trap : { /* trap table management - read entry-table.S before modifying */ @@ -94,10 +94,10 @@ SECTIONS _text = .; _stext = .; .text : { - *(.text.start) - *(.text.entry) - *(.text.break) - *(.text.tlbmiss) + *(.kernel.text.start) + *(.kernel.text.entry) + *(.kernel.text.break) + *(.kernel.text.tlbmiss) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -152,7 +152,7 @@ SECTIONS .sbss : { *(.sbss .sbss.*) } .bss : { *(.bss .bss.*) } - .bss.stack : { *(.bss) } + .bss.kernel.stack : { *(.bss) } __bss_stop = .; _end = . ; diff --git a/arch/frv/mm/tlb-miss.S b/arch/frv/mm/tlb-miss.S index 0764348..e361931 100644 --- a/arch/frv/mm/tlb-miss.S +++ b/arch/frv/mm/tlb-miss.S @@ -16,7 +16,7 @@ #include <asm/highmem.h> #include <asm/spr-regs.h> - .section .text.tlbmiss + .section .kernel.text.tlbmiss,"ax",@progbits .balign 4 .globl __entry_insn_mmu_miss diff --git a/arch/h8300/boot/compressed/head.S b/arch/h8300/boot/compressed/head.S index 985a81a..c4964f1 100644 --- a/arch/h8300/boot/compressed/head.S +++ b/arch/h8300/boot/compressed/head.S @@ -9,7 +9,7 @@ #define SRAM_START 0xff4000 - .section .text.startup + .section .kernel.text.startup,"ax",@progbits .global startup startup: mov.l #SRAM_START+0x8000, sp diff --git a/arch/h8300/boot/compressed/vmlinux.lds b/arch/h8300/boot/compressed/vmlinux.lds index 65e2a0d..5965521 100644 --- a/arch/h8300/boot/compressed/vmlinux.lds +++ b/arch/h8300/boot/compressed/vmlinux.lds @@ -4,7 +4,7 @@ SECTIONS { __stext = . ; __text = .; - *(.text.startup) + *(.kernel.text.startup) *(.text) __etext = . ; } diff --git a/arch/h8300/kernel/init_task.c b/arch/h8300/kernel/init_task.c index cb5dc55..62452f4 100644 --- a/arch/h8300/kernel/init_task.c +++ b/arch/h8300/kernel/init_task.c @@ -36,6 +36,6 @@ EXPORT_SYMBOL(init_task); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/h8300/kernel/vmlinux.lds.S b/arch/h8300/kernel/vmlinux.lds.S index 43a87b9..473e17d 100644 --- a/arch/h8300/kernel/vmlinux.lds.S +++ b/arch/h8300/kernel/vmlinux.lds.S @@ -101,7 +101,7 @@ SECTIONS ___data_start = . ; . = ALIGN(0x2000) ; - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(0x4) ; DATA_DATA . = ALIGN(0x4) ; diff --git a/arch/ia64/include/asm/asmmacro.h b/arch/ia64/include/asm/asmmacro.h index c1642fd..30c8715 100644 --- a/arch/ia64/include/asm/asmmacro.h +++ b/arch/ia64/include/asm/asmmacro.h @@ -70,12 +70,12 @@ name: * path (ivt.S - TLB miss processing) or in places where it might not be * safe to use a "tpa" instruction (mca_asm.S - error recovery). */ - .section ".data.patch.vtop", "a" // declare section & section attributes + .section ".kernel.data.patch.vtop", "a" // declare section & section attributes .previous #define LOAD_PHYSICAL(pr, reg, obj) \ [1:](pr)movl reg = obj; \ - .xdata4 ".data.patch.vtop", 1b-. + .xdata4 ".kernel.data.patch.vtop", 1b-. /* * For now, we always put in the McKinley E9 workaround. On CPUs that don't need it, @@ -84,11 +84,11 @@ name: #define DO_MCKINLEY_E9_WORKAROUND #ifdef DO_MCKINLEY_E9_WORKAROUND - .section ".data.patch.mckinley_e9", "a" + .section ".kernel.data.patch.mckinley_e9", "a" .previous /* workaround for Itanium 2 Errata 9: */ # define FSYS_RETURN \ - .xdata4 ".data.patch.mckinley_e9", 1f-.; \ + .xdata4 ".kernel.data.patch.mckinley_e9", 1f-.; \ 1:{ .mib; \ nop.m 0; \ mov r16=ar.pfs; \ @@ -107,11 +107,11 @@ name: * If physical stack register size is different from DEF_NUM_STACK_REG, * dynamically patch the kernel for correct size. */ - .section ".data.patch.phys_stack_reg", "a" + .section ".kernel.data.patch.phys_stack_reg", "a" .previous #define LOAD_PHYS_STACK_REG_SIZE(reg) \ [1:] adds reg=IA64_NUM_PHYS_STACK_REG*8+8,r0; \ - .xdata4 ".data.patch.phys_stack_reg", 1b-. + .xdata4 ".kernel.data.patch.phys_stack_reg", 1b-. /* * Up until early 2004, use of .align within a function caused bad unwind info. diff --git a/arch/ia64/include/asm/cache.h b/arch/ia64/include/asm/cache.h index e7482bd..7c8d940 100644 --- a/arch/ia64/include/asm/cache.h +++ b/arch/ia64/include/asm/cache.h @@ -24,6 +24,6 @@ # define SMP_CACHE_BYTES (1 << 3) #endif -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif /* _ASM_IA64_CACHE_H */ diff --git a/arch/ia64/include/asm/percpu.h b/arch/ia64/include/asm/percpu.h index 77f30b6..b4a5a2e 100644 --- a/arch/ia64/include/asm/percpu.h +++ b/arch/ia64/include/asm/percpu.h @@ -27,7 +27,7 @@ extern void *per_cpu_init(void); #else /* ! SMP */ -#define PER_CPU_ATTRIBUTES __attribute__((__section__(".data.percpu"))) +#define PER_CPU_ATTRIBUTES __attribute__((__section__(".kernel.data.percpu"))) #define per_cpu_init() (__phys_per_cpu_start) diff --git a/arch/ia64/kernel/Makefile b/arch/ia64/kernel/Makefile index c381ea9..610572a 100644 --- a/arch/ia64/kernel/Makefile +++ b/arch/ia64/kernel/Makefile @@ -72,7 +72,7 @@ GATECFLAGS_gate-syms.o = -r $(obj)/gate-syms.o: $(obj)/gate.lds $(obj)/gate.o FORCE $(call if_changed,gate) -# gate-data.o contains the gate DSO image as data in section .data.gate. +# gate-data.o contains the gate DSO image as data in section .kernel.data.gate. # We must build gate.so before we can assemble it. # Note: kbuild does not track this dependency due to usage of .incbin $(obj)/gate-data.o: $(obj)/gate.so diff --git a/arch/ia64/kernel/gate-data.S b/arch/ia64/kernel/gate-data.S index 258c0a3..7cfa6a2 100644 --- a/arch/ia64/kernel/gate-data.S +++ b/arch/ia64/kernel/gate-data.S @@ -1,3 +1,3 @@ - .section .data.gate, "aw" + .section .kernel.data.gate, "aw" .incbin "arch/ia64/kernel/gate.so" diff --git a/arch/ia64/kernel/gate.S b/arch/ia64/kernel/gate.S index 74b1ccc..eddece2 100644 --- a/arch/ia64/kernel/gate.S +++ b/arch/ia64/kernel/gate.S @@ -20,18 +20,18 @@ * to targets outside the shared object) and to avoid multi-phase kernel builds, we * simply create minimalistic "patch lists" in special ELF sections. */ - .section ".data.patch.fsyscall_table", "a" + .section ".kernel.data.patch.fsyscall_table", "a" .previous #define LOAD_FSYSCALL_TABLE(reg) \ [1:] movl reg=0; \ - .xdata4 ".data.patch.fsyscall_table", 1b-. + .xdata4 ".kernel.data.patch.fsyscall_table", 1b-. - .section ".data.patch.brl_fsys_bubble_down", "a" + .section ".kernel.data.patch.brl_fsys_bubble_down", "a" .previous #define BRL_COND_FSYS_BUBBLE_DOWN(pr) \ [1:](pr)brl.cond.sptk 0; \ ;; \ - .xdata4 ".data.patch.brl_fsys_bubble_down", 1b-. + .xdata4 ".kernel.data.patch.brl_fsys_bubble_down", 1b-. GLOBAL_ENTRY(__kernel_syscall_via_break) .prologue diff --git a/arch/ia64/kernel/gate.lds.S b/arch/ia64/kernel/gate.lds.S index 3cb1abc..9a41f12 100644 --- a/arch/ia64/kernel/gate.lds.S +++ b/arch/ia64/kernel/gate.lds.S @@ -32,21 +32,21 @@ SECTIONS */ . = GATE_ADDR + 0x600; - .data.patch : { + .kernel.data.patch : { __start_gate_mckinley_e9_patchlist = .; - *(.data.patch.mckinley_e9) + *(.kernel.data.patch.mckinley_e9) __end_gate_mckinley_e9_patchlist = .; __start_gate_vtop_patchlist = .; - *(.data.patch.vtop) + *(.kernel.data.patch.vtop) __end_gate_vtop_patchlist = .; __start_gate_fsyscall_patchlist = .; - *(.data.patch.fsyscall_table) + *(.kernel.data.patch.fsyscall_table) __end_gate_fsyscall_patchlist = .; __start_gate_brl_fsys_bubble_down_patchlist = .; - *(.data.patch.brl_fsys_bubble_down) + *(.kernel.data.patch.brl_fsys_bubble_down) __end_gate_brl_fsys_bubble_down_patchlist = .; } :readable diff --git a/arch/ia64/kernel/head.S b/arch/ia64/kernel/head.S index 59301c4..a88dc24 100644 --- a/arch/ia64/kernel/head.S +++ b/arch/ia64/kernel/head.S @@ -181,7 +181,7 @@ swapper_pg_dir: halt_msg: stringz "Halting kernel\n" - .section .text.head,"ax" + .section .kernel.text.head,"ax" .global start_ap diff --git a/arch/ia64/kernel/init_task.c b/arch/ia64/kernel/init_task.c index 5b0e830..0e8c57c 100644 --- a/arch/ia64/kernel/init_task.c +++ b/arch/ia64/kernel/init_task.c @@ -27,7 +27,7 @@ EXPORT_SYMBOL(init_mm); * Initial task structure. * * We need to make sure that this is properly aligned due to the way process stacks are - * handled. This is done by having a special ".data.init_task" section... + * handled. This is done by having a special ".kernel.data.init_task" section... */ #define init_thread_info init_task_mem.s.thread_info @@ -37,7 +37,7 @@ union { struct thread_info thread_info; } s; unsigned long stack[KERNEL_STACK_SIZE/sizeof (unsigned long)]; -} init_task_mem asm ("init_task") __attribute__((section(".data.init_task"))) = {{ +} init_task_mem asm ("init_task") __attribute__((section(".kernel.data.init_task"))) = {{ .task = INIT_TASK(init_task_mem.s.task), .thread_info = INIT_THREAD_INFO(init_task_mem.s.task) }}; diff --git a/arch/ia64/kernel/ivt.S b/arch/ia64/kernel/ivt.S index f675d8e..5bd4502 100644 --- a/arch/ia64/kernel/ivt.S +++ b/arch/ia64/kernel/ivt.S @@ -83,7 +83,7 @@ mov r19=n;; /* prepare to save predicates */ \ br.sptk.many dispatch_to_fault_handler - .section .text.ivt,"ax" + .section .kernel.text.ivt,"ax" .align 32768 // align on 32KB boundary .global ia64_ivt diff --git a/arch/ia64/kernel/minstate.h b/arch/ia64/kernel/minstate.h index 292e214..9bcff80 100644 --- a/arch/ia64/kernel/minstate.h +++ b/arch/ia64/kernel/minstate.h @@ -16,7 +16,7 @@ #define ACCOUNT_SYS_ENTER #endif -.section ".data.patch.rse", "a" +.section ".kernel.data.patch.rse", "a" .previous /* @@ -215,7 +215,7 @@ (pUStk) extr.u r17=r18,3,6; \ (pUStk) sub r16=r18,r22; \ [1:](pKStk) br.cond.sptk.many 1f; \ - .xdata4 ".data.patch.rse",1b-. \ + .xdata4 ".kernel.data.patch.rse",1b-. \ ;; \ cmp.ge p6,p7 = 33,r17; \ ;; \ diff --git a/arch/ia64/kernel/paravirtentry.S b/arch/ia64/kernel/paravirtentry.S index 2f42fcb..418a5f4 100644 --- a/arch/ia64/kernel/paravirtentry.S +++ b/arch/ia64/kernel/paravirtentry.S @@ -24,12 +24,12 @@ #include <asm/asm-offsets.h> #include "entry.h" -#define DATA8(sym, init_value) \ - .pushsection .data.read_mostly ; \ - .align 8 ; \ - .global sym ; \ - sym: ; \ - data8 init_value ; \ +#define DATA8(sym, init_value) \ + .pushsection .kernel.data.read_mostly,"aw",@progbits ; \ + .align 8 ; \ + .global sym ; \ + sym: ; \ + data8 init_value ; \ .popsection #define BRANCH(targ, reg, breg) \ diff --git a/arch/ia64/kernel/vmlinux.lds.S b/arch/ia64/kernel/vmlinux.lds.S index 10a7d47..6e4e8a2 100644 --- a/arch/ia64/kernel/vmlinux.lds.S +++ b/arch/ia64/kernel/vmlinux.lds.S @@ -8,7 +8,7 @@ #define IVT_TEXT \ VMLINUX_SYMBOL(__start_ivt_text) = .; \ - *(.text.ivt) \ + *(.kernel.text.ivt) \ VMLINUX_SYMBOL(__end_ivt_text) = .; OUTPUT_FORMAT("elf64-ia64-little") @@ -51,13 +51,13 @@ SECTIONS KPROBES_TEXT *(.gnu.linkonce.t*) } - .text.head : AT(ADDR(.text.head) - LOAD_OFFSET) - { *(.text.head) } + .kernel.text.head : AT(ADDR(.kernel.text.head) - LOAD_OFFSET) + { *(.kernel.text.head) } .text2 : AT(ADDR(.text2) - LOAD_OFFSET) { *(.text2) } #ifdef CONFIG_SMP - .text.lock : AT(ADDR(.text.lock) - LOAD_OFFSET) - { *(.text.lock) } + .kernel.text.lock : AT(ADDR(.kernel.text.lock) - LOAD_OFFSET) + { *(.kernel.text.lock) } #endif _etext = .; @@ -84,10 +84,10 @@ SECTIONS __stop___mca_table = .; } - .data.patch.phys_stack_reg : AT(ADDR(.data.patch.phys_stack_reg) - LOAD_OFFSET) + .kernel.data.patch.phys_stack_reg : AT(ADDR(.kernel.data.patch.phys_stack_reg) - LOAD_OFFSET) { __start___phys_stack_reg_patchlist = .; - *(.data.patch.phys_stack_reg) + *(.kernel.data.patch.phys_stack_reg) __end___phys_stack_reg_patchlist = .; } @@ -148,24 +148,24 @@ SECTIONS __initcall_end = .; } - .data.patch.vtop : AT(ADDR(.data.patch.vtop) - LOAD_OFFSET) + .kernel.data.patch.vtop : AT(ADDR(.kernel.data.patch.vtop) - LOAD_OFFSET) { __start___vtop_patchlist = .; - *(.data.patch.vtop) + *(.kernel.data.patch.vtop) __end___vtop_patchlist = .; } - .data.patch.rse : AT(ADDR(.data.patch.rse) - LOAD_OFFSET) + .kernel.data.patch.rse : AT(ADDR(.kernel.data.patch.rse) - LOAD_OFFSET) { __start___rse_patchlist = .; - *(.data.patch.rse) + *(.kernel.data.patch.rse) __end___rse_patchlist = .; } - .data.patch.mckinley_e9 : AT(ADDR(.data.patch.mckinley_e9) - LOAD_OFFSET) + .kernel.data.patch.mckinley_e9 : AT(ADDR(.kernel.data.patch.mckinley_e9) - LOAD_OFFSET) { __start___mckinley_e9_bundles = .; - *(.data.patch.mckinley_e9) + *(.kernel.data.patch.mckinley_e9) __end___mckinley_e9_bundles = .; } @@ -193,34 +193,34 @@ SECTIONS __init_end = .; /* The initial task and kernel stack */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) - { *(.data.init_task) } + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) + { *(.kernel.data.init_task) } - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { *(__special_page_section) __start_gate_section = .; - *(.data.gate) + *(.kernel.data.gate) __stop_gate_section = .; } . = ALIGN(PAGE_SIZE); /* make sure the gate page doesn't expose * kernel data */ - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) - { *(.data.read_mostly) } + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) + { *(.kernel.data.read_mostly) } - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) - { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) + { *(.kernel.data.cacheline_aligned) } /* Per-cpu data: */ percpu : { } :percpu . = ALIGN(PERCPU_PAGE_SIZE); __phys_per_cpu_start = .; - .data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) + .kernel.data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } . = __phys_per_cpu_start + PERCPU_PAGE_SIZE; /* ensure percpu data fits diff --git a/arch/ia64/kvm/vmm_ivt.S b/arch/ia64/kvm/vmm_ivt.S index c1d7251..935295e 100644 --- a/arch/ia64/kvm/vmm_ivt.S +++ b/arch/ia64/kvm/vmm_ivt.S @@ -97,7 +97,7 @@ END(kvm_panic) - .section .text.ivt,"ax" + .section .kernel.text.ivt,"ax" .align 32768 // align on 32KB boundary .global kvm_ia64_ivt diff --git a/arch/ia64/xen/xensetup.S b/arch/ia64/xen/xensetup.S index 28fed1f..ebeadf4 100644 --- a/arch/ia64/xen/xensetup.S +++ b/arch/ia64/xen/xensetup.S @@ -14,7 +14,7 @@ #include <linux/init.h> #include <xen/interface/elfnote.h> - .section .data.read_mostly + .section .kernel.data.read_mostly, "a" .align 8 .global xen_domain_type xen_domain_type: diff --git a/arch/m32r/kernel/head.S b/arch/m32r/kernel/head.S index 9091606..6a1fc27 100644 --- a/arch/m32r/kernel/head.S +++ b/arch/m32r/kernel/head.S @@ -23,7 +23,7 @@ __INITDATA /* * References to members of the boot_cpu_data structure. */ -.section .text.head, "ax" +.section .kernel.text.head, "ax" .global start_kernel .global __bss_start .global _end diff --git a/arch/m32r/kernel/init_task.c b/arch/m32r/kernel/init_task.c index 016885c..2b8c780 100644 --- a/arch/m32r/kernel/init_task.c +++ b/arch/m32r/kernel/init_task.c @@ -25,7 +25,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/m32r/kernel/vmlinux.lds.S b/arch/m32r/kernel/vmlinux.lds.S index 9db05df..d0bb3d6 100644 --- a/arch/m32r/kernel/vmlinux.lds.S +++ b/arch/m32r/kernel/vmlinux.lds.S @@ -27,7 +27,7 @@ SECTIONS _text = .; /* Text and read-only data */ .boot : { *(.boot) } = 0 .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -57,17 +57,17 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : { *(.kernel.data.nosave) } . = ALIGN(4096); __nosave_end = .; . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } _edata = .; /* End of data section */ . = ALIGN(8192); /* init_task */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* will be freed after init */ . = ALIGN(4096); /* Init code and data */ diff --git a/arch/m68k/kernel/head.S b/arch/m68k/kernel/head.S index f513f53..eb51357 100644 --- a/arch/m68k/kernel/head.S +++ b/arch/m68k/kernel/head.S @@ -577,7 +577,7 @@ func_define putn,1 #endif .endm -.section ".text.head","ax" +.section ".kernel.text.head","ax" ENTRY(_stext) /* * Version numbers of the bootinfo interface diff --git a/arch/m68k/kernel/process.c b/arch/m68k/kernel/process.c index 632ce01..29f9872 100644 --- a/arch/m68k/kernel/process.c +++ b/arch/m68k/kernel/process.c @@ -47,7 +47,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); EXPORT_SYMBOL(init_mm); union thread_union init_thread_union -__attribute__((section(".data.init_task"), aligned(THREAD_SIZE))) +__attribute__((section(".kernel.data.init_task"), aligned(THREAD_SIZE))) = { INIT_THREAD_INFO(init_task) }; /* initial task structure */ diff --git a/arch/m68k/kernel/sun3-head.S b/arch/m68k/kernel/sun3-head.S index aad0159..28d3a41 100644 --- a/arch/m68k/kernel/sun3-head.S +++ b/arch/m68k/kernel/sun3-head.S @@ -29,7 +29,7 @@ kernel_pmd_table: .skip 0x2000 .globl kernel_pg_dir .equ kernel_pg_dir,kernel_pmd_table - .section .text.head + .section .kernel.text.head, "ax" ENTRY(_stext) ENTRY(_start) diff --git a/arch/m68k/kernel/vmlinux-std.lds b/arch/m68k/kernel/vmlinux-std.lds index f846d4e..dc1e630 100644 --- a/arch/m68k/kernel/vmlinux-std.lds +++ b/arch/m68k/kernel/vmlinux-std.lds @@ -12,7 +12,7 @@ SECTIONS . = 0x1000; _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -35,7 +35,7 @@ SECTIONS } . = ALIGN(16); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } :data .bss : { *(.bss) } /* BSS */ @@ -78,7 +78,7 @@ SECTIONS . = ALIGN(8192); __init_end = .; - .data.init_task : { *(.data.init_task) } /* The initial task and kernel stack */ + .kernel.data.init_task : { *(.kernel.data.init_task) } /* The initial task and kernel stack */ _end = . ; diff --git a/arch/m68k/kernel/vmlinux-sun3.lds b/arch/m68k/kernel/vmlinux-sun3.lds index 8a4919e..2c55ffc 100644 --- a/arch/m68k/kernel/vmlinux-sun3.lds +++ b/arch/m68k/kernel/vmlinux-sun3.lds @@ -12,7 +12,7 @@ SECTIONS . = 0xE002000; _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -69,7 +69,7 @@ __init_begin = .; #endif . = ALIGN(PAGE_SIZE); __init_end = .; - .data.init.task : { *(.data.init_task) } + .kernel.data.init.task : { *(.kernel.data.init_task) } .bss : { *(.bss) } /* BSS */ diff --git a/arch/m68knommu/kernel/init_task.c b/arch/m68knommu/kernel/init_task.c index fe282de..b606519 100644 --- a/arch/m68knommu/kernel/init_task.c +++ b/arch/m68knommu/kernel/init_task.c @@ -36,6 +36,6 @@ EXPORT_SYMBOL(init_task); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/m68knommu/kernel/vmlinux.lds.S b/arch/m68knommu/kernel/vmlinux.lds.S index 69ba9b1..d5d617c 100644 --- a/arch/m68knommu/kernel/vmlinux.lds.S +++ b/arch/m68knommu/kernel/vmlinux.lds.S @@ -55,7 +55,7 @@ SECTIONS { .romvec : { __rom_start = . ; _romvec = .; - *(.data.initvect) + *(.kernel.data.initvect) } > romvec #endif @@ -66,7 +66,7 @@ SECTIONS { TEXT_TEXT SCHED_TEXT LOCK_TEXT - *(.text.lock) + *(.kernel.text.lock) . = ALIGN(16); /* Exception table */ __start___ex_table = .; @@ -148,7 +148,7 @@ SECTIONS { _sdata = . ; DATA_DATA . = ALIGN(8192) ; - *(.data.init_task) + *(.kernel.data.init_task) _edata = . ; } > DATA diff --git a/arch/m68knommu/platform/68360/head-ram.S b/arch/m68knommu/platform/68360/head-ram.S index 2ef0624..88a02ed 100644 --- a/arch/m68knommu/platform/68360/head-ram.S +++ b/arch/m68knommu/platform/68360/head-ram.S @@ -280,7 +280,7 @@ _dprbase: * and then overwritten as needed. */ -.section ".data.initvect","awx" +.section ".kernel.data.initvect","awx" .long RAMEND /* Reset: Initial Stack Pointer - 0. */ .long _start /* Reset: Initial Program Counter - 1. */ .long buserr /* Bus Error - 2. */ diff --git a/arch/m68knommu/platform/68360/head-rom.S b/arch/m68knommu/platform/68360/head-rom.S index 62ecf41..b4da6af 100644 --- a/arch/m68knommu/platform/68360/head-rom.S +++ b/arch/m68knommu/platform/68360/head-rom.S @@ -291,7 +291,7 @@ _dprbase: * and then overwritten as needed. */ -.section ".data.initvect","awx" +.section ".kernel.data.initvect","awx" .long RAMEND /* Reset: Initial Stack Pointer - 0. */ .long _start /* Reset: Initial Program Counter - 1. */ .long buserr /* Bus Error - 2. */ diff --git a/arch/mips/kernel/init_task.c b/arch/mips/kernel/init_task.c index 149cd91..425fa1f 100644 --- a/arch/mips/kernel/init_task.c +++ b/arch/mips/kernel/init_task.c @@ -26,7 +26,7 @@ EXPORT_SYMBOL(init_mm); * The things we do for performance.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"), + __attribute__((__section__(".kernel.data.init_task"), __aligned__(THREAD_SIZE))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/mips/kernel/vmlinux.lds.S b/arch/mips/kernel/vmlinux.lds.S index 58738c8..c9a89d8 100644 --- a/arch/mips/kernel/vmlinux.lds.S +++ b/arch/mips/kernel/vmlinux.lds.S @@ -77,7 +77,7 @@ SECTIONS * object file alignment. Using 32768 */ . = ALIGN(_PAGE_SIZE); - *(.data.init_task) + *(.kernel.data.init_task) DATA_DATA CONSTRUCTORS @@ -99,14 +99,14 @@ SECTIONS . = ALIGN(_PAGE_SIZE); .data_nosave : { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(_PAGE_SIZE); __nosave_end = .; . = ALIGN(1 << CONFIG_MIPS_L1_CACHE_SHIFT); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } _edata = .; /* End of data section */ diff --git a/arch/mips/lasat/image/head.S b/arch/mips/lasat/image/head.S index efb95f2..d2e08b3 100644 --- a/arch/mips/lasat/image/head.S +++ b/arch/mips/lasat/image/head.S @@ -1,7 +1,7 @@ #include <asm/lasat/head.h> .text - .section .text.start, "ax" + .section .kernel.text.start, "ax" .set noreorder .set mips3 diff --git a/arch/mips/lasat/image/romscript.normal b/arch/mips/lasat/image/romscript.normal index 988f8ad..9429398 100644 --- a/arch/mips/lasat/image/romscript.normal +++ b/arch/mips/lasat/image/romscript.normal @@ -4,7 +4,7 @@ SECTIONS { .text : { - *(.text.start) + *(.kernel.text.start) } /* Data in ROM */ diff --git a/arch/mn10300/kernel/head.S b/arch/mn10300/kernel/head.S index 606bd8c..1b4b275 100644 --- a/arch/mn10300/kernel/head.S +++ b/arch/mn10300/kernel/head.S @@ -19,7 +19,7 @@ #include <asm/param.h> #include <asm/unit/serial.h> - .section .text.head,"ax" + .section .kernel.text.head,"ax" ############################################################################### # diff --git a/arch/mn10300/kernel/init_task.c b/arch/mn10300/kernel/init_task.c index 5ac3566..0758121 100644 --- a/arch/mn10300/kernel/init_task.c +++ b/arch/mn10300/kernel/init_task.c @@ -31,7 +31,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/mn10300/kernel/vmlinux.lds.S b/arch/mn10300/kernel/vmlinux.lds.S index b825966..6d139e2 100644 --- a/arch/mn10300/kernel/vmlinux.lds.S +++ b/arch/mn10300/kernel/vmlinux.lds.S @@ -28,7 +28,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : { *( - .text.head + .kernel.text.head .text ) TEXT_TEXT @@ -58,25 +58,25 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : { *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; . = ALIGN(PAGE_SIZE); - .data.page_aligned : { *(.data.idt) } + .kernel.data.page_aligned : { *(.kernel.data.idt) } . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } /* rarely changed data like cpu maps */ . = ALIGN(32); - .data.read_mostly : AT(ADDR(.data.read_mostly)) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly)) { + *(.kernel.data.read_mostly) _edata = .; /* End of data section */ } . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* might get freed after init */ . = ALIGN(PAGE_SIZE); @@ -134,7 +134,7 @@ SECTIONS __bss_start = .; /* BSS */ .bss : { - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) } . = ALIGN(4); diff --git a/arch/parisc/include/asm/cache.h b/arch/parisc/include/asm/cache.h index 32c2cca..d1e16aa 100644 --- a/arch/parisc/include/asm/cache.h +++ b/arch/parisc/include/asm/cache.h @@ -28,7 +28,7 @@ #define SMP_CACHE_BYTES L1_CACHE_BYTES -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) void parisc_cache_init(void); /* initializes cache-flushing */ void disable_sr_hashing_asm(int); /* low level support for above */ diff --git a/arch/parisc/include/asm/system.h b/arch/parisc/include/asm/system.h index ee80c92..7a6ac0c 100644 --- a/arch/parisc/include/asm/system.h +++ b/arch/parisc/include/asm/system.h @@ -174,7 +174,7 @@ static inline void set_eiem(unsigned long val) }) #ifdef CONFIG_SMP -# define __lock_aligned __attribute__((__section__(".data.lock_aligned"))) +# define __lock_aligned __attribute__((__section__(".kernel.data.lock_aligned"))) #endif #define arch_align_stack(x) (x) diff --git a/arch/parisc/kernel/head.S b/arch/parisc/kernel/head.S index 0e3d9f9..9112df2 100644 --- a/arch/parisc/kernel/head.S +++ b/arch/parisc/kernel/head.S @@ -345,7 +345,7 @@ smp_slave_stext: ENDPROC(stext) #ifndef CONFIG_64BIT - .section .data.read_mostly + .section .kernel.data.read_mostly,"aw",@progbits .align 4 .export $global$,data diff --git a/arch/parisc/kernel/init_task.c b/arch/parisc/kernel/init_task.c index 1e25a45..97aceac 100644 --- a/arch/parisc/kernel/init_task.c +++ b/arch/parisc/kernel/init_task.c @@ -48,7 +48,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((aligned(128))) __attribute__((__section__(".data.init_task"))) = + __attribute__((aligned(128))) __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; #if PT_NLEVELS == 3 @@ -57,11 +57,11 @@ union thread_union init_thread_union * guarantee that global objects will be laid out in memory in the same order * as the order of declaration, so put these in different sections and use * the linker script to order them. */ -pmd_t pmd0[PTRS_PER_PMD] __attribute__ ((__section__ (".data.vm0.pmd"), aligned(PAGE_SIZE))); +pmd_t pmd0[PTRS_PER_PMD] __attribute__ ((__section__ (".kernel.data.vm0.pmd"), aligned(PAGE_SIZE))); #endif -pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__ ((__section__ (".data.vm0.pgd"), aligned(PAGE_SIZE))); -pte_t pg0[PT_INITIAL * PTRS_PER_PTE] __attribute__ ((__section__ (".data.vm0.pte"), aligned(PAGE_SIZE))); +pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__ ((__section__ (".kernel.data.vm0.pgd"), aligned(PAGE_SIZE))); +pte_t pg0[PT_INITIAL * PTRS_PER_PTE] __attribute__ ((__section__ (".kernel.data.vm0.pte"), aligned(PAGE_SIZE))); /* * Initial task structure. diff --git a/arch/parisc/kernel/vmlinux.lds.S b/arch/parisc/kernel/vmlinux.lds.S index 1a3b6cc..861f15b 100644 --- a/arch/parisc/kernel/vmlinux.lds.S +++ b/arch/parisc/kernel/vmlinux.lds.S @@ -94,8 +94,8 @@ SECTIONS /* rarely changed data like cpu maps */ . = ALIGN(16); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } . = ALIGN(L1_CACHE_BYTES); @@ -106,14 +106,14 @@ SECTIONS } . = ALIGN(L1_CACHE_BYTES); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } /* PA-RISC locks requires 16-byte alignment */ . = ALIGN(16); - .data.lock_aligned : { - *(.data.lock_aligned) + .kernel.data.lock_aligned : { + *(.kernel.data.lock_aligned) } /* nosave data is really only used for software suspend...it's here @@ -122,7 +122,7 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; .data_nosave : { - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -134,10 +134,10 @@ SECTIONS __bss_start = .; /* page table entries need to be PAGE_SIZE aligned */ . = ALIGN(PAGE_SIZE); - .data.vmpages : { - *(.data.vm0.pmd) - *(.data.vm0.pgd) - *(.data.vm0.pte) + .kernel.data.vmpages : { + *(.kernel.data.vm0.pmd) + *(.kernel.data.vm0.pgd) + *(.kernel.data.vm0.pte) } .bss : { *(.bss) @@ -149,8 +149,8 @@ SECTIONS /* assembler code expects init_task to be 16k aligned */ . = ALIGN(16384); /* init_task */ - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } #ifdef CONFIG_64BIT diff --git a/arch/powerpc/include/asm/cache.h b/arch/powerpc/include/asm/cache.h index 81de6eb..69ead95 100644 --- a/arch/powerpc/include/asm/cache.h +++ b/arch/powerpc/include/asm/cache.h @@ -38,7 +38,7 @@ extern struct ppc64_caches ppc64_caches; #endif /* __powerpc64__ && ! __ASSEMBLY__ */ #if !defined(__ASSEMBLY__) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif #endif /* __KERNEL__ */ diff --git a/arch/powerpc/include/asm/page_64.h b/arch/powerpc/include/asm/page_64.h index 043bfdf..2e5f76b 100644 --- a/arch/powerpc/include/asm/page_64.h +++ b/arch/powerpc/include/asm/page_64.h @@ -157,7 +157,7 @@ do { \ #else #define __page_aligned \ __attribute__((__aligned__(PAGE_SIZE), \ - __section__(".data.page_aligned"))) + __section__(".kernel.data.page_aligned"))) #endif #define VM_DATA_DEFAULT_FLAGS \ diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h index 1a0d628..c115bc7 100644 --- a/arch/powerpc/include/asm/ppc_asm.h +++ b/arch/powerpc/include/asm/ppc_asm.h @@ -193,7 +193,7 @@ name: \ GLUE(.,name): #define _INIT_GLOBAL(name) \ - .section ".text.init.refok"; \ + .section ".kernel.text.init.refok","ax",@progbits; \ .align 2 ; \ .globl name; \ .globl GLUE(.,name); \ @@ -233,7 +233,7 @@ name: \ GLUE(.,name): #define _INIT_STATIC(name) \ - .section ".text.init.refok"; \ + .section ".kernel.text.init.refok","ax",@progbits; \ .align 2 ; \ .section ".opd","aw"; \ name: \ diff --git a/arch/powerpc/kernel/head_32.S b/arch/powerpc/kernel/head_32.S index a1c4cfd..fe8bbb1 100644 --- a/arch/powerpc/kernel/head_32.S +++ b/arch/powerpc/kernel/head_32.S @@ -50,7 +50,7 @@ mtspr SPRN_DBAT##n##L,RB; \ 1: - .section .text.head, "ax" + .section .kernel.text.head, "ax" .stabs "arch/powerpc/kernel/",N_SO,0,0,0f .stabs "head_32.S",N_SO,0,0,0f 0: diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 56d8e5d..d18a199 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -52,7 +52,7 @@ * * This is all going to change RSN when we add bi_recs....... -- Dan */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); diff --git a/arch/powerpc/kernel/head_44x.S b/arch/powerpc/kernel/head_44x.S index b56fecc..1c88bcc 100644 --- a/arch/powerpc/kernel/head_44x.S +++ b/arch/powerpc/kernel/head_44x.S @@ -50,7 +50,7 @@ * r7 - End of kernel command line string * */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); /* diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index 3c9452d..549a236 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -38,7 +38,7 @@ #else #define DO_8xx_CPU6(val, reg) #endif - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S index 11b549a..f9af5b2 100644 --- a/arch/powerpc/kernel/head_fsl_booke.S +++ b/arch/powerpc/kernel/head_fsl_booke.S @@ -53,7 +53,7 @@ * r7 - End of kernel command line string * */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); /* diff --git a/arch/powerpc/kernel/init_task.c b/arch/powerpc/kernel/init_task.c index 688b329..bac0699 100644 --- a/arch/powerpc/kernel/init_task.c +++ b/arch/powerpc/kernel/init_task.c @@ -21,7 +21,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/powerpc/kernel/machine_kexec_64.c b/arch/powerpc/kernel/machine_kexec_64.c index 49e705f..3cd7b25 100644 --- a/arch/powerpc/kernel/machine_kexec_64.c +++ b/arch/powerpc/kernel/machine_kexec_64.c @@ -250,7 +250,7 @@ static void kexec_prepare_cpus(void) * current, but that audit has not been performed. */ static union thread_union kexec_stack - __attribute__((__section__(".data.init_task"))) = { }; + __attribute__((__section__(".kernel.data.init_task"))) = { }; /* Our assembly helper, in kexec_stub.S */ extern NORET_TYPE void kexec_sequence(void *newstack, unsigned long start, diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c index ad06d5c..f4130cc 100644 --- a/arch/powerpc/kernel/vdso.c +++ b/arch/powerpc/kernel/vdso.c @@ -74,7 +74,7 @@ static int vdso_ready; static union { struct vdso_data data; u8 page[PAGE_SIZE]; -} vdso_data_store __attribute__((__section__(".data.page_aligned"))); +} vdso_data_store __attribute__((__section__(".kernel.data.page_aligned"))); struct vdso_data *vdso_data = &vdso_data_store.data; /* Format of the patch table */ diff --git a/arch/powerpc/kernel/vdso32/vdso32_wrapper.S b/arch/powerpc/kernel/vdso32/vdso32_wrapper.S index 556f0ca..e1b3e41 100644 --- a/arch/powerpc/kernel/vdso32/vdso32_wrapper.S +++ b/arch/powerpc/kernel/vdso32/vdso32_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned","aw",@progbits .globl vdso32_start, vdso32_end .balign PAGE_SIZE diff --git a/arch/powerpc/kernel/vdso64/vdso64_wrapper.S b/arch/powerpc/kernel/vdso64/vdso64_wrapper.S index 0529cb9..1d96b15 100644 --- a/arch/powerpc/kernel/vdso64/vdso64_wrapper.S +++ b/arch/powerpc/kernel/vdso64/vdso64_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned","aw",@progbits .globl vdso64_start, vdso64_end .balign PAGE_SIZE diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S index 47bf15c..e8e82fb 100644 --- a/arch/powerpc/kernel/vmlinux.lds.S +++ b/arch/powerpc/kernel/vmlinux.lds.S @@ -52,9 +52,9 @@ SECTIONS /* Text and gots */ .text : AT(ADDR(.text) - LOAD_OFFSET) { ALIGN_FUNCTION(); - *(.text.head) + *(.kernel.text.head) _text = .; - *(.text .fixup .text.init.refok .exit.text.refok __ftr_alt_*) + *(.text .fixup .kernel.text.init.refok .kernel.text.exit.refok __ftr_alt_*) SCHED_TEXT LOCK_TEXT KPROBES_TEXT @@ -180,10 +180,10 @@ SECTIONS } #endif . = ALIGN(PAGE_SIZE); - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } @@ -257,28 +257,28 @@ SECTIONS #else . = ALIGN(16384); #endif - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) } . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) } - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } . = ALIGN(L1_CACHE_BYTES); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) } . = ALIGN(PAGE_SIZE); .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; } diff --git a/arch/s390/include/asm/cache.h b/arch/s390/include/asm/cache.h index 9b86681..2890a0f 100644 --- a/arch/s390/include/asm/cache.h +++ b/arch/s390/include/asm/cache.h @@ -14,6 +14,6 @@ #define L1_CACHE_BYTES 256 #define L1_CACHE_SHIFT 8 -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif diff --git a/arch/s390/kernel/head.S b/arch/s390/kernel/head.S index ec7e35f..cd2bfb7 100644 --- a/arch/s390/kernel/head.S +++ b/arch/s390/kernel/head.S @@ -35,7 +35,7 @@ #define ARCH_OFFSET 0 #endif -.section ".text.head","ax" +.section ".kernel.text.head","ax" #ifndef CONFIG_IPL .org 0 .long 0x00080000,0x80000000+startup # Just a restart PSW diff --git a/arch/s390/kernel/init_task.c b/arch/s390/kernel/init_task.c index 7db95c0..1615535 100644 --- a/arch/s390/kernel/init_task.c +++ b/arch/s390/kernel/init_task.c @@ -30,7 +30,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/s390/kernel/vdso32/vdso32_wrapper.S b/arch/s390/kernel/vdso32/vdso32_wrapper.S index 61639a8..23830ec 100644 --- a/arch/s390/kernel/vdso32/vdso32_wrapper.S +++ b/arch/s390/kernel/vdso32/vdso32_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned", "aw" .globl vdso32_start, vdso32_end .balign PAGE_SIZE diff --git a/arch/s390/kernel/vdso64/vdso64_wrapper.S b/arch/s390/kernel/vdso64/vdso64_wrapper.S index d8e2ac1..6fb0923 100644 --- a/arch/s390/kernel/vdso64/vdso64_wrapper.S +++ b/arch/s390/kernel/vdso64/vdso64_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned", "aw" .globl vdso64_start, vdso64_end .balign PAGE_SIZE diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S index d796d05..873cac2 100644 --- a/arch/s390/kernel/vmlinux.lds.S +++ b/arch/s390/kernel/vmlinux.lds.S @@ -29,7 +29,7 @@ SECTIONS . = 0x00000000; .text : { _text = .; /* Text and read-only data */ - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -66,30 +66,30 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data_nosave : { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; . = ALIGN(PAGE_SIZE); - .data.page_aligned : { - *(.data.idt) + .kernel.data.page_aligned : { + *(.kernel.data.idt) } . = ALIGN(0x100); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } . = ALIGN(0x100); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } _edata = .; /* End of data section */ . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } /* will be freed after init */ diff --git a/arch/sh/include/asm/cache.h b/arch/sh/include/asm/cache.h index 02df18e..f44fed5 100644 --- a/arch/sh/include/asm/cache.h +++ b/arch/sh/include/asm/cache.h @@ -14,7 +14,7 @@ #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifndef __ASSEMBLY__ struct cache_info { diff --git a/arch/sh/kernel/cpu/sh5/entry.S b/arch/sh/kernel/cpu/sh5/entry.S index e640c63..43d031a 100644 --- a/arch/sh/kernel/cpu/sh5/entry.S +++ b/arch/sh/kernel/cpu/sh5/entry.S @@ -2058,10 +2058,10 @@ asm_uaccess_end: /* - * --- .text.init Section + * --- .kernel.text.init Section */ - .section .text.init, "ax" + .section .kernel.text.init, "ax" /* * void trap_init (void) diff --git a/arch/sh/kernel/head_32.S b/arch/sh/kernel/head_32.S index 788605f..d71c37b 100644 --- a/arch/sh/kernel/head_32.S +++ b/arch/sh/kernel/head_32.S @@ -40,7 +40,7 @@ ENTRY(empty_zero_page) 1: .skip PAGE_SIZE - empty_zero_page - 1b - .section .text.head, "ax" + .section .kernel.text.head, "ax" /* * Condition at the entry of _stext: diff --git a/arch/sh/kernel/head_64.S b/arch/sh/kernel/head_64.S index 7ccfb99..cf6ca73 100644 --- a/arch/sh/kernel/head_64.S +++ b/arch/sh/kernel/head_64.S @@ -110,7 +110,7 @@ empty_bad_pte_table: fpu_in_use: .quad 0 - .section .text.head, "ax" + .section .kernel.text.head, "ax" .balign L1_CACHE_BYTES /* * Condition at the entry of __stext: diff --git a/arch/sh/kernel/init_task.c b/arch/sh/kernel/init_task.c index 80c35ff..024f4c5 100644 --- a/arch/sh/kernel/init_task.c +++ b/arch/sh/kernel/init_task.c @@ -21,7 +21,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/sh/kernel/irq.c b/arch/sh/kernel/irq.c index 64b7690..9df37f6 100644 --- a/arch/sh/kernel/irq.c +++ b/arch/sh/kernel/irq.c @@ -158,10 +158,10 @@ asmlinkage int do_IRQ(unsigned int irq, struct pt_regs *regs) #ifdef CONFIG_IRQSTACKS static char softirq_stack[NR_CPUS * THREAD_SIZE] - __attribute__((__section__(".bss.page_aligned"))); + __attribute__((__section__(".bss.kernel.page_aligned"))); static char hardirq_stack[NR_CPUS * THREAD_SIZE] - __attribute__((__section__(".bss.page_aligned"))); + __attribute__((__section__(".bss.kernel.page_aligned"))); /* * allocate per-cpu stacks for hardirq and for softirq processing diff --git a/arch/sh/kernel/vmlinux_32.lds.S b/arch/sh/kernel/vmlinux_32.lds.S index 7b4b82b..04dd19f 100644 --- a/arch/sh/kernel/vmlinux_32.lds.S +++ b/arch/sh/kernel/vmlinux_32.lds.S @@ -28,7 +28,7 @@ SECTIONS } = 0 .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -58,19 +58,19 @@ SECTIONS . = ALIGN(THREAD_SIZE); .data : { /* Data */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) . = ALIGN(L1_CACHE_BYTES); - *(.data.read_mostly) + *(.kernel.data.read_mostly) . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -128,7 +128,7 @@ SECTIONS .bss : { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) *(COMMON) . = ALIGN(4); diff --git a/arch/sh/kernel/vmlinux_64.lds.S b/arch/sh/kernel/vmlinux_64.lds.S index 33fa464..6059a49 100644 --- a/arch/sh/kernel/vmlinux_64.lds.S +++ b/arch/sh/kernel/vmlinux_64.lds.S @@ -42,7 +42,7 @@ SECTIONS } = 0 .text : C_PHYS(.text) { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT *(.text64) *(.text..SHmedia32) @@ -70,19 +70,19 @@ SECTIONS . = ALIGN(THREAD_SIZE); .data : C_PHYS(.data) { /* Data */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) . = ALIGN(L1_CACHE_BYTES); - *(.data.read_mostly) + *(.kernel.data.read_mostly) . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -140,7 +140,7 @@ SECTIONS .bss : C_PHYS(.bss) { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) *(COMMON) . = ALIGN(4); diff --git a/arch/sparc/boot/btfixupprep.c b/arch/sparc/boot/btfixupprep.c index 52a4208..5f8472b 100644 --- a/arch/sparc/boot/btfixupprep.c +++ b/arch/sparc/boot/btfixupprep.c @@ -171,7 +171,7 @@ main1: } } else if (buffer[nbase+4] != '_') continue; - if (!strcmp (sect, ".text.exit")) + if (!strcmp (sect, ".kernel.text.exit")) continue; if (strcmp (sect, ".text") && strcmp (sect, ".init.text") && @@ -325,7 +325,7 @@ main1: (*rr)->next = NULL; } printf("! Generated by btfixupprep. Do not edit.\n\n"); - printf("\t.section\t\".data.init\",#alloc,#write\n\t.align\t4\n\n"); + printf("\t.section\t\".kernel.data.init\",#alloc,#write\n\t.align\t4\n\n"); printf("\t.global\t___btfixup_start\n___btfixup_start:\n\n"); for (i = 0; i < last; i++) { f = array + i; diff --git a/arch/sparc/include/asm/cache.h b/arch/sparc/include/asm/cache.h index 41f85ae..19f1a48 100644 --- a/arch/sparc/include/asm/cache.h +++ b/arch/sparc/include/asm/cache.h @@ -19,7 +19,7 @@ #define SMP_CACHE_BYTES (1 << SMP_CACHE_BYTES_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifdef CONFIG_SPARC32 #include <asm/asi.h> diff --git a/arch/sparc/kernel/head_32.S b/arch/sparc/kernel/head_32.S index f0b4b51..317f636 100644 --- a/arch/sparc/kernel/head_32.S +++ b/arch/sparc/kernel/head_32.S @@ -72,7 +72,7 @@ sun4e_notsup: .align 4 /* The Sparc trap table, bootloader gives us control at _start. */ - .section .text.head,"ax" + .section .kernel.text.head,"ax" .globl start, _stext, _start, __stext .globl trapbase _start: /* danger danger */ diff --git a/arch/sparc/kernel/head_64.S b/arch/sparc/kernel/head_64.S index 8ffee71..82d26b7 100644 --- a/arch/sparc/kernel/head_64.S +++ b/arch/sparc/kernel/head_64.S @@ -467,7 +467,7 @@ jump_to_sun4u_init: jmpl %g2 + %g0, %g0 nop - .section .text.init.refok + .section .kernel.text.init.refok,"ax",@progbits sun4u_init: BRANCH_IF_SUN4V(g1, sun4v_init) diff --git a/arch/sparc/kernel/vmlinux.lds.S b/arch/sparc/kernel/vmlinux.lds.S index 7626708..14e7761 100644 --- a/arch/sparc/kernel/vmlinux.lds.S +++ b/arch/sparc/kernel/vmlinux.lds.S @@ -41,7 +41,7 @@ SECTIONS .text TEXTSTART : { _text = .; - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -59,20 +59,20 @@ SECTIONS *(.data1) } . = ALIGN(SMP_CACHE_BYTES); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { ; + *(.kernel.data.cacheline_aligned) } . = ALIGN(SMP_CACHE_BYTES); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } /* End of data section */ _edata = .; /* init_task */ . = ALIGN(THREAD_SIZE); - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } .fixup : { __start___fixup = .; diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S index cb02486..9ac4b36 100644 --- a/arch/um/include/asm/common.lds.S +++ b/arch/um/include/asm/common.lds.S @@ -49,9 +49,9 @@ } . = ALIGN(32); - .data.percpu : { + .kernel.data.percpu : { __per_cpu_start = . ; - *(.data.percpu) + *(.kernel.data.percpu) __per_cpu_end = . ; } diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S index 9975e1a..1ee6934 100644 --- a/arch/um/kernel/dyn.lds.S +++ b/arch/um/kernel/dyn.lds.S @@ -97,9 +97,9 @@ SECTIONS .fini_array : { *(.fini_array) } .data : { . = ALIGN(KERNEL_STACK_SIZE); /* init_task */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(KERNEL_STACK_SIZE); - *(.data.init_irqstack) + *(.kernel.data.init_irqstack) DATA_DATA *(.data.* .gnu.linkonce.d.*) SORT(CONSTRUCTORS) diff --git a/arch/um/kernel/init_task.c b/arch/um/kernel/init_task.c index 806d381..f7627d0 100644 --- a/arch/um/kernel/init_task.c +++ b/arch/um/kernel/init_task.c @@ -34,9 +34,9 @@ EXPORT_SYMBOL(init_task); */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; union thread_union cpu0_irqstack - __attribute__((__section__(".data.init_irqstack"))) = + __attribute__((__section__(".kernel.data.init_irqstack"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S index 11b8352..a0b6650 100644 --- a/arch/um/kernel/uml.lds.S +++ b/arch/um/kernel/uml.lds.S @@ -53,9 +53,9 @@ SECTIONS .data : { . = ALIGN(KERNEL_STACK_SIZE); /* init_task */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(KERNEL_STACK_SIZE); - *(.data.init_irqstack) + *(.kernel.data.init_irqstack) DATA_DATA *(.gnu.linkonce.d*) CONSTRUCTORS diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 29c5fbf..2c89079 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -29,7 +29,7 @@ #include <asm/boot.h> #include <asm/asm-offsets.h> -.section ".text.head","ax",@progbits +.section ".kernel.text.head","ax",@progbits .globl startup_32 startup_32: diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 1d5dff4..87bfcbe 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -33,7 +33,7 @@ #include <asm/processor-flags.h> #include <asm/asm-offsets.h> -.section ".text.head" +.section ".kernel.text.head","ax",@progbits .code32 .globl startup_32 diff --git a/arch/x86/boot/compressed/vmlinux.scr b/arch/x86/boot/compressed/vmlinux.scr index f02382a..e4dc231 100644 --- a/arch/x86/boot/compressed/vmlinux.scr +++ b/arch/x86/boot/compressed/vmlinux.scr @@ -1,6 +1,6 @@ SECTIONS { - .rodata.compressed : { + .kernel.rodata.compressed : { input_len = .; LONG(input_data_end - input_data) input_data = .; *(.data) diff --git a/arch/x86/boot/compressed/vmlinux_32.lds b/arch/x86/boot/compressed/vmlinux_32.lds index bb3c483..a24e272 100644 --- a/arch/x86/boot/compressed/vmlinux_32.lds +++ b/arch/x86/boot/compressed/vmlinux_32.lds @@ -7,13 +7,13 @@ SECTIONS * address 0. */ . = 0; - .text.head : { + .kernel.text.head : { _head = . ; - *(.text.head) + *(.kernel.text.head) _ehead = . ; } - .rodata.compressed : { - *(.rodata.compressed) + .kernel.rodata.compressed : { + *(.kernel.rodata.compressed) } .text : { _text = .; /* Text */ @@ -21,6 +21,10 @@ SECTIONS *(.text.*) _etext = . ; } + .got : { + *(.got) + *(.got.*) + } .rodata : { _rodata = . ; *(.rodata) /* read-only data */ @@ -40,4 +44,6 @@ SECTIONS *(COMMON) _end = . ; } + /* Be bold, and discard everything not explicitly mentioned */ + /DISCARD/ : { *(*) } } diff --git a/arch/x86/boot/compressed/vmlinux_64.lds b/arch/x86/boot/compressed/vmlinux_64.lds index bef1ac8..9414e60 100644 --- a/arch/x86/boot/compressed/vmlinux_64.lds +++ b/arch/x86/boot/compressed/vmlinux_64.lds @@ -7,13 +7,13 @@ SECTIONS * address 0. */ . = 0; - .text.head : { + .kernel.text.head : { _head = . ; - *(.text.head) + *(.kernel.text.head) _ehead = . ; } - .rodata.compressed : { - *(.rodata.compressed) + .kernel.rodata.compressed : { + *(.kernel.rodata.compressed) } .text : { _text = .; /* Text */ @@ -45,4 +45,6 @@ SECTIONS . = . + 4096 * 6; _ebss = .; } + /* Be bold, and discard everything not explicitly mentioned */ + /DISCARD/ : { *(*) } } diff --git a/arch/x86/include/asm/cache.h b/arch/x86/include/asm/cache.h index 5d367ca..980a79f 100644 --- a/arch/x86/include/asm/cache.h +++ b/arch/x86/include/asm/cache.h @@ -5,7 +5,7 @@ #define L1_CACHE_SHIFT (CONFIG_X86_L1_CACHE_SHIFT) #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifdef CONFIG_X86_VSMP /* vSMP Internode cacheline shift */ @@ -13,7 +13,7 @@ #ifdef CONFIG_SMP #define __cacheline_aligned_in_smp \ __attribute__((__aligned__(1 << (INTERNODE_CACHE_SHIFT)))) \ - __attribute__((__section__(".data.page_aligned"))) + __attribute__((__section__(".kernel.data.page_aligned"))) #endif #endif diff --git a/arch/x86/kernel/acpi/wakeup_32.S b/arch/x86/kernel/acpi/wakeup_32.S index a12e6a9..1b1beb9 100644 --- a/arch/x86/kernel/acpi/wakeup_32.S +++ b/arch/x86/kernel/acpi/wakeup_32.S @@ -1,4 +1,4 @@ - .section .text.page_aligned + .section .kernel.text.page_aligned,"ax",@progbits #include <linux/linkage.h> #include <asm/segment.h> #include <asm/page.h> diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S index eb7515c..19335ea 100644 --- a/arch/x86/kernel/head_32.S +++ b/arch/x86/kernel/head_32.S @@ -81,7 +81,7 @@ INIT_MAP_BEYOND_END = BOOTBITMAP_SIZE + (PAGE_TABLE_SIZE + ALLOCATOR_SLOP)*PAGE_ * any particular GDT layout, because we load our own as soon as we * can. */ -.section .text.head,"ax",@progbits +.section .kernel.text.head,"ax",@progbits ENTRY(startup_32) /* test KEEP_SEGMENTS flag to see if the bootloader is asking us to not reload segments */ @@ -609,7 +609,7 @@ ENTRY(_stext) /* * BSS section */ -.section ".bss.page_aligned","wa" +.section ".bss.kernel.page_aligned","wa" .align PAGE_SIZE_asm #ifdef CONFIG_X86_PAE swapper_pg_pmd: @@ -626,7 +626,7 @@ ENTRY(empty_zero_page) * This starts the data section. */ #ifdef CONFIG_X86_PAE -.section ".data.page_aligned","wa" +.section ".kernel.data.page_aligned","wa" /* Page-aligned for the benefit of paravirt? */ .align PAGE_SIZE_asm ENTRY(swapper_pg_dir) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 26cfdc1..2a9dc1e 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -40,7 +40,7 @@ L4_START_KERNEL = pgd_index(__START_KERNEL_map) L3_START_KERNEL = pud_index(__START_KERNEL_map) .text - .section .text.head + .section .kernel.text.head,"ax",@progbits .code64 .globl startup_64 startup_64: @@ -414,7 +414,7 @@ ENTRY(phys_base) ENTRY(idt_table) .skip 256 * 16 - .section .bss.page_aligned, "aw", @nobits + .section .bss.kernel.page_aligned, "aw", @nobits .align PAGE_SIZE ENTRY(empty_zero_page) .skip PAGE_SIZE diff --git a/arch/x86/kernel/init_task.c b/arch/x86/kernel/init_task.c index df3bf26..a54e0be 100644 --- a/arch/x86/kernel/init_task.c +++ b/arch/x86/kernel/init_task.c @@ -22,7 +22,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* @@ -36,7 +36,7 @@ EXPORT_SYMBOL(init_task); /* * per-CPU TSS segments. Threads are completely 'soft' on Linux, * no more per-task TSS's. The TSS size is kept cacheline-aligned - * so they are allowed to end up in the .data.cacheline_aligned + * so they are allowed to end up in the .kernel.data.cacheline_aligned * section. Since TSS's are completely CPU-local, we want them * on exact cacheline boundaries, to eliminate cacheline ping-pong. */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 18f056a..1418814 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -84,7 +84,7 @@ char ignore_fpu_irq; * for this. */ gate_desc idt_table[256] - __attribute__((__section__(".data.idt"))) = { { { { 0, 0 } } }, }; + __attribute__((__section__(".kernel.data.idt"))) = { { { { 0, 0 } } }, }; #endif DECLARE_BITMAP(used_vectors, NR_VECTORS); diff --git a/arch/x86/kernel/vmlinux_32.lds.S b/arch/x86/kernel/vmlinux_32.lds.S index 82c6755..8ad2b02 100644 --- a/arch/x86/kernel/vmlinux_32.lds.S +++ b/arch/x86/kernel/vmlinux_32.lds.S @@ -31,15 +31,15 @@ SECTIONS . = LOAD_OFFSET + LOAD_PHYSICAL_ADDR; phys_startup_32 = startup_32 - LOAD_OFFSET; - .text.head : AT(ADDR(.text.head) - LOAD_OFFSET) { + .kernel.text.head : AT(ADDR(.kernel.text.head) - LOAD_OFFSET) { _text = .; /* Text and read-only data */ - *(.text.head) + *(.kernel.text.head) } :text = 0x9090 /* read-only */ .text : AT(ADDR(.text) - LOAD_OFFSET) { . = ALIGN(PAGE_SIZE); /* not really needed, already page aligned */ - *(.text.page_aligned) + *(.kernel.text.page_aligned) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -71,32 +71,32 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; } . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) - *(.data.idt) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) + *(.kernel.data.idt) } . = ALIGN(32); - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } /* rarely changed data like cpu maps */ . = ALIGN(32); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) _edata = .; /* End of data section */ } . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) } /* might get freed after init */ @@ -179,11 +179,11 @@ SECTIONS } #endif . = ALIGN(PAGE_SIZE); - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu.page_aligned) - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu.page_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } . = ALIGN(PAGE_SIZE); @@ -192,7 +192,7 @@ SECTIONS .bss : AT(ADDR(.bss) - LOAD_OFFSET) { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) . = ALIGN(4); __bss_stop = .; diff --git a/arch/x86/kernel/vmlinux_64.lds.S b/arch/x86/kernel/vmlinux_64.lds.S index 1a614c0..f3e6dc3 100644 --- a/arch/x86/kernel/vmlinux_64.lds.S +++ b/arch/x86/kernel/vmlinux_64.lds.S @@ -28,7 +28,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : AT(ADDR(.text) - LOAD_OFFSET) { /* First the code that has to be first for bootstrapping */ - *(.text.head) + *(.kernel.text.head) _stext = .; /* Then the rest */ TEXT_TEXT @@ -63,17 +63,17 @@ SECTIONS . = ALIGN(PAGE_SIZE); . = ALIGN(CONFIG_X86_L1_CACHE_BYTES); - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } . = ALIGN(CONFIG_X86_INTERNODE_CACHE_BYTES); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) } #define VSYSCALL_ADDR (-10*1024*1024) -#define VSYSCALL_PHYS_ADDR ((LOADADDR(.data.read_mostly) + SIZEOF(.data.read_mostly) + 4095) & ~(4095)) -#define VSYSCALL_VIRT_ADDR ((ADDR(.data.read_mostly) + SIZEOF(.data.read_mostly) + 4095) & ~(4095)) +#define VSYSCALL_PHYS_ADDR ((LOADADDR(.kernel.data.read_mostly) + SIZEOF(.kernel.data.read_mostly) + 4095) & ~(4095)) +#define VSYSCALL_VIRT_ADDR ((ADDR(.kernel.data.read_mostly) + SIZEOF(.kernel.data.read_mostly) + 4095) & ~(4095)) #define VLOAD_OFFSET (VSYSCALL_ADDR - VSYSCALL_PHYS_ADDR) #define VLOAD(x) (ADDR(x) - VLOAD_OFFSET) @@ -122,13 +122,13 @@ SECTIONS #undef VVIRT . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) }:data.init . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) } /* might get freed after init */ @@ -215,13 +215,13 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; - .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.data.nosave) } + .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; __bss_start = .; /* BSS */ .bss : AT(ADDR(.bss) - LOAD_OFFSET) { - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) } __bss_stop = .; diff --git a/arch/xtensa/kernel/head.S b/arch/xtensa/kernel/head.S index 67e6913..581c735 100644 --- a/arch/xtensa/kernel/head.S +++ b/arch/xtensa/kernel/head.S @@ -234,7 +234,7 @@ should_never_return: * BSS section */ -.section ".bss.page_aligned", "w" +.section ".bss.kernel.page_aligned", "w" ENTRY(swapper_pg_dir) .fill PAGE_SIZE, 1, 0 ENTRY(empty_zero_page) diff --git a/arch/xtensa/kernel/init_task.c b/arch/xtensa/kernel/init_task.c index e07f5c9..d6836f4 100644 --- a/arch/xtensa/kernel/init_task.c +++ b/arch/xtensa/kernel/init_task.c @@ -28,7 +28,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); EXPORT_SYMBOL(init_mm); union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; struct task_struct init_task = INIT_TASK(init_task); diff --git a/arch/xtensa/kernel/vmlinux.lds.S b/arch/xtensa/kernel/vmlinux.lds.S index d506774..01eafe7 100644 --- a/arch/xtensa/kernel/vmlinux.lds.S +++ b/arch/xtensa/kernel/vmlinux.lds.S @@ -121,14 +121,14 @@ SECTIONS DATA_DATA CONSTRUCTORS . = ALIGN(XCHAL_ICACHE_LINESIZE); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) } _edata = .; /* The initial task */ . = ALIGN(8192); - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* Initialization code and data: */ @@ -259,7 +259,7 @@ SECTIONS /* BSS section */ _bss_start = .; - .bss : { *(.bss.page_aligned) *(.bss) } + .bss : { *(.bss.kernel.page_aligned) *(.bss) } _bss_end = .; _end = .; diff --git a/include/asm-frv/init.h b/include/asm-frv/init.h index 8b15838..16bea47 100644 --- a/include/asm-frv/init.h +++ b/include/asm-frv/init.h @@ -1,12 +1,12 @@ #ifndef _ASM_INIT_H #define _ASM_INIT_H -#define __init __attribute__ ((__section__ (".text.init"))) -#define __initdata __attribute__ ((__section__ (".data.init"))) +#define __init __attribute__ ((__section__ (".kernel.text.init"))) +#define __initdata __attribute__ ((__section__ (".kernel.data.init"))) /* For assembly routines */ -#define __INIT .section ".text.init",#alloc,#execinstr +#define __INIT .section ".kernel.text.init",#alloc,#execinstr #define __FINIT .previous -#define __INITDATA .section ".data.init",#alloc,#write +#define __INITDATA .section ".kernel.data.init",#alloc,#write #endif diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index acb72d8..5e98696 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -64,7 +64,7 @@ /* .data section */ #define DATA_DATA \ *(.data) \ - *(.data.init.refok) \ + *(.kernel.data.init.refok) \ *(.ref.data) \ DEV_KEEP(init.data) \ DEV_KEEP(exit.data) \ @@ -255,8 +255,8 @@ *(.text.hot) \ *(.text) \ *(.ref.text) \ - *(.text.init.refok) \ - *(.exit.text.refok) \ + *(.kernel.text.init.refok) \ + *(.kernel.text.exit.refok) \ DEV_KEEP(init.text) \ DEV_KEEP(exit.text) \ CPU_KEEP(init.text) \ @@ -435,11 +435,11 @@ . = ALIGN(align); \ percpu : { } :percpu \ __per_cpu_load = .; \ - .data.percpu 0 : AT(__per_cpu_load - LOAD_OFFSET) { \ - *(.data.percpu.first) \ - *(.data.percpu.shared_aligned) \ - *(.data.percpu) \ - *(.data.percpu.page_aligned) \ + .kernel.data.percpu 0 : AT(__per_cpu_load - LOAD_OFFSET) { \ + *(.kernel.data.percpu.first) \ + *(.kernel.data.percpu.shared_aligned) \ + *(.kernel.data.percpu) \ + *(.kernel.data.percpu.page_aligned) \ ____per_cpu_size = .; \ } \ . = __per_cpu_load + ____per_cpu_size; \ @@ -448,10 +448,10 @@ #define PERCPU(align) \ . = ALIGN(align); \ VMLINUX_SYMBOL(__per_cpu_start) = .; \ - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { \ - *(.data.percpu.page_aligned) \ - *(.data.percpu) \ - *(.data.percpu.shared_aligned) \ + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { \ + *(.kernel.data.percpu.page_aligned) \ + *(.kernel.data.percpu) \ + *(.kernel.data.percpu.shared_aligned) \ } \ VMLINUX_SYMBOL(__per_cpu_end) = .; #endif diff --git a/include/linux/cache.h b/include/linux/cache.h index 97e2488..12fb84d 100644 --- a/include/linux/cache.h +++ b/include/linux/cache.h @@ -31,7 +31,7 @@ #ifndef __cacheline_aligned #define __cacheline_aligned \ __attribute__((__aligned__(SMP_CACHE_BYTES), \ - __section__(".data.cacheline_aligned"))) + __section__(".kernel.data.cacheline_aligned"))) #endif /* __cacheline_aligned */ #ifndef __cacheline_aligned_in_smp diff --git a/include/linux/init.h b/include/linux/init.h index 68cb026..10d256a 100644 --- a/include/linux/init.h +++ b/include/linux/init.h @@ -62,9 +62,9 @@ /* backward compatibility note * A few places hardcode the old section names: - * .text.init.refok - * .data.init.refok - * .exit.text.refok + * .kernel.text.init.refok + * .kernel.data.init.refok + * .kernel.text.exit.refok * They should be converted to use the defines from this file */ @@ -301,7 +301,7 @@ void __init parse_early_param(void); #endif /* Data marked not to be saved by software suspend */ -#define __nosavedata __section(.data.nosave) +#define __nosavedata __section(.kernel.data.nosave) /* This means "can be init if no module support, otherwise module load may call it." */ diff --git a/include/linux/linkage.h b/include/linux/linkage.h index fee9e59..c010d2b 100644 --- a/include/linux/linkage.h +++ b/include/linux/linkage.h @@ -18,8 +18,8 @@ # define asmregparm #endif -#define __page_aligned_data __section(.data.page_aligned) __aligned(PAGE_SIZE) -#define __page_aligned_bss __section(.bss.page_aligned) __aligned(PAGE_SIZE) +#define __page_aligned_data __section(.kernel.data.page_aligned) __aligned(PAGE_SIZE) +#define __page_aligned_bss __section(.bss.kernel.page_aligned) __aligned(PAGE_SIZE) /* * This is used by architectures to keep arguments on the stack diff --git a/include/linux/percpu.h b/include/linux/percpu.h index e1f8708..200d813 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -10,13 +10,13 @@ #ifdef CONFIG_SMP #define DEFINE_PER_CPU(type, name) \ - __attribute__((__section__(".data.percpu"))) \ + __attribute__((__section__(".kernel.data.percpu"))) \ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #ifdef MODULE -#define SHARED_ALIGNED_SECTION ".data.percpu" +#define SHARED_ALIGNED_SECTION ".kernel.data.percpu" #else -#define SHARED_ALIGNED_SECTION ".data.percpu.shared_aligned" +#define SHARED_ALIGNED_SECTION ".kernel.data.percpu.shared_aligned" #endif #define DEFINE_PER_CPU_SHARED_ALIGNED(type, name) \ @@ -24,13 +24,13 @@ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name \ ____cacheline_aligned_in_smp -#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ - __attribute__((__section__(".data.percpu.page_aligned"))) \ +#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ + __attribute__((__section__(".kernel.data.percpu.page_aligned")))\ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #ifdef CONFIG_HAVE_ZERO_BASED_PER_CPU #define DEFINE_PER_CPU_FIRST(type, name) \ - __attribute__((__section__(".data.percpu.first"))) \ + __attribute__((__section__(".kernel.data.percpu.first"))) \ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #else #define DEFINE_PER_CPU_FIRST(type, name) \ diff --git a/include/linux/spinlock.h b/include/linux/spinlock.h index e0c0fcc..b041f4d 100644 --- a/include/linux/spinlock.h +++ b/include/linux/spinlock.h @@ -60,7 +60,7 @@ /* * Must define these before including other files, inline functions need them */ -#define LOCK_SECTION_NAME ".text.lock."KBUILD_BASENAME +#define LOCK_SECTION_NAME ".kernel.text.lock."KBUILD_BASENAME #define LOCK_SECTION_START(extra) \ ".subsection 1\n\t" \ diff --git a/kernel/module.c b/kernel/module.c index 9712c52..6077e44 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -485,7 +485,7 @@ static unsigned int find_pcpusec(Elf_Ehdr *hdr, Elf_Shdr *sechdrs, const char *secstrings) { - return find_sec(hdr, sechdrs, secstrings, ".data.percpu"); + return find_sec(hdr, sechdrs, secstrings, ".kernel.data.percpu"); } static void percpu_modcopy(void *pcpudest, const void *from, unsigned long size) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 8892161..afe9408 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -794,9 +794,9 @@ static const char *data_sections[] = { DATA_SECTIONS, NULL }; /* sections that may refer to an init/exit section with no warning */ static const char *initref_sections[] = { - ".text.init.refok*", - ".exit.text.refok*", - ".data.init.refok*", + ".kernel.text.init.refok*", + ".kernel.text.exit.refok*", + ".kernel.data.init.refok*", NULL }; @@ -915,7 +915,7 @@ static int section_mismatch(const char *fromsec, const char *tosec) * Pattern 0: * Do not warn if funtion/data are marked with __init_refok/__initdata_refok. * The pattern is identified by: - * fromsec = .text.init.refok* | .data.init.refok* + * fromsec = .kernel.text.init.refok* | .kernel.data.init.refok* * * Pattern 1: * If a module parameter is declared __initdata and permissions=0 @@ -939,8 +939,8 @@ static int section_mismatch(const char *fromsec, const char *tosec) * *probe_one, *_console, *_timer * * Pattern 3: - * Whitelist all refereces from .text.head to .init.data - * Whitelist all refereces from .text.head to .init.text + * Whitelist all refereces from .kernel.text.head to .init.data + * Whitelist all refereces from .kernel.text.head to .init.text * * Pattern 4: * Some symbols belong to init section but still it is ok to reference diff --git a/scripts/recordmcount.pl b/scripts/recordmcount.pl index fe83141..403a4a4 100755 --- a/scripts/recordmcount.pl +++ b/scripts/recordmcount.pl @@ -26,7 +26,7 @@ # which will also be the location of that section after final link. # e.g. # -# .section ".text.sched" +# .section ".sched.text" # .globl my_func # my_func: # [...] @@ -39,7 +39,7 @@ # [...] # # Both relocation offsets for the mcounts in the above example will be -# offset from .text.sched. If we make another file called tmp.s with: +# offset from .sched.text. If we make another file called tmp.s with: # # .section __mcount_loc # .quad my_func + 0x5 @@ -51,7 +51,7 @@ # But this gets hard if my_func is not globl (a static function). # In such a case we have: # -# .section ".text.sched" +# .section ".sched.text" # my_func: # [...] # call mcount (offset: 0x5) -- 1.5.6.5 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2009-01-04 19:20 ` Tim Abbott @ 2009-01-12 21:51 ` Andrew Morton 2009-01-12 22:11 ` Andi Kleen 0 siblings, 1 reply; 43+ messages in thread From: Andrew Morton @ 2009-01-12 21:51 UTC (permalink / raw) To: Tim Abbott Cc: andi, jbarnold, linux-kernel, vda.linux, andersk, wdaher, knikanth On Sun, 4 Jan 2009 14:20:35 -0500 (EST) Tim Abbott <tabbott@MIT.EDU> wrote: > On Thu, 1 Jan 2009, Andrew Morton wrote: > > > OK, it hits a lot of files in minor ways. > > > > You could ask Linus to merge it late in the -rc1 merge window (please > > cc me on the email). Or you could just send it to me (with the usual > > cc's) and I'll toss it into my 2.6.29-rc1 pile. I doubt if it'll cause > > much trouble. But please do check its applicability to linux-next - people > > have been moving arch files into different directories lately. > > Andrew, > > I've attached for you a version of the patch the applies to current > linux-next (next-20090102). > > It does not apply to Linus' current master because d3794979a in linux-next > adds use of the ".data.percpu.first" section name, which this patch > updates to ".kernel.data.percpu.first". I think it'll be easiest to just > merge this after d3794979a. > > If a version of d3794979a is not going to be merged first, there will be > two merge conflicts that are easy to resolve as they are changes to code > added by d3794979a, but we'll then need to fix d3794979a before merging > d3794979a. > I don't think anyone will want to carry this in a non-mainline tree for any amount of time. it'd be best to squeeze it into mainline pretty much immediately. But... > This patch was originally written by Denys Vlasenko, but it has been > substantially modified by Anders Kaseorg and Tim Abbott to update it > to apply to the current kernel and fix bugs. Any bugs were added by > Anders Kaseorg and Tim Abbott. > > The purpose of this patch is to make it possible to build the kernel > with "gcc -ffunction-sections -fdata-sections". > > The problem is that with -ffunction-sections -fdata-sections, gcc > creates sections like .text.head and .data.nosave whenever someone has > innocuous code like this: > > static void head(...) {...} > > or this: > > static int nosave = 1; > > somewhere in the kernel. > > The kernel linker scripts are confused by such names, and thus put > these sections in the wrong places. > > This patch renames all "magic" section names used by the kernel to not > have this format, eliminating the possibility of such collisions. > > Ksplice's 'run-pre matching' process is much simpler if the original > kernel was compiled with -ffunction-sections and -fdata-sections. > The changelog gives us no reason to do that. So we `make it possible to build the kernel > with "gcc -ffunction-sections -fdata-sections"'. So what? Where is the value in that? ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2009-01-12 21:51 ` Andrew Morton @ 2009-01-12 22:11 ` Andi Kleen 0 siblings, 0 replies; 43+ messages in thread From: Andi Kleen @ 2009-01-12 22:11 UTC (permalink / raw) To: Andrew Morton Cc: Tim Abbott, andi, jbarnold, linux-kernel, vda.linux, andersk, wdaher, knikanth > So we `make it possible to build the kernel > with "gcc > -ffunction-sections -fdata-sections"'. So what? Where is the value in > that? The original reason was that it allows the linker to drop unused functions automatically. That is a good thing by itself -- e.g. if it was universal we could avoid a lot of #ifdefs. And it also turned out that standard kernel builds get smaller by it iirc. -Andi -- ak@linux.intel.com -- Speaking for myself only. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2008-12-06 0:03 ` [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections Jeff Arnold 2008-12-06 0:03 ` [PATCH 2/7] x86: Add an option to compile " Jeff Arnold 2008-12-06 8:46 ` [PATCH 1/7] Make section names compatible " Andi Kleen @ 2009-02-04 8:15 ` Rusty Russell 2009-02-05 1:11 ` Anders Kaseorg 2 siblings, 1 reply; 43+ messages in thread From: Rusty Russell @ 2009-02-04 8:15 UTC (permalink / raw) To: Jeff Arnold Cc: Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Linus Torvalds On Saturday 06 December 2008 10:33:54 Jeff Arnold wrote: > The purpose of this patch is to make it possible to build the kernel > with "gcc -ffunction-sections -fdata-sections". > > The problem is that with -ffunction-sections -fdata-sections, gcc > creates sections like .text.head and .data.nosave whenever someone has > innocuous code like this: ... > 131 files changed, 356 insertions(+), 346 deletions(-) OK, I updated this patch for latest Linus tree, and there are a few places you missed, but the important q. is whether Linus will take this patch now? There's no good time for this kind of invasive change; but it's not going to get better. And it's the only real roadblock for ksplice, plus it shrinks my vmlinux by 12k when turned on. Linus? PS. Here's the extra ones I found. Since you only enable sections on x86, it's not critical, but nice to cover them all. diff --git a/arch/cris/kernel/vmlinux.lds.S b/arch/cris/kernel/vmlinux.lds.S --- a/arch/cris/kernel/vmlinux.lds.S +++ b/arch/cris/kernel/vmlinux.lds.S @@ -68,7 +68,7 @@ SECTIONS _edata = . ; . = ALIGN(PAGE_SIZE); /* init_task and stack, must be aligned. */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } . = ALIGN(PAGE_SIZE); /* Init code and data. */ __init_begin = .; diff --git a/arch/m32r/kernel/head.S b/arch/m32r/kernel/head.S --- a/arch/m32r/kernel/head.S +++ b/arch/m32r/kernel/head.S @@ -23,7 +23,7 @@ __INITDATA /* * References to members of the boot_cpu_data structure. */ -.section .text.head, "ax" +.section .kernel.text.head, "ax" .global start_kernel .global __bss_start .global _end diff --git a/arch/m32r/kernel/vmlinux.lds.S b/arch/m32r/kernel/vmlinux.lds.S --- a/arch/m32r/kernel/vmlinux.lds.S +++ b/arch/m32r/kernel/vmlinux.lds.S @@ -27,7 +27,7 @@ SECTIONS _text = .; /* Text and read-only data */ .boot : { *(.boot) } = 0 .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT diff --git a/arch/s390/kernel/vdso.c b/arch/s390/kernel/vdso.c --- a/arch/s390/kernel/vdso.c +++ b/arch/s390/kernel/vdso.c @@ -64,7 +64,7 @@ static union { static union { struct vdso_data data; u8 page[PAGE_SIZE]; -} vdso_data_store __attribute__((__section__(".data.page_aligned"))); +} vdso_data_store __attribute__((__section__(".kernel.data.page_aligned"))); struct vdso_data *vdso_data = &vdso_data_store.data; /* diff --git a/arch/s390/kernel/vdso32/vdso32_wrapper.S b/arch/s390/kernel/vdso32/vdso32_wrapper.S --- a/arch/s390/kernel/vdso32/vdso32_wrapper.S +++ b/arch/s390/kernel/vdso32/vdso32_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned" .globl vdso32_start, vdso32_end .balign PAGE_SIZE diff --git a/arch/sparc/kernel/head_32.S b/arch/sparc/kernel/head_32.S --- a/arch/sparc/kernel/head_32.S +++ b/arch/sparc/kernel/head_32.S @@ -72,7 +72,7 @@ sun4e_notsup: .align 4 /* The Sparc trap table, bootloader gives us control at _start. */ - .section .text.head,"ax" + .section .kernel.text.head,"ax" .globl start, _stext, _start, __stext .globl trapbase _start: /* danger danger */ diff --git a/arch/sparc/kernel/init_task.c b/arch/sparc/kernel/init_task.c --- a/arch/sparc/kernel/init_task.c +++ b/arch/sparc/kernel/init_task.c @@ -22,5 +22,5 @@ EXPORT_SYMBOL(init_task); * in etrap.S which assumes it. */ union thread_union init_thread_union - __attribute__((section (".data.init_task"))) + __attribute__((section (".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/sparc/kernel/vmlinux.lds.S b/arch/sparc/kernel/vmlinux.lds.S --- a/arch/sparc/kernel/vmlinux.lds.S +++ b/arch/sparc/kernel/vmlinux.lds.S @@ -41,7 +41,7 @@ SECTIONS .text TEXTSTART : { _text = .; - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2009-02-04 8:15 ` Rusty Russell @ 2009-02-05 1:11 ` Anders Kaseorg 2009-02-05 2:00 ` Anders Kaseorg 0 siblings, 1 reply; 43+ messages in thread From: Anders Kaseorg @ 2009-02-05 1:11 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan, Linus Torvalds On Wed, 4 Feb 2009, Rusty Russell wrote: > PS. Here's the extra ones I found. Since you only enable sections on x86, > it's not critical, but nice to cover them all. It looks like you didn’t base your work on the current copy of the patch posted to this thread [1]. That copy already includes all but two of your changes (arch/s390/kernel/vdso.c and arch/sparc/kernel/init_task.c, in which the lines in question were introduced by recent commits), along with some others. [1] <http://lkml.org/lkml/2009/1/4/196> > --- a/arch/s390/kernel/vdso32/vdso32_wrapper.S > +++ b/arch/s390/kernel/vdso32/vdso32_wrapper.S > @@ -1,7 +1,7 @@ > #include <linux/init.h> > #include <asm/page.h> > > - .section ".data.page_aligned" > + .section ".kernel.data.page_aligned" > > .globl vdso32_start, vdso32_end > .balign PAGE_SIZE This change is wrong; you need to specify .section ".kernel.data.page_aligned", "aw" like we do in [1], or else the assembler will not set the ALLOC flag on that section, leading to a crash at runtime. We will send a new copy of the patch that includes your two changes shortly. Anders ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections 2009-02-05 1:11 ` Anders Kaseorg @ 2009-02-05 2:00 ` Anders Kaseorg 0 siblings, 0 replies; 43+ messages in thread From: Anders Kaseorg @ 2009-02-05 2:00 UTC (permalink / raw) To: Rusty Russell Cc: Jeff Arnold, Andrew Morton, linux-kernel, Denys Vlasenko, Tim Abbott, Waseem Daher, Nikanth Karthikesan, Linus Torvalds From: Denys Vlasenko <vda.linux@googlemail.com> This patch was originally written by Denys Vlasenko, but it has been substantially modified by Anders Kaseorg and Tim Abbott to update it to apply to the current kernel and fix bugs. Any bugs were added by Anders Kaseorg and Tim Abbott. The purpose of this patch is to make it possible to build the kernel with "gcc -ffunction-sections -fdata-sections". The problem is that with -ffunction-sections -fdata-sections, gcc creates sections like .text.head and .data.nosave whenever someone has innocuous code like this: static void head(...) {...} or this: static int nosave = 1; somewhere in the kernel. The kernel linker scripts are confused by such names, and thus put these sections in the wrong places. This patch renames all "magic" section names used by the kernel to not have this format, eliminating the possibility of such collisions. Ksplice's 'run-pre matching' process is much simpler if the original kernel was compiled with -ffunction-sections and -fdata-sections. Changelog since Denys Vlasenko's latest patch: * Renamed .kernel.exit.text.refok to .kernel.text.exit.refok, for better analogy with .kernel.*.init.refok. * Updated from 2.6.27-rc4-git3 to 2.6.29-rc3. * Added some missing section flags for some of the .kernel.* sections (gcc assumes "ax" for sections with names starting with ".text", but does not for sections with names starting with ".kernel.text"). Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com> [andersk@mit.edu] Add missing section flags for .kernel.* sections. Signed-off-by: Anders Kaseorg <andersk@mit.edu> [tabbott@mit.edu] Update from 2.6.27-rc4-git3 to 2.6.28-rc5. [tabbott@mit.edu] Rename .kernel.exit.text.refok. Signed-off-by: Tim Abbott <tabbott@mit.edu> Tested-by: Waseem Daher <wdaher@mit.edu> --- Documentation/mutex-design.txt | 4 +- arch/alpha/kernel/head.S | 2 +- arch/alpha/kernel/init_task.c | 2 +- arch/alpha/kernel/vmlinux.lds.S | 14 ++++---- arch/arm/kernel/head-nommu.S | 3 +- arch/arm/kernel/head.S | 3 +- arch/arm/kernel/init_task.c | 2 +- arch/arm/kernel/vmlinux.lds.S | 14 ++++---- arch/arm/mm/proc-v6.S | 2 +- arch/arm/mm/proc-v7.S | 2 +- arch/arm/mm/tlb-v6.S | 2 +- arch/arm/mm/tlb-v7.S | 2 +- arch/avr32/kernel/init_task.c | 2 +- arch/avr32/kernel/vmlinux.lds.S | 6 ++-- arch/avr32/mm/init.c | 2 +- arch/blackfin/kernel/vmlinux.lds.S | 2 +- arch/cris/kernel/process.c | 2 +- arch/cris/kernel/vmlinux.lds.S | 2 +- arch/frv/kernel/break.S | 4 +- arch/frv/kernel/entry.S | 2 +- arch/frv/kernel/head-mmu-fr451.S | 2 +- arch/frv/kernel/head-uc-fr401.S | 2 +- arch/frv/kernel/head-uc-fr451.S | 2 +- arch/frv/kernel/head-uc-fr555.S | 2 +- arch/frv/kernel/head.S | 4 +- arch/frv/kernel/init_task.c | 2 +- arch/frv/kernel/vmlinux.lds.S | 18 +++++----- arch/frv/mm/tlb-miss.S | 2 +- arch/h8300/boot/compressed/head.S | 2 +- arch/h8300/boot/compressed/vmlinux.lds | 2 +- arch/h8300/kernel/init_task.c | 2 +- arch/h8300/kernel/vmlinux.lds.S | 2 +- arch/ia64/include/asm/asmmacro.h | 12 +++--- arch/ia64/include/asm/cache.h | 2 +- arch/ia64/include/asm/percpu.h | 2 +- arch/ia64/kernel/Makefile | 2 +- arch/ia64/kernel/gate-data.S | 2 +- arch/ia64/kernel/gate.S | 8 ++-- arch/ia64/kernel/gate.lds.S | 10 +++--- arch/ia64/kernel/head.S | 2 +- arch/ia64/kernel/init_task.c | 4 +- arch/ia64/kernel/ivt.S | 2 +- arch/ia64/kernel/minstate.h | 4 +- arch/ia64/kernel/paravirtentry.S | 12 +++--- arch/ia64/kernel/vmlinux.lds.S | 48 +++++++++++++------------- arch/ia64/kvm/vmm_ivt.S | 2 +- arch/ia64/xen/xensetup.S | 2 +- arch/m32r/kernel/head.S | 2 +- arch/m32r/kernel/init_task.c | 2 +- arch/m32r/kernel/vmlinux.lds.S | 8 ++-- arch/m68k/kernel/head.S | 2 +- arch/m68k/kernel/process.c | 2 +- arch/m68k/kernel/sun3-head.S | 2 +- arch/m68k/kernel/vmlinux-std.lds | 6 ++-- arch/m68k/kernel/vmlinux-sun3.lds | 4 +- arch/m68knommu/kernel/init_task.c | 2 +- arch/m68knommu/kernel/vmlinux.lds.S | 6 ++-- arch/m68knommu/platform/68360/head-ram.S | 2 +- arch/m68knommu/platform/68360/head-rom.S | 2 +- arch/mips/kernel/init_task.c | 2 +- arch/mips/kernel/vmlinux.lds.S | 8 ++-- arch/mips/lasat/image/head.S | 2 +- arch/mips/lasat/image/romscript.normal | 2 +- arch/mn10300/kernel/head.S | 2 +- arch/mn10300/kernel/init_task.c | 2 +- arch/mn10300/kernel/vmlinux.lds.S | 16 ++++---- arch/parisc/include/asm/cache.h | 2 +- arch/parisc/include/asm/system.h | 2 +- arch/parisc/kernel/head.S | 2 +- arch/parisc/kernel/init_task.c | 8 ++-- arch/parisc/kernel/vmlinux.lds.S | 26 +++++++------- arch/powerpc/include/asm/cache.h | 2 +- arch/powerpc/include/asm/page_64.h | 2 +- arch/powerpc/include/asm/ppc_asm.h | 4 +- arch/powerpc/kernel/head_32.S | 2 +- arch/powerpc/kernel/head_40x.S | 2 +- arch/powerpc/kernel/head_44x.S | 2 +- arch/powerpc/kernel/head_8xx.S | 2 +- arch/powerpc/kernel/head_fsl_booke.S | 2 +- arch/powerpc/kernel/init_task.c | 2 +- arch/powerpc/kernel/machine_kexec_64.c | 2 +- arch/powerpc/kernel/vdso.c | 2 +- arch/powerpc/kernel/vdso32/vdso32_wrapper.S | 2 +- arch/powerpc/kernel/vdso64/vdso64_wrapper.S | 2 +- arch/powerpc/kernel/vmlinux.lds.S | 28 ++++++++-------- arch/s390/include/asm/cache.h | 2 +- arch/s390/kernel/head.S | 2 +- arch/s390/kernel/init_task.c | 2 +- arch/s390/kernel/vdso.c | 2 +- arch/s390/kernel/vdso32/vdso32_wrapper.S | 2 +- arch/s390/kernel/vdso64/vdso64_wrapper.S | 2 +- arch/s390/kernel/vmlinux.lds.S | 20 +++++----- arch/sh/include/asm/cache.h | 2 +- arch/sh/kernel/cpu/sh5/entry.S | 4 +- arch/sh/kernel/head_32.S | 2 +- arch/sh/kernel/head_64.S | 2 +- arch/sh/kernel/init_task.c | 2 +- arch/sh/kernel/irq.c | 4 +- arch/sh/kernel/vmlinux_32.lds.S | 14 ++++---- arch/sh/kernel/vmlinux_64.lds.S | 14 ++++---- arch/sparc/boot/btfixupprep.c | 4 +- arch/sparc/include/asm/cache.h | 2 +- arch/sparc/kernel/head_32.S | 2 +- arch/sparc/kernel/head_64.S | 2 +- arch/sparc/kernel/init_task.c | 2 +- arch/sparc/kernel/vmlinux.lds.S | 14 ++++---- arch/um/include/asm/common.lds.S | 4 +- arch/um/kernel/dyn.lds.S | 4 +- arch/um/kernel/init_task.c | 4 +- arch/um/kernel/uml.lds.S | 4 +- arch/x86/boot/compressed/head_32.S | 2 +- arch/x86/boot/compressed/head_64.S | 2 +- arch/x86/boot/compressed/vmlinux.scr | 2 +- arch/x86/boot/compressed/vmlinux_32.lds | 14 +++++-- arch/x86/boot/compressed/vmlinux_64.lds | 10 +++-- arch/x86/include/asm/cache.h | 4 +- arch/x86/kernel/acpi/wakeup_32.S | 2 +- arch/x86/kernel/head_32.S | 6 ++-- arch/x86/kernel/head_64.S | 4 +- arch/x86/kernel/init_task.c | 4 +- arch/x86/kernel/traps.c | 2 +- arch/x86/kernel/vmlinux_32.lds.S | 36 ++++++++++---------- arch/x86/kernel/vmlinux_64.lds.S | 26 +++++++------- arch/xtensa/kernel/head.S | 2 +- arch/xtensa/kernel/init_task.c | 2 +- arch/xtensa/kernel/vmlinux.lds.S | 6 ++-- include/asm-frv/init.h | 8 ++-- include/asm-generic/vmlinux.lds.h | 14 ++++---- include/linux/cache.h | 2 +- include/linux/init.h | 8 ++-- include/linux/linkage.h | 4 +- include/linux/percpu.h | 10 +++--- include/linux/spinlock.h | 2 +- kernel/module.c | 2 +- scripts/mod/modpost.c | 12 +++--- scripts/recordmcount.pl | 6 ++-- 136 files changed, 360 insertions(+), 350 deletions(-) diff --git a/Documentation/mutex-design.txt b/Documentation/mutex-design.txt index aa60d1f..b5f800b 100644 --- a/Documentation/mutex-design.txt +++ b/Documentation/mutex-design.txt @@ -66,14 +66,14 @@ of advantages of mutexes: c0377ccb <mutex_lock>: c0377ccb: f0 ff 08 lock decl (%eax) - c0377cce: 78 0e js c0377cde <.text.lock.mutex> + c0377cce: 78 0e js c0377cde <.kernel.text.lock.mutex> c0377cd0: c3 ret the unlocking fastpath is equally tight: c0377cd1 <mutex_unlock>: c0377cd1: f0 ff 00 lock incl (%eax) - c0377cd4: 7e 0f jle c0377ce5 <.text.lock.mutex+0x7> + c0377cd4: 7e 0f jle c0377ce5 <.kernel.text.lock.mutex+0x7> c0377cd6: c3 ret - 'struct mutex' semantics are well-defined and are enforced if diff --git a/arch/alpha/kernel/head.S b/arch/alpha/kernel/head.S index 7ac1f13..3726d71 100644 --- a/arch/alpha/kernel/head.S +++ b/arch/alpha/kernel/head.S @@ -10,7 +10,7 @@ #include <asm/system.h> #include <asm/asm-offsets.h> -.section .text.head, "ax" +.section .kernel.text.head, "ax" .globl swapper_pg_dir .globl _stext swapper_pg_dir=SWAPPER_PGD diff --git a/arch/alpha/kernel/init_task.c b/arch/alpha/kernel/init_task.c index c2938e5..826c26a 100644 --- a/arch/alpha/kernel/init_task.c +++ b/arch/alpha/kernel/init_task.c @@ -17,5 +17,5 @@ EXPORT_SYMBOL(init_mm); EXPORT_SYMBOL(init_task); union thread_union init_thread_union - __attribute__((section(".data.init_thread"))) + __attribute__((section(".kernel.data.init_thread"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/alpha/kernel/vmlinux.lds.S b/arch/alpha/kernel/vmlinux.lds.S index ef37fc1..e3c637e 100644 --- a/arch/alpha/kernel/vmlinux.lds.S +++ b/arch/alpha/kernel/vmlinux.lds.S @@ -16,7 +16,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -93,18 +93,18 @@ SECTIONS /* Freed after init ends here */ /* Note 2 page alignment above. */ - .data.init_thread : { - *(.data.init_thread) + .kernel.data.init_thread : { + *(.kernel.data.init_thread) } . = ALIGN(PAGE_SIZE); - .data.page_aligned : { - *(.data.page_aligned) + .kernel.data.page_aligned : { + *(.kernel.data.page_aligned) } . = ALIGN(64); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } _data = .; diff --git a/arch/arm/kernel/head-nommu.S b/arch/arm/kernel/head-nommu.S index cc87e17..8163b2e 100644 --- a/arch/arm/kernel/head-nommu.S +++ b/arch/arm/kernel/head-nommu.S @@ -32,7 +32,8 @@ * numbers for r1. * */ - .section ".text.head", "ax" + .section ".kernel.text.head", "ax" + .type stext, %function ENTRY(stext) msr cpsr_c, #PSR_F_BIT | PSR_I_BIT | SVC_MODE @ ensure svc mode @ and irqs disabled diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S index 21e17dc..42f2d17 100644 --- a/arch/arm/kernel/head.S +++ b/arch/arm/kernel/head.S @@ -74,7 +74,8 @@ * crap here - that's what the boot loader (or in extreme, well justified * circumstances, zImage) is for. */ - .section ".text.head", "ax" + .section ".kernel.text.head", "ax" + .type stext, %function ENTRY(stext) msr cpsr_c, #PSR_F_BIT | PSR_I_BIT | SVC_MODE @ ensure svc mode @ and irqs disabled diff --git a/arch/arm/kernel/init_task.c b/arch/arm/kernel/init_task.c index e859af3..af0be90 100644 --- a/arch/arm/kernel/init_task.c +++ b/arch/arm/kernel/init_task.c @@ -29,7 +29,7 @@ EXPORT_SYMBOL(init_mm); * The things we do for performance.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S index 0021607..5f73ff8 100644 --- a/arch/arm/kernel/vmlinux.lds.S +++ b/arch/arm/kernel/vmlinux.lds.S @@ -23,10 +23,10 @@ SECTIONS #else . = PAGE_OFFSET + TEXT_OFFSET; #endif - .text.head : { + .kernel.text.head : { _stext = .; _sinittext = .; - *(.text.head) + *(.kernel.text.head) } .init : { /* Init code and data */ @@ -65,8 +65,8 @@ SECTIONS #endif . = ALIGN(4096); __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; #ifndef CONFIG_XIP_KERNEL __init_begin = _stext; @@ -125,7 +125,7 @@ SECTIONS * first, the init task union, aligned * to an 8192 byte boundary. */ - *(.data.init_task) + *(.kernel.data.init_task) #ifdef CONFIG_XIP_KERNEL . = ALIGN(4096); @@ -137,7 +137,7 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(4096); __nosave_end = .; @@ -145,7 +145,7 @@ SECTIONS * then the cacheline aligned data */ . = ALIGN(32); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) /* * The exception fixup table (might need resorting at runtime) diff --git a/arch/arm/mm/proc-v6.S b/arch/arm/mm/proc-v6.S index f0cc599..c2eb6a5 100644 --- a/arch/arm/mm/proc-v6.S +++ b/arch/arm/mm/proc-v6.S @@ -132,7 +132,7 @@ cpu_v6_name: .asciz "ARMv6-compatible processor" .align - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr /* * __v6_setup diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index d1ebec4..19b351b 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -153,7 +153,7 @@ cpu_v7_name: .ascii "ARMv7 Processor" .align - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr /* * __v7_setup diff --git a/arch/arm/mm/tlb-v6.S b/arch/arm/mm/tlb-v6.S index 20f84bb..f033268 100644 --- a/arch/arm/mm/tlb-v6.S +++ b/arch/arm/mm/tlb-v6.S @@ -87,7 +87,7 @@ ENTRY(v6wbi_flush_kern_tlb_range) mcr p15, 0, r2, c7, c5, 4 @ prefetch flush mov pc, lr - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr .type v6wbi_tlb_fns, #object ENTRY(v6wbi_tlb_fns) diff --git a/arch/arm/mm/tlb-v7.S b/arch/arm/mm/tlb-v7.S index 24ba510..1b6d40a 100644 --- a/arch/arm/mm/tlb-v7.S +++ b/arch/arm/mm/tlb-v7.S @@ -80,7 +80,7 @@ ENTRY(v7wbi_flush_kern_tlb_range) mov pc, lr ENDPROC(v7wbi_flush_kern_tlb_range) - .section ".text.init", #alloc, #execinstr + .section ".kernel.text.init", #alloc, #execinstr .type v7wbi_tlb_fns, #object ENTRY(v7wbi_tlb_fns) diff --git a/arch/avr32/kernel/init_task.c b/arch/avr32/kernel/init_task.c index 993d56e..4002a5d 100644 --- a/arch/avr32/kernel/init_task.c +++ b/arch/avr32/kernel/init_task.c @@ -23,7 +23,7 @@ EXPORT_SYMBOL(init_mm); * Initial thread structure. Must be aligned on an 8192-byte boundary. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/avr32/kernel/vmlinux.lds.S b/arch/avr32/kernel/vmlinux.lds.S index 7910d41..1cb0345 100644 --- a/arch/avr32/kernel/vmlinux.lds.S +++ b/arch/avr32/kernel/vmlinux.lds.S @@ -95,15 +95,15 @@ SECTIONS /* * First, the init task union, aligned to an 8K boundary. */ - *(.data.init_task) + *(.kernel.data.init_task) /* Then, the page-aligned data */ . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) /* Then, the cacheline aligned data */ . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) /* And the rest... */ *(.data.rel*) diff --git a/arch/avr32/mm/init.c b/arch/avr32/mm/init.c index e819fa6..5235875 100644 --- a/arch/avr32/mm/init.c +++ b/arch/avr32/mm/init.c @@ -24,7 +24,7 @@ #include <asm/setup.h> #include <asm/sections.h> -#define __page_aligned __attribute__((section(".data.page_aligned"))) +#define __page_aligned __attribute__((section(".kernel.data.page_aligned"))) DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); diff --git a/arch/blackfin/kernel/vmlinux.lds.S b/arch/blackfin/kernel/vmlinux.lds.S index 4b4341d..d1e521c 100644 --- a/arch/blackfin/kernel/vmlinux.lds.S +++ b/arch/blackfin/kernel/vmlinux.lds.S @@ -94,7 +94,7 @@ SECTIONS __sdata = .; /* This gets done first, so the glob doesn't suck it in */ . = ALIGN(32); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) #if !L1_DATA_A_LENGTH . = ALIGN(32); diff --git a/arch/cris/kernel/process.c b/arch/cris/kernel/process.c index 60816e8..43c26bd 100644 --- a/arch/cris/kernel/process.c +++ b/arch/cris/kernel/process.c @@ -51,7 +51,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/cris/kernel/vmlinux.lds.S b/arch/cris/kernel/vmlinux.lds.S index 0d2adfc..526f853 100644 --- a/arch/cris/kernel/vmlinux.lds.S +++ b/arch/cris/kernel/vmlinux.lds.S @@ -68,7 +68,7 @@ SECTIONS _edata = . ; . = ALIGN(PAGE_SIZE); /* init_task and stack, must be aligned. */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } . = ALIGN(PAGE_SIZE); /* Init code and data. */ __init_begin = .; diff --git a/arch/frv/kernel/break.S b/arch/frv/kernel/break.S index bd0bdf9..1b7148a 100644 --- a/arch/frv/kernel/break.S +++ b/arch/frv/kernel/break.S @@ -21,7 +21,7 @@ # # the break handler has its own stack # - .section .bss.stack + .section .bss.kernel.stack .globl __break_user_context .balign THREAD_SIZE __break_stack: @@ -63,7 +63,7 @@ __break_trace_through_exceptions: # entry point for Break Exceptions/Interrupts # ############################################################################### - .section .text.break + .section .kernel.text.break,"ax",@progbits .balign 4 .globl __entry_break __entry_break: diff --git a/arch/frv/kernel/entry.S b/arch/frv/kernel/entry.S index 99060ab..8d54945 100644 --- a/arch/frv/kernel/entry.S +++ b/arch/frv/kernel/entry.S @@ -38,7 +38,7 @@ #define nr_syscalls ((syscall_table_size)/4) - .section .text.entry + .section .kernel.text.entry,"ax",@progbits .balign 4 .macro LEDS val diff --git a/arch/frv/kernel/head-mmu-fr451.S b/arch/frv/kernel/head-mmu-fr451.S index c8f210d..248e557 100644 --- a/arch/frv/kernel/head-mmu-fr451.S +++ b/arch/frv/kernel/head-mmu-fr451.S @@ -31,7 +31,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr401.S b/arch/frv/kernel/head-uc-fr401.S index ee282be..282ab8d 100644 --- a/arch/frv/kernel/head-uc-fr401.S +++ b/arch/frv/kernel/head-uc-fr401.S @@ -30,7 +30,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr451.S b/arch/frv/kernel/head-uc-fr451.S index b10d9c8..2fcb045 100644 --- a/arch/frv/kernel/head-uc-fr451.S +++ b/arch/frv/kernel/head-uc-fr451.S @@ -30,7 +30,7 @@ #define __400_LCR 0xfe000100 #define __400_LSBR 0xfe000c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head-uc-fr555.S b/arch/frv/kernel/head-uc-fr555.S index 39937c1..b5758bf 100644 --- a/arch/frv/kernel/head-uc-fr555.S +++ b/arch/frv/kernel/head-uc-fr555.S @@ -29,7 +29,7 @@ #define __551_LCR 0xfeff1100 #define __551_LSBR 0xfeff1c00 - .section .text.init,"ax" + .section .kernel.text.init,"ax" .balign 4 ############################################################################### diff --git a/arch/frv/kernel/head.S b/arch/frv/kernel/head.S index fecf751..8be8976 100644 --- a/arch/frv/kernel/head.S +++ b/arch/frv/kernel/head.S @@ -27,7 +27,7 @@ # command line string # ############################################################################### - .section .text.head,"ax" + .section .kernel.text.head,"ax" .balign 4 .globl _boot, __head_reference @@ -541,7 +541,7 @@ __head_end: .size _boot, .-_boot # provide a point for GDB to place a break - .section .text.start,"ax" + .section .kernel.text.start,"ax" .globl _start .balign 4 _start: diff --git a/arch/frv/kernel/init_task.c b/arch/frv/kernel/init_task.c index 29429a8..7ac16e1 100644 --- a/arch/frv/kernel/init_task.c +++ b/arch/frv/kernel/init_task.c @@ -24,7 +24,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/frv/kernel/vmlinux.lds.S b/arch/frv/kernel/vmlinux.lds.S index b95c4ea..219d286 100644 --- a/arch/frv/kernel/vmlinux.lds.S +++ b/arch/frv/kernel/vmlinux.lds.S @@ -26,7 +26,7 @@ SECTIONS _sinittext = .; .init.text : { - *(.text.head) + *(.kernel.text.head) #ifndef CONFIG_DEBUG_INFO INIT_TEXT EXIT_TEXT @@ -71,13 +71,13 @@ SECTIONS /* put sections together that have massive alignment issues */ . = ALIGN(THREAD_SIZE); - .data.init_task : { + .kernel.data.init_task : { /* init task record & stack */ - *(.data.init_task) + *(.kernel.data.init_task) } . = ALIGN(L1_CACHE_BYTES); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } .trap : { /* trap table management - read entry-table.S before modifying */ @@ -94,10 +94,10 @@ SECTIONS _text = .; _stext = .; .text : { - *(.text.start) - *(.text.entry) - *(.text.break) - *(.text.tlbmiss) + *(.kernel.text.start) + *(.kernel.text.entry) + *(.kernel.text.break) + *(.kernel.text.tlbmiss) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -152,7 +152,7 @@ SECTIONS .sbss : { *(.sbss .sbss.*) } .bss : { *(.bss .bss.*) } - .bss.stack : { *(.bss) } + .bss.kernel.stack : { *(.bss) } __bss_stop = .; _end = . ; diff --git a/arch/frv/mm/tlb-miss.S b/arch/frv/mm/tlb-miss.S index 0764348..e361931 100644 --- a/arch/frv/mm/tlb-miss.S +++ b/arch/frv/mm/tlb-miss.S @@ -16,7 +16,7 @@ #include <asm/highmem.h> #include <asm/spr-regs.h> - .section .text.tlbmiss + .section .kernel.text.tlbmiss,"ax",@progbits .balign 4 .globl __entry_insn_mmu_miss diff --git a/arch/h8300/boot/compressed/head.S b/arch/h8300/boot/compressed/head.S index 985a81a..c4964f1 100644 --- a/arch/h8300/boot/compressed/head.S +++ b/arch/h8300/boot/compressed/head.S @@ -9,7 +9,7 @@ #define SRAM_START 0xff4000 - .section .text.startup + .section .kernel.text.startup,"ax",@progbits .global startup startup: mov.l #SRAM_START+0x8000, sp diff --git a/arch/h8300/boot/compressed/vmlinux.lds b/arch/h8300/boot/compressed/vmlinux.lds index 65e2a0d..5965521 100644 --- a/arch/h8300/boot/compressed/vmlinux.lds +++ b/arch/h8300/boot/compressed/vmlinux.lds @@ -4,7 +4,7 @@ SECTIONS { __stext = . ; __text = .; - *(.text.startup) + *(.kernel.text.startup) *(.text) __etext = . ; } diff --git a/arch/h8300/kernel/init_task.c b/arch/h8300/kernel/init_task.c index cb5dc55..62452f4 100644 --- a/arch/h8300/kernel/init_task.c +++ b/arch/h8300/kernel/init_task.c @@ -36,6 +36,6 @@ EXPORT_SYMBOL(init_task); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/h8300/kernel/vmlinux.lds.S b/arch/h8300/kernel/vmlinux.lds.S index 43a87b9..473e17d 100644 --- a/arch/h8300/kernel/vmlinux.lds.S +++ b/arch/h8300/kernel/vmlinux.lds.S @@ -101,7 +101,7 @@ SECTIONS ___data_start = . ; . = ALIGN(0x2000) ; - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(0x4) ; DATA_DATA . = ALIGN(0x4) ; diff --git a/arch/ia64/include/asm/asmmacro.h b/arch/ia64/include/asm/asmmacro.h index c1642fd..30c8715 100644 --- a/arch/ia64/include/asm/asmmacro.h +++ b/arch/ia64/include/asm/asmmacro.h @@ -70,12 +70,12 @@ name: * path (ivt.S - TLB miss processing) or in places where it might not be * safe to use a "tpa" instruction (mca_asm.S - error recovery). */ - .section ".data.patch.vtop", "a" // declare section & section attributes + .section ".kernel.data.patch.vtop", "a" // declare section & section attributes .previous #define LOAD_PHYSICAL(pr, reg, obj) \ [1:](pr)movl reg = obj; \ - .xdata4 ".data.patch.vtop", 1b-. + .xdata4 ".kernel.data.patch.vtop", 1b-. /* * For now, we always put in the McKinley E9 workaround. On CPUs that don't need it, @@ -84,11 +84,11 @@ name: #define DO_MCKINLEY_E9_WORKAROUND #ifdef DO_MCKINLEY_E9_WORKAROUND - .section ".data.patch.mckinley_e9", "a" + .section ".kernel.data.patch.mckinley_e9", "a" .previous /* workaround for Itanium 2 Errata 9: */ # define FSYS_RETURN \ - .xdata4 ".data.patch.mckinley_e9", 1f-.; \ + .xdata4 ".kernel.data.patch.mckinley_e9", 1f-.; \ 1:{ .mib; \ nop.m 0; \ mov r16=ar.pfs; \ @@ -107,11 +107,11 @@ name: * If physical stack register size is different from DEF_NUM_STACK_REG, * dynamically patch the kernel for correct size. */ - .section ".data.patch.phys_stack_reg", "a" + .section ".kernel.data.patch.phys_stack_reg", "a" .previous #define LOAD_PHYS_STACK_REG_SIZE(reg) \ [1:] adds reg=IA64_NUM_PHYS_STACK_REG*8+8,r0; \ - .xdata4 ".data.patch.phys_stack_reg", 1b-. + .xdata4 ".kernel.data.patch.phys_stack_reg", 1b-. /* * Up until early 2004, use of .align within a function caused bad unwind info. diff --git a/arch/ia64/include/asm/cache.h b/arch/ia64/include/asm/cache.h index e7482bd..7c8d940 100644 --- a/arch/ia64/include/asm/cache.h +++ b/arch/ia64/include/asm/cache.h @@ -24,6 +24,6 @@ # define SMP_CACHE_BYTES (1 << 3) #endif -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif /* _ASM_IA64_CACHE_H */ diff --git a/arch/ia64/include/asm/percpu.h b/arch/ia64/include/asm/percpu.h index 77f30b6..b4a5a2e 100644 --- a/arch/ia64/include/asm/percpu.h +++ b/arch/ia64/include/asm/percpu.h @@ -27,7 +27,7 @@ extern void *per_cpu_init(void); #else /* ! SMP */ -#define PER_CPU_ATTRIBUTES __attribute__((__section__(".data.percpu"))) +#define PER_CPU_ATTRIBUTES __attribute__((__section__(".kernel.data.percpu"))) #define per_cpu_init() (__phys_per_cpu_start) diff --git a/arch/ia64/kernel/Makefile b/arch/ia64/kernel/Makefile index c381ea9..610572a 100644 --- a/arch/ia64/kernel/Makefile +++ b/arch/ia64/kernel/Makefile @@ -72,7 +72,7 @@ GATECFLAGS_gate-syms.o = -r $(obj)/gate-syms.o: $(obj)/gate.lds $(obj)/gate.o FORCE $(call if_changed,gate) -# gate-data.o contains the gate DSO image as data in section .data.gate. +# gate-data.o contains the gate DSO image as data in section .kernel.data.gate. # We must build gate.so before we can assemble it. # Note: kbuild does not track this dependency due to usage of .incbin $(obj)/gate-data.o: $(obj)/gate.so diff --git a/arch/ia64/kernel/gate-data.S b/arch/ia64/kernel/gate-data.S index 258c0a3..7cfa6a2 100644 --- a/arch/ia64/kernel/gate-data.S +++ b/arch/ia64/kernel/gate-data.S @@ -1,3 +1,3 @@ - .section .data.gate, "aw" + .section .kernel.data.gate, "aw" .incbin "arch/ia64/kernel/gate.so" diff --git a/arch/ia64/kernel/gate.S b/arch/ia64/kernel/gate.S index 74b1ccc..eddece2 100644 --- a/arch/ia64/kernel/gate.S +++ b/arch/ia64/kernel/gate.S @@ -20,18 +20,18 @@ * to targets outside the shared object) and to avoid multi-phase kernel builds, we * simply create minimalistic "patch lists" in special ELF sections. */ - .section ".data.patch.fsyscall_table", "a" + .section ".kernel.data.patch.fsyscall_table", "a" .previous #define LOAD_FSYSCALL_TABLE(reg) \ [1:] movl reg=0; \ - .xdata4 ".data.patch.fsyscall_table", 1b-. + .xdata4 ".kernel.data.patch.fsyscall_table", 1b-. - .section ".data.patch.brl_fsys_bubble_down", "a" + .section ".kernel.data.patch.brl_fsys_bubble_down", "a" .previous #define BRL_COND_FSYS_BUBBLE_DOWN(pr) \ [1:](pr)brl.cond.sptk 0; \ ;; \ - .xdata4 ".data.patch.brl_fsys_bubble_down", 1b-. + .xdata4 ".kernel.data.patch.brl_fsys_bubble_down", 1b-. GLOBAL_ENTRY(__kernel_syscall_via_break) .prologue diff --git a/arch/ia64/kernel/gate.lds.S b/arch/ia64/kernel/gate.lds.S index 3cb1abc..9a41f12 100644 --- a/arch/ia64/kernel/gate.lds.S +++ b/arch/ia64/kernel/gate.lds.S @@ -32,21 +32,21 @@ SECTIONS */ . = GATE_ADDR + 0x600; - .data.patch : { + .kernel.data.patch : { __start_gate_mckinley_e9_patchlist = .; - *(.data.patch.mckinley_e9) + *(.kernel.data.patch.mckinley_e9) __end_gate_mckinley_e9_patchlist = .; __start_gate_vtop_patchlist = .; - *(.data.patch.vtop) + *(.kernel.data.patch.vtop) __end_gate_vtop_patchlist = .; __start_gate_fsyscall_patchlist = .; - *(.data.patch.fsyscall_table) + *(.kernel.data.patch.fsyscall_table) __end_gate_fsyscall_patchlist = .; __start_gate_brl_fsys_bubble_down_patchlist = .; - *(.data.patch.brl_fsys_bubble_down) + *(.kernel.data.patch.brl_fsys_bubble_down) __end_gate_brl_fsys_bubble_down_patchlist = .; } :readable diff --git a/arch/ia64/kernel/head.S b/arch/ia64/kernel/head.S index 59301c4..a88dc24 100644 --- a/arch/ia64/kernel/head.S +++ b/arch/ia64/kernel/head.S @@ -181,7 +181,7 @@ swapper_pg_dir: halt_msg: stringz "Halting kernel\n" - .section .text.head,"ax" + .section .kernel.text.head,"ax" .global start_ap diff --git a/arch/ia64/kernel/init_task.c b/arch/ia64/kernel/init_task.c index 5b0e830..0e8c57c 100644 --- a/arch/ia64/kernel/init_task.c +++ b/arch/ia64/kernel/init_task.c @@ -27,7 +27,7 @@ EXPORT_SYMBOL(init_mm); * Initial task structure. * * We need to make sure that this is properly aligned due to the way process stacks are - * handled. This is done by having a special ".data.init_task" section... + * handled. This is done by having a special ".kernel.data.init_task" section... */ #define init_thread_info init_task_mem.s.thread_info @@ -37,7 +37,7 @@ union { struct thread_info thread_info; } s; unsigned long stack[KERNEL_STACK_SIZE/sizeof (unsigned long)]; -} init_task_mem asm ("init_task") __attribute__((section(".data.init_task"))) = {{ +} init_task_mem asm ("init_task") __attribute__((section(".kernel.data.init_task"))) = {{ .task = INIT_TASK(init_task_mem.s.task), .thread_info = INIT_THREAD_INFO(init_task_mem.s.task) }}; diff --git a/arch/ia64/kernel/ivt.S b/arch/ia64/kernel/ivt.S index f675d8e..5bd4502 100644 --- a/arch/ia64/kernel/ivt.S +++ b/arch/ia64/kernel/ivt.S @@ -83,7 +83,7 @@ mov r19=n;; /* prepare to save predicates */ \ br.sptk.many dispatch_to_fault_handler - .section .text.ivt,"ax" + .section .kernel.text.ivt,"ax" .align 32768 // align on 32KB boundary .global ia64_ivt diff --git a/arch/ia64/kernel/minstate.h b/arch/ia64/kernel/minstate.h index 292e214..9bcff80 100644 --- a/arch/ia64/kernel/minstate.h +++ b/arch/ia64/kernel/minstate.h @@ -16,7 +16,7 @@ #define ACCOUNT_SYS_ENTER #endif -.section ".data.patch.rse", "a" +.section ".kernel.data.patch.rse", "a" .previous /* @@ -215,7 +215,7 @@ (pUStk) extr.u r17=r18,3,6; \ (pUStk) sub r16=r18,r22; \ [1:](pKStk) br.cond.sptk.many 1f; \ - .xdata4 ".data.patch.rse",1b-. \ + .xdata4 ".kernel.data.patch.rse",1b-. \ ;; \ cmp.ge p6,p7 = 33,r17; \ ;; \ diff --git a/arch/ia64/kernel/paravirtentry.S b/arch/ia64/kernel/paravirtentry.S index 2f42fcb..418a5f4 100644 --- a/arch/ia64/kernel/paravirtentry.S +++ b/arch/ia64/kernel/paravirtentry.S @@ -24,12 +24,12 @@ #include <asm/asm-offsets.h> #include "entry.h" -#define DATA8(sym, init_value) \ - .pushsection .data.read_mostly ; \ - .align 8 ; \ - .global sym ; \ - sym: ; \ - data8 init_value ; \ +#define DATA8(sym, init_value) \ + .pushsection .kernel.data.read_mostly,"aw",@progbits ; \ + .align 8 ; \ + .global sym ; \ + sym: ; \ + data8 init_value ; \ .popsection #define BRANCH(targ, reg, breg) \ diff --git a/arch/ia64/kernel/vmlinux.lds.S b/arch/ia64/kernel/vmlinux.lds.S index 10a7d47..6e4e8a2 100644 --- a/arch/ia64/kernel/vmlinux.lds.S +++ b/arch/ia64/kernel/vmlinux.lds.S @@ -8,7 +8,7 @@ #define IVT_TEXT \ VMLINUX_SYMBOL(__start_ivt_text) = .; \ - *(.text.ivt) \ + *(.kernel.text.ivt) \ VMLINUX_SYMBOL(__end_ivt_text) = .; OUTPUT_FORMAT("elf64-ia64-little") @@ -51,13 +51,13 @@ SECTIONS KPROBES_TEXT *(.gnu.linkonce.t*) } - .text.head : AT(ADDR(.text.head) - LOAD_OFFSET) - { *(.text.head) } + .kernel.text.head : AT(ADDR(.kernel.text.head) - LOAD_OFFSET) + { *(.kernel.text.head) } .text2 : AT(ADDR(.text2) - LOAD_OFFSET) { *(.text2) } #ifdef CONFIG_SMP - .text.lock : AT(ADDR(.text.lock) - LOAD_OFFSET) - { *(.text.lock) } + .kernel.text.lock : AT(ADDR(.kernel.text.lock) - LOAD_OFFSET) + { *(.kernel.text.lock) } #endif _etext = .; @@ -84,10 +84,10 @@ SECTIONS __stop___mca_table = .; } - .data.patch.phys_stack_reg : AT(ADDR(.data.patch.phys_stack_reg) - LOAD_OFFSET) + .kernel.data.patch.phys_stack_reg : AT(ADDR(.kernel.data.patch.phys_stack_reg) - LOAD_OFFSET) { __start___phys_stack_reg_patchlist = .; - *(.data.patch.phys_stack_reg) + *(.kernel.data.patch.phys_stack_reg) __end___phys_stack_reg_patchlist = .; } @@ -148,24 +148,24 @@ SECTIONS __initcall_end = .; } - .data.patch.vtop : AT(ADDR(.data.patch.vtop) - LOAD_OFFSET) + .kernel.data.patch.vtop : AT(ADDR(.kernel.data.patch.vtop) - LOAD_OFFSET) { __start___vtop_patchlist = .; - *(.data.patch.vtop) + *(.kernel.data.patch.vtop) __end___vtop_patchlist = .; } - .data.patch.rse : AT(ADDR(.data.patch.rse) - LOAD_OFFSET) + .kernel.data.patch.rse : AT(ADDR(.kernel.data.patch.rse) - LOAD_OFFSET) { __start___rse_patchlist = .; - *(.data.patch.rse) + *(.kernel.data.patch.rse) __end___rse_patchlist = .; } - .data.patch.mckinley_e9 : AT(ADDR(.data.patch.mckinley_e9) - LOAD_OFFSET) + .kernel.data.patch.mckinley_e9 : AT(ADDR(.kernel.data.patch.mckinley_e9) - LOAD_OFFSET) { __start___mckinley_e9_bundles = .; - *(.data.patch.mckinley_e9) + *(.kernel.data.patch.mckinley_e9) __end___mckinley_e9_bundles = .; } @@ -193,34 +193,34 @@ SECTIONS __init_end = .; /* The initial task and kernel stack */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) - { *(.data.init_task) } + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) + { *(.kernel.data.init_task) } - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { *(__special_page_section) __start_gate_section = .; - *(.data.gate) + *(.kernel.data.gate) __stop_gate_section = .; } . = ALIGN(PAGE_SIZE); /* make sure the gate page doesn't expose * kernel data */ - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) - { *(.data.read_mostly) } + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) + { *(.kernel.data.read_mostly) } - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) - { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) + { *(.kernel.data.cacheline_aligned) } /* Per-cpu data: */ percpu : { } :percpu . = ALIGN(PERCPU_PAGE_SIZE); __phys_per_cpu_start = .; - .data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) + .kernel.data.percpu PERCPU_ADDR : AT(__phys_per_cpu_start - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } . = __phys_per_cpu_start + PERCPU_PAGE_SIZE; /* ensure percpu data fits diff --git a/arch/ia64/kvm/vmm_ivt.S b/arch/ia64/kvm/vmm_ivt.S index 3ef1a01..38f8465 100644 --- a/arch/ia64/kvm/vmm_ivt.S +++ b/arch/ia64/kvm/vmm_ivt.S @@ -104,7 +104,7 @@ GLOBAL_ENTRY(kvm_vmm_panic) br.call.sptk.many b6=vmm_panic_handler; END(kvm_vmm_panic) - .section .text.ivt,"ax" + .section .kernel.text.ivt,"ax" .align 32768 // align on 32KB boundary .global kvm_ia64_ivt diff --git a/arch/ia64/xen/xensetup.S b/arch/ia64/xen/xensetup.S index 28fed1f..ebeadf4 100644 --- a/arch/ia64/xen/xensetup.S +++ b/arch/ia64/xen/xensetup.S @@ -14,7 +14,7 @@ #include <linux/init.h> #include <xen/interface/elfnote.h> - .section .data.read_mostly + .section .kernel.data.read_mostly, "a" .align 8 .global xen_domain_type xen_domain_type: diff --git a/arch/m32r/kernel/head.S b/arch/m32r/kernel/head.S index 9091606..6a1fc27 100644 --- a/arch/m32r/kernel/head.S +++ b/arch/m32r/kernel/head.S @@ -23,7 +23,7 @@ __INITDATA /* * References to members of the boot_cpu_data structure. */ -.section .text.head, "ax" +.section .kernel.text.head, "ax" .global start_kernel .global __bss_start .global _end diff --git a/arch/m32r/kernel/init_task.c b/arch/m32r/kernel/init_task.c index 016885c..2b8c780 100644 --- a/arch/m32r/kernel/init_task.c +++ b/arch/m32r/kernel/init_task.c @@ -25,7 +25,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/m32r/kernel/vmlinux.lds.S b/arch/m32r/kernel/vmlinux.lds.S index 9db05df..d0bb3d6 100644 --- a/arch/m32r/kernel/vmlinux.lds.S +++ b/arch/m32r/kernel/vmlinux.lds.S @@ -27,7 +27,7 @@ SECTIONS _text = .; /* Text and read-only data */ .boot : { *(.boot) } = 0 .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -57,17 +57,17 @@ SECTIONS . = ALIGN(4096); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : { *(.kernel.data.nosave) } . = ALIGN(4096); __nosave_end = .; . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } _edata = .; /* End of data section */ . = ALIGN(8192); /* init_task */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* will be freed after init */ . = ALIGN(4096); /* Init code and data */ diff --git a/arch/m68k/kernel/head.S b/arch/m68k/kernel/head.S index f513f53..eb51357 100644 --- a/arch/m68k/kernel/head.S +++ b/arch/m68k/kernel/head.S @@ -577,7 +577,7 @@ func_define putn,1 #endif .endm -.section ".text.head","ax" +.section ".kernel.text.head","ax" ENTRY(_stext) /* * Version numbers of the bootinfo interface diff --git a/arch/m68k/kernel/process.c b/arch/m68k/kernel/process.c index 632ce01..29f9872 100644 --- a/arch/m68k/kernel/process.c +++ b/arch/m68k/kernel/process.c @@ -47,7 +47,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); EXPORT_SYMBOL(init_mm); union thread_union init_thread_union -__attribute__((section(".data.init_task"), aligned(THREAD_SIZE))) +__attribute__((section(".kernel.data.init_task"), aligned(THREAD_SIZE))) = { INIT_THREAD_INFO(init_task) }; /* initial task structure */ diff --git a/arch/m68k/kernel/sun3-head.S b/arch/m68k/kernel/sun3-head.S index aad0159..28d3a41 100644 --- a/arch/m68k/kernel/sun3-head.S +++ b/arch/m68k/kernel/sun3-head.S @@ -29,7 +29,7 @@ kernel_pmd_table: .skip 0x2000 .globl kernel_pg_dir .equ kernel_pg_dir,kernel_pmd_table - .section .text.head + .section .kernel.text.head, "ax" ENTRY(_stext) ENTRY(_start) diff --git a/arch/m68k/kernel/vmlinux-std.lds b/arch/m68k/kernel/vmlinux-std.lds index f846d4e..dc1e630 100644 --- a/arch/m68k/kernel/vmlinux-std.lds +++ b/arch/m68k/kernel/vmlinux-std.lds @@ -12,7 +12,7 @@ SECTIONS . = 0x1000; _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -35,7 +35,7 @@ SECTIONS } . = ALIGN(16); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } :data .bss : { *(.bss) } /* BSS */ @@ -78,7 +78,7 @@ SECTIONS . = ALIGN(8192); __init_end = .; - .data.init_task : { *(.data.init_task) } /* The initial task and kernel stack */ + .kernel.data.init_task : { *(.kernel.data.init_task) } /* The initial task and kernel stack */ _end = . ; diff --git a/arch/m68k/kernel/vmlinux-sun3.lds b/arch/m68k/kernel/vmlinux-sun3.lds index d9368c0..7d9d1c3 100644 --- a/arch/m68k/kernel/vmlinux-sun3.lds +++ b/arch/m68k/kernel/vmlinux-sun3.lds @@ -12,7 +12,7 @@ SECTIONS . = 0xE002000; _text = .; /* Text and read-only data */ .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -70,7 +70,7 @@ __init_begin = .; #endif . = ALIGN(PAGE_SIZE); __init_end = .; - .data.init.task : { *(.data.init_task) } + .kernel.data.init.task : { *(.kernel.data.init_task) } .bss : { *(.bss) } /* BSS */ diff --git a/arch/m68knommu/kernel/init_task.c b/arch/m68knommu/kernel/init_task.c index fe282de..b606519 100644 --- a/arch/m68knommu/kernel/init_task.c +++ b/arch/m68knommu/kernel/init_task.c @@ -36,6 +36,6 @@ EXPORT_SYMBOL(init_task); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/m68knommu/kernel/vmlinux.lds.S b/arch/m68knommu/kernel/vmlinux.lds.S index 69ba9b1..d5d617c 100644 --- a/arch/m68knommu/kernel/vmlinux.lds.S +++ b/arch/m68knommu/kernel/vmlinux.lds.S @@ -55,7 +55,7 @@ SECTIONS { .romvec : { __rom_start = . ; _romvec = .; - *(.data.initvect) + *(.kernel.data.initvect) } > romvec #endif @@ -66,7 +66,7 @@ SECTIONS { TEXT_TEXT SCHED_TEXT LOCK_TEXT - *(.text.lock) + *(.kernel.text.lock) . = ALIGN(16); /* Exception table */ __start___ex_table = .; @@ -148,7 +148,7 @@ SECTIONS { _sdata = . ; DATA_DATA . = ALIGN(8192) ; - *(.data.init_task) + *(.kernel.data.init_task) _edata = . ; } > DATA diff --git a/arch/m68knommu/platform/68360/head-ram.S b/arch/m68knommu/platform/68360/head-ram.S index 2ef0624..88a02ed 100644 --- a/arch/m68knommu/platform/68360/head-ram.S +++ b/arch/m68knommu/platform/68360/head-ram.S @@ -280,7 +280,7 @@ _dprbase: * and then overwritten as needed. */ -.section ".data.initvect","awx" +.section ".kernel.data.initvect","awx" .long RAMEND /* Reset: Initial Stack Pointer - 0. */ .long _start /* Reset: Initial Program Counter - 1. */ .long buserr /* Bus Error - 2. */ diff --git a/arch/m68knommu/platform/68360/head-rom.S b/arch/m68knommu/platform/68360/head-rom.S index 62ecf41..b4da6af 100644 --- a/arch/m68knommu/platform/68360/head-rom.S +++ b/arch/m68knommu/platform/68360/head-rom.S @@ -291,7 +291,7 @@ _dprbase: * and then overwritten as needed. */ -.section ".data.initvect","awx" +.section ".kernel.data.initvect","awx" .long RAMEND /* Reset: Initial Stack Pointer - 0. */ .long _start /* Reset: Initial Program Counter - 1. */ .long buserr /* Bus Error - 2. */ diff --git a/arch/mips/kernel/init_task.c b/arch/mips/kernel/init_task.c index 149cd91..425fa1f 100644 --- a/arch/mips/kernel/init_task.c +++ b/arch/mips/kernel/init_task.c @@ -26,7 +26,7 @@ EXPORT_SYMBOL(init_mm); * The things we do for performance.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"), + __attribute__((__section__(".kernel.data.init_task"), __aligned__(THREAD_SIZE))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/mips/kernel/vmlinux.lds.S b/arch/mips/kernel/vmlinux.lds.S index 58738c8..c9a89d8 100644 --- a/arch/mips/kernel/vmlinux.lds.S +++ b/arch/mips/kernel/vmlinux.lds.S @@ -77,7 +77,7 @@ SECTIONS * object file alignment. Using 32768 */ . = ALIGN(_PAGE_SIZE); - *(.data.init_task) + *(.kernel.data.init_task) DATA_DATA CONSTRUCTORS @@ -99,14 +99,14 @@ SECTIONS . = ALIGN(_PAGE_SIZE); .data_nosave : { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(_PAGE_SIZE); __nosave_end = .; . = ALIGN(1 << CONFIG_MIPS_L1_CACHE_SHIFT); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } _edata = .; /* End of data section */ diff --git a/arch/mips/lasat/image/head.S b/arch/mips/lasat/image/head.S index efb95f2..d2e08b3 100644 --- a/arch/mips/lasat/image/head.S +++ b/arch/mips/lasat/image/head.S @@ -1,7 +1,7 @@ #include <asm/lasat/head.h> .text - .section .text.start, "ax" + .section .kernel.text.start, "ax" .set noreorder .set mips3 diff --git a/arch/mips/lasat/image/romscript.normal b/arch/mips/lasat/image/romscript.normal index 988f8ad..9429398 100644 --- a/arch/mips/lasat/image/romscript.normal +++ b/arch/mips/lasat/image/romscript.normal @@ -4,7 +4,7 @@ SECTIONS { .text : { - *(.text.start) + *(.kernel.text.start) } /* Data in ROM */ diff --git a/arch/mn10300/kernel/head.S b/arch/mn10300/kernel/head.S index 606bd8c..1b4b275 100644 --- a/arch/mn10300/kernel/head.S +++ b/arch/mn10300/kernel/head.S @@ -19,7 +19,7 @@ #include <asm/param.h> #include <asm/unit/serial.h> - .section .text.head,"ax" + .section .kernel.text.head,"ax" ############################################################################### # diff --git a/arch/mn10300/kernel/init_task.c b/arch/mn10300/kernel/init_task.c index 5ac3566..0758121 100644 --- a/arch/mn10300/kernel/init_task.c +++ b/arch/mn10300/kernel/init_task.c @@ -31,7 +31,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/mn10300/kernel/vmlinux.lds.S b/arch/mn10300/kernel/vmlinux.lds.S index b825966..6d139e2 100644 --- a/arch/mn10300/kernel/vmlinux.lds.S +++ b/arch/mn10300/kernel/vmlinux.lds.S @@ -28,7 +28,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : { *( - .text.head + .kernel.text.head .text ) TEXT_TEXT @@ -58,25 +58,25 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : { *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; . = ALIGN(PAGE_SIZE); - .data.page_aligned : { *(.data.idt) } + .kernel.data.page_aligned : { *(.kernel.data.idt) } . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .kernel.data.cacheline_aligned : { *(.kernel.data.cacheline_aligned) } /* rarely changed data like cpu maps */ . = ALIGN(32); - .data.read_mostly : AT(ADDR(.data.read_mostly)) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly)) { + *(.kernel.data.read_mostly) _edata = .; /* End of data section */ } . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* might get freed after init */ . = ALIGN(PAGE_SIZE); @@ -134,7 +134,7 @@ SECTIONS __bss_start = .; /* BSS */ .bss : { - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) } . = ALIGN(4); diff --git a/arch/parisc/include/asm/cache.h b/arch/parisc/include/asm/cache.h index 32c2cca..d1e16aa 100644 --- a/arch/parisc/include/asm/cache.h +++ b/arch/parisc/include/asm/cache.h @@ -28,7 +28,7 @@ #define SMP_CACHE_BYTES L1_CACHE_BYTES -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) void parisc_cache_init(void); /* initializes cache-flushing */ void disable_sr_hashing_asm(int); /* low level support for above */ diff --git a/arch/parisc/include/asm/system.h b/arch/parisc/include/asm/system.h index ee80c92..7a6ac0c 100644 --- a/arch/parisc/include/asm/system.h +++ b/arch/parisc/include/asm/system.h @@ -174,7 +174,7 @@ static inline void set_eiem(unsigned long val) }) #ifdef CONFIG_SMP -# define __lock_aligned __attribute__((__section__(".data.lock_aligned"))) +# define __lock_aligned __attribute__((__section__(".kernel.data.lock_aligned"))) #endif #define arch_align_stack(x) (x) diff --git a/arch/parisc/kernel/head.S b/arch/parisc/kernel/head.S index 0e3d9f9..9112df2 100644 --- a/arch/parisc/kernel/head.S +++ b/arch/parisc/kernel/head.S @@ -345,7 +345,7 @@ smp_slave_stext: ENDPROC(stext) #ifndef CONFIG_64BIT - .section .data.read_mostly + .section .kernel.data.read_mostly,"aw",@progbits .align 4 .export $global$,data diff --git a/arch/parisc/kernel/init_task.c b/arch/parisc/kernel/init_task.c index 1e25a45..97aceac 100644 --- a/arch/parisc/kernel/init_task.c +++ b/arch/parisc/kernel/init_task.c @@ -48,7 +48,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((aligned(128))) __attribute__((__section__(".data.init_task"))) = + __attribute__((aligned(128))) __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; #if PT_NLEVELS == 3 @@ -57,11 +57,11 @@ union thread_union init_thread_union * guarantee that global objects will be laid out in memory in the same order * as the order of declaration, so put these in different sections and use * the linker script to order them. */ -pmd_t pmd0[PTRS_PER_PMD] __attribute__ ((__section__ (".data.vm0.pmd"), aligned(PAGE_SIZE))); +pmd_t pmd0[PTRS_PER_PMD] __attribute__ ((__section__ (".kernel.data.vm0.pmd"), aligned(PAGE_SIZE))); #endif -pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__ ((__section__ (".data.vm0.pgd"), aligned(PAGE_SIZE))); -pte_t pg0[PT_INITIAL * PTRS_PER_PTE] __attribute__ ((__section__ (".data.vm0.pte"), aligned(PAGE_SIZE))); +pgd_t swapper_pg_dir[PTRS_PER_PGD] __attribute__ ((__section__ (".kernel.data.vm0.pgd"), aligned(PAGE_SIZE))); +pte_t pg0[PT_INITIAL * PTRS_PER_PTE] __attribute__ ((__section__ (".kernel.data.vm0.pte"), aligned(PAGE_SIZE))); /* * Initial task structure. diff --git a/arch/parisc/kernel/vmlinux.lds.S b/arch/parisc/kernel/vmlinux.lds.S index 1a3b6cc..861f15b 100644 --- a/arch/parisc/kernel/vmlinux.lds.S +++ b/arch/parisc/kernel/vmlinux.lds.S @@ -94,8 +94,8 @@ SECTIONS /* rarely changed data like cpu maps */ . = ALIGN(16); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } . = ALIGN(L1_CACHE_BYTES); @@ -106,14 +106,14 @@ SECTIONS } . = ALIGN(L1_CACHE_BYTES); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } /* PA-RISC locks requires 16-byte alignment */ . = ALIGN(16); - .data.lock_aligned : { - *(.data.lock_aligned) + .kernel.data.lock_aligned : { + *(.kernel.data.lock_aligned) } /* nosave data is really only used for software suspend...it's here @@ -122,7 +122,7 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; .data_nosave : { - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -134,10 +134,10 @@ SECTIONS __bss_start = .; /* page table entries need to be PAGE_SIZE aligned */ . = ALIGN(PAGE_SIZE); - .data.vmpages : { - *(.data.vm0.pmd) - *(.data.vm0.pgd) - *(.data.vm0.pte) + .kernel.data.vmpages : { + *(.kernel.data.vm0.pmd) + *(.kernel.data.vm0.pgd) + *(.kernel.data.vm0.pte) } .bss : { *(.bss) @@ -149,8 +149,8 @@ SECTIONS /* assembler code expects init_task to be 16k aligned */ . = ALIGN(16384); /* init_task */ - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } #ifdef CONFIG_64BIT diff --git a/arch/powerpc/include/asm/cache.h b/arch/powerpc/include/asm/cache.h index 81de6eb..69ead95 100644 --- a/arch/powerpc/include/asm/cache.h +++ b/arch/powerpc/include/asm/cache.h @@ -38,7 +38,7 @@ extern struct ppc64_caches ppc64_caches; #endif /* __powerpc64__ && ! __ASSEMBLY__ */ #if !defined(__ASSEMBLY__) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif #endif /* __KERNEL__ */ diff --git a/arch/powerpc/include/asm/page_64.h b/arch/powerpc/include/asm/page_64.h index 043bfdf..2e5f76b 100644 --- a/arch/powerpc/include/asm/page_64.h +++ b/arch/powerpc/include/asm/page_64.h @@ -157,7 +157,7 @@ do { \ #else #define __page_aligned \ __attribute__((__aligned__(PAGE_SIZE), \ - __section__(".data.page_aligned"))) + __section__(".kernel.data.page_aligned"))) #endif #define VM_DATA_DEFAULT_FLAGS \ diff --git a/arch/powerpc/include/asm/ppc_asm.h b/arch/powerpc/include/asm/ppc_asm.h index 1a0d628..c115bc7 100644 --- a/arch/powerpc/include/asm/ppc_asm.h +++ b/arch/powerpc/include/asm/ppc_asm.h @@ -193,7 +193,7 @@ name: \ GLUE(.,name): #define _INIT_GLOBAL(name) \ - .section ".text.init.refok"; \ + .section ".kernel.text.init.refok","ax",@progbits; \ .align 2 ; \ .globl name; \ .globl GLUE(.,name); \ @@ -233,7 +233,7 @@ name: \ GLUE(.,name): #define _INIT_STATIC(name) \ - .section ".text.init.refok"; \ + .section ".kernel.text.init.refok","ax",@progbits; \ .align 2 ; \ .section ".opd","aw"; \ name: \ diff --git a/arch/powerpc/kernel/head_32.S b/arch/powerpc/kernel/head_32.S index a1c4cfd..fe8bbb1 100644 --- a/arch/powerpc/kernel/head_32.S +++ b/arch/powerpc/kernel/head_32.S @@ -50,7 +50,7 @@ mtspr SPRN_DBAT##n##L,RB; \ 1: - .section .text.head, "ax" + .section .kernel.text.head, "ax" .stabs "arch/powerpc/kernel/",N_SO,0,0,0f .stabs "head_32.S",N_SO,0,0,0f 0: diff --git a/arch/powerpc/kernel/head_40x.S b/arch/powerpc/kernel/head_40x.S index 56d8e5d..d18a199 100644 --- a/arch/powerpc/kernel/head_40x.S +++ b/arch/powerpc/kernel/head_40x.S @@ -52,7 +52,7 @@ * * This is all going to change RSN when we add bi_recs....... -- Dan */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); diff --git a/arch/powerpc/kernel/head_44x.S b/arch/powerpc/kernel/head_44x.S index b56fecc..1c88bcc 100644 --- a/arch/powerpc/kernel/head_44x.S +++ b/arch/powerpc/kernel/head_44x.S @@ -50,7 +50,7 @@ * r7 - End of kernel command line string * */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); /* diff --git a/arch/powerpc/kernel/head_8xx.S b/arch/powerpc/kernel/head_8xx.S index 3c9452d..549a236 100644 --- a/arch/powerpc/kernel/head_8xx.S +++ b/arch/powerpc/kernel/head_8xx.S @@ -38,7 +38,7 @@ #else #define DO_8xx_CPU6(val, reg) #endif - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); diff --git a/arch/powerpc/kernel/head_fsl_booke.S b/arch/powerpc/kernel/head_fsl_booke.S index 36ffb35..d2c8371 100644 --- a/arch/powerpc/kernel/head_fsl_booke.S +++ b/arch/powerpc/kernel/head_fsl_booke.S @@ -53,7 +53,7 @@ * r7 - End of kernel command line string * */ - .section .text.head, "ax" + .section .kernel.text.head, "ax" _ENTRY(_stext); _ENTRY(_start); /* diff --git a/arch/powerpc/kernel/init_task.c b/arch/powerpc/kernel/init_task.c index 688b329..bac0699 100644 --- a/arch/powerpc/kernel/init_task.c +++ b/arch/powerpc/kernel/init_task.c @@ -21,7 +21,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/powerpc/kernel/machine_kexec_64.c b/arch/powerpc/kernel/machine_kexec_64.c index 49e705f..3cd7b25 100644 --- a/arch/powerpc/kernel/machine_kexec_64.c +++ b/arch/powerpc/kernel/machine_kexec_64.c @@ -250,7 +250,7 @@ static void kexec_prepare_cpus(void) * current, but that audit has not been performed. */ static union thread_union kexec_stack - __attribute__((__section__(".data.init_task"))) = { }; + __attribute__((__section__(".kernel.data.init_task"))) = { }; /* Our assembly helper, in kexec_stub.S */ extern NORET_TYPE void kexec_sequence(void *newstack, unsigned long start, diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c index ad06d5c..f4130cc 100644 --- a/arch/powerpc/kernel/vdso.c +++ b/arch/powerpc/kernel/vdso.c @@ -74,7 +74,7 @@ static int vdso_ready; static union { struct vdso_data data; u8 page[PAGE_SIZE]; -} vdso_data_store __attribute__((__section__(".data.page_aligned"))); +} vdso_data_store __attribute__((__section__(".kernel.data.page_aligned"))); struct vdso_data *vdso_data = &vdso_data_store.data; /* Format of the patch table */ diff --git a/arch/powerpc/kernel/vdso32/vdso32_wrapper.S b/arch/powerpc/kernel/vdso32/vdso32_wrapper.S index 556f0ca..e1b3e41 100644 --- a/arch/powerpc/kernel/vdso32/vdso32_wrapper.S +++ b/arch/powerpc/kernel/vdso32/vdso32_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned","aw",@progbits .globl vdso32_start, vdso32_end .balign PAGE_SIZE diff --git a/arch/powerpc/kernel/vdso64/vdso64_wrapper.S b/arch/powerpc/kernel/vdso64/vdso64_wrapper.S index 0529cb9..1d96b15 100644 --- a/arch/powerpc/kernel/vdso64/vdso64_wrapper.S +++ b/arch/powerpc/kernel/vdso64/vdso64_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned","aw",@progbits .globl vdso64_start, vdso64_end .balign PAGE_SIZE diff --git a/arch/powerpc/kernel/vmlinux.lds.S b/arch/powerpc/kernel/vmlinux.lds.S index 161b9b9..5aa917f 100644 --- a/arch/powerpc/kernel/vmlinux.lds.S +++ b/arch/powerpc/kernel/vmlinux.lds.S @@ -52,9 +52,9 @@ SECTIONS /* Text and gots */ .text : AT(ADDR(.text) - LOAD_OFFSET) { ALIGN_FUNCTION(); - *(.text.head) + *(.kernel.text.head) _text = .; - *(.text .fixup .text.init.refok .exit.text.refok __ftr_alt_*) + *(.text .fixup .kernel.text.init.refok .kernel.text.exit.refok __ftr_alt_*) SCHED_TEXT LOCK_TEXT KPROBES_TEXT @@ -182,10 +182,10 @@ SECTIONS } #endif . = ALIGN(PAGE_SIZE); - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } @@ -259,28 +259,28 @@ SECTIONS #else . = ALIGN(16384); #endif - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) } . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) } - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } . = ALIGN(L1_CACHE_BYTES); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) } . = ALIGN(PAGE_SIZE); .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; } diff --git a/arch/s390/include/asm/cache.h b/arch/s390/include/asm/cache.h index 9b86681..2890a0f 100644 --- a/arch/s390/include/asm/cache.h +++ b/arch/s390/include/asm/cache.h @@ -14,6 +14,6 @@ #define L1_CACHE_BYTES 256 #define L1_CACHE_SHIFT 8 -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #endif diff --git a/arch/s390/kernel/head.S b/arch/s390/kernel/head.S index ec7e35f..cd2bfb7 100644 --- a/arch/s390/kernel/head.S +++ b/arch/s390/kernel/head.S @@ -35,7 +35,7 @@ #define ARCH_OFFSET 0 #endif -.section ".text.head","ax" +.section ".kernel.text.head","ax" #ifndef CONFIG_IPL .org 0 .long 0x00080000,0x80000000+startup # Just a restart PSW diff --git a/arch/s390/kernel/init_task.c b/arch/s390/kernel/init_task.c index 7db95c0..1615535 100644 --- a/arch/s390/kernel/init_task.c +++ b/arch/s390/kernel/init_task.c @@ -30,7 +30,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/s390/kernel/vdso.c b/arch/s390/kernel/vdso.c index 690e178..f1127ef 100644 --- a/arch/s390/kernel/vdso.c +++ b/arch/s390/kernel/vdso.c @@ -64,7 +64,7 @@ __setup("vdso=", vdso_setup); static union { struct vdso_data data; u8 page[PAGE_SIZE]; -} vdso_data_store __attribute__((__section__(".data.page_aligned"))); +} vdso_data_store __attribute__((__section__(".kernel.data.page_aligned"))); struct vdso_data *vdso_data = &vdso_data_store.data; /* diff --git a/arch/s390/kernel/vdso32/vdso32_wrapper.S b/arch/s390/kernel/vdso32/vdso32_wrapper.S index 61639a8..23830ec 100644 --- a/arch/s390/kernel/vdso32/vdso32_wrapper.S +++ b/arch/s390/kernel/vdso32/vdso32_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned", "aw" .globl vdso32_start, vdso32_end .balign PAGE_SIZE diff --git a/arch/s390/kernel/vdso64/vdso64_wrapper.S b/arch/s390/kernel/vdso64/vdso64_wrapper.S index d8e2ac1..6fb0923 100644 --- a/arch/s390/kernel/vdso64/vdso64_wrapper.S +++ b/arch/s390/kernel/vdso64/vdso64_wrapper.S @@ -1,7 +1,7 @@ #include <linux/init.h> #include <asm/page.h> - .section ".data.page_aligned" + .section ".kernel.data.page_aligned", "aw" .globl vdso64_start, vdso64_end .balign PAGE_SIZE diff --git a/arch/s390/kernel/vmlinux.lds.S b/arch/s390/kernel/vmlinux.lds.S index d796d05..873cac2 100644 --- a/arch/s390/kernel/vmlinux.lds.S +++ b/arch/s390/kernel/vmlinux.lds.S @@ -29,7 +29,7 @@ SECTIONS . = 0x00000000; .text : { _text = .; /* Text and read-only data */ - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -66,30 +66,30 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data_nosave : { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; . = ALIGN(PAGE_SIZE); - .data.page_aligned : { - *(.data.idt) + .kernel.data.page_aligned : { + *(.kernel.data.idt) } . = ALIGN(0x100); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { + *(.kernel.data.cacheline_aligned) } . = ALIGN(0x100); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } _edata = .; /* End of data section */ . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } /* will be freed after init */ diff --git a/arch/sh/include/asm/cache.h b/arch/sh/include/asm/cache.h index 02df18e..f44fed5 100644 --- a/arch/sh/include/asm/cache.h +++ b/arch/sh/include/asm/cache.h @@ -14,7 +14,7 @@ #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifndef __ASSEMBLY__ struct cache_info { diff --git a/arch/sh/kernel/cpu/sh5/entry.S b/arch/sh/kernel/cpu/sh5/entry.S index e640c63..43d031a 100644 --- a/arch/sh/kernel/cpu/sh5/entry.S +++ b/arch/sh/kernel/cpu/sh5/entry.S @@ -2058,10 +2058,10 @@ asm_uaccess_end: /* - * --- .text.init Section + * --- .kernel.text.init Section */ - .section .text.init, "ax" + .section .kernel.text.init, "ax" /* * void trap_init (void) diff --git a/arch/sh/kernel/head_32.S b/arch/sh/kernel/head_32.S index 788605f..d71c37b 100644 --- a/arch/sh/kernel/head_32.S +++ b/arch/sh/kernel/head_32.S @@ -40,7 +40,7 @@ ENTRY(empty_zero_page) 1: .skip PAGE_SIZE - empty_zero_page - 1b - .section .text.head, "ax" + .section .kernel.text.head, "ax" /* * Condition at the entry of _stext: diff --git a/arch/sh/kernel/head_64.S b/arch/sh/kernel/head_64.S index 7ccfb99..cf6ca73 100644 --- a/arch/sh/kernel/head_64.S +++ b/arch/sh/kernel/head_64.S @@ -110,7 +110,7 @@ empty_bad_pte_table: fpu_in_use: .quad 0 - .section .text.head, "ax" + .section .kernel.text.head, "ax" .balign L1_CACHE_BYTES /* * Condition at the entry of __stext: diff --git a/arch/sh/kernel/init_task.c b/arch/sh/kernel/init_task.c index 80c35ff..024f4c5 100644 --- a/arch/sh/kernel/init_task.c +++ b/arch/sh/kernel/init_task.c @@ -21,7 +21,7 @@ EXPORT_SYMBOL(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* diff --git a/arch/sh/kernel/irq.c b/arch/sh/kernel/irq.c index 64b7690..9df37f6 100644 --- a/arch/sh/kernel/irq.c +++ b/arch/sh/kernel/irq.c @@ -158,10 +158,10 @@ asmlinkage int do_IRQ(unsigned int irq, struct pt_regs *regs) #ifdef CONFIG_IRQSTACKS static char softirq_stack[NR_CPUS * THREAD_SIZE] - __attribute__((__section__(".bss.page_aligned"))); + __attribute__((__section__(".bss.kernel.page_aligned"))); static char hardirq_stack[NR_CPUS * THREAD_SIZE] - __attribute__((__section__(".bss.page_aligned"))); + __attribute__((__section__(".bss.kernel.page_aligned"))); /* * allocate per-cpu stacks for hardirq and for softirq processing diff --git a/arch/sh/kernel/vmlinux_32.lds.S b/arch/sh/kernel/vmlinux_32.lds.S index 7b4b82b..04dd19f 100644 --- a/arch/sh/kernel/vmlinux_32.lds.S +++ b/arch/sh/kernel/vmlinux_32.lds.S @@ -28,7 +28,7 @@ SECTIONS } = 0 .text : { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -58,19 +58,19 @@ SECTIONS . = ALIGN(THREAD_SIZE); .data : { /* Data */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) . = ALIGN(L1_CACHE_BYTES); - *(.data.read_mostly) + *(.kernel.data.read_mostly) . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -128,7 +128,7 @@ SECTIONS .bss : { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) *(COMMON) . = ALIGN(4); diff --git a/arch/sh/kernel/vmlinux_64.lds.S b/arch/sh/kernel/vmlinux_64.lds.S index 33fa464..6059a49 100644 --- a/arch/sh/kernel/vmlinux_64.lds.S +++ b/arch/sh/kernel/vmlinux_64.lds.S @@ -42,7 +42,7 @@ SECTIONS } = 0 .text : C_PHYS(.text) { - *(.text.head) + *(.kernel.text.head) TEXT_TEXT *(.text64) *(.text..SHmedia32) @@ -70,19 +70,19 @@ SECTIONS . = ALIGN(THREAD_SIZE); .data : C_PHYS(.data) { /* Data */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(L1_CACHE_BYTES); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) . = ALIGN(L1_CACHE_BYTES); - *(.data.read_mostly) + *(.kernel.data.read_mostly) . = ALIGN(PAGE_SIZE); - *(.data.page_aligned) + *(.kernel.data.page_aligned) __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; @@ -140,7 +140,7 @@ SECTIONS .bss : C_PHYS(.bss) { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) *(COMMON) . = ALIGN(4); diff --git a/arch/sparc/boot/btfixupprep.c b/arch/sparc/boot/btfixupprep.c index 52a4208..5f8472b 100644 --- a/arch/sparc/boot/btfixupprep.c +++ b/arch/sparc/boot/btfixupprep.c @@ -171,7 +171,7 @@ main1: } } else if (buffer[nbase+4] != '_') continue; - if (!strcmp (sect, ".text.exit")) + if (!strcmp (sect, ".kernel.text.exit")) continue; if (strcmp (sect, ".text") && strcmp (sect, ".init.text") && @@ -325,7 +325,7 @@ main1: (*rr)->next = NULL; } printf("! Generated by btfixupprep. Do not edit.\n\n"); - printf("\t.section\t\".data.init\",#alloc,#write\n\t.align\t4\n\n"); + printf("\t.section\t\".kernel.data.init\",#alloc,#write\n\t.align\t4\n\n"); printf("\t.global\t___btfixup_start\n___btfixup_start:\n\n"); for (i = 0; i < last; i++) { f = array + i; diff --git a/arch/sparc/include/asm/cache.h b/arch/sparc/include/asm/cache.h index 41f85ae..19f1a48 100644 --- a/arch/sparc/include/asm/cache.h +++ b/arch/sparc/include/asm/cache.h @@ -19,7 +19,7 @@ #define SMP_CACHE_BYTES (1 << SMP_CACHE_BYTES_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifdef CONFIG_SPARC32 #include <asm/asi.h> diff --git a/arch/sparc/kernel/head_32.S b/arch/sparc/kernel/head_32.S index f0b4b51..317f636 100644 --- a/arch/sparc/kernel/head_32.S +++ b/arch/sparc/kernel/head_32.S @@ -72,7 +72,7 @@ sun4e_notsup: .align 4 /* The Sparc trap table, bootloader gives us control at _start. */ - .section .text.head,"ax" + .section .kernel.text.head,"ax" .globl start, _stext, _start, __stext .globl trapbase _start: /* danger danger */ diff --git a/arch/sparc/kernel/head_64.S b/arch/sparc/kernel/head_64.S index 8ffee71..82d26b7 100644 --- a/arch/sparc/kernel/head_64.S +++ b/arch/sparc/kernel/head_64.S @@ -467,7 +467,7 @@ jump_to_sun4u_init: jmpl %g2 + %g0, %g0 nop - .section .text.init.refok + .section .kernel.text.init.refok,"ax",@progbits sun4u_init: BRANCH_IF_SUN4V(g1, sun4v_init) diff --git a/arch/sparc/kernel/init_task.c b/arch/sparc/kernel/init_task.c index f28cb82..2fa8e35 100644 --- a/arch/sparc/kernel/init_task.c +++ b/arch/sparc/kernel/init_task.c @@ -22,5 +22,5 @@ EXPORT_SYMBOL(init_task); * in etrap.S which assumes it. */ union thread_union init_thread_union - __attribute__((section (".data.init_task"))) + __attribute__((section (".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/sparc/kernel/vmlinux.lds.S b/arch/sparc/kernel/vmlinux.lds.S index 7626708..14e7761 100644 --- a/arch/sparc/kernel/vmlinux.lds.S +++ b/arch/sparc/kernel/vmlinux.lds.S @@ -41,7 +41,7 @@ SECTIONS .text TEXTSTART : { _text = .; - *(.text.head) + *(.kernel.text.head) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -59,20 +59,20 @@ SECTIONS *(.data1) } . = ALIGN(SMP_CACHE_BYTES); - .data.cacheline_aligned : { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : { ; + *(.kernel.data.cacheline_aligned) } . = ALIGN(SMP_CACHE_BYTES); - .data.read_mostly : { - *(.data.read_mostly) + .kernel.data.read_mostly : { + *(.kernel.data.read_mostly) } /* End of data section */ _edata = .; /* init_task */ . = ALIGN(THREAD_SIZE); - .data.init_task : { - *(.data.init_task) + .kernel.data.init_task : { + *(.kernel.data.init_task) } .fixup : { __start___fixup = .; diff --git a/arch/um/include/asm/common.lds.S b/arch/um/include/asm/common.lds.S index cb02486..9ac4b36 100644 --- a/arch/um/include/asm/common.lds.S +++ b/arch/um/include/asm/common.lds.S @@ -49,9 +49,9 @@ } . = ALIGN(32); - .data.percpu : { + .kernel.data.percpu : { __per_cpu_start = . ; - *(.data.percpu) + *(.kernel.data.percpu) __per_cpu_end = . ; } diff --git a/arch/um/kernel/dyn.lds.S b/arch/um/kernel/dyn.lds.S index 9975e1a..1ee6934 100644 --- a/arch/um/kernel/dyn.lds.S +++ b/arch/um/kernel/dyn.lds.S @@ -97,9 +97,9 @@ SECTIONS .fini_array : { *(.fini_array) } .data : { . = ALIGN(KERNEL_STACK_SIZE); /* init_task */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(KERNEL_STACK_SIZE); - *(.data.init_irqstack) + *(.kernel.data.init_irqstack) DATA_DATA *(.data.* .gnu.linkonce.d.*) SORT(CONSTRUCTORS) diff --git a/arch/um/kernel/init_task.c b/arch/um/kernel/init_task.c index 806d381..f7627d0 100644 --- a/arch/um/kernel/init_task.c +++ b/arch/um/kernel/init_task.c @@ -34,9 +34,9 @@ EXPORT_SYMBOL(init_task); */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; union thread_union cpu0_irqstack - __attribute__((__section__(".data.init_irqstack"))) = + __attribute__((__section__(".kernel.data.init_irqstack"))) = { INIT_THREAD_INFO(init_task) }; diff --git a/arch/um/kernel/uml.lds.S b/arch/um/kernel/uml.lds.S index 11b8352..a0b6650 100644 --- a/arch/um/kernel/uml.lds.S +++ b/arch/um/kernel/uml.lds.S @@ -53,9 +53,9 @@ SECTIONS .data : { . = ALIGN(KERNEL_STACK_SIZE); /* init_task */ - *(.data.init_task) + *(.kernel.data.init_task) . = ALIGN(KERNEL_STACK_SIZE); - *(.data.init_irqstack) + *(.kernel.data.init_irqstack) DATA_DATA *(.gnu.linkonce.d*) CONSTRUCTORS diff --git a/arch/x86/boot/compressed/head_32.S b/arch/x86/boot/compressed/head_32.S index 29c5fbf..2c89079 100644 --- a/arch/x86/boot/compressed/head_32.S +++ b/arch/x86/boot/compressed/head_32.S @@ -29,7 +29,7 @@ #include <asm/boot.h> #include <asm/asm-offsets.h> -.section ".text.head","ax",@progbits +.section ".kernel.text.head","ax",@progbits .globl startup_32 startup_32: diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index 1d5dff4..87bfcbe 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -33,7 +33,7 @@ #include <asm/processor-flags.h> #include <asm/asm-offsets.h> -.section ".text.head" +.section ".kernel.text.head","ax",@progbits .code32 .globl startup_32 diff --git a/arch/x86/boot/compressed/vmlinux.scr b/arch/x86/boot/compressed/vmlinux.scr index f02382a..e4dc231 100644 --- a/arch/x86/boot/compressed/vmlinux.scr +++ b/arch/x86/boot/compressed/vmlinux.scr @@ -1,6 +1,6 @@ SECTIONS { - .rodata.compressed : { + .kernel.rodata.compressed : { input_len = .; LONG(input_data_end - input_data) input_data = .; *(.data) diff --git a/arch/x86/boot/compressed/vmlinux_32.lds b/arch/x86/boot/compressed/vmlinux_32.lds index bb3c483..a24e272 100644 --- a/arch/x86/boot/compressed/vmlinux_32.lds +++ b/arch/x86/boot/compressed/vmlinux_32.lds @@ -7,13 +7,13 @@ SECTIONS * address 0. */ . = 0; - .text.head : { + .kernel.text.head : { _head = . ; - *(.text.head) + *(.kernel.text.head) _ehead = . ; } - .rodata.compressed : { - *(.rodata.compressed) + .kernel.rodata.compressed : { + *(.kernel.rodata.compressed) } .text : { _text = .; /* Text */ @@ -21,6 +21,10 @@ SECTIONS *(.text.*) _etext = . ; } + .got : { + *(.got) + *(.got.*) + } .rodata : { _rodata = . ; *(.rodata) /* read-only data */ @@ -40,4 +44,6 @@ SECTIONS *(COMMON) _end = . ; } + /* Be bold, and discard everything not explicitly mentioned */ + /DISCARD/ : { *(*) } } diff --git a/arch/x86/boot/compressed/vmlinux_64.lds b/arch/x86/boot/compressed/vmlinux_64.lds index bef1ac8..9414e60 100644 --- a/arch/x86/boot/compressed/vmlinux_64.lds +++ b/arch/x86/boot/compressed/vmlinux_64.lds @@ -7,13 +7,13 @@ SECTIONS * address 0. */ . = 0; - .text.head : { + .kernel.text.head : { _head = . ; - *(.text.head) + *(.kernel.text.head) _ehead = . ; } - .rodata.compressed : { - *(.rodata.compressed) + .kernel.rodata.compressed : { + *(.kernel.rodata.compressed) } .text : { _text = .; /* Text */ @@ -45,4 +45,6 @@ SECTIONS . = . + 4096 * 6; _ebss = .; } + /* Be bold, and discard everything not explicitly mentioned */ + /DISCARD/ : { *(*) } } diff --git a/arch/x86/include/asm/cache.h b/arch/x86/include/asm/cache.h index 5d367ca..980a79f 100644 --- a/arch/x86/include/asm/cache.h +++ b/arch/x86/include/asm/cache.h @@ -5,7 +5,7 @@ #define L1_CACHE_SHIFT (CONFIG_X86_L1_CACHE_SHIFT) #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) -#define __read_mostly __attribute__((__section__(".data.read_mostly"))) +#define __read_mostly __attribute__((__section__(".kernel.data.read_mostly"))) #ifdef CONFIG_X86_VSMP /* vSMP Internode cacheline shift */ @@ -13,7 +13,7 @@ #ifdef CONFIG_SMP #define __cacheline_aligned_in_smp \ __attribute__((__aligned__(1 << (INTERNODE_CACHE_SHIFT)))) \ - __attribute__((__section__(".data.page_aligned"))) + __attribute__((__section__(".kernel.data.page_aligned"))) #endif #endif diff --git a/arch/x86/kernel/acpi/wakeup_32.S b/arch/x86/kernel/acpi/wakeup_32.S index a12e6a9..1b1beb9 100644 --- a/arch/x86/kernel/acpi/wakeup_32.S +++ b/arch/x86/kernel/acpi/wakeup_32.S @@ -1,4 +1,4 @@ - .section .text.page_aligned + .section .kernel.text.page_aligned,"ax",@progbits #include <linux/linkage.h> #include <asm/segment.h> #include <asm/page.h> diff --git a/arch/x86/kernel/head_32.S b/arch/x86/kernel/head_32.S index e835b4e..ae011eb 100644 --- a/arch/x86/kernel/head_32.S +++ b/arch/x86/kernel/head_32.S @@ -81,7 +81,7 @@ INIT_MAP_BEYOND_END = BOOTBITMAP_SIZE + (PAGE_TABLE_SIZE + ALLOCATOR_SLOP)*PAGE_ * any particular GDT layout, because we load our own as soon as we * can. */ -.section .text.head,"ax",@progbits +.section .kernel.text.head,"ax",@progbits ENTRY(startup_32) /* test KEEP_SEGMENTS flag to see if the bootloader is asking us to not reload segments */ @@ -609,7 +609,7 @@ ENTRY(_stext) /* * BSS section */ -.section ".bss.page_aligned","wa" +.section ".bss.kernel.page_aligned","wa" .align PAGE_SIZE_asm #ifdef CONFIG_X86_PAE swapper_pg_pmd: @@ -626,7 +626,7 @@ ENTRY(empty_zero_page) * This starts the data section. */ #ifdef CONFIG_X86_PAE -.section ".data.page_aligned","wa" +.section ".kernel.data.page_aligned","wa" /* Page-aligned for the benefit of paravirt? */ .align PAGE_SIZE_asm ENTRY(swapper_pg_dir) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 0e275d4..02f0d80 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -40,7 +40,7 @@ L4_START_KERNEL = pgd_index(__START_KERNEL_map) L3_START_KERNEL = pud_index(__START_KERNEL_map) .text - .section .text.head + .section .kernel.text.head,"ax",@progbits .code64 .globl startup_64 startup_64: @@ -414,7 +414,7 @@ ENTRY(phys_base) ENTRY(idt_table) .skip 256 * 16 - .section .bss.page_aligned, "aw", @nobits + .section .bss.kernel.page_aligned, "aw", @nobits .align PAGE_SIZE ENTRY(empty_zero_page) .skip PAGE_SIZE diff --git a/arch/x86/kernel/init_task.c b/arch/x86/kernel/init_task.c index df3bf26..a54e0be 100644 --- a/arch/x86/kernel/init_task.c +++ b/arch/x86/kernel/init_task.c @@ -22,7 +22,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); * "init_task" linker map entry.. */ union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; /* @@ -36,7 +36,7 @@ EXPORT_SYMBOL(init_task); /* * per-CPU TSS segments. Threads are completely 'soft' on Linux, * no more per-task TSS's. The TSS size is kept cacheline-aligned - * so they are allowed to end up in the .data.cacheline_aligned + * so they are allowed to end up in the .kernel.data.cacheline_aligned * section. Since TSS's are completely CPU-local, we want them * on exact cacheline boundaries, to eliminate cacheline ping-pong. */ diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 98c2d05..bd6ab41 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -78,7 +78,7 @@ char ignore_fpu_irq; * for this. */ gate_desc idt_table[256] - __attribute__((__section__(".data.idt"))) = { { { { 0, 0 } } }, }; + __attribute__((__section__(".kernel.data.idt"))) = { { { { 0, 0 } } }, }; #endif DECLARE_BITMAP(used_vectors, NR_VECTORS); diff --git a/arch/x86/kernel/vmlinux_32.lds.S b/arch/x86/kernel/vmlinux_32.lds.S index 82c6755..8ad2b02 100644 --- a/arch/x86/kernel/vmlinux_32.lds.S +++ b/arch/x86/kernel/vmlinux_32.lds.S @@ -31,15 +31,15 @@ SECTIONS . = LOAD_OFFSET + LOAD_PHYSICAL_ADDR; phys_startup_32 = startup_32 - LOAD_OFFSET; - .text.head : AT(ADDR(.text.head) - LOAD_OFFSET) { + .kernel.text.head : AT(ADDR(.kernel.text.head) - LOAD_OFFSET) { _text = .; /* Text and read-only data */ - *(.text.head) + *(.kernel.text.head) } :text = 0x9090 /* read-only */ .text : AT(ADDR(.text) - LOAD_OFFSET) { . = ALIGN(PAGE_SIZE); /* not really needed, already page aligned */ - *(.text.page_aligned) + *(.kernel.text.page_aligned) TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -71,32 +71,32 @@ SECTIONS . = ALIGN(PAGE_SIZE); .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { __nosave_begin = .; - *(.data.nosave) + *(.kernel.data.nosave) . = ALIGN(PAGE_SIZE); __nosave_end = .; } . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) - *(.data.idt) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) + *(.kernel.data.idt) } . = ALIGN(32); - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } /* rarely changed data like cpu maps */ . = ALIGN(32); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) _edata = .; /* End of data section */ } . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) } /* might get freed after init */ @@ -179,11 +179,11 @@ SECTIONS } #endif . = ALIGN(PAGE_SIZE); - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { __per_cpu_start = .; - *(.data.percpu.page_aligned) - *(.data.percpu) - *(.data.percpu.shared_aligned) + *(.kernel.data.percpu.page_aligned) + *(.kernel.data.percpu) + *(.kernel.data.percpu.shared_aligned) __per_cpu_end = .; } . = ALIGN(PAGE_SIZE); @@ -192,7 +192,7 @@ SECTIONS .bss : AT(ADDR(.bss) - LOAD_OFFSET) { __init_end = .; __bss_start = .; /* BSS */ - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) . = ALIGN(4); __bss_stop = .; diff --git a/arch/x86/kernel/vmlinux_64.lds.S b/arch/x86/kernel/vmlinux_64.lds.S index 1a614c0..f3e6dc3 100644 --- a/arch/x86/kernel/vmlinux_64.lds.S +++ b/arch/x86/kernel/vmlinux_64.lds.S @@ -28,7 +28,7 @@ SECTIONS _text = .; /* Text and read-only data */ .text : AT(ADDR(.text) - LOAD_OFFSET) { /* First the code that has to be first for bootstrapping */ - *(.text.head) + *(.kernel.text.head) _stext = .; /* Then the rest */ TEXT_TEXT @@ -63,17 +63,17 @@ SECTIONS . = ALIGN(PAGE_SIZE); . = ALIGN(CONFIG_X86_L1_CACHE_BYTES); - .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { - *(.data.cacheline_aligned) + .kernel.data.cacheline_aligned : AT(ADDR(.kernel.data.cacheline_aligned) - LOAD_OFFSET) { + *(.kernel.data.cacheline_aligned) } . = ALIGN(CONFIG_X86_INTERNODE_CACHE_BYTES); - .data.read_mostly : AT(ADDR(.data.read_mostly) - LOAD_OFFSET) { - *(.data.read_mostly) + .kernel.data.read_mostly : AT(ADDR(.kernel.data.read_mostly) - LOAD_OFFSET) { + *(.kernel.data.read_mostly) } #define VSYSCALL_ADDR (-10*1024*1024) -#define VSYSCALL_PHYS_ADDR ((LOADADDR(.data.read_mostly) + SIZEOF(.data.read_mostly) + 4095) & ~(4095)) -#define VSYSCALL_VIRT_ADDR ((ADDR(.data.read_mostly) + SIZEOF(.data.read_mostly) + 4095) & ~(4095)) +#define VSYSCALL_PHYS_ADDR ((LOADADDR(.kernel.data.read_mostly) + SIZEOF(.kernel.data.read_mostly) + 4095) & ~(4095)) +#define VSYSCALL_VIRT_ADDR ((ADDR(.kernel.data.read_mostly) + SIZEOF(.kernel.data.read_mostly) + 4095) & ~(4095)) #define VLOAD_OFFSET (VSYSCALL_ADDR - VSYSCALL_PHYS_ADDR) #define VLOAD(x) (ADDR(x) - VLOAD_OFFSET) @@ -122,13 +122,13 @@ SECTIONS #undef VVIRT . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { - *(.data.init_task) + .kernel.data.init_task : AT(ADDR(.kernel.data.init_task) - LOAD_OFFSET) { + *(.kernel.data.init_task) }:data.init . = ALIGN(PAGE_SIZE); - .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { - *(.data.page_aligned) + .kernel.data.page_aligned : AT(ADDR(.kernel.data.page_aligned) - LOAD_OFFSET) { + *(.kernel.data.page_aligned) } /* might get freed after init */ @@ -215,13 +215,13 @@ SECTIONS . = ALIGN(PAGE_SIZE); __nosave_begin = .; - .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.data.nosave) } + .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.kernel.data.nosave) } . = ALIGN(PAGE_SIZE); __nosave_end = .; __bss_start = .; /* BSS */ .bss : AT(ADDR(.bss) - LOAD_OFFSET) { - *(.bss.page_aligned) + *(.bss.kernel.page_aligned) *(.bss) } __bss_stop = .; diff --git a/arch/xtensa/kernel/head.S b/arch/xtensa/kernel/head.S index 67e6913..581c735 100644 --- a/arch/xtensa/kernel/head.S +++ b/arch/xtensa/kernel/head.S @@ -234,7 +234,7 @@ should_never_return: * BSS section */ -.section ".bss.page_aligned", "w" +.section ".bss.kernel.page_aligned", "w" ENTRY(swapper_pg_dir) .fill PAGE_SIZE, 1, 0 ENTRY(empty_zero_page) diff --git a/arch/xtensa/kernel/init_task.c b/arch/xtensa/kernel/init_task.c index e07f5c9..d6836f4 100644 --- a/arch/xtensa/kernel/init_task.c +++ b/arch/xtensa/kernel/init_task.c @@ -28,7 +28,7 @@ struct mm_struct init_mm = INIT_MM(init_mm); EXPORT_SYMBOL(init_mm); union thread_union init_thread_union - __attribute__((__section__(".data.init_task"))) = + __attribute__((__section__(".kernel.data.init_task"))) = { INIT_THREAD_INFO(init_task) }; struct task_struct init_task = INIT_TASK(init_task); diff --git a/arch/xtensa/kernel/vmlinux.lds.S b/arch/xtensa/kernel/vmlinux.lds.S index d506774..01eafe7 100644 --- a/arch/xtensa/kernel/vmlinux.lds.S +++ b/arch/xtensa/kernel/vmlinux.lds.S @@ -121,14 +121,14 @@ SECTIONS DATA_DATA CONSTRUCTORS . = ALIGN(XCHAL_ICACHE_LINESIZE); - *(.data.cacheline_aligned) + *(.kernel.data.cacheline_aligned) } _edata = .; /* The initial task */ . = ALIGN(8192); - .data.init_task : { *(.data.init_task) } + .kernel.data.init_task : { *(.kernel.data.init_task) } /* Initialization code and data: */ @@ -259,7 +259,7 @@ SECTIONS /* BSS section */ _bss_start = .; - .bss : { *(.bss.page_aligned) *(.bss) } + .bss : { *(.bss.kernel.page_aligned) *(.bss) } _bss_end = .; _end = .; diff --git a/include/asm-frv/init.h b/include/asm-frv/init.h index 8b15838..16bea47 100644 --- a/include/asm-frv/init.h +++ b/include/asm-frv/init.h @@ -1,12 +1,12 @@ #ifndef _ASM_INIT_H #define _ASM_INIT_H -#define __init __attribute__ ((__section__ (".text.init"))) -#define __initdata __attribute__ ((__section__ (".data.init"))) +#define __init __attribute__ ((__section__ (".kernel.text.init"))) +#define __initdata __attribute__ ((__section__ (".kernel.data.init"))) /* For assembly routines */ -#define __INIT .section ".text.init",#alloc,#execinstr +#define __INIT .section ".kernel.text.init",#alloc,#execinstr #define __FINIT .previous -#define __INITDATA .section ".data.init",#alloc,#write +#define __INITDATA .section ".kernel.data.init",#alloc,#write #endif diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index c61fab1..52bfb6b 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -64,7 +64,7 @@ /* .data section */ #define DATA_DATA \ *(.data) \ - *(.data.init.refok) \ + *(.kernel.data.init.refok) \ *(.ref.data) \ DEV_KEEP(init.data) \ DEV_KEEP(exit.data) \ @@ -255,8 +255,8 @@ *(.text.hot) \ *(.text) \ *(.ref.text) \ - *(.text.init.refok) \ - *(.exit.text.refok) \ + *(.kernel.text.init.refok) \ + *(.kernel.text.exit.refok) \ DEV_KEEP(init.text) \ DEV_KEEP(exit.text) \ CPU_KEEP(init.text) \ @@ -433,9 +433,9 @@ #define PERCPU(align) \ . = ALIGN(align); \ VMLINUX_SYMBOL(__per_cpu_start) = .; \ - .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { \ - *(.data.percpu.page_aligned) \ - *(.data.percpu) \ - *(.data.percpu.shared_aligned) \ + .kernel.data.percpu : AT(ADDR(.kernel.data.percpu) - LOAD_OFFSET) { \ + *(.kernel.data.percpu.page_aligned) \ + *(.kernel.data.percpu) \ + *(.kernel.data.percpu.shared_aligned) \ } \ VMLINUX_SYMBOL(__per_cpu_end) = .; diff --git a/include/linux/cache.h b/include/linux/cache.h index 97e2488..12fb84d 100644 --- a/include/linux/cache.h +++ b/include/linux/cache.h @@ -31,7 +31,7 @@ #ifndef __cacheline_aligned #define __cacheline_aligned \ __attribute__((__aligned__(SMP_CACHE_BYTES), \ - __section__(".data.cacheline_aligned"))) + __section__(".kernel.data.cacheline_aligned"))) #endif /* __cacheline_aligned */ #ifndef __cacheline_aligned_in_smp diff --git a/include/linux/init.h b/include/linux/init.h index 68cb026..10d256a 100644 --- a/include/linux/init.h +++ b/include/linux/init.h @@ -62,9 +62,9 @@ /* backward compatibility note * A few places hardcode the old section names: - * .text.init.refok - * .data.init.refok - * .exit.text.refok + * .kernel.text.init.refok + * .kernel.data.init.refok + * .kernel.text.exit.refok * They should be converted to use the defines from this file */ @@ -301,7 +301,7 @@ void __init parse_early_param(void); #endif /* Data marked not to be saved by software suspend */ -#define __nosavedata __section(.data.nosave) +#define __nosavedata __section(.kernel.data.nosave) /* This means "can be init if no module support, otherwise module load may call it." */ diff --git a/include/linux/linkage.h b/include/linux/linkage.h index fee9e59..c010d2b 100644 --- a/include/linux/linkage.h +++ b/include/linux/linkage.h @@ -18,8 +18,8 @@ # define asmregparm #endif -#define __page_aligned_data __section(.data.page_aligned) __aligned(PAGE_SIZE) -#define __page_aligned_bss __section(.bss.page_aligned) __aligned(PAGE_SIZE) +#define __page_aligned_data __section(.kernel.data.page_aligned) __aligned(PAGE_SIZE) +#define __page_aligned_bss __section(.bss.kernel.page_aligned) __aligned(PAGE_SIZE) /* * This is used by architectures to keep arguments on the stack diff --git a/include/linux/percpu.h b/include/linux/percpu.h index 9f2a375..c8c9cb2 100644 --- a/include/linux/percpu.h +++ b/include/linux/percpu.h @@ -10,13 +10,13 @@ #ifdef CONFIG_SMP #define DEFINE_PER_CPU(type, name) \ - __attribute__((__section__(".data.percpu"))) \ + __attribute__((__section__(".kernel.data.percpu"))) \ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #ifdef MODULE -#define SHARED_ALIGNED_SECTION ".data.percpu" +#define SHARED_ALIGNED_SECTION ".kernel.data.percpu" #else -#define SHARED_ALIGNED_SECTION ".data.percpu.shared_aligned" +#define SHARED_ALIGNED_SECTION ".kernel.data.percpu.shared_aligned" #endif #define DEFINE_PER_CPU_SHARED_ALIGNED(type, name) \ @@ -24,8 +24,8 @@ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name \ ____cacheline_aligned_in_smp -#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ - __attribute__((__section__(".data.percpu.page_aligned"))) \ +#define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ + __attribute__((__section__(".kernel.data.percpu.page_aligned")))\ PER_CPU_ATTRIBUTES __typeof__(type) per_cpu__##name #else #define DEFINE_PER_CPU(type, name) \ diff --git a/include/linux/spinlock.h b/include/linux/spinlock.h index e0c0fcc..b041f4d 100644 --- a/include/linux/spinlock.h +++ b/include/linux/spinlock.h @@ -60,7 +60,7 @@ /* * Must define these before including other files, inline functions need them */ -#define LOCK_SECTION_NAME ".text.lock."KBUILD_BASENAME +#define LOCK_SECTION_NAME ".kernel.text.lock."KBUILD_BASENAME #define LOCK_SECTION_START(extra) \ ".subsection 1\n\t" \ diff --git a/kernel/module.c b/kernel/module.c index e8b51d4..8eb8f09 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -484,7 +484,7 @@ static unsigned int find_pcpusec(Elf_Ehdr *hdr, Elf_Shdr *sechdrs, const char *secstrings) { - return find_sec(hdr, sechdrs, secstrings, ".data.percpu"); + return find_sec(hdr, sechdrs, secstrings, ".kernel.data.percpu"); } static void percpu_modcopy(void *pcpudest, const void *from, unsigned long size) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 8892161..afe9408 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -794,9 +794,9 @@ static const char *data_sections[] = { DATA_SECTIONS, NULL }; /* sections that may refer to an init/exit section with no warning */ static const char *initref_sections[] = { - ".text.init.refok*", - ".exit.text.refok*", - ".data.init.refok*", + ".kernel.text.init.refok*", + ".kernel.text.exit.refok*", + ".kernel.data.init.refok*", NULL }; @@ -915,7 +915,7 @@ static int section_mismatch(const char *fromsec, const char *tosec) * Pattern 0: * Do not warn if funtion/data are marked with __init_refok/__initdata_refok. * The pattern is identified by: - * fromsec = .text.init.refok* | .data.init.refok* + * fromsec = .kernel.text.init.refok* | .kernel.data.init.refok* * * Pattern 1: * If a module parameter is declared __initdata and permissions=0 @@ -939,8 +939,8 @@ static int section_mismatch(const char *fromsec, const char *tosec) * *probe_one, *_console, *_timer * * Pattern 3: - * Whitelist all refereces from .text.head to .init.data - * Whitelist all refereces from .text.head to .init.text + * Whitelist all refereces from .kernel.text.head to .init.data + * Whitelist all refereces from .kernel.text.head to .init.text * * Pattern 4: * Some symbols belong to init section but still it is ok to reference diff --git a/scripts/recordmcount.pl b/scripts/recordmcount.pl index fe83141..403a4a4 100755 --- a/scripts/recordmcount.pl +++ b/scripts/recordmcount.pl @@ -26,7 +26,7 @@ # which will also be the location of that section after final link. # e.g. # -# .section ".text.sched" +# .section ".sched.text" # .globl my_func # my_func: # [...] @@ -39,7 +39,7 @@ # [...] # # Both relocation offsets for the mcounts in the above example will be -# offset from .text.sched. If we make another file called tmp.s with: +# offset from .sched.text. If we make another file called tmp.s with: # # .section __mcount_loc # .quad my_func + 0x5 @@ -51,7 +51,7 @@ # But this gets hard if my_func is not globl (a static function). # In such a case we have: # -# .section ".text.sched" +# .section ".sched.text" # my_func: # [...] # call mcount (offset: 0x5) -- 1.6.1.2 ^ permalink raw reply related [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-06 0:03 [PATCH 0/7] Ksplice: Rebootless kernel updates Jeff Arnold 2008-12-06 0:03 ` [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections Jeff Arnold @ 2008-12-17 2:48 ` Tim Abbott 2008-12-17 3:07 ` Andrew Morton 1 sibling, 1 reply; 43+ messages in thread From: Tim Abbott @ 2008-12-17 2:48 UTC (permalink / raw) To: Andrew Morton Cc: Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan On Fri, 5 Dec 2008, Jeff Arnold wrote: > We're interested in having Ksplice included in -mm. If you could consider > this code for inclusion (or let us know what you'd like us to do next), > we'd appreciate it. Hi Andrew, Since it's been a couple of weeks, I thought that I would ask whether you know yet how you'd like to handle this patch series. Thanks, -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-17 2:48 ` [PATCH 0/7] Ksplice: Rebootless kernel updates Tim Abbott @ 2008-12-17 3:07 ` Andrew Morton 2008-12-17 3:53 ` Dave Jones ` (2 more replies) 0 siblings, 3 replies; 43+ messages in thread From: Andrew Morton @ 2008-12-17 3:07 UTC (permalink / raw) To: Tim Abbott Cc: Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Ben Collins, Dave Jones On Tue, 16 Dec 2008 21:48:29 -0500 (EST) Tim Abbott <tabbott@MIT.EDU> wrote: > On Fri, 5 Dec 2008, Jeff Arnold wrote: > > > We're interested in having Ksplice included in -mm. If you could consider > > this code for inclusion (or let us know what you'd like us to do next), > > we'd appreciate it. > > Hi Andrew, > > Since it's been a couple of weeks, I thought that I would ask whether you > know yet how you'd like to handle this patch series. Thanks, > It's quite a lot of tricky code, and fairly high maintenance, I expect. I'd have _thought_ that distros and their high-end customers would be interested in it, but I haven't noticed anything from them. Not that this means much - our processes for gathering this sort of information are rudimentary at best. Has your team been in contact with distros? Are Suse shipping it, or planning to? (cc's a few more distro people) Guys, could you please wander over to the appropriate cube and administer a prod? Thanks. ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-17 3:07 ` Andrew Morton @ 2008-12-17 3:53 ` Dave Jones 2008-12-17 17:19 ` Jeff Arnold 2008-12-17 5:05 ` Tim Abbott 2008-12-17 12:06 ` Ben Collins 2 siblings, 1 reply; 43+ messages in thread From: Dave Jones @ 2008-12-17 3:53 UTC (permalink / raw) To: Andrew Morton Cc: Tim Abbott, Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Ben Collins On Tue, Dec 16, 2008 at 07:07:40PM -0800, Andrew Morton wrote: > I'd have _thought_ that distros and their high-end customers would be > interested in it, but I haven't noticed anything from them. Not that > this means much - our processes for gathering this sort of information > are rudimentary at best. Has your team been in contact with distros? > Are Suse shipping it, or planning to? > > (cc's a few more distro people) I'm not exactly enamoured by this tbh. It's a neat hack, but the idea of it being used by even a small percentage of our users gives me the creeps. It comes down to 'what happens when something goes wrong'. When the end-user is running a ksplice-patched kernel with an unknown number of patches, reproducability can get really complicated. The Fedora position on it has been 'you keep both pieces if something breaks' in the same way we don't support someone who hand-built their own kernel. I understand ksplice now taints the kernel when it applies a patch, which is a step in the right direction, but we don't always get to see tainted flags in bug reports (the non-oops variants for eg). If distros can't get security updates out in a reasonable time, fix the process instead of adding mechanism that does an end-run around it. Which just leaves the "we can't afford downtime" argument, which leads me to question how well reviewed runtime patches are. Having seen some of the non-ksplice runtime patches that appear in the wake of a new security hole, I can't say I have a lot of faith. Perhaps if there was a kernel.org sanctioned ksplice functionality, then such patches would get better review before being deployed. Dunno. Dave -- http://www.codemonkey.org.uk ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-17 3:53 ` Dave Jones @ 2008-12-17 17:19 ` Jeff Arnold 0 siblings, 0 replies; 43+ messages in thread From: Jeff Arnold @ 2008-12-17 17:19 UTC (permalink / raw) To: Dave Jones Cc: Andrew Morton, Tim Abbott, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Ben Collins On Tue, 16 Dec 2008, Dave Jones wrote: Hi Dave. Thanks for your feedback. I thought that it might be helpful for me to provide some information about how Ksplice tries to address these concerns. > It comes down to 'what happens when something goes wrong'. > When the end-user is running a ksplice-patched kernel with an unknown > number of patches, reproducability can get really complicated. Ksplice does everything that we've been able to think of to help with reproducibility and debugging. The taint flag tells you whether any Ksplice updates have been loaded. The list of loaded modules gives you an ID string for every Ksplice update that has been loaded (which, if you have a copy of those updates, tells you everything that Ksplice has done to the kernel, in the order that it has done it). Ksplice verifies the kernel's text before applying each update to make sure that the kernel text exactly matches Ksplice's expectations (Ksplice takes into account code modifications that have already happened, such as from altinstructions and previous Ksplice updates). > but we don't always get to see tainted flags in bug reports (the > non-oops variants for eg). Perhaps it is possible to make taint information appear in bug reports more frequently? It seems as though useful bug reports already need to contain quite a bit of other information. > Which just leaves the "we can't afford downtime" argument, which leads > me to question how well reviewed runtime patches are. I'm not sure whether you're worried about the Ksplice code itself or the patches that people might apply using Ksplice. As for the Ksplice code itself, I suppose that I can only say that we've done our best and we invite people to find problems in it. If you're concerned about the patches that people might apply using Ksplice: yes, I agree that writing new custom code for hot updates is scary. Ksplice has, in fact, been designed around the idea of avoiding new code for hot updates. 88% of CVEs from May 2005 to May 2008 can be corrected without writing a single line of new code [1]--Ksplice constructs the update based entirely on the existing mainline patch. (The remaining CVEs require an average of roughly 20 semicolon-terminated new lines of code per CVE). Updates should still be generated by someone competent, but we've tried to minimize the burden and risk involved. [1] http://www.ksplice.com/cve-evaluation Jeff Arnold jbarnold@mit.edu ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-17 3:07 ` Andrew Morton 2008-12-17 3:53 ` Dave Jones @ 2008-12-17 5:05 ` Tim Abbott 2008-12-17 12:09 ` Ben Collins 2008-12-17 12:06 ` Ben Collins 2 siblings, 1 reply; 43+ messages in thread From: Tim Abbott @ 2008-12-17 5:05 UTC (permalink / raw) To: Andrew Morton Cc: Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Ben Collins, Dave Jones On Tue, 16 Dec 2008, Andrew Morton wrote: > Has your team been in contact with distros? Are Suse shipping it, or > planning to? We've talked to kernel team people from Fedora, Ubuntu, and SUSE so far. Fedora 8, 9, and 10, Debian sid, and Ubuntu jaunty are currently shipping packages for the standalone version of Ksplice. We haven't yet made a big push for end-users--that's still ahead. We've submitted the code now because we think that it's important to try to involve the community sooner rather than later. -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-17 5:05 ` Tim Abbott @ 2008-12-17 12:09 ` Ben Collins 0 siblings, 0 replies; 43+ messages in thread From: Ben Collins @ 2008-12-17 12:09 UTC (permalink / raw) To: Tim Abbott Cc: Andrew Morton, Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Dave Jones On Wed, 2008-12-17 at 00:05 -0500, Tim Abbott wrote: > On Tue, 16 Dec 2008, Andrew Morton wrote: > > > Has your team been in contact with distros? Are Suse shipping it, or > > planning to? > > We've talked to kernel team people from Fedora, Ubuntu, and SUSE so far. > Fedora 8, 9, and 10, Debian sid, and Ubuntu jaunty are currently shipping > packages for the standalone version of Ksplice. Ubuntu jaunty isn't "shipping", just to be clear. > We haven't yet made a big push for end-users--that's still ahead. We've > submitted the code now because we think that it's important to try to > involve the community sooner rather than later. > > -Tim Abbott ^ permalink raw reply [flat|nested] 43+ messages in thread
* Re: [PATCH 0/7] Ksplice: Rebootless kernel updates 2008-12-17 3:07 ` Andrew Morton 2008-12-17 3:53 ` Dave Jones 2008-12-17 5:05 ` Tim Abbott @ 2008-12-17 12:06 ` Ben Collins 2 siblings, 0 replies; 43+ messages in thread From: Ben Collins @ 2008-12-17 12:06 UTC (permalink / raw) To: Andrew Morton Cc: Tim Abbott, Jeff Arnold, linux-kernel, Denys Vlasenko, Anders Kaseorg, Waseem Daher, Nikanth Karthikesan, Dave Jones On Tue, 2008-12-16 at 19:07 -0800, Andrew Morton wrote: > On Tue, 16 Dec 2008 21:48:29 -0500 (EST) Tim Abbott <tabbott@MIT.EDU> wrote: > > > On Fri, 5 Dec 2008, Jeff Arnold wrote: > > > > > We're interested in having Ksplice included in -mm. If you could consider > > > this code for inclusion (or let us know what you'd like us to do next), > > > we'd appreciate it. > > > > Hi Andrew, > > > > Since it's been a couple of weeks, I thought that I would ask whether you > > know yet how you'd like to handle this patch series. Thanks, > > > > It's quite a lot of tricky code, and fairly high maintenance, I expect. > > I'd have _thought_ that distros and their high-end customers would be > interested in it, but I haven't noticed anything from them. Not that > this means much - our processes for gathering this sort of information > are rudimentary at best. Has your team been in contact with distros? > Are Suse shipping it, or planning to? > > (cc's a few more distro people) > > Guys, could you please wander over to the appropriate cube and > administer a prod? We aren't using it and I haven't heard any requests for it. Might check it out though. ^ permalink raw reply [flat|nested] 43+ messages in thread
end of thread, other threads:[~2009-02-16 7:11 UTC | newest] Thread overview: 43+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-12-06 0:03 [PATCH 0/7] Ksplice: Rebootless kernel updates Jeff Arnold 2008-12-06 0:03 ` [PATCH 1/7] Make section names compatible with -ffunction-sections -fdata-sections Jeff Arnold 2008-12-06 0:03 ` [PATCH 2/7] x86: Add an option to compile " Jeff Arnold 2008-12-06 0:03 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Jeff Arnold 2008-12-06 0:03 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Jeff Arnold 2008-12-06 0:03 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Jeff Arnold 2008-12-06 0:03 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Jeff Arnold 2008-12-06 0:04 ` [PATCH 7/7] Ksplice: Support updating x86-32 and x86-64 Jeff Arnold 2008-12-17 5:41 ` Theodore Tso 2008-12-18 2:09 ` Tim Abbott 2009-02-07 2:36 ` Rusty Russell 2009-02-10 1:01 ` Tim Abbott 2009-02-04 11:35 ` [PATCH 6/7] Ksplice: Export symbols needed for Ksplice Rusty Russell 2009-02-13 1:46 ` Tim Abbott 2009-02-16 7:11 ` Rusty Russell 2009-02-04 11:30 ` [PATCH 5/7] Ksplice: Add functions for walking kallsyms symbols Rusty Russell 2009-02-04 21:31 ` Anders Kaseorg 2009-02-04 11:21 ` [PATCH 4/7] Ksplice: Add module_data_address (the analogue of module_text_address) Rusty Russell 2009-02-04 20:48 ` Anders Kaseorg 2009-02-07 12:45 ` Rusty Russell 2009-02-04 9:26 ` [PATCH 3/7] Ksplice: Make find_symbol return a struct kernel_symbol Rusty Russell 2009-02-04 8:38 ` [PATCH 2/7] x86: Add an option to compile with -ffunction-sections -fdata-sections Rusty Russell 2009-02-04 10:26 ` Anders Kaseorg 2009-02-04 10:58 ` Rusty Russell 2009-02-04 20:50 ` Anders Kaseorg 2008-12-06 8:46 ` [PATCH 1/7] Make section names compatible " Andi Kleen 2008-12-31 19:18 ` Tim Abbott 2008-12-31 19:52 ` Andi Kleen 2008-12-31 21:59 ` Tim Abbott 2009-01-01 16:32 ` Andrew Morton 2009-01-04 19:20 ` Tim Abbott 2009-01-12 21:51 ` Andrew Morton 2009-01-12 22:11 ` Andi Kleen 2009-02-04 8:15 ` Rusty Russell 2009-02-05 1:11 ` Anders Kaseorg 2009-02-05 2:00 ` Anders Kaseorg 2008-12-17 2:48 ` [PATCH 0/7] Ksplice: Rebootless kernel updates Tim Abbott 2008-12-17 3:07 ` Andrew Morton 2008-12-17 3:53 ` Dave Jones 2008-12-17 17:19 ` Jeff Arnold 2008-12-17 5:05 ` Tim Abbott 2008-12-17 12:09 ` Ben Collins 2008-12-17 12:06 ` Ben Collins
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox