From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: [RFC] KEXEC: allocate crash note buffers at boot time Date: Tue, 29 Nov 2011 18:56:57 +0000 Message-ID: <4ED52AF9.2080200@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------010304000302060203010307" Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser , Jan Beulich Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org --------------010304000302060203010307 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit Hello, As I have little to no knowledge of this stage of the boot process, is this a sensible way to be setting up the per_cpu areas? I have a sneaking suspicion that it will fall over if a CPU is onlined after boot, and may also fall over if a CPU is offlined and reonlined later. There appears to be no infrastructure currently in place for this type of initialization, which is quite possibly why the code exists in its current form. Thanks, -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com --------------010304000302060203010307 Content-Type: text/x-patch; name="KEXEC-setup-crash-notes-during-boot.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="KEXEC-setup-crash-notes-during-boot.patch" # HG changeset patch # Parent b5ceec1ccccad6e79053a80820132303ff1e136f KEXEC: Allocate crash notes on boot Currently, the buffers for crash notes are allocated per CPU when a KEXEC_CMD_kexec_get_range hypercall is made, referencing the CPU in question. Although it certainly works in general, there are a few edge case problems: 1) There appears to be no guarentee that dom0 will make this hypercall for each pcpu on the system. There appears to be variable behaviour depending on how many cpus dom0 is compiled for, and how many vcpus Xen gives to dom0. 2) The allocation of these buffers occur at the whim of dom0. While this is typically very early on dom0 boot, but not guarenteed. 3) It is possible (although not sensible) for a crash kernel to be loaded without these (or some of these) hypercalls being made. Under these circumstances, a crash would cause the crash note code path will suffer a slew of null pointer deferences. 4) If the hypercalls are made after late in the day, it is possible for the hypercall to return -ENOMEM. As code tends to be more fragile once memory is enhausted, the likelyhood of us needing the crash kernel is greater. In addition, my forthcoming code to support 32bit kdump kernels on 64bit Xen on large (>64GB) boxes will require some guarentees as to where the crash note buffers are actually allocated in physical memory. This is far easier to sort out at boot time, rather than after dom0 has been booted and potentially using the physical memory required. Therefore, allocate the crash note buffers at boot time. Signed-off-by: Andrew Cooper diff -r 0a0c02a61676 xen/common/kexec.c --- a/xen/common/kexec.c +++ b/xen/common/kexec.c @@ -154,6 +154,50 @@ void __init set_kexec_crash_area_size(u6 } } +/* Allocate Xen Crash Note buffers. */ +static __init int kexec_notes_init(void) +{ + int c; + Elf_Note * note; + + /* All CPUs present a PRSTATUS and crash_xen_core note. */ + int nr_bytes = + sizeof_note("CORE", sizeof(ELF_Prstatus)) + + sizeof_note("Xen", sizeof(crash_xen_core_t)); + + /* CPU0 also presents the crash_xen_into note. */ + int cpu0_nr_bytes = nr_bytes + + sizeof_note("Xen", sizeof(crash_xen_info_t)); + + for ( c = 0; c < num_online_cpus(); ++c ) + { + note = xmalloc_bytes( c ? nr_bytes : cpu0_nr_bytes ); + if ( ! note ) + return -ENOMEM; + + per_cpu(crash_notes, c) = note; + + /* Setup CORE note. */ + setup_note(note, "CORE", NT_PRSTATUS, sizeof(ELF_Prstatus)); + note = ELFNOTE_NEXT(note); + + /* Setup Xen CORE note. */ + setup_note(note, "Xen", XEN_ELFNOTE_CRASH_REGS, + sizeof(crash_xen_core_t)); + + if ( 0 == c ) + { + /* Set up Xen Crash Info note. */ + xen_crash_note = note = ELFNOTE_NEXT(note); + setup_note(note, "Xen", XEN_ELFNOTE_CRASH_INFO, + sizeof(crash_xen_info_t)); + } + } + + return 0; +} +__initcall(kexec_notes_init); + static void one_cpu_only(void) { /* Only allow the first cpu to continue - force other cpus to spin */ @@ -306,30 +350,6 @@ static int kexec_get_cpu(xen_kexec_range if ( nr == 0 ) nr_bytes += sizeof_note("Xen", sizeof(crash_xen_info_t)); - if ( per_cpu(crash_notes, nr) == NULL ) - { - Elf_Note *note; - - note = per_cpu(crash_notes, nr) = xmalloc_bytes(nr_bytes); - - if ( note == NULL ) - return -ENOMEM; - - /* Setup CORE note. */ - setup_note(note, "CORE", NT_PRSTATUS, sizeof(ELF_Prstatus)); - - /* Setup Xen CORE note. */ - note = ELFNOTE_NEXT(note); - setup_note(note, "Xen", XEN_ELFNOTE_CRASH_REGS, sizeof(crash_xen_core_t)); - - if (nr == 0) - { - /* Setup system wide Xen info note. */ - xen_crash_note = note = ELFNOTE_NEXT(note); - setup_note(note, "Xen", XEN_ELFNOTE_CRASH_INFO, sizeof(crash_xen_info_t)); - } - } - range->start = __pa((unsigned long)per_cpu(crash_notes, nr)); range->size = nr_bytes; return 0; --------------010304000302060203010307 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------010304000302060203010307--