From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Wright Subject: Re: Kernel bug from 3.0 (was phy disks and vifs timing out in DomU) Date: Wed, 07 Sep 2011 13:57:54 +0100 Message-ID: <4E676A52.8050907@overnetdata.com> References: <4E3266DE.9000606@overnetdata.com> <20110803152841.GA2860@dumpdata.com> <4E4E3957.1040007@overnetdata.com> <20110819125615.GA26558@dumpdata.com> <4E56B132.9050708@overnetdata.com> <20110826142606.GA25511@dumpdata.com> <20110826144438.GA24836@dumpdata.com> <4E5E6843.7050206@citrix.com> <20110831170711.GB13642@dumpdata.com> <1314862972.28989.74.camel@zakaz.uk.xensource.com> <20110901142356.GD23971@dumpdata.com> <4E5FA0C4.7000806@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------070203020808050508080303" Return-path: In-Reply-To: <4E5FA0C4.7000806@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: David Vrabel Cc: Ian Campbell , Todd Deshane , "xen-devel@lists.xensource.com" , Jeremy Fitzhardinge , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------070203020808050508080303 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 01/09/2011 16:12, David Vrabel wrote: > On 01/09/11 15:23, Konrad Rzeszutek Wilk wrote: >> On Thu, Sep 01, 2011 at 08:42:52AM +0100, Ian Campbell wrote: >>> On Wed, 2011-08-31 at 18:07 +0100, Konrad Rzeszutek Wilk wrote: >>>> On Wed, Aug 31, 2011 at 05:58:43PM +0100, David Vrabel wrote: >>>>> On 26/08/11 15:44, Konrad Rzeszutek Wilk wrote: >>>>>> So while I am still looking at the hypervisor code to figure out why >>>>>> it would give me [when trying to map a grant page]: >>>>>> >>>>>> (XEN) mm.c:3846:d0 Could not find L1 PTE for address fbb42000 >>>>> It is failing in guest_map_l1e() because the page for the vmalloc'd >>>>> virtual address PTEs is not present. >>>>> >>>>> The test that fails is: >>>>> >>>>> (l2e_get_flags(l2e) & (_PAGE_PRESENT | _PAGE_PSE)) != _PAGE_PRESENT >>>>> >>>>> I think this is because the GNTTABOP_map_grant_ref hypercall is done >>>>> when task->active_mm != &init_mm and alloc_vm_area() only adds PTEs into >>>>> init_mm so when Xen looks in the page tables it doesn't find the entries >>>>> because they're not there yet. >>>>> >>>>> Putting a call to vmalloc_sync_all() after create_vm_area() and before >>>>> the hypercall makes it work for me. Classic Xen kernels used to have >>>>> such a call. >>>> That sounds quite reasonable. >>> I was wondering why upstream was missing the vmalloc_sync_all() in >>> alloc_vm_area() since the out-of-tree kernels did have it and the >>> function was added by us. I found this: >>> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=ef691947d8a3d479e67652312783aedcf629320a >>> >>> commit ef691947d8a3d479e67652312783aedcf629320a >>> Author: Jeremy Fitzhardinge >>> Date: Wed Dec 1 15:45:48 2010 -0800 >>> >>> vmalloc: remove vmalloc_sync_all() from alloc_vm_area() >>> >>> There's no need for it: it will get faulted into the current pagetable >>> as needed. >>> >>> Signed-off-by: Jeremy Fitzhardinge >>> >>> The flaw in the reasoning here is that you cannot take a kernel fault >>> while processing a hypercall, so hypercall arguments must have been >>> faulted in beforehand and that is what the sync_all was for. >>> >>> It's probably fair to say that the Xen specific caller should take care >>> of that Xen-specific requirement rather than pushing it into common >>> code. On the other hand Xen is the only user and creating a Xen specific >>> helper/wrapper seems a bit pointless. >> Perhaps then doing the vmalloc_sync_all() (or are more precise one: >> vmalloc_sync_one) should be employed in the netback code then? >> >> And obviously guarded by the CONFIG_HIGHMEM case? > Perhaps. But I think the correct thing to do initially is revert the > change and then look at possible improvements. Particularly as the fix > needs to be a backported to stable. > > David > I have implement a patch which does essentially this, i.e. calls vmalloc_sync_all() afer every alloc_vm_area() call (all 5 of them). Now my VMs start correctly, but I still get error messages in the xen dmesg output (attached). Is this expected? Anthony --------------070203020808050508080303 Content-Type: text/plain; name="dmesg.log" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dmesg.log" __ __ _ _ _ _ \ \/ /___ _ __ | || | / | / | \ // _ \ '_ \ | || |_ | | | | / \ __/ | | | |__ _|| |_| | /_/\_\___|_| |_| |_|(_)_(_)_| (XEN) Xen version 4.1.1 (@[unknown]) (gcc version 4.4.3 (GCC) ) Tue Aug 30 19:32:56 GMT 2011 (XEN) Latest ChangeSet: unavailable (XEN) Bootloader: GNU GRUB 0.97 (XEN) Command line: dom0_mem=400M sched=credit (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: none; EDID transfer time: 0 seconds (XEN) EDID info not retrieved because no DDC retrieval method detected (XEN) Disc information: (XEN) Found 2 MBR signatures (XEN) Found 2 EDD information structures (XEN) Xen-e820 RAM map: (XEN) 0000000000000000 - 000000000009b400 (usable) (XEN) 000000000009b400 - 00000000000a0000 (reserved) (XEN) 00000000000e2000 - 0000000000100000 (reserved) (XEN) 0000000000100000 - 00000000aff90000 (usable) (XEN) 00000000aff90000 - 00000000aff9e000 (ACPI data) (XEN) 00000000aff9e000 - 00000000affe0000 (ACPI NVS) (XEN) 00000000affe0000 - 00000000affee000 (reserved) (XEN) 00000000afff0000 - 00000000b0000000 (reserved) (XEN) 00000000fec00000 - 00000000fec01000 (reserved) (XEN) 00000000fee00000 - 00000000fef00000 (reserved) (XEN) 00000000fff00000 - 0000000100000000 (reserved) (XEN) 0000000100000000 - 0000000140000000 (usable) (XEN) System RAM: 3839MB (3931308kB) (XEN) ACPI: RSDP 000FB5D0, 0014 (r0 ACPIAM) (XEN) ACPI: RSDT AFF90000, 0038 (r1 032210 RSDT1037 20100322 MSFT 97) (XEN) ACPI: FACP AFF90200, 0084 (r2 032210 FACP1037 20100322 MSFT 97) (XEN) ACPI: DSDT AFF90460, 7307 (r1 A1270 A1270000 0 INTL 20060113) (XEN) ACPI: FACS AFF9E000, 0040 (XEN) ACPI: APIC AFF90390, 0090 (r1 032210 APIC1037 20100322 MSFT 97) (XEN) ACPI: MCFG AFF90420, 003C (r1 032210 OEMMCFG 20100322 MSFT 97) (XEN) ACPI: OEMB AFF9E040, 0071 (r1 032210 OEMB1037 20100322 MSFT 97) (XEN) ACPI: HPET AFF97770, 0038 (r1 032210 OEMHPET0 20100322 MSFT 97) (XEN) Xen heap: 9MB (9788kB) (XEN) Domain heap initialised (XEN) Processor #0 0:6 APIC version 16 (XEN) Processor #1 0:6 APIC version 16 (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23 (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs (XEN) Table is not found! (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) Detected 3013.752 MHz processor. (XEN) AMD-Vi: IOMMU not found! (XEN) I/O virtualisation disabled (XEN) ENABLING IO-APIC IRQs (XEN) -> Using new ACK method (XEN) Platform timer is 25.000MHz HPET (XEN) Allocated console ring of 16 KiB. (XEN) CPU0: AMD SVM Extension is disabled in BIOS. (XEN) SVM: failed to initialise. (XEN) Brought up 2 CPUs (XEN) *** LOADING DOMAIN 0 *** (XEN) Xen kernel: 32-bit, PAE, lsb (XEN) Dom0 kernel: 32-bit, PAE, lsb, paddr 0x400000 -> 0x1780000 (XEN) PHYSICAL MEMORY ARRANGEMENT: (XEN) Dom0 alloc.: 000000013a000000->000000013c000000 (93225 pages to be allocated) (XEN) Init. ramdisk: 000000013fc29000->000000013ffff600 (XEN) VIRTUAL MEMORY ARRANGEMENT: (XEN) Loaded kernel: c0400000->c1780000 (XEN) Init. ramdisk: c1780000->c1b56600 (XEN) Phys-Mach map: c1b57000->c1bbb000 (XEN) Start info: c1bbb000->c1bbb47c (XEN) Page tables: c1bbc000->c1bd1000 (XEN) Boot stack: c1bd1000->c1bd2000 (XEN) TOTAL: c0000000->c2000000 (XEN) ENTRY ADDRESS: c13d9000 (XEN) Dom0 has maximum 2 VCPUs (XEN) Scrubbing Free RAM: ..................................done. (XEN) Xen trace buffers: disabled (XEN) Std. Loglevel: Errors and warnings (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings) (XEN) Xen is relinquishing VGA console. (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 188kB init memory. (XEN) mm.c:907:d0 Error getting mfn 3a09c (pfn 55555555) from L1 entry 000000003a09c023 for l1e_owner=0, pg_owner=0 (XEN) mm.c:907:d0 Error getting mfn 3a09d (pfn 55555555) from L1 entry 000000003a09d023 for l1e_owner=0, pg_owner=0 (XEN) mm.c:907:d0 Error getting mfn 3a09e (pfn 55555555) from L1 entry 000000003a09e023 for l1e_owner=0, pg_owner=0 (XEN) mm.c:907:d0 Error getting mfn 3a09f (pfn 55555555) from L1 entry 000000003a09f023 for l1e_owner=0, pg_owner=0 (XEN) traps.c:2388:d0 Domain attempted WRMSR c0010004 from 0x0000ab23d6d622da to 0x000000000000abcd. --------------070203020808050508080303 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------070203020808050508080303--