From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gordan Bobic Subject: Re: HVM support for e820_host (Was: Bug: Limitation of <=2GB RAM in domU persists with 4.3.0) Date: Tue, 03 Sep 2013 21:49:40 +0100 Message-ID: <52264B64.1080302@bobich.net> References: <8426aecf79e7f55c21bbe259014591a2@mail.shatteredsilicon.net> <20130724163102.GA6308@phenom.dumpdata.com> <51F051F1.5050806@bobich.net> <51F19D11.1090200@bobich.net> <51F1A54D.6070906@bobich.net> <1374798084.10269.2.camel@hastur.hellion.org.uk> <20130729180431.GQ5848@phenom.dumpdata.com> <20130903145934.GC1487@konrad-lan.dumpdata.com> <52263CBD.1090402@bobich.net> <52264826.3010402@bobich.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <52264826.3010402@bobich.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org I spoke too soon - even with e820_host=0, the same error occurs. What did I break? The code in question is this: if (libxl_defbool_val(d_config->b_info.e820_host)) { ret = libxl__e820_alloc(gc, domid, d_config); if (ret) { LIBXL__LOG_ERRNO(gc->owner, LIBXL__LOG_ERROR, "Failed while collecting E820 with: %d (errno:%d)\n", ret, errno); } } With e820_host=0, that outer black should evaluate to false, should it not? In libxl_create.c, if I am understanding the code correctly, e820_host is defaulted to false, too. What am I missing? Gordan On 09/03/2013 09:35 PM, Gordan Bobic wrote: > First attempt at a test run predictably failed. I added e820_host=1 to a > VM config and tried starting it: > > [root@normandy ~]# xl create /etc/xen/edi > Parsing config from /etc/xen/edi > libxl: error: libxl_x86.c:307:libxl__arch_domain_create: Failed while > collecting E820 with: -3 (errno:-1) > > libxl: error: libxl_create.c:901:domcreate_rebuild_done: cannot > (re-)build domain: -3 > libxl: error: libxl_dm.c:1300:libxl__destroy_device_model: could not > find device-model's pid for dom 1 > libxl: error: libxl.c:1415:libxl__destroy_domid: > libxl__destroy_device_model failed for 1 > > xl-edi.log, qemu-dm-edi.log attached. > Both actually look identical to previous logs before the patch. > > Is this something that is clearly a consequence of the patch being > incomplete? Or did I break something? > > Gordan > > On 09/03/2013 08:47 PM, Gordan Bobic wrote: >> On 09/03/2013 03:59 PM, Konrad Rzeszutek Wilk wrote: >> >>>>>> 2) Further, I'm finding myself motivated to write that >>>>>> auto-set (as opposed to hard coded) vBAR=pBAR patch discussed >>>>>> briefly a week or so ago (have an init script read the BAR >>>>>> info from dom0 and put it in xenstore, plus a patch to >>>>>> make pBAR=vBAR reservations built dynamically rather than >>>>>> statically, based on this data. Now, I'm quite fluent in C, >>>>>> but my familiarity with Xen soruce code is nearly non-existant >>>>>> (limited to studying an old unsupported patch every now and then >>>>>> in order to make it apply to a more recent code release). >>>>>> Can anyone help me out with a high level view WRT where >>>>>> this would be best plumbed in (which files and the flow of >>>>>> control between the affected files)? >>>>> >>>>> hvmloader probably and the libxl e820 code. What from a >>>>> high view needs to happen is that: >>>>> 1). Need to relax the check in libxl for e820_hole >>>>> to also do it for HVM guests. Said code just iterates over the >>>>> host E820 and sanitizes it a bit and makes a E820 hypercall to >>>>> set it for the guest. >> [snip] >> >> OK, I have attached a preliminary patch against 4.3.0 for the libxl >> part. It compiles. I haven't tried running it to see if it actually >> works or does something, but my packages build. >> >> Please let me know if I've missed anything. On it's own, I don't think >> this patch will do much (apart from maybe break HVM hosts with >> e820_host=1 set). >> >>>>> 2). Figure out whether the E820 hypercall (which sets the E820 >>>>> layout for a guest) can be run on HVM guests. I think it >>>>> could not and Mukesh in his PVH patches posted a patch >>>>> to enable that - "..Move e820 fields out of pv_domain struct" >> >> Is this already in 4.3.0 or is this an out-of-tree patch? Do you have a >> link to it handy? >> >>>>> 2). Hvmloader should do an E820 get machine memory hypercall >>>>> to see if there is anything there. If there is - that means >>>>> the toolstack has request a "new" type of E820. Iterate >>>>> over the E820 and make it look like that. >>>>> You can look in the Linux arch/x86/xen/setup.c to see how >>>>> it does that. >>>>> >>>>> The complication there is that hvmloader needs to to fit the >>>>> ACPI code (the guest type one) and such. >>>>> Presumarily you can just re-use the existing spaces that >>>>> the host has marked as E820_RESERVED or E820_ACPI.. >>>> >>>> Yup, I get it. Not only that, but it should also ideally (not >>>> strictly necessary, but it'd be handy) map the IOMEM for devices >>>> it is passed so that pBAR=vBAR (as opposed to just leaving all >>>> the host e820 reserved areas well alone - which would work for >>>> most things). >>> >>> Yes. That is an extra complication that could be done in subsequent >>> patches. But in theory if you have the E820 mirrored from the host the >>> pBAR=vBAR should be easy enough as the values from the host BARs can >>> easily fit in the E820 gaps. >> >> Agreed. Let's leave the pBAR=vBAR part for a separate patch set. I'll >> have to figure out a sensible way to query the IOMEM regions for each of >> the devices passed to the VM and make sure they are in the same hole. >> >>>>> Then there is the SMBIOS would need to move and the BIOS >>>>> might need to be relocated - but I think those are relocatable >>>>> in some form. >> >> [bit above left for later reference] >> >>>>> Well, I am more than happy to help you with this. >>>> >>>> Thanks, much appreciated. :) >>> >>> Yeeey! Vict^H^H^H^volunteer :-)! >>> >>> I am also reachable on IRC (FreeNode mostly) as either darnok or konrad >>> if that would be more convient to discuss this. >> >> Thanks. I'll keep that in mind. :) >> >> Gordan >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >> >