From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: Relocatable Xen early boot code Date: Fri, 12 Jun 2015 14:19:01 +0100 Message-ID: <557ADC45.3040604@citrix.com> References: <20150612111409.GJ19315@olila.local.net-space.pl> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Z3OrU-0003Aj-1N for xen-devel@lists.xenproject.org; Fri, 12 Jun 2015 13:19:08 +0000 In-Reply-To: <20150612111409.GJ19315@olila.local.net-space.pl> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Daniel Kiper , jbeulich@suse.com, keir@xen.org, konrad.wilk@oracle.com Cc: xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 12/06/15 12:14, Daniel Kiper wrote: > Hey, > > During work on multiboot2 protocol support for Xen on EFI platform > I discovered that we need relocatable Xen early boot code (which is > mostly 32-bit code). More you can find here: > > http://lists.xen.org/archives/html/xen-devel/2015-02/msg01257.html > > I would like to focus on solution #1 described above. I have working PoC > which implements it. We also discussed various solutions for this issue > during Xen Hackhaton in Shanghai. As Andrew and Jan asked I tried to > implement solution based on segment registers. However, after revisiting > this issue and further investigation I still have some doubts. You can > read about my findings below. > > Here I do not want to discuss GRUB2 and multiboot2 protocol support details > for relocatable images. It is not needed. It is sufficient to know that it > is able to put loaded image anywhere in available memory below 4 GiB. Loaded > image is informed about its base address according to multiboot2 protocol via > special tag. This is new feature not available in upstream GRUB2. I work on > upstreaming it in parallel. Relevant patches will be posted together with > Xen patches. > > 1) My PoC uses %ebp (last unused register available globally in Xen early > boot code; there is assumption that %ds == %es == %ss) as a storage for > Xen image base address. If boot protocol do not support relocatable images > it is filled with static value calculated during build. Otherwise it is > taken from special multiboot2 tag. > > We need Xen image base address to access memory in Xen image area in at > least three different cases: > a) direct memory access, e.g.: mov %eax,sym_offset(boot_tsc_stamp)(%ebp) > b) memory address calculation, e.g.: lea sym_offset(__page_tables_start)(%ebp),%edx, > c) update static addresses calculated during build, e.g. prebuild page tables. > > If we have Xen image base address in a register then all mentioned operations > are quite simple. > > This idea is based on x86_64 addressing mode which uses %rip as a reference. > > 2) Andrew and Jan suggested that we can use segment descriptor to store Xen > image base address. This way all references to variables in Xen image can > be easily calculated during build and they will be static. However, relevant > segment descriptor must be updated during Xen start. > > At first sight, it looks that %cs, %ds, %es, %ss should be initialized as is > (start: 0, size: 4 GiB, ...). This way we will not break references to trampoline > and all other variables living outside of Xen image. Additionally, we are sure > that all references to variables from C code (xen/arch/x86/boot/reloc.c) are > pointing always to the same place regardless of x86 instruction generated by > compiler and segment register used by this instruction for addressing. So, it > looks that potentially we can use %fs or %gs as segment register to access > variables living in Xen image. Looks good... Case a and b from solution #1 seams > easy to resolve by prefixing by chosen segment register. Case c requires an extra > register to store temporarily Xen image base address. Looks quite easy... However, > looking at xen/arch/x86/boot/cmdline.S (at first sight this is the worst thing) > I am not so happy with segment register solution. Unfortunately all functions > get as argument just displacement in segment. This means that we must rework > all stuff there and pass segment(s) as additional argument(s) because most of > code checks memory in and out of Xen image in parallel. Of course we can use > solution similar to case c described above but I think that then whole stuff > move closer to idea #1. This way we will have something which uses things > from #1 and #2. I do not think that this is solution which we want. > > 3) There is a third solution which is a mixture of #1 and #2. We can use e.g. > %fs:0 (e.g. located somewhere in Xen image) to store Xen image base address. > If this value is needed then we can access it directly, e.g. add %fs:0,%esi, > or copy to temporary register and use as required. > > So, I am still in favor of #1. It is clean and easy. It does not require a lot of work. > #2 has a potential but requires a lot of changes in fragile cmdline.S (maybe others > difficult places). Is it worth? I think that #3 is a backup solution in case we choose > #1 and later it will appear that we need a globally unused register. I really don't understand why we have command line parsing written asm. I had half a mind to rewrite it all in C, (similar to the existing reloc code, but have gcc just create the .s and #include it as before, so external references work in a rational way). It would be far clearer like that. As stated at the hackathon, the problem with using %ebp is that it turns all implicit %ds references into implicit %ss references, and tends to add a SIB+imm32 to each instruction with a memory reference (ebp relative memory references, and r13 for that matter, have restrictions in the way in which they can be encoded). ~Andrew