From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 858CDCD8C9D for ; Thu, 13 Nov 2025 17:43:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.1162007.1489807 (Exim 4.92) (envelope-from ) id 1vJbLb-0002OO-DE; Thu, 13 Nov 2025 17:43:03 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 1162007.1489807; Thu, 13 Nov 2025 17:43:03 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vJbLb-0002OH-AZ; Thu, 13 Nov 2025 17:43:03 +0000 Received: by outflank-mailman (input) for mailman id 1162007; Thu, 13 Nov 2025 17:43:02 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1vJbLa-0002OA-Ap for xen-devel@lists.xenproject.org; Thu, 13 Nov 2025 17:43:02 +0000 Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 314042ce-c0b8-11f0-980a-7dc792cee155; Thu, 13 Nov 2025 18:42:59 +0100 (CET) Received: from m5p.com (mailhost.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:f7]) by mailhost.m5p.com (8.18.1/8.17.1) with ESMTPS id 5ADHgoF9030139 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 13 Nov 2025 12:42:55 -0500 (EST) (envelope-from ehem@m5p.com) Received: (from ehem@localhost) by m5p.com (8.18.1/8.15.2/Submit) id 5ADHgoNp030138; Thu, 13 Nov 2025 09:42:50 -0800 (PST) (envelope-from ehem) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 314042ce-c0b8-11f0-980a-7dc792cee155 Date: Thu, 13 Nov 2025 09:42:50 -0800 From: Elliott Mitchell To: Juergen Gross Cc: xen-devel@lists.xenproject.org Subject: Re: Xen DomU Bootloader Experiences Message-ID: References: <8b3d8e28-6501-462e-b028-8f4dc44027e7@suse.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8b3d8e28-6501-462e-b028-8f4dc44027e7@suse.com> On Thu, Nov 13, 2025 at 07:46:25AM +0100, Juergen Gross wrote: > On 12.11.25 22:13, Elliott Mitchell wrote: > > A few times there have been mentions of a need to choose between boot > > methods for DomUs. There is a need to decide on ones to recommend and > > put effort into supportting. I may not have tried that many nor done > > particularly great amounts of experimentation, but I do have some > > experience with multiple User Domain bootloaders. > > > > PyGRUB > > Xen's bootloader. PyGRUB is quite functional within its limits. In > > particular it simulates the domain's environment in Domain 0. This means > > the security exposure is problematic. Another big concern is that it > > only does GRUB v1 syntax. For a long while Debian had a package for > > generating those files on a modern system, but that package was dropped. > > > > Yet PyGRUB does avoid needing to use external tools to retrieve the > > kernel. If the kernel is updated inside the domain, this does get the > > new kernel. Further being architecture-independent this works on x86, > > ARM*, RISC-V and PowerPC. > > > > As it is the only GRUB-flavor loader available on ARM*, that is the only > > place where I've used PyGRUB. > > There is one further advantage for PyGRUB: it can look into the kernel > _before_ the domU is being created, so it can tell Xen tools whether a > 32- or 64-bit domU is needed based on the selected kernel. > > This is the main reason why PyGRUB is still existing. Okay. While I can believe this is a big advantage in some environments, I suspect this is rarely used. Whereas the security profile of PyGRUB is troublesome, GRUBv1 syntax nearly overwhelms all advantages by itself. Do you think changing the default to not build/install on x86 is inappropriate? I suspect Tianocore/EDK2's 32/64-bit support might be a better choice for an environment where this matters. > > PvGRUB > > I'm sure nearly everyone knows about PvGRUB. By being a proper port of > > GRUB to run directly on Xen, it overcomes PyGRUB's disadvantages. The > > one disadvantage is needing to get patches into an external project for > > changes in Xen. > > > > Two changes to Xen urgently need propogation to PvGRUB. I'm unsure > > whether PvGRUB unmaps its mapping of vcpu_info data. The second is > > needing to work on ARM*, RISC-V and PowerPC. The latter is the one and > > only way in which PvGRUB is inferior to PyGRUB. > > > > As PvGRUB is only available for x86, that is the only place I've used > > PvGRUB. > > Naming is difficult. :-) > > You are talking about grub-pv. pv-grub is the Xen-internal variant based > on Mini-OS and legacy grub 0.97, supporting grub for PV-domUs. > > grub-pv comes basically in three flavors, all x86-only: > > - for 32-bit PV-guests > - for 64-bit PV-guests > - for PVH-guests (32- or 64-bit) > > Adding PVH support to upstream Grub for e.g. Arm should be rather easy. I figured as much, yet this has not yet been done. The other two implementations using GRUBv1 syntax is a severe weakness. I rate this as high priority since GRUBv2 syntax is rather valuable. I would try to lay groundwork for RISC-V and PowerPC too. > > EDK2/Tianocore > > Quite well-known for being the basis of most x86 firmwares, plus being > > part of a typical Qemu setup. Not nearly as well known for being a Xen > > DomU bootloader. > > > > When it was working you would build their ArmVirtXen.dsc file and get > > XEN_EFI.fd as output. You would then use XEN_EFI.fd for the domain's > > kernel. If you looked at the console you saw something which looked and > > acted pretty similar to a UEFI firmware on x86 machines. This was > > extremely functional for OSes which didn't particularly like GRUB. > > Notably I've read of it being able to load a Redmond OS and it was quite > > functional for booting an ARM64 port of FreeBSD. > > > > Sometime after November 16th, 2022 or commit fff6d81270. The built > > images stopped functioning. This is actually rather concerning since it > > may also effects firmwares built for x86 HVM domains. I don't presently > > know whether there are multiple bugs, or a single one effecting all Xen > > builds. > > > > There is also an urgent need to get EDK2/Tianocore updated to match > > Xen/ARM's disallowing mapping the shared information page multiple times. > > As I did not wish to become deeply involved with EDK2/Tianocore I sent a > > patch to xen-devel close to 1.5 years ago. Lack of action suggests there > > is an urgent need for a liason. > > > > > > > > Recommendations: > > PyGRUB is functional within its limits. Problems are GRUBv1 syntax and > > running within Domain 0. Given this I feel the Xen Project should be > > heading towards deprecating PyGRUB. Since PvGRUB works for x86 now, I > > would default to neither building nor installing PyGRUB on x86. For > > other architectures PyGRUB is still useful. > > > > The Xen Project should formally ask the GRUB Project to port PvGRUB to > > ARM, RISC-V and PowerPC. The need for PvGRUB on ARM seems rather urgent. > > Without a proper bootloader VMs aren't too useful. > > Well, I did the grub-pvh implementation. > > Doing that for other architectures shouldn't be rocket science. :-) Okay. I've seen some it written about a number of times and no action. My reading had been the people most knowledgeable about adapting the tools were unsure of how to direct effort. Proper GRUBv2 is high-value for Linux. Tianocore/EDK2 is valuable for everyone, high value for non- Linux. Worst case Tianocore/EDK2 can boot GRUB-UEFI, I doubt GRUB can boot Tianocore/EDK2. Is there any scenario not adaquately addressed by trying for that pair? -- (\___(\___(\______ --=> 8-) EHM <=-- ______/)___/)___/) \BS ( | ehem+sigmsg@m5p.com PGP 87145445 | ) / \_CS\ | _____ -O #include O- _____ | / _/ 8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445