From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42CA3C4338F for ; Tue, 24 Aug 2021 21:15:15 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id AEB6761357 for ; Tue, 24 Aug 2021 21:15:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AEB6761357 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 2572C4B25D; Tue, 24 Aug 2021 17:15:14 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ls3KE6+5y66D; Tue, 24 Aug 2021 17:15:13 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 114F24B168; Tue, 24 Aug 2021 17:15:13 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 468404B168 for ; Tue, 24 Aug 2021 17:15:11 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kfoZKfmL4Qy2 for ; Tue, 24 Aug 2021 17:15:10 -0400 (EDT) Received: from mail-il1-f170.google.com (mail-il1-f170.google.com [209.85.166.170]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 3BC584B13A for ; Tue, 24 Aug 2021 17:15:10 -0400 (EDT) Received: by mail-il1-f170.google.com with SMTP id v2so21881699ilg.12 for ; Tue, 24 Aug 2021 14:15:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:mime-version:content-disposition; bh=ahx89PC8cIKRx7Tmgj3ezIoGMxGJS6rYJPfdg8hea3I=; b=Blewoeg8VpEAOSLu2fSVu1OYBII8wSOT80G8mkJfn17EhHJTy8t0VNwi1CMA35qzJH PRNMksekGBMvfEDa3qJe36pT4LdGCE1FTxqod0mR6XTiXTCpxXoj2DP+T3e1Oru5MOM+ SeTcPAMWPYLnbKsWsLQAoBsq+L6rzQVil6UYcvCVGis77NysmIW+IIDOCT9829zPy+ZM HNSI4par/xzky/hnonF2H6lYrlRgUfMLLTvc5agLCJSH3L8ooyfq+b7JZRgqLbrSBTlT VR1NetWEkKhWRrdPCcXEAo3AFp8CsjP+AWP/HzqUmOPxnNQojfwhKj62gyWzJC0/tDlZ EkfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=ahx89PC8cIKRx7Tmgj3ezIoGMxGJS6rYJPfdg8hea3I=; b=g6tpEWK4sszTWn6sJvjrgqLF0y5laUTV4FlX8NL6e5R69CmxWOMtcCsa43AuKfaqXj jijemxjWL4LOcaWW4loyQlLWU4ITmcmxXbXXZSncOi3MRSiicpAg08jDn44Q02vWkEUO G6uhRKabfpUNiln5ssTkPzYFGSGecK9irgZ2HvGefrvkveFMHsYZMdD/CZrwt9Ysrvpn /xGX6RIFCAL2PDxBRZ7j7zi0osiis9Q7IvdUn+0MRxpq31nKRe2v1N7takw0pykJbcvb LZEYJK2omTQupLsOsKHZDB0X6B7t5R5ea8aIXnnpbGFyXyOldor9oLi3GkNi04eOG6M8 Fupg== X-Gm-Message-State: AOAM530FQj0vEo+L5Zh5pCwhj8gTIbEG8sUO0eq1xqaVD+v2diqTqcOw gMyRivzu3sVUWw+1E0bD3SldXQtIjl9TOA== X-Google-Smtp-Source: ABdhPJzYkTLe3JL/GRn7dx88lWcWUrTlT/x7EdbbtUTbRMUsj0b7GHuF0etxTSoOuXH25YUqoAlx3g== X-Received: by 2002:a92:d282:: with SMTP id p2mr24901934ilp.259.1629839709100; Tue, 24 Aug 2021 14:15:09 -0700 (PDT) Received: from google.com (194.225.68.34.bc.googleusercontent.com. [34.68.225.194]) by smtp.gmail.com with ESMTPSA id z7sm10585059ilz.25.2021.08.24.14.15.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Aug 2021 14:15:08 -0700 (PDT) Date: Tue, 24 Aug 2021 21:15:03 +0000 From: Oliver Upton To: kvmarm@lists.cs.columbia.edu Subject: KVM/arm64: Guest ABI changes do not appear rollback-safe Message-ID: MIME-Version: 1.0 Content-Disposition: inline Cc: kvm@vger.kernel.org, maz@kernel.org, pshier@google.com, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hey folks, Ricardo and I were discussing hypercall support in KVM/arm64 and something seems to be a bit problematic. I do not see anywhere that KVM requires opt-in from the VMM to expose new hypercalls to the guest. To name a few, the TRNG and KVM PTP hypercalls are unconditionally provided to the guest. Exposing new hypercalls to guests in this manner seems very unsafe to me. Suppose an operator is trying to upgrade from kernel N to kernel N+1, which brings in the new 'widget' hypercall. Guests are live migrated onto the N+1 kernel, but the operator finds a defect that warrants a kernel rollback. VMs are then migrated from kernel N+1 -> N. Any guests that discovered the 'widget' hypercall are likely going to get fussy _very_ quickly on the old kernel. Really, we need to ensure that the exposed guest ABI is backwards-compatible. Running a VMM that is blissfully unaware of the 'widget' hypercall should not implicitly expose it to its guest on a new kernel. This conversation was in the context of devising a new UAPI that allows VMMs to trap hypercalls to userspace. While such an interface would easily work around the issue, the onus of ABI compatibility lies with the kernel. So, this is all a long-winded way of asking: how do we dig ourselves out of this situation, and how to we avoid it happening again in the future? I believe the answer to both is to have new VM capabilities for sets of hypercalls exposed to the guest. Unfortunately, the unconditional exposure of TRNG and PTP hypercalls is ABI now, so we'd have to provide an opt-out at this point. For anything new, require opt-in from the VMM before we give it to the guest. Have I missed something blatantly obvious, or do others see this as an issue as well? I'll reply with an example of adding opt-out for PTP. I'm sure other hypercalls could be handled similarly. -- Thanks, Oliver _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71BA5C4338F for ; Tue, 24 Aug 2021 21:17:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2923D6135F for ; Tue, 24 Aug 2021 21:17:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2923D6135F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Subject:Cc:To: From:Date:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=EbgbiZgYRAzhvRSPRqjk2SUch2U80Pqo7WrsWEh3l7g=; b=GT+xeLJ0lIQZ/Y Ybmspu/YMle2iEII2QopRQglTZmRF7uIk7FV5pnq/VuZbY19mFZdws6GtsMzB9F/Jneewo3bD/Y2n xDwQVpnsU0CS8/LA4n9zxlSciQW6FCXqV3OEa5ZQG5SP45ar5oMDrP/4N49WkOg36jxv6eop68Dbk 99/7/zl9yjlkIj0IxlhcVEKfUhZbrcmYMHFB9Cvf+YcrIdECQL/bBQyEZMG4XJzaBvEzxzUIDzLCn Cu5oPGtPKQmsNSsIQWQtvOEtAip0FCcEYNMIb6Z5C/WiDM8e0J3XPfs3/ZZof6Oz4nkvYPQthSTob hk05KbJf/BjMvx3ku8Ew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIdlN-004ieu-6E; Tue, 24 Aug 2021 21:15:17 +0000 Received: from mail-il1-x130.google.com ([2607:f8b0:4864:20::130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIdlJ-004idt-I4 for linux-arm-kernel@lists.infradead.org; Tue, 24 Aug 2021 21:15:15 +0000 Received: by mail-il1-x130.google.com with SMTP id z2so21947738iln.0 for ; Tue, 24 Aug 2021 14:15:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:mime-version:content-disposition; bh=ahx89PC8cIKRx7Tmgj3ezIoGMxGJS6rYJPfdg8hea3I=; b=Blewoeg8VpEAOSLu2fSVu1OYBII8wSOT80G8mkJfn17EhHJTy8t0VNwi1CMA35qzJH PRNMksekGBMvfEDa3qJe36pT4LdGCE1FTxqod0mR6XTiXTCpxXoj2DP+T3e1Oru5MOM+ SeTcPAMWPYLnbKsWsLQAoBsq+L6rzQVil6UYcvCVGis77NysmIW+IIDOCT9829zPy+ZM HNSI4par/xzky/hnonF2H6lYrlRgUfMLLTvc5agLCJSH3L8ooyfq+b7JZRgqLbrSBTlT VR1NetWEkKhWRrdPCcXEAo3AFp8CsjP+AWP/HzqUmOPxnNQojfwhKj62gyWzJC0/tDlZ EkfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=ahx89PC8cIKRx7Tmgj3ezIoGMxGJS6rYJPfdg8hea3I=; b=cyViPhYgwBzpR9MF7LlD2AFusNiCnK2F1/CyEHLbv1wYZrltdVRhj5Kti/f+FJGbss SoBPn0YzsyXUhfnBlLaET60g4agrAv8JwRPvi3KJHB5Z05+do7FPpNb4FhoH+74lsfe6 INGfEXWTklmyCynNIWAysuGAVjo0eJqIQUYIh3vtOVD8MYZLpsNwttcUhmXKrjRHqwuu M54GQvOtCcRlixuhAFbG+iZOhSYYxM3hkev0ttWXX3mdxbnfqa5RyA36OhbIKROmJtZr g1GfQOb6tdzJdbFfZudOGEBAlc8QFQZLB/s4asQp9gDiUFDWdZRjoNhYeEuUreHf63Xk /3aw== X-Gm-Message-State: AOAM532Z4A9/Pvt19AU1MA7EZtgHUHieqoqMisU3Ff2P7kmjQghSqbOa 7XmRELjiWy+4hLliuyTuEdigng== X-Google-Smtp-Source: ABdhPJzYkTLe3JL/GRn7dx88lWcWUrTlT/x7EdbbtUTbRMUsj0b7GHuF0etxTSoOuXH25YUqoAlx3g== X-Received: by 2002:a92:d282:: with SMTP id p2mr24901934ilp.259.1629839709100; Tue, 24 Aug 2021 14:15:09 -0700 (PDT) Received: from google.com (194.225.68.34.bc.googleusercontent.com. [34.68.225.194]) by smtp.gmail.com with ESMTPSA id z7sm10585059ilz.25.2021.08.24.14.15.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Aug 2021 14:15:08 -0700 (PDT) Date: Tue, 24 Aug 2021 21:15:03 +0000 From: Oliver Upton To: kvmarm@lists.cs.columbia.edu Cc: maz@kernel.org, pshier@google.com, ricarkol@google.com, rananta@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com Subject: KVM/arm64: Guest ABI changes do not appear rollback-safe Message-ID: MIME-Version: 1.0 Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210824_141513_643715_EFD93C3B X-CRM114-Status: GOOD ( 15.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hey folks, Ricardo and I were discussing hypercall support in KVM/arm64 and something seems to be a bit problematic. I do not see anywhere that KVM requires opt-in from the VMM to expose new hypercalls to the guest. To name a few, the TRNG and KVM PTP hypercalls are unconditionally provided to the guest. Exposing new hypercalls to guests in this manner seems very unsafe to me. Suppose an operator is trying to upgrade from kernel N to kernel N+1, which brings in the new 'widget' hypercall. Guests are live migrated onto the N+1 kernel, but the operator finds a defect that warrants a kernel rollback. VMs are then migrated from kernel N+1 -> N. Any guests that discovered the 'widget' hypercall are likely going to get fussy _very_ quickly on the old kernel. Really, we need to ensure that the exposed guest ABI is backwards-compatible. Running a VMM that is blissfully unaware of the 'widget' hypercall should not implicitly expose it to its guest on a new kernel. This conversation was in the context of devising a new UAPI that allows VMMs to trap hypercalls to userspace. While such an interface would easily work around the issue, the onus of ABI compatibility lies with the kernel. So, this is all a long-winded way of asking: how do we dig ourselves out of this situation, and how to we avoid it happening again in the future? I believe the answer to both is to have new VM capabilities for sets of hypercalls exposed to the guest. Unfortunately, the unconditional exposure of TRNG and PTP hypercalls is ABI now, so we'd have to provide an opt-out at this point. For anything new, require opt-in from the VMM before we give it to the guest. Have I missed something blatantly obvious, or do others see this as an issue as well? I'll reply with an example of adding opt-out for PTP. I'm sure other hypercalls could be handled similarly. -- Thanks, Oliver _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79F74C4338F for ; Tue, 24 Aug 2021 21:15:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48B4961357 for ; Tue, 24 Aug 2021 21:15:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234723AbhHXVPz (ORCPT ); Tue, 24 Aug 2021 17:15:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233774AbhHXVPy (ORCPT ); Tue, 24 Aug 2021 17:15:54 -0400 Received: from mail-il1-x132.google.com (mail-il1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12917C061757 for ; Tue, 24 Aug 2021 14:15:10 -0700 (PDT) Received: by mail-il1-x132.google.com with SMTP id l10so8163391ilh.8 for ; Tue, 24 Aug 2021 14:15:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:mime-version:content-disposition; bh=ahx89PC8cIKRx7Tmgj3ezIoGMxGJS6rYJPfdg8hea3I=; b=Blewoeg8VpEAOSLu2fSVu1OYBII8wSOT80G8mkJfn17EhHJTy8t0VNwi1CMA35qzJH PRNMksekGBMvfEDa3qJe36pT4LdGCE1FTxqod0mR6XTiXTCpxXoj2DP+T3e1Oru5MOM+ SeTcPAMWPYLnbKsWsLQAoBsq+L6rzQVil6UYcvCVGis77NysmIW+IIDOCT9829zPy+ZM HNSI4par/xzky/hnonF2H6lYrlRgUfMLLTvc5agLCJSH3L8ooyfq+b7JZRgqLbrSBTlT VR1NetWEkKhWRrdPCcXEAo3AFp8CsjP+AWP/HzqUmOPxnNQojfwhKj62gyWzJC0/tDlZ EkfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=ahx89PC8cIKRx7Tmgj3ezIoGMxGJS6rYJPfdg8hea3I=; b=BNzKt8TMjHR5oPU8ePDtdMy0hB7jmQ4hW+UlttENOtFPvdK65mALr5sBSUjlg1pdUs M1MPRxTbGc660qJ2PvFQG0sBh6U0mIACLdmcgI9XvT8x2YU6f2/RBsLO4WVWd/gunYmw LW7RCLTzZstErvcrVSQp7lAbxOPEEQjsqaN3naY2EbfPqDWdABFu2NcPKm8vdVUtH023 oINeb+kTsfOT9MJDQYWAW1N/igk7IU0ofjzsnPtDnQQSnixzEJ5vG+B2JhQyfYYK4HIk ku1yNvQEIsr48iFRh+ByyWO/j4T/t8elOsOQ7mdQ6P3+E6adeiT8Vzx7vovYhafJSiIM 0zeg== X-Gm-Message-State: AOAM533p42rWOHy9DLjgUCs5uLxUkBjsV8RVV/tb8M40SwrmTsHrKzUN iGN7DO8mXktgWi0IR4arnnzwHw== X-Google-Smtp-Source: ABdhPJzYkTLe3JL/GRn7dx88lWcWUrTlT/x7EdbbtUTbRMUsj0b7GHuF0etxTSoOuXH25YUqoAlx3g== X-Received: by 2002:a92:d282:: with SMTP id p2mr24901934ilp.259.1629839709100; Tue, 24 Aug 2021 14:15:09 -0700 (PDT) Received: from google.com (194.225.68.34.bc.googleusercontent.com. [34.68.225.194]) by smtp.gmail.com with ESMTPSA id z7sm10585059ilz.25.2021.08.24.14.15.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Aug 2021 14:15:08 -0700 (PDT) Date: Tue, 24 Aug 2021 21:15:03 +0000 From: Oliver Upton To: kvmarm@lists.cs.columbia.edu Cc: maz@kernel.org, pshier@google.com, ricarkol@google.com, rananta@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com Subject: KVM/arm64: Guest ABI changes do not appear rollback-safe Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hey folks, Ricardo and I were discussing hypercall support in KVM/arm64 and something seems to be a bit problematic. I do not see anywhere that KVM requires opt-in from the VMM to expose new hypercalls to the guest. To name a few, the TRNG and KVM PTP hypercalls are unconditionally provided to the guest. Exposing new hypercalls to guests in this manner seems very unsafe to me. Suppose an operator is trying to upgrade from kernel N to kernel N+1, which brings in the new 'widget' hypercall. Guests are live migrated onto the N+1 kernel, but the operator finds a defect that warrants a kernel rollback. VMs are then migrated from kernel N+1 -> N. Any guests that discovered the 'widget' hypercall are likely going to get fussy _very_ quickly on the old kernel. Really, we need to ensure that the exposed guest ABI is backwards-compatible. Running a VMM that is blissfully unaware of the 'widget' hypercall should not implicitly expose it to its guest on a new kernel. This conversation was in the context of devising a new UAPI that allows VMMs to trap hypercalls to userspace. While such an interface would easily work around the issue, the onus of ABI compatibility lies with the kernel. So, this is all a long-winded way of asking: how do we dig ourselves out of this situation, and how to we avoid it happening again in the future? I believe the answer to both is to have new VM capabilities for sets of hypercalls exposed to the guest. Unfortunately, the unconditional exposure of TRNG and PTP hypercalls is ABI now, so we'd have to provide an opt-out at this point. For anything new, require opt-in from the VMM before we give it to the guest. Have I missed something blatantly obvious, or do others see this as an issue as well? I'll reply with an example of adding opt-out for PTP. I'm sure other hypercalls could be handled similarly. -- Thanks, Oliver