From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Xu Date: Fri, 07 Feb 2020 18:17:48 +0000 Subject: Re: [PATCH v5 12/19] KVM: Move memslot deletion to helper function Message-Id: <20200207181748.GC720553@xz-x1> List-Id: References: <20200121223157.15263-1-sean.j.christopherson@intel.com> <20200121223157.15263-13-sean.j.christopherson@intel.com> <20200206161415.GA695333@xz-x1> <20200206162818.GD13067@linux.intel.com> <20200206165116.GE695333@xz-x1> <20200207175912.GG2401@linux.intel.com> In-Reply-To: <20200207175912.GG2401@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Sean Christopherson Cc: Paolo Bonzini , Paul Mackerras , Christian Borntraeger , Janosch Frank , David Hildenbrand , Cornelia Huck , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, Christoffer Dall , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= On Fri, Feb 07, 2020 at 09:59:12AM -0800, Sean Christopherson wrote: > On Thu, Feb 06, 2020 at 11:51:16AM -0500, Peter Xu wrote: > > On Thu, Feb 06, 2020 at 08:28:18AM -0800, Sean Christopherson wrote: > > > On Thu, Feb 06, 2020 at 11:14:15AM -0500, Peter Xu wrote: > > > > On Tue, Jan 21, 2020 at 02:31:50PM -0800, Sean Christopherson wrote: > > > > > Move memslot deletion into its own routine so that the success path for > > > > > other memslot updates does not need to use kvm_free_memslot(), i.e. can > > > > > explicitly destroy the dirty bitmap when necessary. This paves the way > > > > > for dropping @dont from kvm_free_memslot(), i.e. all callers now pass > > > > > NULL for @dont. > > > > > > > > > > Add a comment above the code to make a copy of the existing memslot > > > > > prior to deletion, it is not at all obvious that the pointer will become > > > > > stale during sorting and/or installation of new memslots. > > > > > > > > Could you help explain a bit on this explicit comment? I can follow > > > > up with the patch itself which looks all correct to me, but I failed > > > > to catch what this extra comment wants to emphasize... > > > > > > It's tempting to write the code like this (I know, because I did it): > > > > > > if (!mem->memory_size) > > > return kvm_delete_memslot(kvm, mem, slot, as_id); > > > > > > new = *slot; > > > > > > Where @slot is a pointer to the memslot to be deleted. At first, second, > > > and third glances, this seems perfectly sane. > > > > > > The issue is that slot was pulled from struct kvm_memslots.memslots, e.g. > > > > > > slot = &slots->memslots[index]; > > > > > > Note that slots->memslots holds actual "struct kvm_memory_slot" objects, > > > not pointers to slots. When update_memslots() sorts the slots, it swaps > > > the actual slot objects, not pointers. I.e. after update_memslots(), even > > > though @slot points at the same address, it's could be pointing at a > > > different slot. As a result kvm_free_memslot() in kvm_delete_memslot() > > > will free the dirty page info and arch-specific points for some random > > > slot, not the intended slot, and will set npages=0 for that random slot. > > > > Ah I see, thanks. Another alternative is we move the "old = *slot" > > copy into kvm_delete_memslot(), which could be even clearer imo. > > The copy is also needed in __kvm_set_memory_region() for the MOVE case. Right. I actually meant to do all "old = *slot" in any function we need to cache the data explicitly, with that we also need another one after calling kvm_delete_memslot() for move. But with the comment as you suggested below it looks good to me too. Thanks, > > > However I'm not sure whether it's a good idea to drop the test-by for > > this. Considering that comment change should not affect it, would you > > mind enrich the comment into something like this (or anything better)? > > > > /* > > * Make a full copy of the old memslot, the pointer will become stale > > * when the memslots are re-sorted by update_memslots() in > > * kvm_delete_memslot(), while to make the kvm_free_memslot() work as > > * expected later on, we still need the cached memory slot. > > */ > > As above, it's more subtle than just the kvm_delete_memslot() case. > > /* > * Make a full copy of the old memslot, the pointer will become stale > * when the memslots are re-sorted by update_memslots() when deleting > * or moving a memslot, and additional modifications to the old memslot > * need to be made after calling update_memslots(). > */ > -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98DB5C35242 for ; Fri, 7 Feb 2020 18:18:01 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 2A4B120726 for ; Fri, 7 Feb 2020 18:18:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AEKvEFJw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A4B120726 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id B2FCF4A534; Fri, 7 Feb 2020 13:18:00 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@redhat.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nELIVK+Xsigr; Fri, 7 Feb 2020 13:17:59 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 869064A524; Fri, 7 Feb 2020 13:17:59 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E4BC14A524 for ; Fri, 7 Feb 2020 13:17:57 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zsxz9F2V6Fqg for ; Fri, 7 Feb 2020 13:17:56 -0500 (EST) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E43794A3A5 for ; Fri, 7 Feb 2020 13:17:56 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581099476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zZJ4qP8Dp5obTldW76MuAHNAAKAcx/3dX73hUB9dtQo=; b=AEKvEFJwZ88mn5VJ6/9cgwIiJ34mmci8v5WOgG2AAfiWBgI/6oDWya/HncVKgw4JiljwwM l/ZrYNGSi/hInjiMLKLJXwdCxKduWfXbIsfqECJLP4NqQTPRMKTD7YDGV+yxgo6IeBBvcs P2Tclz4hO4X2CrTMMwYcbQr/kJ0ZKg4= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-297-m79cGzc4PBmwhbjngUuWXw-1; Fri, 07 Feb 2020 13:17:52 -0500 Received: by mail-qv1-f71.google.com with SMTP id r9so55078qvs.19 for ; Fri, 07 Feb 2020 10:17:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=cPT63VS6JkFJRjpL5MMsWxNfN8Beu47vJy8tbC2r7XM=; b=VJuMquJizhDxFiNaQ4/OsLZpRIZP5N1lUVuaTRkOBEtrfV7q2G767iTPkbGoFAzUbc WVFvKRfwVoqCpU9zyDn/pFQFshbcQoudNXodXise/fjUHiKQDpvYzXWfOYu191uKvowF MtgsujtRQr8I8ZibCTX+oZjRfdDnQbUpAgvmd4KYPrjzf7lpISgAifOTzCksrWIsnW4L lkA11HMjHOZHWPVHhB9nkWL6HrmiZ+gm/gD+Un2YW+oCofGv27fZ+ZhWz8VZ1U2yEHN3 wGoHZYNy7aFiA2av3rfYPJLvDGCLNpa9HCmZYH3xBZ5I/7/AhYScGSB9bDOz9fARV9cK txUQ== X-Gm-Message-State: APjAAAU/UTyvC7cojke8Q5vCF8RH6+5CDKMAKwVe4EgQMj5a7KSY/ONv 5ROVLzWpBesmLzUyR4Cg9AQZUxESIl1r3eOJ3O5VkG48/ROKc7ttPk2B73UAu0chCq7glS4GQhR sp4/AA6EN8UE8B8Q5Zt8l08p2 X-Received: by 2002:a05:6214:1253:: with SMTP id q19mr106732qvv.75.1581099472153; Fri, 07 Feb 2020 10:17:52 -0800 (PST) X-Google-Smtp-Source: APXvYqzRox8Bo7fyFYmNOjYNh/V3B0M3O5rtHPF6if8Y080MuT3a3U4ayt/3NWDSUCkErG4IzY/vdw== X-Received: by 2002:a05:6214:1253:: with SMTP id q19mr106704qvv.75.1581099471868; Fri, 07 Feb 2020 10:17:51 -0800 (PST) Received: from xz-x1 ([2607:9880:19c8:32::2]) by smtp.gmail.com with ESMTPSA id b24sm1725899qto.71.2020.02.07.10.17.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Feb 2020 10:17:51 -0800 (PST) Date: Fri, 7 Feb 2020 13:17:48 -0500 From: Peter Xu To: Sean Christopherson Subject: Re: [PATCH v5 12/19] KVM: Move memslot deletion to helper function Message-ID: <20200207181748.GC720553@xz-x1> References: <20200121223157.15263-1-sean.j.christopherson@intel.com> <20200121223157.15263-13-sean.j.christopherson@intel.com> <20200206161415.GA695333@xz-x1> <20200206162818.GD13067@linux.intel.com> <20200206165116.GE695333@xz-x1> <20200207175912.GG2401@linux.intel.com> MIME-Version: 1.0 In-Reply-To: <20200207175912.GG2401@linux.intel.com> X-MC-Unique: m79cGzc4PBmwhbjngUuWXw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: Wanpeng Li , kvm@vger.kernel.org, David Hildenbrand , linux-mips@vger.kernel.org, Paul Mackerras , kvmarm@lists.cs.columbia.edu, Janosch Frank , Marc Zyngier , Joerg Roedel , Christian Borntraeger , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Jim Mattson , Cornelia Huck , linux-kernel@vger.kernel.org, Paolo Bonzini , Vitaly Kuznetsov , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Fri, Feb 07, 2020 at 09:59:12AM -0800, Sean Christopherson wrote: > On Thu, Feb 06, 2020 at 11:51:16AM -0500, Peter Xu wrote: > > On Thu, Feb 06, 2020 at 08:28:18AM -0800, Sean Christopherson wrote: > > > On Thu, Feb 06, 2020 at 11:14:15AM -0500, Peter Xu wrote: > > > > On Tue, Jan 21, 2020 at 02:31:50PM -0800, Sean Christopherson wrote: > > > > > Move memslot deletion into its own routine so that the success path for > > > > > other memslot updates does not need to use kvm_free_memslot(), i.e. can > > > > > explicitly destroy the dirty bitmap when necessary. This paves the way > > > > > for dropping @dont from kvm_free_memslot(), i.e. all callers now pass > > > > > NULL for @dont. > > > > > > > > > > Add a comment above the code to make a copy of the existing memslot > > > > > prior to deletion, it is not at all obvious that the pointer will become > > > > > stale during sorting and/or installation of new memslots. > > > > > > > > Could you help explain a bit on this explicit comment? I can follow > > > > up with the patch itself which looks all correct to me, but I failed > > > > to catch what this extra comment wants to emphasize... > > > > > > It's tempting to write the code like this (I know, because I did it): > > > > > > if (!mem->memory_size) > > > return kvm_delete_memslot(kvm, mem, slot, as_id); > > > > > > new = *slot; > > > > > > Where @slot is a pointer to the memslot to be deleted. At first, second, > > > and third glances, this seems perfectly sane. > > > > > > The issue is that slot was pulled from struct kvm_memslots.memslots, e.g. > > > > > > slot = &slots->memslots[index]; > > > > > > Note that slots->memslots holds actual "struct kvm_memory_slot" objects, > > > not pointers to slots. When update_memslots() sorts the slots, it swaps > > > the actual slot objects, not pointers. I.e. after update_memslots(), even > > > though @slot points at the same address, it's could be pointing at a > > > different slot. As a result kvm_free_memslot() in kvm_delete_memslot() > > > will free the dirty page info and arch-specific points for some random > > > slot, not the intended slot, and will set npages=0 for that random slot. > > > > Ah I see, thanks. Another alternative is we move the "old = *slot" > > copy into kvm_delete_memslot(), which could be even clearer imo. > > The copy is also needed in __kvm_set_memory_region() for the MOVE case. Right. I actually meant to do all "old = *slot" in any function we need to cache the data explicitly, with that we also need another one after calling kvm_delete_memslot() for move. But with the comment as you suggested below it looks good to me too. Thanks, > > > However I'm not sure whether it's a good idea to drop the test-by for > > this. Considering that comment change should not affect it, would you > > mind enrich the comment into something like this (or anything better)? > > > > /* > > * Make a full copy of the old memslot, the pointer will become stale > > * when the memslots are re-sorted by update_memslots() in > > * kvm_delete_memslot(), while to make the kvm_free_memslot() work as > > * expected later on, we still need the cached memory slot. > > */ > > As above, it's more subtle than just the kvm_delete_memslot() case. > > /* > * Make a full copy of the old memslot, the pointer will become stale > * when the memslots are re-sorted by update_memslots() when deleting > * or moving a memslot, and additional modifications to the old memslot > * need to be made after calling update_memslots(). > */ > -- Peter Xu _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6249EC2BA83 for ; Fri, 7 Feb 2020 18:18:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 33C1F20726 for ; Fri, 7 Feb 2020 18:18:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y/cdiXS9" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727558AbgBGSR7 (ORCPT ); Fri, 7 Feb 2020 13:17:59 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:59889 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727803AbgBGSRz (ORCPT ); Fri, 7 Feb 2020 13:17:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581099474; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cPT63VS6JkFJRjpL5MMsWxNfN8Beu47vJy8tbC2r7XM=; b=Y/cdiXS9qg198XeYOe6OfszY8IiD737uVmSUJE64TzYvg13u9ESEGkmZF/72192DBYdQL7 6/T7/mO1JCuDDQjRzkApSoIq1OhrLPvdDdCt1QZxCBDOmdTL/j4gr7tzbI8xPLS+NSxjZ0 TPF5KfngjJMm20SkOCKHiCB1ZJ8FSQI= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-349-bFyCxqm_M4W_qaDt5AFPRQ-1; Fri, 07 Feb 2020 13:17:52 -0500 X-MC-Unique: bFyCxqm_M4W_qaDt5AFPRQ-1 Received: by mail-qv1-f71.google.com with SMTP id v3so83408qvm.2 for ; Fri, 07 Feb 2020 10:17:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=cPT63VS6JkFJRjpL5MMsWxNfN8Beu47vJy8tbC2r7XM=; b=E6kGi9HDVcN4EYUvGcP6Ao0ooJjUBI1q+JaFPB1IXrLvX/8Y1zUrtsuZOYvic/KeG2 oD4oSXhuTeD5KtySCU5S3DpKZbZfzLuu5Y71KFf2iofQym7G2A1m17FNZZcWN+XJniNB +Ip+fvja4Rf2JoeQJt4PY6HljNUzdxURXT3baXYvExOq6uNAnvWM9aXO7Y71HHD8kOr7 YwtGdXXdcaCsAjJvHx66bGWVA3Li8Bs0Uuo544d4cuuwrHZek00TQ5TeEkMxQ1wqwGvM 36GI6znM9TEnS3Aqgg6BmPI0404IJC6HDo7Rt8VVPcInWBwfRvkoOmRZ8FXJlKlPQPqp sRCQ== X-Gm-Message-State: APjAAAUgoxq+Lqeb9/PoVBVwsbXxlQEoP6Yr5+oiWUPdQgFgJ4K4Vdnu k5kH46bW47FGfD+DXMIBRT2mVstfdVZr6eT88J33tm9AItAJPCT17zM/7KzAzrm2d0yI/STfK0z oteetch/RzL+V/AletRREbQ== X-Received: by 2002:a05:6214:1253:: with SMTP id q19mr106742qvv.75.1581099472154; Fri, 07 Feb 2020 10:17:52 -0800 (PST) X-Google-Smtp-Source: APXvYqzRox8Bo7fyFYmNOjYNh/V3B0M3O5rtHPF6if8Y080MuT3a3U4ayt/3NWDSUCkErG4IzY/vdw== X-Received: by 2002:a05:6214:1253:: with SMTP id q19mr106704qvv.75.1581099471868; Fri, 07 Feb 2020 10:17:51 -0800 (PST) Received: from xz-x1 ([2607:9880:19c8:32::2]) by smtp.gmail.com with ESMTPSA id b24sm1725899qto.71.2020.02.07.10.17.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Feb 2020 10:17:51 -0800 (PST) Date: Fri, 7 Feb 2020 13:17:48 -0500 From: Peter Xu To: Sean Christopherson Cc: Paolo Bonzini , Paul Mackerras , Christian Borntraeger , Janosch Frank , David Hildenbrand , Cornelia Huck , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, Christoffer Dall , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= Subject: Re: [PATCH v5 12/19] KVM: Move memslot deletion to helper function Message-ID: <20200207181748.GC720553@xz-x1> References: <20200121223157.15263-1-sean.j.christopherson@intel.com> <20200121223157.15263-13-sean.j.christopherson@intel.com> <20200206161415.GA695333@xz-x1> <20200206162818.GD13067@linux.intel.com> <20200206165116.GE695333@xz-x1> <20200207175912.GG2401@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20200207175912.GG2401@linux.intel.com> Sender: linux-mips-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org On Fri, Feb 07, 2020 at 09:59:12AM -0800, Sean Christopherson wrote: > On Thu, Feb 06, 2020 at 11:51:16AM -0500, Peter Xu wrote: > > On Thu, Feb 06, 2020 at 08:28:18AM -0800, Sean Christopherson wrote: > > > On Thu, Feb 06, 2020 at 11:14:15AM -0500, Peter Xu wrote: > > > > On Tue, Jan 21, 2020 at 02:31:50PM -0800, Sean Christopherson wrote: > > > > > Move memslot deletion into its own routine so that the success path for > > > > > other memslot updates does not need to use kvm_free_memslot(), i.e. can > > > > > explicitly destroy the dirty bitmap when necessary. This paves the way > > > > > for dropping @dont from kvm_free_memslot(), i.e. all callers now pass > > > > > NULL for @dont. > > > > > > > > > > Add a comment above the code to make a copy of the existing memslot > > > > > prior to deletion, it is not at all obvious that the pointer will become > > > > > stale during sorting and/or installation of new memslots. > > > > > > > > Could you help explain a bit on this explicit comment? I can follow > > > > up with the patch itself which looks all correct to me, but I failed > > > > to catch what this extra comment wants to emphasize... > > > > > > It's tempting to write the code like this (I know, because I did it): > > > > > > if (!mem->memory_size) > > > return kvm_delete_memslot(kvm, mem, slot, as_id); > > > > > > new = *slot; > > > > > > Where @slot is a pointer to the memslot to be deleted. At first, second, > > > and third glances, this seems perfectly sane. > > > > > > The issue is that slot was pulled from struct kvm_memslots.memslots, e.g. > > > > > > slot = &slots->memslots[index]; > > > > > > Note that slots->memslots holds actual "struct kvm_memory_slot" objects, > > > not pointers to slots. When update_memslots() sorts the slots, it swaps > > > the actual slot objects, not pointers. I.e. after update_memslots(), even > > > though @slot points at the same address, it's could be pointing at a > > > different slot. As a result kvm_free_memslot() in kvm_delete_memslot() > > > will free the dirty page info and arch-specific points for some random > > > slot, not the intended slot, and will set npages=0 for that random slot. > > > > Ah I see, thanks. Another alternative is we move the "old = *slot" > > copy into kvm_delete_memslot(), which could be even clearer imo. > > The copy is also needed in __kvm_set_memory_region() for the MOVE case. Right. I actually meant to do all "old = *slot" in any function we need to cache the data explicitly, with that we also need another one after calling kvm_delete_memslot() for move. But with the comment as you suggested below it looks good to me too. Thanks, > > > However I'm not sure whether it's a good idea to drop the test-by for > > this. Considering that comment change should not affect it, would you > > mind enrich the comment into something like this (or anything better)? > > > > /* > > * Make a full copy of the old memslot, the pointer will become stale > > * when the memslots are re-sorted by update_memslots() in > > * kvm_delete_memslot(), while to make the kvm_free_memslot() work as > > * expected later on, we still need the cached memory slot. > > */ > > As above, it's more subtle than just the kvm_delete_memslot() case. > > /* > * Make a full copy of the old memslot, the pointer will become stale > * when the memslots are re-sorted by update_memslots() when deleting > * or moving a memslot, and additional modifications to the old memslot > * need to be made after calling update_memslots(). > */ > -- Peter Xu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2619C2BA83 for ; Fri, 7 Feb 2020 18:18:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BAB2121775 for ; Fri, 7 Feb 2020 18:18:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ckRVSGwm"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XwS795EJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BAB2121775 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1q325ExfPT4NKA/88E9chTN73k36gPpmRmSQNjkza0M=; b=ckRVSGwmNYXJRE WeeQektPIaZON5Kj1p2EbbXE55/Un2wiWJgx1GnfepmVK4/jVmcHwgcxQVTf4Imi+krtpL3c5aS5Y 5uX9o2O3Gsk0NvYSB+/9rrPWV2JnF52APsT0kH9lHqAl2dfRbufzjUeQY8egb5xkNMV8l6J7yDc73 FTi3C2YilaLb87B4kY7LqVQuKjxL/7Lb0EdyiT/9pVjSUt00IaDZ2AagMMeaMB50hFtw/Ew6Hbw1L U3dAWs7xo6hYR+hqiB56zDJbMO2WNDWGJiUgJHG/pj2cTEg9LX8Shq7GQCPXAbtJs2IfmMAHnLr1n AanslaAowii5Is73IN8Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1j08CX-0002Hq-6n; Fri, 07 Feb 2020 18:18:01 +0000 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120] helo=us-smtp-1.mimecast.com) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1j08CS-0002GX-K9 for linux-arm-kernel@lists.infradead.org; Fri, 07 Feb 2020 18:17:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581099475; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zZJ4qP8Dp5obTldW76MuAHNAAKAcx/3dX73hUB9dtQo=; b=XwS795EJykRQWe4WcY7gj7ELnDIg5f79u2HjeSQellpencprlJkCU0sLp/a3tiaZQqjpb1 TNJX1VrrMFiq1T5E/eez0fJt0tR0FYzXgF3OcKr5MGg9ExIUc9sIN7HAAK4orwRdKcdFS9 SNjljzPS7HjG51+C3NRGV8/E1q6Lf/8= Received: from mail-qv1-f70.google.com (mail-qv1-f70.google.com [209.85.219.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-228-b3gXl4w2P6yE44wHINc9gw-1; Fri, 07 Feb 2020 13:17:52 -0500 Received: by mail-qv1-f70.google.com with SMTP id dc2so75207qvb.7 for ; Fri, 07 Feb 2020 10:17:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=cPT63VS6JkFJRjpL5MMsWxNfN8Beu47vJy8tbC2r7XM=; b=aovv6h9IDPR+ToEdIv75bYN4dNchplLKoL9f6rFFvbQRVjdy+kiLFtH6uXI4v+Bnyj /OpK+bTzeVy2G7Xw9jXg/RujmTfHltHqbqhj3pLqZHZcylu11hKyHgB0ONnFNBOpIeQq s8PgQuDpaaNUpnGJRBiy3lkXYtAnUc/m/b6kGi0P3PkpDKog1SXeIXqQd7srnecCXHUR n/lT6UAjJya94/6fl4/xVNatUOgqE8GErq4JcsC0ZtapNWBh6flTiiHOfnTFZgYlnJ9s DHeEkd20FajkJgpbUv4qaIno3EIN4gfdE8w9nOetx8sZhl1synWfnGbUKji35j0sh9FZ FYpA== X-Gm-Message-State: APjAAAXlprkuHR6bkHe7yDEYFqcvE2FLYcvLDuV3xQ0muRtvHX3M1kri W5mtc1zy3Ye/E8HETXLA8jP4EMfu6h1yBp6XbDl2GLul+EV1QHoKfHct/2uVkbFmcQrU7DN52Vc E0+/5BvkWYnNJrpMdOWGgOEsSBmJ0fG8VACk= X-Received: by 2002:a05:6214:1253:: with SMTP id q19mr106747qvv.75.1581099472159; Fri, 07 Feb 2020 10:17:52 -0800 (PST) X-Google-Smtp-Source: APXvYqzRox8Bo7fyFYmNOjYNh/V3B0M3O5rtHPF6if8Y080MuT3a3U4ayt/3NWDSUCkErG4IzY/vdw== X-Received: by 2002:a05:6214:1253:: with SMTP id q19mr106704qvv.75.1581099471868; Fri, 07 Feb 2020 10:17:51 -0800 (PST) Received: from xz-x1 ([2607:9880:19c8:32::2]) by smtp.gmail.com with ESMTPSA id b24sm1725899qto.71.2020.02.07.10.17.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 07 Feb 2020 10:17:51 -0800 (PST) Date: Fri, 7 Feb 2020 13:17:48 -0500 From: Peter Xu To: Sean Christopherson Subject: Re: [PATCH v5 12/19] KVM: Move memslot deletion to helper function Message-ID: <20200207181748.GC720553@xz-x1> References: <20200121223157.15263-1-sean.j.christopherson@intel.com> <20200121223157.15263-13-sean.j.christopherson@intel.com> <20200206161415.GA695333@xz-x1> <20200206162818.GD13067@linux.intel.com> <20200206165116.GE695333@xz-x1> <20200207175912.GG2401@linux.intel.com> MIME-Version: 1.0 In-Reply-To: <20200207175912.GG2401@linux.intel.com> X-MC-Unique: b3gXl4w2P6yE44wHINc9gw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200207_101756_807093_F0E4C186 X-CRM114-Status: GOOD ( 26.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wanpeng Li , kvm@vger.kernel.org, David Hildenbrand , linux-mips@vger.kernel.org, Paul Mackerras , kvmarm@lists.cs.columbia.edu, Janosch Frank , Marc Zyngier , Joerg Roedel , Christian Borntraeger , Julien Thierry , Suzuki K Poulose , kvm-ppc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Jim Mattson , Cornelia Huck , Christoffer Dall , linux-kernel@vger.kernel.org, James Morse , Paolo Bonzini , Vitaly Kuznetsov , Philippe =?utf-8?Q?Mathieu-Daud=C3=A9?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Feb 07, 2020 at 09:59:12AM -0800, Sean Christopherson wrote: > On Thu, Feb 06, 2020 at 11:51:16AM -0500, Peter Xu wrote: > > On Thu, Feb 06, 2020 at 08:28:18AM -0800, Sean Christopherson wrote: > > > On Thu, Feb 06, 2020 at 11:14:15AM -0500, Peter Xu wrote: > > > > On Tue, Jan 21, 2020 at 02:31:50PM -0800, Sean Christopherson wrote: > > > > > Move memslot deletion into its own routine so that the success path for > > > > > other memslot updates does not need to use kvm_free_memslot(), i.e. can > > > > > explicitly destroy the dirty bitmap when necessary. This paves the way > > > > > for dropping @dont from kvm_free_memslot(), i.e. all callers now pass > > > > > NULL for @dont. > > > > > > > > > > Add a comment above the code to make a copy of the existing memslot > > > > > prior to deletion, it is not at all obvious that the pointer will become > > > > > stale during sorting and/or installation of new memslots. > > > > > > > > Could you help explain a bit on this explicit comment? I can follow > > > > up with the patch itself which looks all correct to me, but I failed > > > > to catch what this extra comment wants to emphasize... > > > > > > It's tempting to write the code like this (I know, because I did it): > > > > > > if (!mem->memory_size) > > > return kvm_delete_memslot(kvm, mem, slot, as_id); > > > > > > new = *slot; > > > > > > Where @slot is a pointer to the memslot to be deleted. At first, second, > > > and third glances, this seems perfectly sane. > > > > > > The issue is that slot was pulled from struct kvm_memslots.memslots, e.g. > > > > > > slot = &slots->memslots[index]; > > > > > > Note that slots->memslots holds actual "struct kvm_memory_slot" objects, > > > not pointers to slots. When update_memslots() sorts the slots, it swaps > > > the actual slot objects, not pointers. I.e. after update_memslots(), even > > > though @slot points at the same address, it's could be pointing at a > > > different slot. As a result kvm_free_memslot() in kvm_delete_memslot() > > > will free the dirty page info and arch-specific points for some random > > > slot, not the intended slot, and will set npages=0 for that random slot. > > > > Ah I see, thanks. Another alternative is we move the "old = *slot" > > copy into kvm_delete_memslot(), which could be even clearer imo. > > The copy is also needed in __kvm_set_memory_region() for the MOVE case. Right. I actually meant to do all "old = *slot" in any function we need to cache the data explicitly, with that we also need another one after calling kvm_delete_memslot() for move. But with the comment as you suggested below it looks good to me too. Thanks, > > > However I'm not sure whether it's a good idea to drop the test-by for > > this. Considering that comment change should not affect it, would you > > mind enrich the comment into something like this (or anything better)? > > > > /* > > * Make a full copy of the old memslot, the pointer will become stale > > * when the memslots are re-sorted by update_memslots() in > > * kvm_delete_memslot(), while to make the kvm_free_memslot() work as > > * expected later on, we still need the cached memory slot. > > */ > > As above, it's more subtle than just the kvm_delete_memslot() case. > > /* > * Make a full copy of the old memslot, the pointer will become stale > * when the memslots are re-sorted by update_memslots() when deleting > * or moving a memslot, and additional modifications to the old memslot > * need to be made after calling update_memslots(). > */ > -- Peter Xu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel