From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E60ABC001DF for ; Thu, 3 Aug 2023 13:06:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qRY1R-0000K1-9e; Thu, 03 Aug 2023 09:05:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRY1N-0008Ue-I7 for qemu-devel@nongnu.org; Thu, 03 Aug 2023 09:05:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qRY1L-0002N1-14 for qemu-devel@nongnu.org; Thu, 03 Aug 2023 09:05:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1691067933; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qwoc3pf9uJ89LLqMiMJh16yMSgGITmRChYha7f0k8Qs=; b=Z8985cRHt3v5BROdKeW/FFaRfifa0juBa5SIMGHxEuzfg0VQX5rezDDUoX8fB4KqM44Kqi 43mZzwMc4QIhMtNMA2m9FJ+TSUOds3DDHxMaV0Hc49wqt2tEo3ilYZL+HduiiNHcWx6ycF yvuAKnxqA7ph8oX6ijxCZeOD9+D1geU= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-31-WS6d-jL7PtSZytnzDkmFKQ-1; Thu, 03 Aug 2023 09:05:31 -0400 X-MC-Unique: WS6d-jL7PtSZytnzDkmFKQ-1 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-31785455660so550543f8f.2 for ; Thu, 03 Aug 2023 06:05:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691067930; x=1691672730; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Qwoc3pf9uJ89LLqMiMJh16yMSgGITmRChYha7f0k8Qs=; b=VYxcdFxguaIIkwyPaZ3wVUay0D/VADM8sLbqVPCBQNuRC0vMdnWnowu4aerqxyD7KH JyDOhRyNZ6DXLG44x8YHX8x8lbFNHn/GVglyftZgmpAHdSkEK4DPD/Hs5k/oV32wcsmp oTy2hbXa4cRq8+yEs0LeGT6vtj2jviN8QV0+QgvRiCFhhCxevQVlW3RiJ2k3AROcPmr2 OqYrNrBQ2xhzu+/yj05kYpzVdHigPhZjRvnz2bys8IAwG65TYlrn90w+4bZ/yWQgfRsF B+acA2W6bcS0Ew2zraJ7cFrYQNslBfiZdBB18GucdForHm9hnPbidhPUki584AIMG566 A5SQ== X-Gm-Message-State: ABy/qLYTPai4MrZp8jnzvlr2pMvYd7E1O3rtwaLs3L0kTGUxyNtO4J4P WciF8U+l54nzSG/X+ORT9TccRpMZB7veiU47tcpXc/zadRkXiGEcw2/vXKJ+YsH+8bS3mT+AQ6Q wPR6H1L6FtGXjF+4= X-Received: by 2002:a5d:460f:0:b0:315:9993:1caa with SMTP id t15-20020a5d460f000000b0031599931caamr5314382wrq.12.1691067930463; Thu, 03 Aug 2023 06:05:30 -0700 (PDT) X-Google-Smtp-Source: APBJJlH7GajLmBHd29WLWeB9cAZpW+rixqcOeEXLg2iQbWmZSODsf7unWiCX6sVwncZrmQDYkBj05g== X-Received: by 2002:a5d:460f:0:b0:315:9993:1caa with SMTP id t15-20020a5d460f000000b0031599931caamr5314355wrq.12.1691067930012; Thu, 03 Aug 2023 06:05:30 -0700 (PDT) Received: from ?IPV6:2003:cb:c718:9a00:a5f5:5315:b9fa:64df? (p200300cbc7189a00a5f55315b9fa64df.dip0.t-ipconnect.de. [2003:cb:c718:9a00:a5f5:5315:b9fa:64df]) by smtp.gmail.com with ESMTPSA id p14-20020a1c740e000000b003fe20db88e2sm4281573wmc.18.2023.08.03.06.05.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Aug 2023 06:05:29 -0700 (PDT) Message-ID: Date: Thu, 3 Aug 2023 15:05:28 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Content-Language: en-US To: Isaku Yamahata Cc: Xiaoyao Li , Paolo Bonzini , Sean Christopherson , Igor Mammedov , "Michael S. Tsirkin" , Marcel Apfelbaum , Richard Henderson , Marcelo Tosatti , Markus Armbruster , Eric Blake , =?UTF-8?Q?Daniel_P=2e_Berrang=c3=a9?= , =?UTF-8?Q?Philippe_Mathieu-Daud=c3=a9?= , Peter Xu , Chao Peng , Michael Roth , qemu-devel@nongnu.org, kvm@vger.kernel.org References: <20230731162201.271114-1-xiaoyao.li@intel.com> <20230731162201.271114-9-xiaoyao.li@intel.com> <2addfff0-88bf-59aa-f2f3-8129366a006d@intel.com> <20230802225318.GE1807130@ls.amr.corp.intel.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [RFC PATCH 08/19] HostMem: Add private property to indicate to use kvm gmem In-Reply-To: <20230802225318.GE1807130@ls.amr.corp.intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=170.10.129.124; envelope-from=david@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-0.091, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org On 03.08.23 00:53, Isaku Yamahata wrote: > On Wed, Aug 02, 2023 at 04:14:29PM +0200, > David Hildenbrand wrote: > >> On 02.08.23 10:03, Xiaoyao Li wrote: >>> On 8/2/2023 1:21 AM, David Hildenbrand wrote: >>>> On 31.07.23 18:21, Xiaoyao Li wrote: >>>>> From: Isaku Yamahata >>>>> >>>>> Signed-off-by: Isaku Yamahata >>>>> Signed-off-by: Xiaoyao Li >>>>> --- >>>>>   backends/hostmem.c       | 18 ++++++++++++++++++ >>>>>   include/sysemu/hostmem.h |  2 +- >>>>>   qapi/qom.json            |  4 ++++ >>>>>   3 files changed, 23 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/backends/hostmem.c b/backends/hostmem.c >>>>> index 747e7838c031..dbdbb0aafd45 100644 >>>>> --- a/backends/hostmem.c >>>>> +++ b/backends/hostmem.c >>>>> @@ -461,6 +461,20 @@ static void >>>>> host_memory_backend_set_reserve(Object *o, bool value, Error **errp) >>>>>       } >>>>>       backend->reserve = value; >>>>>   } >>>>> + >>>>> +static bool host_memory_backend_get_private(Object *o, Error **errp) >>>>> +{ >>>>> +    HostMemoryBackend *backend = MEMORY_BACKEND(o); >>>>> + >>>>> +    return backend->private; >>>>> +} >>>>> + >>>>> +static void host_memory_backend_set_private(Object *o, bool value, >>>>> Error **errp) >>>>> +{ >>>>> +    HostMemoryBackend *backend = MEMORY_BACKEND(o); >>>>> + >>>>> +    backend->private = value; >>>>> +} >>>>>   #endif /* CONFIG_LINUX */ >>>>>   static bool >>>>> @@ -541,6 +555,10 @@ host_memory_backend_class_init(ObjectClass *oc, >>>>> void *data) >>>>>           host_memory_backend_get_reserve, >>>>> host_memory_backend_set_reserve); >>>>>       object_class_property_set_description(oc, "reserve", >>>>>           "Reserve swap space (or huge pages) if applicable"); >>>>> +    object_class_property_add_bool(oc, "private", >>>>> +        host_memory_backend_get_private, >>>>> host_memory_backend_set_private); >>>>> +    object_class_property_set_description(oc, "private", >>>>> +        "Use KVM gmem private memory"); >>>>>   #endif /* CONFIG_LINUX */ >>>>>       /* >>>>>        * Do not delete/rename option. This option must be considered >>>>> stable >>>>> diff --git a/include/sysemu/hostmem.h b/include/sysemu/hostmem.h >>>>> index 39326f1d4f9c..d88970395618 100644 >>>>> --- a/include/sysemu/hostmem.h >>>>> +++ b/include/sysemu/hostmem.h >>>>> @@ -65,7 +65,7 @@ struct HostMemoryBackend { >>>>>       /* protected */ >>>>>       uint64_t size; >>>>>       bool merge, dump, use_canonical_path; >>>>> -    bool prealloc, is_mapped, share, reserve; >>>>> +    bool prealloc, is_mapped, share, reserve, private; >>>>>       uint32_t prealloc_threads; >>>>>       ThreadContext *prealloc_context; >>>>>       DECLARE_BITMAP(host_nodes, MAX_NODES + 1); >>>>> diff --git a/qapi/qom.json b/qapi/qom.json >>>>> index 7f92ea43e8e1..e0b2044e3d20 100644 >>>>> --- a/qapi/qom.json >>>>> +++ b/qapi/qom.json >>>>> @@ -605,6 +605,9 @@ >>>>>   # @reserve: if true, reserve swap space (or huge pages) if applicable >>>>>   #     (default: true) (since 6.1) >>>>>   # >>>>> +# @private: if true, use KVM gmem private memory >>>>> +#           (default: false) (since 8.1) >>>>> +# >>>> >>>> But that's not what any of this does. >>>> >>>> This patch only adds a property and doesn't even explain what it intends >>>> to achieve with that. >>>> >>>> How will it be used from a user? What will it affect internally? What >>>> will it modify in regards of the memory backend? >>> >>> How it will be used is in the next patch, patch 09. >>> >>> for kvm_x86_sw_protected_vm type VM, it will allocate private gmem with >>> KVM ioctl if the memory backend has property "private" on. >> >> It feels wired up the wrong way. >> >> When creating/initializing the memory backend, we should also take care of >> allocating the gmem_fd, for example, by doing some gmem allocation callback, >> ideally *internally* creating the RAM memory region / RAMBlock. >> >> And we should fail if that is impossible (gmem does not apply to the VM) or >> creating the gmem_fd failed for other reason. >> >> Like passing a RAM_GMEM flag to memory_region_init_ram_flags_nomigrate() in >> ram_backend_memory_alloc(), to then handle it internally, failing if there >> is an error. > > KVM gmem is tied to VM. not before creating VM. We have to delay of the > allocation of kvm gmem until VM initialization. In vl.c, the flow is 1) Create machine: qemu_create_machine() 2) Configure KVM: configure_accelerators() 3) Create backends: qemu_create_late_backends() Staring at object_create_early(), "memory-backend-" area always late. So maybe, at the time memory backends are created+initialized, everything you need is already in place. > > Hmm, one options is to move gmem_fd from RAMBlock to KVMSlot. Handle the > allocation of KVM gmem (issuing KVM gmem ioctl) there. i.e. in > kvm_set_phys_mem() or kvm_region_add() (or whatever functions of KVM memory > listener). Maybe we can drop ram_block_convert_range() and can have KVM > specific logic instead. Might be doable as well. > > We still need a way for user to specify which guest memory region is subject > to KVM gmem, though. Let's minimize the gmem hacks and come up with something clean. -- Cheers, David / dhildenb