From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Wang Subject: Re: [virtio-dev] Re: [PATCH v9 2/5] virtio-balloon: VIRTIO_BALLOON_F_BALLOON_CHUNKS Date: Tue, 09 May 2017 10:45:59 +0800 Message-ID: <59112D67.9080405@intel.com> References: <1492076108-117229-3-git-send-email-wei.w.wang@intel.com> <20170413184040-mutt-send-email-mst@kernel.org> <58F08A60.2020407@intel.com> <20170415000934-mutt-send-email-mst@kernel.org> <58F43801.7060004@intel.com> <286AC319A985734F985F78AFA26841F7391F6DCD@shsmsx102.ccr.corp.intel.com> <20170426192753-mutt-send-email-mst@kernel.org> <59019055.3040708@intel.com> <20170506012322-mutt-send-email-mst@kernel.org> <286AC319A985734F985F78AFA26841F7391FFBB0@shsmsx102.ccr.corp.intel.com> <20170508203533-mutt-send-email-mst@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "virtio-dev@lists.oasis-open.org" , "linux-kernel@vger.kernel.org" , "qemu-devel@nongnu.org" , "virtualization@lists.linux-foundation.org" , "kvm@vger.kernel.org" , "linux-mm@kvack.org" , "david@redhat.com" , "Hansen, Dave" , "cornelia.huck@de.ibm.com" , "akpm@linux-foundation.org" , "mgorman@techsingularity.net" , "aarcange@redhat.com" , "amit.shah@redhat.com" , "pbonzini@redhat.com" , "liliang.opensource@gmail.com" To: "Michael S. Tsirkin" Return-path: In-Reply-To: <20170508203533-mutt-send-email-mst@kernel.org> Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org On 05/09/2017 01:40 AM, Michael S. Tsirkin wrote: > On Sun, May 07, 2017 at 04:19:28AM +0000, Wang, Wei W wrote: >> On 05/06/2017 06:26 AM, Michael S. Tsirkin wrote: >>> On Thu, Apr 27, 2017 at 02:31:49PM +0800, Wei Wang wrote: >>>> On 04/27/2017 07:20 AM, Michael S. Tsirkin wrote: >>>>> On Wed, Apr 26, 2017 at 11:03:34AM +0000, Wang, Wei W wrote: >>>>>> Hi Michael, could you please give some feedback? >>>>> I'm sorry, I'm not sure feedback on what you are requesting. >>>> Oh, just some trivial things (e.g. use a field in the header, >>>> hdr->chunks to indicate the number of chunks in the payload) that >>>> wasn't confirmed. >>>> >>>> I will prepare the new version with fixing the agreed issues, and we >>>> can continue to discuss those parts if you still find them improper. >>>> >>>> >>>>> The interface looks reasonable now, even though there's a way to >>>>> make it even simpler if we can limit chunk size to 2G (in fact 4G - >>>>> 1). Do you think we can live with this limitation? >>>> Yes, I think we can. So, is it good to change to use the previous >>>> 64-bit chunk format (52-bit base + 12-bit size)? >>> This isn't what I meant. virtio ring has descriptors with a 64 bit address and 32 bit >>> size. >>> >>> If size < 4g is not a significant limitation, why not just use that to pass >>> address/size in a standard s/g list, possibly using INDIRECT? >> OK, I see your point, thanks. Post the two options here for an analysis: >> Option1 (what we have now): >> struct virtio_balloon_page_chunk { >> __le64 chunk_num; >> struct virtio_balloon_page_chunk_entry entry[]; >> }; >> Option2: >> struct virtio_balloon_page_chunk { >> __le64 chunk_num; >> struct scatterlist entry[]; >> }; > This isn't what I meant really :) I meant vring_desc. OK. Repost the code change: Option2: struct virtio_balloon_page_chunk { __le64 chunk_num; struct ving_desc entry[]; }; We pre-allocate a table of desc, and each desc is used to hold a chunk. In that case, the virtqueue_add() function, which deals with sg, is not usable for us. We may need to add a new one, virtqueue_add_indirect_desc(), to add a pre-allocated indirect descriptor table to vring. > >> I don't have an issue to change it to Option2, but I would prefer Option1, >> because I think there is no be obvious difference between the two options, >> while Option1 appears to have little advantages here: >> 1) "struct virtio_balloon_page_chunk_entry" has smaller size than >> "struct scatterlist", so the same size of allocated page chunk buffer >> can hold more entry[] using Option1; >> 2) INDIRECT needs on demand kmalloc(); > Within alloc_indirect? We can fix that with a separate patch. > > >> 3) no 4G size limit; > Do you see lots of >=4g chunks in practice? It wouldn't be much in practice, but we still need the extra code to handle the case - break larger chunks into less-than 4g ones. Best, Wei -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753697AbdEICoH (ORCPT ); Mon, 8 May 2017 22:44:07 -0400 Received: from mga11.intel.com ([192.55.52.93]:15046 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751243AbdEICoG (ORCPT ); Mon, 8 May 2017 22:44:06 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.38,312,1491289200"; d="scan'208";a="84357361" Message-ID: <59112D67.9080405@intel.com> Date: Tue, 09 May 2017 10:45:59 +0800 From: Wei Wang User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: "Michael S. Tsirkin" CC: "virtio-dev@lists.oasis-open.org" , "linux-kernel@vger.kernel.org" , "qemu-devel@nongnu.org" , "virtualization@lists.linux-foundation.org" , "kvm@vger.kernel.org" , "linux-mm@kvack.org" , "david@redhat.com" , "Hansen, Dave" , "cornelia.huck@de.ibm.com" , "akpm@linux-foundation.org" , "mgorman@techsingularity.net" , "aarcange@redhat.com" , "amit.shah@redhat.com" , "pbonzini@redhat.com" , "liliang.opensource@gmail.com" Subject: Re: [virtio-dev] Re: [PATCH v9 2/5] virtio-balloon: VIRTIO_BALLOON_F_BALLOON_CHUNKS References: <1492076108-117229-3-git-send-email-wei.w.wang@intel.com> <20170413184040-mutt-send-email-mst@kernel.org> <58F08A60.2020407@intel.com> <20170415000934-mutt-send-email-mst@kernel.org> <58F43801.7060004@intel.com> <286AC319A985734F985F78AFA26841F7391F6DCD@shsmsx102.ccr.corp.intel.com> <20170426192753-mutt-send-email-mst@kernel.org> <59019055.3040708@intel.com> <20170506012322-mutt-send-email-mst@kernel.org> <286AC319A985734F985F78AFA26841F7391FFBB0@shsmsx102.ccr.corp.intel.com> <20170508203533-mutt-send-email-mst@kernel.org> In-Reply-To: <20170508203533-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 05/09/2017 01:40 AM, Michael S. Tsirkin wrote: > On Sun, May 07, 2017 at 04:19:28AM +0000, Wang, Wei W wrote: >> On 05/06/2017 06:26 AM, Michael S. Tsirkin wrote: >>> On Thu, Apr 27, 2017 at 02:31:49PM +0800, Wei Wang wrote: >>>> On 04/27/2017 07:20 AM, Michael S. Tsirkin wrote: >>>>> On Wed, Apr 26, 2017 at 11:03:34AM +0000, Wang, Wei W wrote: >>>>>> Hi Michael, could you please give some feedback? >>>>> I'm sorry, I'm not sure feedback on what you are requesting. >>>> Oh, just some trivial things (e.g. use a field in the header, >>>> hdr->chunks to indicate the number of chunks in the payload) that >>>> wasn't confirmed. >>>> >>>> I will prepare the new version with fixing the agreed issues, and we >>>> can continue to discuss those parts if you still find them improper. >>>> >>>> >>>>> The interface looks reasonable now, even though there's a way to >>>>> make it even simpler if we can limit chunk size to 2G (in fact 4G - >>>>> 1). Do you think we can live with this limitation? >>>> Yes, I think we can. So, is it good to change to use the previous >>>> 64-bit chunk format (52-bit base + 12-bit size)? >>> This isn't what I meant. virtio ring has descriptors with a 64 bit address and 32 bit >>> size. >>> >>> If size < 4g is not a significant limitation, why not just use that to pass >>> address/size in a standard s/g list, possibly using INDIRECT? >> OK, I see your point, thanks. Post the two options here for an analysis: >> Option1 (what we have now): >> struct virtio_balloon_page_chunk { >> __le64 chunk_num; >> struct virtio_balloon_page_chunk_entry entry[]; >> }; >> Option2: >> struct virtio_balloon_page_chunk { >> __le64 chunk_num; >> struct scatterlist entry[]; >> }; > This isn't what I meant really :) I meant vring_desc. OK. Repost the code change: Option2: struct virtio_balloon_page_chunk { __le64 chunk_num; struct ving_desc entry[]; }; We pre-allocate a table of desc, and each desc is used to hold a chunk. In that case, the virtqueue_add() function, which deals with sg, is not usable for us. We may need to add a new one, virtqueue_add_indirect_desc(), to add a pre-allocated indirect descriptor table to vring. > >> I don't have an issue to change it to Option2, but I would prefer Option1, >> because I think there is no be obvious difference between the two options, >> while Option1 appears to have little advantages here: >> 1) "struct virtio_balloon_page_chunk_entry" has smaller size than >> "struct scatterlist", so the same size of allocated page chunk buffer >> can hold more entry[] using Option1; >> 2) INDIRECT needs on demand kmalloc(); > Within alloc_indirect? We can fix that with a separate patch. > > >> 3) no 4G size limit; > Do you see lots of >=4g chunks in practice? It wouldn't be much in practice, but we still need the extra code to handle the case - break larger chunks into less-than 4g ones. Best, Wei From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47192) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d7v8E-00051Y-U0 for qemu-devel@nongnu.org; Mon, 08 May 2017 22:44:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d7v8B-0004n5-RY for qemu-devel@nongnu.org; Mon, 08 May 2017 22:44:10 -0400 Received: from mga02.intel.com ([134.134.136.20]:65409) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1d7v8B-0004me-Gq for qemu-devel@nongnu.org; Mon, 08 May 2017 22:44:07 -0400 Message-ID: <59112D67.9080405@intel.com> Date: Tue, 09 May 2017 10:45:59 +0800 From: Wei Wang MIME-Version: 1.0 References: <1492076108-117229-3-git-send-email-wei.w.wang@intel.com> <20170413184040-mutt-send-email-mst@kernel.org> <58F08A60.2020407@intel.com> <20170415000934-mutt-send-email-mst@kernel.org> <58F43801.7060004@intel.com> <286AC319A985734F985F78AFA26841F7391F6DCD@shsmsx102.ccr.corp.intel.com> <20170426192753-mutt-send-email-mst@kernel.org> <59019055.3040708@intel.com> <20170506012322-mutt-send-email-mst@kernel.org> <286AC319A985734F985F78AFA26841F7391FFBB0@shsmsx102.ccr.corp.intel.com> <20170508203533-mutt-send-email-mst@kernel.org> In-Reply-To: <20170508203533-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [virtio-dev] Re: [PATCH v9 2/5] virtio-balloon: VIRTIO_BALLOON_F_BALLOON_CHUNKS List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: "virtio-dev@lists.oasis-open.org" , "linux-kernel@vger.kernel.org" , "qemu-devel@nongnu.org" , "virtualization@lists.linux-foundation.org" , "kvm@vger.kernel.org" , "linux-mm@kvack.org" , "david@redhat.com" , "Hansen, Dave" , "cornelia.huck@de.ibm.com" , "akpm@linux-foundation.org" , "mgorman@techsingularity.net" , "aarcange@redhat.com" , "amit.shah@redhat.com" , "pbonzini@redhat.com" , "liliang.opensource@gmail.com" On 05/09/2017 01:40 AM, Michael S. Tsirkin wrote: > On Sun, May 07, 2017 at 04:19:28AM +0000, Wang, Wei W wrote: >> On 05/06/2017 06:26 AM, Michael S. Tsirkin wrote: >>> On Thu, Apr 27, 2017 at 02:31:49PM +0800, Wei Wang wrote: >>>> On 04/27/2017 07:20 AM, Michael S. Tsirkin wrote: >>>>> On Wed, Apr 26, 2017 at 11:03:34AM +0000, Wang, Wei W wrote: >>>>>> Hi Michael, could you please give some feedback? >>>>> I'm sorry, I'm not sure feedback on what you are requesting. >>>> Oh, just some trivial things (e.g. use a field in the header, >>>> hdr->chunks to indicate the number of chunks in the payload) that >>>> wasn't confirmed. >>>> >>>> I will prepare the new version with fixing the agreed issues, and we >>>> can continue to discuss those parts if you still find them improper. >>>> >>>> >>>>> The interface looks reasonable now, even though there's a way to >>>>> make it even simpler if we can limit chunk size to 2G (in fact 4G - >>>>> 1). Do you think we can live with this limitation? >>>> Yes, I think we can. So, is it good to change to use the previous >>>> 64-bit chunk format (52-bit base + 12-bit size)? >>> This isn't what I meant. virtio ring has descriptors with a 64 bit address and 32 bit >>> size. >>> >>> If size < 4g is not a significant limitation, why not just use that to pass >>> address/size in a standard s/g list, possibly using INDIRECT? >> OK, I see your point, thanks. Post the two options here for an analysis: >> Option1 (what we have now): >> struct virtio_balloon_page_chunk { >> __le64 chunk_num; >> struct virtio_balloon_page_chunk_entry entry[]; >> }; >> Option2: >> struct virtio_balloon_page_chunk { >> __le64 chunk_num; >> struct scatterlist entry[]; >> }; > This isn't what I meant really :) I meant vring_desc. OK. Repost the code change: Option2: struct virtio_balloon_page_chunk { __le64 chunk_num; struct ving_desc entry[]; }; We pre-allocate a table of desc, and each desc is used to hold a chunk. In that case, the virtqueue_add() function, which deals with sg, is not usable for us. We may need to add a new one, virtqueue_add_indirect_desc(), to add a pre-allocated indirect descriptor table to vring. > >> I don't have an issue to change it to Option2, but I would prefer Option1, >> because I think there is no be obvious difference between the two options, >> while Option1 appears to have little advantages here: >> 1) "struct virtio_balloon_page_chunk_entry" has smaller size than >> "struct scatterlist", so the same size of allocated page chunk buffer >> can hold more entry[] using Option1; >> 2) INDIRECT needs on demand kmalloc(); > Within alloc_indirect? We can fix that with a separate patch. > > >> 3) no 4G size limit; > Do you see lots of >=4g chunks in practice? It wouldn't be much in practice, but we still need the extra code to handle the case - break larger chunks into less-than 4g ones. Best, Wei