From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Wang Subject: Re: [PATCH v7 kernel 3/5] virtio-balloon: implementation of VIRTIO_BALLOON_F_CHUNK_TRANSFER Date: Sat, 11 Mar 2017 19:59:31 +0800 Message-ID: <58C3E6A3.1000000@intel.com> References: <1488519630-89058-1-git-send-email-wei.w.wang@intel.com> <1488519630-89058-4-git-send-email-wei.w.wang@intel.com> <20170309141411.GZ16328@bombadil.infradead.org> <58C28FF8.5040403@intel.com> <20170310175349-mutt-send-email-mst@kernel.org> <20170310171143.GA16328@bombadil.infradead.org> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Michael S. Tsirkin" , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, qemu-devel@nongnu.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Liang Li , Paolo Bonzini , Cornelia Huck , Amit Shah , Dave Hansen , Andrea Arcangeli , David Hildenbrand , Liang Li To: Matthew Wilcox Return-path: In-Reply-To: <20170310171143.GA16328@bombadil.infradead.org> Sender: owner-linux-mm@kvack.org List-Id: kvm.vger.kernel.org On 03/11/2017 01:11 AM, Matthew Wilcox wrote: > On Fri, Mar 10, 2017 at 05:58:28PM +0200, Michael S. Tsirkin wrote: >> One of the issues of current balloon is the 4k page size >> assumption. For example if you free a huge page you >> have to split it up and pass 4k chunks to host. >> Quite often host can't free these 4k chunks at all (e.g. >> when it's using huge tlb fs). >> It's even sillier for architectures with base page size >4k. > I completely agree with you that we should be able to pass a hugepage > as a single chunk. Also we shouldn't assume that host and guest have > the same page size. I think we can come up with a scheme that actually > lets us encode that into a 64-bit word, something like this: > > bit 0 clear => bits 1-11 encode a page count, bits 12-63 encode a PFN, page size 4k. > bit 0 set, bit 1 clear => bits 2-12 encode a page count, bits 13-63 encode a PFN, page size 8k > bits 0+1 set, bit 2 clear => bits 3-13 for page count, bits 14-63 for PFN, page size 16k. > bits 0-2 set, bit 3 clear => bits 4-14 for page count, bits 15-63 for PFN, page size 32k > bits 0-3 set, bit 4 clear => bits 5-15 for page count, bits 16-63 for PFN, page size 64k > > That means we can always pass 2048 pages (of whatever page size) in a single chunk. And > we support arbitrary power of two page sizes. I suggest something like this: > > u64 page_to_chunk(struct page *page) > { > u64 chunk = page_to_pfn(page) << PAGE_SHIFT; > chunk |= (1UL << compound_order(page)) - 1; > } > > (note this is a single page of order N, so we leave the page count bits > set to 0, meaning one page). > I'm thinking what if the guest needs to transfer these much physically continuous memory to host: 1GB+2MB+64KB+32KB+16KB+4KB. Is it going to use Six 64-bit chunks? Would it be simpler if we just use the 128-bit chunk format (we can drop the previous normal 64-bit format)? Best, Wei -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755630AbdCKL6V (ORCPT ); Sat, 11 Mar 2017 06:58:21 -0500 Received: from mga01.intel.com ([192.55.52.88]:10824 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755348AbdCKL6O (ORCPT ); Sat, 11 Mar 2017 06:58:14 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,146,1486454400"; d="scan'208";a="1107304135" Message-ID: <58C3E6A3.1000000@intel.com> Date: Sat, 11 Mar 2017 19:59:31 +0800 From: Wei Wang User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131028 Thunderbird/17.0.10 MIME-Version: 1.0 To: Matthew Wilcox CC: "Michael S. Tsirkin" , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, qemu-devel@nongnu.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Liang Li , Paolo Bonzini , Cornelia Huck , Amit Shah , Dave Hansen , Andrea Arcangeli , David Hildenbrand , Liang Li Subject: Re: [PATCH v7 kernel 3/5] virtio-balloon: implementation of VIRTIO_BALLOON_F_CHUNK_TRANSFER References: <1488519630-89058-1-git-send-email-wei.w.wang@intel.com> <1488519630-89058-4-git-send-email-wei.w.wang@intel.com> <20170309141411.GZ16328@bombadil.infradead.org> <58C28FF8.5040403@intel.com> <20170310175349-mutt-send-email-mst@kernel.org> <20170310171143.GA16328@bombadil.infradead.org> In-Reply-To: <20170310171143.GA16328@bombadil.infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/11/2017 01:11 AM, Matthew Wilcox wrote: > On Fri, Mar 10, 2017 at 05:58:28PM +0200, Michael S. Tsirkin wrote: >> One of the issues of current balloon is the 4k page size >> assumption. For example if you free a huge page you >> have to split it up and pass 4k chunks to host. >> Quite often host can't free these 4k chunks at all (e.g. >> when it's using huge tlb fs). >> It's even sillier for architectures with base page size >4k. > I completely agree with you that we should be able to pass a hugepage > as a single chunk. Also we shouldn't assume that host and guest have > the same page size. I think we can come up with a scheme that actually > lets us encode that into a 64-bit word, something like this: > > bit 0 clear => bits 1-11 encode a page count, bits 12-63 encode a PFN, page size 4k. > bit 0 set, bit 1 clear => bits 2-12 encode a page count, bits 13-63 encode a PFN, page size 8k > bits 0+1 set, bit 2 clear => bits 3-13 for page count, bits 14-63 for PFN, page size 16k. > bits 0-2 set, bit 3 clear => bits 4-14 for page count, bits 15-63 for PFN, page size 32k > bits 0-3 set, bit 4 clear => bits 5-15 for page count, bits 16-63 for PFN, page size 64k > > That means we can always pass 2048 pages (of whatever page size) in a single chunk. And > we support arbitrary power of two page sizes. I suggest something like this: > > u64 page_to_chunk(struct page *page) > { > u64 chunk = page_to_pfn(page) << PAGE_SHIFT; > chunk |= (1UL << compound_order(page)) - 1; > } > > (note this is a single page of order N, so we leave the page count bits > set to 0, meaning one page). > I'm thinking what if the guest needs to transfer these much physically continuous memory to host: 1GB+2MB+64KB+32KB+16KB+4KB. Is it going to use Six 64-bit chunks? Would it be simpler if we just use the 128-bit chunk format (we can drop the previous normal 64-bit format)? Best, Wei From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:42707) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cmff9-0006Kt-2v for qemu-devel@nongnu.org; Sat, 11 Mar 2017 06:58:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cmff5-0000wC-5U for qemu-devel@nongnu.org; Sat, 11 Mar 2017 06:58:19 -0500 Received: from mga02.intel.com ([134.134.136.20]:25453) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cmff4-0000w0-U3 for qemu-devel@nongnu.org; Sat, 11 Mar 2017 06:58:15 -0500 Message-ID: <58C3E6A3.1000000@intel.com> Date: Sat, 11 Mar 2017 19:59:31 +0800 From: Wei Wang MIME-Version: 1.0 References: <1488519630-89058-1-git-send-email-wei.w.wang@intel.com> <1488519630-89058-4-git-send-email-wei.w.wang@intel.com> <20170309141411.GZ16328@bombadil.infradead.org> <58C28FF8.5040403@intel.com> <20170310175349-mutt-send-email-mst@kernel.org> <20170310171143.GA16328@bombadil.infradead.org> In-Reply-To: <20170310171143.GA16328@bombadil.infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH v7 kernel 3/5] virtio-balloon: implementation of VIRTIO_BALLOON_F_CHUNK_TRANSFER List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Matthew Wilcox Cc: "Michael S. Tsirkin" , virtio-dev@lists.oasis-open.org, kvm@vger.kernel.org, qemu-devel@nongnu.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Liang Li , Paolo Bonzini , Cornelia Huck , Amit Shah , Dave Hansen , Andrea Arcangeli , David Hildenbrand , Liang Li On 03/11/2017 01:11 AM, Matthew Wilcox wrote: > On Fri, Mar 10, 2017 at 05:58:28PM +0200, Michael S. Tsirkin wrote: >> One of the issues of current balloon is the 4k page size >> assumption. For example if you free a huge page you >> have to split it up and pass 4k chunks to host. >> Quite often host can't free these 4k chunks at all (e.g. >> when it's using huge tlb fs). >> It's even sillier for architectures with base page size >4k. > I completely agree with you that we should be able to pass a hugepage > as a single chunk. Also we shouldn't assume that host and guest have > the same page size. I think we can come up with a scheme that actually > lets us encode that into a 64-bit word, something like this: > > bit 0 clear => bits 1-11 encode a page count, bits 12-63 encode a PFN, page size 4k. > bit 0 set, bit 1 clear => bits 2-12 encode a page count, bits 13-63 encode a PFN, page size 8k > bits 0+1 set, bit 2 clear => bits 3-13 for page count, bits 14-63 for PFN, page size 16k. > bits 0-2 set, bit 3 clear => bits 4-14 for page count, bits 15-63 for PFN, page size 32k > bits 0-3 set, bit 4 clear => bits 5-15 for page count, bits 16-63 for PFN, page size 64k > > That means we can always pass 2048 pages (of whatever page size) in a single chunk. And > we support arbitrary power of two page sizes. I suggest something like this: > > u64 page_to_chunk(struct page *page) > { > u64 chunk = page_to_pfn(page) << PAGE_SHIFT; > chunk |= (1UL << compound_order(page)) - 1; > } > > (note this is a single page of order N, so we leave the page count bits > set to 0, meaning one page). > I'm thinking what if the guest needs to transfer these much physically continuous memory to host: 1GB+2MB+64KB+32KB+16KB+4KB. Is it going to use Six 64-bit chunks? Would it be simpler if we just use the 128-bit chunk format (we can drop the previous normal 64-bit format)? Best, Wei