From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nitesh Narayan Lal Subject: Re: [RFC PATCH 0/4] kvm: Report unused guest pages to host Date: Tue, 5 Feb 2019 12:25:11 -0500 Message-ID: <697d3769-9320-a632-749e-56de9474bdf0@redhat.com> References: <20190204181118.12095.38300.stgit@localhost.localdomain> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UNUvqpHAYHR04TsVLVUGxIIA9gwO8kvCl" Cc: rkrcmar@redhat.com, alexander.h.duyck@linux.intel.com, x86@kernel.org, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, pbonzini@redhat.com, tglx@linutronix.de, akpm@linux-foundation.org, Luiz Capitulino , David Hildenbrand , Pankaj Gupta To: Alexander Duyck , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Return-path: In-Reply-To: <20190204181118.12095.38300.stgit@localhost.localdomain> Sender: linux-kernel-owner@vger.kernel.org List-Id: kvm.vger.kernel.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --UNUvqpHAYHR04TsVLVUGxIIA9gwO8kvCl Content-Type: multipart/mixed; boundary="3mzxJ6gNvVUIdJzVqezGb18HN9ja562tQ"; protected-headers="v1" From: Nitesh Narayan Lal To: Alexander Duyck , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: rkrcmar@redhat.com, alexander.h.duyck@linux.intel.com, x86@kernel.org, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, pbonzini@redhat.com, tglx@linutronix.de, akpm@linux-foundation.org, Luiz Capitulino , David Hildenbrand , Pankaj Gupta Message-ID: <697d3769-9320-a632-749e-56de9474bdf0@redhat.com> Subject: Re: [RFC PATCH 0/4] kvm: Report unused guest pages to host References: <20190204181118.12095.38300.stgit@localhost.localdomain> In-Reply-To: <20190204181118.12095.38300.stgit@localhost.localdomain> --3mzxJ6gNvVUIdJzVqezGb18HN9ja562tQ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US On 2/4/19 1:15 PM, Alexander Duyck wrote: > This patch set provides a mechanism by which guests can notify the host= of > pages that are not currently in use. Using this data a KVM host can mor= e > easily balance memory workloads between guests and improve overall syst= em > performance by avoiding unnecessary writing of unused pages to swap. > > In order to support this I have added a new hypercall to provided unuse= d > page hints and made use of mechanisms currently used by PowerPC and s39= 0 > architectures to provide those hints. To reduce the overhead of this ca= ll > I am only using it per huge page instead of of doing a notification per= 4K > page. By doing this we can avoid the expense of fragmenting higher orde= r > pages, and reduce overall cost for the hypercall as it will only be > performed once per huge page. > > Because we are limiting this to huge pages it was necessary to add a > secondary location where we make the call as the buddy allocator can me= rge > smaller pages into a higher order huge page. > > This approach is not usable in all cases. Specifically, when KVM direct= > device assignment is used, the memory for a guest is permanently assign= ed > to physical pages in order to support DMA from the assigned device. In > this case we cannot give the pages back, so the hypercall is disabled b= y > the host. > > Another situation that can lead to issues is if the page were accessed > immediately after free. For example, if page poisoning is enabled the > guest will populate the page *after* freeing it. In this case it does n= ot > make sense to provide a hint about the page being freed so we do not > perform the hypercalls from the guest if this functionality is enabled.= > > My testing up till now has consisted of setting up 4 8GB VMs on a syste= m > with 32GB of memory and 4GB of swap. To stress the memory on the system= I > would run "memhog 8G" sequentially on each of the guests and observe ho= w > long it took to complete the run. The observed behavior is that on the > systems with these patches applied in both the guest and on the host I = was > able to complete the test with a time of 5 to 7 seconds per guest. On a= > system without these patches the time ranged from 7 to 49 seconds per > guest. I am assuming the variability is due to time being spent writing= > pages out to disk in order to free up space for the guest. Hi Alexander, Can you share the host memory usage before and after your run. (In both the cases with your patch-set and without your patch-set) > > --- > > Alexander Duyck (4): > madvise: Expose ability to set dontneed from kernel > kvm: Add host side support for free memory hints > kvm: Add guest side support for free memory hints > mm: Add merge page notifier > > > Documentation/virtual/kvm/cpuid.txt | 4 ++ > Documentation/virtual/kvm/hypercalls.txt | 14 ++++++++ > arch/x86/include/asm/page.h | 25 +++++++++++++++ > arch/x86/include/uapi/asm/kvm_para.h | 3 ++ > arch/x86/kernel/kvm.c | 51 ++++++++++++++++++++++= ++++++++ > arch/x86/kvm/cpuid.c | 6 +++- > arch/x86/kvm/x86.c | 35 +++++++++++++++++++++ > include/linux/gfp.h | 4 ++ > include/linux/mm.h | 2 + > include/uapi/linux/kvm_para.h | 1 + > mm/madvise.c | 13 +++++++- > mm/page_alloc.c | 2 + > 12 files changed, 158 insertions(+), 2 deletions(-) > > -- --=20 Regards Nitesh --3mzxJ6gNvVUIdJzVqezGb18HN9ja562tQ-- --UNUvqpHAYHR04TsVLVUGxIIA9gwO8kvCl Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEkXcoRVGaqvbHPuAGo4ZA3AYyozkFAlxZxvcACgkQo4ZA3AYy ozkEqhAAu4FHz89C2Q2BhyO80m3XgDigU0kqHA+faS1wy8w83dDXh/V4nczbn1Z5 71V8p7YZuD/8LK2yyKk5m62Me5QlHlQIAooBajew4/H60B8aR+kcCEqM7GxtBTqE rKVPDdSws3+9NH53PtODsskZjS2kgX2NHknphl6EYzEzOwrQeO2nxJgNFALnAFz1 bUyUYcEj9gaIYWHRai6kJnjnaIT8FN6u0C7PCjiWxDHKz8Xm3LBvuk2wvgPXPYEQ ctX3vQRttgoYylfthdkWBFQGXk+NCiOx7neGRRgOUcZz/eRdL3vu6BRHQuzS6lKJ TzZWfxTZvKVeYVdP1JGBemrxIA9QqBIKlcGooICQ1MXaFN3sxDd3bq+6zwvCE5Gt VsP3as0v1qX5+Whv2knjUnylamBqZLxS6hd3MHTX8o8ZUtLJXY2QEB6yQrTULy/0 zvjm1xVNxewJEtAes3d5WEmii0XdGogBd9kyfkSCTQJmgP15Ea8gh3mkpIg3CKJv WBpV7acIZLtYiDhrO7knKDvwGOqhCdAfiPgWwACDEf5E+EPLfgkNqi47WcyOylou Un3lME+yCA6xOUIC2Kz1bVnIcvaQE0fDqGtB62fvXN0nMhxhZxSEidU0cO0PLIG5 G1GfmzY2yZVLSJ1yg8FgM5jazu7YMX5Y2oCjvYdZeYIZYmwk4p8= =0CAg -----END PGP SIGNATURE----- --UNUvqpHAYHR04TsVLVUGxIIA9gwO8kvCl--