From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57119) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aPTbt-0003IS-EZ for qemu-devel@nongnu.org; Sat, 30 Jan 2016 06:22:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aPTbp-0000wy-CQ for qemu-devel@nongnu.org; Sat, 30 Jan 2016 06:22:33 -0500 Date: Sat, 30 Jan 2016 10:11:23 +1100 From: David Gibson Message-ID: <20160129231123.GO23043@voom.redhat.com> References: <1453095881-16704-1-git-send-email-david@gibson.dropbear.id.au> <20160119074817.GA16048@in.ibm.com> <20160119110233.GE27454@voom.redhat.com> <56AA827A.2070905@suse.de> <20160129024712.GG23043@voom.redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="B3NBd8mrXZtPJEYR" Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [RFC 0/3] Draft implementation of HPT resizing (qemu side) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexander Graf Cc: lvivier@redhat.com, thuth@redhat.com, qemu-devel@nongnu.org, qemu-ppc@nongnu.org, Bharata B Rao , paulus@samba.org --B3NBd8mrXZtPJEYR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 29, 2016 at 08:18:39AM +0200, Alexander Graf wrote: >=20 >=20 > > Am 29.01.2016 um 04:47 schrieb David Gibson : > >=20 > >> On Thu, Jan 28, 2016 at 10:04:58PM +0100, Alexander Graf wrote: > >>=20 > >>=20 > >>> On 01/19/2016 12:02 PM, David Gibson wrote: > >>>> On Tue, Jan 19, 2016 at 01:18:17PM +0530, Bharata B Rao wrote: > >>>>> On Mon, Jan 18, 2016 at 04:44:38PM +1100, David Gibson wrote: > >>>>> Here is a draft qemu implementation of my proposed PAPR extension f= or > >>>>> allowing runtime resizing of a KVM/ppc64 guest's hash page table. > >>>>> That in turn will allow for more flexible memory hotplug. > >>>>>=20 > >>>>> This should work with the guest kernel side patches I also posted > >>>>> recently [1]. > >>>>>=20 > >>>>> Still required to make this into a full implementation: > >>>>> * Guest needs to auto-resize HPT on memory hotplug events > >>>>>=20 > >>>>> * qemu needs to allocate HPT size based on current rather than > >>>>> maximum memory if the guest is HPT resize aware > >>>>>=20 > >>>>> * KVM host side implementation > >>>>>=20 > >>>>> * PAPR standardization > >>>> So with the current patchset (QEMU and guest kernel changes), I shou= ld > >>>> be able to change the HTAB size of a PR guest right ? I see the below > >>>> failure though: > >>> Uh.. to be honest I haven't really considered the KVM case at all. > >>> I'm kind of surprised it didn't just refuse to do anything. > >>>=20 > >>>> [root@localhost ~]# cat /sys/kernel/debug/powerpc/pft-size > >>>> 24 > >>>> [root@localhost ~]# echo 26 > /sys/kernel/debug/powerpc/pft-size > >>>> [ 65.996845] lpar: Attempting to resize HPT to shift 26 > >>>> [ 65.996845] lpar: Attempting to resize HPT to shift 26 > >>>> [ 66.113596] lpar: HPT resize to shift 26 complete (109 ms / 6 ms) > >>>> [ 66.113596] lpar: HPT resize to shift 26 complete (109 ms / 6 ms) > >>>>=20 > >>>> PR guest just hangs here while I see tons of below messages in > >>>> the 1st level guest: > >>>>=20 > >>>> KVM can't copy data from 0x3fff99e91400! > >>>> ... > >>>> Couldn't emulate instruction 0x00000000 (op 0 xop 0) > >>>> kvmppc_handle_exit_pr: emulation at 700 failed (00000000) > >>> Hm, not sure why that's happening. At first I thought it was because > >>> we weren't updating SDR1 with the address of the new htab, but that's > >>> actually in there. Maybe the KVM PR code isn't rereading it after > >>> initial VM startup. > >>=20 > >> The KVM PR code doesn't care - it just rereads SDR1 on every pteg look= up ;). > >> There's no caching at all. > >=20 > > Ok, no idea why it's not working then. I'll investigate when I get a c= hance. > >=20 > >> Of course, the guest needs to invalidate all pending tlb entries if th= ey're > >> now invalid. > >>=20 > >> Does this work on real hardware? Say, a G5? > >=20 > > As Paulus says it would be possible to do HPT resizing on real > > hardware, but the implementation I've done is specific to PAPR. And > > obviously qemu wouldn't be relevant to that case. >=20 > So why make it specific to papr? Wouldn't it make sense to have it > as a (ppc) generic interface in Linux? Well, I sort of did, in that I added a ppc_md call for it. I just haven't implemented it for anything other than PAPR yet - the PAPR implementation is quite different from what the native one would be, since the hypervisor needs to handle the rehashing. > For the PR PAPR case, QEMU allocates the HTAB, so it needs to make > sure it pushes the changed address as new fake SDR1 value into kvm > when it changes. Yes, I'm doing that - have a look at the qemu series. Not 100% sure it's correct, since I haven't debugged with PR KVM yet. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --B3NBd8mrXZtPJEYR Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJWq/GbAAoJEGw4ysog2bOSd1YP/2M79p41FjQvy2nJ6zkoL4jk ZuKmTVJO7bXGnpSqTenydPHrPOM5ucZ/D7Lhke5A5itdjOZyLT6ziXz9BAhvGyha 8NjjF81wCfwWovFT2SKINxrVA2v8Qb1739vbJtTgxf+2f+X4aua0wIX274oZxO/e z0Z58Qkv34T+HiGJCOBtFIrQhlWt+ye4ym84/DHahevXQEGFMwaM+ehbVEOYaTF5 HzK7v+K7HjO0f7fiq39+q0O5N0o7h03moVgHVegyPo12z+2sdftd+3p8zZV51lyc zZSjY7eVOKxKaBY4Rp+m9VmGixTVeNZuRMpl4HfRPGhHOYGPRk5dQIoyEf6SLOo7 5EecpilFCiw28zYt7tMdP2umPH8/ILxu12gWqPb5H9RaBkd4+JdqUlqecba4CbK6 W3UNyXvjvJBpb55YrhiTvf1GzECrM9fBcRDthSSIv5yIEMo2ll7mlj/qdMIPNQtv +ooySFOb4nqNe39O4AS4rW4NV6+rK4JKyBSjnmNcndkr174bb8uNUKtTJZB8vlzK 0Y4Mod4CixpbIlBYgWfnpjUDYrZXbuGrnTJhvEU3KRJLius6dlbFINUTu+WJocTr LFGD0EnOUApIbWMpZHXsR63QNmjmvKG94Tt830Q3YvxrqMDhiUuUnLwNa3v5kuqT +AkN2WrVNoUO02TUJWdG =CA/B -----END PGP SIGNATURE----- --B3NBd8mrXZtPJEYR--