From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C5EDC47E49 for ; Tue, 22 Oct 2019 06:29:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A96B32184C for ; Tue, 22 Oct 2019 06:29:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Kt8RFytr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A96B32184C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 17E7C6B0007; Tue, 22 Oct 2019 02:29:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 154886B0008; Tue, 22 Oct 2019 02:29:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0433A6B000A; Tue, 22 Oct 2019 02:29:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id D573C6B0007 for ; Tue, 22 Oct 2019 02:29:48 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 58D9B181AEF3F for ; Tue, 22 Oct 2019 06:29:48 +0000 (UTC) X-FDA: 76070444856.04.sky06_72a99fbfcc152 X-HE-Tag: sky06_72a99fbfcc152 X-Filterd-Recvd-Size: 5571 Received: from mail-ot1-f68.google.com (mail-ot1-f68.google.com [209.85.210.68]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Oct 2019 06:29:47 +0000 (UTC) Received: by mail-ot1-f68.google.com with SMTP id s22so13193728otr.6 for ; Mon, 21 Oct 2019 23:29:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kVuB3ZdcTD312TKFiKblRZFbqKpk6CiWqdGLygNSyYI=; b=Kt8RFytrfhZZoo4M+/oorpPr26q0SJUIGT1ad0V21N9+zN85w4W6+ckVjDTUBlPKem 9BhJnufmd+S0x/PDNMwU/9jbEGeoBr17AF33tjrcAtsFL8hTKXat7oNMQfpdjaeuF/qK A4GWmf0dFMu3BdkLwtFCxcUAxu6Tv/ODrnhUKicGjSnD9EFLEkTj4Cb7PMJBTtKtWfTI 8ozI+2RNmv2PINYhQrptvvBBrdW6ZxH7hTl6g/CwKdI2C4Ax2BrDfW6DMvfMAR55uNL4 +J+D5iQNyIyLzsNdds/TaAC6+D3RLo4GiU+GeQkjh+GLv58y/RfXQWDqmX5f4rwzjMJx m/Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kVuB3ZdcTD312TKFiKblRZFbqKpk6CiWqdGLygNSyYI=; b=oSPEIDjoq8nXG2aHEffyWh75k4SelfR9ss8xzGRkFSGfUBMLX8mEhFtyYtPDpJ/YC5 ngUSZmjWn4lFjUBSPm0wxV6GGMZYy2ayRNNxxOjGkXzmlY/Z1UoknprE9XRTjTuT2itf I8wte2YibwdrlqUCg99LxyXaVTtYTyfGfiPaq0eg8FVAHRAI2RhmKtFJd5qKzphH7qPS N3VwMwH0PZGSkX2PleqjoT3ZjnvrMhpPQpqu4fX555X4TEZXazvXIDSD6VF2JhoLCl+U m4CUPxQo5nKAicJpSHrOmXjRd15Pb9MMNgyFicE49WY+3bqxe8/hAYF9y4MQ+t/rxQ1O oFRQ== X-Gm-Message-State: APjAAAWirfDw1JiZRlhzrVRz5VfFRXxOl12OEckA9ByKBIZnRtprP7To nbghGXSx/t5lL0FW99zD2CoTDV9bEnzu74BrPr8= X-Google-Smtp-Source: APXvYqzaxJrbzIMlPV3k+T7wY+FXiNPnEDMD07rPiY7Uh9rrYJMtaW2RPKKderc1uhnmHj42l57t1SwrmQcjoiLDf8M= X-Received: by 2002:a9d:1a5:: with SMTP id e34mr1442226ote.286.1571725786985; Mon, 21 Oct 2019 23:29:46 -0700 (PDT) MIME-Version: 1.0 References: <20190925050649.14926-1-bharata@linux.ibm.com> <20190925050649.14926-3-bharata@linux.ibm.com> <20191018030049.GA907@oak.ozlabs.ibm.com> In-Reply-To: <20191018030049.GA907@oak.ozlabs.ibm.com> From: Bharata B Rao Date: Tue, 22 Oct 2019 11:59:35 +0530 Message-ID: Subject: Re: [PATCH v9 2/8] KVM: PPC: Move pages between normal and secure memory To: Paul Mackerras Cc: Bharata B Rao , linuxram@us.ibm.com, cclaudio@linux.ibm.com, kvm-ppc@vger.kernel.org, linux-mm@kvack.org, jglisse@redhat.com, Aneesh Kumar , paulus@au1.ibm.com, sukadev@linux.vnet.ibm.com, linuxppc-dev , Christoph Hellwig Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Oct 18, 2019 at 8:31 AM Paul Mackerras wrote: > > On Wed, Sep 25, 2019 at 10:36:43AM +0530, Bharata B Rao wrote: > > Manage migration of pages betwen normal and secure memory of secure > > guest by implementing H_SVM_PAGE_IN and H_SVM_PAGE_OUT hcalls. > > > > H_SVM_PAGE_IN: Move the content of a normal page to secure page > > H_SVM_PAGE_OUT: Move the content of a secure page to normal page > > > > Private ZONE_DEVICE memory equal to the amount of secure memory > > available in the platform for running secure guests is created. > > Whenever a page belonging to the guest becomes secure, a page from > > this private device memory is used to represent and track that secure > > page on the HV side. The movement of pages between normal and secure > > memory is done via migrate_vma_pages() using UV_PAGE_IN and > > UV_PAGE_OUT ucalls. > > As we discussed privately, but mentioning it here so there is a > record: I am concerned about this structure > > > +struct kvmppc_uvmem_page_pvt { > > + unsigned long *rmap; > > + struct kvm *kvm; > > + unsigned long gpa; > > +}; > > which keeps a reference to the rmap. The reference could become stale > if the memslot is deleted or moved, and nothing in the patch series > ensures that the stale references are cleaned up. I will add code to release the device PFNs when memslot goes away. In fact the early versions of the patchset had this, but it subsequently got removed. > > If it is possible to do without the long-term rmap reference, and > instead find the rmap via the memslots (with the srcu lock held) each > time we need the rmap, that would be safer, I think, provided that we > can sort out the lock ordering issues. All paths except fault handler access rmap[] under srcu lock. Even in case of fault handler, for those faults induced by us (shared page handling, releasing device pfns), we do hold srcu lock. The difficult case is when we fault due to HV accessing a device page. In this case we come to fault hanler with mmap_sem already held and are not in a position to take kvm srcu lock as that would lead to lock order reversal. Given that we have pages mapped in still, I assume memslot can't go away while we access rmap[], so think we should be ok here. However if that sounds fragile, may be I can go back to my initial design where we weren't using rmap[] to store device PFNs. That will increase the memory usage but we give us an easy option to have per-guest mutex to protect concurrent page-ins/outs/faults. Regards, Bharata. -- http://raobharata.wordpress.com/