From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DA552C25B10 for ; Thu, 9 May 2024 12:58:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.719243.1121866 (Exim 4.92) (envelope-from ) id 1s53La-0005dm-Uc; Thu, 09 May 2024 12:58:06 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 719243.1121866; Thu, 09 May 2024 12:58:06 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s53La-0005df-S8; Thu, 09 May 2024 12:58:06 +0000 Received: by outflank-mailman (input) for mailman id 719243; Thu, 09 May 2024 12:58:05 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s53LZ-0005dZ-Fa for xen-devel@lists.xenproject.org; Thu, 09 May 2024 12:58:05 +0000 Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [2a00:1450:4864:20::335]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id c6069408-0e03-11ef-909c-e314d9c70b13; Thu, 09 May 2024 14:58:04 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-41ebcf01013so4839075e9.0 for ; Thu, 09 May 2024 05:58:04 -0700 (PDT) Received: from localhost ([213.195.114.223]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-41f882085c5sm60337195e9.40.2024.05.09.05.58.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 May 2024 05:58:03 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c6069408-0e03-11ef-909c-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1715259484; x=1715864284; darn=lists.xenproject.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=h5e99J8ljbVH9DUR5p4im2Mm3YPtAJnvq7zveHHzS+M=; b=ESE/NQcCW2xHx2iLvKrww0JVw1U9QJ3B1zBUj8scpwEaRd1JrHfLfkaZoZt9MtKCjI tHFdPUkgLDJNkBMQVokGjc9PRmmWD5zoUz6+McSiRi50pP6OOJv9u67WiF3fA4FAfB/C s4kYkuqAkc1fzdkGGfOoprrs+/0iExby/P1zM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715259484; x=1715864284; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=h5e99J8ljbVH9DUR5p4im2Mm3YPtAJnvq7zveHHzS+M=; b=FT2qfL2qB4bvcFZtJDhyuuKTOAvPWMnYEcmY3kVsIXoJSSwyqFsuO7tQ6s0AfGLktk 3smsJxOKBTWgTLbQF7oveNM2I9V/qMHCw/sHcLf+Zb1uKuFx29aZyEn+Clr+wOM1szLY lllbWVd+/H/h8KpREvw+llFDFewj1MaPNCCWfimVgDkctocVB0Wgg+fkrwaSVxoYLewN AgHIYxF1wOacVLJ+8lqL1NfeG7uqxcXLG752g7N3i3GE8bbX4c5bAobEk2kjcLJkjdf6 SxFhrUC64gKKquPu0C58RcFTow6pOcSSocAAZCkaH11xvgrWHgJKJCu+yAOSqBun21hR hwOQ== X-Forwarded-Encrypted: i=1; AJvYcCUNaMV1ce/69NB1xVsjhCapTn0kqAPVUidmVkMiBsiwiM2ZB0dUjJoTX2olErt6EHEDdXuaWXPtVurm9UWYbTa9V1fIk58epyosEpM8ZTc= X-Gm-Message-State: AOJu0YxXxzrFsYCSna41NGiVny0eUpjHjOivcTEAV99z4Ni8aAqlF6SY mtWXNzqpOD5iTNzFXQgsdA6Da6AdcfGktztsAd2mu1LeceIXdZVKMRjNDbO2h44= X-Google-Smtp-Source: AGHT+IEl4HwV7vax8ufWt5nC2P6TE77OFQE//tV2ZP13qmuSURXdM61V8wgDrPDp+ColIL3d0wC6RQ== X-Received: by 2002:a05:600c:a03:b0:41c:13f6:1ee1 with SMTP id 5b1f17b1804b1-41fbc9326bcmr23489705e9.4.1715259483672; Thu, 09 May 2024 05:58:03 -0700 (PDT) Date: Thu, 9 May 2024 14:58:02 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Julien Grall Cc: Luca Fancellu , Xen-devel , Penny Zheng , Stefano Stabellini , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk Subject: Re: [PATCH 3/7] xen/p2m: put reference for superpage Message-ID: References: <20240423082532.776623-1-luca.fancellu@arm.com> <20240423082532.776623-4-luca.fancellu@arm.com> <9F196831-D294-4227-B86F-E8EEACB5B076@arm.com> <0857d348-1305-40d2-9596-e0e5f4490c4a@xen.org> <64648f8c-3eea-47c5-bdc5-6d4fc6531c60@xen.org> <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <37b842c7-c46e-4948-8139-a07bfc2a6f37@xen.org> On Thu, May 09, 2024 at 01:12:00PM +0100, Julien Grall wrote: > Hi, > > On 09/05/2024 12:28, Roger Pau Monné wrote: > > On Thu, May 09, 2024 at 10:50:56AM +0100, Julien Grall wrote: > > > > > > > > > On 09/05/2024 09:13, Roger Pau Monné wrote: > > > > On Wed, May 08, 2024 at 11:11:04PM +0100, Julien Grall wrote: > > > > Also the interactions with the remote domain would need to be audited, > > > > as the remote domain shattering the superpage would need to be > > > > replicated in the mapping side in order to account for the changes. > > > > > > ... I don't understand this one. How is this different from today's where a > > > domain can foreign map a 2MB which may be using a superpage in the remote > > > domain? > > > > Hm, right, I was wrong with that I think, as long as proper references > > as taken for the superpage entries it should be fine. > > > > > > Not sure all paths will be easy to > > > > audit for preemption if it's more than relinquish_p2m_mapping() that > > > > you need to adjust. > > > > > > I thought about it yesterday. But I came to the conclusion that if we have > > > any concern about removing 1GB foreign superpage then we would already have > > > the problem today as a domain can map contiguously 1GB worth of foreign > > > mapping using small pages. > > > > Yeah, but in that case addition or removal is done in 4K chunks, and > > hence we can preempt during the operation. > > I am not entirely sure how that would work. From my understand, today, most > of the users of the P2M code expects the operation to complete in one go and > if preemption is needed then the caller is responsible to handle it by > breaking up the happy. > > With your suggestion, it sounds like you want to rework how the preemption > today and push it to the P2M code. This would mean we would need to modify > all the callers to check for -EERESTART (or similar) and also tell them how > many pages were handled so the call can be restarted where it stopped. Is it > what you had in mind? TBH, I didn't have a specific location in mind about where to do the split. One solution that could simplify it is allowing foreign entries to only be removed by specific functions, so that the split required in order to do the removal can be handled by the caller knowing it's dealing with a foreign map superpage. But that would require the superpage to be shattered, and hence will require creating lower levle leaf entries in order to do the shattering and the removal in 4K chunks. > I don't expect the work to be trivial, so I wonder if this is really worth > it to try to change the way we preempt. > > > > > OTOH for 1GB given the code here the page could be freed in one go, > > without a chance of preempting the operation. > > > > Maybe you have to shatter superpages into 4K entries and then remove > > them individually, as to allow for preemption to be possible by > > calling put_page() for each 4K chunk? > This would require to allocate some pages from the P2M pool for the tables. > As the pool may be exhausted, it could be problematic when relinquishing the > resources. Indeed, it's not ideal. > It may be possible to find a way to have memory available by removing other > mappings first. But it feels a bit hackish and I would rather prefer if we > avoid allocating any memory when relinquishing. Maybe it could be helpful to provide a function to put a superpage, that internally calls free_domheap_pages() with the appropriate order so that freeing a superpage only takes a single free_domheap_pages() call. That could reduce some of the contention around the heap_lock and d->page_alloc_lock locks. Note also that a foreign unmap resulting in a page free is also not the common case, as that should only happen when the foreign domain has been destroyed, or the page ballooned out, so to benchmark the worst case some effort will be needed in order to model this scenario. Thanks, Roger.