From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4CC2C4320A for ; Thu, 19 Aug 2021 10:06:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5146A61154 for ; Thu, 19 Aug 2021 10:06:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5146A61154 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id CED3A8D0001; Thu, 19 Aug 2021 06:06:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9D7A6B0071; Thu, 19 Aug 2021 06:06:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8BD88D0001; Thu, 19 Aug 2021 06:06:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0034.hostedemail.com [216.40.44.34]) by kanga.kvack.org (Postfix) with ESMTP id 9BF486B006C for ; Thu, 19 Aug 2021 06:06:48 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 374D3822195B for ; Thu, 19 Aug 2021 10:06:48 +0000 (UTC) X-FDA: 78491401296.29.E973B81 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf12.hostedemail.com (Postfix) with ESMTP id B2C3B1005580 for ; Thu, 19 Aug 2021 10:06:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1629367607; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Opv2d9ol21udi4K0v30kbE3GUCegq3iBRM6wdcQpyVk=; b=jIbfNhy+Mef1Fyk3/mmHJqGdW58EkyvyMSoICwAtX3rVwTeYh/QAXkaQiFrV0+AvnUPKvF BaPb24LZBbeqAPGBXR1wvnVYoLu44MyFak0edZhIq9YrguCzr4mrpmJAn8/fqnPM7c/2AR ab0j+Lckvxy8EBw/jbe+wPhhf+ojOSk= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-8z09ZDUANKiY8MWsP89NNw-1; Thu, 19 Aug 2021 06:06:44 -0400 X-MC-Unique: 8z09ZDUANKiY8MWsP89NNw-1 Received: by mail-wr1-f71.google.com with SMTP id y12-20020adfee0c0000b0290154e82fef34so1532064wrn.6 for ; Thu, 19 Aug 2021 03:06:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:organization :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=Opv2d9ol21udi4K0v30kbE3GUCegq3iBRM6wdcQpyVk=; b=ZbexPa5FgNRwvHkjKL1fwYdHFDXnsdjTKutefFLp+0lB0N8jyuSdhignYtS5Z9+3MD wWLbT5r3D1S5dAZb4wvHTkGhPh/UmMzPUOfG4zKdTUS+VbGo6g5I5ESvWs1Xm09uyKvk K9YfvYwrReK6hAPHEOEf/UEgGaL/8pmk0lgmaxkpU39xlBx26O65vy2A0Vm6LEdvFHMI p62i8+7mW95oNdl4FMcJ/PofoRpk84ANDLQmmxJQH6nO7EEBB/iIe1VsCB+juUtDv/PN VHp4k2my9caWQk6BDebLWNXTaT4CLFScazT6kyfgNCPWI84XquYfapuQjw6+XnHuABXw 4CNw== X-Gm-Message-State: AOAM530JWnqD/u+P1BZaAxwHdi63sByBDyG5eZ6fe/PNw3lXrjtasIv9 Z888dQBELy5d/hcQum3WfkLJaVc2DjM/ztotwqIX2ijKx+Kub4mPJO4Gdm/IC/16lRdpwc6Fn06 sRq+1MLmXR8o= X-Received: by 2002:a5d:6483:: with SMTP id o3mr2757135wri.197.1629367602749; Thu, 19 Aug 2021 03:06:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+3/0fv0wKXrwU5A+P76meV3y97/X7hVUmMcENBq24d/DLFeQ19OprL9SSc1+K8XhulZ/xWQ== X-Received: by 2002:a5d:6483:: with SMTP id o3mr2757086wri.197.1629367602415; Thu, 19 Aug 2021 03:06:42 -0700 (PDT) Received: from [192.168.3.132] (p5b0c6bd1.dip0.t-ipconnect.de. [91.12.107.209]) by smtp.gmail.com with ESMTPSA id y1sm2167170wmq.43.2021.08.19.03.06.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 19 Aug 2021 03:06:41 -0700 (PDT) Subject: Re: [PATCH 1/5] mm: Add support for unaccepted memory To: Joerg Roedel Cc: Dave Hansen , Andi Kleen , "Kirill A. Shutemov" , Borislav Petkov , Andy Lutomirski , Sean Christopherson , Andrew Morton , Kuppuswamy Sathyanarayanan , David Rientjes , Vlastimil Babka , Tom Lendacky , Thomas Gleixner , Peter Zijlstra , Paolo Bonzini , Ingo Molnar , Varad Gautam , Dario Faggioli , x86@kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" References: <9748c07c-4e59-89d0-f425-c57f778d1b42@linux.intel.com> <17b6a3a3-bd7d-f57e-8762-96258b16247a@intel.com> <796a4b20-7fa3-3086-efa0-2f728f35ae06@linux.intel.com> <3caf5e73-c104-0057-680c-7851476e67ac@linux.intel.com> <25312492-5d67-e5b0-1a51-b6880f45a550@intel.com> From: David Hildenbrand Organization: Red Hat Message-ID: <39d09b80-f79e-0dab-2be7-31d4db43ab77@redhat.com> Date: Thu, 19 Aug 2021 12:06:40 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: B2C3B1005580 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jIbfNhy+; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf12.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.133.124) smtp.mailfrom=david@redhat.com X-Rspamd-Server: rspam04 X-Stat-Signature: xmt61wrsswbyxwgiw8hcpzqy991rtfpn X-HE-Tag: 1629367607-183404 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 19.08.21 11:55, Joerg Roedel wrote: > Hi David, > > On Tue, Aug 17, 2021 at 05:00:55PM +0200, David Hildenbrand wrote: >> Not sure if already discussed, but what about making sure that free pages >> are not a mixture (partially unaccepted, partially accepted). >> >> You'd have to expose the pages in that granularity to the buddy >> (__free_pages_core), indicating the state. You'd have to reject merging >> pages of differing acceptance state. >> >> Accepting a page would then be handled outside of the zone lock, completely >> controlled by the state. >> >> So a page in the buddy would either be completely accepted or completely >> unaccepted, signaled e.g., by PageOffline(). >> >> Consequently, when allocating a 4KiB page, you'd split an unaccepted 2MiB >> page into separate unaccepted pages. You'd grab one of the unaccepted 4KiB >> pages and accept it before initializing it and handing it out. > > Yes, that is the alternative to over-accepting memory on allocation. But > the problem here is that accepting/validating memory is an expensive > operation which also requires a hypercall. The hypercalls on SNP and TDX > can accept/validate multiple pages in one call. So the recommendation is > to accept memory in bigger chunks, like the 2MB that have been proposed. > The general idea would be to have one thread scanning the free page list and accepting pages in the background. Take a page out, accept it, release it back to the buddy. If we place accepted pages to the front of the free list, allocations would be quite fast. Sure, you'd have some slow early allocations, but I'd guess it really wouldn't make a big impact overall. We'd have to try. It's quite similar to the free page reporting infrastructure, except that with free page reporting we want to place pages to the tail, not to the front. That would be easy to add. > Only accepting memory in allocation size might be too slow, as there is > a lot of code doing order-0 allocations. I think this approach will also > be more intrusive to the page alloctor, as it needs more changes on the > free path to check for acceptance states before pages can be merged. That's already mostly done in this patch IIRC. -- Thanks, David / dhildenb