From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99344C2D0A3 for ; Thu, 29 Oct 2020 07:49:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DE4DE20EDD for ; Thu, 29 Oct 2020 07:49:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VjV4I+bu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE4DE20EDD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EC48D6B005C; Thu, 29 Oct 2020 03:49:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E74F46B0062; Thu, 29 Oct 2020 03:49:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D64766B0068; Thu, 29 Oct 2020 03:49:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id AA3EB6B005C for ; Thu, 29 Oct 2020 03:49:18 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2E4D93622 for ; Thu, 29 Oct 2020 07:49:18 +0000 (UTC) X-FDA: 77424187596.01.oil53_3a10a902728b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 107AE10049A53 for ; Thu, 29 Oct 2020 07:49:18 +0000 (UTC) X-HE-Tag: oil53_3a10a902728b X-Filterd-Recvd-Size: 4186 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Thu, 29 Oct 2020 07:49:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1603957756; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YLhofHK1Sb/YOJxGC+CQPMO1TDqN1agIRTokGdH7Zn4=; b=VjV4I+bu76VmeNEyWXiI5ir1SKtpqMaZ3Rydq1wei2uzWmi0kX5rfEtgJi0k+HHWBbJak5 JrmoyDhAfGKANxjD50Z7ufZ56F9PoCC4bRKKxjt1uiTtZeluPjAbJrFrz1HN2nVXU/UqZ/ VL9PIZswkIBqlQRqUJxoKkdLvVqVaZA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-BvaEgVzcP1aW1CB9H7UtOw-1; Thu, 29 Oct 2020 03:49:14 -0400 X-MC-Unique: BvaEgVzcP1aW1CB9H7UtOw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1639D803F51; Thu, 29 Oct 2020 07:49:13 +0000 (UTC) Received: from [10.36.112.181] (ovpn-112-181.ams2.redhat.com [10.36.112.181]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7132362A0B; Thu, 29 Oct 2020 07:49:11 +0000 (UTC) Subject: Re: [RFC PATCH 0/3] Allocate memmap from hotadded memory (per device) To: Mike Kravetz , Oscar Salvador Cc: mhocko@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, vbabka@suse.cz, pasha.tatashin@soleen.com References: <20201022125835.26396-1-osalvador@suse.de> <3c18e078-25df-3fd4-9988-1b7677d8e05f@redhat.com> <20201027154031.GA11489@linux> <20201027155851.GA11785@linux> <259889fb-f01b-9537-7948-f1a75a372617@oracle.com> From: David Hildenbrand Organization: Red Hat GmbH Message-ID: Date: Thu, 29 Oct 2020 08:49:10 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: <259889fb-f01b-9537-7948-f1a75a372617@oracle.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 28.10.20 19:47, Mike Kravetz wrote: > On 10/27/20 8:58 AM, Oscar Salvador wrote: >> On Tue, Oct 27, 2020 at 04:44:33PM +0100, David Hildenbrand wrote: >>> I'm planning on looking into patch #2/3 later this or next week (this week >>> is open source summit / KVM Forum). >> >> Sure, aprecciated the time ;-) >> >>> >>> One thing to look into right now is how to make this fly this with vmemmap >>> optimizations for hugetlb pages. >>> >>> https://lkml.kernel.org/r/20201026145114.59424-1-songmuchun@bytedance.com >> >> I was about to have a look at that series eitherway, but good you mentioned. >> > > More eyes on that series would be appreciated. > > That series will dynamically free and allocate memmap pages as hugetlb > pages are allocated or freed. I haven't looked through this series, but > my first thought is that we would need to ensure those allocs/frees are > directed to the device. Not sure if there are interfaces for that. Directing to the device might be part of the solution, but does not have to be. You really want to free the pages to the OS in the end, otherwise you lose the whole benefit of the vmemmap optimization. You would want to actually free the pages (making sure whatever generic_online_page() does was done to these special vmemmap pages). But then, you cannot simply skip all X first pages of a memory block when offlining, you can only skip the once that are still vmemmap pages (e.g., marked via page type), and have to isolate/migrate off the no-longer vmemmap pages. -- Thanks, David / dhildenb