From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA284C2D0FA for ; Wed, 13 May 2020 10:04:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 91EE1206D6 for ; Wed, 13 May 2020 10:04:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="PV5yqOxp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91EE1206D6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linuxfoundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 36DBD900121; Wed, 13 May 2020 06:04:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 31D479000F3; Wed, 13 May 2020 06:04:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 20C33900121; Wed, 13 May 2020 06:04:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 0971D9000F3 for ; Wed, 13 May 2020 06:04:50 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BB5B5181AEF00 for ; Wed, 13 May 2020 10:04:49 +0000 (UTC) X-FDA: 76811261898.14.front18_2a1c7692f5a61 X-HE-Tag: front18_2a1c7692f5a61 X-Filterd-Recvd-Size: 4244 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 May 2020 10:04:49 +0000 (UTC) Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E505A206B8; Wed, 13 May 2020 10:04:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1589364288; bh=BkUzNAvRjDhZ5sh9KsFgyFVKv9jNomu1O2R+MD3pou8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=PV5yqOxp7v0teWZ5crR2ehluwOXrr3DcmlDcWZhM/f7bVtRBh3J3jGfx5jnfBhP2B 176rRIBdxukTGVFgS0qqR0iPcOSHajdIJXAg4RNUMFJJLU3l1lzt9L6CKaKl6WEa5S EoNHAQIZWgoUsUYUmjrmADoa3W7ZzdcmSboXjA8g= Date: Wed, 13 May 2020 12:04:46 +0200 From: "gregkh@linuxfoundation.org" To: "Ravich, Leonid" Cc: "Idgar, Or" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" Subject: Re: CMA enhancement - non-default areas in x86 Message-ID: <20200513100446.GB868852@kroah.com> References: <20200513064755.GA763968@kroah.com> <20200513071413.GB766804@kroah.com> <20200513083343.GA772573@kroah.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 13, 2020 at 09:43:45AM +0000, Ravich, Leonid wrote: > > On Wed, May 13, 2020 at 08:29:16AM +0000, Ravich, Leonid wrote: > > > PCIe NTB > > > Documentation/driver-api/ntb.rst > > > > > 1) Basically PCI bridge between to root complex / PCI switches > > > 2) using out of OS memory is one solution but then this memory is > > > Limited for usage by other stack, ex: get_user_pages on this memory > > > will fail, Therefore attempting to use it for block layer with (o_direct) will > > fail. > > > > > > Acutely any generic stack which attempts to "pin" this memory will fail. > > > > So why isn't the BIOS/UEFI properly reserving this from the general operating > > system's pages so that the driver knows to use them instead? > > > > Is UEFI wrong here about these being valid memory ranges for general use? > > If so, why not fix that? If not, how in the world is the OS supposed to know > > these memory ranges are _not_ for general use? I feel like there is > > something missing here... > > > Maybe I am miss understanding something here , but if BIOS/UEFI will reserve this pages > They will be "out of kernel" which will work for propriety driver but this memory will not > be useable for generic driver which will attempt to pin this memory with get_user_pages() . > so we can go and try to fix that (not sure this is the right way) . What do you mean by "propriety" driver vs. "generic" driver? Shouldn't there be some "generic" way that UEFI tells any driver where these memory locations are that can not be used as general memory? If not, try fixing up UEFI for that. > another option here is to use some kernel infrastructure which from one side reserve the memory from general use > on the other hand kernel will be aware of this pages so get_user_pages() will work on this memory . > > from what we saw CMA infrastructure can support such requirements. CMA needs to be told where to reserve the memory at boot time. If you want to use that, great, but something has to tell it, so perhaps just get that info from UEFI as that is the "equilivant" to a device tree, right? Try it all out and see, all of this is pointless without real patches, which is why we almost never have these kinds of discussions without working code. thanks, greg k-h