From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CC8DC47422 for ; Thu, 25 Jan 2024 18:52:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8AC64280004; Thu, 25 Jan 2024 13:52:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 85C9F280003; Thu, 25 Jan 2024 13:52:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72409280004; Thu, 25 Jan 2024 13:52:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5DD7B280003 for ; Thu, 25 Jan 2024 13:52:43 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2B13B1408F3 for ; Thu, 25 Jan 2024 18:52:43 +0000 (UTC) X-FDA: 81718729806.30.B116FCF Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP id CC09C1C001B for ; Thu, 25 Jan 2024 18:52:40 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=esL+wozh; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706208761; a=rsa-sha256; cv=none; b=jyr1V9JImD/gdUccggGpFmdiLTBEiBhBSW/s6N9VKb97pO44tu5tC+hAMJmg2osWCAQgwh KoxMsPXoQPEpU5qL17EhzYvEnQsfZS1NWmntORWLdsulFbO350N3QzaYq78BJDiS4s+bzc qYikf6wU07zIHoqqcDPX3OIQnVMFLPw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=esL+wozh; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706208761; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BqjOuyTldugrTOb8d8FI8yfXjNaBBoto2qDgIBvThho=; b=Ov8x+Z0PkbFlkkiTtiULot8srb5ZgVVgwkTxSaTyetijVV68o9PYAFxOO1fiF+HXrj9h4I g1FuLY8uyVUlYRM8hpZ01h/7YejaBz0B+i34pO2a0lUV8xi3uI0xlsW4VNrKnllhUpnpJk quLXAoSHXe4MX8bli443T9mIw7Lrj2s= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=BqjOuyTldugrTOb8d8FI8yfXjNaBBoto2qDgIBvThho=; b=esL+wozh7IBTAOHbhXfHpRjSRF 9IeGfPxsWV1uzqV5FrF3k9KNr/P+VoHd/4tkAEXlfUdc9C1SqsoCNbdgFSSb+Gg8iy/Cyh6BvHxYH KOP0H5DO4ObBHORmH2YyUDNcNeVneouFhIyS7fbarvj4pJxsGKaxxytObiM8vidyPRqlm8bOrSm4r kWS+IIFPUceRnzNYbeRb7u6RMBdGeg7RPkjstFoQ5WS90+x3Q/qKoB0JjJlBN9SXzBG3vC6WhC1iW 6iQx1gSD080A+AHpepEbWbjCV9l3RuUxecLZB4QdXabFsJTDAIH63Gg4gdTiTcLY9ryT3MYpcLdGn 51dmrN0Q==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rT4py-0000000Apgl-44r1; Thu, 25 Jan 2024 18:52:31 +0000 Date: Thu, 25 Jan 2024 18:52:30 +0000 From: Matthew Wilcox To: David Rientjes Cc: John Hubbard , Zi Yan , Bharata B Rao , Dave Jiang , "Aneesh Kumar K.V" , "Huang, Ying" , Alistair Popple , Christoph Lameter , Andrew Morton , Linus Torvalds , Dave Hansen , Mel Gorman , Jon Grimm , Gregory Price , Brian Morris , Wei Xu , Johannes Weiner , linux-mm@kvack.org Subject: Re: [RFC] Memory tiering kernel alignment Message-ID: References: <75f21150-1e12-4f4b-e578-e170e4fea18b@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <75f21150-1e12-4f4b-e578-e170e4fea18b@google.com> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: CC09C1C001B X-Stat-Signature: a6k6azx4jka6ojhi47fgagk3qbazr8ei X-Rspam-User: X-HE-Tag: 1706208760-662762 X-HE-Meta: U2FsdGVkX1/Vw74uvvJlDe8i6834x5LJPLN7QHxreMXfyUMj8V70LVlBT1fG8LPhP71ZuzTrir0MGdJUnI0ObPZ2fjwQK0m7uGYO34WN3GnLV6Ir84KyMMzfvxIbsYILf7wpkdNnMTQUdYnVBgnKDmTs+WsI1zUr5DWw6lJhLiBIgx1iBRKt8j+NECyh/FyPLILdMUyTIg1f/i9wAoWNmN1XfSXAU7/wLGbzBGo1sjlwJT0UF0l6ZkQ+kF6od6IchQ/n93iJGqm+XuebWJ0QfVwmN5JnmHzloixB9tmmLMLk8pGbcDevF1MfgzPNHgIklKZWpbS1n04z+R26RYTeTb3z85CgpL2i6lgOrCfj6zEwUbA5hLv7VAN37lVzIkOS53U4CuK5C+ZVj2PzP9tFrvqM3AShwPKj+HVpv8ZnS+PpNdbPnobE7JnKp+JrTReKhOFVaEV1yp9IU+wWatbvsMj5bWKpE6CZWBIwFAEzDnZj9i9ud8z/vf6kGtjD8buzG9ZJqFZoSPYSvYGAt866h9kB46q4YGT7HDvl9xI4z/6h4yzykBiEy80UUSUktOEBee3w1hvAJNIyOSqRkV3KgssYKg/Zd+Bon7VZfWfDcadQOCkzeBFuk07nv3RcqocAc50/OwRs2CDyPZcD3xwpfN2OE0ocAyqE1/bI/mzZb/enZH/2OVpzdN42ydZnUkKIljeEk0icd0H/qbQTdCMA8cL5Vv89nil5S/af2qYm76JmWMlI/OqHvJ8clzgz1hQmEh4HA+2R2p/Z+HtSw+Z8OcuQV5HAsdCebVfBRrCMQ8NThg9l3waiXoDiB5zuHvyKgILKlFcKvTQQ7SJiczP7h1WdWs7Cr7RnXB1BUm3nUIFjNC9p4Z8nps19f/skKrAX3Q0Dy5EM/jUpsgmqBkcuQiijBi2x5c6ok7K7KISmTi1rRjhWt2T2d1cJVXIATqgPyBcVk+ls5svrjqJi2Xt 9eVuUOA8 YxecXCj1PE0Cr6BZFOK4YDeiVAWtQX5WsB0GHGQGa87hJFkGiTYhWuiDXjW3Mzh6D+XlNPdePrlCFKSNZzSeeXkUzKZ2xFJdTs4JCsP99mNsaPBs0NBtYd/pY+ZG2IXZYOrr89bGHAMCeGF28uzVcZwD6p0cZWi2Qsy/H1HOkXSdA5b+/9/g6NzI/9N6V5zfBTb1ipkgnCYlKHBBSOHVNzF5hYA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jan 25, 2024 at 10:26:19AM -0800, David Rientjes wrote: > There is a lot of excitement around upcoming CXL type 3 memory expansion > devices and their cost savings potential. As the industry starts to > adopt this technology, one of the key components in strategic planning is > how the upstream Linux kernel will support various tiered configurations > to meet various user needs. I think it goes without saying that this is > quite interesting to cloud providers as well as other hyperscalers :) I'm not excited. I'm disappointed that people are falling for this scam. CXL is the ATM of this decade. The protocol is not fit for the purpose of accessing remote memory, adding 10ns just for an encode/decode cycle. Hands up everybody who's excited about memory latency increasing by 17%. Then there are the lies from the vendors who want you to buy switches. Not one of them are willing to guarantee you the worst case latency through their switches. The concept is wrong. Nobody wants to tie all of their machines together into a giant single failure domain. There's no possible redundancy here. Availability is diminished; how do you upgrade firmware on a switch without taking it down? Nobody can answer my contentions about contention either; preventing a single machine from hogging access to a single CXL endpoint seems like an unsolved problem. CXL is great for its real purpose of attaching GPUs and migrating memory back and forth in a software-transparent way. We should support that, and nothing more. We should reject this technology before it harms our kernel and the entire industry. There's a reason that SGI died. Nobody wants to buy single image machines the size of a data centre.