From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB9AD35B141 for ; Mon, 12 Jan 2026 13:45:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.178 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768225515; cv=none; b=bDCBZdId1I1KvK8HB+ESsd5yuty9/LYQDf+J2VezNfKiBnw1A7NmJJTbaPqQMEBtkDpMIn762H0Ed0JCpSgZmfeAKKFQkZLpZK9F3E4wKbVOrJurSqN1e7Rr8RyYAaVmBA/L7VTT2MS6MNBpk/14NSsF81HbxUYAUPl+Mt55SCY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768225515; c=relaxed/simple; bh=cUHVEQ4ugeQMuGA4/ocXfnET6Ezlx8k5T0UznoHi1+I=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=uo5L6w5dK3yp7z7RjocAl1l2WoHtvOZ7PGWOJKeRMrcHq4xlGdAA4aWWTnjKYyoXtNwgxl9a88de/M3Ud3yHH3L2WPKq69Ns8J2xb3z4lkiEEBXZGNWElm2egF1Nj9YHMFOoQlTtnRat4jbc0hT91yfCScfOlDsiKrgFUysghtc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=GICQ5ZAV; arc=none smtp.client-ip=209.85.160.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="GICQ5ZAV" Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-4ffc5fa443dso32136891cf.2 for ; Mon, 12 Jan 2026 05:45:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1768225513; x=1768830313; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=oQVWjYpPCtwp4dXF89g7hQK6KDriflVp74tvOjx72Jc=; b=GICQ5ZAV2Gr4ghdUMkl1JWB88WIDy84+ElyztjFepUU/xdAmtbj/1y56tkmQmcKRU0 X+/y+LWb+C0pRWfeXjaugN3pS/SQy0OQjA5gpSLyoW/JwG6f2Y4+JallvDWBEWAyUxDI RJtTlQjGTDES1c1bzMb6fz/FCDYlpegNB4E8dsOLqxDFLzcHLs6d81Y6DWfaiMVbCTIn WOKI9rJVPkWfvcCMD5ava1G93AXb4Wvauh1gWyOKAPCYz+MKp8jYws7itoO2wH4yMX5T u5aDzBEqC15GTvQOHI0XQP5dWjnY8CkNB6hw9eBrF2G1tPMumuT+R0yvDtvL/CGagMWV Rn6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768225513; x=1768830313; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oQVWjYpPCtwp4dXF89g7hQK6KDriflVp74tvOjx72Jc=; b=jekAuVebp3A6UqoETc6WSfFDKexT5+Pl89/JCuu1W3n1Qngr4UgH6x/DV87YmKcKMz JG1RmAxITZT/MRJIO3gQUJxnoZ8Rez9aXoC+h50lTIHYi77abxdL6WAEjy8lNtnGQYEh KpbZKSaCNhFZBPZAIT+eHlJaXfM7vNULx930SUIRYTDNiUHHmsg2LbdT2P5Z+7uEH/cx 40lMnGXILdcfHDM+CiR0rASWeaPJ/4LfrUNiglKookki+6DukcFZzNHed0jn+reU2Sbo BtYOFz+V4m5kyDugISIIRsbpJkx5T80aZya59O71y2BU9f9LnE5cAmd/ZowDFJ2qqdVu K+nw== X-Forwarded-Encrypted: i=1; AJvYcCXtuBhrdJHS+RTEPy2MyK4jH67OXyt9qfm3mMjJs3dZFrhyLWnv7M0fsIwupkMYQp+hYZxleLg6zsN9d2s=@vger.kernel.org X-Gm-Message-State: AOJu0YwpnU5hdTKjnnEGzzv7ES3Mdo/Zx6SMS3fbr2qTCNAIgNwUvxhW 2pUUyu209bHwjU85aqS4yi0L5VcTmPdabLlJufOSDFtp6BlgIqA6DQemUmyxaQTli2I= X-Gm-Gg: AY/fxX6uGrDbX680ONmuZ8Ab8Y5rJ1pDbTFc+d5mPmOoVqE5Ua6qIX24VC2RiEYFS5C Wa8BILG3no9Wv+93hHF+a6G5YDLS/UTyF2O2+ZqGsB7jDxyDa4pnLNsipoxNKyj1KB9mJ3mq24b gBURpLSUhgnGgLcKKyjRgtNWsyyFFh7zvpyW2HYsho78o/fhxm2vdVk7zwgWw8EwaXY60C/+8xk 5n8BvwO3/87PeMCncFyQLjC6Ir7+rUtR9pd9gZzgzvMw1aqM3N3BMe3VZeRaekVl5tzipPVhgi2 a+EMUxNMsGgw1nrDw8CTueqJsyHAcjRvR+nFUHM5HNhc7doNfInK/H6vF8RB7rQ8Kt6jO4vMNyw BmMOT3fyVboasJS8zeH1p1XtLKRI0xN/Ee7dVqoo14A0F/j+rCzajpK8bdUOQvsIpqud2+qQOdl IJI9MD2SLRyF7mpGnrFa7EYeKWv//lzLCQoccQbt8ahWlg2XpgnT8K00pYf5a2RfG0hOtp1gnix R0MzQ== X-Google-Smtp-Source: AGHT+IG20+wZaR0cdbZzmqo1CbORIU0XkpxQP0kdASvCc3hNxGR2kbLwFGbi9bRoyfQYpJE34s/36Q== X-Received: by 2002:ac8:7f4e:0:b0:4ed:6803:6189 with SMTP id d75a77b69052e-4ffb49998dfmr285683391cf.53.1768225512643; Mon, 12 Jan 2026 05:45:12 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-112-119.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.112.119]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-4ffa8e36232sm124159891cf.22.2026.01.12.05.45.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jan 2026 05:45:11 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1vfIEI-00000003Q1m-3fo1; Mon, 12 Jan 2026 09:45:10 -0400 Date: Mon, 12 Jan 2026 09:45:10 -0400 From: Jason Gunthorpe To: Zi Yan Cc: Matthew Wilcox , Balbir Singh , Francois Dugast , intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org, Matthew Brost , Madhavan Srinivasan , Nicholas Piggin , Michael Ellerman , "Christophe Leroy (CS GROUP)" , Felix Kuehling , Alex Deucher , Christian =?utf-8?B?S8O2bmln?= , David Airlie , Simona Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Lyude Paul , Danilo Krummrich , Bjorn Helgaas , Logan Gunthorpe , David Hildenbrand , Oscar Salvador , Andrew Morton , Leon Romanovsky , Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Alistair Popple , linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, linux-cxl@vger.kernel.org Subject: Re: [PATCH v4 1/7] mm/zone_device: Add order argument to folio_free callback Message-ID: <20260112134510.GC745888@ziepe.ca> References: <20260111205820.830410-1-francois.dugast@intel.com> <20260111205820.830410-2-francois.dugast@intel.com> <874d29da-2008-47e6-9c27-6c00abbf404a@nvidia.com> <0D532F80-6C4D-4800-9473-485B828B55EC@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <0D532F80-6C4D-4800-9473-485B828B55EC@nvidia.com> On Sun, Jan 11, 2026 at 07:51:01PM -0500, Zi Yan wrote: > On 11 Jan 2026, at 19:19, Balbir Singh wrote: > > > On 1/12/26 08:35, Matthew Wilcox wrote: > >> On Sun, Jan 11, 2026 at 09:55:40PM +0100, Francois Dugast wrote: > >>> The core MM splits the folio before calling folio_free, restoring the > >>> zone pages associated with the folio to an initialized state (e.g., > >>> non-compound, pgmap valid, etc...). The order argument represents the > >>> folio’s order prior to the split which can be used driver side to know > >>> how many pages are being freed. > >> > >> This really feels like the wrong way to fix this problem. > >> > > Hi Matthew, > > I think the wording is confusing, since the actual issue is that: > > 1. zone_device_page_init() calls prep_compound_page() to form a large folio, > 2. but free_zone_device_folio() never reverse the course, > 3. the undo of prep_compound_page() in free_zone_device_folio() needs to > be done before driver callback ->folio_free(), since once ->folio_free() > is called, the folio can be reallocated immediately, > 4. after the undo of prep_compound_page(), folio_order() can no longer provide > the original order information, thus, folio_free() needs that for proper > device side ref manipulation. There is something wrong with the driver if the "folio can be reallocated immediately". The flow generally expects there to be a driver allocator linked to folio_free() 1) Allocator finds free memory 2) zone_device_page_init() allocates the memory and makes refcount=1 3) __folio_put() knows the recount 0. 4) free_zone_device_folio() calls folio_free(), but it doesn't actually need to undo prep_compound_page() because *NOTHING* can use the page pointer at this point. 5) Driver puts the memory back into the allocator and now #1 can happen. It knows how much memory to put back because folio->order is valid from #2 6) #1 happens again, then #2 happens again and the folio is in the right state for use. The successor #2 fully undoes the work of the predecessor #2. If you have races where #1 can happen immediately after #3 then the driver design is fundamentally broken and passing around order isn't going to help anything. If the allocator is using the struct page memory then step #5 should also clean up the struct page with the allocator data before returning it to the allocator. I vaugely remember talking about this before in the context of the Xe driver.. You can't just take an existing VRAM allocator and layer it on top of the folios and have it broadly ignore the folio_free callback. Jsaon