From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.haak.id.au (mail.haak.id.au [172.105.183.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E6C7231A23 for ; Mon, 8 Dec 2025 23:08:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=172.105.183.32 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765235320; cv=none; b=XP6RLfQceNtfNpYNMcqKUinnE0mWkYJYBgmfujex9osXrQJlr9l8QeHOGAaNgZtLGsvPOhYz9fx1qUK57j/vS/PmZKfImusNe0zMrTxWMQZQSA7j8EINzjsy2WTPuwMDNEEuU0E8q8EDatfW7j/ry+zEvR2lTxxooSIUawfLjmI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1765235320; c=relaxed/simple; bh=ep/AkNo4mf2W/fYyOOFTnuqzrhnPcuNkJ/nowgM/mew=; h=Date:From:To:Subject:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=HZAWJAxDdOlLE9EYwRpjRt797sbPIEE885FHMKvYIZtIgIaEpQIqo3vbQQLeC0zCI5Q6SWhmn9R0JK/izRzZZ/TpHi9pkOwlE5YB5QOzXOSNJXNd65YcvTwYqVv7oycOWaEO9kqeWNuuU24xVtMgXORGvQz+pVprRDblzq62mIM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=haak.id.au; spf=pass smtp.mailfrom=haak.id.au; dkim=pass (2048-bit key) header.d=haak.id.au header.i=@haak.id.au header.b=j4i1XK7z; arc=none smtp.client-ip=172.105.183.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=haak.id.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=haak.id.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=haak.id.au header.i=@haak.id.au header.b="j4i1XK7z" Received: from xps15mal (180-150-104-78.b49668.bne.static.aussiebb.net [180.150.104.78]) by mail.haak.id.au (Postfix) with ESMTPSA id D37C283349; Tue, 09 Dec 2025 09:08:35 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=haak.id.au; s=202002; t=1765235316; bh=ep/AkNo4mf2W/fYyOOFTnuqzrhnPcuNkJ/nowgM/mew=; h=Date:From:To:Subject:From; b=j4i1XK7zVjLPGYNVxidOGpai7ZKkso5bj5t3MYYhUxX1U99BOkbGYIael5hPJnNcE YZwQDErN7Izscwpxljbvoe8pykVxUX3hvfEcPrMtk84hppQ+9oTG7ymmclsSbpXTbG mqCWQiNFKftOUGiedEP81JXOOZcaW1cGIWHc4Nk9tZ7i7FBZdNd1uiuPOLk5i4eT7g ZhTW+Am6XxJceKF59eqfe+psAC9eb2nPhP7V9Syv0KBG5U9KEFfSL+eZ9k7qclnKg7 mHoSTaBCG8nDpvys8PiljUbhA2Bayza7OgR8R1uOcgTVgw6gTpfOPjRE++MS6eSrde dACcgVG1op46g== Date: Tue, 9 Dec 2025 09:08:31 +1000 From: Mal Haak To: linux-kernel@vger.kernel.org, surenb@google.com, David Wang <00107082@163.com> Subject: Re: Possible memory leak in 6.17.7 Message-ID: <20251209090831.13c7a639@xps15mal> In-Reply-To: <20251208110829.11840-1-00107082@163.com> References: <20251110182008.71e0858b@xps15mal> <20251208110829.11840-1-00107082@163.com> X-Mailer: Claws Mail 4.3.1 (GTK 3.24.51; x86_64-pc-linux-gnu) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Mon, 8 Dec 2025 19:08:29 +0800 David Wang <00107082@163.com> wrote: > On Mon, 10 Nov 2025 18:20:08 +1000 > Mal Haak wrote: > > Hello, > > > > I have found a memory leak in 6.17.7 but I am unsure how to track it > > down effectively. > > > > > > I think the `memory allocation profiling` feature can help. > https://docs.kernel.org/mm/allocation-profiling.html > > You would need to build a kernel with > CONFIG_MEM_ALLOC_PROFILING=y > CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT=y > > And check /proc/allocinfo for the suspicious allocations which take > more memory than expected. > > (I once caught a nvidia driver memory leak.) > > > FYI > David > Thank you for your suggestion. I have some results. Ran the rsync workload for about 9 hours. It started to look like it was happening. # smem -pw Area Used Cache Noncache firmware/hardware 0.00% 0.00% 0.00% kernel image 0.00% 0.00% 0.00% kernel dynamic memory 80.46% 65.80% 14.66% userspace memory 0.35% 0.16% 0.19% free memory 19.19% 19.19% 0.00% # sort -g /proc/allocinfo|tail|numfmt --to=iec 22M 5609 mm/memory.c:1190 func:folio_prealloc 23M 1932 fs/xfs/xfs_buf.c:226 [xfs] func:xfs_buf_alloc_backing_mem 24M 24135 fs/xfs/xfs_icache.c:97 [xfs] func:xfs_inode_alloc 27M 6693 mm/memory.c:1192 func:folio_prealloc 58M 14784 mm/page_ext.c:271 func:alloc_page_ext 258M 129 mm/khugepaged.c:1069 func:alloc_charge_folio 430M 770788 lib/xarray.c:378 func:xas_alloc 545M 36444 mm/slub.c:3059 func:alloc_slab_page 9.8G 2563617 mm/readahead.c:189 func:ractl_alloc_folio 20G 5164004 mm/filemap.c:2012 func:__filemap_get_folio So I stopped the workload and dropped caches to confirm. # echo 3 > /proc/sys/vm/drop_caches # smem -pw Area Used Cache Noncache firmware/hardware 0.00% 0.00% 0.00% kernel image 0.00% 0.00% 0.00% kernel dynamic memory 33.45% 0.09% 33.36% userspace memory 0.36% 0.16% 0.19% free memory 66.20% 66.20% 0.00% # sort -g /proc/allocinfo|tail|numfmt --to=iec 12M 2987 mm/execmem.c:41 func:execmem_vmalloc 12M 3 kernel/dma/pool.c:96 func:atomic_pool_expand 13M 751 mm/slub.c:3061 func:alloc_slab_page 16M 8 mm/khugepaged.c:1069 func:alloc_charge_folio 18M 4355 mm/memory.c:1190 func:folio_prealloc 24M 6119 mm/memory.c:1192 func:folio_prealloc 58M 14784 mm/page_ext.c:271 func:alloc_page_ext 61M 15448 mm/readahead.c:189 func:ractl_alloc_folio 79M 6726 mm/slub.c:3059 func:alloc_slab_page 11G 2674488 mm/filemap.c:2012 func:__filemap_get_folio So if I'm reading this correctly something is causing folios collect and not be able to be freed? Also it's clear that some of the folio's are counting as cache and some aren't. Like I said 6.17 and 6.18 both have the issue. 6.12 does not. I'm now going to manually walk through previous kernel releases and find where it first starts happening purely because I'm having issues building earlier kernels due to rust stuff and other python incompatibilities making doing a git-bisect a bit fun. I'll do it the packages way until I get closer, then solve the build issues. Thanks, Mal