From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC41CCA0FE2 for ; Tue, 5 Sep 2023 16:26:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1352129AbjIEQ0d (ORCPT ); Tue, 5 Sep 2023 12:26:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39192 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1354391AbjIELRa (ORCPT ); Tue, 5 Sep 2023 07:17:30 -0400 Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0C591AB for ; Tue, 5 Sep 2023 04:17:26 -0700 (PDT) Received: by mail-io1-xd2c.google.com with SMTP id ca18e2360f4ac-792623074edso64995339f.1 for ; Tue, 05 Sep 2023 04:17:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1693912646; x=1694517446; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QFr3guuenGyb3Qn0aVjZt6j2hEqhzlyTzGXreRyxiz4=; b=F+ZMvUZBpiWyG3uIEkTZKCpeg9BCtLmVqUhKHkL7cTgtT51N8qAehYKwrPQtzJN24D myZuG3l62veFZWZF2vkS/sz3bpIgAJKoLJ4MTuQ/fcvI3tG8qaQ7p5wITayKlisWUqWN D+osdDQWVIuxTiJe/7QL84DRKBOKFvfJg6xWo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693912646; x=1694517446; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QFr3guuenGyb3Qn0aVjZt6j2hEqhzlyTzGXreRyxiz4=; b=j7+2tAp12/vUYmTk5aLOnDoES94dUmMvvn80z2T+MEvEyT/iHLB8//aoQIwG6GnRrb NiHvSs8T54h6VI9htghxGz88CLzvji+E1Yf12VxkR2qvpe/J1VT/t/kgvCGSStJXSI+U M689jRPBOgq1WGfm9+xbb4b7QbLdMRDbSRCeFw1o9Psxb9UmI26ia68zctBIPFz+O/2T pX30c5zCBO5WAJ9NApbpDKXm798ig7EJVvdi//qzFvPHYYDCdDzb3mvnXy3KWfexJgyV dXDE1FVUfqaJQcHr0YGB3ctsxtPiVPEfxcocx9vS/9Baq3YswanBEm0tZbt4RpY8jlsq PQUA== X-Gm-Message-State: AOJu0YwAozJdBftqIvHtNHVbaqpRFqG57g4E7Nn8u8TROdcyewEJzKjr DjYESLqvaMUmDpkR6mFqqkLI73t3/kmPm9eLXWs= X-Google-Smtp-Source: AGHT+IH6ATLWvEQISqMYhYAvtisc3efnqHLoJKTqFihvNieOXuIdabwt3JbtmE5N7gFtoTpoNI0Ylw== X-Received: by 2002:a05:6602:19c5:b0:794:eb37:b0da with SMTP id ba5-20020a05660219c500b00794eb37b0damr10714307iob.2.1693912645977; Tue, 05 Sep 2023 04:17:25 -0700 (PDT) Received: from localhost (156.190.123.34.bc.googleusercontent.com. [34.123.190.156]) by smtp.gmail.com with ESMTPSA id n7-20020a6b7207000000b0077e3acd5ea1sm4221764ioc.53.2023.09.05.04.17.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Sep 2023 04:17:25 -0700 (PDT) Date: Tue, 5 Sep 2023 11:17:25 +0000 From: Joel Fernandes To: Catalin Marinas Cc: Christoph Paasch , Andrew Morton , linux-mm@kvack.org, MPTCP Upstream , rcu@vger.kernel.org Subject: Re: kmemleak handling of kfree_rcu Message-ID: <20230905111725.GA3737639@google.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: rcu@vger.kernel.org On Mon, Sep 04, 2023 at 10:22:56PM +0100, Catalin Marinas wrote: > Hi Christoph, > > Please try not to send html, many servers block such emails. > > Also adding the RCU list. > > On Wed, Aug 30, 2023 at 09:37:23AM -0700, Christoph Paasch wrote: > > for the MPTCP upstream project, we are running syzkaller [1] continuously to > > qualify our kernel changes. > > > > We found one issue with kmemleak and its handling of kfree_rcu. > > > > Specifically, it looks like kmemleak falsely reports memory-leaks when the > > object is being freed by kfree_rcu after a certain grace-period. > > > > For example, https://github.com/multipath-tcp/mptcp_net-next/issues/398# > > issuecomment-1584819482 shows how the syzkaller program reliably produces a > > kmemleak report, although the object is not leaked (we confirmed that by simply > > increasing MSECS_MIN_AGE to something larger than the grace-period). > > Not sure which RCU variant you are using but most likely the false > positives are caused by the original reference to the object being lost > and the pointer added to a new location that kmemleak does not track > (e.g. bnode->records[] in the tree-based variant). > > A quick attempt (untested, not even compiled): > > diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c I am not sure if that works. Correct me if I'm wrong but the issue is that the allocated pointer being RCU-freed is no longer reachable by kmemleak (it is scanned but not reachable), however it might still be reachable via an RCU reader. In such a situation, it is a false-positive. The correct fix then should probably be to mark the object as kmemleak_not_leak() until a grace period elapses. This will cause the object to not be reported but still be scanned until eventually the lower layers will remove the object from kmemleak-tracking after the grace period. Per the docs also, that API is used to prevent false-positives. Instead what the below diff appears to do is to mark the bnode cache as a kmemleak object itself, which smells a bit wrong. The bnode is not an allocated object in the traditional sense, it is simple an internal data structure. That may not solve the issue as the false-positive unreachable object is still unreachable. Or did I misunderstand how kmemleak works? (Quite possible). thanks, - Joel > index 1449cb69a0e0..681a1eb7560a 100644 > --- a/kernel/rcu/tree.c > +++ b/kernel/rcu/tree.c > @@ -53,6 +53,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -2934,6 +2935,7 @@ drain_page_cache(struct kfree_rcu_cpu *krcp) > > llist_for_each_safe(pos, n, page_list) { > free_page((unsigned long)pos); > + kmemleak_free(pos); > freed++; > } > > @@ -2962,8 +2964,16 @@ kvfree_rcu_bulk(struct kfree_rcu_cpu *krcp, > rcu_state.name, bnode->records[i], 0); > > vfree(bnode->records[i]); > + /* avoid false negatives */ > + kmemleak_erase(&bnode->records[i]); > } > } > + /* > + * Avoid kmemleak false negatives when such pointers are later > + * re-allocated. > + */ > + for (i = 0; i < bnode->nr_records; i++) > + kmemleak_erase(&bnode->records[i]); > rcu_lock_release(&rcu_callback_map); > } > > @@ -2972,8 +2982,10 @@ kvfree_rcu_bulk(struct kfree_rcu_cpu *krcp, > bnode = NULL; > raw_spin_unlock_irqrestore(&krcp->lock, flags); > > - if (bnode) > + if (bnode) { > free_page((unsigned long) bnode); > + kmemleak_free(bnode); > + } > > cond_resched_tasks_rcu_qs(); > } > @@ -3241,6 +3253,12 @@ static void fill_page_cache_func(struct work_struct *work) > free_page((unsigned long) bnode); > break; > } > + > + /* > + * Scan the bnode->records[] array until the objects are > + * actually freed. > + */ > + kmemleak_alloc(bnode, PAGE_SIZE, 0, GFP_KERNEL); > } > > atomic_set(&krcp->work_in_progress, 0); > @@ -3308,6 +3326,11 @@ add_ptr_to_bulk_krc_lock(struct kfree_rcu_cpu **krcp, > // scenarios. > bnode = (struct kvfree_rcu_bulk_data *) > __get_free_page(GFP_KERNEL | __GFP_NORETRY | __GFP_NOMEMALLOC | __GFP_NOWARN); > + /* > + * Scan the bnode->records[] array until the objects are > + * actually freed. > + */ > + kmemleak_alloc(bnode, PAGE_SIZE, 0, GFP_KERNEL); > raw_spin_lock_irqsave(&(*krcp)->lock, *flags); > } > > > -- > Catalin