From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C0E5C7618B for ; Wed, 24 Jul 2019 16:30:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0EAFC22387 for ; Wed, 24 Jul 2019 16:30:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1563985819; bh=aXnST1Zy+5dz8CWNsZfcAqojaBnHYTD1OJuzvi9y2o4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:List-ID:From; b=Zaf16RHxg0iQEZt0PYV+XsIL8Usa6eBWb90naqpggOdeeMDm+K2uOh39FwYDxHOhd 1Rg7esyd8fPX/klZYurI8pRdCSf+Bmzufj6ZKy5A+Gwa6QOsD5Wx3si0uVDmhxCKB4 JyhKvdYbYHrFaC7ltzLLFRhjrr4flu0Bfxt7MgZo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726997AbfGXQaR (ORCPT ); Wed, 24 Jul 2019 12:30:17 -0400 Received: from mail.kernel.org ([198.145.29.99]:47086 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725826AbfGXQaR (ORCPT ); Wed, 24 Jul 2019 12:30:17 -0400 Received: from sol.localdomain (c-24-5-143-220.hsd1.ca.comcast.net [24.5.143.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AC44A21871; Wed, 24 Jul 2019 16:30:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1563985816; bh=aXnST1Zy+5dz8CWNsZfcAqojaBnHYTD1OJuzvi9y2o4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=sxl6AmiyJogEutNkMx10bst2DtY9JXT9gevPqUof4Qqo6HT+y160gFdVne+tUl+/C 8V5Cjhpk8lxry7d5JIRYedRvu2W5dfAv7kaKt3gVkoEbpo3Cix1mZSwcTtlmtHZDKI hjuujCEhJIvlBksjwJOOhFW6fQhlXigF4uZem8M0= Date: Wed, 24 Jul 2019 09:30:14 -0700 From: Eric Biggers To: Eric Dumazet Cc: Dmitry Vyukov , netdev@vger.kernel.org, "David S. Miller" , Florian Westphal , Ilya Maximets , Eric Dumazet , David Ahern , linux-kernel@vger.kernel.org, syzkaller-bugs@googlegroups.com Subject: Re: Reminder: 99 open syzbot bugs in net subsystem Message-ID: <20190724163014.GC673@sol.localdomain> Mail-Followup-To: Eric Dumazet , Dmitry Vyukov , netdev@vger.kernel.org, "David S. Miller" , Florian Westphal , Ilya Maximets , Eric Dumazet , David Ahern , linux-kernel@vger.kernel.org, syzkaller-bugs@googlegroups.com References: <20190724013813.GB643@sol.localdomain> <63f12327-dd4b-5210-4de2-705af6bc4ba4@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <63f12327-dd4b-5210-4de2-705af6bc4ba4@gmail.com> User-Agent: Mutt/1.12.1 (2019-06-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 24, 2019 at 08:39:05AM +0200, Eric Dumazet wrote: > > > On 7/24/19 3:38 AM, Eric Biggers wrote: > > [This email was generated by a script. Let me know if you have any suggestions > > to make it better, or if you want it re-generated with the latest status.] > > > > Of the currently open syzbot reports against the upstream kernel, I've manually > > marked 99 of them as possibly being bugs in the net subsystem. This category > > only includes the networking bugs that I couldn't assign to a more specific > > component (bpf, xfrm, bluetooth, tls, tipc, sctp, wireless, etc.). I've listed > > these reports below, sorted by an algorithm that tries to list first the reports > > most likely to be still valid, important, and actionable. > > > > Of these 99 bugs, 17 were seen in mainline in the last week. > > > > Of these 99 bugs, 4 were bisected to commits from the following people: > > > > Florian Westphal > > Ilya Maximets > > Eric Dumazet > > David Ahern > > > > If you believe a bug is no longer valid, please close the syzbot report by > > sending a '#syz fix', '#syz dup', or '#syz invalid' command in reply to the > > original thread, as explained at https://goo.gl/tpsmEJ#status > > > > If you believe I misattributed a bug to the net subsystem, please let me know, > > and if possible forward the report to the correct people or mailing list. > > > > Some of the bugs have been fixed already, before syzbot found them. > > Why force human to be gentle to bots and actually replying to them ? > > I usually simply wait that syzbot is finding the bug does not repro anymore, > but now if you send these emails, we will have even more pressure on us. > First, based on experience, I'd guess about 30-45 of these are still valid. 17 were seen in mainline in the last week, but some others are valid too. The ones most likely to still be valid are at the beginning of the list. So let's try not use the presence of outdated bugs as an excuse not to fix current bugs. Second, all these bug reports are still open, regardless of whether reminders are sent or not. I think you're really suggesting that possibly outdated bug reports should be automatically invalidated by syzbot. syzbot already does that for bugs with no reproducer. However, that still leaves a lot of outdated bugs with reproducers. Since the kernel community is basically in continuous bug bankruptcy and lots of syzbot reports are being ignored anyway, I'm in favor of making the invalidation criteria more aggressive, so we can best focus people's efforts. I understand that Dmitry has been against this though, since a significant fraction of bugs that syzbot stopped hitting for some reason actually turn out to be still valid. But we probably have no choice. So I suggest we agree on new criteria for invalidating bugs. I'd suggest assigning a timeout to each bug, based on attributes like "seen in mainline?", "reproducer type", "bisected?", "does it look like a 'bad' crash (e.g. use-after-free)"; similar to the algorithm I'm using to sort the bugs when sorting these reminders. I.e., bugs most likely to still be valid, important, and actionable get longest timeouts. Then if no crash or activity was seen in the timeout, the bug is closed. Any thoughts from anyone? - Eric