From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD244C433DB for ; Mon, 11 Jan 2021 13:51:56 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 94DEA2222B for ; Mon, 11 Jan 2021 13:51:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94DEA2222B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:42884 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kyxbv-0003PM-Ox for qemu-devel@archiver.kernel.org; Mon, 11 Jan 2021 08:51:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52750) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kyxUN-0002j8-KF for qemu-devel@nongnu.org; Mon, 11 Jan 2021 08:44:07 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29114) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kyxUE-00082i-Va for qemu-devel@nongnu.org; Mon, 11 Jan 2021 08:44:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610372633; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tpXo9NxnStxZTiwWw1rETdyPzvAR8Hk9md1s67byzNs=; b=ZprC77DCHLp4aznZd/jARBQ2rrD/y5mFcXGeKkwZa0tP9z+K0BGDSuJ/zyAkroXsEHNPYL Xb4IrbLAW7K3PQiXLZGvMBd1NWmOizm8MUWmmeAeK7DzDP1jKd+hFNrBZxanQTeM8R/3D/ /eo1YL7jZwN1oapNdAu7wfHIGLQ4J/8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-399-3zid_wITNP6NHd4mnMRIOw-1; Mon, 11 Jan 2021 08:43:49 -0500 X-MC-Unique: 3zid_wITNP6NHd4mnMRIOw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7AE69107ACF8; Mon, 11 Jan 2021 13:43:48 +0000 (UTC) Received: from thuth.com (ovpn-112-147.ams2.redhat.com [10.36.112.147]) by smtp.corp.redhat.com (Postfix) with ESMTP id 51B0B17C5F; Mon, 11 Jan 2021 13:43:46 +0000 (UTC) From: Thomas Huth To: qemu-devel@nongnu.org, Peter Maydell Subject: [PULL 04/15] fuzz: accelerate non-crash detection Date: Mon, 11 Jan 2021 14:43:17 +0100 Message-Id: <20210111134328.157775-5-thuth@redhat.com> In-Reply-To: <20210111134328.157775-1-thuth@redhat.com> References: <20210111134328.157775-1-thuth@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=thuth@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII" Received-SPF: pass client-ip=216.205.24.124; envelope-from=thuth@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.251, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander Bulekov , Warner Losh , Qiuhao Li Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" From: Qiuhao Li We spend much time waiting for the timeout program during the minimization process until it passes a time limit. This patch hacks the CLOSED (indicates the redirection file closed) notification in QTest's output if it doesn't crash. Test with quadrupled trace input at: https://bugs.launchpad.net/qemu/+bug/1890333/comments/1 Original version: real 1m37.246s user 0m13.069s sys 0m8.399s Refined version: real 0m45.904s user 0m16.874s sys 0m10.042s Note: Sometimes the mutated or the same trace may trigger a different crash summary (second-to-last line) but indicates the same bug. For example, Bug 1910826 [1], which will trigger a stack overflow, may output summaries like: SUMMARY: AddressSanitizer: stack-overflow /home/qiuhao/hack/qemu/build/../softmmu/physmem.c:488 in flatview_do_translate or SUMMARY: AddressSanitizer: stack-overflow (/home/qiuhao/hack/qemu/build/qemu-system-i386+0x27ca049) in __asan_memcpy Etc. If we use the whole summary line as the token, we may be prevented from further minimization. So in this patch, we only use the first three words which indicate the type of crash: SUMMARY: AddressSanitizer: stack-overflow [1] https://bugs.launchpad.net/qemu/+bug/1910826 Signed-off-by: Qiuhao Li Reviewed-by: Alexander Bulekov Tested-by: Alexander Bulekov Message-Id: Signed-off-by: Thomas Huth --- scripts/oss-fuzz/minimize_qtest_trace.py | 44 +++++++++++++++++------- 1 file changed, 31 insertions(+), 13 deletions(-) diff --git a/scripts/oss-fuzz/minimize_qtest_trace.py b/scripts/oss-fuzz/minimize_qtest_trace.py index 5e405a0d5f..a28913a2a7 100755 --- a/scripts/oss-fuzz/minimize_qtest_trace.py +++ b/scripts/oss-fuzz/minimize_qtest_trace.py @@ -29,8 +29,14 @@ whether the crash occred. Optionally, manually set a string that idenitifes the crash by setting CRASH_TOKEN= """.format((sys.argv[0]))) +deduplication_note = """\n\ +Note: While trimming the input, sometimes the mutated trace triggers a different +type crash but indicates the same bug. Under this situation, our minimizer is +incapable of recognizing and stopped from removing it. In the future, we may +use a more sophisticated crash case deduplication method. +\n""" + def check_if_trace_crashes(trace, path): - global CRASH_TOKEN with open(path, "w") as tracefile: tracefile.write("".join(trace)) @@ -41,18 +47,31 @@ def check_if_trace_crashes(trace, path): trace_path=path), shell=True, stdin=subprocess.PIPE, - stdout=subprocess.PIPE) - stdo = rc.communicate()[0] - output = stdo.decode('unicode_escape') - if rc.returncode == 137: # Timed Out - return False - if len(output.splitlines()) < 2: - return False - + stdout=subprocess.PIPE, + encoding="utf-8") + global CRASH_TOKEN if CRASH_TOKEN is None: - CRASH_TOKEN = output.splitlines()[-2] - - return CRASH_TOKEN in output + try: + outs, _ = rc.communicate(timeout=5) + CRASH_TOKEN = " ".join(outs.splitlines()[-2].split()[0:3]) + except subprocess.TimeoutExpired: + print("subprocess.TimeoutExpired") + return False + print("Identifying Crashes by this string: {}".format(CRASH_TOKEN)) + global deduplication_note + print(deduplication_note) + return True + + for line in iter(rc.stdout.readline, ""): + if "CLOSED" in line: + return False + if CRASH_TOKEN in line: + return True + + print("\nWarning:") + print(" There is no 'CLOSED'or CRASH_TOKEN in the stdout of subprocess.") + print(" Usually this indicates a different type of crash.\n") + return False def minimize_trace(inpath, outpath): @@ -66,7 +85,6 @@ def minimize_trace(inpath, outpath): print("Crashed in {} seconds".format(end-start)) TIMEOUT = (end-start)*5 print("Setting the timeout for {} seconds".format(TIMEOUT)) - print("Identifying Crashes by this string: {}".format(CRASH_TOKEN)) i = 0 newtrace = trace[:] -- 2.27.0