From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97E4F15D1 for ; Sun, 20 Apr 2025 05:48:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745128106; cv=none; b=jjJMZ0z5PakHhP4l/y76hcL64wG7y5kIdf4t36rAIhYEpAoe7wGwx1n8o+le9BrMXThRWtmmtFx31MVZWXLaavcRziMtd+6d5So2wnQD5eXNKcgpg+i82Tb8CDcgIQQ/iExOnHL15RgovoWhvrz3G6OtmFXpDUT4vmmIkVHqIHg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745128106; c=relaxed/simple; bh=V6evirPmNhudxE1mjbEqg6RgIGIJbnfQihlkDSkHIyo=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=P1whH0W3HwYeJkBmOtnjI9ZknBE9zvfMDCd/urkj524F9awDFdRfRwyMt36sVVvDZUZf8EMF06mnTSyrAdtOCPaheOku9aGKyRu+SbjHL5IqbemJzFoH2rgHrNjvSR8L7u8arZsuGUpntGUYP/NhzN77mwMpMv5lhzS3ty5qh70= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=bs9MYK65; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="bs9MYK65" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:In-Reply-To:References; bh=rXevp+2MPoX1mMQip/JohoUxstuFnV/owR3qQ9MjTp4=; b=bs9MYK65JwaWYp3wivEEiVpj7m wcYunp7Xex6nrmoLO9H0unx49yFJlWiIr2UKMrMcKj7Fjt9F7asVTE13eAm0hUGyrgzYvylXbJer5 yl1tj4eOrGi7BEeq2FFfXxqUvY8CC7NUsI9Rh0rkfPhFpe/28q0olbYrdEgXGC29N6POtmFA/XvIy IzRf1dpj64Xkz01ZQWUPqyxCunQA5M2BaInBQecoNG2WnNyRll25dXBVub9iet8jPo6LPnb6dKZUf eymeWgpivjg4x0qdbANtuNtiM/A7J3j4wZoPqZZyrPsjSrrIzivqcPjRnlNbyRld7Pv9ah7qU/1dB CPldL6vg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1u6NXS-00000002Eur-34QI; Sun, 20 Apr 2025 05:48:22 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH 0/5] crash: provide a crash watchdog Date: Sat, 19 Apr 2025 22:48:16 -0700 Message-ID: <20250420054822.533987-1-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain One of the biggest pains we've suffered with CIs has been crashes, filesystem corruptions. Although we have a feature which is *supposed* to do that, its obviosly not working. Fix this by adding support for this stuff, and leveraging for our CIs. We will start with our fstests CIs and can scale this out to the other ones easily now. Luis Chamberlain (5): systemd-remote: use ip address for systemd-remote journal crash: add kernel crash watchdog library fstests_watchdog.py: use the new crash watchdog library crash_watchdog.py: add generic crash watchdog crash_report.py: add a crash report scripts/workflows/fstests/fstests_watchdog.py | 89 ++- scripts/workflows/generic/crash_report.py | 109 +++ scripts/workflows/generic/crash_watchdog.py | 186 +++++ scripts/workflows/generic/get_console.py | 1 + scripts/workflows/generic/lib | 1 + scripts/workflows/lib/crash.py | 724 ++++++++++++++++++ scripts/workflows/lib/systemd_remote.py | 19 +- 7 files changed, 1081 insertions(+), 48 deletions(-) create mode 100755 scripts/workflows/generic/crash_report.py create mode 100755 scripts/workflows/generic/crash_watchdog.py create mode 120000 scripts/workflows/generic/get_console.py create mode 120000 scripts/workflows/generic/lib create mode 100755 scripts/workflows/lib/crash.py -- 2.47.2