[PATCHSET 2/2] fstests: integrate with coredump capturing

[PATCHSET 2/2] fstests: integrate with coredump capturing

[Darrick J. Wong]

Hi all,

Integrate fstests with coredump capturing tools such as systemd-coredump.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=coredump-capture
---
Commits in this patchset:
 * fsstress: don't abort when stat(".") returns EIO
 * check: collect core dumps from systemd-coredump
---
 README         |   20 ++++++++++++++++++++
 check          |    2 ++
 common/rc      |   44 ++++++++++++++++++++++++++++++++++++++++++++
 ltp/fsstress.c |   29 ++++++++++++++++++++++++++---
 4 files changed, 92 insertions(+), 3 deletions(-)

[PATCH 1/2] fsstress: don't abort when stat(".") returns EIO

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

First, start with the premise that fstests is run with a nonzero limit
on the size of core dumps so that we can capture the state of
misbehaving fs utilities like fsck and scrub if they crash.

When fsstress is compiled with DEBUG defined (which is the default), it
will periodically call check_cwd to ensure that the current working
directory hasn't changed out from underneath it.

If the filesystem is XFS and it shuts down, the stat64() calls will
start returning EIO.  In this case, we follow the out: label and call
abort() to exit the program.  Historically this did not produce any core
dumps because $PWD is on the dead filesystem and the write fails.

However, modern systems are often configured to capture coredumps using
some external mechanism, e.g. abrt/systemd-coredump.   In this case, the
capture tool will succeeds in capturing every crashed process, which
fills the crash dump directory with a lot of useless junk.  Worse, if
the capture tool is configured to pass the dumps to fstests, it will
flag the test as failed because something dumped core.

This is really silly, because basic stat requests for the current
working directory can be satisfied from the inode cache without a disk
access.  In this narrow situation, EIO only happens when the fs has shut
down, so just exit the program.  Apply the same exit-on-EIO logic to
post-operation cleanup if we fail to go up one directory.

We really should have a way to query if a filesystem is shut down that
isn't conflated with (possibly transient) EIO errors.  But for now this
is what we have to do. :(

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 ltp/fsstress.c |   29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)


diff --git a/ltp/fsstress.c b/ltp/fsstress.c
index 8dbfb81f95a538..ae31c6a22d4d93 100644
--- a/ltp/fsstress.c
+++ b/ltp/fsstress.c
@@ -1049,8 +1049,21 @@ check_cwd(void)
 
 	ret = stat64(".", &statbuf);
 	if (ret != 0) {
+		int error = errno;
+
 		fprintf(stderr, "fsstress: check_cwd stat64() returned %d with errno: %d (%s)\n",
-			ret, errno, strerror(errno));
+			ret, error, strerror(error));
+
+		/*
+		 * The current working directory is pinned in memory, which
+		 * means that stat should not have had to do any disk accesses
+		 * to retrieve stat information.  Treat an EIO as an indication
+		 * that the filesystem shut down and exit instead of dumping
+		 * core like the abort() below does.
+		 */
+		if (error == EIO)
+			exit(1);
+
 		goto out;
 	}
 
@@ -1284,14 +1297,24 @@ doproc(void)
 		 */
 		if (errtag != 0 && opno % 100 == 0)  {
 			rval = stat64(".", &statbuf);
-			if (rval == EIO)  {
+			if (rval != 0 && errno == EIO)  {
 				fprintf(stderr, "Detected EIO\n");
 				goto errout;
 			}
 		}
 	}
 errout:
-	assert(chdir("..") == 0);
+	rval = chdir("..");
+	if (rval != 0 && errno == EIO) {
+		/*
+		 * If we can't go up a directory due to EIO, treat that as an
+		 * indication that the filesystem shut down and exit instead of
+		 * dumping core like the abort() below does.
+		 */
+		fprintf(stderr, "Detected EIO, cannot clean up\n");
+		exit(1);
+	}
+	assert(rval == 0);
 	free(homedir);
 	if (cleanup) {
 		int ret;

Re: [PATCH 1/2] fsstress: don't abort when stat(".") returns EIO

[Christoph Hellwig]

On Tue, Oct 21, 2025 at 11:42:20AM -0700, Darrick J. Wong wrote:
> We really should have a way to query if a filesystem is shut down that
> isn't conflated with (possibly transient) EIO errors.  But for now this
> is what we have to do. :(

Please send a patch to send a statx patch..

But for this patch:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH 2/2] check: collect core dumps from systemd-coredump

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

On modern RHEL (>=8) and Debian KDE systems, systemd-coredump can be
installed to capture core dumps from crashed programs.  If this is the
case, we would like to capture core dumps from programs that crash
during the test.  Set up an (admittedly overwrought) pipeline to extract
dumps created during the test and then capture them the same way that we
pick up "core" and "core.$pid" files.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 README    |   20 ++++++++++++++++++++
 check     |    2 ++
 common/rc |   44 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 66 insertions(+)


diff --git a/README b/README
index 9e9afe3cbb7ad4..196c79a21bdc0c 100644
--- a/README
+++ b/README
@@ -109,6 +109,11 @@ Ubuntu or Debian
    $ sudo apt-get install exfatprogs f2fs-tools ocfs2-tools udftools xfsdump \
         xfslibs-dev
 
+3. Install packages for optional features:
+
+    systemd coredump capture:
+    $ sudo apt install systemd-coredump systemd jq
+
 Fedora
 ------
 
@@ -124,6 +129,11 @@ Fedora
     $ sudo yum install btrfs-progs exfatprogs f2fs-tools ocfs2-tools xfsdump \
         xfsprogs-devel
 
+3. Install packages for optional features:
+
+    systemd coredump capture:
+    $ sudo yum install systemd systemd-udev jq
+
 RHEL or CentOS
 --------------
 
@@ -159,6 +169,11 @@ RHEL or CentOS
     For ocfs2 build and install:
      - see https://github.com/markfasheh/ocfs2-tools
 
+5. Install packages for optional features:
+
+    systemd coredump capture:
+    $ sudo yum install systemd systemd-udev jq
+
 SUSE Linux Enterprise or openSUSE
 ---------------------------------
 
@@ -176,6 +191,11 @@ SUSE Linux Enterprise or openSUSE
     For XFS install:
      $ sudo zypper install xfsdump xfsprogs-devel
 
+3. Install packages for optional features:
+
+    systemd coredump capture:
+    $ sudo yum install systemd systemd-coredump jq
+
 Build and install test, libs and utils
 --------------------------------------
 
diff --git a/check b/check
index 2d089d351380d2..c897afbb419612 100755
--- a/check
+++ b/check
@@ -930,6 +930,7 @@ function run_section()
 		     $1 == "'$seqnum'" {lasttime=" " $2 "s ... "; exit} \
 		     END {printf "%s", lasttime}' "$check.time"
 		rm -f core $seqres.notrun
+		_start_coredumpctl_collection
 
 		start=`_wallclock`
 		$timestamp && _timestamp
@@ -963,6 +964,7 @@ function run_section()
 		# just "core".  Use globbing to find the most common patterns,
 		# assuming there are no other coredump capture packages set up.
 		local cores=0
+		_finish_coredumpctl_collection
 		for i in core core.*; do
 			test -f "$i" || continue
 			if ((cores++ == 0)); then
diff --git a/common/rc b/common/rc
index 462f433197a3c2..3e7d7646cd6868 100644
--- a/common/rc
+++ b/common/rc
@@ -5001,6 +5001,50 @@ _check_kmemleak()
 	fi
 }
 
+# Current timestamp, in a format that systemd likes
+_systemd_now() {
+	timedatectl show --property=TimeUSec --value
+}
+
+# Do what we need to do to capture core dumps from coredumpctl
+_start_coredumpctl_collection() {
+	command -v coredumpctl &>/dev/null || return
+	command -v timedatectl &>/dev/null || return
+	command -v jq &>/dev/null || return
+
+	sysctl kernel.core_pattern | grep -q systemd-coredump || return
+	COREDUMPCTL_START_TIMESTAMP="$(_systemd_now)"
+}
+
+# Capture core dumps from coredumpctl.
+#
+# coredumpctl list only supports json output as a machine-readable format.  The
+# human-readable format intermingles spaces from the timestamp with actual
+# column separators, so we cannot parse that sanely.  The json output is an
+# array of:
+#        {
+#                "time" : 1749744847150926,
+#                "pid" : 2297,
+#                "uid" : 0,
+#                "gid" : 0,
+#                "sig" : 6,
+#                "corefile" : "present",
+#                "exe" : "/run/fstests/e2fsprogs/fuse2fs",
+#                "size" : 47245
+#        },
+# So we use jq to filter out lost corefiles, then print the pid and exe
+# separated by a pipe and hope that nobody ever puts a pipe in an executable
+# name.
+_finish_coredumpctl_collection() {
+	test -n "$COREDUMPCTL_START_TIMESTAMP" || return
+
+	coredumpctl list --since="$COREDUMPCTL_START_TIMESTAMP" --json=short 2>/dev/null | \
+	jq --raw-output 'map(select(.corefile == "present")) | map("\(.pid)|\(.exe)") | .[]' | while IFS='|' read pid exe; do
+		test -e "core.$pid" || coredumpctl dump --output="core.$pid" "$pid" "$exe" &>> $seqres.full
+	done
+	unset COREDUMPCTL_START_TIMESTAMP
+}
+
 # don't check dmesg log after test
 _disable_dmesg_check()
 {

Re: [PATCH 2/2] check: collect core dumps from systemd-coredump

[Christoph Hellwig]

On Tue, Oct 21, 2025 at 11:42:35AM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> On modern RHEL (>=8) and Debian KDE systems, systemd-coredump can be

Only KDE?  Most of my test systems don't have any graphic
environment set up.

Anyway, the changes looks fine:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 2/2] check: collect core dumps from systemd-coredump

[Darrick J. Wong]

On Tue, Oct 21, 2025 at 09:24:14PM -0700, Christoph Hellwig wrote:
> On Tue, Oct 21, 2025 at 11:42:35AM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > On modern RHEL (>=8) and Debian KDE systems, systemd-coredump can be
> 
> Only KDE?  Most of my test systems don't have any graphic
> environment set up.

Non-KDE Debian doesn't bother installing systemd-coredump.  My actual
test systems have non-graphical Debian, but then I stood up one OL10
system for giggles and <kaboom> coredumps broke. :P

> Anyway, the changes looks fine:
> 
> Reviewed-by: Christoph Hellwig <hch@lst.de>

Thanks!

--D

[PATCHSET 3/3] fstests: integrate with coredump capturing

[Darrick J. Wong]

Hi all,

Integrate fstests with coredump capturing tools such as systemd-coredump.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

With a bit of luck, this should all go splendidly.
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=coredump-capture
---
Commits in this patchset:
 * fsstress: don't abort when stat(".") returns EIO
 * check: collect core dumps from systemd-coredump
---
 check          |    2 ++
 common/rc      |   44 ++++++++++++++++++++++++++++++++++++++++++++
 ltp/fsstress.c |   15 ++++++++++++++-
 3 files changed, 60 insertions(+), 1 deletion(-)

[PATCH 2/2] check: collect core dumps from systemd-coredump

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

On modern RHEL (>=8) and Debian KDE systems, systemd-coredump can be
installed to capture core dumps from crashed programs.  If this is the
case, we would like to capture core dumps from programs that crash
during the test.  Set up an (admittedly overwrought) pipeline to extract
dumps created during the test and then capture them the same way that we
pick up "core" and "core.$pid" files.

Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
---
 check     |    2 ++
 common/rc |   44 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 46 insertions(+)


diff --git a/check b/check
index ce7eacb7c45d9e..77581e438c46b9 100755
--- a/check
+++ b/check
@@ -924,6 +924,7 @@ function run_section()
 		     $1 == "'$seqnum'" {lasttime=" " $2 "s ... "; exit} \
 		     END {printf "%s", lasttime}' "$check.time"
 		rm -f core $seqres.notrun
+		_start_coredumpctl_collection
 
 		start=`_wallclock`
 		$timestamp && _timestamp
@@ -957,6 +958,7 @@ function run_section()
 		# just "core".  Use globbing to find the most common patterns,
 		# assuming there are no other coredump capture packages set up.
 		local cores=0
+		_finish_coredumpctl_collection
 		for i in core core.*; do
 			test -f "$i" || continue
 			if ((cores++ == 0)); then
diff --git a/common/rc b/common/rc
index 04b721b7318a7e..e4c4d05387f44e 100644
--- a/common/rc
+++ b/common/rc
@@ -5034,6 +5034,50 @@ _check_kmemleak()
 	fi
 }
 
+# Current timestamp, in a format that systemd likes
+_systemd_now() {
+	timedatectl show --property=TimeUSec --value
+}
+
+# Do what we need to do to capture core dumps from coredumpctl
+_start_coredumpctl_collection() {
+	command -v coredumpctl &>/dev/null || return
+	command -v timedatectl &>/dev/null || return
+	command -v jq &>/dev/null || return
+
+	sysctl kernel.core_pattern | grep -q systemd-coredump || return
+	COREDUMPCTL_START_TIMESTAMP="$(_systemd_now)"
+}
+
+# Capture core dumps from coredumpctl.
+#
+# coredumpctl list only supports json output as a machine-readable format.  The
+# human-readable format intermingles spaces from the timestamp with actual
+# column separators, so we cannot parse that sanely.  The json output is an
+# array of:
+#        {
+#                "time" : 1749744847150926,
+#                "pid" : 2297,
+#                "uid" : 0,
+#                "gid" : 0,
+#                "sig" : 6,
+#                "corefile" : "present",
+#                "exe" : "/run/fstests/e2fsprogs/fuse2fs",
+#                "size" : 47245
+#        },
+# So we use jq to filter out lost corefiles, then print the pid and exe
+# separated by a pipe and hope that nobody ever puts a pipe in an executable
+# name.
+_finish_coredumpctl_collection() {
+	test -n "$COREDUMPCTL_START_TIMESTAMP" || return
+
+	coredumpctl list --since="$COREDUMPCTL_START_TIMESTAMP" --json=short 2>/dev/null | \
+	jq --raw-output 'map(select(.corefile == "present")) | map("\(.pid)|\(.exe)") | .[]' | while IFS='|' read pid exe; do
+		test -e "core.$pid" || coredumpctl dump --output="core.$pid" "$pid" "$exe" &>> $seqres.full
+	done
+	unset COREDUMPCTL_START_TIMESTAMP
+}
+
 # don't check dmesg log after test
 _disable_dmesg_check()
 {

Re: [PATCH 2/2] check: collect core dumps from systemd-coredump

[Zorro Lang]

On Tue, Jul 29, 2025 at 01:11:06PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> On modern RHEL (>=8) and Debian KDE systems, systemd-coredump can be
> installed to capture core dumps from crashed programs.  If this is the
> case, we would like to capture core dumps from programs that crash
> during the test.  Set up an (admittedly overwrought) pipeline to extract
> dumps created during the test and then capture them the same way that we
> pick up "core" and "core.$pid" files.
> 
> Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> ---
>  check     |    2 ++
>  common/rc |   44 ++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 46 insertions(+)
> 
> 
> diff --git a/check b/check
> index ce7eacb7c45d9e..77581e438c46b9 100755
> --- a/check
> +++ b/check
> @@ -924,6 +924,7 @@ function run_section()
>  		     $1 == "'$seqnum'" {lasttime=" " $2 "s ... "; exit} \
>  		     END {printf "%s", lasttime}' "$check.time"
>  		rm -f core $seqres.notrun
> +		_start_coredumpctl_collection
>  
>  		start=`_wallclock`
>  		$timestamp && _timestamp
> @@ -957,6 +958,7 @@ function run_section()
>  		# just "core".  Use globbing to find the most common patterns,
>  		# assuming there are no other coredump capture packages set up.
>  		local cores=0
> +		_finish_coredumpctl_collection
>  		for i in core core.*; do
>  			test -f "$i" || continue
>  			if ((cores++ == 0)); then
> diff --git a/common/rc b/common/rc
> index 04b721b7318a7e..e4c4d05387f44e 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -5034,6 +5034,50 @@ _check_kmemleak()
>  	fi
>  }
>  
> +# Current timestamp, in a format that systemd likes
> +_systemd_now() {
> +	timedatectl show --property=TimeUSec --value
> +}
> +
> +# Do what we need to do to capture core dumps from coredumpctl
> +_start_coredumpctl_collection() {
> +	command -v coredumpctl &>/dev/null || return
> +	command -v timedatectl &>/dev/null || return
> +	command -v jq &>/dev/null || return
> +
> +	sysctl kernel.core_pattern | grep -q systemd-coredump || return

# rpm -qf `which coredumpctl`
systemd-udev-252-53.el9.x86_64
# rpm -qf `which timedatectl`
systemd-252-53.el9.x86_64
# rpm -qf `which jq`
jq-1.6-17.el9.x86_64
# rpm -qf /usr/lib/systemd/systemd-coredump 
systemd-udev-252-53.el9.x86_64

So we have 3 optional running dependences, how about metion that in README?

Thanks,
Zorro

> +	COREDUMPCTL_START_TIMESTAMP="$(_systemd_now)"
> +}
> +
> +# Capture core dumps from coredumpctl.
> +#
> +# coredumpctl list only supports json output as a machine-readable format.  The
> +# human-readable format intermingles spaces from the timestamp with actual
> +# column separators, so we cannot parse that sanely.  The json output is an
> +# array of:
> +#        {
> +#                "time" : 1749744847150926,
> +#                "pid" : 2297,
> +#                "uid" : 0,
> +#                "gid" : 0,
> +#                "sig" : 6,
> +#                "corefile" : "present",
> +#                "exe" : "/run/fstests/e2fsprogs/fuse2fs",
> +#                "size" : 47245
> +#        },
> +# So we use jq to filter out lost corefiles, then print the pid and exe
> +# separated by a pipe and hope that nobody ever puts a pipe in an executable
> +# name.
> +_finish_coredumpctl_collection() {
> +	test -n "$COREDUMPCTL_START_TIMESTAMP" || return
> +
> +	coredumpctl list --since="$COREDUMPCTL_START_TIMESTAMP" --json=short 2>/dev/null | \
> +	jq --raw-output 'map(select(.corefile == "present")) | map("\(.pid)|\(.exe)") | .[]' | while IFS='|' read pid exe; do
> +		test -e "core.$pid" || coredumpctl dump --output="core.$pid" "$pid" "$exe" &>> $seqres.full
> +	done
> +	unset COREDUMPCTL_START_TIMESTAMP
> +}
> +
>  # don't check dmesg log after test
>  _disable_dmesg_check()
>  {
> 

Re: [PATCH 2/2] check: collect core dumps from systemd-coredump

[Darrick J. Wong]

On Sat, Aug 02, 2025 at 09:47:00PM +0800, Zorro Lang wrote:
> On Tue, Jul 29, 2025 at 01:11:06PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > On modern RHEL (>=8) and Debian KDE systems, systemd-coredump can be
> > installed to capture core dumps from crashed programs.  If this is the
> > case, we would like to capture core dumps from programs that crash
> > during the test.  Set up an (admittedly overwrought) pipeline to extract
> > dumps created during the test and then capture them the same way that we
> > pick up "core" and "core.$pid" files.
> > 
> > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > ---
> >  check     |    2 ++
> >  common/rc |   44 ++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 46 insertions(+)
> > 
> > 
> > diff --git a/check b/check
> > index ce7eacb7c45d9e..77581e438c46b9 100755
> > --- a/check
> > +++ b/check
> > @@ -924,6 +924,7 @@ function run_section()
> >  		     $1 == "'$seqnum'" {lasttime=" " $2 "s ... "; exit} \
> >  		     END {printf "%s", lasttime}' "$check.time"
> >  		rm -f core $seqres.notrun
> > +		_start_coredumpctl_collection
> >  
> >  		start=`_wallclock`
> >  		$timestamp && _timestamp
> > @@ -957,6 +958,7 @@ function run_section()
> >  		# just "core".  Use globbing to find the most common patterns,
> >  		# assuming there are no other coredump capture packages set up.
> >  		local cores=0
> > +		_finish_coredumpctl_collection
> >  		for i in core core.*; do
> >  			test -f "$i" || continue
> >  			if ((cores++ == 0)); then
> > diff --git a/common/rc b/common/rc
> > index 04b721b7318a7e..e4c4d05387f44e 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -5034,6 +5034,50 @@ _check_kmemleak()
> >  	fi
> >  }
> >  
> > +# Current timestamp, in a format that systemd likes
> > +_systemd_now() {
> > +	timedatectl show --property=TimeUSec --value
> > +}
> > +
> > +# Do what we need to do to capture core dumps from coredumpctl
> > +_start_coredumpctl_collection() {
> > +	command -v coredumpctl &>/dev/null || return
> > +	command -v timedatectl &>/dev/null || return
> > +	command -v jq &>/dev/null || return
> > +
> > +	sysctl kernel.core_pattern | grep -q systemd-coredump || return
> 
> # rpm -qf `which coredumpctl`
> systemd-udev-252-53.el9.x86_64
> # rpm -qf `which timedatectl`
> systemd-252-53.el9.x86_64
> # rpm -qf `which jq`
> jq-1.6-17.el9.x86_64
> # rpm -qf /usr/lib/systemd/systemd-coredump 
> systemd-udev-252-53.el9.x86_64
> 
> So we have 3 optional running dependences, how about metion that in README?

Done.

--D

> Thanks,
> Zorro
> 
> > +	COREDUMPCTL_START_TIMESTAMP="$(_systemd_now)"
> > +}
> > +
> > +# Capture core dumps from coredumpctl.
> > +#
> > +# coredumpctl list only supports json output as a machine-readable format.  The
> > +# human-readable format intermingles spaces from the timestamp with actual
> > +# column separators, so we cannot parse that sanely.  The json output is an
> > +# array of:
> > +#        {
> > +#                "time" : 1749744847150926,
> > +#                "pid" : 2297,
> > +#                "uid" : 0,
> > +#                "gid" : 0,
> > +#                "sig" : 6,
> > +#                "corefile" : "present",
> > +#                "exe" : "/run/fstests/e2fsprogs/fuse2fs",
> > +#                "size" : 47245
> > +#        },
> > +# So we use jq to filter out lost corefiles, then print the pid and exe
> > +# separated by a pipe and hope that nobody ever puts a pipe in an executable
> > +# name.
> > +_finish_coredumpctl_collection() {
> > +	test -n "$COREDUMPCTL_START_TIMESTAMP" || return
> > +
> > +	coredumpctl list --since="$COREDUMPCTL_START_TIMESTAMP" --json=short 2>/dev/null | \
> > +	jq --raw-output 'map(select(.corefile == "present")) | map("\(.pid)|\(.exe)") | .[]' | while IFS='|' read pid exe; do
> > +		test -e "core.$pid" || coredumpctl dump --output="core.$pid" "$pid" "$exe" &>> $seqres.full
> > +	done
> > +	unset COREDUMPCTL_START_TIMESTAMP
> > +}
> > +
> >  # don't check dmesg log after test
> >  _disable_dmesg_check()
> >  {
> > 
> 
>