bpftrace script to monitor tasks that are stuck in NFSv4 exception loops

bpftrace script to monitor tasks that are stuck in NFSv4 exception loops

[Trond Myklebust]

[-- Attachment #1: Type: text/plain, Size: 120 bytes --]

See attachment.

-- 
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com



[-- Attachment #2: bpftrace script --]
[-- Type: text/plain, Size: 852 bytes --]

#!/usr/bin/bpftrace
/*
 * Monitor NFSv4 exceptions
 *
 * Copyright 2023 Trond Myklebust
 * Licensed under the GPLv2
 *
 */
struct nfs4_exception {
	struct nfs4_state *state;
	struct inode *inode;
	struct nfs4_stateid_struct *stateid;
	long timeout;
	unsigned short retrans;
	unsigned char task_is_privileged : 1;
	unsigned char delay : 1,
		      recovering : 1,
		      retry : 1;
	bool interruptible;
};

kprobe:nfs4_do_handle_exception {
	$server = (struct nfs_server *)arg0;
	$errorcode = (int32)arg1;
	$exception = (struct nfs4_exception *)arg2;
	$inode = $exception->inode;

	if ($errorcode != 0 && $inode != 0) {
		$dev = (uint32)$inode->i_sb->s_dev;
		printf("%s [%d]: inode=%u:%u:%u, error=%d %s\n",
		       strftime("%F %H:%M:%S", nsecs), pid, $dev >> 20,
		       $dev & ((1 << 20) - 1), $inode->i_ino, $errorcode,
		       kstack());
	}
}

Re: bpftrace script to monitor tasks that are stuck in NFSv4 exception loops

[Benjamin Coddington]

On 11 Oct 2023, at 11:15, Trond Myklebust wrote:

> See attachment.

Cool.  Thanks for this - looks pretty similar to the systemtap stuff we use, but compile times are faster here.

I wonder if bpftrace can do the "guru" things that systemtap can, like modify the stack and memory.

Seems it works with gcc too, just needed CONFIG_DEBUG_INFO_BTF=y.  I will play with it a bit.

Ben