From: "J. Bruce Fields" <bfields@fieldses.org>
To: Dai Ngo <dai.ngo@oracle.com>
Cc: chuck.lever@oracle.com, linux-nfs@vger.kernel.org,
linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH RFC v4 0/2] nfsd: Initial implementation of NFSv4 Courteous Server
Date: Mon, 27 Sep 2021 16:14:33 -0400 [thread overview]
Message-ID: <20210927201433.GA1704@fieldses.org> (raw)
In-Reply-To: <20210924205252.82502-1-dai.ngo@oracle.com>
The file_rwsem is used for /proc/locks; only the code that produces the
/proc/locks output calls down_write, the rest only calls down_read.
I assumed that it was OK to nest read acquisitions of a rwsem, but I
think that's wrong.
I think it should be no big deal to move the lm_expire_lock(.,0) call
outside of the file_rwsem?
--b.
[ 959.807364] ============================================
[ 959.807803] WARNING: possible recursive locking detected
[ 959.808228] 5.15.0-rc2-00009-g4e5af4d2635a #533 Not tainted
[ 959.808675] --------------------------------------------
[ 959.809189] nfsd/5675 is trying to acquire lock:
[ 959.809664] ffffffff8519e470 (file_rwsem){.+.+}-{0:0}, at: locks_remove_posix+0x37f/0x4e0
[ 959.810647]
but task is already holding lock:
[ 959.811097] ffffffff8519e470 (file_rwsem){.+.+}-{0:0}, at: nfsd4_lock+0xcb9/0x3850 [nfsd]
[ 959.812147]
other info that might help us debug this:
[ 959.812698] Possible unsafe locking scenario:
[ 959.813189] CPU0
[ 959.813362] ----
[ 959.813544] lock(file_rwsem);
[ 959.813812] lock(file_rwsem);
[ 959.814078]
*** DEADLOCK ***
[ 959.814386] May be due to missing lock nesting notation
[ 959.814968] 3 locks held by nfsd/5675:
[ 959.815315] #0: ffff888007d42bc8 (&rp->rp_mutex){+.+.}-{3:3}, at: nfs4_preprocess_seqid_op+0x395/0x730 [nfsd]
[ 959.816546] #1: ffff88800f378b70 (&stp->st_mutex#2){+.+.}-{3:3}, at: nfsd4_lock+0x1f91/0x3850 [nfsd]
[ 959.817697] #2: ffffffff8519e470 (file_rwsem){.+.+}-{0:0}, at: nfsd4_lock+0xcb9/0x3850 [nfsd]
[ 959.818755]
stack backtrace:
[ 959.819010] CPU: 2 PID: 5675 Comm: nfsd Not tainted 5.15.0-rc2-00009-g4e5af4d2635a #533
[ 959.819847] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-3.fc34 04/01/2014
[ 959.820637] Call Trace:
[ 959.820759] dump_stack_lvl+0x45/0x59
[ 959.821016] __lock_acquire.cold+0x175/0x3a5
[ 959.821316] ? lockdep_hardirqs_on_prepare+0x400/0x400
[ 959.821741] lock_acquire+0x1a6/0x4b0
[ 959.821976] ? locks_remove_posix+0x37f/0x4e0
[ 959.822316] ? lock_release+0x6d0/0x6d0
[ 959.822591] ? find_held_lock+0x2c/0x110
[ 959.822852] ? lock_is_held_type+0xd5/0x130
[ 959.823139] posix_lock_inode+0x143/0x1ab0
[ 959.823414] ? locks_remove_posix+0x37f/0x4e0
[ 959.823739] ? do_raw_spin_unlock+0x54/0x220
[ 959.824031] ? lockdep_init_map_type+0x2c3/0x7a0
[ 959.824355] ? locks_remove_flock+0x2e0/0x2e0
[ 959.824681] locks_remove_posix+0x37f/0x4e0
[ 959.824984] ? do_lock_file_wait+0x2a0/0x2a0
[ 959.825287] ? lock_downgrade+0x6a0/0x6a0
[ 959.825584] ? nfsd_file_put+0x170/0x170 [nfsd]
[ 959.825941] filp_close+0xed/0x130
[ 959.826191] nfs4_free_lock_stateid+0xcc/0x190 [nfsd]
[ 959.826625] free_ol_stateid_reaplist+0x128/0x1f0 [nfsd]
[ 959.827037] release_openowner+0xee/0x150 [nfsd]
[ 959.827382] ? release_last_closed_stateid+0x460/0x460 [nfsd]
[ 959.827837] ? rwlock_bug.part.0+0x90/0x90
[ 959.828115] __destroy_client+0x39f/0x6f0 [nfsd]
[ 959.828460] ? nfsd4_cb_recall_release+0x20/0x20 [nfsd]
[ 959.828868] nfsd4_fl_expire_lock+0x2bc/0x460 [nfsd]
[ 959.829273] posix_lock_inode+0xa46/0x1ab0
[ 959.829579] ? lockdep_init_map_type+0x2c3/0x7a0
[ 959.829913] ? locks_remove_flock+0x2e0/0x2e0
[ 959.830250] ? __init_waitqueue_head+0x70/0xd0
[ 959.830568] nfsd4_lock+0xcb9/0x3850 [nfsd]
[ 959.830878] ? nfsd4_delegreturn+0x3b0/0x3b0 [nfsd]
[ 959.831248] ? percpu_counter_add_batch+0x77/0x130
[ 959.831594] nfsd4_proc_compound+0xcee/0x21d0 [nfsd]
[ 959.831973] ? nfsd4_release_compoundargs+0x140/0x140 [nfsd]
[ 959.832414] nfsd_dispatch+0x4df/0xc50 [nfsd]
[ 959.832737] ? nfsd_svc+0xca0/0xca0 [nfsd]
[ 959.833051] svc_process_common+0xdeb/0x2480 [sunrpc]
[ 959.833462] ? svc_create+0x20/0x20 [sunrpc]
[ 959.833830] ? nfsd_svc+0xca0/0xca0 [nfsd]
[ 959.834144] ? svc_sock_secure_port+0x36/0x40 [sunrpc]
[ 959.834578] ? svc_recv+0xd9c/0x2490 [sunrpc]
[ 959.834915] svc_process+0x32e/0x4a0 [sunrpc]
[ 959.835249] nfsd+0x306/0x530 [nfsd]
[ 959.835499] ? nfsd_shutdown_threads+0x300/0x300 [nfsd]
[ 959.835899] kthread+0x391/0x470
[ 959.836094] ? _raw_spin_unlock_irq+0x24/0x50
[ 959.836394] ? set_kthread_struct+0x100/0x100
[ 959.836698] ret_from_fork+0x22/0x30
[ 960.750222] nfsd: last server has exited, flushing export cache
[ 960.880355] NFSD: Using nfsdcld client tracking operations.
[ 960.880956] NFSD: starting 15-second grace period (net f0000098)
[ 1403.405511] nfsd: last server has exited, flushing export cache
[ 1403.656335] NFSD: Using nfsdcld client tracking operations.
[ 1403.657585] NFSD: starting 15-second grace period (net f0000098)
[ 1445.741596] nfsd: last server has exited, flushing export cache
[ 1445.981980] NFSD: Using nfsdcld client tracking operations.
[ 1445.983143] NFSD: starting 15-second grace period (net f0000098)
[ 1450.025112] NFSD: all clients done reclaiming, ending NFSv4 grace period (net f0000098)
[ 1472.325551] nfsd: last server has exited, flushing export cache
[ 1472.583073] NFSD: Using nfsdcld client tracking operations.
[ 1472.583998] NFSD: starting 15-second grace period (net f0000098)
[ 1473.175582] NFSD: all clients done reclaiming, ending NFSv4 grace period (net f0000098)
[ 1494.637499] nfsd: last server has exited, flushing export cache
[ 1494.885795] NFSD: Using nfsdcld client tracking operations.
[ 1494.886484] NFSD: starting 15-second grace period (net f0000098)
[ 1495.393667] NFSD: all clients done reclaiming, ending NFSv4 grace period (net f0000098)
[ 1516.781474] nfsd: last server has exited, flushing export cache
[ 1516.902903] NFSD: Using nfsdcld client tracking operations.
[ 1516.903460] NFSD: starting 15-second grace period (net f0000098)
[ 1538.045156] NFSD: all clients done reclaiming, ending NFSv4 grace period (net f0000098)
[ 1559.125362] nfsd: last server has exited, flushing export cache
[ 1559.362856] NFSD: Using nfsdcld client tracking operations.
[ 1559.363658] NFSD: starting 15-second grace period (net f0000098)
[ 1559.480531] NFSD: all clients done reclaiming, ending NFSv4 grace period (net f0000098)
[ 1583.745353] nfsd: last server has exited, flushing export cache
[ 1583.876877] NFSD: Using nfsdcld client tracking operations.
[ 1583.877573] NFSD: starting 15-second grace period (net f0000098)
[ 1586.401321] nfsd: last server has exited, flushing export cache
[ 1586.525629] NFSD: Using nfsdcld client tracking operations.
[ 1586.526388] NFSD: starting 15-second grace period (net f0000098)
[ 1625.993218] nfsd: last server has exited, flushing export cache
[ 1626.442627] NFSD: Using nfsdcld client tracking operations.
[ 1626.444397] NFSD: starting 15-second grace period (net f0000098)
[ 1627.117214] nfsd: last server has exited, flushing export cache
[ 1627.351487] NFSD: Using nfsdcld client tracking operations.
[ 1627.352663] NFSD: starting 15-second grace period (net f0000098)
[ 1627.854410] NFSD: all clients done reclaiming, ending NFSv4 grace period (net f0000098)
[ 3285.818905] clocksource: timekeeping watchdog on CPU3: acpi_pm retried 2 times before success
next prev parent reply other threads:[~2021-09-27 20:14 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-24 20:52 [PATCH RFC v4 0/2] nfsd: Initial implementation of NFSv4 Courteous Server Dai Ngo
2021-09-24 20:52 ` [PATCH RFC v4 1/2] fs/lock: add new callback, lm_expire_lock, to lock_manager_operations Dai Ngo
2021-09-24 20:52 ` [PATCH RFC v4 2/2] nfsd: Initial implementation of NFSv4 Courteous Server Dai Ngo
2021-09-27 20:14 ` J. Bruce Fields [this message]
2021-09-27 20:39 ` [PATCH RFC v4 0/2] " J. Bruce Fields
2021-09-28 6:22 ` dai.ngo
2021-09-28 14:10 ` J. Bruce Fields
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210927201433.GA1704@fieldses.org \
--to=bfields@fieldses.org \
--cc=chuck.lever@oracle.com \
--cc=dai.ngo@oracle.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.