From: "J. Bruce Fields" <bfields@redhat.com>
To: Bryan Schumaker <bjschuma@netapp.com>
Cc: linux-nfs@vger.kernel.org
Subject: Re: [PATCH 03/25] nfsd4: extend state lock over seqid replay logic
Date: Tue, 27 Sep 2011 21:40:04 -0400 [thread overview]
Message-ID: <20110928014004.GF12190@pad.fieldses.org> (raw)
In-Reply-To: <4E82001C.5030005@netapp.com>
On Tue, Sep 27, 2011 at 12:55:56PM -0400, Bryan Schumaker wrote:
> Hi Bruce,
>
> I'm getting the following warning that I was able to bisect to this patch:
Hm. Was it doing a LOCKU at the time, I wonder? It looks like I missed
a case here.... I'll investigate.
--b.
>
> [ 142.149710] ------------[ cut here ]------------
> [ 142.150014] WARNING: at kernel/mutex-debug.c:78 debug_mutex_unlock+0xda/0xe0()
> [ 142.150258] Hardware name: Bochs
> [ 142.150407] Modules linked in: md5 nfsd exportfs nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ext2 snd_hda_intel snd_hda_codec snd_hwdep psmouse i2c_piix4 evdev serio_raw pcspkr virtio_balloon snd_pcm snd_timer snd soundcore snd_page_alloc floppy i2c_core button processor ext4 mbcache jbd2 crc16 pata_acpi uhci_hcd ata_piix libata usbcore scsi_mod virtio_net virtio_pci virtio_blk virtio virtio_ring
> [ 142.152927] Pid: 742, comm: nfsd Not tainted 3.1.0-rc1-SLIM+ #9
> [ 142.152927] Call Trace:
> [ 142.152927] [<ffffffff8105fa4f>] warn_slowpath_common+0x7f/0xc0
> [ 142.152927] [<ffffffff8105faaa>] warn_slowpath_null+0x1a/0x20
> [ 142.152927] [<ffffffff810960ca>] debug_mutex_unlock+0xda/0xe0
> [ 142.152927] [<ffffffff813e4200>] __mutex_unlock_slowpath+0x80/0x140
> [ 142.152927] [<ffffffff813e42ce>] mutex_unlock+0xe/0x10
> [ 142.152927] [<ffffffffa03bd3f5>] nfs4_lock_state+0x35/0x40 [nfsd]
> [ 142.152927] [<ffffffffa03b0b71>] nfsd4_proc_compound+0x2a1/0x690 [nfsd]
> [ 142.152927] [<ffffffffa039f9fb>] nfsd_dispatch+0xeb/0x230 [nfsd]
> [ 142.152927] [<ffffffffa02b1055>] svc_process_common+0x345/0x690 [sunrpc]
> [ 142.152927] [<ffffffff81058d10>] ? try_to_wake_up+0x280/0x280
> [ 142.152927] [<ffffffffa02b16e2>] svc_process+0x102/0x150 [sunrpc]
> [ 142.152927] [<ffffffffa039f0bd>] nfsd+0xbd/0x160 [nfsd]
> [ 142.152927] [<ffffffffa039f000>] ? 0xffffffffa039efff
> [ 142.152927] [<ffffffff8108230c>] kthread+0x8c/0xa0
> [ 142.152927] [<ffffffff813e8694>] kernel_thread_helper+0x4/0x10
> [ 142.152927] [<ffffffff81082280>] ? kthread_worker_fn+0x190/0x190
> [ 142.152927] [<ffffffff813e8690>] ? gs_change+0x13/0x13
> [ 142.152927] ---[ end trace 1b4070dc432138aa ]---
>
> I can duplicate it with this python script, the warning shows up on the server after (during?) the f.close() line:
>
> #!/usr/bin/python
> import sys
> import fcntl
> import struct
> import datetime
>
> f = open(sys.argv[1], 'rw+')
> print "Attempting to lock file:", sys.argv[1]
> lockreq = struct.pack('hhllhh', fcntl.F_WRLCK, 0, 0, 0, 0, 0)
> rv = fcntl.fcntl(f, fcntl.F_SETLKW, lockreq)
> raw_input("Press enter when you are ready to continue... ")
> f.close()
>
> - Bryan
>
> On 09/14/2011 07:44 AM, J. Bruce Fields wrote:
> > There are currently a couple races in the seqid replay code: a
> > retransmission could come while we're still encoding the original reply,
> > or a new seqid-mutating call could come as we're encoding a replay.
> >
> > So, extend the state lock over the encoding (both encoding of a replayed
> > reply and caching of the original encoded reply).
> >
> > I really hate doing this, and previously added the stateowner
> > reference-counting code to avoid it (which was insufficient)--but I
> > don't see a less complicated alternative at the moment.
> >
> > Signed-off-by: J. Bruce Fields <bfields@redhat.com>
> > ---
> > fs/nfsd/nfs4proc.c | 5 +++--
> > fs/nfsd/nfs4state.c | 12 ++++++++----
> > 2 files changed, 11 insertions(+), 6 deletions(-)
> >
> > diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> > index 50bae74..50063a8 100644
> > --- a/fs/nfsd/nfs4proc.c
> > +++ b/fs/nfsd/nfs4proc.c
> > @@ -408,8 +408,8 @@ out:
> > if (open->op_stateowner) {
> > nfs4_get_stateowner(open->op_stateowner);
> > cstate->replay_owner = open->op_stateowner;
> > - }
> > - nfs4_unlock_state();
> > + } else
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -1227,6 +1227,7 @@ encode_op:
> > be32_to_cpu(status));
> >
> > if (cstate->replay_owner) {
> > + nfs4_unlock_state();
> > nfs4_put_stateowner(cstate->replay_owner);
> > cstate->replay_owner = NULL;
> > }
> > diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> > index bc1a9db..6cf729a 100644
> > --- a/fs/nfsd/nfs4state.c
> > +++ b/fs/nfsd/nfs4state.c
> > @@ -3501,7 +3501,8 @@ nfsd4_open_confirm(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> >
> > nfsd4_create_clid_dir(sop->so_client);
> > out:
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -3568,7 +3569,8 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
> > memcpy(&od->od_stateid, &stp->st_stateid, sizeof(stateid_t));
> > status = nfs_ok;
> > out:
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -3609,7 +3611,8 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> > if (list_empty(&so->so_stateids))
> > move_to_close_lru(so);
> > out:
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -4071,7 +4074,8 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> > out:
> > if (status && lock->lk_is_new && lock_sop)
> > release_lockowner(lock_sop);
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
>
next prev parent reply other threads:[~2011-09-28 1:40 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-14 11:44 [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-09-14 11:44 ` [PATCH 01/25] nfsd4: centralize handling of replay owners J. Bruce Fields
2011-09-14 11:44 ` [PATCH 02/25] nfsd4: cleanup seqid op stateowner usage J. Bruce Fields
2011-09-14 11:44 ` [PATCH 03/25] nfsd4: extend state lock over seqid replay logic J. Bruce Fields
2011-09-27 16:55 ` Bryan Schumaker
2011-09-28 1:40 ` J. Bruce Fields [this message]
2011-09-28 1:49 ` J. Bruce Fields
2011-09-28 13:18 ` Bryan Schumaker
2011-09-28 13:44 ` J. Bruce Fields
2011-09-14 11:45 ` [PATCH 04/25] nfsd4: eliminate impossible open replay case J. Bruce Fields
2011-09-14 11:45 ` [PATCH 05/25] nfsd4: drop most stateowner refcounting J. Bruce Fields
2011-09-14 11:45 ` [PATCH 06/25] nfsd4: eliminate unused lt_stateowner J. Bruce Fields
2011-09-14 11:45 ` [PATCH 07/25] nfsd4: share common seqid checks J. Bruce Fields
2011-09-14 11:45 ` [PATCH 08/25] nfsd4: simplify check_open logic J. Bruce Fields
2011-09-14 11:45 ` [PATCH 09/25] nfsd4: move double-confirm test to open_confirm J. Bruce Fields
2011-09-14 11:45 ` [PATCH 10/25] nfsd4: move CLOSE_STATE special case to caller J. Bruce Fields
2011-09-14 11:45 ` [PATCH 11/25] nfsd4: split stateowners into open and lockowners J. Bruce Fields
2011-09-14 11:45 ` [PATCH 12/25] nfsd4: split out some free_generic_stateid code J. Bruce Fields
2011-09-14 11:45 ` [PATCH 13/25] nfsd4: rearrange to avoid a forward reference J. Bruce Fields
2011-09-14 11:45 ` [PATCH 14/25] nfsd4: split up find_stateid J. Bruce Fields
2011-09-14 11:45 ` [PATCH 15/25] nfsd4: split preprocess_seqid, cleanup J. Bruce Fields
2011-09-14 11:45 ` [PATCH 16/25] nfsd4: pass around typemask instead of flags J. Bruce Fields
2011-09-14 11:45 ` [PATCH 17/25] nfsd4: rename init_stateid J. Bruce Fields
2011-09-14 11:45 ` [PATCH 18/25] nfsd4: remove redundant stateid initialization J. Bruce Fields
2011-09-14 11:45 ` [PATCH 19/25] nfsd4: move some of nfs4_stateid into a separate structure J. Bruce Fields
2011-09-14 11:45 ` [PATCH 20/25] nfsd4: add common dl_stid field to delegation J. Bruce Fields
2011-09-14 11:45 ` [PATCH 21/25] nfsd4: share common stid-hashing helper function J. Bruce Fields
2011-09-14 11:45 ` [PATCH 22/25] nfsd4: hash deleg stateid's like any other J. Bruce Fields
2011-09-14 11:45 ` [PATCH 23/25] nfsd4: fix test_stateid for delegation stateid's J. Bruce Fields
2011-09-14 11:45 ` [PATCH 24/25] nfsd4: use deleg changes to cleanup preprocess_stateid_op J. Bruce Fields
2011-09-14 11:45 ` [PATCH 25/25] nfsd4: better stateid hashing J. Bruce Fields
2011-09-19 13:14 ` [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-09-19 13:15 ` [PATCH 1/5] nfsd4: replace oo_confirmed by flag bit J. Bruce Fields
2011-09-19 13:15 ` [PATCH 2/5] nfsd4: match close replays on stateid, not open owner id J. Bruce Fields
2011-09-19 13:15 ` [PATCH 3/5] nfsd4: simplify free_stateid J. Bruce Fields
2011-09-19 13:15 ` [PATCH 4/5] nfsd4: construct stateid from clientid and counter J. Bruce Fields
2011-09-27 16:10 ` Bryan Schumaker
2011-09-28 15:49 ` J. Bruce Fields
2011-10-03 14:43 ` Benny Halevy
2011-10-03 14:57 ` J. Bruce Fields
2011-10-03 15:13 ` Benny Halevy
2011-10-03 15:38 ` J. Bruce Fields
2011-10-04 15:52 ` Benny Halevy
2011-10-04 16:02 ` J. Bruce Fields
2011-09-19 13:15 ` [PATCH 5/5] nfsd4: hash closed stateid's like any other J. Bruce Fields
2011-09-26 22:36 ` [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-09-26 22:44 ` J. Bruce Fields
2011-09-26 22:37 ` J. Bruce Fields
2011-09-26 22:39 ` [PATCH 1/4] nfsd4: move client * to nfs4_stateid, add init_stid helper J. Bruce Fields
2011-09-26 22:39 ` [PATCH 2/4] nfsd4: use idr for stateid's J. Bruce Fields
2011-09-26 22:39 ` [PATCH 3/4] nfsd4: assume test_stateid always has session J. Bruce Fields
2011-09-26 22:39 ` [PATCH 4/4] nfsd4: look up stateid's per clientid J. Bruce Fields
2011-11-08 22:57 ` [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-11-08 22:59 ` [PATCH 1/3] nfsd4: hash lockowners to simplify RELEASE_LOCKOWNER J. Bruce Fields
2011-11-08 22:59 ` [PATCH 2/3] nfsd4: share open and lock owner hash tables J. Bruce Fields
2011-11-08 22:59 ` [PATCH 3/3] nfsd4: add a separate (lockowner, inode) lookup J. Bruce Fields
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110928014004.GF12190@pad.fieldses.org \
--to=bfields@redhat.com \
--cc=bjschuma@netapp.com \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).