From: "J. Bruce Fields" <bfields@redhat.com>
To: Bryan Schumaker <bjschuma@netapp.com>
Cc: linux-nfs@vger.kernel.org
Subject: Re: [PATCH 03/25] nfsd4: extend state lock over seqid replay logic
Date: Tue, 27 Sep 2011 21:40:04 -0400 [thread overview]
Message-ID: <20110928014004.GF12190@pad.fieldses.org> (raw)
In-Reply-To: <4E82001C.5030005@netapp.com>
On Tue, Sep 27, 2011 at 12:55:56PM -0400, Bryan Schumaker wrote:
> Hi Bruce,
>
> I'm getting the following warning that I was able to bisect to this patch:
Hm. Was it doing a LOCKU at the time, I wonder? It looks like I missed
a case here.... I'll investigate.
--b.
>
> [ 142.149710] ------------[ cut here ]------------
> [ 142.150014] WARNING: at kernel/mutex-debug.c:78 debug_mutex_unlock+0xda/0xe0()
> [ 142.150258] Hardware name: Bochs
> [ 142.150407] Modules linked in: md5 nfsd exportfs nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ext2 snd_hda_intel snd_hda_codec snd_hwdep psmouse i2c_piix4 evdev serio_raw pcspkr virtio_balloon snd_pcm snd_timer snd soundcore snd_page_alloc floppy i2c_core button processor ext4 mbcache jbd2 crc16 pata_acpi uhci_hcd ata_piix libata usbcore scsi_mod virtio_net virtio_pci virtio_blk virtio virtio_ring
> [ 142.152927] Pid: 742, comm: nfsd Not tainted 3.1.0-rc1-SLIM+ #9
> [ 142.152927] Call Trace:
> [ 142.152927] [<ffffffff8105fa4f>] warn_slowpath_common+0x7f/0xc0
> [ 142.152927] [<ffffffff8105faaa>] warn_slowpath_null+0x1a/0x20
> [ 142.152927] [<ffffffff810960ca>] debug_mutex_unlock+0xda/0xe0
> [ 142.152927] [<ffffffff813e4200>] __mutex_unlock_slowpath+0x80/0x140
> [ 142.152927] [<ffffffff813e42ce>] mutex_unlock+0xe/0x10
> [ 142.152927] [<ffffffffa03bd3f5>] nfs4_lock_state+0x35/0x40 [nfsd]
> [ 142.152927] [<ffffffffa03b0b71>] nfsd4_proc_compound+0x2a1/0x690 [nfsd]
> [ 142.152927] [<ffffffffa039f9fb>] nfsd_dispatch+0xeb/0x230 [nfsd]
> [ 142.152927] [<ffffffffa02b1055>] svc_process_common+0x345/0x690 [sunrpc]
> [ 142.152927] [<ffffffff81058d10>] ? try_to_wake_up+0x280/0x280
> [ 142.152927] [<ffffffffa02b16e2>] svc_process+0x102/0x150 [sunrpc]
> [ 142.152927] [<ffffffffa039f0bd>] nfsd+0xbd/0x160 [nfsd]
> [ 142.152927] [<ffffffffa039f000>] ? 0xffffffffa039efff
> [ 142.152927] [<ffffffff8108230c>] kthread+0x8c/0xa0
> [ 142.152927] [<ffffffff813e8694>] kernel_thread_helper+0x4/0x10
> [ 142.152927] [<ffffffff81082280>] ? kthread_worker_fn+0x190/0x190
> [ 142.152927] [<ffffffff813e8690>] ? gs_change+0x13/0x13
> [ 142.152927] ---[ end trace 1b4070dc432138aa ]---
>
> I can duplicate it with this python script, the warning shows up on the server after (during?) the f.close() line:
>
> #!/usr/bin/python
> import sys
> import fcntl
> import struct
> import datetime
>
> f = open(sys.argv[1], 'rw+')
> print "Attempting to lock file:", sys.argv[1]
> lockreq = struct.pack('hhllhh', fcntl.F_WRLCK, 0, 0, 0, 0, 0)
> rv = fcntl.fcntl(f, fcntl.F_SETLKW, lockreq)
> raw_input("Press enter when you are ready to continue... ")
> f.close()
>
> - Bryan
>
> On 09/14/2011 07:44 AM, J. Bruce Fields wrote:
> > There are currently a couple races in the seqid replay code: a
> > retransmission could come while we're still encoding the original reply,
> > or a new seqid-mutating call could come as we're encoding a replay.
> >
> > So, extend the state lock over the encoding (both encoding of a replayed
> > reply and caching of the original encoded reply).
> >
> > I really hate doing this, and previously added the stateowner
> > reference-counting code to avoid it (which was insufficient)--but I
> > don't see a less complicated alternative at the moment.
> >
> > Signed-off-by: J. Bruce Fields <bfields@redhat.com>
> > ---
> > fs/nfsd/nfs4proc.c | 5 +++--
> > fs/nfsd/nfs4state.c | 12 ++++++++----
> > 2 files changed, 11 insertions(+), 6 deletions(-)
> >
> > diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
> > index 50bae74..50063a8 100644
> > --- a/fs/nfsd/nfs4proc.c
> > +++ b/fs/nfsd/nfs4proc.c
> > @@ -408,8 +408,8 @@ out:
> > if (open->op_stateowner) {
> > nfs4_get_stateowner(open->op_stateowner);
> > cstate->replay_owner = open->op_stateowner;
> > - }
> > - nfs4_unlock_state();
> > + } else
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -1227,6 +1227,7 @@ encode_op:
> > be32_to_cpu(status));
> >
> > if (cstate->replay_owner) {
> > + nfs4_unlock_state();
> > nfs4_put_stateowner(cstate->replay_owner);
> > cstate->replay_owner = NULL;
> > }
> > diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> > index bc1a9db..6cf729a 100644
> > --- a/fs/nfsd/nfs4state.c
> > +++ b/fs/nfsd/nfs4state.c
> > @@ -3501,7 +3501,8 @@ nfsd4_open_confirm(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> >
> > nfsd4_create_clid_dir(sop->so_client);
> > out:
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -3568,7 +3569,8 @@ nfsd4_open_downgrade(struct svc_rqst *rqstp,
> > memcpy(&od->od_stateid, &stp->st_stateid, sizeof(stateid_t));
> > status = nfs_ok;
> > out:
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -3609,7 +3611,8 @@ nfsd4_close(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> > if (list_empty(&so->so_stateids))
> > move_to_close_lru(so);
> > out:
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
> > @@ -4071,7 +4074,8 @@ nfsd4_lock(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
> > out:
> > if (status && lock->lk_is_new && lock_sop)
> > release_lockowner(lock_sop);
> > - nfs4_unlock_state();
> > + if (!cstate->replay_owner)
> > + nfs4_unlock_state();
> > return status;
> > }
> >
>
next prev parent reply other threads:[~2011-09-28 1:40 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-14 11:44 [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-09-14 11:44 ` [PATCH 01/25] nfsd4: centralize handling of replay owners J. Bruce Fields
2011-09-14 11:44 ` [PATCH 02/25] nfsd4: cleanup seqid op stateowner usage J. Bruce Fields
2011-09-14 11:44 ` [PATCH 03/25] nfsd4: extend state lock over seqid replay logic J. Bruce Fields
2011-09-27 16:55 ` Bryan Schumaker
2011-09-28 1:40 ` J. Bruce Fields [this message]
2011-09-28 1:49 ` J. Bruce Fields
2011-09-28 13:18 ` Bryan Schumaker
2011-09-28 13:44 ` J. Bruce Fields
2011-09-14 11:45 ` [PATCH 04/25] nfsd4: eliminate impossible open replay case J. Bruce Fields
2011-09-14 11:45 ` [PATCH 05/25] nfsd4: drop most stateowner refcounting J. Bruce Fields
2011-09-14 11:45 ` [PATCH 06/25] nfsd4: eliminate unused lt_stateowner J. Bruce Fields
2011-09-14 11:45 ` [PATCH 07/25] nfsd4: share common seqid checks J. Bruce Fields
2011-09-14 11:45 ` [PATCH 08/25] nfsd4: simplify check_open logic J. Bruce Fields
2011-09-14 11:45 ` [PATCH 09/25] nfsd4: move double-confirm test to open_confirm J. Bruce Fields
2011-09-14 11:45 ` [PATCH 10/25] nfsd4: move CLOSE_STATE special case to caller J. Bruce Fields
2011-09-14 11:45 ` [PATCH 11/25] nfsd4: split stateowners into open and lockowners J. Bruce Fields
2011-09-14 11:45 ` [PATCH 12/25] nfsd4: split out some free_generic_stateid code J. Bruce Fields
2011-09-14 11:45 ` [PATCH 13/25] nfsd4: rearrange to avoid a forward reference J. Bruce Fields
2011-09-14 11:45 ` [PATCH 14/25] nfsd4: split up find_stateid J. Bruce Fields
2011-09-14 11:45 ` [PATCH 15/25] nfsd4: split preprocess_seqid, cleanup J. Bruce Fields
2011-09-14 11:45 ` [PATCH 16/25] nfsd4: pass around typemask instead of flags J. Bruce Fields
2011-09-14 11:45 ` [PATCH 17/25] nfsd4: rename init_stateid J. Bruce Fields
2011-09-14 11:45 ` [PATCH 18/25] nfsd4: remove redundant stateid initialization J. Bruce Fields
2011-09-14 11:45 ` [PATCH 19/25] nfsd4: move some of nfs4_stateid into a separate structure J. Bruce Fields
2011-09-14 11:45 ` [PATCH 20/25] nfsd4: add common dl_stid field to delegation J. Bruce Fields
2011-09-14 11:45 ` [PATCH 21/25] nfsd4: share common stid-hashing helper function J. Bruce Fields
2011-09-14 11:45 ` [PATCH 22/25] nfsd4: hash deleg stateid's like any other J. Bruce Fields
2011-09-14 11:45 ` [PATCH 23/25] nfsd4: fix test_stateid for delegation stateid's J. Bruce Fields
2011-09-14 11:45 ` [PATCH 24/25] nfsd4: use deleg changes to cleanup preprocess_stateid_op J. Bruce Fields
2011-09-14 11:45 ` [PATCH 25/25] nfsd4: better stateid hashing J. Bruce Fields
2011-09-19 13:14 ` [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-09-19 13:15 ` [PATCH 1/5] nfsd4: replace oo_confirmed by flag bit J. Bruce Fields
2011-09-19 13:15 ` [PATCH 2/5] nfsd4: match close replays on stateid, not open owner id J. Bruce Fields
2011-09-19 13:15 ` [PATCH 3/5] nfsd4: simplify free_stateid J. Bruce Fields
2011-09-19 13:15 ` [PATCH 4/5] nfsd4: construct stateid from clientid and counter J. Bruce Fields
2011-09-27 16:10 ` Bryan Schumaker
2011-09-28 15:49 ` J. Bruce Fields
2011-10-03 14:43 ` Benny Halevy
2011-10-03 14:57 ` J. Bruce Fields
2011-10-03 15:13 ` Benny Halevy
2011-10-03 15:38 ` J. Bruce Fields
2011-10-04 15:52 ` Benny Halevy
2011-10-04 16:02 ` J. Bruce Fields
2011-09-19 13:15 ` [PATCH 5/5] nfsd4: hash closed stateid's like any other J. Bruce Fields
2011-09-26 22:36 ` [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-09-26 22:44 ` J. Bruce Fields
2011-09-26 22:37 ` J. Bruce Fields
2011-09-26 22:39 ` [PATCH 1/4] nfsd4: move client * to nfs4_stateid, add init_stid helper J. Bruce Fields
2011-09-26 22:39 ` [PATCH 2/4] nfsd4: use idr for stateid's J. Bruce Fields
2011-09-26 22:39 ` [PATCH 3/4] nfsd4: assume test_stateid always has session J. Bruce Fields
2011-09-26 22:39 ` [PATCH 4/4] nfsd4: look up stateid's per clientid J. Bruce Fields
2011-11-08 22:57 ` [PATCH 00/25] nfsd4 state cleanup J. Bruce Fields
2011-11-08 22:59 ` [PATCH 1/3] nfsd4: hash lockowners to simplify RELEASE_LOCKOWNER J. Bruce Fields
2011-11-08 22:59 ` [PATCH 2/3] nfsd4: share open and lock owner hash tables J. Bruce Fields
2011-11-08 22:59 ` [PATCH 3/3] nfsd4: add a separate (lockowner, inode) lookup J. Bruce Fields
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110928014004.GF12190@pad.fieldses.org \
--to=bfields@redhat.com \
--cc=bjschuma@netapp.com \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.