* [Fwd: Re: NFSv4 uninitialized mtime]
@ 2007-06-29 22:28 Erik A. Espinoza
0 siblings, 0 replies; only message in thread
From: Erik A. Espinoza @ 2007-06-29 22:28 UTC (permalink / raw)
To: nfs
[-- Attachment #1: Type: text/plain, Size: 10411 bytes --]
Greetings,
Here are the results
-bash-3.00$ stat badmtime.out
File: `badmtime.out'
Size: 0 Blocks: 1 IO Block: 1048576 regular
empty file
Device: 1ch/28d Inode: 107116 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 5026/anderson) Gid: ( 5026/anderson)
Access: 2007-06-29 14:42:51.026996877 -0700
Modify: 1977-06-21 02:23:11.159764000 -0700
Change: 2007-06-29 14:42:51.039780780 -0700
This was on GIT kernel from git://git.linux-nfs.org/pub/linux/nfs-2.6
based on 2.6.22-rc6. Attached is the tcpdump of the traffic.
Thanks,
Erik
Jeff Layton wrote:
> On Wed, 27 Jun 2007 19:53:27 -0700
> Stuart Anderson <anderson@ligo.caltech.edu> wrote:
>
>> On Wed, Jun 27, 2007 at 09:15:59PM -0400, Jeff Layton wrote:
>>> On Wed, 27 Jun 2007 17:59:57 -0700
>>> Stuart Anderson <anderson@ligo.caltech.edu> wrote:
>>>
>>>> More precisely, applying this patch to the 2.6.20.14 kernel plus the
>>>> revalidate-the-fsid patch did not result in any changes. Does the
>>>> O_EXCL patch require some other supporting patches relative to 2.6.20.14?
>>>>
>>>> or perhaps it is necessary to rebase to a newer kernel before applying it?
>>>> even tough it applies, builds and runs with 2.6.20.14?
>>>>
>>>> Thanks.
>>>>
>>> With Linux, both client and server side were broken in this respect.
>>> The server didn't set the bitmask in the reply and the client didn't
>>> look at it anyway. It's possible that Solaris server is broken in this
>>> regard as well. It would be interesting to see a capture here,
>>> particularly one containing the reply from the CREATE.
>> Possible, but when we switch the client from Linux to Solaris and share
>> the same filesytem from the same server there are no O_EXCL problems.
>>
>
> NFSv3 didn't use this bitmask, so clients tended to just clobber the
> mtime and atime on the subsequent setattr. It's possible that Solaris
> is using the old semantics here with NFSv4. Of course, all of this is
> speculation :-), a capture should give us a better clue.
>
>> I will see if we can capture the bits. Any recomendations on which tool
>> with what options would be most useful to capture the useful traffic?
>>
>
> tcpdump is what I'd recommend. It's pretty standard issue on Linux
> boxes these days. From the client:
>
> # tcpdump -i eth0 -s 0 -w /tmp/nfs4_create.pcap host nfs_server and port 2049
>
> You might need to modify it some depending on your setup. that should
> capture all NFS packets into /tmp/nfs4_create.pcap. Start up tcpdump,
> run the reproducer, and then kill the tcpdump. That should give us a
> pretty small capture file that just has what we need in it. If you want
> to look at it yourself, then you can use wireshark (aka ethereal) on
> Linux.
>
> I have a solaris 11 qemu image. I'll try to reproduce this using it as
> a server today sometime.
>
>>> Exactly what version of Solaris is the server running?
>> # cat /etc/release
>> Solaris 10 11/06 s10x_u3wos_10 X86
>> Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
>> Use is subject to license terms.
>> Assembled 14 November 2006
>> # uname -v
>> Generic_118855-36
>>
>>
>> Somewhat related is the reason that we are tring NFSV4 on Linux clients is
>> that with NFSV3 ACL's do not work between Linux and Solaris if the filesystem
>> on the Solaris server is ZFS, e.g., cp -p will generate interesting error
>> messages. While I would rather stay with NFSV4 if possible, has anyone
>> gotten this configuration to work with V3?
>>
>>>> On Wed, Jun 27, 2007 at 05:49:04PM -0700, Stuart Anderson wrote:
>>>>> Unfortunately this patch did not have any observable affects.
>>>>>
>>>>> On Wed, Jun 27, 2007 at 07:43:25PM -0400, Trond Myklebust wrote:
>>>>>> On Wed, 2007-06-27 at 16:31 -0700, Stuart Anderson wrote:
>>>>>>> The following simple program creates files with un-initialized mtime values
>>>>>>> on a Linux NFSv4 client mounting from a Solaris NFSv4 server--at least the
>>>>>>> mtimes are wildly different each time the program is run. The problem is
>>>>>>> reproducible but does not happen under any of the following circumstances:
>>>>>>>
>>>>>>> 1) Drop O_EXCL from open() call.
>>>>>>> 2) NFS mount using v3.
>>>>>>> 3) Switch NFS v4 client from Linux to Solaris.
>>>>>>>
>>>>>>> This is on an FC4 machine with the 2.6.20.14 kernel plus Trond's recent
>>>>>>> revalidate-the-fsid-on-the-current-dir-not-the-root-dir patch
>>>>>>> and nfs-utils 1.0.9-16 backported from CentOS 5.
>>>>>>>
>>>>>>> Thanks.
>>>>>> Doesn't Jeff's patch fix it?
>>>>>>
>>>>>> Trond
>>>>>>
>>>>>>
>>>>>> From: Jeff Layton <jlayton@redhat.com>
>>>>>> Date: Tue, 5 Jun 2007 14:49:03 -0400
>>>>>> NFS4: on a O_EXCL OPEN make sure SETATTR sets the fields holding the
>>>>>> verifier
>>>>>> Subject: No Subject
>>>>>>
>>>>>> The Linux NFS4 client simply skips over the bitmask in an O_EXCL open
>>>>>> call and so it doesn't bother to reset any fields that may be holding
>>>>>> the verifier. This patch has us save the first two words of the bitmask
>>>>>> (which is all the current client has #defines for). The client then
>>>>>> later checks this bitmask and turns on the appropriate flags in the
>>>>>> sattr->ia_verify field for the following SETATTR call.
>>>>>>
>>>>>> This patch only currently checks to see if the server used the atime
>>>>>> and mtime slots for the verifier (which is what the Linux server uses
>>>>>> for this). I'm not sure of what other fields the server could
>>>>>> reasonably use, but adding checks for others should be trivial.
>>>>>>
>>>>>> Signed-off-by: Jeff Layton <jlayton@redhat.com>
>>>>>> Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
>>>>>> ---
>>>>>>
>>>>>> fs/nfs/nfs4proc.c | 20 ++++++++++++++++++++
>>>>>> fs/nfs/nfs4xdr.c | 9 +++++++--
>>>>>> include/linux/nfs4.h | 1 +
>>>>>> include/linux/nfs_xdr.h | 1 +
>>>>>> 4 files changed, 29 insertions(+), 2 deletions(-)
>>>>>>
>>>>>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
>>>>>> index 3cc7544..fee2d14 100644
>>>>>> --- a/fs/nfs/nfs4proc.c
>>>>>> +++ b/fs/nfs/nfs4proc.c
>>>>>> @@ -943,6 +943,22 @@ static struct nfs4_state *nfs4_open_delegated(struct inode *inode, int flags, st
>>>>>> }
>>>>>>
>>>>>> /*
>>>>>> + * on an EXCLUSIVE create, the server should send back a bitmask with FATTR4-*
>>>>>> + * fields corresponding to attributes that were used to store the verifier.
>>>>>> + * Make sure we clobber those fields in the later setattr call
>>>>>> + */
>>>>>> +static inline void nfs4_exclusive_attrset(struct nfs4_opendata *opendata, struct iattr *sattr)
>>>>>> +{
>>>>>> + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_ACCESS) &&
>>>>>> + !(sattr->ia_valid & ATTR_ATIME_SET))
>>>>>> + sattr->ia_valid |= ATTR_ATIME;
>>>>>> +
>>>>>> + if ((opendata->o_res.attrset[1] & FATTR4_WORD1_TIME_MODIFY) &&
>>>>>> + !(sattr->ia_valid & ATTR_MTIME_SET))
>>>>>> + sattr->ia_valid |= ATTR_MTIME;
>>>>>> +}
>>>>>> +
>>>>>> +/*
>>>>>> * Returns a referenced nfs4_state
>>>>>> */
>>>>>> static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct iattr *sattr, struct rpc_cred *cred, struct nfs4_state **res)
>>>>>> @@ -973,6 +989,9 @@ static int _nfs4_do_open(struct inode *dir, struct path *path, int flags, struct
>>>>>> if (status != 0)
>>>>>> goto err_opendata_free;
>>>>>>
>>>>>> + if (opendata->o_arg.open_flags & O_EXCL)
>>>>>> + nfs4_exclusive_attrset(opendata, sattr);
>>>>>> +
>>>>>> status = -ENOMEM;
>>>>>> state = nfs4_opendata_to_nfs4_state(opendata);
>>>>>> if (state == NULL)
>>>>>> @@ -1784,6 +1803,7 @@ nfs4_proc_create(struct inode *dir, struct dentry *dentry, struct iattr *sattr,
>>>>>> status = nfs4_do_setattr(state->inode, &fattr, sattr, state);
>>>>>> if (status == 0)
>>>>>> nfs_setattr_update_inode(state->inode, sattr);
>>>>>> + nfs_post_op_update_inode(state->inode, &fattr);
>>>>>> }
>>>>>> if (status == 0 && (nd->flags & LOOKUP_OPEN) != 0)
>>>>>> status = nfs4_intent_set_file(nd, &path, state);
>>>>>> diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
>>>>>> index 8003c91..5efd314 100644
>>>>>> --- a/fs/nfs/nfs4xdr.c
>>>>>> +++ b/fs/nfs/nfs4xdr.c
>>>>>> @@ -3269,7 +3269,7 @@ static int decode_delegation(struct xdr_stream *xdr, struct nfs_openres *res)
>>>>>> static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
>>>>>> {
>>>>>> __be32 *p;
>>>>>> - uint32_t bmlen;
>>>>>> + uint32_t savewords, bmlen, i;
>>>>>> int status;
>>>>>>
>>>>>> status = decode_op_hdr(xdr, OP_OPEN);
>>>>>> @@ -3287,7 +3287,12 @@ static int decode_open(struct xdr_stream *xdr, struct nfs_openres *res)
>>>>>> goto xdr_error;
>>>>>>
>>>>>> READ_BUF(bmlen << 2);
>>>>>> - p += bmlen;
>>>>>> + savewords = min_t(uint32_t, bmlen, NFS4_BITMAP_SIZE);
>>>>>> + for (i = 0; i < savewords; ++i)
>>>>>> + READ32(res->attrset[i]);
>>>>>> +
>>>>>> + p += (bmlen - savewords);
>>>>>> +
>>>>>> return decode_delegation(xdr, res);
>>>>>> xdr_error:
>>>>>> dprintk("%s: Bitmap too large! Length = %u\n", __FUNCTION__, bmlen);
>>>>>> diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
>>>>>> index 7e7f33a..8726491 100644
>>>>>> --- a/include/linux/nfs4.h
>>>>>> +++ b/include/linux/nfs4.h
>>>>>> @@ -15,6 +15,7 @@
>>>>>>
>>>>>> #include <linux/types.h>
>>>>>>
>>>>>> +#define NFS4_BITMAP_SIZE 2
>>>>>> #define NFS4_VERIFIER_SIZE 8
>>>>>> #define NFS4_STATEID_SIZE 16
>>>>>> #define NFS4_FHSIZE 128
>>>>>> diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
>>>>>> index 10c26ed..f7100df 100644
>>>>>> --- a/include/linux/nfs_xdr.h
>>>>>> +++ b/include/linux/nfs_xdr.h
>>>>>> @@ -144,6 +144,7 @@ struct nfs_openres {
>>>>>> nfs4_stateid delegation;
>>>>>> __u32 do_recall;
>>>>>> __u64 maxsize;
>>>>>> + __u32 attrset[NFS4_BITMAP_SIZE];
>>>>>> };
>>>>>>
>>>>>> /*
>>>>>
>>>>> --
>>>>> Stuart Anderson anderson@ligo.caltech.edu
>>>>> http://www.ligo.caltech.edu/~anderson
>>>> --
>>>> Stuart Anderson anderson@ligo.caltech.edu
>>>> http://www.ligo.caltech.edu/~anderson
>>>
>>> --
>>> Jeff Layton <jlayton@redhat.com>
>> --
>> Stuart Anderson anderson@ligo.caltech.edu
>> http://www.ligo.caltech.edu/~anderson
>
>
--
Erik A. Espinoza
Systems Administrator
LIGO/Caltech - MS 18-34
Pasadena, CA 91125
Ph: 626-395-8517
[-- Attachment #2: nfs4_create.pcap --]
[-- Type: application/octet-stream, Size: 7478 bytes --]
[-- Attachment #3: Type: text/plain, Size: 286 bytes --]
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
[-- Attachment #4: Type: text/plain, Size: 140 bytes --]
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2007-06-29 22:28 UTC | newest]
Thread overview: (only message) (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-06-29 22:28 [Fwd: Re: NFSv4 uninitialized mtime] Erik A. Espinoza
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.