From: "Paweł Sikora" <pawel.sikora@agmk.net>
To: "J. Bruce Fields" <bfields@fieldses.org>
Cc: Steve Dickson <SteveD@redhat.com>,
linux-nfs@vger.kernel.org, baggins@pld-linux.org,
Trond.Myklebust@netapp.com
Subject: Re: mount.nfs: cannot allocate memory.
Date: Thu, 17 Jan 2013 15:59:57 +0100 [thread overview]
Message-ID: <3670768.htx4NKnPRQ@pawels> (raw)
In-Reply-To: <20130117134959.GE6598@fieldses.org>
On Thursday 17 of January 2013 08:49:59 J. Bruce Fields wrote:
> On Wed, Jan 16, 2013 at 10:18:51PM +0100, Paweł Sikora wrote:
> > On Wednesday 16 of January 2013 15:15:10 J. Bruce Fields wrote:
> > > On Wed, Jan 16, 2013 at 09:07:45PM +0100, Paweł Sikora wrote:
> > > > On Wednesday 16 of January 2013 14:39:32 J. Bruce Fields wrote:
> > > > > On Wed, Jan 16, 2013 at 08:03:14PM +0100, Paweł Sikora wrote:
> > > > > > [259176.973751] NFS: nfs mount opts='soft,addr=10.0.2.28,vers=3,proto=tcp,mountvers=3,mountproto=udp,mountport=50252'
> > > > > > [259176.973757] NFS: parsing nfs mount option 'soft'
> > > > > > [259176.973759] NFS: parsing nfs mount option 'addr=10.0.2.28'
> > > > > > [259176.973765] NFS: parsing nfs mount option 'vers=3'
> > > > > > [259176.973769] NFS: parsing nfs mount option 'proto=tcp'
> > > > > > [259176.973772] NFS: parsing nfs mount option 'mountvers=3'
> > > > > > [259176.973776] NFS: parsing nfs mount option 'mountproto=udp'
> > > > > > [259176.973779] NFS: parsing nfs mount option 'mountport=50252'
> > > > > > [259176.973784] NFS: MNTPATH: '/R10'
> > > > > > [259176.973788] NFS: sending MNT request for nexus:/R10
> > > > > > [259176.974620] NFS: received 1 auth flavors
> > > > > > [259176.974623] NFS: auth flavor[0]: 1
> > > > > > [259176.974640] NFS: MNT request succeeded
> > > > > > [259176.974643] NFS: using auth flavor 1
> > > > > > [259176.974688] --> nfs_init_server()
> > > > > > [259176.974691] --> nfs_get_client(nexus,v3)
> > > > > > [259176.974698] NFS: get client cookie (0xffff88021146f800/0xffff8800ceb06640)
> > > > > > [259176.975704] <-- nfs_init_server() = 0 [new ffff88021146f800]
> > > > > > [259176.975708] --> nfs_probe_fsinfo()
> > > > > > [259176.975711] NFS call fsinfo
> > > > > > [259176.975959] NFS reply fsinfo: -116
> > > > >
> > > > > That's ESTALE. Might be interesting to see the network traffic between
> > > > > client and server.
> > > >
> > > > here's the tcpdump result: http://pluto.agmk.net/kernel/nfs.mount.estale.dump
> > >
> > > On just a very quick skim (you may want to verify to see I've got it
> > > right), frame 30 shows the server returning a filehandle in a MNT reply,
> > > then frame 48 shows the same client that got that MNT reply using the
> > > same filehandle in an FSINFO reply, and getting an NFS3ERR_STALE
> > > response.
> > >
> > > Offhand seems like a server bug. Might conceivably happen if there was
> > > some confusion whether the client was authorized to access that export?
> >
> > i have such nfs problems with only one server which have complicated exports/local-binds:
> >
> > fstab:
> >
> > /dev/md0 / ext3 defaults 1 1
> > /dev/md1 /R0 ext4 defaults,noatime 1 2
> > /dev/md2 /R10 ext4 defaults,noatime 1 2
> > /home /remote/nexus/home none bind
> > /R0/atest_home /home/atest none bind
> > /R0/farm/ftp /var/lib/ftp none bind
> > /R0 /remote/nexus/R0 none bind
> > /R10 /remote/nexus/R10 none bind
> >
> > exports:
> >
> > /home *(rw,sync,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide)
> > /R0 *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide,crossmnt)
> > /R0/farm/ftp *(rw,async,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide,crossmnt)
> > /R10 *(rw,sync,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide,crossmnt)
> > /R10/farm *(rw,sync,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide,crossmnt)
> > /R10/farm/sources *(rw,sync,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide,crossmnt)
> > /R10/farm/soft *(rw,sync,no_wdelay,no_subtree_check,no_root_squash,insecure_locks,nohide,crossmnt)
> >
> > and finally, the /R0/farm contains cross symlinks to R10 via binded dirs:
> >
> > soft -> /remote/nexus/R10/farm/soft
> > sources -> /remote/nexus/R10/farm/sources
> >
> >
> > maybe this crappy setup exposes some bug on the server side?
>
> So in the above setup, /R0 and /remote/nexus/R0, for example, both point
> to the same superblock.
>
> The filehandle contains only a reference to the superblock, with no
> information about how it was arrived at. When nfsd gets the filehandle
> it's resolved in two steps:
>
> - first it asks mountd to tell it a path for the given
> filehandle data
> - then it asks mountd for export options for that path
>
> You can see the former in /proc/net/rpc/nfsd.fh/content, and the latter
> in /proc/net/rpc/nfsd.export/content, so it might be interesting to
> compare those two after a success and after a failure.
>
> Since there are multiple possible paths that each filehandle could be
> mapped to, I suspect the outcome depends on which mountd chooses, which
> could be random. But I don't immediately see how that's causing the
> problem, since all your exports have the same option.
before failing mount attempt i see such values on the server side:
[root@nexus ~]# ./nfs_debug.sh
+ cat /proc/net/rpc/nfsd.fh/content
#domain fsidtype fsid [path]
* 6 0x40694572b34adc7d7535a982d692a864 /R10
* 1 0x00000000 /
* 6 0x0f5a4f040e452cbc5ba412a3a206b7c3 /R0
# * 6 0xa75dfe547f9b43ac0000000000000000
# * 6 0xf0ec5c351974d8650000000000000000
+ cat /proc/net/rpc/nfsd.export/content
#path domain(flags)
/ *(ro,no_root_squash,sync,no_wdelay,no_subtree_check,v4root,fsid=0,uuid=44053428:4cfdc05e:00000000:00000000)
/R0 *(rw,no_root_squash,sync,wdelay,nohide,crossmnt,no_subtree_check,insecure_locks,uuid=044f5a0f:bc2c450e:a312a45b:c3b706a2)
/R10 *(rw,no_root_squash,sync,wdelay,nohide,crossmnt,no_subtree_check,insecure_locks,uuid=72456940:7ddc4ab3:82a93575:64a892d6)
after R0 mount attempt there're two entries for R0 with different uuids. is it correct behaviour?
[root@nexus ~]# ./nfs_debug.sh
+ cat /proc/net/rpc/nfsd.fh/content
#domain fsidtype fsid [path]
* 6 0x40694572b34adc7d7535a982d692a864 /R10
* 1 0x00000000 /
* 6 0x0f5a4f040e452cbc5ba412a3a206b7c3 /R0
# * 6 0xa75dfe547f9b43ac0000000000000000
# * 6 0xf0ec5c351974d8650000000000000000
+ cat /proc/net/rpc/nfsd.export/content
#path domain(flags)
/ *(ro,no_root_squash,sync,no_wdelay,no_subtree_check,v4root,fsid=0,uuid=44053428:4cfdc05e:00000000:00000000)
/R0 *(rw,no_root_squash,sync,wdelay,nohide,crossmnt,no_subtree_check,insecure_locks,uuid=044f5a0f:bc2c450e:a312a45b:c3b706a2)
/R0 *(rw,no_root_squash,sync,wdelay,nohide,crossmnt,no_subtree_check,insecure_locks,uuid=54fe5da7:ac439b7f:00000000:00000000)
/R10 *(rw,no_root_squash,sync,wdelay,nohide,crossmnt,no_subtree_check,insecure_locks,uuid=72456940:7ddc4ab3:82a93575:64a892d6)
next prev parent reply other threads:[~2013-01-17 15:00 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-01-03 9:27 mount.nfs: cannot allocate memory Paweł Sikora
2013-01-03 14:42 ` Steve Dickson
2013-01-04 5:16 ` Paweł Sikora
2013-01-04 6:03 ` fanchaoting
2013-01-04 15:20 ` Paweł Sikora
2013-01-05 10:08 ` Paweł Sikora
2013-01-07 14:47 ` Steve Dickson
2013-01-09 17:44 ` Paweł Sikora
2013-01-16 19:03 ` Paweł Sikora
2013-01-16 19:39 ` J. Bruce Fields
2013-01-16 20:07 ` Paweł Sikora
2013-01-16 20:15 ` J. Bruce Fields
2013-01-16 21:18 ` Paweł Sikora
2013-01-17 13:49 ` J. Bruce Fields
2013-01-17 14:59 ` Paweł Sikora [this message]
2013-01-16 20:14 ` Myklebust, Trond
-- strict thread matches above, loose matches on Subject: below --
2012-06-12 22:44 Mark Lord
2012-06-12 22:50 ` Mark Lord
2012-06-12 22:58 ` Mark Lord
2012-06-12 23:08 ` Mark Lord
2012-06-12 23:28 ` Myklebust, Trond
2012-06-12 23:46 ` Mark Lord
2012-06-12 23:16 ` Dave Jones
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3670768.htx4NKnPRQ@pawels \
--to=pawel.sikora@agmk.net \
--cc=SteveD@redhat.com \
--cc=Trond.Myklebust@netapp.com \
--cc=baggins@pld-linux.org \
--cc=bfields@fieldses.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.