* Re: [pnfs]ds ip
[not found] ` <AANLkTilyAgrY6Sa07ef1QmUhV1OkeKSMug45LwBEeCMg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2010-07-19 7:38 ` Benny Halevy
0 siblings, 0 replies; 2+ messages in thread
From: Benny Halevy @ 2010-07-19 7:38 UTC (permalink / raw)
To: quanli gui; +Cc: NFS list
Gui, please keep the nfs list on the Cc so that others may help and/or
benefit in any other way from our discussion :)
To understand more about your problem we'll need a network dump
on the MDS side showing the communication between the client and the MDS
and probably some more information.
Have you installed the latest nfs utils from below?
git://linux-nfs.org/~bhalevy/pnfs-nfs-utils.git
What's your spnfs configuration?
What's in your /etc/exports file?
What are the mount option on the client side?
Can please run tcpdump as root on the MDS to capture the traffic:
# tcpdump -s 0 -w /tmp/nfs.pcap host <client_address>
Also, let's get the debug info from both client and server:
on the server side:
# echo 32767 > /proc/sys/sunrpc/nfsd_debug
on the client side:
# echo 32767 > /proc/sys/sunrpc/nfs_debug
and then mount the mds and run a simple command like
# date > <your_mount_point>/foo
Thanks,
Benny
On Jul. 13, 2010, 12:59 +0300, quanli gui <gqlxj1987@gmail.com> wrote:
> But the MDS that we use has only one fucntion. The data eventually store
> in the DS, but the data has been found in the functions in the MDS.
> for example, I dprintk some message in write_one() in the spnfs_ops.c.
> static int
> write_one(struct inode *inode, loff_t offset, size_t len, char *buf,
> struct file **filp)
> {
> loff_t bufoffset = 0, soffset, pos, snum, soff, tmp;
> size_t iolen;
> int completed = 0, ds, err;
>
> dprintk("buf %s\n",buf);
>
> while (len > 0) {
> tmp = offset;
> soff = do_div(tmp, spnfs_config->stripe_size);
> snum = tmp;
> ds = do_div(tmp, spnfs_config->num_ds);
> if (spnfs_config->dense_striping == 0)
> soffset = offset;
> else {
> tmp = snum;
> do_div(tmp, spnfs_config->num_ds);
> soffset = tmp * spnfs_config->stripe_size + soff;
> }
> if (len < spnfs_config->stripe_size - soff)
> iolen = len;
> else
> iolen = spnfs_config->stripe_size - soff;
>
> pos = soffset;
> err = vfs_write(filp[ds], buf + bufoffset, iolen, &pos);
> if (err < 0)
> return -EIO;
> filp[ds]->f_pos = pos;
> iolen = err;
> completed += iolen;
> len -= iolen;
> offset += iolen;
> bufoffset += iolen;
> }
>
> return completed;
> }
>
> I cp a file whose name is install.log to /mnt/nfs. The message shows that
> buf Installing m17n-db-1.5.3-1.fc10.noarch
> <4>warning: m17n-db-1.5.3-1.fc10: Header V3 DSA signature: NOKEY, key ID
> 4ebfc273
> <4>Installing setup-2.7.4-1.fc10.noarch
> <4>Installing filesystem-2.4.19-1.fc10.i386
> <4>Installing xml-common-0.6.3-26.fc10.noarch
> <4>Installing xkeyboard-config-1.4-4.fc10.noarch
> <4>Installing ca-certificates-2008-7.noarch
> <4>Installing mozilla-filesystem-1.9-2.fc10.i386
> <4>Installing mailcap-2.1.28-1.fc9.noarch
> <4>Installing iso-codes-3.3-1.fc10.noarch
> <4>Installing nodoka-filesystem-0.3.90-2.fc10.noarch
> <4>Installing nodoka-metacity-theme-0.3.90-2.fc10.noarch
> <4>Installing texlive-texmf-errata-2007-4.fc9.noarch
> <4>Installing control-center-filesystem-2.24.0.1-9.fc10.i386
> <4>Installing desktop-backgrounds-basic-9.0.0-5.noarch
> <4>Installing basesystem-10.0-1.noarch
> <4>Installing texlive-texmf-errata-dvips-2007-4.fc9.noarch
> <4>Installing texlive-texmf-errata-fonts-2007-4.fc9.noarch
> <4>Installing ncurses-base-5.6-20.20080927.fc10.i386
> .....
> The content of the buf is the content of the install.log file. And
> spnfs_ops.c is used in MDS. This shows that client translate the data to
> MDS, MDS write the data to DS. This architecture is different from the
> pnfs architecture that people say of. Please give me some answer.
>
> 2010/7/13 Benny Halevy <bhalevy@panasas.com <mailto:bhalevy@panasas.com>>
>
> On Jul. 13, 2010, 5:05 +0300, quanli gui <gqlxj1987@gmail.com
> <mailto:gqlxj1987@gmail.com>> wrote:
> > I use
> > #define dprintk printk
>
> You don't have to modify the kernel to see the debug prints...
> You can turn the debug prints on and off in run time
> by writing the numeric bitmask (in ASCII) into
> /proc/sys/sunrpc/nfs_debug
> for example:
> # echo 32767 > /proc/sys/sunrpc/nfs_debug
>
> > to show some messages in the pnfs process. In the pnfs process, the
> > client would use some functions in nfs4filelayoutdev.c to connect with
> > ds. But from the printk messages, I found that the functions that
> > connect to the ds printk the ip message, which is belong to the
> mds. Do
> > the client only connect to the mds? Do the mds become the storage
> server
> > which connect to the client? This conficts the pnfs
> architecture.Please
> > give me some answers.
>
> The client should communicate with the DS for I/O operations.
> If the MSD has dual function, both as a DS and a MDS, the DS ip address
> may be the same as the MDS's.
>
> Can you give an example of a particular message you're worried about?
>
> What do you have running on the server side?
>
> Benny
>
>
^ permalink raw reply [flat|nested] 2+ messages in thread