re-export NFS mounted disk to be NFS mounted?

re-export NFS mounted disk to be NFS mounted?

[Yu Chen]

Hello,

I have a cluster, the disk space is on a Xserver, it's NFS exported,  
and mounted on the cluster's head node without problem (mounted on / 
mnt/nfs), then I exported the "/mnt/nfs" directory, then tried to  
mount it on the nodes in the cluster (mount -t nfs headnode:/mnt/nfs / 
mnt/tmp), it gave error: mount ... failed, reason given by server:  
Permission denied.

my Xserver nfs exports entry has this: /Volumes/DataRAID  -alldirs - 
maproot=nobody -sec=sys -network my.headnode.network -mask 255.255.255.0

my headnode mounted /Volumes/DataRAID on /mnt/nfs then exported as nfs  
exports entry:
/mnt/nfs 10.0.0.0/255.255.255.0(rw,sync)

Anybody has any suggestions?

Thanks in advance.
CY

Re: re-export NFS mounted disk to be NFS mounted?

[Yuri Csapo]

AFAIK that is not possible on Linux with the stock kernel nfs server. It 
*should*  be possible with the userspace server, but I'm not sure it's 
supported. Maybe someone who has actually done it (as opposed to playing 
with it) should comment.

OTOH it doesn't seem like a good idea anyway... NFS's performance is 
really bad and what you're doing will multiply bad x 2. I know sometimes 
you need to work with what you have but if your cluster is in any way 
I/O sensitive you should think about getting some local disk space for 
it, at least.

Yu Chen wrote:
> Hello,
> 
> I have a cluster, the disk space is on a Xserver, it's NFS exported, and 
> mounted on the cluster's head node without problem (mounted on 
> /mnt/nfs), then I exported the "/mnt/nfs" directory, then tried to mount 
> it on the nodes in the cluster (mount -t nfs headnode:/mnt/nfs 
> /mnt/tmp), it gave error: mount ... failed, reason given by server: 
> Permission denied.
> 
> my Xserver nfs exports entry has this: /Volumes/DataRAID  -alldirs 
> -maproot=nobody -sec=sys -network my.headnode.network -mask 255.255.255.0
> 
> my headnode mounted /Volumes/DataRAID on /mnt/nfs then exported as nfs 
> exports entry:
> /mnt/nfs 10.0.0.0/255.255.255.0(rw,sync)
> 
> Anybody has any suggestions?
> 
> Thanks in advance.
> CY
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-admin" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Yuri Csapo
Academic Computing & Networking
Colorado School of Mines
CT-256
Phone:  (303) 273-3503
Fax:      (303) 273-3475
Email:   ycsapo@mines.edu

Please use the following link to open a service request:
http://helpdesk.mines.edu
===========================================
With a PC, I always felt limited
by the software available.
On Unix, I am limited only by my knowledge.
--Peter J. Schoenster

Re: re-export NFS mounted disk to be NFS mounted?

[Yu Chen]

Thanks Yuri,

Other than NFS, what else can I do, I heard a lot about NFS's  
performance, but I just don't know an alternative yet (AFP doesn't  
work that well on a server as what I read from mail lists, and from  
trying).

Guess I will go to local disk solution.

CY

On Mar 23, 2009, at 12:15 PM, Yuri Csapo wrote:

> AFAIK that is not possible on Linux with the stock kernel nfs  
> server. It *should*  be possible with the userspace server, but I'm  
> not sure it's supported. Maybe someone who has actually done it (as  
> opposed to playing with it) should comment.
>
> OTOH it doesn't seem like a good idea anyway... NFS's performance is  
> really bad and what you're doing will multiply bad x 2. I know  
> sometimes you need to work with what you have but if your cluster is  
> in any way I/O sensitive you should think about getting some local  
> disk space for it, at least.
>
> Yu Chen wrote:
>> Hello,
>> I have a cluster, the disk space is on a Xserver, it's NFS  
>> exported, and mounted on the cluster's head node without problem  
>> (mounted on /mnt/nfs), then I exported the "/mnt/nfs" directory,  
>> then tried to mount it on the nodes in the cluster (mount -t nfs  
>> headnode:/mnt/nfs /mnt/tmp), it gave error: mount ... failed,  
>> reason given by server: Permission denied.
>> my Xserver nfs exports entry has this: /Volumes/DataRAID  -alldirs - 
>> maproot=nobody -sec=sys -network my.headnode.network -mask  
>> 255.255.255.0
>> my headnode mounted /Volumes/DataRAID on /mnt/nfs then exported as  
>> nfs exports entry:
>> /mnt/nfs 10.0.0.0/255.255.255.0(rw,sync)
>> Anybody has any suggestions?
>> Thanks in advance.
>> CY
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux- 
>> admin" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> -- 
> Yuri Csapo
> Academic Computing & Networking
> Colorado School of Mines
> CT-256
> Phone:  (303) 273-3503
> Fax:      (303) 273-3475
> Email:   ycsapo@mines.edu
>
> Please use the following link to open a service request:
> http://helpdesk.mines.edu
> ===========================================
> With a PC, I always felt limited
> by the software available.
> On Unix, I am limited only by my knowledge.
> --Peter J. Schoenster
>

Re: re-export NFS mounted disk to be NFS mounted?

[Yuri Csapo]

Well, a lot of it depends on the hardware you have, but re-exporting is 
really not a good idea no matter which protocol you use. Unless the 
cluster is a computational cluster which only needs to read/write disks 
sporadically, I'd strongly suggest you give it its own disk space, 
directly attached to the head node (or use a SAN, but that's beside the 
scope here I think).

We have a number of clusters here and there are 2 different solutions we 
use (that I'm aware of):

- For the big supercomputer we use a SAN which all nodes can see with 
Lustre as a filesystem. You can see hardware details here:

http://geco.mines.edu/hardware.shtml

- For small clusters (typically 10-node) we use disks directly attached 
to the head and NFS-exported to the nodes. This is just one hop of NFS 
so it's not that bad; still we work around it (after a fashion) by 
having local disks on each node that can be used as a temporary, 
node-specific scratch area.

Hope this helps...

Yuri

Yu Chen wrote:
> Thanks Yuri,
> 
> Other than NFS, what else can I do, I heard a lot about NFS's 
> performance, but I just don't know an alternative yet (AFP doesn't work 
> that well on a server as what I read from mail lists, and from trying).
> 
> Guess I will go to local disk solution.
> 
> CY
> 
> On Mar 23, 2009, at 12:15 PM, Yuri Csapo wrote:
> 
>> AFAIK that is not possible on Linux with the stock kernel nfs server. 
>> It *should*  be possible with the userspace server, but I'm not sure 
>> it's supported. Maybe someone who has actually done it (as opposed to 
>> playing with it) should comment.
>>
>> OTOH it doesn't seem like a good idea anyway... NFS's performance is 
>> really bad and what you're doing will multiply bad x 2. I know 
>> sometimes you need to work with what you have but if your cluster is 
>> in any way I/O sensitive you should think about getting some local 
>> disk space for it, at least.
>>
>> Yu Chen wrote:
>>> Hello,
>>> I have a cluster, the disk space is on a Xserver, it's NFS exported, 
>>> and mounted on the cluster's head node without problem (mounted on 
>>> /mnt/nfs), then I exported the "/mnt/nfs" directory, then tried to 
>>> mount it on the nodes in the cluster (mount -t nfs headnode:/mnt/nfs 
>>> /mnt/tmp), it gave error: mount ... failed, reason given by server: 
>>> Permission denied.
>>> my Xserver nfs exports entry has this: /Volumes/DataRAID  -alldirs 
>>> -maproot=nobody -sec=sys -network my.headnode.network -mask 
>>> 255.255.255.0
>>> my headnode mounted /Volumes/DataRAID on /mnt/nfs then exported as 
>>> nfs exports entry:
>>> /mnt/nfs 10.0.0.0/255.255.255.0(rw,sync)
>>> Anybody has any suggestions?
>>> Thanks in advance.
>>> CY
>>> -- 
>>> To unsubscribe from this list: send the line "unsubscribe 
>>> linux-admin" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> -- 
>> Yuri Csapo
>> Academic Computing & Networking
>> Colorado School of Mines
>> CT-256
>> Phone:  (303) 273-3503
>> Fax:      (303) 273-3475
>> Email:   ycsapo@mines.edu
>>
>> Please use the following link to open a service request:
>> http://helpdesk.mines.edu
>> ===========================================
>> With a PC, I always felt limited
>> by the software available.
>> On Unix, I am limited only by my knowledge.
>> --Peter J. Schoenster
>>

-- 
Yuri Csapo
Academic Computing & Networking
Colorado School of Mines
CT-256
Phone:  (303) 273-3503
Fax:      (303) 273-3475
Email:   ycsapo@mines.edu

Please use the following link to open a service request:
http://helpdesk.mines.edu
===========================================
With a PC, I always felt limited
by the software available.
On Unix, I am limited only by my knowledge.
--Peter J. Schoenster

Re: re-export NFS mounted disk to be NFS mounted?

[Yu Chen]

Thank for the explaining, Yuri.

Our cluster mainly is computational, not much disk activity, so I  
guess I will do the NFS thing.

CY


On Mar 23, 2009, at 1:23 PM, Yuri Csapo wrote:

> Well, a lot of it depends on the hardware you have, but re-exporting  
> is really not a good idea no matter which protocol you use. Unless  
> the cluster is a computational cluster which only needs to read/ 
> write disks sporadically, I'd strongly suggest you give it its own  
> disk space, directly attached to the head node (or use a SAN, but  
> that's beside the scope here I think).
>
> We have a number of clusters here and there are 2 different  
> solutions we use (that I'm aware of):
>
> - For the big supercomputer we use a SAN which all nodes can see  
> with Lustre as a filesystem. You can see hardware details here:
>
> http://geco.mines.edu/hardware.shtml
>
> - For small clusters (typically 10-node) we use disks directly  
> attached to the head and NFS-exported to the nodes. This is just one  
> hop of NFS so it's not that bad; still we work around it (after a  
> fashion) by having local disks on each node that can be used as a  
> temporary, node-specific scratch area.
>
> Hope this helps...
>
> Yuri
>
> Yu Chen wrote:
>> Thanks Yuri,
>> Other than NFS, what else can I do, I heard a lot about NFS's  
>> performance, but I just don't know an alternative yet (AFP doesn't  
>> work that well on a server as what I read from mail lists, and from  
>> trying).
>> Guess I will go to local disk solution.
>> CY
>> On Mar 23, 2009, at 12:15 PM, Yuri Csapo wrote:
>>> AFAIK that is not possible on Linux with the stock kernel nfs  
>>> server. It *should*  be possible with the userspace server, but  
>>> I'm not sure it's supported. Maybe someone who has actually done  
>>> it (as opposed to playing with it) should comment.
>>>
>>> OTOH it doesn't seem like a good idea anyway... NFS's performance  
>>> is really bad and what you're doing will multiply bad x 2. I know  
>>> sometimes you need to work with what you have but if your cluster  
>>> is in any way I/O sensitive you should think about getting some  
>>> local disk space for it, at least.
>>>
>>> Yu Chen wrote:
>>>> Hello,
>>>> I have a cluster, the disk space is on a Xserver, it's NFS  
>>>> exported, and mounted on the cluster's head node without problem  
>>>> (mounted on /mnt/nfs), then I exported the "/mnt/nfs" directory,  
>>>> then tried to mount it on the nodes in the cluster (mount -t nfs  
>>>> headnode:/mnt/nfs /mnt/tmp), it gave error: mount ... failed,  
>>>> reason given by server: Permission denied.
>>>> my Xserver nfs exports entry has this: /Volumes/DataRAID  - 
>>>> alldirs -maproot=nobody -sec=sys -network my.headnode.network - 
>>>> mask 255.255.255.0
>>>> my headnode mounted /Volumes/DataRAID on /mnt/nfs then exported  
>>>> as nfs exports entry:
>>>> /mnt/nfs 10.0.0.0/255.255.255.0(rw,sync)
>>>> Anybody has any suggestions?
>>>> Thanks in advance.
>>>> CY
>>>> -- 
>>>> To unsubscribe from this list: send the line "unsubscribe linux- 
>>>> admin" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> -- 
>>> Yuri Csapo
>>> Academic Computing & Networking
>>> Colorado School of Mines
>>> CT-256
>>> Phone:  (303) 273-3503
>>> Fax:      (303) 273-3475
>>> Email:   ycsapo@mines.edu
>>>
>>> Please use the following link to open a service request:
>>> http://helpdesk.mines.edu
>>> ===========================================
>>> With a PC, I always felt limited
>>> by the software available.
>>> On Unix, I am limited only by my knowledge.
>>> --Peter J. Schoenster
>>>
>
> -- 
> Yuri Csapo
> Academic Computing & Networking
> Colorado School of Mines
> CT-256
> Phone:  (303) 273-3503
> Fax:      (303) 273-3475
> Email:   ycsapo@mines.edu
>
> Please use the following link to open a service request:
> http://helpdesk.mines.edu
> ===========================================
> With a PC, I always felt limited
> by the software available.
> On Unix, I am limited only by my knowledge.
> --Peter J. Schoenster
>