[Lustre-devel] moving /proc to $MNT/.lustre

[Lustre-devel] moving /proc to $MNT/.lustre

[Peter Bojanic]

Hi Nathan,

(Copying Lustre-devel)

On 2007-12-13, at 15:57 , Nathan Rutman wrote:

> Peter - I filed bug 14471 and Tom is running with it, but it's kind  
> of a big UI change and so I think someone In Authority should give  
> the go-ahead.  I think andreas and I are in agreement that it makes  
> sense.  The first step should be to do a nice DLD detailing  
> potential impact on all our tools, testing, debug etc.

Are you absolutely certain about this? This is going to break a ton of  
people's scripts.

This needs a few more nods from the architecture group -- I'd like to  
see at least eeb chime in.

By means of discussing it here, we're pretty much announcing the  
change to the Lustre community. But, if we proceed, you should also  
post a note to lustre-discuss for wider dissemination.

Thanks,
Bojanic

[Lustre-devel] moving /proc to $MNT/.lustre

[Nathan Rutman]

Peter Bojanic wrote:
> Hi Nathan,
>
> (Copying Lustre-devel)
>
> On 2007-12-13, at 15:57 , Nathan Rutman wrote:
>
>   
>> Peter - I filed bug 14471 and Tom is running with it, but it's kind  
>> of a big UI change and so I think someone In Authority should give  
>> the go-ahead.  I think andreas and I are in agreement that it makes  
>> sense.  The first step should be to do a nice DLD detailing  
>> potential impact on all our tools, testing, debug etc.
>>     
>
> Are you absolutely certain about this? This is going to break a ton of  
> people's scripts.
>
>   
well, that's why I asked.  As I said, Andreas and I are in agreement, 
and it certainly makes sense from a portability point of view, as well 
as consistency with future features (snapshots, audit logs, etc.), and 
the final elimination of our various /proc locking headaches.  But yes, 
it would break user's scripts  - that's a 1-time thing, and I think not 
too terrible.

> This needs a few more nods from the architecture group -- I'd like to  
> see at least eeb chime in.
>
> By means of discussing it here, we're pretty much announcing the  
> change to the Lustre community. But, if we proceed, you should also  
> post a note to lustre-discuss for wider dissemination.
>
> Thanks,
> Bojanic
>
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-devel
>   

[Lustre-devel] moving /proc to $MNT/.lustre

[Alex Zhuravlev]

Nathan Rutman wrote:
> well, that's why I asked.  As I said, Andreas and I are in agreement, 
> and it certainly makes sense from a portability point of view, as well 
> as consistency with future features (snapshots, audit logs, etc.), and 
> the final elimination of our various /proc locking headaches.  But yes, 
> it would break user's scripts  - that's a 1-time thing, and I think not 
> too terrible.

it's not quite clear how do we do with userspace servers. there was an idea
to use named pipes, but I'm not sure it's very portable.

thanks, Alex

[Lustre-devel] moving /proc to $MNT/.lustre

[Brian J. Murrell]

On Mon, 2008-01-07 at 09:50 -0800, Nathan Rutman wrote:
> well, that's why I asked.  As I said, Andreas and I are in agreement, 
> and it certainly makes sense from a portability point of view, as well 
> as consistency with future features (snapshots, audit logs, etc.), and 
> the final elimination of our various /proc locking headaches.  But yes, 
> it would break user's scripts  - that's a 1-time thing, and I think not 
> too terrible.

Is it possible to support both for a release or two to give people time
to migrate and have an actual implementation to test against as they
work to port their scripts?  The alternative is that given that we don't
provide public beta binaries or nightly snapshot binaries, we'd be
requiring people who want to port, test and release their ports on "flag
day" to build from CVS to test.

b.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20080107/c1d2062b/attachment.pgp>

[Lustre-devel] moving /proc to $MNT/.lustre

[Andreas Dilger]

On Jan 07, 2008  13:10 -0500, Brian J. Murrell wrote:
> On Mon, 2008-01-07 at 09:50 -0800, Nathan Rutman wrote:
> > well, that's why I asked.  As I said, Andreas and I are in agreement, 
> > and it certainly makes sense from a portability point of view, as well 
> > as consistency with future features (snapshots, audit logs, etc.), and 
> > the final elimination of our various /proc locking headaches.  But yes, 
> > it would break user's scripts  - that's a 1-time thing, and I think not 
> > too terrible.
> 
> Is it possible to support both for a release or two to give people time
> to migrate and have an actual implementation to test against as they
> work to port their scripts?  The alternative is that given that we don't
> provide public beta binaries or nightly snapshot binaries, we'd be
> requiring people who want to port, test and release their ports on "flag
> day" to build from CVS to test.

It wasn't mentioned here, but this is already planned.  There will be
new commands "lctl get_param" and "lctl set_param" (or similar) that
will be usable by scripts to get/set Lustre tunables.  This will work
with both /proc and .../.lustre files so will allow scripts to move
over to the new mechanism.

For user-space servers there will be no alternative but to use the lctl
mechanism since /proc entries will not exist at all.  Then again, there
will not be any existing systems using the old mechanism since uOSS will
only work with ZFS.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

[Lustre-devel] moving /proc to $MNT/.lustre

[Oleg Drokin]

Hello!

On Jan 7, 2008, at 3:07 PM, Andreas Dilger wrote:
>> Is it possible to support both for a release or two to give people  
>> time
>> to migrate and have an actual implementation to test against as they
>> work to port their scripts?  The alternative is that given that we  
>> don't
>> provide public beta binaries or nightly snapshot binaries, we'd be
>> requiring people who want to port, test and release their ports on  
>> "flag
>> day" to build from CVS to test.
> It wasn't mentioned here, but this is already planned.  There will be
> new commands "lctl get_param" and "lctl set_param" (or similar) that
> will be usable by scripts to get/set Lustre tunables.  This will work
> with both /proc and .../.lustre files so will allow scripts to move
> over to the new mechanism.

What's the plan to prevent various backups software (and also tar &  
friends)
from backing up and restoring (esp. restoring of course) values in these
files?

Bye,
     Oleg

[Lustre-devel] moving /proc to $MNT/.lustre

[Andreas Dilger]

On Jan 07, 2008  22:39 -0500, Oleg Drokin wrote:
> On Jan 7, 2008, at 3:07 PM, Andreas Dilger wrote:
>>> Is it possible to support both for a release or two to give people time
>>> to migrate and have an actual implementation to test against as they
>>> work to port their scripts?  The alternative is that given that we don't
>>> provide public beta binaries or nightly snapshot binaries, we'd be
>>> requiring people who want to port, test and release their ports on "flag
>>> day" to build from CVS to test.
>> It wasn't mentioned here, but this is already planned.  There will be
>> new commands "lctl get_param" and "lctl set_param" (or similar) that
>> will be usable by scripts to get/set Lustre tunables.  This will work
>> with both /proc and .../.lustre files so will allow scripts to move
>> over to the new mechanism.
>
> What's the plan to prevent various backups software (and also tar & 
> friends)
> from backing up and restoring (esp. restoring of course) values in these
> files?

The typical way is that .lustre would not appear in readdir/getdirents
but could be accessed if explicitly named.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

[Lustre-devel] moving /proc to $MNT/.lustre

[Alex Zhuravlev]

Andreas Dilger wrote:
>> What's the plan to prevent various backups software (and also tar & 
>> friends)
>> from backing up and restoring (esp. restoring of course) values in these
>> files?
> 
> The typical way is that .lustre would not appear in readdir/getdirents
> but could be accessed if explicitly named.

wouldn't it make sense to get rid of .lustre and use lctl then?
I guess we'll have to do this anyway if we want to use all our
testing infrastructure with userspace servers.

thanks, Alex

[Lustre-devel] moving /proc to $MNT/.lustre

[Nathan Rutman]

Alex Zhuravlev wrote:
> Andreas Dilger wrote:
>>> What's the plan to prevent various backups software (and also tar & 
>>> friends)
>>> from backing up and restoring (esp. restoring of course) values in 
>>> these
>>> files?
>>
>> The typical way is that .lustre would not appear in readdir/getdirents
>> but could be accessed if explicitly named.
>
> wouldn't it make sense to get rid of .lustre and use lctl then?
> I guess we'll have to do this anyway if we want to use all our
> testing infrastructure with userspace servers.
No - there will be plenty of other things in .lustre as well:
snapshots
log files
status indications

[Lustre-devel] moving /proc to $MNT/.lustre

[Alex Zhuravlev]

Nathan Rutman wrote:
> No - there will be plenty of other things in .lustre as well:
> snapshots
> log files
> status indications

hmm. and how are we going to access that in case of userspace server?
via client?

thanks, Alex

[Lustre-devel] moving /proc to $MNT/.lustre

[Nicholas Henke]

Andreas Dilger wrote:
> 
> It wasn't mentioned here, but this is already planned.  There will be
> new commands "lctl get_param" and "lctl set_param" (or similar) that
> will be usable by scripts to get/set Lustre tunables.  This will work
> with both /proc and .../.lustre files so will allow scripts to move
> over to the new mechanism.
> 
> For user-space servers there will be no alternative but to use the lctl
> mechanism since /proc entries will not exist at all.  Then again, there
> will not be any existing systems using the old mechanism since uOSS will
> only work with ZFS.
> 

How is get_param/set_param going to work for the cases like clearing the LRU - where one might not 
know the names of all of the parameters:

for LRU in /proc/fs/lustre/ldlm/namespaces/*/lru_size; do echo clear > $LRU; done

Nic

[Lustre-devel] moving /proc to $MNT/.lustre

[Nathan Rutman]

Nicholas Henke wrote:
> Andreas Dilger wrote:
>>
>> It wasn't mentioned here, but this is already planned.  There will be
>> new commands "lctl get_param" and "lctl set_param" (or similar) that
>> will be usable by scripts to get/set Lustre tunables.  This will work
>> with both /proc and .../.lustre files so will allow scripts to move
>> over to the new mechanism.
>>
>> For user-space servers there will be no alternative but to use the lctl
>> mechanism since /proc entries will not exist at all.  Then again, there
>> will not be any existing systems using the old mechanism since uOSS will
>> only work with ZFS.
>>
>
> How is get_param/set_param going to work for the cases like clearing 
> the LRU - where one might not know the names of all of the parameters:
>
> for LRU in /proc/fs/lustre/ldlm/namespaces/*/lru_size; do echo clear > 
> $LRU; done
>
> Nic
In this particular case, there will not be a "namespaces" subdirectory 
-- there will be different files in the .lustre directories of different 
mount points:
/mnt/test/.lustre/ldlm/lru_size
/mnt/fast2/.lustre/ldlm/lru_size
etc

But in general, if there are any non-predefined path names for userspace 
server parameters, we'll have to add a lctl method of listing them.