Re: Elastic Quota File System (EQFS)

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Hi all,

>> > On one school server, theres 10MB quota. (Okay, its admins are
>> > BOFHs^H^H^H^H^HSISAL). Everyone tries to run mozilla there (because
>> > its installed as default!), and immediately fills his/her quota with
>> > cache files, leading to failed login next time (gnome just will not
>> > start if it can't write to ~).
>> >
>> > Imagine mozilla automatically marking cache files "elastic".
>> >
>> > That would solve the problem -- mozilla caches would go away when disk
>> > space was demanded, still mozilla's on-disk caching would be effective
>> > when there is enough disk space.

Also this is applicable to mailboxes - automize the marking of old mails of
the mailing list as elastic, whenever the threshold is reached, you might
either want to put those mails as compressed archive or simply delete it.
This has two advantages:
 - No email bounces for the reason of 'mailbox full' and
 - Optimized utlization of the mailbox

Yes, this can be done using scripts too, but what if you are to use other
users' space that they are not using? Of course script cannot do that! You
need to have some FS support or a libquota hack.


>>
>> How does Mozilla (or any process) react when its files are deleted from
>> under it?  Would the file remain until all the open processes close the
>> file or would it just "disappear"?
>
>Of course, if mozilla marked them "elastic" it should better be
>prepared for they disappearance. I'd disappear them with simple
>unlink(), so they'd physically survive as long as someone held them
>open.
>
>>  Would it delete entire directories or
>> just some of the files?  How does it choose?  (First up against the
delete
>> when the drive space fills...)
>
>Probably just some of the files... Or you could delete directory, too,
>if it was marked "elastic". What to delete first... probably file with
>oldest access time? Or random file, with chance of being selected
>proportional to file size?

One can either even mark the whole directory as elastic. When it comes to
taking action, I think I'II opt for random deletion of the files for not
being unfair to any particular user.

AG

Re: Elastic Quota File System (EQFS)

[Fao, Sean]

Amit Gud wrote:

>Hi all,
>
>  
>
>>>>On one school server, theres 10MB quota. (Okay, its admins are
>>>>BOFHs^H^H^H^H^HSISAL). Everyone tries to run mozilla there (because
>>>>its installed as default!), and immediately fills his/her quota with
>>>>cache files, leading to failed login next time (gnome just will not
>>>>start if it can't write to ~).
>>>>
>>>>Imagine mozilla automatically marking cache files "elastic".
>>>>
>>>>That would solve the problem -- mozilla caches would go away when disk
>>>>space was demanded, still mozilla's on-disk caching would be effective
>>>>when there is enough disk space.
>>>>        
>>>>
>
>Also this is applicable to mailboxes - automize the marking of old mails of
>the mailing list as elastic, whenever the threshold is reached, you might
>either want to put those mails as compressed archive or simply delete it.
>This has two advantages:
> - No email bounces for the reason of 'mailbox full' and
> - Optimized utlization of the mailbox
>
>Yes, this can be done using scripts too, but what if you are to use other
>users' space that they are not using? Of course script cannot do that! You
>need to have some FS support or a libquota hack.
>  
>
What you're suggesting is not something that could be controlled by the 
file system because there are far too many methods for storing email.  
Consider, for instance, if all email messages are stored in a single 
database file.  Unless you include parsing code for all email storage 
methods, there's no possible way for your design to work.  As you should 
be able to see, this "feature" should be implemented in the email server 
code; *not* the file system.

Personally, I would rather not use this feature, at all; but, I'm also 
entitled to leaving the elastic bit turned off on all my messages.

Sean

Re: Elastic Quota File System (EQFS)

[Horst von Brand]

"Amit Gud" <gud@eth.net> said:

[...]

> Also this is applicable to mailboxes - automize the marking of old mails of
> the mailing list as elastic, whenever the threshold is reached, you might
> either want to put those mails as compressed archive or simply delete it.

Right. And which to do is a policy decision, left to the individual owner
of this particular file or collection of files. Democracy at work ;-)

> This has two advantages:
>  - No email bounces for the reason of 'mailbox full' and

It will fill up eventually...

>  - Optimized utlization of the mailbox

> Yes, this can be done using scripts too,

Having the system fool around with my mail because you get too much junk
and can't be bothered to delete it is what I'd consider an hostile act.

Case closed, anyway. It belongs in the kernel only if there is no
reasonable way to do it in userspace.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> Case closed, anyway. It belongs in the kernel only if there is no
> reasonable way to do it in userspace.

But... there's no reasonable way to do this in userspace.

Two pieces of kernel support are needed:

1) some way to indicate "this file is elastic" (okay perhaps xattrs
can do this already)

and either

2a) file selection/deletion in kernel

or

2b) assume that disk does not fill up faster than 1GB/sec, allways
keep 1GB free, make "deleting" daemon poll each second [ugly,
unreliable]

or

2c) just before returning -ENOSPC, synchronously tell userspace to
free space, and retry the operation.

BTW 2c) would be also usefull for undelete. Unfortunately 2c looks
very complex, too; it might be easier to do 2a than 2c.
								Pavel

-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: Elastic Quota File System (EQFS)

[Alan]

On Fri, 2004-06-25 at 09:25, Pavel Machek wrote:
> Hi!
> 
> > Case closed, anyway. It belongs in the kernel only if there is no
> > reasonable way to do it in userspace.
> 
> But... there's no reasonable way to do this in userspace.
> 
> Two pieces of kernel support are needed:
> 
> 1) some way to indicate "this file is elastic" (okay perhaps xattrs
> can do this already)
> 
> and either
> 
> 2a) file selection/deletion in kernel
> 
> or
> 
> 2b) assume that disk does not fill up faster than 1GB/sec, allways
> keep 1GB free, make "deleting" daemon poll each second [ugly,
> unreliable]
> 
> or
> 
> 2c) just before returning -ENOSPC, synchronously tell userspace to
> free space, and retry the operation.
> 
> BTW 2c) would be also usefull for undelete. Unfortunately 2c looks
> very complex, too; it might be easier to do 2a than 2c.

Why does the kernel have to get involved with file deletion?

All it needs is to run at sufficient privs.

If you are overflowing drives that easily, it is time to buy a bigger
drive.  It is not the time to start deleting stuff at random.  Data is
usually put on a drive for a reason.  Having a human decide what to
delete is *much* better than letting some automated process do it in
background.

This sounds like a hack to get around a badly designed system with too
few resources.

Windows has an option to delete files "that are not needed".  It tends
to delete things that you wanted, but had not thought about in a while.

This really strikes me as a bad idea.  It has lots of "special" things
that programs will have to deal with for this particular case. 
This makes things much more complex in userspace for a problem that
needs to be dealt with in meatspace.

-- 
"Ye have locked yerselves up in cages of fear--and, behold, do ye now
complain that ye lack FREEDOM!"
  - Lord Omar in THE EPISTLE TO THE PARANOIDS Chaper 1 Verse 1

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Alan wrote:

>On Fri, 2004-06-25 at 09:25, Pavel Machek wrote:
>  
>
>>Hi!
>>
>>    
>>
>>>Case closed, anyway. It belongs in the kernel only if there is no
>>>reasonable way to do it in userspace.
>>>      
>>>
>>But... there's no reasonable way to do this in userspace.
>>
>>Two pieces of kernel support are needed:
>>
>>1) some way to indicate "this file is elastic" (okay perhaps xattrs
>>can do this already)
>>
>>and either
>>
>>2a) file selection/deletion in kernel
>>
>>or
>>
>>2b) assume that disk does not fill up faster than 1GB/sec, allways
>>keep 1GB free, make "deleting" daemon poll each second [ugly,
>>unreliable]
>>
>>or
>>
>>2c) just before returning -ENOSPC, synchronously tell userspace to
>>free space, and retry the operation.
>>
>>BTW 2c) would be also usefull for undelete. Unfortunately 2c looks
>>very complex, too; it might be easier to do 2a than 2c.
>>    
>>
>
>Why does the kernel have to get involved with file deletion?
>
>  
>
Involvement of the kernel depends on  how we are treating the elastic 
files:

    -   when files are made elastic, are we increasing the user's quota? 
if so kernel becomes neccessary for if
       -   elastic file is deleted: decrement the user's quota
       -   elastic file is chowned: do the needfull (decrement previous 
owners and increment new owners quota)
   
    -   when files are made elastic, we are just not accounting the file 
size for the user's quota. In this case I dont see any involvement of 
the kernel,  but will need some other hacks.


>All it needs is to run at sufficient privs.
>
>If you are overflowing drives that easily, it is time to buy a bigger
>drive.  It is not the time to start deleting stuff at random.  Data is
>usually put on a drive for a reason.  Having a human decide what to
>delete is *much* better than letting some automated process do it in
>background.
>
>  
>
Admin assigns quotas to the user with using some case studies, like he 
won't be allocating 500 Mb for a C language course students. We are 
trying to harness some user behavior here. I wish I had some Gartner or 
Forrestor figures to show you that how often _all_ the users use up 
their 100% quota at any given point of time. I'm assuming that this 
*does* happen very rarely and I also assume that my assumption is very 
close to correct. The point is their is some 'free' space available for 
the users who use up all their quotas. Of course this free space is 
likely to be someday fall short for the over-spilling users, but then 
its time to increase quotas of all the users, and its then admin's duty 
to look at this. Perhaps this could become a good indication for the 
admin for increasing the quota of all the users.

>This sounds like a hack to get around a badly designed system with too
>few resources.
>
>Windows has an option to delete files "that are not needed".  It tends
>to delete things that you wanted, but had not thought about in a while.
>
>This really strikes me as a bad idea.  It has lots of "special" things
>that programs will have to deal with for this particular case. 
>This makes things much more complex in userspace for a problem that
>needs to be dealt with in meatspace.
>
>  
>
I do believe that this system can be done in userspace, but it has its 
own flaws then. Suppose theres a daemon, call it eqfsd. It forks parent 
listens a char device, child watches the disk space usage. A kernel 
module reports the file deletes, chowns to the char device, parent does 
the needfull. Child 'periodically' checks that the threshold is not 
reached. Here what can be done is suppose a user can transfer data with  
say 5 Mbps speed to his account....then we can easily get the minimum 
time required to fill up the remaining free space. Child sleeps for this 
much time minus some value for safety.  i.e. child sleeps for: ( (D - 
Ui) / 5 ) - safety_value seconds.

I'm sure this gotta be slow and *will* slow down the system, so I still 
insist that this should be in kspace.

It cannot be denied that there _are_ applications for such a system that 
we already discussed and theres a class of users who will find the 
system useful.


AG

Re: Elastic Quota File System (EQFS)

[Fao, Sean]

Amit Gud wrote:

> It cannot be denied that there _are_ applications for such a system 
> that we already discussed and theres a class of users who will find 
> the system useful.


I personally see no use whatsoever.  Why not just allocate 100% of the 
file system to everybody and ignore quota's, entirely?  Each user will 
use whatever he/she requires and when space starts to run out, users 
will manually clean up what they don't need.

I am totally against the automatic deletion of files and believe that 
all users will _eventually_ walk in on a Monday morning to find out that 
the OS took it upon itself to delete a file that was flagged as elastic, 
that shouldn't have been.  I also tend to believe that the exact 
time/date that the file was removed could conceivably occur six months 
prior to that Monday morning, without the users knowledge.  Now the 
burden will again be placed on to system administrators.  This time, to 
locate and recover the lost file(s) by sorting through months of 
archives.  Personally, I prefer setting quota's on an individual bases, 
to finding a needle in a haystack

In my mind, you either have a quota or you don't; there's no in between.

Sean

Re: Elastic Quota File System (EQFS)

[Kevin Fox]

Not to argue whether its a good idea but, one use for this would be
undeleting. rm becomes a set EQFS bit. You keep all your old files that
way until space is really needed and then they get reclaimed. You might
not want that to count against your quota.

On Fri, 2004-06-25 at 13:22, Fao, Sean wrote:
> Amit Gud wrote:
> 
> > It cannot be denied that there _are_ applications for such a system 
> > that we already discussed and theres a class of users who will find 
> > the system useful.
> 
> 
> I personally see no use whatsoever.  Why not just allocate 100% of the 
> file system to everybody and ignore quota's, entirely?  Each user will 
> use whatever he/she requires and when space starts to run out, users 
> will manually clean up what they don't need.
> 
> I am totally against the automatic deletion of files and believe that 
> all users will _eventually_ walk in on a Monday morning to find out that 
> the OS took it upon itself to delete a file that was flagged as elastic, 
> that shouldn't have been.  I also tend to believe that the exact 
> time/date that the file was removed could conceivably occur six months 
> prior to that Monday morning, without the users knowledge.  Now the 
> burden will again be placed on to system administrators.  This time, to 
> locate and recover the lost file(s) by sorting through months of 
> archives.  Personally, I prefer setting quota's on an individual bases, 
> to finding a needle in a haystack
> 
> In my mind, you either have a quota or you don't; there's no in between.
> 
> Sean
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Fao, Sean wrote:

> Amit Gud wrote:
>
>> It cannot be denied that there _are_ applications for such a system 
>> that we already discussed and theres a class of users who will find 
>> the system useful.
>
>
>
> I personally see no use whatsoever. Why not just allocate 100% of the 
> file system to everybody and ignore quota's, entirely?  Each user will 
> use whatever he/she requires and when space starts to run out, users 
> will manually clean up what they don't need.
>
We should get our basics right first. We _do_ need quotas!! Without any 
quota system how are we going to avoid a malicious user  from taking 
away all the space to keep other people starving? In EQFS also this can 
happen, but we are giving *controlled flexibility* to the user. He is 
having some stretching power but not beyond a certain limit. And do you 
think users are sincere enough to clean up there files when they are done?

> I am totally against the automatic deletion of files and believe that 
> all users will _eventually_ walk in on a Monday morning to find out 
> that the OS took it upon itself to delete a file that was flagged as 
> elastic, that shouldn't have been.  

User is the king, he decides what files should be elastic and what not. 
This can always be controlled.


AG

Re: Elastic Quota File System (EQFS)

[Fao, Sean]

Amit Gud wrote:

> Fao, Sean wrote:
>
>> Amit Gud wrote:
>>
>>> It cannot be denied that there _are_ applications for such a system 
>>> that we already discussed and theres a class of users who will find 
>>> the system useful.
>>
>>
>>
>>
>> I personally see no use whatsoever. Why not just allocate 100% of the 
>> file system to everybody and ignore quota's, entirely?  Each user 
>> will use whatever he/she requires and when space starts to run out, 
>> users will manually clean up what they don't need.
>>
> We should get our basics right first. We _do_ need quotas!! Without 
> any quota system how are we going to avoid a malicious user  from 
> taking away all the space to keep other people starving? In EQFS also 
> this can happen, but we are giving *controlled flexibility* to the 
> user. He is having some stretching power but not beyond a certain 
> limit. And do you think users are sincere enough to clean up there 
> files when they are done?


And I suppose you think that users will be sincere enough to mark files 
as elastic?  I, for one, already said that I absolutely would *not* mark 
a single file as elastic.  If I'm using 110 MB and you need an 
additional 10 MB for storage, you won't be getting it from me because I 
don't want to come in some morning to find that a file has disappeared.

The system that you're asking for is a system without quotas.  Think 
about what you're saying.

>
>> I am totally against the automatic deletion of files and believe that 
>> all users will _eventually_ walk in on a Monday morning to find out 
>> that the OS took it upon itself to delete a file that was flagged as 
>> elastic, that shouldn't have been.  
>
>
> User is the king, he decides what files should be elastic and what 
> not. This can always be controlled.
>
Controlled how?  Who is anybody to inform me of what files I need/don't 
need?

Sean

Re: Elastic Quota File System (EQFS)

[Stephen Wille Padnos]

Fao, Sean wrote:

> Amit Gud wrote:
>
>> Fao, Sean wrote:
>>
>>> Amit Gud wrote:
>>>
>>>> It cannot be denied that there _are_ applications for such a system 
>>>> that we already discussed and theres a class of users who will find 
>>>> the system useful.
>>>
>>> I personally see no use whatsoever. Why not just allocate 100% of 
>>> the file system to everybody and ignore quota's, entirely?  Each 
>>> user will use whatever he/she requires and when space starts to run 
>>> out, users will manually clean up what they don't need.
>>
>> We should get our basics right first. We _do_ need quotas!! Without 
>> any quota system how are we going to avoid a malicious user  from 
>> taking away all the space to keep other people starving? In EQFS also 
>> this can happen, but we are giving *controlled flexibility* to the 
>> user. He is having some stretching power but not beyond a certain 
>> limit. And do you think users are sincere enough to clean up there 
>> files when they are done?
>
> And I suppose you think that users will be sincere enough to mark 
> files as elastic?  I, for one, already said that I absolutely would 
> *not* mark a single file as elastic.  If I'm using 110 MB and you need 
> an additional 10 MB for storage, you won't be getting it from me 
> because I don't want to come in some morning to find that a file has 
> disappeared.
>
> The system that you're asking for is a system without quotas.  Think 
> about what you're saying.

I think you missed one of the main points - you don't get any extra 
space until you mark some of your files as elastic.
You're right - under this system, nobody would get any space from 
deletion of your files because you would use the system as a normal hard 
quota system - you would mark no files as elastic, and would therefore 
be limited to your quota (in the example you gave, you would not be 
using 110M, because your quota would have limited you to 100M).  If you 
were so kind as to mark something as elastic (say, that recently 
doneloaded install tarball of the Gimp), then you would remove the 
storage taken by those files from your quota usage and would have more 
space available, with the risk that the elastic files might not stick 
around.

Under no circumstance would you lose any file that fits under your quota.

>>> I am totally against the automatic deletion of files and believe 
>>> that all users will _eventually_ walk in on a Monday morning to find 
>>> out that the OS took it upon itself to delete a file that was 
>>> flagged as elastic, that shouldn't have been.  
>>
>> User is the king, he decides what files should be elastic and what 
>> not. This can always be controlled.
>
> Controlled how?  Who is anybody to inform me of what files I 
> need/don't need?

Controlled by you using one of the methods that have been suggested:
a .elastic file/directory structure
/scratch/ space usage
a filesystem that can keep track of these things, and a program like chmod
xattrs and other userspace tools

etc.

- Steve

Re: Elastic Quota File System (EQFS)

[Fao, Sean]

Stephen Wille Padnos wrote:

> Fao, Sean wrote:
>
>> Amit Gud wrote:
>>
>>> Fao, Sean wrote:
>>>
>>>> Amit Gud wrote:
>>>>
>>>>> It cannot be denied that there _are_ applications for such a 
>>>>> system that we already discussed and theres a class of users who 
>>>>> will find the system useful.
>>>>
>>>>
>>>> I personally see no use whatsoever. Why not just allocate 100% of 
>>>> the file system to everybody and ignore quota's, entirely?  Each 
>>>> user will use whatever he/she requires and when space starts to run 
>>>> out, users will manually clean up what they don't need.
>>>
>>>
>>> We should get our basics right first. We _do_ need quotas!! Without 
>>> any quota system how are we going to avoid a malicious user  from 
>>> taking away all the space to keep other people starving? In EQFS 
>>> also this can happen, but we are giving *controlled flexibility* to 
>>> the user. He is having some stretching power but not beyond a 
>>> certain limit. And do you think users are sincere enough to clean up 
>>> there files when they are done?
>>
>>
>> And I suppose you think that users will be sincere enough to mark 
>> files as elastic?  I, for one, already said that I absolutely would 
>> *not* mark a single file as elastic.  If I'm using 110 MB and you 
>> need an additional 10 MB for storage, you won't be getting it from me 
>> because I don't want to come in some morning to find that a file has 
>> disappeared.
>>
>> The system that you're asking for is a system without quotas.  Think 
>> about what you're saying.
>
>
> I think you missed one of the main points - you don't get any extra 
> space until you mark some of your files as elastic.
> You're right - under this system, nobody would get any space from 
> deletion of your files because you would use the system as a normal 
> hard quota system - you would mark no files as elastic, and would 
> therefore be limited to your quota (in the example you gave, you would 
> not be using 110M, because your quota would have limited you to 
> 100M).  If you were so kind as to mark something as elastic (say, that 
> recently doneloaded install tarball of the Gimp), then you would 
> remove the storage taken by those files from your quota usage and 
> would have more space available, with the risk that the elastic files 
> might not stick around.


You're right; I did missunderstand one of the main points.  I was under 
the impression that I could take whatever I needed, as long as it were 
available.  This does change my view on the subject, somewhat.

Thanks for clearing that up,

Sean

Re: Elastic Quota File System (EQFS)

[Rob Couto]

On Saturday 26 June 2004 07:16 pm, Stephen Wille Padnos wrote:

> I think you missed one of the main points - you don't get any extra
> space until you mark some of your files as elastic.
> You're right - under this system, nobody would get any space from
> deletion of your files because you would use the system as a normal hard
> quota system - you would mark no files as elastic, and would therefore
> be limited to your quota (in the example you gave, you would not be
> using 110M, because your quota would have limited you to 100M).  If you
> were so kind as to mark something as elastic (say, that recently
> doneloaded install tarball of the Gimp), then you would remove the
> storage taken by those files from your quota usage and would have more
> space available, with the risk that the elastic files might not stick
> around.
>
> Under no circumstance would you lose any file that fits under your quota.

-snip-

> Controlled by you using one of the methods that have been suggested:
> a .elastic file/directory structure
> /scratch/ space usage
> a filesystem that can keep track of these things, and a program like chmod
> xattrs and other userspace tools
>
> etc.
>
> - Steve

It looks (to my untrained eyes) like a user-driven caching "algorithm", where 
I can keep these KDE tarballs around next to the kernel sources, and a few 
shiny new slackware ISOs, and all are of course  replaceable, but I mark them 
elastic or put them in /scratch/... to recover my space at the cost of an 
increased probability that I'll have to download some of them again. I like 
it.

-- 
Rob Couto [rpc@cafe4111.org]
computer safety tip: use only a non-conducting, static-free hammer.
--

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> >This sounds like a hack to get around a badly designed system with too
> I do believe that this system can be done in userspace, but it has its 
> own flaws then. Suppose theres a daemon, call it eqfsd. It forks parent 
> listens a char device, child watches the disk space usage. A kernel 
> module reports the file deletes, chowns to the char device, parent does 
> the needfull. Child 'periodically' checks that the threshold is not 
> reached. Here what can be done is suppose a user can transfer data with  
> say 5 Mbps speed to his account....then we can easily get the
> minimum 

I believe I can write data at 1GB/sec at common hardware (~RAM
speed). You'll either need a *lot* of free space, or very fast
polling, and it will still break when the daemon is swapped out, when
scheduler decides writer is more important etc.
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: Elastic Quota File System (EQFS)

[Horst von Brand]

Pavel Machek <pavel@ucw.cz> said:
> Hi!
> 
> > Case closed, anyway. It belongs in the kernel only if there is no
> > reasonable way to do it in userspace.
> 
> But... there's no reasonable way to do this in userspace.

Let's see...

> Two pieces of kernel support are needed:
> 
> 1) some way to indicate "this file is elastic" (okay perhaps xattrs
> can do this already)

Or just a list of elastic files in ~/.elastic. Even better, could mark them
as "Just delete", "compress"; could go as far as giving (fallback?) globs
to select files for each treatment ("If space gets tight, delete *~ files,
and compress *.tex that haven't been read in a week"). Sounds like a fun
Perl project...

> and either
> 
> 2a) file selection/deletion in kernel

A daemon or cron job running as root can do that just fine. Or you can set
it up for your own files.

> or

> 2b) assume that disk does not fill up faster than 1GB/sec, allways
> keep 1GB free, make "deleting" daemon poll each second [ugly,
> unreliable]

Buy a larger disk. Make sure sum of all hard quotas is less than filesystem
size. Need that anyway; so it reduces to a one-user problem with per-user
solutions. 

> or

> 2c) just before returning -ENOSPC, synchronously tell userspace to
> free space, and retry the operation.

Ugly.

> BTW 2c) would be also usefull for undelete. Unfortunately 2c looks
> very complex, too; it might be easier to do 2a than 2c.

As said, all this buys very little for a lot of hairy code in the kernel,
which will be rarely used (and whose bugs will show up when it is badly
needed to work right). Besides, I strongly oppose automatic file
disposal. I love the Unix way exactly because I decide what to do (no "I
know better than you what to do", "you dumb user aren't supposed to do
that, so you have no right to know", no "undelete deleted files, but only
sometimes; it just might still be around, but if space got tight it isn't"
(this is even worse...))
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Horst von Brand wrote:

>Pavel Machek <pavel@ucw.cz> said:
>  
>
>>Hi!
>>
>>    
>>
>>>Case closed, anyway. It belongs in the kernel only if there is no
>>>reasonable way to do it in userspace.
>>>      
>>>
>>But... there's no reasonable way to do this in userspace.
>>    
>>
>
>Let's see...
>
>  
>
>>Two pieces of kernel support are needed:
>>
>>1) some way to indicate "this file is elastic" (okay perhaps xattrs
>>can do this already)
>>    
>>
>
>Or just a list of elastic files in ~/.elastic. Even better, could mark them
>as "Just delete", "compress"; could go as far as giving (fallback?) globs
>to select files for each treatment ("If space gets tight, delete *~ files,
>and compress *.tex that haven't been read in a week"). Sounds like a fun
>Perl project...
>
>  
>
.elastic is a file or directory? If its file, daemon has to do the ugly 
insertion deletion of the file enteries. If its a directory, daemon has 
to do the updating of the files in case of mv, plus follow the links for 
unlink, chown if we are storing the files as symlinks and we will be 
wasting the inodes.

Just think of the load on the system if we run a daemon, which sleeps 
for a time depending on the data transfer rate and the amount of free 
space left, and if the free space left is very close to the margin... 
the system might even freeze. The primary intention of taking 
filesystems into kernel is the speed and the security that we get, 
otherwise we do have userspace filesystems!

What we need is an application-transparent system which of course should 
be plugable within a system and which also shouldn't hamper the system 
throughput in a major way.

I really don't see user space implementation in the picture.

AG

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> > > Case closed, anyway. It belongs in the kernel only if there is no
> > > reasonable way to do it in userspace.
> > 
> > But... there's no reasonable way to do this in userspace.
> 
> Let's see...
> 
> > Two pieces of kernel support are needed:
> > 
> > 1) some way to indicate "this file is elastic" (okay perhaps xattrs
> > can do this already)
> 
> Or just a list of elastic files in ~/.elastic. Even better, could mark them
> as "Just delete", "compress"; could go as far as giving (fallback?) globs
> to select files for each treatment ("If space gets tight, delete *~ files,
> and compress *.tex that haven't been read in a week"). Sounds like a fun
> Perl project...

.elastic is ugly but okay.

> > and either
> > 
> > 2a) file selection/deletion in kernel
> 
> A daemon or cron job running as root can do that just fine. Or you can set
> it up for your own files.

If I make it cron job once an hour, users will still get -ENOSPC for
up-to hour. You can make it down to second, but you'll still get
-ENOSPC from time to time. If you want to eliminate -ENOSPC
altogether, you'll delete file from kernel just before returning
-ENOSPC and retry operation. 

> > 2b) assume that disk does not fill up faster than 1GB/sec, allways
> > keep 1GB free, make "deleting" daemon poll each second [ugly,
> > unreliable]
> 
> Buy a larger disk. Make sure sum of all hard quotas is less than filesystem
> size. Need that anyway; so it reduces to a one-user problem with per-user
> solutions. 

Well, having sum of all hard quotas > filesystem size was point of
this excersize...

> > BTW 2c) would be also usefull for undelete. Unfortunately 2c looks
> > very complex, too; it might be easier to do 2a than 2c.
> 
> As said, all this buys very little for a lot of hairy code in the kernel,
> which will be rarely used (and whose bugs will show up when it is badly
> needed to work right). Besides, I strongly oppose automatic file

It may be little gain for lots of effort. I'm just trying to say that
it is not complete nonsense.

I guess we are basically agreeing with each other... It may make nice
student project, and if patch is very non-intrusive, I guess its
okay. If it turns to be hairy, its not worth bothering.

> that, so you have no right to know", no "undelete deleted files, but only
> sometimes; it just might still be around, but if space got tight it isn't"
> (this is even worse...))

Well, few times I wished unix had undelete... It actually *has* one,
and its called power button if you are realize your mistake within 5
seconds.
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Elastic Quota File System (EQFS)

[Amit Gud]

Greetings,

	I think I should discuss this in the list...

	Recently I'm into developing an Elastic Quota File System (EQFS). This
file system works on a simple concept ... give it to others if you're not
using it, let others use it, but on the guarantee that you get it back when
you need it!!

	Here I'm talking about disk quotas. In any typical network, e.g.
sourceforge, each user is given a fixed amount of quota. 100 Mb in case of
sourceforge. 100 Mb is way over some project requirements and too small for
some projects. EQFS tries to solve this problem by exploiting the users'
usage behavior at runtime. That is the user's quota which he doesn't need
is given to the users who need it, but on 100% assurance that the originl
user can any time reclaim his/her quota.

	Before getting into implementation details I want to have public opinion
about this system. All EQFS tries to do is it maximizes the disk space
usage, which otherwise is wasted if the user doesn't really need the
allocated user..on the other hand it helps avoid the starvation of the user
who needs more space. It also helps administrator to get away with the
problem of variable quota needs..as EQFS itself adjusts according to the
user needs.


regs,
AG


	
http://www.ddsl.net

Re: Elastic Quota File System (EQFS)

[Mark Watts]

> Greetings,
>
> 	I think I should discuss this in the list...
>
> 	Recently I'm into developing an Elastic Quota File System (EQFS). This
> file system works on a simple concept ... give it to others if you're not
> using it, let others use it, but on the guarantee that you get it back when
> you need it!!

How do you intend to guarantee this?
Randomly deleting a users files to free up disk space is a Bad (tm) idea, so 
what other mechanism are you going to employ?

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Ok, this is what I propose:

    Lets say there are just 2 users with 100 megs of individual quota, user
A is using 20 megs and user B is running out of his quota. Now what B could
do is delete some files himself and make some free space for storing other
files. Now what I say is instead of deleting the files, he declares those
files as elastic.

Now, moment he makes that files elastic, that much amount of space is added
to his quota. Here Mark Cooke's equation applies with some modifications:
N no. of users,
Qi allocated quota of ith user
Ui individual disk usage of ith user ( should be <= allocated quota of ith
user ),
D disk threshold; thats the amount of disk space admin wants to allow the
users to use (should be >= sum of all users' allocated quota, i.e. summation
Qi ; for i = 0 to N - 1).

Total usage of all the users (here A & B) should be at _anytime_ less than
D. i.e. summation Ui <= D; for i = 0 to N - 1.

The point to note here is that we are not bothering how much quota has been
allocated to an individual user by the admin, but we are more interested in
the usage pattern followed by the users. E.g. if user B wants additional
space of say 25 megs, he picks up 25 megs of his files and 'marks' them
elastic. Now his quota is increased to 125 megs and he can now add more 25
megs of files; at the same time allocated quota for user A is left
unaffected. Applying the above equation total usage now is A: 20 megs, B:
125 megs, now total 145 <= D, say 200 megs. Thus this should be ok for the
system, since the usage is within bounds.

Now what happens if Ui > D? This can happen when user A tries to recliam his
space. i.e. if user A adds say more 70 megs of files, so the total usage is
now - A: 90 megs, B: 125 megs; 215 ! <= D. The moment the total usage
crosses the value, 'action' will be taken on the elastic files. Here elastic
files are of user B so only those will be affected and users A's data will
be untouched, so in a way this will be completely transparent to user A.
What action should be taken can be specified by the user while making the
files elastic. He can either opt to delete the file, compress it or move it
to some place (backup) where he know he has write access. The corresponding
action will be taken until the threshold is met.

Will this work?? We are relying on the 'free' space ( i.e. D - Ui ) for the
users to benefit. The chances of having a greater value for D - Ui increases
with the increase in the number of users, i.e. N. Here we are talking about
2 users but think of 10000+ users where all the users will probably never
use up _all_ the allocated disk space. This user behavior can be well
exploited.

EQFS can be best fitted in the mail servers. Here e.g. I make whole
linux-kernel mailing list elastic. As long as Ui <= D I get to keep all the
messages, whenever Ui > D, messages with latest dates will be 'acted' upon.

For variable quota needs, admin can allocate different quotas for different
users, but this can get tiresome when N is large. With EQFS, he can allocate
fixed quota for each user ( old and new ) , set up a value for D and relax.
The users will automatically get the quota they need. One may ask that this
can be done by just setting up value of D, checking it against summation Ui
and not allocating individual quotas at all. But when summation Ui crosses D
value, whose file to act on? Moreover with both individual quotas and D, we
give users 'controlled' flexibility just like elastic - it can be stretched
but not beyond a certain range.

What happens when an user tries to eat up all the free ( D - Ui ) space?
This answer is implementation dependent because you need to make a decision:
should an user be allowed to make a file elastic when Ui == D . I think by
saying 'yes' we eliminate some users' mischief of eating up all free space.

Queries, comments, suggestions welcome.

regs,
AG

> On Wed, 2004-06-23 at 18:53, Mark Watts wrote:
> > > Greetings,
> > >
> > > I think I should discuss this in the list...
> > >
> > > Recently I'm into developing an Elastic Quota File System (EQFS). This
> > > file system works on a simple concept ... give it to others if you're
not
> > > using it, let others use it, but on the guarantee that you get it back
when
> > > you need it!!
> >
> > How do you intend to guarantee this?
> > Randomly deleting a users files to free up disk space is a Bad (tm)
idea, so
> > what other mechanism are you going to employ?
>
> Hi Mark, Amit,
>
> Simple example of a flexible quota scheme:
>
> N users with Q megabytes of guaranteed quota
> D total megs of disk storage
> The difference D - N*Q is the amount you can be flexible with.
>
>
> The above is a somewhat different scheme than the 'give your unused
> quota back to others' part of Amit's post though.
>
> If Amit does actually mean to have a situation where the remaining
> guaranteed quota is less than the actual remaining free space, there is
> *no way* to satisfy the guarantee.
>
> Imagine the worst case scenario if all users suddenly want their
> guaranteed quota.  The only way to free up space is deleting files from
> over-quota users - something which would be unacceptable operationally,
> IMHO.
>
>
> That said, I'll read your tech description with interest when it comes
> out,
>
> Mark
>

Re: Elastic Quota File System (EQFS)

[Olaf Dabrunz]

On 24-Jun-04, Amit Gud wrote:
> Ok, this is what I propose:
> 
>     Lets say there are just 2 users with 100 megs of individual quota, user
> A is using 20 megs and user B is running out of his quota. Now what B could
> do is delete some files himself and make some free space for storing other
> files. Now what I say is instead of deleting the files, he declares those
> files as elastic.
> 
> Now, moment he makes that files elastic, that much amount of space is added
> to his quota. Here Mark Cooke's equation applies with some modifications:
> N no. of users,
> Qi allocated quota of ith user
> Ui individual disk usage of ith user ( should be <= allocated quota of ith
> user ),
> D disk threshold; thats the amount of disk space admin wants to allow the
> users to use (should be >= sum of all users' allocated quota, i.e. summation
> Qi ; for i = 0 to N - 1).
> 
> Total usage of all the users (here A & B) should be at _anytime_ less than
> D. i.e. summation Ui <= D; for i = 0 to N - 1.
> 
> The point to note here is that we are not bothering how much quota has been
> allocated to an individual user by the admin, but we are more interested in
> the usage pattern followed by the users. E.g. if user B wants additional
> space of say 25 megs, he picks up 25 megs of his files and 'marks' them
> elastic. Now his quota is increased to 125 megs and he can now add more 25
> megs of files; at the same time allocated quota for user A is left
> unaffected. Applying the above equation total usage now is A: 20 megs, B:
> 125 megs, now total 145 <= D, say 200 megs. Thus this should be ok for the
> system, since the usage is within bounds.
> 
> Now what happens if Ui > D? This can happen when user A tries to recliam his
> space. i.e. if user A adds say more 70 megs of files, so the total usage is
> now - A: 90 megs, B: 125 megs; 215 ! <= D. The moment the total usage
> crosses the value, 'action' will be taken on the elastic files. Here elastic
> files are of user B so only those will be affected and users A's data will
> be untouched, so in a way this will be completely transparent to user A.
> What action should be taken can be specified by the user while making the
> files elastic. He can either opt to delete the file, compress it or move it
> to some place (backup) where he know he has write access. The corresponding

- having files disappear at the discretion of the filesystem seems to be
  bad behaviour: either I need this file, then I do not want it to just
  disappear, or I do not need it, and then I can delete it myself.

  Since my idea of which files I need and which I do not need changes
  over time, I believe it is far better that I can control which files I
  need and which I do not need whenever other constraints (e.g. quota
  filled up) make this decision necessary. Also, then I can opt to try
  to convince someone to increase my quota.

- moving the file to some other place (backup) does not seem to be a
  viable option:

  - If the backup media is always accessible, then why can't the user
    store the "elastic" files there immediately?
    -> advantages:
      - the user knows where his file is
      - applications that remember the path to a file will be able to
	access it

  - If the backup media will only be accessible after manually inserting
    it into some drive, this amounts to sending an E-Mail to the backup
    admin and then pass a list of backup files to the backup software.

    But now getting the file back involves a considerable amount of
    manual and administrative work. And it involves bugging the backup
    admin, who now becomes the bottleneck of your EQFS.

So this narrows down to the effective handling of backup procedures and
the effective administration of fixed quotas and centralization of data.

If you have many users it is also likely that there are more people
interested in big data-files. So you need to help these people organize
themselves e.g. by helping them to create mailing-list, web-pages or
letting them install servers that makes the data centrally available
with some interface that they can use to select parts of the data.

I would rather suggest that if the file does not fit within a given
quota, the user should apply for more quota and give reasons for that.

I believe that flexible or "elastic" allocation of ressources is a good
idea in general, but it only works if you have cheap and easy ways to
control both allocation and deallocation. So in the case of CBQ in
networks this works, since bandwidth can easily and quickly be allocated
and deallocated.

But for filesystem space this requires something like a "slower (= less
expensive), bigger, always accessible" third level of storage in the
"RAM, disk, ..." hierarchy. And then you would need an easy or even
transparent way to access files on this third level storage. And you
need to make sure that, although you obviously *need* the data for
something, you still can afford to increase retrieval times by several
orders of magnitude at the discretion of the filesystem.

But usually all this can be done by scripts as well.

Still, there is a scenario and a combination of features for such a
filesystem that IMHO would make it useful:

- Provide allocation of overquota as you described it.
- Let the filesystem move (parts of) the "elastic" files to some
  third-level backing-store on an as-needed basis. This provides you
  with a not-so-cheap (but cheaper than manual handling) resource
  management facility.

Now you can use the third-level storage as a backing store for
hard-drive space, analoguous to what swap-space provides for RAM. And
you can "swap in" parts of files from there and cache them on the hard
drive. So "elastic" files are actually files that are "swappable" to
backing store.

This assumes that the "elastic" files meet the requirements for a
"working set" in a similar fashion as for RAM-based data. I.e. the swap
operations need only be invoked relatively seldom.

If this is not the case, your site/customer needs to consider buying
more hard drive space (and maybe also RAM).


The tradeoff for the user now is:
    - do not have the big file(s) OR
    - have them and be able to use them in a random-access fashion from
      any application, but maybe only with a (quite) slow access time,
      but without additional administrative/manual hassle

Maybe this is a good tradeoff for a significant amount of users. Maybe
there are sites/customers that have the required backing store (or would
consider buying into this). I do not know. Find a sponsor, do some field
research and give it a try.

> action will be taken until the threshold is met.
> 
> Will this work?? We are relying on the 'free' space ( i.e. D - Ui ) for the
> users to benefit. The chances of having a greater value for D - Ui increases
> with the increase in the number of users, i.e. N. Here we are talking about
> 2 users but think of 10000+ users where all the users will probably never
> use up _all_ the allocated disk space. This user behavior can be well
> exploited.
> 
> EQFS can be best fitted in the mail servers. Here e.g. I make whole
> linux-kernel mailing list elastic. As long as Ui <= D I get to keep all the
> messages, whenever Ui > D, messages with latest dates will be 'acted' upon.
> 
> For variable quota needs, admin can allocate different quotas for different
> users, but this can get tiresome when N is large. With EQFS, he can allocate
> fixed quota for each user ( old and new ) , set up a value for D and relax.
> The users will automatically get the quota they need. One may ask that this
> can be done by just setting up value of D, checking it against summation Ui
> and not allocating individual quotas at all. But when summation Ui crosses D
> value, whose file to act on? Moreover with both individual quotas and D, we
> give users 'controlled' flexibility just like elastic - it can be stretched
> but not beyond a certain range.
> 
> What happens when an user tries to eat up all the free ( D - Ui ) space?
> This answer is implementation dependent because you need to make a decision:
> should an user be allowed to make a file elastic when Ui == D . I think by
> saying 'yes' we eliminate some users' mischief of eating up all free space.
> 
> Queries, comments, suggestions welcome.
> 
> regs,
> AG
> 
> > On Wed, 2004-06-23 at 18:53, Mark Watts wrote:
> > > > Greetings,
> > > >
> > > > I think I should discuss this in the list...
> > > >
> > > > Recently I'm into developing an Elastic Quota File System (EQFS). This
> > > > file system works on a simple concept ... give it to others if you're
> not
> > > > using it, let others use it, but on the guarantee that you get it back
> when
> > > > you need it!!
> > >
> > > How do you intend to guarantee this?
> > > Randomly deleting a users files to free up disk space is a Bad (tm)
> idea, so
> > > what other mechanism are you going to employ?
> >
> > Hi Mark, Amit,
> >
> > Simple example of a flexible quota scheme:
> >
> > N users with Q megabytes of guaranteed quota
> > D total megs of disk storage
> > The difference D - N*Q is the amount you can be flexible with.
> >
> >
> > The above is a somewhat different scheme than the 'give your unused
> > quota back to others' part of Amit's post though.
> >
> > If Amit does actually mean to have a situation where the remaining
> > guaranteed quota is less than the actual remaining free space, there is
> > *no way* to satisfy the guarantee.
> >
> > Imagine the worst case scenario if all users suddenly want their
> > guaranteed quota.  The only way to free up space is deleting files from
> > over-quota users - something which would be unacceptable operationally,
> > IMHO.
> >
> >
> > That said, I'll read your tech description with interest when it comes
> > out,
> >
> > Mark
> >
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Olaf Dabrunz (od/odabrunz), SUSE Linux AG, Nürnberg

Re: Elastic Quota File System (EQFS)

[Sam Elstob]

Olaf Dabrunz wrote:
> On 24-Jun-04, Amit Gud wrote:
> 
>>Ok, this is what I propose:
>>
>>    Lets say there are just 2 users with 100 megs of individual quota, user
>>A is using 20 megs and user B is running out of his quota. Now what B could
>>do is delete some files himself and make some free space for storing other
>>files. Now what I say is instead of deleting the files, he declares those
>>files as elastic.
>>
>>Now, moment he makes that files elastic, that much amount of space is added
>>to his quota. Here Mark Cooke's equation applies with some modifications:
>>N no. of users,
>>Qi allocated quota of ith user
>>Ui individual disk usage of ith user ( should be <= allocated quota of ith
>>user ),
>>D disk threshold; thats the amount of disk space admin wants to allow the
>>users to use (should be >= sum of all users' allocated quota, i.e. summation
>>Qi ; for i = 0 to N - 1).
>>
>>Total usage of all the users (here A & B) should be at _anytime_ less than
>>D. i.e. summation Ui <= D; for i = 0 to N - 1.
>>
>>The point to note here is that we are not bothering how much quota has been
>>allocated to an individual user by the admin, but we are more interested in
>>the usage pattern followed by the users. E.g. if user B wants additional
>>space of say 25 megs, he picks up 25 megs of his files and 'marks' them
>>elastic. Now his quota is increased to 125 megs and he can now add more 25
>>megs of files; at the same time allocated quota for user A is left
>>unaffected. Applying the above equation total usage now is A: 20 megs, B:
>>125 megs, now total 145 <= D, say 200 megs. Thus this should be ok for the
>>system, since the usage is within bounds.
>>
>>Now what happens if Ui > D? This can happen when user A tries to recliam his
>>space. i.e. if user A adds say more 70 megs of files, so the total usage is
>>now - A: 90 megs, B: 125 megs; 215 ! <= D. The moment the total usage
>>crosses the value, 'action' will be taken on the elastic files. Here elastic
>>files are of user B so only those will be affected and users A's data will
>>be untouched, so in a way this will be completely transparent to user A.
>>What action should be taken can be specified by the user while making the
>>files elastic. He can either opt to delete the file, compress it or move it
>>to some place (backup) where he know he has write access. The corresponding
> 
> 
> - having files disappear at the discretion of the filesystem seems to be
>   bad behaviour: either I need this file, then I do not want it to just
>   disappear, or I do not need it, and then I can delete it myself.
> 
>   Since my idea of which files I need and which I do not need changes
>   over time, I believe it is far better that I can control which files I
>   need and which I do not need whenever other constraints (e.g. quota
>   filled up) make this decision necessary. Also, then I can opt to try
>   to convince someone to increase my quota.
> 
> - moving the file to some other place (backup) does not seem to be a
>   viable option:
> 
>   - If the backup media is always accessible, then why can't the user
>     store the "elastic" files there immediately?
>     -> advantages:
>       - the user knows where his file is
>       - applications that remember the path to a file will be able to
> 	access it
> 
>   - If the backup media will only be accessible after manually inserting
>     it into some drive, this amounts to sending an E-Mail to the backup
>     admin and then pass a list of backup files to the backup software.
> 
>     But now getting the file back involves a considerable amount of
>     manual and administrative work. And it involves bugging the backup
>     admin, who now becomes the bottleneck of your EQFS.
> 
> So this narrows down to the effective handling of backup procedures and
> the effective administration of fixed quotas and centralization of data.
> 
> If you have many users it is also likely that there are more people
> interested in big data-files. So you need to help these people organize
> themselves e.g. by helping them to create mailing-list, web-pages or
> letting them install servers that makes the data centrally available
> with some interface that they can use to select parts of the data.
> 
> I would rather suggest that if the file does not fit within a given
> quota, the user should apply for more quota and give reasons for that.
> 
> I believe that flexible or "elastic" allocation of ressources is a good
> idea in general, but it only works if you have cheap and easy ways to
> control both allocation and deallocation. So in the case of CBQ in
> networks this works, since bandwidth can easily and quickly be allocated
> and deallocated.
> 
> But for filesystem space this requires something like a "slower (= less
> expensive), bigger, always accessible" third level of storage in the
> "RAM, disk, ..." hierarchy. And then you would need an easy or even
> transparent way to access files on this third level storage. And you
> need to make sure that, although you obviously *need* the data for
> something, you still can afford to increase retrieval times by several
> orders of magnitude at the discretion of the filesystem.
> 
> But usually all this can be done by scripts as well.
> 
> Still, there is a scenario and a combination of features for such a
> filesystem that IMHO would make it useful:
> 
> - Provide allocation of overquota as you described it.
> - Let the filesystem move (parts of) the "elastic" files to some
>   third-level backing-store on an as-needed basis. This provides you
>   with a not-so-cheap (but cheaper than manual handling) resource
>   management facility.
> 
> Now you can use the third-level storage as a backing store for
> hard-drive space, analoguous to what swap-space provides for RAM. And
> you can "swap in" parts of files from there and cache them on the hard
> drive. So "elastic" files are actually files that are "swappable" to
> backing store.
> 
> This assumes that the "elastic" files meet the requirements for a
> "working set" in a similar fashion as for RAM-based data. I.e. the swap
> operations need only be invoked relatively seldom.
> 
> If this is not the case, your site/customer needs to consider buying
> more hard drive space (and maybe also RAM).
> 
> 
> The tradeoff for the user now is:
>     - do not have the big file(s) OR
>     - have them and be able to use them in a random-access fashion from
>       any application, but maybe only with a (quite) slow access time,
>       but without additional administrative/manual hassle
> 
> Maybe this is a good tradeoff for a significant amount of users. Maybe
> there are sites/customers that have the required backing store (or would
> consider buying into this). I do not know. Find a sponsor, do some field
> research and give it a try.
> 
> 
>>action will be taken until the threshold is met.
>>
>>Will this work?? We are relying on the 'free' space ( i.e. D - Ui ) for the
>>users to benefit. The chances of having a greater value for D - Ui increases
>>with the increase in the number of users, i.e. N. Here we are talking about
>>2 users but think of 10000+ users where all the users will probably never
>>use up _all_ the allocated disk space. This user behavior can be well
>>exploited.
>>
>>EQFS can be best fitted in the mail servers. Here e.g. I make whole
>>linux-kernel mailing list elastic. As long as Ui <= D I get to keep all the
>>messages, whenever Ui > D, messages with latest dates will be 'acted' upon.
>>
>>For variable quota needs, admin can allocate different quotas for different
>>users, but this can get tiresome when N is large. With EQFS, he can allocate
>>fixed quota for each user ( old and new ) , set up a value for D and relax.
>>The users will automatically get the quota they need. One may ask that this
>>can be done by just setting up value of D, checking it against summation Ui
>>and not allocating individual quotas at all. But when summation Ui crosses D
>>value, whose file to act on? Moreover with both individual quotas and D, we
>>give users 'controlled' flexibility just like elastic - it can be stretched
>>but not beyond a certain range.
>>
>>What happens when an user tries to eat up all the free ( D - Ui ) space?
>>This answer is implementation dependent because you need to make a decision:
>>should an user be allowed to make a file elastic when Ui == D . I think by
>>saying 'yes' we eliminate some users' mischief of eating up all free space.
>>
>>Queries, comments, suggestions welcome.
>>
>>regs,
>>AG
>>
>>
>>>On Wed, 2004-06-23 at 18:53, Mark Watts wrote:
>>>
>>>>>Greetings,
>>>>>
>>>>>I think I should discuss this in the list...
>>>>>
>>>>>Recently I'm into developing an Elastic Quota File System (EQFS). This
>>>>>file system works on a simple concept ... give it to others if you're
>>
>>not
>>
>>>>>using it, let others use it, but on the guarantee that you get it back
>>
>>when
>>
>>>>>you need it!!
>>>>
>>>>How do you intend to guarantee this?
>>>>Randomly deleting a users files to free up disk space is a Bad (tm)
>>
>>idea, so
>>
>>>>what other mechanism are you going to employ?
>>>
>>>Hi Mark, Amit,
>>>
>>>Simple example of a flexible quota scheme:
>>>
>>>N users with Q megabytes of guaranteed quota
>>>D total megs of disk storage
>>>The difference D - N*Q is the amount you can be flexible with.
>>>
>>>
>>>The above is a somewhat different scheme than the 'give your unused
>>>quota back to others' part of Amit's post though.
>>>
>>>If Amit does actually mean to have a situation where the remaining
>>>guaranteed quota is less than the actual remaining free space, there is
>>>*no way* to satisfy the guarantee.
>>>
>>>Imagine the worst case scenario if all users suddenly want their
>>>guaranteed quota.  The only way to free up space is deleting files from
>>>over-quota users - something which would be unacceptable operationally,
>>>IMHO.
>>>
>>>
>>>That said, I'll read your tech description with interest when it comes
>>>out,
>>>
>>>Mark
>>>
>>
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Amit

I agree that the situation with 10000+ users is where such a filesystem 
becomse useful.  It is the difference between actual disk usage and sum 
of all disk quota that provides the elastisicty.

Imagine an ISP providing free mail with an advertised quota of 1GB per 
user.  With 100,000 users, 100TB of storage would be needed.  However, 
it is likely that most users would not use their full quota meaning a 
lot of wasted storage capacity, at the expense of the mail provider. 
What if the EFS could allow the sum of all advertised quotas to total 
more than the actual online disk capacity.  Assuming average quota usage 
settles down and doesn't suddenly change on mass the actual online 
storage capacity of the system could be much lower (and thus cheaper) 
than the case where every user is at full quota.  When the actual disk 
usage reaches, say 70%, of the attached capacity the administrator could 
be notified and action taken e.g. attach more disks and extend the 
volume, or backup and delete dead accounts.

I'm talking about the case where large (10000>) numbers of users are 
involved with a habit of using less than some advertised quota. 
Obviously the adverised quota is still available to those who want it, 
but overall the necessary disk space is less than the sum of all 
advertised disk quotas.

Sam Elstob

Re: Elastic Quota File System (EQFS)

[Olaf Dabrunz]

On 24-Jun-04, Sam Elstob wrote:
> Amit
> 
> I agree that the situation with 10000+ users is where such a filesystem 
> becomse useful.  It is the difference between actual disk usage and sum 
> of all disk quota that provides the elastisicty.
> 
> Imagine an ISP providing free mail with an advertised quota of 1GB per 
> user.  With 100,000 users, 100TB of storage would be needed.  However, 
> it is likely that most users would not use their full quota meaning a 
> lot of wasted storage capacity, at the expense of the mail provider. 
> What if the EFS could allow the sum of all advertised quotas to total 
> more than the actual online disk capacity.  Assuming average quota usage 
> settles down and doesn't suddenly change on mass the actual online 
> storage capacity of the system could be much lower (and thus cheaper) 
> than the case where every user is at full quota.  When the actual disk 
> usage reaches, say 70%, of the attached capacity the administrator could 
> be notified and action taken e.g. attach more disks and extend the 
> volume, or backup and delete dead accounts.

This case is easily solved without any new mechanism. Simply assign a
quota of 1GB to each user while providing physical disk space only for
the expected filling + free space margin. Handle additional space
requirements as you described above.

What Amid tries to solve is obviously a different "scenario": how to put
more files on the disks than fit in the physically available space.

He tries to do that by
   - squeezing in more data by compressing other data
   - deleting files
   - backing up files

And he provides a mechanism how a user can buy more space by marking
other files as "shrink/delete/back up at filesystems discretion". So in
effect he says: buy more space by marking files as "not needed, may be
removed (deleted or moved to some unreachable place) at any time". Then
I would rather opt to "clean up" my files myself whenever I need more
space within my quota.

(The compression case is actually not working out, since it will only
get back as much space as the compression achieves. So the elastic quota
may only be expanded by the space saved when the file is compressed. To
find out this ratio, you have to compress the file. But then you can as
well have your filesystem transparently compress the file.)

> I'm talking about the case where large (10000>) numbers of users are 
> involved with a habit of using less than some advertised quota. 
> Obviously the adverised quota is still available to those who want it, 
> but overall the necessary disk space is less than the sum of all 
> advertised disk quotas.
> 
> Sam Elstob

-- 
Olaf Dabrunz (od/odabrunz), SUSE Linux AG, Nürnberg

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> > What action should be taken can be specified by the user while making the
> > files elastic. He can either opt to delete the file, compress it or move it
> > to some place (backup) where he know he has write access. The corresponding
> 
> - having files disappear at the discretion of the filesystem seems to be
>   bad behaviour: either I need this file, then I do not want it to just
>   disappear, or I do not need it, and then I can delete it myself.


Actually, think .o or mozilla cache files... You don't need them, but if you have them,
things are faster.

				Pavel

-- 
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         

Re: Elastic Quota File System (EQFS)

[Fao, Sean]

Pavel Machek wrote:

>>>What action should be taken can be specified by the user while making the
>>>files elastic. He can either opt to delete the file, compress it or move it
>>>to some place (backup) where he know he has write access. The corresponding
>>>      
>>>
>>- having files disappear at the discretion of the filesystem seems to be
>>  bad behaviour: either I need this file, then I do not want it to just
>>  disappear, or I do not need it, and then I can delete it myself.
>>    
>>
>
>
>Actually, think .o or mozilla cache files... You don't need them, but if you have them,
>things are faster.
>  
>
I fail to understand the point you're trying to make.  Are you 
suggesting that a feature doesn't necessarily have to be implemented, 
just because it's there?  If so, the proposed idea on the "elastic" file 
system differs greatly.  Cached content, for instance, speeds up the 
browsing experience *without* hindering the ability of the application 
to function normally.  Caching is a true enhancement --in most 
circumstances.  I can personally see no way to implement EQFS without 
greatly exasperating end users with its shortcomings.

Sean

Re: Elastic Quota File System (EQFS)

[Timothy Miller]

Fao, Sean wrote:
> Pavel Machek wrote:
> 
>>>> What action should be taken can be specified by the user while 
>>>> making the
>>>> files elastic. He can either opt to delete the file, compress it or 
>>>> move it
>>>> to some place (backup) where he know he has write access. The 
>>>> corresponding
>>>>     
>>>
>>> - having files disappear at the discretion of the filesystem seems to be
>>>  bad behaviour: either I need this file, then I do not want it to just
>>>  disappear, or I do not need it, and then I can delete it myself.
>>>   
>>
>>
>>
>> Actually, think .o or mozilla cache files... You don't need them, but 
>> if you have them,
>> things are faster.
>>  
>>
> I fail to understand the point you're trying to make.  Are you 
> suggesting that a feature doesn't necessarily have to be implemented, 
> just because it's there?  If so, the proposed idea on the "elastic" file 
> system differs greatly.  Cached content, for instance, speeds up the 
> browsing experience *without* hindering the ability of the application 
> to function normally.  Caching is a true enhancement --in most 
> circumstances.  I can personally see no way to implement EQFS without 
> greatly exasperating end users with its shortcomings.
> 
> Sean
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: Elastic Quota File System (EQFS)

[Timothy Miller]

Fao, Sean wrote:

> I fail to understand the point you're trying to make.  Are you 
> suggesting that a feature doesn't necessarily have to be implemented, 
> just because it's there?  If so, the proposed idea on the "elastic" file 
> system differs greatly.  Cached content, for instance, speeds up the 
> browsing experience *without* hindering the ability of the application 
> to function normally.  Caching is a true enhancement --in most 
> circumstances.  I can personally see no way to implement EQFS without 
> greatly exasperating end users with its shortcomings.


What you need is a small number of fast, expensive drives, and a large 
array of cheap drives, and use the fast drives as a cache for user files.

But this is completely different from elastic quotas.

My solution to this would be to have large arrays of cheap disks and put 
lots of RAM in the server and let RAM be the cache.

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> >>>What action should be taken can be specified by the user while making the
> >>>files elastic. He can either opt to delete the file, compress it or move 
> >>>it
> >>>to some place (backup) where he know he has write access. The 
> >>>corresponding
> >>>     
> >>>
> >>- having files disappear at the discretion of the filesystem seems to be
> >> bad behaviour: either I need this file, then I do not want it to just
> >> disappear, or I do not need it, and then I can delete it myself.
> >>   
> >>
> >
> >
> >Actually, think .o or mozilla cache files... You don't need them, but if 
> >you have them,
> >things are faster.
> > 
> >
> I fail to understand the point you're trying to make.  Are you 
> suggesting that a feature doesn't necessarily have to be implemented, 
> just because it's there?  If so, the proposed idea on the "elastic" file 
> system differs greatly.  Cached content, for instance, speeds up the 
> browsing experience *without* hindering the ability of the application 
> to function normally.  Caching is a true enhancement --in most 
> circumstances.  I can personally see no way to implement EQFS without 
> greatly exasperating end users with its shortcomings.

Okay, lets make it explicit.

On one school server, theres 10MB quota. (Okay, its admins are
BOFHs^H^H^H^H^HSISAL). Everyone tries to run mozilla there (because
its installed as default!), and immediately fills his/her quota with
cache files, leading to failed login next time (gnome just will not
start if it can't write to ~).

Imagine mozilla automatically marking cache files "elastic".

That would solve the problem -- mozilla caches would go away when disk
space was demanded, still mozilla's on-disk caching would be effective
when there is enough disk space.
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: Elastic Quota File System (EQFS)

[alan]

On Thu, 24 Jun 2004, Pavel Machek wrote:

> Okay, lets make it explicit.
> 
> On one school server, theres 10MB quota. (Okay, its admins are
> BOFHs^H^H^H^H^HSISAL). Everyone tries to run mozilla there (because
> its installed as default!), and immediately fills his/her quota with
> cache files, leading to failed login next time (gnome just will not
> start if it can't write to ~).
> 
> Imagine mozilla automatically marking cache files "elastic".
> 
> That would solve the problem -- mozilla caches would go away when disk
> space was demanded, still mozilla's on-disk caching would be effective
> when there is enough disk space.

How does Mozilla (or any process) react when its files are deleted from 
under it?  Would the file remain until all the open processes close the 
file or would it just "disappear"?  Would it delete entire directories or 
just some of the files?  How does it choose?  (First up against the delete 
when the drive space fills...)

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> > On one school server, theres 10MB quota. (Okay, its admins are
> > BOFHs^H^H^H^H^HSISAL). Everyone tries to run mozilla there (because
> > its installed as default!), and immediately fills his/her quota with
> > cache files, leading to failed login next time (gnome just will not
> > start if it can't write to ~).
> > 
> > Imagine mozilla automatically marking cache files "elastic".
> > 
> > That would solve the problem -- mozilla caches would go away when disk
> > space was demanded, still mozilla's on-disk caching would be effective
> > when there is enough disk space.
> 
> How does Mozilla (or any process) react when its files are deleted from 
> under it?  Would the file remain until all the open processes close the 
> file or would it just "disappear"? 

Of course, if mozilla marked them "elastic" it should better be
prepared for they disappearance. I'd disappear them with simple
unlink(), so they'd physically survive as long as someone held them
open.

>  Would it delete entire directories or 
> just some of the files?  How does it choose?  (First up against the delete 
> when the drive space fills...)

Probably just some of the files... Or you could delete directory, too,
if it was marked "elastic". What to delete first... probably file with
oldest access time? Or random file, with chance of being selected
proportional to file size?

I'm not implementing it, I'm just arguing that it is usefull.
								Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: Elastic Quota File System (EQFS)

[alan]

On Fri, 25 Jun 2004, Pavel Machek wrote:

> Hi!
> 
> > > On one school server, theres 10MB quota. (Okay, its admins are
> > > BOFHs^H^H^H^H^HSISAL). Everyone tries to run mozilla there (because
> > > its installed as default!), and immediately fills his/her quota with
> > > cache files, leading to failed login next time (gnome just will not
> > > start if it can't write to ~).
> > > 
> > > Imagine mozilla automatically marking cache files "elastic".
> > > 
> > > That would solve the problem -- mozilla caches would go away when disk
> > > space was demanded, still mozilla's on-disk caching would be effective
> > > when there is enough disk space.
> > 
> > How does Mozilla (or any process) react when its files are deleted from 
> > under it?  Would the file remain until all the open processes close the 
> > file or would it just "disappear"? 
> 
> Of course, if mozilla marked them "elastic" it should better be
> prepared for they disappearance. I'd disappear them with simple
> unlink(), so they'd physically survive as long as someone held them
> open.

Hard to get the client to support a feature in an experimental file 
system.  You are more likely to handle this like a system policy than 
something set by the client.  (Especially since an app designed to fill 
disk space could just mark its files as elastic to get around quotas.)

> >  Would it delete entire directories or 
> > just some of the files?  How does it choose?  (First up against the delete 
> > when the drive space fills...)
> 
> Probably just some of the files... Or you could delete directory, too,
> if it was marked "elastic". What to delete first... probably file with
> oldest access time? Or random file, with chance of being selected
> proportional to file size?
> 
> I'm not implementing it, I'm just arguing that it is usefull.

I think that would make a bunch of headaches for the app designer.  
Imaging having a cache directory with an indexed database in a directory 
marked as elastic.  What happens whenone part of the multi-file database 
gets nuked in the middle on operation?  You are going to have to 
handle much more error conditions for weird special cases to deal with 
files wandering away without having the app just halt.  (Most programs I 
have seen just halt when a file they need is not found.)

A better option in this case is to reduce the default size of Mozilla's 
cache or expand the size of the quota for each user to deal with the added 
space requirements.

If you are concerned about disk usage from caches, you can always create 
a script that removes the cache(s) when the user logs out.

Re: Elastic Quota File System (EQFS)

[Pavel Machek]

Hi!

> > Of course, if mozilla marked them "elastic" it should better be
> > prepared for they disappearance. I'd disappear them with simple
> > unlink(), so they'd physically survive as long as someone held them
> > open.
> 
> Hard to get the client to support a feature in an experimental file 
> system.  You are more likely to handle this like a system policy than 
> something set by the client.  (Especially since an app designed to fill 
> disk space could just mark its files as elastic to get around
> quotas.)

I believe application support is right thing to do... It may not be
the easiest thing to do, through.

> > >  Would it delete entire directories or 
> > > just some of the files?  How does it choose?  (First up against the delete 
> > > when the drive space fills...)
> > 
> > Probably just some of the files... Or you could delete directory, too,
> > if it was marked "elastic". What to delete first... probably file with
> > oldest access time? Or random file, with chance of being selected
> > proportional to file size?
> > 
> > I'm not implementing it, I'm just arguing that it is usefull.
> 
> I think that would make a bunch of headaches for the app designer.  
> Imaging having a cache directory with an indexed database in a directory 
> marked as elastic.  What happens whenone part of the multi-file database 
> gets nuked in the middle on operation?  You are going to have to 
> handle much more error conditions for weird special cases to deal with 
> files wandering away without having the app just halt.  (Most programs I 
> have seen just halt when a file they need is not found.)

Well, that's arguably application bug. Anyway, noone says that right
thing to do is easy.

> A better option in this case is to reduce the default size of Mozilla's 
> cache or expand the size of the quota for each user to deal with the added 
> space requirements.
> 
> If you are concerned about disk usage from caches, you can always create 
> a script that removes the cache(s) when the user logs out.

That's not the right thing.. that way you loose caching effects around
logins even when there's plenty of space.

There's quite a lot of data -- at least on my systems -- that can be
removed with "only" loss of performance...

1) browser caches

2) package lists, downloaded packages

3) object files

heck, if you know you have reliable network connection 4), you could
even mark stuff like /usr/bin/mozilla elastic, and re-install it from
the network when it is needed... Doing anything more complex than 1)
requires extensive changes all around the kernel and userland, and
you'd probably not call that system unix any more.

I'm not saying that "elastic" feature should go into 2.6 or 2.8 or
whatever, but it still seems interesting to me.
									Pavel
-- 
People were complaining that M$ turns users into beta-testers...
...jr ghea gurz vagb qrirybcref, naq gurl frrz gb yvxr vg gung jnl!

Re: Elastic Quota File System (EQFS)

[Fao, Sean]

>>A better option in this case is to reduce the default size of Mozilla's 
>>cache or expand the size of the quota for each user to deal with the added 
>>space requirements.
>>
>>If you are concerned about disk usage from caches, you can always create 
>>a script that removes the cache(s) when the user logs out.
>>    
>>
>
>That's not the right thing.. that way you loose caching effects around
>logins even when there's plenty of space.
>
>There's quite a lot of data -- at least on my systems -- that can be
>removed with "only" loss of performance...
>
>1) browser caches
>
>2) package lists, downloaded packages
>
>3) object files
>
>heck, if you know you have reliable network connection 4), you could
>even mark stuff like /usr/bin/mozilla elastic, and re-install it from
>the network when it is needed... Doing anything more complex than 1)
>requires extensive changes all around the kernel and userland, and
>you'd probably not call that system unix any more.
>  
>
All the suggested benefits listed above could easily be implemented in a 
script.  For instance, one could design a script that checks the amount 
of disk space at logout.  If the amount of disk space remaining is less 
than X (where X is value predefined by an administrator), the script 
could _suggest_ that corrective action be taken and allow the *user* to 
_decide_ what he/she wants to delete/move.

In a workstation environment, my honest opinion is that quota's are set 
entirely too low if you're running that close to your limit.  On a 
server, I do not see deleting files at the decision of the OS, to be 
beneficial.  Nor do I see any reason to develop the suggested FS to 
implement what should be taken care of by a knowledgeable administrator.

The idea of an elastic file system is interesting until you start 
considering how it would be implemented and what affects it would have 
in a production environment.

Sean

Re: Elastic Quota File System (EQFS)

[Josh Boyer]

On Thu, 2004-06-24 at 19:15, Pavel Machek wrote:

> > A better option in this case is to reduce the default size of Mozilla's 
> > cache or expand the size of the quota for each user to deal with the added 
> > space requirements.
> > 
> > If you are concerned about disk usage from caches, you can always create 
> > a script that removes the cache(s) when the user logs out.
> 
> That's not the right thing.. that way you loose caching effects around
> logins even when there's plenty of space.
> 
> There's quite a lot of data -- at least on my systems -- that can be
> removed with "only" loss of performance...
> 
> 1) browser caches
> 
> 2) package lists, downloaded packages
> 
> 3) object files
> 
> heck, if you know you have reliable network connection 4), you could
> even mark stuff like /usr/bin/mozilla elastic, and re-install it from
> the network when it is needed... Doing anything more complex than 1)
> requires extensive changes all around the kernel and userland, and
> you'd probably not call that system unix any more.
> 
> I'm not saying that "elastic" feature should go into 2.6 or 2.8 or
> whatever, but it still seems interesting to me.

Couldn't most of this be done in userspace with xattrs and a "elastic
quota" daemon?  Mark such files as elastic with an xattr, and when space
is needed for user N, the daemon comes along and deletes the marked
files.  You could even make the deamon semi-smart and take things such
as filesize, least recently used files, etc into account.

Or maybe I am missing something... wouldn't be the first time.

josh

Re: Elastic Quota File System (EQFS)

[Jörn Engel]

On Fri, 25 June 2004 07:07:48 -0500, Josh Boyer wrote:
> 
> Couldn't most of this be done in userspace with xattrs and a "elastic
> quota" daemon?  Mark such files as elastic with an xattr, and when space
> is needed for user N, the daemon comes along and deletes the marked
> files.  You could even make the deamon semi-smart and take things such
> as filesize, least recently used files, etc into account.
> 
> Or maybe I am missing something...

"when space is needed" is hard to detect for the daemon.

Jörn

-- 
To my face you have the audacity to advise me to become a thief - the worst
kind of thief that is conceivable, a thief of spiritual things, a thief of
ideas! It is insufferable, intolerable!
-- M. Binet in Scarabouche

Re: Elastic Quota File System (EQFS)

[Timothy Miller]

I have a much simpler idea that both implements the EQFS and doesn't 
touch the kernel.

Each user is given a quota which applies to their home directory.  (This 
quota is not elastic and if everyone met their quota, everything would 
fit.)  In addition, there is another directory or file system (could be 
on the same disk or even the same partition) to which their quota 
doesn't apply AT ALL.  Let's call this "scratch" space.

Periodically, a daemon checks the disk usage, and whenever the disk 
usage approaches, say, 90%, its starts deleting the oldest files from 
the scratch space until its gets below the watermark.

So anything in "/scratch/$USER/" is free to be deleted by the daemon.

BTW, they did something similar to this when I was in college (I 
graduated in 1996), although they deleted from /scratch manually.

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Timothy Miller wrote:

>
> I have a much simpler idea that both implements the EQFS and doesn't 
> touch the kernel.
>
> Each user is given a quota which applies to their home directory.  
> (This quota is not elastic and if everyone met their quota, everything 
> would fit.)  In addition, there is another directory or file system 
> (could be on the same disk or even the same partition) to which their 
> quota doesn't apply AT ALL.  Let's call this "scratch" space.
>
I guess the system should be more transparent to the users and their 
applications. Here its not convenient to generate .o files or caches in 
/scratch/$USER/ .

AG

Re: Elastic Quota File System (EQFS)

[Helge Hafting]

On Sat, Jun 26, 2004 at 12:14:50AM +0530, Amit Gud wrote:
> Timothy Miller wrote:
> 
> >
> >I have a much simpler idea that both implements the EQFS and doesn't 
> >touch the kernel.
> >
> >Each user is given a quota which applies to their home directory.  
> >(This quota is not elastic and if everyone met their quota, everything 
> >would fit.)  In addition, there is another directory or file system 
> >(could be on the same disk or even the same partition) to which their 
> >quota doesn't apply AT ALL.  Let's call this "scratch" space.
> >
> I guess the system should be more transparent to the users and their 
> applications. Here its not convenient to generate .o files or caches in 
> /scratch/$USER/ .
> 
Symlinks.  I mozilla stores a cache in .mozilla/something,
then make .mozilla/something a symlink to /scratch/user/.mozilla/something

In a big installation, a script can do this for all users
as well as similiar tricks for other caching apps.

Helge Hafting

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Horst von Brand wrote:

> Pavel Machek <pavel@ucw.cz> said:
>  
>
>> Hi!
>>
>>   
>>
>>> Case closed, anyway. It belongs in the kernel only if there is no
>>> reasonable way to do it in userspace.
>>>     
>>
>> But... there's no reasonable way to do this in userspace.
>>   
>
>
> Let's see...
>
>  
>
>> Two pieces of kernel support are needed:
>>
>> 1) some way to indicate "this file is elastic" (okay perhaps xattrs
>> can do this already)
>>   
>
>
> Or just a list of elastic files in ~/.elastic. Even better, could mark 
> them
> as "Just delete", "compress"; could go as far as giving (fallback?) globs
> to select files for each treatment ("If space gets tight, delete *~ 
> files,
> and compress *.tex that haven't been read in a week"). Sounds like a fun
> Perl project...
>
>  
>
.elastic is a file or directory? If its file, daemon has to do the ugly 
insertion deletion of the file enteries. If its a directory, daemon has 
to do the updating of the files in case of mv, plus follow the links for 
unlink, chown if we are storing the files as symlinks and we will be 
wasting the inodes.

Just think of the load on the system if we run a daemon, which sleeps 
for a time depending on the data transfer rate and the amount of free 
space left, and if the free space left is very close to the margin... 
the system might even freeze. The primary intention of taking 
filesystems into kernel is the speed and the security that we get, 
otherwise we do have userspace filesystems!

What we need is an application-transparent system which of course should 
be plugable within a system and which also shouldn't hamper the system 
throughput in a major way.

I really don't see user space implementation in the picture.

AG

Re: Elastic Quota File System (EQFS)

[Olaf Dabrunz]

On 25-Jun-04, Timothy Miller wrote:
> 
> I have a much simpler idea that both implements the EQFS and doesn't 
> touch the kernel.
> 
> Each user is given a quota which applies to their home directory.  (This 
> quota is not elastic and if everyone met their quota, everything would 
> fit.)  In addition, there is another directory or file system (could be 
> on the same disk or even the same partition) to which their quota 
> doesn't apply AT ALL.  Let's call this "scratch" space.
> 
> Periodically, a daemon checks the disk usage, and whenever the disk 
> usage approaches, say, 90%, its starts deleting the oldest files from 
> the scratch space until its gets below the watermark.
> 
> So anything in "/scratch/$USER/" is free to be deleted by the daemon.
> 
> BTW, they did something similar to this when I was in college (I 
> graduated in 1996), although they deleted from /scratch manually.

An easy setup for this is to put /home on a different filesystem than
/tmp, use quota on /home but leave quota off for /tmp. Then most Unix
systems can easily be configured to clean up /tmp periodically, and also
the user is easily aware of the nature of files in /tmp (i.e. they are
"elastic" in some sense).

My university was (and still is) using this setup on many servers.

-- 
Olaf Dabrunz (od/odabrunz), SUSE Linux AG, Nürnberg

Re: Elastic Quota File System (EQFS)

[Lionel Bouton]

Pavel Machek wrote the following on 06/25/2004 12:03 AM :

>Of course, if mozilla marked them "elastic" it should better be
>prepared for they disappearance. I'd disappear them with simple
>unlink(), so they'd physically survive as long as someone held them
>open.
>
>  
>

Doesn't work reliably : the deletion is done in order to reclaim space 
that is needed now. You may want to retry unlinking files until you 
reach the free space needed, but this is clearly a receipe for famine : 
process can wait on writes an unspecified amount of time.


-- 
Lionel Bouton - inet6
---------------------------------------------------------------------
   o              Siege social: 51, rue de Verdun - 92158 Suresnes
  /      _ __ _   Acces Bureaux: 33 rue Benoit Malon - 92150 Suresnes
 / /\  /_  / /_   France
 \/  \/_  / /_/   Tel. +33 (0) 1 41 44 85 36
  Inetsys S.A.    Fax  +33 (0) 1 46 97 20 10
 

Re: Elastic Quota File System (EQFS)

[V13]

On Friday 25 June 2004 10:52, Lionel Bouton wrote:
> Pavel Machek wrote the following on 06/25/2004 12:03 AM :
> >Of course, if mozilla marked them "elastic" it should better be
> >prepared for they disappearance. I'd disappear them with simple
> >unlink(), so they'd physically survive as long as someone held them
> >open.
>
> Doesn't work reliably : the deletion is done in order to reclaim space
> that is needed now. You may want to retry unlinking files until you
> reach the free space needed, but this is clearly a receipe for famine :
> process can wait on writes an unspecified amount of time.

This could be solved like OOM by killing all related processes.

<<V13>>

Re: Elastic Quota File System (EQFS)

[Lionel Bouton]

V13 wrote the following on 06/27/2004 08:18 PM :

>On Friday 25 June 2004 10:52, Lionel Bouton wrote:
>  
>
>>Pavel Machek wrote the following on 06/25/2004 12:03 AM :
>>    
>>
>>>Of course, if mozilla marked them "elastic" it should better be
>>>prepared for they disappearance. I'd disappear them with simple
>>>unlink(), so they'd physically survive as long as someone held them
>>>open.
>>>      
>>>
>>Doesn't work reliably : the deletion is done in order to reclaim space
>>that is needed now. You may want to retry unlinking files until you
>>reach the free space needed, but this is clearly a receipe for famine :
>>process can wait on writes an unspecified amount of time.
>>    
>>
>
>This could be solved like OOM by killing all related processes.
>  
>

I don't want to see mozilla shut down because it was filling a cache 
file during a big download...

Note :
In practice I don't see it as a real problem with good-manered 
applications : as we are speaking of mutualised storage space, 
statistically the system should find files to remove unless it is 
burried under open filedescriptors for elastic files. But this is not 
really robust : a very simple DoS against this is to allocate all the 
available storage space in elastic files and maintain the 
filedescriptors open.

To continue on the "kill process" subject : I think making aware the 
process of the problem is much more sane. I'd really like mozilla to 
tell me : "Sorry, my download cannot continue, the system removed the 
download file in order to free some disk space".
IMHO one way to make this work *reliably* is to allow the fs to remove 
the files from disk directly and not simply unlink them and wait for the 
last fd to be closed. The fs must then return an EBADF (I don't know if 
a new error code tailored for this case is affordable) for subsequent 
read/writes and applications using elastic files must be written to 
gracefully recover from this.
It seems much more logical to me to make applications aware of the 
exotic nature of elastic files and handle the associated behavior. 
Conceptually elastic files seem different enough from regular files that 
you would want to handle them separately at the application level. In 
fact I believe elastic files should be created by elastic aware 
applications and not by adventurous users/admins. For example I'd really 
prefer mozilla to choose to create elastic files when configured to do 
so and not have an admin make a chattr on the cache directory...

Re: Elastic Quota File System (EQFS)

[Amit Gud]

Lionel Bouton wrote:

> V13 wrote the following on 06/27/2004 08:18 PM :
>
> >On Friday 25 June 2004 10:52, Lionel Bouton wrote:
> >
> >
> >>Pavel Machek wrote the following on 06/25/2004 12:03 AM :
> >>
> >>
> >>>Of course, if mozilla marked them "elastic" it should better be
> >>>prepared for they disappearance. I'd disappear them with simple
> >>>unlink(), so they'd physically survive as long as someone held them
> >>>open.
> >>>
> >>>
> >>Doesn't work reliably : the deletion is done in order to reclaim space
> >>that is needed now. You may want to retry unlinking files until you
> >>reach the free space needed, but this is clearly a receipe for famine :
> >>process can wait on writes an unspecified amount of time.
> >>
> >>
> >
> >This could be solved like OOM by killing all related processes.
> >
> >
>
> I don't want to see mozilla shut down because it was filling a cache
> file during a big download...
>
> Note :
> In practice I don't see it as a real problem with good-manered
> applications : as we are speaking of mutualised storage space,
> statistically the system should find files to remove unless it is
> burried under open filedescriptors for elastic files. But this is not
> really robust : a very simple DoS against this is to allocate all the
> available storage space in elastic files and maintain the
> filedescriptors open.
>
> To continue on the "kill process" subject : I think making aware the
> process of the problem is much more sane. I'd really like mozilla to
> tell me : "Sorry, my download cannot continue, the system removed the
> download file in order to free some disk space".
> IMHO one way to make this work *reliably* is to allow the fs to remove
> the files from disk directly and not simply unlink them and wait for the
> last fd to be closed. The fs must then return an EBADF (I don't know if
> a new error code tailored for this case is affordable) for subsequent
> read/writes and applications using elastic files must be written to
> gracefully recover from this.
> It seems much more logical to me to make applications aware of the
> exotic nature of elastic files and handle the associated behavior.
> Conceptually elastic files seem different enough from regular files that
> you would want to handle them separately at the application level. In
> fact I believe elastic files should be created by elastic aware
> applications and not by adventurous users/admins. For example I'd really
> prefer mozilla to choose to create elastic files when configured to do
> so and not have an admin make a chattr on the cache directory...
>

Right. For the moment its safe enough to allow only elastic-aware
applications to
create the elastic files. A new error code would ceratinly be helpful here.
Here we do take away user control of stating which files he don't want in
case the threshold is reached, but this would definately cause other
applications work smoothly provided there are no commonly shared files. But
once most applications become elastic-aware, he can comfortably do so. Also
we can even have a flag, something like -elas, in gcc for creating object
files as elastic.

AG

Re: Elastic Quota File System (EQFS)

[Bernd Eckenfels]

In article <001901c459cd$bc436e40$868209ca@home> you wrote:
> The point to note here is that we are not bothering how much quota has been
> allocated to an individual user by the admin, but we are more interested in
> the usage pattern followed by the users. E.g. if user B wants additional
> space of say 25 megs, he picks up 25 megs of his files and 'marks' them
> elastic. Now his quota is increased to 125 megs and he can now add more 25
> megs of files; at the same time allocated quota for user A is left
> unaffected.

I would better understand if you do not incease the quota, but simply dont
count the elastic files in the consumption.

Greetings
Bernd
-- 
eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/

Re: Elastic Quota File System (EQFS)

[Rik van Riel]

On Wed, 23 Jun 2004, Amit Gud wrote:

> That is the user's quota which he doesn't need is given to the users who
> need it, but on 100% assurance that the originl user can any time
> reclaim his/her quota.

Where do you put to store the "extra data" of the over-quota
users if it can't be stored on the filesystem ?

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan

Elastic Quota File System (EQFS)

[gud]

Greetings,

	I think I should discuss this in the list...

	Recently I'm into developing an Elastic Quota File System (EQFS). This
file system works on a simple concept ... give it to others if you're not
using it, let others use it, but on the guarantee that you get it back when
you need it!!

	Here I'm talking about disk quotas. In any typical network, e.g.
sourceforge, each user is given a fixed amount of quota. 100 Mb in case of
sourceforge. 100 Mb is way over some project requirements and too small for
some projects. EQFS tries to solve this problem by exploiting the users'
usage behavior at runtime. That is the user's quota which he doesn't need
is given to the users who need it, but on 100% assurance that the originl
user can any time reclaim his/her quota.

	Before getting into implementation details I want to have public opinion
about this system. All EQFS tries to do is it maximizes the disk space
usage, which otherwise is wasted if the user doesn't really need the
allocated user..on the other hand it helps avoid the starvation of the user
who needs more space. It also helps administrator to get away with the
problem of variable quota needs..as EQFS itself adjusts according to the
user needs.


regs,
AG


	
http://www.ddsl.net