Re: Elastic Quota File System (EQFS)

Amit Gud
2004-06-25T17:35:46Z

Alan wrote:

>On Fri, 2004-06-25 at 09:25, Pavel Machek wrote:
>
>
>>Hi!
>>
>>
>>
>>>Case closed, anyway. It belongs in the kernel only if there is no
>>>reasonable way to do it in userspace.
>>>
>>>
>>But... there's no reasonable way to do this in userspace.
>>
>>Two pieces of kernel support are needed:
>>
>>1) some way to indicate "this file is elastic" (okay perhaps xattrs
>>can do this already)
>>
>>and either
>>
>>2a) file selection/deletion in kernel
>>
>>or
>>
>>2b) assume that disk does not fill up faster than 1GB/sec, allways
>>keep 1GB free, make "deleting" daemon poll each second [ugly,
>>unreliable]
>>
>>or
>>
>>2c) just before returning -ENOSPC, synchronously tell userspace to
>>free space, and retry the operation.
>>
>>BTW 2c) would be also usefull for undelete. Unfortunately 2c looks
>>very complex, too; it might be easier to do 2a than 2c.
>>
>>
>
>Why does the kernel have to get involved with file deletion?
>
>
>
Involvement of the kernel depends on how we are treating the elastic
files:

- when files are made elastic, are we increasing the user's quota?
if so kernel becomes neccessary for if
- elastic file is deleted: decrement the user's quota
- elastic file is chowned: do the needfull (decrement previous
owners and increment new owners quota)

- when files are made elastic, we are just not accounting the file
size for the user's quota. In this case I dont see any involvement of
the kernel, but will need some other hacks.

>All it needs is to run at sufficient privs.
>
>If you are overflowing drives that easily, it is time to buy a bigger
>drive. It is not the time to start deleting stuff at random. Data is
>usually put on a drive for a reason. Having a human decide what to
>delete is *much* better than letting some automated process do it in
>background.
>
>
>
Admin assigns quotas to the user with using some case studies, like he
won't be allocating 500 Mb for a C language course students. We are
trying to harness some user behavior here. I wish I had some Gartner or
Forrestor figures to show you that how often _all_ the users use up
their 100% quota at any given point of time. I'm assuming that this
*does* happen very rarely and I also assume that my assumption is very
close to correct. The point is their is some 'free' space available for
the users who use up all their quotas. Of course this free space is
likely to be someday fall short for the over-spilling users, but then
its time to increase quotas of all the users, and its then admin's duty
to look at this. Perhaps this could become a good indication for the
admin for increasing the quota of all the users.

>This sounds like a hack to get around a badly designed system with too
>few resources.
>
>Windows has an option to delete files "that are not needed". It tends
>to delete things that you wanted, but had not thought about in a while.
>
>This really strikes me as a bad idea. It has lots of "special" things
>that programs will have to deal with for this particular case.
>This makes things much more complex in userspace for a problem that
>needs to be dealt with in meatspace.
>
>
>
I do believe that this system can be done in userspace, but it has its
own flaws then. Suppose theres a daemon, call it eqfsd. It forks parent
listens a char device, child watches the disk space usage. A kernel
module reports the file deletes, chowns to the char device, parent does
the needfull. Child 'periodically' checks that the threshold is not
reached. Here what can be done is suppose a user can transfer data with
say 5 Mbps speed to his account....then we can easily get the minimum
time required to fill up the remaining free space. Child sleeps for this
much time minus some value for safety. i.e. child sleeps for: ( (D -
Ui) / 5 ) - safety_value seconds.

I'm sure this gotta be slow and *will* slow down the system, so I still
insist that this should be in kspace.

It cannot be denied that there _are_ applications for such a system that
we already discussed and theres a class of users who will find the
system useful.

AG