Reserved Blocks

Reserved Blocks

[Sam Vilain]

How can I reserve space on a filesystem for the superuser?  There
doesn't seem to be an option to mkreiserfs or reiserfstune for it.

Is this essential UNIX filesystem feature implemented?
--
   Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/
    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

  America is the only country that went from barbarism to decadence
without civilization in between.
OSCAR WILDE

Re: Reserved Blocks

[Oleg Drokin]

Hello!

On Mon, Sep 16, 2002 at 11:27:42AM +0100, Sam Vilain wrote:

> How can I reserve space on a filesystem for the superuser?  There
> doesn't seem to be an option to mkreiserfs or reiserfstune for it.

This is not implemented.

Bye,
    Oleg

Re: Reserved Blocks

[Nikita Danilov]

Sam Vilain writes:
 > How can I reserve space on a filesystem for the superuser?  There
 > doesn't seem to be an option to mkreiserfs or reiserfstune for it.

I am afraid, there is no support for this, currently. But implementation
should be fairly trivial, through.

 > 
 > Is this essential UNIX filesystem feature implemented?

Well, nobody asked for it yet. :)

 > --

Nikita.

 >    Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/

Re: Reserved Blocks

[Nikita Danilov]

Sam Vilain writes:
 > Nikita Danilov <Nikita@Namesys.COM> wrote:
 > 
 > >  > How can I reserve space on a filesystem for the superuser?  There
 > >  > doesn't seem to be an option to mkreiserfs or reiserfstune for it.
 > > I am afraid, there is no support for this, currently. But implementation
 > > should be fairly trivial, through.
 > 
 > How trivial?  One new C source file?  One new function and a handful
 > of hooks?  Adding in hooks to already existing kernel functions?  Or
 > adding a new field to reserved space in the superblock of the
 > filesystem?

I guess, it doesn't worth breaking existing disk format, so adding new
field to the on-disk super-block is out of question.

What I thought about is adding new mount option -o root_blk=1000, and
adding one `if' statement to the

fs/reiserfs/bitmap.c:reiserfs_allocate_blocknrs().

Also, if there are not enough free blocks at the -mount- time---issue a
warning. What about this? Oleg?

 > 
 > >  > Is this essential UNIX filesystem feature implemented?
 > > Well, nobody asked for it yet. :)
 > 
 > Allow me to ask on behalf of all the SysAdmins out there.  This is an
 > extremely useful standard safeguard against users filling up /var (or,
 > for single filesystem machines, /).  The superuser not being able to
 > create temporary files in /tmp, /var/tmp, or even /etc/mtab can cause
 > the system to behave erratically on all but the most mature
 > distributions.

I am not questioning this feature usefulness, it is just that there are
always more features to add than resources available, so adding feature
that nobody asked for is not very likely.

 > 
 > Perhaps you should have a section in your FAQ that covers
 > standard/traditional filesystem features not present in ReiserFS, or
 > at what kernel version they were merged in - eg 2.4.19pre3 for inode
 > attributes, and not yet for Filesystem Quotas or SuperUser Reserved
 > Blocks percentage.
 > --
 >    Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/
 >     7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
 >     278A A425 30A9 05B5 2F13
 > 
 > Real software engineers don't program in assembler.  They become
 > queasy at the very thought.

Nikita.

Re: Reserved Blocks

[Oleg Drokin]

Hello!

On Mon, Sep 16, 2002 at 05:15:20PM +0400, Nikita Danilov wrote:

> What I thought about is adding new mount option -o root_blk=1000, and
> adding one `if' statement to the
> fs/reiserfs/bitmap.c:reiserfs_allocate_blocknrs().

Actually two mount options. Or even three.
One option is to specify uid for whom to reserve. Or may be not uid but gid,
too.

> Also, if there are not enough free blocks at the -mount- time---issue a
> warning. What about this? Oleg?

Sounds easy enough.

But is easily defeated, unfortunatelly.
Since the spans across already allocated blocks without any control,
the only moment root-owned process makes tree to occupy one more block to
create some directory, for example, malicious process immediately writes
file tail into that free space...

>  > >  > Is this essential UNIX filesystem feature implemented?
>  > > Well, nobody asked for it yet. :)
>  > Allow me to ask on behalf of all the SysAdmins out there.  This is an
>  > extremely useful standard safeguard against users filling up /var (or,

No. It does not work, I think. Well, it works, but can be easily defeated.

E.g. if I need to overflow your partition where logs are stored, I'd do someting
like that:
while :; do cat /etc/passwd | logger -t nobody ; done
Repeat with different -p swithces if you have several filesystems for different
logs.

Also consider there is set uid application that outputs data.
I just do setuid_app >>/fs_to_fill/somefile
and repeat it that much times to fill the fs.

So this protection you are asking for is mostly imagenary.

>  > Perhaps you should have a section in your FAQ that covers
>  > standard/traditional filesystem features not present in ReiserFS, or
>  > at what kernel version they were merged in - eg 2.4.19pre3 for inode
>  > attributes, and not yet for Filesystem Quotas or SuperUser Reserved
>  > Blocks percentage.

FS quotas are implemented. Link to URL is in our FAQ.

Bye,
    Oleg

Re: Reserved Blocks

[Paul Jarc]

Oleg Drokin <green@namesys.com> wrote:
> Also consider there is set uid application that outputs data.
> I just do setuid_app >>/fs_to_fill/somefile
> and repeat it that much times to fill the fs.

That won't work.  The open() for the ">>" output redirection is done
by the shell, not the setuid program.  But if the setuid program lets
the user specify a filename to write to, then it could be vulnerable,
and it would still be vulnerable even if the program check permissions
with access().


paul

Re: Reserved Blocks

[Oleg Drokin]

Hello!

On Wed, Sep 25, 2002 at 02:07:41PM -0400, Paul Jarc wrote:
> > Also consider there is set uid application that outputs data.
> > I just do setuid_app >>/fs_to_fill/somefile
> > and repeat it that much times to fill the fs.
> That won't work.  The open() for the ">>" output redirection is done

It should.

> by the shell, not the setuid program.  But if the setuid program lets

That's right. But the program that writes into that file is setuid program.
And blocks are allocated by that setuid program.
This is only works if you can create files of fs you want to fiil of course.
At least that's my current belief ;)

Bye,
    Oleg

Re: Reserved Blocks

[Matthias Urlichs]

Hi,

Oleg Drokin:
> That's right. But the program that writes into that file is setuid program.
> And blocks are allocated by that setuid program.

Blocks (possibly more than one) may also be allocated by opening the file,
especially if it happens to have a long name.

If you want limits, use quotas. They may not be perfect, but at least they
work.  ;-)

-- 
Matthias Urlichs     |     noris network AG     |     http://smurf.noris.de/

Re: Reserved Blocks

[Oleg Drokin]

Hello!

On Thu, Sep 26, 2002 at 08:55:13AM +0200, Matthias Urlichs wrote:

> Oleg Drokin:
> > That's right. But the program that writes into that file is setuid program.
> > And blocks are allocated by that setuid program.
> Blocks (possibly more than one) may also be allocated by opening the file,

I'd agree if you replace "opening" with "creating".

> especially if it happens to have a long name.

It cannot be longer than 255 bytes anyway.

> If you want limits, use quotas. They may not be perfect, but at least they
> work.  ;-)

That's true ;)

Bye,
    Oleg

Re: Reserved Blocks

[Matthias Urlichs]

Hi,

Oleg Drokin:
> > Blocks (possibly more than one) may also be allocated by opening the file,
> 
> I'd agree if you replace "opening" with "creating".
> 
Thanks for the correction -- of course.

> > especially if it happens to have a long name.
> 
> It cannot be longer than 255 bytes anyway.
> 
On the other hand, you can write a nontrivial amount of bytes to the file
before Reiserfs truly allocates a block for them. (Unless you disable
that, of course.)



-- 
Matthias Urlichs     |     noris network AG     |     http://smurf.noris.de/

Re: Reserved Blocks

[Oleg Drokin]

Hello!

On Thu, Sep 26, 2002 at 09:59:42AM +0200, Matthias Urlichs wrote:
> > > Blocks (possibly more than one) may also be allocated by opening the file,
> > I'd agree if you replace "opening" with "creating".
> Thanks for the correction -- of course.

But one can use existing file.
Notice my original example used '>>' shell redirector.

> > > especially if it happens to have a long name.
> > It cannot be longer than 255 bytes anyway.
> On the other hand, you can write a nontrivial amount of bytes to the file
> before Reiserfs truly allocates a block for them. (Unless you disable
> that, of course.)

This is not true for reiserfs in 2.4/2.5 kernels.
There first plain block is allocated and on file close it is converted
to the in-tree "direct item" if possible.

Bye,
    Oleg

Re: Reserved Blocks

[Matthias Urlichs]

Hi,

Oleg Drokin:
> This is not true for reiserfs in 2.4/2.5 kernels.
> There first plain block is allocated and on file close it is converted
> to the in-tree "direct item" if possible.
> 
Ah, OK. I didn't know that you do it that way.

-- 
Matthias Urlichs     |     noris network AG     |     http://smurf.noris.de/

Re: Reserved Blocks

[Sam Vilain]

On Mon, 16 Sep 2002 17:15:20 +0400
Nikita Danilov <Nikita@Namesys.COM> wrote:

> I am not questioning this feature usefulness, it is just that there are
> always more features to add than resources available, so adding feature
> that nobody asked for is not very likely.

Still, it is courteous to at least state how far you've got.
--
   Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/
    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

  I dont have any solution, but I certainly admire the problem.
ASHLEIGH BRILLIANT

Re: Reserved Blocks

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 2018 bytes --]

On Mon, 16 Sep 2002 11:27:42 BST, Sam Vilain <sam@vilain.net>  said:
> How can I reserve space on a filesystem for the superuser?  There
> doesn't seem to be an option to mkreiserfs or reiserfstune for it.
> 
> Is this essential UNIX filesystem feature implemented?

"essential feature"?  Hardly.

The *ORIGINAL* reason for the Berkeley FFS doing the "10% reserve"
was to work around the fact that the original FFS code had severe fragmentation
issues if the filesystem ran over 90% full (but at under 90%, there were
enough free blocks usually available to not fragment TOO badly - the "knee"
in the performance curve seems to have been around 92-94% full).  As such,
this shouldn't be at all required for a modern filesystem, as it should be
hardened against fragmentation to start with.

A lot of sites have started abusing the "10% reserve" feature in order to
protect themselves from people hosing things up by filling the disk - these
are usually boxes that have "one big / filesystem" for everything.  I
suggest that this is a *BAD* practice, and that the *proper* protection
here is to have separate /, /usr, /var, /tmp and /home file systems.

If you're worried about security enough to not want users doing a
denial-of-service attack by filling the disk, then DON'T HAVE a world-writable
directory on a partition that will cause trouble if it gets full.

Among other things, if there's a world-writable directory on the same
filesystem, you get exposed to a whole raft of hard-link attacks that
simply Wont Work if (for example) /tmp is a separate filesystem.  You'd
be amazed how many tools would fail in interesting ways if you were able to
do 'ln /etc/passwd /tmp/totally-foobar' because they expect exactly one link
to the /etc/passwd inode.  And if you thought the average symlink attack was
bad, a hardlink is worse - root can stat() the temp file and say "Oh, it's OK,
I own it" and proceed to do Something Very Dumb...
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reserved Blocks

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1388 bytes --]

On Mon, 16 Sep 2002 17:13:52 +0200, Russell Coker said:

> Alternately use quotas, speaking of which, do Quotas work on ReiserFS without 
> additional patches yet?  Quotas really are an essential feature.

Quotas are a Good Thing - but note that quite often you get into the
situation where you want to "overcommit" a partition (for instance, 2G of
disk, and 10 users each of which have a 500M limit).  It helps, but you're
still open to attacks by collusion (intentional, accidental, or by hijacking
3 other userids to help).

> Solar Designer's OpenWall kernel patch addresses this issue by prohibiting 
....
> Also the LSM patch includes NSA Security enhanced Linux.  In the default SE 
> Linux policy a user in the default un-priviledged role (user_r) with the 
> defalt security domain for that role (user_t) will not be allowed to even 

Ahh, both are very nice, but alas, neither is part of the base kernel yet.

> A machine with an adequate security setup and with correctly configured quotas 
> should be able to operate correctly with hostile local users even if it only 
> has a single file system.

Granted.  However, my point was that I've never seen a Unixoid system that was
safe to run with only one big partition without extensive kernel hacking to
make it safe. I'll be more than happy to change my stance as soon as SELinux
ships as part of the base system ;)

/Valdis

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Reserved Blocks

[Russell Coker]

On Mon, 16 Sep 2002 17:35, Valdis.Kletnieks@vt.edu wrote:
> On Mon, 16 Sep 2002 17:13:52 +0200, Russell Coker said:
> > Alternately use quotas, speaking of which, do Quotas work on ReiserFS
> > without additional patches yet?  Quotas really are an essential feature.
>
> Quotas are a Good Thing - but note that quite often you get into the
> situation where you want to "overcommit" a partition (for instance, 2G of
> disk, and 10 users each of which have a 500M limit).  It helps, but you're
> still open to attacks by collusion (intentional, accidental, or by
> hijacking 3 other userids to help).

Group quotas should help that.  Put all the untrusted users in a group that 
has a quota which is small enough to not allow any damage.

> Ahh, both are very nice, but alas, neither is part of the base kernel yet.

LSM will be part of the base kernel soon, the code merge has already 
started...  Whether SE Linux and OpenWall get included is still up for 
discussion.  But that's OK, once LSM is in the base kernel it'll be trivial 
to add SE Linux (no more patch conflicts).

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: Reserved Blocks

[Sam Vilain]

Russell Coker <russell@coker.com.au> wrote:

> A machine with an adequate security setup and with correctly configured quotas 
> should be able to operate correctly with hostile local users even if it only 
> has a single file system.

Perhaps.  Quotas are non-trivial to set up, though.  As you pointed
out, they're not in the standard kernel (and patching the latest
kernel with the quotas patch isn't straightforward either).

Reserved blocks should be considered a degenerate quota system.  But I
still think you should be able to have both at the same time. Checking
for reserved blocks could be combined with the quotas code, although I
do like the mount option suggested for its inherit simplicity.

Whether or not a test system will perform well with hostile local
users is not particularly what I'm interested in.  I'm worrying about
the frequency that systems with friendly users stop running because
their filesystems have filled because the programs they wrote ran
away, in real world situations.  I've seen /var/tmp filled enough on
networks I have looked after that I consider this `vital' for my
ongoing use of ReiserFS. 
--
   Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/
    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

  I dont have any solution, but I certainly admire the problem.
ASHLEIGH BRILLIANT

Re: Reserved Blocks

[Russell Coker]

On Mon, 16 Sep 2002 18:23, Sam Vilain wrote:
> Russell Coker <russell@coker.com.au> wrote:
> > A machine with an adequate security setup and with correctly configured
> > quotas should be able to operate correctly with hostile local users even
> > if it only has a single file system.
>
> Perhaps.  Quotas are non-trivial to set up, though.  As you pointed
> out, they're not in the standard kernel (and patching the latest
> kernel with the quotas patch isn't straightforward either).

Quotas are not THAT difficult to setup.  Quotas for Ext2 have been in the 
kernel for ages (long before ReiserFS even had it's current name).

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: Reserved Blocks

[Sam Vilain]

On Mon, 16 Sep 2002 10:50:53 -0400
Valdis.Kletnieks@vt.edu wrote:

> > Is this essential UNIX filesystem feature implemented?
> "essential feature"?  Hardly.
> The *ORIGINAL* reason for the Berkeley FFS doing the "10% reserve"
> was to work around the fact that the original FFS code had severe fragmentation
> issues if the filesystem ran over 90% full (but at under 90%, there were
> enough free blocks usually available to not fragment TOO badly - the "knee"
> in the performance curve seems to have been around 92-94% full).  As such,
> this shouldn't be at all required for a modern filesystem, as it should be
> hardened against fragmentation to start with.
> A lot of sites have started abusing the "10% reserve" feature in order to
> protect themselves from people hosing things up by filling the disk - these
> are usually boxes that have "one big / filesystem" for everything.  I
> suggest that this is a *BAD* practice, and that the *proper* protection
> here is to have separate /, /usr, /var, /tmp and /home file systems.

I knew I'd get some fool saying this.  Let's stop this flamewar in the
making now.

Yes, I realise what you are saying.  The effect is worst on systems
where the root filesystem is writable by users and /etc and /tmp are
on the same filesystem (which is probably the default configuration of
many Linux distributions out there).  As a sysadmin, you've just taken
over one of those systems.  How, exactly, do you go about splitting
the filesystems up?

Pragmatic concerns aside, it is still necessary on systems with a
sound filesystem arrangement. /var/tmp is still on the same partition
for everyone. Programs need space in /var/tmp, /var/run, etc.
Therefore, without quotas or superuser reserved blocks, any user can
fill up /var/tmp, which may be needed to write pid files and all the
`per-system state information' we've come to expect in /var over the
years.

Besides, even a Balanced tree algorithm will not perform very well
allocating space when a partition is 99% full.  That should just stand
to reason.  If reiserfs lets me shrink the reserved blocks percentage
to 5% or 3% rather than 10%, fantastic.

If you've not administered enough systems to appreciate why you'd want
this, then fine.  But don't use statements like ``this shouldn't be at
all required for a modern filesystem, as it should be hardened against
fragmentation to start with'', because that statement is 95% truth and
5% bullshit.  And 0% consideration for real users.

Thank you for your input.
--
   Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/
    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

Real computer scientists love the concept of users.  Users are always
real impressed by the stuff computer scientists are talking about; it
sure sounds better than the stuff they are being forced to use now.

Re: Reserved Blocks

[Russell Coker]

On Mon, 16 Sep 2002 18:14, Sam Vilain wrote:
> Pragmatic concerns aside, it is still necessary on systems with a
> sound filesystem arrangement. /var/tmp is still on the same partition
> for everyone. Programs need space in /var/tmp, /var/run, etc.
> Therefore, without quotas or superuser reserved blocks, any user can
> fill up /var/tmp, which may be needed to write pid files and all the
> `per-system state information' we've come to expect in /var over the
> years.

I sometimes have /tmp be a symlink to /var/tmp/tmp, then /var/tmp may be a 
separate file system, or just part of the /var file system.

-- 
http://www.coker.com.au/bonnie++/     Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/       Postal SMTP/POP benchmark
http://www.coker.com.au/projects.html Projects I am working on
http://www.coker.com.au/~russell/     My home page

Re: Reserved Blocks

[Sam Vilain]

Valdis.Kletnieks@vt.edu wrote:

> > many Linux distributions out there).  As a sysadmin, you've just taken
> > over one of those systems.  How, exactly, do you go about splitting
> > the filesystems up?
> This of course depends on how much unallocated disk space you have handy.
> The method of attack will differ based on whether you have a free partition
> or not. You may be able to mount a "new" filesystem and copy it, or you may
> be forced to dump to auxillary storage and reload.

I know that, and you've just agreed that it's not straightforward, and
not possible if you don't have any unpartitioned space available to
you.

> > to reason.  If reiserfs lets me shrink the reserved blocks percentage
> > to 5% or 3% rather than 10%, fantastic.
> If in this day and age you're quibbling over the difference between
> 10% and 5%, I wonder if you've already tuned the 'NBPI' value to match
> [...]

I'm just pointing out that the limit is still there, it's just a
little smaller.  `Fantastic' was perhaps too strong a word.

> Maxstor just announced 320G drives.  If you need that last 32G
> *that* badly, it's time to buy another one - and then think about
> the backup issues. ;)                        ^^^^^^^^^^^^^^^^^^^^
  ^^^^^^^^^^^^^^^^^^
Heh.  Definitely.  Large disks are dangerous :-).  I prefer to buy
more smaller ones and mirror 'em.

> > If you've not administered enough systems to appreciate why you'd
> > want this, then fine.
> Hmm.. let's just say that I'd already been doing Unix for a while
> when I had Sun change a purchase order out from under me from
> Sun-2's to Sun-3's instead.

Oh, you got that too?  I was a bit pissed off when they insisted we
replaced our 490s with Sparc 5's and fucked up our nice tidy X.25
cabling.  I remember decommissioning the last shoebox QIC-150.  Oh,
and those nice Sun vacuum driven 9 tracks made loading reel to reel a
pleasure.

OK, I'll admit you may have a few years on me, but it's not the length
of time it's the love gone into the networks you've looked after, how
much you learned from them, and how much love you were able to return
to the networks.  Which is unquantifiable - so let's leave it at that.

> No, splitting up filesystems isn't perfect - especially if you don't
> have an underlying system like LVM so you can grow partitions as
> needed.  However, I still say that it's the best of a number of
> non-perfect solutions.

Well, I say implement all that don't introduce undue complexity and
see which one yields the best results after 5 to 10 years.  The
skilled system administrator will appreciate them all.

So there :-P
--
   Sam Vilain, sam@vilain.net     WWW: http://sam.vilain.net/
    7D74 2A09 B2D3 C30F F78E      GPG: http://sam.vilain.net/sam.asc
    278A A425 30A9 05B5 2F13

Real Programmers don`t write in RPG.  RPG is for gum-chewing dimwits
who maintain ancient payroll programs.