super root shell/mode/api

super root shell/mode/api

[Andrea]

Hello!

I'm a C/C++ Open Source Software Developer / SysAdmin.

Excuse my English, it's not my first language.

First thank You for Linux, I'm using it since 1998!

Some days ago I've lost some important data on a Linux
machine, because of an out of memory and swap situation.

When I was coding on c-64 and it crashes I pressed my
Cartridge button and could save what I wanted, execute
code and so on. Some may remember this old school times :)

Now 20 years has passed and a web page can block my whole
machine?!?!?!?

1989 c-64 higher data safety then 2009 Linux?!?!?

I've read the OOM discussion and I know that it's not easy 
to find a perfect solution, but don't implementing any 
solution is the worst solution at all.

I know there is an OOM handling, but the only thing that
happened was the hard disk light flashing for more or less 30 minutes
and I was forced to press the reset button and my data was lost :(

I think it would be simply awesome to have a Linux Kernel mode
similar to the c-64 cartridge concept.

Maybe call it in honor to the c-64 'cartridge freeze mode' or so :)

You hit a button combination and you enter in a Linux Kernel ncurses menu
and/or shell and/or GUI, where you can for example:

save processes, memory e.g. from 0x00000 - 0xFFFFF
protected ssh/telnet access
examine memory dumps
hex editor
assembler/disassembler
statistics
password protected area
anti root-kit/virus tool (the 'cartridge freeze mode' can be loaded from protected memory or even protected media for this case)
anti virus modules
swap out processes that take too much memory
all thinks you can do with SysRq Keys
terminate processes
search for graphics
screen-shots
backup
hibernation
core dumps
top like screen
virtual machines 
grep text in ascii, unicode
continue everything: freeze and defreeze
the shell could be accessible always without freezing everything, like a super root shell/mode/api
you can load user executables/modules for this mode too

That would rock!

Please consider doing something similar!

Thanks for reading!

Regards

Andrea Gedda



      

Re: super root shell/mode/api

[Andi Kleen]

Andrea <andrea256it@yahoo.it> writes:
>
> I know there is an OOM handling, but the only thing that
> happened was the hard disk light flashing for more or less 30 minutes
> and I was forced to press the reset button and my data was lost :(

One trick I found very useful to make OOM much less painful is to
make the swap partition small (not zero though).

The traditional suggestion of 2x RAM is far too large on modern
systems and since Linux is rather inefficient at swapping having
so much swap space just prolongs the death struggle.

With a smaller swap partition (not more than 100-200MB) the OOM killer kicks in 
relatively quickly when something goes wrong and kills the offending process.

> I think it would be simply awesome to have a Linux Kernel mode
> similar to the c-64 cartridge concept.
>
> Maybe call it in honor to the c-64 'cartridge freeze mode' or so :)
>
> You hit a button combination and you enter in a Linux Kernel ncurses menu
> and/or shell and/or GUI, where you can for example:

You could do all that today by using a suitable kexec/kdump kernel setup with
sysrq-C. 

The kdump kernel can do all that based on the image of the previous kernel.
Some of it is very easy (e.g. for disassemble just run "crash"), other
parts don't make sense (e.g. swap out processes -- the parent kernel
already did that)

Traditional distributions just dump the image, but there's no
principle reason it couldn't do more.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: super root shell/mode/api

[Alan Cox]

> Some days ago I've lost some important data on a Linux
> machine, because of an out of memory and swap situation.
> 
> When I was coding on c-64 and it crashes I pressed my
> Cartridge button and could save what I wanted, execute
> code and so on. Some may remember this old school times :)
> 
> Now 20 years has passed and a web page can block my whole
> machine?!?!?!?

Your distribution failed to configure your system sensibly in that case.
Linux has supported a strict overcommit mode for some years, and in that
mode a process isn't permitted to drive the system so far out of memory
it locks up or hangs.

You can also set some limits on a given process (which with certain
versions of firefox - especially old ones or if you have the flash plugin
is a good idea).

First call  - ask your distribution why it isn't using strict overcommit.

Re: super root shell/mode/api

[Henrique de Moraes Holschuh]

On Mon, 18 May 2009, Alan Cox wrote:
> Your distribution failed to configure your system sensibly in that case.
> Linux has supported a strict overcommit mode for some years, and in that
> mode a process isn't permitted to drive the system so far out of memory
> it locks up or hangs.

Err... strict overcommit is vm.overcommit_memory=2, right?  That means no
overcommit at all (as far as the documentation goes, anyway).

Which distros enable that by default?

> You can also set some limits on a given process (which with certain
> versions of firefox - especially old ones or if you have the flash plugin
> is a good idea).

Indeed.  Or just limit the users, so that root always has room to work.

-- 
  "One disk to rule them all, One disk to find them. One disk to bring
  them all and in the darkness grind them. In the Land of Redmond
  where the shadows lie." -- The Silicon Valley Tarot
  Henrique Holschuh

Re: super root shell/mode/api

[Willy Tarreau]

On Mon, May 18, 2009 at 01:12:31PM -0300, Henrique de Moraes Holschuh wrote:
> On Mon, 18 May 2009, Alan Cox wrote:
> > Your distribution failed to configure your system sensibly in that case.
> > Linux has supported a strict overcommit mode for some years, and in that
> > mode a process isn't permitted to drive the system so far out of memory
> > it locks up or hangs.
> 
> Err... strict overcommit is vm.overcommit_memory=2, right?  That means no
> overcommit at all (as far as the documentation goes, anyway).

exactly

> Which distros enable that by default?

I don't know if mainline distros do this, but some distros dedicated to
embedded systems have been using that for ages, almost since Alan published
his first overcommit patch a long time ago. It's the only way to reach
very long uptimes on servers, as it also protects you against your own
mistakes (eg: stupid actions such as "vi access.log" when the file is
larger than memory).

Willy

Re: super root shell/mode/api

[Sitsofe Wheeler]

On Wed, May 20, 2009 at 06:36:04AM +0200, Willy Tarreau wrote:
> 
> I don't know if mainline distros do this, but some distros dedicated to
> embedded systems have been using that for ages, almost since Alan published
> his first overcommit patch a long time ago. It's the only way to reach
> very long uptimes on servers, as it also protects you against your own
> mistakes (eg: stupid actions such as "vi access.log" when the file is
> larger than memory).

Shouldn't the vi case be taken care of by good ulimits? Certainly that's
what I've seen openSUSE do by default (although strangely SLES 10
doesn't)...

The last I heard about overcommit was that there were always some
legitimate programs that could run that would be stopped by having it on
(evolution always seemed to be mentioned for some reason). As such it
would be surprising to see any of the desktop distros enabling it by
default.

I've been wondering for a while with my EeePC if strict overcommit is a
no brainer if you are running without swap... In such systems the only
things that can be forced out of RAM are mmap'd files (?) so does
overcommit even happen?

-- 
Sitsofe | http://sucs.org/~sits/

Re: super root shell/mode/api

[Andrea]

Thanks.

I'll take a look.

In order to be able to dump/terminate viruses, attacks and OOM processes an
admin should have the possibility to *quickly* SIGSTP *all* processes except
vga consoles and to switch from X11 to vga, maybe with SysRq.

Is this possible?

Andrea Gedda


      

Re: super root shell/mode/api

[Bodo Eggert]

Andrea <andrea256it@yahoo.it> wrote:

> In order to be able to dump/terminate viruses, attacks and OOM processes an
> admin should have the possibility to *quickly* SIGSTP *all* processes except
> vga consoles and to switch from X11 to vga, maybe with SysRq.
> 
> Is this possible?

If there is a malware with root privileges, this would be of no use. You are
0wned.

If there is a malware with user privileges, stopping these processes will
be enough.

So why bother?

Re: super root shell/mode/api

[Andrea]

> If there is a malware with root privileges, this would be of no use. You are
> 0wned.
> 
> If there is a malware with user privileges, stopping these processes will
> be enough.
> 
> So why bother?

That's exactly the problem a remote attacker or virus
can gain root and you are completely powerless. You want
to save data? The attacker just logs you out before you
can run any command. You can't even backup or save
data! You are owned. Yes.

With this super shell/mode/menu in less then one second, you stop
everything - a global SIGSTP - and gain control over your machine!

You can save all memory, e.g. for controlling what happened 
or data recovery, sigstop without hurry all processes that seems
a problem and so on.

Then when you have saved everything, made a backup of the hdd,
stopped the processes with viruses or too much memory you can
unfreeze the system - a global SIGCONT and it was like stopping
the time. Attackers, viruses, OOM processes are stopped in less
then one second. You can go back to the system whenever you
want. You the console user finally have power. 

The value of such a tool is only evident once you had it.

I can give you a lot of other reasons, but I must work now,
maybe in some days.

Regards.

Andrea Gedda



      

Re: super root shell/mode/api

[Alan Cox]

> With this super shell/mode/menu in less then one second, you stop
> everything - a global SIGSTP - and gain control over your machine!

Too late, the attacker has already patched your "super shell" out and
disabled it. I don't think you can outreact a 3GHz processor.

Re: super root shell/mode/api

[Bodo Eggert]

On Tue, 19 May 2009, Andrea wrote:

>> If there is a malware with root privileges, this would be of no use. You are
>> 0wned.
>>
>> If there is a malware with user privileges, stopping these processes will
>> be enough.
>>
>> So why bother?
>
> That's exactly the problem a remote attacker or virus
> can gain root and you are completely powerless. You want
> to save data? The attacker just logs you out before you
> can run any command. You can't even backup or save
> data! You are owned. Yes.
>
> With this super shell/mode/menu in less then one second, you stop
> everything - a global SIGSTP - and gain control over your machine!

The problem is: You can only do the first step. The second step is 
prevented by the attacker replacing your super-root shell and the linux 
kernel with his specially crafted versions.

That's why you need a hypervisor or a virtual machine to do the job.

> You can save all memory, e.g. for controlling what happened
> or data recovery, sigstop without hurry all processes that seems
> a problem and so on.

You can't, since the attacker modified the "save memory" function to
exclude the malware and all your personal documents - or simply to
not work at all.

Re: super root shell/mode/api

[Joao Correia]

On Tue, May 19, 2009 at 3:36 PM, Andrea <andrea256it@yahoo.it> wrote:
> That's exactly the problem a remote attacker or virus
> can gain root and you are completely powerless. You want
> to save data? The attacker just logs you out before you
> can run any command. You can't even backup or save
> data! You are owned. Yes.


This isn't Hollywood. Pull the cable.

Best regards,
Joao Correia