allow non root users to set io priority "idle" ?

allow non root users to set io priority "idle" ?

[dragoran]

Its possible to set the io priority using tools like ionice and syscalls.
But this works only as root.
Why can't a non root user change the priority of his processes to idle?
I understand that realtime priority requires root but why idle?
For instance the beagle trys to set its priority to idle but fails 
because it does not run as root.
Any reason for this that I have missed? I can't think of a case where 
setting the io priority to idle would have a negative impact for other 
users or the whole system.

P.S:
Please CC me when replying.

Re: allow non root users to set io priority "idle" ?

[Andi Kleen]

dragoran <drago01@gmail.com> writes:

> Its possible to set the io priority using tools like ionice and syscalls.
> But this works only as root.
> Why can't a non root user change the priority of his processes to idle?
> I understand that realtime priority requires root but why idle?
> For instance the beagle trys to set its priority to idle but fails
> because it does not run as root.
> Any reason for this that I have missed? I can't think of a case where
> setting the io priority to idle would have a negative impact for other
> users or the whole system.

Very low priority can starve others when it holds some kernel resource
needed by another task.  

Consider three tasks: one very low priority, one high priority: Low
priority task holds some kernel resource, middle task eats as much CPU
as it gets; high priority task wants to get the resource. High
priority will need to wait for low running, which could take a long
time. With true SCHED_IDLE (i believe the current implementation is
not true) this could be never or at least a very long time.

There are ways to defend against this problem (known as priority inheritance),
but the kernel doesn't do them consistently. The same issue could also
happen for user space managed resources.

For IO I suppose the same could happen too. e.g. low priority
task wants to write out a page and keeps it locked until the IO 
is finished. High priority task wants to access the page and has
to wait until it is unlocked. Middle task generates an endless
stream of IO that makes the idle priority writeout never finish.

In general idle priorities are quite risky, even for root.

-Andi

Re: allow non root users to set io priority "idle" ?

[dragoran]

Andi Kleen wrote:
> dragoran <drago01@gmail.com> writes:
>
>   
>> Its possible to set the io priority using tools like ionice and syscalls.
>> But this works only as root.
>> Why can't a non root user change the priority of his processes to idle?
>> I understand that realtime priority requires root but why idle?
>> For instance the beagle trys to set its priority to idle but fails
>> because it does not run as root.
>> Any reason for this that I have missed? I can't think of a case where
>> setting the io priority to idle would have a negative impact for other
>> users or the whole system.
>>     
>
> Very low priority can starve others when it holds some kernel resource
> needed by another task.  
>
> Consider three tasks: one very low priority, one high priority: Low
> priority task holds some kernel resource, middle task eats as much CPU
> as it gets; high priority task wants to get the resource. High
> priority will need to wait for low running, which could take a long
> time. With true SCHED_IDLE (i believe the current implementation is
> not true) this could be never or at least a very long time.
>
> There are ways to defend against this problem (known as priority inheritance),
> but the kernel doesn't do them consistently. The same issue could also
> happen for user space managed resources.
>
> For IO I suppose the same could happen too. e.g. low priority
> task wants to write out a page and keeps it locked until the IO 
> is finished. High priority task wants to access the page and has
> to wait until it is unlocked. Middle task generates an endless
> stream of IO that makes the idle priority writeout never finish.
couldn't this be fixed by bumping idle tasks to middle while they hold a 
pagelock?
ex:
task A - hp
task B - mp
task C - lp (idle)
task C locks a page -> becomes a mp (middle prio) task until its finished
task C unlocks the page -> back to idle

> In general idle priorities are quite risky, even for root.
>
> -Andi
>
>   

Re: allow non root users to set io priority "idle" ?

[Andi Kleen]

> couldn't this be fixed by bumping idle tasks to middle while they hold a 

Usually to high.

But it's all complicated and hasn't been done consistently
(there are real time mutexes in the -rt kernel for example, 
but there are lots of other locks and they have higher overhead too) 
and it's unclear we really want to do all this complexity anyways.

Also as I said the problem could then still happen in user space
which then would all need to be fixed to handle PI too.

In some cases the relationship is also not as simple as a single 
lock. And for IO handling it would be likely quite hard.

I personally always found idle priorities quite dubious because
even if they worked reliable for the CPU they will clear your cache/
load your memory controller and impact all other programs because
of this. And for the disk they will cause additional seeks which are 
also very costly.

-Andi

Re: allow non root users to set io priority "idle" ?

[dragoran]

Andi Kleen wrote:
>> couldn't this be fixed by bumping idle tasks to middle while they hold a 
>>     
>
> Usually to high.
>
> But it's all complicated and hasn't been done consistently
> (there are real time mutexes in the -rt kernel for example, 
> but there are lots of other locks and they have higher overhead too) 
> and it's unclear we really want to do all this complexity anyways.
>
> Also as I said the problem could then still happen in user space
> which then would all need to be fixed to handle PI too.
>
> In some cases the relationship is also not as simple as a single 
> lock. And for IO handling it would be likely quite hard.
>
> I personally always found idle priorities quite dubious because
> even if they worked reliable for the CPU they will clear your cache/
> load your memory controller and impact all other programs because
> of this. And for the disk they will cause additional seeks which are 
> also very costly.
>
>   
ok, thx so that means that the best that can be done for now is to run 
beagle as best effort with prio 7 (like its done now).

Re: allow non root users to set io priority "idle" ?

[Jens Axboe]

On Mon, Aug 06 2007, Andi Kleen wrote:
> > couldn't this be fixed by bumping idle tasks to middle while they hold a 
> 
> Usually to high.
> 
> But it's all complicated and hasn't been done consistently
> (there are real time mutexes in the -rt kernel for example, 
> but there are lots of other locks and they have higher overhead too) 
> and it's unclear we really want to do all this complexity anyways.
> 
> Also as I said the problem could then still happen in user space
> which then would all need to be fixed to handle PI too.
> 
> In some cases the relationship is also not as simple as a single 
> lock. And for IO handling it would be likely quite hard.
> 
> I personally always found idle priorities quite dubious because
> even if they worked reliable for the CPU they will clear your cache/
> load your memory controller and impact all other programs because
> of this. And for the disk they will cause additional seeks which are 
> also very costly.

But that is why the idle priority implementation in CFQ adds a grace
period before idle prio tasks are run. So that concern should not be an
issue, if so the grace period needs to be enlarged. That at least covers
the seek side of things. If idle io tasks run, then the IO load on the
system must be very low to zero. Hence other IO relevant resource
contention isn't an iissue.

-- 
Jens Axboe

Re: allow non root users to set io priority "idle" ?

[dragoran]

so there is no real reason not to allow it for non root users?
removing the check is easy (3 lines) ....
or are there any other issues/problems?

On 8/7/07, Jens Axboe <jens.axboe@oracle.com> wrote:
> On Mon, Aug 06 2007, Andi Kleen wrote:
> > > couldn't this be fixed by bumping idle tasks to middle while they hold a
> >
> > Usually to high.
> >
> > But it's all complicated and hasn't been done consistently
> > (there are real time mutexes in the -rt kernel for example,
> > but there are lots of other locks and they have higher overhead too)
> > and it's unclear we really want to do all this complexity anyways.
> >
> > Also as I said the problem could then still happen in user space
> > which then would all need to be fixed to handle PI too.
> >
> > In some cases the relationship is also not as simple as a single
> > lock. And for IO handling it would be likely quite hard.
> >
> > I personally always found idle priorities quite dubious because
> > even if they worked reliable for the CPU they will clear your cache/
> > load your memory controller and impact all other programs because
> > of this. And for the disk they will cause additional seeks which are
> > also very costly.
>
> But that is why the idle priority implementation in CFQ adds a grace
> period before idle prio tasks are run. So that concern should not be an
> issue, if so the grace period needs to be enlarged. That at least covers
> the seek side of things. If idle io tasks run, then the IO load on the
> system must be very low to zero. Hence other IO relevant resource
> contention isn't an iissue.
>
> --
> Jens Axboe
>
>

Re: allow non root users to set io priority "idle" ?

[dragoran]

so there is no real reason not to allow it for non root users?
removing the check is easy (3 lines) ....
or are there any other issues/problems?

(resend sorry for the top post stupid gmail ;) )

Re: allow non root users to set io priority "idle" ?

[Andi Kleen]

On Wed, Aug 08, 2007 at 12:18:12AM +0200, dragoran wrote:
> so there is no real reason not to allow it for non root users?
> removing the check is easy (3 lines) ....
> or are there any other issues/problems?

Jens was only talking about the seek issue I mentioned I believe;
not about the PI problem.

-Andi

Re: allow non root users to set io priority "idle" ?

[Jens Axboe]

On Wed, Aug 08 2007, dragoran wrote:
> so there is no real reason not to allow it for non root users?
> removing the check is easy (3 lines) ....
> or are there any other issues/problems?

Andi already explained to you why it can't be enabled for non-root
users. I merely talked about why idle priority works.

-- 
Jens Axboe

Re: allow non root users to set io priority "idle" ?

[dragoran]

Jens Axboe wrote:
> On Wed, Aug 08 2007, dragoran wrote:
>   
>> so there is no real reason not to allow it for non root users?
>> removing the check is easy (3 lines) ....
>> or are there any other issues/problems?
>>     
>
> Andi already explained to you why it can't be enabled for non-root
> users. I merely talked about why idle priority works.
>
>   
ok, I missunderstood you.
shouldn't read mails at 0:18 am ;)

Re: allow non root users to set io priority "idle" ?

[Lee Revell]

On 06 Aug 2007 13:11:01 +0200, Andi Kleen <andi@firstfloor.org> wrote:
> For IO I suppose the same could happen too. e.g. low priority
> task wants to write out a page and keeps it locked until the IO
> is finished. High priority task wants to access the page and has
> to wait until it is unlocked. Middle task generates an endless
> stream of IO that makes the idle priority writeout never finish.

I don't think it's a problem for high priority (RT) tasks - it's well
known in the real time Linux community that you never, ever do IO from
a thread that has to satisfy RT constraints.  A correct RT linux app
does its IO from a SCHED_NORMAL thread, with *plenty* of buffering,
and feeds the RT constrained SCHED_FIFO threads using a lock free
ringbuffer.

SCHED_IDLE starving SCHED_NORMAL is an issue of course.  But
SCHED_IDLE seems a lot more useful for read than write which I would
expect to take fewer locks.  For example I'd expect Beagle to want to
read at SCHED_IDLE but write out its indices at SCHED_NORMAL.

Would it make any sense to allow anyone to set SCHED_IDLE for reads
but require root to change IO priority for writes?

Lee

Re: allow non root users to set io priority "idle" ?

[Andi Kleen]

> I don't think it's a problem for high priority (RT) tasks - it's well
> known in the real time Linux community that you never, ever do IO from
> a thread that has to satisfy RT constraints.  

The point here is to protect the system from a potentially malicious user

> which I would
> expect to take fewer locks.  

Even a single lock is one too much. My example lock -- the page lock bit --
certainly applies to both reads and writes.

-Andi

Re: allow non root users to set io priority "idle" ?

[Sheplyakov Alexei]

[-- Attachment #1: Type: text/plain, Size: 1072 bytes --]

On Mon, Aug 06, 2007 at 01:11:01PM +0200, Andi Kleen wrote:

> Very low priority can starve others when it holds some kernel resource
> needed by another task.  

Nevertheless ordinary users are permitted to lower priority ([re]nice)
and resource limits (setrlimit).

> For IO I suppose the same could happen too. e.g. low priority
> task wants to write out a page and keeps it locked until the IO 
> is finished. High priority task wants to access the page and has
> to wait until it is unlocked. Middle task generates an endless
> stream of IO that makes the idle priority writeout never finish.

I don't quite understand. There are a lot of other ways to starve such
high-priority process: 
1. renice the low-priority process
2. send it a signal
3. ptrace it
4. use something like cpulimit (http://cpulimit.sourceforge.net/) to stall it
....

So I think current behaviour of ionice is inconsistent (and pointless).

P.S. 
Please CC me, I'm not subscribed.

Best regards,
 Alexei

-- 
All science is either physics or stamp collecting.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: allow non root users to set io priority "idle" ?

[Andi Kleen]

On Wed, Aug 08, 2007 at 01:51:32PM +0400, Sheplyakov Alexei wrote:
> On Mon, Aug 06, 2007 at 01:11:01PM +0200, Andi Kleen wrote:
> 
> > Very low priority can starve others when it holds some kernel resource
> > needed by another task.  
> 
> Nevertheless ordinary users are permitted to lower priority ([re]nice)

Low priority as allowed by renice is ok -- the resource holder will 
run eventually and the system make progress.

> I don't quite understand. There are a lot of other ways to starve such
> high-priority process: 
> 1. renice the low-priority process

That requires user action.

e.g. if you google a bit you can find a famous story where the NASA
had to do that remotely to solve a PI problem on one of the Mars rovers. But I'm
sure they weren't very pleased about it. And not everybody has a JPL
control room to patch things up in the backhand.

> 2. send it a signal
> 3. ptrace it

Only root can do these.

-Andi

Re: allow non root users to set io priority "idle" ?

[Sheplyakov Alexei]

[-- Attachment #1: Type: text/plain, Size: 1434 bytes --]

On Wed, Aug 08, 2007 at 12:20:56PM +0200, Andi Kleen wrote:

> > I don't quite understand. There are a lot of other ways to starve such
> > high-priority process: 
> > 1. renice the low-priority process
> 
> That requires user action.

As well as lowering IO priority. 

> > 2. send it a signal
> > 3. ptrace it
> 
> Only root can do these.

The original question (unless I'm missing something) was "why _the owner_
of a process is not enabled to lower its IO priority?" The owner of the
process certainly can send signal and ptrace it.

> Low priority as allowed by renice is ok -- the resource holder will 
> run eventually and the system make progress.

System (whatever that means) should not depend on random users' processes.

> e.g. if you google a bit you can find a famous story where the NASA
> had to do that remotely to solve a PI problem on one of the Mars rovers.

Certainly there are a scenarios when lowering priority (CPU or IO) have 
negative effect. "If it hurts -- then don't do it". However, it is not
kernels job to decide if the user is right. If the user asks to do
something "stupid" (run his process with idle IO priority, whipe out his
home directory, you name it) -- just do that (as long as the action is 
permitted by the security policy).

P.S. 
Please CC me, I'm not subscribed.

Best regards,
 Alexei

-- 
All science is either physics or stamp collecting.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: allow non root users to set io priority "idle" ?

[Andi Kleen]

> The original question (unless I'm missing something) was "why _the owner_
> of a process is not enabled to lower its IO priority?" The owner of the
> process certainly can send signal and ptrace it.

Ah I misparsed you.

Sending signals and ptraces is atomic regarding system operations. This means
the kernel will finish doing whatever it is doing (or bail out releasing
all resources) before processing them. So stopping a process this
way can never starve its IO.  Ok there are special cases like user space
FUSE or NFS servers that can block operations, but these are expected to run 
as root.

-Andi

Re: allow non root users to set io priority "idle" ?

[Jens Axboe]

On Wed, Aug 08 2007, Sheplyakov Alexei wrote:
> So I think current behaviour of ionice is inconsistent (and pointless).

It is not, as has been explained more than once in this thread, we
cannot allow ordinary users to eg /etc/passwd infinitely. See?

It'd be nice if we could solve this of course, until then true idle io
must be a cautious root-only thing. So it may be inconsistent and a bit
confusing, but it's definitely not pointless.

-- 
Jens Axboe

Re: allow non root users to set io priority "idle" ?

[Bodo Eggert]

Andi Kleen <andi@firstfloor.org> wrote:

>> couldn't this be fixed by bumping idle tasks to middle while they hold a
> 
> Usually to high.

Then use the lowest non-idle priority. The result will not be more b0rken
than nice -n 19.

> But it's all complicated and hasn't been done consistently
> (there are real time mutexes in the -rt kernel for example,
> but there are lots of other locks and they have higher overhead too)
> and it's unclear we really want to do all this complexity anyways.

If you have an rt application, it will block normal tasks like a normal
task will block idle tasks. You need to handle that situation anyway.

> Also as I said the problem could then still happen in user space
> which then would all need to be fixed to handle PI too.

Don't do that then. If I ask an idle priority task, I should not expect an
answer unless the system is idle. And besides that, I should not depend on
an answer at all, since the task might be stuck while reading a NFS file
from that smoking and burning server.

> In some cases the relationship is also not as simple as a single
> lock. And for IO handling it would be likely quite hard.

As long as IO works correctly while having realtime tasks, there is no
problem, and if it doesn't work, it needs to be fixed anyway.

> I personally always found idle priorities quite dubious because
> even if they worked reliable for the CPU they will clear your cache/
> load your memory controller and impact all other programs because
> of this. And for the disk they will cause additional seeks which are
> also very costly.

A very-low-normal-priority tasks would cause all these effects anyway, but
it would do so more frequently.
-- 
Top 100 things you don't want the sysadmin to say:
74. I remember the last time I saw it do that...

Friß, Spammer: jf@7eggert.dyndns.org rqfpzq@XhuRCfa.7eggert.dyndns.org