One Interrupted Threads per Interrupt line?

One Interrupted Threads per Interrupt line?

[Monica Puig-Pey]

Hi,

I'm using Ubuntu 10.04 with the 2.6.31-11-rt patch installed. I'm 
developing a PCI device driver using hardware interrupts.
I read in the “Internals of the RT Patch” document 
(http://www.kernel.org/doc/ols/2007/ols2007v2-pages-161-172.pdf ) that 
when a device driver requests an IRQ, a thread is created to service 
this interrupt line (Threaded interrupts). Only ONE thread can be 
created per interrupt line, so shared interrupts are still handled by a 
single thread.

My driver requests IRQ 16, which is shared with others handlers. Typing 
on the shell  $ps -eo pid,pri,rtprio,cmd I can see my driver requesting 
irq 16 and with 50 RTPRIO. At the same time I can see the others drivers 
requesting IRQ 16 with different real time priorities than mine( 80, 85, 
50). Does this mean that there are more than ONE Threaded Interrupts for 
IRQ 16? or it's just the thread priority when running each handler for 
IRQ 16?
Finally, how could I change the 50 RT PRIO for my handler pci_9111?

Here you have the shell:

   PID PRI RTPRIO CMD

   658 120     80 [irq/16-uhci_hcd]
  3852 125     85 [irq/16-HDA Inte]
  4303  90     50 [irq/16-radeon@p]
  5863  90     50 [irq/16-pci_9111]
  5865  19      - ps -eo pid,pri,rtprio,cmd


Thank you very much,

Mónica
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

I/O operations priority in RTOS

[Monica Puig-Pey]

Hello,
I'm studying how to develop drivers in a real time OS and how do they 
work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
I would like to know the priority when executing open(), read(), write() 
and close() operations.
In my example the thread which is using the driver runs with 10 RTPRIO, 
but I don't know what happens in kernel context with the priority when 
running the I/O operations.
Thank you for your help, I don't know where to learn about this.

Mónica
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

kernel threads in drivers using the RT patch

[Monica Puig-Pey]

Hello,

I'm studying how to develop drivers in a real time OS and how do they 
work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
I'm trying to run a kernel thread from my driver (kernel context). I 
create the kthread in the module_init, I change its real time priority 
and then make it run .
Code shown below:

thread_handler=*(struct task_struct *)kthread_create(my_tasklet_handler, 
(void*)&datos,"thread_handler");
   if (IS_ERR(&thread_handler)){
      return PTR_ERR(&thread_handler);
      printk(KERN_WARNING "<pci> (init_module)  Error creating Thread 
handler\n");
   }
params->sched_priority=20;
result = sched_setscheduler(&thread_handler, SCHED_FIFO, params);
result=wake_up_process(&thread_handler);

Params is global struct sched_param * and struct thread_handler  a 
global task_struct.

I stop the Kthread in the module_exit using 
res=kthread_stop(&thread_handler);

The kthread "thread_handler" has a while (1) loop and it is stopped in a 
semaphore, waiting for the interrupt handler. When it wakes up each time 
it calls kthread_should_stop() and depending on that break out of the 
loop or run.

My problem is, when trying to establish the rtprio kthread, the whole 
computer get blocked and I only can restart it. If I delete this part, 
then kthread runs, and works well. With the ps command I can see it has 
a 24 NON real time priority. Then the problem come when uninstalling my 
module with rmmod. In the  kthread_stop function, everything gets blocked.

Could someone tell what am I doing wrong? Or just give me some help 
about kernel threads in the RT Patch?

Thanks a lot

Mónica
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: I/O operations priority in RTOS

[Nicholas Mc Guire]

On Sat, 04 Jun 2011, Monica Puig-Pey wrote:

> Hello,
> I'm studying how to develop drivers in a real time OS and how do they  
> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
> I would like to know the priority when executing open(), read(), write()  
> and close() operations.
> In my example the thread which is using the driver runs with 10 RTPRIO,  
> but I don't know what happens in kernel context with the priority when  
> running the I/O operations.
> Thank you for your help, I don't know where to learn about this.
>

Priorities are not associated with functions but rather with processes/threads
so the priority with which the functions open/read/write are executed depend
on the process that invokes these functions.

That said there is an indirect impact of the function on priority as soon as
the funciton needs to share any resources that need protection by locking. The
moment that happens the effective execution priority of the function no longer only depends on the task that calls open but may indirectly depedn on a totally
unrelated task that happens to hold a lock protecting some shared resource.

So the consequence for rt-drivers is that they must ensure that they minimize
critical sections, or prevent them all together (i.e. by using lock free 
methods). Notably this puts restrictions on the use of dynamic resources at 
runtime, that is after the driver was initialized - basically you want all 
resources that the rt-driver needs allocated at driver initialization time. 
Also when using bottom half mechanisms you need to take into account the 
priority of the kernel thread that manages the defered work items, so 
rt-drivers may have a different structure than normal drivers.

hofrat

Re: I/O operations priority in RTOS

[Armin Steinhoff]

Nicholas Mc Guire wrote:
> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>
>> Hello,
>> I'm studying how to develop drivers in a real time OS and how do they
>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>> I would like to know the priority when executing open(), read(), write()
>> and close() operations.
>> In my example the thread which is using the driver runs with 10 RTPRIO,
>> but I don't know what happens in kernel context with the priority when
>> running the I/O operations.
>> Thank you for your help, I don't know where to learn about this.
>>
> []
> Also when using bottom half mechanisms you need to take into account the
> priority of the kernel thread that manages the defered work items, so
> rt-drivers may have a different structure than normal drivers.

That's the reason why I prefer UIO based user space drivers !

--Armin

> hofrat
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: I/O operations priority in RTOS

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1320 bytes --]

On 2011-06-05 10:46, Armin Steinhoff wrote:
> Nicholas Mc Guire wrote:
>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>
>>> Hello,
>>> I'm studying how to develop drivers in a real time OS and how do they
>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>> I would like to know the priority when executing open(), read(), write()
>>> and close() operations.
>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>> but I don't know what happens in kernel context with the priority when
>>> running the I/O operations.
>>> Thank you for your help, I don't know where to learn about this.
>>>
>> []
>> Also when using bottom half mechanisms you need to take into account the
>> priority of the kernel thread that manages the defered work items, so
>> rt-drivers may have a different structure than normal drivers.
> 
> That's the reason why I prefer UIO based user space drivers !

...which are forced to use threaded IRQs over -rt as well.

Taking threaded IRQs out of the picture, bottom-half mechanisms usually
come into play when dispatch workload that cannot be assigned to a
specific user context is significant. But then UIO is the wrong approach
anyway as it doesn't match well the requirements of multi-user device
interfaces.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: I/O operations priority in RTOS

[Nicholas Mc Guire]

On Sun, 05 Jun 2011, Armin Steinhoff wrote:

> Nicholas Mc Guire wrote:
>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>
>>> Hello,
>>> I'm studying how to develop drivers in a real time OS and how do they
>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>> I would like to know the priority when executing open(), read(), write()
>>> and close() operations.
>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>> but I don't know what happens in kernel context with the priority when
>>> running the I/O operations.
>>> Thank you for your help, I don't know where to learn about this.
>>>
>> []
>> Also when using bottom half mechanisms you need to take into account the
>> priority of the kernel thread that manages the defered work items, so
>> rt-drivers may have a different structure than normal drivers.
>
> That's the reason why I prefer UIO based user space drivers !
>
...and how to resolve DMA ? if DMA were resolved cleanly I would agree.

hofrat

Re: I/O operations priority in RTOS

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

On 2011-06-05 11:28, Nicholas Mc Guire wrote:
> On Sun, 05 Jun 2011, Armin Steinhoff wrote:
> 
>> Nicholas Mc Guire wrote:
>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>
>>>> Hello,
>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>> I would like to know the priority when executing open(), read(), write()
>>>> and close() operations.
>>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>>> but I don't know what happens in kernel context with the priority when
>>>> running the I/O operations.
>>>> Thank you for your help, I don't know where to learn about this.
>>>>
>>> []
>>> Also when using bottom half mechanisms you need to take into account the
>>> priority of the kernel thread that manages the defered work items, so
>>> rt-drivers may have a different structure than normal drivers.
>>
>> That's the reason why I prefer UIO based user space drivers !
>>
> ...and how to resolve DMA ? if DMA were resolved cleanly I would agree.

Regarding that limitation, there is some hope: "next-generation" UIO is
called VFIO. Useful for exclusively assigning virtual PCI functions of
network adapters etc. to user space stacks or hypervisors like QEMU/KVM
(for device pass-through). But it's not mainline yet. And it obviously
requires an IOMMU.

But the key point remains: "exclusively". Anything else cannot be
modeled efficiently via UIO or VFIO.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: I/O operations priority in RTOS

[Nicholas Mc Guire]

On Sun, 05 Jun 2011, Jan Kiszka wrote:

> On 2011-06-05 11:28, Nicholas Mc Guire wrote:
> > On Sun, 05 Jun 2011, Armin Steinhoff wrote:
> > 
> >> Nicholas Mc Guire wrote:
> >>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
> >>>
> >>>> Hello,
> >>>> I'm studying how to develop drivers in a real time OS and how do they
> >>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
> >>>> I would like to know the priority when executing open(), read(), write()
> >>>> and close() operations.
> >>>> In my example the thread which is using the driver runs with 10 RTPRIO,
> >>>> but I don't know what happens in kernel context with the priority when
> >>>> running the I/O operations.
> >>>> Thank you for your help, I don't know where to learn about this.
> >>>>
> >>> []
> >>> Also when using bottom half mechanisms you need to take into account the
> >>> priority of the kernel thread that manages the defered work items, so
> >>> rt-drivers may have a different structure than normal drivers.
> >>
> >> That's the reason why I prefer UIO based user space drivers !
> >>
> > ...and how to resolve DMA ? if DMA were resolved cleanly I would agree.
> 
> Regarding that limitation, there is some hope: "next-generation" UIO is
> called VFIO. Useful for exclusively assigning virtual PCI functions of
> network adapters etc. to user space stacks or hypervisors like QEMU/KVM
> (for device pass-through). But it's not mainline yet. And it obviously
> requires an IOMMU.
> 
> But the key point remains: "exclusively". Anything else cannot be
> modeled efficiently via UIO or VFIO.
> 
for many RT apps that is an acceptable limit - in fact sometimes a requirement - having dynamic access and handling this at runtime is not necessarily an advantage for rt. Giving guarantees on timing for non-exclusive access in the general case would probably need to be based on a priority access model any way (i.e. running non-RT network traffic on a RT-link as "rt-idle" bandwidth usage - but full shared access at runtime would be very hard, if not impossible, to model at the device interface level without breaking RT properties. So I think its legitimate to keep it out of the device all together and let the user apps have a non-rt fight about access...

Any pointers to VFIO ?

hofrat

Re: I/O operations priority in RTOS

[Armin Steinhoff]

Nicholas Mc Guire wrote:
> On Sun, 05 Jun 2011, Armin Steinhoff wrote:
>
>> Nicholas Mc Guire wrote:
>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>
>>>> Hello,
>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>> I would like to know the priority when executing open(), read(), write()
>>>> and close() operations.
>>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>>> but I don't know what happens in kernel context with the priority when
>>>> running the I/O operations.
>>>> Thank you for your help, I don't know where to learn about this.
>>>>
>>> []
>>> Also when using bottom half mechanisms you need to take into account the
>>> priority of the kernel thread that manages the defered work items, so
>>> rt-drivers may have a different structure than normal drivers.
>> That's the reason why I prefer UIO based user space drivers !
>>
> ...and how to resolve DMA ? if DMA were resolved cleanly I would agree.

https://opensource.qualcomm.com/wiki/UIO-DMA

I haven't tested until now  ... what do you think about it ?

--Armin


> hofrat
>

Re: I/O operations priority in RTOS

[Armin Steinhoff]

Jan Kiszka wrote:
> On 2011-06-05 10:46, Armin Steinhoff wrote:
>> Nicholas Mc Guire wrote:
>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>
>>>> Hello,
>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>> I would like to know the priority when executing open(), read(), write()
>>>> and close() operations.
>>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>>> but I don't know what happens in kernel context with the priority when
>>>> running the I/O operations.
>>>> Thank you for your help, I don't know where to learn about this.
>>>>
>>> []
>>> Also when using bottom half mechanisms you need to take into account the
>>> priority of the kernel thread that manages the defered work items, so
>>> rt-drivers may have a different structure than normal drivers.
>> That's the reason why I prefer UIO based user space drivers !
> ...which are forced to use threaded IRQs over -rt as well.

What's wrong with that approach? You have an ISR in the kernel module 
and an interrupt thread at user space.
Seem OK for me since a long time with QNX :)
> Taking threaded IRQs out of the picture, bottom-half mechanisms usually
> come into play when dispatch workload that cannot be assigned to a
> specific user context is significant. But then UIO is the wrong approach
> anyway as it doesn't match well the requirements of multi-user device
> interfaces.
   Support of multi-user device interfaces at user space is just a 
design issue ... I use this concept since a long time with QNX :)

--Armin



> Jan
>

Re: I/O operations priority in RTOS

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2610 bytes --]

On 2011-06-06 00:29, Nicholas Mc Guire wrote:
> On Sun, 05 Jun 2011, Jan Kiszka wrote:
> 
>> On 2011-06-05 11:28, Nicholas Mc Guire wrote:
>>> On Sun, 05 Jun 2011, Armin Steinhoff wrote:
>>>
>>>> Nicholas Mc Guire wrote:
>>>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>>>
>>>>>> Hello,
>>>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>>>> I would like to know the priority when executing open(), read(), write()
>>>>>> and close() operations.
>>>>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>>>>> but I don't know what happens in kernel context with the priority when
>>>>>> running the I/O operations.
>>>>>> Thank you for your help, I don't know where to learn about this.
>>>>>>
>>>>> []
>>>>> Also when using bottom half mechanisms you need to take into account the
>>>>> priority of the kernel thread that manages the defered work items, so
>>>>> rt-drivers may have a different structure than normal drivers.
>>>>
>>>> That's the reason why I prefer UIO based user space drivers !
>>>>
>>> ...and how to resolve DMA ? if DMA were resolved cleanly I would agree.
>>
>> Regarding that limitation, there is some hope: "next-generation" UIO is
>> called VFIO. Useful for exclusively assigning virtual PCI functions of
>> network adapters etc. to user space stacks or hypervisors like QEMU/KVM
>> (for device pass-through). But it's not mainline yet. And it obviously
>> requires an IOMMU.
>>
>> But the key point remains: "exclusively". Anything else cannot be
>> modeled efficiently via UIO or VFIO.
>>
> for many RT apps that is an acceptable limit - in fact sometimes a requirement - having dynamic access and handling this at runtime is not necessarily an advantage for rt. Giving guarantees on timing for non-exclusive access in the general case would probably need to be based on a priority access model any way (i.e. running non-RT network traffic on a RT-link as "rt-idle" bandwidth usage - but full shared access at runtime would be very hard, if not impossible, to model at the device interface level without breaking RT properties. So I think its legitimate to keep it out of the device all together and let the user apps have a non-rt fight about access...

There are good examples - in the RT domain - for both models. Sharing a
device does not necessarily mean making things dynamic or even
unpredictable.

> 
> Any pointers to VFIO ?

E.g. http://thread.gmane.org/gmane.linux.kernel.pci/10088

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: I/O operations priority in RTOS

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2302 bytes --]

On 2011-06-06 00:39, Armin Steinhoff wrote:
> Jan Kiszka wrote:
>> On 2011-06-05 10:46, Armin Steinhoff wrote:
>>> Nicholas Mc Guire wrote:
>>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>>
>>>>> Hello,
>>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>>> I would like to know the priority when executing open(), read(),
>>>>> write()
>>>>> and close() operations.
>>>>> In my example the thread which is using the driver runs with 10
>>>>> RTPRIO,
>>>>> but I don't know what happens in kernel context with the priority when
>>>>> running the I/O operations.
>>>>> Thank you for your help, I don't know where to learn about this.
>>>>>
>>>> []
>>>> Also when using bottom half mechanisms you need to take into account
>>>> the
>>>> priority of the kernel thread that manages the defered work items, so
>>>> rt-drivers may have a different structure than normal drivers.
>>> That's the reason why I prefer UIO based user space drivers !
>> ...which are forced to use threaded IRQs over -rt as well.
> 
> What's wrong with that approach? You have an ISR in the kernel module
> and an interrupt thread at user space.

Under -rt, you have an ISR kernel thread in addition to whatever threads
you create in user space. I thought your point about UIO was avoiding
the need to worry about priorities, but maybe I misunderstood this.

> Seem OK for me since a long time with QNX :)
>> Taking threaded IRQs out of the picture, bottom-half mechanisms usually
>> come into play when dispatch workload that cannot be assigned to a
>> specific user context is significant. But then UIO is the wrong approach
>> anyway as it doesn't match well the requirements of multi-user device
>> interfaces.
>   Support of multi-user device interfaces at user space is just a design
> issue ... I use this concept since a long time with QNX :)

There is always a tradeoff. IIRC, even QNX is no pure microkernel
anymore when it comes to performance sensitive services that are shared
between many processes. And Linux is no pure monolithic kernel anymore
when it comes to easily implementing user space file systems or
exclusively assigning devices to processes.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: I/O operations priority in RTOS

[Nicholas Mc Guire]

> 
> There are good examples - in the RT domain - for both models. Sharing a
> device does not necessarily mean making things dynamic or even
> unpredictable.
> 

then I maybe dont get the "exclusive" issue you note - if the highest priority task does not have exclusive access - assuming that there is not contention free sharing - how can you give guarantees ? I can imagin sharing if the highest priority has unconstraint access to the reource and any other access is controled by the highest priority task (i.e. queue replication like ARINC 653 queueing ports) but if you have multiple instances of different priority accessing a resource I don't see how this could be done while giving guarantees on timing (assuming that it is not possible to provide all access as physically atomic instructions)

Do you have an example at hand of such a shared I/O device scheme that can guarantee rt timing ?

> > 
> > Any pointers to VFIO ?
> 
> E.g. http://thread.gmane.org/gmane.linux.kernel.pci/10088
>
thanks. 

hofrat

Re: I/O operations priority in RTOS

[Armin Steinhoff]

Jan Kiszka wrote:
> On 2011-06-05 11:28, Nicholas Mc Guire wrote:
>> On Sun, 05 Jun 2011, Armin Steinhoff wrote:
>>
>>> Nicholas Mc Guire wrote:
>>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>>
>>>>> Hello,
>>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>>> I would like to know the priority when executing open(), read(), write()
>>>>> and close() operations.
>>>>> In my example the thread which is using the driver runs with 10 RTPRIO,
>>>>> but I don't know what happens in kernel context with the priority when
>>>>> running the I/O operations.
>>>>> Thank you for your help, I don't know where to learn about this.
>>>>>
>>>> []
>>>> Also when using bottom half mechanisms you need to take into account the
>>>> priority of the kernel thread that manages the defered work items, so
>>>> rt-drivers may have a different structure than normal drivers.
>>> That's the reason why I prefer UIO based user space drivers !
>>>
>> ...and how to resolve DMA ? if DMA were resolved cleanly I would agree.
> Regarding that limitation, there is some hope: "next-generation" UIO is
> called VFIO.

I found in a posting about VFIO that  " ioctls are used for all the 
basic interactions"  ... that means VFIO works with a lot of context 
switches.
That's not the case with UIO.  So why should VFIO be a mainline approach 
or next generation UIO at all?

--Armin

Changing Kernel thread priorities

[Monica Puig-Pey]

I am writing a driver which has one kernel thread associated with it.
I want to change the priority of this thread, so that I can specify the 
order in which it is scheduled following an interrupt.
I'm using:

  sched_setscheduler(struct task_struct *, int, struct sched_param *);

but it doesn't work. I tried to change the priority from the 
init_module, and also from the Kernel Thread, but there is no way.

Kernel version is 2.6.31-11-rt

What do I call to change a kernel thread priority?

Thanks you very much

Mónica

Re: Changing Kernel thread priorities

[Rolando Martins]

Hi,
I use the following:

PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
print $1; }' | xargs echo)
for i in $PIDs
do
    ret=$(chrt -f -p 99 $i)
done

This will change the kernel thread associated with an irq handler to
RT FIFO prio 99.
Just change the script to your specific interrupt.

Hope it helps,
Rolando

On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey <puigpeym@unican.es> wrote:
> I am writing a driver which has one kernel thread associated with it.
> I want to change the priority of this thread, so that I can specify the
> order in which it is scheduled following an interrupt.
> I'm using:
>
>  sched_setscheduler(struct task_struct *, int, struct sched_param *);
>
> but it doesn't work. I tried to change the priority from the init_module,
> and also from the Kernel Thread, but there is no way.
>
> Kernel version is 2.6.31-11-rt
>
> What do I call to change a kernel thread priority?
>
> Thanks you very much
>
> Mónica
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Monica Puig-Pey]

El 06/06/11 13:54, Rolando Martins escribió:
> Hi,
> I use the following:
>
> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
> print $1; }' | xargs echo)
> for i in $PIDs
> do
>      ret=$(chrt -f -p 99 $i)
> done
>
> This will change the kernel thread associated with an irq handler to
> RT FIFO prio 99.
> Just change the script to your specific interrupt.
>
> Hope it helps,
> Rolando
>
> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey<puigpeym@unican.es>  wrote:
>> I am writing a driver which has one kernel thread associated with it.
>> I want to change the priority of this thread, so that I can specify the
>> order in which it is scheduled following an interrupt.
>> I'm using:
>>
>>   sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>
>> but it doesn't work. I tried to change the priority from the init_module,
>> and also from the Kernel Thread, but there is no way.
>>
>> Kernel version is 2.6.31-11-rt
>>
>> What do I call to change a kernel thread priority?
>>
>> Thanks you very much
>>
>> Mónica
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

I need to change the priority from inside the driver, when creating the 
kernel thread.
Your script is useful but it is done in user context,
Any other help please?

Mónica


--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Johannes Bauer]

Hi!

i thought i sent you the discussion already...

But again, here is how Remy Bohmer does it using the same sched_setscheduler call!
http://www.mail-archive.com/linux-rt-users@vger.kernel.org/msg01046.html

If it does not help you, i woul dsuggest to post your code, maybe this would help people to help you...

regards JB



----- Ursprüngliche Nachricht -----
Von: "Monica Puig-Pey" <puigpeym@unican.es>
Erhalten: 06.06.2011 13:58
An: "Rolando Martins" <rolando.martins@gmail.com>

El 06/06/11 13:54, Rolando Martins escribió:
> Hi,
> I use the following:
>
> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
> print $1; }' | xargs echo)
> for i in $PIDs
> do
> ret=$(chrt -f -p 99 $i)
> done
>
> This will change the kernel thread associated with an irq handler to
> RT FIFO prio 99.
> Just change the script to your specific interrupt.
>
> Hope it helps,
> Rolando
>
> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey wrote:
>> I am writing a driver which has one kernel thread associated with it.
>> I want to change the priority of this thread, so that I can specify the
>> order in which it is scheduled following an interrupt.
>> I'm using:
>>
>> sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>
>> but it doesn't work. I tried to change the priority from the init_module,
>> and also from the Kernel Thread, but there is no way.
>>
>> Kernel version is 2.6.31-11-rt
>>
>> What do I call to change a kernel thread priority?
>>
>> Thanks you very much
>>
>> Mónica
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>

I need to change the priority from inside the driver, when creating the 
kernel thread.
Your script is useful but it is done in user context,
Any other help please?

Mónica


--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Mark Hounschell]

On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
> El 06/06/11 13:54, Rolando Martins escribió:
>> Hi,
>> I use the following:
>>
>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>> print $1; }' | xargs echo)
>> for i in $PIDs
>> do
>> ret=$(chrt -f -p 99 $i)
>> done
>>
>> This will change the kernel thread associated with an irq handler to
>> RT FIFO prio 99.
>> Just change the script to your specific interrupt.
>>
>> Hope it helps,
>> Rolando
>>
>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey<puigpeym@unican.es>
>> wrote:
>>> I am writing a driver which has one kernel thread associated with it.
>>> I want to change the priority of this thread, so that I can specify the
>>> order in which it is scheduled following an interrupt.
>>> I'm using:
>>>
>>> sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>>
>>> but it doesn't work. I tried to change the priority from the
>>> init_module,
>>> and also from the Kernel Thread, but there is no way.
>>>
>>> Kernel version is 2.6.31-11-rt
>>>
>>> What do I call to change a kernel thread priority?
>>>
>>> Thanks you very much
>>>
>>> Mónica
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-rt-users" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>
> I need to change the priority from inside the driver, when creating the
> kernel thread.
> Your script is useful but it is done in user context,
> Any other help please?

What I do is record the PID of the thread in the driver, then create an 
IOCTL for your driver that user land can call that either returns the 
PID so you can do it in user land, or cause the IOCTL code to do it in 
the driver.

The same can be done with the affinity of the IRQ if you record the IRQ 
number.

Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Armin Steinhoff]

Monica Puig-Pey wrote:
> I am writing a driver which has one kernel thread associated with it.
> I want to change the priority of this thread, so that I can specify 
> the order in which it is scheduled following an interrupt.
> I'm using:
>
>  sched_setscheduler(struct task_struct *, int, struct sched_param *);  
> -> sys_sched_setscheduler

http://ftp.au.debian.org/linux-mandocs/2.6.0-test7-full/sys_sched_setscheduler.html

Hope this helps ..

--Armin

>
> but it doesn't work. I tried to change the priority from the 
> init_module, and also from the Kernel Thread, but there is no way.
>
> Kernel version is 2.6.31-11-rt
>
> What do I call to change a kernel thread priority?
>
> Thanks you very much
>
> Mónica
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe 
> linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: I/O operations priority in RTOS

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]

On 2011-06-06 02:21, Nicholas Mc Guire wrote:
>>
>> There are good examples - in the RT domain - for both models. Sharing a
>> device does not necessarily mean making things dynamic or even
>> unpredictable.
>>
> 
> then I maybe dont get the "exclusive" issue you note - if the highest priority task does not have exclusive access - assuming that there is not contention free sharing - how can you give guarantees ? I can imagin sharing if the highest priority has unconstraint access to the reource and any other access is controled by the highest priority task (i.e. queue replication like ARINC 653 queueing ports) but if you have multiple instances of different priority accessing a resource I don't see how this could be done while giving guarantees on timing (assuming that it is not possible to provide all access as physically atomic instructions)

Just the classic way: If there is a critical path while accessing the
device, make it exclusive via locking. If the length of the critical
path is controllable by a low-prio task, you either need to include that
workload in the WCET estimation or you need to apply some limits/quotas
to prevent unbounded path lengths.

> 
> Do you have an example at hand of such a shared I/O device scheme that can guarantee rt timing ?

Let's make it simple: What prevents sharing the process image exposed by
an abstract I/O devices in form of a MMIO region between multiple user
processes? That you may not rely on the hardware to avoid that processes
tap on each other's shoes? But that can be sorted out by privileged
driver that enforces access rules, dispatches variable update events to
listeners or ensures that only consistent modifications are transferred
to the physical process.

Where do we loose RT here?

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: I/O operations priority in RTOS

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 2445 bytes --]

On 2011-06-06 09:41, Armin Steinhoff wrote:
> Jan Kiszka wrote:
>> On 2011-06-05 11:28, Nicholas Mc Guire wrote:
>>> On Sun, 05 Jun 2011, Armin Steinhoff wrote:
>>>
>>>> Nicholas Mc Guire wrote:
>>>>> On Sat, 04 Jun 2011, Monica Puig-Pey wrote:
>>>>>
>>>>>> Hello,
>>>>>> I'm studying how to develop drivers in a real time OS and how do they
>>>>>> work. I'm using  Ubuntu 10.04 with the 2.6.31-11-rt patch installed.
>>>>>> I would like to know the priority when executing open(), read(),
>>>>>> write()
>>>>>> and close() operations.
>>>>>> In my example the thread which is using the driver runs with 10
>>>>>> RTPRIO,
>>>>>> but I don't know what happens in kernel context with the priority
>>>>>> when
>>>>>> running the I/O operations.
>>>>>> Thank you for your help, I don't know where to learn about this.
>>>>>>
>>>>> []
>>>>> Also when using bottom half mechanisms you need to take into
>>>>> account the
>>>>> priority of the kernel thread that manages the defered work items, so
>>>>> rt-drivers may have a different structure than normal drivers.
>>>> That's the reason why I prefer UIO based user space drivers !
>>>>
>>> ...and how to resolve DMA ? if DMA were resolved cleanly I would agree.
>> Regarding that limitation, there is some hope: "next-generation" UIO is
>> called VFIO.
> 
> I found in a posting about VFIO that  " ioctls are used for all the
> basic interactions"  ... that means VFIO works with a lot of context
> switches.

That mostly refers to configuring VFIO devices (just check the code when
in doubt).

One exception is EIO. VFIO works generically, without any device
specific kernel stubs. So legacy interrupts are first generically
masked, then reported to user space to do the device specific handling,
and then ack'ed by user space via an IOCTL to remove the mask again. But
that only matters for legacy INTx, not for MSI[-X].

> That's not the case with UIO.  So why should VFIO be a mainline approach
> or next generation UIO at all?

Because it is promising to solve the DMA challenge without totally
ignoring security (in contrast to uio-dma). And because it already has
two major use cases (high-speed user space networking and virtualization).

Conceptually, VFIO should be able to cover most of those scenarios that
UIO currently addresses. But UIO is kernel ABI and wouldn't go away even
if VFIO was able to completely replace it.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]

Re: Changing Kernel thread priorities

[Peter Zijlstra]

"Monica Puig-Pey" <puigpeym@unican.es> wrote:
> 
> I need to change the priority from inside the driver, when creating the 
> kernel thread.

No you don't. How does you driver know about what priority is correct
wrt all the other running RT tasks on the system?

Determining the right priority in a fixed priority scheduling system is
a system wide problem, nobody but the administrator can possibly even
begin to solve it.

There's a reason all RT irq threads are started at 50, its plain
impossible to do better.

Re: Changing Kernel thread priorities

[Johannes Bauer]

Absolutly correct!

However, if you are running the system on an embedded platform, where the _WHOLE_ system (including priorities) is preconfigured and never touched, starting a irq thread with the right prio from start is a more straightforward method than having to invoke a script that changes it using userspace chrt tool.

Regards JB
----- Ursprüngliche Nachricht -----
Von: "Peter Zijlstra" <peterz@infradead.org>
Erhalten: 07.06.2011 00:36
An: hannes_bauer@aon.at

"Monica Puig-Pey" wrote:
> 
> I need to change the priority from inside the driver, when creating the 
> kernel thread.

No you don't. How does you driver know about what priority is correct
wrt all the other running RT tasks on the system?

Determining the right priority in a fixed priority scheduling system is
a system wide problem, nobody but the administrator can possibly even
begin to solve it.

There's a reason all RT irq threads are started at 50, its plain
impossible to do better.

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Monica Puig-Pey]

El 07/06/11 10:27, Johannes Bauer escribió:
> Absolutly correct!
>
> However, if you are running the system on an embedded platform, where the _WHOLE_ system (including priorities) is preconfigured and never touched, starting a irq thread with the right prio from start is a more straightforward method than having to invoke a script that changes it using userspace chrt tool.
>
> Regards JB
> ----- Ursprüngliche Nachricht -----
> Von: "Peter Zijlstra"<peterz@infradead.org>
> Erhalten: 07.06.2011 00:36
> An: hannes_bauer@aon.at
>
> "Monica Puig-Pey" wrote:
>>
>> I need to change the priority from inside the driver, when creating the
>> kernel thread.
>
> No you don't. How does you driver know about what priority is correct
> wrt all the other running RT tasks on the system?
>
> Determining the right priority in a fixed priority scheduling system is
> a system wide problem, nobody but the administrator can possibly even
> begin to solve it.
>
> There's a reason all RT irq threads are started at 50, its plain
> impossible to do better.
>

That's it!
If I work with embedded system where I know all my tasks running and 
there is not a user how could I do it?

What I tried is create the kernel thread in my init_module using:

	#include <linux/kthread.h>

	struct task_struct *kthread_create(int (*threadfn)(void *data),
				   void *data,
				   const char namefmt[], ...)
and then running it with:

	#include <linux/sched.h>
	
	extern int wake_up_process(struct task_struct *tsk);

These functions stars a Kthread which has a NON RT priority. I can see 
this using the ps command from user space.
Because it's not a real time thread is why I want, better need, to 
change its priority, to have only real time threads running in my 
driver. I want to use the Kthread as a bottom half for the interrupts.

How could I create real time kernel threads then? is kthread_create 
wrong for real time enviroment?


-- 
__________________________________________________________________________________

Mónica Puig-Pey González       E-mail: puigpeym@unican.es

Grupo de Computadores y Tiempo Real, Departamento de Electrónica y 
Computadores.
Facultad de Ciencias - Universidad de Cantabria
Av. de los Castros s/n. 39005 - Santander, España
__________________________________________________________________________________

Re: Changing Kernel thread priorities

[Monica Puig-Pey]

El 06/06/11 18:49, Mark Hounschell escribió:
> On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
>> El 06/06/11 13:54, Rolando Martins escribió:
>>> Hi,
>>> I use the following:
>>>
>>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>>> print $1; }' | xargs echo)
>>> for i in $PIDs
>>> do
>>> ret=$(chrt -f -p 99 $i)
>>> done
>>>
>>> This will change the kernel thread associated with an irq handler to
>>> RT FIFO prio 99.
>>> Just change the script to your specific interrupt.
>>>
>>> Hope it helps,
>>> Rolando
>>>
>>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey<puigpeym@unican.es>
>>> wrote:
>>>> I am writing a driver which has one kernel thread associated with it.
>>>> I want to change the priority of this thread, so that I can specify the
>>>> order in which it is scheduled following an interrupt.
>>>> I'm using:
>>>>
>>>> sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>>>
>>>> but it doesn't work. I tried to change the priority from the
>>>> init_module,
>>>> and also from the Kernel Thread, but there is no way.
>>>>
>>>> Kernel version is 2.6.31-11-rt
>>>>
>>>> What do I call to change a kernel thread priority?
>>>>
>>>> Thanks you very much
>>>>
>>>> Mónica
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> linux-rt-users" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
>>
>> I need to change the priority from inside the driver, when creating the
>> kernel thread.
>> Your script is useful but it is done in user context,
>> Any other help please?
>
> What I do is record the PID of the thread in the driver, then create an
> IOCTL for your driver that user land can call that either returns the
> PID so you can do it in user land, or cause the IOCTL code to do it in
> the driver.
>
> The same can be done with the affinity of the IRQ if you record the IRQ
> number.
>
> Mark
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

But I don't have de PID of my Kthread, I only have the task_struc * that 
gives me the function:

  struct task_struct *kthread_create(int (*threadfn)(void *data),
                    void *data,
                    const char namefmt[], ...)

How could I get the PID, and which function should I use in the IOCTL 
(kernel context) for changing its priority?

Thank you so much for your help

Mónica
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Thomas Gleixner]

On Tue, 7 Jun 2011, Johannes Bauer wrote:

Please stop top-posting and use proper line breaks at 78

> > Peter Zijlstra wrote:
> > > "Monica Puig-Pey" wrote:

> > > I need to change the priority from inside the driver, when creating the 
> > > kernel thread.
> >
> > No you don't. How does you driver know about what priority is correct
> > wrt all the other running RT tasks on the system?
> > 
> > Determining the right priority in a fixed priority scheduling system is
> > a system wide problem, nobody but the administrator can possibly even
> > begin to solve it.
> > 
> > There's a reason all RT irq threads are started at 50, its plain
> > impossible to do better.

> Absolutly correct!
> 
> However, if you are running the system on an embedded platform,
> where the _WHOLE_ system (including priorities) is preconfigured and
> never touched, starting a irq thread with the right prio from start
> is a more straightforward method than having to invoke a script that
> changes it using userspace chrt tool.

Feel free to do that for your embedded system and carry the patch for
yourself if you think it's worth to avoid the extra init script.

But we do _not_ add stuff like this to the mainline simply because
there is no way to find a prio setting which is appropriate for all
users of a particular driver.

Aside of that the extra init script is definitely less annoying to
maintain than the crap you need to hack into random drivers.

Thanks,

	tglx

Re: Changing Kernel thread priorities

[Thomas Gleixner]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1154 bytes --]

On Tue, 7 Jun 2011, Monica Puig-Pey wrote:
> El 07/06/11 10:27, Johannes Bauer escribió:
> > There's a reason all RT irq threads are started at 50, its plain
> > impossible to do better.
> > 
> 
> That's it!
> If I work with embedded system where I know all my tasks running and there is
> not a user how could I do it?

init scripts run from user space and you can adjust the priority there.
 
> What I tried is create the kernel thread in my init_module using:
> 
> 	#include <linux/kthread.h>
> 
> 	struct task_struct *kthread_create(int (*threadfn)(void *data),
> 				   void *data,
> 				   const char namefmt[], ...)
> and then running it with:
> 
> 	#include <linux/sched.h>
> 	
> 	extern int wake_up_process(struct task_struct *tsk);
> 
> These functions stars a Kthread which has a NON RT priority. I can see this
> using the ps command from user space.
> Because it's not a real time thread is why I want, better need, to change its
> priority, to have only real time threads running in my driver. I want to use
> the Kthread as a bottom half for the interrupts.

Use threaded interrupt handlers. That's what they are made for.

Thanks,

	tglx

Re: Changing Kernel thread priorities

[Mark Hounschell]

On 06/07/2011 04:40 AM, Monica Puig-Pey wrote:
> El 06/06/11 18:49, Mark Hounschell escribió:
>> On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
>>> El 06/06/11 13:54, Rolando Martins escribió:
>>>> Hi,
>>>> I use the following:
>>>>
>>>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>>>> print $1; }' | xargs echo)
>>>> for i in $PIDs
>>>> do
>>>> ret=$(chrt -f -p 99 $i)
>>>> done
>>>>
>>>> This will change the kernel thread associated with an irq handler to
>>>> RT FIFO prio 99.
>>>> Just change the script to your specific interrupt.
>>>>
>>>> Hope it helps,
>>>> Rolando
>>>>
>>>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey<puigpeym@unican.es>
>>>> wrote:
>>>>> I am writing a driver which has one kernel thread associated with it.
>>>>> I want to change the priority of this thread, so that I can specify the
>>>>> order in which it is scheduled following an interrupt.
>>>>> I'm using:
>>>>>
>>>>> sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>>>>
>>>>> but it doesn't work. I tried to change the priority from the
>>>>> init_module,
>>>>> and also from the Kernel Thread, but there is no way.
>>>>>
>>>>> Kernel version is 2.6.31-11-rt
>>>>>
>>>>> What do I call to change a kernel thread priority?
>>>>>
>>>>> Thanks you very much
>>>>>
>>>>> Mónica
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>> linux-rt-users" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>
>>>
>>> I need to change the priority from inside the driver, when creating the
>>> kernel thread.
>>> Your script is useful but it is done in user context,
>>> Any other help please?
>>
>> What I do is record the PID of the thread in the driver, then create an
>> IOCTL for your driver that user land can call that either returns the
>> PID so you can do it in user land, or cause the IOCTL code to do it in
>> the driver.
>>
>> The same can be done with the affinity of the IRQ if you record the IRQ
>> number.
>>
>> Mark
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-rt-users" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> But I don't have de PID of my Kthread, I only have the task_struc * that
> gives me the function:
>
> struct task_struct *kthread_create(int (*threadfn)(void *data),
> void *data,
> const char namefmt[], ...)
>
> How could I get the PID, and which function should I use in the IOCTL
> (kernel context) for changing its priority?
>

The PID can be obtained from within the interrupt handler its self via 
current->pid.
Obviously an interrupt has to occur first but after one interrupt you have it.

Actually I had forgot how I handled this. Where I change the RT priority 
and cpu affinity is in what used to be called the Bottom Half and the IOCTL 
referred to above simply tells the BH to do it and with what values.

Regards
Mark



--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Peter Zijlstra]

On Tue, 2011-06-07 at 11:40 +0200, Armin Steinhoff wrote:
> Hi,
> 
> when I read all these confusing statements here ( in german it looks 
> like an "Eiertanz")  ... I can only say:
> 
> - do the basic stuff in a minimal kernel driver
> - use UIO (or VFIO for PCI devices)

I see no requirement for any of those horrid things to be used. You can
write a full on proper kernel driver, it just cannot set kernel thread
priorities to a sane value (let them all default to 50 or so).

Then have a user space script or whatever set the kthread priorities.

> and you get clean control about your real-time priorities.
> 
> I think changing the priorities of "interrupt threads" inside the kernel 
> could lead to strange race conditions in the kernel.

No, changing the priority in the kernel is a perfectly sound operation,
it just doesn't make any sense to do so since its impossible to
determine a proper priority.

Therefore setting a priority is a pure user policy and should not be
done by the driver itself -- it simply cannot do it right so why bother
doing it.

Re: Changing Kernel thread priorities

[Armin Steinhoff]

Hi,

when I read all these confusing statements here ( in german it looks 
like an "Eiertanz")  ... I can only say:

- do the basic stuff in a minimal kernel driver
- use UIO (or VFIO for PCI devices)

and you get clean control about your real-time priorities.

I think changing the priorities of "interrupt threads" inside the kernel 
could lead to strange race conditions in the kernel.
That seems to be the reason for that "Eiertanz" here :)

--Armin





Thomas Gleixner wrote:
> On Tue, 7 Jun 2011, Johannes Bauer wrote:
>
> Please stop top-posting and use proper line breaks at 78
>
>>> Peter Zijlstra wrote:
>>>> "Monica Puig-Pey" wrote:
>>>> I need to change the priority from inside the driver, when creating the
>>>> kernel thread.
>>> No you don't. How does you driver know about what priority is correct
>>> wrt all the other running RT tasks on the system?
>>>
>>> Determining the right priority in a fixed priority scheduling system is
>>> a system wide problem, nobody but the administrator can possibly even
>>> begin to solve it.
>>>
>>> There's a reason all RT irq threads are started at 50, its plain
>>> impossible to do better.
>> Absolutly correct!
>>
>> However, if you are running the system on an embedded platform,
>> where the _WHOLE_ system (including priorities) is preconfigured and
>> never touched, starting a irq thread with the right prio from start
>> is a more straightforward method than having to invoke a script that
>> changes it using userspace chrt tool.
> Feel free to do that for your embedded system and carry the patch for
> yourself if you think it's worth to avoid the extra init script.
>
> But we do _not_ add stuff like this to the mainline simply because
> there is no way to find a prio setting which is appropriate for all
> users of a particular driver.
>
> Aside of that the extra init script is definitely less annoying to
> maintain than the crap you need to hack into random drivers.
>
> Thanks,
>
> 	tglx
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Changing Kernel thread priorities

[Mark Hounschell]

On 06/07/2011 05:14 AM, Mark Hounschell wrote:
> On 06/07/2011 04:40 AM, Monica Puig-Pey wrote:
>> El 06/06/11 18:49, Mark Hounschell escribió:
>>> On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
>>>> El 06/06/11 13:54, Rolando Martins escribió:
>>>>> Hi,
>>>>> I use the following:
>>>>>
>>>>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>>>>> print $1; }' | xargs echo)
>>>>> for i in $PIDs
>>>>> do
>>>>> ret=$(chrt -f -p 99 $i)
>>>>> done
>>>>>
>>>>> This will change the kernel thread associated with an irq handler to
>>>>> RT FIFO prio 99.
>>>>> Just change the script to your specific interrupt.
>>>>>
>>>>> Hope it helps,
>>>>> Rolando
>>>>>
>>>>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey<puigpeym@unican.es>
>>>>> wrote:
>>>>>> I am writing a driver which has one kernel thread associated with it.
>>>>>> I want to change the priority of this thread, so that I can specify the
>>>>>> order in which it is scheduled following an interrupt.
>>>>>> I'm using:
>>>>>>
>>>>>> sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>>>>>
>>>>>> but it doesn't work. I tried to change the priority from the
>>>>>> init_module,
>>>>>> and also from the Kernel Thread, but there is no way.
>>>>>>
>>>>>> Kernel version is 2.6.31-11-rt
>>>>>>
>>>>>> What do I call to change a kernel thread priority?
>>>>>>
>>>>>> Thanks you very much
>>>>>>
>>>>>> Mónica
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>> linux-rt-users" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>
>>>> I need to change the priority from inside the driver, when creating the
>>>> kernel thread.
>>>> Your script is useful but it is done in user context,
>>>> Any other help please?
>>>
>>> What I do is record the PID of the thread in the driver, then create an
>>> IOCTL for your driver that user land can call that either returns the
>>> PID so you can do it in user land, or cause the IOCTL code to do it in
>>> the driver.
>>>
>>> The same can be done with the affinity of the IRQ if you record the IRQ
>>> number.
>>>
>>> Mark
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-rt-users" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> But I don't have de PID of my Kthread, I only have the task_struc * that
>> gives me the function:
>>
>> struct task_struct *kthread_create(int (*threadfn)(void *data),
>> void *data,
>> const char namefmt[], ...)
>>
>> How could I get the PID, and which function should I use in the IOCTL
>> (kernel context) for changing its priority?
>>
>
> The PID can be obtained from within the interrupt handler its self via
> current->pid.
> Obviously an interrupt has to occur first but after one interrupt you have it.
>
> Actually I had forgot how I handled this. Where I change the RT priority
> and cpu affinity is in what used to be called the Bottom Half and the IOCTL
> referred to above simply tells the BH to do it and with what values.
>

In interrupt handler code snippet:

struct task_struct *TSK;
struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };

         TSK = current;

Code snippet from BH:

         if (((rtom_rtprio != 0) &&
              (rtom_rtprio != PARAM.sched_priority)) ||
              (my_rtom_rtprio[BOARD] != rtom_rtprio)) {

                 PARAM.sched_priority = rtom_rtprio;
                 my_rtom_rtprio[COUNT] = rtom_rtprio;
                 sched_setscheduler(TSK, SCHED_FIFO, &PARAM);

                 set_cpus_allowed(TSK, rtom_devices[BOARD].irq_cpu_mask);
                 rtom_devices[BOARD].irq_task_pid = TSK->pid;
        }

rtom_rtprio and irq_cpu_mask are set by userland via an IOCTL. An interrupt 
must occur for this to happen and my BOARD never shares IRQs.

Mark

--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Remy Bohmer]

[-- Attachment #1: Type: text/plain, Size: 2535 bytes --]

Hi All,

2011/6/7 Peter Zijlstra <peterz@infradead.org>:
> On Tue, 2011-06-07 at 11:40 +0200, Armin Steinhoff wrote:
>> Hi,
>>
>> when I read all these confusing statements here ( in german it looks
>> like an "Eiertanz")  ... I can only say:
>>
>> - do the basic stuff in a minimal kernel driver
>> - use UIO (or VFIO for PCI devices)
>
> I see no requirement for any of those horrid things to be used. You can
> write a full on proper kernel driver, it just cannot set kernel thread
> priorities to a sane value (let them all default to 50 or so).
>
> Then have a user space script or whatever set the kthread priorities.
>> and you get clean control about your real-time priorities.
>> I think changing the priorities of "interrupt threads" inside the kernel
>> could lead to strange race conditions in the kernel.

Well, I 100% agree that it must be under full userspace control to be
able to set the priorities. But, the kernel default assumption of
starting everything at 50 is wrong as well.
Imagine the following situation:
* Realtime application is running and has threads active in the range
of prios 20 - 90.
* Now bring up a network device, it immediately starts spamming the
system at prio 50 _before_ you have the chance to set it below 20 by
means of chrt.
* RT behaviour is gone!

So, in that case and in many other hotplug cases, you ruin the RT
behaviour of the system just by the
default-50-is-probably-right-assumption of the kernel.
For systems where you have everything under control as a user/system
designer, hotplug can also be under control as well.

To solve this we have the patchset in use as attached to this mail. It
is a newer version of the old set mentioned earlier in this mail
thread.
The opinion of Thomas about this subject is quite clear so I will not
post it as a cleaned up patchset, although I believe we reworked all
his previous major remarks about this set.

For everyone else who can do something useful with it: go ahead. It
should apply to 2.6.33.9-rt31
It creates entries in /proc/irq for you to setup the priorities after
booting. (/proc/irq/hirq-prio and /proc/irq/sirq-prio), but also per
driver in /proc/irq/<irq-id>/<name>/irqprio
In /proc/irq/[hs]irq-prio you can, for example, enter the following text:
* at91_udc:22,ohci_hcd:usb1:22,atmel_spi:22,33

This results in:
* starting the at91_udc at prio 22
* starting ohci:usb1 also at prio 22
* atmel_spi at 22
* overall default moved from 50 to 33

Kind regards,

Remy

[-- Attachment #2: irqprio-patches.tar.gz --]
[-- Type: application/x-gzip, Size: 5565 bytes --]

Re: Changing Kernel thread priorities

[Peter Zijlstra]

On Tue, 2011-06-07 at 13:02 +0200, Remy Bohmer wrote:
> Hi All,
> 
> 2011/6/7 Peter Zijlstra <peterz@infradead.org>:
> > On Tue, 2011-06-07 at 11:40 +0200, Armin Steinhoff wrote:
> >> Hi,
> >>
> >> when I read all these confusing statements here ( in german it looks
> >> like an "Eiertanz")  ... I can only say:
> >>
> >> - do the basic stuff in a minimal kernel driver
> >> - use UIO (or VFIO for PCI devices)
> >
> > I see no requirement for any of those horrid things to be used. You can
> > write a full on proper kernel driver, it just cannot set kernel thread
> > priorities to a sane value (let them all default to 50 or so).
> >
> > Then have a user space script or whatever set the kthread priorities.
> >> and you get clean control about your real-time priorities.
> >> I think changing the priorities of "interrupt threads" inside the kernel
> >> could lead to strange race conditions in the kernel.
> 
> Well, I 100% agree that it must be under full userspace control to be
> able to set the priorities. But, the kernel default assumption of
> starting everything at 50 is wrong as well.
> Imagine the following situation:
> * Realtime application is running and has threads active in the range
> of prios 20 - 90.
> * Now bring up a network device, it immediately starts spamming the
> system at prio 50 _before_ you have the chance to set it below 20 by
> means of chrt.
> * RT behaviour is gone!

Good point I guess, Thomas should we default to 1 for everything?

Re: Changing Kernel thread priorities

[Lucas De Marchi]

On Tue, Jun 7, 2011 at 11:14, Peter Zijlstra <peterz@infradead.org> wrote:
> > Well, I 100% agree that it must be under full userspace control to be
> > able to set the priorities. But, the kernel default assumption of
> > starting everything at 50 is wrong as well.
> > Imagine the following situation:
> > * Realtime application is running and has threads active in the range
> > of prios 20 - 90.
> > * Now bring up a network device, it immediately starts spamming the
> > system at prio 50 _before_ you have the chance to set it below 20 by
> > means of chrt.
> > * RT behaviour is gone!

Why is the application using priorities in the range 20-90 if it's
well known that we default RT kernel threads to 50?

IMO it's a mistake in the application.


Lucas De Marchi

Re: Changing Kernel thread priorities

[Thomas Gleixner]

On Tue, 7 Jun 2011, Peter Zijlstra wrote:
> On Tue, 2011-06-07 at 13:02 +0200, Remy Bohmer wrote:
> > Well, I 100% agree that it must be under full userspace control to be
> > able to set the priorities. But, the kernel default assumption of
> > starting everything at 50 is wrong as well.
> > Imagine the following situation:
> > * Realtime application is running and has threads active in the range
> > of prios 20 - 90.
> > * Now bring up a network device, it immediately starts spamming the
> > system at prio 50 _before_ you have the chance to set it below 20 by
> > means of chrt.
> > * RT behaviour is gone!
> 
> Good point I guess, Thomas should we default to 1 for everything?

No objections.

Re: Changing Kernel thread priorities

[Monica Puig-Pey]

El 07/06/11 11:46, Mark Hounschell escribió:
> On 06/07/2011 05:14 AM, Mark Hounschell wrote:
>> On 06/07/2011 04:40 AM, Monica Puig-Pey wrote:
>>> El 06/06/11 18:49, Mark Hounschell escribió:
>>>> On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
>>>>> El 06/06/11 13:54, Rolando Martins escribió:
>>>>>> Hi,
>>>>>> I use the following:
>>>>>>
>>>>>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>>>>>> print $1; }' | xargs echo)
>>>>>> for i in $PIDs
>>>>>> do
>>>>>> ret=$(chrt -f -p 99 $i)
>>>>>> done
>>>>>>
>>>>>> This will change the kernel thread associated with an irq handler to
>>>>>> RT FIFO prio 99.
>>>>>> Just change the script to your specific interrupt.
>>>>>>
>>>>>> Hope it helps,
>>>>>> Rolando
>>>>>>
>>>>>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey
>>>>>> wrote:
>>>>>>> I am writing a driver which has one kernel thread associated with
>>>>>>> it.
>>>>>>> I want to change the priority of this thread, so that I can
>>>>>>> specify the
>>>>>>> order in which it is scheduled following an interrupt.
>>>>>>> I'm using:
>>>>>>>
>>>>>>> sched_setscheduler(struct task_struct *, int, struct sched_param *);
>>>>>>>
>>>>>>> but it doesn't work. I tried to change the priority from the
>>>>>>> init_module,
>>>>>>> and also from the Kernel Thread, but there is no way.
>>>>>>>
>>>>>>> Kernel version is 2.6.31-11-rt
>>>>>>>
>>>>>>> What do I call to change a kernel thread priority?
>>>>>>>
>>>>>>> Thanks you very much
>>>>>>>
>>>>>>> Mónica
>>>>>>>
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>> linux-rt-users" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>>
>>>>>
>>>>> I need to change the priority from inside the driver, when creating
>>>>> the
>>>>> kernel thread.
>>>>> Your script is useful but it is done in user context,
>>>>> Any other help please?
>>>>
>>>> What I do is record the PID of the thread in the driver, then create an
>>>> IOCTL for your driver that user land can call that either returns the
>>>> PID so you can do it in user land, or cause the IOCTL code to do it in
>>>> the driver.
>>>>
>>>> The same can be done with the affinity of the IRQ if you record the IRQ
>>>> number.
>>>>
>>>> Mark
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> linux-rt-users" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>> But I don't have de PID of my Kthread, I only have the task_struc * that
>>> gives me the function:
>>>
>>> struct task_struct *kthread_create(int (*threadfn)(void *data),
>>> void *data,
>>> const char namefmt[], ...)
>>>
>>> How could I get the PID, and which function should I use in the IOCTL
>>> (kernel context) for changing its priority?
>>>
>>
>> The PID can be obtained from within the interrupt handler its self via
>> current->pid.
>> Obviously an interrupt has to occur first but after one interrupt you
>> have it.
>>
>> Actually I had forgot how I handled this. Where I change the RT priority
>> and cpu affinity is in what used to be called the Bottom Half and the
>> IOCTL
>> referred to above simply tells the BH to do it and with what values.
>>
>
> In interrupt handler code snippet:
>
> struct task_struct *TSK;
> struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };
>
> TSK = current;
>
> Code snippet from BH:
>
> if (((rtom_rtprio != 0) &&
> (rtom_rtprio != PARAM.sched_priority)) ||
> (my_rtom_rtprio[BOARD] != rtom_rtprio)) {
>
> PARAM.sched_priority = rtom_rtprio;
> my_rtom_rtprio[COUNT] = rtom_rtprio;
> sched_setscheduler(TSK, SCHED_FIFO, &PARAM);
>
> set_cpus_allowed(TSK, rtom_devices[BOARD].irq_cpu_mask);
> rtom_devices[BOARD].irq_task_pid = TSK->pid;
> }
>
> rtom_rtprio and irq_cpu_mask are set by userland via an IOCTL. An
> interrupt must occur for this to happen and my BOARD never shares IRQs.
>
> Mark
>

Thanks, your idea seems to be very useful to me. I was doing it very 
similar, but I didn't use the "current" variable to know the task_struct *.

I have tried your suggestion on an easy example, but it didn't work.
Insmod returns through the kernel

	[11334.895499] kthread: Unknown symbol sched_setscheduler

Code shown below:

#include <linux/module.h>	
#include <linux/kernel.h>
#include <linux/ioport.h>

#include <linux/wait.h>
#include <linux/kthread.h>
#include <asm/io.h>
#include <linux/sched.h>


struct task_struct *ts;

int thread(void *data)
{
   struct task_struct *TSK;
   struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };
   TSK = current;

   PARAM.sched_priority = 50;
   sched_setscheduler(TSK, SCHED_FIFO, &PARAM); // <-- unknown symbol??

   while(1){
     printk("Hi I am kernel thread!\n");
     msleep(100);
     if (kthread_should_stop())
       break;		
}
         return 0;
}


int init_module(void)
{
   printk(KERN_INFO "init_module() called\n");
   ts=kthread_run(thread,NULL,"kthread");
          return 0;
}

void cleanup_module(void)
{
   printk(KERN_INFO "cleanup_module() called\n");
   kthread_stop(ts);
}

Re: Changing Kernel thread priorities

[Mark Hounschell]

On 06/07/2011 02:34 PM, Monica Puig-Pey wrote:
> El 07/06/11 11:46, Mark Hounschell escribió:
>> On 06/07/2011 05:14 AM, Mark Hounschell wrote:
>>> On 06/07/2011 04:40 AM, Monica Puig-Pey wrote:
>>>> El 06/06/11 18:49, Mark Hounschell escribió:
>>>>> On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
>>>>>> El 06/06/11 13:54, Rolando Martins escribió:
>>>>>>> Hi,
>>>>>>> I use the following:
>>>>>>>
>>>>>>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>>>>>>> print $1; }' | xargs echo)
>>>>>>> for i in $PIDs
>>>>>>> do
>>>>>>> ret=$(chrt -f -p 99 $i)
>>>>>>> done
>>>>>>>
>>>>>>> This will change the kernel thread associated with an irq handler to
>>>>>>> RT FIFO prio 99.
>>>>>>> Just change the script to your specific interrupt.
>>>>>>>
>>>>>>> Hope it helps,
>>>>>>> Rolando
>>>>>>>
>>>>>>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey
>>>>>>> wrote:
>>>>>>>> I am writing a driver which has one kernel thread associated with
>>>>>>>> it.
>>>>>>>> I want to change the priority of this thread, so that I can
>>>>>>>> specify the
>>>>>>>> order in which it is scheduled following an interrupt.
>>>>>>>> I'm using:
>>>>>>>>
>>>>>>>> sched_setscheduler(struct task_struct *, int, struct sched_param
>>>>>>>> *);
>>>>>>>>
>>>>>>>> but it doesn't work. I tried to change the priority from the
>>>>>>>> init_module,
>>>>>>>> and also from the Kernel Thread, but there is no way.
>>>>>>>>
>>>>>>>> Kernel version is 2.6.31-11-rt
>>>>>>>>
>>>>>>>> What do I call to change a kernel thread priority?
>>>>>>>>
>>>>>>>> Thanks you very much
>>>>>>>>
>>>>>>>> Mónica
>>>>>>>>
>>>>>>>> --
>>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>>> linux-rt-users" in
>>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>>>
>>>>>>
>>>>>> I need to change the priority from inside the driver, when creating
>>>>>> the
>>>>>> kernel thread.
>>>>>> Your script is useful but it is done in user context,
>>>>>> Any other help please?
>>>>>
>>>>> What I do is record the PID of the thread in the driver, then
>>>>> create an
>>>>> IOCTL for your driver that user land can call that either returns the
>>>>> PID so you can do it in user land, or cause the IOCTL code to do it in
>>>>> the driver.
>>>>>
>>>>> The same can be done with the affinity of the IRQ if you record the
>>>>> IRQ
>>>>> number.
>>>>>
>>>>> Mark
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>> linux-rt-users" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
>>>> But I don't have de PID of my Kthread, I only have the task_struc *
>>>> that
>>>> gives me the function:
>>>>
>>>> struct task_struct *kthread_create(int (*threadfn)(void *data),
>>>> void *data,
>>>> const char namefmt[], ...)
>>>>
>>>> How could I get the PID, and which function should I use in the IOCTL
>>>> (kernel context) for changing its priority?
>>>>
>>>
>>> The PID can be obtained from within the interrupt handler its self via
>>> current->pid.
>>> Obviously an interrupt has to occur first but after one interrupt you
>>> have it.
>>>
>>> Actually I had forgot how I handled this. Where I change the RT priority
>>> and cpu affinity is in what used to be called the Bottom Half and the
>>> IOCTL
>>> referred to above simply tells the BH to do it and with what values.
>>>
>>
>> In interrupt handler code snippet:
>>
>> struct task_struct *TSK;
>> struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };
>>
>> TSK = current;
>>
>> Code snippet from BH:
>>
>> if (((rtom_rtprio != 0) &&
>> (rtom_rtprio != PARAM.sched_priority)) ||
>> (my_rtom_rtprio[BOARD] != rtom_rtprio)) {
>>
>> PARAM.sched_priority = rtom_rtprio;
>> my_rtom_rtprio[COUNT] = rtom_rtprio;
>> sched_setscheduler(TSK, SCHED_FIFO, &PARAM);
>>
>> set_cpus_allowed(TSK, rtom_devices[BOARD].irq_cpu_mask);
>> rtom_devices[BOARD].irq_task_pid = TSK->pid;
>> }
>>
>> rtom_rtprio and irq_cpu_mask are set by userland via an IOCTL. An
>> interrupt must occur for this to happen and my BOARD never shares IRQs.
>>
>> Mark
>>
>
> Thanks, your idea seems to be very useful to me. I was doing it very
> similar, but I didn't use the "current" variable to know the task_struct *.
>
> I have tried your suggestion on an easy example, but it didn't work.
> Insmod returns through the kernel
>
> [11334.895499] kthread: Unknown symbol sched_setscheduler
>
> Code shown below:
>
> #include <linux/module.h>
> #include <linux/kernel.h>
> #include <linux/ioport.h>
>
> #include <linux/wait.h>
> #include <linux/kthread.h>
> #include <asm/io.h>
> #include <linux/sched.h>
>
>
> struct task_struct *ts;
>
> int thread(void *data)
> {
> struct task_struct *TSK;
> struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };
> TSK = current;
>
> PARAM.sched_priority = 50;
> sched_setscheduler(TSK, SCHED_FIFO, &PARAM); // <-- unknown symbol??
>
> while(1){
> printk("Hi I am kernel thread!\n");
> msleep(100);
> if (kthread_should_stop())
> break;
> }
> return 0;
> }
>
>
> int init_module(void)
> {
> printk(KERN_INFO "init_module() called\n");
> ts=kthread_run(thread,NULL,"kthread");
> return 0;
> }
>
> void cleanup_module(void)
> {
> printk(KERN_INFO "cleanup_module() called\n");
> kthread_stop(ts);
> }
> .
>

In kernel/sched.c

EXPORT_SYMBOL_GPL(sched_setscheduler);

If your driver is not GPL, you can't use it.

Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Nicholas Mc Guire]

On Tue, 07 Jun 2011, Remy Bohmer wrote:

> Hi All,
> 
> 2011/6/7 Peter Zijlstra <peterz@infradead.org>:
> > On Tue, 2011-06-07 at 11:40 +0200, Armin Steinhoff wrote:
> >> Hi,
> >>
> >> when I read all these confusing statements here ( in german it looks
> >> like an "Eiertanz") ?... I can only say:
> >>
> >> - do the basic stuff in a minimal kernel driver
> >> - use UIO (or VFIO for PCI devices)
> >
> > I see no requirement for any of those horrid things to be used. You can
> > write a full on proper kernel driver, it just cannot set kernel thread
> > priorities to a sane value (let them all default to 50 or so).
> >
> > Then have a user space script or whatever set the kthread priorities.
> >> and you get clean control about your real-time priorities.
> >> I think changing the priorities of "interrupt threads" inside the kernel
> >> could lead to strange race conditions in the kernel.
> 
> Well, I 100% agree that it must be under full userspace control to be
> able to set the priorities. But, the kernel default assumption of
> starting everything at 50 is wrong as well.
> Imagine the following situation:
> * Realtime application is running and has threads active in the range
> of prios 20 - 90.
> * Now bring up a network device, it immediately starts spamming the
> system at prio 50 _before_ you have the chance to set it below 20 by
> means of chrt.
> * RT behaviour is gone!
> 
> So, in that case and in many other hotplug cases, you ruin the RT
> behaviour of the system just by the
> default-50-is-probably-right-assumption of the kernel.
> For systems where you have everything under control as a user/system
> designer, hotplug can also be under control as well.
>

I dont't quite see that - the 50 default is well dokumented so you can
plan it into the rt design at system level. It simply means that you
would need to put your hard-rt tasks in the range of 50<prio<99. 

The actual value is quite irrelevant aslong as it is well defined, and 
leaves a range free above suitably large for a rt task set (if you need
more than 5 distinct priorities for a rt task-set it generally means you
are using implicid locking any way)

I dont see the utility of adding a further interface that provides the same
level of configuration that the current user space tools do, and having one
interface for interrupt threads and one for tasks does not really simplify
things - from a rt-task-set perspective the differentiation betwen rt-tasks
and rt-related interrupt threads does not really make sens so keeping the
management in one interface seems more resonable to me.

hofrat 

Re: Changing Kernel thread priorities

[Nicholas Mc Guire]

On Tue, 07 Jun 2011, Thomas Gleixner wrote:

> On Tue, 7 Jun 2011, Peter Zijlstra wrote:
> > On Tue, 2011-06-07 at 13:02 +0200, Remy Bohmer wrote:
> > > Well, I 100% agree that it must be under full userspace control to be
> > > able to set the priorities. But, the kernel default assumption of
> > > starting everything at 50 is wrong as well.
> > > Imagine the following situation:
> > > * Realtime application is running and has threads active in the range
> > > of prios 20 - 90.
> > > * Now bring up a network device, it immediately starts spamming the
> > > system at prio 50 _before_ you have the chance to set it below 20 by
> > > means of chrt.
> > > * RT behaviour is gone!
> > 
> > Good point I guess, Thomas should we default to 1 for everything?
> 
> No objections.

I think that splitting the range makes more sense - you are now potentially
reverting the argument brought up before.

"My drivers behave properly until I load any RT-task"

hofrat

Re: Changing Kernel thread priorities

[Remy Bohmer]

Hi,

>> So, in that case and in many other hotplug cases, you ruin the RT
>> behaviour of the system just by the
>> default-50-is-probably-right-assumption of the kernel.
>> For systems where you have everything under control as a user/system
>> designer, hotplug can also be under control as well.
>>
>
> I dont't quite see that - the 50 default is well dokumented so you can
> plan it into the rt design at system level. It simply means that you
> would need to put your hard-rt tasks in the range of 50<prio<99.

This is another assumption as well. You assume here that any task
bound to certain latencies must be above _all_ interrupt handlers, and
tasks not bound to certain latencies must be set below _all_ interrupt
handlers.
In fact, in real life,  the prioritisation will be much more fine
grained during runtime. One can have a RT task that depends on the
networking stack, but not on mass-storage. In that case you want to
move them out of the range of 50.
The whole idea about threaded interrupts is, is that you can give them
priorities such that they suit your specific application. It is
therefore common practice that priorities does not stay at 49/50.

In real life you may want, for EXAMPLE, this setup:
* prio 70: high priority motor control loop
* prio 60: network device irq
* prio 59: network softirqs
* prio 55: some realtime task depending on networkingstack
* prio 54: mass storage irq
* prio 53: block device softirq
* prio 52: some realtime task depending on mass-storage
* prio 50: all remaining irq threads
* prio 49: all remaining softirqs

Assume here you do a ifconfig down and ifconfig up, in the current
kernel behaviour you will see that the irq thread switches from prio
60 to 50.
The irq-thread will become of a lower priority compared to its related
softirqs due to this reason, which can result in a complete die of
this network interface... even before it ever came back up again...

As mentioned before by Thomas, the configuration is a policy issue and
must be set from user-context. I understand what he means by that and
I agree, but there still has to be a mechanism to make the kernel
remember the configuration set by the user to prevent all kinds of
race conditions. You cannot demand from the user to run after
executing a shell command like ifconfig or modprobe to run some sort
of init-script that repairs what the kernel assumed wrong. The wrong
assumptions the kernel does results in: deadlocks, priority inversion
issues between irq-threads and softirqs and realtime behaviour impact.
Even UDEV cannot_fix_this_problem_ since it runs _after_ the kernel
has set the wrong priorities of the irq threads and the problem it
imposes already may have occurred.

Setting the priorities right once is already complicated enough, it
makes it far more complex if all kinds of race conditions and
limitations need to be taken into account due to this mentioned
auto-reset-to-50(-or-1)-assumptions of the kernel...
True, there must be a safe default for booting, and 49/50 is good
enough for that.

So, you might disagree the way our patches to solve this problem are
implemented, I can buy that, it is just at the level of 'works-for-me'
. Since this discussion frequently appears back on the mailinglist
makes clear that this _is_ an issue that is relevant. Instead of
ignoring or denying this issue, we should figure out what is the
_best_way_ to solve it.
I hope to see great and constructive suggestions soon on this list, I
am very willing to implement it :-)

Kind regards,

Remy

Re: Changing Kernel thread priorities

[Thomas Gleixner]

On Wed, 8 Jun 2011, Remy Bohmer wrote:
> In real life you may want, for EXAMPLE, this setup:
> * prio 70: high priority motor control loop
> * prio 60: network device irq
> * prio 59: network softirqs
> * prio 55: some realtime task depending on networkingstack
> * prio 54: mass storage irq
> * prio 53: block device softirq
> * prio 52: some realtime task depending on mass-storage
> * prio 50: all remaining irq threads
> * prio 49: all remaining softirqs
> 
> Assume here you do a ifconfig down and ifconfig up, in the current
> kernel behaviour you will see that the irq thread switches from prio
> 60 to 50.
> The irq-thread will become of a lower priority compared to its related
> softirqs due to this reason, which can result in a complete die of
> this network interface... even before it ever came back up again...

Not really. If that's the case it needs to be investigated and
fixed.

> As mentioned before by Thomas, the configuration is a policy issue and
> must be set from user-context. I understand what he means by that and
> I agree, but there still has to be a mechanism to make the kernel
> remember the configuration set by the user to prevent all kinds of
> race conditions. You cannot demand from the user to run after

Which race conditions?

> executing a shell command like ifconfig or modprobe to run some sort
> of init-script that repairs what the kernel assumed wrong. The wrong
> assumptions the kernel does results in: deadlocks, priority inversion
> issues between irq-threads and softirqs and realtime behaviour impact.

If you do an ifdown/up then your prio 55 task is totally irrelevant
until the interface is back to full operation again, which includes
setting the priority right.

There is another gotcha with your approach. It only ever works when
the interrupt descriptors are static and not dynamically
allocated/freed. If they are fully dynamic then you have no
possibility to store the prio information after a full teardown of a
device.

So moving the base priority down to 1 or 2 is probably the most
sensible solution to avoid that a newly brought up interrupt thread
interferes with anything in the rt domain and it's not rocket science
to adjust the priority in a ifup.post or with an udev rule.

Thanks,

	tglx

Re: Changing Kernel thread priorities

[Nicholas Mc Guire]

On Wed, 08 Jun 2011, Remy Bohmer wrote:

> Hi,
> 
> >> So, in that case and in many other hotplug cases, you ruin the RT
> >> behaviour of the system just by the
> >> default-50-is-probably-right-assumption of the kernel.
> >> For systems where you have everything under control as a user/system
> >> designer, hotplug can also be under control as well.
> >>
> >
> > I dont't quite see that - the 50 default is well dokumented so you can
> > plan it into the rt design at system level. It simply means that you
> > would need to put your hard-rt tasks in the range of 50<prio<99.
> 
> This is another assumption as well. You assume here that any task
> bound to certain latencies must be above _all_ interrupt handlers, and
> tasks not bound to certain latencies must be set below _all_ interrupt
> handlers.
> In fact, in real life,  the prioritisation will be much more fine
> grained during runtime. One can have a RT task that depends on the
> networking stack, but not on mass-storage. In that case you want to
> move them out of the range of 50.

No implications are made on the "correct" priority - but as there is no
correct priority setting a well defined default and leaving it to the
user is the most resonable solution. The main issue with not changing it
from the current value is simply that it may change behavior of existing
systems that update and that is, if it provides no obvious benifit, not
a good thing to happen.

> The whole idea about threaded interrupts is, is that you can give them
> priorities such that they suit your specific application. It is
> therefore common practice that priorities does not stay at 49/50.
> 
> In real life you may want, for EXAMPLE, this setup:
> * prio 70: high priority motor control loop
> * prio 60: network device irq
> * prio 59: network softirqs
> * prio 55: some realtime task depending on networkingstack
> * prio 54: mass storage irq
> * prio 53: block device softirq
> * prio 52: some realtime task depending on mass-storage
> * prio 50: all remaining irq threads
> * prio 49: all remaining softirqs
> 
> Assume here you do a ifconfig down and ifconfig up, in the current
> kernel behaviour you will see that the irq thread switches from prio
> 60 to 50.
> The irq-thread will become of a lower priority compared to its related
> softirqs due to this reason, which can result in a complete die of
> this network interface... even before it ever came back up again...
> 
> As mentioned before by Thomas, the configuration is a policy issue and
> must be set from user-context. I understand what he means by that and
> I agree, but there still has to be a mechanism to make the kernel
> remember the configuration set by the user to prevent all kinds of
> race conditions. You cannot demand from the user to run after
> executing a shell command like ifconfig or modprobe to run some sort
> of init-script that repairs what the kernel assumed wrong. The wrong
> assumptions the kernel does results in: deadlocks, priority inversion
> issues between irq-threads and softirqs and realtime behaviour impact.

but you equally can not assume that a device comming back on-line always
should have the same priority. i.e. DAQ-card is opened by process A prioX
and then closed to be reopened by procsee B prio Y - the priority of the
DAQ-card IRQ  thread is not related and persistence here is dependant on 
the particular circumstances.

> Even UDEV cannot_fix_this_problem_ since it runs _after_ the kernel
> has set the wrong priorities of the irq threads and the problem it
> imposes already may have occurred.
> 
> Setting the priorities right once is already complicated enough, it
> makes it far more complex if all kinds of race conditions and
> limitations need to be taken into account due to this mentioned
> auto-reset-to-50(-or-1)-assumptions of the kernel...
> True, there must be a safe default for booting, and 49/50 is good
> enough for that.
> 
> So, you might disagree the way our patches to solve this problem are
> implemented, I can buy that, it is just at the level of 'works-for-me'
> . Since this discussion frequently appears back on the mailinglist
> makes clear that this _is_ an issue that is relevant. Instead of
> ignoring or denying this issue, we should figure out what is the
> _best_way_ to solve it.
> I hope to see great and constructive suggestions soon on this list, I
> am very willing to implement it :-)
>

so maybe a comlet set of requirements is needed first rather than ad-hoc
solutions for special cases - Im quite sure the non-persistant case is only
one of a few issues one can find here.

hofrat

Re: Changing Kernel thread priorities

[Monica Puig-Pey]

El 07/06/11 20:55, Mark Hounschell escribió:
> On 06/07/2011 02:34 PM, Monica Puig-Pey wrote:
>> El 07/06/11 11:46, Mark Hounschell escribió:
>>> On 06/07/2011 05:14 AM, Mark Hounschell wrote:
>>>> On 06/07/2011 04:40 AM, Monica Puig-Pey wrote:
>>>>> El 06/06/11 18:49, Mark Hounschell escribió:
>>>>>> On 06/06/2011 07:58 AM, Monica Puig-Pey wrote:
>>>>>>> El 06/06/11 13:54, Rolando Martins escribió:
>>>>>>>> Hi,
>>>>>>>> I use the following:
>>>>>>>>
>>>>>>>> PIDs=$(ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq" | awk '{
>>>>>>>> print $1; }' | xargs echo)
>>>>>>>> for i in $PIDs
>>>>>>>> do
>>>>>>>> ret=$(chrt -f -p 99 $i)
>>>>>>>> done
>>>>>>>>
>>>>>>>> This will change the kernel thread associated with an irq
>>>>>>>> handler to
>>>>>>>> RT FIFO prio 99.
>>>>>>>> Just change the script to your specific interrupt.
>>>>>>>>
>>>>>>>> Hope it helps,
>>>>>>>> Rolando
>>>>>>>>
>>>>>>>> On Mon, Jun 6, 2011 at 12:47 PM, Monica Puig-Pey
>>>>>>>> wrote:
>>>>>>>>> I am writing a driver which has one kernel thread associated with
>>>>>>>>> it.
>>>>>>>>> I want to change the priority of this thread, so that I can
>>>>>>>>> specify the
>>>>>>>>> order in which it is scheduled following an interrupt.
>>>>>>>>> I'm using:
>>>>>>>>>
>>>>>>>>> sched_setscheduler(struct task_struct *, int, struct sched_param
>>>>>>>>> *);
>>>>>>>>>
>>>>>>>>> but it doesn't work. I tried to change the priority from the
>>>>>>>>> init_module,
>>>>>>>>> and also from the Kernel Thread, but there is no way.
>>>>>>>>>
>>>>>>>>> Kernel version is 2.6.31-11-rt
>>>>>>>>>
>>>>>>>>> What do I call to change a kernel thread priority?
>>>>>>>>>
>>>>>>>>> Thanks you very much
>>>>>>>>>
>>>>>>>>> Mónica
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>>>> linux-rt-users" in
>>>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>>>>
>>>>>>>
>>>>>>> I need to change the priority from inside the driver, when creating
>>>>>>> the
>>>>>>> kernel thread.
>>>>>>> Your script is useful but it is done in user context,
>>>>>>> Any other help please?
>>>>>>
>>>>>> What I do is record the PID of the thread in the driver, then
>>>>>> create an
>>>>>> IOCTL for your driver that user land can call that either returns the
>>>>>> PID so you can do it in user land, or cause the IOCTL code to do
>>>>>> it in
>>>>>> the driver.
>>>>>>
>>>>>> The same can be done with the affinity of the IRQ if you record the
>>>>>> IRQ
>>>>>> number.
>>>>>>
>>>>>> Mark
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>> linux-rt-users" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>> But I don't have de PID of my Kthread, I only have the task_struc *
>>>>> that
>>>>> gives me the function:
>>>>>
>>>>> struct task_struct *kthread_create(int (*threadfn)(void *data),
>>>>> void *data,
>>>>> const char namefmt[], ...)
>>>>>
>>>>> How could I get the PID, and which function should I use in the IOCTL
>>>>> (kernel context) for changing its priority?
>>>>>
>>>>
>>>> The PID can be obtained from within the interrupt handler its self via
>>>> current->pid.
>>>> Obviously an interrupt has to occur first but after one interrupt you
>>>> have it.
>>>>
>>>> Actually I had forgot how I handled this. Where I change the RT
>>>> priority
>>>> and cpu affinity is in what used to be called the Bottom Half and the
>>>> IOCTL
>>>> referred to above simply tells the BH to do it and with what values.
>>>>
>>>
>>> In interrupt handler code snippet:
>>>
>>> struct task_struct *TSK;
>>> struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };
>>>
>>> TSK = current;
>>>
>>> Code snippet from BH:
>>>
>>> if (((rtom_rtprio != 0) &&
>>> (rtom_rtprio != PARAM.sched_priority)) ||
>>> (my_rtom_rtprio[BOARD] != rtom_rtprio)) {
>>>
>>> PARAM.sched_priority = rtom_rtprio;
>>> my_rtom_rtprio[COUNT] = rtom_rtprio;
>>> sched_setscheduler(TSK, SCHED_FIFO, &PARAM);
>>>
>>> set_cpus_allowed(TSK, rtom_devices[BOARD].irq_cpu_mask);
>>> rtom_devices[BOARD].irq_task_pid = TSK->pid;
>>> }
>>>
>>> rtom_rtprio and irq_cpu_mask are set by userland via an IOCTL. An
>>> interrupt must occur for this to happen and my BOARD never shares IRQs.
>>>
>>> Mark
>>>
>>
>> Thanks, your idea seems to be very useful to me. I was doing it very
>> similar, but I didn't use the "current" variable to know the
>> task_struct *.
>>
>> I have tried your suggestion on an easy example, but it didn't work.
>> Insmod returns through the kernel
>>
>> [11334.895499] kthread: Unknown symbol sched_setscheduler
>>
>> Code shown below:
>>
>> #include <linux/module.h>
>> #include <linux/kernel.h>
>> #include <linux/ioport.h>
>>
>> #include <linux/wait.h>
>> #include <linux/kthread.h>
>> #include <asm/io.h>
>> #include <linux/sched.h>
>>
>>
>> struct task_struct *ts;
>>
>> int thread(void *data)
>> {
>> struct task_struct *TSK;
>> struct sched_param PARAM = {.sched_priority = MAX_RT_PRIO };
>> TSK = current;
>>
>> PARAM.sched_priority = 50;
>> sched_setscheduler(TSK, SCHED_FIFO, &PARAM); // <-- unknown symbol??
>>
>> while(1){
>> printk("Hi I am kernel thread!\n");
>> msleep(100);
>> if (kthread_should_stop())
>> break;
>> }
>> return 0;
>> }
>>
>>
>> int init_module(void)
>> {
>> printk(KERN_INFO "init_module() called\n");
>> ts=kthread_run(thread,NULL,"kthread");
>> return 0;
>> }
>>
>> void cleanup_module(void)
>> {
>> printk(KERN_INFO "cleanup_module() called\n");
>> kthread_stop(ts);
>> }
>> .
>>
>
> In kernel/sched.c
>
> EXPORT_SYMBOL_GPL(sched_setscheduler);
>
> If your driver is not GPL, you can't use it.
>
> Mark

I did it, it worked!!! thank you so much!!!!! :)
-- 
__________________________________________________________________________________

Mónica Puig-Pey González       E-mail: puigpeym@unican.es

Grupo de Computadores y Tiempo Real, Departamento de Electrónica y 
Computadores.
Facultad de Ciencias - Universidad de Cantabria
Av. de los Castros s/n. 39005 - Santander, España
__________________________________________________________________________________
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Changing Kernel thread priorities

[Remy Bohmer]

Hi Thomas,

2011/6/8 Thomas Gleixner <tglx@linutronix.de>:
> On Wed, 8 Jun 2011, Remy Bohmer wrote:
>> In real life you may want, for EXAMPLE, this setup:
>> * prio 70: high priority motor control loop
>> * prio 60: network device irq
>> * prio 59: network softirqs
>> * prio 55: some realtime task depending on networkingstack
>> * prio 54: mass storage irq
>> * prio 53: block device softirq
>> * prio 52: some realtime task depending on mass-storage
>> * prio 50: all remaining irq threads
>> * prio 49: all remaining softirqs
>>
>> Assume here you do a ifconfig down and ifconfig up, in the current
>> kernel behaviour you will see that the irq thread switches from prio
>> 60 to 50.
>> The irq-thread will become of a lower priority compared to its related
>> softirqs due to this reason, which can result in a complete die of
>> this network interface... even before it ever came back up again...
>
> Not really. If that's the case it needs to be investigated and
> fixed.

I, of course, agree with that, but these cases are usually extremely
hard to find, and occur typically only in the once-a-month-condition
that you cannot reproduce...
Do you remember why the priority of the softirqs was moved down from
50 to 49 ? IIRC this was because of the very same reason and IIRC
still valid

We do not have control over all kernel code, and new drivers are
continuously being developed that make wrong implicit assumptions
about the order of irq->sirq->everything else. Of course this is
wrong, and there is no excuse, but it is a fact of life...
In practice the softirq prio can be set to a higher value than 50 (or
1), and a hirq thread that is started at 50 (or 2) will result in
situations that are not expected.

>> As mentioned before by Thomas, the configuration is a policy issue and
>> must be set from user-context. I understand what he means by that and
>> I agree, but there still has to be a mechanism to make the kernel
>> remember the configuration set by the user to prevent all kinds of
>> race conditions. You cannot demand from the user to run after
>
> Which race conditions?

Race conditions that occur when a softirq preempts a related hardirq
what the driver did not expect or was designed for.

>> executing a shell command like ifconfig or modprobe to run some sort
>> of init-script that repairs what the kernel assumed wrong. The wrong
>> assumptions the kernel does results in: deadlocks, priority inversion
>> issues between irq-threads and softirqs and realtime behaviour impact.
>
> If you do an ifdown/up then your prio 55 task is totally irrelevant
> until the interface is back to full operation again, which includes
> setting the priority right.

I already expected that remark after I pressed the send button of that
mail... This was just meant as an example, in which you can probably
shoot more holes in. It is not about the example, it is about the
essence of what I am trying to explain here.

> There is another gotcha with your approach. It only ever works when
> the interrupt descriptors are static and not dynamically
> allocated/freed. If they are fully dynamic then you have no
> possibility to store the prio information after a full teardown of a
> device.

It depends how it is being implemented.
A mechanism to specify the policy does not mean everything has to be
already in place the policy is about at the moment you specify the
policy.
In other words, a policy may describe situations that are going to
happen in the future, not necessarily situations that are actual now.

For example, something like this:
* A user specifies a table with policy information about what each
interrupt handler in the system should do when they are being created.
* When the interrupt handler is being installed, it is looked up in
the table at what priority and scheduling policy it needs to run. If
not specified, go for a default.
* Additionally: When the table is being updated, the already running
threads can being adjusted to the new policy.

> So moving the base priority down to 1 or 2 is probably the most
> sensible solution to avoid that a newly brought up interrupt thread
> interferes with anything in the rt domain and it's not rocket science
> to adjust the priority in a ifup.post or with an udev rule.

At prio 1 or 2, _every_ RT-thread in the system is to be assumed to be
more low-latency bound compared to _any_ interrupt handler. And you
assume here that no user RT-thread in the system shall use any
functionality of any driver that has an interrupt handler (otherwise
you get the priority inversions issue)
As mentioned in this thread before by someone else, you will get this
old issue back: 'My drivers start to behave weird when I create a
RT-thread...'
The prio inversion issue between hirq/sirq will even become more
worse, since there will be a smaller chance that softirqs will stay at
prio 1 and thus there is less guarantee that they will stay below the
hirq-prio all the time.

Furthermore, I prefer the principle: _Nothing_ goes above interrupt
(thread) priority unless there is a very special reason for it and it
has been investigated that it is safe to do so. And a user-thread that
requires functionality of a certain driver shall be set below the
priority of the hirq-thread of that driver. The prio of the softirq
must _always_ be between that user-thread and hirq-thread if there is
a relation between the driver and softirq.

In that light I think prio 1/2 is more worse compared to 49/50. I
think the current _default_ is okay, it makes the system at least
boot.

Kind regards,

Remy

Re: Changing Kernel thread priorities

[Thomas Gleixner]

On Fri, 10 Jun 2011, Remy Bohmer wrote:
> 2011/6/8 Thomas Gleixner <tglx@linutronix.de>:
> > On Wed, 8 Jun 2011, Remy Bohmer wrote:
> >> In real life you may want, for EXAMPLE, this setup:
> >> * prio 70: high priority motor control loop
> >> * prio 60: network device irq
> >> * prio 59: network softirqs
> >> * prio 55: some realtime task depending on networkingstack
> >> * prio 54: mass storage irq
> >> * prio 53: block device softirq
> >> * prio 52: some realtime task depending on mass-storage
> >> * prio 50: all remaining irq threads
> >> * prio 49: all remaining softirqs
> >>
> >> Assume here you do a ifconfig down and ifconfig up, in the current
> >> kernel behaviour you will see that the irq thread switches from prio
> >> 60 to 50.
> >> The irq-thread will become of a lower priority compared to its related
> >> softirqs due to this reason, which can result in a complete die of
> >> this network interface... even before it ever came back up again...
> >
> > Not really. If that's the case it needs to be investigated and
> > fixed.
> 
> I, of course, agree with that, but these cases are usually extremely
> hard to find, and occur typically only in the once-a-month-condition
> that you cannot reproduce...
> Do you remember why the priority of the softirqs was moved down from
> 50 to 49 ? IIRC this was because of the very same reason and IIRC
> still valid

No, it's not. The root cause was a problem with the network softirq
and a network driver, the softirq ->49 was a temporary workaround
until we had enough information to find the real root cause. I wish
I'd never committed that change at all.

> We do not have control over all kernel code, and new drivers are
> continuously being developed that make wrong implicit assumptions
> about the order of irq->sirq->everything else. Of course this is
> wrong, and there is no excuse, but it is a fact of life...
> In practice the softirq prio can be set to a higher value than 50 (or
> 1), and a hirq thread that is started at 50 (or 2) will result in
> situations that are not expected.
> 
> >> As mentioned before by Thomas, the configuration is a policy issue and
> >> must be set from user-context. I understand what he means by that and
> >> I agree, but there still has to be a mechanism to make the kernel
> >> remember the configuration set by the user to prevent all kinds of
> >> race conditions. You cannot demand from the user to run after
> >
> > Which race conditions?
> 
> Race conditions that occur when a softirq preempts a related hardirq
> what the driver did not expect or was designed for.

And making it the other way round hides the problem, which is even
worse. We want stuff to explode right away. You can run into the same
problem when the softirq holds a lock and the high prio irq thread
boosts it.

> > So moving the base priority down to 1 or 2 is probably the most
> > sensible solution to avoid that a newly brought up interrupt thread
> > interferes with anything in the rt domain and it's not rocket science
> > to adjust the priority in a ifup.post or with an udev rule.
> 
> At prio 1 or 2, _every_ RT-thread in the system is to be assumed to be
> more low-latency bound compared to _any_ interrupt handler. And you
> assume here that no user RT-thread in the system shall use any
> functionality of any driver that has an interrupt handler (otherwise
> you get the priority inversions issue)

Sigh. People who use RT threads should better know what they do and
configure their damned system correct. We cannot provide a solution
which takes every incarnatation of lusers into account.

> As mentioned in this thread before by someone else, you will get this
> old issue back: 'My drivers start to behave weird when I create a
> RT-thread...'

And I do not care at all. The answer is: Do not use an RT-thread when
you are not knowing what you are doing.

> The prio inversion issue between hirq/sirq will even become more
> worse, since there will be a smaller chance that softirqs will stay at
> prio 1 and thus there is less guarantee that they will stay below the
> hirq-prio all the time.

There is no such thing and if it's there, then it needs to be found
and fixed.

> Furthermore, I prefer the principle: _Nothing_ goes above interrupt
> (thread) priority unless there is a very special reason for it and it
> has been investigated that it is safe to do so. And a user-thread that
> requires functionality of a certain driver shall be set below the
> priority of the hirq-thread of that driver. The prio of the softirq
> must _always_ be between that user-thread and hirq-thread if there is
> a relation between the driver and softirq.
> 
> In that light I think prio 1/2 is more worse compared to 49/50. I
> think the current _default_ is okay, it makes the system at least
> boot.

It boots with 50 or whatever you set it to as well.

Thanks,

	tglx

Re: Changing Kernel thread priorities

[Remy Bohmer]

Hi,

> No, it's not. The root cause was a problem with the network softirq
> and a network driver, the softirq ->49 was a temporary workaround
> until we had enough information to find the real root cause. I wish
> I'd never committed that change at all.

Clear. Did not know it was already solved. I thought it was still an
issue. This changes things :-)

>> Race conditions that occur when a softirq preempts a related hardirq
>> what the driver did not expect or was designed for.
>
> And making it the other way round hides the problem, which is even
> worse. We want stuff to explode right away.

100% Agreed

> You can run into the same
> problem when the softirq holds a lock and the high prio irq thread
> boosts it.

OK.

Thanks for the explanation. I see no reason any more why setting the
prios default to 1 would be a bad thing.
The rest of the configuration in that case can then indeed done be
done by udev and other userland friends.

Kind regards,

Remy