Re: 2.6.18-rt7 on HP Z800

[Nivedita Singhvi <niv@us.ibm.com>]

On 08/05/2010 01:09 AM, Simpson, John (UK) (Contractor) wrote:
>
> I've had no response to this question. Can anyone help? I'm also baffled
> as to why the smp_affinity stuff doesn't work.
>
> -----Original Message-----
> From: linux-rt-users-owner@vger.kernel.org
> [mailto:linux-rt-users-owner@vger.kernel.org] On Behalf Of Simpson, John
> (UK) (Contractor)
> Sent: 04 August 2010 09:30
> To: linux-rt-users@vger.kernel.org
> Subject: 2.6.18-rt7 on HP Z800
>
>
>                 *** WARNING ***
>
> This mail has originated outside your organization, either from an
> external partner or the Global Internet.
>       Keep this in mind if you answer this message.
>
> As previously posted, I'm using 2.6.18 for compatibility with third
> party drivers for an FPGA card. This a scene-generation app using an
> Nvidia Quadro FX5800 requiring a high-frame rate with no dropped frames;
> hence the need for RT (we've had previous success using proprietary
> stuff but thought we'd give this a go).
>
> It's working fine (with very occasional crashes, most of which seem to
> have been referred to in later RT kernel fixes, but we can put up with
> it).
>
> Some notes for posterity:
>
>      Turn off irqbalance service
>      Deselect power management stuff in kernel
>      Select SATA IDE emulation in BIOS (otherwise no SCSI/CD stuff)
>      Rebuilt Broadcom tg3 and set pci=nomsi for network drivers to
> function
>      isolcpus used on boot
>      App run on isolated cpus using 'taskset'
>
> Like I said, it works fine, I haven't seen any dropped frames yet, but
> there's stuff I don't understand from using other RT OSs...
>
>      All interrupts run on CPU0, all the time, there's no requirement to
> use smp_affinity, in fact if you do the whole thing grinds to a halt.  I
> guess this is fine.
>      The isolcpus option seems to work fine except for the following
> running on all eight CPUs:
>          migration
>          posix_cpu_timer
>          softirq-high
>          softirq-timer
>          softirq-net-tx
>          softirq-net-rx
>          softirq-block
>          softirq-tasklet
>          softirq-hrtimer
>          softirq-rcu
>          watchdog
>          desched
>          aio
>          cqueue
>          kblockd
>          events
>
>      I guess this is preemptible stuff which must run per cpu, but
> there's a bit in a Red Hat page
> http://rt.et.redhat.com/wiki/index.php?title=RHEL-RT_AffinityHowto&print
> able=yes (starting "In MRG Realtime") which apparently refers to these
> elements in their RT kernel suggesting that these should be dealt with
> (but then doesn't seem to say how).  Do I need to do anything about
> these processes?

 From the above, everything is working and you are not actually having
an issue, but your only question is whether that latter list of tasks
is something you need to do something about? If that's teh case, my
answer would be no -- those are per-cpu activities which are monitoring
things on that cpu.

What the RH page is referring to is binding hard interrupts to certain
CPUs (by echoing a bitmask selecting that CPU/ or a set of CPUs into the
/proc/irq/#/smp_affinity field. Also pin your applications to appropriate
CPUs using the taskset utility as described on that page.

All the above really underlines is the need to architect your real-time
solution from the top-down, and take the entire system into account, to
help meet real-time guarantees.

Hope that helps...

thanks,
Nivedita