Re: [patch] x86, apic: use 0x20 for the IRQ_MOVE_CLEANUP_VECTOR instead of 0x1f

[ebiederm@xmission.com (Eric W. Biederman)]

"H. Peter Anvin" <hpa@zytor.com> writes:

> On 01/11/2010 04:28 PM, Eric W. Biederman wrote:
>> 
>> Sorry.  I suck at multitasking.
>> 
>> Without changes assign_irq_vector will reuse vectors in the range
>> IRQ0_VECTOR to IRQ15_VECTOR in the code as it we currently ship it,
>> when we switch irq0-15 into ioapic mode.
>> 
>> Switching the loop to cover IRQ0_VECTOR to IRQ15_VECTOR is not a
>> problem.  I don't think it will find anything free as we assign those
>> vectors on all cpus, but the data structures are fine.
>> 
>> I am uncomfortable with the suggestion of sharing the priority of the
>> IRQ_MOVE_CLEANUP_VECTOR with other interrupts.  I know if it had be
>> clear from the documentation that it was safe to share the irq level
>> with other interrupts I would not have reserved the entire interrupt
>> level for the IRQ_MOVE_CLEANUP_VECTOR.
>> 
>
> What are the properties that you're looking for, and in what
> documentation?  We reviewed the Software Development Manual here at
> Intel, and it rather explicitly states:
>
> "Each interrupt priority level (sometimes interpreted by software as an
> interrupt priority class) encompasses 16 vectors. Prioritizing
> interrupts within a priority level is determined by the vector number.
> The higher the vector number, the higher the priority within that
> priority level.  In determining the priority of a vector and ranking
> of vectors within a priority group, the vector number is often divided
> into two parts, with the high 4 bits of the vector indicating its
> priority and the low 4 bit indicating its ranking within the priority
> group."
>
> [Intel® 64 and IA-32 Architectures Software Developer’s Manual
> Volume 3A: System Programming Guide, Part 1; September 2009, Order
> Number 253668-032US; section 10.9.3, page 10-57f.]
>
> So 0x20 is the lowest-priority vector within priority group 0x2.

After having the documentation quoted at me.  I am having a distinct
memory of one piece of documentation saying:
"interrupts within a priority level can be delivered in any order"

So I am guessing there is not any ordering of interrupts in the same
priority level until they get to the local apic.

What guarantee we need is the interesting question.

The cleanup ipi is sent when we have seen an interrupt arrive at it's
newly configured location.  It is possible that there is still an
interrupt in flight to the old configured location (think NUMA where
the interrupt has been migrated from off node to on node).  We want
the guarantee that the ipi arrives after the inflight irq.  Which
means on the wire ordering as well as in the local apic ordering is
interesting.

I am slammed with other stuff right now so I don't think I will have
time to find the old documentation I was looking at for a couple of
more days.

Eric