Re: [patch 08/16] genirq: Introduce callback irq_chip.irq_write_msi_msg

["Yun Wu (Abel)" <wuyun.wu@huawei.com>]

On 2014/11/18 22:29, Jiang Liu wrote:

> 
> 
> On 2014/11/18 22:22, Yun Wu (Abel) wrote:
>> On 2014/11/18 22:03, Jiang Liu wrote:
>>
>>> On 2014/11/18 21:52, Yun Wu (Abel) wrote:
>>>> On 2014/11/18 21:43, Jiang Liu wrote:
>>>>
>>>>> On 2014/11/18 21:33, Yun Wu (Abel) wrote:
>>>>>> On 2014/11/18 18:19, Thomas Gleixner wrote:
>>>>>>
>>>>>>> On Tue, 18 Nov 2014, Yun Wu (Abel) wrote:
>>>>>>>> On 2014/11/12 21:43, Thomas Gleixner wrote:
>>>>>>>>>  struct irq_chip {
>>>>>>>>> @@ -359,6 +360,7 @@ struct irq_chip {
>>>>>>>>>  	void		(*irq_release_resources)(struct irq_data *data);
>>>>>>>>>  
>>>>>>>>>  	void		(*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>>> +	void		(*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
>>>>>>>>
>>>>>>>> Hmm... It's really weird.
>>>>>>>> I don't think it's the interrupt controllers' responsibility to write messages
>>>>>>>> for all the endpoint devices since the methods of configuring message registers
>>>>>>>> may different between these devices. And theoretically, the endpoint devices
>>>>>>>> themselves should take the responsibility to configure their message registers.
>>>>>>>> To say the least, the write_msg callback here still need to call some certain
>>>>>>>> interfaces provided by the corresponding device.
>>>>>>>>
>>>>>>>> There would be lots of ARM new devices capable of sending message
>>>>>>>> based interrupts to interrupt controllers, does all the drivers of
>>>>>>>> the devices need to expose a write_msg callback to interrupt
>>>>>>>> controllers?
>>>>>>>
>>>>>>> Well, writing the message _IS_ part of the interrupt controller.
>>>>>>>
>>>>>>> So in order to enable non PCI based MSI we want to have generic
>>>>>>> infrastructure with minimal per device/device class callbacks and of
>>>>>>> course you need to provide that callback for your special device.
>>>>>>>
>>>>>>> We already have non PCI based MSI controllers in x86 today and we need
>>>>>>> to handle the whole stuff with tons of copied coded extra for each of
>>>>>>> those. So consolidating it into common infrastructure allows us to get
>>>>>>> rid of the pointless copied code and reduce the per device effort to
>>>>>>> the relevant hardware specific callbacks. irq_write_msi_msg being one
>>>>>>> of those.
>>>>>>>
>>>>>>
>>>>>> At least, we have the same goal.
>>>>>> I will illustrate my thoughts by an example.
>>>>>> The current code is something like:
>>>>>>
>>>>>> Device A
>>>>>> ========
>>>>>> void A_write_msg() { ... }
>>>>>>
>>>>>> Group B
>>>>>> (a group of devices behave same on writing messages, i.e. PCI)
>>>>>> =======
>>>>>> void B_write_msg() { ... }
>>>>>>
>>>>>> Controller
>>>>>> ==========
>>>>>> irq_chip.irq_write_msi_msg () {
>>>>>> 	if (A)
>>>>>> 		A_write_msg();
>>>>>> 	if (B)
>>>>>> 		B_write_msg();
>>>>>> }
>>>>>>
>>>>>> It's horrible when new devices come out, since we need to modify the
>>>>>> controller part for each new device.
>>>>>> What I suggested is:
>>>>>>
>>>>>> MSI Core
>>>>>> ========
>>>>>> struct msi_ops { .write_msg, };
>>>>>> struct msi_desc { .msi_ops, };
>>>>>>
>>>>>> write_msg() {
>>>>>> 	X = get_dev();
>>>>>> 	irq_chip.compose_msg(X);	// IRQ chips' responsibility
>>>>>> 	X_msi_ops.write_msg();		// nothing to do with IRQ chips
>>>>>> }
>>>>>>
>>>>>> Device A
>>>>>> ========
>>>>>> void A_write_msg() { ... }
>>>>>> A_msi_ops.write_msg = A_write_msg;
>>>>>>
>>>>>> Group B
>>>>>> =======
>>>>>> void B_write_msg() { ... }
>>>>>> B_msi_ops.write_msg = B_write_msg;
>>>>>>
>>>>>> Please correct me if I misunderstood anything.
>>>>> Hi Yun,
>>>>> 	We provide an irq_chip for each type of interrupt controller
>>>>> instead of devices. For the example mentioned above, if device A
>>>>> and Group B has different interrupt controllers, we just need to
>>>>> implement irq_chip_A and irq_chip_B and set irq_chip.irq_write_msi_msg()
>>>>> to suitable callbacks.
>>>>> 	The framework already achieves what you you want:)
>>>>
>>>> What if device A and group B have the same interrupt controller?
>>> Device doesn't care about interrupt controllers, it just cares about
>>> interrupts used by itself. It's the interrupt source (controller)
>>> enumerators' responsibility to discover interrupt source, associate
>>> methods to control the interrupt source and assign irq numbers for
>>> interrupt sources.
>>
>> Yes, indeed.
>>
>>> There are two ways to associate irq numbers with device:
>>> 1) arch code/bus drivers statically assigns irq number for devices
>>>    when creating device objects, such as PCI legacy interrupt
>>>    (INTA-INTD), IOAPIC interrupts.
>>
>> And OF interfaces.
>>
>>> 2) device drivers ask interrupt source enumerators to dynamically
>>>    create irq numbers, such pci_enable_msix_range() and friends.
>>> So device driver definitely doesn't need to known about irq_chip
>>> methods to control interrupt sources.
>>>
>>
>> The above you described is absolutely right, but not the things I want
>> to know. :)
>> Take GICv3 ITS for example, it deals with both PCI and non PCI message
>> interrupts. IIUC, several irq_chips need to be implemented in the ITS
>> driver (i.e. pci_msi_chip, A_msi_chip and B_msi_chip). What should we
>> do to the ITS driver if new MSI-capable devices come out?
> Marc has posted a patchset to enable ITS based on the hierarchy
> irqdomain framework, please refer to "[PATCH 00/15] arm64: PCI/MSI:
> GICv3 ITS support (stacked domain edition)" at
> https://lkml.org/lkml/2014/11/11/620
> 

IIUC, Marc's patch now only supports PCI MSI/MSI-X...

Thanks,
	Abel