RE: [PATCH net-next v8 6/9] net: txgbe: Support GPIO to SFP socket

["Jiawen Wu" <jiawenwu@trustnetic.com>]

Forgot to Cc: Michael Walle

On Friday, May 19, 2023 4:24 PM, Jiawen Wu wrote:
> On Thursday, May 18, 2023 8:49 PM, Andrew Lunn wrote:
> > > > I _think_ you are mixing upstream IRQs and downstream IRQs.
> > > >
> > > > Interrupts are arranged in trees. The CPU itself only has one or two
> > > > interrupts. e.g. for ARM you have FIQ and IRQ. When the CPU gets an
> > > > interrupt, you look in the interrupt controller to see what external
> > > > or internal interrupt triggered the CPU interrupt. And that interrupt
> > > > controller might indicate the interrupt came from another interrupt
> > > > controller. Hence the tree structure. And each node in the tree is
> > > > considered an interrupt domain.
> > > >
> > > > A GPIO controller can also be an interrupt controller. It has an
> > > > upstream interrupt, going to the controller above it. And it has
> > > > downstream interrupts, the GPIO lines coming into it which can cause
> > > > an interrupt. And the GPIO interrupt controller is a domain.
> > > >
> > > > So what exactly does gpio_regmap_config.irq_domain mean? Is it the
> > > > domain of the upstream interrupt controller? Is it an empty domain
> > > > structure to be used by the GPIO interrupt controller? It is very
> > > > unlikely to have anything to do with the SFP devices below it.
> > >
> > > Sorry, since I don't know much about interrupt,  it is difficult to understand
> > > regmap-irq in a short time. There are many questions about regmap-irq.
> > >
> > > When I want to add an IRQ chip for regmap, for the further irq_domain,
> > > I need to pass a parameter of IRQ, and this IRQ will be requested with handler:
> > > regmap_irq_thread(). Which IRQ does it mean?
> >
> > That is your upstream IRQ, the interrupt indicating one of your GPIO
> > lines has changed state.
> >
> > > In the previous code of using
> > > devm_gpiochip_add_data(), I set the MSI-X interrupt as gpio-irq's parent, but
> > > it was used to set chained handler only. Should the parent be this IRQ? I found
> > > the error with irq_free_descs and irq_domain_remove when I remove txgbe.ko.
> >
> > Do you have one MSI-X dedicated for GPIOs. Or is it your general MAC
> > interrupt, and you need to read an interrupt controller register to
> > determine it was GPIOs which triggered the interrupt?
> >
> > If you are getting errors when removing the driver it means you are
> > missing some level of undoing what us done in probe. Are you sure
> > regmap_del_irq_chip() is being called on unload?
> >
> > > As you said, the interrupt of each tree node has its domain. Can I understand
> > > that there are two layer in the interrupt tree for MSI-X and GPIOs, and requesting
> > > them separately is not conflicting? Although I thought so, but after I implement
> > > gpio-regmap, SFP driver even could not find gpio_desc. Maybe I missed something
> > > on registering gpio-regmap...
> >
> > That is probably some sort of naming issue. You might want to add some
> > prints in swnode_find_gpio() and gpiochip_find() to see what it is
> > looking for vs what the name actually is.
> 
> It's true for the problem of name, but there is another problem. SFP driver has
> successfully got gpio_desc, then it failed to get gpio_irq from gpio_desc (with error
> return -517). I traced the function gpiod_to_irq():
> 
> 	gc = desc->gdev->chip;
> 	offset = gpio_chip_hwgpio(desc);
> 	if (gc->to_irq) {
> 		int retirq = gc->to_irq(gc, offset);
> 
> 		/* Zero means NO_IRQ */
> 		if (!retirq)
> 			return -ENXIO;
> 
> 		return retirq;
> 	}
> 
> 'gc->to_irq = gpiochip_to_irq' was set in [4]gpiochip_irqchip_add_domain().
> So:
> 
> 	static int gpiochip_to_irq(struct gpio_chip *gc, unsigned int offset)
> 	{
> 		struct irq_domain *domain = gc->irq.domain;
> 
> 	#ifdef CONFIG_GPIOLIB_IRQCHIP
> 		/*
> 		 * Avoid race condition with other code, which tries to lookup
> 		 * an IRQ before the irqchip has been properly registered,
> 		 * i.e. while gpiochip is still being brought up.
> 		 */
> 		if (!gc->irq.initialized)
> 			return -EPROBE_DEFER;
> 	#endif
> 
> gc->irq.initialized is set to true at the end of [3]gpiochip_add_irqchip() only.
> Firstly, it checks if irqchip is NULL:
> 
> 	static int gpiochip_add_irqchip(struct gpio_chip *gc,
> 					struct lock_class_key *lock_key,
> 					struct lock_class_key *request_key)
> 	{
> 		struct fwnode_handle *fwnode = dev_fwnode(&gc->gpiodev->dev);
> 		struct irq_chip *irqchip = gc->irq.chip;
> 		unsigned int type;
> 		unsigned int i;
> 
> 		if (!irqchip)
> 			return 0;
> 
> The result shows that it was NULL, so gc->irq.initialized = false.
> Above all, return irq = -EPROBE_DEFER.
> 
> So let's sort the function calls. In chronological order, [1] calls [2], [2] calls
> [3], then [1] calls [4]. The irq_chip was added to irq_domain->host_data->irq_chip
> before [1]. But I don't find where to convert gpio_chip->irq.domain->host_data->irq_chip
> to gpio_chip->irq.chip, it seems like it should happen after [4] ? But if it wants to use
> 'gc->to_irq' successfully, it should happen before [3]?
> 
> [1] gpio_regmap_register()
> [2] gpiochip_add_data()
> [3] gpiochip_add_irqchip()
> [4] gpiochip_irqchip_add_domain()
> 
> I'm sorry that I described the problem in a confusing way, apologize if I missed
> some code that caused this confusion.