Re: [PATCH] usb: dwc3: Potential fix of possible dwc3 interrupt storm

[Thinh Nguyen <Thinh.Nguyen@synopsys.com>]

On Mon, Sep 16, 2024, Selvarasu Ganesan wrote:
> 
> On 9/13/2024 11:21 PM, Thinh Nguyen wrote:
> > Hi,
> >
> > On Fri, Sep 13, 2024, Selvarasu Ganesan wrote:
> >> Hi Thinh,
> >>
> >> So far, there have been no reported error instances. But, we suspecting
> >> that the issue may be related to our glue driver. In our glue driver, we
> >> access the reference of evt->flags when starting or stopping the gadget
> >> based on a VBUS notification. We apologize for sharing this information
> >> so late, as we only became aware of it recently.
> >>
> >> The following sequence outlines the possible scenarios of race conditions:
> >>
> >> Thread#1 (Our glue Driver Sequence)
> >> ===================================
> >> ->USB VBUS notification
> >> ->Start/Stop gadget
> >> ->dwc->ev_buf->flags |= BIT(20); (It's for our reference)
> >> ->Call dwc3 exynos runtime suspend/resume
> >> ->dwc->ev_buf->flags &= ~BIT(20);
> >> ->Call dwc3 core runtime suspend/resume
> >>
> >> Thread#2
> >> ========
> >> ->dwc3_interrupt()
> >> ->evt->flags |= DWC3_EVENT_PENDING;
> >> ->dwc3_thread_interrupt()
> >> ->evt->flags &= ~DWC3_EVENT_PENDING;
> >>
> > This is great! That's likely the problem. Glad you found it.
> >
> >>
> >> After our internal discussions, we have decided to remove the
> >> unnecessary access to evt->flag in our glue driver. We have made these
> >> changes and initiated testing.
> >>
> >> Thank you for your help so far to understand more into our glue driver code.
> >>
> >> And We are thinking that it would be fine to reset evt->flag when the
> >> USB controller is started, along with the changes you suggested earlier.
> >> This additional measure will help prevent similar issues from occurring
> >> in the future.
> >>
> >> Please let us know your thoughts on this proposal. If it is not
> >> necessary, we understand and will proceed accordingly.
> >>
> > You can submit the change I suggested. That's a valid change. However,
> > we should not include the reset of the DWC3_EVENT_PENDING flag. Had we
> > done this, you may not found the issue above. It serves no purpose for
> > the core driver logic and will be an extra burden for us to maintain. (I
> > don't want to scratch my head in the future to figure out why that
> > change was needed or concern whether it can be removed without causing
> > regression).
> 
> Yeah I understand.
> 
> Please reconfirm the below changes once with commit message. I will post 
> new version if this changes are fine.
> 
> 
> [PATCH] usb: dwc3: core: Stop processing of pending events if controller 
> is halted
> 
> This commit addresses an issue where events were being processed when
> the controller was in a halted state. To fix this issue by stop
> processing the events as the event count was considered stale or
> invalid when the controller was halted.
> 
> Fixes: fc8bb91bc83e ("usb: dwc3: implement runtime PM")
> Cc: stable <stable@kernel.org>
> Signed-off-by: Selvarasu Ganesan <selvarasu.g@samsung.com>
> Suggested-by: Thinh Nguyen <Thinh.Nguyen@synopsys.com>
> ---
>   drivers/usb/dwc3/core.c   | 17 +++++++++++++++--
>   drivers/usb/dwc3/core.h   |  4 ----
>   drivers/usb/dwc3/gadget.c | 22 +++++++++++-----------
>   3 files changed, 26 insertions(+), 17 deletions(-)
> 
> diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c
> index 9eb085f359ce..f47b20bc2d1f 100644
> --- a/drivers/usb/dwc3/core.c
> +++ b/drivers/usb/dwc3/core.c
> @@ -544,6 +544,7 @@ static int dwc3_alloc_event_buffers(struct dwc3 
> *dwc, unsigned int length)
>   int dwc3_event_buffers_setup(struct dwc3 *dwc)
>   {
>          struct dwc3_event_buffer        *evt;
> +       u32                             reg;
> 
>          if (!dwc->ev_buf)
>                  return 0;
> @@ -556,8 +557,10 @@ int dwc3_event_buffers_setup(struct dwc3 *dwc)
>                          upper_32_bits(evt->dma));
>          dwc3_writel(dwc->regs, DWC3_GEVNTSIZ(0),
>                          DWC3_GEVNTSIZ_SIZE(evt->length));
> -       dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(0), 0);
> 
> +       /* Clear any stale event */

Do the same thing here as in dwc3_event_buffers_cleanup().

> +       reg = dwc3_readl(dwc->regs, DWC3_GEVNTCOUNT(0));
> +       dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(0), reg);
>          return 0;
>   }
> 
> @@ -2499,7 +2502,11 @@ static int dwc3_runtime_resume(struct device *dev)
> 
>          switch (dwc->current_dr_role) {
>          case DWC3_GCTL_PRTCAP_DEVICE:
> -               dwc3_gadget_process_pending_events(dwc);
> +               if (dwc->pending_events) {
> +                       pm_runtime_put(dwc->dev);
> +                       dwc->pending_events = false;
> +                       enable_irq(dwc->irq_gadget);
> +               }
>                  break;
>          case DWC3_GCTL_PRTCAP_HOST:
>          default:
> @@ -2589,6 +2596,12 @@ static void dwc3_complete(struct device *dev)
>   static const struct dev_pm_ops dwc3_dev_pm_ops = {
>          SET_SYSTEM_SLEEP_PM_OPS(dwc3_suspend, dwc3_resume)
>          .complete = dwc3_complete,
> +
> +       /*
> +        * Runtime suspend halts the controller on disconnection. It 
> replies on
> +        * platforms with custom connection notification to start the 
> controller
> +        * again.
> +        */
>          SET_RUNTIME_PM_OPS(dwc3_runtime_suspend, dwc3_runtime_resume,
>                          dwc3_runtime_idle)
>   };
> diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h
> index c71240e8f7c7..9c508e0c5cdf 100644
> --- a/drivers/usb/dwc3/core.h
> +++ b/drivers/usb/dwc3/core.h
> @@ -1675,7 +1675,6 @@ static inline void dwc3_otg_host_init(struct dwc3 
> *dwc)
>   #if !IS_ENABLED(CONFIG_USB_DWC3_HOST)
>   int dwc3_gadget_suspend(struct dwc3 *dwc);
>   int dwc3_gadget_resume(struct dwc3 *dwc);
> -void dwc3_gadget_process_pending_events(struct dwc3 *dwc);
>   #else
>   static inline int dwc3_gadget_suspend(struct dwc3 *dwc)
>   {
> @@ -1687,9 +1686,6 @@ static inline int dwc3_gadget_resume(struct dwc3 *dwc)
>          return 0;
>   }
> 
> -static inline void dwc3_gadget_process_pending_events(struct dwc3 *dwc)
> -{
> -}
>   #endif /* !IS_ENABLED(CONFIG_USB_DWC3_HOST) */
> 
>   #if IS_ENABLED(CONFIG_USB_DWC3_ULPI)
> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
> index 291bc549935b..a32c3a292353 100644
> --- a/drivers/usb/dwc3/gadget.c
> +++ b/drivers/usb/dwc3/gadget.c
> @@ -4483,6 +4483,17 @@ static irqreturn_t dwc3_check_event_buf(struct 
> dwc3_event_buffer *evt)
>                  return IRQ_HANDLED;
> 
>          count = dwc3_readl(dwc->regs, DWC3_GEVNTCOUNT(0));

If we properly cleanup event count in dwc3_event_buffers_cleanup() as
noted above, then you don't need this condition below. You can remove
the below check:

> +
> +       /*
> +        * If the controller is halted, the event count is 
> stale/invalid. Ignore
> +        * them. This happens if the interrupt assertion is from an 
> out-of-band
> +        * resume notification.
> +        */
> +       if (!dwc->pullups_connected && count) {
> +               dwc3_writel(dwc->regs, DWC3_GEVNTCOUNT(0), count);
> +               return IRQ_HANDLED;
> +       }
> +
>          count &= DWC3_GEVNTCOUNT_MASK;
>          if (!count)
>                  return IRQ_NONE;
> @@ -4728,14 +4739,3 @@ int dwc3_gadget_resume(struct dwc3 *dwc)
> 
>          return dwc3_gadget_soft_connect(dwc);
>   }
> -
> -void dwc3_gadget_process_pending_events(struct dwc3 *dwc)
> -{
> -       if (dwc->pending_events) {
> -               dwc3_interrupt(dwc->irq_gadget, dwc->ev_buf);
> -               dwc3_thread_interrupt(dwc->irq_gadget, dwc->ev_buf);
> -               pm_runtime_put(dwc->dev);
> -               dwc->pending_events = false;
> -               enable_irq(dwc->irq_gadget);
> -       }
> -}

The rest looks fine.

Thanks,
Thinh