* [PATCH v2 1/3] usb: dwc2: gadget: restart descriptor DMA isochronous OUT endpoints after BNA
2026-11-16 7:04 [PATCH v2 0/3] usb: dwc2: gadget: descriptor DMA fixes for BNA, found on Sophgo SG2002 Vadim Yuzifovich
@ 2026-11-16 7:04 ` Vadim Yuzifovich
2026-11-16 7:29 ` sashiko-bot
2026-11-16 7:04 ` [PATCH v2 2/3] usb: dwc2: gadget: restart descriptor DMA bulk and interrupt " Vadim Yuzifovich
2026-11-16 7:04 ` [PATCH v2 3/3] usb: dwc2: gadget: make descriptors visible before the endpoint is enabled Vadim Yuzifovich
2 siblings, 1 reply; 7+ messages in thread
From: Vadim Yuzifovich @ 2026-11-16 7:04 UTC (permalink / raw)
To: gregkh, Thinh.Nguyen
Cc: chenyou910331, inochiama, linux-usb, sophgo, linux-kernel,
Vadim Yuzifovich
In descriptor DMA mode the core stops an isochronous endpoint when it
reaches a descriptor that is not ready (BNA), for example after an
interrupt latency longer than the queued requests cover.
dwc2_gadget_handle_isoc_bna() resets the endpoint and leaves the restart
to the next OUTTKNEPDIS. On a Sophgo SG2002 (DWC_otg 4.20a) that
interrupt does not come. The core keeps the OUT packet that met the BNA,
shows the endpoint as disabled, and tries the descriptor at the start of
the chain again and again, raising BNA each time, as fast as the handler
clears it.
With a UAC1 capture endpoint and eight requests queued, one 8 ms
interrupt latency (a line printed on the 115200 baud serial console)
caused one BNA. The endpoint stayed disabled and the BNA interrupt kept
the CPU busy until the host gave up and reset the port, which took the
other functions of the composite gadget down as well.
An endpoint-disabled interrupt after the BNA makes it worse:
dwc2_gadget_handle_ep_disabled() has an isochronous part written for
buffer DMA, which completes requests frame by frame and advances
target_frame up to the current frame. That moves target_frame off
TARGET_FRAME_INITIAL, and the OUT token handler then takes the chain for
running and never restarts it.
So restart an OUT endpoint from the BNA handler, and from the
endpoint-disabled handler when it finds the endpoint disabled with
requests queued, and do not run the buffer DMA completion in descriptor
DMA mode. IN endpoints keep restarting on the next NAK.
The restart has to deal with the packet the core holds. The core takes
the descriptor at the start of the chain the moment it is host ready,
before the endpoint is enabled, writes the packet to it and then stops:
a chain that is enabled after that takes one packet at most and raises
BNA on the next. Traced on the SG2002, a restart that just rebuilt the
chain and enabled the endpoint gave one BNA and one restart for every
packet from then on, about 2000 interrupts a second, with the audio
mostly intact; and since the chain is filled front to back, the core
took descriptor 0 while it still carried the L bit.
Therefore keep descriptor 0 host busy while the chain is filled, write
the chain address, and only then mark descriptor 0 host ready. If the
core takes it before the endpoint is enabled, wait for the packet to
land, give its request back, and build the chain once more: no packet
is waiting for that one, and it starts like the first chain of a
stream.
The BNA handler must not flush the Rx FIFO for this. The FIFO is shared
by all OUT endpoints and nothing stops the host: when an OUT token
arrived while the handler ran, the flush left the core with a FIFO it
never drained again. GINTSTS.RXFLVL stayed set, every isochronous OUT
packet was dropped with OUTTKNEPDIS and no XferCompl, and bulk OUT on
the mass storage function was NAKed until its host driver timed out
and reset the port, after which EP0 did not answer. That happened about
once in 100 BNAs here. The flush does not drop the packet that met the
BNA either; the core delivers it to the new chain all the same. So
drop the flush.
A BNA can also arrive in the same interrupt as the endpoint-disabled,
OUT token or NAK interrupt that starts a chain, and since the core
keeps raising it, one is latched just before the restart and seen
after it. Handle BNA before those interrupts, and ignore a BNA for an
OUT endpoint that is enabled, so that the BNA handler never gives back
a request of a running chain or resets the endpoint state under it.
Fixes: 729cac693eec ("usb: dwc2: Change ISOC DDMA flow")
Signed-off-by: Vadim Yuzifovich <vadim@yuzi.co>
---
v2:
- Handle BNA before the endpoint-disabled, OUT token and NAK interrupts
in dwc2_hsotg_epint().
- Ignore a BNA for an OUT endpoint that is enabled.
- Keep descriptor 0 host busy until the chain is written and the chain
address is set; wait for a packet the core holds from before the BNA,
give it back and build the chain again.
- Fixes tag. Changelog rewritten for what the trace showed.
- Do not flush the Rx FIFO in the BNA handler: with an OUT token
arriving meanwhile, the core stopped draining the FIFO.
drivers/usb/dwc2/gadget.c | 168 ++++++++++++++++++++++++++++++++++----
1 file changed, 152 insertions(+), 16 deletions(-)
diff --git a/drivers/usb/dwc2/gadget.c b/drivers/usb/dwc2/gadget.c
index 6c8a9f05f..8adc90047 100644
--- a/drivers/usb/dwc2/gadget.c
+++ b/drivers/usb/dwc2/gadget.c
@@ -900,6 +900,7 @@ static void dwc2_gadget_config_nonisoc_xfer_ddma(struct dwc2_hsotg_ep *hs_ep,
* @hs_ep: The isochronous endpoint.
* @dma_buff: usb requests dma buffer.
* @len: usb request transfer length.
+ * @hold: leave the descriptor host busy; the caller marks it host ready.
*
* Fills next free descriptor with the data of the arrived usb request,
* frame info, sets Last and IOC bits increments next_desc. If filled
@@ -907,7 +908,8 @@ static void dwc2_gadget_config_nonisoc_xfer_ddma(struct dwc2_hsotg_ep *hs_ep,
* status.
*/
static int dwc2_gadget_fill_isoc_desc(struct dwc2_hsotg_ep *hs_ep,
- dma_addr_t dma_buff, unsigned int len)
+ dma_addr_t dma_buff, unsigned int len,
+ bool hold)
{
struct dwc2_dma_desc *desc;
struct dwc2_hsotg *hsotg = hs_ep->parent;
@@ -956,7 +958,8 @@ static int dwc2_gadget_fill_isoc_desc(struct dwc2_hsotg_ep *hs_ep,
}
desc->status &= ~DEV_DMA_BUFF_STS_MASK;
- desc->status |= (DEV_DMA_BUFF_STS_HREADY << DEV_DMA_BUFF_STS_SHIFT);
+ desc->status |= (hold ? DEV_DMA_BUFF_STS_HBUSY :
+ DEV_DMA_BUFF_STS_HREADY) << DEV_DMA_BUFF_STS_SHIFT;
/* Increment frame number by interval for IN */
if (hs_ep->dir_in)
@@ -970,6 +973,52 @@ static int dwc2_gadget_fill_isoc_desc(struct dwc2_hsotg_ep *hs_ep,
return 0;
}
+static void dwc2_gadget_complete_isoc_request_ddma(struct dwc2_hsotg_ep *hs_ep);
+
+/*
+ * dwc2_gadget_isoc_out_held_packet - let the core finish an OUT packet it
+ * holds from before a BNA.
+ * @hs_ep: The isochronous OUT endpoint, its new chain written and descriptor 0
+ * just marked host ready, the endpoint not enabled yet.
+ *
+ * A packet that met a descriptor that was not ready stays with the core. The
+ * core raises BNA, shows the endpoint as disabled, and keeps trying the
+ * descriptor at the start of the chain. As soon as that one is host ready the
+ * core writes the packet to it, without the endpoint being enabled, and then
+ * stops: a chain enabled after that takes no further packet. So wait for the
+ * packet to land and give its request back. The caller then starts a new
+ * chain, which no packet is waiting for.
+ *
+ * Return: true if the core took descriptor 0.
+ */
+static bool dwc2_gadget_isoc_out_held_packet(struct dwc2_hsotg_ep *hs_ep)
+{
+ struct dwc2_dma_desc *desc = &hs_ep->desc_list[0];
+ int i;
+
+ /* The core takes the descriptor within a few bus cycles, if at all. */
+ for (i = 0; i < 10; i++) {
+ if (READ_ONCE(desc->status) >> DEV_DMA_BUFF_STS_SHIFT !=
+ DEV_DMA_BUFF_STS_HREADY)
+ break;
+ udelay(1);
+ }
+ if (i == 10)
+ return false;
+
+ /* One packet of at most 1024 bytes: the DMA takes microseconds. */
+ for (i = 0; i < 100; i++) {
+ if (READ_ONCE(desc->status) >> DEV_DMA_BUFF_STS_SHIFT ==
+ DEV_DMA_BUFF_STS_DMADONE)
+ break;
+ udelay(1);
+ }
+
+ dwc2_gadget_complete_isoc_request_ddma(hs_ep);
+
+ return true;
+}
+
/*
* dwc2_gadget_start_isoc_ddma - start isochronous transfer in DDMA
* @hs_ep: The isochronous endpoint.
@@ -982,6 +1031,7 @@ static void dwc2_gadget_start_isoc_ddma(struct dwc2_hsotg_ep *hs_ep)
struct dwc2_hsotg *hsotg = hs_ep->parent;
struct dwc2_hsotg_req *hs_req, *treq;
int index = hs_ep->index;
+ bool again = false;
int ret;
int i;
u32 dma_reg;
@@ -989,6 +1039,7 @@ static void dwc2_gadget_start_isoc_ddma(struct dwc2_hsotg_ep *hs_ep)
u32 ctrl;
struct dwc2_dma_desc *desc;
+restart:
if (list_empty(&hs_ep->queue)) {
hs_ep->target_frame = TARGET_FRAME_INITIAL;
dev_dbg(hsotg->dev, "%s: No requests in queue\n", __func__);
@@ -1003,6 +1054,12 @@ static void dwc2_gadget_start_isoc_ddma(struct dwc2_hsotg_ep *hs_ep)
<< DEV_DMA_BUFF_STS_SHIFT);
}
+ /*
+ * Descriptor 0 stays host busy until the rest of the chain is written
+ * and the chain address is set. After a BNA the core may take it the
+ * moment it is host ready, see dwc2_gadget_isoc_out_held_packet(), and
+ * must not find it as the last of a chain that is still being built.
+ */
hs_ep->next_desc = 0;
list_for_each_entry_safe(hs_req, treq, &hs_ep->queue, queue) {
dma_addr_t dma_addr = hs_req->req.dma;
@@ -1012,7 +1069,8 @@ static void dwc2_gadget_start_isoc_ddma(struct dwc2_hsotg_ep *hs_ep)
dma_addr = sg_dma_address(hs_req->req.sg);
}
ret = dwc2_gadget_fill_isoc_desc(hs_ep, dma_addr,
- hs_req->req.length);
+ hs_req->req.length,
+ !hs_ep->next_desc);
if (ret)
break;
}
@@ -1024,6 +1082,16 @@ static void dwc2_gadget_start_isoc_ddma(struct dwc2_hsotg_ep *hs_ep)
/* write descriptor chain address to control register */
dwc2_writel(hsotg, hs_ep->desc_list_dma, dma_reg);
+ desc = &hs_ep->desc_list[0];
+ desc->status &= ~DEV_DMA_BUFF_STS_MASK;
+ desc->status |= DEV_DMA_BUFF_STS_HREADY << DEV_DMA_BUFF_STS_SHIFT;
+
+ if (!hs_ep->dir_in && !again &&
+ dwc2_gadget_isoc_out_held_packet(hs_ep)) {
+ again = true;
+ goto restart;
+ }
+
ctrl = dwc2_readl(hsotg, depctl);
ctrl |= DXEPCTL_EPENA | DXEPCTL_CNAK;
dwc2_writel(hsotg, ctrl, depctl);
@@ -1491,7 +1559,7 @@ static int dwc2_hsotg_ep_queue(struct usb_ep *ep, struct usb_request *req,
dma_addr = sg_dma_address(hs_req->req.sg);
}
dwc2_gadget_fill_isoc_desc(hs_ep, dma_addr,
- hs_req->req.length);
+ hs_req->req.length, false);
}
return 0;
}
@@ -2259,26 +2327,66 @@ static void dwc2_gadget_complete_isoc_request_ddma(struct dwc2_hsotg_ep *hs_ep)
}
}
+/*
+ * dwc2_gadget_restart_isoc_out_ddma - build the chain of a disabled ISOC OUT
+ * endpoint again from its queue and enable it.
+ * @hs_ep: The isochronous OUT endpoint, in its initial state.
+ *
+ * After a BNA the core keeps raising BNA for the endpoint, as fast as the
+ * handler clears it, and no OUTTKNEPDIS comes to restart the chain. So the
+ * chain is restarted here at once.
+ */
+static void dwc2_gadget_restart_isoc_out_ddma(struct dwc2_hsotg_ep *hs_ep)
+{
+ struct dwc2_hsotg *hsotg = hs_ep->parent;
+
+ if (hs_ep->dir_in || list_empty(&hs_ep->queue) ||
+ hs_ep->target_frame != TARGET_FRAME_INITIAL ||
+ dwc2_readl(hsotg, DOEPCTL(hs_ep->index)) & DXEPCTL_EPENA)
+ return;
+
+ hs_ep->target_frame = dwc2_hsotg_read_frameno(hsotg);
+ dwc2_gadget_start_isoc_ddma(hs_ep);
+}
+
/*
* dwc2_gadget_handle_isoc_bna - handle BNA interrupt for ISOC.
* @hs_ep: The isochronous endpoint.
*
- * If EP ISOC OUT then need to flush RX FIFO to remove source of BNA
- * interrupt. Reset target frame and next_desc to allow to start
- * ISOC's on NAK interrupt for IN direction or on OUTTKNEPDIS
- * interrupt for OUT direction.
+ * Give back the request that met the BNA and reset target frame and
+ * next_desc. An IN endpoint starts again on the next NAK interrupt. An OUT
+ * endpoint is restarted here, see dwc2_gadget_restart_isoc_out_ddma().
+ *
+ * The Rx FIFO is not flushed for an OUT endpoint. It is shared by all OUT
+ * endpoints and the host goes on sending, so a flush here can hit a packet
+ * that is being received, after which the core no longer drains the FIFO.
+ * The packet that met the BNA is not dropped by a flush either: the core
+ * holds it and writes it to the first descriptor of the new chain.
*/
static void dwc2_gadget_handle_isoc_bna(struct dwc2_hsotg_ep *hs_ep)
{
struct dwc2_hsotg *hsotg = hs_ep->parent;
- if (!hs_ep->dir_in)
- dwc2_flush_rx_fifo(hsotg);
+ /*
+ * The core disables an OUT endpoint that raises BNA and goes on
+ * raising BNA for it until it is enabled again. So a BNA can be
+ * latched between the moment the interrupt handler clears DOEPINT and
+ * the restart below, and is then seen with the new chain running.
+ * Leave that chain alone. If the endpoint is still about to be
+ * disabled, the endpoint-disabled interrupt or the next BNA restarts
+ * it.
+ */
+ if (!hs_ep->dir_in &&
+ dwc2_readl(hsotg, DOEPCTL(hs_ep->index)) & DXEPCTL_EPENA)
+ return;
+
dwc2_hsotg_complete_request(hsotg, hs_ep, get_ep_head(hs_ep), 0);
hs_ep->target_frame = TARGET_FRAME_INITIAL;
hs_ep->next_desc = 0;
hs_ep->compl_desc = 0;
+
+ dwc2_gadget_restart_isoc_out_ddma(hs_ep);
}
/**
@@ -2902,6 +3010,27 @@ static void dwc2_gadget_handle_ep_disabled(struct dwc2_hsotg_ep *hs_ep)
if (!hs_ep->isochronous)
return;
+ /*
+ * In descriptor DMA the requests sit in the descriptor chain rather
+ * than one per frame, so the frame by frame completion below does not
+ * apply. The core disables an isochronous endpoint by itself when it
+ * meets a descriptor that was not filled in time (BNA). Put such an
+ * endpoint back in its initial state and build the chain again from
+ * the queue: at once for OUT, on the next NAK for IN. The loop below
+ * would move target_frame off TARGET_FRAME_INITIAL, which tells the
+ * token handlers that a chain runs: the endpoint then stays disabled
+ * for good, and every request queued again goes into a dead chain.
+ */
+ if (using_desc_dma(hsotg)) {
+ if (!(dwc2_readl(hsotg, epctl_reg) & DXEPCTL_EPENA)) {
+ hs_ep->target_frame = TARGET_FRAME_INITIAL;
+ hs_ep->next_desc = 0;
+ hs_ep->compl_desc = 0;
+ dwc2_gadget_restart_isoc_out_ddma(hs_ep);
+ }
+ return;
+ }
+
if (list_empty(&hs_ep->queue)) {
dev_dbg(hsotg->dev, "%s: complete_ep 0x%p, ep->queue empty!\n",
__func__, hs_ep);
@@ -3151,6 +3280,19 @@ static void dwc2_hsotg_epint(struct dwc2_hsotg *hsotg, unsigned int idx,
}
}
+ /*
+ * Handle BNA before the endpoint-disabled, OUT token and NAK
+ * interrupts that can come with it. Those handlers start a new chain
+ * for an isochronous endpoint in descriptor DMA mode, and the BNA
+ * handler must not then give back a request of the running chain and
+ * reset the endpoint state under it.
+ */
+ if (ints & DXEPINT_BNAINTR) {
+ dev_dbg(hsotg->dev, "%s: BNA interrupt\n", __func__);
+ if (hs_ep->isochronous)
+ dwc2_gadget_handle_isoc_bna(hs_ep);
+ }
+
if (ints & DXEPINT_EPDISBLD)
dwc2_gadget_handle_ep_disabled(hs_ep);
@@ -3208,12 +3350,6 @@ static void dwc2_hsotg_epint(struct dwc2_hsotg *hsotg, unsigned int idx,
if (ints & DXEPINT_BACK2BACKSETUP)
dev_dbg(hsotg->dev, "%s: B2BSetup/INEPNakEff\n", __func__);
- if (ints & DXEPINT_BNAINTR) {
- dev_dbg(hsotg->dev, "%s: BNA interrupt\n", __func__);
- if (hs_ep->isochronous)
- dwc2_gadget_handle_isoc_bna(hs_ep);
- }
-
if (dir_in && !hs_ep->isochronous) {
/* not sure if this is important, but we'll clear it anyway */
if (ints & DXEPINT_INTKNTXFEMP) {
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
--
2.56.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v2 2/3] usb: dwc2: gadget: restart descriptor DMA bulk and interrupt endpoints after BNA
2026-11-16 7:04 [PATCH v2 0/3] usb: dwc2: gadget: descriptor DMA fixes for BNA, found on Sophgo SG2002 Vadim Yuzifovich
2026-11-16 7:04 ` [PATCH v2 1/3] usb: dwc2: gadget: restart descriptor DMA isochronous OUT endpoints after BNA Vadim Yuzifovich
@ 2026-11-16 7:04 ` Vadim Yuzifovich
2026-11-16 7:13 ` sashiko-bot
2026-11-16 7:04 ` [PATCH v2 3/3] usb: dwc2: gadget: make descriptors visible before the endpoint is enabled Vadim Yuzifovich
2 siblings, 1 reply; 7+ messages in thread
From: Vadim Yuzifovich @ 2026-11-16 7:04 UTC (permalink / raw)
To: gregkh, Thinh.Nguyen
Cc: chenyou910331, inochiama, linux-usb, sophgo, linux-kernel,
Vadim Yuzifovich
In descriptor DMA mode the core raises BNA and disables an endpoint when
the descriptor it fetches is not marked host ready. For isochronous
endpoints dwc2 handles that; for bulk and interrupt endpoints it only
logs the interrupt at debug level. Nothing was transferred and no
XferCompl follows, so the request stays current, the requests queued
behind it never start, and the function driver waits for a completion
that never comes.
On a Sophgo SG2002 (DWC_otg 4.20a) with a video pipeline loading the
DRAM this happened every minute or two to the bulk IN endpoint of
f_mass_storage: debugfs showed the endpoint disabled with an 18-byte
REQUEST SENSE reply current and nothing done, the 13-byte CSW queued
behind it, and f_mass_storage asleep waiting for its buffer. The host's
command timed out after 30 s and it reset the port. Read in the BNA
handler, the descriptor is 0x0e000012: host ready, last, short,
interrupt on completion, 18 bytes. The chain is written completely
before the endpoint is enabled, so the core fetched the descriptor
before the CPU's stores to it reached memory. The next patch closes
that window on this SoC, but a BNA on such an endpoint should not leave
it dead on any platform.
When a bulk or interrupt endpoint other than EP0 raises BNA, is disabled
and not halted, start its current request again, which fills the
descriptor anew. Start it from what has been transferred of it, the way
the next part of a request longer than one chain is started, so that a
BNA on a later part does not send or fill the start of the buffer
again. Report the BNA with a rate limited warning instead of
at debug level: unlike on an isochronous endpoint it is not expected.
Signed-off-by: Vadim Yuzifovich <vadim@yuzi.co>
---
v2:
- Restart the request from req.actual (continuing) instead of from its
beginning.
- Follows the move of the isochronous BNA handling in patch 1; the bulk
and interrupt restart stays after the endpoint-disabled handler.
drivers/usb/dwc2/gadget.c | 49 +++++++++++++++++++++++++++++++++++++++
1 file changed, 49 insertions(+)
diff --git a/drivers/usb/dwc2/gadget.c b/drivers/usb/dwc2/gadget.c
index 8adc90047..a7eaf75ff 100644
--- a/drivers/usb/dwc2/gadget.c
+++ b/drivers/usb/dwc2/gadget.c
@@ -2389,6 +2389,46 @@ static void dwc2_gadget_handle_isoc_bna(struct dwc2_hsotg_ep *hs_ep)
dwc2_gadget_restart_isoc_out_ddma(hs_ep);
}
+/*
+ * dwc2_gadget_handle_nonisoc_bna - handle BNA interrupt for bulk and
+ * interrupt endpoints in descriptor DMA mode.
+ * @hs_ep: The endpoint, not EP0 and not isochronous.
+ * @ints: The endpoint interrupts that came with the BNA, for the log.
+ *
+ * The core raises BNA and disables the endpoint when the descriptor it
+ * fetches is not marked host ready. The chain of a bulk or interrupt
+ * endpoint is complete before the endpoint is enabled, and afterwards the
+ * descriptor reads host ready, so the core read it before the CPU's writes
+ * to it reached memory. Nothing was transferred and no XferCompl
+ * follows: the request stays current, the ones queued behind it never
+ * start, and the function waits until the host gives up and resets the
+ * port. Start the request again from what has been transferred of it, as
+ * for the next part of a request longer than one chain; that fills the
+ * descriptor anew.
+ */
+static void dwc2_gadget_handle_nonisoc_bna(struct dwc2_hsotg_ep *hs_ep,
+ u32 ints)
+{
+ struct dwc2_hsotg *hsotg = hs_ep->parent;
+ struct dwc2_hsotg_req *hs_req = hs_ep->req;
+ u32 epctl_reg = hs_ep->dir_in ? DIEPCTL(hs_ep->index) :
+ DOEPCTL(hs_ep->index);
+ u32 epctl = dwc2_readl(hsotg, epctl_reg);
+ u32 status = hs_ep->desc_list ? READ_ONCE(hs_ep->desc_list[0].status) : 0;
+ bool restart = hs_req && !hs_ep->halted &&
+ !(epctl & (DXEPCTL_EPENA | DXEPCTL_STALL));
+
+ dev_warn_ratelimited(hsotg->dev,
+ "%s: BNA, DxEPINT %08x DxEPCTL %08x descriptor %08x, %s\n",
+ hs_ep->name, ints, epctl, status,
+ restart ? "request restarted" : "not restarted");
+
+ if (!restart)
+ return;
+
+ dwc2_hsotg_start_req(hsotg, hs_ep, hs_req, true);
+}
+
/**
* dwc2_hsotg_rx_data - receive data from the FIFO for an endpoint
* @hsotg: The device state.
@@ -3350,6 +3390,15 @@ static void dwc2_hsotg_epint(struct dwc2_hsotg *hsotg, unsigned int idx,
if (ints & DXEPINT_BACK2BACKSETUP)
dev_dbg(hsotg->dev, "%s: B2BSetup/INEPNakEff\n", __func__);
+ /*
+ * A bulk or interrupt endpoint is restarted here, after the
+ * endpoint-disabled handler above has flushed the TxFIFO of an IN
+ * endpoint.
+ */
+ if ((ints & DXEPINT_BNAINTR) && !hs_ep->isochronous &&
+ using_desc_dma(hsotg) && idx)
+ dwc2_gadget_handle_nonisoc_bna(hs_ep, ints);
+
if (dir_in && !hs_ep->isochronous) {
/* not sure if this is important, but we'll clear it anyway */
if (ints & DXEPINT_INTKNTXFEMP) {
--
2.56.0
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v2 3/3] usb: dwc2: gadget: make descriptors visible before the endpoint is enabled
2026-11-16 7:04 [PATCH v2 0/3] usb: dwc2: gadget: descriptor DMA fixes for BNA, found on Sophgo SG2002 Vadim Yuzifovich
2026-11-16 7:04 ` [PATCH v2 1/3] usb: dwc2: gadget: restart descriptor DMA isochronous OUT endpoints after BNA Vadim Yuzifovich
2026-11-16 7:04 ` [PATCH v2 2/3] usb: dwc2: gadget: restart descriptor DMA bulk and interrupt " Vadim Yuzifovich
@ 2026-11-16 7:04 ` Vadim Yuzifovich
2026-11-16 7:27 ` sashiko-bot
2 siblings, 1 reply; 7+ messages in thread
From: Vadim Yuzifovich @ 2026-11-16 7:04 UTC (permalink / raw)
To: gregkh, Thinh.Nguyen
Cc: chenyou910331, inochiama, linux-usb, sophgo, linux-kernel,
Vadim Yuzifovich
In descriptor DMA mode the core fetches the first descriptor of a chain
as soon as the endpoint is enabled. dwc2 fills the chain in coherent
memory and then writes DxEPDMA and DxEPCTL.EPENA with writel(), whose
barrier orders the earlier stores before the MMIO write. On a Sophgo
SG2002 (T-Head C906, DWC_otg 4.20a) that is not enough: the core still
finds a descriptor that is not host ready, raises BNA and disables the
endpoint, although the descriptor reads host ready, last, short, IOC
when the interrupt is handled.
With a host sending TEST UNIT READY to a CD-ROM LUN as fast as it can
(about 2600 a second, each followed by REQUEST SENSE) and a 1080p60
H.264 encode loading the DRAM, about 0.8% of transfer starts raised BNA,
99.5% of them within 20 us of the enable (the latency of the interrupt
itself), with DxEPDMA still at the head of the chain. Without the
encode there was none in 254000 commands. BNAs per variant, measured in
turn on one boot with a diagnostic build, at about 780000 transfer
starts a minute each:
unchanged 6215, 6510
mb(), read the last descriptor, mb() 0
th.sync.s 0
udelay(2) 2
descriptor written as two word stores,
status last, dma_wmb() between 3728
Mapping coherent memory non-bufferable on the C906 did not change the
rate either. So the stores are still on their way to DRAM when the core
reads, and a load from the descriptor waits for them.
Read the last descriptor of a chain back between two full barriers
before the endpoint is enabled, for non-isochronous transfers and for
the start of an isochronous chain. An isochronous chain marks its first
descriptor host ready last, so that one is read back as well. It costs
one uncached load per transfer. With it there was no BNA in 6.2 million
transfer starts under the same load, and none in a 60 minute run at
1080p60.
Signed-off-by: Vadim Yuzifovich <vadim@yuzi.co>
---
v2:
- Read the last descriptor back also when the isochronous chain is full
and next_desc has wrapped to 0.
- Read descriptor 0 of an isochronous chain back as well.
drivers/usb/dwc2/gadget.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/drivers/usb/dwc2/gadget.c b/drivers/usb/dwc2/gadget.c
index a7eaf75ff..cd1773836 100644
--- a/drivers/usb/dwc2/gadget.c
+++ b/drivers/usb/dwc2/gadget.c
@@ -854,6 +854,27 @@ static void dwc2_gadget_fill_nonisoc_xfer_ddma_one(struct dwc2_hsotg_ep *hs_ep,
}
}
+/*
+ * dwc2_gadget_desc_visible - make descriptors the CPU has just written
+ * visible to the core before it is told to fetch them.
+ * @desc: The descriptor written last.
+ *
+ * The core fetches the first descriptor of a chain as soon as the endpoint is
+ * enabled. writel() orders the CPU's earlier stores before that MMIO write,
+ * but on the Sophgo SG2002 (T-Head C906) the stores to coherent memory can
+ * still be on their way to DRAM when the core reads it. The core then finds a
+ * descriptor that is not host ready and raises BNA. Reading the last
+ * descriptor back between two full barriers waits for the stores to land.
+ */
+static void dwc2_gadget_desc_visible(struct dwc2_dma_desc *desc)
+{
+ /* Order the stores to the chain before the load below. */
+ mb();
+ (void)READ_ONCE(desc->status);
+ /* Finish the load before the MMIO write that enables the endpoint. */
+ mb();
+}
+
/*
* dwc2_gadget_config_nonisoc_xfer_ddma - prepare non ISOC DMA desc chain.
* @hs_ep: The endpoint
@@ -881,6 +902,7 @@ static void dwc2_gadget_config_nonisoc_xfer_ddma(struct dwc2_hsotg_ep *hs_ep,
if (!ureq || !ureq->num_sgs) {
dwc2_gadget_fill_nonisoc_xfer_ddma_one(hs_ep, &desc,
dma_buff, len, true);
+ dwc2_gadget_desc_visible(desc - 1);
return;
}
@@ -893,6 +915,7 @@ static void dwc2_gadget_config_nonisoc_xfer_ddma(struct dwc2_hsotg_ep *hs_ep,
}
hs_ep->desc_count = desc_count;
+ dwc2_gadget_desc_visible(desc - 1);
}
/*
@@ -1079,12 +1102,23 @@ static void dwc2_gadget_start_isoc_ddma(struct dwc2_hsotg_ep *hs_ep)
depctl = hs_ep->dir_in ? DIEPCTL(index) : DOEPCTL(index);
dma_reg = hs_ep->dir_in ? DIEPDMA(index) : DOEPDMA(index);
+ /*
+ * The rest of the chain has to be visible before descriptor 0 is
+ * host ready. The queue is not empty, so at least one descriptor was
+ * filled: next_desc 0 means that the chain is full and the index
+ * wrapped.
+ */
+ i = hs_ep->next_desc ? hs_ep->next_desc - 1 :
+ MAX_DMA_DESC_NUM_HS_ISOC - 1;
+ dwc2_gadget_desc_visible(&hs_ep->desc_list[i]);
+
/* write descriptor chain address to control register */
dwc2_writel(hsotg, hs_ep->desc_list_dma, dma_reg);
desc = &hs_ep->desc_list[0];
desc->status &= ~DEV_DMA_BUFF_STS_MASK;
desc->status |= DEV_DMA_BUFF_STS_HREADY << DEV_DMA_BUFF_STS_SHIFT;
+ dwc2_gadget_desc_visible(desc);
if (!hs_ep->dir_in && !again &&
dwc2_gadget_isoc_out_held_packet(hs_ep)) {
--
2.56.0
^ permalink raw reply related [flat|nested] 7+ messages in thread