From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtpbgau1.qq.com (smtpbgau1.qq.com [54.206.16.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0AD2D3009EE; Wed, 25 Mar 2026 09:14:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=54.206.16.166 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774430045; cv=none; b=CMTIKnNGK9QEeBwnhjN5GM/iV7Mp/6x0Cf+Gn3XjUZo4dKlRzkHyJMP0fL02m/WStc0nbKBy+8gsR5CQkmGJkMX8wDYahTuOuZOq/pjxWc2dp6o1vGQRADQFpTZET1u1AB5XcgxynWTDwgpMZYjG8+qo0mC4ZJfDSUTy6y+iVHw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774430045; c=relaxed/simple; bh=73B4l3R4CWelf8r+zUQseP5uZ/RvFyZZC7y1/gxjjV0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=g1lFJ1t4iuEslBscKM4W8vv8QQ1ehyXlMclaL3icN2NP8dYllYyakg3ZBR9D0h3V/i3uT7sQzTVhu5arTfi7CIILpx72blLNc0oV75J5B7dgTA7mKl6UZhoJ1X5MAGcZ2didl3+UwJtQF851Fl6RrY4Spomqls6Oc3kJHciTRvM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=mucse.com; spf=pass smtp.mailfrom=mucse.com; arc=none smtp.client-ip=54.206.16.166 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=mucse.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=mucse.com X-QQ-mid: zesmtpsz4t1774429941t89bba290 X-QQ-Originating-IP: xxN4Gei91kj4CPJH9xQ3eB9cwwO7q31na3d7uVUn8+I= Received: from localhost.localdomain ( [203.174.112.180]) by bizesmtp.qq.com (ESMTP) with id ; Wed, 25 Mar 2026 17:12:19 +0800 (CST) X-QQ-SSF: 0000000000000000000000000000000 X-QQ-GoodBg: 0 X-BIZMAIL-ID: 4100074904878717650 EX-QQ-RecipientCnt: 9 From: Dong Yibo To: andrew+netdev@lunn.ch, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, danishanwar@ti.com Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, dong100@mucse.com Subject: [PATCH net-next 3/4] net: rnpgbe: Add RX packet reception support Date: Wed, 25 Mar 2026 17:12:03 +0800 Message-Id: <20260325091204.94015-4-dong100@mucse.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20260325091204.94015-1-dong100@mucse.com> References: <20260325091204.94015-1-dong100@mucse.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-QQ-SENDSIZE: 520 Feedback-ID: zesmtpsz:mucse.com:qybglogicsvrgz:qybglogicsvrgz3a-1 X-QQ-XMAILINFO: MQCbYOYzcCnkg2Wi7x+7qCSpOYW/vaC1NTEbn+hmGH8iu0M/jO+I0fnz KIwI6zNu65jzp9GoGoCrx+v07M6DLuuaXq/Lh5ei4/1uz7dAovPdQaPxDxzvGSNe4MxEv9s SvrSUf33RX6A6+1SSZZO+3QGI8OvmuCx+P2+Im6w11EXELAK9/wt/AS4MSJdu4X9ZhWtsH2 +hRTfpRQKbAoWaxmPi1ASBZRBFp45aaO/vj+ay4fmUamZ2+SB2KI7BCQTgbj7UulnNVnZPt kUpVGTP7CYGaB3FJvNZe5iK7GsH2EmeQ/QDy8srKFejtYfcI9fFauNd4kIHO6iDHQp6Eyg2 4Rr2i0J0N2YB7QDdritXQADUG6NwmIc9Pmlhyp9x85/PQ3Nl7pJbZQznC5/buSlzwF8zrRe uwOjaRqFl9SrtqiejChT4ZIXTkkpor30xlbLz4tG3dBz6FnB/Ly/fsVl97PTMTwdjtgHFW7 8RVuX5qHymD5T5eIW+pSTyAAtAyjnVG5DINb20xEo7tmOExcPqAwkkxiHfAAEmmlwzs2NQb QDwcRQODbhY+WtFhWnZttNWYgzTBvTRoVr1/RlWtRerBIkbJwTmDo7jcK1jlKrD69I6Y9xn se2fRZNZ+5XVQzUAzj0SBduFzb56H8Jve5S0oT2XbIc0acn7cTbd5MkXWf3eAU48esV22mL /mWBvOBVBZnWP0/SyJ7Veknxs1eJkrr0rDQHz7k69HwUmuWMYXdjho6L8/2XlKNNhKa1qB9 U6sFVj36o6oIKpYM6D2YhfHg9fnVEirvFPkbuRQ81DE35Sk14u8NhVqTEzpc/tIrPmS9+aK HZdyN3UEGUz0ygbjjMyw/M0yxaYczfl7/tXEOuO0HG2+/8r707bk7b1rGh8++JvOID75hPk RAz+zVZnOZancA0s5BqX9v1SKIZqaE64f6pyFV0T8f/QHrfmxm0Nvj9sPsylZ1um5CWJ1k5 xXgQnpdbZV2fgRh5nyRTTGeiZH8LN1pp62WGFgd7ofKwM11o7vVXNEeTgBhPb+/IQuRFwr1 AZbk+eOULWk/YGZGC5VM+uaVzoyc0lzNQslztegw== X-QQ-XMRINFO: MPJ6Tf5t3I/ylTmHUqvI8+Wpn+Gzalws3A== X-QQ-RECHKSPAM: 0 Add basic RX packet reception infrastructure to the rnpgbe driver: - Add RX descriptor structure (union rnpgbe_rx_desc) with write-back format for hardware status - Add RX buffer management using page_pool for efficient page recycling - Implement NAPI poll callback (rnpgbe_poll) for RX processing - Add RX ring setup and cleanup functions - Implement packet building from page buffer - Add RX statistics tracking Signed-off-by: Dong Yibo --- drivers/net/ethernet/mucse/Kconfig | 1 + drivers/net/ethernet/mucse/rnpgbe/rnpgbe.h | 49 +- drivers/net/ethernet/mucse/rnpgbe/rnpgbe_hw.h | 1 + .../net/ethernet/mucse/rnpgbe/rnpgbe_lib.c | 622 +++++++++++++++++- .../net/ethernet/mucse/rnpgbe/rnpgbe_lib.h | 32 +- .../net/ethernet/mucse/rnpgbe/rnpgbe_main.c | 9 + 6 files changed, 710 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/mucse/Kconfig b/drivers/net/ethernet/mucse/Kconfig index 0b3e853d625f..be0fdf268484 100644 --- a/drivers/net/ethernet/mucse/Kconfig +++ b/drivers/net/ethernet/mucse/Kconfig @@ -19,6 +19,7 @@ if NET_VENDOR_MUCSE config MGBE tristate "Mucse(R) 1GbE PCI Express adapters support" depends on PCI + select PAGE_POOL help This driver supports Mucse(R) 1GbE PCI Express family of adapters. diff --git a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe.h b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe.h index 7d28ef3bdd86..13838e370165 100644 --- a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe.h +++ b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe.h @@ -61,7 +61,32 @@ struct rnpgbe_tx_desc { #define M_TXD_CMD_EOP 0x010000 /* End of Packet */ }; +union rnpgbe_rx_desc { + struct { + __le64 pkt_addr; /* Packet buffer address */ + __le64 resv_cmd; /* cmd status */ + }; + struct { + __le32 rss_hash; /* RSS HASH */ + __le16 mark; /* mark info */ + __le16 rev1; + __le16 len; /* Packet length */ + __le16 padding_len; + __le16 vlan; /* VLAN tag */ + __le16 cmd; /* cmd status */ +#define M_RXD_STAT_DD BIT(1) /* Descriptor Done */ +#define M_RXD_STAT_EOP BIT(0) /* End of Packet */ + } wb; +}; + #define M_TX_DESC(R, i) (&(((struct rnpgbe_tx_desc *)((R)->desc))[i])) +#define M_RX_DESC(R, i) (&(((union rnpgbe_rx_desc *)((R)->desc))[i])) + +static inline __le16 rnpgbe_test_staterr(union rnpgbe_rx_desc *rx_desc, + const u16 stat_err_bits) +{ + return rx_desc->wb.cmd & cpu_to_le16(stat_err_bits); +} struct mucse_tx_buffer { struct rnpgbe_tx_desc *next_to_watch; @@ -77,13 +102,24 @@ struct mucse_queue_stats { u64 bytes; }; +struct mucse_rx_buffer { + struct sk_buff *skb; + dma_addr_t dma; + struct page *page; + u32 page_offset; +}; + struct mucse_ring { struct mucse_ring *next; struct mucse_q_vector *q_vector; struct net_device *netdev; struct device *dev; + struct page_pool *page_pool; void *desc; - struct mucse_tx_buffer *tx_buffer_info; + union { + struct mucse_tx_buffer *tx_buffer_info; + struct mucse_rx_buffer *rx_buffer_info; + }; void __iomem *ring_addr; void __iomem *tail; void __iomem *irq_mask; @@ -109,6 +145,15 @@ static inline u16 mucse_desc_unused(struct mucse_ring *ring) return ((ntc > ntu) ? 0 : ring->count) + ntc - ntu - 1; } +static inline u16 mucse_desc_unused_rx(struct mucse_ring *ring) +{ + u16 ntc = ring->next_to_clean; + u16 ntu = ring->next_to_use; + + /* 16 * 16 = 256 tlp-max-payload size */ + return ((ntc > ntu) ? 0 : ring->count) + ntc - ntu - 16; +} + static inline __le64 build_ctob(u32 vlan_cmd, u32 mac_ip_len, u32 size) { return cpu_to_le64(((u64)vlan_cmd << 32) | ((u64)mac_ip_len << 16) | @@ -142,6 +187,7 @@ struct mucse_stats { #define MAX_Q_VECTORS 8 #define M_DEFAULT_TXD 512 +#define M_DEFAULT_RXD 512 #define M_DEFAULT_TX_WORK 256 struct mucse { @@ -161,6 +207,7 @@ struct mucse { int tx_work_limit; int num_tx_queues; int num_q_vectors; + int rx_ring_item_count; int num_rx_queues; }; diff --git a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_hw.h b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_hw.h index f060c39e9690..ce092edf920a 100644 --- a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_hw.h +++ b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_hw.h @@ -16,6 +16,7 @@ #define M_DEFAULT_N210_MHZ 62 #define TX_AXI_RW_EN 0xc +#define RX_AXI_RW_EN 0x03 #define RNPGBE_DMA_AXI_EN 0x0010 #define RNPGBE_LEGACY_TIME 0xd000 #define RNPGBE_LEGACY_ENABLE 0xd004 diff --git a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.c b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.c index 9153e38fdd15..3dbb697a0667 100644 --- a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.c +++ b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.c @@ -3,7 +3,9 @@ #include #include +#include #include +#include #include "rnpgbe_lib.h" #include "rnpgbe.h" @@ -147,6 +149,353 @@ static bool rnpgbe_clean_tx_irq(struct mucse_q_vector *q_vector, return total_bytes == 0; } +static bool mucse_alloc_mapped_page(struct mucse_ring *rx_ring, + struct mucse_rx_buffer *bi) +{ + struct page *page = bi->page; + dma_addr_t dma; + + if (page) + return true; + + page = page_pool_dev_alloc_pages(rx_ring->page_pool); + if (unlikely(!page)) + return false; + dma = page_pool_get_dma_addr(page); + + bi->dma = dma; + bi->page = page; + bi->page_offset = RNPGBE_SKB_PAD; + + return true; +} + +static void mucse_update_rx_tail(struct mucse_ring *rx_ring, + u32 val) +{ + rx_ring->next_to_use = val; + /* + * Force memory writes to complete before letting h/w + * know there are new descriptors to fetch. (Only + * applicable for weak-ordered memory model archs, + * such as IA-64). + */ + wmb(); + writel(val, rx_ring->tail); +} + +/** + * rnpgbe_alloc_rx_buffers - Replace used receive buffers + * @rx_ring: ring to place buffers on + * @cleaned_count: number of buffers to replace + * @return: true if alloc failed + **/ +static bool rnpgbe_alloc_rx_buffers(struct mucse_ring *rx_ring, + u16 cleaned_count) +{ + u64 fun_id = ((u64)(rx_ring->pfvfnum) << 56); + union rnpgbe_rx_desc *rx_desc; + u16 i = rx_ring->next_to_use; + struct mucse_rx_buffer *bi; + bool err = false; + u16 bufsz; + /* nothing to do */ + if (!cleaned_count) + return err; + + rx_desc = M_RX_DESC(rx_ring, i); + bi = &rx_ring->rx_buffer_info[i]; + i -= rx_ring->count; + bufsz = mucse_rx_bufsz(rx_ring); + + do { + if (!mucse_alloc_mapped_page(rx_ring, bi)) { + err = true; + break; + } + + dma_sync_single_range_for_device(rx_ring->dev, bi->dma, + bi->page_offset, bufsz, + DMA_FROM_DEVICE); + rx_desc->pkt_addr = cpu_to_le64(bi->dma + bi->page_offset + + fun_id); + + /* clean dd */ + rx_desc->resv_cmd = 0; + rx_desc++; + bi++; + i++; + if (unlikely(!i)) { + rx_desc = M_RX_DESC(rx_ring, 0); + bi = rx_ring->rx_buffer_info; + i -= rx_ring->count; + } + cleaned_count--; + } while (cleaned_count); + + i += rx_ring->count; + + if (rx_ring->next_to_use != i) + mucse_update_rx_tail(rx_ring, i); + + return err; +} + +/** + * rnpgbe_get_buffer - Get the rx_buffer to be used + * @rx_ring: pointer to rx ring + * @skb: pointer skb for this packet + * @size: data size in this desc + * @return: rx_buffer. + **/ +static struct mucse_rx_buffer *rnpgbe_get_buffer(struct mucse_ring *rx_ring, + struct sk_buff **skb, + const unsigned int size) +{ + struct mucse_rx_buffer *rx_buffer; + + rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean]; + *skb = rx_buffer->skb; + prefetchw(page_address(rx_buffer->page) + rx_buffer->page_offset); + /* we are reusing so sync this buffer for CPU use */ + dma_sync_single_range_for_cpu(rx_ring->dev, rx_buffer->dma, + rx_buffer->page_offset, size, + DMA_FROM_DEVICE); + + return rx_buffer; +} + +/** + * rnpgbe_add_rx_frag - Add no-linear data to the skb + * @rx_buffer: pointer to rx_buffer + * @skb: pointer skb for this packet + * @size: data size in this desc + **/ +static void rnpgbe_add_rx_frag(struct mucse_rx_buffer *rx_buffer, + struct sk_buff *skb, + unsigned int size) +{ + unsigned int truesize = SKB_DATA_ALIGN(RNPGBE_SKB_PAD + size); + + skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_buffer->page, + rx_buffer->page_offset, size, truesize); +} + +/** + * rnpgbe_build_skb - Try to build a sbk based on rx_buffer + * @rx_buffer: pointer to rx_buffer + * @size: data size in this desc + * @return: skb for this rx_buffer + **/ +static struct sk_buff *rnpgbe_build_skb(struct mucse_rx_buffer *rx_buffer, + unsigned int size) +{ + unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + + SKB_DATA_ALIGN(size + RNPGBE_SKB_PAD); + void *va = page_address(rx_buffer->page) + rx_buffer->page_offset; + struct sk_buff *skb; + + net_prefetch(va); + /* build an skb around the page buffer */ + skb = build_skb(va - RNPGBE_SKB_PAD, truesize); + if (unlikely(!skb)) + return NULL; + /* update pointers within the skb to store the data */ + skb_reserve(skb, RNPGBE_SKB_PAD); + __skb_put(skb, size); + skb_mark_for_recycle(skb); + + return skb; +} + +/** + * rnpgbe_pull_tail - Pull header to linear portion of buffer + * @skb: current socket buffer containing buffer in progress + **/ +static void rnpgbe_pull_tail(struct sk_buff *skb) +{ + skb_frag_t *frag = &skb_shinfo(skb)->frags[0]; + unsigned int pull_len; + unsigned char *va; + + va = skb_frag_address(frag); + pull_len = eth_get_headlen(skb->dev, va, M_RX_HDR_SIZE); + /* align pull length to size of long to optimize memcpy performance */ + skb_copy_to_linear_data(skb, va, ALIGN(pull_len, sizeof(long))); + /* update all of the pointers */ + skb_frag_size_sub(frag, pull_len); + skb_frag_off_add(frag, pull_len); + skb->data_len -= pull_len; + skb->tail += pull_len; +} + +/** + * rnpgbe_is_non_eop - Process handling of non-EOP buffers + * @rx_ring: rx ring being processed + * @rx_desc: rx descriptor for current buffer + * @skb: current socket buffer containing buffer in progress + * + * This function updates next to clean. If the buffer is an EOP buffer + * this function exits returning false, otherwise it will place the + * sk_buff in the next buffer to be chained and return true indicating + * that this is in fact a non-EOP buffer. + * + * @return: true for not end of packet + **/ +static bool rnpgbe_is_non_eop(struct mucse_ring *rx_ring, + union rnpgbe_rx_desc *rx_desc, + struct sk_buff *skb) +{ + u32 ntc = rx_ring->next_to_clean + 1; + + /* fetch, update, and store next to clean */ + ntc = (ntc < rx_ring->count) ? ntc : 0; + rx_ring->next_to_clean = ntc; + prefetch(M_RX_DESC(rx_ring, ntc)); + /* if we are the last buffer then there is nothing else to do */ + if (likely(rnpgbe_test_staterr(rx_desc, M_RXD_STAT_EOP))) + return false; + /* place skb in next buffer to be received */ + rx_ring->rx_buffer_info[ntc].skb = skb; + /* we should clean it since we used all info in it */ + rx_desc->wb.cmd = 0; + + return true; +} + +/** + * rnpgbe_cleanup_headers - Correct corrupted or empty headers + * @skb: current socket buffer containing buffer in progress + * @return: true if an error was encountered and skb was freed. + **/ +static bool rnpgbe_cleanup_headers(struct sk_buff *skb) +{ + if (IS_ERR(skb)) + return true; + /* place header in linear portion of buffer */ + if (!skb_headlen(skb)) + rnpgbe_pull_tail(skb); + /* if eth_skb_pad returns an error the skb was freed */ + if (eth_skb_pad(skb)) + return true; + + return false; +} + +/** + * rnpgbe_process_skb_fields - Setup skb header fields from desc + * @rx_ring: structure containing ring specific data + * @skb: skb currently being received and modified + * + * rnpgbe_process_skb_fields checks the ring, descriptor information + * in order to setup the hash, chksum, vlan, protocol, and other + * fields within the skb. + **/ +static void rnpgbe_process_skb_fields(struct mucse_ring *rx_ring, + struct sk_buff *skb) +{ + struct net_device *dev = rx_ring->netdev; + + skb_record_rx_queue(skb, rx_ring->queue_index); + skb->protocol = eth_type_trans(skb, dev); +} + +/** + * rnpgbe_clean_rx_irq - Clean completed descriptors from Rx ring + * @q_vector: structure containing interrupt and ring information + * @rx_ring: rx descriptor ring to transact packets on + * @budget: total limit on number of packets to process + * + * rnpgbe_clean_rx_irq tries to check dd in desc, handle this desc + * if dd is set which means data is write-back by hw + * + * @return: amount of work completed. + **/ +static int rnpgbe_clean_rx_irq(struct mucse_q_vector *q_vector, + struct mucse_ring *rx_ring, + int budget) +{ + unsigned int total_rx_bytes = 0, total_rx_packets = 0; + u16 cleaned_count = mucse_desc_unused_rx(rx_ring); + bool fail_alloc = false; + + while (likely(total_rx_packets < budget)) { + struct mucse_rx_buffer *rx_buffer; + union rnpgbe_rx_desc *rx_desc; + struct sk_buff *skb; + unsigned int size; + + if (cleaned_count >= M_RX_BUFFER_WRITE) { + if (rnpgbe_alloc_rx_buffers(rx_ring, cleaned_count)) { + fail_alloc = true; + cleaned_count = mucse_desc_unused_rx(rx_ring); + } else { + cleaned_count = 0; + } + } + rx_desc = M_RX_DESC(rx_ring, rx_ring->next_to_clean); + + if (!rnpgbe_test_staterr(rx_desc, M_RXD_STAT_DD)) + break; + + /* This memory barrier is needed to keep us from reading + * any other fields out of the rx_desc until we know the + * descriptor has been written back + */ + dma_rmb(); + size = le16_to_cpu(rx_desc->wb.len); + if (!size) + break; + + rx_buffer = rnpgbe_get_buffer(rx_ring, &skb, size); + + if (skb) + rnpgbe_add_rx_frag(rx_buffer, skb, size); + else + skb = rnpgbe_build_skb(rx_buffer, size); + /* exit if we failed to retrieve a buffer */ + if (!skb) { + dma_sync_single_range_for_device(rx_ring->dev, + rx_buffer->dma, + rx_buffer->page_offset, + size, + DMA_FROM_DEVICE); + page_pool_recycle_direct(rx_ring->page_pool, + rx_buffer->page); + break; + } + + rx_buffer->page = NULL; + rx_buffer->skb = NULL; + cleaned_count++; + + if (rnpgbe_is_non_eop(rx_ring, rx_desc, skb)) + continue; + + /* verify the packet layout is correct */ + if (rnpgbe_cleanup_headers(skb)) { + /* we should clean it since we used all info in it */ + rx_desc->wb.cmd = 0; + continue; + } + + /* probably a little skewed due to removing CRC */ + total_rx_bytes += skb->len; + rnpgbe_process_skb_fields(rx_ring, skb); + rx_desc->wb.cmd = 0; + napi_gro_receive(&q_vector->napi, skb); + /* update budget accounting */ + total_rx_packets++; + } + + u64_stats_update_begin(&rx_ring->syncp); + rx_ring->stats.packets += total_rx_packets; + rx_ring->stats.bytes += total_rx_bytes; + u64_stats_update_end(&rx_ring->syncp); + /* keep polling if alloc mem failed */ + return fail_alloc ? budget : total_rx_packets; +} + /** * rnpgbe_poll - NAPI Rx polling callback * @napi: structure for representing this polling device @@ -159,6 +508,7 @@ static int rnpgbe_poll(struct napi_struct *napi, int budget) { struct mucse_q_vector *q_vector = container_of(napi, struct mucse_q_vector, napi); + int per_ring_budget, work_done = 0; bool clean_complete = true; struct mucse_ring *ring; @@ -167,12 +517,27 @@ static int rnpgbe_poll(struct napi_struct *napi, int budget) clean_complete = false; } + if (q_vector->rx.count > 1) + per_ring_budget = max(budget / q_vector->rx.count, 1); + else + per_ring_budget = budget; + + mucse_for_each_ring(ring, q_vector->rx) { + int cleaned = 0; + + cleaned = rnpgbe_clean_rx_irq(q_vector, ring, per_ring_budget); + work_done += cleaned; + if (cleaned >= per_ring_budget) + clean_complete = false; + } + if (!clean_complete) return budget; - rnpgbe_irq_enable_queues(q_vector); + if (likely(napi_complete_done(napi, work_done))) + rnpgbe_irq_enable_queues(q_vector); - return 0; + return min(work_done, budget - 1); } /** @@ -338,12 +703,16 @@ static int rnpgbe_alloc_q_vector(struct mucse *mucse, } for (idx = 0; idx < rxr_count; idx++) { + ring->dev = &mucse->pdev->dev; mucse_add_ring(ring, &q_vector->rx); + ring->count = mucse->rx_ring_item_count; + ring->netdev = mucse->netdev; ring->queue_index = eth_queue_idx + idx; ring->rnpgbe_queue_idx = rxr_idx; ring->ring_addr = hw->hw_addr + RING_OFFSET(rxr_idx); ring->irq_mask = ring->ring_addr + RNPGBE_DMA_INT_MASK; ring->trig = ring->ring_addr + RNPGBE_DMA_INT_TRIG; + ring->pfvfnum = hw->pfvfnum; mucse->rx_ring[ring->queue_index] = ring; rxr_idx += step; ring++; @@ -776,6 +1145,16 @@ static void rnpgbe_clean_all_tx_rings(struct mucse *mucse) rnpgbe_clean_tx_ring(mucse->tx_ring[i]); } +/** + * rnpgbe_clean_all_rx_rings - Free Rx Buffers for all queues + * @mucse: board private structure + **/ +static void rnpgbe_clean_all_rx_rings(struct mucse *mucse) +{ + for (int i = 0; i < mucse->num_rx_queues; i++) + rnpgbe_clean_rx_ring(mucse->rx_ring[i]); +} + void rnpgbe_down(struct mucse *mucse) { struct net_device *netdev = mucse->netdev; @@ -785,6 +1164,7 @@ void rnpgbe_down(struct mucse *mucse) rnpgbe_irq_disable(mucse); netif_tx_disable(netdev); rnpgbe_napi_disable_all(mucse); + rnpgbe_clean_all_rx_rings(mucse); } /** @@ -799,6 +1179,8 @@ void rnpgbe_up_complete(struct mucse *mucse) rnpgbe_napi_enable_all(mucse); rnpgbe_irq_enable(mucse); netif_tx_start_all_queues(netdev); + for (int i = 0; i < mucse->num_rx_queues; i++) + mucse_ring_wr32(mucse->rx_ring[i], RNPGBE_RX_START, 1); } /** @@ -1152,5 +1534,241 @@ void rnpgbe_get_stats64(struct net_device *netdev, stats->tx_bytes += bytes; } } + + for (i = 0; i < mucse->num_rx_queues; i++) { + struct mucse_ring *ring = READ_ONCE(mucse->rx_ring[i]); + u64 bytes, packets; + unsigned int start; + + if (ring) { + do { + start = u64_stats_fetch_begin(&ring->syncp); + packets = ring->stats.packets; + bytes = ring->stats.bytes; + } while (u64_stats_fetch_retry(&ring->syncp, start)); + stats->rx_packets += packets; + stats->rx_bytes += bytes; + } + } rcu_read_unlock(); } + +static int mucse_alloc_page_pool(struct mucse_ring *rx_ring) +{ + int ret = 0; + + struct page_pool_params pp_params = { + .flags = PP_FLAG_DMA_MAP | PP_FLAG_DMA_SYNC_DEV, + .order = 0, + .pool_size = rx_ring->count, + .nid = dev_to_node(rx_ring->dev), + .dev = rx_ring->dev, + .dma_dir = DMA_FROM_DEVICE, + .offset = 0, + .max_len = PAGE_SIZE, + }; + + rx_ring->page_pool = page_pool_create(&pp_params); + if (IS_ERR(rx_ring->page_pool)) { + ret = PTR_ERR(rx_ring->page_pool); + rx_ring->page_pool = NULL; + } + + return ret; +} + +/** + * rnpgbe_setup_rx_resources - allocate Rx resources (Descriptors) + * @rx_ring: rx descriptor ring (for a specific queue) to setup + * @mucse: pointer to private structure + * + * @return: 0 on success, negative on failure + **/ +static int rnpgbe_setup_rx_resources(struct mucse_ring *rx_ring, + struct mucse *mucse) +{ + struct device *dev = rx_ring->dev; + int size; + + size = sizeof(struct mucse_rx_buffer) * rx_ring->count; + + rx_ring->rx_buffer_info = vzalloc(size); + + if (!rx_ring->rx_buffer_info) + goto err_return; + /* Round up to nearest 4K */ + rx_ring->size = rx_ring->count * sizeof(union rnpgbe_rx_desc); + rx_ring->size = ALIGN(rx_ring->size, 4096); + rx_ring->desc = dma_alloc_coherent(dev, rx_ring->size, &rx_ring->dma, + GFP_KERNEL); + if (!rx_ring->desc) + goto err_free_buffer; + + rx_ring->next_to_clean = 0; + rx_ring->next_to_use = 0; + + if (mucse_alloc_page_pool(rx_ring)) + goto err_free_desc; + + return 0; +err_free_desc: + dma_free_coherent(dev, rx_ring->size, rx_ring->desc, + rx_ring->dma); + rx_ring->desc = NULL; +err_free_buffer: + vfree(rx_ring->rx_buffer_info); +err_return: + rx_ring->rx_buffer_info = NULL; + return -ENOMEM; +} + +/** + * rnpgbe_clean_rx_ring - Free Rx Buffers per Queue + * @rx_ring: ring to free buffers from + **/ +void rnpgbe_clean_rx_ring(struct mucse_ring *rx_ring) +{ + u16 i = rx_ring->next_to_clean; + struct mucse_rx_buffer *rx_buffer = &rx_ring->rx_buffer_info[i]; + + /* Free all the Rx ring sk_buffs */ + while (i != rx_ring->next_to_use) { + if (rx_buffer->skb) { + struct sk_buff *skb = rx_buffer->skb; + + dev_kfree_skb(skb); + rx_buffer->skb = NULL; + } + dma_sync_single_range_for_cpu(rx_ring->dev, rx_buffer->dma, + rx_buffer->page_offset, + mucse_rx_bufsz(rx_ring), + DMA_FROM_DEVICE); + if (rx_buffer->page) { + page_pool_put_full_page(rx_ring->page_pool, + rx_buffer->page, false); + rx_buffer->page = NULL; + } + i++; + rx_buffer++; + if (i == rx_ring->count) { + i = 0; + rx_buffer = rx_ring->rx_buffer_info; + } + } + + rx_ring->next_to_clean = 0; + rx_ring->next_to_use = 0; +} + +/** + * rnpgbe_free_rx_resources - Free Rx Resources + * @rx_ring: ring to clean the resources from + * + * Free all receive software resources + **/ +static void rnpgbe_free_rx_resources(struct mucse_ring *rx_ring) +{ + rnpgbe_clean_rx_ring(rx_ring); + vfree(rx_ring->rx_buffer_info); + rx_ring->rx_buffer_info = NULL; + /* if not set, then don't free */ + if (!rx_ring->desc) + return; + + dma_free_coherent(rx_ring->dev, rx_ring->size, rx_ring->desc, + rx_ring->dma); + rx_ring->desc = NULL; + if (rx_ring->page_pool) { + page_pool_destroy(rx_ring->page_pool); + rx_ring->page_pool = NULL; + } +} + +/** + * rnpgbe_setup_all_rx_resources - allocate all queues Rx resources + * @mucse: pointer to private structure + * + * @return: 0 on success, negative on failure + **/ +int rnpgbe_setup_all_rx_resources(struct mucse *mucse) +{ + int i, err = 0; + + for (i = 0; i < mucse->num_rx_queues; i++) { + err = rnpgbe_setup_rx_resources(mucse->rx_ring[i], mucse); + if (!err) + continue; + + goto err_setup_rx; + } + + return 0; +err_setup_rx: + while (i--) + rnpgbe_free_rx_resources(mucse->rx_ring[i]); + return err; +} + +/** + * rnpgbe_free_all_rx_resources - Free Rx Resources for All Queues + * @mucse: pointer to private structure + * + * Free all receive software resources + **/ +void rnpgbe_free_all_rx_resources(struct mucse *mucse) +{ + for (int i = 0; i < (mucse->num_rx_queues); i++) { + if (mucse->rx_ring[i]->desc) + rnpgbe_free_rx_resources(mucse->rx_ring[i]); + } +} + +/** + * rnpgbe_configure_rx_ring - Configure Rx ring info to hw + * @mucse: pointer to private structure + * @ring: structure containing ring specific data + * + * Configure the Rx descriptor ring after a reset. + **/ +static void rnpgbe_configure_rx_ring(struct mucse *mucse, + struct mucse_ring *ring) +{ + struct mucse_hw *hw = &mucse->hw; + + /* disable queue to avoid issues while updating state */ + mucse_ring_wr32(ring, RNPGBE_RX_START, 0); + /* set descripts registers*/ + mucse_ring_wr32(ring, RNPGBE_RX_BASE_ADDR_LO, (u32)ring->dma); + mucse_ring_wr32(ring, RNPGBE_RX_BASE_ADDR_HI, + (u32)((u64)ring->dma >> 32) | (hw->pfvfnum << 24)); + mucse_ring_wr32(ring, RNPGBE_RX_LEN, ring->count); + ring->tail = ring->ring_addr + RNPGBE_RX_TAIL; + ring->next_to_clean = mucse_ring_rd32(ring, RNPGBE_RX_HEAD); + ring->next_to_use = ring->next_to_clean; + mucse_ring_wr32(ring, RNPGBE_RX_SG_LEN, M_DEFAULT_SG); + mucse_ring_wr32(ring, RNPGBE_RX_FETCH, M_DEFAULT_RX_FETCH); + mucse_ring_wr32(ring, RNPGBE_RX_TIMEOUT_TH, 0); + mucse_ring_wr32(ring, RNPGBE_RX_INT_TIMER, + M_DEFAULT_INT_TIMER_R * hw->cycles_per_us); + mucse_ring_wr32(ring, RNPGBE_RX_INT_PKTCNT, M_DEFAULT_RX_INT_PKTCNT); + rnpgbe_alloc_rx_buffers(ring, mucse_desc_unused_rx(ring)); +} + +/** + * rnpgbe_configure_rx - Configure Receive Unit after Reset + * @mucse: pointer to private structure + * + * Configure the Rx unit after a reset. + **/ +void rnpgbe_configure_rx(struct mucse *mucse) +{ + struct mucse_hw *hw = &mucse->hw; + u32 dma_axi_ctl; + + for (int i = 0; i < mucse->num_rx_queues; i++) + rnpgbe_configure_rx_ring(mucse, mucse->rx_ring[i]); + + dma_axi_ctl = mucse_hw_rd32(hw, RNPGBE_DMA_AXI_EN); + dma_axi_ctl |= RX_AXI_RW_EN; + mucse_hw_wr32(hw, RNPGBE_DMA_AXI_EN, dma_axi_ctl); +} diff --git a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.h b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.h index 2c2796764c2d..29520ad716ca 100644 --- a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.h +++ b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_lib.h @@ -8,11 +8,27 @@ struct mucse; struct mucse_ring; #define RING_OFFSET(n) (0x1000 + 0x100 * (n)) +#define RNPGBE_RX_START 0x10 #define RNPGBE_TX_START 0x18 #define RNPGBE_DMA_INT_MASK 0x24 #define TX_INT_MASK BIT(1) #define RX_INT_MASK BIT(0) #define INT_VALID (BIT(16) | BIT(17)) +#define RNPGBE_RX_BASE_ADDR_HI 0x30 +#define RNPGBE_RX_BASE_ADDR_LO 0x34 +#define RNPGBE_RX_LEN 0x38 +#define RNPGBE_RX_HEAD 0x3c +#define RNPGBE_RX_TAIL 0x40 +#define M_DEFAULT_RX_FETCH 0x100020 +#define RNPGBE_RX_FETCH 0x44 +#define M_DEFAULT_INT_TIMER_R 30 +#define RNPGBE_RX_INT_TIMER 0x48 +#define M_DEFAULT_RX_INT_PKTCNT 64 +#define RNPGBE_RX_INT_PKTCNT 0x4c +#define RNPGBE_RX_ARB_DEF_LVL 0x50 +#define RNPGBE_RX_TIMEOUT_TH 0x54 +#define M_DEFAULT_SG 96 /* unit 16b, 1536 bytes */ +#define RNPGBE_RX_SG_LEN 0x58 #define RNPGBE_TX_BASE_ADDR_HI 0x60 #define RNPGBE_TX_BASE_ADDR_LO 0x64 #define RNPGBE_TX_LEN 0x68 @@ -33,13 +49,23 @@ struct mucse_ring; #define M_MAX_DATA_PER_TXD (0x1 << M_MAX_TXD_PWR) #define TXD_USE_COUNT(S) DIV_ROUND_UP((S), M_MAX_DATA_PER_TXD) #define DESC_NEEDED (MAX_SKB_FRAGS + 4) +#define RNPGBE_SKB_PAD (NET_SKB_PAD + NET_IP_ALIGN) +#define M_RXBUFFER_1536 1536 +#define M_RX_BUFFER_WRITE 16 +#define M_RX_HDR_SIZE 256 + +static inline unsigned int mucse_rx_bufsz(struct mucse_ring *ring) +{ + /* 1536 is enough for mtu 1500 packets */ + return (M_RXBUFFER_1536 - NET_IP_ALIGN); +} + /* hw require this not zero */ #define M_DEFAULT_MAC_IP_LEN 20 #define mucse_for_each_ring(pos, head)\ for (typeof((head).ring) __pos = (head).ring;\ __pos ? ({ pos = __pos; 1; }) : 0;\ __pos = __pos->next) - int rnpgbe_init_interrupt_scheme(struct mucse *mucse); void rnpgbe_clear_interrupt_scheme(struct mucse *mucse); int register_mbx_irq(struct mucse *mucse); @@ -50,10 +76,14 @@ void rnpgbe_irq_disable(struct mucse *mucse); void rnpgbe_down(struct mucse *mucse); void rnpgbe_up_complete(struct mucse *mucse); void rnpgbe_configure_tx(struct mucse *mucse); +void rnpgbe_configure_rx(struct mucse *mucse); int rnpgbe_setup_all_tx_resources(struct mucse *mucse); void rnpgbe_free_all_tx_resources(struct mucse *mucse); netdev_tx_t rnpgbe_xmit_frame_ring(struct sk_buff *skb, struct mucse_ring *tx_ring); void rnpgbe_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *stats); +void rnpgbe_clean_rx_ring(struct mucse_ring *rx_ring); +int rnpgbe_setup_all_rx_resources(struct mucse *mucse); +void rnpgbe_free_all_rx_resources(struct mucse *mucse); #endif diff --git a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c index 6c9ff8a6a0bf..413eefae65dd 100644 --- a/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c +++ b/drivers/net/ethernet/mucse/rnpgbe/rnpgbe_main.c @@ -35,6 +35,7 @@ static struct pci_device_id rnpgbe_pci_tbl[] = { static void rnpgbe_configure(struct mucse *mucse) { rnpgbe_configure_tx(mucse); + rnpgbe_configure_rx(mucse); } /** @@ -63,11 +64,17 @@ static int rnpgbe_open(struct net_device *netdev) err = rnpgbe_setup_all_tx_resources(mucse); if (err) goto err_free_irqs; + err = rnpgbe_setup_all_rx_resources(mucse); + if (err) + goto err_free_tx; + rnpgbe_configure(mucse); rnpgbe_up_complete(mucse); return 0; +err_free_tx: + rnpgbe_free_all_tx_resources(mucse); err_free_irqs: rnpgbe_free_irq(mucse); return err; @@ -89,6 +96,7 @@ static int rnpgbe_close(struct net_device *netdev) rnpgbe_down(mucse); rnpgbe_free_irq(mucse); rnpgbe_free_all_tx_resources(mucse); + rnpgbe_free_all_rx_resources(mucse); return 0; } @@ -121,6 +129,7 @@ static const struct net_device_ops rnpgbe_netdev_ops = { static void rnpgbe_sw_init(struct mucse *mucse) { mucse->tx_ring_item_count = M_DEFAULT_TXD; + mucse->rx_ring_item_count = M_DEFAULT_RXD; mucse->tx_work_limit = M_DEFAULT_TX_WORK; } -- 2.25.1