Netdev List
 help / color / mirror / Atom feed
* Re: [PATCH net-next] mlx4: optimize xmit path
@ 2014-09-28 18:07 Alexei Starovoitov
  2014-09-28 18:52 ` Eric Dumazet
  0 siblings, 1 reply; 12+ messages in thread
From: Alexei Starovoitov @ 2014-09-28 18:07 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Or Gerlitz, David S. Miller, Jesper Dangaard Brouer, Eric Dumazet,
	John Fastabend, Linux Netdev List, Amir Vadai, Or Gerlitz

On Sat, Sep 27, 2014 at 3:56 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> First I implemented skb->xmit_more support, and pktgen throughput
> went from ~5Mpps to ~10Mpps.
>
> Then, looking closely at this driver I found false sharing problems that
> should be addressed by this patch, as my pktgen now reaches 14.7 Mpps
> on a single TX queue, with a burst factor of 8.
>
> So this patch in a whole permits to improve raw performance on a single
> TX queue from about 5 Mpps to 14.7 Mpps.

this is great improvement!
Thank you for leading this effort.
10G line rate is definitely nice :)
Hopefully Or can demo similar numbers with 40G nic as well :)

> +       if (ring->bf_enabled && desc_size <= MAX_BF && !bounce &&
> +           !vlan_tx_tag_present(skb) && send_doorbell) {

feels something wrong here, since it checks for
send_doorbell, but iowrite() happens in the 'else' part
of this branch with another 'if (send_doorbell)'

The previous code seems equally confusing to me.

> +               tx_desc->ctrl.bf_qpn = ring->doorbell_qpn |
> +                                      cpu_to_be32(real_size);
>
>                 op_own |= htonl((bf_index & 0xffff) << 8);
> -               /* Ensure new descirptor hits memory
> -               * before setting ownership of this descriptor to HW */
> +               /* Ensure new descriptor hits memory
> +                * before setting ownership of this descriptor to HW
> +                */
>                 wmb();
>                 tx_desc->ctrl.owner_opcode = op_own;
>
>                 wmb();
>
> -               mlx4_bf_copy(ring->bf.reg + ring->bf.offset, (unsigned long *) &tx_desc->ctrl,
> -                    desc_size);
> +               mlx4_bf_copy(ring->bf.reg + ring->bf.offset,
> +                            &tx_desc->ctrl,
> +                            desc_size);
>
>                 wmb();
>
>                 ring->bf.offset ^= ring->bf.buf_size;
>         } else {
> -               /* Ensure new descirptor hits memory
> -               * before setting ownership of this descriptor to HW */
> +               tx_desc->ctrl.vlan_tag = cpu_to_be16(vlan_tag);
> +               tx_desc->ctrl.ins_vlan = MLX4_WQE_CTRL_INS_VLAN *
> +                       !!vlan_tx_tag_present(skb);
> +               tx_desc->ctrl.fence_size = real_size;
> +
> +               /* Ensure new descriptor hits memory
> +                * before setting ownership of this descriptor to HW
> +                */
>                 wmb();
>                 tx_desc->ctrl.owner_opcode = op_own;
> -               wmb();
> -               iowrite32be(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
> +
> +               if (send_doorbell) {
> +                       wmb(); /* ensure owner_opcode is written */
> +                       iowrite32(ring->doorbell_qpn,
> +                                 ring->bf.uar->map + MLX4_SEND_DOORBELL);
> +               }

shinfo, prefetch and access_once additions all look useful to me.

Thanks!

^ permalink raw reply	[flat|nested] 12+ messages in thread
* [RFC PATCH net-next] net: pktgen: packet bursting via skb->xmit_more
@ 2014-09-26  0:46 Alexei Starovoitov
  2014-09-26  1:20 ` Eric Dumazet
  0 siblings, 1 reply; 12+ messages in thread
From: Alexei Starovoitov @ 2014-09-26  0:46 UTC (permalink / raw)
  To: David S. Miller
  Cc: Jesper Dangaard Brouer, Eric Dumazet, John Fastabend, netdev

This patch demonstrates the effect of delaying update of HW tailptr.
(based on earlier patch by Jesper)

burst=1 is a default. It sends one packet with xmit_more=false
burst=2 sends one packet with xmit_more=true and
        2nd copy of the same packet with xmit_more=false
burst=3 sends two copies of the same packet with xmit_more=true and
        3rd copy with xmit_more=false

Performance with ixgbe:

usec 30:
burst=1  tx:9.2 Mpps
burst=2  tx:13.6 Mpps
burst=3  tx:14.5 Mpps full 10G line rate

usec 1 (default):
burst=1,4,100 tx:3.9 Mpps

usec 0:
burst=1  tx:4.9 Mpps
burst=2  tx:6.6 Mpps
burst=3  tx:7.9 Mpps
burst=4  tx:8.7 Mpps
burst=8  tx:10.3 Mpps
burst=128  tx:12.4 Mpps

Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
---

tx queue size, irq affinity left in default.
pause frames are off.

Nice to finally see line rate generated by one cpu

Comparing to Jesper patch this one amortizes the cost
of spin_lock and atomic_inc by doing HARD_TX_LOCK and
atomic_add(N) once across N packets.

 net/core/pktgen.c |   33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index 5c728aa..47557ba 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -387,6 +387,7 @@ struct pktgen_dev {
 	u16 queue_map_min;
 	u16 queue_map_max;
 	__u32 skb_priority;	/* skb priority field */
+	int burst;		/* number of duplicated packets to burst */
 	int node;               /* Memory node */
 
 #ifdef CONFIG_XFRM
@@ -613,6 +614,9 @@ static int pktgen_if_show(struct seq_file *seq, void *v)
 	if (pkt_dev->traffic_class)
 		seq_printf(seq, "     traffic_class: 0x%02x\n", pkt_dev->traffic_class);
 
+	if (pkt_dev->burst > 1)
+		seq_printf(seq, "     burst: %d\n", pkt_dev->burst);
+
 	if (pkt_dev->node >= 0)
 		seq_printf(seq, "     node: %d\n", pkt_dev->node);
 
@@ -1124,6 +1128,16 @@ static ssize_t pktgen_if_write(struct file *file,
 			pkt_dev->dst_mac_count);
 		return count;
 	}
+	if (!strcmp(name, "burst")) {
+		len = num_arg(&user_buffer[i], 10, &value);
+		if (len < 0)
+			return len;
+
+		i += len;
+		pkt_dev->burst = value < 1 ? 1 : value;
+		sprintf(pg_result, "OK: burst=%d", pkt_dev->burst);
+		return count;
+	}
 	if (!strcmp(name, "node")) {
 		len = num_arg(&user_buffer[i], 10, &value);
 		if (len < 0)
@@ -3299,7 +3313,8 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 {
 	struct net_device *odev = pkt_dev->odev;
 	struct netdev_queue *txq;
-	int ret;
+	int burst_cnt, ret;
+	bool more;
 
 	/* If device is offline, then don't send */
 	if (unlikely(!netif_running(odev) || !netif_carrier_ok(odev))) {
@@ -3347,8 +3362,14 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 		pkt_dev->last_ok = 0;
 		goto unlock;
 	}
-	atomic_inc(&(pkt_dev->skb->users));
-	ret = netdev_start_xmit(pkt_dev->skb, odev, txq, false);
+	atomic_add(pkt_dev->burst, &pkt_dev->skb->users);
+
+	burst_cnt = 0;
+
+xmit_more:
+	more = ++burst_cnt < pkt_dev->burst;
+
+	ret = netdev_start_xmit(pkt_dev->skb, odev, txq, more);
 
 	switch (ret) {
 	case NETDEV_TX_OK:
@@ -3356,6 +3377,8 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 		pkt_dev->sofar++;
 		pkt_dev->seq_num++;
 		pkt_dev->tx_bytes += pkt_dev->last_pkt_size;
+		if (more)
+			goto xmit_more;
 		break;
 	case NET_XMIT_DROP:
 	case NET_XMIT_CN:
@@ -3374,6 +3397,9 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 		atomic_dec(&(pkt_dev->skb->users));
 		pkt_dev->last_ok = 0;
 	}
+
+	if (unlikely(pkt_dev->burst - burst_cnt > 0))
+		atomic_sub(pkt_dev->burst - burst_cnt, &pkt_dev->skb->users);
 unlock:
 	HARD_TX_UNLOCK(odev, txq);
 
@@ -3572,6 +3598,7 @@ static int pktgen_add_device(struct pktgen_thread *t, const char *ifname)
 	pkt_dev->svlan_p = 0;
 	pkt_dev->svlan_cfi = 0;
 	pkt_dev->svlan_id = 0xffff;
+	pkt_dev->burst = 1;
 	pkt_dev->node = -1;
 
 	err = pktgen_setup_dev(t->net, pkt_dev, ifname);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 12+ messages in thread
end of thread, other threads:[~2014-09-29  5:08 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-28 18:07 [PATCH net-next] mlx4: optimize xmit path Alexei Starovoitov
2014-09-28 18:52 ` Eric Dumazet
2014-09-28 20:49   ` Alexei Starovoitov
2014-09-29  2:22     ` Eric Dumazet
2014-09-29  5:08       ` Alexei Starovoitov
  -- strict thread matches above, loose matches on Subject: below --
2014-09-26  0:46 [RFC PATCH net-next] net: pktgen: packet bursting via skb->xmit_more Alexei Starovoitov
2014-09-26  1:20 ` Eric Dumazet
2014-09-26  7:42   ` Eric Dumazet
2014-09-27 20:43     ` Eric Dumazet
2014-09-27 20:55       ` Or Gerlitz
2014-09-27 21:30         ` Eric Dumazet
2014-09-27 22:56           ` [PATCH net-next] mlx4: optimize xmit path Eric Dumazet
2014-09-27 23:44             ` Hannes Frederic Sowa
2014-09-28  0:05               ` Eric Dumazet
2014-09-28  0:22                 ` Hannes Frederic Sowa
2014-09-28 12:42             ` Eric Dumazet
2014-09-28 14:35             ` Or Gerlitz
2014-09-28 16:03               ` Eric Dumazet

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox