[PATCH net] net: rps: send out pending IPI's on CPU hotplug

[PATCH net] net: rps: send out pending IPI's on CPU hotplug

[ashwanth]

IPI's from the victim cpu are not handled in dev_cpu_callback.
So these pending IPI's would be sent to the remote cpu only when
NET_RX is scheduled on the victim cpu and since this trigger is
unpredictable it would result in packet latencies on the remote cpu.

This patch adds support to send the pending ipi's of victim cpu.

Signed-off-by: Ashwanth Goli <ashwanth@codeaurora.org>
---
  net/core/dev.c | 31 ++++++++++++++++++++++---------
  1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/net/core/dev.c b/net/core/dev.c
index fca407b..e6bfa54 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4948,6 +4948,19 @@ __sum16 __skb_gro_checksum_complete(struct 
sk_buff *skb)
  }
  EXPORT_SYMBOL(__skb_gro_checksum_complete);

+static void net_rps_send_ipi(struct softnet_data *remsd)
+{
+#ifdef CONFIG_RPS
+	while (remsd) {
+		struct softnet_data *next = remsd->rps_ipi_next;
+
+		if (cpu_online(remsd->cpu))
+			smp_call_function_single_async(remsd->cpu, &remsd->csd);
+		remsd = next;
+	}
+#endif
+}
+
  /*
   * net_rps_action_and_irq_enable sends any pending IPI's for rps.
   * Note: called with local irq disabled, but exits with local irq 
enabled.
@@ -4963,14 +4976,7 @@ static void net_rps_action_and_irq_enable(struct 
softnet_data *sd)
  		local_irq_enable();

  		/* Send pending IPI's to kick RPS processing on remote cpus. */
-		while (remsd) {
-			struct softnet_data *next = remsd->rps_ipi_next;
-
-			if (cpu_online(remsd->cpu))
-				smp_call_function_single_async(remsd->cpu,
-							   &remsd->csd);
-			remsd = next;
-		}
+		net_rps_send_ipi(remsd);
  	} else
  #endif
  		local_irq_enable();
@@ -8192,7 +8198,7 @@ static int dev_cpu_dead(unsigned int oldcpu)
  	struct sk_buff **list_skb;
  	struct sk_buff *skb;
  	unsigned int cpu;
-	struct softnet_data *sd, *oldsd;
+	struct softnet_data *sd, *oldsd, *remsd;

  	local_irq_disable();
  	cpu = smp_processor_id();
@@ -8233,6 +8239,13 @@ static int dev_cpu_dead(unsigned int oldcpu)
  	raise_softirq_irqoff(NET_TX_SOFTIRQ);
  	local_irq_enable();

+#ifdef CONFIG_RPS
+	remsd = oldsd->rps_ipi_list;
+	oldsd->rps_ipi_list = NULL;
+#endif
+	/* send out pending IPI's on offline CPU */
+	net_rps_send_ipi(remsd);
+
  	/* Process offline CPU's input_pkt_queue */
  	while ((skb = __skb_dequeue(&oldsd->process_queue))) {
  		netif_rx_ni(skb);
-- 
1.9.1

Re: [PATCH net] net: rps: send out pending IPI's on CPU hotplug

[Eric Dumazet]

On Tue, 2017-06-06 at 20:47 +0530, ashwanth@codeaurora.org wrote:
> IPI's from the victim cpu are not handled in dev_cpu_callback.
> So these pending IPI's would be sent to the remote cpu only when
> NET_RX is scheduled on the victim cpu and since this trigger is
> unpredictable it would result in packet latencies on the remote cpu.
> 
> This patch adds support to send the pending ipi's of victim cpu.
> 
> Signed-off-by: Ashwanth Goli <ashwanth@codeaurora.org>
> ---

Acked-by: Eric Dumazet <edumazet@google.com>

Thanks !

Re: [PATCH net] net: rps: send out pending IPI's on CPU hotplug

[David Miller]

From: ashwanth@codeaurora.org
Date: Tue, 06 Jun 2017 20:47:36 +0530

> IPI's from the victim cpu are not handled in dev_cpu_callback.
> So these pending IPI's would be sent to the remote cpu only when
> NET_RX is scheduled on the victim cpu and since this trigger is
> unpredictable it would result in packet latencies on the remote cpu.
> 
> This patch adds support to send the pending ipi's of victim cpu.
> 
> Signed-off-by: Ashwanth Goli <ashwanth@codeaurora.org>
> ---
>  net/core/dev.c | 31 ++++++++++++++++++++++---------
>  1 file changed, 22 insertions(+), 9 deletions(-)
> 
> diff --git a/net/core/dev.c b/net/core/dev.c
> index fca407b..e6bfa54 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -4948,6 +4948,19 @@ __sum16 __skb_gro_checksum_complete(struct
> sk_buff *skb)

This patch has been severely corrupted by your email client.

Please read Documentation/email-clients.txt, fix things up, send
a test email to yourself, and only resubmit this patch once you
are able yourself to successfully apply the patch that arrives
in that test email.

Thank you.