Netdev List
 help / color / mirror / Atom feed
* Re: [PATCH 1/2] rps: core implementation
From: Eric Dumazet @ 2009-11-20  6:49 UTC (permalink / raw)
  To: Tom Herbert; +Cc: Andi Kleen, David Miller, netdev
In-Reply-To: <65634d660911192241q30a3928bg44854fc929dae218@mail.gmail.com>

Tom Herbert a écrit :
> 
> So is send_remote_softirq also broken according to this?

I found this function not usable as is, anyway.
I tried to use it for XPS but failed.

Before calling it, we must put the work in a remote queue,
but if work has to be done by current cpu, we cannot remove this
work to put it in our queue. (We dont have a status from send_remote_softirq())



^ permalink raw reply

* Re: [PATCH] RDMA/addr: Use appropriate locking with for_each_netdev()
From: Roland Dreier @ 2009-11-20  6:48 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Sean Hefty, David S. Miller, Linux Netdev List, Roland Dreier,
	Hal Rosenstock, linux-rdma
In-Reply-To: <4B0628E0.6000404-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>


 > Any news of this patch ?
 > 
 > We need it for 2.6.32 and also to prepare 2.6.33 with upcoming patches.

Sorry for not explicitly emailing about this patch.  I applied it as
part of Sean's series, but I just added it to my 2.6.33 queue.  At this
point in the release cycle it doesn't look severe enough for 2.6.32,
unless I miss something.

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply

* Re: [PATCH 1/2] rps: core implementation
From: Tom Herbert @ 2009-11-20  6:41 UTC (permalink / raw)
  To: Andi Kleen; +Cc: David Miller, netdev
In-Reply-To: <20091119100805.GA22938@basil.fritz.box>

>> What are the standard deadlocks?  Looks like __send_remote_softirq
>> will call __smp_call_function with irq's disabled...
>
> The traditional deadlock (that was before the queue smp_call_function)
> was
>
> A                        B
>                         grab lock
> interrupts off
> spin on lock
>                         send IPI
>                         wait for specific CPU
>
> never answers because
> interrupts are off
>                         hangs forever
>
>
> I think with the queued smp_call_function it's better because
> the locks are only hold much shorter and that particular scenario
> is gone, but I'm not sure the problem has fully gone away.
>
> At least there are still plenty of WARN_ON( ... irqs_disabled()) in
> kernel/smp.c
>

So is send_remote_softirq also broken according to this?

^ permalink raw reply

* Re: [PATCH 1/1] Defer skb allocation for both mergeable buffers and big packets in virtio_net
From: Eric Dumazet @ 2009-11-20  6:19 UTC (permalink / raw)
  To: Shirley Ma
  Cc: Michael S. Tsirkin, Avi Kivity, Rusty Russell, netdev, kvm,
	linux-kernel
In-Reply-To: <1258697745.7416.20.camel@localhost.localdomain>

Shirley Ma a écrit :
> This patch is generated against 2.6 git tree. I didn't break up this
> patch since it has one functionality. Please review it.
> 
> Thanks
> Shirley
> 
> Signed-off-by: Shirley Ma <xma@us.ibm.com>
> ------
>  
> +void virtio_free_pages(void *buf)
> +{
> +	struct page *page = (struct page *)buf;
> +
> +	while (page) {
> +		__free_pages(page, 0);
> +		page = (struct page *)page->private;
> +	}
> +}
> +

Interesting use after free :)

^ permalink raw reply

* [PATCH 1/1] Defer skb allocation for both mergeable buffers and big packets in virtio_net
From: Shirley Ma @ 2009-11-20  6:15 UTC (permalink / raw)
  To: Michael S. Tsirkin, Avi Kivity, Rusty Russell, netdev, kvm,
	linux-kernel

This patch is generated against 2.6 git tree. I didn't break up this
patch since it has one functionality. Please review it.

Thanks
Shirley

Signed-off-by: Shirley Ma <xma@us.ibm.com>
------

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index b9e002f..6fb788b 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -80,33 +80,48 @@ static inline struct skb_vnet_hdr *skb_vnet_hdr(struct sk_buff *skb)
 	return (struct skb_vnet_hdr *)skb->cb;
 }
 
-static void give_a_page(struct virtnet_info *vi, struct page *page)
+static void give_pages(struct virtnet_info *vi, struct page *page)
 {
-	page->private = (unsigned long)vi->pages;
+	struct page *npage = (struct page *)page->private;
+
+	if (!npage)
+		page->private = (unsigned long)vi->pages;
+	else {
+		/* give a page list */
+		while (npage) {
+			if (npage->private == (unsigned long)0) {
+				npage->private = (unsigned long)vi->pages;
+				break;
+			}
+			npage = (struct page *)npage->private;
+		}
+	}
 	vi->pages = page;
 }
 
-static void trim_pages(struct virtnet_info *vi, struct sk_buff *skb)
-{
-	unsigned int i;
-
-	for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
-		give_a_page(vi, skb_shinfo(skb)->frags[i].page);
-	skb_shinfo(skb)->nr_frags = 0;
-	skb->data_len = 0;
-}
-
 static struct page *get_a_page(struct virtnet_info *vi, gfp_t gfp_mask)
 {
 	struct page *p = vi->pages;
 
-	if (p)
+	if (p) {
 		vi->pages = (struct page *)p->private;
-	else
+		/* use private to chain big packets */
+		p->private = (unsigned long)0;
+	} else
 		p = alloc_page(gfp_mask);
 	return p;
 }
 
+void virtio_free_pages(void *buf)
+{
+	struct page *page = (struct page *)buf;
+
+	while (page) {
+		__free_pages(page, 0);
+		page = (struct page *)page->private;
+	}
+}
+
 static void skb_xmit_done(struct virtqueue *svq)
 {
 	struct virtnet_info *vi = svq->vdev->priv;
@@ -118,12 +133,36 @@ static void skb_xmit_done(struct virtqueue *svq)
 	netif_wake_queue(vi->dev);
 }
 
-static void receive_skb(struct net_device *dev, struct sk_buff *skb,
+static int set_skb_frags(struct sk_buff *skb, struct page *page,
+				int offset, int len)
+{
+	int i = skb_shinfo(skb)->nr_frags;
+	skb_frag_t *f;
+
+	i = skb_shinfo(skb)->nr_frags;
+	f = &skb_shinfo(skb)->frags[i];
+	f->page = page;
+	f->page_offset = offset;
+
+	if (len > (PAGE_SIZE - f->page_offset))
+		f->size = PAGE_SIZE - f->page_offset;
+	else
+		f->size = len;
+
+	skb_shinfo(skb)->nr_frags++;
+	skb->data_len += f->size;
+	skb->len += f->size;
+
+	len -= f->size;
+	return len;
+}
+
+static void receive_skb(struct net_device *dev, void *buf,
 			unsigned len)
 {
 	struct virtnet_info *vi = netdev_priv(dev);
-	struct skb_vnet_hdr *hdr = skb_vnet_hdr(skb);
-	int err;
+	struct skb_vnet_hdr *hdr;
+	struct sk_buff *skb;
 	int i;
 
 	if (unlikely(len < sizeof(struct virtio_net_hdr) + ETH_HLEN)) {
@@ -132,39 +171,71 @@ static void receive_skb(struct net_device *dev, struct sk_buff *skb,
 		goto drop;
 	}
 
-	if (vi->mergeable_rx_bufs) {
-		unsigned int copy;
-		char *p = page_address(skb_shinfo(skb)->frags[0].page);
+	if (!vi->mergeable_rx_bufs && !vi->big_packets) {
+		skb = (struct sk_buff *)buf;
+
+		__skb_unlink(skb, &vi->recv);
+
+		hdr = skb_vnet_hdr(skb);
+		len -= sizeof(hdr->hdr);
+		skb_trim(skb, len);
+	} else {
+		struct page *page = (struct page *)buf;
+		int copy, hdr_len, num_buf, offset;
+		char *p;
+
+		p = page_address(page);
 
-		if (len > PAGE_SIZE)
-			len = PAGE_SIZE;
-		len -= sizeof(struct virtio_net_hdr_mrg_rxbuf);
+		skb = netdev_alloc_skb(vi->dev, GOOD_COPY_LEN + NET_IP_ALIGN);
+		if (unlikely(!skb)) {
+			dev->stats.rx_dropped++;
+			return;
+		}
+		skb_reserve(skb, NET_IP_ALIGN);
+		hdr = skb_vnet_hdr(skb);
 
-		memcpy(&hdr->mhdr, p, sizeof(hdr->mhdr));
-		p += sizeof(hdr->mhdr);
+		if (vi->mergeable_rx_bufs) {
+			hdr_len = sizeof(hdr->mhdr);
+			memcpy(&hdr->mhdr, p, hdr_len);
+			num_buf = hdr->mhdr.num_buffers;
+			offset = hdr_len;
+			if (len > PAGE_SIZE)
+				len = PAGE_SIZE;
+		} else {
+			/* big packtes 6 bytes alignment between virtio_net
+			 * header and data */
+			hdr_len = sizeof(hdr->hdr);
+			memcpy(&hdr->hdr, p, hdr_len);
+			offset = hdr_len + 6;
+		}
+
+		p += offset;
 
+		len -= hdr_len;
 		copy = len;
 		if (copy > skb_tailroom(skb))
 			copy = skb_tailroom(skb);
-
 		memcpy(skb_put(skb, copy), p, copy);
 
 		len -= copy;
 
-		if (!len) {
-			give_a_page(vi, skb_shinfo(skb)->frags[0].page);
-			skb_shinfo(skb)->nr_frags--;
-		} else {
-			skb_shinfo(skb)->frags[0].page_offset +=
-				sizeof(hdr->mhdr) + copy;
-			skb_shinfo(skb)->frags[0].size = len;
-			skb->data_len += len;
-			skb->len += len;
+		if (!len)
+			give_pages(vi, page);
+		else {
+			len = set_skb_frags(skb, page, copy + offset, len);
+			/* process big packets */
+			while (len > 0) {
+				page = (struct page *)page->private;
+				if (!page)
+					break;
+				len = set_skb_frags(skb, page, 0, len);
+			}
+			if (page && page->private)
+				give_pages(vi, (struct page *)page->private);
 		}
 
-		while (--hdr->mhdr.num_buffers) {
-			struct sk_buff *nskb;
-
+		/* process mergeable buffers */
+		while (vi->mergeable_rx_bufs && --num_buf) {
 			i = skb_shinfo(skb)->nr_frags;
 			if (i >= MAX_SKB_FRAGS) {
 				pr_debug("%s: packet too long %d\n", dev->name,
@@ -173,41 +244,20 @@ static void receive_skb(struct net_device *dev, struct sk_buff *skb,
 				goto drop;
 			}
 
-			nskb = vi->rvq->vq_ops->get_buf(vi->rvq, &len);
-			if (!nskb) {
+			page = vi->rvq->vq_ops->get_buf(vi->rvq, &len);
+			if (!page) {
 				pr_debug("%s: rx error: %d buffers missing\n",
 					 dev->name, hdr->mhdr.num_buffers);
 				dev->stats.rx_length_errors++;
 				goto drop;
 			}
 
-			__skb_unlink(nskb, &vi->recv);
-			vi->num--;
-
-			skb_shinfo(skb)->frags[i] = skb_shinfo(nskb)->frags[0];
-			skb_shinfo(nskb)->nr_frags = 0;
-			kfree_skb(nskb);
-
 			if (len > PAGE_SIZE)
 				len = PAGE_SIZE;
 
-			skb_shinfo(skb)->frags[i].size = len;
-			skb_shinfo(skb)->nr_frags++;
-			skb->data_len += len;
-			skb->len += len;
-		}
-	} else {
-		len -= sizeof(hdr->hdr);
-
-		if (len <= MAX_PACKET_LEN)
-			trim_pages(vi, skb);
+			set_skb_frags(skb, page, 0, len);
 
-		err = pskb_trim(skb, len);
-		if (err) {
-			pr_debug("%s: pskb_trim failed %i %d\n", dev->name,
-				 len, err);
-			dev->stats.rx_dropped++;
-			goto drop;
+			vi->num--;
 		}
 	}
 
@@ -271,107 +321,105 @@ drop:
 	dev_kfree_skb(skb);
 }
 
-static bool try_fill_recv_maxbufs(struct virtnet_info *vi, gfp_t gfp)
+/* Returns false if we couldn't fill entirely (OOM). */
+static bool try_fill_recv(struct virtnet_info *vi, gfp_t gfp)
 {
-	struct sk_buff *skb;
 	struct scatterlist sg[2+MAX_SKB_FRAGS];
-	int num, err, i;
+	int err = 0;
 	bool oom = false;
 
 	sg_init_table(sg, 2+MAX_SKB_FRAGS);
 	do {
-		struct skb_vnet_hdr *hdr;
-
-		skb = netdev_alloc_skb(vi->dev, MAX_PACKET_LEN + NET_IP_ALIGN);
-		if (unlikely(!skb)) {
-			oom = true;
-			break;
-		}
-
-		skb_reserve(skb, NET_IP_ALIGN);
-		skb_put(skb, MAX_PACKET_LEN);
-
-		hdr = skb_vnet_hdr(skb);
-		sg_set_buf(sg, &hdr->hdr, sizeof(hdr->hdr));
-
-		if (vi->big_packets) {
-			for (i = 0; i < MAX_SKB_FRAGS; i++) {
-				skb_frag_t *f = &skb_shinfo(skb)->frags[i];
-				f->page = get_a_page(vi, gfp);
-				if (!f->page)
-					break;
-
-				f->page_offset = 0;
-				f->size = PAGE_SIZE;
-
-				skb->data_len += PAGE_SIZE;
-				skb->len += PAGE_SIZE;
-
-				skb_shinfo(skb)->nr_frags++;
+		/* allocate skb for MAX_PACKET_LEN len */
+		if (!vi->big_packets && !vi->mergeable_rx_bufs) {
+			struct skb_vnet_hdr *hdr;
+			struct sk_buff *skb;
+
+			skb = netdev_alloc_skb(vi->dev,
+					       MAX_PACKET_LEN + NET_IP_ALIGN);
+			if (unlikely(!skb)) {
+				oom = true;
+				break;
 			}
-		}
-
-		num = skb_to_sgvec(skb, sg+1, 0, skb->len) + 1;
-		skb_queue_head(&vi->recv, skb);
-
-		err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, num, skb);
-		if (err < 0) {
-			skb_unlink(skb, &vi->recv);
-			trim_pages(vi, skb);
-			kfree_skb(skb);
-			break;
-		}
-		vi->num++;
-	} while (err >= num);
-	if (unlikely(vi->num > vi->max))
-		vi->max = vi->num;
-	vi->rvq->vq_ops->kick(vi->rvq);
-	return !oom;
-}
-
-/* Returns false if we couldn't fill entirely (OOM). */
-static bool try_fill_recv(struct virtnet_info *vi, gfp_t gfp)
-{
-	struct sk_buff *skb;
-	struct scatterlist sg[1];
-	int err;
-	bool oom = false;
 
-	if (!vi->mergeable_rx_bufs)
-		return try_fill_recv_maxbufs(vi, gfp);
+			skb_reserve(skb, NET_IP_ALIGN);
+			skb_put(skb, MAX_PACKET_LEN);
 
-	do {
-		skb_frag_t *f;
+			hdr = skb_vnet_hdr(skb);
+			sg_set_buf(sg, &hdr->hdr, sizeof(hdr->hdr));
 
-		skb = netdev_alloc_skb(vi->dev, GOOD_COPY_LEN + NET_IP_ALIGN);
-		if (unlikely(!skb)) {
-			oom = true;
-			break;
-		}
-
-		skb_reserve(skb, NET_IP_ALIGN);
-
-		f = &skb_shinfo(skb)->frags[0];
-		f->page = get_a_page(vi, gfp);
-		if (!f->page) {
-			oom = true;
-			kfree_skb(skb);
-			break;
-		}
+			skb_to_sgvec(skb, sg+1, 0, skb->len);
+			skb_queue_head(&vi->recv, skb);
 
-		f->page_offset = 0;
-		f->size = PAGE_SIZE;
-
-		skb_shinfo(skb)->nr_frags++;
+			err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, 2, skb);
+			if (err < 0) {
+				skb_unlink(skb, &vi->recv);
+				kfree_skb(skb);
+				break;
+			}
 
-		sg_init_one(sg, page_address(f->page), PAGE_SIZE);
-		skb_queue_head(&vi->recv, skb);
+		} else {
+			struct page *first_page = NULL;
+			struct page *page;
+			int i = MAX_SKB_FRAGS + 2;
+			char *p;
+
+			/*
+			 * chain pages for big packets, allocate skb
+			 * late for both big packets and mergeable
+			 * buffers
+			 */
+more:			page = get_a_page(vi, gfp);
+			if (!page) {
+				if (first_page)
+					give_pages(vi, first_page);
+				oom = true;
+				break;
+			}
 
-		err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0, 1, skb);
-		if (err < 0) {
-			skb_unlink(skb, &vi->recv);
-			kfree_skb(skb);
-			break;
+			p = page_address(page);
+			if (vi->mergeable_rx_bufs) {
+				sg_init_one(sg, p, PAGE_SIZE);
+				err = vi->rvq->vq_ops->add_buf(vi->rvq, sg, 0,
+							       1, page);
+				if (err < 0) {
+					give_pages(vi, page);
+					break;
+				}
+			} else {
+				int hdr_len = sizeof(struct virtio_net_hdr);
+
+				/*
+				 * allocate MAX_SKB_FRAGS + 1 pages for
+				 * big packets
+				 */
+				page->private = (unsigned long)first_page;
+				first_page = page;
+				if (--i == 1) {
+					int offset = hdr_len + 6;
+
+					/*
+					 * share one page between virtio_net
+					 * header and data, and reserve 6 bytes
+					 * for alignment
+					 */
+					sg_set_buf(sg, p, hdr_len);
+					sg_set_buf(sg+1, p + offset,
+						   PAGE_SIZE - offset);
+					err = vi->rvq->vq_ops->add_buf(vi->rvq,
+							sg, 0,
+							MAX_SKB_FRAGS + 2,
+							first_page);
+					if (err < 0) {
+						give_pages(vi, first_page);
+						break;
+					}
+
+				} else {
+					sg_set_buf(&sg[i], p, PAGE_SIZE);
+					goto more;
+				}
+			}
 		}
 		vi->num++;
 	} while (err > 0);
@@ -411,14 +459,13 @@ static void refill_work(struct work_struct *work)
 static int virtnet_poll(struct napi_struct *napi, int budget)
 {
 	struct virtnet_info *vi = container_of(napi, struct virtnet_info, napi);
-	struct sk_buff *skb = NULL;
+	void *buf = NULL;
 	unsigned int len, received = 0;
 
 again:
 	while (received < budget &&
-	       (skb = vi->rvq->vq_ops->get_buf(vi->rvq, &len)) != NULL) {
-		__skb_unlink(skb, &vi->recv);
-		receive_skb(vi->dev, skb, len);
+	       (buf = vi->rvq->vq_ops->get_buf(vi->rvq, &len)) != NULL) {
+		receive_skb(vi->dev, buf, len);
 		vi->num--;
 		received++;
 	}
@@ -959,6 +1006,7 @@ static void __devexit virtnet_remove(struct virtio_device *vdev)
 {
 	struct virtnet_info *vi = vdev->priv;
 	struct sk_buff *skb;
+	int freed;
 
 	/* Stop all the virtqueues. */
 	vdev->config->reset(vdev);
@@ -970,11 +1018,17 @@ static void __devexit virtnet_remove(struct virtio_device *vdev)
 	}
 	__skb_queue_purge(&vi->send);
 
-	BUG_ON(vi->num != 0);
-
 	unregister_netdev(vi->dev);
 	cancel_delayed_work_sync(&vi->refill);
 
+	if (vi->mergeable_rx_bufs || vi->big_packets) {
+		freed = vi->rvq->vq_ops->destroy_buf(vi->rvq,
+						     virtio_free_pages);
+		vi->num -= freed;
+	}
+
+	BUG_ON(vi->num != 0);
+
 	vdev->config->del_vqs(vi->vdev);
 
 	while (vi->pages)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index fbd2ecd..aec7fe7 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -334,6 +334,29 @@ static bool vring_enable_cb(struct virtqueue *_vq)
 	return true;
 }
 
+static int vring_destroy_buf(struct virtqueue *_vq, void (*callback)(void *))
+{
+	struct vring_virtqueue *vq = to_vvq(_vq);
+	void *ret;
+	unsigned int i;
+	int freed = 0;
+
+	START_USE(vq);
+
+	for (i = 0; i < vq->vring.num; i++) {
+		if (vq->data[i]) {
+			/* detach_buf clears data, so grab it now. */
+			ret = vq->data[i];
+			detach_buf(vq, i);
+			callback(ret);
+			freed++;
+		}
+	}
+
+	END_USE(vq);
+	return freed;
+}
+
 irqreturn_t vring_interrupt(int irq, void *_vq)
 {
 	struct vring_virtqueue *vq = to_vvq(_vq);
@@ -360,6 +383,7 @@ static struct virtqueue_ops vring_vq_ops = {
 	.kick = vring_kick,
 	.disable_cb = vring_disable_cb,
 	.enable_cb = vring_enable_cb,
+	.destroy_buf = vring_destroy_buf,
 };
 
 struct virtqueue *vring_new_virtqueue(unsigned int num,
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 057a2e0..7b1e86c 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -71,6 +71,7 @@ struct virtqueue_ops {
 
 	void (*disable_cb)(struct virtqueue *vq);
 	bool (*enable_cb)(struct virtqueue *vq);
+	int (*destroy_buf)(struct virtqueue *vq, void (*callback)(void *));
 };
 
 /**



^ permalink raw reply related

* [PATCH 0/1] Defer skb allocation for both mergeable buffers and big packets in virtio_net
From: Shirley Ma @ 2009-11-20  6:09 UTC (permalink / raw)
  To: Michael S. Tsirkin, Avi Kivity, Rusty Russell, netdev, kvm,
	linux-kernel

Guest virtio_net receives packets from its pre-allocated vring 
buffers, then it delivers these packets to upper layer protocols
as skb buffs. So it's not necessary to pre-allocate skb for each
mergable buffer, then frees it when it's useless. 

This patch has deferred skb allocation when receiving packets for
both big packets and mergeable buffers. It reduces skb pre-allocations 
and skb_frees.

Based on Mickael & Avi's suggestion. A destroy function has been created
to push virtio free buffs to vring for unused pages, and used page private
to maintain page list.

I didn't touch small packet skb allocation to avoid extra copies for small
packets.

This patch has tested and measured against 2.6.32-rc5 git. It is built again
 2.6.32-rc7 kernel. Tests have been done for small packets, big packets and
mergeable buffers.

The single netperf TCP_STREAM performance improved for host to guest. 
It also reduces UDP packets drop rate.

The netperf laptop results were:

mtu=1500
netperf -H xxx -l 120

		w/o patch	w/i patch (two runs)	
guest to host:  3336.84Mb/s   3730.14Mb/s ~ 3582.88Mb/s

host to guest:  3165.10Mb/s   3370.39Mb/s ~ 3407.96Mb/s

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Changli Gao @ 2009-11-20  5:50 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <4B062C51.8090100@gmail.com>

On Fri, Nov 20, 2009 at 1:42 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Changli Gao a écrit :
>> On Fri, Nov 20, 2009 at 1:24 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>> Changli Gao a écrit :
>>>
>>> Please re-read my patch, you misunderstood it, or I dont get you.
>>
>> I think I didn't misunderstand it. If local socket only sends packets,
>> which don't need replies from receiver, so new NIC RX IRQ, and NET_RX
>> softirq won't be triggered. Who will call xps_flush() to free the
>> memory used by locally generated packets?
>>
>
> Changli, when we transmit a skb on NIC, NIC is supposed to have a TX completion
> call back, to free this skb.
>
> These completion calls are running from net_rx_action(), if driver is NAPI enabled.
>
> Only NAPI enabled drivers are allowed to use XPS infrastructure.
>
> I sent 100.000.000 packets in my pktgen+tg3 tests, without receiving a single packet
> in return, I can tell you all packets were correctly freed :)
>
>

Oh, I am so sorry. It seems I missed sth. I need to review the new
NAPI. Thanks for your patience and help.

-- 
Regards,
Changli Gao(xiaosuo@gmail.com)

^ permalink raw reply

* Re: [PATCH] RDMA/addr: Use appropriate locking with for_each_netdev()
From: Eric Dumazet @ 2009-11-20  5:43 UTC (permalink / raw)
  To: Stephen Hemminger
  Cc: Sean Hefty, 'Roland Dreier', David S. Miller,
	Linux Netdev List, Roland Dreier, Hal Rosenstock, linux-rdma
In-Reply-To: <20091119214047.1a966c79@nehalam>

Stephen Hemminger a écrit :
> On Fri, 20 Nov 2009 06:28:00 +0100
> Eric Dumazet <eric.dumazet@gmail.com> wrote:
> 
>> +		read_lock(&dev_base_lock);
>>  		for_each_netdev(&init_net, dev) {
>>  			if (ipv6_chk_addr(&init_net,
>>  					  &((struct sockaddr_in6 *) addr)->sin6_addr,
>> @@ -139,6 +140,7 @@ int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
>>  				break;
>>  			}
>>  		}
>> +		read_unlock(&dev_base_lock);
> 
> what about for_each_netdev_rcu() here instead...
> 

Yes, in 2.6.33 :)

For 2.6.32, we need this patch

Thanks

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20  5:42 UTC (permalink / raw)
  To: Changli Gao; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <412e6f7f0911192134m24beca36m1887513513f130b8@mail.gmail.com>

Changli Gao a écrit :
> On Fri, Nov 20, 2009 at 1:24 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> Changli Gao a écrit :
>>
>> Please re-read my patch, you misunderstood it, or I dont get you.
> 
> I think I didn't misunderstand it. If local socket only sends packets,
> which don't need replies from receiver, so new NIC RX IRQ, and NET_RX
> softirq won't be triggered. Who will call xps_flush() to free the
> memory used by locally generated packets?
> 

Changli, when we transmit a skb on NIC, NIC is supposed to have a TX completion
call back, to free this skb.

These completion calls are running from net_rx_action(), if driver is NAPI enabled.

Only NAPI enabled drivers are allowed to use XPS infrastructure.

I sent 100.000.000 packets in my pktgen+tg3 tests, without receiving a single packet
in return, I can tell you all packets were correctly freed :)



^ permalink raw reply

* Re: [PATCH] RDMA/addr: Use appropriate locking with for_each_netdev()
From: Stephen Hemminger @ 2009-11-20  5:40 UTC (permalink / raw)
  To: Eric Dumazet
  Cc: Sean Hefty, 'Roland Dreier', David S. Miller,
	Linux Netdev List, Roland Dreier, Hal Rosenstock, linux-rdma
In-Reply-To: <4B0628E0.6000404-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

On Fri, 20 Nov 2009 06:28:00 +0100
Eric Dumazet <eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> +		read_lock(&dev_base_lock);
>  		for_each_netdev(&init_net, dev) {
>  			if (ipv6_chk_addr(&init_net,
>  					  &((struct sockaddr_in6 *) addr)->sin6_addr,
> @@ -139,6 +140,7 @@ int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
>  				break;
>  			}
>  		}
> +		read_unlock(&dev_base_lock);

what about for_each_netdev_rcu() here instead...



-- 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Changli Gao @ 2009-11-20  5:34 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <4B0627FA.3060103@gmail.com>

On Fri, Nov 20, 2009 at 1:24 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Changli Gao a écrit :
>
> Please re-read my patch, you misunderstood it, or I dont get you.

I think I didn't misunderstand it. If local socket only sends packets,
which don't need replies from receiver, so new NIC RX IRQ, and NET_RX
softirq won't be triggered. Who will call xps_flush() to free the
memory used by locally generated packets?

>
> If xps_consume_skb(skb) is ever called (from one to XXX times),
> then we xps_flush() them, from net_rx_action()
>
> net_rx_action()
> {
>        while (has_work) {
>                perform_napi_things(); // calls xps_consume_skb()
>        }
>        xps_flush(); // post things to remote cpus, and dont leak memory
> }
>
> This net_rx_action() is same for forwarding and localy generated packets.
>



-- 
Regards,
Changli Gao(xiaosuo@gmail.com)

^ permalink raw reply

* Re: [PATCH] RDMA/addr: Use appropriate locking with for_each_netdev()
From: Eric Dumazet @ 2009-11-20  5:28 UTC (permalink / raw)
  To: Sean Hefty
  Cc: 'Roland Dreier', David S. Miller, Linux Netdev List,
	Roland Dreier, Hal Rosenstock, linux-rdma
In-Reply-To: <8C10E584257A46DB9A0AD193520CF4A7-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>

Sean Hefty a écrit :
>> Would it be possible for you to take Eric's patch as the first in your
>> set (keeping his From: of course) and base your fixes on top of that?
> 
> Will do.
> 

Any news of this patch ?

We need it for 2.6.32 and also to prepare 2.6.33 with upcoming patches.

Thanks

[PATCH] RDMA/addr: Use appropriate locking with for_each_netdev()

for_each_netdev() should be used with RTNL or dev_base_lock held,
or risk a crash.

Signed-off-by: Eric Dumazet <eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/addr.c |    9 +++++++--
 1 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/core/addr.c b/drivers/infiniband/core/addr.c
index bd07803..5ca0b2c 100644
--- a/drivers/infiniband/core/addr.c
+++ b/drivers/infiniband/core/addr.c
@@ -131,6 +131,7 @@ int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
 
 #if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
 	case AF_INET6:
+		read_lock(&dev_base_lock);
 		for_each_netdev(&init_net, dev) {
 			if (ipv6_chk_addr(&init_net,
 					  &((struct sockaddr_in6 *) addr)->sin6_addr,
@@ -139,6 +140,7 @@ int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
 				break;
 			}
 		}
+		read_unlock(&dev_base_lock);
 		break;
 #endif
 	}
@@ -391,15 +393,17 @@ static int addr_resolve_local(struct sockaddr *src_in,
 	{
 		struct in6_addr *a;
 
+		read_lock(&dev_base_lock);
 		for_each_netdev(&init_net, dev)
 			if (ipv6_chk_addr(&init_net,
 					  &((struct sockaddr_in6 *) dst_in)->sin6_addr,
 					  dev, 1))
 				break;
 
-		if (!dev)
+		if (!dev) {
+			read_unlock(&dev_base_lock);
 			return -EADDRNOTAVAIL;
-
+		}
 		a = &((struct sockaddr_in6 *) src_in)->sin6_addr;
 
 		if (ipv6_addr_any(a)) {
@@ -416,6 +420,7 @@ static int addr_resolve_local(struct sockaddr *src_in,
 			if (!ret)
 				memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
 		}
+		read_unlock(&dev_base_lock);
 		break;
 	}
 #endif

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20  5:24 UTC (permalink / raw)
  To: Changli Gao; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <412e6f7f0911192111jbc8b237sc619a54510219336@mail.gmail.com>

Changli Gao a écrit :
> You call xps_flush() in net_rx_aciton(). It means that if no new
> packet arrives, xps_flush() won't be called forever, and the memory
> used by skbs will be hold forever. Did I misunderstand? Your algorithm
> only works with packet forwarding but sending packets from local
> sockets.
> 

Please re-read my patch, you misunderstood it, or I dont get you.

If xps_consume_skb(skb) is ever called (from one to XXX times),
then we xps_flush() them, from net_rx_action()

net_rx_action()
{
	while (has_work) {
		perform_napi_things(); // calls xps_consume_skb()
	}
	xps_flush(); // post things to remote cpus, and dont leak memory
}

This net_rx_action() is same for forwarding and localy generated packets.

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Changli Gao @ 2009-11-20  5:11 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <4B0621FC.6060004@gmail.com>

On Fri, Nov 20, 2009 at 12:58 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Changli Gao a écrit :
>> On Fri, Nov 20, 2009 at 7:46 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>> diff --git a/net/core/dev.c b/net/core/dev.c
>>> index 9977288..9e134f6 100644
>>> --- a/net/core/dev.c
>>> +++ b/net/core/dev.c
>>> @@ -2000,6 +2001,7 @@ gso:
>>>         */
>>>        rcu_read_lock_bh();
>>>
>>> +       skb->sending_cpu = cpu = smp_processor_id();
>>>        txq = dev_pick_tx(dev, skb);
>>>        q = rcu_dereference(txq->qdisc);
>>
>> I think assigning cpu to skb->sending_cpu just before calling
>> hard_start_xmit is better, because the CPU which dequeues the skb will
>> be another one.
>
> I want to record the application CPU, because I want the application CPU
> to call sock_wfree(), not the CPU that happened to dequeue skb to transmit it
> in case of txq contention.
>

got it.

>>
>>> @@ -2024,8 +2026,6 @@ gso:
>>>           Either shot noqueue qdisc, it is even simpler 8)
>>>         */
>>>        if (dev->flags & IFF_UP) {
>>> -               int cpu = smp_processor_id(); /* ok because BHs are off */
>>> -
>>>                if (txq->xmit_lock_owner != cpu) {
>>>
>>>                        HARD_TX_LOCK(dev, txq, cpu);
>>> @@ -2967,7 +2967,7 @@ static void net_rx_action(struct softirq_action *h)
>>>        }
>>>  out:
>>>        local_irq_enable();
>>> -
>>> +       xps_flush();
>>
>> If there isn't any new skbs, the memory will be hold forever. I know
>> you want to eliminate unnecessary IPI, how about sending IPI only when
>> the remote xps_pcpu_queues are changed from empty to nonempty?
>
> I dont understand your remark, and dont see the problem, yet.
>
> I send IPI only on cpus I know I have at least one skb queueud for them.
> For each cpu taking TX completion interrupts I have :
>
> One bitmask (xps_cpus) of cpus I will eventually send IPI at end of net_rx_action()
>

You call xps_flush() in net_rx_aciton(). It means that if no new
packet arrives, xps_flush() won't be called forever, and the memory
used by skbs will be hold forever. Did I misunderstand? Your algorithm
only works with packet forwarding but sending packets from local
sockets.

-- 
Regards,
Changli Gao(xiaosuo@gmail.com)

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20  5:08 UTC (permalink / raw)
  To: Tom Herbert; +Cc: David S. Miller, Linux Netdev List
In-Reply-To: <65634d660911191641o4210a797mf1e8168dd8dd8b60@mail.gmail.com>

Tom Herbert a écrit :
> 
> 
> On Thu, Nov 19, 2009 at 3:46 PM, Eric Dumazet <eric.dumazet@gmail.com
> <mailto:eric.dumazet@gmail.com>> wrote:
> 
>     Here is first version of XPS.
> 
> Very cool!  The infrastructure to move lists of skb's from cpu to
> another and the IPI to kick processing look like something that could be
> consolidated between rps and xps :-)

Sure, but RPS seems quite slow to integrate (no offense Tom) :)
I wanted to cook a patch on top on yours, but got no sign you were
about to release RPS version 4 soon.

> 
>     Goal of XPS is to free TX completed skbs by the cpu that submitted
>     the transmit.
> 
>     Because I chose to union skb->iif with skb->sending_cpu, I chose
>     to introduce a new xps_consume_skb(skb), and not generalize
>     consume_skb() itself.
> 
>     This means that selected drivers must use new function to benefit
>     from XPS
> 
> 
> Is this better than modifying consume_skb so this can be used by any driver?

consume_skb() is also used by RX side, and this side doesnt want XPS.

Adding a flag in skb to differentiate the use might be possible,
but we add a new test in hot paths...

>  
> 
>      
> 
>     Preliminary tests are quite good, especially on NUMA machines.
> 
>     Only NAPI drivers can use this new infrastructure (xps_consume_skb()
>     cannot
>     be called from hardirq context, only from softirq)
> 
> Is this a strict requirement, especially considering devices with
> separate TX interrupts?  For a hardirq we could we just put the skb on
> local percpu queue and schedule a softirq to do the ipi.

I chose this way because any sane and up2date driver should really
not use hardirqs TX completion. I want fast processing, without
masking local interrupts in xps_consume_skb(), and wihout testing
our context.

Note : if TX completion and RX are run in different NAPI contexts,
there is no problem using xps_consume_skb().


Thanks for reviewing Tom.

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-20  4:58 UTC (permalink / raw)
  To: Changli Gao; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <412e6f7f0911191812uf0abc61w2f0d44f4d71bd55@mail.gmail.com>

Changli Gao a écrit :
> On Fri, Nov 20, 2009 at 7:46 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> diff --git a/net/core/dev.c b/net/core/dev.c
>> index 9977288..9e134f6 100644
>> --- a/net/core/dev.c
>> +++ b/net/core/dev.c
>> @@ -2000,6 +2001,7 @@ gso:
>>         */
>>        rcu_read_lock_bh();
>>
>> +       skb->sending_cpu = cpu = smp_processor_id();
>>        txq = dev_pick_tx(dev, skb);
>>        q = rcu_dereference(txq->qdisc);
> 
> I think assigning cpu to skb->sending_cpu just before calling
> hard_start_xmit is better, because the CPU which dequeues the skb will
> be another one.

I want to record the application CPU, because I want the application CPU
to call sock_wfree(), not the CPU that happened to dequeue skb to transmit it
in case of txq contention.

> 
>> @@ -2024,8 +2026,6 @@ gso:
>>           Either shot noqueue qdisc, it is even simpler 8)
>>         */
>>        if (dev->flags & IFF_UP) {
>> -               int cpu = smp_processor_id(); /* ok because BHs are off */
>> -
>>                if (txq->xmit_lock_owner != cpu) {
>>
>>                        HARD_TX_LOCK(dev, txq, cpu);
>> @@ -2967,7 +2967,7 @@ static void net_rx_action(struct softirq_action *h)
>>        }
>>  out:
>>        local_irq_enable();
>> -
>> +       xps_flush();
> 
> If there isn't any new skbs, the memory will be hold forever. I know
> you want to eliminate unnecessary IPI, how about sending IPI only when
> the remote xps_pcpu_queues are changed from empty to nonempty?

I dont understand your remark, and dont see the problem, yet.

I send IPI only on cpus I know I have at least one skb queueud for them.
For each cpu taking TX completion interrupts I have :

One bitmask (xps_cpus) of cpus I will eventually send IPI at end of net_rx_action()

One array of skb lists per remote cpu, allocated on cpu node memory, thanks
to __alloc_percpu() at boot time.

I say _eventually_ because the algo is :

+		if (cpu_online(cpu)) {
+			spin_lock(&q->list.lock);
+			prevlen = skb_queue_len(&q->list);
+			skb_queue_splice_init(&head[cpu], &q->list);
+			spin_unlock(&q->list.lock);
+			/*
+			 * We hope remote cpu will be fast enough to transfert
+			 * this list to its completion queue before our
+			 * next xps_flush() call
+			 */
+			if (!prevlen)
+				__smp_call_function_single(cpu, &q->csd, 0);
+			continue;

So I send an IPI only if needed, once for the whole skb list.

With my pktgen (no skb cloning setup) tests, and

ethtool -C eth3 tx-usecs 1000 tx-frames 100

I really saw batches of 100 frames given from CPU X (NIC interrupts) to CPU Y (pktgen cpu)

What memory is hold forever ?

^ permalink raw reply

* Re: [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Changli Gao @ 2009-11-20  2:12 UTC (permalink / raw)
  To: Eric Dumazet; +Cc: David S. Miller, Tom Herbert, Linux Netdev List
In-Reply-To: <4B05D8DC.7020907@gmail.com>

On Fri, Nov 20, 2009 at 7:46 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 9977288..9e134f6 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -2000,6 +2001,7 @@ gso:
>         */
>        rcu_read_lock_bh();
>
> +       skb->sending_cpu = cpu = smp_processor_id();
>        txq = dev_pick_tx(dev, skb);
>        q = rcu_dereference(txq->qdisc);

I think assigning cpu to skb->sending_cpu just before calling
hard_start_xmit is better, because the CPU which dequeues the skb will
be another one.

>
> @@ -2024,8 +2026,6 @@ gso:
>           Either shot noqueue qdisc, it is even simpler 8)
>         */
>        if (dev->flags & IFF_UP) {
> -               int cpu = smp_processor_id(); /* ok because BHs are off */
> -
>                if (txq->xmit_lock_owner != cpu) {
>
>                        HARD_TX_LOCK(dev, txq, cpu);
> @@ -2967,7 +2967,7 @@ static void net_rx_action(struct softirq_action *h)
>        }
>  out:
>        local_irq_enable();
> -
> +       xps_flush();

If there isn't any new skbs, the memory will be hold forever. I know
you want to eliminate unnecessary IPI, how about sending IPI only when
the remote xps_pcpu_queues are changed from empty to nonempty?


-- 
Regards,
Changli Gao(xiaosuo@gmail.com)

^ permalink raw reply

* [PATCH 2.6.32-rc6] drivers/net: ks8851_mll ethernet network driver -resubmit
From: Choi, David @ 2009-11-20  1:34 UTC (permalink / raw)
  To: davem; +Cc: netdev, linux-kernel, shemminger, greg, Choi, David

Hello all,

This patch is for ks8851 16bit MLL Ethernet network device driver in order to 
fix bugs and to enhance functions. This resubmit removes a printk statement causing
the compile warning in 64bit processors.

>From 	: David J. Choi <david.choi@micrel.com>

Summary of Changes:  

	-Fix to receive multicast packets by setting the corresponding hardware 
	 bit during initialization.
	-Fix to re-enable the interface [by interface up command(ifup)] while the 
	 interface is down.
	-Fix to be able to down the interface by passing the last parameter 
	 correctly to request_irq().
	-Remove to read 4 extra bytes from the receiving queue after reading a 
	 packet, even though it does not cause a predictable issue now.
	-Remove occurrences of transmission done interrupt in order to tx 
	 throughput enhancement.
	-Enable IP checksum for packet receiving by setting the corresponding 
	 hardware bit during initialization.
	-Relocate ks_enable_int()/ks_disable_int() in order not to declare those 
	 functions at the beginning of the file.
	-Rename ks_enable()/_disable() to ks_enable_qmu()/ks_disable_qmu() in 
	 order to give more meaningful names and relocate them not declaire 
	 those functions at the beginning of the file.

Signed-off-by: David J. Choi <david.choi@micrel.com>

---
--- linux-2.6.32-rc6/drivers/net/ks8851_mll.c.orig	2009-11-12 09:57:33.000000000 -0800
+++ linux-2.6.32-rc6/drivers/net/ks8851_mll.c	2009-11-19 17:04:33.000000000 -0800
@@ -568,6 +568,16 @@ static inline void ks_outblk(struct ks_n
 		iowrite16(*wptr++, ks->hw_addr);
 }
 
+static void ks_disable_int(struct ks_net *ks)
+{
+	ks_wrreg16(ks, KS_IER, 0x0000);
+}  /* ks_disable_int */
+
+static void ks_enable_int(struct ks_net *ks)
+{
+	ks_wrreg16(ks, KS_IER, ks->rc_ier);
+}  /* ks_enable_int */
+
 /**
  * ks_tx_fifo_space - return the available hardware buffer size.
  * @ks: The chip information
@@ -681,6 +691,47 @@ static void ks_soft_reset(struct ks_net 
 }
 
 
+void ks_enable_qmu(struct ks_net *ks)
+{
+	u16 w;
+
+	w = ks_rdreg16(ks, KS_TXCR);
+	/* Enables QMU Transmit (TXCR). */
+	ks_wrreg16(ks, KS_TXCR, w | TXCR_TXE);
+
+	/*
+	 * RX Frame Count Threshold Enable and Auto-Dequeue RXQ Frame
+	 * Enable
+	 */
+
+	w = ks_rdreg16(ks, KS_RXQCR);
+	ks_wrreg16(ks, KS_RXQCR, w | RXQCR_RXFCTE);
+
+	/* Enables QMU Receive (RXCR1). */
+	w = ks_rdreg16(ks, KS_RXCR1);
+	ks_wrreg16(ks, KS_RXCR1, w | RXCR1_RXE);
+	ks->enabled = true;
+}  /* ks_enable_qmu */
+
+static void ks_disable_qmu(struct ks_net *ks)
+{
+	u16	w;
+
+	w = ks_rdreg16(ks, KS_TXCR);
+
+	/* Disables QMU Transmit (TXCR). */
+	w  &= ~TXCR_TXE;
+	ks_wrreg16(ks, KS_TXCR, w);
+
+	/* Disables QMU Receive (RXCR1). */
+	w = ks_rdreg16(ks, KS_RXCR1);
+	w &= ~RXCR1_RXE ;
+	ks_wrreg16(ks, KS_RXCR1, w);
+
+	ks->enabled = false;
+
+}  /* ks_disable_qmu */
+
 /**
  * ks_read_qmu - read 1 pkt data from the QMU.
  * @ks: The chip information
@@ -752,7 +803,7 @@ static void ks_rcv(struct ks_net *ks, st
 			(frame_hdr->len < RX_BUF_SIZE) && frame_hdr->len)) {
 			skb_reserve(skb, 2);
 			/* read data block including CRC 4 bytes */
-			ks_read_qmu(ks, (u16 *)skb->data, frame_hdr->len + 4);
+			ks_read_qmu(ks, (u16 *)skb->data, frame_hdr->len);
 			skb_put(skb, frame_hdr->len);
 			skb->dev = netdev;
 			skb->protocol = eth_type_trans(skb, netdev);
@@ -861,7 +912,7 @@ static int ks_net_open(struct net_device
 		ks_dbg(ks, "%s - entry\n", __func__);
 
 	/* reset the HW */
-	err = request_irq(ks->irq, ks_irq, KS_INT_FLAGS, DRV_NAME, ks);
+	err = request_irq(ks->irq, ks_irq, KS_INT_FLAGS, DRV_NAME, netdev);
 
 	if (err) {
 		printk(KERN_ERR "Failed to request IRQ: %d: %d\n",
@@ -869,6 +920,15 @@ static int ks_net_open(struct net_device
 		return err;
 	}
 
+	/* wake up powermode to normal mode */
+	ks_set_powermode(ks, PMECR_PM_NORMAL);
+	mdelay(1);	/* wait for normal mode to take effect */
+
+	ks_wrreg16(ks, KS_ISR, 0xffff);
+	ks_enable_int(ks);
+	ks_enable_qmu(ks);
+	netif_start_queue(ks->netdev);
+
 	if (netif_msg_ifup(ks))
 		ks_dbg(ks, "network device %s up\n", netdev->name);
 
@@ -892,19 +952,14 @@ static int ks_net_stop(struct net_device
 
 	netif_stop_queue(netdev);
 
-	kfree(ks->frame_head_info);
-
 	mutex_lock(&ks->lock);
 
 	/* turn off the IRQs and ack any outstanding */
 	ks_wrreg16(ks, KS_IER, 0x0000);
 	ks_wrreg16(ks, KS_ISR, 0xffff);
 
-	/* shutdown RX process */
-	ks_wrreg16(ks, KS_RXCR1, 0x0000);
-
-	/* shutdown TX process */
-	ks_wrreg16(ks, KS_TXCR, 0x0000);
+	/* shutdown RX/TX QMU */
+	ks_disable_qmu(ks);
 
 	/* set powermode to soft power down to save power */
 	ks_set_powermode(ks, PMECR_PM_SOFTDOWN);
@@ -929,17 +984,8 @@ static int ks_net_stop(struct net_device
  */
 static void ks_write_qmu(struct ks_net *ks, u8 *pdata, u16 len)
 {
-	unsigned fid = ks->fid;
-
-	fid = ks->fid;
-	ks->fid = (ks->fid + 1) & TXFR_TXFID_MASK;
-
-	/* reduce the tx interrupt occurrances. */
-	if (!fid)
-		fid |= TXFR_TXIC;       /* irq on completion */
-
 	/* start header at txb[0] to align txw entries */
-	ks->txh.txw[0] = cpu_to_le16(fid);
+	ks->txh.txw[0] = 0;
 	ks->txh.txw[1] = cpu_to_le16(len);
 
 	/* 1. set sudo-DMA mode */
@@ -957,16 +1003,6 @@ static void ks_write_qmu(struct ks_net *
 		;
 }
 
-static void ks_disable_int(struct ks_net *ks)
-{
-	ks_wrreg16(ks, KS_IER, 0x0000);
-}  /* ks_disable_int */
-
-static void ks_enable_int(struct ks_net *ks)
-{
-	ks_wrreg16(ks, KS_IER, ks->rc_ier);
-}  /* ks_enable_int */
-
 /**
  * ks_start_xmit - transmit packet
  * @skb		: The buffer to transmit
@@ -1410,25 +1446,6 @@ static int ks_read_selftest(struct ks_ne
 	return ret;
 }
 
-static void ks_disable(struct ks_net *ks)
-{
-	u16	w;
-
-	w = ks_rdreg16(ks, KS_TXCR);
-
-	/* Disables QMU Transmit (TXCR). */
-	w  &= ~TXCR_TXE;
-	ks_wrreg16(ks, KS_TXCR, w);
-
-	/* Disables QMU Receive (RXCR1). */
-	w = ks_rdreg16(ks, KS_RXCR1);
-	w &= ~RXCR1_RXE ;
-	ks_wrreg16(ks, KS_RXCR1, w);
-
-	ks->enabled = false;
-
-}  /* ks_disable */
-
 static void ks_setup(struct ks_net *ks)
 {
 	u16	w;
@@ -1463,7 +1480,7 @@ static void ks_setup(struct ks_net *ks)
 	w = TXCR_TXFCE | TXCR_TXPE | TXCR_TXCRC | TXCR_TCGIP;
 	ks_wrreg16(ks, KS_TXCR, w);
 
-	w = RXCR1_RXFCE | RXCR1_RXBE | RXCR1_RXUE;
+	w = RXCR1_RXFCE | RXCR1_RXBE | RXCR1_RXUE | RXCR1_RXME | RXCR1_RXIPFCC;
 
 	if (ks->promiscuous)         /* bPromiscuous */
 		w |= (RXCR1_RXAE | RXCR1_RXINVF);
@@ -1486,28 +1503,6 @@ static void ks_setup_int(struct ks_net *
 	ks->rc_ier = (IRQ_LCI | IRQ_TXI | IRQ_RXI);
 }  /* ks_setup_int */
 
-void ks_enable(struct ks_net *ks)
-{
-	u16 w;
-
-	w = ks_rdreg16(ks, KS_TXCR);
-	/* Enables QMU Transmit (TXCR). */
-	ks_wrreg16(ks, KS_TXCR, w | TXCR_TXE);
-
-	/*
-	 * RX Frame Count Threshold Enable and Auto-Dequeue RXQ Frame
-	 * Enable
-	 */
-
-	w = ks_rdreg16(ks, KS_RXQCR);
-	ks_wrreg16(ks, KS_RXQCR, w | RXQCR_RXFCTE);
-
-	/* Enables QMU Receive (RXCR1). */
-	w = ks_rdreg16(ks, KS_RXCR1);
-	ks_wrreg16(ks, KS_RXCR1, w | RXCR1_RXE);
-	ks->enabled = true;
-}  /* ks_enable */
-
 static int ks_hw_init(struct ks_net *ks)
 {
 #define	MHEADER_SIZE	(sizeof(struct type_frame_head) * MAX_RECV_FRAMES)
@@ -1612,11 +1607,9 @@ static int __devinit ks8851_probe(struct
 
 	ks_soft_reset(ks, GRR_GSR);
 	ks_hw_init(ks);
-	ks_disable(ks);
+	ks_disable_qmu(ks);
 	ks_setup(ks);
 	ks_setup_int(ks);
-	ks_enable_int(ks);
-	ks_enable(ks);
 	memcpy(netdev->dev_addr, ks->mac_addr, 6);
 
 	data = ks_rdreg16(ks, KS_OBCR);
@@ -1658,6 +1651,7 @@ static int __devexit ks8851_remove(struc
 	struct ks_net *ks = netdev_priv(netdev);
 	struct resource *iomem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
 
+	kfree(ks->frame_head_info);
 	unregister_netdev(netdev);
 	iounmap(ks->hw_addr);
 	free_netdev(netdev);
--

^ permalink raw reply

* Re: [BUG] netxen: Stops working between 2.6.30 and 2.6.31-rc1
From: Eric W. Biederman @ 2009-11-20  1:19 UTC (permalink / raw)
  To: Jens Rosenboom; +Cc: Dhananjay Phadke, netdev@vger.kernel.org, Amit Salecha
In-Reply-To: <20091119183607.GK14661@jayr.de>

Jens Rosenboom <me@jayr.de> writes:

> On Thu, Nov 19, 2009 at 10:07:21AM -0800, Dhananjay Phadke wrote:
>> > My netxen 10G card stops working somewhere between 2.6.30 and 2.6.31-rc1.
>> > With the
>> > newer kernel I can see packets been received on the switch it is
>> > connected to, but
>> > the kernel doesn't report any sent packets in the interface counters and
>> > nothing
>> > is being received either.
>> > 
>> > I've tried to bisect this, but only seems the end up with kernels that do
>> > not boot
>> > at all because some SCSI stuff goes bad.
>> 
>> Any particular reason for using -rc1 kernel and not 2.6.31 stable kernel?
>
> Sorry, I forgot to mention that all later kernels that I tested
> including 2.6.31 and the current net-2.6 also fail, so the badness 
> comes in somewhere in between 2.6.30 and 2.6.31-rc1.
>
> I also noticed that the newer kernel allocate four interrupts for the
> card instead of only one, but none of them seem to get triggered, the
> /proc/interrupts counters all stay at zero.

Hmm.  Have you tried disabling msi's? aka putting nomsi on the kernel
command line.

If you aren't getting interrupts it might be that your board simply
has problems with receiving msi interrupts.  That at least used to
be common.

Eric

^ permalink raw reply

* Re: [net-next-2.6 PATCH v2] allow access to sysfs_groups member
From: Eric W. Biederman @ 2009-11-20  1:11 UTC (permalink / raw)
  To: Kurt Van Dijck; +Cc: David Miller, netdev
In-Reply-To: <20091118214226.GA283@e-circ.dyndns.org>

Kurt Van Dijck <kurt.van.dijck@eia.be> writes:

> On Wed, Nov 18, 2009 at 01:08:59PM -0800, David Miller wrote:
>>
>> Unfortunately, the code in this function is much different
>> in net-next-2.6 which is where I want to add this, and your
>> patch doesn't apply.
>> 
>> Please rebase your patch on that tree, thank you.
> As I mentioned, I'm not experienced with using git yet.
> Is there a fine manual I can inspect? I get lost in the man-pages.

git clone git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next-2.6.git

Should get you started.

In this case please look at how I modified the bonding driver to do
what you are trying to do.  That is the conflict in net-next and
I think it was actually less code.

The other very useful command is: git log -u some/path/

Git has fine manual pages and git command --help works for the builtin
git commands.

Eric

^ permalink raw reply

* [PATCH NEXT] netxen: remove PCI IDs of CNA device
From: Amit Kumar Salecha @ 2009-11-20  1:08 UTC (permalink / raw)
  To: davem; +Cc: netdev, dhananjay.phadke

Remove PCI vendor and device IDs for QLE8240 and QLE8242
CNA devices. CNA devices will have separate driver.

Signed-off-by: Amit Kumar Salecha <amit.salecha@qlogic.com>
---
 drivers/net/netxen/netxen_nic_main.c |    6 ------
 1 files changed, 0 insertions(+), 6 deletions(-)

diff --git a/drivers/net/netxen/netxen_nic_main.c b/drivers/net/netxen/netxen_nic_main.c
index 12d1037..bfbf75c 100644
--- a/drivers/net/netxen/netxen_nic_main.c
+++ b/drivers/net/netxen/netxen_nic_main.c
@@ -95,11 +95,6 @@ static void netxen_config_indev_addr(struct net_device *dev, unsigned long);
 #define ENTRY(device) \
 	{PCI_DEVICE(PCI_VENDOR_ID_NETXEN, (device)), \
 	.class = PCI_CLASS_NETWORK_ETHERNET << 8, .class_mask = ~0}
-#define ENTRY2(device) \
-	{PCI_DEVICE(PCI_VENDOR_ID_QLOGIC, (device)), \
-	.class = PCI_CLASS_NETWORK_ETHERNET << 8, .class_mask = ~0}
-
-#define PCI_DEVICE_ID_QLOGIC_QLE824X	0x8020
 
 static struct pci_device_id netxen_pci_tbl[] __devinitdata = {
 	ENTRY(PCI_DEVICE_ID_NX2031_10GXSR),
@@ -110,7 +105,6 @@ static struct pci_device_id netxen_pci_tbl[] __devinitdata = {
 	ENTRY(PCI_DEVICE_ID_NX2031_XG_MGMT),
 	ENTRY(PCI_DEVICE_ID_NX2031_XG_MGMT2),
 	ENTRY(PCI_DEVICE_ID_NX3031),
-	ENTRY2(PCI_DEVICE_ID_QLOGIC_QLE824X),
 	{0,}
 };
 
-- 
1.6.0.2


^ permalink raw reply related

* [net-2.6 PATCH] e1000e: do not initiate autonegotiation during OEM configuration
From: Jeff Kirsher @ 2009-11-20  0:17 UTC (permalink / raw)
  To: davem; +Cc: netdev, gospo, Bruce Allan, Jeff Kirsher

From: Bruce Allan <bruce.w.allan@intel.com>

When configuring the OEM bits in the PHY on 82577/82578, do not restart
autonegotiation if the firmware is blocking it (e.g. when an IDE-R session
is active) because the link must not go down.

Signed-off-by: Bruce Allan <bruce.w.allan@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---

 drivers/net/e1000e/ich8lan.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/drivers/net/e1000e/ich8lan.c b/drivers/net/e1000e/ich8lan.c
index 92cf103..eff3f47 100644
--- a/drivers/net/e1000e/ich8lan.c
+++ b/drivers/net/e1000e/ich8lan.c
@@ -1118,7 +1118,8 @@ static s32 e1000_oem_bits_config_ich8lan(struct e1000_hw *hw, bool d0_state)
 			oem_reg |= HV_OEM_BITS_LPLU;
 	}
 	/* Restart auto-neg to activate the bits */
-	oem_reg |= HV_OEM_BITS_RESTART_AN;
+	if (!e1000_check_reset_block(hw))
+		oem_reg |= HV_OEM_BITS_RESTART_AN;
 	ret_val = hw->phy.ops.write_phy_reg_locked(hw, HV_OEM_BITS, oem_reg);
 
 out:


^ permalink raw reply related

* [PATCH net-next-2.6] net: Xmit Packet Steering (XPS)
From: Eric Dumazet @ 2009-11-19 23:46 UTC (permalink / raw)
  To: David S. Miller; +Cc: Tom Herbert, Linux Netdev List

Here is first version of XPS.

Goal of XPS is to free TX completed skbs by the cpu that submitted the transmit.

Because I chose to union skb->iif with skb->sending_cpu, I chose
to introduce a new xps_consume_skb(skb), and not generalize consume_skb() itself.

This means that selected drivers must use new function to benefit from XPS

Preliminary tests are quite good, especially on NUMA machines.

Only NAPI drivers can use this new infrastructure (xps_consume_skb() cannot
be called from hardirq context, only from softirq)

I converted tg3 and pktgen for my tests

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
 drivers/net/tg3.c      |    2
 include/linux/skbuff.h |   14 +++
 net/core/Makefile      |    1
 net/core/dev.c         |    8 +-
 net/core/pktgen.c      |    1
 net/core/xps.c         |  145 +++++++++++++++++++++++++++++++++++++++
 6 files changed, 165 insertions(+), 6 deletions(-)

diff --git a/drivers/net/tg3.c b/drivers/net/tg3.c
index 6e6db95..bc756e6 100644
--- a/drivers/net/tg3.c
+++ b/drivers/net/tg3.c
@@ -4379,7 +4379,7 @@ static void tg3_tx(struct tg3_napi *tnapi)
 			sw_idx = NEXT_TX(sw_idx);
 		}
 
-		dev_kfree_skb(skb);
+		xps_consume_skb(skb);
 
 		if (unlikely(tx_bug)) {
 			tg3_tx_recover(tp);
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 63f4742..e8e4795 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -366,7 +366,10 @@ struct sk_buff {
 	struct nf_bridge_info	*nf_bridge;
 #endif
 
-	int			iif;
+	union {
+		int		iif;
+		int		sending_cpu;
+	};
 #ifdef CONFIG_NET_SCHED
 	__u16			tc_index;	/* traffic control index */
 #ifdef CONFIG_NET_CLS_ACT
@@ -441,6 +444,15 @@ static inline struct rtable *skb_rtable(const struct sk_buff *skb)
 
 extern void kfree_skb(struct sk_buff *skb);
 extern void consume_skb(struct sk_buff *skb);
+#if defined(CONFIG_SMP)
+extern void xps_consume_skb(struct sk_buff *skb);
+extern void xps_flush(void);
+extern void xps_init(void);
+#else
+#define xps_consume_skb(skb) consume_skb(skb)
+static inline void xps_flush(void) {}
+static inline void xps_init(void) {}
+#endif
 extern void	       __kfree_skb(struct sk_buff *skb);
 extern struct sk_buff *__alloc_skb(unsigned int size,
 				   gfp_t priority, int fclone, int node);
diff --git a/net/core/Makefile b/net/core/Makefile
index 796f46e..eacd3d8 100644
--- a/net/core/Makefile
+++ b/net/core/Makefile
@@ -19,4 +19,5 @@ obj-$(CONFIG_NET_DMA) += user_dma.o
 obj-$(CONFIG_FIB_RULES) += fib_rules.o
 obj-$(CONFIG_TRACEPOINTS) += net-traces.o
 obj-$(CONFIG_NET_DROP_MONITOR) += drop_monitor.o
+obj-$(CONFIG_SMP) += xps.o
 
diff --git a/net/core/dev.c b/net/core/dev.c
index 9977288..9e134f6 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -1965,6 +1965,7 @@ int dev_queue_xmit(struct sk_buff *skb)
 	struct netdev_queue *txq;
 	struct Qdisc *q;
 	int rc = -ENOMEM;
+	int cpu;
 
 	/* GSO will handle the following emulations directly. */
 	if (netif_needs_gso(dev, skb))
@@ -2000,6 +2001,7 @@ gso:
 	 */
 	rcu_read_lock_bh();
 
+	skb->sending_cpu = cpu = smp_processor_id();
 	txq = dev_pick_tx(dev, skb);
 	q = rcu_dereference(txq->qdisc);
 
@@ -2024,8 +2026,6 @@ gso:
 	   Either shot noqueue qdisc, it is even simpler 8)
 	 */
 	if (dev->flags & IFF_UP) {
-		int cpu = smp_processor_id(); /* ok because BHs are off */
-
 		if (txq->xmit_lock_owner != cpu) {
 
 			HARD_TX_LOCK(dev, txq, cpu);
@@ -2967,7 +2967,7 @@ static void net_rx_action(struct softirq_action *h)
 	}
 out:
 	local_irq_enable();
-
+	xps_flush();
 #ifdef CONFIG_NET_DMA
 	/*
 	 * There may not be any more sk_buffs coming right now, so push
@@ -5798,7 +5798,7 @@ static int __init net_dev_init(void)
 		queue->backlog.gro_list = NULL;
 		queue->backlog.gro_count = 0;
 	}
-
+	xps_init();
 	dev_boot_phase = 0;
 
 	/* The loopback device is special if any other network devices
diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index d38470a..b41b794 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -3435,6 +3435,7 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
 			pkt_dev->clone_count--;	/* back out increment, OOM */
 			return;
 		}
+		pkt_dev->skb->sending_cpu = smp_processor_id();
 		pkt_dev->last_pkt_size = pkt_dev->skb->len;
 		pkt_dev->allocated_skbs++;
 		pkt_dev->clone_count = 0;	/* reset counter */
diff --git a/net/core/xps.c b/net/core/xps.c
index e69de29..e580159 100644
--- a/net/core/xps.c
+++ b/net/core/xps.c
@@ -0,0 +1,145 @@
+/*
+ * XPS : Xmit Packet Steering
+ *
+ * TX completion packet freeing is performed on cpu that sent packet.
+ */
+#if defined(CONFIG_SMP)
+
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/kmemcheck.h>
+#include <linux/mm.h>
+#include <linux/interrupt.h>
+#include <linux/in.h>
+#include <linux/inet.h>
+#include <linux/slab.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+
+
+struct xps_pcpu_queue {
+	struct call_single_data csd;
+	struct sk_buff_head	list;
+};
+
+static DEFINE_PER_CPU(cpumask_t, xps_cpus);
+
+static DEFINE_PER_CPU(struct xps_pcpu_queue, xps_pcpu_queue);
+
+static struct sk_buff_head *xps_array; /* nr_cpu_ids elems */
+
+
+
+/* called from softirq context only */
+void xps_consume_skb(struct sk_buff *skb)
+{
+	unsigned int remote;
+	int thiscpu;
+	struct sk_buff_head *head;
+	/*
+	 * Might be stupid to dirty this cache line, but might
+	 * also be stupid to send an IPI if skb is not to be freed :(
+	 * One solution to this problem would be to move ->users in first cache line
+	 * (shared with ->next & ->sending_cpu fields ), so that this cpu dirties
+	 * only one cache line per queued skb.
+	 */
+	if (!atomic_dec_and_test(&skb->users))
+		return;
+
+	remote = skb->sending_cpu;
+	thiscpu = smp_processor_id();
+	if (remote >= nr_cpu_ids || remote == thiscpu) {
+		__kfree_skb(skb);
+		return;
+	}
+	head = &per_cpu_ptr(xps_array, thiscpu)[remote];
+
+	__skb_queue_head(head, skb);
+
+	/* IPI to remote processor will be sent later by xps_flush(),
+	 * to coalesce as much as possible skbs
+	 */
+	cpu_set(remote, __get_cpu_var(xps_cpus));
+}
+EXPORT_SYMBOL(xps_consume_skb);
+
+/*
+ * called at end of net_rx_action()
+ * preemption (and cpu migration/offline/online) disabled
+ */
+void xps_flush(void)
+{
+	int cpu, prevlen;
+	struct sk_buff_head *head = per_cpu_ptr(xps_array, smp_processor_id());
+	struct xps_pcpu_queue *q;
+	struct sk_buff *skb;
+
+	for_each_cpu_mask_nr(cpu, __get_cpu_var(xps_cpus)) {
+		q = &per_cpu(xps_pcpu_queue, cpu);
+		if (cpu_online(cpu)) {
+			spin_lock(&q->list.lock);
+			prevlen = skb_queue_len(&q->list);
+			skb_queue_splice_init(&head[cpu], &q->list);
+			spin_unlock(&q->list.lock);
+			/*
+			 * We hope remote cpu will be fast enough to transfert
+			 * this list to its completion queue before our
+			 * next xps_flush() call
+			 */
+			if (!prevlen)
+				__smp_call_function_single(cpu, &q->csd, 0);
+			continue;
+		}
+		/*
+		 * ok, we must free these skbs, even if we tried to avoid it :)
+		 */
+		while ((skb = __skb_dequeue(&head[cpu])) != NULL)
+			__kfree_skb(skb);
+	}
+	cpus_clear(__get_cpu_var(xps_cpus));
+}
+
+/*
+ * called from hardirq (IPI) context
+ */
+static void remote_free_skb_list(void *arg)
+{
+	struct sk_buff *last;
+	struct softnet_data *sd;
+	struct xps_pcpu_queue *q = arg; /* &__get_cpu_var(xps_pcpu_queue); */
+
+	spin_lock(&q->list.lock);
+
+	last = q->list.prev;
+	sd = &__get_cpu_var(softnet_data);
+	last->next = sd->completion_queue;
+	sd->completion_queue = q->list.next;
+	__skb_queue_head_init(&q->list);
+
+	spin_unlock(&q->list.lock);
+
+	raise_softirq_irqoff(NET_TX_SOFTIRQ);
+}
+
+void __init xps_init(void)
+{
+	int cpu, remote;
+	struct sk_buff_head *head;
+
+	xps_array = __alloc_percpu(nr_cpu_ids * sizeof(struct sk_buff_head),
+				   __alignof__(struct sk_buff_head));
+	if (!xps_array)
+		panic("XPS: Could not allocate xps_array\n");
+
+	for_each_possible_cpu(cpu) {
+		skb_queue_head_init(&per_cpu(xps_pcpu_queue, cpu).list);
+		per_cpu(xps_pcpu_queue, cpu).csd.func = remote_free_skb_list;
+		per_cpu(xps_pcpu_queue, cpu).csd.info = &per_cpu(xps_pcpu_queue, cpu);
+		head = per_cpu_ptr(xps_array, cpu);
+		for (remote = 0; remote < nr_cpu_ids; remote++)
+			__skb_queue_head_init(head + remote);
+	}
+}
+
+#endif /* CONFIG_SMP */

^ permalink raw reply related

* Re: Network hangs with 2.6.30.5
From: David Miller @ 2009-11-19 23:40 UTC (permalink / raw)
  To: ilpo.jarvinen; +Cc: jarkao2, holger.hoffstaette, netdev, eric.dumazet, zbr
In-Reply-To: <alpine.DEB.2.00.0910021104130.13543@wel-95.cs.helsinki.fi>

From: "Ilpo Järvinen" <ilpo.jarvinen@helsinki.fi>
Date: Fri, 2 Oct 2009 11:11:55 +0300 (EEST)

> On Thu, 1 Oct 2009, David Miller wrote:
> 
>> From: Jarek Poplawski <jarkao2@gmail.com>
>> Date: Mon, 7 Sep 2009 07:21:43 +0000
>> 
>> I also looked through all the TCP commits in 2.6.29 to 2.6.30
>> and I could not find anything else that might cause stalls with
>> time-wait recycled connections.
> 
> What about the more than 64k connections change a9d8f9110d7e953c2f2 (or 
> its fixes), it might be another possibility? ...It certainly does 
> something related to reuse and happens to be in the correct time frame... 
> (I've added Evgeniy).

So I've been studying this one quite a bit.

This change could only cause problems in timewait recycling if it:

1) Would accept using a bind bucket that previously it would not

2) Caused socket corruption due to bad locking

And I can find neither problem with the Evgeniy's bind changes.

So we've back to square one I think.

^ permalink raw reply

* Re: wanPMC-CxT1E1
From: Greg KH @ 2009-11-19 22:25 UTC (permalink / raw)
  To: Bob Beers; +Cc: netdev, Krzysztof Halasa
In-Reply-To: <4f6ba3b0911021316v16ff7431s4dfa939743541df6@mail.gmail.com>

On Mon, Nov 02, 2009 at 04:16:25PM -0500, Bob Beers wrote:
> >>
> >> commit d54e08030785153c8c0f4eb4f1cf320d60fff286
> >> Author: Bob Beers <bob.beers@gmail.com>
> >> Date:   Mon Nov 2 15:06:19 2009 -0500
> >>
> >>     Add CXT1E1 [1,2,4] channel wan card driver.
> >>
> >>  drivers/net/wan/Makefile                     |    1 +
> >
> > This file shouldn't need to be modified.  But I can fix that up by hand.
> >
> 
> Yeah, sorry about that.

Sorry for the long delay, my fault :(

Anyway, I need a "Signed-off-by:" line for the patch so that I can apply
it.  Also, who is the original author of the driver, any ideas?  We
should at least credit them with this, right?

Can you resend with a signed-off-by: line, and the other fixes all
rolled up into one patch?

thanks,

greg k-h

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox