* Re: [PATCH 3/3] [RFC] Changes for MQ vhost
From: Krishna Kumar2 @ 2011-03-01 16:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: anthony, arnd, avi, davem, eric.dumazet, horms, kvm, netdev,
rusty
In-Reply-To: <20110228100423.GC28006@redhat.com>
"Michael S. Tsirkin" <mst@redhat.com> wrote on 02/28/2011 03:34:23 PM:
> > The number of vhost threads is <= #txqs. Threads handle more
> > than one txq when #txqs is more than MAX_VHOST_THREADS (4).
>
> It is this sharing that prevents us from just reusing multiple vhost
> descriptors?
Sorry, I didn't understand this question.
> 4 seems a bit arbitrary - do you have an explanation
> on why this is a good number?
I was not sure what is the best way - a sysctl parameter? Or should the
maximum depend on number of host cpus? But that results in too many
threads, e.g. if I have 16 cpus and 16 txqs.
> > + struct task_struct *worker; /* worker for this vq */
> > + spinlock_t *work_lock; /* points to a dev->work_lock[] entry
*/
> > + struct list_head *work_list; /* points to a dev->work_list[]
entry */
> > + int qnum; /* 0 for RX, 1 -> n-1 for TX */
>
> Is this right?
Will fix this.
> > @@ -122,12 +128,33 @@ struct vhost_dev {
> > int nvqs;
> > struct file *log_file;
> > struct eventfd_ctx *log_ctx;
> > - spinlock_t work_lock;
> > - struct list_head work_list;
> > - struct task_struct *worker;
> > + spinlock_t *work_lock[MAX_VHOST_THREADS];
> > + struct list_head *work_list[MAX_VHOST_THREADS];
>
> This looks a bit strange. Won't sticking everything in a single
> array of structures rather than multiple arrays be better for cache
> utilization?
Correct. In that context, which is better:
struct {
spinlock_t *work_lock;
struct list_head *work_list;
} work[MAX_VHOST_THREADS];
or, to make sure work_lock/work_list is cache-aligned:
struct work_lock_list {
spinlock_t work_lock;
struct list_head work_list;
} ____cacheline_aligned_in_smp;
and define:
struct vhost_dev {
...
struct work_lock_list work[MAX_VHOST_THREADS];
};
Second method uses a little more space but each vhost needs only
one (read-only) cache line. I tested with this and can confirm it
aligns each element on a cache-line. BW improved slightly (upto
3%), remote SD improves by upto -4% or so.
> > +static inline int get_nvhosts(int nvqs)
>
> nvhosts -> nthreads?
Yes.
> > +static inline int vhost_get_thread_index(int index, int numtxqs, int
nvhosts)
> > +{
> > + return (index % numtxqs) % nvhosts;
> > +}
> > +
>
> As the only caller passes MAX_VHOST_THREADS,
> just use that?
Yes, nice catch.
> > struct vhost_net {
> > struct vhost_dev dev;
> > - struct vhost_virtqueue vqs[VHOST_NET_VQ_MAX];
> > - struct vhost_poll poll[VHOST_NET_VQ_MAX];
> > + struct vhost_virtqueue *vqs;
> > + struct vhost_poll *poll;
> > + struct socket **socks;
> > /* Tells us whether we are polling a socket for TX.
> > * We only do this when socket buffer fills up.
> > * Protected by tx vq lock. */
> > - enum vhost_net_poll_state tx_poll_state;
> > + enum vhost_net_poll_state *tx_poll_state;
>
> another array?
Yes... I am also allocating twice the space than what is required
to make it's usage simple. Please let me know what you feel about
this.
Thanks,
- KK
^ permalink raw reply
* Re: [PATCH 2/3] [RFC] Changes for MQ virtio-net
From: Krishna Kumar2 @ 2011-03-01 16:02 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: anthony, arnd, avi, davem, eric.dumazet, horms, kvm, netdev,
rusty
In-Reply-To: <20110228094320.GB28006@redhat.com>
"Michael S. Tsirkin" <mst@redhat.com> wrote on 02/28/2011 03:13:20 PM:
Thank you once again for your feedback on both these patches.
I will send the qemu patch tomorrow. I will also send the next
version incorporating these suggestions once we finalize some
minor points.
> Overall looks good.
> The numtxqs meaning the number of rx queues needs some cleanup.
> init/cleanup routines need more symmetry.
> Error handling on setup also seems slightly buggy or at least
asymmetrical.
> Finally, this will use up a large number of MSI vectors,
> while TX interrupts mostly stay unused.
>
> Some comments below.
>
> > +/* Maximum number of individual RX/TX queues supported */
> > +#define VIRTIO_MAX_TXQS 16
> > +
>
> This also does not seem to belong in the header.
Both virtio-net and vhost need some check to make sure very
high values are not passed by userspace. Is this not required?
> > +#define VIRTIO_NET_F_NUMTXQS 21 /* Device supports multiple
TX queue */
>
> VIRTIO_NET_F_MULTIQUEUE ?
Yes, that's a better name.
> > @@ -34,6 +38,8 @@ struct virtio_net_config {
> > __u8 mac[6];
> > /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
> > __u16 status;
> > + /* number of RX/TX queues */
> > + __u16 numtxqs;
>
> The interface here is a bit ugly:
> - this is really both # of tx and rx queues but called numtxqs
> - there's a hardcoded max value
> - 0 is assumed to be same as 1
> - assumptions above are undocumented.
>
> One way to address this could be num_queue_pairs, and something like
> /* The actual number of TX and RX queues is num_queue_pairs +
1 each. */
> __u16 num_queue_pairs;
> (and tweak code to match).
>
> Alternatively, have separate registers for the number of tx and rx
queues.
OK, so virtio_net_config has num_queue_pairs, and this gets converted to
numtxqs in virtnet_info?
> > +struct virtnet_info {
> > + struct send_queue **sq;
> > + struct receive_queue **rq;
> > +
> > + /* read-mostly variables */
> > + int numtxqs ____cacheline_aligned_in_smp;
>
> Why do you think this alignment is a win?
Actually this code was from the earlier patchset (MQ TX only) where
the layout was different. Now rq and sq are allocated as follows:
vi->sq = kzalloc(numtxqs * sizeof(*vi->sq), GFP_KERNEL);
for (i = 0; i < numtxqs; i++) {
vi->sq[i] = kzalloc(sizeof(*vi->sq[i]), GFP_KERNEL);
Since the two pointers becomes read-only during use, there is no cache
line dirty'ing. I will remove this directive.
> > +/*
> > + * Note for 'qnum' below:
> > + * first 'numtxqs' vqs are RX, next 'numtxqs' vqs are TX.
> > + */
>
> Another option to consider is to have them RX,TX,RX,TX:
> this way vq->queue_index / 2 gives you the
> queue pair number, no need to read numtxqs. On the other hand, it makes
the
> #RX==#TX assumption even more entrenched.
OK. I was following how many drivers were allocating RX and TX's
together - eg ixgbe_adapter has tx_ring and rx_ring arrays; bnx2
has rx_buf_ring and tx_buf_ring arrays, etc. Also, vhost has some
code that processes tx first before rx (e.g. vhost_net_stop/flush),
so this approach seemed helpful. I am OK either way, what do you
suggest?
> > + err = vi->vdev->config->find_vqs(vi->vdev, totalvqs, vqs,
callbacks,
> > + (const char
**)names);
> > + if (err)
> > + goto free_params;
> > +
>
> This would use up quite a lot of vectors. However,
> tx interrupt is, in fact, slow path. So, assuming we don't have
> enough vectors to use per vq, I think it's a good idea to
> support reducing MSI vector usage by mapping all TX VQs to the same
vector
> and separate vectors for RX.
> The hypervisor actually allows this, but we don't have an API at the
virtio
> level to pass that info to virtio pci ATM.
> Any idea what a good API to use would be?
Yes, it is a waste to have these vectors for tx ints. I initially
thought of adding a flag to virtio_device to pass to vp_find_vqs,
but it won't work, so a new API is needed. I can work with you on
this in the background if you like.
> > + for (i = 0; i < numtxqs; i++) {
> > + vi->rq[i]->rvq = vqs[i];
> > + vi->sq[i]->svq = vqs[i + numtxqs];
>
> This logic is spread all over. We need some kind of macro to
> get queue number of vq number and back.
Will add this.
> > + if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VQ)) {
> > + vi->cvq = vqs[i + numtxqs];
> > +
> > + if (virtio_has_feature(vi->vdev,
VIRTIO_NET_F_CTRL_VLAN))
> > + vi->dev->features |=
NETIF_F_HW_VLAN_FILTER;
>
> This bit does not seem to belong in initialize_vqs.
I will move it back to probe.
> > + err = virtio_config_val(vdev, VIRTIO_NET_F_NUMTXQS,
> > + offsetof(struct
virtio_net_config, numtxqs),
> > + &numtxqs);
> > +
> > + /* We need atleast one txq */
> > + if (err || !numtxqs)
> > + numtxqs = 1;
>
> err is okay, but should we just fail on illegal values?
> Or change the semantics:
> n = 0;
> err = virtio_config_val(vdev, VIRTIO_NET_F_NUMTXQS,
> offsetof(struct virtio_net_config, numtxqs),
> &n);
> numtxq = n + 1;
Will this be better:
int num_queue_pairs = 2;
int numtxqs;
err = virtio_config_val(vdev, VIRTIO_NET_F_MULTIQUEUE,
offsetof(struct virtio_net_config,
num_queue_pairs), &num_queue_pairs);
<ignore error, if any>
numtxqs = num_queue_pairs / 2;
> > + if (numtxqs > VIRTIO_MAX_TXQS)
> > + return -EINVAL;
>
> Do we strictly need this?
> I think we should just use whatever hardware has,
> or alternatively somehow ignore the unused queues
> (easy for tx, not sure about rx).
vq's are matched between qemu, virtio-net and vhost. Isn't some check
required that userspace has not passed a bad value?
> > + if (vi->rq[i]->num == 0) {
> > + err = -ENOMEM;
> > + goto free_recv_bufs;
> > + }
> > }
> If this fails for vq > 0, you have to detach bufs.
Right, will fix this.
> > free_vqs:
> > + for (i = 0; i < numtxqs; i++)
> > + cancel_delayed_work_sync(&vi->rq[i]->refill);
> > vdev->config->del_vqs(vdev);
> > -free:
> > + free_rq_sq(vi);
>
> If we have a wrapper to init all vqs, pls add a wrapper to clean up
> all vqs as well.
Will add that.
> > + for (i = 0; i < vi->numtxqs; i++) {
> > + struct virtqueue *rvq = vi->rq[i]->rvq;
> > +
> > + while (1) {
> > + buf = virtqueue_detach_unused_buf
(rvq);
> > + if (!buf)
> > + break;
> > + if (vi->mergeable_rx_bufs || vi->
big_packets)
> > + give_pages(vi->rq[i],
buf);
> > + else
> > + dev_kfree_skb(buf);
> > + --vi->rq[i]->num;
> > + }
> > + BUG_ON(vi->rq[i]->num != 0);
> > }
> > - BUG_ON(vi->num != 0);
> > +
> > + free_rq_sq(vi);
>
>
> This looks wrong here. This function should detach
> and free all bufs, not internal malloc stuff.
That is being done by free_receive_buf after free_unused_bufs()
returns. I hope this addresses your point.
> I think we should have free_unused_bufs that handles
> a single queue, and call it in a loop.
OK, so define free_unused_bufs() as:
static void free_unused_bufs(struct virtnet_info *vi, struct virtqueue
*svq,
struct virtqueue *rvq)
{
/* Use svq and rvq with the remaining code unchanged */
}
Thanks,
- KK
^ permalink raw reply
* [PATCH net-next-2.6] benet: use GFP_KERNEL allocations when possible
From: Eric Dumazet @ 2011-03-01 15:48 UTC (permalink / raw)
To: Ajit Khaparde; +Cc: netdev
Extend be_alloc_pages() with a gfp parameter, so that we use GFP_KERNEL
allocations instead of GFP_ATOMIC when not running in softirq context.
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
drivers/net/benet/be_main.c | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/net/benet/be_main.c b/drivers/net/benet/be_main.c
index 0bdccb1..ef66dc6 100644
--- a/drivers/net/benet/be_main.c
+++ b/drivers/net/benet/be_main.c
@@ -1169,20 +1169,20 @@ static inline void be_rx_compl_reset(struct be_eth_rx_compl *rxcp)
rxcp->dw[offsetof(struct amap_eth_rx_compl, valid) / 32] = 0;
}
-static inline struct page *be_alloc_pages(u32 size)
+static inline struct page *be_alloc_pages(u32 size, gfp_t gfp)
{
- gfp_t alloc_flags = GFP_ATOMIC;
u32 order = get_order(size);
+
if (order > 0)
- alloc_flags |= __GFP_COMP;
- return alloc_pages(alloc_flags, order);
+ gfp |= __GFP_COMP;
+ return alloc_pages(gfp, order);
}
/*
* Allocate a page, split it to fragments of size rx_frag_size and post as
* receive buffers to BE
*/
-static void be_post_rx_frags(struct be_rx_obj *rxo)
+static void be_post_rx_frags(struct be_rx_obj *rxo, gfp_t gfp)
{
struct be_adapter *adapter = rxo->adapter;
struct be_rx_page_info *page_info_tbl = rxo->page_info_tbl;
@@ -1196,7 +1196,7 @@ static void be_post_rx_frags(struct be_rx_obj *rxo)
page_info = &rxo->page_info_tbl[rxq->head];
for (posted = 0; posted < MAX_RX_POST && !page_info->page; posted++) {
if (!pagep) {
- pagep = be_alloc_pages(adapter->big_page_size);
+ pagep = be_alloc_pages(adapter->big_page_size, gfp);
if (unlikely(!pagep)) {
rxo->stats.rx_post_fail++;
break;
@@ -1753,7 +1753,7 @@ static int be_poll_rx(struct napi_struct *napi, int budget)
/* Refill the queue */
if (atomic_read(&rxo->q.used) < RX_FRAGS_REFILL_WM)
- be_post_rx_frags(rxo);
+ be_post_rx_frags(rxo, GFP_ATOMIC);
/* All consumed */
if (work_done < budget) {
@@ -1890,7 +1890,7 @@ static void be_worker(struct work_struct *work)
if (rxo->rx_post_starved) {
rxo->rx_post_starved = false;
- be_post_rx_frags(rxo);
+ be_post_rx_frags(rxo, GFP_KERNEL);
}
}
if (!adapter->ue_detected && !lancer_chip(adapter))
@@ -2138,7 +2138,7 @@ static int be_open(struct net_device *netdev)
u16 link_speed;
for_all_rx_queues(adapter, rxo, i) {
- be_post_rx_frags(rxo);
+ be_post_rx_frags(rxo, GFP_KERNEL);
napi_enable(&rxo->rx_eq.napi);
}
napi_enable(&tx_eq->napi);
^ permalink raw reply related
* Re: [PATCH (sh-2.6) 1/4] clksource: Generic timer infrastructure
From: Stuart Menefy @ 2011-03-01 15:20 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Peppe CAVALLARO, linux-sh@vger.kernel.org, netdev@vger.kernel.org,
John Stultz, Thomas Gleixner, linux-kernel@vger.kernel.org
In-Reply-To: <201102241820.55873.arnd@arndb.de>
Hi Arnd
Thanks for the comments.
On 24/02/11 17:20, Arnd Bergmann wrote:
> On Tuesday 22 February 2011, Peppe CAVALLARO wrote:
>> From: Stuart Menefy <stuart.menefy@st.com>
>>
>> Many devices targeted at the embedded market provide a number of
>> generic timers which are capable of generating interrupts at a
>> requested rate. These can then be used in the implementation of drivers
>> for other peripherals which require a timer interrupt, without having
>> to provide an additional timer as part of that peripheral.
>>
>> A code provides a simple abstraction layer which allows a timer to be
>> registered, and for a driver to request a timer.
>>
>> Currently this doesn't provide any of the additional information, such
>> as precision or position in clock framework which might be required
>> for a fully featured driver.
>
> This code should probably be discussed on a more broader
> platform than the netdev and linux-sh mailing lists,
> as the scope is neither sh nor network specific.
>
> You should at least add linux-kernel@vger.kernel.org, possibly
> also linux-arch@vger.kernel.org.
>
> Further, John and Thomas are responsible for the timekeeping
> infrastructure, and they are probably interested in this
> as well.
>
> Why is this code useful to you? In the scenarios I've seen, the
> board can always assign a timer to a specific device in a fixed
> way that can be describe in a board file or device tree.
What we were trying to do was separate the code which actually manipulates
the timer hardware from the code which wants that timer service. In this
case it was a network device driver which is used on multiple SoC devices,
while the timer hardware tends to differ from device to device.
The other user of this code which we have is an OProfile driver, which
with this change can now be independent of the hardware it is running on,
while the previous version manipulated the timer hardware directly.
> Also, what is the difference between this and clkdev?
clkdev can be used to find a struct clk, which is fine if you just want to
read the time. In this instance we want to get interrupts from the timer
hardware, which isn't supported by the clk infrastructure.
If anything this duplicates clockevents. The main reason for not using
clockevents was that nobody else does! Currently clockevents are
used strictly for time keeping within the kernel, and most architectures
only register those which are intended to be used for this purpose.
We felt a bit nervous about adding code to register all the device's timers
as clockevents, and having the network device driver pick up one of those
for its own use.
>> Signed-off-by: Stuart Menefy <stuart.menefy@st.com>
>> Hacked-by: Giuseppe Cavallaro <peppe.cavallaro@st.com>
>> ---
>> drivers/clocksource/Makefile | 1 +
>> drivers/clocksource/generictimer.c | 60 ++++++++++++++++++++++++++++++++++++
>> include/linux/generictimer.h | 41 ++++++++++++++++++++++++
>> 3 files changed, 102 insertions(+), 0 deletions(-)
>> create mode 100644 drivers/clocksource/generictimer.c
>> create mode 100644 include/linux/generictimer.h
>
> I don't think it fits that well into the drivers/clocksource directory,
> because you don't actually register a struct clock_event_device or
> struct clocksource.
True. The intent was that this would be a third interface onto timer
hardware, alongside clocksources and clockevents.
> I don't know if this could also be merged with the clocksource infrastructure,
> but your code currently doesn't do that.
It would probably be clockevent rather than clocksource, but it could be if
people felt that was a better way to go.
>> +struct generic_timer *generic_timer_claim(void (*handler) (void *), void *data)
>> +{
>> + struct generic_timer *gt = NULL;
>> +
>> + if (!handler) {
>> + pr_err("%s: invalid handler\n", __func__);
>> + return NULL;
>> + }
>> +
>> + mutex_lock(>_mutex);
>> + if (!list_empty(>_list)) {
>> + struct list_head *list = gt_list.next;
>> + list_del(list);
>> + gt = container_of(list, struct generic_timer, list);
>> + }
>> + mutex_unlock(>_mutex);
>> +
>> + if (!gt)
>> + return NULL;
>> +
>> + /* Prepare the new handler */
>> + gt->priv_handler = handler;
>> + gt->data = data;
>> +
>> + return gt;
>> +}
>
> This does not seem very generic. You put timers into the list and take
> them out again, but don't have any way to deal with timers that match
> specific purposes. It obviously works for your specific use case where
> you register exactly one timer, and use that in exactly one driver.
>
> If more drivers were converted to generic_timer, which is obviously
> the intention, then you might have a situation with very different
> timers (fixed/variable tick, high/low frequencies, accurate/inaccurate),
> or you might have fewer timers than users.
Agreed, this was a first 'keep it simple' pass, maybe its too simple.
^ permalink raw reply
* [PATCH 1/2 net-next][v2] bonding: fix incorrect transmit queue offset
From: Andy Gospodarek @ 2011-03-01 15:31 UTC (permalink / raw)
To: Phil Oester; +Cc: David Miller, bhutchings, andy, netdev, fubar
In-Reply-To: <20110225225636.GA18792@linuxace.com>
On Fri, Feb 25, 2011 at 02:56:36PM -0800, Phil Oester wrote:
> On Wed, Feb 23, 2011 at 03:54:51PM -0800, David Miller wrote:
> > From: Ben Hutchings <bhutchings@solarflare.com>
> > Date: Wed, 23 Feb 2011 23:37:49 +0000
> >
> > > On Wed, 2011-02-23 at 15:13 -0800, David Miller wrote:
> > >> From: Phil Oester <kernel@linuxace.com>
> > >> Date: Wed, 23 Feb 2011 15:08:44 -0800
> > >>
> > >> > On Wed, Feb 23, 2011 at 02:42:49PM -0500, Andy Gospodarek wrote:
> > >> >> + while (txq >= dev->real_num_tx_queues) {
> > >> >> + /* let the user know if we do not have enough tx queues */
> > >> >> + if (net_ratelimit())
> > >> >> + pr_warning("%s selects invalid tx queue %d. Consider"
> > >> >> + " setting module option tx_queues > %d.",
> > >> >> + dev->name, txq, dev->real_num_tx_queues);
> > >> >> + txq -= dev->real_num_tx_queues;
> > >> >> + }
> > >> >
> > >> > Think this would be better as a WARN_ONCE, as otherwise syslog will still
> > >> > get flooded with this - even when ratelimited. See get_rps_cpu in
> > >> > net/core/dev.c as an example.o
> > >>
> > >> Agreed.
> > >
> > > This shouldn't WARN at all. It is perfectly valid (though non-optimal)
> > > to have different numbers of queues on two different multiqueue devices.
> >
> > That's also a good point.
>
> The patch works as expected. Do we have any agreement on a final version?
>
Thanks for the testing, Phil.
I'm in favor of this patch as it does alert the admin that bonding may
not have enough default queues, but it is not as verbose (backtrace et
al) and likely to create bug reports as a message from WARN_ON.
Signed-off-by: Andy Gospodarek <andy@greyhouse.net>
---
drivers/net/bonding/bond_main.c | 26 +++++++++++++++++++-------
1 files changed, 19 insertions(+), 7 deletions(-)
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index 584f97b..acc05d6 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -4643,15 +4643,27 @@ out:
return res;
}
+/*
+ * This helper function exists to help dev_pick_tx get the correct
+ * destination queue. Using a helper function skips the a call to
+ * skb_tx_hash and will put the skbs in the queue we expect on their
+ * way down to the bonding driver.
+ */
static u16 bond_select_queue(struct net_device *dev, struct sk_buff *skb)
{
- /*
- * This helper function exists to help dev_pick_tx get the correct
- * destination queue. Using a helper function skips the a call to
- * skb_tx_hash and will put the skbs in the queue we expect on their
- * way down to the bonding driver.
- */
- return skb->queue_mapping;
+ u16 txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : 0;
+
+ if (txq >= dev->real_num_tx_queues) {
+ /* let the user know if we do not have enough tx queues */
+ if (net_ratelimit())
+ pr_warning("%s selects invalid tx queue %d. Consider"
+ " setting module option tx_queues > %d.",
+ dev->name, txq, dev->real_num_tx_queues);
+ do
+ txq -= dev->real_num_tx_queues;
+ while (txq >= dev->real_num_tx_queues);
+ }
+ return txq;
}
static netdev_tx_t bond_start_xmit(struct sk_buff *skb, struct net_device *dev)
^ permalink raw reply related
* Re: [patch net-next-2.6] bonding: remove skb_share_check in handle_frame
From: Changli Gao @ 2011-03-01 15:12 UTC (permalink / raw)
To: Jiri Pirko; +Cc: netdev, davem, fubar, eric.dumazet, nicolas.2p.debian, andy
In-Reply-To: <20110301092907.GG2855@psychotron.redhat.com>
On Tue, Mar 1, 2011 at 5:29 PM, Jiri Pirko <jpirko@redhat.com> wrote:
> Unapplicable, sorry (wrong branch :(). Here's corrected patch:
>
> Subject: [PATCH net-next-2.6 v2] bonding: remove skb_share_check in handle_frame
>
> No need to do share check here.
>
I don't think so. Although you avoid netif_rx(), you can't avoid
ptype_all handlers. In fact, all the RX handlers should has this
check(), if they may modify the skb.
--
Regards,
Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
* Re: [PATCH 2/4] slub,rcu: don't assume the size of struct rcu_head
From: Christoph Lameter @ 2011-03-01 15:11 UTC (permalink / raw)
To: Pekka Enberg
Cc: Lai Jiangshan, Ingo Molnar, Paul E. McKenney, Eric Dumazet,
David S. Miller, Matt Mackall, linux-mm, linux-kernel, netdev
In-Reply-To: <AANLkTimXy2Yaj+NTDMNTWuLqHHfKZJhVDpeXj3CfMvBf@mail.gmail.com>
On Tue, 1 Mar 2011, Pekka Enberg wrote:
> The SLAB and SLUB patches are fine by me if there are going to be real
> users for this. Christoph, Paul?
The solution is a bit overkill. It would be much simpler to add a union to
struct page that has lru and the rcu in there similar things can be done
for SLAB and the network layer. A similar issue already exists for the
spinlock in struct page. Lets follow the existing way of handling this.
Struct page may be larger for debugging purposes already because of the
need for extended spinlock data.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 15:07 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298991160.3284.108.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 03:52:40PM +0100, Eric Dumazet wrote:
> Le mardi 01 mars 2011 à 09:30 -0500, Thomas Graf a écrit :
> > On Tue, Mar 01, 2011 at 09:22:35AM -0500, Thomas Graf wrote:
> > > On Tue, Mar 01, 2011 at 03:06:59PM +0100, Eric Dumazet wrote:
> > > > Would be nice to cpu affine named to _not_ run on CPU11, just to
> > > > specialize it for TX completions and have softirq time percentage and
> > > > "perf top -C 11 " results
> >
> > CPU 1 isolated as well (named running with mask 0,2-10)
> >
> > ----------------------------------------------------------------------------------------------------------------------
> > PerfTop: 580 irqs/sec kernel:100.0% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
> > ----------------------------------------------------------------------------------------------------------------------
> >
> > samples pcnt function DSO
> > _______ _____ ___________________________ ___________________________________________________________
> >
> > 283.00 9.2% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> > 256.00 8.4% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
> > 190.00 6.2% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> > 182.00 5.9% get_page_from_freelist /lib/modules/2.6.38-rc5+/build/vmlinux
> > 157.00 5.1% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
> > 143.00 4.7% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
> > 133.00 4.3% sock_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
> > 133.00 4.3% __udp4_lib_lookup /lib/modules/2.6.38-rc5+/build/vmlinux
> > 131.00 4.3% sk_run_filter /lib/modules/2.6.38-rc5+/build/vmlinux
>
> sk_run_filter ? Do you have a packet filter running ?
dhclient was running. With dhclient killed:
----------------------------------------------------------------------------------------------------------------------
PerfTop: 726 irqs/sec kernel:99.9% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
----------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ___________________________ ___________________________________________________________
472.00 10.6% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
419.00 9.4% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
280.00 6.3% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
259.00 5.8% get_page_from_freelist /lib/modules/2.6.38-rc5+/build/vmlinux
248.00 5.6% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
238.00 5.4% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
204.00 4.6% sock_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
189.00 4.3% __udp4_lib_lookup /lib/modules/2.6.38-rc5+/build/vmlinux
178.00 4.0% getnstimeofday /lib/modules/2.6.38-rc5+/build/vmlinux
169.00 3.8% __alloc_skb /lib/modules/2.6.38-rc5+/build/vmlinux
144.00 3.2% read_tsc /lib/modules/2.6.38-rc5+/build/vmlinux
143.00 3.2% sock_def_readable /lib/modules/2.6.38-rc5+/build/vmlinux
138.00 3.1% udp_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
115.00 2.6% kmem_cache_alloc_node_trace /lib/modules/2.6.38-rc5+/build/vmlinux
114.00 2.6% __netif_receive_skb /lib/modules/2.6.38-rc5+/build/vmlinux
109.00 2.5% _raw_spin_lock /lib/modules/2.6.38-rc5+/build/vmlinux
100.00 2.3% is_swiotlb_buffer /lib/modules/2.6.38-rc5+/build/vmlinux
90.00 2.0% __phys_addr /lib/modules/2.6.38-rc5+/build/vmlinux
82.00 1.8% __udp4_lib_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
80.00 1.8% kmem_cache_alloc_node /lib/modules/2.6.38-rc5+/build/vmlinux
73.00 1.6% ip_route_input_common /lib/modules/2.6.38-rc5+/build/vmlinux
60.00 1.4% memcpy /lib/modules/2.6.38-rc5+/build/vmlinux
59.00 1.3% dma_issue_pending_all /lib/modules/2.6.38-rc5+/build/vmlinux
58.00 1.3% ip_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
57.00 1.3% be_post_rx_frags /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
49.00 1.1% __alloc_pages_nodemask /lib/modules/2.6.38-rc5+/build/vmlinux
45.00 1.0% alloc_pages_current /lib/modules/2.6.38-rc5+/build/vmlinux
27.00 0.6% get_rps_cpu /lib/modules/2.6.38-rc5+/build/vmlinux
23.00 0.5% napi_complete /lib/modules/2.6.38-rc5+/build/vmlinux
22.00 0.5% ip_local_deliver /lib/modules/2.6.38-rc5+/build/vmlinux
18.00 0.4% selinux_socket_sock_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
17.00 0.4% native_read_tsc /lib/modules/2.6.38-rc5+/build/vmlinux
16.00 0.4% local_bh_enable /lib/modules/2.6.38-rc5+/build/vmlinux
16.00 0.4% next_zones_zonelist /lib/modules/2.6.38-rc5+/build/vmlinux
14.00 0.3% sk_filter /lib/modules/2.6.38-rc5+/build/vmlinux
13.00 0.3% eth_type_trans /lib/modules/2.6.38-rc5+/build/vmlinux
10.00 0.2% __kmalloc_node_track_caller /lib/modules/2.6.38-rc5+/build/vmlinux
10.00 0.2% _raw_spin_lock_irqsave /lib/modules/2.6.38-rc5+/build/vmlinux
9.00 0.2% raw_local_deliver /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.2% __udp_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.2% netif_receive_skb /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.2% ip_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
7.00 0.2% net_rx_action /lib/modules/2.6.38-rc5+/build/vmlinux
6.00 0.1% swiotlb_map_page /lib/modules/2.6.38-rc5+/build/vmlinux
6.00 0.1% __sk_mem_schedule /lib/modules/2.6.38-rc5+/build/vmlinux
6.00 0.1% dso__find_symbol /usr/bin/perf
6.00 0.1% __netdev_alloc_skb /lib/modules/2.6.38-rc5+/build/vmlinux
^ permalink raw reply
* source route ignored in favor of local interface
From: Joe Buehler @ 2011-03-01 14:57 UTC (permalink / raw)
To: netdev
I have a LINUX box talking on many different networks at the same time. Since
IP addresses on the networks can overlap (they are completely different
networks) we use source routing and NAT to get packets going in and out of the
right interfaces.
Everything works great, with one exception. If I try to talk to a remote host
that happens to have the same IP address as one of my interfaces, the kernel
routes the packet to the local interface.
It looks to me as though the problem is that the source routes are lower
priority than the local interfaces. As soon as the kernel sees a destination
address that matches a local interface it routes to the local interface and pays
no attention to the source route.
I consider this a bug. Is there any way to change this behavior?
The kernel involved is 2.6.27.7, with patches from Cavium for support of their
hardware.
Joe Buehler
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 14:52 UTC (permalink / raw)
To: Thomas Graf
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301143050.GB10761@canuck.infradead.org>
Le mardi 01 mars 2011 à 09:30 -0500, Thomas Graf a écrit :
> On Tue, Mar 01, 2011 at 09:22:35AM -0500, Thomas Graf wrote:
> > On Tue, Mar 01, 2011 at 03:06:59PM +0100, Eric Dumazet wrote:
> > > Would be nice to cpu affine named to _not_ run on CPU11, just to
> > > specialize it for TX completions and have softirq time percentage and
> > > "perf top -C 11 " results
>
> CPU 1 isolated as well (named running with mask 0,2-10)
>
> ----------------------------------------------------------------------------------------------------------------------
> PerfTop: 580 irqs/sec kernel:100.0% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
> ----------------------------------------------------------------------------------------------------------------------
>
> samples pcnt function DSO
> _______ _____ ___________________________ ___________________________________________________________
>
> 283.00 9.2% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 256.00 8.4% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
> 190.00 6.2% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 182.00 5.9% get_page_from_freelist /lib/modules/2.6.38-rc5+/build/vmlinux
> 157.00 5.1% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
> 143.00 4.7% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
> 133.00 4.3% sock_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
> 133.00 4.3% __udp4_lib_lookup /lib/modules/2.6.38-rc5+/build/vmlinux
> 131.00 4.3% sk_run_filter /lib/modules/2.6.38-rc5+/build/vmlinux
sk_run_filter ? Do you have a packet filter running ?
^ permalink raw reply
* Re: [PATCH]drivers:isdn:istream.c Fix typo pice to piece
From: Jiri Kosina @ 2011-03-01 14:47 UTC (permalink / raw)
To: Justin P. Mattock; +Cc: mac, isdn, netdev
In-Reply-To: <alpine.LNX.2.00.1103011521160.32580@pobox.suse.cz>
On Tue, 1 Mar 2011, Jiri Kosina wrote:
> > The below patch changes a typo "pice" to "piece"
> >
> > Signed-off-by: Justin P. Mattock <justinmattock@gmail.com>
> >
> > ---
> > drivers/isdn/hardware/eicon/istream.c | 2 +-
> > 1 files changed, 1 insertions(+), 1 deletions(-)
> >
> > diff --git a/drivers/isdn/hardware/eicon/istream.c b/drivers/isdn/hardware/eicon/istream.c
> > index 18f8798..7bd5baa 100644
> > --- a/drivers/isdn/hardware/eicon/istream.c
> > +++ b/drivers/isdn/hardware/eicon/istream.c
> > @@ -62,7 +62,7 @@ void diva_xdi_provide_istream_info (ADAPTER* a,
> > stream interface.
> > If synchronous service was requested, then function
> > does return amount of data written to stream.
> > - 'final' does indicate that pice of data to be written is
> > + 'final' does indicate that piece of data to be written is
> > final part of frame (necessary only by structured datatransfer)
> > return 0 if zero lengh packet was written
> > return -1 if stream is full
>
> Applied.
Bah, I had broken local clone of linux-next, I see it's already there.
Sorry for the noise.
--
Jiri Kosina
SUSE Labs, Novell Inc.
^ permalink raw reply
* Re: Bug inkvm_set_irq
From: Jean-Philippe Menil @ 2011-03-01 14:39 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: netdev, kvm, virtualization
In-Reply-To: <20110301070333.GA5972@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 19679 bytes --]
Le 01/03/2011 08:03, Michael S. Tsirkin a écrit :
> On Mon, Feb 28, 2011 at 11:34:16PM +0100, Jean-Philippe Menil wrote:
>> Hi,
>>
>> here is another trace with kvm.ko compiled with debug flags.
>>
>> the bug:
>> [12099.503414] BUG: unable to handle kernel paging request at
>> 000000000b6635e9
>> [12099.503462] IP: [<ffffffffa03ee877>] kvm_set_irq+0x37/0x140 [kvm]
>> [12099.503521] PGD 45d8d2067 PUD 45d58e067 PMD 0
>> [12099.503560] Oops: 0000 [#1] SMP
>> [12099.503591] last sysfs file:
>> /sys/devices/system/cpu/cpu11/cache/index2/shared_cpu_map
>> [12099.503641] CPU 0
>> [12099.503648] Modules linked in: netconsole configfs vhost_net
>> macvtap macvlan tun veth powernow_k8 mperf cpufreq_userspace
>> cpufreq_stats cpufreq_powersave cpufreq_ondemand freq_table
>> cpufreq_conservative fuse xt_physdev ip6t_LOG ip6table_filter
>> ip6_tables ipt_LOG xt_multiport xt_limit xt_tcpudp xt_state
>> iptable_filter ip_tables x_tables nf_conntrack_tftp nf_conntrack_ftp
>> nf_conntrack_ipv4 nf_defrag_ipv4 8021q bridge stp ext2 mbcache
>> dm_round_robin dm_multipath nf_conntrack_ipv6 nf_conntrack
>> nf_defrag_ipv6 kvm_amd kvm ipv6 snd_pcm snd_timer snd soundcore
>> snd_page_alloc shpchp pci_hotplug tpm_tis i2c_nforce2 tpm i2c_core
>> pcspkr evdev psmouse joydev tpm_bios processor ghes dcdbas hed
>> button serio_raw thermal_sys xfs exportfs dm_mod sg sr_mod cdrom
>> usbhid hid usb_storage ses sd_mod enclosure megaraid_sas ohci_hcd
>> lpfc scsi_transport_fc bnx2 scsi_tgt scsi_mod ehci_hcd [last
>> unloaded: scsi_wait_scan]
>> [12099.504277]
>> [12099.504302] Pid: 1742, comm: kworker/0:2 Not tainted
>> 2.6.37.2-dsiun-110105+ #2 Dell Inc. PowerEdge M605/0K543T
>> [12099.504373] RIP: 0010:[<ffffffffa03ee877>] [<ffffffffa03ee877>]
>> kvm_set_irq+0x37/0x140 [kvm]
>> [12099.504444] RSP: 0018:ffff88045e013d00 EFLAGS: 00010246
>> [12099.504474] RAX: 000000000b6634c1 RBX: 0000000000000018 RCX:
>> 0000000000000001
>> [12099.504508] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
>> ffff880419b600c0
>> [12099.504541] RBP: ffff88045e013dd0 R08: ffff88045e012000 R09:
>> 0000000000000000
>> [12099.504575] R10: 0000000000000000 R11: 00000000ffffffff R12:
>> ffff880419b600c0
>> [12099.504609] R13: ffff880419b600c0 R14: ffffffffa03efaa0 R15:
>> 0000000000000001
>> [12099.504643] FS: 00007f3abaa05710(0000) GS:ffff88007f800000(0000)
>> knlGS:0000000000000000
>> [12099.504693] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [12099.504724] CR2: 000000000b6635e9 CR3: 000000045e2bc000 CR4:
>> 00000000000006f0
>> [12099.504757] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
>> 0000000000000000
>> [12099.504791] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
>> 0000000000000400
>> [12099.504825] Process kworker/0:2 (pid: 1742, threadinfo
>> ffkvm_set_irqff88045e012000, task ffff88045ffb0d60)
>> [12099.504874] Stack:
>> [12099.504897] 00000000000119c0 00000000000119c0 00000000000119c0
>> ffff88045ffb0d60
>> [12099.504953] ffff88045ffb1010 ffff88045e013fd8 ffff88045ffb1018
>> ffff88045e012010
>> [12099.505009] 00000000000119c0 ffff88045e013fd8 00000000000119c0
>> 00000000000119c0
>> [12099.505065] Call Trace:
>> [12099.505099] [<ffffffff813818ce>] ? common_interrupt+0xe/0x13
>> [12099.505145] [<ffffffffa03efaa0>] ? irqfd_inject+0x0/0x50 [kvm]
>> [12099.505145] [<ffffffffa03efaca>] irqfd_inject+0x2a/0x50 [kvm]
>> [12099.505145] [<ffffffff8106b7bb>] process_one_work+0x11b/0x450
>> [12099.505145] [<ffffffff8106bf37>] worker_thread+0x157/0x410
>> [12099.505145] [<ffffffff8103a569>] ? __wake_up_common+0x59/0x90
>> [12099.505145] [<ffffffff8106bde0>] ? worker_thread+0x0/0x410
>> [12099.505145] [<ffffffff8106f996>] kthread+0x96/0xa0
>> [12099.505145] [<ffffffff81003c64>] kernel_thread_helper+0x4/0x10
>> [12099.505145] [<ffffffff8106f900>] ? kthread+0x0/0xa0
>> [12099.505145] [<ffffffff81003c60>] ? kernel_thread_helper+0x0/0x10
>> [12099.505145] Code: 55 49 89 fd 41 54 53 89 d3 48 81 ec a8 00 00 00
>> 8b 15 a6 75 03 00 89 b5 3c ff ff ff 85 d2 0f 85 d5 00 00 00 49 8b 85
>> 58 24 00 00<3b> 98 28 01 00 00 73 61 89 db 48 8b 84 d8 30 01 00 00
>> 48 85 c0
>> [12099.505145] RIP [<ffffffffa03ee877>] kvm_set_irq+0x37/0x140 [kvm]
>> [12099.505145] RSP<ffff88045e013d00>
>> [12099.505145] CR2: 000000000b6635e9
>>
>>
>> markup_oops result:
>>
>> root@ayrshire:~# cat bug.txt | perl markup_oops.pl -m
>> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
>> /boot/vmlinuz-2.6.37.2-dsiun-110105+
>> vmaoffset = 18446744072103034880 ffffffffa03ee841: 48 89 e5 mov
>> %rsp,%rbp
>> ffffffffa03ee844: 41 57 push %r15
>> ffffffffa03ee846: 41 89 cf mov %ecx,%r15d | %r15
>> => 1 %ecx = 1
>> ffffffffa03ee849: 41 56 push %r14 | %r14
>> => ffffffffa03efaa0
>> ffffffffa03ee84b: 41 55 push %r13
>> ffffffffa03ee84d: 49 89 fd mov %rdi,%r13 | %edi
>> = ffff880419b600c0 %r13 => ffff880419b600c0
>> ffffffffa03ee850: 41 54 push %r12 | %r12
>> => ffff880419b600c0
>> ffffffffa03ee852: 53 push %rbx
>> ffffffffa03ee853: 89 d3 mov %edx,%ebx | %ebx => 18
>> ffffffffa03ee855: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
>> ffffffffa03ee85c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
>> # ffffffffa03ee862<kvm_set_irq+0x22>
>> ffffffffa03ee862: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) |
>> %esi = 0
>> ffffffffa03ee868: 85 d2 test %edx,%edx | %edx => 0
>> ffffffffa03ee86a: 0f 85 d5 00 00 00 jne ffffffffa03ee945
>> <kvm_set_irq+0x105>
>> ffffffffa03ee870: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
>> %eax => b6634c1 %r13 = ffff880419b600c0
>> *ffffffffa03ee877: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx |
>> %eax = b6634c1 %ebx = 18<--- faulting instruction
>> ffffffffa03ee87d: 73 61 jae ffffffffa03ee8e0
>> <kvm_set_irq+0xa0>
>> ffffffffa03ee87f: 89 db mov %ebx,%ebx
>> ffffffffa03ee881: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
>> ffffffffa03ee888: 00
>> ffffffffa03ee889: 48 85 c0 test %rax,%rax
>> ffffffffa03ee88c: 74 52 je ffffffffa03ee8e0
>> <kvm_set_irq+0xa0>
>> ffffffffa03ee88e: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
>> ffffffffa03ee895: 31 db xor %ebx,%ebx
>> ffffffffa03ee897: 48 8b 08 mov (%rax),%rcx
>> ffffffffa03ee89a: 83 c3 01 add $0x1,%ebx
>> ffffffffa03ee89d: 0f 18 09 prefetcht0 (%rcx)
>> ffffffffa03ee8a0: 48 8b 48 e0 mov -0x20(%rax),%rcx
>> ffffffffa03ee8a4: 48 89 0a mov %rcx,(%rdx)
>> ffffffffa03ee8a7: 48 8b 48 e8 mov -0x18(%rax),%rcx
>> ffffffffa03ee8ab: 48 89 4a 08 mov %rcx,0x8(%rdx)
>> ffffffffa03ee8af: 48 8b 48 f0 mov -0x10(%rax),%rcx
>> ffffffffa03ee8b3: 48 89 4a 10 mov %rcx,0x10(%rdx)
>> ffffffffa03ee8b7: 48 8b 48 f8 mov -0x8(%rax),%rcx
>> ffffffffa03ee8bb: 48 89 4a 18 mov %rcx,0x18(%rdx)
>> ffffffffa03ee8bf: 48 8b 08 mov (%rax),%rcx
>>
>> The relvant part of objdump for kvm_set_irq:
>> root@ayrshire:~# objdump -ldS
>> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko>
>> dump.txt
>>
>> 0000000000006840<kvm_set_irq>:
>> kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:148
>> 6840: 55 push %rbp
>> 6841: 48 89 e5 mov %rsp,%rbp
>> 6844: 41 57 push %r15
>> 6846: 41 89 cf mov %ecx,%r15d
>> 6849: 41 56 push %r14
>> 684b: 41 55 push %r13
>> 684d: 49 89 fd mov %rdi,%r13
>> 6850: 41 54 push %r12
>> 6852: 53 push %rbx
>> 6853: 89 d3 mov %edx,%ebx
>> 6855: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
>> trace_kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/include/trace/events/kvm.h:10
>> 685c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
>> # 6862<kvm_set_irq+0x22>
>> kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:148
>> 6862: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp)
>> trace_kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/include/trace/events/kvm.h:10
>> 6868: 85 d2 test %edx,%edx
>> 686a: 0f 85 d5 00 00 00 jne 6945<kvm_set_irq+0x105>
>> kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
>> 6870: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
>> 6877: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
>> 687d: 73 61 jae 68e0<kvm_set_irq+0xa0>
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:163
>> 687f: 89 db mov %ebx,%ebx
>> 6881: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
>> 6888: 00
>> 6889: 48 85 c0 test %rax,%rax
>> 688c: 74 52 je 68e0<kvm_set_irq+0xa0>
>> 688e: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
>> 6895: 31 db xor %ebx,%ebx
>> 6897: 48 8b 08 mov (%rax),%rcx
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:164
>> 689a: 83 c3 01 add $0x1,%ebx
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:163
>> 689d: 0f 18 09 prefetcht0 (%rcx)
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:164
>> 68a0: 48 8b 48 e0 mov -0x20(%rax),%rcx
>> 68a4: 48 89 0a mov %rcx,(%rdx)
>> 68a7: 48 8b 48 e8 mov -0x18(%rax),%rcx
>> 68ab: 48 89 4a 08 mov %rcx,0x8(%rdx)
>> 68af: 48 8b 48 f0 mov -0x10(%rax),%rcx
>> 68b3: 48 89 4a 10 mov %rcx,0x10(%rdx)
>> 68b7: 48 8b 48 f8 mov -0x8(%rax),%rcx
>> 68bb: 48 89 4a 18 mov %rcx,0x18(%rdx)
>> 68bf: 48 8b 08 mov (%rax),%rcx
>> 68c2: 48 89 4a 20 mov %rcx,0x20(%rdx)
>> 68c6: 48 8b 48 08 mov 0x8(%rax),%rcx
>> 68ca: 48 89 4a 28 mov %rcx,0x28(%rdx)
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:163
>> 68ce: 48 8b 00 mov (%rax),%rax
>> 68d1: 48 83 c2 30 add $0x30,%rdx
>> 68d5: 48 85 c0 test %rax,%rax
>> 68d8: 75 bd jne 6897<kvm_set_irq+0x57>
>> 68da: eb 06 jmp 68e2<kvm_set_irq+0xa2>
>> 68dc: 0f 1f 40 00 nopl 0x0(%rax)
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
>> 68e0: 31 db xor %ebx,%ebx
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:169
>> 68e2: 4c 8d b5 40 ff ff ff lea -0xc0(%rbp),%r14
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
>> 68e9: 41 bc ff ff ff ff mov $0xffffffff,%r12d
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:167
>> 68ef: 85 db test %ebx,%ebx
>> 68f1: 74 3d je 6930<kvm_set_irq+0xf0>
>> 68f3: 83 eb 01 sub $0x1,%ebx
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:169
>> 68f6: 44 89 f9 mov %r15d,%ecx
>> 68f9: 8b 95 3c ff ff ff mov -0xc4(%rbp),%edx
>> 68ff: 48 63 c3 movslq %ebx,%rax
>> 6902: 4c 89 ee mov %r13,%rsi
>> 6905: 48 8d 04 40 lea (%rax,%rax,2),%rax
>> 6909: 48 c1 e0 04 shl $0x4,%rax
>> 690d: 49 8d 3c 06 lea (%r14,%rax,1),%rdi
>> 6911: ff 94 05 48 ff ff ff callq *-0xb8(%rbp,%rax,1)
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:170
>> 6918: 85 c0 test %eax,%eax
>> 691a: 78 d3 js 68ef<kvm_set_irq+0xaf>
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:173
>> 691c: 45 85 e4 test %r12d,%r12d
>> 691f: ba 00 00 00 00 mov $0x0,%edx
>> 6924: 44 0f 48 e2 cmovs %edx,%r12d
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:167
>> 6928: 85 db test %ebx,%ebx
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:173
>> 692a: 46 8d 24 20 lea (%rax,%r12,1),%r12d
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:167
>> 692e: 75 c3 jne 68f3<kvm_set_irq+0xb3>
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:177
>> 6930: 48 81 c4 a8 00 00 00 add $0xa8,%rsp
>> 6937: 44 89 e0 mov %r12d,%eax
>> 693a: 5b pop %rbx
>> 693b: 41 5c pop %r12
>> 693d: 41 5d pop %r13
>> 693f: 41 5e pop %r14
>> 6941: 41 5f pop %r15
>> 6943: c9 leaveq
>> 6944: c3 retq
>> trace_kvm_set_irq():
>> /usr/src/GIT/linux-2.6-stable/include/trace/events/kvm.h:10
>> 6945: 4c 8b 25 00 00 00 00 mov 0x0(%rip),%r12
>> # 694c<kvm_set_irq+0x10c>
>> 694c: 4d 85 e4 test %r12,%r12
>> 694f: 0f 84 1b ff ff ff je 6870<kvm_set_irq+0x30>
>> 6955: 49 8b 04 24 mov (%r12),%rax
>> 6959: 49 8b 7c 24 08 mov 0x8(%r12),%rdi
>> 695e: 49 83 c4 10 add $0x10,%r12
>> 6962: 8b 8d 3c ff ff ff mov -0xc4(%rbp),%ecx
>> 6968: 44 89 fa mov %r15d,%edx
>> 696b: 89 de mov %ebx,%esi
>> 696d: ff d0 callq *%rax
>> 696f: 49 8b 04 24 mov (%r12),%rax
>> 6973: 48 85 c0 test %rax,%rax
>> 6976: 75 e1 jne 6959<kvm_set_irq+0x119>
>> 6978: e9 f3 fe ff ff jmpq 6870<kvm_set_irq+0x30>
>> kvm_set_irq():
>> 697d: 0f 1f 00 nopl (%rax)
>>
>> So, if i've read correctly, the offset is 0x6877 ?
>>
>> root@ayrshire:~# addr2line -e
>> /lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
>> 0x6877
>> /usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
>>
>>
>> Is it the correct way to analyse this?
>>
>> Regards.
>
> Yes. So we have:
>
> irq_rt = rcu_dereference(kvm->irq_routing);
>
>> ffffffffa03ee870: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
>> %eax => b6634c1 %r13 = ffff880419b600c0
>
> if (irq< irq_rt->nr_rt_entries)
>
>> *ffffffffa03ee877: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx |
>> %eax = b6634c1 %ebx = 18<--- faulting instruction
>
> The problem then is that while the kvm pointer is
> ffff880419b600c0 which looks sane,
> the value we read from kvm->irq_routing is b6634c1 which
> does not make sense. When we dereference that, kaboom.
>
> Is the kvm pointer wrong or the memory corrupted?
> Try printing the kvm pointer during
> initialization, e.g. in kvm_vm_ioctl_create_vcpu,
> then and compare to markup_oops.
>
>
Hi,
so this time the bug is:
[17882.612303] BUG: unable to handle kernel paging request at
0000000000002458
[17882.612342] IP: [<ffffffffa03898a0>] kvm_set_irq+0x30/0x140 [kvm]
markup_oops give me this:
root@ayrshire:~# cat bug-0103.txt | perl markup_oops.pl -m
/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
/boot/vmlinuz-2.6.37.2-dsiun-110105+
vmaoffset = 18446744072102621184 ffffffffa0389871: 48 89 e5
mov %rsp,%rbp
ffffffffa0389874: 41 57 push %r15
ffffffffa0389876: 41 89 cf mov %ecx,%r15d | %r15 =>
1 %ecx = 1
ffffffffa0389879: 41 56 push %r14 | %r14 =>
ffffffffa038aad0
ffffffffa038987b: 41 55 push %r13
ffffffffa038987d: 49 89 fd mov %rdi,%r13 | %edi = 0
%r13 => 0
ffffffffa0389880: 41 54 push %r12 | %r12 => 0
ffffffffa0389882: 53 push %rbx
ffffffffa0389883: 89 d3 mov %edx,%ebx | %ebx => 1a
ffffffffa0389885: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
ffffffffa038988c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx #
ffffffffa0389892 <kvm_set_irq+0x22>
ffffffffa0389892: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) |
%esi = 0
ffffffffa0389898: 85 d2 test %edx,%edx | %edx => 0
ffffffffa038989a: 0f 85 d5 00 00 00 jne ffffffffa0389975
<kvm_set_irq+0x105>
*ffffffffa03898a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
%eax = 0 %r13 = 0 <--- faulting instruction
ffffffffa03898a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
ffffffffa03898ad: 73 61 jae ffffffffa0389910
<kvm_set_irq+0xa0>
ffffffffa03898af: 89 db mov %ebx,%ebx
ffffffffa03898b1: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
ffffffffa03898b8: 00
ffffffffa03898b9: 48 85 c0 test %rax,%rax
ffffffffa03898bc: 74 52 je ffffffffa0389910
<kvm_set_irq+0xa0>
ffffffffa03898be: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
ffffffffa03898c5: 31 db xor %ebx,%ebx
ffffffffa03898c7: 48 8b 08 mov (%rax),%rcx
ffffffffa03898ca: 83 c3 01 add $0x1,%ebx
ffffffffa03898cd: 0f 18 09 prefetcht0 (%rcx)
ffffffffa03898d0: 48 8b 48 e0 mov -0x20(%rax),%rcx
ffffffffa03898d4: 48 89 0a mov %rcx,(%rdx)
ffffffffa03898d7: 48 8b 48 e8 mov -0x18(%rax),%rcx
ffffffffa03898db: 48 89 4a 08 mov %rcx,0x8(%rdx)
ffffffffa03898df: 48 8b 48 f0 mov -0x10(%rax),%rcx
ffffffffa03898e3: 48 89 4a 10 mov %rcx,0x10(%rdx)
ffffffffa03898e7: 48 8b 48 f8 mov -0x8(%rax),%rcx
ffffffffa03898eb: 48 89 4a 18 mov %rcx,0x18(%rdx)
wich correspond to offset 68a0 (from objdump):
kvm_set_irq():
/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
68a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax
/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
68a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
root@ayrshire:~# addr2line -e
/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko 0x68a0
/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
So here kvm->irq_routing is null.
How can it be?
Regards.
[-- Attachment #2: jean-philippe_menil.vcf --]
[-- Type: text/x-vcard, Size: 361 bytes --]
begin:vcard
fn:Jean-Philippe Menil
n:Menil;Jean-Philippe
org;quoted-printable:Universit=C3=A9 de Nantes;IRTS - DSI
adr;quoted-printable:;;2 rue de la Houssini=C3=A8re;Nantes;Loire-Atlantique;44382;France
email;internet:jean-philippe.menil@univ-nantes.fr
title;quoted-printable:Administrateur R=C3=A9seau
url:http://www.cri.univ-nantes.fr
version:2.1
end:vcard
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 14:30 UTC (permalink / raw)
To: Eric Dumazet, Herbert Xu, David Miller, rick.jones2, therbert,
wsommerfeld
In-Reply-To: <20110301142235.GA10761@canuck.infradead.org>
On Tue, Mar 01, 2011 at 09:22:35AM -0500, Thomas Graf wrote:
> On Tue, Mar 01, 2011 at 03:06:59PM +0100, Eric Dumazet wrote:
> > Would be nice to cpu affine named to _not_ run on CPU11, just to
> > specialize it for TX completions and have softirq time percentage and
> > "perf top -C 11 " results
CPU 1 isolated as well (named running with mask 0,2-10)
----------------------------------------------------------------------------------------------------------------------
PerfTop: 580 irqs/sec kernel:100.0% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
----------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ___________________________ ___________________________________________________________
283.00 9.2% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
256.00 8.4% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
190.00 6.2% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
182.00 5.9% get_page_from_freelist /lib/modules/2.6.38-rc5+/build/vmlinux
157.00 5.1% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
143.00 4.7% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
133.00 4.3% sock_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
133.00 4.3% __udp4_lib_lookup /lib/modules/2.6.38-rc5+/build/vmlinux
131.00 4.3% sk_run_filter /lib/modules/2.6.38-rc5+/build/vmlinux
114.00 3.7% getnstimeofday /lib/modules/2.6.38-rc5+/build/vmlinux
112.00 3.7% __alloc_skb /lib/modules/2.6.38-rc5+/build/vmlinux
103.00 3.4% read_tsc /lib/modules/2.6.38-rc5+/build/vmlinux
100.00 3.3% __netif_receive_skb /lib/modules/2.6.38-rc5+/build/vmlinux
95.00 3.1% udp_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
82.00 2.7% sock_def_readable /lib/modules/2.6.38-rc5+/build/vmlinux
79.00 2.6% kmem_cache_alloc_node_trace /lib/modules/2.6.38-rc5+/build/vmlinux
72.00 2.3% _raw_spin_lock /lib/modules/2.6.38-rc5+/build/vmlinux
67.00 2.2% __phys_addr /lib/modules/2.6.38-rc5+/build/vmlinux
63.00 2.1% is_swiotlb_buffer /lib/modules/2.6.38-rc5+/build/vmlinux
51.00 1.7% __udp4_lib_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
48.00 1.6% memcpy /lib/modules/2.6.38-rc5+/build/vmlinux
47.00 1.5% ip_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
46.00 1.5% kmem_cache_alloc_node /lib/modules/2.6.38-rc5+/build/vmlinux
44.00 1.4% dma_issue_pending_all /lib/modules/2.6.38-rc5+/build/vmlinux
40.00 1.3% ip_route_input_common /lib/modules/2.6.38-rc5+/build/vmlinux
36.00 1.2% __alloc_pages_nodemask /lib/modules/2.6.38-rc5+/build/vmlinux
33.00 1.1% be_post_rx_frags /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
24.00 0.8% alloc_pages_current /lib/modules/2.6.38-rc5+/build/vmlinux
21.00 0.7% packet_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
20.00 0.7% local_bh_enable /lib/modules/2.6.38-rc5+/build/vmlinux
17.00 0.6% consume_skb /lib/modules/2.6.38-rc5+/build/vmlinux
16.00 0.5% next_zones_zonelist /lib/modules/2.6.38-rc5+/build/vmlinux
14.00 0.5% selinux_socket_sock_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
13.00 0.4% ip_local_deliver /lib/modules/2.6.38-rc5+/build/vmlinux
11.00 0.4% sk_filter /lib/modules/2.6.38-rc5+/build/vmlinux
10.00 0.3% get_rps_cpu /lib/modules/2.6.38-rc5+/build/vmlinux
9.00 0.3% native_read_tsc /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.3% local_bh_disable /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.3% eth_type_trans /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.3% napi_complete /lib/modules/2.6.38-rc5+/build/vmlinux
7.00 0.2% netif_receive_skb /lib/modules/2.6.38-rc5+/build/vmlinux
7.00 0.2% dso__find_symbol /usr/bin/perf
7.00 0.2% __kmalloc_node_track_caller /lib/modules/2.6.38-rc5+/build/vmlinux
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 14:22 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298988419.3284.107.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 03:06:59PM +0100, Eric Dumazet wrote:
> Would be nice to cpu affine named to _not_ run on CPU11, just to
> specialize it for TX completions and have softirq time percentage and
> "perf top -C 11 " results
----------------------------------------------------------------------------------------------------------------------
PerfTop: 995 irqs/sec kernel:97.7% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 11)
----------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ___________________________ ___________________________________________________________
335.00 23.3% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
253.00 17.6% be_tx_compl_process /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
177.00 12.3% skb_release_data /lib/modules/2.6.38-rc5+/build/vmlinux
132.00 9.2% kfree /lib/modules/2.6.38-rc5+/build/vmlinux
127.00 8.8% kfree_skb /lib/modules/2.6.38-rc5+/build/vmlinux
105.00 7.3% be_poll_tx_mcc /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
99.00 6.9% kmem_cache_free /lib/modules/2.6.38-rc5+/build/vmlinux
36.00 2.5% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
20.00 1.4% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
19.00 1.3% skb_release_head_state /lib/modules/2.6.38-rc5+/build/vmlinux
13.00 0.9% unmap_tx_frag /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
11.00 0.8% rb_next /usr/bin/perf
10.00 0.7% dso__find_symbol /usr/bin/perf
9.00 0.6% is_swiotlb_buffer /lib/modules/2.6.38-rc5+/build/vmlinux
9.00 0.6% __strcmp_sse42 /lib64/libc-2.12.so
8.00 0.6% __kfree_skb /lib/modules/2.6.38-rc5+/build/vmlinux
8.00 0.6% __strstr_sse42 /lib64/libc-2.12.so
6.00 0.4% _int_malloc /lib64/libc-2.12.so
^ permalink raw reply
* Re: [PATCH]drivers:isdn:istream.c Fix typo pice to piece
From: Jiri Kosina @ 2011-03-01 14:21 UTC (permalink / raw)
To: Justin P. Mattock; +Cc: mac, isdn, netdev, linux-kernel
In-Reply-To: <1298269881-2754-1-git-send-email-justinmattock@gmail.com>
On Sun, 20 Feb 2011, Justin P. Mattock wrote:
> The below patch changes a typo "pice" to "piece"
>
> Signed-off-by: Justin P. Mattock <justinmattock@gmail.com>
>
> ---
> drivers/isdn/hardware/eicon/istream.c | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/isdn/hardware/eicon/istream.c b/drivers/isdn/hardware/eicon/istream.c
> index 18f8798..7bd5baa 100644
> --- a/drivers/isdn/hardware/eicon/istream.c
> +++ b/drivers/isdn/hardware/eicon/istream.c
> @@ -62,7 +62,7 @@ void diva_xdi_provide_istream_info (ADAPTER* a,
> stream interface.
> If synchronous service was requested, then function
> does return amount of data written to stream.
> - 'final' does indicate that pice of data to be written is
> + 'final' does indicate that piece of data to be written is
> final part of frame (necessary only by structured datatransfer)
> return 0 if zero lengh packet was written
> return -1 if stream is full
Applied.
--
Jiri Kosina
SUSE Labs, Novell Inc.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 14:06 UTC (permalink / raw)
To: Thomas Graf
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301135053.GA6684@canuck.infradead.org>
Le mardi 01 mars 2011 à 08:50 -0500, Thomas Graf a écrit :
> On Tue, Mar 01, 2011 at 08:19:51PM +0800, Herbert Xu wrote:
> > On Tue, Mar 01, 2011 at 07:18:29AM -0500, Thomas Graf wrote:
> > >
> > >
> > > ... makes it use CPU 5 for rxq2 and the qps goes up from 250kqps to 270kqps
> >
> > I think the increase here comes from the larger number of packets
> > in flight more than anything.
> >
> > The bottleneck is still the TX queue (both software and hardware).
>
> Disabled netfilter and reran test
>
> Now does ~316kqps (rx was split over 2 queues)
Would be nice to cpu affine named to _not_ run on CPU11, just to
specialize it for TX completions and have softirq time percentage and
"perf top -C 11 " results
Thanks
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 13:58 UTC (permalink / raw)
To: Eric Dumazet
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298987566.3284.105.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 02:52:46PM +0100, Eric Dumazet wrote:
> Le mardi 01 mars 2011 à 21:18 +0800, Herbert Xu a écrit :
>
> > Interesting. So I wonder which lock is showing up at the top
> > of the profile with a single socket then. As it's definitely
> > going away with multiple sockets, that means it's not the TX
> > queue lock.
> >
>
> This CPU also runs named process, so this is socket lock and receive
> queue lock.
It can't be the socket lock because it's an IRQ-disabling variant.
The receive queue lock, I'll buy that.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 13:52 UTC (permalink / raw)
To: Herbert Xu
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301131823.GB8028@gondor.apana.org.au>
Le mardi 01 mars 2011 à 21:18 +0800, Herbert Xu a écrit :
> Interesting. So I wonder which lock is showing up at the top
> of the profile with a single socket then. As it's definitely
> going away with multiple sockets, that means it's not the TX
> queue lock.
>
This CPU also runs named process, so this is socket lock and receive
queue lock.
Named threads all do : recvmsg()/sendmsg() in a loop, so all are waiting
a frame before doing some work.
Because of single receive queue, extra context switches occur (all
threads but one have to sleep again per query)
For about 80 kqps (standard linux-2.6 kernel, no patches), I have
following vmstat output
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
4 1 0 2184048 63496 1595056 0 0 0 2060 64592 294528 19 11 67 4
6 1 0 2184040 63496 1595056 0 0 0 1960 64686 293928 19 11 66 4
3 1 0 2184040 63496 1595056 0 0 0 2344 64556 294268 20 11 66 4
4 1 0 2184040 63496 1595056 0 0 0 2400 64626 293859 19 11 67 4
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 13:50 UTC (permalink / raw)
To: Herbert Xu
Cc: Eric Dumazet, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301121951.GA7267@gondor.apana.org.au>
On Tue, Mar 01, 2011 at 08:19:51PM +0800, Herbert Xu wrote:
> On Tue, Mar 01, 2011 at 07:18:29AM -0500, Thomas Graf wrote:
> >
> >
> > ... makes it use CPU 5 for rxq2 and the qps goes up from 250kqps to 270kqps
>
> I think the increase here comes from the larger number of packets
> in flight more than anything.
>
> The bottleneck is still the TX queue (both software and hardware).
Disabled netfilter and reran test
Now does ~316kqps (rx was split over 2 queues)
----------------------------------------------------------------------------------------------------------------------
PerfTop: 30608 irqs/sec kernel:66.1% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
----------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ ___________________________________________________________
19237.00 5.6% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
17170.00 5.0% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
11411.00 3.3% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
11320.00 3.3% isc_rwlock_lock /usr/lib64/libisc.so.62.0.1
10669.00 3.1% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
10655.00 3.1% get_page_from_freelist /lib/modules/2.6.38-rc5+/build/vmlinux
9523.00 2.8% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
8677.00 2.5% __udp4_lib_lookup /lib/modules/2.6.38-rc5+/build/vmlinux
8379.00 2.4% sock_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
8226.00 2.4% sk_run_filter /lib/modules/2.6.38-rc5+/build/vmlinux
6724.00 1.9% __netif_receive_skb /lib/modules/2.6.38-rc5+/build/vmlinux
6553.00 1.9% __alloc_skb /lib/modules/2.6.38-rc5+/build/vmlinux
6205.00 1.8% udp_queue_rcv_skb /lib/modules/2.6.38-rc5+/build/vmlinux
6038.00 1.7% _raw_spin_lock /lib/modules/2.6.38-rc5+/build/vmlinux
5868.00 1.7% isc_rwlock_unlock /usr/lib64/libisc.so.62.0.1
5696.00 1.6% dns_rbt_findnode /usr/lib64/libdns.so.69.0.1
5647.00 1.6% read_tsc /lib/modules/2.6.38-rc5+/build/vmlinux
5633.00 1.6% getnstimeofday /lib/modules/2.6.38-rc5+/build/vmlinux
5448.00 1.6% kmem_cache_alloc_node_trace /lib/modules/2.6.38-rc5+/build/vmlinux
5272.00 1.5% finish_task_switch /lib/modules/2.6.38-rc5+/build/vmlinux
4719.00 1.4% sock_def_readable /lib/modules/2.6.38-rc5+/build/vmlinux
4002.00 1.2% is_swiotlb_buffer /lib/modules/2.6.38-rc5+/build/vmlinux
3914.00 1.1% memcpy /lib/modules/2.6.38-rc5+/build/vmlinux
3717.00 1.1% isc_stats_increment /usr/lib64/libisc.so.62.0.1
3706.00 1.1% __udp4_lib_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
3653.00 1.1% ip_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
3598.00 1.0% kmem_cache_alloc_node /lib/modules/2.6.38-rc5+/build/vmlinux
3407.00 1.0% ip_route_input_common /lib/modules/2.6.38-rc5+/build/vmlinux
2683.00 0.8% be_post_rx_frags /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
2666.00 0.8% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so
2331.00 0.7% __phys_addr /lib/modules/2.6.38-rc5+/build/vmlinux
2230.00 0.6% __alloc_pages_nodemask /lib/modules/2.6.38-rc5+/build/vmlinux
2023.00 0.6% dns_name_fullcompare /usr/lib64/libdns.so.69.0.1
1972.00 0.6% packet_rcv /lib/modules/2.6.38-rc5+/build/vmlinux
1902.00 0.6% eth_type_trans /lib/modules/2.6.38-rc5+/build/vmlinux
1860.00 0.5% __pthread_mutex_unlock /lib64/libpthread-2.12.so
1804.00 0.5% fget_light /lib/modules/2.6.38-rc5+/build/vmlinux
1739.00 0.5% alloc_pages_current /lib/modules/2.6.38-rc5+/build/vmlinux
1736.00 0.5% dns_rbtnodechain_init /usr/lib64/libdns.so.69.0.1
----------------------------------------------------------------------------------------------------------------------
PerfTop: 29038 irqs/sec kernel:48.0% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 11)
----------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ ___________________________________________________________
12833.00 7.5% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
10771.00 6.3% isc_rwlock_lock /usr/lib64/libisc.so.62.0.1
8713.00 5.1% be_tx_compl_process /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
6452.00 3.8% kfree /lib/modules/2.6.38-rc5+/build/vmlinux
5935.00 3.5% skb_release_data /lib/modules/2.6.38-rc5+/build/vmlinux
5552.00 3.2% kmem_cache_free /lib/modules/2.6.38-rc5+/build/vmlinux
5292.00 3.1% isc_rwlock_unlock /usr/lib64/libisc.so.62.0.1
4893.00 2.9% dns_rbt_findnode /usr/lib64/libdns.so.69.0.1
4413.00 2.6% kfree_skb /lib/modules/2.6.38-rc5+/build/vmlinux
3802.00 2.2% be_poll_tx_mcc /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
3515.00 2.1% isc_stats_increment /usr/lib64/libisc.so.62.0.1
3016.00 1.8% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
2202.00 1.3% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
2027.00 1.2% _raw_spin_lock /lib/modules/2.6.38-rc5+/build/vmlinux
1935.00 1.1% finish_task_switch /lib/modules/2.6.38-rc5+/build/vmlinux
1906.00 1.1% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so
1837.00 1.1% dns_name_fullcompare /usr/lib64/libdns.so.69.0.1
1702.00 1.0% dns_rbtnodechain_init /usr/lib64/libdns.so.69.0.1
1561.00 0.9% fget_light /lib/modules/2.6.38-rc5+/build/vmlinux
1559.00 0.9% dns_name_getlabelsequence /usr/lib64/libdns.so.69.0.1
1491.00 0.9% dns_name_equal /usr/lib64/libdns.so.69.0.1
1464.00 0.9% __pthread_mutex_unlock /lib64/libpthread-2.12.so
1454.00 0.9% dns_acl_match /usr/lib64/libdns.so.69.0.1
1293.00 0.8% dns_zone_attach /usr/lib64/libdns.so.69.0.1
1245.00 0.7% be_xmit /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
1159.00 0.7% dns_message_rendersection /usr/lib64/libdns.so.69.0.1
1115.00 0.7% isc___mempool_get /usr/lib64/libisc.so.62.0.1
1100.00 0.6% copy_user_generic_string /lib/modules/2.6.38-rc5+/build/vmlinux
1030.00 0.6% dns_name_fromwire /usr/lib64/libdns.so.69.0.1
1015.00 0.6% dns_name_hash /usr/lib64/libdns.so.69.0.1
1013.00 0.6% isc_radix_search /usr/lib64/libisc.so.62.0.1
970.00 0.6% __ip_route_output_key /lib/modules/2.6.38-rc5+/build/vmlinux
917.00 0.5% fput /lib/modules/2.6.38-rc5+/build/vmlinux
817.00 0.5% dev_queue_xmit /lib/modules/2.6.38-rc5+/build/vmlinux
812.00 0.5% sk_run_filter /lib/modules/2.6.38-rc5+/build/vmlinux
806.00 0.5% avc_has_perm_noaudit /lib/modules/2.6.38-rc5+/build/vmlinux
802.00 0.5% sock_wfree /lib/modules/2.6.38-rc5+/build/vmlinux
793.00 0.5% dns_name_towire /usr/lib64/libdns.so.69.0.1
754.00 0.4% sock_alloc_send_pskb /lib/modules/2.6.38-rc5+/build/vmlinux
752.00 0.4% dns_message_parse /usr/lib64/libdns.so.69.0.1
749.00 0.4% dns_rdata_towire /usr/lib64/libdns.so.69.0.1
728.00 0.4% dns_rdataset_init /usr/lib64/libdns.so.69.0.1
709.00 0.4% isc___mempool_put /usr/lib64/libisc.so.62.0.1
699.00 0.4% skb_release_head_state /lib/modules/2.6.38-rc5+/build/vmlinux
685.00 0.4% _raw_spin_lock_bh /lib/modules/2.6.38-rc5+/build/vmlinux
683.00 0.4% dns_name_concatenate /usr/lib64/libdns.so.69.0.1
678.00 0.4% __ip_append_data /lib/modules/2.6.38-rc5+/build/vmlinux
673.00 0.4% tick_nohz_stop_sched_tick /lib/modules/2.6.38-rc5+/build/vmlinux
662.00 0.4% sys_sendmsg /lib/modules/2.6.38-rc5+/build/vmlinux
654.00 0.4% dns_compress_findglobal /usr/lib64/libdns.so.69.0.1
654.00 0.4% memcpy /lib64/libc-2.12.so
637.00 0.4% dns_compress_invalidate /usr/lib64/libdns.so.69.0.1
597.00 0.3% isc__buffer_init /usr/lib64/libisc.so.62.0.1
595.00 0.3% dns_zone_detach /usr/lib64/libdns.so.69.0.1
WARNING: failed to keep up with mmap data.
WARNING: failed to keep up with mmap data.
----------------------------------------------------------------------------------------------------------------------
PerfTop: 29539 irqs/sec kernel:47.0% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 11)
----------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ ___________________________________________________________
14478.00 7.5% intel_idle /lib/modules/2.6.38-rc5+/build/vmlinux
12279.00 6.3% isc_rwlock_lock /usr/lib64/libisc.so.62.0.1
9844.00 5.1% be_tx_compl_process /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
7368.00 3.8% kfree /lib/modules/2.6.38-rc5+/build/vmlinux
6696.00 3.5% skb_release_data /lib/modules/2.6.38-rc5+/build/vmlinux
6240.00 3.2% kmem_cache_free /lib/modules/2.6.38-rc5+/build/vmlinux
6034.00 3.1% isc_rwlock_unlock /usr/lib64/libisc.so.62.0.1
5547.00 2.9% dns_rbt_findnode /usr/lib64/libdns.so.69.0.1
5012.00 2.6% kfree_skb /lib/modules/2.6.38-rc5+/build/vmlinux
4290.00 2.2% be_poll_tx_mcc /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
4024.00 2.1% isc_stats_increment /usr/lib64/libisc.so.62.0.1
3417.00 1.8% _raw_spin_unlock_irqrestore /lib/modules/2.6.38-rc5+/build/vmlinux
2470.00 1.3% __do_softirq /lib/modules/2.6.38-rc5+/build/vmlinux
2312.00 1.2% _raw_spin_lock /lib/modules/2.6.38-rc5+/build/vmlinux
2138.00 1.1% finish_task_switch /lib/modules/2.6.38-rc5+/build/vmlinux
2136.00 1.1% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so
2061.00 1.1% dns_name_fullcompare /usr/lib64/libdns.so.69.0.1
1961.00 1.0% dns_rbtnodechain_init /usr/lib64/libdns.so.69.0.1
1797.00 0.9% dns_name_getlabelsequence /usr/lib64/libdns.so.69.0.1
1743.00 0.9% fget_light /lib/modules/2.6.38-rc5+/build/vmlinux
1723.00 0.9% dns_name_equal /usr/lib64/libdns.so.69.0.1
1673.00 0.9% __pthread_mutex_unlock /lib64/libpthread-2.12.so
1671.00 0.9% dns_acl_match /usr/lib64/libdns.so.69.0.1
1488.00 0.8% dns_zone_attach /usr/lib64/libdns.so.69.0.1
1428.00 0.7% be_xmit /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
1369.00 0.7% dns_message_rendersection /usr/lib64/libdns.so.69.0.1
1278.00 0.7% isc___mempool_get /usr/lib64/libisc.so.62.0.1
1251.00 0.6% copy_user_generic_string /lib/modules/2.6.38-rc5+/build/vmlinux
1193.00 0.6% dns_name_fromwire /usr/lib64/libdns.so.69.0.1
1182.00 0.6% isc_radix_search /usr/lib64/libisc.so.62.0.1
^ permalink raw reply
* Re: [GIT/PATCH v3] xen network backend driver
From: Ian Campbell @ 2011-03-01 13:38 UTC (permalink / raw)
To: Ben Hutchings
Cc: netdev@vger.kernel.org, xen-devel, Jeremy Fitzhardinge,
Herbert Xu, Konrad Rzeszutek Wilk, Francois Romieu
In-Reply-To: <1298982526.3437.5.camel@localhost>
On Tue, 2011-03-01 at 12:28 +0000, Ben Hutchings wrote:
> On Tue, 2011-03-01 at 10:06 +0000, Ian Campbell wrote:
> > On Mon, 2011-02-28 at 18:53 +0000, Ben Hutchings wrote:
> > > On Mon, 2011-02-28 at 17:27 +0000, Ian Campbell wrote:
> >
> > > This should be defined as unsigned long (ideally it would be u64, but
> > > that can't be updated atomically on 32-bit systems).
> > [...]
> > > Don't update last_rx; it's only needed on slave devices of a bond, and
> > > the bonding driver takes care of it now.
> >
> > I made these two changes.
> >
> > > [...]
> > > > +static int xenvif_change_mtu(struct net_device *dev, int mtu)
> > > > +{
> > > > + struct xenvif *vif = netdev_priv(dev);
> > > > + int max = vif->can_sg ? 65535 - ETH_HLEN : ETH_DATA_LEN;
> > > > + if (mtu > max)
> > > > + return -EINVAL;
> > > > + dev->mtu = mtu;
> > > > + return 0;
> > > > +}
> > > [...]
> > >
> > > Since any VLAN tag must be inserted inline, shouldn't the MTU limit be
> > > 65535 - VLAN_ETH_HLEN?
> >
> > In that case shouldn't the other case also be ETH_FRAME_LEN -
> > VLAN_ETH_HLEN?
> [...]
>
> IEEE 802.3, in its infinite wisdom, says that the maximum frame length
> is 1514, except that when an 802.1q tag is present it is 1518. So the
> MTU for standard Ethernet remains 1500, regardless of the use of VLANs.
Super ;-)
I've made that change too and repushed the branch with all 3 of your
suggestions. The incremental patch is below.
The following changes since commit 2e820f58f7ad8eaca2f194ccdfea0de63e9c6d78:
Ian Campbell (1):
xen/irq: implement bind_interdomain_evtchn_to_irqhandler for backend drivers
are available in the git repository at:
git://xenbits.xen.org/people/ianc/linux-2.6.git upstream/dom0/backend/netback
Bastian Blank (1):
xen: netback: Fix null-pointer access in netback_uevent
Christophe Saout (1):
xen: netback: use dev_name() instead of removed ->bus_id.
Dongxiao Xu (5):
xen: netback: Move global/static variables into struct xen_netbk.
xen: netback: Introduce a new struct type page_ext.
xen: netback: Multiple tasklets support.
xen: netback: Use Kernel thread to replace the tasklet.
xen: netback: Set allocated memory to zero from vmalloc.
Ian Campbell (61):
xen: netback: Initial import of linux-2.6.18-xen.hg netback driver.
xen: netback: first cut at porting to upstream and cleaning up
xen: netback: add ethtool stat to track copied skbs.
xen: netback: make queue length parameter writeable in sysfs
xen: netback: parent sysfs device should be set before registering.
xen: rename netbk module xen-netback.
xen: netback: remove unused xen_network_done code
xen: netback: factor disconnect from backend into new function.
xen: netback: wait for hotplug scripts to complete before signalling connected to frontend
xen: netback: Always pull through PKT_PROT_LEN bytes into the linear part of an skb.
xen: netback: Allow setting of large MTU before rings have connected.
xen: netback: correctly setup skb->ip_summed on receive
xen: netback: handle NET_SKBUFF_DATA_USES_OFFSET correctly
xen: netback: drop frag member from struct netbk_rx_meta
xen: netback: linearise SKBs as we copy them into guest memory on guest-RX.
xen: netback: drop more relics of flipping mode
xen: netback: check if foreign pages are actually netback-created foreign pages.
xen: netback: do not unleash netback threads until initialisation is complete
xen: netback: save interrupt state in add_to_net_schedule_list_tail
xen: netback: increase size of rx_meta array.
xen: netback: take net_schedule_list_lock when removing entry from net_schedule_list
xen: netback: Drop GSO SKBs which do not have csum_blank.
xen: netback: completely remove tx_queue_timer
Revert "xen: netback: Drop GSO SKBs which do not have csum_blank."
xen: netback: handle incoming GSO SKBs which are not CHECKSUM_PARTIAL
xen: netback: rationalise types used in count_skb_slots
xen: netback: refactor logic for moving to a new receive buffer.
xen: netback: refactor code to get next rx buffer into own function.
xen: netback: simplify use of netbk_add_frag_responses
xen: netback: cleanup coding style
xen: netback: drop private ?PRINTK macros in favour of pr_*
xen: netback: move under drivers/net/xen-netback/
xen: netback: remove queue_length module option
xen: netback: correct error return from ethtool hooks.
xen: netback: avoid leading _ in function parameter names.
xen: netback: drop unused debug interrupt handler.
xen: netif: properly namespace the Xen netif protocol header.
xen: netif: improve Kconfig help text for front- and backend drivers.
xen: netback: drop ethtool drvinfo callback
xen: netback: use xen_netbk prefix where appropriate
xen: netback: refactor to make all xen_netbk knowledge internal to netback.c
xen: netback: use xenvif_ prefix where appropriate
xen: netback: add reference from xenvif to xen_netbk
xen: netback: refactor to separate network device from worker pools
xen: netback: switch to kthread mode and drop tasklet mode
xen: netback: handle frames whose head crosses a page boundary
xen: netback: return correct values from start_xmit
xen: netback: remove useless memset to zero.
xen: netback: use register_netdev()
xen: netback: simplify unwinding netback_init's work on failure.
xen: netback: use core network carrier flag.
xen: netback: s/xenvif_queue_full/xenvif_rx_queue_full/
xen: netback: add xenvif_rx_schedulable
xen: netback: further separate xen_netbk and xenvif
xen: netback: use netdev_LEVEL instead of pr_LEVEL
xen: netback: drop rx_notify and notify_list array in favour of a normal list
xen: netback: Make dependency on PageForeign conditional
xen: netback: completely drop foreign page support
xen: netback: ethtool stats fields should be unsigned long
xen: netback: do not update last_rx on receive.
xen: netback: Allow headroom for VLAN header in SG MTU calculation.
James Harper (1):
xen: netback: avoid null-pointer access in netback_uevent
Jan Beulich (1):
xen: netback: unmap tx ring gref when mapping of rx ring gref failed
Jeremy Fitzhardinge (21):
xen: netback: don't include xen/evtchn.h
xen: netback: use mod_timer
xen: netback: use NET_SKB_PAD rather than "16"
xen: netback: completely drop flip support
xen: netback: demacro MASK_PEND_IDX
xen: netback: convert PEND_RING_IDX into a proper typedef name
xen: netback: rename NR_PENDING_REQS to nr_pending_reqs()
xen: netback: pre-initialize list and spinlocks; use empty list to indicate not on list
xen: netback: remove CONFIG_XEN_NETDEV_PIPELINED_TRANSMITTER
xen: netback: make netif_get/put inlines
xen: netback: move code around
xen: netback: document PKT_PROT_LEN
xen: netback: convert to net_device_ops
xen: netback: reinstate missing code
xen: netback: remove debug noise
xen: netback: don't screw around with packet gso state
xen: netback: use dev_get/set_drvdata() inteface
xen: netback: include linux/sched.h for TASK_* definitions
xen: netback: use get_sset_count rather than obsolete get_stats_count
xen: netback: minor code formatting fixup
xen: netback: only initialize for PV domains
Keir Fraser (1):
xen: netback: Fixes for delayed copy of tx network packets.
Konrad Rzeszutek Wilk (1):
Fix compile warnings: ignoring return value of 'xenbus_register_backend' ..
Paul Durrant (8):
xen: netback: Fix basic indentation issue
xen: netback: Add a new style of passing GSO packets to frontends.
xen: netback: Make frontend features distinct from netback feature flags.
xen: netback: Re-define PKT_PROT_LEN to be bigger.
xen: netback: Don't count packets we don't actually receive.
xen: netback: Remove the 500ms timeout to restart the netif queue.
xen: netback: Add a missing test to tx_work_todo.
xen: netback: Re-factor net_tx_action_dealloc() slightly.
Steven Smith (2):
xen: netback: make sure that pg->mapping is never NULL for a page mapped from a foreign domain.
xen: netback: try to pull a minimum of 72 bytes into the skb data area
drivers/net/Kconfig | 38 +-
drivers/net/Makefile | 1 +
drivers/net/xen-netback/Makefile | 3 +
drivers/net/xen-netback/common.h | 162 ++++
drivers/net/xen-netback/interface.c | 424 +++++++++
drivers/net/xen-netback/netback.c | 1745 +++++++++++++++++++++++++++++++++++
drivers/net/xen-netback/xenbus.c | 490 ++++++++++
drivers/net/xen-netfront.c | 20 +-
include/xen/interface/io/netif.h | 80 +-
9 files changed, 2909 insertions(+), 54 deletions(-)
create mode 100644 drivers/net/xen-netback/Makefile
create mode 100644 drivers/net/xen-netback/common.h
create mode 100644 drivers/net/xen-netback/interface.c
create mode 100644 drivers/net/xen-netback/netback.c
create mode 100644 drivers/net/xen-netback/xenbus.c
Ian.
diff --git a/drivers/net/xen-netback/common.h b/drivers/net/xen-netback/common.h
index 21f4c0c..406fbef 100644
--- a/drivers/net/xen-netback/common.h
+++ b/drivers/net/xen-netback/common.h
@@ -99,7 +99,7 @@ struct xenvif {
struct timer_list credit_timeout;
/* Statistics */
- int rx_gso_checksum_fixup;
+ unsigned long rx_gso_checksum_fixup;
/* Miscellaneous private stuff. */
struct list_head schedule_list;
diff --git a/drivers/net/xen-netback/interface.c b/drivers/net/xen-netback/interface.c
index 1614ba5..887e2ce 100644
--- a/drivers/net/xen-netback/interface.c
+++ b/drivers/net/xen-netback/interface.c
@@ -32,6 +32,7 @@
#include <linux/ethtool.h>
#include <linux/rtnetlink.h>
+#include <linux/if_vlan.h>
#include <xen/events.h>
#include <asm/xen/hypercall.h>
@@ -107,7 +108,6 @@ static int xenvif_start_xmit(struct sk_buff *skb, struct net_device *dev)
void xenvif_receive_skb(struct xenvif *vif, struct sk_buff *skb)
{
netif_rx_ni(skb);
- vif->dev->last_rx = jiffies;
}
void xenvif_notify_tx_completion(struct xenvif *vif)
@@ -157,7 +157,7 @@ static int xenvif_close(struct net_device *dev)
static int xenvif_change_mtu(struct net_device *dev, int mtu)
{
struct xenvif *vif = netdev_priv(dev);
- int max = vif->can_sg ? 65535 - ETH_HLEN : ETH_DATA_LEN;
+ int max = vif->can_sg ? 65535 - VLAN_ETH_HLEN : ETH_DATA_LEN;
if (mtu > max)
return -EINVAL;
^ permalink raw reply related
* Re: [Lxc-users] Bad checksums and lost packets with macvlan on dummy
From: Daniel Lezcano @ 2011-03-01 13:29 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Andrian Nord, lxc-users, Patrick McHardy, Linux Netdev List
In-Reply-To: <1298879114.8726.131.camel@edumazet-laptop>
On 02/28/2011 08:45 AM, Eric Dumazet wrote:
> Le dimanche 27 février 2011 à 21:35 +0100, Daniel Lezcano a écrit :
>> On 02/27/2011 08:50 PM, Eric Dumazet wrote:
>>> Le dimanche 27 février 2011 à 16:14 +0100, Daniel Lezcano a écrit :
>>>> On 02/23/2011 06:13 PM, Andrian Nord wrote:
>>>>> On Mon, Feb 21, 2011 at 05:07:31PM +0100, Daniel Lezcano wrote:
>>>>>> I Cc'ed the netdev mailing list and Patrick in case my analysis is wrong
>>>>>> or incomplete.
>>>>> I'm confirming, that this happens only when macvlan's are onto dummy net
>>>>> device. In case of some physical interface under macvlan there is no lost
>>>>> packages and no broken checksums.
>>>> I did some tests with a 2.6.35 kernel version and it seems the checksum
>>>> errors do not appear.
>>>> I noticed there are some changes in the dummy setup function:
>>>>
>>>> dev->features |= NETIF_F_SG | NETIF_F_FRAGLIST | NETIF_F_TSO;
>>>> dev->features |= NETIF_F_NO_CSUM | NETIF_F_HIGHDMA | NETIF_F_LLTX;
>>>>
>>>>
>>>> May be that was introduced by commit:
>>>>
>>>> commit 6d81f41c58c69ddde497e9e640ba5805aa26e78c
>>>> Author: Eric Dumazet<eric.dumazet@gmail.com>
>>>> Date: Mon Sep 27 20:50:33 2010 +0000
>>>>
>>>> dummy: percpu stats and lockless xmit
>>>>
>>>> Converts dummy network device driver to :
>>>>
>>>> - percpu stats
>>>>
>>>> - 64bit stats
>>>>
>>>> - lockless xmit (NETIF_F_LLTX)
>>>>
>>>> - performance features added (NETIF_F_SG | NETIF_F_FRAGLIST |
>>>> NETIF_F_TSO | NETIF_F_NO_CSUM | NETIF_F_HIGHDMA)
>>>>
>>>> Signed-off-by: Eric Dumazet<eric.dumazet@gmail.com>
>>>> Signed-off-by: David S. Miller<davem@davemloft.net>
>>>>
>>>>
>>>> Eric,
>>>>
>>>> Andrian is observing, with a couple of macvlan (in bridge mode) on top
>>>> of a dummy interface, a lot of checksums error and packets drop.
>>>> Each macvlan is in a different network namespace and the dummy interface
>>>> is in the init_net.
>>>>
>>>> Any ideas ?
>>> Not sure I understand... I thought dummy was dropping all frames
>>> anyway ?
>>>
>>> static netdev_tx_t dummy_xmit(struct sk_buff *skb, struct net_device *dev)
>>> {
>>> struct pcpu_dstats *dstats = this_cpu_ptr(dev->dstats);
>>>
>>> u64_stats_update_begin(&dstats->syncp);
>>> dstats->tx_packets++;
>>> dstats->tx_bytes += skb->len;
>>> u64_stats_update_end(&dstats->syncp);
>>>
>>> dev_kfree_skb(skb);
>>> return NETDEV_TX_OK;
>>> }
>>>
>>>
>>> Maybe you could describe the setup ?
>> Yes, it is very simple.
>>
>> There are two network namespaces.
>>
>> macvlan1 is in network namespace 1
>> macvlan2 is in network namespace 2
>>
>> Both are in "bridge" mode, so they can communicate together.
>> The lower device is dummy0 in the init network namespace.
>>
>> IMO the problem is coming from the macvlan driver:
>>
>> dev->features = lowerdev->features& MACVLAN_FEATURES
>>
>> As dummy0 has the offloading capabilities set on, the macvlan driver
>> inherit these features.
>>
>> In the normal case, dummy0 is supposed to drop the packets. But with
>> macvlan these packets are broadcasted to the other macvlan ports, so no
>> checksum is computed when the packets are transmitted between macvlan1
>> and macvlan2.
> So where frames get bad checksums ?
>
> In this "bridge" mode, I suspect the broadcast is done _before_ sending
> frame to dummy, so maybe macvlan should not inherit from lowerdev in
> this particular case ?
Hi Eric,
yes, you are right, the packets are sent before.
In the 'macvlan_queue_xmit', the code checks the dev is in 'bridge'
mode. If so, it looks if there is a destination port for the packet and
then calls the 'forward' callback which is 'dev_forward_skb'.
I was able to reproduce the same problem with qemu and an emulated
'e1000' card instead of dummy0. The packets are dropped too.
Patrick, do you have any suggestions to fix this ?
Thanks
-- Daniel
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 13:27 UTC (permalink / raw)
To: Eric Dumazet, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301120112.GL9763@canuck.infradead.org>
On Tue, Mar 01, 2011 at 07:01:12AM -0500, Thomas Graf wrote:
> On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
>
> This is how perf top looks like with SO_REUSEPORT
>
> ----------------------------------------------------------------------------------------------------------------------------------
> PerfTop: 27498 irqs/sec kernel:50.5% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
> ----------------------------------------------------------------------------------------------------------------------------------
>
> samples pcnt function DSO
> _______ _____ _____________________________ __________________
>
> 16464.00 6.0% isc_rwlock_lock libisc.so.62.0.1
> 15462.00 5.7% intel_idle [kernel.kallsyms]
So was this a RHEL6 kernel? I wonder if that is what's making it
perform better.
I guess we'll find out tomorrow.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 13:18 UTC (permalink / raw)
To: Eric Dumazet
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298984609.3284.98.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 02:03:29PM +0100, Eric Dumazet wrote:
>
> I believe its now done properly (in net-next-2.6) with commit
> 4f57c087de9b46182 (net: implement mechanism for HW based QOS)
Nope, that has nothing to do with this.
> > The odd packet reordering each time your scheduler decides to
> > migrate the process isn't a big deal IMHO. If your scheduler
> > is constantly moving things you've got bigger problems to worry
> > about.
>
> Well, BENET has one TX queue anyway...
Interesting. So I wonder which lock is showing up at the top
of the profile with a single socket then. As it's definitely
going away with multiple sockets, that means it's not the TX
queue lock.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 13:11 UTC (permalink / raw)
To: Eric Dumazet
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298984669.3284.99.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 02:04:29PM +0100, Eric Dumazet wrote:
> Well, some machines have 4096 cpus ;)
Well just change it to use the multiplication then :)
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 13:04 UTC (permalink / raw)
To: Herbert Xu
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301123250.GA7368@gondor.apana.org.au>
Le mardi 01 mars 2011 à 20:32 +0800, Herbert Xu a écrit :
> On Tue, Mar 01, 2011 at 07:53:05PM +0800, Herbert Xu wrote:
> > On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
> > >
> > > CPU 11 handles all TX completions : Its a potential bottleneck.
> > >
> > > I might ressurect XPS patch ;)
> >
> > Actually this has been my gripe all along with our TX multiqueue
> > support. We should not decide the queue based on the socket, but
> > on the current CPU.
> >
> > We already do the right thing for forwarded packets because there
> > is no socket to latch onto, we just need to fix it for locally
> > generated traffic.
> >
> > The odd packet reordering each time your scheduler decides to
> > migrate the process isn't a big deal IMHO. If your scheduler
> > is constantly moving things you've got bigger problems to worry
> > about.
>
> If anybody wants to play here is a patch to do exactly that:
>
> net: Determine TX queue purely by current CPU
>
> Distributing packets generated on one CPU to multiple queues
> makes no sense. Nor does putting packets from multiple CPUs
> into a single queue.
>
> While this may introduce packet reordering should the scheduler
> decide to migrate a thread, it isn't a big deal because migration
> is meant to be a rare event, and nothing will die as long as the
> ordering doesn't occur all the time.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
>
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 8ae6631..87bd20a 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -2164,22 +2164,12 @@ static u32 hashrnd __read_mostly;
> u16 __skb_tx_hash(const struct net_device *dev, const struct sk_buff *skb,
> unsigned int num_tx_queues)
> {
> - u32 hash;
> + u32 hash = raw_smp_processor_id();
>
> - if (skb_rx_queue_recorded(skb)) {
> - hash = skb_get_rx_queue(skb);
> - while (unlikely(hash >= num_tx_queues))
> - hash -= num_tx_queues;
> - return hash;
> - }
> + while (unlikely(hash >= num_tx_queues))
> + hash -= num_tx_queues;
>
> - if (skb->sk && skb->sk->sk_hash)
> - hash = skb->sk->sk_hash;
> - else
> - hash = (__force u16) skb->protocol ^ skb->rxhash;
> - hash = jhash_1word(hash, hashrnd);
> -
> - return (u16) (((u64) hash * num_tx_queues) >> 32);
> + return hash;
> }
> EXPORT_SYMBOL(__skb_tx_hash);
>
> Cheers,
Well, some machines have 4096 cpus ;)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox