* 答复: [PATCH] set fake_rtable's dst to NULL to avoid kernel Oops.
From: Peter Huang (Peng) @ 2012-03-29 6:40 UTC (permalink / raw)
To: 'Eric Dumazet'
Cc: linux-kernel, harry.majun, zhoukang7, 'netdev'
In-Reply-To: <1333002975.2325.82.camel@edumazet-glaptop>
We already check current kernel-3.3, it has the same problem.
I am not very sure that if this modify could cause other problems or not,
Because I don't know where fake_rtable was used.
-----邮件原件-----
发件人: Eric Dumazet [mailto:eric.dumazet@gmail.com]
发送时间: 2012年3月29日 14:36
收件人: Peter Huang (Peng)
抄送: linux-kernel@vger.kernel.org; harry.majun@huawei.com; zhoukang7@huawei.com; netdev
主题: Re: [PATCH] set fake_rtable's dst to NULL to avoid kernel Oops.
On Thu, 2012-03-29 at 14:21 +0800, Peter Huang (Peng) wrote:
> In our environment, we encountered a kernel Oops problem, and caused a
> restart.
>
CC netdev, since its more appropriate
> Below are what happened:
> kernel: 2.6.32.36-0.5-xen OS:xen + dom-0 + guest(rhel5.5)
> 1.destroy one VM.
> 2.ipsan path have some problem and make destroy process delayed about 10s.
> 3.customer defined script find that VM no longer exsit through libvirt API.
> 4.br0(related to the VM we are destoryed before) was deleted by the script.
> 5.delayed VM destroy process come to tap device releasing, this will
> decrement
> skb->_skb_dst's reference count(skb->_skb_dst points to fake_rtable), but
> br0
> deleting already released this struct, and unfortunately OS reused this
> memory
> and marked it read-only.
> 6.Oops happened, and caused restart.
>
> After analyzing the stack dump info, we find out that during our VM destroy,
> lots of ipv6 multicast pkts
> exsited, and skb->_skb_dst pointed to (stuct)fake_rtable.
> through kernel source greping, will only find one reference to fake_rtable's
> MTU setting.
>
> So I'm wondering that what fake_rtable stands for, and where we are using
> it.
> If fake_rtable's dst is not used, we can make dst as NULL to avoid our
> problem,.
> I also added the patch which modified the skb->_skb_dst to NULL when
> "skb->_skb_dst == (unsigned long)&to->br->fake_rtable".
>
> BTW, we also verified a similar senario on kernel-3.3, that br0 has attached
> eth0 and eth1, eth1 was
> connected to our guest which will multicast ipv6 packets, and you can get an
> "WARNING: at net/core/dst.c:274 dst_release+0x6d/0x70()"
> by using the fake_rtable_verify.c attached,
> #gcc fake_rtable_verify.c
> #./a.out &
> #sleep 30 //make sure ipv6 pkts was in tap00's receiving queue.
> #ifconfig br0 down
> #brctl delbr br0 //delete br0, will also delete net_device's fake_rtable.
> #sleep 50
> #kill -9 `pidof a.out` //tap00's delete will do dst_release, and this will
> write to the memory already freed.
>
> Below is the Oops stack dump info:
> ////////////////////////////////////////////////////////////////////////////
> ///
> RIP: e030:[<ffffffff802ddbd1>]
> <ffffffff802ddbd1>{dst_release+0x11}
> RSP: e02b:ffff88008b185b70 EFLAGS: 00010286
> RAX: 00000000ffffffff RBX: ffff880033d184c0 RCX: 0000000000000000
> RDX: ffff88008b54f080 RSI: 0000000012df12df RDI: ffff88008b54efc0
> RBP: ffff8800f4a3f500 R08: 0000000000000001 R09: 0000000000000000
> R10: 0000000000000002 R11: ffffffff8018c1e0 R12: ffff8800f4a3f400
> R13: 0000000000000001 R14: ffff8800f4a3f4e0 R15: ffff8800351030c0
> FS: 00007f4cbd080700(0000) GS:ffff880002008000(0000) knlGS:0000000000000000
> CS: e033 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffff88008b54f080 CR3: 000000008a27c000 CR4: 0000000000002620
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> <ffffffff80009b05>{dump_trace+0x65}
> <ffffffff8037d897>{notifier_call_chain+0x37}
> <ffffffff8005a1ed>{notify_die+0x2d}
> <ffffffff8037bd0b>{__die+0x8b}
> <ffffffff8001bed1>{no_context+0xd1}
> <ffffffff8001c1f5>{__bad_area_nosemaphore+0x175}
> <ffffffff8037b298>{page_fault+0x28}
> <ffffffff802ddbd1>{dst_release+0x11}
> <ffffffff802cd69d>{skb_release_head_state+0xbd}
> <ffffffff802cd369>{__kfree_skb+0x9}
> <ffffffff802edaab>{pfifo_fast_reset+0x5b}
> <ffffffff802edbd3>{qdisc_reset+0x13}
> <ffffffff802edcc7>{dev_deactivate_queue+0x57}
> <ffffffff802ee4bf>{dev_deactivate+0x3f}
> <ffffffff802d9575>{dev_close+0x65}
> <ffffffff802d960e>{rollback_registered+0x3e}
> <ffffffff802d9715>{unregister_netdevice+0x15}
> <ffffffffa0807655>{tun:tun_chr_close+0xe5}
> <ffffffff800d9edd>{__fput+0xcd}
> <ffffffff800d6076>{filp_close+0x56}
> <ffffffff8003fd9a>{put_files_struct+0x7a}
> <ffffffff80040fb2>{do_exit+0x752}
> <ffffffff800410ef>{do_group_exit+0x3f}
> <ffffffff8004d9d9>{get_signal_to_deliver+0x229}
> <ffffffff80006acd>{do_notify_resume+0x11d}
> <ffffffff8000763c>{int_signal+0x12}
> [<00007f4cbc7fd57d>]
> ////////////////////////////////////////////////////////////////////////////
> ///
>
> Signed-off-by: Peter Huang(Peng) <peter.huangpeng@huawei.com>
> ---
> diff -Nur a/net/bridge/br_forward.c b/net/bridge/br_forward.c
> @@ -91,6 +91,9 @@
> skb->dev = to->dev;
> skb_forward_csum(skb);
>
> + if (skb->_skb_dst == (unsigned long)&to->br->fake_rtable)
> + skb_dst_set(skb, NULL);
> +
> NF_HOOK(NFPROTO_BRIDGE, NF_BR_FORWARD, skb, indev, skb->dev,
> br_forward_finish);
> }
Did you check current kernel has this bug ?
I remember we already fix this, maybe you need a backport.
^ permalink raw reply
* Re: [PATCH] f_phonet: fix skb truesize underestimation
From: Eric Dumazet @ 2012-03-29 6:46 UTC (permalink / raw)
To: Rémi Denis-Courmont
Cc: ext David Miller, gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r,
linux-usb-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
balbi-l0cyMroinI0
In-Reply-To: <1377643.vUarymgFD1@hector>
On Thu, 2012-03-29 at 09:30 +0300, Rémi Denis-Courmont wrote:
> Le mardi 27 mars 2012 22:51:09 ext David Miller a écrit :
> > From: Eric Dumazet <eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Date: Tue, 27 Mar 2012 06:04:02 -0700
> >
> > > Now skb_add_rx_frag() has a truesize parameter, we can fix f_phonet to
> > > properly account truesize of each fragment : a full page.
> > >
> > > Signed-off-by: Eric Dumazet <eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> >
> > Applied.
>
> Sorry for not reviewing. My evil corporate mail transfer agent apparently ate
> that patch :-(
>
> I guess I should use my personal email instead...
>
Thats true, I got a funny reject from your mail server, stating :
"Your domain is blocked with bad reputation defined by CommTouch: ...
(state 13)"
gmail.com has bad reputation ? Come on !
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] f_phonet: fix skb truesize underestimation
From: David Miller @ 2012-03-29 6:51 UTC (permalink / raw)
To: remi.denis-courmont-xNZwKgViW5gAvxtiuMwx3w
Cc: eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w,
gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r,
linux-usb-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
balbi-l0cyMroinI0
In-Reply-To: <1377643.vUarymgFD1@hector>
From: Rémi Denis-Courmont <remi.denis-courmont-xNZwKgViW5gAvxtiuMwx3w@public.gmane.org>
Date: Thu, 29 Mar 2012 09:30:32 +0300
> I guess I should use my personal email instead...
nokia.com tends to eat 2 out of every 3 mailing list postings
on vger.kernel.org, so yes I'd say using person email would
be a good idea
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [BUGFIX][PATCH 0/3] memcg: tcp memcontrol fixes.
From: KAMEZAWA Hiroyuki @ 2012-03-29 7:01 UTC (permalink / raw)
To: Glauber Costa, netdev; +Cc: David Miller, Andrew Morton, kamezawa.hiroyu
I'm very sorry if you received this e-mail twice.
==
This series is 3 bugfixes for memcg's kmem.tcp memory controller.
Maybe this should go via network tree.
(CC akpm for noticing an ugly change in res_counter.)
All patches are generated onto today linus's git tree.
Brief description:
Patch 1/3 .... tcp memcontrol doesn't see memcg's use_hierarchy value. Fix it.
Patch 2/3 and 3/3 ....
Because tcp memcontrol doesn't do any accounting when limit=RESOUCE_MAX,
there will be account leakage when limit is changed. This can trigger
WARN_ON() in res_counter which checks usage >= 0.
Patch 2/3 .... don't call static_key_slow_dec(&memcg_socket_limit_enabled) until
a cgroup under accounted is destroyed.
Patch 3/3 .... add res_counter_uncharge_nowarn() to ignore leakage.
Thanks,
-Kame
^ permalink raw reply
* [PATCH 1/3] [BUGFIX] memcg/tcp : fix to see use_hierarchy in tcp memcontrol cgroup
From: KAMEZAWA Hiroyuki @ 2012-03-29 7:03 UTC (permalink / raw)
To: KAMEZAWA Hiroyuki; +Cc: Glauber Costa, netdev, David Miller, Andrew Morton
In-Reply-To: <4F7408B7.9090706@jp.fujitsu.com>
Now, tcp memory control cgroup ignores memcg's use_hierarchy value
and act as use_hierarchy=1 always. After this patch, tcp memcontrol will
work as memcg is designed.
Note:
I know there is a discussion to remove use_hierarchy but this is BUG, now.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
---
include/linux/memcontrol.h | 3 +++
mm/memcontrol.c | 5 +++++
net/ipv4/tcp_memcontrol.c | 2 +-
3 files changed, 9 insertions(+), 1 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index f94efd2..e116b7c 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -199,6 +199,9 @@ void mem_cgroup_split_huge_fixup(struct page *head);
bool mem_cgroup_bad_page_check(struct page *page);
void mem_cgroup_print_bad_page(struct page *page);
#endif
+
+bool mem_cgroup_use_hierarchy(struct mem_cgroup *memcg);
+
#else /* CONFIG_CGROUP_MEM_RES_CTLR */
struct mem_cgroup;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 7d698df..467881f 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -867,6 +867,11 @@ struct mem_cgroup *try_get_mem_cgroup_from_mm(struct mm_struct *mm)
return memcg;
}
+bool mem_cgroup_use_hierarchy(struct mem_cgroup *memcg)
+{
+ return memcg->use_hierarchy;
+}
+
/**
* mem_cgroup_iter - iterate over memory cgroup hierarchy
* @root: hierarchy root
diff --git a/net/ipv4/tcp_memcontrol.c b/net/ipv4/tcp_memcontrol.c
index e795272..32764a6 100644
--- a/net/ipv4/tcp_memcontrol.c
+++ b/net/ipv4/tcp_memcontrol.c
@@ -75,7 +75,7 @@ int tcp_init_cgroup(struct cgroup *cgrp, struct cgroup_subsys *ss)
tcp->tcp_memory_pressure = 0;
parent_cg = tcp_prot.proto_cgroup(parent);
- if (parent_cg)
+ if (parent_cg && mem_cgroup_use_hierarchy(parent))
res_parent = parent_cg->memory_allocated;
res_counter_init(&tcp->tcp_memory_allocated, res_parent);
--
1.7.4.1
^ permalink raw reply related
* [BUGFIX][PATCH 2/3] memcg/tcp: remove static_branch_slow_dec() at changing limit
From: KAMEZAWA Hiroyuki @ 2012-03-29 7:07 UTC (permalink / raw)
To: KAMEZAWA Hiroyuki; +Cc: Glauber Costa, netdev, David Miller, Andrew Morton
In-Reply-To: <4F7408B7.9090706@jp.fujitsu.com>
tcp memcontrol uses static_branch to optimize limit=RESOURCE_MAX case.
If all cgroup's limit=RESOUCE_MAX, resource usage is not accounted.
But it's buggy now.
For example, do following
# while sleep 1;do
echo 9223372036854775807 > /cgroup/memory/A/memory.kmem.tcp.limit_in_bytes;
echo 300M > /cgroup/memory/A/memory.kmem.tcp.limit_in_bytes;
done
and run network application under A. tcp's usage is sometimes accounted
and sometimes not accounted because of frequent changes of static_branch.
Then, finally, you can see broken tcp.usage_in_bytes.
WARN_ON() is printed because res_counter->usage goes below 0.
==
kernel: ------------[ cut here ]----------
kernel: WARNING: at kernel/res_counter.c:96 res_counter_uncharge_locked+0x37/0x40()
<snip>
kernel: Pid: 17753, comm: bash Tainted: G W 3.3.0+ #99
kernel: Call Trace:
kernel: <IRQ> [<ffffffff8104cc9f>] warn_slowpath_common+0x7f/0xc0
kernel: [<ffffffff810d7e88>] ? rb_reserve__next_event+0x68/0x470
kernel: [<ffffffff8104ccfa>] warn_slowpath_null+0x1a/0x20
kernel: [<ffffffff810b4e37>] res_counter_uncharge_locked+0x37/0x40
...
==
This patch removes static_branch_slow_dec() at changing res_counter's
limit to RESOUCE_MAX. By this, once accounting started, the accountting
will continue until the tcp cgroup is destroyed.
I think this will not be problem in real use.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
---
include/net/tcp_memcontrol.h | 1 +
net/ipv4/tcp_memcontrol.c | 24 ++++++++++++++++++------
2 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/include/net/tcp_memcontrol.h b/include/net/tcp_memcontrol.h
index 48410ff..f47e3c7 100644
--- a/include/net/tcp_memcontrol.h
+++ b/include/net/tcp_memcontrol.h
@@ -9,6 +9,7 @@ struct tcp_memcontrol {
/* those two are read-mostly, leave them at the end */
long tcp_prot_mem[3];
int tcp_memory_pressure;
+ bool accounting;
};
struct cg_proto *tcp_proto_cgroup(struct mem_cgroup *memcg);
diff --git a/net/ipv4/tcp_memcontrol.c b/net/ipv4/tcp_memcontrol.c
index 32764a6..cd0b47d 100644
--- a/net/ipv4/tcp_memcontrol.c
+++ b/net/ipv4/tcp_memcontrol.c
@@ -49,6 +49,20 @@ static void memcg_tcp_enter_memory_pressure(struct sock *sk)
}
EXPORT_SYMBOL(memcg_tcp_enter_memory_pressure);
+static void tcp_start_accounting(struct tcp_memcontrol *tcp)
+{
+ if (tcp->accounting)
+ return;
+ tcp->accounting = true;
+ static_key_slow_inc(&memcg_socket_limit_enabled);
+}
+
+static void tcp_end_accounting(struct tcp_memcontrol *tcp)
+{
+ if (tcp->accounting)
+ static_key_slow_dec(&memcg_socket_limit_enabled);
+}
+
int tcp_init_cgroup(struct cgroup *cgrp, struct cgroup_subsys *ss)
{
/*
@@ -73,6 +87,7 @@ int tcp_init_cgroup(struct cgroup *cgrp, struct cgroup_subsys *ss)
tcp->tcp_prot_mem[1] = net->ipv4.sysctl_tcp_mem[1];
tcp->tcp_prot_mem[2] = net->ipv4.sysctl_tcp_mem[2];
tcp->tcp_memory_pressure = 0;
+ tcp->accounting = false;
parent_cg = tcp_prot.proto_cgroup(parent);
if (parent_cg && mem_cgroup_use_hierarchy(parent))
@@ -110,8 +125,7 @@ void tcp_destroy_cgroup(struct cgroup *cgrp)
val = res_counter_read_u64(&tcp->tcp_memory_allocated, RES_LIMIT);
- if (val != RESOURCE_MAX)
- static_key_slow_dec(&memcg_socket_limit_enabled);
+ tcp_end_accounting(tcp);
}
EXPORT_SYMBOL(tcp_destroy_cgroup);
@@ -142,10 +156,8 @@ static int tcp_update_limit(struct mem_cgroup *memcg, u64 val)
tcp->tcp_prot_mem[i] = min_t(long, val >> PAGE_SHIFT,
net->ipv4.sysctl_tcp_mem[i]);
- if (val == RESOURCE_MAX && old_lim != RESOURCE_MAX)
- static_key_slow_dec(&memcg_socket_limit_enabled);
- else if (old_lim == RESOURCE_MAX && val != RESOURCE_MAX)
- static_key_slow_inc(&memcg_socket_limit_enabled);
+ if (old_lim == RESOURCE_MAX && val != RESOURCE_MAX)
+ tcp_start_accounting(tcp);
return 0;
}
--
1.7.4.1
^ permalink raw reply related
* [BUGFIX][PATCH 3/3] memcg/tcp: ignore tcp usage before accounting started
From: KAMEZAWA Hiroyuki @ 2012-03-29 7:10 UTC (permalink / raw)
To: KAMEZAWA Hiroyuki; +Cc: Glauber Costa, netdev, David Miller, Andrew Morton
In-Reply-To: <4F7408B7.9090706@jp.fujitsu.com>
tcp memcontrol starts accouting after res->limit is set. So, if a sockets
starts before setting res->limit, there are already used resource.
After setting res->limit, the resource (already used) will be uncharged and
make res_counter below 0 because they are not charged. This causes warning.
This patch fixes that by adding res_counter_uncharge_nowarn().
(*) We cannot avoid this while we have 'account start' switch.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
---
include/linux/res_counter.h | 2 ++
include/net/sock.h | 3 ++-
kernel/res_counter.c | 18 ++++++++++++++++++
3 files changed, 22 insertions(+), 1 deletions(-)
diff --git a/include/linux/res_counter.h b/include/linux/res_counter.h
index da81af0..e081948 100644
--- a/include/linux/res_counter.h
+++ b/include/linux/res_counter.h
@@ -134,6 +134,8 @@ int __must_check res_counter_charge_nofail(struct res_counter *counter,
void res_counter_uncharge_locked(struct res_counter *counter, unsigned long val);
void res_counter_uncharge(struct res_counter *counter, unsigned long val);
+void res_counter_uncharge_nowarn(struct res_counter *counter,
+ unsigned long val);
/**
* res_counter_margin - calculate chargeable space of a counter
diff --git a/include/net/sock.h b/include/net/sock.h
index a6ba1f8..a1b3f4802 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1048,7 +1048,8 @@ static inline void memcg_memory_allocated_add(struct cg_proto *prot,
static inline void memcg_memory_allocated_sub(struct cg_proto *prot,
unsigned long amt)
{
- res_counter_uncharge(prot->memory_allocated, amt << PAGE_SHIFT);
+ res_counter_uncharge_nowarn(prot->memory_allocated,
+ amt << PAGE_SHIFT);
}
static inline u64 memcg_memory_allocated_read(struct cg_proto *prot)
diff --git a/kernel/res_counter.c b/kernel/res_counter.c
index d508363..2bb01ac 100644
--- a/kernel/res_counter.c
+++ b/kernel/res_counter.c
@@ -113,6 +113,24 @@ void res_counter_uncharge(struct res_counter *counter, unsigned long val)
local_irq_restore(flags);
}
+void res_counter_uncharge_nowarn(struct res_counter *counter,
+ unsigned long val)
+{
+ struct res_counter *c;
+ unsigned long flags;
+
+ local_irq_save(flags);
+
+ for (c = counter; c != NULL; c = c->parent) {
+ spin_lock(&c->lock);
+ if (c->usage < val)
+ val = c->usage;
+ res_counter_uncharge_locked(c, val);
+ spin_unlock(&c->lock);
+ }
+ local_irq_restore(flags);
+}
+
static inline unsigned long long *
res_counter_member(struct res_counter *counter, int member)
--
1.7.4.1
^ permalink raw reply related
* [Q/RFC] BPF use in broader scope
From: Jiri Pirko @ 2012-03-29 7:44 UTC (permalink / raw)
To: netdev; +Cc: eric.dumazet, davem, bhutchings, shemminger
Hi all.
I came to an idea of using BPF infrastructure currently used in kernel,
for computing hashes selecting TX ports in team device. Since the same
data (skb) are alalyzed/used as for socket filtering, BPF seems so be quite
suitable for this. It would allow userspace daemon to specify various
kinds of TX selection algorithms.
Here are proposed things to be done:
1) introduce in-kernel api for creating sk-unattached filters (I have
the patch cooked up already)
2) extend current BPF machine to allow XOR operation. Not sure if this
is doable or what the best of doing this is.
3) add possibility to pass some data to the machine via
pre-filling "Scratch Memory Store". I think this can be done easily
moving "u32 mem[BPF_MEMWORDS];" to bpf_func caller and pass it as the
second function parameter. That should not break anything.
Then the computed hash can be either stored into Scratch memory or returned
directly (where ordinary sk filters return len).
Does this seems reasonable? Thoughts, comments?
Thanks!
Jirka
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: David Miller @ 2012-03-29 7:49 UTC (permalink / raw)
To: jpirko; +Cc: netdev, eric.dumazet, bhutchings, shemminger
In-Reply-To: <20120329074443.GB2098@minipsycho>
From: Jiri Pirko <jpirko@redhat.com>
Date: Thu, 29 Mar 2012 09:44:43 +0200
> Here are proposed things to be done:
> 1) introduce in-kernel api for creating sk-unattached filters (I have
> the patch cooked up already)
>
> 2) extend current BPF machine to allow XOR operation. Not sure if this
> is doable or what the best of doing this is.
>
> 3) add possibility to pass some data to the machine via
> pre-filling "Scratch Memory Store". I think this can be done easily
> moving "u32 mem[BPF_MEMWORDS];" to bpf_func caller and pass it as the
> second function parameter. That should not break anything.
>
> Then the computed hash can be either stored into Scratch memory or returned
> directly (where ordinary sk filters return len).
>
> Does this seems reasonable? Thoughts, comments?
No fundamental objections, but we have all of these JITs now to
update when adding new operations or semantics, so be careful.
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: Jiri Pirko @ 2012-03-29 7:54 UTC (permalink / raw)
To: David Miller; +Cc: netdev, eric.dumazet, bhutchings, shemminger
In-Reply-To: <20120329.034957.655153582806618222.davem@davemloft.net>
Thu, Mar 29, 2012 at 09:49:57AM CEST, davem@davemloft.net wrote:
>From: Jiri Pirko <jpirko@redhat.com>
>Date: Thu, 29 Mar 2012 09:44:43 +0200
>
>> Here are proposed things to be done:
>> 1) introduce in-kernel api for creating sk-unattached filters (I have
>> the patch cooked up already)
>>
>> 2) extend current BPF machine to allow XOR operation. Not sure if this
>> is doable or what the best of doing this is.
>>
>> 3) add possibility to pass some data to the machine via
>> pre-filling "Scratch Memory Store". I think this can be done easily
>> moving "u32 mem[BPF_MEMWORDS];" to bpf_func caller and pass it as the
>> second function parameter. That should not break anything.
>>
>> Then the computed hash can be either stored into Scratch memory or returned
>> directly (where ordinary sk filters return len).
>>
>> Does this seems reasonable? Thoughts, comments?
>
>No fundamental objections, but we have all of these JITs now to
>update when adding new operations or semantics, so be careful.
Yep, I'm aware. I must admit that the JIT code scares me a litte :(
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: Eric Dumazet @ 2012-03-29 7:58 UTC (permalink / raw)
To: Jiri Pirko; +Cc: netdev, davem, bhutchings, shemminger
In-Reply-To: <20120329074443.GB2098@minipsycho>
On Thu, 2012-03-29 at 09:44 +0200, Jiri Pirko wrote:
> Hi all.
>
> I came to an idea of using BPF infrastructure currently used in kernel,
> for computing hashes selecting TX ports in team device. Since the same
> data (skb) are alalyzed/used as for socket filtering, BPF seems so be quite
> suitable for this. It would allow userspace daemon to specify various
> kinds of TX selection algorithms.
>
> Here are proposed things to be done:
> 1) introduce in-kernel api for creating sk-unattached filters (I have
> the patch cooked up already)
>
> 2) extend current BPF machine to allow XOR operation. Not sure if this
> is doable or what the best of doing this is.
>
> 3) add possibility to pass some data to the machine via
> pre-filling "Scratch Memory Store". I think this can be done easily
> moving "u32 mem[BPF_MEMWORDS];" to bpf_func caller and pass it as the
> second function parameter. That should not break anything.
>
> Then the computed hash can be either stored into Scratch memory or returned
> directly (where ordinary sk filters return len).
>
> Does this seems reasonable? Thoughts, comments?
>
Sure it seems good ideas.
Maybe add :
4) be able to extend skb_flow_dissect() using BPF hooks
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: Eric Dumazet @ 2012-03-29 8:02 UTC (permalink / raw)
To: Jiri Pirko; +Cc: David Miller, netdev, bhutchings, shemminger
In-Reply-To: <20120329075410.GC2098@minipsycho>
On Thu, 2012-03-29 at 09:54 +0200, Jiri Pirko wrote:
> Yep, I'm aware. I must admit that the JIT code scares me a litte :(
>
If you add a new XOR instruction in interpreter only, JIT compiler will
automatically aborts, so no risk.
Each arch maintainer will add the support for the new instructions as
separate patches.
So you can focus on net/core/filter.c file only.
^ permalink raw reply
* Re: [PATCH] tcp: bind() use stronger condition for bind_conflict
From: Alexandru Copot @ 2012-03-29 8:22 UTC (permalink / raw)
To: Eric Dumazet
Cc: Daniel Baluta, Flavio Leitner, davem, kuznet, jmorris, yoshfuji,
kaber, netdev, linux-kernel
In-Reply-To: <1332965525.2325.13.camel@edumazet-glaptop>
On Wed, Mar 28, 2012 at 11:12 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Some performance data would be welcomed, in the case many sockets are
> already bound...
I've done some tests running a program that creates and binds(0) 90000 sockets.
The total running time, on average, is:
* without this patch: 0.352 s
* with the patch: 0.355 s
Also, recording this program with perf shows a small increase of 0.6%
for inet_csk_get_port
relative to inet_bind, after applying the patch.
So the performance is almost the same, even for such a large number of sockets.
Alex Copot
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: Jiri Pirko @ 2012-03-29 8:31 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, netdev, bhutchings, shemminger, matt
In-Reply-To: <1333008145.2325.275.camel@edumazet-glaptop>
Thu, Mar 29, 2012 at 10:02:25AM CEST, eric.dumazet@gmail.com wrote:
>On Thu, 2012-03-29 at 09:54 +0200, Jiri Pirko wrote:
>
>> Yep, I'm aware. I must admit that the JIT code scares me a litte :(
>>
>
>If you add a new XOR instruction in interpreter only, JIT compiler will
>automatically aborts, so no risk.
>
>Each arch maintainer will add the support for the new instructions as
>separate patches.
>
>So you can focus on net/core/filter.c file only.
>
Ok - I can do this for 2). But for 3) JITs need to be modified. So I
would like to kindly ask you and Matt if you can do this modification so
bpf_func takes pointer to mem (scratch store) as second parameter. I'm
sure it's very easy for you to do.
Ccing Matt.
Thanks.
>
>
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: David Miller @ 2012-03-29 8:43 UTC (permalink / raw)
To: jpirko; +Cc: eric.dumazet, netdev, bhutchings, shemminger, matt
In-Reply-To: <20120329083149.GD2098@minipsycho>
From: Jiri Pirko <jpirko@redhat.com>
Date: Thu, 29 Mar 2012 10:31:49 +0200
> Thu, Mar 29, 2012 at 10:02:25AM CEST, eric.dumazet@gmail.com wrote:
>>On Thu, 2012-03-29 at 09:54 +0200, Jiri Pirko wrote:
>>
>>> Yep, I'm aware. I must admit that the JIT code scares me a litte :(
>>>
>>
>>If you add a new XOR instruction in interpreter only, JIT compiler will
>>automatically aborts, so no risk.
>>
>>Each arch maintainer will add the support for the new instructions as
>>separate patches.
>>
>>So you can focus on net/core/filter.c file only.
>>
>
> Ok - I can do this for 2). But for 3) JITs need to be modified. So I
> would like to kindly ask you and Matt if you can do this modification so
> bpf_func takes pointer to mem (scratch store) as second parameter. I'm
> sure it's very easy for you to do.
The ARM JIT just went into Linus's tree as well.
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: Eric Dumazet @ 2012-03-29 8:45 UTC (permalink / raw)
To: Jiri Pirko; +Cc: David Miller, netdev, bhutchings, shemminger, matt
In-Reply-To: <20120329083149.GD2098@minipsycho>
On Thu, 2012-03-29 at 10:31 +0200, Jiri Pirko wrote:
> Thu, Mar 29, 2012 at 10:02:25AM CEST, eric.dumazet@gmail.com wrote:
> >On Thu, 2012-03-29 at 09:54 +0200, Jiri Pirko wrote:
> >
> >> Yep, I'm aware. I must admit that the JIT code scares me a litte :(
> >>
> >
> >If you add a new XOR instruction in interpreter only, JIT compiler will
> >automatically aborts, so no risk.
> >
> >Each arch maintainer will add the support for the new instructions as
> >separate patches.
> >
> >So you can focus on net/core/filter.c file only.
> >
>
> Ok - I can do this for 2). But for 3) JITs need to be modified. So I
> would like to kindly ask you and Matt if you can do this modification so
> bpf_func takes pointer to mem (scratch store) as second parameter. I'm
> sure it's very easy for you to do.
I am not sure why you want this.
This adds register pressure (at least for x86) ...
^ permalink raw reply
* Re: 答复: [PATCH] set fake_rtable's dst to NULL to avoid kernel Oops.
From: Eric Dumazet @ 2012-03-29 8:52 UTC (permalink / raw)
To: Peter Huang (Peng); +Cc: linux-kernel, harry.majun, zhoukang7, 'netdev'
In-Reply-To: <002601cd0d76$c4987440$4dc95cc0$%huangpeng@huawei.com>
On Thu, 2012-03-29 at 14:40 +0800, Peter Huang (Peng) wrote:
> We already check current kernel-3.3, it has the same problem.
>
> I am not very sure that if this modify could cause other problems or not,
> Because I don't know where fake_rtable was used.
Check net/bridge/br_netfilter.c and commits e688a6048076 (net: introduce
DST_NOPEER dst flag ) 4adf0af6818f3ea5 (bridge: send correct MTU value
in PMTU (revised))
Apparently bug is because struct net_bridge is freed while its embedded
fake_rtable is still used by some packets.
I am not sure we are allowed to NULLify skb->dst, it might break
netfilter.
Maybe real fix would be to use a non embedded dst.
^ permalink raw reply
* Re: maranello for linux-current [b43 driver]
From: Florian Fainelli @ 2012-03-29 9:07 UTC (permalink / raw)
To: Jim Cromie; +Cc: Bo Han, netdev-u79uwXL29TY76Z2rM5mHXA, linux-wireless
In-Reply-To: <CAJfuBxz9kVAkkSjFk2PAYLxOsgmrCx-HV0OM7EwV9J5mSzB8dA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
(Adding linux-wireless in CC)
Le 03/29/12 00:24, Jim Cromie a écrit :
> you may have heard of maranello, a project to add
> partial packet recovery to 80211, by adding block checksums
> on transmit, validating them or re-requesting retransmit
> of corrupted blocks.
>
> http://www.cs.umd.edu/projects/maranello/
>
> the code at the page is for 2.6.29-rc2
>
> Ive taken it, did some refactoring to ease merging,
> and merged in each linux release, up to 3.3
>
> The refactoring is:
>
> - move code from include/net/mac80211.h utils.c to
> drivers/net/wireless/maranello.*
> this makes it somewhat common, but more isolated overall,
> and reduces merge conflicts due to continued evolution of those files.
> I imagine new code would eventually go back to the original files, but
> not for a while.
>
> - move most of new code in each *.c file to *_mnlo.c
> fn decls added to corresponding *.h
> again, this code probably would be moved back to original *.c, but not yet.
>
>
> After all the merges, I compiled, and found some errors.
> I had problems building plain 2.6.29-rc2, and several later releases,
> so punted on compiling every step.
>
> The compile errors are fixed on top of the merges, but ideally should be
> rearranged to fix each release. This requires more git-fu than I possess,
> and IIUC, netdev ML is more interested in patches to linux-current.
>
> Repeating: this is only compile tested, I dont have a b43 based wifi card
> in my desk/lap-tops.
>
> I do have a WRT54G router, currently running openWRT 10.03
> which does have a b43 wifi chip. I intend to try putting this code
> there eventually,
> but that may take a while yet, and cannot test maranello itself anyway.
>
> Having split maranello code into new files, I have some misgivings about
> that approach - it simplified merges, but at the cost of not seeing
> the underlying conflicts. The compiler told me about some, surely others lurk.
>
> So I guess Im seeking advice:
>
> - what did I break
> - what happens when the driver is tested
> - whether to repeat this merge-effort w/o the split
>
> - why did earlier releases fail to build
> [jimc@groucho linux-2.6]$ git checkout v2.6.29
> [jimc@groucho build-dell-2]$ make xconfig
> Makefile:23: *** mixed implicit and normal rules. Stop.
> I also see breakage on unrelated stuff (that builds fine on mainline)
> [jimc@groucho build-dell-2]$ make
> AS arch/x86/xen/xen-asm_64.o
> /home/jimc/projects/lx/linux-2.6/arch/x86/xen/xen-asm_64.S:
> Assembler messages:
> /home/jimc/projects/lx/linux-2.6/arch/x86/xen/xen-asm_64.S:48:
> Error: unsupported for `mov'
> ...
> the tree is here:
> https://github.com/jimc/linux-2.6/tree/maranello/released-split
>
> Im happy to open it up for collaboration if theres interest,
> I'll have to figure out how to do that though..
> Of course you can clone and fork it.
>
> Its probably premature to concentrate on rework for inclusion,
> I think itd be better to get it working on mainline 1st,
> but the long view might inform the short-term steps.
>
> thanks
> Jim Cromie
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH net-next v2 5/5] r8169: support RTL8411
From: Francois Romieu @ 2012-03-29 9:12 UTC (permalink / raw)
To: Hayes Wang; +Cc: netdev, linux-kernel
In-Reply-To: <1332923214-5071-5-git-send-email-hayeswang@realtek.com>
Hayes Wang <hayeswang@realtek.com> :
> Support the new chip RTL8411.
>
> Signed-off-by: Hayes Wang <hayeswang@realtek.com>
> ---
> drivers/net/ethernet/realtek/r8169.c | 142 +++++++++++++++++++++++++++++++++-
> 1 files changed, 139 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
> index cde5cea..d00f4e9 100644
> --- a/drivers/net/ethernet/realtek/r8169.c
> +++ b/drivers/net/ethernet/realtek/r8169.c
[...]
> @@ -250,7 +252,10 @@ static const struct {
> JUMBO_9K, false),
> [RTL_GIGA_MAC_VER_37] =
> _R("RTL8402", RTL_TD_1, FIRMWARE_8402_1,
> - JUMBO_1K, true)
> + JUMBO_1K, true),
> + [RTL_GIGA_MAC_VER_38] =
> + _R("RTL8411", RTL_TD_1, FIRMWARE_8411_1,
> + JUMBO_9K, false)
rtl_init_jumbo_ops probably needs to be updated as well.
Nit: please keep a colon in 'JUMBO_9K, false),'.
^
There was one after the (not so) last record before patch 03/05 and I would
prefer it to stay because new devices are regularly added. The '-' line is
only noise.
Thanks.
--
Ueimor
^ permalink raw reply
* Re: [PATCH 1/3] [BUGFIX] memcg/tcp : fix to see use_hierarchy in tcp memcontrol cgroup
From: Glauber Costa @ 2012-03-29 9:14 UTC (permalink / raw)
To: KAMEZAWA Hiroyuki; +Cc: netdev, David Miller, Andrew Morton
In-Reply-To: <4F74095B.70105@jp.fujitsu.com>
On 03/29/2012 09:03 AM, KAMEZAWA Hiroyuki wrote:
>
> Now, tcp memory control cgroup ignores memcg's use_hierarchy value
> and act as use_hierarchy=1 always. After this patch, tcp memcontrol will
> work as memcg is designed.
>
> Note:
> I know there is a discussion to remove use_hierarchy but this is BUG, now.
>
Kame,
Are you sure about that?
I just tried it myself, and it seems to work:
root@inf5072-11:~/glommer-temporary/a/b# cat memory.kmem.tcp.usage_in_bytes
724992
root@inf5072-11:~/glommer-temporary/a/b# cat
../memory.kmem.tcp.usage_in_bytes
0
Did you got this conclusion through testing or code inspection?
As a matter of fact, that's why I believe the current behavior is indeed
correct:
the res_counter is initialized as:
parent_cg = tcp_prot.proto_cgroup(parent);
if (parent_cg)
res_parent = parent_cg->memory_allocated;
res_counter_init(&tcp->tcp_memory_allocated, res_parent);
now, parent is drawn from parent_mem_cgroup(), that reads as follows:
struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *memcg)
{
if (!memcg->res.parent)
return NULL;
return mem_cgroup_from_res_counter(memcg->res.parent, res);
}
so if we have use_hierarchy = 0, res.parent should be NULL (because that
is the way we initialize it)
^ permalink raw reply
* Re: [PATCH 1/3] [BUGFIX] memcg/tcp : fix to see use_hierarchy in tcp memcontrol cgroup
From: KAMEZAWA Hiroyuki @ 2012-03-29 9:16 UTC (permalink / raw)
To: Glauber Costa; +Cc: netdev, David Miller, Andrew Morton
In-Reply-To: <4F7427E4.2020307@parallels.com>
(2012/03/29 18:14), Glauber Costa wrote:
> On 03/29/2012 09:03 AM, KAMEZAWA Hiroyuki wrote:
>>
>> Now, tcp memory control cgroup ignores memcg's use_hierarchy value
>> and act as use_hierarchy=1 always. After this patch, tcp memcontrol will
>> work as memcg is designed.
>>
>> Note:
>> I know there is a discussion to remove use_hierarchy but this is BUG, now.
>>
>
> Kame,
>
> Are you sure about that?
>
> I just tried it myself, and it seems to work:
>
> root@inf5072-11:~/glommer-temporary/a/b# cat memory.kmem.tcp.usage_in_bytes
> 724992
> root@inf5072-11:~/glommer-temporary/a/b# cat
> ../memory.kmem.tcp.usage_in_bytes
> 0
>
>
> Did you got this conclusion through testing or code inspection?
>
> As a matter of fact, that's why I believe the current behavior is indeed
> correct:
>
> the res_counter is initialized as:
>
> parent_cg = tcp_prot.proto_cgroup(parent);
> if (parent_cg)
> res_parent = parent_cg->memory_allocated;
>
> res_counter_init(&tcp->tcp_memory_allocated, res_parent);
>
> now, parent is drawn from parent_mem_cgroup(), that reads as follows:
>
> struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *memcg)
> {
> if (!memcg->res.parent)
> return NULL;
> return mem_cgroup_from_res_counter(memcg->res.parent, res);
> }
>
>
> so if we have use_hierarchy = 0, res.parent should be NULL (because that
> is the way we initialize it)
>
Ah, sorry. you're right. please forget this patch.
To be honest, I wrote this after seeing WARN_ON in patch 2 and 3 and
misunderstood the code at writing patch 2 and 3.
Thanks,
-Kame
^ permalink raw reply
* Re: [BUGFIX][PATCH 3/3] memcg/tcp: ignore tcp usage before accounting started
From: Glauber Costa @ 2012-03-29 9:21 UTC (permalink / raw)
To: KAMEZAWA Hiroyuki; +Cc: netdev, David Miller, Andrew Morton
In-Reply-To: <4F740AEF.7090900@jp.fujitsu.com>
On 03/29/2012 09:10 AM, KAMEZAWA Hiroyuki wrote:
> tcp memcontrol starts accouting after res->limit is set. So, if a sockets
> starts before setting res->limit, there are already used resource.
> After setting res->limit, the resource (already used) will be uncharged and
> make res_counter below 0 because they are not charged. This causes warning.
>
> This patch fixes that by adding res_counter_uncharge_nowarn().
> (*) We cannot avoid this while we have 'account start' switch.
>
> Signed-off-by: KAMEZAWA Hiroyuki<kamezawa.hiroyu@jp.fujitsu.com>
Fine by me.
Acked-by: Glauber Costa <glommer@parallels.com>
^ permalink raw reply
* Re: maranello for linux-current [b43 driver]
From: Jim Cromie @ 2012-03-29 9:29 UTC (permalink / raw)
To: Florian Fainelli; +Cc: Bo Han, netdev, linux-wireless
In-Reply-To: <4F74265F.8060508@openwrt.org>
On Thu, Mar 29, 2012 at 3:07 AM, Florian Fainelli <florian@openwrt.org> wrote:
> (Adding linux-wireless in CC)
>
> Le 03/29/12 00:24, Jim Cromie a écrit :
>>
>> you may have heard of maranello, a project to add
>> partial packet recovery to 80211, by adding block checksums
>> on transmit, validating them or re-requesting retransmit
>> of corrupted blocks.
>>
>> http://www.cs.umd.edu/projects/maranello/
>>
>> the code at the page is for 2.6.29-rc2
>>
>> Ive taken it, did some refactoring to ease merging,
>> and merged in each linux release, up to 3.3
>>
I redid it, this time no refactoring, just conflict resolution
and compile fixes.
Now pushed to:
https://github.com/jimc/linux-2.6/tree/maranello/merges
>> So I guess Im seeking advice:
>>
>> - what did I break
>> - what happens when the driver is tested
Theres a lot to fix still.
- // comments
- long lines
- unused vars
- various FIXMEs
- refactorings for better reuse in other drivers
(this is large question, probably with lots of answers)
What drivers have FW whose source is available,
and thus hackable for maranello support ?
thanks.
Jim Cromie
^ permalink raw reply
* Re: [Q/RFC] BPF use in broader scope
From: Jiri Pirko @ 2012-03-29 9:31 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, netdev, bhutchings, shemminger, matt
In-Reply-To: <1333010732.2325.339.camel@edumazet-glaptop>
Thu, Mar 29, 2012 at 10:45:32AM CEST, eric.dumazet@gmail.com wrote:
>On Thu, 2012-03-29 at 10:31 +0200, Jiri Pirko wrote:
>> Thu, Mar 29, 2012 at 10:02:25AM CEST, eric.dumazet@gmail.com wrote:
>> >On Thu, 2012-03-29 at 09:54 +0200, Jiri Pirko wrote:
>> >
>> >> Yep, I'm aware. I must admit that the JIT code scares me a litte :(
>> >>
>> >
>> >If you add a new XOR instruction in interpreter only, JIT compiler will
>> >automatically aborts, so no risk.
>> >
>> >Each arch maintainer will add the support for the new instructions as
>> >separate patches.
>> >
>> >So you can focus on net/core/filter.c file only.
>> >
>>
>> Ok - I can do this for 2). But for 3) JITs need to be modified. So I
>> would like to kindly ask you and Matt if you can do this modification so
>> bpf_func takes pointer to mem (scratch store) as second parameter. I'm
>> sure it's very easy for you to do.
>
>I am not sure why you want this.
>
>This adds register pressure (at least for x86) ...
Well I think that there would become handy to be able to pass some data
to bpf_func (other than skb). But it's just an idea.
>
>
>
^ permalink raw reply
* [PATCH] fix a bug in emitting the 16-bit immediate operand of AND
From: zhuangfeiran @ 2012-03-29 9:27 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-kernel, eric.dumazet
When K >= 0xFFFF0000, AND needs the two least significant bytes of K as
its operand, but EMIT2() gives it the least significant byte of K and
0x2. EMIT() should be used here to replace EMIT2().
Signed-off-by: Feiran Zhuang <zhuangfeiran@ict.ac.cn>
---
arch/x86/net/bpf_jit_comp.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 5671752..5a5b6e4 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -289,7 +289,7 @@ void bpf_jit_compile(struct sk_filter *fp)
EMIT2(0x24, K & 0xFF); /* and imm8,%al */
} else if (K >= 0xFFFF0000) {
EMIT2(0x66, 0x25); /* and imm16,%ax */
- EMIT2(K, 2);
+ EMIT(K, 2);
} else {
EMIT1_off32(0x25, K); /* and imm32,%eax */
}
--
1.7.5.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox