* [RFC PATCH net-next 2/5] bpf: Refactor codes handling percpu map
From: Martin KaFai Lau @ 2016-10-02 3:58 UTC (permalink / raw)
To: netdev; +Cc: FB Kernel Team
In-Reply-To: <1475380713-4106095-1-git-send-email-kafai@fb.com>
Refactor the codes that populate the value
of a htab_elem in a BPF_MAP_TYPE_PERCPU_HASH
typed bpf_map.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
kernel/bpf/hashtab.c | 47 +++++++++++++++++++++--------------------------
1 file changed, 21 insertions(+), 26 deletions(-)
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 570eeca..a5e3915 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -420,6 +420,24 @@ static void free_htab_elem(struct bpf_htab *htab, struct htab_elem *l)
}
}
+static void pcpu_copy_value(struct bpf_htab *htab, void __percpu *pptr,
+ void *value, bool onallcpus)
+{
+ if (!onallcpus) {
+ /* copy true value_size bytes */
+ memcpy(this_cpu_ptr(pptr), value, htab->map.value_size);
+ } else {
+ u32 size = round_up(htab->map.value_size, 8);
+ int off = 0, cpu;
+
+ for_each_possible_cpu(cpu) {
+ bpf_long_memcpy(per_cpu_ptr(pptr, cpu),
+ value + off, size);
+ off += size;
+ }
+ }
+}
+
static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,
void *value, u32 key_size, u32 hash,
bool percpu, bool onallcpus,
@@ -479,18 +497,8 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,
}
}
- if (!onallcpus) {
- /* copy true value_size bytes */
- memcpy(this_cpu_ptr(pptr), value, htab->map.value_size);
- } else {
- int off = 0, cpu;
+ pcpu_copy_value(htab, pptr, value, onallcpus);
- for_each_possible_cpu(cpu) {
- bpf_long_memcpy(per_cpu_ptr(pptr, cpu),
- value + off, size);
- off += size;
- }
- }
if (!prealloc)
htab_elem_set_ptr(l_new, key_size, pptr);
} else {
@@ -606,22 +614,9 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key,
goto err;
if (l_old) {
- void __percpu *pptr = htab_elem_get_ptr(l_old, key_size);
- u32 size = htab->map.value_size;
-
/* per-cpu hash map can update value in-place */
- if (!onallcpus) {
- memcpy(this_cpu_ptr(pptr), value, size);
- } else {
- int off = 0, cpu;
-
- size = round_up(size, 8);
- for_each_possible_cpu(cpu) {
- bpf_long_memcpy(per_cpu_ptr(pptr, cpu),
- value + off, size);
- off += size;
- }
- }
+ pcpu_copy_value(htab, htab_elem_get_ptr(l_old, key_size),
+ value, onallcpus);
} else {
l_new = alloc_htab_elem(htab, key, value, key_size,
hash, true, onallcpus, false);
--
2.5.1
^ permalink raw reply related
* [RFC PATCH net-next 1/5] bpf: LRU List
From: Martin KaFai Lau @ 2016-10-02 3:58 UTC (permalink / raw)
To: netdev; +Cc: FB Kernel Team
Introduce bpf_lru_list which will provide LRU capability to
the bpf_htab in the later patch.
General Thoughts:
1. Target use case. Read is more often than update.
(i.e. bpf_lookup_elem() is more often than bpf_update_elem()).
If bpf_prog does a bpf_lookup_elem() first and then an in-place
update, it still counts as a read operation to the LRU list concern.
2. It may be useful to think of it as a LRU cache
3. Optimize the read case
3.1 No lock in read case
3.2 The LRU maintenance is only done during bpf_update_elem()
4. If there is a percpu LRU list, it will lose the system-wise LRU
property. A completely isolated percpu LRU list has the best
performance but the memory utilization is not ideal considering
the work load may be imbalance.
5. Hence, this patch starts the LRU implementation with a global LRU
list with batched operations before accessing the global LRU list.
6. There is a percpu list which is named 'struct bpf_lru_locallist' in
the code. The local list is to batch enough operations before
acquiring the lock of the global LRU list. More details on this later.
Global LRU List:
1. It has three sub-lists: active-list, inactive-list and free-list.
2. The two list idea, active and inactive, is borrowed from the
page cache.
3. All nodes are pre-allocated and all sit at the free-list (of the
global LRU list) at the beginning. The pre-allocated reasoning
is similar to the existing BPF_MAP_TYPE_HASH. However,
opting-out prealloc (BPF_F_NO_PREALLOC) is not supported in
the LRU map.
Active/Inactive List (of the global LRU list):
1. The active list, as its name says it, maintains the active set of
the nodes. We can think of it as the working set or more frequently
accessed nodes. The access frequency is approximated by a ref-bit.
The ref-bit is set during the bpf_lookup_elem().
2. The inactive list, as its name also says it, maintains a less
active set of nodes. They are the candidates to be removed
from the bpf_htab when we are running out of free nodes.
3. The ordering of these two lists is acting as a rough clock.
The tail of the inactive list is the older nodes and
should be released first if the bpf_htab needs free element.
Rotating the Active/Inactive List (of the global LRU list):
1. It is the basic operation to maintain the LRU property of
the global list.
2. The active list is only rotated when the inactive list is running
low. This idea is similar to the current page cache.
Inactive running low is currently defined as
"# of inactive < # of active".
3. The active list rotation always starts from the tail. It moves
node without ref-bit set to the head of the inactive list.
It moves node with ref-bit set back to the head of the active
list and then clears its ref-bit.
4. The inactive rotation is pretty simply.
It walks the inactive list and moves the nodes back to the head of
active list if its ref-bit is set. The ref-bit is cleared after moving
to the active list.
If the node does not have ref-bit set, it just leave it as it is
because it is already in the inactive list.
Shrinking the Inactive List (of the global LRU list):
1. Shrinking is the operation to get free nodes when the bpf_htab is
full.
2. It usually only shrinks the inactive list to get free nodes.
3. During shrinking, it will walk the inactive list from the tail,
delete the nodes without ref-bit set from bpf_htab.
4. If no free node found after step (3), it will forcefully get
one node from the tail of inactive or active list. Forcefully is
in the sense that it ignores the ref-bit.
Local List:
1. Each CPU has a 'struct bpf_lru_locallist'. The purpose is to
batch enough operations before acquiring the lock of the
global LRU.
2. A local list has two sub-lists, free-list and pending-list.
3. During bpf_update_elem(), it will try to get from the free-list
of (the current CPU local list).
4. If the local free-list is empty, it will acquire from the
global LRU list. The global LRU list can either satisfy it
by its global free-list or by shrinking the global inactive
list. Since we have acquired the global LRU list lock,
it will try to get at most LOCAL_FREE_TARGET elements
to the local free list.
5. When a new element is added to the bpf_htab, it will
first sit at the pending-list (of the local list) first.
The pending-list will be flushed to the global LRU list
when it needs to acquire free nodes from the global list
next time.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
kernel/bpf/Makefile | 2 +-
kernel/bpf/bpf_lru_list.c | 585 ++++++++++++++++++++++++++++++++++++++++++++++
kernel/bpf/bpf_lru_list.h | 74 ++++++
3 files changed, 660 insertions(+), 1 deletion(-)
create mode 100644 kernel/bpf/bpf_lru_list.c
create mode 100644 kernel/bpf/bpf_lru_list.h
diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
index eed911d..c4d89d6 100644
--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -1,7 +1,7 @@
obj-y := core.o
obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o
-obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o
+obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o
ifeq ($(CONFIG_PERF_EVENTS),y)
obj-$(CONFIG_BPF_SYSCALL) += stackmap.o
endif
diff --git a/kernel/bpf/bpf_lru_list.c b/kernel/bpf/bpf_lru_list.c
new file mode 100644
index 0000000..4581872
--- /dev/null
+++ b/kernel/bpf/bpf_lru_list.c
@@ -0,0 +1,585 @@
+/* Copyright (c) 2016 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General Public
+ * License as published by the Free Software Foundation.
+ */
+#include <linux/cpumask.h>
+#include <linux/spinlock.h>
+#include <linux/percpu.h>
+
+#include "bpf_lru_list.h"
+
+/* The max number of elements to process when walking a list */
+#define MAX_WALKS (128)
+
+/* Whenever we acquire the global LRU's lock to get free nodes,
+ * we try to get LOCAL_FREE_TARGET number of free elements to
+ * the local list.
+ */
+#define LOCAL_FREE_TARGET (128)
+
+/* Helpers to get the local list index */
+#define LOCAL_LIST_IDX(t) ((t) - BPF_LOCAL_LIST_T_OFFSET)
+#define LOCAL_FREE_LIST_IDX LOCAL_LIST_IDX(BPF_LRU_LOCAL_LIST_T_FREE)
+#define LOCAL_PENDING_LIST_IDX LOCAL_LIST_IDX(BPF_LRU_LOCAL_LIST_T_PENDING)
+#define IS_LOCAL_LIST_TYPE(t) ((t) >= BPF_LOCAL_LIST_T_OFFSET)
+
+static int get_next_cpu(int cpu)
+{
+ cpu = cpumask_next(cpu, cpu_possible_mask);
+ if (cpu >= nr_cpu_ids)
+ cpu = cpumask_first(cpu_possible_mask);
+ return cpu;
+}
+
+/* Local list helpers */
+static struct list_head *local_free_list(struct bpf_lru_locallist *loc_l)
+{
+ return &loc_l->lists[LOCAL_FREE_LIST_IDX];
+}
+
+static struct list_head *local_pending_list(struct bpf_lru_locallist *loc_l)
+{
+ return &loc_l->lists[LOCAL_PENDING_LIST_IDX];
+}
+
+/* bpf_lru_node helpers */
+static bool bpf_lru_node_is_ref(const struct bpf_lru_node *node)
+{
+ return node->ref;
+}
+
+static void bpf_lru_list_count_inc(struct bpf_lru_list *l,
+ enum bpf_lru_list_type type)
+{
+ if (type < NR_BPF_LRU_LIST_COUNT)
+ l->counts[type]++;
+}
+
+static void bpf_lru_list_count_dec(struct bpf_lru_list *l,
+ enum bpf_lru_list_type type)
+{
+ if (type < NR_BPF_LRU_LIST_COUNT)
+ l->counts[type]--;
+}
+
+/* Move nodes out of the global LRU list to the local free list.
+ *
+ * @l: The global LRU list
+ * @loc_l: The local list
+ * @node: The node which is already in the global LRU list
+ */
+static void __bpf_lru_node_move_out(struct bpf_lru_list *l,
+ struct bpf_lru_locallist *loc_l,
+ struct bpf_lru_node *node)
+{
+ if (WARN_ON_ONCE(IS_LOCAL_LIST_TYPE(node->type)))
+ return;
+
+ /* If the removing node is the next_inactive_rotation candidate,
+ * move the next_inactive_rotation pointer also.
+ */
+ if (&node->list == l->next_inactive_rotation)
+ l->next_inactive_rotation = l->next_inactive_rotation->prev;
+
+ bpf_lru_list_count_dec(l, node->type);
+
+ node->type = BPF_LRU_LOCAL_LIST_T_FREE;
+ node->cpu = loc_l->cpu;
+ list_move(&node->list, local_free_list(loc_l));
+}
+
+/* Move nodes from local list to the global LRU list
+ *
+ * @l: The global LRU list
+ * @node: The node which is already in the local list
+ * @tgt_type: The target list type
+ */
+static void __bpf_lru_node_move_in(struct bpf_lru_list *l,
+ struct bpf_lru_node *node,
+ enum bpf_lru_list_type tgt_type)
+{
+ if (WARN_ON_ONCE(!IS_LOCAL_LIST_TYPE(node->type)) ||
+ WARN_ON_ONCE(IS_LOCAL_LIST_TYPE(tgt_type)))
+ return;
+
+ bpf_lru_list_count_inc(l, tgt_type);
+ node->type = tgt_type;
+ node->ref = 0;
+ list_move(&node->list, &l->lists[tgt_type]);
+}
+
+/* Move nodes between or within active and inactive list (like
+ * active to inactive, inactive to active or tail of active back to
+ * the head of active).
+ *
+ * @l: The global LRU list
+ * @node: A node which is already in the global LRU list
+ * @tgt_type: The target list type
+ */
+static void __bpf_lru_node_move(struct bpf_lru_list *l,
+ struct bpf_lru_node *node,
+ enum bpf_lru_list_type tgt_type)
+{
+ if (WARN_ON_ONCE(IS_LOCAL_LIST_TYPE(node->type)) ||
+ WARN_ON_ONCE(IS_LOCAL_LIST_TYPE(tgt_type)))
+ return;
+
+ if (node->type != tgt_type) {
+ bpf_lru_list_count_dec(l, node->type);
+ bpf_lru_list_count_inc(l, tgt_type);
+ node->type = tgt_type;
+ }
+ node->ref = 0;
+
+ /* If the moving node is the next_inactive_rotation candidate,
+ * move the next_inactive_rotation pointer also.
+ */
+ if (&node->list == l->next_inactive_rotation)
+ l->next_inactive_rotation = l->next_inactive_rotation->prev;
+
+ list_move(&node->list, &l->lists[tgt_type]);
+}
+
+static bool bpf_lru_list_inactive_low(const struct bpf_lru_list *l)
+{
+ return l->counts[BPF_LRU_LIST_T_INACTIVE] <
+ l->counts[BPF_LRU_LIST_T_ACTIVE];
+}
+
+/* Rotate the active list (of the global LRU list):
+ * 1. Start from tail
+ * 2. If the node has the ref bit set, it will be rotated
+ * back to the head of active list with the ref bit cleared.
+ * Give this node one more chance to survive in the active list.
+ * 3. If the ref bit is not set, move it to the head of the
+ * inactive list.
+ * 4. It will at most scan MAX_WALKS nodes
+ */
+static void __bpf_lru_list_rotate_active(struct bpf_lru_list *l)
+{
+ struct list_head *active = &l->lists[BPF_LRU_LIST_T_ACTIVE];
+ struct bpf_lru_node *node, *tmp_node, *first_node;
+ int i = 0;
+
+ first_node = list_first_entry(active, struct bpf_lru_node, list);
+ list_for_each_entry_safe_reverse(node, tmp_node, active, list) {
+ if (bpf_lru_node_is_ref(node))
+ __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_ACTIVE);
+ else
+ __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_INACTIVE);
+
+ if (++i == MAX_WALKS || node == first_node)
+ break;
+ }
+}
+
+/* Rotate the inactive list (of the global LRU list). It starts from the
+ * next_inactive_rotation.
+ * 1. If the node has ref bit set, it will be moved to the head
+ * of active list with the ref bit cleared.
+ * 2. If the node does not have ref bit set, it will leave it
+ * at its current location (i.e. do nothing) so that it can
+ * be considered during the next inactive_shrink.
+ * 3. It will at most scan MAX_WALKS nodes.
+ */
+static void __bpf_lru_list_rotate_inactive(struct bpf_lru_list *l)
+{
+ struct list_head *inactive = &l->lists[BPF_LRU_LIST_T_INACTIVE];
+ struct list_head *cur, *next, *last;
+ struct bpf_lru_node *node;
+ unsigned int i = 0;
+
+ if (list_empty(inactive))
+ return;
+
+ last = l->next_inactive_rotation->next;
+ if (last == inactive)
+ last = last->next;
+
+ cur = l->next_inactive_rotation;
+ while (i < MAX_WALKS) {
+ if (cur == inactive) {
+ cur = cur->prev;
+ continue;
+ }
+
+ node = list_entry(cur, struct bpf_lru_node, list);
+ next = cur->prev;
+ if (bpf_lru_node_is_ref(node))
+ __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_ACTIVE);
+ if (cur == last)
+ break;
+ cur = next;
+ i++;
+ }
+
+ l->next_inactive_rotation = next;
+}
+
+/* Shrink the inactive list (of the global LRU list)
+ * It starts from the tail of the inactive list and only
+ * move the nodes without the ref bit set to the
+ * free list (of the percpu _local_ list).
+ */
+static unsigned int __bpf_lru_list_shrink_inactive(struct bpf_lru *lru,
+ struct bpf_lru_locallist *loc_l,
+ unsigned int tgt_nshrink)
+{
+ struct bpf_lru_list *l = &lru->lru_list;
+ struct list_head *inactive = &l->lists[BPF_LRU_LIST_T_INACTIVE];
+ struct bpf_lru_node *node, *tmp_node, *first_node;
+ unsigned int nshrinked = 0;
+ int i = 0;
+
+ first_node = list_first_entry(inactive, struct bpf_lru_node, list);
+ list_for_each_entry_safe_reverse(node, tmp_node, inactive, list) {
+ if (bpf_lru_node_is_ref(node))
+ __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_ACTIVE);
+ else if (lru->del_from_htab(lru->del_arg, node)) {
+ __bpf_lru_node_move_out(l, loc_l, node);
+ if (++nshrinked == tgt_nshrink)
+ break;
+ }
+
+ if (++i == MAX_WALKS)
+ break;
+ }
+
+ return nshrinked;
+}
+
+/* 1. Rotate the active list (if needed)
+ * 2. Always rotate the inactive list
+ */
+static void __bpf_lru_list_rotate(struct bpf_lru_list *l)
+{
+ if (bpf_lru_list_inactive_low(l))
+ __bpf_lru_list_rotate_active(l);
+
+ __bpf_lru_list_rotate_inactive(l);
+}
+
+/* Calls __bpf_lru_list_shrink_inactive() to shrink some
+ * ref-bit-cleared nodes and move them to the
+ * free list (of a _local_ list).
+ *
+ * If the _local_ list has no free nodes after calling
+ * __bpf_lru_list_shrink_inactive(). It will just remove
+ * one node from either inactive or active list without
+ * honoring the ref-bit. It prefers inactive list to active
+ * list in this situation.
+ */
+static unsigned int __bpf_lru_list_shrink(struct bpf_lru *lru,
+ struct bpf_lru_locallist *loc_l,
+ unsigned int tgt_nshrink)
+{
+ struct bpf_lru_list *l = &lru->lru_list;
+ struct bpf_lru_node *node, *tmp_node;
+ struct list_head *force_shrink_list;
+ unsigned int nshrinked;
+
+ nshrinked = __bpf_lru_list_shrink_inactive(lru, loc_l, tgt_nshrink);
+ if (nshrinked)
+ return nshrinked;
+
+ /* Do a force shrink by ignoring the reference bit */
+ if (!list_empty(&l->lists[BPF_LRU_LIST_T_INACTIVE]))
+ force_shrink_list = &l->lists[BPF_LRU_LIST_T_INACTIVE];
+ else
+ force_shrink_list = &l->lists[BPF_LRU_LIST_T_ACTIVE];
+
+ list_for_each_entry_safe_reverse(node, tmp_node, force_shrink_list,
+ list) {
+ if (lru->del_from_htab(lru->del_arg, node)) {
+ __bpf_lru_node_move_out(l, loc_l, node);
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+static unsigned int __bpf_lru_list_get_free(struct bpf_lru_list *l,
+ struct bpf_lru_locallist *loc_l,
+ unsigned int tgt_nfree)
+{
+ struct bpf_lru_node *node, *tmp_node;
+ unsigned int nfree = 0;
+
+ list_for_each_entry_safe(node, tmp_node, &l->lists[BPF_LRU_LIST_T_FREE],
+ list) {
+ __bpf_lru_node_move_out(l, loc_l, node);
+
+ if (++nfree == tgt_nfree)
+ break;
+ }
+
+ return nfree;
+}
+
+static void __local_list_flush(struct bpf_lru_list *l,
+ struct bpf_lru_locallist *loc_l)
+{
+ struct bpf_lru_node *node, *tmp_node;
+
+ list_for_each_entry_safe_reverse(node, tmp_node,
+ local_pending_list(loc_l), list) {
+ if (bpf_lru_node_is_ref(node))
+ __bpf_lru_node_move_in(l, node, BPF_LRU_LIST_T_ACTIVE);
+ else
+ __bpf_lru_node_move_in(l, node,
+ BPF_LRU_LIST_T_INACTIVE);
+ }
+}
+
+static void bpf_lru_list_push_free(struct bpf_lru_list *l,
+ struct bpf_lru_node *node)
+{
+ unsigned long flags;
+
+ if (WARN_ON_ONCE(IS_LOCAL_LIST_TYPE(node->type)))
+ return;
+
+ raw_spin_lock_irqsave(&l->lock, flags);
+ __bpf_lru_node_move(l, node, BPF_LRU_LIST_T_FREE);
+ raw_spin_unlock_irqrestore(&l->lock, flags);
+}
+
+static void bpf_lru_list_pop_free(struct bpf_lru *lru,
+ struct bpf_lru_locallist *loc_l)
+{
+ struct bpf_lru_list *l = &lru->lru_list;
+ unsigned int nfree;
+
+ raw_spin_lock(&l->lock);
+
+ __local_list_flush(l, loc_l);
+
+ __bpf_lru_list_rotate(l);
+
+ nfree = __bpf_lru_list_get_free(l, loc_l, LOCAL_FREE_TARGET);
+ if (nfree < LOCAL_FREE_TARGET)
+ __bpf_lru_list_shrink(lru, loc_l, LOCAL_FREE_TARGET - nfree);
+
+ raw_spin_unlock(&l->lock);
+}
+
+static void __local_list_add_pending(struct bpf_lru *lru,
+ struct bpf_lru_locallist *loc_l,
+ struct bpf_lru_node *node,
+ u32 hash)
+{
+ *(u32 *)((void *)node + lru->hash_offset) = hash;
+ node->type = BPF_LRU_LOCAL_LIST_T_PENDING;
+ node->ref = 0;
+ list_add(&node->list, local_pending_list(loc_l));
+}
+
+struct bpf_lru_node *__local_list_pop_free(struct bpf_lru_locallist *loc_l)
+{
+ struct bpf_lru_node *node;
+
+ node = list_first_entry_or_null(local_free_list(loc_l),
+ struct bpf_lru_node,
+ list);
+ if (node)
+ list_del(&node->list);
+
+ return node;
+}
+
+struct bpf_lru_node *__local_list_pop_pending(struct bpf_lru *lru,
+ struct bpf_lru_locallist *loc_l)
+{
+ struct bpf_lru_node *node;
+ bool force = false;
+
+ignore_ref:
+ /* Get from the tail (i.e. older element) of the pending list. */
+ list_for_each_entry_reverse(node, local_pending_list(loc_l),
+ list) {
+ if ((!bpf_lru_node_is_ref(node) || force) &&
+ lru->del_from_htab(lru->del_arg, node)) {
+ list_del(&node->list);
+ return node;
+ }
+ }
+
+ if (!force) {
+ force = true;
+ goto ignore_ref;
+ }
+
+ return NULL;
+}
+
+struct bpf_lru_node *bpf_lru_pop_free(struct bpf_lru *lru, u32 hash)
+{
+ struct bpf_lru_locallist *loc_l, *steal_loc_l;
+ struct bpf_lru_node *node;
+ int steal, first_steal;
+ unsigned long flags;
+
+ loc_l = this_cpu_ptr(lru->local_list);
+
+ raw_spin_lock_irqsave(&loc_l->lock, flags);
+
+ node = __local_list_pop_free(loc_l);
+ if (!node) {
+ bpf_lru_list_pop_free(lru, loc_l);
+ node = __local_list_pop_free(loc_l);
+ }
+
+ if (node)
+ __local_list_add_pending(lru, loc_l, node, hash);
+
+ raw_spin_unlock_irqrestore(&loc_l->lock, flags);
+
+ if (node)
+ return node;
+
+ /* No free nodes found from the local free list and
+ * the global LRU list.
+ *
+ * Steal from the local free/pending list of the
+ * current CPU and remote CPU in RR. It starts
+ * with the loc_l->next_steal CPU.
+ */
+
+ first_steal = loc_l->next_steal;
+ steal = first_steal;
+ do {
+ steal_loc_l = per_cpu_ptr(lru->local_list, steal);
+
+ raw_spin_lock_irqsave(&steal_loc_l->lock, flags);
+
+ node = __local_list_pop_free(steal_loc_l);
+ if (!node)
+ node = __local_list_pop_pending(lru, steal_loc_l);
+
+ raw_spin_unlock_irqrestore(&steal_loc_l->lock, flags);
+
+ steal = get_next_cpu(steal);
+ } while (!node && steal != first_steal);
+
+ loc_l->next_steal = steal;
+
+ if (node) {
+ raw_spin_lock_irqsave(&loc_l->lock, flags);
+ __local_list_add_pending(lru, loc_l, node, hash);
+ raw_spin_unlock_irqrestore(&loc_l->lock, flags);
+ }
+
+ return node;
+}
+
+void bpf_lru_push_free(struct bpf_lru *lru, struct bpf_lru_node *node)
+{
+ unsigned long flags;
+
+ if (WARN_ON_ONCE(node->type == BPF_LRU_LIST_T_FREE) ||
+ WARN_ON_ONCE(node->type == BPF_LRU_LOCAL_LIST_T_FREE))
+ return;
+
+ if (node->type == BPF_LRU_LOCAL_LIST_T_PENDING) {
+ struct bpf_lru_locallist *loc_l;
+
+ loc_l = per_cpu_ptr(lru->local_list, node->cpu);
+
+ raw_spin_lock_irqsave(&loc_l->lock, flags);
+
+ if (unlikely(node->type != BPF_LRU_LOCAL_LIST_T_PENDING)) {
+ raw_spin_unlock_irqrestore(&loc_l->lock, flags);
+ goto check_lru_list;
+ }
+
+ node->type = BPF_LRU_LOCAL_LIST_T_FREE;
+ node->ref = 0;
+ list_move(&node->list, local_free_list(loc_l));
+
+ raw_spin_unlock_irqrestore(&loc_l->lock, flags);
+ return;
+ }
+
+check_lru_list:
+ bpf_lru_list_push_free(&lru->lru_list, node);
+}
+
+void bpf_lru_populate(struct bpf_lru *lru, void *buf, u32 node_offset,
+ u32 elem_size, u32 nr_elems)
+{
+ struct bpf_lru_list *l = &lru->lru_list;
+ u32 i;
+
+ for (i = 0; i < nr_elems; i++) {
+ struct bpf_lru_node *node;
+
+ node = (struct bpf_lru_node *)(buf + node_offset);
+ node->type = BPF_LRU_LIST_T_FREE;
+ node->ref = 0;
+ list_add(&node->list, &l->lists[BPF_LRU_LIST_T_FREE]);
+ buf += elem_size;
+ }
+}
+
+static void bpf_lru_locallist_init(struct bpf_lru_locallist *loc_l, int cpu)
+{
+ int i;
+
+ for (i = 0; i < NR_BPF_LRU_LOCAL_LIST_T; i++)
+ INIT_LIST_HEAD(&loc_l->lists[i]);
+
+ loc_l->cpu = cpu;
+ loc_l->next_steal = cpu;
+
+ raw_spin_lock_init(&loc_l->lock);
+}
+
+static void bpf_lru_list_init(struct bpf_lru_list *l)
+{
+ int i;
+
+ for (i = 0; i < NR_BPF_LRU_LIST_T; i++)
+ INIT_LIST_HEAD(&l->lists[i]);
+
+ for (i = 0; i < NR_BPF_LRU_LIST_COUNT; i++)
+ l->counts[i] = 0;
+
+ l->next_inactive_rotation = &l->lists[BPF_LRU_LIST_T_INACTIVE];
+
+ raw_spin_lock_init(&l->lock);
+}
+
+int bpf_lru_init(struct bpf_lru *lru,
+ u32 hash_offset,
+ del_from_htab_func del_from_htab,
+ void *del_arg)
+{
+ int cpu;
+
+ lru->local_list = alloc_percpu(struct bpf_lru_locallist);
+ if (!lru->local_list)
+ return -ENOMEM;
+
+ for_each_possible_cpu(cpu) {
+ struct bpf_lru_locallist *loc_l;
+
+ loc_l = per_cpu_ptr(lru->local_list, cpu);
+ bpf_lru_locallist_init(loc_l, cpu);
+ }
+
+ bpf_lru_list_init(&lru->lru_list);
+
+ lru->del_from_htab = del_from_htab;
+ lru->del_arg = del_arg;
+ lru->hash_offset = hash_offset;
+
+ return 0;
+}
+
+void bpf_lru_destroy(struct bpf_lru *lru)
+{
+ free_percpu(lru->local_list);
+}
diff --git a/kernel/bpf/bpf_lru_list.h b/kernel/bpf/bpf_lru_list.h
new file mode 100644
index 0000000..dfe46cd
--- /dev/null
+++ b/kernel/bpf/bpf_lru_list.h
@@ -0,0 +1,74 @@
+/* Copyright (c) 2016 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General Public
+ * License as published by the Free Software Foundation.
+ */
+#ifndef __BPF_LRU_LIST_H_
+#define __BPF_LRU_LIST_H_
+
+#include <linux/list.h>
+#include <linux/spinlock_types.h>
+
+#define NR_BPF_LRU_LIST_T (3)
+#define NR_BPF_LRU_LIST_COUNT (2)
+#define NR_BPF_LRU_LOCAL_LIST_T (2)
+#define BPF_LOCAL_LIST_T_OFFSET NR_BPF_LRU_LIST_T
+
+enum bpf_lru_list_type {
+ BPF_LRU_LIST_T_ACTIVE,
+ BPF_LRU_LIST_T_INACTIVE,
+ BPF_LRU_LIST_T_FREE,
+ BPF_LRU_LOCAL_LIST_T_FREE,
+ BPF_LRU_LOCAL_LIST_T_PENDING,
+};
+
+struct bpf_lru_node {
+ struct list_head list;
+ u16 cpu;
+ u8 type;
+ u8 ref;
+};
+
+struct bpf_lru_list {
+ struct list_head lists[NR_BPF_LRU_LIST_T];
+ unsigned int counts[BPF_LRU_LIST_T_FREE];
+ /* The next inacitve list rotation starts from here */
+ struct list_head *next_inactive_rotation;
+ raw_spinlock_t lock ____cacheline_aligned_in_smp;
+};
+
+struct bpf_lru_locallist {
+ struct list_head lists[NR_BPF_LRU_LOCAL_LIST_T];
+ u16 cpu;
+ u16 next_steal;
+ raw_spinlock_t lock;
+};
+
+typedef bool (*del_from_htab_func)(void *arg, struct bpf_lru_node *node);
+
+struct bpf_lru {
+ struct bpf_lru_list lru_list;
+ struct bpf_lru_locallist __percpu *local_list;
+ del_from_htab_func del_from_htab;
+ void *del_arg;
+ unsigned int hash_offset;
+};
+
+static inline void bpf_lru_node_set_ref(struct bpf_lru_node *node)
+{
+ node->ref = 1;
+}
+
+int bpf_lru_init(struct bpf_lru *lru,
+ u32 hash,
+ del_from_htab_func del_from_htab,
+ void *delete_arg);
+void bpf_lru_populate(struct bpf_lru *lru, void *buf, u32 node_offset,
+ u32 elem_size, u32 nr_elems);
+void bpf_lru_destroy(struct bpf_lru *lru);
+struct bpf_lru_node *bpf_lru_pop_free(struct bpf_lru *lru, u32 hash);
+void bpf_lru_push_free(struct bpf_lru *lru, struct bpf_lru_node *node);
+void bpf_lru_promote(struct bpf_lru *lru, struct bpf_lru_node *node);
+
+#endif
--
2.5.1
^ permalink raw reply related
* [RFC PATCH net-next 5/5] bpf: Add tests for the LRU bpf_htab
From: Martin KaFai Lau @ 2016-10-02 3:58 UTC (permalink / raw)
To: netdev; +Cc: FB Kernel Team
In-Reply-To: <1475380713-4106095-1-git-send-email-kafai@fb.com>
This patch has some unit tests and a test_lru_dist.
The test_lru_dist reads in the numeric keys from a file.
The files used here are generated by a modified fio-genzipf tool
originated from the fio test suit. The sample data file can be
found here: https://github.com/iamkafai/bpf-lru
The zipf.* data files have 100k numeric keys and the key is also
ranged from 1 to 100k.
The test_lru_dist outputs the number of unique keys (nr_unique).
F.e. The following means, 61239 of them is unique out of 100k keys.
nr_misses means it cannot be found in the LRU map, so nr_misses
must be >= nr_unique.
[root@arch-vm1 ~]# ./test_lru_dist zipf.100k.a0_01.out 40000
task:0 do_test_lru_dist:......
task:0 BPF LRU: nr_unique:61239(/100000) nr_misses:67054(/100000)
task:0 Perfect LRU: nr_unique:61239(/100000 nr_misses:66993(/100000)
[root@arch-vm1 ~]# ./test_lru_dist zipf.100k.a1_01.out 4000
task:0 BPF LRU: nr_unique:23093(/100000) nr_misses:31603(/100000)
task:0 Perfect LRU: nr_unique:23093(/100000 nr_misses:34328(/100000)
test_lru_dist also simulates a perfect LRU map as a comparison.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
samples/bpf/Makefile | 4 +-
samples/bpf/map_perf_test_kern.c | 20 ++
samples/bpf/map_perf_test_user.c | 17 +
samples/bpf/test_lru_dist.c | 322 ++++++++++++++++++
samples/bpf/test_lru_map.c | 690 +++++++++++++++++++++++++++++++++++++++
5 files changed, 1052 insertions(+), 1 deletion(-)
create mode 100644 samples/bpf/test_lru_dist.c
create mode 100644 samples/bpf/test_lru_map.c
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 12b7304..55052af 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -2,7 +2,7 @@
obj- := dummy.o
# List of programs to build
-hostprogs-y := test_verifier test_maps
+hostprogs-y := test_verifier test_maps test_lru_map test_lru_dist
hostprogs-y += sock_example
hostprogs-y += fds_example
hostprogs-y += sockex1
@@ -30,6 +30,8 @@ hostprogs-y += sampleip
test_verifier-objs := test_verifier.o libbpf.o
test_maps-objs := test_maps.o libbpf.o
+test_lru_map-objs := test_lru_map.o libbpf.o
+test_lru_dist-objs := test_lru_dist.o libbpf.o
sock_example-objs := sock_example.o libbpf.o
fds_example-objs := bpf_load.o libbpf.o fds_example.o
sockex1-objs := bpf_load.o libbpf.o sockex1_user.o
diff --git a/samples/bpf/map_perf_test_kern.c b/samples/bpf/map_perf_test_kern.c
index 311538e..abb11af 100644
--- a/samples/bpf/map_perf_test_kern.c
+++ b/samples/bpf/map_perf_test_kern.c
@@ -19,6 +19,13 @@ struct bpf_map_def SEC("maps") hash_map = {
.max_entries = MAX_ENTRIES,
};
+struct bpf_map_def SEC("maps") lru_hash_map = {
+ .type = BPF_MAP_TYPE_LRU_HASH,
+ .key_size = sizeof(u32),
+ .value_size = sizeof(long),
+ .max_entries = 10000,
+};
+
struct bpf_map_def SEC("maps") percpu_hash_map = {
.type = BPF_MAP_TYPE_PERCPU_HASH,
.key_size = sizeof(u32),
@@ -53,6 +60,7 @@ int stress_hmap(struct pt_regs *ctx)
value = bpf_map_lookup_elem(&hash_map, &key);
if (value)
bpf_map_delete_elem(&hash_map, &key);
+
return 0;
}
@@ -96,5 +104,17 @@ int stress_percpu_hmap_alloc(struct pt_regs *ctx)
bpf_map_delete_elem(&percpu_hash_map_alloc, &key);
return 0;
}
+
+SEC("kprobe/sys_getpid")
+int stress_lru_hmap_alloc(struct pt_regs *ctx)
+{
+ u32 key = bpf_get_prandom_u32();
+ long val = 1;
+
+ bpf_map_update_elem(&lru_hash_map, &key, &val, BPF_ANY);
+
+ return 0;
+}
+
char _license[] SEC("license") = "GPL";
u32 _version SEC("version") = LINUX_VERSION_CODE;
diff --git a/samples/bpf/map_perf_test_user.c b/samples/bpf/map_perf_test_user.c
index 3147377..db7fa74 100644
--- a/samples/bpf/map_perf_test_user.c
+++ b/samples/bpf/map_perf_test_user.c
@@ -35,6 +35,7 @@ static __u64 time_get_ns(void)
#define PERCPU_HASH_PREALLOC (1 << 1)
#define HASH_KMALLOC (1 << 2)
#define PERCPU_HASH_KMALLOC (1 << 3)
+#define LRU_HASH_PREALLOC (1 << 4)
static int test_flags = ~0;
@@ -50,6 +51,19 @@ static void test_hash_prealloc(int cpu)
cpu, MAX_CNT * 1000000000ll / (time_get_ns() - start_time));
}
+static void test_lru_hash_prealloc(int cpu)
+{
+ __u64 start_time;
+ int i;
+
+ start_time = time_get_ns();
+ for (i = 0; i < MAX_CNT; i++)
+ syscall(__NR_getpid);
+ printf("%d:lru_hash_map_perf pre-alloc %lld events per sec\n",
+ cpu, MAX_CNT * 1000000000ll / (time_get_ns() - start_time));
+}
+
+
static void test_percpu_hash_prealloc(int cpu)
{
__u64 start_time;
@@ -105,6 +119,9 @@ static void loop(int cpu)
if (test_flags & PERCPU_HASH_KMALLOC)
test_percpu_hash_kmalloc(cpu);
+
+ if (test_flags & LRU_HASH_PREALLOC)
+ test_lru_hash_prealloc(cpu);
}
static void run_perf_test(int tasks)
diff --git a/samples/bpf/test_lru_dist.c b/samples/bpf/test_lru_dist.c
new file mode 100644
index 0000000..376e14c
--- /dev/null
+++ b/samples/bpf/test_lru_dist.c
@@ -0,0 +1,322 @@
+/*
+ * Copyright (c) 2016 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General Public
+ * License as published by the Free Software Foundation.
+ */
+#define _GNU_SOURCE
+#include <linux/types.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <linux/bpf.h>
+#include <errno.h>
+#include <string.h>
+#include <assert.h>
+#include <sched.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdlib.h>
+#include <time.h>
+#include "libbpf.h"
+
+#define offsetof(TYPE, MEMBER) ((size_t)&((TYPE *)0)->MEMBER)
+
+#define container_of(ptr, type, member) ({ \
+ const typeof( ((type *)0)->member ) *__mptr = (ptr); \
+ (type *)( (char *)__mptr - offsetof(type,member) );})
+
+struct list_head {
+ struct list_head *next, *prev;
+};
+
+static inline void INIT_LIST_HEAD(struct list_head *list)
+{
+ list->next = list;
+ list->prev = list;
+}
+
+static inline int list_empty(const struct list_head *head)
+{
+ return head->next == head;
+}
+
+static inline void __list_add(struct list_head *new,
+ struct list_head *prev,
+ struct list_head *next)
+{
+ next->prev = new;
+ new->next = next;
+ new->prev = prev;
+ prev->next = new;
+}
+
+static inline void list_add(struct list_head *new, struct list_head *head)
+{
+ __list_add(new, head, head->next);
+}
+
+static inline void __list_del(struct list_head *prev, struct list_head *next)
+{
+ next->prev = prev;
+ prev->next = next;
+}
+
+static inline void __list_del_entry(struct list_head *entry)
+{
+ __list_del(entry->prev, entry->next);
+}
+
+static inline void list_move(struct list_head *list, struct list_head *head)
+{
+ __list_del_entry(list);
+ list_add(list, head);
+}
+
+#define list_entry(ptr, type, member) \
+ container_of(ptr, type, member)
+
+
+#define list_last_entry(ptr, type, member) \
+ list_entry((ptr)->prev, type, member)
+
+struct pfect_lru_node {
+ struct list_head list;
+ unsigned long long key;
+};
+
+struct pfect_lru {
+ struct list_head list;
+ struct pfect_lru_node *free_nodes;
+ unsigned int cur_size;
+ unsigned int lru_size;
+ unsigned int nr_unique;
+ unsigned int nr_misses;
+ unsigned int total;
+ int map_fd;
+};
+
+static void pfect_lru_init(struct pfect_lru *lru, unsigned int lru_size,
+ unsigned int nr_possible_elems)
+{
+ lru->map_fd = bpf_create_map(BPF_MAP_TYPE_HASH,
+ sizeof(unsigned long long),
+ sizeof(struct pfect_lru_node *),
+ nr_possible_elems, 0);
+ assert(lru->map_fd != -1);
+
+ lru->free_nodes = malloc(lru_size * sizeof(struct pfect_lru_node));
+ assert(lru->free_nodes);
+
+ INIT_LIST_HEAD(&lru->list);
+ lru->cur_size = 0;
+ lru->lru_size = lru_size;
+ lru->nr_unique = lru->nr_misses = lru->total = 0;
+}
+
+static void pfect_lru_destroy(struct pfect_lru *lru)
+{
+ close(lru->map_fd);
+ free(lru->free_nodes);
+}
+
+static int pfect_lru_lookup_or_insert(struct pfect_lru *lru,
+ unsigned long long key)
+{
+ struct pfect_lru_node *node = NULL;
+ int seen = 0;
+
+ lru->total++;
+ if (!bpf_lookup_elem(lru->map_fd, &key, &node)) {
+ if (node) {
+ list_move(&node->list, &lru->list);
+ return 1;
+ }
+ seen = 1;
+ }
+
+ if (lru->cur_size < lru->lru_size) {
+ node = &lru->free_nodes[lru->cur_size++];
+ INIT_LIST_HEAD(&node->list);
+ } else {
+ struct pfect_lru_node *null_node = NULL;
+
+ node = list_last_entry(&lru->list,
+ struct pfect_lru_node,
+ list);
+ bpf_update_elem(lru->map_fd, &node->key, &null_node, BPF_EXIST);
+ }
+
+ node->key = key;
+ list_move(&node->list, &lru->list);
+
+ lru->nr_misses++;
+ if (seen) {
+ assert(!bpf_update_elem(lru->map_fd, &key, &node, BPF_EXIST));
+ } else {
+ lru->nr_unique++;
+ assert(!bpf_update_elem(lru->map_fd, &key, &node, BPF_NOEXIST));
+ }
+
+ return seen;
+}
+
+static unsigned int read_keys(const char *dist_file,
+ unsigned long long **keys)
+{
+ struct stat fst;
+ unsigned long long *retkeys;
+ unsigned int counts = 0;
+ int dist_fd;
+ char *b, *l;
+ int i;
+
+ dist_fd = open(dist_file, 0);
+ assert(dist_fd != -1);
+
+ assert(fstat(dist_fd, &fst) == 0);
+ b = malloc(fst.st_size);
+ assert(b);
+
+ assert(read(dist_fd, b, fst.st_size) == fst.st_size);
+ close(dist_fd);
+ for (i = 0; i < fst.st_size; i++) {
+ if (b[i] == '\n')
+ counts++;
+ }
+ counts++; /* in case the last line has no \n */
+
+ retkeys = malloc(counts * sizeof(unsigned long long));
+ assert(retkeys);
+
+ counts = 0;
+ for (l = strtok(b, "\n"); l; l = strtok(NULL, "\n"))
+ retkeys[counts++] = strtoull(l, NULL, 10);
+ free(b);
+
+ *keys = retkeys;
+
+ return counts;
+}
+
+static void do_test_lru_dist(int lru_map_fd, int task,
+ const unsigned long long *keys,
+ unsigned int key_counts, unsigned int lru_size)
+{
+ unsigned int nr_misses = 0;
+ struct pfect_lru pfect_lru;
+ unsigned long long key_offset = task * key_counts;
+ unsigned long long key, value = 1234;
+ unsigned int i;
+
+ printf("task:%d %s:......\n", task, __func__);
+
+ pfect_lru_init(&pfect_lru, lru_size, key_counts);
+
+ for (i = 0; i < key_counts; i++) {
+ key = keys[i] + key_offset;
+
+ pfect_lru_lookup_or_insert(&pfect_lru, key);
+
+ if (!bpf_lookup_elem(lru_map_fd, &key, &value))
+ continue;
+
+ if (bpf_update_elem(lru_map_fd, &key, &value, BPF_NOEXIST)) {
+ printf("bpf_update_elem(lru_map_fd, %llu): errno:%d\n",
+ key, errno);
+ assert(0);
+ }
+
+ nr_misses++;
+ }
+
+ printf(" task:%d BPF LRU: nr_unique:%u(/%u) nr_misses:%u(/%u)\n",
+ task, pfect_lru.nr_unique, key_counts, nr_misses, key_counts);
+ printf(" task:%d Perfect LRU: nr_unique:%u(/%u nr_misses:%u(/%u)\n",
+ task, pfect_lru.nr_unique, pfect_lru.total,
+ pfect_lru.nr_misses, pfect_lru.total);
+
+ pfect_lru_destroy(&pfect_lru);
+}
+
+static void test_lru_dist(int map_type, const unsigned long long *keys,
+ unsigned int key_counts, unsigned int lru_size)
+{
+ cpu_set_t cpuset;
+ int lru_map_fd;
+
+ CPU_ZERO(&cpuset);
+ CPU_SET(0, &cpuset);
+ assert(!sched_setaffinity(0, sizeof(cpuset), &cpuset));
+
+ lru_map_fd = bpf_create_map(map_type, sizeof(unsigned long long),
+ sizeof(unsigned long long),
+ lru_size, 0);
+ assert(lru_map_fd != -1);
+ do_test_lru_dist(lru_map_fd, 0, keys, key_counts, lru_size);
+ close(lru_map_fd);
+}
+
+static void test_parallel_lru_dist(int map_type, int nr_tasks,
+ const unsigned long long *keys,
+ unsigned int key_counts,
+ unsigned int lru_size)
+{
+ cpu_set_t cpuset;
+ pid_t pid[nr_tasks];
+ int lru_map_fd;
+ int i;
+
+
+ lru_map_fd = bpf_create_map(map_type, sizeof(unsigned long long),
+ sizeof(unsigned long long),
+ nr_tasks * lru_size, 0);
+ assert(lru_map_fd != -1);
+
+ for (i = 0; i < nr_tasks; i++) {
+ pid[i] = fork();
+ if (pid[i] == 0) {
+ CPU_ZERO(&cpuset);
+ CPU_SET(i, &cpuset);
+ assert(!sched_setaffinity(0, sizeof(cpuset), &cpuset));
+ do_test_lru_dist(lru_map_fd, i, keys, key_counts,
+ lru_size);
+ exit(0);
+ } else if (pid[i] == -1) {
+ printf("couldn't spawn #%d process\n", i);
+ exit(1);
+ }
+ }
+ for (i = 0; i < nr_tasks; i++) {
+ int status;
+
+ assert(waitpid(pid[i], &status, 0) == pid[i]);
+ assert(status == 0);
+ }
+
+ close(lru_map_fd);
+}
+
+int main(int argc, char **argv)
+{
+ struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};
+ const char *dist_file = argv[1];
+ int lru_size = atoi(argv[2]);
+ unsigned long long *keys = NULL;
+ unsigned int counts;
+
+ setbuf(stdout, NULL);
+
+ assert(!setrlimit(RLIMIT_MEMLOCK, &r));
+
+ counts = read_keys(dist_file, &keys);
+ test_lru_dist(BPF_MAP_TYPE_LRU_HASH, keys, counts, lru_size);
+ if (argc > 3)
+ test_parallel_lru_dist(BPF_MAP_TYPE_LRU_HASH, atoi(argv[3]),
+ keys, counts, lru_size);
+
+ free(keys);
+
+ return 0;
+}
diff --git a/samples/bpf/test_lru_map.c b/samples/bpf/test_lru_map.c
new file mode 100644
index 0000000..82cfffc
--- /dev/null
+++ b/samples/bpf/test_lru_map.c
@@ -0,0 +1,690 @@
+/*
+ * Copyright (c) 2016 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General Public
+ * License as published by the Free Software Foundation.
+ */
+#define _GNU_SOURCE
+#include <stdio.h>
+#include <unistd.h>
+#include <linux/bpf.h>
+#include <errno.h>
+#include <string.h>
+#include <assert.h>
+#include <sched.h>
+#include <sys/wait.h>
+#include <stdlib.h>
+#include <time.h>
+#include "libbpf.h"
+
+#define min(a, b) ((a) < (b) ? (a) : (b))
+#define LOCAL_FREE_TARGET (128)
+
+static long nr_cpus;
+
+static int create_map(int map_type, unsigned int size)
+{
+ int map_fd;
+
+ map_fd = bpf_create_map(map_type, sizeof(unsigned long long),
+ sizeof(unsigned long long), size, 0);
+
+ if (map_fd == -1)
+ perror("bpf_create_map");
+
+ return map_fd;
+}
+
+static int map_subset(int map0, int map1)
+{
+ unsigned long long next_key = 0;
+ unsigned long long value0[nr_cpus], value1[nr_cpus];
+ int ret;
+
+ while (!bpf_get_next_key(map1, &next_key, &next_key)) {
+ assert(!bpf_lookup_elem(map1, &next_key, value1));
+ ret = bpf_lookup_elem(map0, &next_key, value0);
+ if (ret) {
+ printf("key:%llu not found from map. %s(%d)\n",
+ next_key, strerror(errno), errno);
+ return 0;
+ }
+ if (value0[0] != value1[0]) {
+ printf("key:%llu value0:%llu != value1:%llu\n",
+ next_key, value0[0], value1[0]);
+ return 0;
+ }
+ }
+ return 1;
+}
+
+static int map_equal(int lru_map, int expected)
+{
+ return map_subset(lru_map, expected) && map_subset(expected, lru_map);
+}
+
+/* Size of the LRU amp is 2
+ * Add key=1 (+1 key)
+ * Add key=2 (+1 key)
+ * Lookup Key=1
+ * Add Key=3
+ * => Key=2 will be removed by LRU
+ * Iterate map. Only found key=1 and key=3
+ */
+static void test_lru_sanity0(int map_type)
+{
+ unsigned long long key, value[nr_cpus];
+ cpu_set_t cpuset;
+ int map_fd, expected_map_fd;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ CPU_ZERO(&cpuset);
+ CPU_SET(0, &cpuset);
+ assert(!sched_setaffinity(0, sizeof(cpuset), &cpuset));
+
+ map_fd = create_map(map_type, 2);
+ assert(map_fd != -1);
+ expected_map_fd = create_map(BPF_MAP_TYPE_HASH, 2);
+ assert(expected_map_fd != -1);
+
+ value[0] = 1234;
+
+ /* insert key=1 element */
+
+ key = 1;
+ assert(!bpf_update_elem(map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_update_elem(expected_map_fd, &key, value, BPF_NOEXIST));
+
+ /* BPF_NOEXIST means: add new element if it doesn't exist */
+ assert(bpf_update_elem(map_fd, &key, value, BPF_NOEXIST) == -1 &&
+ /* key=1 already exists */
+ errno == EEXIST);
+
+ assert(bpf_update_elem(map_fd, &key, value, -1) == -1 &&
+ errno == EINVAL);
+
+ /* insert key=2 element */
+
+ /* check that key=2 is not found */
+ key = 2;
+ assert(bpf_lookup_elem(map_fd, &key, value) == -1 && errno == ENOENT);
+
+ /* BPF_EXIST means: update existing element */
+ assert(bpf_update_elem(map_fd, &key, value, BPF_EXIST) == -1 &&
+ /* key=2 is not there */
+ errno == ENOENT);
+
+ assert(!bpf_update_elem(map_fd, &key, value, BPF_NOEXIST));
+
+ /* insert key=3 element */
+
+ /* check that key=3 is not found */
+ key = 3;
+ assert(bpf_lookup_elem(map_fd, &key, value) == -1 && errno == ENOENT);
+
+ /* check that key=1 can be found and mark the ref bit to
+ * stop LRU from removing key=1
+ */
+ key = 1;
+ assert(!bpf_lookup_elem(map_fd, &key, value));
+ assert(value[0] == 1234);
+
+ key = 3;
+ assert(!bpf_update_elem(map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_update_elem(expected_map_fd, &key, value, BPF_NOEXIST));
+
+ /* key=2 has been removed from the LRU */
+ key = 2;
+ assert(bpf_lookup_elem(map_fd, &key, value) == -1);
+
+ assert(map_equal(map_fd, expected_map_fd));
+
+ close(map_fd);
+
+ printf("Pass\n");
+}
+
+/* Size of the LRU map is 1.5*LOCAL_FREE_TARGET
+ * Insert 1 to LOCAL_FREE_TARGET (+LOCAL_FREE_TARGET keys)
+ * Lookup 1 to LOCAL_FREE_TARGET/2
+ * Insert 1+LOCAL_FREE_TARGET to 2*LOCAL_FREE_TARGET (+LOCAL_FREE_TARGET keys)
+ * => 1+LOCAL_FREE_TARGET/2 to LOCALFREE_TARGET will be removed by LRU
+ */
+static void test_lru_sanity1(int map_type)
+{
+ unsigned long long key, end_key, value[nr_cpus];
+ unsigned int map_size;
+ int lru_map_fd, expected_map_fd;
+ unsigned int batch_size;
+ cpu_set_t cpuset;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ batch_size = LOCAL_FREE_TARGET / 2;
+ assert(batch_size * 2 == LOCAL_FREE_TARGET);
+
+ CPU_ZERO(&cpuset);
+ CPU_SET(0, &cpuset);
+ assert(sched_setaffinity(0, sizeof(cpuset), &cpuset) == 0);
+
+ map_size = LOCAL_FREE_TARGET + batch_size;
+ lru_map_fd = create_map(map_type, map_size);
+ assert(lru_map_fd != -1);
+ expected_map_fd = create_map(BPF_MAP_TYPE_HASH, map_size);
+ assert(expected_map_fd != -1);
+
+ value[0] = 1234;
+
+ /* Insert 1 to LOCAL_FREE_TARGET (+LOCAL_FREE_TARGET keys) */
+ end_key = 1 + LOCAL_FREE_TARGET;
+ for (key = 1; key < end_key; key++)
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+
+ /* Lookup 1 to LOCAL_FREE_TARGET/2 */
+ end_key = 1 + batch_size;
+ for (key = 1; key < end_key; key++) {
+ assert(!bpf_lookup_elem(lru_map_fd, &key, value));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ /* Insert 1+LOCAL_FREE_TARGET to 2*LOCAL_FREE_TARGET
+ * => 1+LOCAL_FREE_TARGET/2 to LOCALFREE_TARGET will be
+ * removed by LRU
+ */
+ key = 1 + LOCAL_FREE_TARGET;
+ end_key = key + LOCAL_FREE_TARGET;
+ for (; key < end_key; key++) {
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ assert(map_equal(lru_map_fd, expected_map_fd));
+
+ close(expected_map_fd);
+ close(lru_map_fd);
+
+ printf("Pass\n");
+}
+
+/* Size of the LRU map 1.5 * LOCAL_FREE_TARGET
+ * Insert 1 to LOCAL_FREE_TARGET (+LOCAL_FREE_TARGET keys)
+ * Update 1 to LOCAL_FREE_TARGET/2
+ * => The original 1 to LOCAL_FREE_TARGET/2 will be removed due to
+ * the LRU shrink process
+ * Re-insert 1 to LOCAL_FREE_TARGET/2 again and do a lookup immeidately
+ * Insert 1+LOCAL_FREE_TARGET to LOCAL_FREE_TARGET*3/2
+ * Insert 1+LOCAL_FREE_TARGET*3/2 to LOCAL_FREE_TARGET*5/2
+ * => Key 1+LOCAL_FREE_TARGET to LOCAL_FREE_TARGET*3/2
+ * will be removed from LRU because it has never
+ * been lookup and ref bit is not set
+ */
+static void test_lru_sanity2(int map_type)
+{
+ unsigned long long key, value[nr_cpus];
+ unsigned long long end_key;
+ int lru_map_fd, expected_map_fd;
+ unsigned int batch_size;
+ unsigned int map_size;
+ cpu_set_t cpuset;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ batch_size = LOCAL_FREE_TARGET / 2;
+ assert(batch_size * 2 == LOCAL_FREE_TARGET);
+
+ CPU_ZERO(&cpuset);
+ CPU_SET(0, &cpuset);
+ assert(sched_setaffinity(0, sizeof(cpuset), &cpuset) == 0);
+
+ map_size = LOCAL_FREE_TARGET + batch_size;
+ lru_map_fd = create_map(map_type, map_size);
+ assert(lru_map_fd != -1);
+ expected_map_fd = create_map(BPF_MAP_TYPE_HASH, map_size);
+ assert(expected_map_fd != -1);
+
+ value[0] = 1234;
+
+ /* Insert 1 to LOCAL_FREE_TARGET (+LOCAL_FREE_TARGET keys) */
+ end_key = 1 + LOCAL_FREE_TARGET;
+ for (key = 1; key < end_key; key++)
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+
+ /* Any bpf_update_elem will require to acquire a new node
+ * from LRU first.
+ *
+ * The local list is running out of free nodes.
+ * It gets from the global LRU list which tries to
+ * shrink the inactive list to get LOCAL_FREE_TARGET
+ * number of free nodes.
+ *
+ * Hence, the oldest key 1 to LOCAL_FREE_TARGET/2
+ * are removed from the LRU list.
+ */
+ key = 1;
+ if (map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH) {
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_delete_elem(lru_map_fd, &key));
+ } else
+ assert(bpf_update_elem(lru_map_fd, &key, value, BPF_EXIST));
+
+
+ /* Re-insert 1 to LOCAL_FREE_TARGET/2 again and do a lookup
+ * immeidately.
+ */
+ end_key = 1 + batch_size;
+ value[0] = 4321;
+ for (key = 1; key < end_key; key++) {
+ assert(bpf_lookup_elem(lru_map_fd, &key, value));
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_lookup_elem(lru_map_fd, &key, value));
+ assert(value[0] == 4321);
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ value[0] = 1234;
+
+ /* Insert 1+LOCAL_FREE_TARGET to LOCAL_FREE_TARGET*3/2 */
+ end_key = 1 + LOCAL_FREE_TARGET + batch_size;
+ for (key = 1 + LOCAL_FREE_TARGET; key < end_key; key++)
+ /* These newly added but not referenced keys will be
+ * gone during the next LRU shrink.
+ */
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+
+ /* Insert 1+LOCAL_FREE_TARGET*3/2 to LOCAL_FREE_TARGET*5/2 */
+ end_key = key + LOCAL_FREE_TARGET;
+ for (; key < end_key; key++) {
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ assert(map_equal(lru_map_fd, expected_map_fd));
+
+ close(expected_map_fd);
+ close(lru_map_fd);
+
+ printf("Pass\n");
+}
+
+/* Size of the LRU map is 2*LOCAL_FREE_TARGET
+ * It is to test the active/inactive list rotation
+ * Insert 1 to 2*LOCAL_FREE_TARGET (+2*LOCAL_FREE_TARGET keys)
+ * Lookup key 1 to LOCAL_FREE_TARGET*3/2
+ * Add 1+2*LOCAL_FREE_TARGET to LOCAL_FREE_TARGET*5/2 (+LOCAL_FREE_TARGET/2 keys)
+ * => key 1+LOCAL_FREE_TARGET*3/2 to 2*LOCAL_FREE_TARGET are removed from LRU
+ */
+static void test_lru_sanity3(int map_type)
+{
+ unsigned long long key, end_key, value[nr_cpus];
+ int lru_map_fd, expected_map_fd;
+ unsigned int batch_size;
+ unsigned int map_size;
+ cpu_set_t cpuset;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ batch_size = LOCAL_FREE_TARGET / 2;
+ assert(batch_size * 2 == LOCAL_FREE_TARGET);
+
+ CPU_ZERO(&cpuset);
+ CPU_SET(0, &cpuset);
+ assert(sched_setaffinity(0, sizeof(cpuset), &cpuset) == 0);
+
+ map_size = LOCAL_FREE_TARGET * 2;
+ lru_map_fd = create_map(map_type, map_size);
+ assert(lru_map_fd != -1);
+ expected_map_fd = create_map(BPF_MAP_TYPE_HASH, map_size);
+ assert(expected_map_fd != -1);
+
+ value[0] = 1234;
+
+ /* Insert 1 to 2*LOCAL_FREE_TARGET (+2*LOCAL_FREE_TARGET keys) */
+ end_key = 1 + (2 * LOCAL_FREE_TARGET);
+ for (key = 1; key < end_key; key++)
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+
+ /* Lookup key 1 to LOCAL_FREE_TARGET*3/2 */
+ end_key = LOCAL_FREE_TARGET + batch_size;
+ for (key = 1; key < end_key; key++) {
+ assert(!bpf_lookup_elem(lru_map_fd, &key, value));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ /* Add 1+2*LOCAL_FREE_TARGET to LOCAL_FREE_TARGET*5/2
+ * (+LOCAL_FREE_TARGET/2 keys)
+ */
+ key = 2 * LOCAL_FREE_TARGET + 1;
+ end_key = key + batch_size;
+ for (; key < end_key; key++) {
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ assert(map_equal(lru_map_fd, expected_map_fd));
+
+ close(expected_map_fd);
+ close(lru_map_fd);
+
+ printf("Pass\n");
+}
+
+/* Test deletion */
+static void test_lru_sanity4(int map_type)
+{
+ int lru_map_fd, expected_map_fd;
+ unsigned long long key, value[nr_cpus];
+ unsigned long long end_key;
+ cpu_set_t cpuset;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ CPU_ZERO(&cpuset);
+ CPU_SET(0, &cpuset);
+ assert(sched_setaffinity(0, sizeof(cpuset), &cpuset) == 0);
+
+ lru_map_fd = create_map(map_type, 3 * LOCAL_FREE_TARGET);
+ assert(lru_map_fd != -1);
+ expected_map_fd = create_map(BPF_MAP_TYPE_HASH,
+ 3 * LOCAL_FREE_TARGET);
+
+ value[0] = 1234;
+
+ for (key = 1; key <= 2 * LOCAL_FREE_TARGET; key++)
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+
+ key = 1;
+ assert(bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+
+ for (key = 1; key <= LOCAL_FREE_TARGET; key++) {
+ assert(!bpf_lookup_elem(lru_map_fd, &key, value));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ for (; key <= 2 * LOCAL_FREE_TARGET; key++) {
+ assert(!bpf_delete_elem(lru_map_fd, &key));
+ assert(bpf_delete_elem(lru_map_fd, &key));
+ }
+
+ end_key = key + 2 * LOCAL_FREE_TARGET;
+ for (; key < end_key; key++) {
+ assert(!bpf_update_elem(lru_map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_update_elem(expected_map_fd, &key, value,
+ BPF_NOEXIST));
+ }
+
+ assert(map_equal(lru_map_fd, expected_map_fd));
+
+ close(expected_map_fd);
+ close(lru_map_fd);
+
+ printf("Pass\n");
+}
+
+static void do_test_lru_small0(int cpu, int map_fd)
+{
+ unsigned long long key, value[nr_cpus];
+
+ /* Ensure the last key inserted by previous CPU can be found */
+ key = cpu;
+ assert(!bpf_lookup_elem(map_fd, &key, value));
+
+ value[0] = 1234;
+
+ key = cpu + 1;
+ assert(!bpf_update_elem(map_fd, &key, value, BPF_NOEXIST));
+ assert(!bpf_lookup_elem(map_fd, &key, value));
+
+ /* Cannot find the last key because it was removed by LRU */
+ key = cpu;
+ assert(bpf_lookup_elem(map_fd, &key, value));
+}
+
+static void test_lru_small0(int map_type)
+{
+ unsigned long long key, value[nr_cpus];
+ int map_fd;
+ int i;
+
+ printf("%s (map_type%d): ", __func__, map_type);
+
+ map_fd = create_map(map_type, 1);
+ assert(map_fd != -1);
+
+ value[0] = 1234;
+ key = 0;
+ assert(!bpf_update_elem(map_fd, &key, value, BPF_NOEXIST));
+
+ for (i = 0; i < nr_cpus; i++) {
+ cpu_set_t cpuset;
+ pid_t pid;
+
+ pid = fork();
+ if (pid == 0) {
+ CPU_ZERO(&cpuset);
+ CPU_SET(i, &cpuset);
+ assert(!sched_setaffinity(0, sizeof(cpuset), &cpuset));
+ do_test_lru_small0(i, map_fd);
+ exit(0);
+ } else if (pid == -1) {
+ printf("couldn't spawn #%d process\n", i);
+ exit(1);
+ } else {
+ int status;
+
+ assert(waitpid(pid, &status, 0) == pid);
+ assert(status == 0);
+ }
+ }
+
+ close(map_fd);
+
+ printf("Pass\n");
+}
+
+static void test_lru_loss0(int map_type)
+{
+ unsigned long long key, value[nr_cpus];
+ unsigned int old_unused_losses = 0;
+ unsigned int new_unused_losses = 0;
+ unsigned int used_losses = 0;
+ int map_fd;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ map_fd = create_map(map_type, 900);
+ assert(map_fd != -1);
+
+ value[0] = 1234;
+
+ for (key = 1; key <= 1000; key++) {
+ int start_key, end_key;
+
+ assert(bpf_update_elem(map_fd, &key, value, BPF_NOEXIST) == 0);
+
+ start_key = 101;
+ end_key = min(key, 900);
+
+ while (start_key <= end_key) {
+ bpf_lookup_elem(map_fd, &start_key, value);
+ start_key++;
+ }
+ }
+
+ for (key = 1; key <= 1000; key++) {
+ if (bpf_lookup_elem(map_fd, &key, value)) {
+ if (key <= 100)
+ old_unused_losses++;
+ else if (key <= 900)
+ used_losses++;
+ else
+ new_unused_losses++;
+ }
+ }
+
+ close(map_fd);
+
+ printf("older-elem-losses:%d(/100) active-elem-losses:%d(/800) "
+ "newer-elem-losses:%d(/100)\n",
+ old_unused_losses, used_losses, new_unused_losses);
+}
+
+static void test_lru_loss1(int map_type)
+{
+ unsigned long long key, value[nr_cpus];
+ int map_fd;
+ unsigned int nr_losses = 0;
+
+ printf("%s (map_type:%d): ", __func__, map_type);
+
+ map_fd = create_map(map_type, 1000);
+ assert(map_fd != -1);
+
+ value[0] = 1234;
+
+ for (key = 1; key <= 1000; key++)
+ assert(!bpf_update_elem(map_fd, &key, value, BPF_NOEXIST));
+
+ for (key = 1; key <= 1000; key++) {
+ if (bpf_lookup_elem(map_fd, &key, value))
+ nr_losses++;
+ }
+
+ close(map_fd);
+
+ printf("nr_losses:%d(/1000)\n", nr_losses);
+}
+
+static void do_test_lru_parallel_loss(int task, void *data)
+{
+ const unsigned int nr_stable_elems = 1000;
+ const unsigned int nr_repeats = 100000;
+
+ int map_fd = *(int *)data;
+ unsigned long long stable_base;
+ unsigned long long key, value[nr_cpus];
+ unsigned long long next_ins_key;
+ unsigned int nr_losses = 0;
+ unsigned int i;
+
+ stable_base = task * nr_repeats * 2 + 1;
+ next_ins_key = stable_base;
+ value[0] = 1234;
+ for (i = 0; i < nr_stable_elems; i++) {
+ assert(bpf_update_elem(map_fd, &next_ins_key, value,
+ BPF_NOEXIST) == 0);
+ next_ins_key++;
+ }
+
+ for (i = 0; i < nr_repeats; i++) {
+ int rn;
+
+ rn = rand();
+
+ if (rn % 10) {
+ key = rn % nr_stable_elems + stable_base;
+ bpf_lookup_elem(map_fd, &key, value);
+ } else {
+ bpf_update_elem(map_fd, &next_ins_key, value,
+ BPF_NOEXIST);
+ next_ins_key++;
+ }
+ }
+
+ key = stable_base;
+ for (i = 0; i < nr_stable_elems; i++) {
+ if (bpf_lookup_elem(map_fd, &key, value))
+ nr_losses++;
+ key++;
+ }
+
+ printf(" task:%d nr_losses:%u\n", task, nr_losses);
+}
+
+static void run_parallel(int tasks, void (*fn)(int i, void *data), void *data)
+{
+ cpu_set_t cpuset;
+ pid_t pid[tasks];
+ int i;
+
+ for (i = 0; i < tasks; i++) {
+ pid[i] = fork();
+ if (pid[i] == 0) {
+ CPU_ZERO(&cpuset);
+ CPU_SET(i, &cpuset);
+ assert(!sched_setaffinity(0, sizeof(cpuset), &cpuset));
+ fn(i, data);
+ exit(0);
+ } else if (pid[i] == -1) {
+ printf("couldn't spawn #%d process\n", i);
+ exit(1);
+ }
+ }
+ for (i = 0; i < tasks; i++) {
+ int status;
+
+ assert(waitpid(pid[i], &status, 0) == pid[i]);
+ assert(status == 0);
+ }
+}
+
+static void test_lru_parallel_loss(int map_type, int nr_tasks)
+{
+ int map_fd;
+
+ printf("%s (map_type:%d):\n", __func__, map_type);
+
+ /* Give 20% more than the active working set */
+ map_fd = create_map(map_type, nr_tasks * (1000 + 200));
+
+ assert(map_fd != -1);
+
+ run_parallel(nr_tasks, do_test_lru_parallel_loss, &map_fd);
+
+ close(map_fd);
+}
+
+int main(int argc, char **argv)
+{
+ struct rlimit r = {RLIM_INFINITY, RLIM_INFINITY};
+ int map_types[] = {BPF_MAP_TYPE_LRU_HASH,
+ BPF_MAP_TYPE_LRU_PERCPU_HASH};
+ int i;
+
+ setbuf(stdout, NULL);
+
+ assert(!setrlimit(RLIMIT_MEMLOCK, &r));
+
+ srand(time(NULL));
+
+ nr_cpus = sysconf(_SC_NPROCESSORS_CONF);
+ assert(nr_cpus != -1);
+ printf("nr_cpus:%ld\n\n", nr_cpus);
+
+ for (i = 0; i < sizeof(map_types) / sizeof(*map_types); i++) {
+ test_lru_sanity0(map_types[i]);
+ test_lru_sanity1(map_types[i]);
+ test_lru_sanity2(map_types[i]);
+ test_lru_sanity3(map_types[i]);
+ test_lru_sanity4(map_types[i]);
+
+ test_lru_small0(map_types[i]);
+
+ test_lru_loss0(map_types[i]);
+ test_lru_loss1(map_types[i]);
+ test_lru_parallel_loss(map_types[i], nr_cpus);
+
+ printf("\n");
+ }
+
+
+ return 0;
+}
--
2.5.1
^ permalink raw reply related
* [RFC PATCH net-next 4/5] bpf: Add BPF_MAP_TYPE_LRU_PERCPU_HASH
From: Martin KaFai Lau @ 2016-10-02 3:58 UTC (permalink / raw)
To: netdev; +Cc: FB Kernel Team
In-Reply-To: <1475380713-4106095-1-git-send-email-kafai@fb.com>
Provide a LRU version of the existing BPF_MAP_TYPE_PERCPU_HASH
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
include/uapi/linux/bpf.h | 1 +
kernel/bpf/hashtab.c | 129 ++++++++++++++++++++++++++++++++++++++++++++---
kernel/bpf/syscall.c | 8 ++-
3 files changed, 130 insertions(+), 8 deletions(-)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index c153ff8..b443675 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -86,6 +86,7 @@ enum bpf_map_type {
BPF_MAP_TYPE_STACK_TRACE,
BPF_MAP_TYPE_CGROUP_ARRAY,
BPF_MAP_TYPE_LRU_HASH,
+ BPF_MAP_TYPE_LRU_PERCPU_HASH,
};
enum bpf_prog_type {
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index 1493d98..92c546e 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -62,7 +62,14 @@ static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node);
static bool htab_is_lru(const struct bpf_htab *htab)
{
- return htab->map.map_type == BPF_MAP_TYPE_LRU_HASH;
+ return htab->map.map_type == BPF_MAP_TYPE_LRU_HASH ||
+ htab->map.map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH;
+}
+
+static bool htab_is_percpu(const struct bpf_htab *htab)
+{
+ return htab->map.map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ htab->map.map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH;
}
static inline void htab_elem_set_ptr(struct htab_elem *l, u32 key_size,
@@ -85,7 +92,7 @@ static void htab_free_elems(struct bpf_htab *htab)
{
int i;
- if (htab->map.map_type != BPF_MAP_TYPE_PERCPU_HASH)
+ if (!htab_is_percpu(htab))
goto free_elems;
for (i = 0; i < htab->map.max_entries; i++) {
@@ -122,7 +129,7 @@ static int prealloc_init(struct bpf_htab *htab)
if (!htab->elems)
return -ENOMEM;
- if (htab->map.map_type != BPF_MAP_TYPE_PERCPU_HASH)
+ if (!htab_is_percpu(htab))
goto skip_percpu_elems;
for (i = 0; i < htab->map.max_entries; i++) {
@@ -194,8 +201,10 @@ static int alloc_extra_elems(struct bpf_htab *htab)
/* Called from syscall */
static struct bpf_map *htab_map_alloc(union bpf_attr *attr)
{
- bool percpu = attr->map_type == BPF_MAP_TYPE_PERCPU_HASH;
- bool lru = attr->map_type == BPF_MAP_TYPE_LRU_HASH;
+ bool percpu = (attr->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ attr->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH);
+ bool lru = (attr->map_type == BPF_MAP_TYPE_LRU_HASH ||
+ attr->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH);
bool prealloc = !(attr->map_flags & BPF_F_NO_PREALLOC);
struct bpf_htab *htab;
int err, i;
@@ -793,12 +802,84 @@ err:
return ret;
}
+static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key,
+ void *value, u64 map_flags,
+ bool onallcpus)
+{
+ struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
+ struct htab_elem *l_new = NULL, *l_old;
+ struct hlist_head *head;
+ unsigned long flags;
+ struct bucket *b;
+ u32 key_size, hash;
+ int ret;
+
+ if (unlikely(map_flags > BPF_EXIST))
+ /* unknown flags */
+ return -EINVAL;
+
+ WARN_ON_ONCE(!rcu_read_lock_held());
+
+ key_size = map->key_size;
+
+ hash = htab_map_hash(key, key_size);
+
+ b = __select_bucket(htab, hash);
+ head = &b->head;
+
+ /* For LRU, we need to alloc before taking bucket's
+ * spinlock because LRU's elem alloc may need
+ * to remove older elem from htab and this removal
+ * operation will need a bucket lock.
+ */
+ if (map_flags != BPF_EXIST) {
+ l_new = prealloc_lru_pop(htab, key, hash);
+ if (!l_new)
+ return -ENOMEM;
+ }
+
+ /* bpf_map_update_elem() can be called in_irq() */
+ raw_spin_lock_irqsave(&b->lock, flags);
+
+ l_old = lookup_elem_raw(head, hash, key, key_size);
+
+ ret = check_flags(htab, l_old, map_flags);
+ if (ret)
+ goto err;
+
+ if (l_old) {
+ bpf_lru_node_set_ref(&l_old->lru_node);
+
+ /* per-cpu hash map can update value in-place */
+ pcpu_copy_value(htab, htab_elem_get_ptr(l_old, key_size),
+ value, onallcpus);
+ } else {
+ pcpu_copy_value(htab, htab_elem_get_ptr(l_new, key_size),
+ value, onallcpus);
+ hlist_add_head_rcu(&l_new->hash_node, head);
+ l_new = NULL;
+ }
+ ret = 0;
+err:
+ raw_spin_unlock_irqrestore(&b->lock, flags);
+ if (l_new)
+ bpf_lru_push_free(&htab->lru, &l_new->lru_node);
+ return ret;
+}
+
static int htab_percpu_map_update_elem(struct bpf_map *map, void *key,
void *value, u64 map_flags)
{
return __htab_percpu_map_update_elem(map, key, value, map_flags, false);
}
+static int htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key,
+ void *value, u64 map_flags)
+{
+ return __htab_lru_percpu_map_update_elem(map, key, value, map_flags,
+ false);
+}
+
/* Called from syscall or from eBPF program */
static int htab_map_delete_elem(struct bpf_map *map, void *key)
{
@@ -945,8 +1026,21 @@ static void *htab_percpu_map_lookup_elem(struct bpf_map *map, void *key)
return NULL;
}
+static void *htab_lru_percpu_map_lookup_elem(struct bpf_map *map, void *key)
+{
+ struct htab_elem *l = __htab_map_lookup_elem(map, key);
+
+ if (l) {
+ bpf_lru_node_set_ref(&l->lru_node);
+ return this_cpu_ptr(htab_elem_get_ptr(l, map->key_size));
+ }
+
+ return NULL;
+}
+
int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value)
{
+ struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
struct htab_elem *l;
void __percpu *pptr;
int ret = -ENOENT;
@@ -962,6 +1056,8 @@ int bpf_percpu_hash_copy(struct bpf_map *map, void *key, void *value)
l = __htab_map_lookup_elem(map, key);
if (!l)
goto out;
+ if (htab_is_lru(htab))
+ bpf_lru_node_set_ref(&l->lru_node);
pptr = htab_elem_get_ptr(l, map->key_size);
for_each_possible_cpu(cpu) {
bpf_long_memcpy(value + off,
@@ -977,10 +1073,16 @@ out:
int bpf_percpu_hash_update(struct bpf_map *map, void *key, void *value,
u64 map_flags)
{
+ struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
int ret;
rcu_read_lock();
- ret = __htab_percpu_map_update_elem(map, key, value, map_flags, true);
+ if (htab_is_lru(htab))
+ ret = __htab_lru_percpu_map_update_elem(map, key, value,
+ map_flags, true);
+ else
+ ret = __htab_percpu_map_update_elem(map, key, value, map_flags,
+ true);
rcu_read_unlock();
return ret;
@@ -1000,11 +1102,26 @@ static struct bpf_map_type_list htab_percpu_type __read_mostly = {
.type = BPF_MAP_TYPE_PERCPU_HASH,
};
+static const struct bpf_map_ops htab_lru_percpu_ops = {
+ .map_alloc = htab_map_alloc,
+ .map_free = htab_map_free,
+ .map_get_next_key = htab_map_get_next_key,
+ .map_lookup_elem = htab_lru_percpu_map_lookup_elem,
+ .map_update_elem = htab_lru_percpu_map_update_elem,
+ .map_delete_elem = htab_lru_map_delete_elem,
+};
+
+static struct bpf_map_type_list htab_lru_percpu_type __read_mostly = {
+ .ops = &htab_lru_percpu_ops,
+ .type = BPF_MAP_TYPE_LRU_PERCPU_HASH,
+};
+
static int __init register_htab_map(void)
{
bpf_register_map_type(&htab_type);
bpf_register_map_type(&htab_percpu_type);
bpf_register_map_type(&htab_lru_type);
+ bpf_register_map_type(&htab_lru_percpu_type);
return 0;
}
late_initcall(register_htab_map);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 228f962..bc69eb4 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -295,6 +295,7 @@ static int map_lookup_elem(union bpf_attr *attr)
goto free_key;
if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH ||
map->map_type == BPF_MAP_TYPE_PERCPU_ARRAY)
value_size = round_up(map->value_size, 8) * num_possible_cpus();
else
@@ -305,7 +306,8 @@ static int map_lookup_elem(union bpf_attr *attr)
if (!value)
goto free_key;
- if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH) {
+ if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH) {
err = bpf_percpu_hash_copy(map, key, value);
} else if (map->map_type == BPF_MAP_TYPE_PERCPU_ARRAY) {
err = bpf_percpu_array_copy(map, key, value);
@@ -369,6 +371,7 @@ static int map_update_elem(union bpf_attr *attr)
goto free_key;
if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH ||
map->map_type == BPF_MAP_TYPE_PERCPU_ARRAY)
value_size = round_up(map->value_size, 8) * num_possible_cpus();
else
@@ -388,7 +391,8 @@ static int map_update_elem(union bpf_attr *attr)
*/
preempt_disable();
__this_cpu_inc(bpf_prog_active);
- if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH) {
+ if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ map->map_type == BPF_MAP_TYPE_LRU_PERCPU_HASH) {
err = bpf_percpu_hash_update(map, key, value, attr->flags);
} else if (map->map_type == BPF_MAP_TYPE_PERCPU_ARRAY) {
err = bpf_percpu_array_update(map, key, value, attr->flags);
--
2.5.1
^ permalink raw reply related
* [RFC PATCH net-next 3/5] bpf: ADD BPF_MAP_TYPE_LRU_HASH
From: Martin KaFai Lau @ 2016-10-02 3:58 UTC (permalink / raw)
To: netdev; +Cc: FB Kernel Team
In-Reply-To: <1475380713-4106095-1-git-send-email-kafai@fb.com>
Provide a LRU version of the existing BPF_MAP_TYPE_HASH.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
---
include/uapi/linux/bpf.h | 1 +
kernel/bpf/hashtab.c | 234 ++++++++++++++++++++++++++++++++++++++++++++---
2 files changed, 222 insertions(+), 13 deletions(-)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index f896dfa..c153ff8 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -85,6 +85,7 @@ enum bpf_map_type {
BPF_MAP_TYPE_PERCPU_ARRAY,
BPF_MAP_TYPE_STACK_TRACE,
BPF_MAP_TYPE_CGROUP_ARRAY,
+ BPF_MAP_TYPE_LRU_HASH,
};
enum bpf_prog_type {
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index a5e3915..1493d98 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -15,6 +15,7 @@
#include <linux/filter.h>
#include <linux/vmalloc.h>
#include "percpu_freelist.h"
+#include "bpf_lru_list.h"
struct bucket {
struct hlist_head head;
@@ -25,7 +26,10 @@ struct bpf_htab {
struct bpf_map map;
struct bucket *buckets;
void *elems;
- struct pcpu_freelist freelist;
+ union {
+ struct pcpu_freelist freelist;
+ struct bpf_lru lru;
+ };
void __percpu *extra_elems;
atomic_t count; /* number of elements in this hashtable */
u32 n_buckets; /* number of hash buckets */
@@ -48,11 +52,19 @@ struct htab_elem {
union {
struct rcu_head rcu;
enum extra_elem_state state;
+ struct bpf_lru_node lru_node;
};
u32 hash;
char key[0] __aligned(8);
};
+static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node);
+
+static bool htab_is_lru(const struct bpf_htab *htab)
+{
+ return htab->map.map_type == BPF_MAP_TYPE_LRU_HASH;
+}
+
static inline void htab_elem_set_ptr(struct htab_elem *l, u32 key_size,
void __percpu *pptr)
{
@@ -87,7 +99,22 @@ free_elems:
vfree(htab->elems);
}
-static int prealloc_elems_and_freelist(struct bpf_htab *htab)
+static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key,
+ u32 hash)
+{
+ struct bpf_lru_node *node = bpf_lru_pop_free(&htab->lru, hash);
+ struct htab_elem *l;
+
+ if (node) {
+ l = container_of(node, struct htab_elem, lru_node);
+ memcpy(l->key, key, htab->map.key_size);
+ return l;
+ }
+
+ return NULL;
+}
+
+static int prealloc_init(struct bpf_htab *htab)
{
int err = -ENOMEM, i;
@@ -110,12 +137,26 @@ static int prealloc_elems_and_freelist(struct bpf_htab *htab)
}
skip_percpu_elems:
- err = pcpu_freelist_init(&htab->freelist);
+ if (htab_is_lru(htab))
+ err = bpf_lru_init(&htab->lru,
+ offsetof(struct htab_elem, hash) -
+ offsetof(struct htab_elem, lru_node),
+ htab_lru_map_delete_node,
+ htab);
+ else
+ err = pcpu_freelist_init(&htab->freelist);
+
if (err)
goto free_elems;
- pcpu_freelist_populate(&htab->freelist, htab->elems, htab->elem_size,
- htab->map.max_entries);
+ if (htab_is_lru(htab))
+ bpf_lru_populate(&htab->lru, htab->elems,
+ offsetof(struct htab_elem, lru_node),
+ htab->elem_size, htab->map.max_entries);
+ else
+ pcpu_freelist_populate(&htab->freelist, htab->elems,
+ htab->elem_size, htab->map.max_entries);
+
return 0;
free_elems:
@@ -123,6 +164,16 @@ free_elems:
return err;
}
+static void prealloc_destroy(struct bpf_htab *htab)
+{
+ htab_free_elems(htab);
+
+ if (htab_is_lru(htab))
+ bpf_lru_destroy(&htab->lru);
+ else
+ pcpu_freelist_destroy(&htab->freelist);
+}
+
static int alloc_extra_elems(struct bpf_htab *htab)
{
void __percpu *pptr;
@@ -144,6 +195,8 @@ static int alloc_extra_elems(struct bpf_htab *htab)
static struct bpf_map *htab_map_alloc(union bpf_attr *attr)
{
bool percpu = attr->map_type == BPF_MAP_TYPE_PERCPU_HASH;
+ bool lru = attr->map_type == BPF_MAP_TYPE_LRU_HASH;
+ bool prealloc = !(attr->map_flags & BPF_F_NO_PREALLOC);
struct bpf_htab *htab;
int err, i;
u64 cost;
@@ -152,6 +205,9 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr)
/* reserved bits should not be used */
return ERR_PTR(-EINVAL);
+ if (lru && !prealloc)
+ return ERR_PTR(-ENOTSUPP);
+
htab = kzalloc(sizeof(*htab), GFP_USER);
if (!htab)
return ERR_PTR(-ENOMEM);
@@ -241,14 +297,14 @@ static struct bpf_map *htab_map_alloc(union bpf_attr *attr)
raw_spin_lock_init(&htab->buckets[i].lock);
}
- if (!percpu) {
+ if (!percpu && !lru) {
err = alloc_extra_elems(htab);
if (err)
goto free_buckets;
}
- if (!(attr->map_flags & BPF_F_NO_PREALLOC)) {
- err = prealloc_elems_and_freelist(htab);
+ if (prealloc) {
+ err = prealloc_init(htab);
if (err)
goto free_extra_elems;
}
@@ -323,6 +379,47 @@ static void *htab_map_lookup_elem(struct bpf_map *map, void *key)
return NULL;
}
+static void *htab_lru_map_lookup_elem(struct bpf_map *map, void *key)
+{
+ struct htab_elem *l = __htab_map_lookup_elem(map, key);
+
+ if (l) {
+ bpf_lru_node_set_ref(&l->lru_node);
+ return l->key + round_up(map->key_size, 8);
+ }
+
+ return NULL;
+}
+
+/* It is called from the bpf_lru_list when the LRU needs to delete
+ * older elements from the htab.
+ */
+static bool htab_lru_map_delete_node(void *arg, struct bpf_lru_node *node)
+{
+ struct bpf_htab *htab = (struct bpf_htab *)arg;
+ struct htab_elem *l, *tgt_l;
+ struct hlist_head *head;
+ unsigned long flags;
+ struct bucket *b;
+
+ tgt_l = container_of(node, struct htab_elem, lru_node);
+ b = __select_bucket(htab, tgt_l->hash);
+ head = &b->head;
+
+ raw_spin_lock_irqsave(&b->lock, flags);
+
+ hlist_for_each_entry_rcu(l, head, hash_node)
+ if (l == tgt_l) {
+ hlist_del_rcu(&l->hash_node);
+ break;
+ }
+
+
+ raw_spin_unlock_irqrestore(&b->lock, flags);
+
+ return l == tgt_l;
+}
+
/* Called from syscall */
static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
{
@@ -579,6 +676,70 @@ err:
return ret;
}
+static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value,
+ u64 map_flags)
+{
+ struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
+ struct htab_elem *l_new, *l_old = NULL;
+ struct hlist_head *head;
+ unsigned long flags;
+ struct bucket *b;
+ u32 key_size, hash;
+ int ret;
+
+ if (unlikely(map_flags > BPF_EXIST))
+ /* unknown flags */
+ return -EINVAL;
+
+ WARN_ON_ONCE(!rcu_read_lock_held());
+
+ key_size = map->key_size;
+
+ hash = htab_map_hash(key, key_size);
+
+ b = __select_bucket(htab, hash);
+ head = &b->head;
+
+ /* For LRU, we need to alloc before taking bucket's
+ * spinlock because getting free nodes from LRU may need
+ * to remove older elements from htab and this removal
+ * operation will need a bucket lock.
+ */
+ l_new = prealloc_lru_pop(htab, key, hash);
+ if (!l_new)
+ return -ENOMEM;
+ memcpy(l_new->key + round_up(map->key_size, 8), value, map->value_size);
+
+ /* bpf_map_update_elem() can be called in_irq() */
+ raw_spin_lock_irqsave(&b->lock, flags);
+
+ l_old = lookup_elem_raw(head, hash, key, key_size);
+
+ ret = check_flags(htab, l_old, map_flags);
+ if (ret)
+ goto err;
+
+ /* add new element to the head of the list, so that
+ * concurrent search will find it before old elem
+ */
+ hlist_add_head_rcu(&l_new->hash_node, head);
+ if (l_old) {
+ bpf_lru_node_set_ref(&l_new->lru_node);
+ hlist_del_rcu(&l_old->hash_node);
+ }
+ ret = 0;
+
+err:
+ raw_spin_unlock_irqrestore(&b->lock, flags);
+
+ if (ret)
+ bpf_lru_push_free(&htab->lru, &l_new->lru_node);
+ else if (l_old)
+ bpf_lru_push_free(&htab->lru, &l_old->lru_node);
+
+ return ret;
+}
+
static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key,
void *value, u64 map_flags,
bool onallcpus)
@@ -671,6 +832,39 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key)
return ret;
}
+static int htab_lru_map_delete_elem(struct bpf_map *map, void *key)
+{
+ struct bpf_htab *htab = container_of(map, struct bpf_htab, map);
+ struct hlist_head *head;
+ struct bucket *b;
+ struct htab_elem *l;
+ unsigned long flags;
+ u32 hash, key_size;
+ int ret = -ENOENT;
+
+ WARN_ON_ONCE(!rcu_read_lock_held());
+
+ key_size = map->key_size;
+
+ hash = htab_map_hash(key, key_size);
+ b = __select_bucket(htab, hash);
+ head = &b->head;
+
+ raw_spin_lock_irqsave(&b->lock, flags);
+
+ l = lookup_elem_raw(head, hash, key, key_size);
+
+ if (l) {
+ hlist_del_rcu(&l->hash_node);
+ ret = 0;
+ }
+
+ raw_spin_unlock_irqrestore(&b->lock, flags);
+ if (l)
+ bpf_lru_push_free(&htab->lru, &l->lru_node);
+ return ret;
+}
+
static void delete_all_elements(struct bpf_htab *htab)
{
int i;
@@ -702,12 +896,11 @@ static void htab_map_free(struct bpf_map *map)
* not have executed. Wait for them.
*/
rcu_barrier();
- if (htab->map.map_flags & BPF_F_NO_PREALLOC) {
+ if (htab->map.map_flags & BPF_F_NO_PREALLOC)
delete_all_elements(htab);
- } else {
- htab_free_elems(htab);
- pcpu_freelist_destroy(&htab->freelist);
- }
+ else
+ prealloc_destroy(htab);
+
free_percpu(htab->extra_elems);
kvfree(htab->buckets);
kfree(htab);
@@ -727,6 +920,20 @@ static struct bpf_map_type_list htab_type __read_mostly = {
.type = BPF_MAP_TYPE_HASH,
};
+static const struct bpf_map_ops htab_lru_ops = {
+ .map_alloc = htab_map_alloc,
+ .map_free = htab_map_free,
+ .map_get_next_key = htab_map_get_next_key,
+ .map_lookup_elem = htab_lru_map_lookup_elem,
+ .map_update_elem = htab_lru_map_update_elem,
+ .map_delete_elem = htab_lru_map_delete_elem,
+};
+
+static struct bpf_map_type_list htab_lru_type __read_mostly = {
+ .ops = &htab_lru_ops,
+ .type = BPF_MAP_TYPE_LRU_HASH,
+};
+
/* Called from eBPF program */
static void *htab_percpu_map_lookup_elem(struct bpf_map *map, void *key)
{
@@ -797,6 +1004,7 @@ static int __init register_htab_map(void)
{
bpf_register_map_type(&htab_type);
bpf_register_map_type(&htab_percpu_type);
+ bpf_register_map_type(&htab_lru_type);
return 0;
}
late_initcall(register_htab_map);
--
2.5.1
^ permalink raw reply related
* Assertion faileda at samples/bpf/test_maps
From: Louie Lu @ 2016-10-02 6:55 UTC (permalink / raw)
To: netdev
Hi everyone,
I'm now playing net-next samples/bpf things, and I found that
test_maps will give an assertion
```
test_maps: /home/grd/linux/net-next/samples/bpf/test_maps.c:146:
test_percpu_hashmap_sanity: Assertion `bpf_lookup_elem(map_fd, &key,
value) == -1 && errno == ENOENT' failed.
[1] 20706 abort (core dumped) ./test_maps
```
I've done the check at [[iovisor-dev] Assertion fails at
samples/bpf/test_maps](http://lists.iovisor.org/pipermail/iovisor-dev/2016-June/000281.html)
which tells me to set ulimit to unlimited or patching by setrlimit().
The problem is bpf_lookup_elem return -1, and errno return 0.
I'm using Intel(R) Core(TM) i7-2640M and net-next commit:
803783849fed11, 4.8.0-rc7-ARCH-02283-g8037838.
And, a strange this is when I done this at QEMU, test_maps perform
will and pass the test.
Is that my hardware problem or something else?
Thanks.
Louie Lu.
^ permalink raw reply
* [PATCH] ptp: Fix resource leak in case of error
From: Christophe JAILLET @ 2016-10-02 7:04 UTC (permalink / raw)
To: richardcochran; +Cc: netdev, linux-kernel, kernel-janitors, Christophe JAILLET
A call to 'ida_simple_remove()' is missing in the error handling path.
This as been spotted with the following coccinelle script which tries to
detect missing 'ida_simple_remove()' call in error handling paths.
///////////////
@@
expression x;
identifier l;
@@
* x = ida_simple_get(...);
...
if (...) {
...
}
...
if (...) {
...
goto l;
}
...
* l: ... when != ida_simple_remove(...);
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
---
Compile-tested only
---
drivers/ptp/ptp_clock.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/ptp/ptp_clock.c b/drivers/ptp/ptp_clock.c
index 2e481b9e8ea5..86280b7e41f3 100644
--- a/drivers/ptp/ptp_clock.c
+++ b/drivers/ptp/ptp_clock.c
@@ -263,6 +263,7 @@ struct ptp_clock *ptp_clock_register(struct ptp_clock_info *info,
no_device:
mutex_destroy(&ptp->tsevq_mux);
mutex_destroy(&ptp->pincfg_mux);
+ ida_simple_remove(&ptp_clocks_map, index);
no_slot:
kfree(ptp);
no_memory:
--
2.7.4
^ permalink raw reply related
* Re: rhashtable - rhashtable_insert_fast failed
From: Helge Deller @ 2016-10-02 9:49 UTC (permalink / raw)
To: Herbert Xu; +Cc: Phil Sutter, netdev@vger.kernel.org, Thomas Graf
In-Reply-To: <20160608023957.GA12209@gondor.apana.org.au>
Hi Herbert,
On 08.06.2016 04:39, Herbert Xu wrote:
> On Tue, Jun 07, 2016 at 04:47:28PM +0200, Helge Deller wrote:
>> On 07.06.2016 16:16, Herbert Xu wrote:
>>> On Tue, Jun 07, 2016 at 04:13:50PM +0200, Helge Deller wrote:
>>>>
>>>> What warnings do you mean specifically? Some specific CONFIG_ option ?
>>>
>>> Look for GFP_NOWARN in lib/rhashtable.c and delete it.
>>
>> Ok, removed it.
>> It generates a kernel warning:
>
> Thanks. This is exactly the problem that I'm yet to fix, namely
> we're trying to allocate a hash table that's too big to be done
> using physically contiguous memory.
I'm still seeing this problem with v4.8-rc8
Helge
^ permalink raw reply
* [PATCH 1/2] net: mv643xx_eth: use phydev from struct net_device
From: Philippe Reynes @ 2016-10-02 10:06 UTC (permalink / raw)
To: sebastian.hesselbarth, davem; +Cc: netdev, linux-kernel, Philippe Reynes
The private structure contain a pointer to phydev, but the structure
net_device already contain such pointer. So we can remove the pointer
phydev in the private structure, and update the driver to use the
one contained in struct net_device.
Signed-off-by: Philippe Reynes <tremyfr@gmail.com>
---
drivers/net/ethernet/marvell/mv643xx_eth.c | 88 ++++++++++++++--------------
1 files changed, 43 insertions(+), 45 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mv643xx_eth.c b/drivers/net/ethernet/marvell/mv643xx_eth.c
index 5583118..ae7370d 100644
--- a/drivers/net/ethernet/marvell/mv643xx_eth.c
+++ b/drivers/net/ethernet/marvell/mv643xx_eth.c
@@ -384,8 +384,6 @@ struct mv643xx_eth_private {
struct net_device *dev;
- struct phy_device *phy;
-
struct timer_list mib_counters_timer;
spinlock_t mib_counters_lock;
struct mib_counters mib_counters;
@@ -1236,7 +1234,7 @@ static void mv643xx_eth_adjust_link(struct net_device *dev)
DISABLE_AUTO_NEG_FOR_FLOW_CTRL |
DISABLE_AUTO_NEG_FOR_DUPLEX;
- if (mp->phy->autoneg == AUTONEG_ENABLE) {
+ if (dev->phydev->autoneg == AUTONEG_ENABLE) {
/* enable auto negotiation */
pscr &= ~autoneg_disable;
goto out_write;
@@ -1244,7 +1242,7 @@ static void mv643xx_eth_adjust_link(struct net_device *dev)
pscr |= autoneg_disable;
- if (mp->phy->speed == SPEED_1000) {
+ if (dev->phydev->speed == SPEED_1000) {
/* force gigabit, half duplex not supported */
pscr |= SET_GMII_SPEED_TO_1000;
pscr |= SET_FULL_DUPLEX_MODE;
@@ -1253,12 +1251,12 @@ static void mv643xx_eth_adjust_link(struct net_device *dev)
pscr &= ~SET_GMII_SPEED_TO_1000;
- if (mp->phy->speed == SPEED_100)
+ if (dev->phydev->speed == SPEED_100)
pscr |= SET_MII_SPEED_TO_100;
else
pscr &= ~SET_MII_SPEED_TO_100;
- if (mp->phy->duplex == DUPLEX_FULL)
+ if (dev->phydev->duplex == DUPLEX_FULL)
pscr |= SET_FULL_DUPLEX_MODE;
else
pscr &= ~SET_FULL_DUPLEX_MODE;
@@ -1500,11 +1498,12 @@ static int
mv643xx_eth_get_settings_phy(struct mv643xx_eth_private *mp,
struct ethtool_cmd *cmd)
{
+ struct net_device *dev = mp->dev;
int err;
- err = phy_read_status(mp->phy);
+ err = phy_read_status(dev->phydev);
if (err == 0)
- err = phy_ethtool_gset(mp->phy, cmd);
+ err = phy_ethtool_gset(dev->phydev, cmd);
/*
* The MAC does not support 1000baseT_Half.
@@ -1553,23 +1552,21 @@ mv643xx_eth_get_settings_phyless(struct mv643xx_eth_private *mp,
static void
mv643xx_eth_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
{
- struct mv643xx_eth_private *mp = netdev_priv(dev);
wol->supported = 0;
wol->wolopts = 0;
- if (mp->phy)
- phy_ethtool_get_wol(mp->phy, wol);
+ if (dev->phydev)
+ phy_ethtool_get_wol(dev->phydev, wol);
}
static int
mv643xx_eth_set_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
{
- struct mv643xx_eth_private *mp = netdev_priv(dev);
int err;
- if (mp->phy == NULL)
+ if (!dev->phydev)
return -EOPNOTSUPP;
- err = phy_ethtool_set_wol(mp->phy, wol);
+ err = phy_ethtool_set_wol(dev->phydev, wol);
/* Given that mv643xx_eth works without the marvell-specific PHY driver,
* this debugging hint is useful to have.
*/
@@ -1583,7 +1580,7 @@ mv643xx_eth_get_settings(struct net_device *dev, struct ethtool_cmd *cmd)
{
struct mv643xx_eth_private *mp = netdev_priv(dev);
- if (mp->phy != NULL)
+ if (dev->phydev)
return mv643xx_eth_get_settings_phy(mp, cmd);
else
return mv643xx_eth_get_settings_phyless(mp, cmd);
@@ -1592,10 +1589,9 @@ mv643xx_eth_get_settings(struct net_device *dev, struct ethtool_cmd *cmd)
static int
mv643xx_eth_set_settings(struct net_device *dev, struct ethtool_cmd *cmd)
{
- struct mv643xx_eth_private *mp = netdev_priv(dev);
int ret;
- if (mp->phy == NULL)
+ if (!dev->phydev)
return -EINVAL;
/*
@@ -1603,7 +1599,7 @@ mv643xx_eth_set_settings(struct net_device *dev, struct ethtool_cmd *cmd)
*/
cmd->advertising &= ~ADVERTISED_1000baseT_Half;
- ret = phy_ethtool_sset(mp->phy, cmd);
+ ret = phy_ethtool_sset(dev->phydev, cmd);
if (!ret)
mv643xx_eth_adjust_link(dev);
return ret;
@@ -1622,12 +1618,10 @@ static void mv643xx_eth_get_drvinfo(struct net_device *dev,
static int mv643xx_eth_nway_reset(struct net_device *dev)
{
- struct mv643xx_eth_private *mp = netdev_priv(dev);
-
- if (mp->phy == NULL)
+ if (!dev->phydev)
return -EINVAL;
- return genphy_restart_aneg(mp->phy);
+ return genphy_restart_aneg(dev->phydev);
}
static int
@@ -2326,19 +2320,20 @@ static inline void oom_timer_wrapper(unsigned long data)
static void port_start(struct mv643xx_eth_private *mp)
{
+ struct net_device *dev = mp->dev;
u32 pscr;
int i;
/*
* Perform PHY reset, if there is a PHY.
*/
- if (mp->phy != NULL) {
+ if (dev->phydev) {
struct ethtool_cmd cmd;
- mv643xx_eth_get_settings(mp->dev, &cmd);
- phy_init_hw(mp->phy);
- mv643xx_eth_set_settings(mp->dev, &cmd);
- phy_start(mp->phy);
+ mv643xx_eth_get_settings(dev, &cmd);
+ phy_init_hw(dev->phydev);
+ mv643xx_eth_set_settings(dev, &cmd);
+ phy_start(dev->phydev);
}
/*
@@ -2350,7 +2345,7 @@ static void port_start(struct mv643xx_eth_private *mp)
wrlp(mp, PORT_SERIAL_CONTROL, pscr);
pscr |= DO_NOT_FORCE_LINK_FAIL;
- if (mp->phy == NULL)
+ if (!dev->phydev)
pscr |= FORCE_LINK_PASS;
wrlp(mp, PORT_SERIAL_CONTROL, pscr);
@@ -2534,8 +2529,8 @@ static int mv643xx_eth_stop(struct net_device *dev)
del_timer_sync(&mp->rx_oom);
netif_carrier_off(dev);
- if (mp->phy)
- phy_stop(mp->phy);
+ if (dev->phydev)
+ phy_stop(dev->phydev);
free_irq(dev->irq, dev);
port_reset(mp);
@@ -2553,13 +2548,12 @@ static int mv643xx_eth_stop(struct net_device *dev)
static int mv643xx_eth_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
{
- struct mv643xx_eth_private *mp = netdev_priv(dev);
int ret;
- if (mp->phy == NULL)
+ if (!dev->phydev)
return -ENOTSUPP;
- ret = phy_mii_ioctl(mp->phy, ifr, cmd);
+ ret = phy_mii_ioctl(dev->phydev, ifr, cmd);
if (!ret)
mv643xx_eth_adjust_link(dev);
return ret;
@@ -3006,7 +3000,8 @@ static struct phy_device *phy_scan(struct mv643xx_eth_private *mp,
static void phy_init(struct mv643xx_eth_private *mp, int speed, int duplex)
{
- struct phy_device *phy = mp->phy;
+ struct net_device *dev = mp->dev;
+ struct phy_device *phy = dev->phydev;
if (speed == 0) {
phy->autoneg = AUTONEG_ENABLE;
@@ -3024,6 +3019,7 @@ static void phy_init(struct mv643xx_eth_private *mp, int speed, int duplex)
static void init_pscr(struct mv643xx_eth_private *mp, int speed, int duplex)
{
+ struct net_device *dev = mp->dev;
u32 pscr;
pscr = rdlp(mp, PORT_SERIAL_CONTROL);
@@ -3033,7 +3029,7 @@ static void init_pscr(struct mv643xx_eth_private *mp, int speed, int duplex)
}
pscr = MAX_RX_PACKET_9700BYTE | SERIAL_PORT_CONTROL_RESERVED;
- if (mp->phy == NULL) {
+ if (!dev->phydev) {
pscr |= DISABLE_AUTO_NEG_SPEED_GMII;
if (speed == SPEED_1000)
pscr |= SET_GMII_SPEED_TO_1000;
@@ -3072,6 +3068,7 @@ static int mv643xx_eth_probe(struct platform_device *pdev)
struct mv643xx_eth_platform_data *pd;
struct mv643xx_eth_private *mp;
struct net_device *dev;
+ struct phy_device *phydev = NULL;
struct resource *res;
int err;
@@ -3127,18 +3124,18 @@ static int mv643xx_eth_probe(struct platform_device *pdev)
err = 0;
if (pd->phy_node) {
- mp->phy = of_phy_connect(mp->dev, pd->phy_node,
- mv643xx_eth_adjust_link, 0,
- PHY_INTERFACE_MODE_GMII);
- if (!mp->phy)
+ phydev = of_phy_connect(mp->dev, pd->phy_node,
+ mv643xx_eth_adjust_link, 0,
+ PHY_INTERFACE_MODE_GMII);
+ if (!phydev)
err = -ENODEV;
else
- phy_addr_set(mp, mp->phy->mdio.addr);
+ phy_addr_set(mp, phydev->mdio.addr);
} else if (pd->phy_addr != MV643XX_ETH_PHY_NONE) {
- mp->phy = phy_scan(mp, pd->phy_addr);
+ phydev = phy_scan(mp, pd->phy_addr);
- if (IS_ERR(mp->phy))
- err = PTR_ERR(mp->phy);
+ if (IS_ERR(phydev))
+ err = PTR_ERR(phydev);
else
phy_init(mp, pd->speed, pd->duplex);
}
@@ -3222,10 +3219,11 @@ out:
static int mv643xx_eth_remove(struct platform_device *pdev)
{
struct mv643xx_eth_private *mp = platform_get_drvdata(pdev);
+ struct net_device *dev = mp->dev;
unregister_netdev(mp->dev);
- if (mp->phy != NULL)
- phy_disconnect(mp->phy);
+ if (dev->phydev)
+ phy_disconnect(dev->phydev);
cancel_work_sync(&mp->tx_timeout_task);
if (!IS_ERR(mp->clk))
--
1.7.4.4
^ permalink raw reply related
* [PATCH 2/2] net: mv643xx_eth: use new api ethtool_{get|set}_link_ksettings
From: Philippe Reynes @ 2016-10-02 10:06 UTC (permalink / raw)
To: sebastian.hesselbarth, davem; +Cc: netdev, linux-kernel, Philippe Reynes
In-Reply-To: <1475402809-11593-1-git-send-email-tremyfr@gmail.com>
The ethtool api {get|set}_settings is deprecated.
We move this driver to new api {get|set}_link_ksettings.
Signed-off-by: Philippe Reynes <tremyfr@gmail.com>
---
drivers/net/ethernet/marvell/mv643xx_eth.c | 84 +++++++++++++++++----------
1 files changed, 53 insertions(+), 31 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mv643xx_eth.c b/drivers/net/ethernet/marvell/mv643xx_eth.c
index ae7370d..18e6bb6 100644
--- a/drivers/net/ethernet/marvell/mv643xx_eth.c
+++ b/drivers/net/ethernet/marvell/mv643xx_eth.c
@@ -1495,56 +1495,69 @@ static const struct mv643xx_eth_stats mv643xx_eth_stats[] = {
};
static int
-mv643xx_eth_get_settings_phy(struct mv643xx_eth_private *mp,
- struct ethtool_cmd *cmd)
+mv643xx_eth_get_link_ksettings_phy(struct mv643xx_eth_private *mp,
+ struct ethtool_link_ksettings *cmd)
{
struct net_device *dev = mp->dev;
int err;
+ u32 supported, advertising;
err = phy_read_status(dev->phydev);
if (err == 0)
- err = phy_ethtool_gset(dev->phydev, cmd);
+ err = phy_ethtool_ksettings_get(dev->phydev, cmd);
/*
* The MAC does not support 1000baseT_Half.
*/
- cmd->supported &= ~SUPPORTED_1000baseT_Half;
- cmd->advertising &= ~ADVERTISED_1000baseT_Half;
+ ethtool_convert_link_mode_to_legacy_u32(&supported,
+ cmd->link_modes.supported);
+ ethtool_convert_link_mode_to_legacy_u32(&advertising,
+ cmd->link_modes.advertising);
+ supported &= ~SUPPORTED_1000baseT_Half;
+ advertising &= ~ADVERTISED_1000baseT_Half;
+ ethtool_convert_legacy_u32_to_link_mode(cmd->link_modes.supported,
+ supported);
+ ethtool_convert_legacy_u32_to_link_mode(cmd->link_modes.advertising,
+ advertising);
return err;
}
static int
-mv643xx_eth_get_settings_phyless(struct mv643xx_eth_private *mp,
- struct ethtool_cmd *cmd)
+mv643xx_eth_get_link_ksettings_phyless(struct mv643xx_eth_private *mp,
+ struct ethtool_link_ksettings *cmd)
{
u32 port_status;
+ u32 supported, advertising;
port_status = rdlp(mp, PORT_STATUS);
- cmd->supported = SUPPORTED_MII;
- cmd->advertising = ADVERTISED_MII;
+ supported = SUPPORTED_MII;
+ advertising = ADVERTISED_MII;
switch (port_status & PORT_SPEED_MASK) {
case PORT_SPEED_10:
- ethtool_cmd_speed_set(cmd, SPEED_10);
+ cmd->base.speed = SPEED_10;
break;
case PORT_SPEED_100:
- ethtool_cmd_speed_set(cmd, SPEED_100);
+ cmd->base.speed = SPEED_100;
break;
case PORT_SPEED_1000:
- ethtool_cmd_speed_set(cmd, SPEED_1000);
+ cmd->base.speed = SPEED_1000;
break;
default:
- cmd->speed = -1;
+ cmd->base.speed = -1;
break;
}
- cmd->duplex = (port_status & FULL_DUPLEX) ? DUPLEX_FULL : DUPLEX_HALF;
- cmd->port = PORT_MII;
- cmd->phy_address = 0;
- cmd->transceiver = XCVR_INTERNAL;
- cmd->autoneg = AUTONEG_DISABLE;
- cmd->maxtxpkt = 1;
- cmd->maxrxpkt = 1;
+ cmd->base.duplex = (port_status & FULL_DUPLEX) ?
+ DUPLEX_FULL : DUPLEX_HALF;
+ cmd->base.port = PORT_MII;
+ cmd->base.phy_address = 0;
+ cmd->base.autoneg = AUTONEG_DISABLE;
+
+ ethtool_convert_legacy_u32_to_link_mode(cmd->link_modes.supported,
+ supported);
+ ethtool_convert_legacy_u32_to_link_mode(cmd->link_modes.advertising,
+ advertising);
return 0;
}
@@ -1576,19 +1589,23 @@ mv643xx_eth_set_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
}
static int
-mv643xx_eth_get_settings(struct net_device *dev, struct ethtool_cmd *cmd)
+mv643xx_eth_get_link_ksettings(struct net_device *dev,
+ struct ethtool_link_ksettings *cmd)
{
struct mv643xx_eth_private *mp = netdev_priv(dev);
if (dev->phydev)
- return mv643xx_eth_get_settings_phy(mp, cmd);
+ return mv643xx_eth_get_link_ksettings_phy(mp, cmd);
else
- return mv643xx_eth_get_settings_phyless(mp, cmd);
+ return mv643xx_eth_get_link_ksettings_phyless(mp, cmd);
}
static int
-mv643xx_eth_set_settings(struct net_device *dev, struct ethtool_cmd *cmd)
+mv643xx_eth_set_link_ksettings(struct net_device *dev,
+ const struct ethtool_link_ksettings *cmd)
{
+ struct ethtool_link_ksettings c = *cmd;
+ u32 advertising;
int ret;
if (!dev->phydev)
@@ -1597,9 +1614,13 @@ mv643xx_eth_set_settings(struct net_device *dev, struct ethtool_cmd *cmd)
/*
* The MAC does not support 1000baseT_Half.
*/
- cmd->advertising &= ~ADVERTISED_1000baseT_Half;
+ ethtool_convert_link_mode_to_legacy_u32(&advertising,
+ c.link_modes.advertising);
+ advertising &= ~ADVERTISED_1000baseT_Half;
+ ethtool_convert_legacy_u32_to_link_mode(c.link_modes.advertising,
+ advertising);
- ret = phy_ethtool_sset(dev->phydev, cmd);
+ ret = phy_ethtool_ksettings_set(dev->phydev, &c);
if (!ret)
mv643xx_eth_adjust_link(dev);
return ret;
@@ -1746,8 +1767,6 @@ static int mv643xx_eth_get_sset_count(struct net_device *dev, int sset)
}
static const struct ethtool_ops mv643xx_eth_ethtool_ops = {
- .get_settings = mv643xx_eth_get_settings,
- .set_settings = mv643xx_eth_set_settings,
.get_drvinfo = mv643xx_eth_get_drvinfo,
.nway_reset = mv643xx_eth_nway_reset,
.get_link = ethtool_op_get_link,
@@ -1761,6 +1780,8 @@ static const struct ethtool_ops mv643xx_eth_ethtool_ops = {
.get_ts_info = ethtool_op_get_ts_info,
.get_wol = mv643xx_eth_get_wol,
.set_wol = mv643xx_eth_set_wol,
+ .get_link_ksettings = mv643xx_eth_get_link_ksettings,
+ .set_link_ksettings = mv643xx_eth_set_link_ksettings,
};
@@ -2328,11 +2349,12 @@ static void port_start(struct mv643xx_eth_private *mp)
* Perform PHY reset, if there is a PHY.
*/
if (dev->phydev) {
- struct ethtool_cmd cmd;
+ struct ethtool_link_ksettings cmd;
- mv643xx_eth_get_settings(dev, &cmd);
+ mv643xx_eth_get_link_ksettings(dev, &cmd);
phy_init_hw(dev->phydev);
- mv643xx_eth_set_settings(dev, &cmd);
+ mv643xx_eth_set_link_ksettings(
+ dev, (const struct ethtool_link_ksettings *)&cmd);
phy_start(dev->phydev);
}
--
1.7.4.4
^ permalink raw reply related
* [GIT] Networking
From: David Miller @ 2016-10-02 13:01 UTC (permalink / raw)
To: torvalds; +Cc: akpm, netdev, linux-kernel
1) Fix wrong TCP checksums on MTU probing when checksum offloading is
disabled, from Douglas Caetano dos Santos.
2) Fix qdisc backlog updates in qfq and sfb schedulers, from Cong Wang.
3) Route lookup flow key protocol value is wrong in ip6gre_xmit_other(),
fix from Lance Richardson.
4) Scheduling while atomic in multicast routing code of ipv4 and ipv6,
fix from Nikolay Aleksandrov.
5) Fix packet alignment in fec driver, from Eric Nelson.
6) Fix perf regression in sctp due to struct layout and cache misses,
from Xin Long.
Please pull, thanks a lot!
The following changes since commit b1f2beb87bb034bb209773807994279f90cace78:
Merge tag 'media/v4.8-7' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media (2016-09-22 09:04:49 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/davem/net.git
for you to fetch changes up to 1cceda7849809a8857fd9f26efe8846506c710e1:
sctp: fix the issue sctp_diag uses lock_sock in rcu_read_lock (2016-09-30 02:08:57 -0400)
----------------------------------------------------------------
David S. Miller (4):
Merge tag 'linux-can-fixes-for-4.8-20160922' of git://git.kernel.org/.../mkl/linux-can
Merge branch 'fec-align'
Merge branch 'act_ife-fixes'
Merge branch 'sctp-fixes'
Douglas Caetano dos Santos (1):
tcp: fix wrong checksum calculation on MTU probing
Eric Dumazet (1):
tcp: fix a compile error in DBGUNDO()
Eric Nelson (3):
net: fec: remove QUIRK_HAS_RACC from i.mx25
net: fec: remove QUIRK_HAS_RACC from i.mx27
net: fec: align IP header in hardware
Florian Fainelli (1):
Revert "net: ethernet: bcmgenet: use phydev from struct net_device"
Jorgen Hansen (1):
VSOCK: Don't dec ack backlog twice for rejected connections
Lance Richardson (1):
ip6_gre: fix flowi6_proto value in ip6gre_xmit_other()
Milton Miller (1):
tg3: Avoid NULL pointer dereference in tg3_io_error_detected()
Nikolay Aleksandrov (1):
ipmr, ip6mr: fix scheduling while atomic and a deadlock with ipmr_get_route
Sergei Miroshnichenko (1):
can: dev: fix deadlock reported after bus-off
WANG Cong (2):
sch_qfq: keep backlog updated with qlen
sch_sfb: keep backlog updated with qlen
Xin Long (4):
sctp: move sent_count to the memory hole in sctp_chunk
sctp: remove prsctp_param from sctp_chunk
sctp: change to check peer prsctp_capable when using prsctp polices
sctp: fix the issue sctp_diag uses lock_sock in rcu_read_lock
Yotam Gigi (2):
act_ife: Fix external mac header on encode
act_ife: Fix false encoding
drivers/net/can/dev.c | 27 +++++++++++++++++----------
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 45 +++++++++++++++++++++++++--------------------
drivers/net/ethernet/broadcom/genet/bcmgenet.h | 1 +
drivers/net/ethernet/broadcom/genet/bcmmii.c | 24 +++++++++++++-----------
drivers/net/ethernet/broadcom/tg3.c | 10 +++++-----
drivers/net/ethernet/freescale/fec_main.c | 15 ++++++++++++---
include/linux/can/dev.h | 3 ++-
include/linux/mroute.h | 2 +-
include/linux/mroute6.h | 2 +-
include/net/sctp/structs.h | 13 +++----------
net/ipv4/ipmr.c | 3 ++-
net/ipv4/route.c | 3 ++-
net/ipv4/tcp_input.c | 3 +--
net/ipv4/tcp_output.c | 12 +++++++-----
net/ipv6/ip6_gre.c | 1 -
net/ipv6/ip6mr.c | 5 +++--
net/ipv6/route.c | 4 +++-
net/sched/act_ife.c | 7 +++----
net/sched/sch_qfq.c | 3 +++
net/sched/sch_sfb.c | 3 +++
net/sctp/chunk.c | 11 ++++++++---
net/sctp/outqueue.c | 12 ++++++------
net/sctp/sctp_diag.c | 58 ++++++++++++++++++++++++++++++++++++++++------------------
net/sctp/sm_make_chunk.c | 15 ---------------
net/sctp/socket.c | 10 +++++++---
net/vmw_vsock/af_vsock.c | 6 +++---
26 files changed, 171 insertions(+), 127 deletions(-)
^ permalink raw reply
* Re: [PATCH] net: rtnl: avoid uninitialized data in IFLA_VF_VLAN_LIST handling
From: Tariq Toukan @ 2016-10-02 9:03 UTC (permalink / raw)
To: Arnd Bergmann, Eric Dumazet
Cc: David S. Miller, Roopa Prabhu, Nicolas Dichtel,
Nikolay Aleksandrov, Jiri Pirko, Brenden Blanco,
Hannes Frederic Sowa, Nogah Frankel, netdev, LKML, Moshe Shemesh
In-Reply-To: <201609301838.05349.arnd@arndb.de>
Hi Arnd,
On 30/09/2016 7:38 PM, Arnd Bergmann wrote:
> On Friday 30 September 2016, Eric Dumazet wrote:
>>> @@ -1753,6 +1753,9 @@ static int do_setvfinfo(struct net_device *dev, struct nlattr **tb)
>>>
>>> len++;
>>> }
>>> + if (len == 0)
>>> + return -EINVAL;
>>> +
>>> err = ops->ndo_set_vf_vlan(dev, ivvl[0]->vf, ivvl[0]->vlan,
>>> ivvl[0]->qos, ivvl[0]->vlan_proto);
>>> if (err < 0)
>>> --
>>> 2.9.0
>>>
Reviewed-by: Moshe Shemesh <moshe@mellanox.com>
Thanks for the fix, I will add the flag to my makefile.
>> So, if I read this code, we build an array, but call ndo_set_vf_vlan()
>> only using first element ?
>>
>> Looks like the bug should be fixed in a different way.
> I was wondering about this too, but didn't understand enough about it to say
> if it was intentional or not.
It is intentional.
The change we need here is the option to set the VF vlan protocol, so
our hw will insert either S-TAG or C-TAG per VF. The previous version of
this patch didn't use any array.
Then we got a comment, which makes sense, that the new UAPI interface
should be able to support a future driver that might want to set both
S-TAG and C-TAG per VF.
We could have used a single struct on kernel side instead of a list of
size 1, but this would cause bad/not scalable code in the parsing stage.
As the UAPI is a list, in the parsing stage we thought it is more
correct to use the constant MAX_VLAN_LIST_LEN and iterate as of a
general case, even though currently the list size is one.
> I just realized that I forgot to add Moshe and Tariq
> on Cc (I relied on scripts/get_maintainer.pl, but didn't double-check).
> I've added them to Cc now, hope they can clarify this.
>
> Arnd
Regards,
Moshe and Tariq
^ permalink raw reply
* [PATCH 1/2] net: stmmac: use phydev from struct net_device
From: Philippe Reynes @ 2016-10-02 20:07 UTC (permalink / raw)
To: peppe.cavallaro, alexandre.torgue, davem
Cc: netdev, linux-kernel, Philippe Reynes
The private structure contain a pointer to phydev, but the structure
net_device already contain such pointer. So we can remove the pointer
phydev in the private structure, and update the driver to use the
one contained in struct net_device.
Signed-off-by: Philippe Reynes <tremyfr@gmail.com>
---
.../net/ethernet/stmicro/stmmac/dwmac-socfpga.c | 4 +-
drivers/net/ethernet/stmicro/stmmac/stmmac.h | 1 -
.../net/ethernet/stmicro/stmmac/stmmac_ethtool.c | 18 +++++-----
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 39 ++++++++++----------
4 files changed, 30 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c
index bec6963..5ad1dfb 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c
@@ -367,8 +367,8 @@ static int socfpga_dwmac_resume(struct device *dev)
* control register 0, and can be modified by the phy driver
* framework.
*/
- if (priv->phydev)
- phy_resume(priv->phydev);
+ if (ndev->phydev)
+ phy_resume(ndev->phydev);
return stmmac_resume(dev);
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
index 8dc9056..f94e028 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h
@@ -90,7 +90,6 @@ struct stmmac_priv {
struct mac_device_info *hw;
spinlock_t lock;
- struct phy_device *phydev ____cacheline_aligned_in_smp;
int oldlink;
int speed;
int oldduplex;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
index 1e06173..ac823bf 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
@@ -273,7 +273,7 @@ static int stmmac_ethtool_getsettings(struct net_device *dev,
struct ethtool_cmd *cmd)
{
struct stmmac_priv *priv = netdev_priv(dev);
- struct phy_device *phy = priv->phydev;
+ struct phy_device *phy = dev->phydev;
int rc;
if (priv->hw->pcs & STMMAC_PCS_RGMII ||
@@ -359,7 +359,7 @@ static int stmmac_ethtool_setsettings(struct net_device *dev,
struct ethtool_cmd *cmd)
{
struct stmmac_priv *priv = netdev_priv(dev);
- struct phy_device *phy = priv->phydev;
+ struct phy_device *phy = dev->phydev;
int rc;
if (priv->hw->pcs & STMMAC_PCS_RGMII ||
@@ -468,12 +468,12 @@ stmmac_get_pauseparam(struct net_device *netdev,
if (!adv_lp.pause)
return;
} else {
- if (!(priv->phydev->supported & SUPPORTED_Pause) ||
- !(priv->phydev->supported & SUPPORTED_Asym_Pause))
+ if (!(netdev->phydev->supported & SUPPORTED_Pause) ||
+ !(netdev->phydev->supported & SUPPORTED_Asym_Pause))
return;
}
- pause->autoneg = priv->phydev->autoneg;
+ pause->autoneg = netdev->phydev->autoneg;
if (priv->flow_ctrl & FLOW_RX)
pause->rx_pause = 1;
@@ -487,7 +487,7 @@ stmmac_set_pauseparam(struct net_device *netdev,
struct ethtool_pauseparam *pause)
{
struct stmmac_priv *priv = netdev_priv(netdev);
- struct phy_device *phy = priv->phydev;
+ struct phy_device *phy = netdev->phydev;
int new_pause = FLOW_OFF;
if (priv->hw->pcs && priv->hw->mac->pcs_get_adv_lp) {
@@ -547,7 +547,7 @@ static void stmmac_get_ethtool_stats(struct net_device *dev,
}
}
if (priv->eee_enabled) {
- int val = phy_get_eee_err(priv->phydev);
+ int val = phy_get_eee_err(dev->phydev);
if (val)
priv->xstats.phy_eee_wakeup_error_n = val;
}
@@ -666,7 +666,7 @@ static int stmmac_ethtool_op_get_eee(struct net_device *dev,
edata->eee_active = priv->eee_active;
edata->tx_lpi_timer = priv->tx_lpi_timer;
- return phy_ethtool_get_eee(priv->phydev, edata);
+ return phy_ethtool_get_eee(dev->phydev, edata);
}
static int stmmac_ethtool_op_set_eee(struct net_device *dev,
@@ -691,7 +691,7 @@ static int stmmac_ethtool_op_set_eee(struct net_device *dev,
priv->tx_lpi_timer = edata->tx_lpi_timer;
}
- return phy_ethtool_set_eee(priv->phydev, edata);
+ return phy_ethtool_set_eee(dev->phydev, edata);
}
static u32 stmmac_usec2riwt(u32 usec, struct stmmac_priv *priv)
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 4c8c60a..d815c82 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -221,7 +221,8 @@ static inline u32 stmmac_rx_dirty(struct stmmac_priv *priv)
*/
static inline void stmmac_hw_fix_mac_speed(struct stmmac_priv *priv)
{
- struct phy_device *phydev = priv->phydev;
+ struct net_device *ndev = priv->dev;
+ struct phy_device *phydev = ndev->phydev;
if (likely(priv->plat->fix_mac_speed))
priv->plat->fix_mac_speed(priv->plat->bsp_priv, phydev->speed);
@@ -279,6 +280,7 @@ static void stmmac_eee_ctrl_timer(unsigned long arg)
*/
bool stmmac_eee_init(struct stmmac_priv *priv)
{
+ struct net_device *ndev = priv->dev;
unsigned long flags;
bool ret = false;
@@ -295,7 +297,7 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
int tx_lpi_timer = priv->tx_lpi_timer;
/* Check if the PHY supports EEE */
- if (phy_init_eee(priv->phydev, 1)) {
+ if (phy_init_eee(ndev->phydev, 1)) {
/* To manage at run-time if the EEE cannot be supported
* anymore (for example because the lp caps have been
* changed).
@@ -327,7 +329,7 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
tx_lpi_timer);
}
/* Set HW EEE according to the speed */
- priv->hw->mac->set_eee_pls(priv->hw, priv->phydev->link);
+ priv->hw->mac->set_eee_pls(priv->hw, ndev->phydev->link);
ret = true;
spin_unlock_irqrestore(&priv->lock, flags);
@@ -691,7 +693,7 @@ static void stmmac_release_ptp(struct stmmac_priv *priv)
static void stmmac_adjust_link(struct net_device *dev)
{
struct stmmac_priv *priv = netdev_priv(dev);
- struct phy_device *phydev = priv->phydev;
+ struct phy_device *phydev = dev->phydev;
unsigned long flags;
int new_state = 0;
unsigned int fc = priv->flow_ctrl, pause_time = priv->pause;
@@ -874,8 +876,6 @@ static int stmmac_init_phy(struct net_device *dev)
pr_debug("stmmac_init_phy: %s: attached to PHY (UID 0x%x)"
" Link = %d\n", dev->name, phydev->phy_id, phydev->link);
- priv->phydev = phydev;
-
return 0;
}
@@ -1800,8 +1800,8 @@ static int stmmac_open(struct net_device *dev)
stmmac_init_tx_coalesce(priv);
- if (priv->phydev)
- phy_start(priv->phydev);
+ if (dev->phydev)
+ phy_start(dev->phydev);
/* Request the IRQ lines */
ret = request_irq(dev->irq, stmmac_interrupt,
@@ -1848,8 +1848,8 @@ wolirq_error:
init_error:
free_dma_desc_resources(priv);
dma_desc_error:
- if (priv->phydev)
- phy_disconnect(priv->phydev);
+ if (dev->phydev)
+ phy_disconnect(dev->phydev);
return ret;
}
@@ -1868,10 +1868,9 @@ static int stmmac_release(struct net_device *dev)
del_timer_sync(&priv->eee_ctrl_timer);
/* Stop and disconnect the PHY */
- if (priv->phydev) {
- phy_stop(priv->phydev);
- phy_disconnect(priv->phydev);
- priv->phydev = NULL;
+ if (dev->phydev) {
+ phy_stop(dev->phydev);
+ phy_disconnect(dev->phydev);
}
netif_stop_queue(dev);
@@ -2873,9 +2872,9 @@ static int stmmac_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
case SIOCGMIIPHY:
case SIOCGMIIREG:
case SIOCSMIIREG:
- if (!priv->phydev)
+ if (!dev->phydev)
return -EINVAL;
- ret = phy_mii_ioctl(priv->phydev, rq, cmd);
+ ret = phy_mii_ioctl(dev->phydev, rq, cmd);
break;
case SIOCSHWTSTAMP:
ret = stmmac_hwtstamp_ioctl(dev, rq);
@@ -3428,8 +3427,8 @@ int stmmac_suspend(struct device *dev)
if (!ndev || !netif_running(ndev))
return 0;
- if (priv->phydev)
- phy_stop(priv->phydev);
+ if (ndev->phydev)
+ phy_stop(ndev->phydev);
spin_lock_irqsave(&priv->lock, flags);
@@ -3523,8 +3522,8 @@ int stmmac_resume(struct device *dev)
spin_unlock_irqrestore(&priv->lock, flags);
- if (priv->phydev)
- phy_start(priv->phydev);
+ if (ndev->phydev)
+ phy_start(ndev->phydev);
return 0;
}
--
1.7.4.4
^ permalink raw reply related
* [PATCH 2/2] net: stmmac: use new api ethtool_{get|set}_link_ksettings
From: Philippe Reynes @ 2016-10-02 20:07 UTC (permalink / raw)
To: peppe.cavallaro, alexandre.torgue, davem
Cc: netdev, linux-kernel, Philippe Reynes
In-Reply-To: <1475438871-26735-1-git-send-email-tremyfr@gmail.com>
The ethtool api {get|set}_settings is deprecated.
We move this driver to new api {get|set}_link_ksettings.
Signed-off-by: Philippe Reynes <tremyfr@gmail.com>
---
.../net/ethernet/stmicro/stmmac/stmmac_ethtool.c | 97 +++++++++++--------
1 files changed, 56 insertions(+), 41 deletions(-)
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
index ac823bf..3fe9340 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c
@@ -269,8 +269,8 @@ static void stmmac_ethtool_getdrvinfo(struct net_device *dev,
strlcpy(info->version, DRV_MODULE_VERSION, sizeof(info->version));
}
-static int stmmac_ethtool_getsettings(struct net_device *dev,
- struct ethtool_cmd *cmd)
+static int stmmac_ethtool_get_link_ksettings(struct net_device *dev,
+ struct ethtool_link_ksettings *cmd)
{
struct stmmac_priv *priv = netdev_priv(dev);
struct phy_device *phy = dev->phydev;
@@ -279,15 +279,16 @@ static int stmmac_ethtool_getsettings(struct net_device *dev,
if (priv->hw->pcs & STMMAC_PCS_RGMII ||
priv->hw->pcs & STMMAC_PCS_SGMII) {
struct rgmii_adv adv;
+ u32 supported, advertising, lp_advertising;
if (!priv->xstats.pcs_link) {
- ethtool_cmd_speed_set(cmd, SPEED_UNKNOWN);
- cmd->duplex = DUPLEX_UNKNOWN;
+ cmd->base.speed = SPEED_UNKNOWN;
+ cmd->base.duplex = DUPLEX_UNKNOWN;
return 0;
}
- cmd->duplex = priv->xstats.pcs_duplex;
+ cmd->base.duplex = priv->xstats.pcs_duplex;
- ethtool_cmd_speed_set(cmd, priv->xstats.pcs_speed);
+ cmd->base.speed = priv->xstats.pcs_speed;
/* Get and convert ADV/LP_ADV from the HW AN registers */
if (!priv->hw->mac->pcs_get_adv_lp)
@@ -297,45 +298,59 @@ static int stmmac_ethtool_getsettings(struct net_device *dev,
/* Encoding of PSE bits is defined in 802.3z, 37.2.1.4 */
+ ethtool_convert_link_mode_to_legacy_u32(
+ &supported, cmd->link_modes.supported);
+ ethtool_convert_link_mode_to_legacy_u32(
+ &advertising, cmd->link_modes.advertising);
+ ethtool_convert_link_mode_to_legacy_u32(
+ &lp_advertising, cmd->link_modes.lp_advertising);
+
if (adv.pause & STMMAC_PCS_PAUSE)
- cmd->advertising |= ADVERTISED_Pause;
+ advertising |= ADVERTISED_Pause;
if (adv.pause & STMMAC_PCS_ASYM_PAUSE)
- cmd->advertising |= ADVERTISED_Asym_Pause;
+ advertising |= ADVERTISED_Asym_Pause;
if (adv.lp_pause & STMMAC_PCS_PAUSE)
- cmd->lp_advertising |= ADVERTISED_Pause;
+ lp_advertising |= ADVERTISED_Pause;
if (adv.lp_pause & STMMAC_PCS_ASYM_PAUSE)
- cmd->lp_advertising |= ADVERTISED_Asym_Pause;
+ lp_advertising |= ADVERTISED_Asym_Pause;
/* Reg49[3] always set because ANE is always supported */
- cmd->autoneg = ADVERTISED_Autoneg;
- cmd->supported |= SUPPORTED_Autoneg;
- cmd->advertising |= ADVERTISED_Autoneg;
- cmd->lp_advertising |= ADVERTISED_Autoneg;
+ cmd->base.autoneg = ADVERTISED_Autoneg;
+ supported |= SUPPORTED_Autoneg;
+ advertising |= ADVERTISED_Autoneg;
+ lp_advertising |= ADVERTISED_Autoneg;
if (adv.duplex) {
- cmd->supported |= (SUPPORTED_1000baseT_Full |
- SUPPORTED_100baseT_Full |
- SUPPORTED_10baseT_Full);
- cmd->advertising |= (ADVERTISED_1000baseT_Full |
- ADVERTISED_100baseT_Full |
- ADVERTISED_10baseT_Full);
+ supported |= (SUPPORTED_1000baseT_Full |
+ SUPPORTED_100baseT_Full |
+ SUPPORTED_10baseT_Full);
+ advertising |= (ADVERTISED_1000baseT_Full |
+ ADVERTISED_100baseT_Full |
+ ADVERTISED_10baseT_Full);
} else {
- cmd->supported |= (SUPPORTED_1000baseT_Half |
- SUPPORTED_100baseT_Half |
- SUPPORTED_10baseT_Half);
- cmd->advertising |= (ADVERTISED_1000baseT_Half |
- ADVERTISED_100baseT_Half |
- ADVERTISED_10baseT_Half);
+ supported |= (SUPPORTED_1000baseT_Half |
+ SUPPORTED_100baseT_Half |
+ SUPPORTED_10baseT_Half);
+ advertising |= (ADVERTISED_1000baseT_Half |
+ ADVERTISED_100baseT_Half |
+ ADVERTISED_10baseT_Half);
}
if (adv.lp_duplex)
- cmd->lp_advertising |= (ADVERTISED_1000baseT_Full |
- ADVERTISED_100baseT_Full |
- ADVERTISED_10baseT_Full);
+ lp_advertising |= (ADVERTISED_1000baseT_Full |
+ ADVERTISED_100baseT_Full |
+ ADVERTISED_10baseT_Full);
else
- cmd->lp_advertising |= (ADVERTISED_1000baseT_Half |
- ADVERTISED_100baseT_Half |
- ADVERTISED_10baseT_Half);
- cmd->port = PORT_OTHER;
+ lp_advertising |= (ADVERTISED_1000baseT_Half |
+ ADVERTISED_100baseT_Half |
+ ADVERTISED_10baseT_Half);
+ cmd->base.port = PORT_OTHER;
+
+ ethtool_convert_legacy_u32_to_link_mode(
+ cmd->link_modes.supported, supported);
+ ethtool_convert_legacy_u32_to_link_mode(
+ cmd->link_modes.advertising, advertising);
+ ethtool_convert_legacy_u32_to_link_mode(
+ cmd->link_modes.lp_advertising, lp_advertising);
return 0;
}
@@ -350,13 +365,13 @@ static int stmmac_ethtool_getsettings(struct net_device *dev,
"link speed / duplex setting\n", dev->name);
return -EBUSY;
}
- cmd->transceiver = XCVR_INTERNAL;
- rc = phy_ethtool_gset(phy, cmd);
+ rc = phy_ethtool_ksettings_get(phy, cmd);
return rc;
}
-static int stmmac_ethtool_setsettings(struct net_device *dev,
- struct ethtool_cmd *cmd)
+static int
+stmmac_ethtool_set_link_ksettings(struct net_device *dev,
+ const struct ethtool_link_ksettings *cmd)
{
struct stmmac_priv *priv = netdev_priv(dev);
struct phy_device *phy = dev->phydev;
@@ -367,7 +382,7 @@ static int stmmac_ethtool_setsettings(struct net_device *dev,
u32 mask = ADVERTISED_Autoneg | ADVERTISED_Pause;
/* Only support ANE */
- if (cmd->autoneg != AUTONEG_ENABLE)
+ if (cmd->base.autoneg != AUTONEG_ENABLE)
return -EINVAL;
mask &= (ADVERTISED_1000baseT_Half |
@@ -389,7 +404,7 @@ static int stmmac_ethtool_setsettings(struct net_device *dev,
}
spin_lock(&priv->lock);
- rc = phy_ethtool_sset(phy, cmd);
+ rc = phy_ethtool_ksettings_set(phy, cmd);
spin_unlock(&priv->lock);
return rc;
@@ -850,8 +865,6 @@ static int stmmac_set_tunable(struct net_device *dev,
static const struct ethtool_ops stmmac_ethtool_ops = {
.begin = stmmac_check_if_running,
.get_drvinfo = stmmac_ethtool_getdrvinfo,
- .get_settings = stmmac_ethtool_getsettings,
- .set_settings = stmmac_ethtool_setsettings,
.get_msglevel = stmmac_ethtool_getmsglevel,
.set_msglevel = stmmac_ethtool_setmsglevel,
.get_regs = stmmac_ethtool_gregs,
@@ -871,6 +884,8 @@ static const struct ethtool_ops stmmac_ethtool_ops = {
.set_coalesce = stmmac_set_coalesce,
.get_tunable = stmmac_get_tunable,
.set_tunable = stmmac_set_tunable,
+ .get_link_ksettings = stmmac_ethtool_get_link_ksettings,
+ .set_link_ksettings = stmmac_ethtool_set_link_ksettings,
};
void stmmac_set_ethtool_ops(struct net_device *netdev)
--
1.7.4.4
^ permalink raw reply related
* linux-next: manual merge of the net-next tree with Linus' tree
From: Stephen Rothwell @ 2016-10-02 22:32 UTC (permalink / raw)
To: David Miller, Networking; +Cc: linux-next, linux-kernel, Xin Long
Hi all,
Today's linux-next merge of the net-next tree got a conflict in:
net/sctp/outqueue.c
between commit:
be4947bf46cb ("sctp: change to check peer prsctp_capable when using prsctp polices")
from Linus' tree and commit:
2c89791eeb6f ("sctp: remove the unnecessary state check in sctp_outq_tail")
from the net-next tree.
I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging. You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.
--
Cheers,
Stephen Rothwell
diff --cc net/sctp/outqueue.c
index 107233da5cc9,3ec6da8bbb53..000000000000
--- a/net/sctp/outqueue.c
+++ b/net/sctp/outqueue.c
@@@ -299,42 -298,19 +298,19 @@@ void sctp_outq_tail(struct sctp_outq *q
* immediately.
*/
if (sctp_chunk_is_data(chunk)) {
- /* Is it OK to queue data chunks? */
- /* From 9. Termination of Association
- *
- * When either endpoint performs a shutdown, the
- * association on each peer will stop accepting new
- * data from its user and only deliver data in queue
- * at the time of sending or receiving the SHUTDOWN
- * chunk.
- */
- switch (q->asoc->state) {
- case SCTP_STATE_CLOSED:
- case SCTP_STATE_SHUTDOWN_PENDING:
- case SCTP_STATE_SHUTDOWN_SENT:
- case SCTP_STATE_SHUTDOWN_RECEIVED:
- case SCTP_STATE_SHUTDOWN_ACK_SENT:
- /* Cannot send after transport endpoint shutdown */
- error = -ESHUTDOWN;
- break;
-
- default:
- pr_debug("%s: outqueueing: outq:%p, chunk:%p[%s])\n",
- __func__, q, chunk, chunk && chunk->chunk_hdr ?
- sctp_cname(SCTP_ST_CHUNK(chunk->chunk_hdr->type)) :
- "illegal chunk");
-
- sctp_chunk_hold(chunk);
- sctp_outq_tail_data(q, chunk);
- if (chunk->asoc->peer.prsctp_capable &&
- SCTP_PR_PRIO_ENABLED(chunk->sinfo.sinfo_flags))
- chunk->asoc->sent_cnt_removable++;
- if (chunk->chunk_hdr->flags & SCTP_DATA_UNORDERED)
- SCTP_INC_STATS(net, SCTP_MIB_OUTUNORDERCHUNKS);
- else
- SCTP_INC_STATS(net, SCTP_MIB_OUTORDERCHUNKS);
- break;
- }
+ pr_debug("%s: outqueueing: outq:%p, chunk:%p[%s])\n",
+ __func__, q, chunk, chunk && chunk->chunk_hdr ?
+ sctp_cname(SCTP_ST_CHUNK(chunk->chunk_hdr->type)) :
+ "illegal chunk");
+
+ sctp_outq_tail_data(q, chunk);
- if (chunk->asoc->prsctp_enable &&
++ if (chunk->asoc->peer.prsctp_capable &&
+ SCTP_PR_PRIO_ENABLED(chunk->sinfo.sinfo_flags))
+ chunk->asoc->sent_cnt_removable++;
+ if (chunk->chunk_hdr->flags & SCTP_DATA_UNORDERED)
+ SCTP_INC_STATS(net, SCTP_MIB_OUTUNORDERCHUNKS);
+ else
+ SCTP_INC_STATS(net, SCTP_MIB_OUTORDERCHUNKS);
} else {
list_add_tail(&chunk->list, &q->control_chunk_list);
SCTP_INC_STATS(net, SCTP_MIB_OUTCTRLCHUNKS);
^ permalink raw reply
* linux-next: manual merge of the net-next tree with Linus' tree
From: Stephen Rothwell @ 2016-10-02 22:37 UTC (permalink / raw)
To: David Miller, Networking
Cc: linux-next, linux-kernel, Xin Long, Lorenzo Colitti
Hi all,
Today's linux-next merge of the net-next tree got a conflict in:
net/sctp/sctp_diag.c
between commit:
1cceda784980 ("sctp: fix the issue sctp_diag uses lock_sock in rcu_read_lock")
from Linus' tree and commit:
d545caca827b ("net: inet: diag: expose the socket mark to privileged processes.")
from the net-next tree.
I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging. You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.
--
Cheers,
Stephen Rothwell
diff --cc net/sctp/sctp_diag.c
index cef0cee182d4,807158e32f5f..000000000000
--- a/net/sctp/sctp_diag.c
+++ b/net/sctp/sctp_diag.c
@@@ -299,9 -314,10 +303,10 @@@ static int sctp_sock_dump(struct sock *
sk_user_ns(NETLINK_CB(cb->skb).sk),
NETLINK_CB(cb->skb).portid,
cb->nlh->nlmsg_seq,
- NLM_F_MULTI, cb->nlh) < 0) {
+ NLM_F_MULTI, cb->nlh,
+ commp->net_admin) < 0) {
cb->args[3] = 1;
- err = 2;
+ err = 1;
goto release;
}
cb->args[3] = 1;
@@@ -309,8 -325,9 +314,9 @@@
if (inet_sctp_diag_fill(sk, assoc, skb, r,
sk_user_ns(NETLINK_CB(cb->skb).sk),
NETLINK_CB(cb->skb).portid,
- cb->nlh->nlmsg_seq, 0, cb->nlh) < 0) {
+ cb->nlh->nlmsg_seq, 0, cb->nlh,
+ commp->net_admin) < 0) {
- err = 2;
+ err = 1;
goto release;
}
next:
^ permalink raw reply
* Re: [PATCH] Add netdev all_adj_list refcnt propagation to fix panic
From: David Miller @ 2016-10-03 0:17 UTC (permalink / raw)
To: acollins; +Cc: dsa, netdev, vfalico, nikolay
In-Reply-To: <715a8d0a-9245-fd83-bf58-70f863c721f7@cradlepoint.com>
From: Andrew Collins <acollins@cradlepoint.com>
Date: Wed, 28 Sep 2016 13:04:42 -0600
> On 09/28/2016 12:06 PM, David Ahern wrote:
>> Andrew Collins posted this patch as RFC in March:
>> http://patchwork.ozlabs.org/patch/603101/
>>
>> It has apparently fallen through the cracks and never applied.
>>
>> It solves a refcnt problem (thanks Nik for pointing out this patch)
>
> I have been running the patch in production for the past few months
> without issue, in spite of my concerns about the potential side
> effects of the change. That said, I suspect my use cases are a subset
> of yours.
>
> If nobody has a better fix yet, I'm happy to respin this patch as
> non-RFC with proper signoff.
Please do so.
^ permalink raw reply
* Re: [PATCH net] Panic when tc_lookup_action_n finds a partially initialized action.
From: Jamal Hadi Salim @ 2016-10-03 1:18 UTC (permalink / raw)
To: Krister Johansen; +Cc: netdev, Cong Wang
In-Reply-To: <20161002031349.GB2635@templeofstupid.com>
On 16-10-01 11:13 PM, Krister Johansen wrote:
> A tc_action_ops structure is visibile as soon as it is placed in the
> act_base list. When tcf_regsiter_action adds an item to this list and
> drops act_mod_lock, registration is not complete until
> register_pernet_subsys() finishes.
>
> If two threads attempt to modify a tc action in a way that triggers a
> module load, the thread that wins the race ends up defeferencing a NULL
> pointer after tcf_action_init_1() invokes a_o->init(). In the
> particular case that this submitter encountered, the panic occurred in
> tcf_gact_init() when net_generic() returned a NULL tc_action_net
> pointer. The gact_net_id needed to fetch the correct pointer was not
> yet set, because the register_pernet_subsys() call was pending in
> another thread.
>
> Fixes: ddf97ccdd7cb ("net_sched: add network namespace support for tc actions")
> Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
Looks reasonable to me but will let Cong a closer look since he added
that code.
cheers,
jamal
^ permalink raw reply
* Re: [PATCH v2 net-next 1/2] net: centralize net_device min/max MTU checking
From: David Miller @ 2016-10-03 2:43 UTC (permalink / raw)
To: jkbs; +Cc: jarod, linux-kernel, netdev
In-Reply-To: <87intdwwx7.fsf@redhat.com>
From: Jakub Sitnicki <jkbs@redhat.com>
Date: Fri, 30 Sep 2016 11:37:24 +0200
> On Wed, Sep 28, 2016 at 10:20 PM GMT, Jarod Wilson wrote:
>> While looking into an MTU issue with sfc, I started noticing that almost
>> every NIC driver with an ndo_change_mtu function implemented almost
>> exactly the same range checks, and in many cases, that was the only
>> practical thing their ndo_change_mtu function was doing. Quite a few
>> drivers have either 68, 64, 60 or 46 as their minimum MTU value checked,
>> and then various sizes from 1500 to 65535 for their maximum MTU value. We
>> can remove a whole lot of redundant code here if we simple store min_mtu
>> and max_mtu in net_device, and check against those in net/core/dev.c's
>> dev_set_mtu().
>>
>> In theory, there should be zero functional change with this patch, it just
>> puts the infrastructure in place. Subsequent patches will attempt to start
>> using said infrastructure, with theoretically zero change in
>> functionality.
>>
>> CC: "David S. Miller" <davem@davemloft.net>
>> CC: netdev@vger.kernel.org
>> Signed-off-by: Jarod Wilson <jarod@redhat.com>
>> ---
>
> [...]
>
>> diff --git a/net/core/dev.c b/net/core/dev.c
>> index c0c291f..5343799 100644
>> --- a/net/core/dev.c
>> +++ b/net/core/dev.c
>> @@ -6493,9 +6493,17 @@ int dev_set_mtu(struct net_device *dev, int new_mtu)
>> if (new_mtu == dev->mtu)
>> return 0;
>>
>> - /* MTU must be positive. */
>> - if (new_mtu < 0)
>> + if (new_mtu < dev->min_mtu) {
>
> Ouch, integral promotions. Looks like you need to keep the < 0 check.
> Otherwise new_mtu gets promoted to unsigned int and negative values will
> pass the check.
Agreed, the < 0 test must be reintroduced.
^ permalink raw reply
* Re: [PATCH] netfilter: bridge: clarify bridge/netfilter message
From: David Miller @ 2016-10-03 2:44 UTC (permalink / raw)
To: stefan; +Cc: netdev, bridge, linux-kernel, pablo
In-Reply-To: <20160928220528.17478-1-stefan@agner.ch>
From: Stefan Agner <stefan@agner.ch>
Date: Wed, 28 Sep 2016 15:05:28 -0700
> When using bridge without bridge netfilter enabled the message
> displayed is rather confusing and leads to belive that a deprecated
> feature is in use. Use IS_MODULE to be explicit that the message only
> affects users which use bridge netfilter as module and reword the
> message.
>
> Signed-off-by: Stefan Agner <stefan@agner.ch>
Applied to net-next, thanks.
^ permalink raw reply
* [PATCH] vmxnet3: Wake queue from reset work
From: Benjamin Poirier @ 2016-10-03 2:47 UTC (permalink / raw)
To: netdev; +Cc: Shrikrishna Khare, VMware, Inc.
vmxnet3_reset_work() expects tx queues to be stopped (via
vmxnet3_quiesce_dev -> netif_tx_disable). However, this races with the
netif_wake_queue() call in netif_tx_timeout() such that the driver's
start_xmit routine may be called unexpectedly, triggering one of the BUG_ON
in vmxnet3_map_pkt with a stack trace like this:
RIP: 0010:[<ffffffffa00cf4bc>] vmxnet3_map_pkt+0x3ac/0x4c0 [vmxnet3]
[<ffffffffa00cf7e0>] vmxnet3_tq_xmit+0x210/0x4e0 [vmxnet3]
[<ffffffff813ab144>] dev_hard_start_xmit+0x2e4/0x4c0
[<ffffffff813c956e>] sch_direct_xmit+0x17e/0x1e0
[<ffffffff813c96a7>] __qdisc_run+0xd7/0x130
[<ffffffff813a6a7a>] net_tx_action+0x10a/0x200
[<ffffffff810691df>] __do_softirq+0x11f/0x260
[<ffffffff81472fdc>] call_softirq+0x1c/0x30
[<ffffffff81004695>] do_softirq+0x65/0xa0
[<ffffffff81069b89>] local_bh_enable_ip+0x99/0xa0
[<ffffffffa031ff36>] destroy_conntrack+0x96/0x110 [nf_conntrack]
[<ffffffff813d65e2>] nf_conntrack_destroy+0x12/0x20
[<ffffffff8139c6d5>] skb_release_head_state+0xb5/0xf0
[<ffffffff8139d299>] skb_release_all+0x9/0x20
[<ffffffff8139cfe9>] __kfree_skb+0x9/0x90
[<ffffffffa00d0069>] vmxnet3_quiesce_dev+0x209/0x340 [vmxnet3]
[<ffffffffa00d020a>] vmxnet3_reset_work+0x6a/0xa0 [vmxnet3]
[<ffffffff8107d7cc>] process_one_work+0x16c/0x350
[<ffffffff810804fa>] worker_thread+0x17a/0x410
[<ffffffff810848c6>] kthread+0x96/0xa0
[<ffffffff81472ee4>] kernel_thread_helper+0x4/0x10
Signed-off-by: Benjamin Poirier <bpoirier@suse.com>
---
drivers/net/vmxnet3/vmxnet3_drv.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/vmxnet3/vmxnet3_drv.c b/drivers/net/vmxnet3/vmxnet3_drv.c
index 4244b9d..515f7aa 100644
--- a/drivers/net/vmxnet3/vmxnet3_drv.c
+++ b/drivers/net/vmxnet3/vmxnet3_drv.c
@@ -3186,7 +3186,6 @@ vmxnet3_tx_timeout(struct net_device *netdev)
netdev_err(adapter->netdev, "tx hang\n");
schedule_work(&adapter->work);
- netif_wake_queue(adapter->netdev);
}
@@ -3213,6 +3212,7 @@ vmxnet3_reset_work(struct work_struct *data)
}
rtnl_unlock();
+ netif_wake_queue(adapter->netdev);
clear_bit(VMXNET3_STATE_BIT_RESETTING, &adapter->state);
}
--
2.9.3
^ permalink raw reply related
* Re: [PATCH] ipv6 addrconf: remove addrconf_sysctl_hop_limit()
From: David Miller @ 2016-10-03 3:48 UTC (permalink / raw)
To: zenczykowski; +Cc: maze, netdev, ek, lorenzo
In-Reply-To: <1475134423-23635-1-git-send-email-zenczykowski@gmail.com>
From: Maciej Żenczykowski <zenczykowski@gmail.com>
Date: Thu, 29 Sep 2016 00:33:43 -0700
> From: Maciej Żenczykowski <maze@google.com>
>
> This is an effective no-op in terms of user observable behaviour.
>
> By preventing the overwrite of non-null extra1/extra2 fields
> in addrconf_sysctl() we can enable the use of proc_dointvec_minmax().
>
> This allows us to eliminate the constant min/max (1..255) trampoline
> function that is addrconf_sysctl_hop_limit().
>
> This is nice because it simplifies the code, and allows future
> sysctls with constant min/max limits to also not require trampolines.
>
> We still can't eliminate the trampoline for mtu because it isn't
> actually a constant (it depends on other tunables of the device)
> and thus requires at-write-time logic to enforce range.
>
> Signed-off-by: Maciej Żenczykowski <maze@google.com>
Applied, thanks.
^ permalink raw reply
* [net-next:master 249/527] emac-mac.c:undefined reference to `bad_dma_ops'
From: kbuild test robot @ 2016-10-03 4:06 UTC (permalink / raw)
To: Timur Tabi; +Cc: kbuild-all, netdev
[-- Attachment #1: Type: text/plain, Size: 2377 bytes --]
Hi Timur,
FYI, the error/warning still remains.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git master
head: d4ef9f72128d414ad83b27b49312faa971d77382
commit: b9b17debc69d27cd55e21ee51a5ba7fc50a426cf [249/527] net: emac: emac gigabit ethernet controller driver
config: m32r-allyesconfig (attached as .config)
compiler: m32r-linux-gcc (GCC) 6.2.0
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
git checkout b9b17debc69d27cd55e21ee51a5ba7fc50a426cf
# save the attached .config to linux build tree
make.cross ARCH=m32r
All errors (new ones prefixed by >>):
drivers/built-in.o: In function `emac_probe':
emac.c:(.text+0x3de178): undefined reference to `bad_dma_ops'
emac.c:(.text+0x3de180): undefined reference to `bad_dma_ops'
drivers/built-in.o: In function `emac_tx_q_descs_free':
>> emac-mac.c:(.text+0x3de924): undefined reference to `bad_dma_ops'
emac-mac.c:(.text+0x3de928): undefined reference to `bad_dma_ops'
drivers/built-in.o: In function `emac_rx_q_free_descs':
emac-mac.c:(.text+0x3de9f8): undefined reference to `bad_dma_ops'
drivers/built-in.o:emac-mac.c:(.text+0x3de9fc): more undefined references to `bad_dma_ops' follow
sound/built-in.o: In function `snd_pcm_lib_default_mmap':
(.text+0xfc64): undefined reference to `dma_common_mmap'
sound/built-in.o: In function `snd_pcm_lib_default_mmap':
(.text+0xfc64): relocation truncated to fit: R_M32R_26_PCREL_RELA against undefined symbol `dma_common_mmap'
sound/built-in.o: In function `cygnus_pcm_preallocate_dma_buffer':
cygnus-pcm.c:(.text+0x10fff0): undefined reference to `bad_dma_ops'
cygnus-pcm.c:(.text+0x10fff4): undefined reference to `bad_dma_ops'
cygnus-pcm.c:(.text+0x110028): undefined reference to `bad_dma_ops'
sound/built-in.o: In function `cygnus_dma_free_dma_buffers':
cygnus-pcm.c:(.text+0x110128): undefined reference to `bad_dma_ops'
cygnus-pcm.c:(.text+0x110130): undefined reference to `bad_dma_ops'
sound/built-in.o:cygnus-pcm.c:(.text+0x1101c8): more undefined references to `bad_dma_ops' follow
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 36410 bytes --]
^ permalink raw reply
* Re: [PATCH] cxgb4: mark cxgb_setup_tc() static
From: David Miller @ 2016-10-03 5:20 UTC (permalink / raw)
To: baoyou.xie
Cc: hariprasad, netdev, linux-kernel, arnd, xie.baoyou, han.fei,
tang.qiang007
In-Reply-To: <1475220865-18321-1-git-send-email-baoyou.xie@linaro.org>
From: Baoyou Xie <baoyou.xie@linaro.org>
Date: Fri, 30 Sep 2016 15:34:25 +0800
> We get 1 warning when building kernel with W=1:
> drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c:2715:5: warning: no previous prototype for 'cxgb_setup_tc' [-Wmissing-prototypes]
>
> In fact, this function is only used in the file in which it is
> declared and don't need a declaration, but can be made static.
> so this patch marks this function with 'static'.
>
> Signed-off-by: Baoyou Xie <baoyou.xie@linaro.org>
Applied.
^ permalink raw reply
* Re: [PATCH] net: ethernet: mediatek: mark symbols static where possible
From: David Miller @ 2016-10-03 5:24 UTC (permalink / raw)
To: baoyou.xie
Cc: nbd, blogic, matthias.bgg, netdev, linux-arm-kernel,
linux-mediatek, linux-kernel, arnd, xie.baoyou, han.fei,
tang.qiang007
In-Reply-To: <1475221730-18773-1-git-send-email-baoyou.xie@linaro.org>
From: Baoyou Xie <baoyou.xie@linaro.org>
Date: Fri, 30 Sep 2016 15:48:50 +0800
> We get 2 warnings when building kernel with W=1:
> drivers/net/ethernet/mediatek/mtk_eth_soc.c:2041:5: warning: no previous prototype for 'mtk_get_link_ksettings' [-Wmissing-prototypes]
> drivers/net/ethernet/mediatek/mtk_eth_soc.c:2052:5: warning: no previous prototype for 'mtk_set_link_ksettings' [-Wmissing-prototypes]
>
> In fact, these functions are only used in the file in which they are
> declared and don't need a declaration, but can be made static.
> So this patch marks these functions with 'static'.
>
> Signed-off-by: Baoyou Xie <baoyou.xie@linaro.org>
Applied, thanks.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox