* [PATCH v15 net-next 10/11] bpf: verifier (add verifier core)
From: Alexei Starovoitov @ 2014-09-26 7:17 UTC (permalink / raw)
To: David S. Miller
Cc: Ingo Molnar, Linus Torvalds, Andy Lutomirski, Daniel Borkmann,
Hannes Frederic Sowa, Chema Gonzalez, Eric Dumazet,
Peter Zijlstra, Pablo Neira Ayuso, H. Peter Anvin, Andrew Morton,
Kees Cook, linux-api, netdev, linux-kernel
In-Reply-To: <1411715827-10430-1-git-send-email-ast@plumgrid.com>
This patch adds verifier core which simulates execution of every insn and
records the state of registers and program stack. Every branch instruction seen
during simulation is pushed into state stack. When verifier reaches BPF_EXIT,
it pops the state from the stack and continues until it reaches BPF_EXIT again.
For program:
1: bpf_mov r1, xxx
2: if (r1 == 0) goto 5
3: bpf_mov r0, 1
4: goto 6
5: bpf_mov r0, 2
6: bpf_exit
The verifier will walk insns: 1, 2, 3, 4, 6
then it will pop the state recorded at insn#2 and will continue: 5, 6
This way it walks all possible paths through the program and checks all
possible values of registers. While doing so, it checks for:
- invalid instructions
- uninitialized register access
- uninitialized stack access
- misaligned stack access
- out of range stack access
- invalid calling convention
- instruction encoding is not using reserved fields
Kernel subsystem configures the verifier with two callbacks:
- bool (*is_valid_access)(int off, int size, enum bpf_access_type type);
that provides information to the verifer which fields of 'ctx'
are accessible (remember 'ctx' is the first argument to eBPF program)
- const struct bpf_func_proto *(*get_func_proto)(enum bpf_func_id func_id);
returns argument constraints of kernel helper functions that eBPF program
may call, so that verifier can checks that R1-R5 types match the prototype
More details in Documentation/networking/filter.txt and in kernel/bpf/verifier.c
Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
---
include/linux/bpf.h | 47 +++
kernel/bpf/verifier.c | 1075 ++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 1121 insertions(+), 1 deletion(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 9dfeb36f8971..3cf91754a957 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -46,6 +46,31 @@ void bpf_register_map_type(struct bpf_map_type_list *tl);
void bpf_map_put(struct bpf_map *map);
struct bpf_map *bpf_map_get(struct fd f);
+/* function argument constraints */
+enum bpf_arg_type {
+ ARG_ANYTHING = 0, /* any argument is ok */
+
+ /* the following constraints used to prototype
+ * bpf_map_lookup/update/delete_elem() functions
+ */
+ ARG_CONST_MAP_PTR, /* const argument used as pointer to bpf_map */
+ ARG_PTR_TO_MAP_KEY, /* pointer to stack used as map key */
+ ARG_PTR_TO_MAP_VALUE, /* pointer to stack used as map value */
+
+ /* the following constraints used to prototype bpf_memcmp() and other
+ * functions that access data on eBPF program stack
+ */
+ ARG_PTR_TO_STACK, /* any pointer to eBPF program stack */
+ ARG_CONST_STACK_SIZE, /* number of bytes accessed from stack */
+};
+
+/* type of values returned from helper functions */
+enum bpf_return_type {
+ RET_INTEGER, /* function returns integer */
+ RET_VOID, /* function doesn't return anything */
+ RET_PTR_TO_MAP_VALUE_OR_NULL, /* returns a pointer to map elem value or NULL */
+};
+
/* eBPF function prototype used by verifier to allow BPF_CALLs from eBPF programs
* to in-kernel helper functions and for adjusting imm32 field in BPF_CALL
* instructions after verifying
@@ -53,11 +78,33 @@ struct bpf_map *bpf_map_get(struct fd f);
struct bpf_func_proto {
u64 (*func)(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5);
bool gpl_only;
+ enum bpf_return_type ret_type;
+ enum bpf_arg_type arg1_type;
+ enum bpf_arg_type arg2_type;
+ enum bpf_arg_type arg3_type;
+ enum bpf_arg_type arg4_type;
+ enum bpf_arg_type arg5_type;
+};
+
+/* bpf_context is intentionally undefined structure. Pointer to bpf_context is
+ * the first argument to eBPF programs.
+ * For socket filters: 'struct bpf_context *' == 'struct sk_buff *'
+ */
+struct bpf_context;
+
+enum bpf_access_type {
+ BPF_READ = 1,
+ BPF_WRITE = 2
};
struct bpf_verifier_ops {
/* return eBPF function prototype for verification */
const struct bpf_func_proto *(*get_func_proto)(enum bpf_func_id func_id);
+
+ /* return true if 'size' wide access at offset 'off' within bpf_context
+ * with 'type' (read or write) is allowed
+ */
+ bool (*is_valid_access)(int off, int size, enum bpf_access_type type);
};
struct bpf_prog_type_list {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c689ab8e2713..a086dd3210a8 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -125,6 +125,70 @@
* are set to NOT_INIT to indicate that they are no longer readable.
*/
+/* types of values stored in eBPF registers */
+enum bpf_reg_type {
+ NOT_INIT = 0, /* nothing was written into register */
+ UNKNOWN_VALUE, /* reg doesn't contain a valid pointer */
+ PTR_TO_CTX, /* reg points to bpf_context */
+ CONST_PTR_TO_MAP, /* reg points to struct bpf_map */
+ PTR_TO_MAP_VALUE, /* reg points to map element value */
+ PTR_TO_MAP_VALUE_OR_NULL,/* points to map elem value or NULL */
+ FRAME_PTR, /* reg == frame_pointer */
+ PTR_TO_STACK, /* reg == frame_pointer + imm */
+ CONST_IMM, /* constant integer value */
+};
+
+struct reg_state {
+ enum bpf_reg_type type;
+ union {
+ /* valid when type == CONST_IMM | PTR_TO_STACK */
+ int imm;
+
+ /* valid when type == CONST_PTR_TO_MAP | PTR_TO_MAP_VALUE |
+ * PTR_TO_MAP_VALUE_OR_NULL
+ */
+ struct bpf_map *map_ptr;
+ };
+};
+
+enum bpf_stack_slot_type {
+ STACK_INVALID, /* nothing was stored in this stack slot */
+ STACK_SPILL, /* 1st byte of register spilled into stack */
+ STACK_SPILL_PART, /* other 7 bytes of register spill */
+ STACK_MISC /* BPF program wrote some data into this slot */
+};
+
+struct bpf_stack_slot {
+ enum bpf_stack_slot_type stype;
+ struct reg_state reg_st;
+};
+
+/* state of the program:
+ * type of all registers and stack info
+ */
+struct verifier_state {
+ struct reg_state regs[MAX_BPF_REG];
+ struct bpf_stack_slot stack[MAX_BPF_STACK];
+};
+
+/* linked list of verifier states used to prune search */
+struct verifier_state_list {
+ struct verifier_state state;
+ struct verifier_state_list *next;
+};
+
+/* verifier_state + insn_idx are pushed to stack when branch is encountered */
+struct verifier_stack_elem {
+ /* verifer state is 'st'
+ * before processing instruction 'insn_idx'
+ * and after processing instruction 'prev_insn_idx'
+ */
+ struct verifier_state st;
+ int insn_idx;
+ int prev_insn_idx;
+ struct verifier_stack_elem *next;
+};
+
#define MAX_USED_MAPS 64 /* max number of maps accessed by one eBPF program */
/* single container for all structs
@@ -132,6 +196,9 @@
*/
struct verifier_env {
struct bpf_prog *prog; /* eBPF program being verified */
+ struct verifier_stack_elem *head; /* stack of verifier states to be processed */
+ int stack_size; /* number of states to be processed */
+ struct verifier_state cur_state; /* current verifier state */
struct bpf_map *used_maps[MAX_USED_MAPS]; /* array of map's used by eBPF program */
u32 used_map_cnt; /* number of used maps */
};
@@ -160,6 +227,45 @@ static void verbose(const char *fmt, ...)
va_end(args);
}
+/* string representation of 'enum bpf_reg_type' */
+static const char * const reg_type_str[] = {
+ [NOT_INIT] = "?",
+ [UNKNOWN_VALUE] = "inv",
+ [PTR_TO_CTX] = "ctx",
+ [CONST_PTR_TO_MAP] = "map_ptr",
+ [PTR_TO_MAP_VALUE] = "map_value",
+ [PTR_TO_MAP_VALUE_OR_NULL] = "map_value_or_null",
+ [FRAME_PTR] = "fp",
+ [PTR_TO_STACK] = "fp",
+ [CONST_IMM] = "imm",
+};
+
+static void print_verifier_state(struct verifier_env *env)
+{
+ enum bpf_reg_type t;
+ int i;
+
+ for (i = 0; i < MAX_BPF_REG; i++) {
+ t = env->cur_state.regs[i].type;
+ if (t == NOT_INIT)
+ continue;
+ verbose(" R%d=%s", i, reg_type_str[t]);
+ if (t == CONST_IMM || t == PTR_TO_STACK)
+ verbose("%d", env->cur_state.regs[i].imm);
+ else if (t == CONST_PTR_TO_MAP || t == PTR_TO_MAP_VALUE ||
+ t == PTR_TO_MAP_VALUE_OR_NULL)
+ verbose("(ks=%d,vs=%d)",
+ env->cur_state.regs[i].map_ptr->key_size,
+ env->cur_state.regs[i].map_ptr->value_size);
+ }
+ for (i = 0; i < MAX_BPF_STACK; i++) {
+ if (env->cur_state.stack[i].stype == STACK_SPILL)
+ verbose(" fp%d=%s", -MAX_BPF_STACK + i,
+ reg_type_str[env->cur_state.stack[i].reg_st.type]);
+ }
+ verbose("\n");
+}
+
static const char *const bpf_class_string[] = {
[BPF_LD] = "ld",
[BPF_LDX] = "ldx",
@@ -305,6 +411,735 @@ static void print_bpf_insn(struct bpf_insn *insn)
}
}
+static int pop_stack(struct verifier_env *env, int *prev_insn_idx)
+{
+ struct verifier_stack_elem *elem;
+ int insn_idx;
+
+ if (env->head == NULL)
+ return -1;
+
+ memcpy(&env->cur_state, &env->head->st, sizeof(env->cur_state));
+ insn_idx = env->head->insn_idx;
+ if (prev_insn_idx)
+ *prev_insn_idx = env->head->prev_insn_idx;
+ elem = env->head->next;
+ kfree(env->head);
+ env->head = elem;
+ env->stack_size--;
+ return insn_idx;
+}
+
+static struct verifier_state *push_stack(struct verifier_env *env, int insn_idx,
+ int prev_insn_idx)
+{
+ struct verifier_stack_elem *elem;
+
+ elem = kmalloc(sizeof(struct verifier_stack_elem), GFP_KERNEL);
+ if (!elem)
+ goto err;
+
+ memcpy(&elem->st, &env->cur_state, sizeof(env->cur_state));
+ elem->insn_idx = insn_idx;
+ elem->prev_insn_idx = prev_insn_idx;
+ elem->next = env->head;
+ env->head = elem;
+ env->stack_size++;
+ if (env->stack_size > 1024) {
+ verbose("BPF program is too complex\n");
+ goto err;
+ }
+ return &elem->st;
+err:
+ /* pop all elements and return */
+ while (pop_stack(env, NULL) >= 0);
+ return NULL;
+}
+
+#define CALLER_SAVED_REGS 6
+static const int caller_saved[CALLER_SAVED_REGS] = {
+ BPF_REG_0, BPF_REG_1, BPF_REG_2, BPF_REG_3, BPF_REG_4, BPF_REG_5
+};
+
+static void init_reg_state(struct reg_state *regs)
+{
+ int i;
+
+ for (i = 0; i < MAX_BPF_REG; i++) {
+ regs[i].type = NOT_INIT;
+ regs[i].imm = 0;
+ regs[i].map_ptr = NULL;
+ }
+
+ /* frame pointer */
+ regs[BPF_REG_FP].type = FRAME_PTR;
+
+ /* 1st arg to a function */
+ regs[BPF_REG_1].type = PTR_TO_CTX;
+}
+
+static void mark_reg_unknown_value(struct reg_state *regs, u32 regno)
+{
+ BUG_ON(regno >= MAX_BPF_REG);
+ regs[regno].type = UNKNOWN_VALUE;
+ regs[regno].imm = 0;
+ regs[regno].map_ptr = NULL;
+}
+
+enum reg_arg_type {
+ SRC_OP, /* register is used as source operand */
+ DST_OP, /* register is used as destination operand */
+ DST_OP_NO_MARK /* same as above, check only, don't mark */
+};
+
+static int check_reg_arg(struct reg_state *regs, u32 regno,
+ enum reg_arg_type t)
+{
+ if (regno >= MAX_BPF_REG) {
+ verbose("R%d is invalid\n", regno);
+ return -EINVAL;
+ }
+
+ if (t == SRC_OP) {
+ /* check whether register used as source operand can be read */
+ if (regs[regno].type == NOT_INIT) {
+ verbose("R%d !read_ok\n", regno);
+ return -EACCES;
+ }
+ } else {
+ /* check whether register used as dest operand can be written to */
+ if (regno == BPF_REG_FP) {
+ verbose("frame pointer is read only\n");
+ return -EACCES;
+ }
+ if (t == DST_OP)
+ mark_reg_unknown_value(regs, regno);
+ }
+ return 0;
+}
+
+static int bpf_size_to_bytes(int bpf_size)
+{
+ if (bpf_size == BPF_W)
+ return 4;
+ else if (bpf_size == BPF_H)
+ return 2;
+ else if (bpf_size == BPF_B)
+ return 1;
+ else if (bpf_size == BPF_DW)
+ return 8;
+ else
+ return -EINVAL;
+}
+
+/* check_stack_read/write functions track spill/fill of registers,
+ * stack boundary and alignment are checked in check_mem_access()
+ */
+static int check_stack_write(struct verifier_state *state, int off, int size,
+ int value_regno)
+{
+ struct bpf_stack_slot *slot;
+ int i;
+
+ if (value_regno >= 0 &&
+ (state->regs[value_regno].type == PTR_TO_MAP_VALUE ||
+ state->regs[value_regno].type == PTR_TO_STACK ||
+ state->regs[value_regno].type == PTR_TO_CTX)) {
+
+ /* register containing pointer is being spilled into stack */
+ if (size != 8) {
+ verbose("invalid size of register spill\n");
+ return -EACCES;
+ }
+
+ slot = &state->stack[MAX_BPF_STACK + off];
+ slot->stype = STACK_SPILL;
+ /* save register state */
+ slot->reg_st = state->regs[value_regno];
+ for (i = 1; i < 8; i++) {
+ slot = &state->stack[MAX_BPF_STACK + off + i];
+ slot->stype = STACK_SPILL_PART;
+ slot->reg_st.type = UNKNOWN_VALUE;
+ slot->reg_st.map_ptr = NULL;
+ }
+ } else {
+
+ /* regular write of data into stack */
+ for (i = 0; i < size; i++) {
+ slot = &state->stack[MAX_BPF_STACK + off + i];
+ slot->stype = STACK_MISC;
+ slot->reg_st.type = UNKNOWN_VALUE;
+ slot->reg_st.map_ptr = NULL;
+ }
+ }
+ return 0;
+}
+
+static int check_stack_read(struct verifier_state *state, int off, int size,
+ int value_regno)
+{
+ int i;
+ struct bpf_stack_slot *slot;
+
+ slot = &state->stack[MAX_BPF_STACK + off];
+
+ if (slot->stype == STACK_SPILL) {
+ if (size != 8) {
+ verbose("invalid size of register spill\n");
+ return -EACCES;
+ }
+ for (i = 1; i < 8; i++) {
+ if (state->stack[MAX_BPF_STACK + off + i].stype !=
+ STACK_SPILL_PART) {
+ verbose("corrupted spill memory\n");
+ return -EACCES;
+ }
+ }
+
+ if (value_regno >= 0)
+ /* restore register state from stack */
+ state->regs[value_regno] = slot->reg_st;
+ return 0;
+ } else {
+ for (i = 0; i < size; i++) {
+ if (state->stack[MAX_BPF_STACK + off + i].stype !=
+ STACK_MISC) {
+ verbose("invalid read from stack off %d+%d size %d\n",
+ off, i, size);
+ return -EACCES;
+ }
+ }
+ if (value_regno >= 0)
+ /* have read misc data from the stack */
+ mark_reg_unknown_value(state->regs, value_regno);
+ return 0;
+ }
+}
+
+/* check read/write into map element returned by bpf_map_lookup_elem() */
+static int check_map_access(struct verifier_env *env, u32 regno, int off,
+ int size)
+{
+ struct bpf_map *map = env->cur_state.regs[regno].map_ptr;
+
+ if (off < 0 || off + size > map->value_size) {
+ verbose("invalid access to map value, value_size=%d off=%d size=%d\n",
+ map->value_size, off, size);
+ return -EACCES;
+ }
+ return 0;
+}
+
+/* check access to 'struct bpf_context' fields */
+static int check_ctx_access(struct verifier_env *env, int off, int size,
+ enum bpf_access_type t)
+{
+ if (env->prog->aux->ops->is_valid_access &&
+ env->prog->aux->ops->is_valid_access(off, size, t))
+ return 0;
+
+ verbose("invalid bpf_context access off=%d size=%d\n", off, size);
+ return -EACCES;
+}
+
+/* check whether memory at (regno + off) is accessible for t = (read | write)
+ * if t==write, value_regno is a register which value is stored into memory
+ * if t==read, value_regno is a register which will receive the value from memory
+ * if t==write && value_regno==-1, some unknown value is stored into memory
+ * if t==read && value_regno==-1, don't care what we read from memory
+ */
+static int check_mem_access(struct verifier_env *env, u32 regno, int off,
+ int bpf_size, enum bpf_access_type t,
+ int value_regno)
+{
+ struct verifier_state *state = &env->cur_state;
+ int size, err = 0;
+
+ size = bpf_size_to_bytes(bpf_size);
+ if (size < 0)
+ return size;
+
+ if (off % size != 0) {
+ verbose("misaligned access off %d size %d\n", off, size);
+ return -EACCES;
+ }
+
+ if (state->regs[regno].type == PTR_TO_MAP_VALUE) {
+ err = check_map_access(env, regno, off, size);
+ if (!err && t == BPF_READ && value_regno >= 0)
+ mark_reg_unknown_value(state->regs, value_regno);
+
+ } else if (state->regs[regno].type == PTR_TO_CTX) {
+ err = check_ctx_access(env, off, size, t);
+ if (!err && t == BPF_READ && value_regno >= 0)
+ mark_reg_unknown_value(state->regs, value_regno);
+
+ } else if (state->regs[regno].type == FRAME_PTR) {
+ if (off >= 0 || off < -MAX_BPF_STACK) {
+ verbose("invalid stack off=%d size=%d\n", off, size);
+ return -EACCES;
+ }
+ if (t == BPF_WRITE)
+ err = check_stack_write(state, off, size, value_regno);
+ else
+ err = check_stack_read(state, off, size, value_regno);
+ } else {
+ verbose("R%d invalid mem access '%s'\n",
+ regno, reg_type_str[state->regs[regno].type]);
+ return -EACCES;
+ }
+ return err;
+}
+
+static int check_xadd(struct verifier_env *env, struct bpf_insn *insn)
+{
+ struct reg_state *regs = env->cur_state.regs;
+ int err;
+
+ if ((BPF_SIZE(insn->code) != BPF_W && BPF_SIZE(insn->code) != BPF_DW) ||
+ insn->imm != 0) {
+ verbose("BPF_XADD uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ /* check src1 operand */
+ err = check_reg_arg(regs, insn->src_reg, SRC_OP);
+ if (err)
+ return err;
+
+ /* check src2 operand */
+ err = check_reg_arg(regs, insn->dst_reg, SRC_OP);
+ if (err)
+ return err;
+
+ /* check whether atomic_add can read the memory */
+ err = check_mem_access(env, insn->dst_reg, insn->off,
+ BPF_SIZE(insn->code), BPF_READ, -1);
+ if (err)
+ return err;
+
+ /* check whether atomic_add can write into the same memory */
+ return check_mem_access(env, insn->dst_reg, insn->off,
+ BPF_SIZE(insn->code), BPF_WRITE, -1);
+}
+
+/* when register 'regno' is passed into function that will read 'access_size'
+ * bytes from that pointer, make sure that it's within stack boundary
+ * and all elements of stack are initialized
+ */
+static int check_stack_boundary(struct verifier_env *env,
+ int regno, int access_size)
+{
+ struct verifier_state *state = &env->cur_state;
+ struct reg_state *regs = state->regs;
+ int off, i;
+
+ if (regs[regno].type != PTR_TO_STACK)
+ return -EACCES;
+
+ off = regs[regno].imm;
+ if (off >= 0 || off < -MAX_BPF_STACK || off + access_size > 0 ||
+ access_size <= 0) {
+ verbose("invalid stack type R%d off=%d access_size=%d\n",
+ regno, off, access_size);
+ return -EACCES;
+ }
+
+ for (i = 0; i < access_size; i++) {
+ if (state->stack[MAX_BPF_STACK + off + i].stype != STACK_MISC) {
+ verbose("invalid indirect read from stack off %d+%d size %d\n",
+ off, i, access_size);
+ return -EACCES;
+ }
+ }
+ return 0;
+}
+
+static int check_func_arg(struct verifier_env *env, u32 regno,
+ enum bpf_arg_type arg_type, struct bpf_map **mapp)
+{
+ struct reg_state *reg = env->cur_state.regs + regno;
+ enum bpf_reg_type expected_type;
+ int err = 0;
+
+ if (arg_type == ARG_ANYTHING)
+ return 0;
+
+ if (reg->type == NOT_INIT) {
+ verbose("R%d !read_ok\n", regno);
+ return -EACCES;
+ }
+
+ if (arg_type == ARG_PTR_TO_STACK || arg_type == ARG_PTR_TO_MAP_KEY ||
+ arg_type == ARG_PTR_TO_MAP_VALUE) {
+ expected_type = PTR_TO_STACK;
+ } else if (arg_type == ARG_CONST_STACK_SIZE) {
+ expected_type = CONST_IMM;
+ } else if (arg_type == ARG_CONST_MAP_PTR) {
+ expected_type = CONST_PTR_TO_MAP;
+ } else {
+ verbose("unsupported arg_type %d\n", arg_type);
+ return -EFAULT;
+ }
+
+ if (reg->type != expected_type) {
+ verbose("R%d type=%s expected=%s\n", regno,
+ reg_type_str[reg->type], reg_type_str[expected_type]);
+ return -EACCES;
+ }
+
+ if (arg_type == ARG_CONST_MAP_PTR) {
+ /* bpf_map_xxx(map_ptr) call: remember that map_ptr */
+ *mapp = reg->map_ptr;
+
+ } else if (arg_type == ARG_PTR_TO_MAP_KEY) {
+ /* bpf_map_xxx(..., map_ptr, ..., key) call:
+ * check that [key, key + map->key_size) are within
+ * stack limits and initialized
+ */
+ if (!*mapp) {
+ /* in function declaration map_ptr must come before
+ * map_key, so that it's verified and known before
+ * we have to check map_key here. Otherwise it means
+ * that kernel subsystem misconfigured verifier
+ */
+ verbose("invalid map_ptr to access map->key\n");
+ return -EACCES;
+ }
+ err = check_stack_boundary(env, regno, (*mapp)->key_size);
+
+ } else if (arg_type == ARG_PTR_TO_MAP_VALUE) {
+ /* bpf_map_xxx(..., map_ptr, ..., value) call:
+ * check [value, value + map->value_size) validity
+ */
+ if (!*mapp) {
+ /* kernel subsystem misconfigured verifier */
+ verbose("invalid map_ptr to access map->value\n");
+ return -EACCES;
+ }
+ err = check_stack_boundary(env, regno, (*mapp)->value_size);
+
+ } else if (arg_type == ARG_CONST_STACK_SIZE) {
+ /* bpf_xxx(..., buf, len) call will access 'len' bytes
+ * from stack pointer 'buf'. Check it
+ * note: regno == len, regno - 1 == buf
+ */
+ if (regno == 0) {
+ /* kernel subsystem misconfigured verifier */
+ verbose("ARG_CONST_STACK_SIZE cannot be first argument\n");
+ return -EACCES;
+ }
+ err = check_stack_boundary(env, regno - 1, reg->imm);
+ }
+
+ return err;
+}
+
+static int check_call(struct verifier_env *env, int func_id)
+{
+ struct verifier_state *state = &env->cur_state;
+ const struct bpf_func_proto *fn = NULL;
+ struct reg_state *regs = state->regs;
+ struct bpf_map *map = NULL;
+ struct reg_state *reg;
+ int i, err;
+
+ /* find function prototype */
+ if (func_id < 0 || func_id >= __BPF_FUNC_MAX_ID) {
+ verbose("invalid func %d\n", func_id);
+ return -EINVAL;
+ }
+
+ if (env->prog->aux->ops->get_func_proto)
+ fn = env->prog->aux->ops->get_func_proto(func_id);
+
+ if (!fn) {
+ verbose("unknown func %d\n", func_id);
+ return -EINVAL;
+ }
+
+ /* eBPF programs must be GPL compatible to use GPL-ed functions */
+ if (!env->prog->aux->is_gpl_compatible && fn->gpl_only) {
+ verbose("cannot call GPL only function from proprietary program\n");
+ return -EINVAL;
+ }
+
+ /* check args */
+ err = check_func_arg(env, BPF_REG_1, fn->arg1_type, &map);
+ if (err)
+ return err;
+ err = check_func_arg(env, BPF_REG_2, fn->arg2_type, &map);
+ if (err)
+ return err;
+ err = check_func_arg(env, BPF_REG_3, fn->arg3_type, &map);
+ if (err)
+ return err;
+ err = check_func_arg(env, BPF_REG_4, fn->arg4_type, &map);
+ if (err)
+ return err;
+ err = check_func_arg(env, BPF_REG_5, fn->arg5_type, &map);
+ if (err)
+ return err;
+
+ /* reset caller saved regs */
+ for (i = 0; i < CALLER_SAVED_REGS; i++) {
+ reg = regs + caller_saved[i];
+ reg->type = NOT_INIT;
+ reg->imm = 0;
+ }
+
+ /* update return register */
+ if (fn->ret_type == RET_INTEGER) {
+ regs[BPF_REG_0].type = UNKNOWN_VALUE;
+ } else if (fn->ret_type == RET_VOID) {
+ regs[BPF_REG_0].type = NOT_INIT;
+ } else if (fn->ret_type == RET_PTR_TO_MAP_VALUE_OR_NULL) {
+ regs[BPF_REG_0].type = PTR_TO_MAP_VALUE_OR_NULL;
+ /* remember map_ptr, so that check_map_access()
+ * can check 'value_size' boundary of memory access
+ * to map element returned from bpf_map_lookup_elem()
+ */
+ if (map == NULL) {
+ verbose("kernel subsystem misconfigured verifier\n");
+ return -EINVAL;
+ }
+ regs[BPF_REG_0].map_ptr = map;
+ } else {
+ verbose("unknown return type %d of func %d\n",
+ fn->ret_type, func_id);
+ return -EINVAL;
+ }
+ return 0;
+}
+
+/* check validity of 32-bit and 64-bit arithmetic operations */
+static int check_alu_op(struct reg_state *regs, struct bpf_insn *insn)
+{
+ u8 opcode = BPF_OP(insn->code);
+ int err;
+
+ if (opcode == BPF_END || opcode == BPF_NEG) {
+ if (opcode == BPF_NEG) {
+ if (BPF_SRC(insn->code) != 0 ||
+ insn->src_reg != BPF_REG_0 ||
+ insn->off != 0 || insn->imm != 0) {
+ verbose("BPF_NEG uses reserved fields\n");
+ return -EINVAL;
+ }
+ } else {
+ if (insn->src_reg != BPF_REG_0 || insn->off != 0 ||
+ (insn->imm != 16 && insn->imm != 32 && insn->imm != 64)) {
+ verbose("BPF_END uses reserved fields\n");
+ return -EINVAL;
+ }
+ }
+
+ /* check src operand */
+ err = check_reg_arg(regs, insn->dst_reg, SRC_OP);
+ if (err)
+ return err;
+
+ /* check dest operand */
+ err = check_reg_arg(regs, insn->dst_reg, DST_OP);
+ if (err)
+ return err;
+
+ } else if (opcode == BPF_MOV) {
+
+ if (BPF_SRC(insn->code) == BPF_X) {
+ if (insn->imm != 0 || insn->off != 0) {
+ verbose("BPF_MOV uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ /* check src operand */
+ err = check_reg_arg(regs, insn->src_reg, SRC_OP);
+ if (err)
+ return err;
+ } else {
+ if (insn->src_reg != BPF_REG_0 || insn->off != 0) {
+ verbose("BPF_MOV uses reserved fields\n");
+ return -EINVAL;
+ }
+ }
+
+ /* check dest operand */
+ err = check_reg_arg(regs, insn->dst_reg, DST_OP);
+ if (err)
+ return err;
+
+ if (BPF_SRC(insn->code) == BPF_X) {
+ if (BPF_CLASS(insn->code) == BPF_ALU64) {
+ /* case: R1 = R2
+ * copy register state to dest reg
+ */
+ regs[insn->dst_reg] = regs[insn->src_reg];
+ } else {
+ regs[insn->dst_reg].type = UNKNOWN_VALUE;
+ regs[insn->dst_reg].map_ptr = NULL;
+ }
+ } else {
+ /* case: R = imm
+ * remember the value we stored into this reg
+ */
+ regs[insn->dst_reg].type = CONST_IMM;
+ regs[insn->dst_reg].imm = insn->imm;
+ }
+
+ } else if (opcode > BPF_END) {
+ verbose("invalid BPF_ALU opcode %x\n", opcode);
+ return -EINVAL;
+
+ } else { /* all other ALU ops: and, sub, xor, add, ... */
+
+ bool stack_relative = false;
+
+ if (BPF_SRC(insn->code) == BPF_X) {
+ if (insn->imm != 0 || insn->off != 0) {
+ verbose("BPF_ALU uses reserved fields\n");
+ return -EINVAL;
+ }
+ /* check src1 operand */
+ err = check_reg_arg(regs, insn->src_reg, SRC_OP);
+ if (err)
+ return err;
+ } else {
+ if (insn->src_reg != BPF_REG_0 || insn->off != 0) {
+ verbose("BPF_ALU uses reserved fields\n");
+ return -EINVAL;
+ }
+ }
+
+ /* check src2 operand */
+ err = check_reg_arg(regs, insn->dst_reg, SRC_OP);
+ if (err)
+ return err;
+
+ if ((opcode == BPF_MOD || opcode == BPF_DIV) &&
+ BPF_SRC(insn->code) == BPF_K && insn->imm == 0) {
+ verbose("div by zero\n");
+ return -EINVAL;
+ }
+
+ /* pattern match 'bpf_add Rx, imm' instruction */
+ if (opcode == BPF_ADD && BPF_CLASS(insn->code) == BPF_ALU64 &&
+ regs[insn->dst_reg].type == FRAME_PTR &&
+ BPF_SRC(insn->code) == BPF_K)
+ stack_relative = true;
+
+ /* check dest operand */
+ err = check_reg_arg(regs, insn->dst_reg, DST_OP);
+ if (err)
+ return err;
+
+ if (stack_relative) {
+ regs[insn->dst_reg].type = PTR_TO_STACK;
+ regs[insn->dst_reg].imm = insn->imm;
+ }
+ }
+
+ return 0;
+}
+
+static int check_cond_jmp_op(struct verifier_env *env,
+ struct bpf_insn *insn, int *insn_idx)
+{
+ struct reg_state *regs = env->cur_state.regs;
+ struct verifier_state *other_branch;
+ u8 opcode = BPF_OP(insn->code);
+ int err;
+
+ if (opcode > BPF_EXIT) {
+ verbose("invalid BPF_JMP opcode %x\n", opcode);
+ return -EINVAL;
+ }
+
+ if (BPF_SRC(insn->code) == BPF_X) {
+ if (insn->imm != 0) {
+ verbose("BPF_JMP uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ /* check src1 operand */
+ err = check_reg_arg(regs, insn->src_reg, SRC_OP);
+ if (err)
+ return err;
+ } else {
+ if (insn->src_reg != BPF_REG_0) {
+ verbose("BPF_JMP uses reserved fields\n");
+ return -EINVAL;
+ }
+ }
+
+ /* check src2 operand */
+ err = check_reg_arg(regs, insn->dst_reg, SRC_OP);
+ if (err)
+ return err;
+
+ /* detect if R == 0 where R was initialized to zero earlier */
+ if (BPF_SRC(insn->code) == BPF_K &&
+ (opcode == BPF_JEQ || opcode == BPF_JNE) &&
+ regs[insn->dst_reg].type == CONST_IMM &&
+ regs[insn->dst_reg].imm == insn->imm) {
+ if (opcode == BPF_JEQ) {
+ /* if (imm == imm) goto pc+off;
+ * only follow the goto, ignore fall-through
+ */
+ *insn_idx += insn->off;
+ return 0;
+ } else {
+ /* if (imm != imm) goto pc+off;
+ * only follow fall-through branch, since
+ * that's where the program will go
+ */
+ return 0;
+ }
+ }
+
+ other_branch = push_stack(env, *insn_idx + insn->off + 1, *insn_idx);
+ if (!other_branch)
+ return -EFAULT;
+
+ /* detect if R == 0 where R is returned value from bpf_map_lookup_elem() */
+ if (BPF_SRC(insn->code) == BPF_K &&
+ insn->imm == 0 && (opcode == BPF_JEQ ||
+ opcode == BPF_JNE) &&
+ regs[insn->dst_reg].type == PTR_TO_MAP_VALUE_OR_NULL) {
+ if (opcode == BPF_JEQ) {
+ /* next fallthrough insn can access memory via
+ * this register
+ */
+ regs[insn->dst_reg].type = PTR_TO_MAP_VALUE;
+ /* branch targer cannot access it, since reg == 0 */
+ other_branch->regs[insn->dst_reg].type = CONST_IMM;
+ other_branch->regs[insn->dst_reg].imm = 0;
+ } else {
+ other_branch->regs[insn->dst_reg].type = PTR_TO_MAP_VALUE;
+ regs[insn->dst_reg].type = CONST_IMM;
+ regs[insn->dst_reg].imm = 0;
+ }
+ } else if (BPF_SRC(insn->code) == BPF_K &&
+ (opcode == BPF_JEQ || opcode == BPF_JNE)) {
+
+ if (opcode == BPF_JEQ) {
+ /* detect if (R == imm) goto
+ * and in the target state recognize that R = imm
+ */
+ other_branch->regs[insn->dst_reg].type = CONST_IMM;
+ other_branch->regs[insn->dst_reg].imm = insn->imm;
+ } else {
+ /* detect if (R != imm) goto
+ * and in the fall-through state recognize that R = imm
+ */
+ regs[insn->dst_reg].type = CONST_IMM;
+ regs[insn->dst_reg].imm = insn->imm;
+ }
+ }
+ if (log_level)
+ print_verifier_state(env);
+ return 0;
+}
+
/* return the map pointer stored inside BPF_LD_IMM64 instruction */
static struct bpf_map *ld_imm64_to_map_ptr(struct bpf_insn *insn)
{
@@ -313,6 +1148,37 @@ static struct bpf_map *ld_imm64_to_map_ptr(struct bpf_insn *insn)
return (struct bpf_map *) (unsigned long) imm64;
}
+/* verify BPF_LD_IMM64 instruction */
+static int check_ld_imm(struct verifier_env *env, struct bpf_insn *insn)
+{
+ struct reg_state *regs = env->cur_state.regs;
+ int err;
+
+ if (BPF_SIZE(insn->code) != BPF_DW) {
+ verbose("invalid BPF_LD_IMM insn\n");
+ return -EINVAL;
+ }
+ if (insn->off != 0) {
+ verbose("BPF_LD_IMM64 uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ err = check_reg_arg(regs, insn->dst_reg, DST_OP);
+ if (err)
+ return err;
+
+ if (insn->src_reg == 0)
+ /* generic move 64-bit immediate into a register */
+ return 0;
+
+ /* replace_map_fd_with_map_ptr() should have caught bad ld_imm64 */
+ BUG_ON(insn->src_reg != BPF_PSEUDO_MAP_FD);
+
+ regs[insn->dst_reg].type = CONST_PTR_TO_MAP;
+ regs[insn->dst_reg].map_ptr = ld_imm64_to_map_ptr(insn);
+ return 0;
+}
+
/* non-recursive DFS pseudo code
* 1 procedure DFS-iterative(G,v):
* 2 label v as discovered
@@ -498,6 +1364,212 @@ err_free:
return ret;
}
+static int do_check(struct verifier_env *env)
+{
+ struct verifier_state *state = &env->cur_state;
+ struct bpf_insn *insns = env->prog->insnsi;
+ struct reg_state *regs = state->regs;
+ int insn_cnt = env->prog->len;
+ int insn_idx, prev_insn_idx = 0;
+ int insn_processed = 0;
+ bool do_print_state = false;
+
+ init_reg_state(regs);
+ insn_idx = 0;
+ for (;;) {
+ struct bpf_insn *insn;
+ u8 class;
+ int err;
+
+ if (insn_idx >= insn_cnt) {
+ verbose("invalid insn idx %d insn_cnt %d\n",
+ insn_idx, insn_cnt);
+ return -EFAULT;
+ }
+
+ insn = &insns[insn_idx];
+ class = BPF_CLASS(insn->code);
+
+ if (++insn_processed > 32768) {
+ verbose("BPF program is too large. Proccessed %d insn\n",
+ insn_processed);
+ return -E2BIG;
+ }
+
+ if (log_level && do_print_state) {
+ verbose("\nfrom %d to %d:", prev_insn_idx, insn_idx);
+ print_verifier_state(env);
+ do_print_state = false;
+ }
+
+ if (log_level) {
+ verbose("%d: ", insn_idx);
+ print_bpf_insn(insn);
+ }
+
+ if (class == BPF_ALU || class == BPF_ALU64) {
+ err = check_alu_op(regs, insn);
+ if (err)
+ return err;
+
+ } else if (class == BPF_LDX) {
+ if (BPF_MODE(insn->code) != BPF_MEM ||
+ insn->imm != 0) {
+ verbose("BPF_LDX uses reserved fields\n");
+ return -EINVAL;
+ }
+ /* check src operand */
+ err = check_reg_arg(regs, insn->src_reg, SRC_OP);
+ if (err)
+ return err;
+
+ err = check_reg_arg(regs, insn->dst_reg, DST_OP_NO_MARK);
+ if (err)
+ return err;
+
+ /* check that memory (src_reg + off) is readable,
+ * the state of dst_reg will be updated by this func
+ */
+ err = check_mem_access(env, insn->src_reg, insn->off,
+ BPF_SIZE(insn->code), BPF_READ,
+ insn->dst_reg);
+ if (err)
+ return err;
+
+ } else if (class == BPF_STX) {
+ if (BPF_MODE(insn->code) == BPF_XADD) {
+ err = check_xadd(env, insn);
+ if (err)
+ return err;
+ insn_idx++;
+ continue;
+ }
+
+ if (BPF_MODE(insn->code) != BPF_MEM ||
+ insn->imm != 0) {
+ verbose("BPF_STX uses reserved fields\n");
+ return -EINVAL;
+ }
+ /* check src1 operand */
+ err = check_reg_arg(regs, insn->src_reg, SRC_OP);
+ if (err)
+ return err;
+ /* check src2 operand */
+ err = check_reg_arg(regs, insn->dst_reg, SRC_OP);
+ if (err)
+ return err;
+
+ /* check that memory (dst_reg + off) is writeable */
+ err = check_mem_access(env, insn->dst_reg, insn->off,
+ BPF_SIZE(insn->code), BPF_WRITE,
+ insn->src_reg);
+ if (err)
+ return err;
+
+ } else if (class == BPF_ST) {
+ if (BPF_MODE(insn->code) != BPF_MEM ||
+ insn->src_reg != BPF_REG_0) {
+ verbose("BPF_ST uses reserved fields\n");
+ return -EINVAL;
+ }
+ /* check src operand */
+ err = check_reg_arg(regs, insn->dst_reg, SRC_OP);
+ if (err)
+ return err;
+
+ /* check that memory (dst_reg + off) is writeable */
+ err = check_mem_access(env, insn->dst_reg, insn->off,
+ BPF_SIZE(insn->code), BPF_WRITE,
+ -1);
+ if (err)
+ return err;
+
+ } else if (class == BPF_JMP) {
+ u8 opcode = BPF_OP(insn->code);
+
+ if (opcode == BPF_CALL) {
+ if (BPF_SRC(insn->code) != BPF_K ||
+ insn->off != 0 ||
+ insn->src_reg != BPF_REG_0 ||
+ insn->dst_reg != BPF_REG_0) {
+ verbose("BPF_CALL uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ err = check_call(env, insn->imm);
+ if (err)
+ return err;
+
+ } else if (opcode == BPF_JA) {
+ if (BPF_SRC(insn->code) != BPF_K ||
+ insn->imm != 0 ||
+ insn->src_reg != BPF_REG_0 ||
+ insn->dst_reg != BPF_REG_0) {
+ verbose("BPF_JA uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ insn_idx += insn->off + 1;
+ continue;
+
+ } else if (opcode == BPF_EXIT) {
+ if (BPF_SRC(insn->code) != BPF_K ||
+ insn->imm != 0 ||
+ insn->src_reg != BPF_REG_0 ||
+ insn->dst_reg != BPF_REG_0) {
+ verbose("BPF_EXIT uses reserved fields\n");
+ return -EINVAL;
+ }
+
+ /* eBPF calling convetion is such that R0 is used
+ * to return the value from eBPF program.
+ * Make sure that it's readable at this time
+ * of bpf_exit, which means that program wrote
+ * something into it earlier
+ */
+ err = check_reg_arg(regs, BPF_REG_0, SRC_OP);
+ if (err)
+ return err;
+
+ insn_idx = pop_stack(env, &prev_insn_idx);
+ if (insn_idx < 0) {
+ break;
+ } else {
+ do_print_state = true;
+ continue;
+ }
+ } else {
+ err = check_cond_jmp_op(env, insn, &insn_idx);
+ if (err)
+ return err;
+ }
+ } else if (class == BPF_LD) {
+ u8 mode = BPF_MODE(insn->code);
+
+ if (mode == BPF_ABS || mode == BPF_IND) {
+ verbose("LD_ABS is not supported yet\n");
+ return -EINVAL;
+ } else if (mode == BPF_IMM) {
+ err = check_ld_imm(env, insn);
+ if (err)
+ return err;
+
+ insn_idx++;
+ } else {
+ verbose("invalid BPF_LD mode\n");
+ return -EINVAL;
+ }
+ } else {
+ verbose("unknown insn class %d\n", class);
+ return -EINVAL;
+ }
+
+ insn_idx++;
+ }
+
+ return 0;
+}
+
/* look for pseudo eBPF instructions that access map FDs and
* replace them with actual map pointers
*/
@@ -651,9 +1723,10 @@ int bpf_check(struct bpf_prog *prog, union bpf_attr *attr)
if (ret < 0)
goto skip_full_check;
- /* ret = do_check(env); */
+ ret = do_check(env);
skip_full_check:
+ while (pop_stack(env, NULL) >= 0);
if (log_level && log_len >= log_size - 1) {
BUG_ON(log_len >= log_size);
--
1.7.9.5
^ permalink raw reply related
* [PATCH v15 net-next 11/11] bpf: mini eBPF library, test stubs and verifier testsuite
From: Alexei Starovoitov @ 2014-09-26 7:17 UTC (permalink / raw)
To: David S. Miller
Cc: Ingo Molnar, Linus Torvalds, Andy Lutomirski, Daniel Borkmann,
Hannes Frederic Sowa, Chema Gonzalez, Eric Dumazet,
Peter Zijlstra, Pablo Neira Ayuso, H. Peter Anvin, Andrew Morton,
Kees Cook, linux-api, netdev, linux-kernel
In-Reply-To: <1411715827-10430-1-git-send-email-ast@plumgrid.com>
1.
the library includes a trivial set of BPF syscall wrappers:
int bpf_create_map(int key_size, int value_size, int max_entries);
int bpf_update_elem(int fd, void *key, void *value);
int bpf_lookup_elem(int fd, void *key, void *value);
int bpf_delete_elem(int fd, void *key);
int bpf_get_next_key(int fd, void *key, void *next_key);
int bpf_prog_load(enum bpf_prog_type prog_type,
const struct sock_filter_int *insns, int insn_len,
const char *license);
bpf_prog_load() stores verifier log into global bpf_log_buf[] array
and BPF_*() macros to build instructions
2.
test stubs configure eBPF infra with 'unspec' map and program types.
These are fake types used by user space testsuite only.
3.
verifier tests valid and invalid programs and expects predefined
error log messages from kernel.
40 tests so far.
$ sudo ./test_verifier
#0 add+sub+mul OK
#1 unreachable OK
#2 unreachable2 OK
#3 out of range jump OK
#4 out of range jump2 OK
#5 test1 ld_imm64 OK
...
Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
---
kernel/bpf/Makefile | 4 +
kernel/bpf/test_stub.c | 116 +++++++++
lib/Kconfig.debug | 3 +-
samples/bpf/Makefile | 12 +
samples/bpf/libbpf.c | 94 ++++++++
samples/bpf/libbpf.h | 172 ++++++++++++++
samples/bpf/test_verifier.c | 548 +++++++++++++++++++++++++++++++++++++++++++
7 files changed, 948 insertions(+), 1 deletion(-)
create mode 100644 kernel/bpf/test_stub.c
create mode 100644 samples/bpf/Makefile
create mode 100644 samples/bpf/libbpf.c
create mode 100644 samples/bpf/libbpf.h
create mode 100644 samples/bpf/test_verifier.c
diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
index 3c726b0995b7..45427239f375 100644
--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -1 +1,5 @@
obj-y := core.o syscall.o verifier.o
+
+ifdef CONFIG_TEST_BPF
+obj-y += test_stub.o
+endif
diff --git a/kernel/bpf/test_stub.c b/kernel/bpf/test_stub.c
new file mode 100644
index 000000000000..fcaddff4003e
--- /dev/null
+++ b/kernel/bpf/test_stub.c
@@ -0,0 +1,116 @@
+/* Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General Public
+ * License as published by the Free Software Foundation.
+ */
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+#include <linux/bpf.h>
+
+/* test stubs for BPF_MAP_TYPE_UNSPEC and for BPF_PROG_TYPE_UNSPEC
+ * to be used by user space verifier testsuite
+ */
+struct bpf_context {
+ u64 arg1;
+ u64 arg2;
+};
+
+static u64 test_func(u64 r1, u64 r2, u64 r3, u64 r4, u64 r5)
+{
+ return 0;
+}
+
+static struct bpf_func_proto test_funcs[] = {
+ [BPF_FUNC_unspec] = {
+ .func = test_func,
+ .gpl_only = true,
+ .ret_type = RET_PTR_TO_MAP_VALUE_OR_NULL,
+ .arg1_type = ARG_CONST_MAP_PTR,
+ .arg2_type = ARG_PTR_TO_MAP_KEY,
+ },
+};
+
+static const struct bpf_func_proto *test_func_proto(enum bpf_func_id func_id)
+{
+ if (func_id < 0 || func_id >= ARRAY_SIZE(test_funcs))
+ return NULL;
+ return &test_funcs[func_id];
+}
+
+static const struct bpf_context_access {
+ int size;
+ enum bpf_access_type type;
+} test_ctx_access[] = {
+ [offsetof(struct bpf_context, arg1)] = {
+ FIELD_SIZEOF(struct bpf_context, arg1),
+ BPF_READ
+ },
+ [offsetof(struct bpf_context, arg2)] = {
+ FIELD_SIZEOF(struct bpf_context, arg2),
+ BPF_READ
+ },
+};
+
+static bool test_is_valid_access(int off, int size, enum bpf_access_type type)
+{
+ const struct bpf_context_access *access;
+
+ if (off < 0 || off >= ARRAY_SIZE(test_ctx_access))
+ return false;
+
+ access = &test_ctx_access[off];
+ if (access->size == size && (access->type & type))
+ return true;
+
+ return false;
+}
+
+static struct bpf_verifier_ops test_ops = {
+ .get_func_proto = test_func_proto,
+ .is_valid_access = test_is_valid_access,
+};
+
+static struct bpf_prog_type_list tl_prog = {
+ .ops = &test_ops,
+ .type = BPF_PROG_TYPE_UNSPEC,
+};
+
+static struct bpf_map *test_map_alloc(union bpf_attr *attr)
+{
+ struct bpf_map *map;
+
+ map = kzalloc(sizeof(*map), GFP_USER);
+ if (!map)
+ return ERR_PTR(-ENOMEM);
+
+ map->key_size = attr->key_size;
+ map->value_size = attr->value_size;
+ map->max_entries = attr->max_entries;
+ return map;
+}
+
+static void test_map_free(struct bpf_map *map)
+{
+ kfree(map);
+}
+
+static struct bpf_map_ops test_map_ops = {
+ .map_alloc = test_map_alloc,
+ .map_free = test_map_free,
+};
+
+static struct bpf_map_type_list tl_map = {
+ .ops = &test_map_ops,
+ .type = BPF_MAP_TYPE_UNSPEC,
+};
+
+static int __init register_test_ops(void)
+{
+ bpf_register_map_type(&tl_map);
+ bpf_register_prog_type(&tl_prog);
+ return 0;
+}
+late_initcall(register_test_ops);
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index a28590083622..3ac43f34437b 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1672,7 +1672,8 @@ config TEST_BPF
against the BPF interpreter or BPF JIT compiler depending on the
current setting. This is in particular useful for BPF JIT compiler
development, but also to run regression tests against changes in
- the interpreter code.
+ the interpreter code. It also enables test stubs for eBPF maps and
+ verifier used by user space verifier testsuite.
If unsure, say N.
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
new file mode 100644
index 000000000000..634391797856
--- /dev/null
+++ b/samples/bpf/Makefile
@@ -0,0 +1,12 @@
+# kbuild trick to avoid linker error. Can be omitted if a module is built.
+obj- := dummy.o
+
+# List of programs to build
+hostprogs-y := test_verifier
+
+test_verifier-objs := test_verifier.o libbpf.o
+
+# Tell kbuild to always build the programs
+always := $(hostprogs-y)
+
+HOSTCFLAGS += -I$(objtree)/usr/include
diff --git a/samples/bpf/libbpf.c b/samples/bpf/libbpf.c
new file mode 100644
index 000000000000..ff6504420738
--- /dev/null
+++ b/samples/bpf/libbpf.c
@@ -0,0 +1,94 @@
+/* eBPF mini library */
+#include <stdlib.h>
+#include <stdio.h>
+#include <linux/unistd.h>
+#include <unistd.h>
+#include <string.h>
+#include <linux/netlink.h>
+#include <linux/bpf.h>
+#include <errno.h>
+#include "libbpf.h"
+
+static __u64 ptr_to_u64(void *ptr)
+{
+ return (__u64) (unsigned long) ptr;
+}
+
+int bpf_create_map(enum bpf_map_type map_type, int key_size, int value_size,
+ int max_entries)
+{
+ union bpf_attr attr = {
+ .map_type = map_type,
+ .key_size = key_size,
+ .value_size = value_size,
+ .max_entries = max_entries
+ };
+
+ return syscall(__NR_bpf, BPF_MAP_CREATE, &attr, sizeof(attr));
+}
+
+int bpf_update_elem(int fd, void *key, void *value)
+{
+ union bpf_attr attr = {
+ .map_fd = fd,
+ .key = ptr_to_u64(key),
+ .value = ptr_to_u64(value),
+ };
+
+ return syscall(__NR_bpf, BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
+}
+
+int bpf_lookup_elem(int fd, void *key, void *value)
+{
+ union bpf_attr attr = {
+ .map_fd = fd,
+ .key = ptr_to_u64(key),
+ .value = ptr_to_u64(value),
+ };
+
+ return syscall(__NR_bpf, BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
+}
+
+int bpf_delete_elem(int fd, void *key)
+{
+ union bpf_attr attr = {
+ .map_fd = fd,
+ .key = ptr_to_u64(key),
+ };
+
+ return syscall(__NR_bpf, BPF_MAP_DELETE_ELEM, &attr, sizeof(attr));
+}
+
+int bpf_get_next_key(int fd, void *key, void *next_key)
+{
+ union bpf_attr attr = {
+ .map_fd = fd,
+ .key = ptr_to_u64(key),
+ .next_key = ptr_to_u64(next_key),
+ };
+
+ return syscall(__NR_bpf, BPF_MAP_GET_NEXT_KEY, &attr, sizeof(attr));
+}
+
+#define ROUND_UP(x, n) (((x) + (n) - 1u) & ~((n) - 1u))
+
+char bpf_log_buf[LOG_BUF_SIZE];
+
+int bpf_prog_load(enum bpf_prog_type prog_type,
+ const struct bpf_insn *insns, int prog_len,
+ const char *license)
+{
+ union bpf_attr attr = {
+ .prog_type = prog_type,
+ .insns = ptr_to_u64((void *) insns),
+ .insn_cnt = prog_len / sizeof(struct bpf_insn),
+ .license = ptr_to_u64((void *) license),
+ .log_buf = ptr_to_u64(bpf_log_buf),
+ .log_size = LOG_BUF_SIZE,
+ .log_level = 1,
+ };
+
+ bpf_log_buf[0] = 0;
+
+ return syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
+}
diff --git a/samples/bpf/libbpf.h b/samples/bpf/libbpf.h
new file mode 100644
index 000000000000..8a31babeca5d
--- /dev/null
+++ b/samples/bpf/libbpf.h
@@ -0,0 +1,172 @@
+/* eBPF mini library */
+#ifndef __LIBBPF_H
+#define __LIBBPF_H
+
+struct bpf_insn;
+
+int bpf_create_map(enum bpf_map_type map_type, int key_size, int value_size,
+ int max_entries);
+int bpf_update_elem(int fd, void *key, void *value);
+int bpf_lookup_elem(int fd, void *key, void *value);
+int bpf_delete_elem(int fd, void *key);
+int bpf_get_next_key(int fd, void *key, void *next_key);
+
+int bpf_prog_load(enum bpf_prog_type prog_type,
+ const struct bpf_insn *insns, int insn_len,
+ const char *license);
+
+#define LOG_BUF_SIZE 8192
+extern char bpf_log_buf[LOG_BUF_SIZE];
+
+/* ALU ops on registers, bpf_add|sub|...: dst_reg += src_reg */
+
+#define BPF_ALU64_REG(OP, DST, SRC) \
+ ((struct bpf_insn) { \
+ .code = BPF_ALU64 | BPF_OP(OP) | BPF_X, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = 0, \
+ .imm = 0 })
+
+#define BPF_ALU32_REG(OP, DST, SRC) \
+ ((struct bpf_insn) { \
+ .code = BPF_ALU | BPF_OP(OP) | BPF_X, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = 0, \
+ .imm = 0 })
+
+/* ALU ops on immediates, bpf_add|sub|...: dst_reg += imm32 */
+
+#define BPF_ALU64_IMM(OP, DST, IMM) \
+ ((struct bpf_insn) { \
+ .code = BPF_ALU64 | BPF_OP(OP) | BPF_K, \
+ .dst_reg = DST, \
+ .src_reg = 0, \
+ .off = 0, \
+ .imm = IMM })
+
+#define BPF_ALU32_IMM(OP, DST, IMM) \
+ ((struct bpf_insn) { \
+ .code = BPF_ALU | BPF_OP(OP) | BPF_K, \
+ .dst_reg = DST, \
+ .src_reg = 0, \
+ .off = 0, \
+ .imm = IMM })
+
+/* Short form of mov, dst_reg = src_reg */
+
+#define BPF_MOV64_REG(DST, SRC) \
+ ((struct bpf_insn) { \
+ .code = BPF_ALU64 | BPF_MOV | BPF_X, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = 0, \
+ .imm = 0 })
+
+/* Short form of mov, dst_reg = imm32 */
+
+#define BPF_MOV64_IMM(DST, IMM) \
+ ((struct bpf_insn) { \
+ .code = BPF_ALU64 | BPF_MOV | BPF_K, \
+ .dst_reg = DST, \
+ .src_reg = 0, \
+ .off = 0, \
+ .imm = IMM })
+
+/* BPF_LD_IMM64 macro encodes single 'load 64-bit immediate' insn */
+#define BPF_LD_IMM64(DST, IMM) \
+ BPF_LD_IMM64_RAW(DST, 0, IMM)
+
+#define BPF_LD_IMM64_RAW(DST, SRC, IMM) \
+ ((struct bpf_insn) { \
+ .code = BPF_LD | BPF_DW | BPF_IMM, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = 0, \
+ .imm = (__u32) (IMM) }), \
+ ((struct bpf_insn) { \
+ .code = 0, /* zero is reserved opcode */ \
+ .dst_reg = 0, \
+ .src_reg = 0, \
+ .off = 0, \
+ .imm = ((__u64) (IMM)) >> 32 })
+
+#define BPF_PSEUDO_MAP_FD 1
+
+/* pseudo BPF_LD_IMM64 insn used to refer to process-local map_fd */
+#define BPF_LD_MAP_FD(DST, MAP_FD) \
+ BPF_LD_IMM64_RAW(DST, BPF_PSEUDO_MAP_FD, MAP_FD)
+
+
+/* Memory load, dst_reg = *(uint *) (src_reg + off16) */
+
+#define BPF_LDX_MEM(SIZE, DST, SRC, OFF) \
+ ((struct bpf_insn) { \
+ .code = BPF_LDX | BPF_SIZE(SIZE) | BPF_MEM, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = OFF, \
+ .imm = 0 })
+
+/* Memory store, *(uint *) (dst_reg + off16) = src_reg */
+
+#define BPF_STX_MEM(SIZE, DST, SRC, OFF) \
+ ((struct bpf_insn) { \
+ .code = BPF_STX | BPF_SIZE(SIZE) | BPF_MEM, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = OFF, \
+ .imm = 0 })
+
+/* Memory store, *(uint *) (dst_reg + off16) = imm32 */
+
+#define BPF_ST_MEM(SIZE, DST, OFF, IMM) \
+ ((struct bpf_insn) { \
+ .code = BPF_ST | BPF_SIZE(SIZE) | BPF_MEM, \
+ .dst_reg = DST, \
+ .src_reg = 0, \
+ .off = OFF, \
+ .imm = IMM })
+
+/* Conditional jumps against registers, if (dst_reg 'op' src_reg) goto pc + off16 */
+
+#define BPF_JMP_REG(OP, DST, SRC, OFF) \
+ ((struct bpf_insn) { \
+ .code = BPF_JMP | BPF_OP(OP) | BPF_X, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = OFF, \
+ .imm = 0 })
+
+/* Conditional jumps against immediates, if (dst_reg 'op' imm32) goto pc + off16 */
+
+#define BPF_JMP_IMM(OP, DST, IMM, OFF) \
+ ((struct bpf_insn) { \
+ .code = BPF_JMP | BPF_OP(OP) | BPF_K, \
+ .dst_reg = DST, \
+ .src_reg = 0, \
+ .off = OFF, \
+ .imm = IMM })
+
+/* Raw code statement block */
+
+#define BPF_RAW_INSN(CODE, DST, SRC, OFF, IMM) \
+ ((struct bpf_insn) { \
+ .code = CODE, \
+ .dst_reg = DST, \
+ .src_reg = SRC, \
+ .off = OFF, \
+ .imm = IMM })
+
+/* Program exit */
+
+#define BPF_EXIT_INSN() \
+ ((struct bpf_insn) { \
+ .code = BPF_JMP | BPF_EXIT, \
+ .dst_reg = 0, \
+ .src_reg = 0, \
+ .off = 0, \
+ .imm = 0 })
+
+#endif
diff --git a/samples/bpf/test_verifier.c b/samples/bpf/test_verifier.c
new file mode 100644
index 000000000000..d10992e2740e
--- /dev/null
+++ b/samples/bpf/test_verifier.c
@@ -0,0 +1,548 @@
+/*
+ * Testsuite for eBPF verifier
+ *
+ * Copyright (c) 2014 PLUMgrid, http://plumgrid.com
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of version 2 of the GNU General Public
+ * License as published by the Free Software Foundation.
+ */
+#include <stdio.h>
+#include <unistd.h>
+#include <linux/bpf.h>
+#include <errno.h>
+#include <linux/unistd.h>
+#include <string.h>
+#include <linux/filter.h>
+#include "libbpf.h"
+
+#define MAX_INSNS 512
+#define ARRAY_SIZE(x) (sizeof(x) / sizeof(*(x)))
+
+struct bpf_test {
+ const char *descr;
+ struct bpf_insn insns[MAX_INSNS];
+ int fixup[32];
+ const char *errstr;
+ enum {
+ ACCEPT,
+ REJECT
+ } result;
+};
+
+static struct bpf_test tests[] = {
+ {
+ "add+sub+mul",
+ .insns = {
+ BPF_MOV64_IMM(BPF_REG_1, 1),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, 2),
+ BPF_MOV64_IMM(BPF_REG_2, 3),
+ BPF_ALU64_REG(BPF_SUB, BPF_REG_1, BPF_REG_2),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_1, -1),
+ BPF_ALU64_IMM(BPF_MUL, BPF_REG_1, 3),
+ BPF_MOV64_REG(BPF_REG_0, BPF_REG_1),
+ BPF_EXIT_INSN(),
+ },
+ .result = ACCEPT,
+ },
+ {
+ "unreachable",
+ .insns = {
+ BPF_EXIT_INSN(),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "unreachable",
+ .result = REJECT,
+ },
+ {
+ "unreachable2",
+ .insns = {
+ BPF_JMP_IMM(BPF_JA, 0, 0, 1),
+ BPF_JMP_IMM(BPF_JA, 0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "unreachable",
+ .result = REJECT,
+ },
+ {
+ "out of range jump",
+ .insns = {
+ BPF_JMP_IMM(BPF_JA, 0, 0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "jump out of range",
+ .result = REJECT,
+ },
+ {
+ "out of range jump2",
+ .insns = {
+ BPF_JMP_IMM(BPF_JA, 0, 0, -2),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "jump out of range",
+ .result = REJECT,
+ },
+ {
+ "test1 ld_imm64",
+ .insns = {
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 1),
+ BPF_LD_IMM64(BPF_REG_0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 1),
+ BPF_LD_IMM64(BPF_REG_0, 1),
+ BPF_MOV64_IMM(BPF_REG_0, 2),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid BPF_LD_IMM insn",
+ .result = REJECT,
+ },
+ {
+ "test2 ld_imm64",
+ .insns = {
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 1),
+ BPF_LD_IMM64(BPF_REG_0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 1),
+ BPF_LD_IMM64(BPF_REG_0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid BPF_LD_IMM insn",
+ .result = REJECT,
+ },
+ {
+ "test3 ld_imm64",
+ .insns = {
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, 1),
+ BPF_RAW_INSN(BPF_LD | BPF_IMM | BPF_DW, 0, 0, 0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 0),
+ BPF_LD_IMM64(BPF_REG_0, 1),
+ BPF_LD_IMM64(BPF_REG_0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid bpf_ld_imm64 insn",
+ .result = REJECT,
+ },
+ {
+ "test4 ld_imm64",
+ .insns = {
+ BPF_RAW_INSN(BPF_LD | BPF_IMM | BPF_DW, 0, 0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid bpf_ld_imm64 insn",
+ .result = REJECT,
+ },
+ {
+ "test5 ld_imm64",
+ .insns = {
+ BPF_RAW_INSN(BPF_LD | BPF_IMM | BPF_DW, 0, 0, 0, 0),
+ },
+ .errstr = "invalid bpf_ld_imm64 insn",
+ .result = REJECT,
+ },
+ {
+ "no bpf_exit",
+ .insns = {
+ BPF_ALU64_REG(BPF_MOV, BPF_REG_0, BPF_REG_2),
+ },
+ .errstr = "jump out of range",
+ .result = REJECT,
+ },
+ {
+ "loop (back-edge)",
+ .insns = {
+ BPF_JMP_IMM(BPF_JA, 0, 0, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "back-edge",
+ .result = REJECT,
+ },
+ {
+ "loop2 (back-edge)",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_3, BPF_REG_0),
+ BPF_JMP_IMM(BPF_JA, 0, 0, -4),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "back-edge",
+ .result = REJECT,
+ },
+ {
+ "conditional loop",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_1, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_0),
+ BPF_MOV64_REG(BPF_REG_3, BPF_REG_0),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_1, 0, -3),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "back-edge",
+ .result = REJECT,
+ },
+ {
+ "read uninitialized register",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_0, BPF_REG_2),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R2 !read_ok",
+ .result = REJECT,
+ },
+ {
+ "read invalid register",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_0, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R15 is invalid",
+ .result = REJECT,
+ },
+ {
+ "program doesn't init R0 before exit",
+ .insns = {
+ BPF_ALU64_REG(BPF_MOV, BPF_REG_2, BPF_REG_1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R0 !read_ok",
+ .result = REJECT,
+ },
+ {
+ "stack out of bounds",
+ .insns = {
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, 8, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid stack",
+ .result = REJECT,
+ },
+ {
+ "invalid call insn1",
+ .insns = {
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL | BPF_X, 0, 0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "BPF_CALL uses reserved",
+ .result = REJECT,
+ },
+ {
+ "invalid call insn2",
+ .insns = {
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 1, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "BPF_CALL uses reserved",
+ .result = REJECT,
+ },
+ {
+ "invalid function call",
+ .insns = {
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, 1234567),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid func 1234567",
+ .result = REJECT,
+ },
+ {
+ "uninitialized stack1",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_unspec),
+ BPF_EXIT_INSN(),
+ },
+ .fixup = {2},
+ .errstr = "invalid indirect read from stack",
+ .result = REJECT,
+ },
+ {
+ "uninitialized stack2",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, -8),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid read from stack",
+ .result = REJECT,
+ },
+ {
+ "check valid spill/fill",
+ .insns = {
+ /* spill R1(ctx) into stack */
+ BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_1, -8),
+
+ /* fill it back into R2 */
+ BPF_LDX_MEM(BPF_DW, BPF_REG_2, BPF_REG_10, -8),
+
+ /* should be able to access R0 = *(R2 + 8) */
+ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, 8),
+ BPF_EXIT_INSN(),
+ },
+ .result = ACCEPT,
+ },
+ {
+ "check corrupted spill/fill",
+ .insns = {
+ /* spill R1(ctx) into stack */
+ BPF_STX_MEM(BPF_DW, BPF_REG_10, BPF_REG_1, -8),
+
+ /* mess up with R1 pointer on stack */
+ BPF_ST_MEM(BPF_B, BPF_REG_10, -7, 0x23),
+
+ /* fill back into R0 should fail */
+ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_10, -8),
+
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "corrupted spill",
+ .result = REJECT,
+ },
+ {
+ "invalid src register in STX",
+ .insns = {
+ BPF_STX_MEM(BPF_B, BPF_REG_10, -1, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R15 is invalid",
+ .result = REJECT,
+ },
+ {
+ "invalid dst register in STX",
+ .insns = {
+ BPF_STX_MEM(BPF_B, 14, BPF_REG_10, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R14 is invalid",
+ .result = REJECT,
+ },
+ {
+ "invalid dst register in ST",
+ .insns = {
+ BPF_ST_MEM(BPF_B, 14, -1, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R14 is invalid",
+ .result = REJECT,
+ },
+ {
+ "invalid src register in LDX",
+ .insns = {
+ BPF_LDX_MEM(BPF_B, BPF_REG_0, 12, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R12 is invalid",
+ .result = REJECT,
+ },
+ {
+ "invalid dst register in LDX",
+ .insns = {
+ BPF_LDX_MEM(BPF_B, 11, BPF_REG_1, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "R11 is invalid",
+ .result = REJECT,
+ },
+ {
+ "junk insn",
+ .insns = {
+ BPF_RAW_INSN(0, 0, 0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid BPF_LD_IMM",
+ .result = REJECT,
+ },
+ {
+ "junk insn2",
+ .insns = {
+ BPF_RAW_INSN(1, 0, 0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "BPF_LDX uses reserved fields",
+ .result = REJECT,
+ },
+ {
+ "junk insn3",
+ .insns = {
+ BPF_RAW_INSN(-1, 0, 0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid BPF_ALU opcode f0",
+ .result = REJECT,
+ },
+ {
+ "junk insn4",
+ .insns = {
+ BPF_RAW_INSN(-1, -1, -1, -1, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "invalid BPF_ALU opcode f0",
+ .result = REJECT,
+ },
+ {
+ "junk insn5",
+ .insns = {
+ BPF_RAW_INSN(0x7f, -1, -1, -1, -1),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "BPF_ALU uses reserved fields",
+ .result = REJECT,
+ },
+ {
+ "misaligned read from stack",
+ .insns = {
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_2, -4),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "misaligned access",
+ .result = REJECT,
+ },
+ {
+ "invalid map_fd for function call",
+ .insns = {
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_ALU64_REG(BPF_MOV, BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_unspec),
+ BPF_EXIT_INSN(),
+ },
+ .errstr = "fd 0 is not pointing to valid bpf_map",
+ .result = REJECT,
+ },
+ {
+ "don't check return value before access",
+ .insns = {
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_unspec),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup = {3},
+ .errstr = "R0 invalid mem access 'map_value_or_null'",
+ .result = REJECT,
+ },
+ {
+ "access memory with incorrect alignment",
+ .insns = {
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_unspec),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 1),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 4, 0),
+ BPF_EXIT_INSN(),
+ },
+ .fixup = {3},
+ .errstr = "misaligned access",
+ .result = REJECT,
+ },
+ {
+ "sometimes access memory with incorrect alignment",
+ .insns = {
+ BPF_ST_MEM(BPF_DW, BPF_REG_10, -8, 0),
+ BPF_MOV64_REG(BPF_REG_2, BPF_REG_10),
+ BPF_ALU64_IMM(BPF_ADD, BPF_REG_2, -8),
+ BPF_LD_MAP_FD(BPF_REG_1, 0),
+ BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 0, 0, BPF_FUNC_unspec),
+ BPF_JMP_IMM(BPF_JEQ, BPF_REG_0, 0, 2),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 0),
+ BPF_EXIT_INSN(),
+ BPF_ST_MEM(BPF_DW, BPF_REG_0, 0, 1),
+ BPF_EXIT_INSN(),
+ },
+ .fixup = {3},
+ .errstr = "R0 invalid mem access",
+ .result = REJECT,
+ },
+};
+
+static int probe_filter_length(struct bpf_insn *fp)
+{
+ int len = 0;
+
+ for (len = MAX_INSNS - 1; len > 0; --len)
+ if (fp[len].code != 0 || fp[len].imm != 0)
+ break;
+
+ return len + 1;
+}
+
+static int create_map(void)
+{
+ long long key, value = 0;
+ int map_fd;
+
+ map_fd = bpf_create_map(BPF_MAP_TYPE_UNSPEC, sizeof(key), sizeof(value), 1024);
+ if (map_fd < 0) {
+ printf("failed to create map '%s'\n", strerror(errno));
+ }
+
+ return map_fd;
+}
+
+static int test(void)
+{
+ int prog_fd, i;
+
+ for (i = 0; i < ARRAY_SIZE(tests); i++) {
+ struct bpf_insn *prog = tests[i].insns;
+ int prog_len = probe_filter_length(prog);
+ int *fixup = tests[i].fixup;
+ int map_fd = -1;
+
+ if (*fixup) {
+ map_fd = create_map();
+
+ do {
+ prog[*fixup].imm = map_fd;
+ fixup++;
+ } while (*fixup);
+ }
+ printf("#%d %s ", i, tests[i].descr);
+
+ prog_fd = bpf_prog_load(BPF_PROG_TYPE_UNSPEC, prog,
+ prog_len * sizeof(struct bpf_insn),
+ "GPL");
+
+ if (tests[i].result == ACCEPT) {
+ if (prog_fd < 0) {
+ printf("FAIL\nfailed to load prog '%s'\n",
+ strerror(errno));
+ printf("%s", bpf_log_buf);
+ goto fail;
+ }
+ } else {
+ if (prog_fd >= 0) {
+ printf("FAIL\nunexpected success to load\n");
+ printf("%s", bpf_log_buf);
+ goto fail;
+ }
+ if (strstr(bpf_log_buf, tests[i].errstr) == 0) {
+ printf("FAIL\nunexpected error message: %s",
+ bpf_log_buf);
+ goto fail;
+ }
+ }
+
+ printf("OK\n");
+fail:
+ if (map_fd >= 0)
+ close(map_fd);
+ close(prog_fd);
+
+ }
+
+ return 0;
+}
+
+int main(void)
+{
+ return test();
+}
--
1.7.9.5
^ permalink raw reply related
* Re: [RFC PATCH net-next] net: pktgen: packet bursting via skb->xmit_more
From: Eric Dumazet @ 2014-09-26 7:42 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: David S. Miller, Jesper Dangaard Brouer, Eric Dumazet,
John Fastabend, netdev
In-Reply-To: <1411694414.16953.70.camel@edumazet-glaptop2.roam.corp.google.com>
On Thu, 2014-09-25 at 18:20 -0700, Eric Dumazet wrote:
> On Thu, 2014-09-25 at 17:46 -0700, Alexei Starovoitov wrote:
> > This patch demonstrates the effect of delaying update of HW tailptr.
> > (based on earlier patch by Jesper)
> >
> > burst=1 is a default. It sends one packet with xmit_more=false
> > burst=2 sends one packet with xmit_more=true and
> > 2nd copy of the same packet with xmit_more=false
> > burst=3 sends two copies of the same packet with xmit_more=true and
> > 3rd copy with xmit_more=false
>
> >
> > Cc: Jesper Dangaard Brouer <brouer@redhat.com>
> > Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
> > ---
>
> Perfect, this is what I had in mind, thanks !
>
> Acked-by: Eric Dumazet <edumazet@google.com>
>
By the way with this patch, I now reach 10 Mpps on mlx4
base line : 5 Mpps
+ skb->xmit_more and quick hack in pjtgen (spinlock/unlock per packet)
-> 7 Mpps
+ burst of 16 packets, no spinlock per packet -> 10 Mpps
^ permalink raw reply
* Re: [PATCH v2 net-next 0/9] ipv6: fib6: socket dst_entry improvments and cleanups
From: Hannes Frederic Sowa @ 2014-09-26 7:58 UTC (permalink / raw)
To: David Miller
Cc: netdev, eric.dumazet, hideaki, vyasevich, nicolas.dichtel, kafai
In-Reply-To: <20140926.002853.1097612772155886372.davem@davemloft.net>
On Fri, Sep 26, 2014, at 06:28, David Miller wrote:
> From: Hannes Frederic Sowa <hannes@stressinduktion.org>
> Date: Sun, 21 Sep 2014 16:11:44 +0200
>
> > Eric Dumazet noticed that rt6_nodes wich are neither RTF_NONEXTHOP nor
> > RTF_GATEWAY but DST_HOST ones cause major routing lookup churn because
> > their rt6_genid is never renewed, thus ip6_dst_check always considers
> > them outdated. This is a major problem, because these kind of routes
> > are normally used to in input handling.
>
> This series is a disappointment for me from the perspective of the
> fact that we have a regression in mainline and this is too complex
> of a set of changes for there.
>
> If we relookup the thing every TCP input packet, we might as well
> not do the input route caching in the socket.
I can understand.
Toss this series, I'll try to do better tomorrow and send changes for
net and submit net-next cleanups when your queue is a bit smaller.
Bye,
Hannes
^ permalink raw reply
* Re: [RFC PATCH net-next] net: pktgen: packet bursting via skb->xmit_more
From: Jesper Dangaard Brouer @ 2014-09-26 8:05 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: David S. Miller, Eric Dumazet, John Fastabend, netdev, brouer
In-Reply-To: <1411692382-8898-1-git-send-email-ast@plumgrid.com>
On Thu, 25 Sep 2014 17:46:22 -0700
Alexei Starovoitov <ast@plumgrid.com> wrote:
> This patch demonstrates the effect of delaying update of HW tailptr.
> (based on earlier patch by Jesper)
>
> burst=1 is a default. It sends one packet with xmit_more=false
> burst=2 sends one packet with xmit_more=true and
> 2nd copy of the same packet with xmit_more=false
> burst=3 sends two copies of the same packet with xmit_more=true and
> 3rd copy with xmit_more=false
>
> Performance with ixgbe:
>
> usec 30:
> burst=1 tx:9.2 Mpps
> burst=2 tx:13.6 Mpps
> burst=3 tx:14.5 Mpps full 10G line rate
Perfect, full wirespeed! :-)
> usec 1 (default):
> burst=1,4,100 tx:3.9 Mpps
Here you are being limited by the TX ring queue cleanup, being too slow.
As desc here:
http://netoptimizer.blogspot.dk/2014/06/pktgen-for-network-overload-testing.html
> usec 0:
> burst=1 tx:4.9 Mpps
> burst=2 tx:6.6 Mpps
> burst=3 tx:7.9 Mpps
> burst=4 tx:8.7 Mpps
> burst=8 tx:10.3 Mpps
> burst=128 tx:12.4 Mpps
>
> Cc: Jesper Dangaard Brouer <brouer@redhat.com>
> Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
> ---
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
> tx queue size, irq affinity left in default.
> pause frames are off.
>
> Nice to finally see line rate generated by one cpu
Yes,
> Comparing to Jesper patch this one amortizes the cost
> of spin_lock and atomic_inc by doing HARD_TX_LOCK and
> atomic_add(N) once across N packets.
Nice additional optimizations :-)
> net/core/pktgen.c | 33 ++++++++++++++++++++++++++++++---
> 1 file changed, 30 insertions(+), 3 deletions(-)
>
> diff --git a/net/core/pktgen.c b/net/core/pktgen.c
> index 5c728aa..47557ba 100644
> --- a/net/core/pktgen.c
> +++ b/net/core/pktgen.c
> @@ -387,6 +387,7 @@ struct pktgen_dev {
> u16 queue_map_min;
> u16 queue_map_max;
> __u32 skb_priority; /* skb priority field */
> + int burst; /* number of duplicated packets to burst */
> int node; /* Memory node */
>
> #ifdef CONFIG_XFRM
[...]
> @@ -3299,7 +3313,8 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
> {
> struct net_device *odev = pkt_dev->odev;
> struct netdev_queue *txq;
> - int ret;
> + int burst_cnt, ret;
> + bool more;
>
> /* If device is offline, then don't send */
> if (unlikely(!netif_running(odev) || !netif_carrier_ok(odev))) {
> @@ -3347,8 +3362,14 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
> pkt_dev->last_ok = 0;
> goto unlock;
> }
> - atomic_inc(&(pkt_dev->skb->users));
> - ret = netdev_start_xmit(pkt_dev->skb, odev, txq, false);
> + atomic_add(pkt_dev->burst, &pkt_dev->skb->users);
> +
> + burst_cnt = 0;
> +
> +xmit_more:
> + more = ++burst_cnt < pkt_dev->burst;
> +
> + ret = netdev_start_xmit(pkt_dev->skb, odev, txq, more);
>
> switch (ret) {
> case NETDEV_TX_OK:
> @@ -3356,6 +3377,8 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
> pkt_dev->sofar++;
> pkt_dev->seq_num++;
> pkt_dev->tx_bytes += pkt_dev->last_pkt_size;
> + if (more)
> + goto xmit_more;
I think this will break my VLAN hack mode, that allows me to shoot
pktgen after the qdisc layer, but I'm okay with that, as I can just
avoid using this new burst mode and then it will still work for me.
> break;
> case NET_XMIT_DROP:
> case NET_XMIT_CN:
> @@ -3374,6 +3397,9 @@ static void pktgen_xmit(struct pktgen_dev *pkt_dev)
> atomic_dec(&(pkt_dev->skb->users));
> pkt_dev->last_ok = 0;
> }
> +
> + if (unlikely(pkt_dev->burst - burst_cnt > 0))
> + atomic_sub(pkt_dev->burst - burst_cnt, &pkt_dev->skb->users);
> unlock:
> HARD_TX_UNLOCK(odev, txq);
>
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply
* Re: [PATCH v3 net-next 1/3] ipv4: rename ip_options_echo to __ip_options_echo()
From: Joe Perches @ 2014-09-26 8:08 UTC (permalink / raw)
To: Julia Lawall
Cc: Eric Dumazet, Eric Dumazet, David S. Miller, netdev,
Yuchung Cheng, Neal Cardwell, Christoph Paasch
In-Reply-To: <alpine.DEB.2.02.1409260724520.2078@localhost6.localdomain6>
On Fri, 2014-09-26 at 07:29 +0200, Julia Lawall wrote:
> On Thu, 25 Sep 2014, Joe Perches wrote:
> > On Thu, 2014-09-25 at 13:38 -0700, Eric Dumazet wrote:
> > > On Thu, 2014-09-25 at 13:30 -0700, Joe Perches wrote:
[]
> > > > I wonder how much effort, if any, should be made to convert
> > > > struct sk_buff * to const struct sk_buff * where appropriate.
> > > >
> > > > For instance:n
> > > >
> > > > This __ip_options_echo could use const struct sk_buff *skb
> > > > if fib_compute_spec_dst was changed to const struct sk_buff *skb.
> > >
> > > Well, this seems something certainly doable, as a follow up ;)
> >
> > It's doable, but it seems a non-trivial inspection task.
> >
> > I believe coccinelle does not have the ability to automate this.
>
> What are the exact conditions when the change is possible?
>
> I guess something like the value is only used for accessing its fields.
> and is not passed to any other function?
I expect the entire call tree needs to be known
and inspected for modification of fields.
^ permalink raw reply
* Re: [PATCH v1 2/4] net: fec: ptp: Use hardware algorithm to adjust PTP counter.
From: Richard Cochran @ 2014-09-26 8:22 UTC (permalink / raw)
To: luwei.zhou@freescale.com
Cc: davem@davemloft.net, netdev@vger.kernel.org, shawn.guo@linaro.org,
bhutchings@solarflare.com, Fabio.Estevam@freescale.com,
fugang.duan@freescale.com, Frank.Li@freescale.com,
stephen@networkplumber.org
In-Reply-To: <cd5a482f0db446f9adce6906b19a9c56@BY2PR03MB441.namprd03.prod.outlook.com>
On Fri, Sep 26, 2014 at 05:53:41AM +0000, luwei.zhou@freescale.com wrote:
> [Zhou Luwei-B45643] Thanks. It can pass compile without warning and
> test. In fact I used other tools to increase SYNC rate to 16 per
> second and it works well.
Some PTP profiles use 128 per second.
Thanks,
Richard
^ permalink raw reply
* Re: [PATCH net-next] net: reorganize sk_buff for faster __copy_skb_header()
From: Jesper Dangaard Brouer @ 2014-09-26 8:35 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, netdev, Amir Vadai, John Fastabend, brouer
In-Reply-To: <1411680004.16953.57.camel@edumazet-glaptop2.roam.corp.google.com>
On Thu, 25 Sep 2014 14:20:04 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> With proliferation of bit fields in sk_buff, __copy_skb_header() became
> quite expensive, showing as the most expensive function in a GSO
> workload.
>
> __copy_skb_header() performance is also critical for non GSO TCP
> operations, as it is used from skb_clone()
>
> This patch carefully moves all the fields that were not copied in a
> separate zone : cloned, nohdr, fclone, peeked, head_frag, xmit_more
>
> Then I moved all other fields and all other copied fields in a section
> delimited by headers_start[0]/headers_end[0] section so that we
> can use a single memcpy() call, inlined by compiler using long
> word load/stores.
>
> I also tried to make all copies in the natural orders of sk_buff,
> to help hardware prefetching.
>
> I made sure sk_buff size did not change.
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> ---
I'm impressed, network ninja!
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
It would have been nice if you would have included the performance
improvement you saw, e.g. from:
http://thread.gmane.org/gmane.linux.network/332035/focus=332086
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply
* Re: [PATCH net-next] dql: dql_queued() should write first to reduce bus transactions
From: Jesper Dangaard Brouer @ 2014-09-26 8:47 UTC (permalink / raw)
To: Eric Dumazet
Cc: Tom Herbert, Linux Netdev List, David S. Miller, Alexander Duyck,
Toke Høiland-Jørgensen, Florian Westphal,
Jamal Hadi Salim, Dave Taht, John Fastabend, Daniel Borkmann,
Hannes Frederic Sowa, brouer
In-Reply-To: <1411711496.16953.92.camel@edumazet-glaptop2.roam.corp.google.com>
On Thu, 25 Sep 2014 23:04:56 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:
[...]
>
> It turned out the problem was caused by compiler trying to be
> smart, but involving a bad MESI transaction :
>
[...]
>
> I got an incredible 10 % gain [1] by making sure cpu do not attempt
> to get the cache line in Shared mode, but directly requests for
> ownership.
[...]
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> ---
I'm very impressed - thank you Eric for finding this!!!
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply
* Re: [PATCH net-next] bonding: make global bonding stats more reliable
From: Nikolay Aleksandrov @ 2014-09-26 8:52 UTC (permalink / raw)
To: Andy Gospodarek, netdev; +Cc: j.vosburgh, vfalico
In-Reply-To: <1411650996-2087-1-git-send-email-gospo@cumulusnetworks.com>
On 25/09/14 15:16, Andy Gospodarek wrote:
> As the code stands today, bonding stats are based simply on the stats
> from the member interfaces. If a member was to be removed from a bond,
> the stats would instantly drop. This would be confusing to an admin
> would would suddonly see interface stats drop while traffic is still
> flowing.
>
> In addition to preventing the stats drops mentioned above, new members
> will now be added to the bond and only traffic received after the member
> was added to the bond will be counted as part of bonding stats.
>
> Signed-off-by: Andy Gospodarek <gospo@cumulusnetworks.com>
> ---
Hi Andy,
> drivers/net/bonding/bond_main.c | 63 +++++++++++++++++++++++++++--------------
> drivers/net/bonding/bonding.h | 3 ++
> 2 files changed, 44 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
> index 5390475..4543c74 100644
> --- a/drivers/net/bonding/bond_main.c
> +++ b/drivers/net/bonding/bond_main.c
> @@ -1149,11 +1149,19 @@ static struct slave *bond_alloc_slave(struct bonding *bond)
> if (!slave)
> return NULL;
>
> + slave->slave_stats = kzalloc(sizeof(struct rtnl_link_stats64),
> + GFP_KERNEL);
> + if (!slave->slave_stats) {
> + kfree(slave);
> + return NULL;
> + }
> +
> if (BOND_MODE(bond) == BOND_MODE_8023AD) {
> SLAVE_AD_INFO(slave) = kzalloc(sizeof(struct ad_slave_info),
> GFP_KERNEL);
> if (!SLAVE_AD_INFO(slave)) {
> kfree(slave);
> + kfree(slave->slave_stats);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Wrong order. 1. slave_stats, 2. slave.
> return NULL;
> }
> }
> @@ -1167,6 +1175,7 @@ static void bond_free_slave(struct slave *slave)
> if (BOND_MODE(bond) == BOND_MODE_8023AD)
> kfree(SLAVE_AD_INFO(slave));
>
> + kfree(slave->slave_stats);
> kfree(slave);
> }
>
> @@ -1344,6 +1353,8 @@ int bond_enslave(struct net_device *bond_dev, struct net_device *slave_dev)
> }
>
> slave_dev->priv_flags |= IFF_BONDING;
> + /* initialize slave stats */
> + dev_get_stats(new_slave->dev, new_slave->slave_stats);
>
> if (bond_is_lb(bond)) {
> /* bond_alb_init_slave() must be called before all other stages since
> @@ -3085,38 +3096,43 @@ static struct rtnl_link_stats64 *bond_get_stats(struct net_device *bond_dev,
> struct list_head *iter;
> struct slave *slave;
>
> - memset(stats, 0, sizeof(*stats));
> + memcpy(stats, bond->bond_stats, sizeof(*stats));
>
> bond_for_each_slave(bond, slave, iter) {
> const struct rtnl_link_stats64 *sstats =
> dev_get_stats(slave->dev, &temp);
> + struct rtnl_link_stats64 *pstats = slave->slave_stats;
> +
> + stats->rx_packets += sstats->rx_packets - pstats->rx_packets;
> + stats->rx_bytes += sstats->rx_bytes - pstats->rx_bytes;
> + stats->rx_errors += sstats->rx_errors - pstats->rx_errors;
> + stats->rx_dropped += sstats->rx_dropped - pstats->rx_dropped;
>
> - stats->rx_packets += sstats->rx_packets;
> - stats->rx_bytes += sstats->rx_bytes;
> - stats->rx_errors += sstats->rx_errors;
> - stats->rx_dropped += sstats->rx_dropped;
> + stats->tx_packets += sstats->tx_packets - pstats->tx_packets;;
> + stats->tx_bytes += sstats->tx_bytes - pstats->tx_bytes;
> + stats->tx_errors += sstats->tx_errors - pstats->tx_errors;
> + stats->tx_dropped += sstats->tx_dropped - pstats->tx_dropped;
>
> - stats->tx_packets += sstats->tx_packets;
> - stats->tx_bytes += sstats->tx_bytes;
> - stats->tx_errors += sstats->tx_errors;
> - stats->tx_dropped += sstats->tx_dropped;
> + stats->multicast += sstats->multicast - pstats->multicast;
> + stats->collisions += sstats->collisions - pstats->collisions;
>
> - stats->multicast += sstats->multicast;
> - stats->collisions += sstats->collisions;
> + stats->rx_length_errors += sstats->rx_length_errors - pstats->rx_length_errors;
> + stats->rx_over_errors += sstats->rx_over_errors - pstats->rx_over_errors;
> + stats->rx_crc_errors += sstats->rx_crc_errors - pstats->rx_crc_errors;
> + stats->rx_frame_errors += sstats->rx_frame_errors - pstats->rx_frame_errors;
> + stats->rx_fifo_errors += sstats->rx_fifo_errors - pstats->rx_fifo_errors;
> + stats->rx_missed_errors += sstats->rx_missed_errors - pstats->rx_missed_errors;
>
> - stats->rx_length_errors += sstats->rx_length_errors;
> - stats->rx_over_errors += sstats->rx_over_errors;
> - stats->rx_crc_errors += sstats->rx_crc_errors;
> - stats->rx_frame_errors += sstats->rx_frame_errors;
> - stats->rx_fifo_errors += sstats->rx_fifo_errors;
> - stats->rx_missed_errors += sstats->rx_missed_errors;
> + stats->tx_aborted_errors += sstats->tx_aborted_errors - pstats->tx_aborted_errors;
> + stats->tx_carrier_errors += sstats->tx_carrier_errors - pstats->tx_carrier_errors;
> + stats->tx_fifo_errors += sstats->tx_fifo_errors - pstats->tx_fifo_errors;
> + stats->tx_heartbeat_errors += sstats->tx_heartbeat_errors - pstats->tx_heartbeat_errors;
> + stats->tx_window_errors += sstats->tx_window_errors - pstats->tx_window_errors;
>
> - stats->tx_aborted_errors += sstats->tx_aborted_errors;
> - stats->tx_carrier_errors += sstats->tx_carrier_errors;
> - stats->tx_fifo_errors += sstats->tx_fifo_errors;
> - stats->tx_heartbeat_errors += sstats->tx_heartbeat_errors;
> - stats->tx_window_errors += sstats->tx_window_errors;
> + /* save off the slave stats for the next run */
> + memcpy(pstats, sstats, sizeof(*sstats));
> }
> + memcpy(bond->bond_stats, stats, sizeof(*stats));
>
> return stats;
> }
> @@ -4258,6 +4274,9 @@ static int bond_init(struct net_device *bond_dev)
> bond_dev->addr_assign_type == NET_ADDR_PERM)
> eth_hw_addr_random(bond_dev);
>
> + /* initialize persistent stats for the bond */
> + bond->bond_stats = kzalloc(sizeof(struct rtnl_link_stats64),
> + GFP_ATOMIC);
^^^^^^^^^^^^^^^^^^^^^^^^
I don't think this will get freed if the bond device is destroyed.
> return 0;
> }
>
> diff --git a/drivers/net/bonding/bonding.h b/drivers/net/bonding/bonding.h
> index 6140bf0..fe25265 100644
> --- a/drivers/net/bonding/bonding.h
> +++ b/drivers/net/bonding/bonding.h
> @@ -24,6 +24,7 @@
> #include <linux/inetdevice.h>
> #include <linux/etherdevice.h>
> #include <linux/reciprocal_div.h>
> +#include <linux/if_link.h>
>
> #include "bond_3ad.h"
> #include "bond_alb.h"
> @@ -175,6 +176,7 @@ struct slave {
> struct netpoll *np;
> #endif
> struct kobject kobj;
> + struct rtnl_link_stats64 *slave_stats;
> };
>
> /*
> @@ -224,6 +226,7 @@ struct bonding {
> /* debugging support via debugfs */
> struct dentry *debug_dir;
> #endif /* CONFIG_DEBUG_FS */
> + struct rtnl_link_stats64 *bond_stats;
> };
>
> #define bond_slave_get_rcu(dev) \
>
^ permalink raw reply
* RE: [net-next PATCH 1/1 V4] qdisc: bulk dequeue support for qdiscs with TCQ_F_ONETXQUEUE
From: David Laight @ 2014-09-26 9:23 UTC (permalink / raw)
To: 'Eric Dumazet', Tom Herbert
Cc: Jesper Dangaard Brouer, Linux Netdev List, David S. Miller,
Alexander Duyck, Toke Høiland-Jørgensen,
Florian Westphal, Jamal Hadi Salim, Dave Taht, John Fastabend,
Daniel Borkmann, Hannes Frederic Sowa
In-Reply-To: <1411688409.16953.64.camel@edumazet-glaptop2.roam.corp.google.com>
From: Eric Dumazet
> On Wed, 2014-09-24 at 19:12 -0700, Eric Dumazet wrote:
...
> It turned out the problem I noticed was caused by compiler trying to be
> smart, but involving a bad MESI transaction.
>
> 0.05 mov 0xc0(%rax),%edi // LOAD dql->num_queued
> 0.48 mov %edx,0xc8(%rax) // STORE dql->last_obj_cnt = count
> 58.23 add %edx,%edi
> 0.58 cmp %edi,0xc4(%rax)
> 0.76 mov %edi,0xc0(%rax) // STORE dql->num_queued += count
> 0.72 js bd8
>
>
> I get an incredible 10 % gain by making sure cpu wont get the cache line
> in Shared mode.
That is a stunning difference between requesting 'exclusive' access
and upgrading 'shared' to exclusive.
Stinks of a cpu bug?
Or is the reported stall a side effect of waiting for the earlier
'cache line read' to complete in order to issue the 'upgrade to exclusive'.
In which case gcc's instruction scheduler probably needs to be taught
to schedule writes before reads.
> (I also tried a barrier() in netdev_tx_sent_queue() between the
>
> dql_queued(&dev_queue->dql, bytes);
> -
> + barrier();
> if (likely(dql_avail(&dev_queue->dql) >= 0))
>
> But following patch seems cleaner
>
> diff --git a/include/linux/dynamic_queue_limits.h b/include/linux/dynamic_queue_limits.h
> index 5621547d631b..978fbe332090 100644
> --- a/include/linux/dynamic_queue_limits.h
> +++ b/include/linux/dynamic_queue_limits.h
> @@ -80,7 +80,7 @@ static inline void dql_queued(struct dql *dql, unsigned int count)
> /* Returns how many objects can be queued, < 0 indicates over limit. */
> static inline int dql_avail(const struct dql *dql)
> {
> - return dql->adj_limit - dql->num_queued;
> + return ACCESS_ONCE(dql->adj_limit) - ACCESS_ONCE(dql->num_queued);
Dunno, that could have an impact on other calls where the values
are already in registers - I suspect ACCESS_ONCE() forces an access.
David
^ permalink raw reply
* Re: [PATCH net-next 2/3] ipv6: mld: do not overwrite uri when receiving an mldv2 query
From: Daniel Borkmann @ 2014-09-26 9:29 UTC (permalink / raw)
To: David L Stevens; +Cc: davem, hannes, netdev
In-Reply-To: <5424A408.5000208@oracle.com>
On 09/26/2014 01:23 AM, David L Stevens wrote:
...
> Now, probably that discussion should've happened when the tunables were put in, but
> having the sysctl's is still useful for setting the values when there is no querier
> present.
>
> When there is a querier, however, the original code IMO makes more sense, especially
> in the absence of any input from an administrator.
>
> I'm generally for allowing administrators complete flexibility, even if they use it
> for evil, so I think I'd prefer something along the lines of:
>
> 1) have an initial default of 1sec (v2) or 10sec (v1)
> 2) if an administrator sets the sysctl, override any
> other choice with that setting
> 3) if an administator has not set it, use the querier value
>
> That combination allows the querier to effectively set an appropriate interval for
> the entire network, allows an admin to change it per-host if desired, and uses the
> suggested defaults when there is no querier or admin intervention.
>
> Or maybe split the sysctls into one that forces the value and one that just sets
> a default which can be overridden by queriers.
>
> I don't think your patches are incorrect, but I don't think the original behavior
> is either. With your interpretation, the URI (but not the MRD or QRV), must be
> changed on every individual host to tune a network away from the default values.
> The current code doesn't have that problem.
I'm fine with either suggestion. Actually the _current_ situation we're in is
that in IPv4 we _always_ use the current, uncached _sysctl_ tuned setting of URI
(independent of any protocol version); while in IPv6 we use the _cached_ sysctl
URI in case of MLDv1 and _always_ overwrite the URI in case of MLDv2 (even for
MLDv1). Are you suggesting that, we then better adapt using the maxdelay value
everywhere and adapt URI to it, plus having a boolean knob defaulting to off for
an admin to enforce always using the provided sysctl default setting and not
the snooped MLD?
^ permalink raw reply
* [PATCH net-next] r8169:add support for RTL8168EP
From: Chun-Hao Lin @ 2014-09-26 10:18 UTC (permalink / raw)
To: netdev; +Cc: nic_swsd, linux-kernel, Chun-Hao Lin
RTL8168EP is Realtek PCIe Gigabit Ethernet controller.
It is a successor chip of RTL8168DP.
This patch add support for this chip.
Signed-off-by: Chun-Hao Lin <hau@realtek.com>
---
drivers/net/ethernet/realtek/r8169.c | 611 +++++++++++++++++++++++++++++------
1 file changed, 514 insertions(+), 97 deletions(-)
diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
index 1d81238..0ead9a7 100644
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -155,6 +155,9 @@ enum mac_version {
RTL_GIGA_MAC_VER_46,
RTL_GIGA_MAC_VER_47,
RTL_GIGA_MAC_VER_48,
+ RTL_GIGA_MAC_VER_49,
+ RTL_GIGA_MAC_VER_50,
+ RTL_GIGA_MAC_VER_51,
RTL_GIGA_MAC_NONE = 0xff,
};
@@ -302,6 +305,15 @@ static const struct {
[RTL_GIGA_MAC_VER_48] =
_R("RTL8107e", RTL_TD_1, FIRMWARE_8107E_2,
JUMBO_1K, false),
+ [RTL_GIGA_MAC_VER_49] =
+ _R("RTL8168ep/8111ep", RTL_TD_1, NULL,
+ JUMBO_9K, false),
+ [RTL_GIGA_MAC_VER_50] =
+ _R("RTL8168ep/8111ep", RTL_TD_1, NULL,
+ JUMBO_9K, false),
+ [RTL_GIGA_MAC_VER_51] =
+ _R("RTL8168ep/8111ep", RTL_TD_1, NULL,
+ JUMBO_9K, false),
};
#undef _R
@@ -400,6 +412,10 @@ enum rtl_registers {
FuncEvent = 0xf0,
FuncEventMask = 0xf4,
FuncPresetState = 0xf8,
+ IBCR0 = 0xf8,
+ IBCR2 = 0xf9,
+ IBIMR0 = 0xfa,
+ IBISR0 = 0xfb,
FuncForceEvent = 0xfc,
};
@@ -467,6 +483,7 @@ enum rtl8168_registers {
#define ERIAR_EXGMAC (0x00 << ERIAR_TYPE_SHIFT)
#define ERIAR_MSIX (0x01 << ERIAR_TYPE_SHIFT)
#define ERIAR_ASF (0x02 << ERIAR_TYPE_SHIFT)
+#define ERIAR_OOB (0x02 << ERIAR_TYPE_SHIFT)
#define ERIAR_MASK_SHIFT 12
#define ERIAR_MASK_0001 (0x1 << ERIAR_MASK_SHIFT)
#define ERIAR_MASK_0011 (0x3 << ERIAR_MASK_SHIFT)
@@ -935,93 +952,10 @@ static const struct rtl_cond name = { \
\
static bool name ## _check(struct rtl8169_private *tp)
-DECLARE_RTL_COND(rtl_ocpar_cond)
-{
- void __iomem *ioaddr = tp->mmio_addr;
-
- return RTL_R32(OCPAR) & OCPAR_FLAG;
-}
-
-static u32 ocp_read(struct rtl8169_private *tp, u8 mask, u16 reg)
-{
- void __iomem *ioaddr = tp->mmio_addr;
-
- RTL_W32(OCPAR, ((u32)mask & 0x0f) << 12 | (reg & 0x0fff));
-
- return rtl_udelay_loop_wait_high(tp, &rtl_ocpar_cond, 100, 20) ?
- RTL_R32(OCPDR) : ~0;
-}
-
-static void ocp_write(struct rtl8169_private *tp, u8 mask, u16 reg, u32 data)
-{
- void __iomem *ioaddr = tp->mmio_addr;
-
- RTL_W32(OCPDR, data);
- RTL_W32(OCPAR, OCPAR_FLAG | ((u32)mask & 0x0f) << 12 | (reg & 0x0fff));
-
- rtl_udelay_loop_wait_low(tp, &rtl_ocpar_cond, 100, 20);
-}
-
-DECLARE_RTL_COND(rtl_eriar_cond)
-{
- void __iomem *ioaddr = tp->mmio_addr;
-
- return RTL_R32(ERIAR) & ERIAR_FLAG;
-}
-
-static void rtl8168_oob_notify(struct rtl8169_private *tp, u8 cmd)
-{
- void __iomem *ioaddr = tp->mmio_addr;
-
- RTL_W8(ERIDR, cmd);
- RTL_W32(ERIAR, 0x800010e8);
- msleep(2);
-
- if (!rtl_udelay_loop_wait_low(tp, &rtl_eriar_cond, 100, 5))
- return;
-
- ocp_write(tp, 0x1, 0x30, 0x00000001);
-}
-
#define OOB_CMD_RESET 0x00
#define OOB_CMD_DRIVER_START 0x05
#define OOB_CMD_DRIVER_STOP 0x06
-static u16 rtl8168_get_ocp_reg(struct rtl8169_private *tp)
-{
- return (tp->mac_version == RTL_GIGA_MAC_VER_31) ? 0xb8 : 0x10;
-}
-
-DECLARE_RTL_COND(rtl_ocp_read_cond)
-{
- u16 reg;
-
- reg = rtl8168_get_ocp_reg(tp);
-
- return ocp_read(tp, 0x0f, reg) & 0x00000800;
-}
-
-static void rtl8168_driver_start(struct rtl8169_private *tp)
-{
- rtl8168_oob_notify(tp, OOB_CMD_DRIVER_START);
-
- rtl_msleep_loop_wait_high(tp, &rtl_ocp_read_cond, 10, 10);
-}
-
-static void rtl8168_driver_stop(struct rtl8169_private *tp)
-{
- rtl8168_oob_notify(tp, OOB_CMD_DRIVER_STOP);
-
- rtl_msleep_loop_wait_low(tp, &rtl_ocp_read_cond, 10, 10);
-}
-
-static int r8168dp_check_dash(struct rtl8169_private *tp)
-{
- u16 reg = rtl8168_get_ocp_reg(tp);
-
- return (ocp_read(tp, 0x0f, reg) & 0x00008000) ? 1 : 0;
-}
-
static bool rtl_ocp_reg_failure(struct rtl8169_private *tp, u32 reg)
{
if (reg & 0xffff0001) {
@@ -1163,6 +1097,13 @@ static int r8169_mdio_read(struct rtl8169_private *tp, int reg)
return value;
}
+DECLARE_RTL_COND(rtl_ocpar_cond)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+
+ return RTL_R32(OCPAR) & OCPAR_FLAG;
+}
+
static void r8168dp_1_mdio_access(struct rtl8169_private *tp, int reg, u32 data)
{
void __iomem *ioaddr = tp->mmio_addr;
@@ -1298,6 +1239,13 @@ static u16 rtl_ephy_read(struct rtl8169_private *tp, int reg_addr)
RTL_R32(EPHYAR) & EPHYAR_DATA_MASK : ~0;
}
+DECLARE_RTL_COND(rtl_eriar_cond)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+
+ return RTL_R32(ERIAR) & ERIAR_FLAG;
+}
+
static void rtl_eri_write(struct rtl8169_private *tp, int addr, u32 mask,
u32 val, int type)
{
@@ -1329,6 +1277,45 @@ static void rtl_w1w0_eri(struct rtl8169_private *tp, int addr, u32 mask, u32 p,
rtl_eri_write(tp, addr, mask, (val & ~m) | p, type);
}
+static u32 ocp_read(struct rtl8169_private *tp, u8 mask, u16 reg)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+
+ if (tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
+ return rtl_eri_read(tp, reg, ERIAR_OOB);
+ } else if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ RTL_W32(OCPAR, ((u32)mask & 0x0f) << 12 | (reg & 0x0fff));
+ return rtl_udelay_loop_wait_high(tp, &rtl_ocpar_cond, 100, 20)
+ ? RTL_R32(OCPDR) : ~0;
+ }
+
+ return ~0;
+}
+
+static void ocp_write(struct rtl8169_private *tp, u8 mask, u16 reg, u32 data)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+
+ if (tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
+ rtl_eri_write(tp, reg, ((u32)mask & 0x0f) << ERIAR_MASK_SHIFT,
+ data, ERIAR_OOB);
+ } else if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ RTL_W32(OCPDR, data);
+ RTL_W32(OCPAR, OCPAR_FLAG | ((u32)mask & 0x0f) << 12 |
+ (reg & 0x0fff));
+
+ rtl_udelay_loop_wait_low(tp, &rtl_ocpar_cond, 100, 20);
+ }
+}
+
struct exgmac_reg {
u16 addr;
u16 mask;
@@ -1361,6 +1348,103 @@ static u8 rtl8168d_efuse_read(struct rtl8169_private *tp, int reg_addr)
RTL_R32(EFUSEAR) & EFUSEAR_DATA_MASK : ~0;
}
+static u16 rtl8168_get_ocp_reg(struct rtl8169_private *tp)
+{
+ return (tp->mac_version == RTL_GIGA_MAC_VER_31) ? 0xb8 : 0x10;
+}
+
+DECLARE_RTL_COND(rtl_ocp_read_cond)
+{
+ u16 reg;
+
+ reg = rtl8168_get_ocp_reg(tp);
+
+ return ocp_read(tp, 0x0f, reg) & 0x00000800;
+}
+
+DECLARE_RTL_COND(rtl_ep_ocp_read_cond)
+{
+ return ocp_read(tp, 0x0f, 0x124) & 0x00000001;
+}
+
+DECLARE_RTL_COND(rtl_ocp_tx_cond)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+
+ return RTL_R8(IBISR0) & 0x02;
+}
+
+static void rtl8168_oob_notify(struct rtl8169_private *tp, u8 cmd)
+{
+ rtl_eri_write(tp, 0xe8, ERIAR_MASK_0001, cmd, ERIAR_OOB);
+
+ ocp_write(tp, 0x1, 0x30, 0x01);
+}
+
+static void rtl8168_driver_start(struct rtl8169_private *tp)
+{
+ if (tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
+ u32 tmp;
+
+ ocp_write(tp, 0x01, 0x180, OOB_CMD_DRIVER_START);
+ tmp = ocp_read(tp, 0x01, 0x30);
+ tmp |= 0x01;
+ ocp_write(tp, 0x01, 0x30, tmp);
+ rtl_msleep_loop_wait_high(tp, &rtl_ep_ocp_read_cond, 10, 10);
+ } else if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ rtl8168_oob_notify(tp, OOB_CMD_DRIVER_START);
+ rtl_msleep_loop_wait_high(tp, &rtl_ocp_read_cond, 10, 10);
+ }
+}
+
+static void rtl8168_driver_stop(struct rtl8169_private *tp)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+
+ if (tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
+ u32 tmp;
+
+ RTL_W8(IBCR2, RTL_R8(IBCR2) & ~0x01);
+ rtl_msleep_loop_wait_low(tp, &rtl_ocp_tx_cond, 50, 2000);
+ RTL_W8(IBISR0, RTL_R8(IBISR0) | 0x20);
+ RTL_W8(IBCR0, RTL_R8(IBCR0) & ~0x01);
+
+ ocp_write(tp, 0x01, 0x180, OOB_CMD_DRIVER_STOP);
+ tmp = ocp_read(tp, 0x01, 0x30);
+ tmp |= 0x01;
+ ocp_write(tp, 0x01, 0x30, tmp);
+ rtl_msleep_loop_wait_low(tp, &rtl_ep_ocp_read_cond, 10, 10);
+ } else if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ rtl8168_oob_notify(tp, OOB_CMD_DRIVER_STOP);
+ rtl_msleep_loop_wait_low(tp, &rtl_ocp_read_cond, 10, 10);
+ }
+}
+
+static int r8168_check_dash(struct rtl8169_private *tp)
+{
+ if (tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
+ return (ocp_read(tp, 0x0f, 0x128) & 0x00000001) ? 1 : 0;
+ } else if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ u16 reg = rtl8168_get_ocp_reg(tp);
+
+ return (ocp_read(tp, 0x0f, reg) & 0x00008000) ? 1 : 0;
+ }
+
+ return 0;
+}
+
static u16 rtl_get_events(struct rtl8169_private *tp)
{
void __iomem *ioaddr = tp->mmio_addr;
@@ -1548,6 +1632,9 @@ static u32 __rtl8169_get_wol(struct rtl8169_private *tp)
switch (tp->mac_version) {
case RTL_GIGA_MAC_VER_45:
case RTL_GIGA_MAC_VER_46:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
if (rtl_eri_read(tp, 0xdc, ERIAR_EXGMAC) & MagicPacket_v2)
wolopts |= WAKE_MAGIC;
break;
@@ -1603,6 +1690,9 @@ static void __rtl8169_set_wol(struct rtl8169_private *tp, u32 wolopts)
switch (tp->mac_version) {
case RTL_GIGA_MAC_VER_45:
case RTL_GIGA_MAC_VER_46:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
tmp = ARRAY_SIZE(cfg) - 1;
if (wolopts & WAKE_MAGIC)
rtl_w1w0_eri(tp,
@@ -2109,6 +2199,11 @@ static void rtl8169_get_mac_version(struct rtl8169_private *tp,
u32 val;
int mac_version;
} mac_info[] = {
+ /* 8168EP family. */
+ { 0x7cf00000, 0x50200000, RTL_GIGA_MAC_VER_51 },
+ { 0x7cf00000, 0x50100000, RTL_GIGA_MAC_VER_50 },
+ { 0x7cf00000, 0x50000000, RTL_GIGA_MAC_VER_49 },
+
/* 8168H family. */
{ 0x7cf00000, 0x54100000, RTL_GIGA_MAC_VER_46 },
{ 0x7cf00000, 0x54000000, RTL_GIGA_MAC_VER_45 },
@@ -3724,6 +3819,143 @@ static void rtl8168h_2_hw_phy_config(struct rtl8169_private *tp)
rtl_writephy(tp, 0x1f, 0x0000);
}
+static void rtl8168ep_1_hw_phy_config(struct rtl8169_private *tp)
+{
+ rtl_apply_firmware(tp);
+
+ /* Enable PHY auto speed down */
+ rtl_writephy(tp, 0x1f, 0x0a44);
+ rtl_w1w0_phy(tp, 0x11, 0x000c, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* patch 10M & ALDPS */
+ rtl_writephy(tp, 0x1f, 0x0bcc);
+ rtl_w1w0_phy(tp, 0x14, 0x0000, 0x0100);
+ rtl_writephy(tp, 0x1f, 0x0a44);
+ rtl_w1w0_phy(tp, 0x11, 0x00c0, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x8084);
+ rtl_w1w0_phy(tp, 0x14, 0x0000, 0x6000);
+ rtl_w1w0_phy(tp, 0x10, 0x1003, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Enable EEE auto-fallback function */
+ rtl_writephy(tp, 0x1f, 0x0a4b);
+ rtl_w1w0_phy(tp, 0x11, 0x0004, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Enable UC LPF tune function */
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x8012);
+ rtl_w1w0_phy(tp, 0x14, 0x8000, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* set rg_sel_sdm_rate */
+ rtl_writephy(tp, 0x1f, 0x0C42);
+ rtl_w1w0_phy(tp, 0x11, 0x4000, 0x2000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Check ALDPS bit, disable it if enabled */
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ if (rtl_readphy(tp, 0x10) & 0x0004)
+ rtl_w1w0_phy(tp, 0x10, 0x0000, 0x0004);
+
+ rtl_writephy(tp, 0x1f, 0x0000);
+}
+
+static void rtl8168ep_2_hw_phy_config(struct rtl8169_private *tp)
+{
+ rtl_apply_firmware(tp);
+
+ /* patch 10M & ALDPS */
+ rtl_writephy(tp, 0x1f, 0x0bcc);
+ rtl_w1w0_phy(tp, 0x14, 0x0000, 0x0100);
+ rtl_writephy(tp, 0x1f, 0x0a44);
+ rtl_w1w0_phy(tp, 0x11, 0x00c0, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x8084);
+ rtl_w1w0_phy(tp, 0x14, 0x0000, 0x6000);
+ rtl_w1w0_phy(tp, 0x10, 0x1003, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Enable UC LPF tune function */
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x8012);
+ rtl_w1w0_phy(tp, 0x14, 0x8000, 0x0000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Set rg_sel_sdm_rate */
+ rtl_writephy(tp, 0x1f, 0x0C42);
+ rtl_w1w0_phy(tp, 0x11, 0x4000, 0x2000);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Channel estimation parameters */
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x80f3);
+ rtl_w1w0_phy(tp, 0x14, 0x8b00, ~0x8bff);
+ rtl_writephy(tp, 0x13, 0x80f0);
+ rtl_w1w0_phy(tp, 0x14, 0x3a00, ~0x3aff);
+ rtl_writephy(tp, 0x13, 0x80ef);
+ rtl_w1w0_phy(tp, 0x14, 0x0500, ~0x05ff);
+ rtl_writephy(tp, 0x13, 0x80f6);
+ rtl_w1w0_phy(tp, 0x14, 0x6e00, ~0x6eff);
+ rtl_writephy(tp, 0x13, 0x80ec);
+ rtl_w1w0_phy(tp, 0x14, 0x6800, ~0x68ff);
+ rtl_writephy(tp, 0x13, 0x80ed);
+ rtl_w1w0_phy(tp, 0x14, 0x7c00, ~0x7cff);
+ rtl_writephy(tp, 0x13, 0x80f2);
+ rtl_w1w0_phy(tp, 0x14, 0xf400, ~0xf4ff);
+ rtl_writephy(tp, 0x13, 0x80f4);
+ rtl_w1w0_phy(tp, 0x14, 0x8500, ~0x85ff);
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x8110);
+ rtl_w1w0_phy(tp, 0x14, 0xa800, ~0xa8ff);
+ rtl_writephy(tp, 0x13, 0x810f);
+ rtl_w1w0_phy(tp, 0x14, 0x1d00, ~0x1dff);
+ rtl_writephy(tp, 0x13, 0x8111);
+ rtl_w1w0_phy(tp, 0x14, 0xf500, ~0xf5ff);
+ rtl_writephy(tp, 0x13, 0x8113);
+ rtl_w1w0_phy(tp, 0x14, 0x6100, ~0x61ff);
+ rtl_writephy(tp, 0x13, 0x8115);
+ rtl_w1w0_phy(tp, 0x14, 0x9200, ~0x92ff);
+ rtl_writephy(tp, 0x13, 0x810e);
+ rtl_w1w0_phy(tp, 0x14, 0x0400, ~0x04ff);
+ rtl_writephy(tp, 0x13, 0x810c);
+ rtl_w1w0_phy(tp, 0x14, 0x7C00, ~0x7Cff);
+ rtl_writephy(tp, 0x13, 0x810b);
+ rtl_w1w0_phy(tp, 0x14, 0x5a00, ~0x5aff);
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ rtl_writephy(tp, 0x13, 0x80d1);
+ rtl_w1w0_phy(tp, 0x14, 0xff00, ~0xffff);
+ rtl_writephy(tp, 0x13, 0x80cd);
+ rtl_w1w0_phy(tp, 0x14, 0x9e00, ~0x9eff);
+ rtl_writephy(tp, 0x13, 0x80d3);
+ rtl_w1w0_phy(tp, 0x14, 0x0e00, ~0x0eff);
+ rtl_writephy(tp, 0x13, 0x80d5);
+ rtl_w1w0_phy(tp, 0x14, 0xca00, ~0xcaff);
+ rtl_writephy(tp, 0x13, 0x80d7);
+ rtl_w1w0_phy(tp, 0x14, 0x8400, ~0x84ff);
+
+ /* Force PWM-mode */
+ rtl_writephy(tp, 0x1f, 0x0bcd);
+ rtl_writephy(tp, 0x14, 0x5065);
+ rtl_writephy(tp, 0x14, 0xd065);
+ rtl_writephy(tp, 0x1f, 0x0bC8);
+ rtl_writephy(tp, 0x12, 0x00ed);
+ rtl_writephy(tp, 0x1f, 0x0bcd);
+ rtl_writephy(tp, 0x14, 0x1065);
+ rtl_writephy(tp, 0x14, 0x9065);
+ rtl_writephy(tp, 0x14, 0x1065);
+ rtl_writephy(tp, 0x1f, 0x0000);
+
+ /* Check ALDPS bit, disable it if enabled */
+ rtl_writephy(tp, 0x1f, 0x0a43);
+ if (rtl_readphy(tp, 0x10) & 0x0004)
+ rtl_w1w0_phy(tp, 0x10, 0x0000, 0x0004);
+
+ rtl_writephy(tp, 0x1f, 0x0000);
+}
+
static void rtl8102e_hw_phy_config(struct rtl8169_private *tp)
{
static const struct phy_reg phy_reg_init[] = {
@@ -3923,6 +4155,14 @@ static void rtl_hw_phy_config(struct net_device *dev)
rtl8168h_2_hw_phy_config(tp);
break;
+ case RTL_GIGA_MAC_VER_49:
+ rtl8168ep_1_hw_phy_config(tp);
+ break;
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
+ rtl8168ep_2_hw_phy_config(tp);
+ break;
+
case RTL_GIGA_MAC_VER_41:
default:
break;
@@ -4137,6 +4377,9 @@ static void rtl_init_mdio_ops(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_46:
case RTL_GIGA_MAC_VER_47:
case RTL_GIGA_MAC_VER_48:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
ops->write = r8168g_mdio_write;
ops->read = r8168g_mdio_read;
break;
@@ -4195,6 +4438,9 @@ static void rtl_wol_suspend_quirk(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_46:
case RTL_GIGA_MAC_VER_47:
case RTL_GIGA_MAC_VER_48:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
RTL_W32(RxConfig, RTL_R32(RxConfig) |
AcceptBroadcast | AcceptMulticast | AcceptMyPhys);
break;
@@ -4265,7 +4511,7 @@ static void r810x_pll_power_up(struct rtl8169_private *tp)
break;
case RTL_GIGA_MAC_VER_47:
case RTL_GIGA_MAC_VER_48:
- RTL_W8(PMCH, RTL_R8(PMCH) | 0xC0);
+ RTL_W8(PMCH, RTL_R8(PMCH) | 0xc0);
break;
default:
RTL_W8(PMCH, RTL_R8(PMCH) | 0x80);
@@ -4339,8 +4585,11 @@ static void r8168_pll_power_down(struct rtl8169_private *tp)
if ((tp->mac_version == RTL_GIGA_MAC_VER_27 ||
tp->mac_version == RTL_GIGA_MAC_VER_28 ||
- tp->mac_version == RTL_GIGA_MAC_VER_31) &&
- r8168dp_check_dash(tp)) {
+ tp->mac_version == RTL_GIGA_MAC_VER_31 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) &&
+ r8168_check_dash(tp)) {
return;
}
@@ -4369,12 +4618,16 @@ static void r8168_pll_power_down(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_33:
case RTL_GIGA_MAC_VER_45:
case RTL_GIGA_MAC_VER_46:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
RTL_W8(PMCH, RTL_R8(PMCH) & ~0x80);
break;
case RTL_GIGA_MAC_VER_40:
case RTL_GIGA_MAC_VER_41:
+ case RTL_GIGA_MAC_VER_49:
rtl_w1w0_eri(tp, 0x1a8, ERIAR_MASK_1111, 0x00000000,
0xfc000000, ERIAR_EXGMAC);
+ RTL_W8(PMCH, RTL_R8(PMCH) & ~0x80);
break;
}
}
@@ -4395,10 +4648,14 @@ static void r8168_pll_power_up(struct rtl8169_private *tp)
break;
case RTL_GIGA_MAC_VER_45:
case RTL_GIGA_MAC_VER_46:
- RTL_W8(PMCH, RTL_R8(PMCH) | 0xC0);
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
+ RTL_W8(PMCH, RTL_R8(PMCH) | 0xc0);
break;
case RTL_GIGA_MAC_VER_40:
case RTL_GIGA_MAC_VER_41:
+ case RTL_GIGA_MAC_VER_49:
+ RTL_W8(PMCH, RTL_R8(PMCH) | 0xc0);
rtl_w1w0_eri(tp, 0x1a8, ERIAR_MASK_1111, 0xfc000000,
0x00000000, ERIAR_EXGMAC);
break;
@@ -4472,6 +4729,9 @@ static void rtl_init_pll_power_ops(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_44:
case RTL_GIGA_MAC_VER_45:
case RTL_GIGA_MAC_VER_46:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
ops->down = r8168_pll_power_down;
ops->up = r8168_pll_power_up;
break;
@@ -4526,6 +4786,9 @@ static void rtl_init_rxcfg(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_46:
case RTL_GIGA_MAC_VER_47:
case RTL_GIGA_MAC_VER_48:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
RTL_W32(RxConfig, RX128_INT_EN | RX_DMA_BURST | RX_EARLY_OFF);
break;
default:
@@ -4691,6 +4954,9 @@ static void rtl_init_jumbo_ops(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_46:
case RTL_GIGA_MAC_VER_47:
case RTL_GIGA_MAC_VER_48:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
default:
ops->disable = NULL;
ops->enable = NULL;
@@ -4805,7 +5071,10 @@ static void rtl8169_hw_reset(struct rtl8169_private *tp)
tp->mac_version == RTL_GIGA_MAC_VER_45 ||
tp->mac_version == RTL_GIGA_MAC_VER_46 ||
tp->mac_version == RTL_GIGA_MAC_VER_47 ||
- tp->mac_version == RTL_GIGA_MAC_VER_48) {
+ tp->mac_version == RTL_GIGA_MAC_VER_48 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
RTL_W8(ChipCmd, RTL_R8(ChipCmd) | StopReq);
rtl_udelay_loop_wait_high(tp, &rtl_txcfg_empty_cond, 100, 666);
} else {
@@ -5731,6 +6000,125 @@ static void rtl_hw_start_8168h_1(struct rtl8169_private *tp)
r8168_mac_ocp_write(tp, 0xc09e, 0x0000);
}
+static void rtl_hw_start_8168ep(struct rtl8169_private *tp)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+ struct pci_dev *pdev = tp->pci_dev;
+
+ RTL_W32(TxConfig, RTL_R32(TxConfig) | TXCFG_AUTO_FIFO);
+
+ rtl_eri_write(tp, 0x3a0, ERIAR_MASK_1111, 0x00000000, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0x3a4, ERIAR_MASK_1111, 0x00000000, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0x3a8, ERIAR_MASK_1111, 0x00000000, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0x3ac, ERIAR_MASK_1111, 0x00000000, ERIAR_EXGMAC);
+
+ rtl_eri_write(tp, 0xc8, ERIAR_MASK_0101, 0x00080002, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0xcc, ERIAR_MASK_0001, 0x2f, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0xd0, ERIAR_MASK_0001, 0x5f, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0xe8, ERIAR_MASK_1111, 0x00100006, ERIAR_EXGMAC);
+
+ rtl_csi_access_enable_1(tp);
+
+ rtl_tx_performance_tweak(pdev, 0x5 << MAX_READ_REQUEST_SHIFT);
+
+ rtl_w1w0_eri(tp, 0xdc, ERIAR_MASK_0001, 0x00, 0x01, ERIAR_EXGMAC);
+ rtl_w1w0_eri(tp, 0xdc, ERIAR_MASK_0001, 0x01, 0x00, ERIAR_EXGMAC);
+
+ rtl_w1w0_eri(tp, 0xd4, ERIAR_MASK_1111, 0x1f80, 0x00, ERIAR_EXGMAC);
+
+ rtl_eri_write(tp, 0x5f0, ERIAR_MASK_0011, 0x4f87, ERIAR_EXGMAC);
+
+ RTL_W8(ChipCmd, CmdTxEnb | CmdRxEnb);
+ RTL_W32(MISC, RTL_R32(MISC) & ~RXDV_GATED_EN);
+ RTL_W8(MaxTxPacketSize, EarlySize);
+
+ rtl_eri_write(tp, 0xc0, ERIAR_MASK_0011, 0x0000, ERIAR_EXGMAC);
+ rtl_eri_write(tp, 0xb8, ERIAR_MASK_0011, 0x0000, ERIAR_EXGMAC);
+
+ /* Adjust EEE LED frequency */
+ RTL_W8(EEE_LED, RTL_R8(EEE_LED) & ~0x07);
+
+ rtl_w1w0_eri(tp, 0x2fc, ERIAR_MASK_0001, 0x01, 0x06, ERIAR_EXGMAC);
+
+ RTL_W8(DLLPR, RTL_R8(DLLPR) & ~TX_10M_PS_EN);
+
+ rtl_pcie_state_l2l3_enable(tp, false);
+}
+
+static void rtl_hw_start_8168ep_1(struct rtl8169_private *tp)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+ static const struct ephy_info e_info_8168ep_1[] = {
+ { 0x00, 0xffff, 0x10ab },
+ { 0x06, 0xffff, 0xf030 },
+ { 0x08, 0xffff, 0x2006 },
+ { 0x0d, 0xffff, 0x1666 },
+ { 0x0c, 0x3ff0, 0x0000 }
+ };
+
+ /* disable aspm and clock request before access ephy */
+ RTL_W8(Config2, RTL_R8(Config2) & ~ClkReqEn);
+ RTL_W8(Config5, RTL_R8(Config5) & ~ASPM_en);
+ rtl_ephy_init(tp, e_info_8168ep_1, ARRAY_SIZE(e_info_8168ep_1));
+
+ rtl_hw_start_8168ep(tp);
+}
+
+static void rtl_hw_start_8168ep_2(struct rtl8169_private *tp)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+ static const struct ephy_info e_info_8168ep_2[] = {
+ { 0x00, 0xffff, 0x10a3 },
+ { 0x19, 0xffff, 0xfc00 },
+ { 0x1e, 0xffff, 0x20ea }
+ };
+
+ /* disable aspm and clock request before access ephy */
+ RTL_W8(Config2, RTL_R8(Config2) & ~ClkReqEn);
+ RTL_W8(Config5, RTL_R8(Config5) & ~ASPM_en);
+ rtl_ephy_init(tp, e_info_8168ep_2, ARRAY_SIZE(e_info_8168ep_2));
+
+ rtl_hw_start_8168ep(tp);
+
+ RTL_W8(DLLPR, RTL_R8(DLLPR) & ~PFM_EN);
+ RTL_W8(DLLPR, RTL_R8(MISC_1) & ~PFM_D3COLD_EN);
+}
+
+static void rtl_hw_start_8168ep_3(struct rtl8169_private *tp)
+{
+ void __iomem *ioaddr = tp->mmio_addr;
+ u32 data;
+ static const struct ephy_info e_info_8168ep_3[] = {
+ { 0x00, 0xffff, 0x10a3 },
+ { 0x19, 0xffff, 0x7c00 },
+ { 0x1e, 0xffff, 0x20eb },
+ { 0x0d, 0xffff, 0x1666 }
+ };
+
+ /* disable aspm and clock request before access ephy */
+ RTL_W8(Config2, RTL_R8(Config2) & ~ClkReqEn);
+ RTL_W8(Config5, RTL_R8(Config5) & ~ASPM_en);
+ rtl_ephy_init(tp, e_info_8168ep_3, ARRAY_SIZE(e_info_8168ep_3));
+
+ rtl_hw_start_8168ep(tp);
+
+ RTL_W8(DLLPR, RTL_R8(DLLPR) & ~PFM_EN);
+ RTL_W8(DLLPR, RTL_R8(MISC_1) & ~PFM_D3COLD_EN);
+
+ data = r8168_mac_ocp_read(tp, 0xd3e2);
+ data &= 0xf000;
+ data |= 0x0271;
+ r8168_mac_ocp_write(tp, 0xd3e2, data);
+
+ data = r8168_mac_ocp_read(tp, 0xd3e4);
+ data &= 0xff00;
+ r8168_mac_ocp_write(tp, 0xd3e4, data);
+
+ data = r8168_mac_ocp_read(tp, 0xe860);
+ data |= 0x0080;
+ r8168_mac_ocp_write(tp, 0xe860, data);
+}
+
static void rtl_hw_start_8168(struct net_device *dev)
{
struct rtl8169_private *tp = netdev_priv(dev);
@@ -5846,6 +6234,18 @@ static void rtl_hw_start_8168(struct net_device *dev)
rtl_hw_start_8168h_1(tp);
break;
+ case RTL_GIGA_MAC_VER_49:
+ rtl_hw_start_8168ep_1(tp);
+ break;
+
+ case RTL_GIGA_MAC_VER_50:
+ rtl_hw_start_8168ep_2(tp);
+ break;
+
+ case RTL_GIGA_MAC_VER_51:
+ rtl_hw_start_8168ep_3(tp);
+ break;
+
default:
printk(KERN_ERR PFX "%s: unknown chipset (mac_version = %d).\n",
dev->name, tp->mac_version);
@@ -7366,9 +7766,13 @@ static void rtl_remove_one(struct pci_dev *pdev)
struct net_device *dev = pci_get_drvdata(pdev);
struct rtl8169_private *tp = netdev_priv(dev);
- if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
- tp->mac_version == RTL_GIGA_MAC_VER_28 ||
- tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ if ((tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) &&
+ r8168_check_dash(tp)) {
rtl8168_driver_stop(tp);
}
@@ -7524,6 +7928,9 @@ static void rtl_hw_initialize(struct rtl8169_private *tp)
case RTL_GIGA_MAC_VER_46:
case RTL_GIGA_MAC_VER_47:
case RTL_GIGA_MAC_VER_48:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
rtl_hw_init_8168g(tp);
break;
@@ -7664,6 +8071,9 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
switch (tp->mac_version) {
case RTL_GIGA_MAC_VER_45:
case RTL_GIGA_MAC_VER_46:
+ case RTL_GIGA_MAC_VER_49:
+ case RTL_GIGA_MAC_VER_50:
+ case RTL_GIGA_MAC_VER_51:
if (rtl_eri_read(tp, 0xdc, ERIAR_EXGMAC) & MagicPacket_v2)
tp->features |= RTL_FEATURE_WOL;
if ((RTL_R8(Config3) & LinkUp) != 0)
@@ -7703,11 +8113,14 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
if (tp->mac_version == RTL_GIGA_MAC_VER_45 ||
tp->mac_version == RTL_GIGA_MAC_VER_46 ||
tp->mac_version == RTL_GIGA_MAC_VER_47 ||
- tp->mac_version == RTL_GIGA_MAC_VER_48) {
+ tp->mac_version == RTL_GIGA_MAC_VER_48 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) {
u16 mac_addr[3];
- *(u32 *)&mac_addr[0] = rtl_eri_read(tp, 0xE0, ERIAR_EXGMAC);
- *(u16 *)&mac_addr[2] = rtl_eri_read(tp, 0xE4, ERIAR_EXGMAC);
+ *(u32 *)&mac_addr[0] = rtl_eri_read(tp, 0xe0, ERIAR_EXGMAC);
+ *(u16 *)&mac_addr[2] = rtl_eri_read(tp, 0xe4, ERIAR_EXGMAC);
if (is_valid_ether_addr((u8 *)mac_addr))
rtl_rar_set(tp, (u8 *)mac_addr);
@@ -7780,9 +8193,13 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
rtl_chip_infos[chipset].jumbo_tx_csum ? "ok" : "ko");
}
- if (tp->mac_version == RTL_GIGA_MAC_VER_27 ||
- tp->mac_version == RTL_GIGA_MAC_VER_28 ||
- tp->mac_version == RTL_GIGA_MAC_VER_31) {
+ if ((tp->mac_version == RTL_GIGA_MAC_VER_27 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_28 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_31 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_49 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_50 ||
+ tp->mac_version == RTL_GIGA_MAC_VER_51) &&
+ r8168_check_dash(tp)) {
rtl8168_driver_start(tp);
}
--
1.8.3.2
^ permalink raw reply related
* Re: [PATCH net-next] dql: dql_queued() should write first to reduce bus transactions
From: Hannes Frederic Sowa @ 2014-09-26 11:06 UTC (permalink / raw)
To: Eric Dumazet, Tom Herbert
Cc: Jesper Dangaard Brouer, Linux Netdev List, David S. Miller,
Alexander Duyck, Toke Høiland-Jørgensen,
Florian Westphal, Jamal Hadi Salim, Dave Taht, John Fastabend,
Daniel Borkmann
In-Reply-To: <1411711496.16953.92.camel@edumazet-glaptop2.roam.corp.google.com>
On Fri, Sep 26, 2014, at 08:04, Eric Dumazet wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> While doing high throughput test on a BQL enabled NIC,
> I found a very high cost in ndo_start_xmit() when accessing BQL data.
>
> It turned out the problem was caused by compiler trying to be
> smart, but involving a bad MESI transaction :
>
> 0.05 │ mov 0xc0(%rax),%edi // LOAD dql->num_queued
> 0.48 │ mov %edx,0xc8(%rax) // STORE dql->last_obj_cnt = count
> 58.23 │ add %edx,%edi
> 0.58 │ cmp %edi,0xc4(%rax)
> 0.76 │ mov %edi,0xc0(%rax) // STORE dql->num_queued += count
> 0.72 │ js bd8
>
>
> I got an incredible 10 % gain [1] by making sure cpu do not attempt
> to get the cache line in Shared mode, but directly requests for
> ownership.
>
> New code :
> mov %edx,0xc8(%rax) // STORE dql->last_obj_cnt = count
> add %edx,0xc0(%rax) // RMW dql->num_queued += count
> mov 0xc4(%rax),%ecx // LOAD dql->adj_limit
> mov 0xc0(%rax),%edx // LOAD dql->num_queued
> cmp %edx,%ecx
>
> The TX completion was running from another cpu, with high interrupts
> rate.
>
> Note that I am using barrier() as a soft hint, as mb() here could be
> too heavy cost.
>
> [1] This was a netperf TCP_STREAM with TSO disabled, but GSO enabled.
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> ---
> include/linux/dynamic_queue_limits.h | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/dynamic_queue_limits.h
> b/include/linux/dynamic_queue_limits.h
> index 5621547d631b..a4be70398ce1 100644
> --- a/include/linux/dynamic_queue_limits.h
> +++ b/include/linux/dynamic_queue_limits.h
> @@ -73,14 +73,22 @@ static inline void dql_queued(struct dql *dql,
> unsigned int count)
> {
> BUG_ON(count > DQL_MAX_OBJECT);
>
> - dql->num_queued += count;
> dql->last_obj_cnt = count;
> +
> + /* We want to force a write first, so that cpu do not attempt
> + * to get cache line containing last_obj_cnt, num_queued,
> adj_limit
> + * in Shared state, but directly does a Request For Ownership
> + * It is only a hint, we use barrier() only.
> + */
> + barrier();
> +
> + dql->num_queued += count;
> }
I thought that prefetchw() would be the canonical way to solve write
stalls in CPUs and prepare the specific cache line to be written to.
Interesting, thanks Eric.
Thanks,
Hannes
^ permalink raw reply
* [PATCH 1/6 -stable] ipvs: avoid netns exit crash on ip_vs_conn_drop_conntrack
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
In-Reply-To: <1411731824-15051-1-git-send-email-pablo@netfilter.org>
From: Julian Anastasov <ja@ssi.bg>
[ upstream commit 2627b7e15c5064ddd5e578e4efd948d48d531a3f ]
commit 8f4e0a18682d91 ("IPVS netns exit causes crash in conntrack")
added second ip_vs_conn_drop_conntrack call instead of just adding
the needed check. As result, the first call still can cause
crash on netns exit. Remove it.
Cc: <stable@vger.kernel.org> # 3.14.x
Cc: <stable@vger.kernel.org> # 3.12.x
Cc: <stable@vger.kernel.org> # 3.10.x
Cc: <stable@vger.kernel.org> # 3.4.x
Cc: <stable@vger.kernel.org> # 3.2.x
Signed-off-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Hans Schillstrom <hans@schillstrom.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
net/netfilter/ipvs/ip_vs_conn.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c
index a8eb0a8..610e19c 100644
--- a/net/netfilter/ipvs/ip_vs_conn.c
+++ b/net/netfilter/ipvs/ip_vs_conn.c
@@ -797,7 +797,6 @@ static void ip_vs_conn_expire(unsigned long data)
ip_vs_control_del(cp);
if (cp->flags & IP_VS_CONN_F_NFCT) {
- ip_vs_conn_drop_conntrack(cp);
/* Do not access conntracks during subsys cleanup
* because nf_conntrack_find_get can not be used after
* conntrack cleanup for the net.
--
1.7.10.4
^ permalink raw reply related
* [PATCH 3/6 -stable] ipvs: Maintain all DSCP and ECN bits for ipv6 tun forwarding
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
In-Reply-To: <1411731824-15051-1-git-send-email-pablo@netfilter.org>
From: Alex Gartrell <agartrell@fb.com>
[ upstream commit 76f084bc10004b3050b2cff9cfac29148f1f6088 ]
Previously, only the four high bits of the tclass were maintained in the
ipv6 case. This matches the behavior of ipv4, though whether or not we
should reflect ECN bits may be up for debate.
Cc: <stable@vger.kernel.org> # 3.16.x
Cc: <stable@vger.kernel.org> # 3.14.x
Cc: <stable@vger.kernel.org> # 3.12.x
Cc: <stable@vger.kernel.org> # 3.10.x
Cc: <stable@vger.kernel.org> # 3.4.x
Signed-off-by: Alex Gartrell <agartrell@fb.com>
Acked-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
net/netfilter/ipvs/ip_vs_xmit.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c
index 73ba1cc..6f70bdd 100644
--- a/net/netfilter/ipvs/ip_vs_xmit.c
+++ b/net/netfilter/ipvs/ip_vs_xmit.c
@@ -967,8 +967,8 @@ ip_vs_tunnel_xmit_v6(struct sk_buff *skb, struct ip_vs_conn *cp,
iph->nexthdr = IPPROTO_IPV6;
iph->payload_len = old_iph->payload_len;
be16_add_cpu(&iph->payload_len, sizeof(*old_iph));
- iph->priority = old_iph->priority;
memset(&iph->flow_lbl, 0, sizeof(iph->flow_lbl));
+ ipv6_change_dsfield(iph, 0, ipv6_get_dsfield(old_iph));
iph->daddr = cp->daddr.in6;
iph->saddr = saddr;
iph->hop_limit = old_iph->hop_limit;
--
1.7.10.4
^ permalink raw reply related
* [PATCH 4/6 -stable] netfilter: x_tables: allow to use default cgroup match
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
In-Reply-To: <1411731824-15051-1-git-send-email-pablo@netfilter.org>
From: Daniel Borkmann <dborkman@redhat.com>
[ upstream commit caa8ad94edf686d02b555c65a6162c0d1b434958 ]
There's actually no good reason why we cannot use cgroup id 0,
so lets just remove this artificial barrier.
Cc: <stable@vger.kernel.org> # 3.16.x
Cc: <stable@vger.kernel.org> # 3.14.x
Reported-by: Alexey Perevalov <a.perevalov@samsung.com>
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Tested-by: Alexey Perevalov <a.perevalov@samsung.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
net/netfilter/xt_cgroup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/netfilter/xt_cgroup.c b/net/netfilter/xt_cgroup.c
index f4e8330..7198d66 100644
--- a/net/netfilter/xt_cgroup.c
+++ b/net/netfilter/xt_cgroup.c
@@ -31,7 +31,7 @@ static int cgroup_mt_check(const struct xt_mtchk_param *par)
if (info->invert & ~1)
return -EINVAL;
- return info->id ? 0 : -EINVAL;
+ return 0;
}
static bool
--
1.7.10.4
^ permalink raw reply related
* [PATCH 2/6 -stable] netfilter: xt_hashlimit: perform garbage collection from process context
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
In-Reply-To: <1411731824-15051-1-git-send-email-pablo@netfilter.org>
From: Eric Dumazet <edumazet@google.com>
[ upstream commit 7bd8490eef9776ced7632345df5133384b6be0fe ]
xt_hashlimit cannot be used with large hash tables, because garbage
collector is run from a timer. If table is really big, its possible
to hold cpu for more than 500 msec, which is unacceptable.
Switch to a work queue, and use proper scheduling points to remove
latencies spikes.
Later, we also could switch to a smoother garbage collection done
at lookup time, one bucket at a time...
Cc: <stable@vger.kernel.org> # 3.16.x
Cc: <stable@vger.kernel.org> # 3.14.x
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Florian Westphal <fw@strlen.de>
Cc: Patrick McHardy <kaber@trash.net>
Reviewed-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
net/netfilter/xt_hashlimit.c | 31 ++++++++++++++++---------------
1 file changed, 16 insertions(+), 15 deletions(-)
diff --git a/net/netfilter/xt_hashlimit.c b/net/netfilter/xt_hashlimit.c
index a3910fc..47dc683 100644
--- a/net/netfilter/xt_hashlimit.c
+++ b/net/netfilter/xt_hashlimit.c
@@ -104,7 +104,7 @@ struct xt_hashlimit_htable {
spinlock_t lock; /* lock for list_head */
u_int32_t rnd; /* random seed for hash */
unsigned int count; /* number entries in table */
- struct timer_list timer; /* timer for gc */
+ struct delayed_work gc_work;
/* seq_file stuff */
struct proc_dir_entry *pde;
@@ -213,7 +213,7 @@ dsthash_free(struct xt_hashlimit_htable *ht, struct dsthash_ent *ent)
call_rcu_bh(&ent->rcu, dsthash_free_rcu);
ht->count--;
}
-static void htable_gc(unsigned long htlong);
+static void htable_gc(struct work_struct *work);
static int htable_create(struct net *net, struct xt_hashlimit_mtinfo1 *minfo,
u_int8_t family)
@@ -273,9 +273,9 @@ static int htable_create(struct net *net, struct xt_hashlimit_mtinfo1 *minfo,
}
hinfo->net = net;
- setup_timer(&hinfo->timer, htable_gc, (unsigned long)hinfo);
- hinfo->timer.expires = jiffies + msecs_to_jiffies(hinfo->cfg.gc_interval);
- add_timer(&hinfo->timer);
+ INIT_DEFERRABLE_WORK(&hinfo->gc_work, htable_gc);
+ queue_delayed_work(system_power_efficient_wq, &hinfo->gc_work,
+ msecs_to_jiffies(hinfo->cfg.gc_interval));
hlist_add_head(&hinfo->node, &hashlimit_net->htables);
@@ -300,29 +300,30 @@ static void htable_selective_cleanup(struct xt_hashlimit_htable *ht,
{
unsigned int i;
- /* lock hash table and iterate over it */
- spin_lock_bh(&ht->lock);
for (i = 0; i < ht->cfg.size; i++) {
struct dsthash_ent *dh;
struct hlist_node *n;
+
+ spin_lock_bh(&ht->lock);
hlist_for_each_entry_safe(dh, n, &ht->hash[i], node) {
if ((*select)(ht, dh))
dsthash_free(ht, dh);
}
+ spin_unlock_bh(&ht->lock);
+ cond_resched();
}
- spin_unlock_bh(&ht->lock);
}
-/* hash table garbage collector, run by timer */
-static void htable_gc(unsigned long htlong)
+static void htable_gc(struct work_struct *work)
{
- struct xt_hashlimit_htable *ht = (struct xt_hashlimit_htable *)htlong;
+ struct xt_hashlimit_htable *ht;
+
+ ht = container_of(work, struct xt_hashlimit_htable, gc_work.work);
htable_selective_cleanup(ht, select_gc);
- /* re-add the timer accordingly */
- ht->timer.expires = jiffies + msecs_to_jiffies(ht->cfg.gc_interval);
- add_timer(&ht->timer);
+ queue_delayed_work(system_power_efficient_wq,
+ &ht->gc_work, msecs_to_jiffies(ht->cfg.gc_interval));
}
static void htable_remove_proc_entry(struct xt_hashlimit_htable *hinfo)
@@ -341,7 +342,7 @@ static void htable_remove_proc_entry(struct xt_hashlimit_htable *hinfo)
static void htable_destroy(struct xt_hashlimit_htable *hinfo)
{
- del_timer_sync(&hinfo->timer);
+ cancel_delayed_work_sync(&hinfo->gc_work);
htable_remove_proc_entry(hinfo);
htable_selective_cleanup(hinfo, select_all);
kfree(hinfo->name);
--
1.7.10.4
^ permalink raw reply related
* [PATCH 6/6 -stable] netfilter: nf_tables: don't update chain with unset counters
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
In-Reply-To: <1411731824-15051-1-git-send-email-pablo@netfilter.org>
[ upstream commit b88825de8545ad252c31543fef13cadf4de7a2bc ]
Fix possible replacement of the per-cpu chain counters by null
pointer when updating an existing chain in the commit path.
Cc: <stable@vger.kernel.org> # 3.16.x
Reported-by: Matteo Croce <technoboy85@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
net/netfilter/nf_tables_api.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index c6f9d3d..9aa31f1 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -899,6 +899,9 @@ static struct nft_stats __percpu *nft_stats_alloc(const struct nlattr *attr)
static void nft_chain_stats_replace(struct nft_base_chain *chain,
struct nft_stats __percpu *newstats)
{
+ if (newstats == NULL)
+ return;
+
if (chain->stats) {
struct nft_stats __percpu *oldstats =
nft_dereference(chain->stats);
--
1.7.10.4
^ permalink raw reply related
* [PATCH 0/6 -stable] netfilter/ipvs patches
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
Hi David,
This patchset for -stable contains:
1) Fix a crash in IPVS when exitting from netns. This applies cleanly
starting 3.2.
2) Perform garbage collection from process context in hashlimit,
otherwise this match can hold the cpu for 500ms with large tables.
Patch applies to 3.14 and 3.16.
3) Maintain all DSCP and ECN bits for ipv6 tun forwarding in ipvs.
Patch applies starting 3.4.
4) The cgroup match rejects the id 0 for no good reason and Samsumg
people needs this to get cgroup match working in previous kernels.
This one-liner applies starting 3.14.
5) Fix wrong family in hook registration in ipvs, according to my
scripts, this patch applies cleanly starting 3.10.
6) Fix crash in nf_tables when updating unset chain counters.
Patch applies to 3.16 only.
I'm sending this batch for -stable submission to netdev in the way I
interpreted from Documentation/networking/netdev-FAQ.txt.
Every patch contains the upstream commit and the tag that indicates
the kernel version as specified by the tag:
Cc: <stable@vger.kernel.org> # x.y.x
Which is the tag I've been using in -stable submissions so far.
Let me know if the way I made is fine with you. Alternatively, I can
make upcoming submissions for -stable in a better way. I can also submit
netfilter/ipvs patches myself, including you, the netdev and the stable
lists in the CC, so you can ack this if you consider they are OK and
people can follow track of what is submitted to -stable from my side.
Thanks!
^ permalink raw reply
* [PATCH 5/6 -stable] ipvs: fix ipv6 hook registration for local replies
From: Pablo Neira Ayuso @ 2014-09-26 11:43 UTC (permalink / raw)
To: netfilter-devel; +Cc: davem, netdev
In-Reply-To: <1411731824-15051-1-git-send-email-pablo@netfilter.org>
From: Julian Anastasov <ja@ssi.bg>
[ upstream commit eb90b0c734ad793d5f5bf230a9e9a4dcc48df8aa ]
commit fc604767613b6d2036cdc35b660bc39451040a47
("ipvs: changes for local real server") from 2.6.37
introduced DNAT support to local real server but the
IPv6 LOCAL_OUT handler ip_vs_local_reply6() is
registered incorrectly as IPv4 hook causing any outgoing
IPv4 traffic to be dropped depending on the IP header values.
Chris tracked down the problem to CONFIG_IP_VS_IPV6=y
Bug report: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1349768
Cc: <stable@vger.kernel.org> # 3.16.x
Cc: <stable@vger.kernel.org> # 3.14.x
Cc: <stable@vger.kernel.org> # 3.12.x
Cc: <stable@vger.kernel.org> # 3.10.x
Reported-by: Chris J Arges <chris.j.arges@canonical.com>
Tested-by: Chris J Arges <chris.j.arges@canonical.com>
Signed-off-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
net/netfilter/ipvs/ip_vs_core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c
index e683675..5c34e8d 100644
--- a/net/netfilter/ipvs/ip_vs_core.c
+++ b/net/netfilter/ipvs/ip_vs_core.c
@@ -1906,7 +1906,7 @@ static struct nf_hook_ops ip_vs_ops[] __read_mostly = {
{
.hook = ip_vs_local_reply6,
.owner = THIS_MODULE,
- .pf = NFPROTO_IPV4,
+ .pf = NFPROTO_IPV6,
.hooknum = NF_INET_LOCAL_OUT,
.priority = NF_IP6_PRI_NAT_DST + 1,
},
--
1.7.10.4
^ permalink raw reply related
* Re: [PATCH net-next] dql: dql_queued() should write first to reduce bus transactions
From: Eric Dumazet @ 2014-09-26 12:02 UTC (permalink / raw)
To: Hannes Frederic Sowa
Cc: Tom Herbert, Jesper Dangaard Brouer, Linux Netdev List,
David S. Miller, Alexander Duyck,
Toke Høiland-Jørgensen, Florian Westphal,
Jamal Hadi Salim, Dave Taht, John Fastabend, Daniel Borkmann
In-Reply-To: <1411729595.1776139.172032293.6FCC3EBB@webmail.messagingengine.com>
On Fri, 2014-09-26 at 13:06 +0200, Hannes Frederic Sowa wrote:
> I thought that prefetchw() would be the canonical way to solve write
> stalls in CPUs and prepare the specific cache line to be written to.
> Interesting, thanks Eric.
True, and I have a tcp_ack() (tcp_clean_rtx_queue() more exactly) patch
that indeed uses prefetchw() to bring tcp_shinfo() in cache before the
tcp_unlink_write_queue(skb, sk);
Performance went from 570000 packets per second to 820000 packets per
second on a TSO=off GSO=off workload.
But this applies after the https://patchwork.ozlabs.org/patch/392877/
patch (tcp: change tcp_skb_pcount() location )
^ permalink raw reply
* Re: [PATCH] net: stmmac: fix stmmac_pci_probe failed when CONFIG_HAVE_CLK is selected
From: Giuseppe CAVALLARO @ 2014-09-26 12:05 UTC (permalink / raw)
To: Kweh, Hock Leong, David Miller
Cc: rayagond@vayavyalabs.com, vbridgers2013@gmail.com, wens@csie.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
Ong, Boon Leong, tobias.johannes.klausmann@mni.thm.de
In-Reply-To: <F54AEECA5E2B9541821D670476DAE19C2B7EB35E@PGSMSX102.gar.corp.intel.com>
On 9/24/2014 12:48 PM, Kweh, Hock Leong wrote:
>> -----Original Message-----
>> From: Giuseppe CAVALLARO [mailto:peppe.cavallaro@st.com]
>> Sent: Wednesday, September 24, 2014 2:10 PM
>>>
>>> Hi peppe,
>>>
>>> Appreciate for the explanation. Just to clarify that I am not asking not to
>> pass in the priv->stmmac_clk.
>>> In fact, the fix will fail at case 2 if driver cannot obtain the priv->stmmac_clk,
>> but just not the case 1.
>>> For case 1, seem like it does not require the stmmac_clk then I think
>>> it should be OK not to fail it when driver did not get stmmac_clk but have
>> the clk_csr set.
>>
>> ok we can do that but this clock is also managed when the iface is down.
>> Maybe it could be convenient to manage it for power consumption.
>> What do you think?
>
> Hi peppe,
>
> I don't really get what you mean here. Are you telling that you are OK with the fix?
> Or you are referring to the bottom idea which introduce clock registration APIs to
> stmmac_pci driver?
I just meant that if no clock is passed then the probe doesn't fail and
we can keep a dev_warn. Pls surround the code with a comment and repost
the patch. Let me know and sorry for the delay on replying.
peppe
>
> Regarding the power management, isn't this taking care by the PCI framework itself
> for the PCI devices / PCI cards?
>
> Sorry, may be would need you to provide a big picture to this. Thanks. :)
>
>>
>>> Anyway, I can change the fix by adding the clock registration APIs
>>> being call at the stmmac_pci.c probe there before calling
>>> stmmac_dvr_probe. By doing this, it created a dependency to the pci
>>> driver that must have CONFIG_HAVE_CLK to be turned on. Besides, I
>> would need you guys to provide me information on other platforms about
>> what is the best value to set? Can I just set to zero since the stmmac_pci
>> driver is always using the priv->plat->clk_csr?
^ permalink raw reply
* RE: [PATCH] net: stmmac: fix stmmac_pci_probe failed when CONFIG_HAVE_CLK is selected
From: Kweh, Hock Leong @ 2014-09-26 12:12 UTC (permalink / raw)
To: Giuseppe CAVALLARO, David Miller
Cc: rayagond@vayavyalabs.com, vbridgers2013@gmail.com, wens@csie.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
Ong, Boon Leong, tobias.johannes.klausmann@mni.thm.de
In-Reply-To: <54255692.8020402@st.com>
> -----Original Message-----
> From: Giuseppe CAVALLARO [mailto:peppe.cavallaro@st.com]
> Sent: Friday, September 26, 2014 8:06 PM
> > > From: Giuseppe CAVALLARO [mailto:peppe.cavallaro@st.com]
> > > Sent: Wednesday, September 24, 2014 2:10 PM
>
> I just meant that if no clock is passed then the probe doesn't fail and we can
> keep a dev_warn. Pls surround the code with a comment and repost the
> patch. Let me know and sorry for the delay on replying.
>
> peppe
>
Now I got you. Okay, will do that and send out the V2 patch. Thanks.
^ permalink raw reply
* Re: [PATCH net-next 2/3] ipv6: mld: do not overwrite uri when receiving an mldv2 query
From: David L Stevens @ 2014-09-26 12:13 UTC (permalink / raw)
To: Daniel Borkmann; +Cc: davem, hannes, netdev
In-Reply-To: <542531FA.2070701@redhat.com>
On 09/26/2014 05:29 AM, Daniel Borkmann wrote:
> On 09/26/2014 01:23 AM, David L Stevens wrote:
> I'm fine with either suggestion. Actually the _current_ situation we're in is
> that in IPv4 we _always_ use the current, uncached _sysctl_ tuned setting of URI
> (independent of any protocol version); while in IPv6 we use the _cached_ sysctl
> URI in case of MLDv1 and _always_ overwrite the URI in case of MLDv2 (even for
> MLDv1). Are you suggesting that, we then better adapt using the maxdelay value
> everywhere and adapt URI to it, plus having a boolean knob defaulting to off for
> an admin to enforce always using the provided sysctl default setting and not
> the snooped MLD?
Yes.
Definitely, IGMP and MLD, all versions, should do the same thing and I think that
ought to use the querier MRC, if present and not overridden by an admin.
Further, I think a version switch or failure to hear from a querier for qrv*qi
ought to reset everything.
+-DLS
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox