[RFC: add openvswitch actions using BPF 0/9]

[RFC: add openvswitch actions using BPF 0/9]

[Andy Zhou]

User space changes of BPF OVS action agains curret OVS master.

Andy Zhou (9):
  hack: Do not compile datapath
  odp: add a new ODP action: OVS_ACTION_ATTR_BPF_PROG
  tests: add a OVS_ACTION_ATTR_BPF_PROG ation unit test case
  autoconf: support -with-llc options
  bpf: add the first BPF program.
  lib: import into libbpf to ovs/lib
  ofproto-dpif: Add eBPF program loader and runtime infrasturcure.
  ofproto-dpif: Add datapath eBPF support detection
  ofproto-dpif-xlate: generate BPF output action (Hack)

 INSTALL.BPF.md                                    |  42 ++
 Makefile.am                                       |   5 +-
 acinclude.m4                                      |  18 +-
 bpf/automake.mk                                   |  27 ++
 bpf/bpf-shared.h                                  |  12 +
 bpf/ovs-actions.c                                 |  13 +
 bpf/ovs-bpf-helpers.h                             |  35 ++
 configure.ac                                      |   2 +
 datapath/linux/compat/include/linux/openvswitch.h |  29 +-
 lib/automake.mk                                   |  22 ++
 lib/dpif-netdev.c                                 |   1 +
 lib/dpif.c                                        |   1 +
 lib/libbpf.c                                      | 100 +++++
 lib/libbpf.h                                      | 185 +++++++++
 lib/odp-execute.c                                 |   1 +
 lib/odp-util.c                                    |  33 ++
 ofproto/automake.mk                               |   2 +
 ofproto/ofproto-dpif-bpf.c                        | 454 ++++++++++++++++++++++
 ofproto/ofproto-dpif-bpf.h                        |  42 ++
 ofproto/ofproto-dpif-xlate.c                      |  18 +-
 ofproto/ofproto-dpif.c                            |  21 +
 tests/odp.at                                      |   1 +
 vswitchd/automake.mk                              |   5 +
 23 files changed, 1064 insertions(+), 5 deletions(-)
 create mode 100644 INSTALL.BPF.md
 create mode 100644 bpf/automake.mk
 create mode 100644 bpf/bpf-shared.h
 create mode 100644 bpf/ovs-actions.c
 create mode 100644 bpf/ovs-bpf-helpers.h
 create mode 100644 lib/libbpf.c
 create mode 100644 lib/libbpf.h
 create mode 100644 ofproto/ofproto-dpif-bpf.c
 create mode 100644 ofproto/ofproto-dpif-bpf.h

-- 
1.9.1

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

[RFC: add openvswitch actions using BPF 1/9] hack: Do not compile datapath

[Andy Zhou]

The OVS eBPF patch sets are developed against master 'net-next' tree,
which is currently targeting for kernel version 3.19.

The datapath in OVS mater does not currently support 3.19 kernel.
But we should not really using OVS datapath, The changes in the
follow patches are only changes in user space programs, which should
work with the kernel module built from the 'net-next' tree.

This patch disables building of the datapath and disables the
configuration time kernel version check.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 Makefile.am  | 3 ++-
 acinclude.m4 | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index 3e5e0b2..1363abe 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -7,7 +7,8 @@
 
 AUTOMAKE_OPTIONS = foreign subdir-objects
 ACLOCAL_AMFLAGS = -I m4
-SUBDIRS = datapath
+# SUBDIRS = datapath
+SUBDIRS =
 
 AM_CPPFLAGS = $(SSL_CFLAGS)
 AM_LDFLAGS = $(SSL_LDFLAGS)
diff --git a/acinclude.m4 b/acinclude.m4
index cb3e148..f419056 100644
--- a/acinclude.m4
+++ b/acinclude.m4
@@ -134,7 +134,7 @@ AC_DEFUN([OVS_CHECK_LINUX], [
     AC_MSG_RESULT([$kversion])
 
     if test "$version" -ge 3; then
-       if test "$version" = 3 && test "$patchlevel" -le 18; then
+       if test "$version" = 3 && test "$patchlevel" -le 19; then
           : # Linux 3.x
        else
          AC_ERROR([Linux kernel in $KBUILD is version $kversion, but version newer than 3.18.x is not supported (please refer to the FAQ for advice)])
-- 
1.9.1

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

[RFC: add openvswitch actions using BPF 2/9] odp: add a new ODP action: OVS_ACTION_ATTR_BPF_PROG

[Andy Zhou]

Add a new ODP action and implements the corresponding Netlink message
handling of this new attribute.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 datapath/linux/compat/include/linux/openvswitch.h | 29 +++++++++++++++++++-
 lib/dpif-netdev.c                                 |  1 +
 lib/dpif.c                                        |  1 +
 lib/odp-execute.c                                 |  1 +
 lib/odp-util.c                                    | 33 +++++++++++++++++++++++
 5 files changed, 64 insertions(+), 1 deletion(-)

diff --git a/datapath/linux/compat/include/linux/openvswitch.h b/datapath/linux/compat/include/linux/openvswitch.h
index a59e109..7284023 100644
--- a/datapath/linux/compat/include/linux/openvswitch.h
+++ b/datapath/linux/compat/include/linux/openvswitch.h
@@ -569,6 +569,18 @@ struct ovs_action_push_mpls {
 };
 
 /**
+ * struct ovs_action_bpf_prog - %OVS_ACTION_ATTR_BPF_PROG action argument.
+ *
+ *  XXX provides bpf program id and execution context.
+ *
+ */
+struct ovs_action_bpf_prog {
+	__be32 prog_fd;
+	__be32 arg0;
+	__be32 arg1;
+};
+
+/**
  * struct ovs_action_push_vlan - %OVS_ACTION_ATTR_PUSH_VLAN action argument.
  * @vlan_tpid: Tag protocol identifier (TPID) to push.
  * @vlan_tci: Tag control identifier (TCI) to push.  The CFI bit must be set
@@ -685,7 +697,7 @@ enum ovs_action_attr {
 				       * data immediately followed by a mask.
 				       * The data must be zero for the unmasked
 				       * bits. */
-
+	OVS_ACTION_ATTR_BPF_PROG,     /* struct ovs_action_bpf_prog. */
 #ifndef __KERNEL__
 	OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
 	OVS_ACTION_ATTR_TUNNEL_POP,    /* u32 port number. */
@@ -695,4 +707,19 @@ enum ovs_action_attr {
 
 #define OVS_ACTION_ATTR_MAX (__OVS_ACTION_ATTR_MAX - 1)
 
+/* integer value in 'imm' field of BPF_CALL instruction selects which OVS helper
+ * function eBPF program intends to call
+ */
+enum ovs_bpf_func_id {
+	OVS_BPF_FUNC_unspec,
+	OVS_BPF_FUNC_output,          /* int ovs_bpf_output(ctxt) */
+	__OVS_BPF_FUNC_MAX_ID,
+};
+
+struct ovs_bpf_action_ctxt {
+	void *skb;
+	uint32_t arg0;
+	uint32_t arg1;
+};
+
 #endif /* _LINUX_OPENVSWITCH_H */
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 54bad02..e991716 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -3240,6 +3240,7 @@ dp_execute_cb(void *aux_, struct dpif_packet **packets, int cnt,
     case OVS_ACTION_ATTR_SAMPLE:
     case OVS_ACTION_ATTR_HASH:
     case OVS_ACTION_ATTR_UNSPEC:
+    case OVS_ACTION_ATTR_BPF_PROG:
     case __OVS_ACTION_ATTR_MAX:
         OVS_NOT_REACHED();
     }
diff --git a/lib/dpif.c b/lib/dpif.c
index 6ecd1d9..8aac7dd 100644
--- a/lib/dpif.c
+++ b/lib/dpif.c
@@ -1132,6 +1132,7 @@ dpif_execute_helper_cb(void *aux_, struct dpif_packet **packets, int cnt,
     case OVS_ACTION_ATTR_SET_MASKED:
     case OVS_ACTION_ATTR_SAMPLE:
     case OVS_ACTION_ATTR_UNSPEC:
+    case OVS_ACTION_ATTR_BPF_PROG:
     case __OVS_ACTION_ATTR_MAX:
         OVS_NOT_REACHED();
     }
diff --git a/lib/odp-execute.c b/lib/odp-execute.c
index 98ac18c..ee06362 100644
--- a/lib/odp-execute.c
+++ b/lib/odp-execute.c
@@ -591,6 +591,7 @@ odp_execute_actions(void *dp, struct dpif_packet **packets, int cnt, bool steal,
             break;
 
         case OVS_ACTION_ATTR_UNSPEC:
+        case OVS_ACTION_ATTR_BPF_PROG:
         case __OVS_ACTION_ATTR_MAX:
             OVS_NOT_REACHED();
         }
diff --git a/lib/odp-util.c b/lib/odp-util.c
index 37d73a9..1477d13 100644
--- a/lib/odp-util.c
+++ b/lib/odp-util.c
@@ -88,6 +88,7 @@ odp_action_len(uint16_t type)
     case OVS_ACTION_ATTR_SET: return -2;
     case OVS_ACTION_ATTR_SET_MASKED: return -2;
     case OVS_ACTION_ATTR_SAMPLE: return -2;
+    case OVS_ACTION_ATTR_BPF_PROG: return sizeof(struct ovs_action_bpf_prog);
 
     case OVS_ACTION_ATTR_UNSPEC:
     case __OVS_ACTION_ATTR_MAX:
@@ -185,6 +186,17 @@ format_odp_sample_action(struct ds *ds, const struct nlattr *attr)
     ds_put_format(ds, "))");
 }
 
+static void
+format_bpf_prog_action(struct ds *ds, const struct nlattr *a)
+{
+    struct ovs_action_bpf_prog *bpf_act = (struct ovs_action_bpf_prog *)
+                                          nl_attr_get(a);
+
+    ds_put_cstr(ds, "bpf");
+    ds_put_format(ds, "(%u,%u,%u)", ntohl(bpf_act->prog_fd),
+                  ntohl(bpf_act->arg0), ntohl(bpf_act->arg1));
+}
+
 static const char *
 slow_path_reason_to_string(uint32_t reason)
 {
@@ -682,6 +694,9 @@ format_odp_action(struct ds *ds, const struct nlattr *a)
     case OVS_ACTION_ATTR_SAMPLE:
         format_odp_sample_action(ds, a);
         break;
+    case OVS_ACTION_ATTR_BPF_PROG:
+        format_bpf_prog_action(ds, a);
+        break;
     case OVS_ACTION_ATTR_UNSPEC:
     case __OVS_ACTION_ATTR_MAX:
     default:
@@ -1006,6 +1021,24 @@ parse_odp_action(const char *s, const struct simap *port_names,
         return parse_odp_userspace_action(s, actions);
     }
 
+    {
+        struct ovs_action_bpf_prog act_bpf;
+        uint32_t fd;
+        uint32_t arg0, arg1;
+        int n = -1;
+
+        if (ovs_scan(s, "bpf(%"PRIu32",%"PRIu32",%"PRIu32")%n", &fd, &arg0,
+                     &arg1, &n)) {
+            memset(&act_bpf, 0, sizeof(act_bpf));
+            act_bpf.prog_fd = htonl(fd);
+            act_bpf.arg0 = htonl(arg0);
+            act_bpf.arg1 = htonl(arg1);
+            nl_msg_put_unspec(actions, OVS_ACTION_ATTR_BPF_PROG,
+                              &act_bpf, sizeof(act_bpf));
+            return n;
+        }
+    }
+
     if (!strncmp(s, "set(", 4)) {
         size_t start_ofs;
         int retval;
-- 
1.9.1

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

[RFC: add openvswitch actions using BPF 3/9] tests: add a OVS_ACTION_ATTR_BPF_PROG ation unit test case

[Andy Zhou]

Test parsing and formating of the newly added action.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 tests/odp.at | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/odp.at b/tests/odp.at
index 8f96c6a..29a977f 100644
--- a/tests/odp.at
+++ b/tests/odp.at
@@ -282,6 +282,7 @@ tnl_pop(4)
 tnl_push(tnl_port(4),header(size=42,type=3,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=47,tos=0,ttl=64,frag=0x40),gre((flags=0x20,proto=0x6558),key=0x1e241)),out_port(1))
 tnl_push(tnl_port(4),header(size=46,type=3,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=47,tos=0,ttl=64,frag=0x40),gre((flags=0xa0,proto=0x6558),csum=0x0,key=0x1e241)),out_port(1))
 tnl_push(tnl_port(6),header(size=50,type=4,eth(dst=f8:bc:12:44:34:b6,src=f8:bc:12:46:58:e0,dl_type=0x0800),ipv4(src=1.1.2.88,dst=1.1.2.92,proto=17,tos=0,ttl=64,frag=0x40),udp(src=0,dst=4789),vxlan(flags=0x8000000,vni=0x1c700)),out_port(1))
+bpf(1,1,1)
 ])
 AT_CHECK_UNQUOTED([ovstest test-odp parse-actions < actions.txt], [0],
   [`cat actions.txt`
-- 
1.9.1

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

[RFC: add openvswitch actions using BPF 4/9] autoconf: support -with-llc options

[Andy Zhou]

Add a configuration option to allow specifing the LLVM backend module
for compiling BPF C source code into BPF instruction sets.

See INSTALL.BPF.md for usgae information.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 INSTALL.BPF.md | 42 ++++++++++++++++++++++++++++++++++++++++++
 Makefile.am    |  1 +
 acinclude.m4   | 16 ++++++++++++++++
 configure.ac   |  2 ++
 4 files changed, 61 insertions(+)
 create mode 100644 INSTALL.BPF.md

diff --git a/INSTALL.BPF.md b/INSTALL.BPF.md
new file mode 100644
index 0000000..bc1f5ad
--- /dev/null
+++ b/INSTALL.BPF.md
@@ -0,0 +1,42 @@
+# Dependencies
+
+- Need Clang + llvm-dev version 3.X for any (2 <= X <= 4)
+- http://llvm.org/apt/
+
+apt-get install libelf-dev clang-3.4 llvm-3.4-dev
+
+# Build LLVM plugin
+
+*[Upstream] git://git.kernel.org/pub/scm/linux/kernel/git/ast/bpf*
+
+git clone git://github.com/joestringer/linux
+NN=$PWD/linux
+
+cd $NN/tools/bpf/llvm/bld
+make LLVM_CONFIG=`which llvm-config-3.4`
+
+# Configure OVS to use BPF LLC
+
+OVS=/path/to/openvswitch
+cd $OVS
+./configure --with-llc=$NN/tools/bpf/llvm/bld/Debug+Asserts/bin/llc
+
+# Build OVS and BPF module
+
+make
+cd datapath/bpf
+make
+
+# Load BPF module
+
+cd $NN/
+make M=samples/bpf
+samples/bpf/simple_load $OVS/datapath/bpf/simple.bpf
+
+Note down which fd was used for the program. This will be used in next step.
+
+# Install flow to use BPF
+
+$OVS/utilities/ovs-dpctl add-flow "eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00)" "bpf(<FD>, 0)"
+
+Success?
diff --git a/Makefile.am b/Makefile.am
index 1363abe..ee8fe03 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -71,6 +71,7 @@ docs = \
 	DESIGN.md \
 	FAQ.md \
 	INSTALL.md \
+	INSTALL.BPF.md \
 	INSTALL.Debian.md \
 	INSTALL.Docker.md \
 	INSTALL.DPDK.md \
diff --git a/acinclude.m4 b/acinclude.m4
index f419056..e3b5356 100644
--- a/acinclude.m4
+++ b/acinclude.m4
@@ -157,6 +157,22 @@ AC_DEFUN([OVS_CHECK_LINUX], [
   AM_CONDITIONAL(LINUX_ENABLED, test -n "$KBUILD")
 ])
 
+dnl OVS_CHECK_BPF
+dnl
+dnl Check clang set ups for compiling BPF
+AC_DEFUN([OVS_CHECK_LLC], [
+  AC_ARG_WITH([llc],
+              [AC_HELP_STRING([--with-llc=/path/to/llc],
+                              [Specify the bpf clang backend binrary])])
+
+  if (test X"$with_llc" != X); then
+    LLC=$with_llc
+      if (test ! -x $LLC); then
+         AC_MSG_ERROR([BPF llc backend is configured but not found ])
+      fi
+  fi
+])
+
 dnl OVS_CHECK_DPDK
 dnl
 dnl Configure DPDK source tree
diff --git a/configure.ac b/configure.ac
index d2d02ca..7510d56 100644
--- a/configure.ac
+++ b/configure.ac
@@ -160,8 +160,10 @@ OVS_ENABLE_SPARSE
 
 AC_ARG_VAR(KARCH, [Kernel Architecture String])
 AC_SUBST(KARCH)
+AC_ARG_VAR(LLC, [LLVM bpf backend processor])
 OVS_CHECK_LINUX
 OVS_CHECK_DPDK
+OVS_CHECK_LLC
 OVS_CHECK_PRAGMA_MESSAGE
 AC_SUBST([OVS_CFLAGS])
 AC_SUBST([OVS_LDFLAGS])
-- 
1.9.1

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

[RFC: add openvswitch actions using BPF 5/9] bpf: add the first BPF program.

[Andy Zhou]

Added a BPF implementation of output action. The intention is to have
all OVS related BPF programs live under the 'bpf' folder.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 Makefile.am           |  1 +
 bpf/automake.mk       | 27 +++++++++++++++++++++++++++
 bpf/bpf-shared.h      | 12 ++++++++++++
 bpf/ovs-actions.c     | 13 +++++++++++++
 bpf/ovs-bpf-helpers.h | 35 +++++++++++++++++++++++++++++++++++
 5 files changed, 88 insertions(+)
 create mode 100644 bpf/automake.mk
 create mode 100644 bpf/bpf-shared.h
 create mode 100644 bpf/ovs-actions.c
 create mode 100644 bpf/ovs-bpf-helpers.h

diff --git a/Makefile.am b/Makefile.am
index ee8fe03..dce9056 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -354,6 +354,7 @@ dist-docs:
 
 include m4/automake.mk
 include lib/automake.mk
+include bpf/automake.mk
 include ofproto/automake.mk
 include utilities/automake.mk
 include tests/automake.mk
diff --git a/bpf/automake.mk b/bpf/automake.mk
new file mode 100644
index 0000000..20fc776
--- /dev/null
+++ b/bpf/automake.mk
@@ -0,0 +1,27 @@
+# Copyright (C) 2015 Nicira, Inc.
+#
+# Copying and distribution of this file, with or without modification,
+# are permitted in any medium without royalty provided the copyright
+# notice and this notice are preserved.  This file is offered as-is,
+# without warranty of any kind.
+
+if LINUX
+sbin_PROGRAMS += bpf/ovs-actions.bpf
+
+
+EXTRA_DIST += $(srcdir)/bpf/ovs-bpf-helpers.h \
+	      $(srcdir)/bpf/bpf-shared.h \
+	      $(srcdir)/bpf/ovs-actions.c
+
+DEP_FILES  = $(srcdir)/bpf/ovs-bpf-helpers.h \
+	     $(srcdir)/bpf/bpf-shared.h \
+             $(srcdir)/datapath/linux/compat/include/linux/openvswitch.h
+
+BPF_INCLUDES=-I. -I$(srcdir)/datapath/linux/compat/include -I/usr/include
+
+bpf/ovs-actions.bpf: $(srcdir)/bpf/ovs-actions.c $(DEP_FILES)
+	$(AM_V_GEN)clang -DHAVE_CONFIG_H $(BPF_INCLUDES) $(NOSTDINC_FLAGS) \
+		$(AM_CFLAGS) $(EXTRA_CFLAGS) -Wno-unused-value -Wno-pointer-sign \
+		-O2 -emit-llvm -c $< -o -| $(LLC) -filetype=obj -o $@
+
+endif
diff --git a/bpf/bpf-shared.h b/bpf/bpf-shared.h
new file mode 100644
index 0000000..a710c3c
--- /dev/null
+++ b/bpf/bpf-shared.h
@@ -0,0 +1,12 @@
+
+/* Shared data structures between bpf and C programs. */
+
+/* a helper structure used by eBPF C program
+ * to describe map attributes to elf_bpf loader
+ */
+struct bpf_map_def {
+	unsigned int type;
+	unsigned int key_size;
+	unsigned int value_size;
+	unsigned int max_entries;
+};
diff --git a/bpf/ovs-actions.c b/bpf/ovs-actions.c
new file mode 100644
index 0000000..05a0a71
--- /dev/null
+++ b/bpf/ovs-actions.c
@@ -0,0 +1,13 @@
+#include <config.h>
+#include "ovs-bpf-helpers.h"
+
+int output_action(struct ovs_bpf_action_ctxt *ctxt);
+
+SEC("ovs/output")
+int
+output_action(struct ovs_bpf_action_ctxt *ctxt)
+{
+	return ovs_bpf_helper_output(ctxt->skb, ctxt->arg0);
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/bpf/ovs-bpf-helpers.h b/bpf/ovs-bpf-helpers.h
new file mode 100644
index 0000000..ba10084
--- /dev/null
+++ b/bpf/ovs-bpf-helpers.h
@@ -0,0 +1,35 @@
+#ifndef __OVS_BPF_HELPERS_H
+#define __OVS_BPF_HELPERS_H
+
+#include <stdint.h>
+#include <linux/openvswitch.h>
+struct sk_buff;
+
+/* helper macro to place programs, maps, license in
+ * different sections in elf_bpf file. Section names
+ * are interpreted by elf_bpf loader
+ */
+#define SEC(NAME) __attribute__((section(NAME), used))
+
+/* helper functions called from eBPF programs written in C */
+static void *(*bpf_map_lookup_elem)(void *map, void *key) =
+	(void *) OVS_BPF_FUNC_map_lookup_elem;
+static int (*bpf_map_update_elem)(void *map, void *key, void *value,
+				  unsigned long long flags) =
+	(void *) OVS_BPF_FUNC_map_update_elem;
+static int (*bpf_map_delete_elem)(void *map, void *key) =
+	(void *) OVS_BPF_FUNC_map_delete_elem;
+static int (*ovs_bpf_helper_output)(struct sk_buff *skb, uint32_t out_port) =
+	(void *) OVS_BPF_FUNC_output;
+
+/* llvm builtin functions that eBPF C program may use to
+ * emit BPF_LD_ABS and BPF_LD_IND instructions
+ */
+unsigned long long load_byte(void *skb,
+			     unsigned long long off) asm("llvm.bpf.load.byte");
+unsigned long long load_half(void *skb,
+			     unsigned long long off) asm("llvm.bpf.load.half");
+unsigned long long load_word(void *skb,
+			     unsigned long long off) asm("llvm.bpf.load.word");
+
+#endif
-- 
1.9.1

_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

[RFC: add openvswitch actions using BPF 6/9] lib: import into libbpf to ovs/lib

[Andy Zhou]

Those files are basically imported from 'net-next: sample/bpf'
directory.  This is a stop gap measure unil libbpf is upstreamed.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 lib/automake.mk |  22 +++++++
 lib/libbpf.c    | 100 ++++++++++++++++++++++++++++++
 lib/libbpf.h    | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 307 insertions(+)
 create mode 100644 lib/libbpf.c
 create mode 100644 lib/libbpf.h

diff --git a/lib/automake.mk b/lib/automake.mk
index 2a5844b..d9488c0 100644
--- a/lib/automake.mk
+++ b/lib/automake.mk
@@ -317,6 +317,28 @@ lib_libsflow_la_CFLAGS += -Wno-unused-parameter
 endif
 
 if LINUX
+lib_LTLIBRARIES += lib/libbpf.la
+lib_libbpf_la_LDFLAGS = \
+        -version-info $(LT_CURRENT):$(LT_REVISION):$(LT_AGE) \
+	-Wl,--version-script=$(top_builddir)/lib/libbpf.sym \
+        $(AM_LDFLAGS)
+lib_libbpf_la_SOURCES = \
+	lib/libbpf.h \
+	lib/libbpf.c
+lib_libbpf_la_CPPFLAGS = $(AM_CPPFLAGS)
+# GCC Bug 60784 affects both GCC 4.8 and GCC 4.9. -Wmissing-field-initializers will
+# give spurious warning when compiling libbpf. Turn it off for now.
+lib_libbpf_la_CFLAGS = $(filter-out -Wmissing-field-initializers, $(AM_CFLAGS))
+lib_libbpf_la_CFLAGS += -Wno-missing-field-initializers
+if HAVE_WNO_UNUSED
+lib_libbpf_la_CFLAGS += -Wno-unused
+endif
+if HAVE_WNO_UNUSED_PARAMETER
+lib_libbpf_la_CFLAGS += -Wno-unused-parameter
+endif
+endif
+
+if LINUX
 lib_libopenvswitch_la_SOURCES += \
 	lib/dpif-netlink.c \
 	lib/dpif-netlink.h \
diff --git a/lib/libbpf.c b/lib/libbpf.c
new file mode 100644
index 0000000..045bdba
--- /dev/null
+++ b/lib/libbpf.c
@@ -0,0 +1,100 @@
+/* eBPF mini library */
+#include <stdlib.h>
+#include <stdio.h>
+#include <linux/unistd.h>
+#include <unistd.h>
+#include <string.h>
+#include <linux/bpf.h>
+#include <errno.h>
+#include <net/ethernet.h>
+#include <net/if.h>
+#include <linux/if_packet.h>
+#include <arpa/inet.h>
+#include "libbpf.h"
+
+#include <config.h>
+
+static __u64 ptr_to_u64(void *ptr)
+{
+    return (__u64) (unsigned long) ptr;
+}
+
+int bpf_create_map(enum bpf_map_type map_type, int key_size, int value_size,
+                   int max_entries)
+{
+    union bpf_attr attr = {
+        .map_type = map_type,
+        .key_size = key_size,
+        .value_size = value_size,
+        .max_entries = max_entries,
+    };
+
+    return syscall(__NR_bpf, BPF_MAP_CREATE, &attr, sizeof(attr));
+}
+
+int bpf_update_elem(int fd, void *key, void *value, unsigned long long flags)
+{
+    union bpf_attr attr = {
+        .map_fd = fd,
+        .key = ptr_to_u64(key),
+        .value = ptr_to_u64(value),
+        .flags = flags,
+    };
+
+    return syscall(__NR_bpf, BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr));
+}
+
+int bpf_lookup_elem(int fd, void *key, void *value)
+{
+    union bpf_attr attr = {
+        .map_fd = fd,
+        .key = ptr_to_u64(key),
+        .value = ptr_to_u64(value),
+    };
+
+    return syscall(__NR_bpf, BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr));
+}
+
+int bpf_delete_elem(int fd, void *key)
+{
+    union bpf_attr attr = {
+        .map_fd = fd,
+        .key = ptr_to_u64(key),
+    };
+
+    return syscall(__NR_bpf, BPF_MAP_DELETE_ELEM, &attr, sizeof(attr));
+}
+
+int bpf_get_next_key(int fd, void *key, void *next_key)
+{
+    union bpf_attr attr = {
+        .map_fd = fd,
+        .key = ptr_to_u64(key),
+        .next_key = ptr_to_u64(next_key),
+    };
+
+    return syscall(__NR_bpf, BPF_MAP_GET_NEXT_KEY, &attr, sizeof(attr));
+}
+
+#define ROUND_UP(x, n) (((x) + (n) - 1u) & ~((n) - 1u))
+
+char bpf_log_buf[LOG_BUF_SIZE];
+
+int bpf_prog_load(enum bpf_prog_type prog_type,
+          const struct bpf_insn *insns, int prog_len,
+          const char *license)
+{
+    union bpf_attr attr = {
+        .prog_type = prog_type,
+        .insns = ptr_to_u64((void *) insns),
+        .insn_cnt = prog_len / sizeof(struct bpf_insn),
+        .license = ptr_to_u64((void *) license),
+        .log_buf = ptr_to_u64(bpf_log_buf),
+        .log_size = LOG_BUF_SIZE,
+        .log_level = 1,
+    };
+
+    bpf_log_buf[0] = 0;
+
+    return syscall(__NR_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
+}
diff --git a/lib/libbpf.h b/lib/libbpf.h
new file mode 100644
index 0000000..fe6f31e
--- /dev/null
+++ b/lib/libbpf.h
@@ -0,0 +1,185 @@
+/* eBPF mini library */
+#ifndef __LIBBPF_H
+#define __LIBBPF_H
+
+struct bpf_insn;
+
+int bpf_create_map(enum bpf_map_type map_type, int key_size, int value_size,
+           int max_entries);
+int bpf_update_elem(int fd, void *key, void *value, unsigned long long flags);
+int bpf_lookup_elem(int fd, void *key, void *value);
+int bpf_delete_elem(int fd, void *key);
+int bpf_get_next_key(int fd, void *key, void *next_key);
+
+int bpf_prog_load(enum bpf_prog_type prog_type,
+          const struct bpf_insn *insns, int insn_len,
+          const char *license);
+
+#define LOG_BUF_SIZE 65536
+extern char bpf_log_buf[LOG_BUF_SIZE];
+
+/* ALU ops on registers, bpf_add|sub|...: dst_reg += src_reg */
+
+#define BPF_ALU64_REG(OP, DST, SRC)             \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ALU64 | BPF_OP(OP) | BPF_X,    \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = 0,                 \
+        .imm   = 0 })
+
+#define BPF_ALU32_REG(OP, DST, SRC)             \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ALU | BPF_OP(OP) | BPF_X,      \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = 0,                 \
+        .imm   = 0 })
+
+/* ALU ops on immediates, bpf_add|sub|...: dst_reg += imm32 */
+
+#define BPF_ALU64_IMM(OP, DST, IMM)             \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ALU64 | BPF_OP(OP) | BPF_K,    \
+        .dst_reg = DST,                 \
+        .src_reg = 0,                   \
+        .off   = 0,                 \
+        .imm   = IMM })
+
+#define BPF_ALU32_IMM(OP, DST, IMM)             \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ALU | BPF_OP(OP) | BPF_K,      \
+        .dst_reg = DST,                 \
+        .src_reg = 0,                   \
+        .off   = 0,                 \
+        .imm   = IMM })
+
+/* Short form of mov, dst_reg = src_reg */
+
+#define BPF_MOV64_REG(DST, SRC)                 \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ALU64 | BPF_MOV | BPF_X,       \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = 0,                 \
+        .imm   = 0 })
+
+/* Short form of mov, dst_reg = imm32 */
+
+#define BPF_MOV64_IMM(DST, IMM)                 \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ALU64 | BPF_MOV | BPF_K,       \
+        .dst_reg = DST,                 \
+        .src_reg = 0,                   \
+        .off   = 0,                 \
+        .imm   = IMM })
+
+/* BPF_LD_IMM64 macro encodes single 'load 64-bit immediate' insn */
+#define BPF_LD_IMM64(DST, IMM)                  \
+    BPF_LD_IMM64_RAW(DST, 0, IMM)
+
+#define BPF_LD_IMM64_RAW(DST, SRC, IMM)             \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_LD | BPF_DW | BPF_IMM,     \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = 0,                 \
+        .imm   = (__u32) (IMM) }),          \
+    ((struct bpf_insn) {                    \
+        .code  = 0, /* zero is reserved opcode */   \
+        .dst_reg = 0,                   \
+        .src_reg = 0,                   \
+        .off   = 0,                 \
+        .imm   = ((__u64) (IMM)) >> 32 })
+
+#define BPF_PSEUDO_MAP_FD   1
+
+/* pseudo BPF_LD_IMM64 insn used to refer to process-local map_fd */
+#define BPF_LD_MAP_FD(DST, MAP_FD)              \
+    BPF_LD_IMM64_RAW(DST, BPF_PSEUDO_MAP_FD, MAP_FD)
+
+
+/* Direct packet access, R0 = *(uint *) (skb->data + imm32) */
+
+#define BPF_LD_ABS(SIZE, IMM)                   \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_LD | BPF_SIZE(SIZE) | BPF_ABS, \
+        .dst_reg = 0,                   \
+        .src_reg = 0,                   \
+        .off   = 0,                 \
+        .imm   = IMM })
+
+/* Memory load, dst_reg = *(uint *) (src_reg + off16) */
+
+#define BPF_LDX_MEM(SIZE, DST, SRC, OFF)            \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_LDX | BPF_SIZE(SIZE) | BPF_MEM,    \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = OFF,                   \
+        .imm   = 0 })
+
+/* Memory store, *(uint *) (dst_reg + off16) = src_reg */
+
+#define BPF_STX_MEM(SIZE, DST, SRC, OFF)            \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_STX | BPF_SIZE(SIZE) | BPF_MEM,    \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = OFF,                   \
+        .imm   = 0 })
+
+/* Memory store, *(uint *) (dst_reg + off16) = imm32 */
+
+#define BPF_ST_MEM(SIZE, DST, OFF, IMM)             \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_ST | BPF_SIZE(SIZE) | BPF_MEM, \
+        .dst_reg = DST,                 \
+        .src_reg = 0,                   \
+        .off   = OFF,                   \
+        .imm   = IMM })
+
+/* Conditional jumps against registers, if (dst_reg 'op' src_reg) goto pc + off16 */
+
+#define BPF_JMP_REG(OP, DST, SRC, OFF)              \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_JMP | BPF_OP(OP) | BPF_X,      \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = OFF,                   \
+        .imm   = 0 })
+
+/* Conditional jumps against immediates, if (dst_reg 'op' imm32) goto pc + off16 */
+
+#define BPF_JMP_IMM(OP, DST, IMM, OFF)              \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_JMP | BPF_OP(OP) | BPF_K,      \
+        .dst_reg = DST,                 \
+        .src_reg = 0,                   \
+        .off   = OFF,                   \
+        .imm   = IMM })
+
+/* Raw code statement block */
+
+#define BPF_RAW_INSN(CODE, DST, SRC, OFF, IMM)          \
+    ((struct bpf_insn) {                    \
+        .code  = CODE,                  \
+        .dst_reg = DST,                 \
+        .src_reg = SRC,                 \
+        .off   = OFF,                   \
+        .imm   = IMM })
+
+/* Program exit */
+
+#define BPF_EXIT_INSN()                     \
+    ((struct bpf_insn) {                    \
+        .code  = BPF_JMP | BPF_EXIT,            \
+        .dst_reg = 0,                   \
+        .src_reg = 0,                   \
+        .off   = 0,                 \
+        .imm   = 0 })
+
+/* create RAW socket and bind to interface 'name' */
+int open_raw_sock(const char *name);
+
+#endif
-- 
1.9.1

[RFC: add openvswitch actions using BPF 7/9] ofproto-dpif: Add eBPF program loader and runtime infrasturcure.

[Andy Zhou]

Added a eBPF program loader in ofproto-dpif layer. It only loades
the OVS programs.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 ofproto/automake.mk        |   2 +
 ofproto/ofproto-dpif-bpf.c | 454 +++++++++++++++++++++++++++++++++++++++++++++
 ofproto/ofproto-dpif-bpf.h |  42 +++++
 vswitchd/automake.mk       |   5 +
 4 files changed, 503 insertions(+)
 create mode 100644 ofproto/ofproto-dpif-bpf.c
 create mode 100644 ofproto/ofproto-dpif-bpf.h

diff --git a/ofproto/automake.mk b/ofproto/automake.mk
index 0058ff3..86719de 100644
--- a/ofproto/automake.mk
+++ b/ofproto/automake.mk
@@ -28,6 +28,8 @@ ofproto_libofproto_la_SOURCES = \
 	ofproto/ofproto.h \
 	ofproto/ofproto-dpif.c \
 	ofproto/ofproto-dpif.h \
+	ofproto/ofproto-dpif-bpf.c \
+	ofproto/ofproto-dpif-bpf.h \
 	ofproto/ofproto-dpif-ipfix.c \
 	ofproto/ofproto-dpif-ipfix.h \
 	ofproto/ofproto-dpif-mirror.c \
diff --git a/ofproto/ofproto-dpif-bpf.c b/ofproto/ofproto-dpif-bpf.c
new file mode 100644
index 0000000..12f6766
--- /dev/null
+++ b/ofproto/ofproto-dpif-bpf.c
@@ -0,0 +1,454 @@
+/*
+ * Copyright (c) 2015 Nicira, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <config.h>
+
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <libelf.h>
+#include <gelf.h>
+#include <errno.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdbool.h>
+#include <linux/bpf.h>
+#include <linux/filter.h>
+#include <pthread.h>
+#include "simap.h"
+#include "util.h"
+#include "dirs.h"
+#include "unixctl.h"
+#include "dynamic-string.h"
+#include "ovs-atomic.h"
+#include "openvswitch/thread.h"
+#include "openvswitch/vlog.h"
+#include "ofproto-dpif-bpf.h"
+#include "libbpf.h"
+#include "bpf/bpf-shared.h"
+
+#define DEFAULT_BPF_ACTION_FILE "ovs-actions.bpf"
+#define BPF_MAP_STUB_SIZE 32
+
+VLOG_DEFINE_THIS_MODULE(bpf);
+
+struct bpf_info {
+    struct simap bpf_maps;
+    struct simap bpf_progs;
+};
+
+static struct bpf_info ovs_bpfs__ = {
+    SIMAP_INITIALIZER(&ovs_bpfs__.bpf_maps),
+    SIMAP_INITIALIZER(&ovs_bpfs__.bpf_progs),
+};
+
+static bool bpf_enable__ = false;
+static struct ovs_mutex mutex = OVS_MUTEX_INITIALIZER;
+static struct bpf_info *const ovs_bpfs OVS_GUARDED_BY(mutex) = &ovs_bpfs__;
+
+#define BPF_UNIXCTL_CHECK(conn) \
+    if (bpf_unixctl_check_enable(conn) == -1) return
+
+/*
+ * Get the default eBPF action file name.
+ *
+ * Returns the file name. Caller is responsible for free the memory.  */
+static char *
+bpf_default_path(void)
+{
+    return xasprintf("%s/%s", ovs_pkgdatadir(), DEFAULT_BPF_ACTION_FILE);
+}
+
+static int
+bpf_unixctl_check_enable(struct unixctl_conn *conn)
+{
+    if (!bpf_enable__) {
+        unixctl_command_reply(conn, "Current datapath doest not support eBPF actions.\n");
+        return -1;
+    }
+
+    return 0;
+}
+
+static void 
+add_map(char *map_name,  int fd)
+{
+    ovs_mutex_lock(&mutex);
+    simap_put(&ovs_bpfs->bpf_maps, map_name, fd);
+    ovs_mutex_unlock(&mutex);
+}
+
+static void
+add_prog(char *prog_name, int fd)
+{
+    ovs_mutex_lock(&mutex);
+    simap_put(&ovs_bpfs->bpf_progs, prog_name, fd);
+    ovs_mutex_unlock(&mutex);
+}
+
+static int
+is_ovs_section(const char *shname) {
+   return !memcmp(shname, "ovs", 3);
+}
+
+static int
+load_maps(struct bpf_map_def *maps, int len, int map_fds[])
+{
+    int i, fd;
+
+    for (i = 0; i < len / sizeof(struct bpf_map_def); i++) {
+        char *name;
+
+        fd = bpf_create_map(maps[i].type, maps[i].key_size,
+                            maps[i].value_size, maps[i].max_entries);
+
+        if (fd < 0) {
+            return fd;
+        }
+
+        name = xasprintf("map%d(type[%u],ks[%u],vs[%u],max[%u])",
+                         i, maps[i].type, maps[i].key_size,
+                         maps[i].value_size, maps[i].max_entries);
+
+        map_fds[i] = fd;
+        add_map(name, fd);
+        free(name);
+    }
+
+    return 0;
+}
+
+static int
+load_prog(int type, struct bpf_insn *prog, int size, char *license)
+{
+    return bpf_prog_load(type, prog, size, license);
+}
+
+static int
+parse_relo_and_apply(Elf_Data *data, Elf_Data *symbols,
+                     GElf_Shdr *shdr, struct bpf_insn *insn, int map_fds[])
+{
+    int i, nrels;
+
+    nrels = shdr->sh_size / shdr->sh_entsize;
+
+    for (i = 0; i < nrels; i++) {
+        GElf_Sym sym;
+        GElf_Rel rel;
+        unsigned int insn_idx;
+
+        gelf_getrel(data, i, &rel);
+
+        insn_idx = rel.r_offset / sizeof(struct bpf_insn);
+
+        gelf_getsym(symbols, GELF_R_SYM(rel.r_info), &sym);
+
+        if (insn[insn_idx].code != (BPF_LD | BPF_IMM | BPF_DW)) {
+            VLOG_ERR("invalid relo for insn[%d].code 0x%x\n",
+                     insn_idx, insn[insn_idx].code);
+            return 1;
+        }
+        insn[insn_idx].src_reg = BPF_PSEUDO_MAP_FD;
+        insn[insn_idx].imm = map_fds[sym.st_value / sizeof(struct bpf_map_def)];
+    }
+
+    return 0;
+}
+
+static int
+get_sec(Elf *elf, int i, GElf_Ehdr *ehdr, char **shname,
+        GElf_Shdr *shdr, Elf_Data **data)
+{
+    Elf_Scn *scn;
+
+    scn = elf_getscn(elf, i);
+    if (!scn) {
+        return -1;
+    }
+
+    if (gelf_getshdr(scn, shdr) != shdr) {
+        return -2;
+    }
+
+    *shname = elf_strptr(elf, ehdr->e_shstrndx, shdr->sh_name);
+    if (!*shname || !shdr->sh_size) {
+        return -3;
+    }
+
+    *data = elf_getdata(scn, 0);
+    if (!*data || elf_getdata(scn, *data) != NULL) {
+        return -4;
+    }
+
+    return 0;
+}
+
+static int
+ofproto_dpif_load_bpf(const char *bpf_file OVS_UNUSED)
+{
+    int fd, i;
+    Elf *elf;
+    GElf_Ehdr ehdr;
+    GElf_Shdr shdr, shdr_prog;
+    Elf_Data *data, *data_prog, *symbols;
+    char *shname, *shname_prog, *license, *err;
+    int *map_fds;
+
+    err = license = NULL;
+    symbols = NULL;
+
+    if (elf_version(EV_CURRENT) == EV_NONE) {
+        err = xasprintf("Elf library out of date");
+        goto error;
+    }
+
+    fd = open(bpf_file, O_RDONLY, 0);
+    if (fd < 0) {
+        err = xasprintf("Failed to read %s", bpf_file);
+        goto error;
+    }
+
+    elf = elf_begin(fd, ELF_C_READ, NULL);
+    if (!elf) {
+        err = xasprintf("Elf file %s has internal error", bpf_file);
+        goto error;
+    }
+
+    
+    if (gelf_getehdr(elf, &ehdr) != &ehdr) {
+        err = xasprintf("Elf file %s has internal error", bpf_file);
+        goto error;
+    }
+
+    map_fds = xmalloc(sizeof(int) * ehdr.e_shnum);
+    for (i = 1;  i < ehdr.e_shnum; i++) {
+        if (get_sec(elf, i, &ehdr, &shname, &shdr, &data)) {
+            continue;
+        }
+
+        if ((strcmp(shname, "license") == 0) && (license == NULL)) {
+            license = xmalloc(data->d_size);
+            memcpy(license, data->d_buf, data->d_size);
+        } else if (strcmp(shname, "maps") == 0) {
+            if (load_maps(data->d_buf, data->d_size, map_fds)) {
+                err = xasprintf("Failed to load maps");
+                goto error;
+            }
+        } else if (shdr.sh_type == SHT_SYMTAB) {
+            symbols = data;
+        }
+    }
+
+    /* Load programs that need map fixups (relocations).  */
+    for (i = 1;  i < ehdr.e_shnum; i++) {
+        if (get_sec(elf, i, &ehdr, &shname, &shdr, &data)) {
+            continue;
+        }
+
+        if (shdr.sh_type == SHT_REL) {
+            struct bpf_insn *insns;
+
+            if (get_sec(elf, shdr.sh_info, &ehdr, &shname_prog,
+                        &shdr_prog, &data_prog)) {
+                continue;
+            }
+
+            insns = (struct bpf_insn *) data_prog->d_buf;
+            
+            if (is_ovs_section(shname_prog)) {
+                int fd;
+                if (parse_relo_and_apply(data, symbols, &shdr, insns, map_fds)) {
+                    err = xasprintf("Failed to relocate ovs programs.");
+                    goto error;
+                }
+                fd = load_prog(BPF_PROG_TYPE_OPENVSWITCH, insns,
+                               data_prog->d_size, license);
+
+                if (fd) {
+                    add_prog(shname_prog, fd);
+                } else {
+                    err = xasprintf("Failed load ovs program %s", shname_prog);
+                    goto error;
+                }
+            }
+        }
+    }
+    free(map_fds);
+    map_fds = NULL;
+
+    /* Load OVS programs that don't use maps. */
+    for (i = 1;  i < ehdr.e_shnum; i++) {
+        if (get_sec(elf, i, &ehdr, &shname, &shdr, &data)) {
+            continue;
+        }
+
+        if (is_ovs_section(shname) && !ofproto_dpif_bpf_lookup(shname)) {
+            int fd;
+
+            fd = load_prog(BPF_PROG_TYPE_OPENVSWITCH, data->d_buf,
+                           data->d_size, license);
+
+            if (fd) {
+                add_prog(shname, fd);
+            } else {
+                err = xasprintf("Failed load ovs program %s", shname);
+                goto error;
+            }
+        }
+    }
+
+error:
+    if (fd) {
+        close(fd);
+    }
+
+    if (license) {
+        free(license);
+    }
+
+    if (map_fds) {
+        free(map_fds);
+    }
+
+    if (err) {
+        VLOG_ERR("%s", err); 
+        free(err);
+        return -1;
+    }
+
+    return 0;
+}
+
+void
+ofproto_dpif_bpf_close()
+{
+    struct simap_node *node;
+
+    ovs_mutex_lock(&mutex);
+
+    SIMAP_FOR_EACH(node, &ovs_bpfs->bpf_progs) {
+        //int fd = (int)node->data;
+
+        //fclose(fd);
+        simap_delete(&ovs_bpfs->bpf_progs, node);
+    }
+
+    SIMAP_FOR_EACH(node, &ovs_bpfs->bpf_maps) {
+        //int fd = (int)node->data;
+        //fclose(fd);
+        simap_delete(&ovs_bpfs->bpf_maps, node);
+    }
+
+    ovs_mutex_unlock(&mutex);
+}
+
+int
+ofproto_dpif_bpf_lookup(const char *bpf_prog_name)
+{
+    struct simap_node *node;
+    int fd = 0;
+
+    ovs_mutex_lock(&mutex);
+    node = simap_find(&ovs_bpfs->bpf_progs, bpf_prog_name);
+    if (node) {
+        fd =(int)node->data;
+    }
+    ovs_mutex_unlock(&mutex);
+
+    return fd;
+}
+
+static void
+bpf_unixctl_show(struct unixctl_conn *conn, int argc OVS_UNUSED,
+                 const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED)
+{
+    struct ds ds = DS_EMPTY_INITIALIZER;
+    struct simap_node *node;
+    char *bpf_file;
+
+    BPF_UNIXCTL_CHECK(conn);
+
+    bpf_file = bpf_default_path();
+    ds_put_format(&ds, "eBPF file: %s \n", bpf_file);
+    free(bpf_file);
+
+    ovs_mutex_lock(&mutex);
+
+    ds_put_format(&ds, "eBPF maps:\n");
+    SIMAP_FOR_EACH(node, &ovs_bpfs->bpf_maps) {
+        ds_put_format(&ds, "\t%-40s%d\n", node->name, (int)node->data);
+    }
+
+    ds_put_format(&ds, "eBPF programs:\n");
+    SIMAP_FOR_EACH(node, &ovs_bpfs->bpf_progs) {
+        ds_put_format(&ds, "\t%-40s%d\n", node->name, (int)node->data);
+    }
+
+    ovs_mutex_unlock(&mutex);
+
+    unixctl_command_reply(conn, ds_cstr(&ds));
+    ds_destroy(&ds);
+}
+
+static void
+bpf_unixctl_clear(struct unixctl_conn *conn, int argc OVS_UNUSED,
+                 const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED)
+{
+    BPF_UNIXCTL_CHECK(conn);
+
+    ofproto_dpif_bpf_close();
+    unixctl_command_reply(conn, "OK");
+}
+
+static void
+bpf_unixctl_reload(struct unixctl_conn *conn, int argc OVS_UNUSED,
+                 const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED)
+{
+    char *bpf_fpath = NULL;
+    int err = 0;
+
+    BPF_UNIXCTL_CHECK(conn);
+
+    ofproto_dpif_bpf_close();
+
+    bpf_fpath = bpf_default_path();
+    err = ofproto_dpif_load_bpf(bpf_fpath);
+    free(bpf_fpath);
+    unixctl_command_reply(conn, err ? "Failed" : "OK");
+}
+
+int
+ofproto_dpif_bpf_init(bool enable)
+{
+    char *bpf_fpath = NULL;
+    int err = 0;
+
+    bpf_enable__ = enable;
+
+    unixctl_command_register("bpf/show",  "", 0, 0, bpf_unixctl_show, NULL);
+    unixctl_command_register("bpf/clear", "", 0, 0, bpf_unixctl_clear, NULL);
+    unixctl_command_register("bpf/reload","", 1, 1, bpf_unixctl_reload, NULL);
+
+    err = 0;
+    if (enable) {
+        bpf_fpath = bpf_default_path();
+        err = ofproto_dpif_load_bpf(bpf_fpath);
+        free(bpf_fpath);
+    }
+
+    return err;
+}
diff --git a/ofproto/ofproto-dpif-bpf.h b/ofproto/ofproto-dpif-bpf.h
new file mode 100644
index 0000000..14acb88
--- /dev/null
+++ b/ofproto/ofproto-dpif-bpf.h
@@ -0,0 +1,42 @@
+/*
+ * Copyright (c) 2015 Nicira, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at:
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef OFPROTO_DPIF_BPF_H
+#define OFPROTO_DPIF_BPF_H
+
+#ifdef __linux__
+int ofproto_dpif_bpf_init(bool enable);
+void ofproto_dpif_bpf_close(void);
+int ofproto_dpif_bpf_lookup(const char *bpf_prog_name);
+#else
+static inline int
+ofproto_dpif_bpf_init(bool enable)
+{
+    return 0;
+}
+
+static void
+ofproto_dpif_bpf_close(void)
+{
+}
+
+static int
+ofproto_dpif_bpf_lookup(const char *bpf_prog_name)
+{
+    return -2;
+}
+#endif
+
+#endif
diff --git a/vswitchd/automake.mk b/vswitchd/automake.mk
index 80affe9..a94720b 100644
--- a/vswitchd/automake.mk
+++ b/vswitchd/automake.mk
@@ -19,6 +19,11 @@ vswitchd_ovs_vswitchd_LDFLAGS = $(AM_LDFLAGS) $(DPDK_vswitchd_LDFLAGS)
 EXTRA_DIST += vswitchd/INTERNALS
 MAN_ROOTS += vswitchd/ovs-vswitchd.8.in
 
+if LINUX
+vswitchd_ovs_vswitchd_LDADD += lib/libbpf.la
+vswitchd_ovs_vswitchd_LDADD += -lelf
+endif
+
 # vswitch schema and IDL
 EXTRA_DIST += vswitchd/vswitch.ovsschema
 pkgdata_DATA += vswitchd/vswitch.ovsschema
-- 
1.9.1

[RFC: add openvswitch actions using BPF 8/9] ofproto-dpif: Add datapath eBPF support detection

[Andy Zhou]

First cut in added datpath eBPF support detector. This feature has not
been fully developed. The stub implementation simply assumes the
datpath support eBPF.

Signed-off-by: Andy Zhou <azhou@nicira.com>

----
Current interface only support 'yes' or 'no' detection. Should we
consider 'support level'?
---
 ofproto/ofproto-dpif.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
index b909fd9..4bcd034 100644
--- a/ofproto/ofproto-dpif.c
+++ b/ofproto/ofproto-dpif.c
@@ -58,6 +58,7 @@
 #include "ofproto-dpif-sflow.h"
 #include "ofproto-dpif-upcall.h"
 #include "ofproto-dpif-xlate.h"
+#include "ofproto-dpif-bpf.h"
 #include "poll-loop.h"
 #include "ovs-rcu.h"
 #include "ovs-router.h"
@@ -293,6 +294,9 @@ struct dpif_backer {
      * actions. */
     bool masked_set_action;
 
+    /* True if the datapath supports bpf actions */
+    bool enable_bpf_action;
+
     /* Maximum number of MPLS label stack entries that the datapath supports
      * in a match */
     size_t max_mpls_depth;
@@ -909,6 +913,7 @@ static bool check_variable_length_userdata(struct dpif_backer *backer);
 static size_t check_max_mpls_depth(struct dpif_backer *backer);
 static bool check_recirc(struct dpif_backer *backer);
 static bool check_ufid(struct dpif_backer *backer);
+static bool check_bpf_actions(struct dpif_backer *backer);
 static bool check_masked_set_action(struct dpif_backer *backer);
 
 static int
@@ -1007,6 +1012,7 @@ open_dpif_backer(const char *type, struct dpif_backer **backerp)
     backer->max_mpls_depth = check_max_mpls_depth(backer);
     backer->masked_set_action = check_masked_set_action(backer);
     backer->enable_ufid = check_ufid(backer);
+    backer->enable_bpf_action = check_bpf_actions(backer);
     backer->rid_pool = recirc_id_pool_create();
     ovs_mutex_init(&backer->recirc_mutex);
     cmap_init(&backer->recirc_map);
@@ -1032,6 +1038,9 @@ open_dpif_backer(const char *type, struct dpif_backer **backerp)
     backer->variable_length_userdata = check_variable_length_userdata(backer);
     backer->dp_version_string = dpif_get_dp_version(backer->dpif);
 
+    /* Load eBPF actions if datapath supports it.  */
+    error = ofproto_dpif_bpf_init(backer->enable_bpf_action);
+
     return error;
 }
 
@@ -1110,7 +1119,19 @@ check_ufid(struct dpif_backer *backer)
     return enable_ufid;
 }
 
+/* Tests whether 'dpif' supports eBPF base flow actions.
+ *
+ * Returns true if 'dpif' supports eBPF based actions.
+ */
+static bool
+check_bpf_actions( struct dpif_backer *backer OVS_UNUSED)
+{
+    /* XXX */
+    return true;
+}
+
 /* Tests whether 'backer''s datapath supports variable-length
+ *
  * OVS_USERSPACE_ATTR_USERDATA in OVS_ACTION_ATTR_USERSPACE actions.  We need
  * to disable some features on older datapaths that don't support this
  * feature.
-- 
1.9.1

[RFC: add openvswitch actions using BPF 9/9] ofproto-dpif-xlate: generate BPF output action (Hack)

[Andy Zhou]

This is a hack in xlate code to always generate both BPF output
and the original output action. Since the currnet BPF output 'action'
simply generate a kenrel log message about the output port number.

With this patch, Every time datapath output a packet, it also generate
log about the port number in the kernel log.

Signed-off-by: Andy Zhou <azhou@nicira.com>
---
 ofproto/ofproto-dpif-xlate.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
index 0786513..41a7502 100644
--- a/ofproto/ofproto-dpif-xlate.c
+++ b/ofproto/ofproto-dpif-xlate.c
@@ -15,6 +15,7 @@
 #include <config.h>
 
 #include "ofproto/ofproto-dpif-xlate.h"
+#include "ofproto/ofproto-dpif-bpf.h"
 
 #include <errno.h>
 #include <arpa/inet.h>
@@ -2833,14 +2834,27 @@ compose_output_action__(struct xlate_ctx *ctx, ofp_port_t ofp_port,
                                         OVS_ACTION_ATTR_TUNNEL_POP,
                                         odp_tnl_port);
                 } else {
+                    int fd;
+                    struct ovs_action_bpf_prog bpf_output;
+
+                    fd = ofproto_dpif_bpf_lookup("ovs/output");
+
+                    if (fd > 0) {
+                        bpf_output.prog_fd = htonl(fd);
+                        bpf_output.arg0 = htonl(out_port);
+                        bpf_output.arg1 = htonl(0);
+                        nl_msg_push_unspec(ctx->xout->odp_actions,
+                                           OVS_ACTION_ATTR_BPF_PROG,
+                                           &bpf_output, sizeof(bpf_output));
+                    }
                     /* Tunnel push-pop action is not compatible with
                      * IPFIX action. */
                     add_ipfix_output_action(ctx, out_port);
                     nl_msg_put_odp_port(ctx->xout->odp_actions,
                                         OVS_ACTION_ATTR_OUTPUT,
                                         out_port);
-               }
-           }
+                }
+            }
         }
 
         ctx->sflow_odp_port = odp_port;
-- 
1.9.1

Re: [ovs-dev] [RFC: add openvswitch actions using BPF 1/9] hack: Do not compile datapath

[Thomas Graf]

On 02/04/15 at 02:49pm, Andy Zhou wrote:
> The OVS eBPF patch sets are developed against master 'net-next' tree,
> which is currently targeting for kernel version 3.19.
> 
> The datapath in OVS mater does not currently support 3.19 kernel.
> But we should not really using OVS datapath, The changes in the
> follow patches are only changes in user space programs, which should
> work with the kernel module built from the 'net-next' tree.
> 
> This patch disables building of the datapath and disables the
> configuration time kernel version check.
> 
> Signed-off-by: Andy Zhou <azhou@nicira.com>

You can drop this when the datapath sync series is fully merged.

Re: [ovs-dev] [RFC: add openvswitch actions using BPF 2/9] odp: add a new ODP action: OVS_ACTION_ATTR_BPF_PROG

[Thomas Graf]

On 02/04/15 at 02:49pm, Andy Zhou wrote:
>  /**
> + * struct ovs_action_bpf_prog - %OVS_ACTION_ATTR_BPF_PROG action argument.
> + *
> + *  XXX provides bpf program id and execution context.
> + *
> + */
> +struct ovs_action_bpf_prog {
> +	__be32 prog_fd;
> +	__be32 arg0;
> +	__be32 arg1;
> +};

I assume you intend to replace this with a nested Netlink attribute with a
series of U32 attributes to carry args in the future?

> +enum ovs_bpf_func_id {
> +	OVS_BPF_FUNC_unspec,
> +	OVS_BPF_FUNC_output,          /* int ovs_bpf_output(ctxt) */
> +	__OVS_BPF_FUNC_MAX_ID,
> +};

The mix of cap / no-cap looks a bit weird.

> +static void
> +format_bpf_prog_action(struct ds *ds, const struct nlattr *a)
> +{
> +    struct ovs_action_bpf_prog *bpf_act = (struct ovs_action_bpf_prog *)
> +                                          nl_attr_get(a);
> +
> +    ds_put_cstr(ds, "bpf");
> +    ds_put_format(ds, "(%u,%u,%u)", ntohl(bpf_act->prog_fd),
> +                  ntohl(bpf_act->arg0), ntohl(bpf_act->arg1));
> +}

Maybe ds_put_format(ds, "bpf(... ?

Re: [ovs-dev] [RFC: add openvswitch actions using BPF 2/9] odp: add a new ODP action: OVS_ACTION_ATTR_BPF_PROG

[Thomas Graf]

On 02/04/15 at 02:49pm, Andy Zhou wrote:
> +struct ovs_action_bpf_prog {
> +	__be32 prog_fd;
> +	__be32 arg0;
> +	__be32 arg1;
> +};

Given the 64 bit registers of BPF. Should we have all args in the
Netlink message be 64 bit as well?

Re: [ovs-dev] [RFC: add openvswitch actions using BPF 5/9] bpf: add the first BPF program.

[Thomas Graf]

First of all, I *love* this. I have some questions on versioning of
helpers in the kernel. I will put comments in the respective kernel
patches directly.

On 02/04/15 at 02:49pm, Andy Zhou wrote:
> +EXTRA_DIST += $(srcdir)/bpf/ovs-bpf-helpers.h \
> +	      $(srcdir)/bpf/bpf-shared.h \
> +	      $(srcdir)/bpf/ovs-actions.c
> +
> +DEP_FILES  = $(srcdir)/bpf/ovs-bpf-helpers.h \
> +	     $(srcdir)/bpf/bpf-shared.h \
> +             $(srcdir)/datapath/linux/compat/include/linux/openvswitch.h

Some tab / space mixing here.

> +bpf/ovs-actions.bpf: $(srcdir)/bpf/ovs-actions.c $(DEP_FILES)
> +	$(AM_V_GEN)clang -DHAVE_CONFIG_H $(BPF_INCLUDES) $(NOSTDINC_FLAGS) \
> +		$(AM_CFLAGS) $(EXTRA_CFLAGS) -Wno-unused-value -Wno-pointer-sign \
> +		-O2 -emit-llvm -c $< -o -| $(LLC) -filetype=obj -o $@

I assume you will convert this to a Makefile template

Re: [ovs-dev] [RFC: add openvswitch actions using BPF 9/9] ofproto-dpif-xlate: generate BPF output action (Hack)

[Thomas Graf]

On 02/04/15 at 02:49pm, Andy Zhou wrote:
> This is a hack in xlate code to always generate both BPF output
> and the original output action. Since the currnet BPF output 'action'
> simply generate a kenrel log message about the output port number.
> 
> With this patch, Every time datapath output a packet, it also generate
> log about the port number in the kernel log.

This sounds great. Having the possibility the overwrite existing
datapath actions with BPF programs sounds like an excellent first
step towards BPF usage in the datapath.

Re: [RFC: add openvswitch actions using BPF 1/9] hack: Do not compile datapath

[Andy Zhou]

Yes. That's the plan.

On Thu, Feb 5, 2015 at 5:44 AM, Thomas Graf <tgraf@noironetworks.com> wrote:
> On 02/04/15 at 02:49pm, Andy Zhou wrote:
>> The OVS eBPF patch sets are developed against master 'net-next' tree,
>> which is currently targeting for kernel version 3.19.
>>
>> The datapath in OVS mater does not currently support 3.19 kernel.
>> But we should not really using OVS datapath, The changes in the
>> follow patches are only changes in user space programs, which should
>> work with the kernel module built from the 'net-next' tree.
>>
>> This patch disables building of the datapath and disables the
>> configuration time kernel version check.
>>
>> Signed-off-by: Andy Zhou <azhou@nicira.com>
>
> You can drop this when the datapath sync series is fully merged.
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev

Re: [ovs-dev] [RFC: add openvswitch actions using BPF 5/9] bpf: add the first BPF program.

[Andy Zhou]

On Thu, Feb 5, 2015 at 6:18 AM, Thomas Graf <tgraf@noironetworks.com> wrote:
> First of all, I *love* this. I have some questions on versioning of
> helpers in the kernel. I will put comments in the respective kernel
> patches directly.
>
Thanks for the feedbacks.
> On 02/04/15 at 02:49pm, Andy Zhou wrote:
>> +EXTRA_DIST += $(srcdir)/bpf/ovs-bpf-helpers.h \
>> +           $(srcdir)/bpf/bpf-shared.h \
>> +           $(srcdir)/bpf/ovs-actions.c
>> +
>> +DEP_FILES  = $(srcdir)/bpf/ovs-bpf-helpers.h \
>> +          $(srcdir)/bpf/bpf-shared.h \
>> +             $(srcdir)/datapath/linux/compat/include/linux/openvswitch.h
>
> Some tab / space mixing here.
Will fix.
>
>> +bpf/ovs-actions.bpf: $(srcdir)/bpf/ovs-actions.c $(DEP_FILES)
>> +     $(AM_V_GEN)clang -DHAVE_CONFIG_H $(BPF_INCLUDES) $(NOSTDINC_FLAGS) \
>> +             $(AM_CFLAGS) $(EXTRA_CFLAGS) -Wno-unused-value -Wno-pointer-sign \
>> +             -O2 -emit-llvm -c $< -o -| $(LLC) -filetype=obj -o $@
>
> I assume you will convert this to a Makefile template
Yes. It needs to be addressed.  Not sure if additional changes are
required now that
BPF backend is merged into LLVM upstream,

Re: [RFC: add openvswitch actions using BPF 4/9] autoconf: support -with-llc options

[Joe Stringer]

On 4 February 2015 at 14:49, Andy Zhou <azhou@nicira.com> wrote:
> Add a configuration option to allow specifing the LLVM backend module
> for compiling BPF C source code into BPF instruction sets.
>
> See INSTALL.BPF.md for usgae information.
>
> Signed-off-by: Joe Stringer <joestringer@nicira.com>
> Signed-off-by: Andy Zhou <azhou@nicira.com>
> ---
>  INSTALL.BPF.md | 42 ++++++++++++++++++++++++++++++++++++++++++
>  Makefile.am    |  1 +
>  acinclude.m4   | 16 ++++++++++++++++
>  configure.ac   |  2 ++
>  4 files changed, 61 insertions(+)
>  create mode 100644 INSTALL.BPF.md
>
> diff --git a/INSTALL.BPF.md b/INSTALL.BPF.md
> new file mode 100644
> index 0000000..bc1f5ad
> --- /dev/null
> +++ b/INSTALL.BPF.md
> @@ -0,0 +1,42 @@
> +# Dependencies
> +
> +- Need Clang + llvm-dev version 3.X for any (2 <= X <= 4)
> +- http://llvm.org/apt/
> +
> +apt-get install libelf-dev clang-3.4 llvm-3.4-dev
> +
> +# Build LLVM plugin
> +
> +*[Upstream] git://git.kernel.org/pub/scm/linux/kernel/git/ast/bpf*
> +
> +git clone git://github.com/joestringer/linux
> +NN=$PWD/linux
> +
> +cd $NN/tools/bpf/llvm/bld
> +make LLVM_CONFIG=`which llvm-config-3.4`
> +
> +# Configure OVS to use BPF LLC
> +
> +OVS=/path/to/openvswitch
> +cd $OVS
> +./configure --with-llc=$NN/tools/bpf/llvm/bld/Debug+Asserts/bin/llc

Naturally we'd need to update these instructions, this becomes much
simpler if we just require clang-3.7.

Re: [RFC: add openvswitch actions using BPF 7/9] ofproto-dpif: Add eBPF program loader and runtime infrasturcure.

[Joe Stringer]

On 4 February 2015 at 14:49, Andy Zhou <azhou@nicira.com> wrote:
> Added a eBPF program loader in ofproto-dpif layer. It only loades
> the OVS programs.
>
> Signed-off-by: Andy Zhou <azhou@nicira.com>
> ---

A little more precise: It only loads BPF programs with sections that
begin with the 'ovs' prefix.

There's a few scattered OVS style things like using ovs_scan(),
removing commented lines.

> +#define BPF_UNIXCTL_CHECK(conn) \
> +    if (bpf_unixctl_check_enable(conn) == -1) return

It seems unusual for OVS code to skimp on some "if(foo()) return"
logic by using a macro.

> +static int
> +bpf_unixctl_check_enable(struct unixctl_conn *conn)

Another minor style thing here, I think "check_enable" is ambiguous.
It's actually checking for datapath support for the feature. (I
realise there are other places in OVS that we do this, and I've been
meaning to clean them up).

> diff --git a/vswitchd/automake.mk b/vswitchd/automake.mk
> index 80affe9..a94720b 100644
> --- a/vswitchd/automake.mk
> +++ b/vswitchd/automake.mk
> @@ -19,6 +19,11 @@ vswitchd_ovs_vswitchd_LDFLAGS = $(AM_LDFLAGS) $(DPDK_vswitchd_LDFLAGS)
>  EXTRA_DIST += vswitchd/INTERNALS
>  MAN_ROOTS += vswitchd/ovs-vswitchd.8.in
>
> +if LINUX
> +vswitchd_ovs_vswitchd_LDADD += lib/libbpf.la
> +vswitchd_ovs_vswitchd_LDADD += -lelf
> +endif
> +

Do we perform a check for the libelf dev headers and library anywhere
in the build process?