[PATCH 0/5] examples: add a new makefile to build all examples

[PATCH 0/5] examples: add a new makefile to build all examples

[Olivier Matz]

This patch series adds a makefile to build all examples supported
by the configuration. It helps to check that all examples compile
after a dpdk modification.

After applying the patches, it is possible to build all examples by doing
the following:

  user@droids:~/dpdk.org$ cd examples
  user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
      RTE_TARGET=x86_64-default-linuxapp-gcc


Olivier Matz (5):
  mk: introduce rte.extsubdir.mk
  examples: use rte.extsubdir.mk to process subdirectories
  examples: add a makefile to build all examples
  examples: fix qos_sched makefile
  examples: fix netmap_compat example

 examples/Makefile                                | 68 ++++++++++++++++++++++++
 examples/l2fwd-ivshmem/Makefile                  |  9 +---
 examples/multi_process/Makefile                  | 16 +++---
 examples/multi_process/client_server_mp/Makefile | 15 +++---
 examples/netmap_compat/bridge/Makefile           |  5 +-
 examples/qos_sched/Makefile                      |  2 -
 examples/quota_watermark/Makefile                | 12 ++---
 mk/rte.extsubdir.mk                              | 49 +++++++++++++++++
 8 files changed, 138 insertions(+), 38 deletions(-)
 create mode 100644 examples/Makefile
 create mode 100644 mk/rte.extsubdir.mk

-- 
1.9.2

[PATCH 1/5] mk: introduce rte.extsubdir.mk

[Olivier Matz]

This makefile can be included by a project that needs to build several
applications or libraries that are located in different directories.

Signed-off-by: Olivier Matz <olivier.matz-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
---
 mk/rte.extsubdir.mk | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 mk/rte.extsubdir.mk

diff --git a/mk/rte.extsubdir.mk b/mk/rte.extsubdir.mk
new file mode 100644
index 0000000..97f4baf
--- /dev/null
+++ b/mk/rte.extsubdir.mk
@@ -0,0 +1,49 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2014 6WIND S.A.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of 6WIND S.A. nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+MAKEFLAGS += --no-print-directory
+
+# name of build directory in each example directory
+O ?= $(RTE_TARGET)
+
+.PHONY: all
+all: $(DIRS-y)
+
+.PHONY: clean
+clean: $(DIRS-y)
+
+.PHONY: $(DIRS-y)
+$(DIRS-y):
+	@echo "== $@"
+	$(Q)$(MAKE) -C $(@) \
+		M=$(CURDIR)/$(@)/Makefile \
+		O=$(CURDIR)/$(@)/$(O) \
+		S=$(CURDIR)/$(@) \
+		$(filter-out $(DIRS-y),$(MAKECMDGOALS))
-- 
1.9.2

[PATCH 2/5] examples: use rte.extsubdir.mk to process subdirectories

[Olivier Matz]

Signed-off-by: Olivier Matz <olivier.matz-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
---
 examples/l2fwd-ivshmem/Makefile                  |  9 +--------
 examples/multi_process/Makefile                  | 16 +++++++---------
 examples/multi_process/client_server_mp/Makefile | 15 ++++++---------
 examples/quota_watermark/Makefile                | 12 +++---------
 4 files changed, 17 insertions(+), 35 deletions(-)

diff --git a/examples/l2fwd-ivshmem/Makefile b/examples/l2fwd-ivshmem/Makefile
index 7286b37..df59ed8 100644
--- a/examples/l2fwd-ivshmem/Makefile
+++ b/examples/l2fwd-ivshmem/Makefile
@@ -37,14 +37,7 @@ endif
 RTE_TARGET ?= x86_64-ivshmem-linuxapp-gcc
 
 include $(RTE_SDK)/mk/rte.vars.mk
-unexport RTE_SRCDIR RTE_OUTPUT RTE_EXTMK
 
 DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += host guest
 
-.PHONY: all clean $(DIRS-y)
-
-all: $(DIRS-y)
-clean: $(DIRS-y)
-
-$(DIRS-y):
-	$(MAKE) -C $@ $(MAKECMDGOALS)
+include $(RTE_SDK)/mk/rte.extsubdir.mk
diff --git a/examples/multi_process/Makefile b/examples/multi_process/Makefile
index ba96a7e..f2c8e68 100644
--- a/examples/multi_process/Makefile
+++ b/examples/multi_process/Makefile
@@ -33,15 +33,13 @@ ifeq ($(RTE_SDK),)
 $(error "Please define RTE_SDK environment variable")
 endif
 
-include $(RTE_SDK)/mk/rte.vars.mk
-unexport RTE_SRCDIR RTE_OUTPUT RTE_EXTMK
-
-DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += $(wildcard *_mp)
+# Default target, can be overriden by command line or environment
+RTE_TARGET ?= x86_64-ivshmem-linuxapp-gcc
 
-.PHONY: all clean $(DIRS-y)
+include $(RTE_SDK)/mk/rte.vars.mk
 
-all: $(DIRS-y)
-clean: $(DIRS-y)
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += client_server_mp
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += simple_mp
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += symmetric_mp
 
-$(DIRS-y):
-	$(MAKE) -C $@ $(MAKECMDGOALS)
+include $(RTE_SDK)/mk/rte.extsubdir.mk
diff --git a/examples/multi_process/client_server_mp/Makefile b/examples/multi_process/client_server_mp/Makefile
index 24d31b0..b8d6b3f 100644
--- a/examples/multi_process/client_server_mp/Makefile
+++ b/examples/multi_process/client_server_mp/Makefile
@@ -33,15 +33,12 @@ ifeq ($(RTE_SDK),)
 $(error "Please define RTE_SDK environment variable")
 endif
 
-include $(RTE_SDK)/mk/rte.vars.mk
-unexport RTE_SRCDIR RTE_OUTPUT RTE_EXTMK
-
-DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += $(wildcard mp_*)
+# Default target, can be overriden by command line or environment
+RTE_TARGET ?= x86_64-ivshmem-linuxapp-gcc
 
-.PHONY: all clean $(DIRS-y)
+include $(RTE_SDK)/mk/rte.vars.mk
 
-all: $(DIRS-y)
-clean: $(DIRS-y)
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += mp_client
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += mp_server
 
-$(DIRS-y):
-	$(MAKE) -C $@ $(MAKECMDGOALS)
+include $(RTE_SDK)/mk/rte.extsubdir.mk
diff --git a/examples/quota_watermark/Makefile b/examples/quota_watermark/Makefile
index 5596dcc..e4d54c2 100644
--- a/examples/quota_watermark/Makefile
+++ b/examples/quota_watermark/Makefile
@@ -37,14 +37,8 @@ endif
 RTE_TARGET ?= x86_64-default-linuxapp-gcc
 
 include $(RTE_SDK)/mk/rte.vars.mk
-unexport RTE_SRCDIR RTE_OUTPUT RTE_EXTMK
 
-DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += $(wildcard qw*)
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += qw
+DIRS-$(CONFIG_RTE_EXEC_ENV_LINUXAPP) += qwctl
 
-.PHONY: all clean $(DIRS-y)
-
-all: $(DIRS-y)
-clean: $(DIRS-y)
-
-$(DIRS-y):
-	$(MAKE) -C $@ $(MAKECMDGOALS)
+include $(RTE_SDK)/mk/rte.extsubdir.mk
-- 
1.9.2

[PATCH 3/5] examples: add a makefile to build all examples

[Olivier Matz]

It is now possible to build all examples by doing the following:

  user@droids:~/dpdk.org$ cd examples
  user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
      RTE_TARGET=x86_64-default-linuxapp-gcc

Signed-off-by: Olivier Matz <olivier.matz-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
---
 examples/Makefile | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
 create mode 100644 examples/Makefile

diff --git a/examples/Makefile b/examples/Makefile
new file mode 100644
index 0000000..ec2532b
--- /dev/null
+++ b/examples/Makefile
@@ -0,0 +1,68 @@
+#   BSD LICENSE
+#
+#   Copyright(c) 2014 6WIND S.A.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of 6WIND S.A. nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overriden by command line or environment
+RTE_TARGET ?= x86_64-default-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+DIRS-y += cmdline
+ifneq ($(ICP_ROOT),)
+DIRS-y += dpdk_qat
+endif
+DIRS-y += exception_path
+DIRS-y += helloworld
+DIRS-y += ip_reassembly
+DIRS-$(RTE_MBUF_SCATTER_GATHER) += ipv4_frag
+DIRS-$(RTE_MBUF_SCATTER_GATHER) += ipv4_multicast
+DIRS-y += kni
+DIRS-y += l2fwd
+DIRS-$(CONFIG_RTE_LIBRTE_IVSHMEM) += l2fwd-ivshmem
+DIRS-y += l3fwd
+DIRS-y += l3fwd-power
+DIRS-y += l3fwd-vf
+DIRS-y += link_status_interrupt
+DIRS-y += load_balancer
+DIRS-y += multi_process
+DIRS-y += netmap_compat/bridge
+DIRS-$(CONFIG_RTE_LIBRTE_METER) += qos_meter
+DIRS-$(CONFIG_RTE_LIBRTE_SCHED) += qos_sched
+DIRS-y += quota_watermark
+DIRS-y += timer
+DIRS-y += vhost
+DIRS-$(CONFIG_RTE_LIBRTE_XEN_DOM0) += vhost_xen
+DIRS-y += vmdq
+DIRS-y += vmdq_dcb
+
+include $(RTE_SDK)/mk/rte.extsubdir.mk
-- 
1.9.2

[PATCH 4/5] examples: fix qos_sched makefile

[Olivier Matz]

The example does not compile as the linker complains about duplicated
symbols.

Remove -lsched from LDLIBS, it is already present in rte.app.mk and
added by the DPDK framework automatically.

Signed-off-by: Olivier Matz <olivier.matz-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
---
 examples/qos_sched/Makefile | 2 --
 1 file changed, 2 deletions(-)

diff --git a/examples/qos_sched/Makefile b/examples/qos_sched/Makefile
index b91fe37..9366efe 100755
--- a/examples/qos_sched/Makefile
+++ b/examples/qos_sched/Makefile
@@ -54,6 +54,4 @@ CFLAGS += $(WERROR_FLAGS)
 CFLAGS_args.o := -D_GNU_SOURCE
 CFLAGS_cfg_file.o := -D_GNU_SOURCE
 
-LDLIBS += -lrte_sched
-
 include $(RTE_SDK)/mk/rte.extapp.mk
-- 
1.9.2

[PATCH 5/5] examples: fix netmap_compat example

[Olivier Matz]

It is not allowed to reference a an absolute file name in SRCS-y.
A VPATH has to be used, else the dependencies won't be checked
properly.

Signed-off-by: Olivier Matz <olivier.matz-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
---
 examples/netmap_compat/bridge/Makefile | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/examples/netmap_compat/bridge/Makefile b/examples/netmap_compat/bridge/Makefile
index 74feb1e..ebc6b1c 100644
--- a/examples/netmap_compat/bridge/Makefile
+++ b/examples/netmap_compat/bridge/Makefile
@@ -41,9 +41,12 @@ include $(RTE_SDK)/mk/rte.vars.mk
 # binary name
 APP = bridge
 
+# for compat_netmap.c
+VPATH := $(SRCDIR)/../lib
+
 # all source are stored in SRCS-y
 SRCS-y := bridge.c
-SRCS-y += $(SRCDIR)/../lib/compat_netmap.c
+SRCS-y += compat_netmap.c
 
 CFLAGS += -O3 -I$(SRCDIR)/../lib -I$(SRCDIR)/../netmap
 CFLAGS += $(WERROR_FLAGS)
-- 
1.9.2

Re: [PATCH 0/5] examples: add a new makefile to build all examples

[Neil Horman]

On Fri, Apr 25, 2014 at 02:00:44PM +0200, Olivier Matz wrote:
> This patch series adds a makefile to build all examples supported
> by the configuration. It helps to check that all examples compile
> after a dpdk modification.
> 
> After applying the patches, it is possible to build all examples by doing
> the following:
> 
>   user@droids:~/dpdk.org$ cd examples
>   user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
>       RTE_TARGET=x86_64-default-linuxapp-gcc

What you actually want here is RTE_TARGET=build, as the Makefile looks for the
${RTE_TARGET}/.config file, it doesn't just parse whatever target file you
specify.

That begs the bigger point, RTE_TARGET will always be =build, so why not just
add a top level make rule for the examples subdirectory, then RTE_SDK and
RTE_TARGET will just be automatically set.


Regards
Neil

Re: [PATCH 0/5] examples: add a new makefile to build all examples

[Olivier MATZ]

Hi Neil,

First, thank you for your reviews.

On Friday, April 25, 2014 10:05:26 AM Neil Horman wrote:
> > After applying the patches, it is possible to build all examples by doing
> > the following:
> >   user@droids:~/dpdk.org$ cd examples
> >   user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
> >       RTE_TARGET=x86_64-default-linuxapp-gcc
> 
> What you actually want here is RTE_TARGET=build, as the Makefile looks for
> the ${RTE_TARGET}/.config file, it doesn't just parse whatever target file
> you specify.
> 
> That begs the bigger point, RTE_TARGET will always be =build, so why not
> just add a top level make rule for the examples subdirectory, then RTE_SDK
> and RTE_TARGET will just be automatically set.

When we compile an example or an external application, RTE_SDK references
a binary DPDK, or more precisely a DPDK source workspace that has been
compiled with "make install T=my-targets".

You're right that RTE_TARGET corresponds to the name of the build directory
but the logic is that the name of the directory is the name of the target.

To be clearer, the commit log should probably be updated with the
following list of commands:

  user@droids:~/dpdk.org$ make install T=x86_64-default-linuxapp-gcc
  user@droids:~/dpdk.org$ cd examples
  user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
        RTE_TARGET=x86_64-default-linuxapp-gcc

About adding a top level makefile rule, it's a good idea as it can
be documented in "make help", but I wasn't very confident about where
to put the output files. I can check if doing something like below is
possible or too complex:

  user@droids:~/dpdk.org$ make install T=x86_64-default-linuxapp-gcc
  user@droids:~/dpdk.org$ cd x86_64-default-linuxapp-gcc
  user@droids:~/dpdk.org/x86_64-default-linuxapp-gcc$ make examples
  # generate examples in $PWD/examples

Regards,
Olivier

Re: [PATCH 0/5] examples: add a new makefile to build all examples

[Neil Horman]

On Fri, Apr 25, 2014 at 06:24:48PM +0200, Olivier MATZ wrote:
> Hi Neil,
> 
> First, thank you for your reviews.
> 
You're welcome, glad to do it.

> On Friday, April 25, 2014 10:05:26 AM Neil Horman wrote:
> > > After applying the patches, it is possible to build all examples by doing
> > > the following:
> > >   user@droids:~/dpdk.org$ cd examples
> > >   user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
> > >       RTE_TARGET=x86_64-default-linuxapp-gcc
> > 
> > What you actually want here is RTE_TARGET=build, as the Makefile looks for
> > the ${RTE_TARGET}/.config file, it doesn't just parse whatever target file
> > you specify.
> > 
> > That begs the bigger point, RTE_TARGET will always be =build, so why not
> > just add a top level make rule for the examples subdirectory, then RTE_SDK
> > and RTE_TARGET will just be automatically set.
> 
> When we compile an example or an external application, RTE_SDK references
> a binary DPDK, or more precisely a DPDK source workspace that has been
> compiled with "make install T=my-targets".
> 
> You're right that RTE_TARGET corresponds to the name of the build directory
> but the logic is that the name of the directory is the name of the target.
> 
> To be clearer, the commit log should probably be updated with the
> following list of commands:
> 
>   user@droids:~/dpdk.org$ make install T=x86_64-default-linuxapp-gcc
>   user@droids:~/dpdk.org$ cd examples
>   user@droids:~/dpdk.org/examples$ make RTE_SDK=${PWD}/.. \
>         RTE_TARGET=x86_64-default-linuxapp-gcc
> 
Ah, I see, yes, that does make it more clear.  You're moving the built files to
a private directory so that you can target multiple platforms in parallel.
Please add the install step to the log file, I think that helps alot.

> About adding a top level makefile rule, it's a good idea as it can
> be documented in "make help", but I wasn't very confident about where
> to put the output files. I can check if doing something like below is
> possible or too complex:
> 
>   user@droids:~/dpdk.org$ make install T=x86_64-default-linuxapp-gcc
>   user@droids:~/dpdk.org$ cd x86_64-default-linuxapp-gcc
>   user@droids:~/dpdk.org/x86_64-default-linuxapp-gcc$ make examples
>   # generate examples in $PWD/examples
> 
That might work.  My thought was to simply add an examples rule like this to the
top level make file

examples:
	ifeq ($(T),)
	T := build
	endif
	$MAKE -C examples

The syntax isn't perfect, but you get the idea, basically just a rule to hand
off control to the examples subdirectory, that defaults the target directory to
build, should no other be specified.  The result is that the examples are built
for the last compiled target.

Regards
Neil
 
> Regards,
> Olivier
> 
>