dpdk proposal installation process

dpdk proposal installation process

[Arevalo, Mario Alfredo C]

Hi folks,

      Good day, this is a proposal in order to improve the dpdk install process,
I would like to know your point of view about the next points according to
previous conversations :) in order to create a new patches version.

1) I think the first thing that I have to be aware is "compatibility", the
new changes won't affect the current dpdk behaviour.

2) Create new makefile rules, these rules is going to install dpdk files in
default paths, however the linux distributions don't use the same paths for their
files, the linux distribution and the architecture can be factor for different
path as Panu commented in previous conversations, he is right, then all variables
could be overridden, the variables names for the user can be included in documentation.
Also an option could be a configuration file for paths, however I'm not sure.

3) The default paths for dpdk in order to follow a hierarchy, however the variable
with those values can be overridden.

-install-bin          --> /usr/bin.
-install-headers  --> /usr/include/dpdk
-install-lib           --> /usr/lib64
-install-doc         --> /usr/share/doc/dpdk
-install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
                                else KERNEL_DIR=/boot/modules).
-install-sdk         --> /usr/share/dpdk and call install-headers ).
-install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)

4) I'm going to take account all feedback about variables, paths etc for the new version :).

Thank you so much for your help.


Mario.

Re: dpdk proposal installation process

[Bruce Richardson]

On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C wrote:
> Hi folks,
> 
>       Good day, this is a proposal in order to improve the dpdk install process,
> I would like to know your point of view about the next points according to
> previous conversations :) in order to create a new patches version.
> 
> 1) I think the first thing that I have to be aware is "compatibility", the
> new changes won't affect the current dpdk behaviour.
> 
> 2) Create new makefile rules, these rules is going to install dpdk files in
> default paths, however the linux distributions don't use the same paths for their
> files, the linux distribution and the architecture can be factor for different
> path as Panu commented in previous conversations, he is right, then all variables
> could be overridden, the variables names for the user can be included in documentation.
> Also an option could be a configuration file for paths, however I'm not sure.
> 
> 3) The default paths for dpdk in order to follow a hierarchy, however the variable
> with those values can be overridden.
> 
> -install-bin          --> /usr/bin.
> -install-headers  --> /usr/include/dpdk
> -install-lib           --> /usr/lib64
> -install-doc         --> /usr/share/doc/dpdk
> -install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
>                                 else KERNEL_DIR=/boot/modules).
> -install-sdk         --> /usr/share/dpdk and call install-headers ).
> -install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
> 
> 4) I'm going to take account all feedback about variables, paths etc for the new version :).
> 
> Thank you so much for your help.
> 
> 
> Mario.

Hi Mario,

that seems like a lot of commands to add - are they all individually needed?

In terms of where things go, should the "usr" part not a) be configurable via
a parameter, and b) default to "/usr/local" as that's where user-installed
software from outside the packaging system normally gets put.

/Bruce

Re: dpdk proposal installation process

[Olivier MATZ]

Hi Mario,

On 10/20/2015 11:17 AM, Bruce Richardson wrote:
> On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C wrote:
>> Hi folks,
>>
>>       Good day, this is a proposal in order to improve the dpdk install process,
>> I would like to know your point of view about the next points according to
>> previous conversations :) in order to create a new patches version.
>>
>> 1) I think the first thing that I have to be aware is "compatibility", the
>> new changes won't affect the current dpdk behaviour.

Yes. As I stated in a previous mail, I think nobody uses the current
"make install" without specifying T= as the default value is to build
and install for all targets.

My suggestion is:

- rename the previous "install" target. The name could probably
  be "mbuild" (for multiple builds). Other ideas are welcome.

- when "make install" is invoked with T= argument, call the mbuild
  target to have the same behavior than before. This compat layer
  could be removed in the future.

- when "make install" is invoked without T=, it installs the fhs.

>> 2) Create new makefile rules, these rules is going to install dpdk files in
>> default paths, however the linux distributions don't use the same paths for their
>> files, the linux distribution and the architecture can be factor for different
>> path as Panu commented in previous conversations, he is right, then all variables
>> could be overridden, the variables names for the user can be included in documentation.
>> Also an option could be a configuration file for paths, however I'm not sure.

I think having variables is ok.

>> 3) The default paths for dpdk in order to follow a hierarchy, however the variable
>> with those values can be overridden.
>>
>> -install-bin          --> /usr/bin.
>> -install-headers  --> /usr/include/dpdk
>> -install-lib           --> /usr/lib64

I remember Panu suggested to have /usr/lib by default.
I also think /usr/lib a better default value: some distributions
use /usr/lib for 64 bits libs, but we never have 32 bits libs in
/usr/lib64.

>> -install-doc         --> /usr/share/doc/dpdk
>> -install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
>>                                 else KERNEL_DIR=/boot/modules).

I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?

>> -install-sdk         --> /usr/share/dpdk and call install-headers ).
>> -install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
>>
>> 4) I'm going to take account all feedback about variables, paths etc for the new version :).
>>
>> Thank you so much for your help.
>>
>>
>> Mario.
> 
> Hi Mario,
> 
> that seems like a lot of commands to add - are they all individually needed?
> 
> In terms of where things go, should the "usr" part not a) be configurable via
> a parameter, and b) default to "/usr/local" as that's where user-installed
> software from outside the packaging system normally gets put.

A PREFIX variable would do the job.
About the default to /usr or /usr/local, I agree that /usr/local looks
more usual, and I don't think it's a problem for packaging as soon as
it can be overridden.


Regards,
Olivier

Re: dpdk proposal installation process

[Panu Matilainen]

On 10/21/2015 10:15 PM, Olivier MATZ wrote:
> Hi Mario,
>
> On 10/20/2015 11:17 AM, Bruce Richardson wrote:
>> On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C wrote:
>>> Hi folks,
>>>
>>>        Good day, this is a proposal in order to improve the dpdk install process,
>>> I would like to know your point of view about the next points according to
>>> previous conversations :) in order to create a new patches version.
>>>
>>> 1) I think the first thing that I have to be aware is "compatibility", the
>>> new changes won't affect the current dpdk behaviour.
>
> Yes. As I stated in a previous mail, I think nobody uses the current
> "make install" without specifying T= as the default value is to build
> and install for all targets.
>
> My suggestion is:
>
> - rename the previous "install" target. The name could probably
>    be "mbuild" (for multiple builds). Other ideas are welcome.
>
> - when "make install" is invoked with T= argument, call the mbuild
>    target to have the same behavior than before. This compat layer
>    could be removed in the future.
>
> - when "make install" is invoked without T=, it installs the fhs.

Nice, this sounds like the best of both worlds.

>
>>> 2) Create new makefile rules, these rules is going to install dpdk files in
>>> default paths, however the linux distributions don't use the same paths for their
>>> files, the linux distribution and the architecture can be factor for different
>>> path as Panu commented in previous conversations, he is right, then all variables
>>> could be overridden, the variables names for the user can be included in documentation.
>>> Also an option could be a configuration file for paths, however I'm not sure.
>
> I think having variables is ok.
>
>>> 3) The default paths for dpdk in order to follow a hierarchy, however the variable
>>> with those values can be overridden.
>>>
>>> -install-bin          --> /usr/bin.
>>> -install-headers  --> /usr/include/dpdk
>>> -install-lib           --> /usr/lib64
>
> I remember Panu suggested to have /usr/lib by default.
> I also think /usr/lib a better default value: some distributions
> use /usr/lib for 64 bits libs, but we never have 32 bits libs in
> /usr/lib64.

Yes, just stick /usr/lib there and be done with it, lib64 is not a good 
default for these very reasons.

>>> -install-doc         --> /usr/share/doc/dpdk
>>> -install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
>>>                                  else KERNEL_DIR=/boot/modules).
>
> I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
>
>>> -install-sdk         --> /usr/share/dpdk and call install-headers ).
>>> -install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
>>>
>>> 4) I'm going to take account all feedback about variables, paths etc for the new version :).
>>>
>>> Thank you so much for your help.
>>>
>>>
>>> Mario.
>>
>> Hi Mario,
>>
>> that seems like a lot of commands to add - are they all individually needed?
>>
>> In terms of where things go, should the "usr" part not a) be configurable via
>> a parameter, and b) default to "/usr/local" as that's where user-installed
>> software from outside the packaging system normally gets put.
>
> A PREFIX variable would do the job.
> About the default to /usr or /usr/local, I agree that /usr/local looks
> more usual, and I don't think it's a problem for packaging as soon as
> it can be overridden.

Yeah, PREFIX support would be nice, and defaulting that to /usr/local 
would be the right thing.

	- Panu -

>
>
> Regards,
> Olivier
>

Re: dpdk proposal installation process

[Bruce Richardson]

On Thu, Oct 22, 2015 at 08:55:41AM +0300, Panu Matilainen wrote:
> On 10/21/2015 10:15 PM, Olivier MATZ wrote:
> >Hi Mario,
> >
> >On 10/20/2015 11:17 AM, Bruce Richardson wrote:
> >>On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C wrote:
> >>>Hi folks,
> >>>
> >>>       Good day, this is a proposal in order to improve the dpdk install process,
> >>>I would like to know your point of view about the next points according to
> >>>previous conversations :) in order to create a new patches version.
> >>>
> >>>1) I think the first thing that I have to be aware is "compatibility", the
> >>>new changes won't affect the current dpdk behaviour.
> >
> >Yes. As I stated in a previous mail, I think nobody uses the current
> >"make install" without specifying T= as the default value is to build
> >and install for all targets.
> >
> >My suggestion is:
> >
> >- rename the previous "install" target. The name could probably
> >   be "mbuild" (for multiple builds). Other ideas are welcome.
> >
> >- when "make install" is invoked with T= argument, call the mbuild
> >   target to have the same behavior than before. This compat layer
> >   could be removed in the future.
> >
> >- when "make install" is invoked without T=, it installs the fhs.
> 
> Nice, this sounds like the best of both worlds.
> 
> >
> >>>2) Create new makefile rules, these rules is going to install dpdk files in
> >>>default paths, however the linux distributions don't use the same paths for their
> >>>files, the linux distribution and the architecture can be factor for different
> >>>path as Panu commented in previous conversations, he is right, then all variables
> >>>could be overridden, the variables names for the user can be included in documentation.
> >>>Also an option could be a configuration file for paths, however I'm not sure.
> >
> >I think having variables is ok.
> >
> >>>3) The default paths for dpdk in order to follow a hierarchy, however the variable
> >>>with those values can be overridden.
> >>>
> >>>-install-bin          --> /usr/bin.
> >>>-install-headers  --> /usr/include/dpdk
> >>>-install-lib           --> /usr/lib64
> >
> >I remember Panu suggested to have /usr/lib by default.
> >I also think /usr/lib a better default value: some distributions
> >use /usr/lib for 64 bits libs, but we never have 32 bits libs in
> >/usr/lib64.
> 
> Yes, just stick /usr/lib there and be done with it, lib64 is not a good
> default for these very reasons.
> 
> >>>-install-doc         --> /usr/share/doc/dpdk
> >>>-install-mod        --> if RTE_EXEC_ENV=linuxapp then KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
> >>>                                 else KERNEL_DIR=/boot/modules).
> >
> >I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
> >
> >>>-install-sdk         --> /usr/share/dpdk and call install-headers ).
> >>>-install-fhs          --> call install-libraries, install-mod, install-bin and install-doc (maybe install-headers)
> >>>
> >>>4) I'm going to take account all feedback about variables, paths etc for the new version :).
> >>>
> >>>Thank you so much for your help.
> >>>
> >>>
> >>>Mario.
> >>
> >>Hi Mario,
> >>
> >>that seems like a lot of commands to add - are they all individually needed?
> >>
> >>In terms of where things go, should the "usr" part not a) be configurable via
> >>a parameter, and b) default to "/usr/local" as that's where user-installed
> >>software from outside the packaging system normally gets put.
> >
> >A PREFIX variable would do the job.
> >About the default to /usr or /usr/local, I agree that /usr/local looks
> >more usual, and I don't think it's a problem for packaging as soon as
> >it can be overridden.
> 
> Yeah, PREFIX support would be nice, and defaulting that to /usr/local would
> be the right thing.
> 
> 	- Panu -
> 
> >
> >
> >Regards,
> >Olivier
> >
> 

Can I throw a completely different suggestion into the mix?

Can we make use of the fact that make config creates a directory called "build"
by default. Then running "make" alone in that directory does the expected
behaviour of a compile of the whole sdk. How about having "make install" in the
build directory behave like a generic "make install" call for other packages?

I'm imagining the following sequence of steps to install:

	./configure --machine=[default|native|other] 
		# configure is a simple script that just calls "make config T=..."
	cd build
	make
	make install

Thoughts?

/Bruce

Re: dpdk proposal installation process

[Arevalo, Mario Alfredo C]

> -----Original Message-----
> From: Richardson, Bruce
> Sent: Thursday, October 22, 2015 9:57 AM
> To: Panu Matilainen
> Cc: Olivier MATZ; Arevalo, Mario Alfredo C; dev@dpdk.org
> Subject: Re: [dpdk-dev] dpdk proposal installation process
> 
> On Thu, Oct 22, 2015 at 08:55:41AM +0300, Panu Matilainen wrote:
> > On 10/21/2015 10:15 PM, Olivier MATZ wrote:
> > >Hi Mario,
> > >
> > >On 10/20/2015 11:17 AM, Bruce Richardson wrote:
> > >>On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C
> wrote:
> > >>>Hi folks,
> > >>>
> > >>>       Good day, this is a proposal in order to improve the dpdk
> > >>>install process, I would like to know your point of view about the
> > >>>next points according to previous conversations :) in order to create a
> new patches version.
> > >>>
> > >>>1) I think the first thing that I have to be aware is
> > >>>"compatibility", the new changes won't affect the current dpdk
> behaviour.
> > >
> > >Yes. As I stated in a previous mail, I think nobody uses the current
> > >"make install" without specifying T= as the default value is to build
> > >and install for all targets.
> > >
> > >My suggestion is:
> > >
> > >- rename the previous "install" target. The name could probably
> > >   be "mbuild" (for multiple builds). Other ideas are welcome.
> > >
> > >- when "make install" is invoked with T= argument, call the mbuild
> > >   target to have the same behavior than before. This compat layer
> > >   could be removed in the future.
> > >
> > >- when "make install" is invoked without T=, it installs the fhs.
> >
> > Nice, this sounds like the best of both worlds.
> >
> > >
> > >>>2) Create new makefile rules, these rules is going to install dpdk
> > >>>files in default paths, however the linux distributions don't use
> > >>>the same paths for their files, the linux distribution and the
> > >>>architecture can be factor for different path as Panu commented in
> > >>>previous conversations, he is right, then all variables could be
> overridden, the variables names for the user can be included in
> documentation.
> > >>>Also an option could be a configuration file for paths, however I'm not
> sure.
> > >
> > >I think having variables is ok.
> > >
> > >>>3) The default paths for dpdk in order to follow a hierarchy,
> > >>>however the variable with those values can be overridden.
> > >>>
> > >>>-install-bin          --> /usr/bin.
> > >>>-install-headers  --> /usr/include/dpdk
> > >>>-install-lib           --> /usr/lib64
> > >
> > >I remember Panu suggested to have /usr/lib by default.
> > >I also think /usr/lib a better default value: some distributions use
> > >/usr/lib for 64 bits libs, but we never have 32 bits libs in
> > >/usr/lib64.
> >
> > Yes, just stick /usr/lib there and be done with it, lib64 is not a
> > good default for these very reasons.
> >
> > >>>-install-doc         --> /usr/share/doc/dpdk
> > >>>-install-mod        --> if RTE_EXEC_ENV=linuxapp then
> KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
> > >>>                                 else KERNEL_DIR=/boot/modules).
> > >
> > >I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
> > >
> > >>>-install-sdk         --> /usr/share/dpdk and call install-headers ).
> > >>>-install-fhs          --> call install-libraries, install-mod, install-bin and install-
> doc (maybe install-headers)
> > >>>
> > >>>4) I'm going to take account all feedback about variables, paths etc for
> the new version :).
> > >>>
> > >>>Thank you so much for your help.
> > >>>
> > >>>
> > >>>Mario.
> > >>
> > >>Hi Mario,
> > >>
> > >>that seems like a lot of commands to add - are they all individually
> needed?
> > >>
> > >>In terms of where things go, should the "usr" part not a) be
> > >>configurable via a parameter, and b) default to "/usr/local" as
> > >>that's where user-installed software from outside the packaging system
> normally gets put.
> > >
> > >A PREFIX variable would do the job.
> > >About the default to /usr or /usr/local, I agree that /usr/local
> > >looks more usual, and I don't think it's a problem for packaging as
> > >soon as it can be overridden.
> >
> > Yeah, PREFIX support would be nice, and defaulting that to /usr/local
> > would be the right thing.
> >
> > 	- Panu -
> >
> > >
> > >
> > >Regards,
> > >Olivier
> > >
> >
> 
> Can I throw a completely different suggestion into the mix?
> 
> Can we make use of the fact that make config creates a directory called
> "build"
> by default. Then running "make" alone in that directory does the expected
> behaviour of a compile of the whole sdk. How about having "make install" in
> the build directory behave like a generic "make install" call for other
> packages?
> 
> I'm imagining the following sequence of steps to install:
> 
> 	./configure --machine=[default|native|other]
> 		# configure is a simple script that just calls "make config T=..."
> 	cd build
> 	make
> 	make install
> 
> Thoughts?
> 
> /Bruce

Hi Guys,

Thank you so much for your feedback, about your last comments,
I understood the next:

1)
-"make install" using "T" variable:
This command will use a rule for the current behaviour called "mbuild"

-"make install" without "T" variable:
This command will install everything: headers, libraries, modules, apps, docs, and
sdk files (config files, make files, scripts and examples).

2) About variables, you rigth, KMOD_DIR sounds better than KERNEL_DIR,
and LIB_DIR should be /usr/lib by default.

3)
-To use a "prefix variable"
-This prefix variable will be "/usr/local" by default.
-This prefix variable can be overridden.

Example:
$(DESTDIR)/$(PREFIX)/$(INCLUDE_DIR)

4) What do you think about the Bruce's proposal?

compile and install using configure-make-make_install

[Bruce Richardson]

Just clarify a bit further the idea I had in mind, I've put together the following
two example patches. After applying these patches the following sequence of
operations works to install dpdk libraries and headers into /opt (/opt/lib/dpdk
and /opt/include/dpdk respectively). Also the machine type is changed from native
to default, and the settings for KNI are disabled.

	./configure --prefix=/opt --machine=default \
		--config=RTE_LIBRTE_KNI=n \
		--config=CONFIG_RTE_KNI_KMOD=n
	cd build
	make 
	sudo make install

NOTE: I'm not suggesting this is ready for inclusion in DPDK in any shape or
form, I'm just trying to make clear what my suggestion was referring to, and
to trigger more discussion on the topic.

Bruce Richardson (2):
  gen-build-mk: add "make install" option to build dir
  add example configure script

 configure               | 59 +++++++++++++++++++++++++++++++++++++++++++++++++
 scripts/gen-build-mk.sh | 13 +++++++++++
 2 files changed, 72 insertions(+)
 create mode 100755 configure

-- 
2.4.3

[RFC-PATCH 1/2] gen-build-mk: add "make install" option to build dir

[Bruce Richardson]

Add in a "make install" command to the makefile for a DPDK build
directory. Once inside the directory, make and make install will
behave as normal for opensource packages, compiling and installing
in the filesystem the DPDK code.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 scripts/gen-build-mk.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/scripts/gen-build-mk.sh b/scripts/gen-build-mk.sh
index 5c12813..2387cf6 100755
--- a/scripts/gen-build-mk.sh
+++ b/scripts/gen-build-mk.sh
@@ -36,6 +36,9 @@
 #   $1: path of project src root
 #   $2: path of build dir (can be relative to $1)
 
+if [ -z "$RTE_PREFIX" ] ; then
+	RTE_PREFIX=/usr/local
+fi
 echo "# Automatically generated by gen-build-mk.sh"
 echo
 echo "ifdef O"
@@ -46,8 +49,18 @@ echo "endif"
 echo
 echo "MAKEFLAGS += --no-print-directory"
 echo
+echo "INSTALL = install -v"
+echo
 echo "all:"
 echo "	@\$(MAKE) -C $1 O=$2"
 echo
+echo "install:"
+echo "	@\$(INSTALL) -d $RTE_PREFIX/lib/dpdk $RTE_PREFIX/include/dpdk"
+echo "	@\$(INSTALL) -d $RTE_PREFIX/include/dpdk/exec-env $RTE_PREFIX/include/dpdk/generic"
+echo "	@\$(INSTALL) lib/* $RTE_PREFIX/lib/dpdk"
+echo "	@\$(INSTALL) include/*.h $RTE_PREFIX/include/dpdk"
+echo "	@\$(INSTALL) include/exec-env/*.h $RTE_PREFIX/include/dpdk/exec-env"
+echo "	@\$(INSTALL) include/generic/*.h $RTE_PREFIX/include/dpdk/generic"
+echo
 echo "%::"
 echo "	@\$(MAKE) -C $1 O=$2 \$@"
-- 
2.4.3

[RFC-PATCH 2/2] add example configure script

[Bruce Richardson]

To demonstrate the idea of using a configure-make-make install type
operation for installing DPDK, add in a simple configure script in
python which runs "make config T=<target>" to do most of the work and
then tweaks the .config file afterwards.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 configure | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100755 configure

diff --git a/configure b/configure
new file mode 100755
index 0000000..e820bdd
--- /dev/null
+++ b/configure
@@ -0,0 +1,59 @@
+#! /usr/bin/env python
+
+#   BSD LICENSE
+#
+#   Copyright(c) 2015 Intel Corporation. All rights reserved.
+#   All rights reserved.
+#
+#   Redistribution and use in source and binary forms, with or without
+#   modification, are permitted provided that the following conditions
+#   are met:
+#
+#     * Redistributions of source code must retain the above copyright
+#       notice, this list of conditions and the following disclaimer.
+#     * Redistributions in binary form must reproduce the above copyright
+#       notice, this list of conditions and the following disclaimer in
+#       the documentation and/or other materials provided with the
+#       distribution.
+#     * Neither the name of Intel Corporation nor the names of its
+#       contributors may be used to endorse or promote products derived
+#       from this software without specific prior written permission.
+#
+#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+config_append=""
+
+from getopt import getopt
+import sys, os
+
+optlist, args = getopt(sys.argv[1:], '', ['prefix=','machine=','config='])
+
+for option, arg in optlist:
+	if option == "--prefix":
+		os.environ['RTE_PREFIX'] = arg
+	elif option == "--machine":
+		config_append += "CONFIG_RTE_MACHINE=\"%s\"\n" % arg
+	elif option == "--config":
+		if not arg.startswith("CONFIG_"):
+			config_append += "CONFIG_%s\n" % arg
+		else:
+			config_append += "%s\n" % arg
+
+if "RTE_TARGET" in os.environ:
+	os.system("make config T=$RTE_TARGET")
+else:
+	os.system("make config T=x86_64-native-linuxapp-gcc")
+f = open("build/.config", "a")
+f.write(config_append)
+f.close()
+
-- 
2.4.3

Re: compile and install using configure-make-make_install

[Panu Matilainen]

On 10/27/2015 04:25 PM, Bruce Richardson wrote:
> Just clarify a bit further the idea I had in mind, I've put together the following
> two example patches. After applying these patches the following sequence of
> operations works to install dpdk libraries and headers into /opt (/opt/lib/dpdk
> and /opt/include/dpdk respectively). Also the machine type is changed from native
> to default, and the settings for KNI are disabled.
>
> 	./configure --prefix=/opt --machine=default \
> 		--config=RTE_LIBRTE_KNI=n \
> 		--config=CONFIG_RTE_KNI_KMOD=n
> 	cd build
> 	make
> 	sudo make install

I think I like the idea of a configure script that hides (some of) the 
DPDK config peculiarities behind a more familiar looking interface. For 
example it could be used for figuring out the machine-os-compiler 
triplet automatically for the common cases.

However having "make install" behave differently depending on which 
directory its invoked from I dunno. Its unlike anything else out there 
(that I know of) and I thought the idea was to make DPDK behave more 
like a normal OSS citizen :)

	- Panu -

Re: compile and install using configure-make-make_install

[Bruce Richardson]

On Tue, Nov 03, 2015 at 09:35:49AM +0200, Panu Matilainen wrote:
> On 10/27/2015 04:25 PM, Bruce Richardson wrote:
> >Just clarify a bit further the idea I had in mind, I've put together the following
> >two example patches. After applying these patches the following sequence of
> >operations works to install dpdk libraries and headers into /opt (/opt/lib/dpdk
> >and /opt/include/dpdk respectively). Also the machine type is changed from native
> >to default, and the settings for KNI are disabled.
> >
> >	./configure --prefix=/opt --machine=default \
> >		--config=RTE_LIBRTE_KNI=n \
> >		--config=CONFIG_RTE_KNI_KMOD=n
> >	cd build
> >	make
> >	sudo make install
> 
> I think I like the idea of a configure script that hides (some of) the DPDK
> config peculiarities behind a more familiar looking interface. For example
> it could be used for figuring out the machine-os-compiler triplet
> automatically for the common cases.
> 
> However having "make install" behave differently depending on which
> directory its invoked from I dunno. Its unlike anything else out there (that
> I know of) and I thought the idea was to make DPDK behave more like a normal
> OSS citizen :)
> 
> 	- Panu -
> 

Yes, I would agree with your complaint.

Unfortunately, I can't see any quick-fix way for it in the short term. Ideally,
I think we should look to rename the existing "make install " option to something
else, and then later look to add in a proper make install like above.

For example: I would suggest in one release e.g. 2.2 to rename make install to
e.g. "make sdk" or "make target" [I like make target, since it implies that you
specify what the target should be with T="..."].
At that point, we add a new "make install" rule which simply prints out
"make install is deprecated, use 'make target' instead."

Then we wait a couple of releases to allow everyone to switch over to the new rule.
Once done, then we can look to introduce a new, more standard, "make install" type
target. [And in this case, I think limiting it to a build subdir or some such
would also be helpful, just to help those doing a multi-version upgrade jump]

/Bruce

Re: dpdk proposal installation process

[Thomas Monjalon]

2015-10-26 16:18, Arevalo, Mario Alfredo C:
> Hi Guys,
> 
> Thank you so much for your feedback, about your last comments,
> I understood the next:
> 
> 1)
> -"make install" using "T" variable:
> This command will use a rule for the current behaviour called "mbuild"

I think the ability of building several targets in one command make
the build system more complex than it should be.
A target should be defined by its configured options, not a default
template config.
So I suggest to drop it.

> -"make install" without "T" variable:
> This command will install everything: headers, libraries, modules, apps, docs, and
> sdk files (config files, make files, scripts and examples).
> 
> 2) About variables, you rigth, KMOD_DIR sounds better than KERNEL_DIR,
> and LIB_DIR should be /usr/lib by default.
> 
> 3)
> -To use a "prefix variable"
> -This prefix variable will be "/usr/local" by default.
> -This prefix variable can be overridden.
> 
> Example:
> $(DESTDIR)/$(PREFIX)/$(INCLUDE_DIR)

It is a good step.
In order to be more standard, we should try to comply with these guidelines:
https://www.gnu.org/prep/standards/html_node/Directory-Variables.html
https://www.gnu.org/prep/standards/html_node/DESTDIR.html

> 4) What do you think about the Bruce's proposal?

I love the idea of having a configure script.
I would add some checks to solve the dependencies (external and internal).

Re: dpdk proposal installation process

[Marc]

On 22 October 2015 at 16:57, Bruce Richardson <bruce.richardson@intel.com>
wrote:

> On Thu, Oct 22, 2015 at 08:55:41AM +0300, Panu Matilainen wrote:
> > On 10/21/2015 10:15 PM, Olivier MATZ wrote:
> > >Hi Mario,
> > >
> > >On 10/20/2015 11:17 AM, Bruce Richardson wrote:
> > >>On Tue, Oct 20, 2015 at 12:21:00AM +0000, Arevalo, Mario Alfredo C
> wrote:
> > >>>Hi folks,
> > >>>
> > >>>       Good day, this is a proposal in order to improve the dpdk
> install process,
> > >>>I would like to know your point of view about the next points
> according to
> > >>>previous conversations :) in order to create a new patches version.
> > >>>
> > >>>1) I think the first thing that I have to be aware is
> "compatibility", the
> > >>>new changes won't affect the current dpdk behaviour.
> > >
> > >Yes. As I stated in a previous mail, I think nobody uses the current
> > >"make install" without specifying T= as the default value is to build
> > >and install for all targets.
> > >
> > >My suggestion is:
> > >
> > >- rename the previous "install" target. The name could probably
> > >   be "mbuild" (for multiple builds). Other ideas are welcome.
> > >
> > >- when "make install" is invoked with T= argument, call the mbuild
> > >   target to have the same behavior than before. This compat layer
> > >   could be removed in the future.
> > >
> > >- when "make install" is invoked without T=, it installs the fhs.
> >
> > Nice, this sounds like the best of both worlds.
> >
> > >
> > >>>2) Create new makefile rules, these rules is going to install dpdk
> files in
> > >>>default paths, however the linux distributions don't use the same
> paths for their
> > >>>files, the linux distribution and the architecture can be factor for
> different
> > >>>path as Panu commented in previous conversations, he is right, then
> all variables
> > >>>could be overridden, the variables names for the user can be included
> in documentation.
> > >>>Also an option could be a configuration file for paths, however I'm
> not sure.
> > >
> > >I think having variables is ok.
> > >
> > >>>3) The default paths for dpdk in order to follow a hierarchy, however
> the variable
> > >>>with those values can be overridden.
> > >>>
> > >>>-install-bin          --> /usr/bin.
> > >>>-install-headers  --> /usr/include/dpdk
> > >>>-install-lib           --> /usr/lib64
> > >
> > >I remember Panu suggested to have /usr/lib by default.
> > >I also think /usr/lib a better default value: some distributions
> > >use /usr/lib for 64 bits libs, but we never have 32 bits libs in
> > >/usr/lib64.
> >
> > Yes, just stick /usr/lib there and be done with it, lib64 is not a good
> > default for these very reasons.
> >
> > >>>-install-doc         --> /usr/share/doc/dpdk
> > >>>-install-mod        --> if RTE_EXEC_ENV=linuxapp then
> KERNEL_DIR=/lib/modules/$(uname -r)/extra/drivers/dpdk
> > >>>                                 else KERNEL_DIR=/boot/modules).
> > >
> > >I'm not sure KERNEL_DIR is the proper name. Maybe KMOD_DIR?
> > >
> > >>>-install-sdk         --> /usr/share/dpdk and call install-headers ).
> > >>>-install-fhs          --> call install-libraries, install-mod,
> install-bin and install-doc (maybe install-headers)
> > >>>
> > >>>4) I'm going to take account all feedback about variables, paths etc
> for the new version :).
> > >>>
> > >>>Thank you so much for your help.
> > >>>
> > >>>
> > >>>Mario.
> > >>
> > >>Hi Mario,
> > >>
> > >>that seems like a lot of commands to add - are they all individually
> needed?
> > >>
> > >>In terms of where things go, should the "usr" part not a) be
> configurable via
> > >>a parameter, and b) default to "/usr/local" as that's where
> user-installed
> > >>software from outside the packaging system normally gets put.
> > >
> > >A PREFIX variable would do the job.
> > >About the default to /usr or /usr/local, I agree that /usr/local looks
> > >more usual, and I don't think it's a problem for packaging as soon as
> > >it can be overridden.
> >
> > Yeah, PREFIX support would be nice, and defaulting that to /usr/local
> would
> > be the right thing.
> >
> >       - Panu -
> >
> > >
> > >
> > >Regards,
> > >Olivier
> > >
> >
>
> Can I throw a completely different suggestion into the mix?
>
> Can we make use of the fact that make config creates a directory called
> "build"
> by default. Then running "make" alone in that directory does the expected
> behaviour of a compile of the whole sdk. How about having "make install"
> in the
> build directory behave like a generic "make install" call for other
> packages?
>
> I'm imagining the following sequence of steps to install:
>
>         ./configure --machine=[default|native|other]
>                 # configure is a simple script that just calls "make
> config T=..."
>         cd build
>

Why not the inverse, configure in the folder where you build so that you
have all the compilation environment in the target folder  (as in
autoconf+automake and as of now in DPDK). You can have easily parallel
builds in different folders.



>         make
>         make install
>

If you want this workflow, why not directly using autoconf + maybe the
config file there is now (since there are a ton of parameters)? Putting
general configuration parameters into configure.ac and leave the rest to
the config files.

The PREFIX and installation of files is something that automake+autoconf
solves too (probably without libtool for DPDK).

In any case, for install-fhs, I would name it install-all, to make it
consistent with typical autotools build envs, which is what users are used
to.

Marc



> Thoughts?
>
> /Bruce
>