[meta-networking][PATCH 1/2 v4] netmap-modules: make deterministic builds for drivers

[meta-networking][PATCH 1/2 v4] netmap-modules: make deterministic builds for drivers

[Mark Hatle]

From: Jackie Huang <jackie.huang@windriver.com>

The driver builds are optional, but for deterministic builds,
we should should be able to explicitly enable/disable the
builds for them in a proper place (maybe in BSP).

We use PACKAGECONFIG for listing enabled drivers but can't use
it's arguments since there is no option for each driver, and
the options for drivers are:
 --no-drivers    do not compile any driver
 --no-drivers=   do not compile the given drivers (comma sep.)
 --drivers=      only compile the given drivers (comma sep.)

So add additional logic to add proper configs to EXTRA_OECONF

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 .../recipes-kernel/netmap/netmap-modules_git.bb    | 48 +++++++++++++++++++++-
 1 file changed, 47 insertions(+), 1 deletion(-)

diff --git a/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb b/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
index 7405d51..c4f54e9 100644
--- a/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
+++ b/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
@@ -20,7 +20,53 @@ EXTRA_OECONF = "--kernel-dir=${STAGING_KERNEL_BUILDDIR} \
                 --driver-suffix="-netmap" \
                 "
 
-EXTRA_OECONF += "--no-drivers=ixgbe,virtio_net.c"
+# The driver builds are optional, but for deterministic builds,
+# we should be able to explicitly enable/disable the builds
+# for them in a proper place (maybe in BSP).
+# We use PACKAGECONFIG for listing enabled drivers but can't use
+# it's arguments since there is no option for each driver, and
+# the options for drivers are:
+#  --no-drivers    do not compile any driver
+#  --no-drivers=   do not compile the given drivers (comma sep.)
+#  --drivers=      only compile the given drivers (comma sep.)
+#
+# So use the following python code to add proper configs to
+# EXTRA_OECONF and skip arch test for kernel modules.
+#
+# The default is no-drivers, and all supported drivers are listed
+# as PACKAGECONFIG flags.
+PACKAGECONFIG ??= ""
+
+# List all supported drivers, get from "./configure --show-drivers",
+# Add "driver_" prefix for easy distinguishing from other configure
+# options.
+PACKAGECONFIG[driver_ixgbe] = ",,,"
+PACKAGECONFIG[driver_igb] = ",,,"
+PACKAGECONFIG[driver_e1000e] = ",,,"
+PACKAGECONFIG[driver_e1000] = ",,,"
+PACKAGECONFIG[driver_veth.c] = ",,,"
+PACKAGECONFIG[driver_forcedeth.c] = ",,,"
+PACKAGECONFIG[driver_virtio_net.c] = ",,,"
+PACKAGECONFIG[driver_r8169.c] = ",,,"
+
+python __anonymous () {
+    pkgconfigs = d.getVar("PACKAGECONFIG", True).split()
+    pkgconfigflags = d.getVarFlags("PACKAGECONFIG").keys()
+
+    # This is needed since there may be non-driver flags in PACKAGECONFIG
+    all_drivers_list = []
+    for f in pkgconfigflags:
+        if f.startswith('driver_'):
+            all_drivers_list.append(f)
+    drivers_list = []
+    for i in pkgconfigs:
+        if i in all_drivers_list:
+            drivers_list.append(i[len('driver_'):])
+
+    config_drivers = "--drivers=" + ",".join(drivers_list)
+    extra_oeconf_drivers = bb.utils.contains_any('PACKAGECONFIG', all_drivers_list, config_drivers, '--no-drivers', d)
+    d.appendVar("EXTRA_OECONF", extra_oeconf_drivers)
+}
 
 LDFLAGS := "${@'${LDFLAGS}'.replace('-Wl,-O1', '')}"
 LDFLAGS := "${@'${LDFLAGS}'.replace('-Wl,--as-needed', '')}"
-- 
1.9.3

[meta-networking][PATCH 2/2 v4] netmap-modules: Modules may not have the same arch as userspace

[Mark Hatle]

From: Jackie Huang <jackie.huang@windriver.com>

Kernel modules may not have the same architecture as user space.  So we
tell INSANE_SKIP to skip checking the arch for the modules.  This is
consistent with other kernel modules and the kernel recipe.

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
 meta-networking/recipes-kernel/netmap/netmap-modules_git.bb | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb b/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
index c4f54e9..8930285 100644
--- a/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
+++ b/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
@@ -66,6 +66,11 @@ python __anonymous () {
     config_drivers = "--drivers=" + ",".join(drivers_list)
     extra_oeconf_drivers = bb.utils.contains_any('PACKAGECONFIG', all_drivers_list, config_drivers, '--no-drivers', d)
     d.appendVar("EXTRA_OECONF", extra_oeconf_drivers)
+
+    # skip the arch test for kernel modules
+    if drivers_list:
+        for driver in drivers_list:
+            d.setVar("INSANE_SKIP_kernel-module-%s-netmap" % bb.utils.prune_suffix(driver, ['.c'], d), "arch")
 }
 
 LDFLAGS := "${@'${LDFLAGS}'.replace('-Wl,-O1', '')}"
-- 
1.9.3

Re: [meta-networking][PATCH 1/2 v4] netmap-modules: make deterministic builds for drivers

[Mark Hatle]

V4:

Based on Martin's feedback, I reworked this and split the commit into two pieces.

V3: (not previously sent to the list)

renamed 'dri_' to 'driver_' as we already had some confusion that these were
'direct rendering interface' drivers and how were they in networking.. :P

--Mark

On 11/20/15 11:32 AM, Mark Hatle wrote:
> From: Jackie Huang <jackie.huang@windriver.com>
> 
> The driver builds are optional, but for deterministic builds,
> we should should be able to explicitly enable/disable the
> builds for them in a proper place (maybe in BSP).
> 
> We use PACKAGECONFIG for listing enabled drivers but can't use
> it's arguments since there is no option for each driver, and
> the options for drivers are:
>  --no-drivers    do not compile any driver
>  --no-drivers=   do not compile the given drivers (comma sep.)
>  --drivers=      only compile the given drivers (comma sep.)
> 
> So add additional logic to add proper configs to EXTRA_OECONF
> 
> Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
>  .../recipes-kernel/netmap/netmap-modules_git.bb    | 48 +++++++++++++++++++++-
>  1 file changed, 47 insertions(+), 1 deletion(-)
> 
> diff --git a/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb b/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
> index 7405d51..c4f54e9 100644
> --- a/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
> +++ b/meta-networking/recipes-kernel/netmap/netmap-modules_git.bb
> @@ -20,7 +20,53 @@ EXTRA_OECONF = "--kernel-dir=${STAGING_KERNEL_BUILDDIR} \
>                  --driver-suffix="-netmap" \
>                  "
>  
> -EXTRA_OECONF += "--no-drivers=ixgbe,virtio_net.c"
> +# The driver builds are optional, but for deterministic builds,
> +# we should be able to explicitly enable/disable the builds
> +# for them in a proper place (maybe in BSP).
> +# We use PACKAGECONFIG for listing enabled drivers but can't use
> +# it's arguments since there is no option for each driver, and
> +# the options for drivers are:
> +#  --no-drivers    do not compile any driver
> +#  --no-drivers=   do not compile the given drivers (comma sep.)
> +#  --drivers=      only compile the given drivers (comma sep.)
> +#
> +# So use the following python code to add proper configs to
> +# EXTRA_OECONF and skip arch test for kernel modules.
> +#
> +# The default is no-drivers, and all supported drivers are listed
> +# as PACKAGECONFIG flags.
> +PACKAGECONFIG ??= ""
> +
> +# List all supported drivers, get from "./configure --show-drivers",
> +# Add "driver_" prefix for easy distinguishing from other configure
> +# options.
> +PACKAGECONFIG[driver_ixgbe] = ",,,"
> +PACKAGECONFIG[driver_igb] = ",,,"
> +PACKAGECONFIG[driver_e1000e] = ",,,"
> +PACKAGECONFIG[driver_e1000] = ",,,"
> +PACKAGECONFIG[driver_veth.c] = ",,,"
> +PACKAGECONFIG[driver_forcedeth.c] = ",,,"
> +PACKAGECONFIG[driver_virtio_net.c] = ",,,"
> +PACKAGECONFIG[driver_r8169.c] = ",,,"
> +
> +python __anonymous () {
> +    pkgconfigs = d.getVar("PACKAGECONFIG", True).split()
> +    pkgconfigflags = d.getVarFlags("PACKAGECONFIG").keys()
> +
> +    # This is needed since there may be non-driver flags in PACKAGECONFIG
> +    all_drivers_list = []
> +    for f in pkgconfigflags:
> +        if f.startswith('driver_'):
> +            all_drivers_list.append(f)
> +    drivers_list = []
> +    for i in pkgconfigs:
> +        if i in all_drivers_list:
> +            drivers_list.append(i[len('driver_'):])
> +
> +    config_drivers = "--drivers=" + ",".join(drivers_list)
> +    extra_oeconf_drivers = bb.utils.contains_any('PACKAGECONFIG', all_drivers_list, config_drivers, '--no-drivers', d)
> +    d.appendVar("EXTRA_OECONF", extra_oeconf_drivers)
> +}
>  
>  LDFLAGS := "${@'${LDFLAGS}'.replace('-Wl,-O1', '')}"
>  LDFLAGS := "${@'${LDFLAGS}'.replace('-Wl,--as-needed', '')}"
>