[PATCH 1/2] README.qemu: qemuppc64 is not supported

[PATCH 1/2] README.qemu: qemuppc64 is not supported

[Randy MacLeod]

Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
---
 README.qemu | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.qemu b/README.qemu
index fca04b1..9f56b7d 100644
--- a/README.qemu
+++ b/README.qemu
@@ -7,7 +7,7 @@ are currently supported in 32 and 64 bit variants:
 
   * ARM (qemuarm + qemuarm64)
   * x86 (qemux86 + qemux86-64)
-  * PowerPC (qemuppc + qemuppc64)
+  * PowerPC (qemuppc only)
   * MIPS (qemumips + qemumips64)
 
 Use of the QEMU images is covered in the Yocto Project Reference Manual.
-- 
2.9.3

[PATCH 2/2] openssl: force soft link to avoid rare race

[Randy MacLeod]

This patch works around a rare parallel build race condition using
the force option when soft linking.

The error seen is:

ln: failed to create symbolic link 'libssl.so': File exists
make[4]: *** [Makefile.shared:171: link_a.gnu] Error 1
make[4]: Leaving directory
'/.../build/tmp-glibc/work/x86_64-linux/openssl-native/1.0.2k-r0/openssl-1.0.2k'

Just add the -f flag to the platform independent soft link code to
avoid the collision.  This is reasonable since this Makefile removes
the link target before creating a new soft link. The Makefile was
written this way to support platforms that don't allow forcing a
softlink to overwrite an existing link. Only builds on Linux are
supported so that's not a requirement for oe-core recipes.

The openssl team is rewriting their build files so it's not appropriate
for openssl upstream and fixing the root cause of the race condition
was also not pursued.

Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
---
 ...penssl-force-soft-link-to-avoid-rare-race.patch | 48 ++++++++++++++++++++++
 .../recipes-connectivity/openssl/openssl_1.0.2l.bb |  1 +
 2 files changed, 49 insertions(+)
 create mode 100644 meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch

diff --git a/meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch b/meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch
new file mode 100644
index 0000000..f3c1ac7
--- /dev/null
+++ b/meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch
@@ -0,0 +1,48 @@
+From 3d9199423d48766649a2b2ebb3924e892ed16fa4 Mon Sep 17 00:00:00 2001
+From: Randy MacLeod <Randy.MacLeod@windriver.com>
+Date: Tue, 20 Jun 2017 15:32:08 -0400
+Subject: [PATCH] openssl: Force soft link to avoid rare race
+
+This patch works around a rare parallel build race condition. 
+The error seen is:
+
+ln: failed to create symbolic link 'libssl.so': File exists
+make[4]: *** [Makefile.shared:171: link_a.gnu] Error 1
+make[4]: Leaving directory
+'/.../build/tmp-glibc/work/x86_64-linux/openssl-native/1.0.2k-r0/openssl-1.0.2k'
+
+The openssl team is rewriting their build files so it's not
+appropriate for openssl upstream and fixing the root cause of
+the Makefile race condition was also not pursued.
+
+Upstream-status: Inappropriate [build rules rewrite in progress]
+
+Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
+---
+ Makefile.shared | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/Makefile.shared b/Makefile.shared
+index e8d222a..1bff92f 100644
+--- a/Makefile.shared
++++ b/Makefile.shared
+@@ -118,14 +118,14 @@
+ 		if [ -n "$$SHLIB_COMPAT" ]; then \
+ 			for x in $$SHLIB_COMPAT; do \
+ 				( $(SET_X); rm -f $$SHLIB$$x$$SHLIB_SUFFIX; \
+-				  ln -s $$prev $$SHLIB$$x$$SHLIB_SUFFIX ); \
++				  ln -sf $$prev $$SHLIB$$x$$SHLIB_SUFFIX ); \
+ 				prev=$$SHLIB$$x$$SHLIB_SUFFIX; \
+ 			done; \
+ 		fi; \
+ 		if [ -n "$$SHLIB_SOVER" ]; then \
+ 			[ -e "$$SHLIB$$SHLIB_SUFFIX" ] || \
+ 			( $(SET_X); rm -f $$SHLIB$$SHLIB_SUFFIX; \
+-			  ln -s $$prev $$SHLIB$$SHLIB_SUFFIX ); \
++			  ln -sf $$prev $$SHLIB$$SHLIB_SUFFIX ); \
+ 		fi; \
+ 	fi
+ 
+-- 
+2.9.3
+
diff --git a/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
index 8c34ea6..c537aa4 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
@@ -41,6 +41,7 @@ SRC_URI += "file://find.pl;subdir=openssl-${PV}/util/ \
             file://openssl-util-perlpath.pl-cwd.patch \
             file://Use-SHA256-not-MD5-as-default-digest.patch \
             file://0001-Fix-build-with-clang-using-external-assembler.patch \
+            file://0001-openssl-force-soft-link-to-avoid-rare-race.patch  \
             "
 SRC_URI[md5sum] = "f85123cd390e864dfbe517e7616e6566"
 SRC_URI[sha256sum] = "ce07195b659e75f4e1db43552860070061f156a98bb37b672b101ba6e3ddf30c"
-- 
2.9.3

Re: [PATCH 2/2] openssl: force soft link to avoid rare race

[Randy MacLeod]

On 2017-10-16 11:00 AM, Randy MacLeod wrote:
> This patch works around a rare parallel build race condition using
> the force option when soft linking.

Rare here means a few times out of > 5000 builds.

I pushed this change into the WR oe-core branch 3 weeks ago (Sept 25th)
and haven't see any build failures of this type since then
whereas I would expect to see about 5-10 without the work-around.

../Randy

> 
> The error seen is:
> 
> ln: failed to create symbolic link 'libssl.so': File exists
> make[4]: *** [Makefile.shared:171: link_a.gnu] Error 1
> make[4]: Leaving directory
> '/.../build/tmp-glibc/work/x86_64-linux/openssl-native/1.0.2k-r0/openssl-1.0.2k'
> 
> Just add the -f flag to the platform independent soft link code to
> avoid the collision.  This is reasonable since this Makefile removes
> the link target before creating a new soft link. The Makefile was
> written this way to support platforms that don't allow forcing a
> softlink to overwrite an existing link. Only builds on Linux are
> supported so that's not a requirement for oe-core recipes.
> 
> The openssl team is rewriting their build files so it's not appropriate
> for openssl upstream and fixing the root cause of the race condition
> was also not pursued.
> 
> Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
> ---
>   ...penssl-force-soft-link-to-avoid-rare-race.patch | 48 ++++++++++++++++++++++
>   .../recipes-connectivity/openssl/openssl_1.0.2l.bb |  1 +
>   2 files changed, 49 insertions(+)
>   create mode 100644 meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch
> 
> diff --git a/meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch b/meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch
> new file mode 100644
> index 0000000..f3c1ac7
> --- /dev/null
> +++ b/meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch
> @@ -0,0 +1,48 @@
> +From 3d9199423d48766649a2b2ebb3924e892ed16fa4 Mon Sep 17 00:00:00 2001
> +From: Randy MacLeod <Randy.MacLeod@windriver.com>
> +Date: Tue, 20 Jun 2017 15:32:08 -0400
> +Subject: [PATCH] openssl: Force soft link to avoid rare race
> +
> +This patch works around a rare parallel build race condition.
> +The error seen is:
> +
> +ln: failed to create symbolic link 'libssl.so': File exists
> +make[4]: *** [Makefile.shared:171: link_a.gnu] Error 1
> +make[4]: Leaving directory
> +'/.../build/tmp-glibc/work/x86_64-linux/openssl-native/1.0.2k-r0/openssl-1.0.2k'
> +
> +The openssl team is rewriting their build files so it's not
> +appropriate for openssl upstream and fixing the root cause of
> +the Makefile race condition was also not pursued.
> +
> +Upstream-status: Inappropriate [build rules rewrite in progress]
> +
> +Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
> +---
> + Makefile.shared | 4 ++--
> + 1 file changed, 2 insertions(+), 2 deletions(-)
> +
> +diff --git a/Makefile.shared b/Makefile.shared
> +index e8d222a..1bff92f 100644
> +--- a/Makefile.shared
> ++++ b/Makefile.shared
> +@@ -118,14 +118,14 @@
> + 		if [ -n "$$SHLIB_COMPAT" ]; then \
> + 			for x in $$SHLIB_COMPAT; do \
> + 				( $(SET_X); rm -f $$SHLIB$$x$$SHLIB_SUFFIX; \
> +-				  ln -s $$prev $$SHLIB$$x$$SHLIB_SUFFIX ); \
> ++				  ln -sf $$prev $$SHLIB$$x$$SHLIB_SUFFIX ); \
> + 				prev=$$SHLIB$$x$$SHLIB_SUFFIX; \
> + 			done; \
> + 		fi; \
> + 		if [ -n "$$SHLIB_SOVER" ]; then \
> + 			[ -e "$$SHLIB$$SHLIB_SUFFIX" ] || \
> + 			( $(SET_X); rm -f $$SHLIB$$SHLIB_SUFFIX; \
> +-			  ln -s $$prev $$SHLIB$$SHLIB_SUFFIX ); \
> ++			  ln -sf $$prev $$SHLIB$$SHLIB_SUFFIX ); \
> + 		fi; \
> + 	fi
> +
> +--
> +2.9.3
> +
> diff --git a/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
> index 8c34ea6..c537aa4 100644
> --- a/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
> +++ b/meta/recipes-connectivity/openssl/openssl_1.0.2l.bb
> @@ -41,6 +41,7 @@ SRC_URI += "file://find.pl;subdir=openssl-${PV}/util/ \
>               file://openssl-util-perlpath.pl-cwd.patch \
>               file://Use-SHA256-not-MD5-as-default-digest.patch \
>               file://0001-Fix-build-with-clang-using-external-assembler.patch \
> +            file://0001-openssl-force-soft-link-to-avoid-rare-race.patch  \
>               "
>   SRC_URI[md5sum] = "f85123cd390e864dfbe517e7616e6566"
>   SRC_URI[sha256sum] = "ce07195b659e75f4e1db43552860070061f156a98bb37b672b101ba6e3ddf30c"
> 


-- 
# Randy MacLeod.  WR Linux
# Wind River an Intel Company

✗ patchtest: failure for "README.qemu: qemuppc64 is not ..." and 1 more

[Patchwork]

== Series Details ==

Series: "README.qemu: qemuppc64 is not ..." and 1 more
Revision: 1
URL   : https://patchwork.openembedded.org/series/9358/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence] 
  Suggested fix    Add Upstream-Status: <status> to the header of meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch (possible values: Pending, Submitted, Accepted, Backport, Denied, Inappropriate)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: ✗ patchtest: failure for "README.qemu: qemuppc64 is not ..." and 1 more

[Randy MacLeod]

On 2017-10-16 12:00 PM, Patchwork wrote:
> == Series Details ==
> 
> Series: "README.qemu: qemuppc64 is not ..." and 1 more
> Revision: 1
> URL   : https://patchwork.openembedded.org/series/9358/
> State : failure
> 
> == Summary ==
> 
> 
> Thank you for submitting this patch series to OpenEmbedded Core. This is
> an automated response. Several tests have been executed on the proposed
> series by patchtest resulting in the following failures:
> 
> 
> 
> * Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence]
>    Suggested fix    Add Upstream-Status: <status> to the header of meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch (possible values: Pending, Submitted, Accepted, Backport, Denied, Inappropriate)
> 

I have:
    Upstream-status: Inappropriate [build rules rewrite in progress]

patchtest rules should be case insensitive most of the time and
particularly for the first character of words in a labels so
  [Uu]pstream-[Ss]tatus: is okay but not uPsTrEaM-sTaTuS:

Also is there a list server for privately reviewing a patch by email?
I looked at setting up patchtest locally but got distracted when the
setup seemed to take more than a few minutes. Maybe I just need
to focus for longer...


../Randy

> 
> 
> If you believe any of these test results are incorrect, please reply to the
> mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> Otherwise we would appreciate you correcting the issues and submitting a new
> version of the patchset if applicable. Please ensure you add/increment the
> version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> [PATCH v3] -> ...).
> 
> ---
> Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
> 
> 


-- 
# Randy MacLeod.  WR Linux
# Wind River an Intel Company

Re: ✗ patchtest: failure for "README.qemu: qemuppc64 is not ..." and 1 more

[Leonardo Sandoval]

On Mon, 16 Oct 2017 14:39:45 -0400
Randy MacLeod <randy.macleod@windriver.com> wrote:

> On 2017-10-16 12:00 PM, Patchwork wrote:
> > == Series Details ==
> > 
> > Series: "README.qemu: qemuppc64 is not ..." and 1 more
> > Revision: 1
> > URL   : https://patchwork.openembedded.org/series/9358/
> > State : failure
> > 
> > == Summary ==
> > 
> > 
> > Thank you for submitting this patch series to OpenEmbedded Core. This is
> > an automated response. Several tests have been executed on the proposed
> > series by patchtest resulting in the following failures:
> > 
> > 
> > 
> > * Issue             Added patch file is missing Upstream-Status in the header [test_upstream_status_presence]
> >    Suggested fix    Add Upstream-Status: <status> to the header of meta/recipes-connectivity/openssl/openssl-1.0.2l/0001-openssl-force-soft-link-to-avoid-rare-race.patch (possible values: Pending, Submitted, Accepted, Backport, Denied, Inappropriate)
> > 
> 
> I have:
>     Upstream-status: Inappropriate [build rules rewrite in progress]
> 
> patchtest rules should be case insensitive most of the time and
> particularly for the first character of words in a labels so
>   [Uu]pstream-[Ss]tatus: is okay but not uPsTrEaM-sTaTuS:

This is a topic what have been discussed before and agreed to be strict and make it case sensitive. One of the reason (perhaps not the strongest) for the latter is that it makes much easier the scripting part but there may be other reasons that I can recall.

> 
> Also is there a list server for privately reviewing a patch by email?
> I looked at setting up patchtest locally but got distracted when the
> setup seemed to take more than a few minutes. Maybe I just need
> to focus for longer...

on your current branch, just run the patchtest script located at scripts/contrib/, that would provide basically the same log as the one gotten from patchwork.

> 
> 
> ../Randy
> 
> > 
> > 
> > If you believe any of these test results are incorrect, please reply to the
> > mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> > Otherwise we would appreciate you correcting the issues and submitting a new
> > version of the patchset if applicable. Please ensure you add/increment the
> > version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> > [PATCH v3] -> ...).
> > 
> > ---
> > Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> > Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
> > 
> > 
> 
> 
> -- 
> # Randy MacLeod.  WR Linux
> # Wind River an Intel Company
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


-- 
Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>

Re: ✗ patchtest: failure for "README.qemu: qemuppc64 is not ..." and 1 more

[Leonardo Sandoval]

> 
> > 
> > Also is there a list server for privately reviewing a patch by email?
> > I looked at setting up patchtest locally but got distracted when the
> > setup seemed to take more than a few minutes. Maybe I just need
> > to focus for longer...
> 
> on your current branch, just run the patchtest script located at scripts/contrib/, that would provide basically the same log as the one gotten from patchwork.

Randy,

just test the patchtest script under contrib and I see some exceptions coming from the execution which turn into failures and false positives. this script creates a virtual environment then clones repos and start testing patches starting from master to HEAD. This virtual environment is what is causing problems. I will fix it soon but for the time being a server is doing the test for you :)

Leo

> 
> > 
> > 
> > ../Randy
> > 
> > > 
> > > 
> > > If you believe any of these test results are incorrect, please reply to the
> > > mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
> > > Otherwise we would appreciate you correcting the issues and submitting a new
> > > version of the patchset if applicable. Please ensure you add/increment the
> > > version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
> > > [PATCH v3] -> ...).
> > > 
> > > ---
> > > Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
> > > Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
> > > 
> > > 
> > 
> > 
> > -- 
> > # Randy MacLeod.  WR Linux
> > # Wind River an Intel Company
> > -- 
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 
> -- 
> Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
> -- 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


-- 
Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>