Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH] package/python-numpy: fix occasional build failure with lapack
@ 2019-05-15 14:41 Giulio Benetti
  2019-05-15 15:12 ` Yann E. MORIN
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-15 14:41 UTC (permalink / raw)
  To: buildroot

python-numpy build fails only if lapack is built before python-numpy
itself, and this doesn't always happen because lapack dependency is
missing in BR2_PYTHON_NUMPY_DEPENDENCIES.
Then build failure is due to missing BR2_PACKAGE_LAPACK_COMPLEX that
provides some functions in lapack libraries needed by python-numpy.

So:
- add lapack to BR2_PYTHON_NUMPY_DEPENDENCIES when
  BR2_PACKAGE_LAPACK = y
- substitute ifeq check "$(BR2_PACKAGE_LAPACK),y" with
  "$(BR2_PACKAGE_LAPACK_COMPLEX),y" because python-numpy needs COMPLEX
  functions and BR2_PACKAGE_LAPACK_COMPLEX inherits BR2_PACKAGE_LAPACK

Fixes:
http://autobuild.buildroot.net/results/50f/50f7f09a9f830cd7b94f8fc83c09fc3d39297d3d/

Signed-off-by: Giulio Benetti <giulio.benetti@micronovasrl.com>
---
 package/python-numpy/python-numpy.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/python-numpy/python-numpy.mk b/package/python-numpy/python-numpy.mk
index 28dccf8be5..a61246615a 100644
--- a/package/python-numpy/python-numpy.mk
+++ b/package/python-numpy/python-numpy.mk
@@ -15,8 +15,8 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
 			numpy/core/src/multiarray/dragon4.c
 PYTHON_NUMPY_SETUP_TYPE = setuptools
 
-ifeq ($(BR2_PACKAGE_CLAPACK),y)
-PYTHON_NUMPY_DEPENDENCIES += clapack
+ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
+PYTHON_NUMPY_DEPENDENCIES += clapack lapack
 PYTHON_NUMPY_SITE_CFG_LIBS += blas lapack
 else
 PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH] package/python-numpy: fix occasional build failure with lapack
  2019-05-15 14:41 [Buildroot] [PATCH] package/python-numpy: fix occasional build failure with lapack Giulio Benetti
@ 2019-05-15 15:12 ` Yann E. MORIN
  2019-05-15 19:58   ` Giulio Benetti
  0 siblings, 1 reply; 16+ messages in thread
From: Yann E. MORIN @ 2019-05-15 15:12 UTC (permalink / raw)
  To: buildroot

Giulio, All,

On 2019-05-15 16:41 +0200, Giulio Benetti spake thusly:
> python-numpy build fails only if lapack is built before python-numpy
> itself, and this doesn't always happen because lapack dependency is
> missing in BR2_PYTHON_NUMPY_DEPENDENCIES.
> Then build failure is due to missing BR2_PACKAGE_LAPACK_COMPLEX that
> provides some functions in lapack libraries needed by python-numpy.
> 
> So:
> - add lapack to BR2_PYTHON_NUMPY_DEPENDENCIES when
>   BR2_PACKAGE_LAPACK = y
> - substitute ifeq check "$(BR2_PACKAGE_LAPACK),y" with
>   "$(BR2_PACKAGE_LAPACK_COMPLEX),y" because python-numpy needs COMPLEX
>   functions and BR2_PACKAGE_LAPACK_COMPLEX inherits BR2_PACKAGE_LAPACK
> 
> Fixes:
> http://autobuild.buildroot.net/results/50f/50f7f09a9f830cd7b94f8fc83c09fc3d39297d3d/
> 
> Signed-off-by: Giulio Benetti <giulio.benetti@micronovasrl.com>
> ---
>  package/python-numpy/python-numpy.mk | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/package/python-numpy/python-numpy.mk b/package/python-numpy/python-numpy.mk
> index 28dccf8be5..a61246615a 100644
> --- a/package/python-numpy/python-numpy.mk
> +++ b/package/python-numpy/python-numpy.mk
> @@ -15,8 +15,8 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
>  			numpy/core/src/multiarray/dragon4.c
>  PYTHON_NUMPY_SETUP_TYPE = setuptools
>  
> -ifeq ($(BR2_PACKAGE_CLAPACK),y)
> -PYTHON_NUMPY_DEPENDENCIES += clapack
> +ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
> +PYTHON_NUMPY_DEPENDENCIES += clapack lapack

This is not correct, because tehre is no relation between lapack and
clapack. So, we can have lapack enabled but not clapack. Your code will
cause a build failure when clapack is not enabled.

So, you probably want something like:

    ifeq ($(BR2_PACKAGE_CLAPACK),y)
    PYTHON_NUMPY_DEPENDENCIES_LAPACK += clapack
    endif
    ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
    PYTHON_NUMPY_DEPENDENCIES_LAPACK += lapack
    endif
    ifneq ($(PYTHON_NUMPY_DEPENDENCIES_LAPACK),)
    PYTHON_NUMPY_DEPENDENCIES += $(PYTHON_NUMPY_DEPENDENCIES_LAPACK)
    else
    PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
    endif

But beware of the above, it might not yet be correct: if clapack and
lapack (without complex) are both enabled, then you may still en up in
the current situation.

So, you may need to refine it even further, with something like:

    ifeq ($(BR2_PACKAGE_CLAPACK),y)
    PYTHON_NUMPY_DEPENDENCIES += clapack
    PYTHON_NUMPY_ENV += BLAS=clapack LAPACK=clapack
    else ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
    PYTHON_NUMPY_DEPENDENCIES += lapack
    PYTHON_NUMPY_ENV += BLAS=lapack LAPACK=lapack
    else
    PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
    endif

(check what BLAS= and LAPACK= expect as values.)

Also, is the depenency on clapack really needed? Can python-numpy really
use clapack?

Regards,
Yann E. MORIN.

>  PYTHON_NUMPY_SITE_CFG_LIBS += blas lapack
>  else
>  PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
> -- 
> 2.17.1
> 

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH] package/python-numpy: fix occasional build failure with lapack
  2019-05-15 15:12 ` Yann E. MORIN
@ 2019-05-15 19:58   ` Giulio Benetti
  2019-05-15 20:50     ` Giulio Benetti
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-15 19:58 UTC (permalink / raw)
  To: buildroot

Hello Yann,

Il 15/05/2019 17:12, Yann E. MORIN ha scritto:
> Giulio, All,
> 
> On 2019-05-15 16:41 +0200, Giulio Benetti spake thusly:
>> python-numpy build fails only if lapack is built before python-numpy
>> itself, and this doesn't always happen because lapack dependency is
>> missing in BR2_PYTHON_NUMPY_DEPENDENCIES.
>> Then build failure is due to missing BR2_PACKAGE_LAPACK_COMPLEX that
>> provides some functions in lapack libraries needed by python-numpy.
>>
>> So:
>> - add lapack to BR2_PYTHON_NUMPY_DEPENDENCIES when
>>    BR2_PACKAGE_LAPACK = y
>> - substitute ifeq check "$(BR2_PACKAGE_LAPACK),y" with
>>    "$(BR2_PACKAGE_LAPACK_COMPLEX),y" because python-numpy needs COMPLEX
>>    functions and BR2_PACKAGE_LAPACK_COMPLEX inherits BR2_PACKAGE_LAPACK
>>
>> Fixes:
>> http://autobuild.buildroot.net/results/50f/50f7f09a9f830cd7b94f8fc83c09fc3d39297d3d/
>>
>> Signed-off-by: Giulio Benetti <giulio.benetti@micronovasrl.com>
>> ---
>>   package/python-numpy/python-numpy.mk | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/package/python-numpy/python-numpy.mk b/package/python-numpy/python-numpy.mk
>> index 28dccf8be5..a61246615a 100644
>> --- a/package/python-numpy/python-numpy.mk
>> +++ b/package/python-numpy/python-numpy.mk
>> @@ -15,8 +15,8 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
>>   			numpy/core/src/multiarray/dragon4.c
>>   PYTHON_NUMPY_SETUP_TYPE = setuptools
>>   
>> -ifeq ($(BR2_PACKAGE_CLAPACK),y)
>> -PYTHON_NUMPY_DEPENDENCIES += clapack
>> +ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
>> +PYTHON_NUMPY_DEPENDENCIES += clapack lapack
> 
> This is not correct, because tehre is no relation between lapack and
> clapack. So, we can have lapack enabled but not clapack. Your code will
> cause a build failure when clapack is not enabled.

Right, they should be exclusive.

> So, you probably want something like:
> 
>      ifeq ($(BR2_PACKAGE_CLAPACK),y)
>      PYTHON_NUMPY_DEPENDENCIES_LAPACK += clapack
>      endif
>      ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
>      PYTHON_NUMPY_DEPENDENCIES_LAPACK += lapack
>      endif
>      ifneq ($(PYTHON_NUMPY_DEPENDENCIES_LAPACK),)
>      PYTHON_NUMPY_DEPENDENCIES += $(PYTHON_NUMPY_DEPENDENCIES_LAPACK)
>      else
>      PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
>      endif
> 
> But beware of the above, it might not yet be correct: if clapack and
> lapack (without complex) are both enabled, then you may still en up in
> the current situation.

Right.

> So, you may need to refine it even further, with something like:
> 
>      ifeq ($(BR2_PACKAGE_CLAPACK),y)
>      PYTHON_NUMPY_DEPENDENCIES += clapack
>      PYTHON_NUMPY_ENV += BLAS=clapack LAPACK=clapack
>      else ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
>      PYTHON_NUMPY_DEPENDENCIES += lapack
>      PYTHON_NUMPY_ENV += BLAS=lapack LAPACK=lapack
>      else
>      PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
>      endif
> 
> (check what BLAS= and LAPACK= expect as values.)
> 
> Also, is the depenency on clapack really needed? Can python-numpy really
> use clapack?

Judging from here:
https://www.numpy.org/devdocs/user/building.html#blas
clapack seems to not be supported.
But AIK clapack should be the same as lapack but written entirely in C 
instead of Fortran(later translated in C in lapack during build), so it 
should work with clapack too.
I try to build with that and check.

Thanks for reviewing and proposing alternatives.

Kind regards
-- 
Giulio Benetti
CTO

MICRONOVA SRL
Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
Tel. 049/8931563 - Fax 049/8931346
Cod.Fiscale - P.IVA 02663420285
Capitale Sociale ? 26.000 i.v.
Iscritta al Reg. Imprese di Padova N. 02663420285
Numero R.E.A. 258642

> Regards,
> Yann E. MORIN.
> 
>>   PYTHON_NUMPY_SITE_CFG_LIBS += blas lapack
>>   else
>>   PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
>> -- 
>> 2.17.1
>>
> 

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH] package/python-numpy: fix occasional build failure with lapack
  2019-05-15 19:58   ` Giulio Benetti
@ 2019-05-15 20:50     ` Giulio Benetti
  2019-05-15 21:03       ` [Buildroot] [PATCH v2] " Giulio Benetti
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-15 20:50 UTC (permalink / raw)
  To: buildroot

Il 15/05/2019 21:58, Giulio Benetti ha scritto:
> Hello Yann,
> 
> Il 15/05/2019 17:12, Yann E. MORIN ha scritto:
>> Giulio, All,
>>
>> On 2019-05-15 16:41 +0200, Giulio Benetti spake thusly:
>>> python-numpy build fails only if lapack is built before python-numpy
>>> itself, and this doesn't always happen because lapack dependency is
>>> missing in BR2_PYTHON_NUMPY_DEPENDENCIES.
>>> Then build failure is due to missing BR2_PACKAGE_LAPACK_COMPLEX that
>>> provides some functions in lapack libraries needed by python-numpy.
>>>
>>> So:
>>> - add lapack to BR2_PYTHON_NUMPY_DEPENDENCIES when
>>>     BR2_PACKAGE_LAPACK = y
>>> - substitute ifeq check "$(BR2_PACKAGE_LAPACK),y" with
>>>     "$(BR2_PACKAGE_LAPACK_COMPLEX),y" because python-numpy needs COMPLEX
>>>     functions and BR2_PACKAGE_LAPACK_COMPLEX inherits BR2_PACKAGE_LAPACK
>>>
>>> Fixes:
>>> http://autobuild.buildroot.net/results/50f/50f7f09a9f830cd7b94f8fc83c09fc3d39297d3d/
>>>
>>> Signed-off-by: Giulio Benetti <giulio.benetti@micronovasrl.com>
>>> ---
>>>    package/python-numpy/python-numpy.mk | 4 ++--
>>>    1 file changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/package/python-numpy/python-numpy.mk b/package/python-numpy/python-numpy.mk
>>> index 28dccf8be5..a61246615a 100644
>>> --- a/package/python-numpy/python-numpy.mk
>>> +++ b/package/python-numpy/python-numpy.mk
>>> @@ -15,8 +15,8 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
>>>    			numpy/core/src/multiarray/dragon4.c
>>>    PYTHON_NUMPY_SETUP_TYPE = setuptools
>>>    
>>> -ifeq ($(BR2_PACKAGE_CLAPACK),y)
>>> -PYTHON_NUMPY_DEPENDENCIES += clapack
>>> +ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
>>> +PYTHON_NUMPY_DEPENDENCIES += clapack lapack
>>
>> This is not correct, because tehre is no relation between lapack and
>> clapack. So, we can have lapack enabled but not clapack. Your code will
>> cause a build failure when clapack is not enabled.
> 
> Right, they should be exclusive.
> 
>> So, you probably want something like:
>>
>>       ifeq ($(BR2_PACKAGE_CLAPACK),y)
>>       PYTHON_NUMPY_DEPENDENCIES_LAPACK += clapack
>>       endif
>>       ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
>>       PYTHON_NUMPY_DEPENDENCIES_LAPACK += lapack
>>       endif
>>       ifneq ($(PYTHON_NUMPY_DEPENDENCIES_LAPACK),)
>>       PYTHON_NUMPY_DEPENDENCIES += $(PYTHON_NUMPY_DEPENDENCIES_LAPACK)
>>       else
>>       PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
>>       endif
>>
>> But beware of the above, it might not yet be correct: if clapack and
>> lapack (without complex) are both enabled, then you may still en up in
>> the current situation.
> 
> Right.
> 
>> So, you may need to refine it even further, with something like:
>>
>>       ifeq ($(BR2_PACKAGE_CLAPACK),y)
>>       PYTHON_NUMPY_DEPENDENCIES += clapack
>>       PYTHON_NUMPY_ENV += BLAS=clapack LAPACK=clapack
>>       else ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
>>       PYTHON_NUMPY_DEPENDENCIES += lapack
>>       PYTHON_NUMPY_ENV += BLAS=lapack LAPACK=lapack
>>       else
>>       PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
>>       endif
>>
>> (check what BLAS= and LAPACK= expect as values.)
>>
>> Also, is the depenency on clapack really needed? Can python-numpy really
>> use clapack?
> 
> Judging from here:
> https://www.numpy.org/devdocs/user/building.html#blas
> clapack seems to not be supported.
> But AIK clapack should be the same as lapack but written entirely in C
> instead of Fortran(later translated in C in lapack during build), so it
> should work with clapack too.
> I try to build with that and check.

I've successfully built python-numpy with these accelerators:
- lapack
- clapack(c verion of lapack(fortran))
- openblas

Speaking in IRC with Yann came out the possibility to specify LAPACK 
order with NPY_LAPACK_ORDER.

Another thing that could be done(after fixing current patch) is to add 
the choice in Config.in of which accelerator to use between these:
- lapack
- clapack
- openblas

IMHO I would like to impose LAPACK and BLAS instead of setting 
NPY_LAPACK_ORDER and NPY_BLAS_ORDER, since Buildroot is meant to be 
kinda "static" after building.

So I would:
- send a V2 patch to fix build failure using lapack only
then:
- follow with a second patch adding such choice

What about this?

Kind regards
-- 
Giulio Benetti
CTO

MICRONOVA SRL
Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
Tel. 049/8931563 - Fax 049/8931346
Cod.Fiscale - P.IVA 02663420285
Capitale Sociale ? 26.000 i.v.
Iscritta al Reg. Imprese di Padova N. 02663420285
Numero R.E.A. 258642

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-15 20:50     ` Giulio Benetti
@ 2019-05-15 21:03       ` Giulio Benetti
  2019-05-18 20:13         ` Thomas Petazzoni
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-15 21:03 UTC (permalink / raw)
  To: buildroot

python-numpy build fails only if lapack is built before python-numpy
itself, and this doesn't always happen because lapack dependency is
missing in BR2_PYTHON_NUMPY_DEPENDENCIES. clapack is present
instead, but it's wrong since we're checking $(BR2_PACKAGE_LAPACK) not
$(BR2_PACKAGE_CLAPACK). Once clapack is substituted with lapack build
failure is due to missing BR2_PACKAGE_LAPACK_COMPLEX that
provides some functions in lapack libraries needed by python-numpy.

So:
- substitute clapack with lapack in BR2_PYTHON_NUMPY_DEPENDENCIES when
  BR2_PACKAGE_LAPACK = y
- substitute ifeq check "$(BR2_PACKAGE_LAPACK),y" with
  "$(BR2_PACKAGE_LAPACK_COMPLEX),y" because python-numpy needs COMPLEX
  functions and BR2_PACKAGE_LAPACK_COMPLEX inherits BR2_PACKAGE_LAPACK

Fixes:
http://autobuild.buildroot.net/results/50f/50f7f09a9f830cd7b94f8fc83c09fc3d39297d3d/

Signed-off-by: Giulio Benetti <giulio.benetti@micronovasrl.com>
---
 package/python-numpy/python-numpy.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/python-numpy/python-numpy.mk b/package/python-numpy/python-numpy.mk
index 28dccf8be5..af210041a1 100644
--- a/package/python-numpy/python-numpy.mk
+++ b/package/python-numpy/python-numpy.mk
@@ -15,8 +15,8 @@ PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
 			numpy/core/src/multiarray/dragon4.c
 PYTHON_NUMPY_SETUP_TYPE = setuptools
 
-ifeq ($(BR2_PACKAGE_CLAPACK),y)
-PYTHON_NUMPY_DEPENDENCIES += clapack
+ifeq ($(BR2_PACKAGE_LAPACK_COMPLEX),y)
+PYTHON_NUMPY_DEPENDENCIES += lapack
 PYTHON_NUMPY_SITE_CFG_LIBS += blas lapack
 else
 PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-15 21:03       ` [Buildroot] [PATCH v2] " Giulio Benetti
@ 2019-05-18 20:13         ` Thomas Petazzoni
  2019-05-19 14:47           ` Giulio Benetti
  0 siblings, 1 reply; 16+ messages in thread
From: Thomas Petazzoni @ 2019-05-18 20:13 UTC (permalink / raw)
  To: buildroot

Hello,

On Wed, 15 May 2019 23:03:42 +0200
Giulio Benetti <giulio.benetti@micronovasrl.com> wrote:

> python-numpy build fails only if lapack is built before python-numpy
> itself, and this doesn't always happen because lapack dependency is
> missing in BR2_PYTHON_NUMPY_DEPENDENCIES. clapack is present
> instead, but it's wrong since we're checking $(BR2_PACKAGE_LAPACK) not
> $(BR2_PACKAGE_CLAPACK).

I don't follow here: in the current code, we're checking
BR2_PACKAGE_CLAPACK and adding clapack to PYTHON_NUMPY_DEPENDENCIES,
which is consistent.

In order to better understand the problem. Without this patch:

 - Does python-numpy builds fine when neither clapack nor lapack are
   enabled ?

 - Does python-numpy builds fine when clapack is enabled, but not
   lapack ?

 - What happens when both clapack and lapack are enabled ?

Also, perhaps we need:

	select BR2_PACKAGE_LAPACK_COMPLEX if BR2_PACKAGE_LAPACK

in the Config.in file, so that it's a bit easier for users: they don't
have to know they need that specific sub-option of lapack.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-18 20:13         ` Thomas Petazzoni
@ 2019-05-19 14:47           ` Giulio Benetti
  2019-05-20 17:48             ` Giulio Benetti
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-19 14:47 UTC (permalink / raw)
  To: buildroot

Hello Thomas, All,

Il 18/05/2019 22:13, Thomas Petazzoni ha scritto:
> Hello,
> 
> On Wed, 15 May 2019 23:03:42 +0200
> Giulio Benetti <giulio.benetti@micronovasrl.com> wrote:
> 
>> python-numpy build fails only if lapack is built before python-numpy
>> itself, and this doesn't always happen because lapack dependency is
>> missing in BR2_PYTHON_NUMPY_DEPENDENCIES. clapack is present
>> instead, but it's wrong since we're checking $(BR2_PACKAGE_LAPACK) not
>> $(BR2_PACKAGE_CLAPACK).
> 
> I don't follow here: in the current code, we're checking
> BR2_PACKAGE_CLAPACK and adding clapack to PYTHON_NUMPY_DEPENDENCIES,
> which is consistent.

You're right, I've done a mistake, I've substituted clapack with lapack.
So need to send a v3 with this corrected.

The problem was due to cohexistence of lapack and clapack, but they are 
both(like openblas) accelerators that can exist once(I mean or lapack, 
or clapack).
Here I was asking about inserting a menu to choose which accelerator to use:
http://lists.busybox.net/pipermail/buildroot/2019-May/250273.html

Can you take a look and answer with your opinion?

> In order to better understand the problem. Without this patch:
> 
>   - Does python-numpy builds fine when neither clapack nor lapack are
>     enabled ?

Yes it does(it works without accelerators).

>   - Does python-numpy builds fine when clapack is enabled, but not
>     lapack ?

Yes

>   - What happens when both clapack and lapack are enabled ?

Python-numpy will choose the accelerator according to this list:
https://www.numpy.org/devdocs/user/building.html#accelerated-blas-lapack-libraries

> Also, perhaps we need:
> 
> 	select BR2_PACKAGE_LAPACK_COMPLEX if BR2_PACKAGE_LAPACK
> 
> in the Config.in file, so that it's a bit easier for users: they don't
> have to know they need that specific sub-option of lapack.

If lapack selected yes, but that should come after adding a choice menu 
IMHO.

Kind regards
-- 
Giulio Benetti
CTO

MICRONOVA SRL
Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
Tel. 049/8931563 - Fax 049/8931346
Cod.Fiscale - P.IVA 02663420285
Capitale Sociale ? 26.000 i.v.
Iscritta al Reg. Imprese di Padova N. 02663420285
Numero R.E.A. 258642

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-19 14:47           ` Giulio Benetti
@ 2019-05-20 17:48             ` Giulio Benetti
  2019-05-20 17:49               ` Giulio Benetti
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-20 17:48 UTC (permalink / raw)
  To: buildroot

Hello Thomas,

Il 19/05/2019 16:47, Giulio Benetti ha scritto:
> Hello Thomas, All,
> 
> Il 18/05/2019 22:13, Thomas Petazzoni ha scritto:
>> Hello,
>>
>> On Wed, 15 May 2019 23:03:42 +0200
>> Giulio Benetti <giulio.benetti@micronovasrl.com> wrote:
>>
>>> python-numpy build fails only if lapack is built before python-numpy
>>> itself, and this doesn't always happen because lapack dependency is
>>> missing in BR2_PYTHON_NUMPY_DEPENDENCIES. clapack is present
>>> instead, but it's wrong since we're checking $(BR2_PACKAGE_LAPACK) not
>>> $(BR2_PACKAGE_CLAPACK).
>>
>> I don't follow here: in the current code, we're checking
>> BR2_PACKAGE_CLAPACK and adding clapack to PYTHON_NUMPY_DEPENDENCIES,
>> which is consistent.
> 
> You're right, I've done a mistake, I've substituted clapack with lapack.
> So need to send a v3 with this corrected.
> 
> The problem was due to cohexistence of lapack and clapack, but they are 
> both(like openblas) accelerators that can exist once(I mean or lapack, 
> or clapack).
> Here I was asking about inserting a menu to choose which accelerator to 
> use:
> http://lists.busybox.net/pipermail/buildroot/2019-May/250273.html
> 
> Can you take a look and answer with your opinion?
> 
>> In order to better understand the problem. Without this patch:
>>
>> ? - Does python-numpy builds fine when neither clapack nor lapack are
>> ??? enabled ?
> 
> Yes it does(it works without accelerators).
> 
>> ? - Does python-numpy builds fine when clapack is enabled, but not
>> ??? lapack ?
> 
> Yes
> 
>> ? - What happens when both clapack and lapack are enabled ?
> 
> Python-numpy will choose the accelerator according to this list:
> https://www.numpy.org/devdocs/user/building.html#accelerated-blas-lapack-libraries 

This ^^^^^^^^^^^^^ is wrong.
The problem is that both lapack and clapack library names are equal:
liblapack.so

So what happens here is that python-numpy will use clapack headers,
then it will link with liblapack.so causing linking error.
Because liblapack.so is from lapack(not clapack) due to alphabetical
order when building.

Basically clapack is the c implementation of lapack, but in lapack it's
been implemented lapacke(a c interface) that makes clapack useless at
this point.

I see 2 options:
- remove clapack completely because it's the same as lapacke(embedded in
lapack)
- make clapack and lapack exclusive selectable in their Config.in like:
   (clapack Config.in)
   depends on !BR2_PACKAGE_LAPACK
   and:
   (lapack Config.in)
   depends on !BR2_PACKAGE_CLAPACK

But IMHO we should proceed with dropping clapack.

> 
>> Also, perhaps we need:
>>
>> ????select BR2_PACKAGE_LAPACK_COMPLEX if BR2_PACKAGE_LAPACK

This ^^^^^^^^^^ instead never built succesfully.
It built successfully only because I was trying to build with:
make python-numpy-dirclean python-numpy
instead of:
make clean all

So I had a mess of LAPACK libraries in my environment(sorry).

Best regards--
Giulio Benetti
CTO

MICRONOVA SRL
Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
Tel. 049/8931563 - Fax 049/8931346
Cod.Fiscale - P.IVA 02663420285
Capitale Sociale ? 26.000 i.v.
Iscritta al Reg. Imprese di Padova N. 02663420285
Numero R.E.A. 258642

>> in the Config.in file, so that it's a bit easier for users: they don't
>> have to know they need that specific sub-option of lapack.
> 
> If lapack selected yes, but that should come after adding a choice menu 
> IMHO.
> 
> Kind regards

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-20 17:48             ` Giulio Benetti
@ 2019-05-20 17:49               ` Giulio Benetti
  2019-05-26  9:43                 ` Arnout Vandecappelle
  0 siblings, 1 reply; 16+ messages in thread
From: Giulio Benetti @ 2019-05-20 17:49 UTC (permalink / raw)
  To: buildroot

Adding Samuel+,

Il 20/05/2019 19:48, Giulio Benetti ha scritto:
> Hello Thomas,
> 
> Il 19/05/2019 16:47, Giulio Benetti ha scritto:
>> Hello Thomas, All,
>>
>> Il 18/05/2019 22:13, Thomas Petazzoni ha scritto:
>>> Hello,
>>>
>>> On Wed, 15 May 2019 23:03:42 +0200
>>> Giulio Benetti <giulio.benetti@micronovasrl.com> wrote:
>>>
>>>> python-numpy build fails only if lapack is built before python-numpy
>>>> itself, and this doesn't always happen because lapack dependency is
>>>> missing in BR2_PYTHON_NUMPY_DEPENDENCIES. clapack is present
>>>> instead, but it's wrong since we're checking $(BR2_PACKAGE_LAPACK) not
>>>> $(BR2_PACKAGE_CLAPACK).
>>>
>>> I don't follow here: in the current code, we're checking
>>> BR2_PACKAGE_CLAPACK and adding clapack to PYTHON_NUMPY_DEPENDENCIES,
>>> which is consistent.
>>
>> You're right, I've done a mistake, I've substituted clapack with lapack.
>> So need to send a v3 with this corrected.
>>
>> The problem was due to cohexistence of lapack and clapack, but they 
>> are both(like openblas) accelerators that can exist once(I mean or 
>> lapack, or clapack).
>> Here I was asking about inserting a menu to choose which accelerator 
>> to use:
>> http://lists.busybox.net/pipermail/buildroot/2019-May/250273.html
>>
>> Can you take a look and answer with your opinion?
>>
>>> In order to better understand the problem. Without this patch:
>>>
>>> ? - Does python-numpy builds fine when neither clapack nor lapack are
>>> ??? enabled ?
>>
>> Yes it does(it works without accelerators).
>>
>>> ? - Does python-numpy builds fine when clapack is enabled, but not
>>> ??? lapack ?
>>
>> Yes
>>
>>> ? - What happens when both clapack and lapack are enabled ?
>>
>> Python-numpy will choose the accelerator according to this list:
>> https://www.numpy.org/devdocs/user/building.html#accelerated-blas-lapack-libraries 
> 
> 
> This ^^^^^^^^^^^^^ is wrong.
> The problem is that both lapack and clapack library names are equal:
> liblapack.so
> 
> So what happens here is that python-numpy will use clapack headers,
> then it will link with liblapack.so causing linking error.
> Because liblapack.so is from lapack(not clapack) due to alphabetical
> order when building.
> 
> Basically clapack is the c implementation of lapack, but in lapack it's
> been implemented lapacke(a c interface) that makes clapack useless at
> this point.
> 
> I see 2 options:
> - remove clapack completely because it's the same as lapacke(embedded in
> lapack)
> - make clapack and lapack exclusive selectable in their Config.in like:
>  ? (clapack Config.in)
>  ? depends on !BR2_PACKAGE_LAPACK
>  ? and:
>  ? (lapack Config.in)
>  ? depends on !BR2_PACKAGE_CLAPACK
> 
> But IMHO we should proceed with dropping clapack.
> 
>>
>>> Also, perhaps we need:
>>>
>>> ????select BR2_PACKAGE_LAPACK_COMPLEX if BR2_PACKAGE_LAPACK
> 
> This ^^^^^^^^^^ instead never built succesfully.
> It built successfully only because I was trying to build with:
> make python-numpy-dirclean python-numpy
> instead of:
> make clean all
> 
> So I had a mess of LAPACK libraries in my environment(sorry).
> 
> Best regards--
> Giulio Benetti
> CTO
> 
> MICRONOVA SRL
> Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
> Tel. 049/8931563 - Fax 049/8931346
> Cod.Fiscale - P.IVA 02663420285
> Capitale Sociale ? 26.000 i.v.
> Iscritta al Reg. Imprese di Padova N. 02663420285
> Numero R.E.A. 258642
> 
>>> in the Config.in file, so that it's a bit easier for users: they don't
>>> have to know they need that specific sub-option of lapack.
>>
>> If lapack selected yes, but that should come after adding a choice 
>> menu IMHO.
>>
>> Kind regards
> 
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

-- 
Giulio Benetti
CTO

MICRONOVA SRL
Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
Tel. 049/8931563 - Fax 049/8931346
Cod.Fiscale - P.IVA 02663420285
Capitale Sociale ? 26.000 i.v.
Iscritta al Reg. Imprese di Padova N. 02663420285
Numero R.E.A. 258642

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-20 17:49               ` Giulio Benetti
@ 2019-05-26  9:43                 ` Arnout Vandecappelle
  2019-05-26 10:11                   ` Giulio Benetti
  2019-05-26 11:43                   ` Peter Korsgaard
  0 siblings, 2 replies; 16+ messages in thread
From: Arnout Vandecappelle @ 2019-05-26  9:43 UTC (permalink / raw)
  To: buildroot



On 20/05/2019 19:49, Giulio Benetti wrote:
> Adding Samuel+,
> 
> Il 20/05/2019 19:48, Giulio Benetti ha scritto:
>> Hello Thomas,
>>
>> Il 19/05/2019 16:47, Giulio Benetti ha scritto:
>>> Hello Thomas, All,
>>>
>>> Il 18/05/2019 22:13, Thomas Petazzoni ha scritto:
[snip]
>>>> ? - What happens when both clapack and lapack are enabled ?
>>>
>>> Python-numpy will choose the accelerator according to this list:
>>> https://www.numpy.org/devdocs/user/building.html#accelerated-blas-lapack-libraries
>> This ^^^^^^^^^^^^^ is wrong.
>> The problem is that both lapack and clapack library names are equal:
>> liblapack.so

 So, you are saying that if you build both lapack and clapack, you get only one
.so file?


>> So what happens here is that python-numpy will use clapack headers,
>> then it will link with liblapack.so causing linking error.
>> Because liblapack.so is from lapack(not clapack) due to alphabetical
>> order when building.
>>
>> Basically clapack is the c implementation of lapack, but in lapack it's
>> been implemented lapacke(a c interface) that makes clapack useless at
>> this point.
>>
>> I see 2 options:
>> - remove clapack completely because it's the same as lapacke(embedded in
>> lapack)
>> - make clapack and lapack exclusive selectable in their Config.in like:
>> ?? (clapack Config.in)
>> ?? depends on !BR2_PACKAGE_LAPACK
>> ?? and:
>> ?? (lapack Config.in)
>> ?? depends on !BR2_PACKAGE_CLAPACK

 We indeed have to make the mutually exclusive. There are a number of ways that
we can do that.

A. Add a dependency (not like above because it's a circular dependency).

B. Turn it into a virtual package.

C. Turn it into a choice (in addition to making it a virtual package).

 The latter is probably the proper way. Cfr. how it was done for
openssl/libressl and libjpeg/turbo-jpeg.


>> But IMHO we should proceed with dropping clapack.

 Why? it is maintained, and it is a dependency of armadillo...

 I had a deeper look, and actually lapack and clapack are the same, just
compiled differently: lapack is built with the fortran compiler, clapack is
generated with f2c (by upstream) and built with the C compiler. AFAIU, anything
that uses lapack should also be able to use clapack and vice versa.

 We have two packages like that: numpy and armadillo. They both seem to support
both lapack and clapack.

 So, it seems indeed removing clapack is the appropriate thing. Except: lapack
requires a fortran compiler, which is not always available. So in that sense, it
would actually be more appropriate to remove lapack...

 But maybe it would be better to normally use lapack, and only enable clapack
when lapack isn't available (i.e. when there's no fortran compiler).

 Regards,
 Arnout

>>>> Also, perhaps we need:
>>>>
>>>> ????select BR2_PACKAGE_LAPACK_COMPLEX if BR2_PACKAGE_LAPACK
>>
>> This ^^^^^^^^^^ instead never built succesfully.
>> It built successfully only because I was trying to build with:
>> make python-numpy-dirclean python-numpy
>> instead of:
>> make clean all
>>
>> So I had a mess of LAPACK libraries in my environment(sorry).
>>
>> Best regards--
>> Giulio Benetti
>> CTO
>>
>> MICRONOVA SRL
>> Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
>> Tel. 049/8931563 - Fax 049/8931346
>> Cod.Fiscale - P.IVA 02663420285
>> Capitale Sociale ? 26.000 i.v.
>> Iscritta al Reg. Imprese di Padova N. 02663420285
>> Numero R.E.A. 258642
>>
>>>> in the Config.in file, so that it's a bit easier for users: they don't
>>>> have to know they need that specific sub-option of lapack.
>>>
>>> If lapack selected yes, but that should come after adding a choice menu IMHO.
>>>
>>> Kind regards
>>
>> _______________________________________________
>> buildroot mailing list
>> buildroot at busybox.net
>> http://lists.busybox.net/mailman/listinfo/buildroot
> 

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-26  9:43                 ` Arnout Vandecappelle
@ 2019-05-26 10:11                   ` Giulio Benetti
  2019-05-26 11:43                   ` Peter Korsgaard
  1 sibling, 0 replies; 16+ messages in thread
From: Giulio Benetti @ 2019-05-26 10:11 UTC (permalink / raw)
  To: buildroot

Hello Arnout,

readding Samuel+,

Il 26/05/2019 11:43, Arnout Vandecappelle ha scritto:
> 
> 
> On 20/05/2019 19:49, Giulio Benetti wrote:
>> Adding Samuel+,
>>
>> Il 20/05/2019 19:48, Giulio Benetti ha scritto:
>>> Hello Thomas,
>>>
>>> Il 19/05/2019 16:47, Giulio Benetti ha scritto:
>>>> Hello Thomas, All,
>>>>
>>>> Il 18/05/2019 22:13, Thomas Petazzoni ha scritto:
> [snip]
>>>>>  ? - What happens when both clapack and lapack are enabled ?
>>>>
>>>> Python-numpy will choose the accelerator according to this list:
>>>> https://www.numpy.org/devdocs/user/building.html#accelerated-blas-lapack-libraries
>>> This ^^^^^^^^^^^^^ is wrong.
>>> The problem is that both lapack and clapack library names are equal:
>>> liblapack.so
> 
>   So, you are saying that if you build both lapack and clapack, you get only one
> .so file?

Exactly and lapack during install overwrite clapacks's liblapack.so with 
its liblapack.so

> 
>>> So what happens here is that python-numpy will use clapack headers,
>>> then it will link with liblapack.so causing linking error.
>>> Because liblapack.so is from lapack(not clapack) due to alphabetical
>>> order when building.
>>>
>>> Basically clapack is the c implementation of lapack, but in lapack it's
>>> been implemented lapacke(a c interface) that makes clapack useless at
>>> this point.
>>>
>>> I see 2 options:
>>> - remove clapack completely because it's the same as lapacke(embedded in
>>> lapack)
>>> - make clapack and lapack exclusive selectable in their Config.in like:
>>>  ?? (clapack Config.in)
>>>  ?? depends on !BR2_PACKAGE_LAPACK
>>>  ?? and:
>>>  ?? (lapack Config.in)
>>>  ?? depends on !BR2_PACKAGE_CLAPACK
> 
>   We indeed have to make the mutually exclusive. There are a number of ways that
> we can do that.
> 
> A. Add a dependency (not like above because it's a circular dependency).
> 
> B. Turn it into a virtual package.
> 
> C. Turn it into a choice (in addition to making it a virtual package).
> 
>   The latter is probably the proper way. Cfr. how it was done for
> openssl/libressl and libjpeg/turbo-jpeg.

I agree with using option C. Otherwise there will always be problems 
like this.

> 
>>> But IMHO we should proceed with dropping clapack.
> 
>   Why? it is maintained, and it is a dependency of armadillo...
> 
>   I had a deeper look, and actually lapack and clapack are the same, just
> compiled differently: lapack is built with the fortran compiler, clapack is
> generated with f2c (by upstream) and built with the C compiler. AFAIU, anything
> that uses lapack should also be able to use clapack and vice versa.
> 
>   We have two packages like that: numpy and armadillo. They both seem to support
> both lapack and clapack.
> 
>   So, it seems indeed removing clapack is the appropriate thing. Except: lapack
> requires a fortran compiler, which is not always available. 

Yes, I thought about it too and this is why it makes more sense option C 
IMHO.

> So in that sense, it
> would actually be more appropriate to remove lapack...
> 
>   But maybe it would be better to normally use lapack, and only enable clapack
> when lapack isn't available (i.e. when there's no fortran compiler).

Yes that can be done. But at this point I would make it with option C.

Do I proceed that way?

PS: I always try to add to CC Samuel and everytime it gets removed, is 
there a reason?

Best regards
-- 
Giulio Benetti
CTO

MICRONOVA SRL
Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
Tel. 049/8931563 - Fax 049/8931346
Cod.Fiscale - P.IVA 02663420285
Capitale Sociale ? 26.000 i.v.
Iscritta al Reg. Imprese di Padova N. 02663420285
Numero R.E.A. 258642


>   Regards,
>   Arnout
> 
>>>>> Also, perhaps we need:
>>>>>
>>>>>  ????select BR2_PACKAGE_LAPACK_COMPLEX if BR2_PACKAGE_LAPACK
>>>
>>> This ^^^^^^^^^^ instead never built succesfully.
>>> It built successfully only because I was trying to build with:
>>> make python-numpy-dirclean python-numpy
>>> instead of:
>>> make clean all
>>>
>>> So I had a mess of LAPACK libraries in my environment(sorry).
>>>
>>> Best regards--
>>> Giulio Benetti
>>> CTO
>>>
>>> MICRONOVA SRL
>>> Sede: Via A. Niedda 3 - 35010 Vigonza (PD)
>>> Tel. 049/8931563 - Fax 049/8931346
>>> Cod.Fiscale - P.IVA 02663420285
>>> Capitale Sociale ? 26.000 i.v.
>>> Iscritta al Reg. Imprese di Padova N. 02663420285
>>> Numero R.E.A. 258642
>>>
>>>>> in the Config.in file, so that it's a bit easier for users: they don't
>>>>> have to know they need that specific sub-option of lapack.
>>>>
>>>> If lapack selected yes, but that should come after adding a choice menu IMHO.
>>>>
>>>> Kind regards
>>>
>>> _______________________________________________
>>> buildroot mailing list
>>> buildroot at busybox.net
>>> http://lists.busybox.net/mailman/listinfo/buildroot
>>

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-26  9:43                 ` Arnout Vandecappelle
  2019-05-26 10:11                   ` Giulio Benetti
@ 2019-05-26 11:43                   ` Peter Korsgaard
  2019-05-26 12:29                     ` Arnout Vandecappelle
  1 sibling, 1 reply; 16+ messages in thread
From: Peter Korsgaard @ 2019-05-26 11:43 UTC (permalink / raw)
  To: buildroot

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >>> But IMHO we should proceed with dropping clapack.

 >  Why? it is maintained, and it is a dependency of armadillo...

 >  I had a deeper look, and actually lapack and clapack are the same, just
 > compiled differently: lapack is built with the fortran compiler, clapack is
 > generated with f2c (by upstream) and built with the C compiler. AFAIU, anything
 > that uses lapack should also be able to use clapack and vice versa.

 >  We have two packages like that: numpy and armadillo. They both seem to support
 > both lapack and clapack.

 >  So, it seems indeed removing clapack is the appropriate thing. Except: lapack
 > requires a fortran compiler, which is not always available. So in that sense, it
 > would actually be more appropriate to remove lapack...

 >  But maybe it would be better to normally use lapack, and only enable clapack
 > when lapack isn't available (i.e. when there's no fortran compiler).

Not knowing anything about (c)lapack (or fortran), is there any
performance advantage using lapack over clapack? Otherwise just always
using clapack seems like the simplest solution.

-- 
Bye, Peter Korsgaard

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-26 11:43                   ` Peter Korsgaard
@ 2019-05-26 12:29                     ` Arnout Vandecappelle
  2019-05-28  5:34                       ` Benjamin Kamath
  2019-05-28 14:57                       ` Bernd Kuhls
  0 siblings, 2 replies; 16+ messages in thread
From: Arnout Vandecappelle @ 2019-05-26 12:29 UTC (permalink / raw)
  To: buildroot



On 26/05/2019 13:43, Peter Korsgaard wrote:
>>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
> 
> Hi,
> 
>  >>> But IMHO we should proceed with dropping clapack.
> 
>  >  Why? it is maintained, and it is a dependency of armadillo...
> 
>  >  I had a deeper look, and actually lapack and clapack are the same, just
>  > compiled differently: lapack is built with the fortran compiler, clapack is
>  > generated with f2c (by upstream) and built with the C compiler. AFAIU, anything
>  > that uses lapack should also be able to use clapack and vice versa.
> 
>  >  We have two packages like that: numpy and armadillo. They both seem to support
>  > both lapack and clapack.
> 
>  >  So, it seems indeed removing clapack is the appropriate thing. Except: lapack
>  > requires a fortran compiler, which is not always available. So in that sense, it
>  > would actually be more appropriate to remove lapack...
> 
>  >  But maybe it would be better to normally use lapack, and only enable clapack
>  > when lapack isn't available (i.e. when there's no fortran compiler).
> 
> Not knowing anything about (c)lapack (or fortran), is there any
> performance advantage using lapack over clapack? Otherwise just always
> using clapack seems like the simplest solution.

 Sounds OK to me.

 Bernd, you're the only one who ever made a real contribution to lapack. Thoughts?

 Benjamin, you originally contributed lapack. Any ideas?

 Regards,
 Arnout

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-26 12:29                     ` Arnout Vandecappelle
@ 2019-05-28  5:34                       ` Benjamin Kamath
  2019-07-03 20:48                         ` Romain Naour
  2019-05-28 14:57                       ` Bernd Kuhls
  1 sibling, 1 reply; 16+ messages in thread
From: Benjamin Kamath @ 2019-05-28  5:34 UTC (permalink / raw)
  To: buildroot

Sub options ate my last reply. Re-posting to list.

On Sun, May 26, 2019 at 5:29 AM Arnout Vandecappelle <arnout@mind.be> wrote:
>
>
>
> On 26/05/2019 13:43, Peter Korsgaard wrote:
> >>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
> >
> > Hi,
> >
> >  >>> But IMHO we should proceed with dropping clapack.
> >
> >  >  Why? it is maintained, and it is a dependency of armadillo...
> >
> >  >  I had a deeper look, and actually lapack and clapack are the same, just
> >  > compiled differently: lapack is built with the fortran compiler, clapack is
> >  > generated with f2c (by upstream) and built with the C compiler. AFAIU, anything
> >  > that uses lapack should also be able to use clapack and vice versa.
> >
> >  >  We have two packages like that: numpy and armadillo. They both seem to support
> >  > both lapack and clapack.
> >
> >  >  So, it seems indeed removing clapack is the appropriate thing. Except: lapack
> >  > requires a fortran compiler, which is not always available. So in that sense, it
> >  > would actually be more appropriate to remove lapack...
> >
> >  >  But maybe it would be better to normally use lapack, and only enable clapack
> >  > when lapack isn't available (i.e. when there's no fortran compiler).
> >
> > Not knowing anything about (c)lapack (or fortran), is there any
> > performance advantage using lapack over clapack? Otherwise just always
> > using clapack seems like the simplest solution.
>
>  Sounds OK to me.
>
>  Bernd, you're the only one who ever made a real contribution to lapack. Thoughts?
>
>  Benjamin, you originally contributed lapack. Any ideas?

I think lapack also might serve as a useful reference design for
anyone trying to bring an external Fortran package into their build.

If one is much more difficult to maintain, it makes sense to get rid
of that one. I think in this case clapack might be pretty well
maintained, but in general it's nice to avoid intermediate sources as
they are just another level of indirection where problems can occur.

Don't feel very strongly one way or the other. At this point, both
will be in the history so breadcrumbs are there if someone wants to
use the package that has support dropped.

Cheers,
Ben

>
>  Regards,
>  Arnout

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-26 12:29                     ` Arnout Vandecappelle
  2019-05-28  5:34                       ` Benjamin Kamath
@ 2019-05-28 14:57                       ` Bernd Kuhls
  1 sibling, 0 replies; 16+ messages in thread
From: Bernd Kuhls @ 2019-05-28 14:57 UTC (permalink / raw)
  To: buildroot

Am Sun, 26 May 2019 14:29:26 +0200 schrieb Arnout Vandecappelle:

>  Bernd, you're the only one who ever made a real contribution to lapack.
>  Thoughts?

Hi Arnout,

while it is true that I sent a version bump for lapack nearly two years 
ago this was part of a broader approach to bump many outdated packages.
I never used lapack and therefore I am unable to contribute to the 
current discussion, sorry.

Regards, Bernd

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH v2] package/python-numpy: fix occasional build failure with lapack
  2019-05-28  5:34                       ` Benjamin Kamath
@ 2019-07-03 20:48                         ` Romain Naour
  0 siblings, 0 replies; 16+ messages in thread
From: Romain Naour @ 2019-07-03 20:48 UTC (permalink / raw)
  To: buildroot

Hi Giulio, All,

Le 28/05/2019 ? 07:34, Benjamin Kamath a ?crit?:
> Sub options ate my last reply. Re-posting to list.
> 
> On Sun, May 26, 2019 at 5:29 AM Arnout Vandecappelle <arnout@mind.be> wrote:
>>
>>
>>
>> On 26/05/2019 13:43, Peter Korsgaard wrote:
>>>>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:
>>>
>>> Hi,
>>>
>>>  >>> But IMHO we should proceed with dropping clapack.
>>>
>>>  >  Why? it is maintained, and it is a dependency of armadillo...
>>>
>>>  >  I had a deeper look, and actually lapack and clapack are the same, just
>>>  > compiled differently: lapack is built with the fortran compiler, clapack is
>>>  > generated with f2c (by upstream) and built with the C compiler. AFAIU, anything
>>>  > that uses lapack should also be able to use clapack and vice versa.
>>>
>>>  >  We have two packages like that: numpy and armadillo. They both seem to support
>>>  > both lapack and clapack.
>>>
>>>  >  So, it seems indeed removing clapack is the appropriate thing. Except: lapack
>>>  > requires a fortran compiler, which is not always available. So in that sense, it
>>>  > would actually be more appropriate to remove lapack...

Maybe we should convince Thomas to generate toolchains from toolchain-builder
project with gFortran :

http://patchwork.ozlabs.org/patch/1113806/

>>>
>>>  >  But maybe it would be better to normally use lapack, and only enable clapack
>>>  > when lapack isn't available (i.e. when there's no fortran compiler).
>>>
>>> Not knowing anything about (c)lapack (or fortran), is there any
>>> performance advantage using lapack over clapack? Otherwise just always
>>> using clapack seems like the simplest solution.
>>
>>  Sounds OK to me.
>>
>>  Bernd, you're the only one who ever made a real contribution to lapack. Thoughts?
>>
>>  Benjamin, you originally contributed lapack. Any ideas?
> 
> I think lapack also might serve as a useful reference design for
> anyone trying to bring an external Fortran package into their build.
> 
> If one is much more difficult to maintain, it makes sense to get rid
> of that one. I think in this case clapack might be pretty well
> maintained, but in general it's nice to avoid intermediate sources as
> they are just another level of indirection where problems can occur.
> 
> Don't feel very strongly one way or the other. At this point, both
> will be in the history so breadcrumbs are there if someone wants to
> use the package that has support dropped.

Actually python-numpy doesn't seems to work at all due to a runtime issue:

http://patchwork.ozlabs.org/patch/1114198/

http://patchwork.ozlabs.org/patch/1112759/
(It would be great if python-numpy is runtime tested in gitlab)

Also, using uClibc fail too for another reason.

Sorry, I missed this thread while working on theses patches...

As a side project, I'm testing python-scipy package that require gFortran to
build. For now Python is stuck while importing the library for some reason...

Best regards,
Romain

> 
> Cheers,
> Ben
> 
>>
>>  Regards,
>>  Arnout
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2019-07-03 20:48 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-05-15 14:41 [Buildroot] [PATCH] package/python-numpy: fix occasional build failure with lapack Giulio Benetti
2019-05-15 15:12 ` Yann E. MORIN
2019-05-15 19:58   ` Giulio Benetti
2019-05-15 20:50     ` Giulio Benetti
2019-05-15 21:03       ` [Buildroot] [PATCH v2] " Giulio Benetti
2019-05-18 20:13         ` Thomas Petazzoni
2019-05-19 14:47           ` Giulio Benetti
2019-05-20 17:48             ` Giulio Benetti
2019-05-20 17:49               ` Giulio Benetti
2019-05-26  9:43                 ` Arnout Vandecappelle
2019-05-26 10:11                   ` Giulio Benetti
2019-05-26 11:43                   ` Peter Korsgaard
2019-05-26 12:29                     ` Arnout Vandecappelle
2019-05-28  5:34                       ` Benjamin Kamath
2019-07-03 20:48                         ` Romain Naour
2019-05-28 14:57                       ` Bernd Kuhls

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox