[PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Jarod Wilson]

Building for 32-bit arm, things fall down, due to lack of arch-specific
memory barriers.

In file included from
/builddir/build/BUILD/rdma-core-12/libibumad/sysfs.c:46:0:
include/infiniband/arch.h:132:2: error: #error No architecture specific
memory barrier defines found!
 #error No architecture specific memory barrier defines found!

Reported-by: Honggang Li <honli-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Jarod Wilson <jarod-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 redhat/rdma-core.spec | 1 +
 1 file changed, 1 insertion(+)

diff --git a/redhat/rdma-core.spec b/redhat/rdma-core.spec
index 246079e..4d31d53 100644
--- a/redhat/rdma-core.spec
+++ b/redhat/rdma-core.spec
@@ -166,6 +166,7 @@ level work required to open an RDMA connection between two machines.
 
 %package -n libibumad
 Summary: OpenFabrics Alliance InfiniBand umad (userspace management datagram) library
+ExcludeArch: %{arm}
 Requires: %{name}%{?_isa} = %{version}-%{release}
 
 %description -n libibumad
-- 
2.10.0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Jason Gunthorpe]

On Mon, Jan 09, 2017 at 04:39:44PM -0500, Jarod Wilson wrote:
> Building for 32-bit arm, things fall down, due to lack of arch-specific
> memory barriers.

Since we now have rxe that should work on ARM I think we need to fix
this upstream..

Do you have time to test some patches on ARM?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Jarod Wilson]

On 2017-01-09 5:31 PM, Jason Gunthorpe wrote:
> On Mon, Jan 09, 2017 at 04:39:44PM -0500, Jarod Wilson wrote:
>> Building for 32-bit arm, things fall down, due to lack of arch-specific
>> memory barriers.
>
> Since we now have rxe that should work on ARM I think we need to fix
> this upstream..
>
> Do you have time to test some patches on ARM?

Afraid I don't have hardware. To the best of my knowledge, the only ARM 
systems my group has (and/or are getting soon) with RDMA hardware is all 
ARM64.

-- 
Jarod Wilson
jarod-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Jason Gunthorpe]

On Mon, Jan 09, 2017 at 10:27:52PM -0500, Jarod Wilson wrote:
> On 2017-01-09 5:31 PM, Jason Gunthorpe wrote:
> >On Mon, Jan 09, 2017 at 04:39:44PM -0500, Jarod Wilson wrote:
> >>Building for 32-bit arm, things fall down, due to lack of arch-specific
> >>memory barriers.
> >
> >Since we now have rxe that should work on ARM I think we need to fix
> >this upstream..
> >
> >Do you have time to test some patches on ARM?
> 
> Afraid I don't have hardware. To the best of my knowledge, the only ARM
> systems my group has (and/or are getting soon) with RDMA hardware is all
> ARM64.

RDMA hardware doesn't work in ARM so that is OK. Cross compiling is
fine, ideal would be to test RXE on an arbitary ARM system, but I
wouldn't call that critical.

I just don't want to setup a cross compiler for this :\

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Amrani, Ram]

> RDMA hardware doesn't work in ARM so that is OK. Cross compiling is fine,
> ideal would be to test RXE on an arbitary ARM system, but I wouldn't call
> that critical.
> 
> I just don't want to setup a cross compiler for this :\
> 
> Jason
> --

Hi Jason, Doug, All,

I wanted to bring to your attention that our RDMA solution *is* working in an ARM based environment:
the Cavium ThunderX platform. This is a 64bit ARMv8 *with* cache coherency. Please keep that in mind
when considering RDMA-relevant platforms.

Thanks,
Ram

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

On Thu, Jan 12, 2017 at 12:32:37PM +0000, Amrani, Ram wrote:
> > RDMA hardware doesn't work in ARM so that is OK. Cross compiling is fine,
> > ideal would be to test RXE on an arbitary ARM system, but I wouldn't call
> > that critical.
> >
> > I just don't want to setup a cross compiler for this :\
> >
> > Jason
> > --
>
> Hi Jason, Doug, All,
>
> I wanted to bring to your attention that our RDMA solution *is* working in an ARM based environment:
> the Cavium ThunderX platform. This is a 64bit ARMv8 *with* cache coherency. Please keep that in mind
> when considering RDMA-relevant platforms.

They are referring to 32bit ARM in their discussion. AFAIK all devices in RDMA
are working and supported in 64bit ARM environment.

>
> Thanks,
> Ram
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Amrani, Ram]

> > > RDMA hardware doesn't work in ARM so that is OK. Cross compiling is
> > > fine, ideal would be to test RXE on an arbitary ARM system, but I
> > > wouldn't call that critical.
> > >
> > > I just don't want to setup a cross compiler for this :\
> > >
> > > Jason
> > > --
> >
> > Hi Jason, Doug, All,
> >
> > I wanted to bring to your attention that our RDMA solution *is* working
> in an ARM based environment:
> > the Cavium ThunderX platform. This is a 64bit ARMv8 *with* cache
> > coherency. Please keep that in mind when considering RDMA-relevant
> platforms.
> 
> They are referring to 32bit ARM in their discussion. AFAIK all devices in
> RDMA are working and supported in 64bit ARM environment.
 
I understand that the thread relates to 32-bit. I wanted to make sure that it 
doesn't impact 64-bit, as the sentence - "RDMA hardware doesn't work
in ARM " may imply that ARM doesn't support RDMA at all and that 64-bit
may be dropped.

Ram

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Jason Gunthorpe]

On Thu, Jan 12, 2017 at 01:15:19PM +0000, Amrani, Ram wrote:

> > They are referring to 32bit ARM in their discussion. AFAIK all devices in
> > RDMA are working and supported in 64bit ARM environment.
>  
> I understand that the thread relates to 32-bit. I wanted to make sure that it 
> doesn't impact 64-bit, as the sentence - "RDMA hardware doesn't work
> in ARM " may imply that ARM doesn't support RDMA at all and that 64-bit
> may be dropped.

hhns is already 64 bit ARM only hardware, it will be supported..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Hal Rosenstock]

On 1/9/2017 5:31 PM, Jason Gunthorpe wrote:
> On Mon, Jan 09, 2017 at 04:39:44PM -0500, Jarod Wilson wrote:
>> Building for 32-bit arm, things fall down, due to lack of arch-specific
>> memory barriers.
> 
> Since we now have rxe that should work on ARM I think we need to fix
> this upstream..
> 
> Do you have time to test some patches on ARM?

Looks to me that issue was introduced by:

commit 1df0888f6a736e1612ce8b054d6c17651ebd003f
Author: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Date:   Fri Sep 2 12:57:57 2016 -0600

    Remove most checks of __BYTE_ORDER

    For a long time now endian.h has defined sane fixed with conversion
    macros, so lets just use them instead of rolling our own.

    Also, htonll is defined in this source tree under infiniband/arch.h,
    so all users of that macro can just use the header.

    Someday we should also get rid of all the endless wrappers..

    Signed-off-by: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>

where byte order macros are from libibverbs arch.h which also includes
the memory barrier stuff.

So can't arch.h be separated out into 2 headers and have the various
parts of rdma-core include one or both of these headers as needed ?

-- Hal

> Jason
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 2260 bytes --]

On Tue, 2017-01-10 at 11:30 -0500, Hal Rosenstock wrote:
> On 1/9/2017 5:31 PM, Jason Gunthorpe wrote:
> > 
> > On Mon, Jan 09, 2017 at 04:39:44PM -0500, Jarod Wilson wrote:
> > > 
> > > Building for 32-bit arm, things fall down, due to lack of arch-
> > > specific
> > > memory barriers.
> > 
> > Since we now have rxe that should work on ARM I think we need to
> > fix
> > this upstream..
> > 
> > Do you have time to test some patches on ARM?
> 
> Looks to me that issue was introduced by:
> 
> commit 1df0888f6a736e1612ce8b054d6c17651ebd003f
> Author: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
> Date:   Fri Sep 2 12:57:57 2016 -0600
> 
>     Remove most checks of __BYTE_ORDER
> 
>     For a long time now endian.h has defined sane fixed with
> conversion
>     macros, so lets just use them instead of rolling our own.
> 
>     Also, htonll is defined in this source tree under
> infiniband/arch.h,
>     so all users of that macro can just use the header.
> 
>     Someday we should also get rid of all the endless wrappers..
> 
>     Signed-off-by: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
> 
> where byte order macros are from libibverbs arch.h which also
> includes
> the memory barrier stuff.
> 
> So can't arch.h be separated out into 2 headers and have the various
> parts of rdma-core include one or both of these headers as needed ?

No, that won't help.  At most it would shift where the compile breaks.
 When Steve Wise did the patch to add the arm64 memory barriers, we
discussed on list the fact that not having a proper memory barrier
define for an arch should be a fatal error, so we also made it such
that without an arch specific memory barrier define, we fail the build.
 If you split the headers, then somewhere you are going to come to a
file that needs the memory barriers defined in the headers, and when
you include the header, the compile error returns.  The proper fix is
just to get 32bit arm defines in place.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-core] redhat/spec: can't build ibumad on 32-bit arm

[Jason Gunthorpe]

On Tue, Jan 10, 2017 at 12:36:04PM -0500, Doug Ledford wrote:
> file that needs the memory barriers defined in the headers, and when
> you include the header, the compile error returns. ??The proper fix is
> just to get 32bit arm defines in place.

It is a little worse than that, 32 bit ARM is not DMA coherent at all
so it simply cannot work. The fix is to not compile providers that
require DMA coherence on arches that cannot provide it.

Bit tricky, but almost done..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html