libnl vs. libnl2 madness

libnl vs. libnl2 madness

[Stefan Schmidt]

Hello.

The last hours I was trying to compile wpa_supplicant with nl80211 support. That
needs netlink support through libnl or libnl2.

Enabling the option to use libnl2, not that easy to find, still breaks with
missing header files. Checking the libnl2 source shows that they are provided.
After some head on the table banging I found:

commit 880e00d3b7ccf66d9421a06bc28e553e07842b59
Author: Michael 'Mickey' Lauer <mickey@vanille-media.de>
Date:   Tue Nov 24 16:33:06 2009 +0000

    libnl2: change includedir to not step over libnl1; also convert to new staging

which brings in this:

+includedir = ${prefix}/include/libnl2

Resulting in header files getting staged in to libnl2/netlink/fooo instead of
netlink/foo.

This obviously breaks every application that uses libnl2. Reverting this part
makes wpa_supplicant happy.

Can somebody bring some light into this? Do we have libnl1 only recipes in our
metadata?

If libnl2 overrides header files from libnl with other content this is of course
bad. As is to move away header files from the usual location and breaking apps.

regards
Stefan Schmidt

Re: libnl vs. libnl2 madness

[Thomas Zimmermann]

On Thursday 03 February 2011 10:32:53 Stefan Schmidt wrote:
> Hello.
>
> The last hours I was trying to compile wpa_supplicant with nl80211 support.
> That needs netlink support through libnl or libnl2.
>
> Enabling the option to use libnl2, not that easy to find, still breaks with
> missing header files. Checking the libnl2 source shows that they are
> provided. After some head on the table banging I found:
>
> commit 880e00d3b7ccf66d9421a06bc28e553e07842b59
> Author: Michael 'Mickey' Lauer <mickey@vanille-media.de>
> Date:   Tue Nov 24 16:33:06 2009 +0000
>
>     libnl2: change includedir to not step over libnl1; also convert to new
> staging

At that time there was just one recipe depending on libnl2 and that was FSO2.
I think one month ago JaMa has build the first SHR image without libnl(1), 
until then it was impossible because of some important parts. I'm not sure but 
i think the main blocker was bluez4 (Can't check now).

But i think there are a lot minor packages that still require libnl and don't 
support libnl2.

Regards
Thomas

Re: libnl vs. libnl2 madness

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3058 bytes --]

On Thu, Feb 03, 2011 at 10:54:08AM +0100, Thomas Zimmermann wrote:
> On Thursday 03 February 2011 10:32:53 Stefan Schmidt wrote:
> > Hello.
> >
> > The last hours I was trying to compile wpa_supplicant with nl80211 support.
> > That needs netlink support through libnl or libnl2.
> >
> > Enabling the option to use libnl2, not that easy to find, still breaks with
> > missing header files. Checking the libnl2 source shows that they are
> > provided. After some head on the table banging I found:
> >
> > commit 880e00d3b7ccf66d9421a06bc28e553e07842b59
> > Author: Michael 'Mickey' Lauer <mickey@vanille-media.de>
> > Date:   Tue Nov 24 16:33:06 2009 +0000
> >
> >     libnl2: change includedir to not step over libnl1; also convert to new
> > staging
> 
> At that time there was just one recipe depending on libnl2 and that was FSO2.
> I think one month ago JaMa has build the first SHR image without libnl(1), 

yes after couple of changes
http://git.openembedded.org/cgit.cgi/openembedded/log/?h=org.openembedded.dev&qt=grep&q=libnl

> until then it was impossible because of some important parts. I'm not sure but 
> i think the main blocker was bluez4 (Can't check now).

Yes older bluez4 with nl plugin (which was removed later).

and:
http://gitorious.org/~jama/angstrom/jama-shr-experimental/commit/70b13d30b335b8fd67116d3bc0834382a77bc319
81      ANGSTROM_BLACKLIST_pn-bmon = "bmon depends on libnl we want libnl2"
82	ANGSTROM_BLACKLIST_pn-crda = "crda depends on libnl we want libnl2"
83	ANGSTROM_BLACKLIST_pn-hostap-daemon = "hostap-daemon depends on libnl we want libnl2"
84	ANGSTROM_BLACKLIST_pn-ibrdtn = "ibrdtn depends on libnl we want libnl2"
85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"
86	ANGSTROM_BLACKLIST_pn-lowpan-tools = "lowpan-tools depends on libnl we want libnl2"
87	ANGSTROM_BLACKLIST_pn-networkmanager = "networkmanager depends on libnl we want libnl2"
88	ANGSTROM_BLACKLIST_pn-networkmanager-openvpn = "networkmanager-openvpn depends on libnl we want libnl2"
89	ANGSTROM_BLACKLIST_pn-pstree = "pstree depends on libnl we want libnl2"
90	ANGSTROM_BLACKLIST_pn-rfkill = "rfkill depends on libnl we want libnl2"

I didn't check if those can be built with libnl2 or if they work properly with libnl2.

But be aware that we had enough issues in runtime (build passes ok) caused by libnl1 built in same sysroot 
as libnl2 (that's why I had to blacklist libnl2 in SHR).

From GNUtoo's report about issues with libnl1 in runtime..(when both libnl are built in same sysroot and libnl1 was rebuilt later)
<snip>
root@htcdream ~ # phoneuid
BUG: route/class_api.c:42
phoneuid: route/class_api.c:42: rtnl_class_register: Assertion `0' failed.
Aborted
</snip>

And also be aware that after blacklisting it, doing just bitbake -c clean libnl libnl2 wasn't 
enough to rebuild all apps properly against libnl2, I had to manually remove all traces of 
libnl1 like shlibs etc (not sure why).

Regards,

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

Re: libnl vs. libnl2 madness

[Stefan Schmidt]

Hello.

On Thu, 2011-02-03 at 11:12, Martin Jansa wrote:
> On Thu, Feb 03, 2011 at 10:54:08AM +0100, Thomas Zimmermann wrote:
> > On Thursday 03 February 2011 10:32:53 Stefan Schmidt wrote:
> > 
> > At that time there was just one recipe depending on libnl2 and that was FSO2.
> > I think one month ago JaMa has build the first SHR image without libnl(1), 
> 
> yes after couple of changes
> http://git.openembedded.org/cgit.cgi/openembedded/log/?h=org.openembedded.dev&qt=grep&q=libnl
> 
> > until then it was impossible because of some important parts. I'm not sure but 
> > i think the main blocker was bluez4 (Can't check now).
> 
> Yes older bluez4 with nl plugin (which was removed later).

Good, so bluez is of the table.

> and:
> http://gitorious.org/~jama/angstrom/jama-shr-experimental/commit/70b13d30b335b8fd67116d3bc0834382a77bc319
> 81      ANGSTROM_BLACKLIST_pn-bmon = "bmon depends on libnl we want libnl2"
> 82	ANGSTROM_BLACKLIST_pn-crda = "crda depends on libnl we want libnl2"

No idea about these two.

> 83	ANGSTROM_BLACKLIST_pn-hostap-daemon = "hostap-daemon depends on libnl we want libnl2"

Should be fine to build with CONFIG_LIBNL20 set. Its the same codebase as
wpa-supplicant.

> 84	ANGSTROM_BLACKLIST_pn-ibrdtn = "ibrdtn depends on libnl we want libnl2"

Maybe I should get a stab at this as I'm the main user of it in OE. :)

> 85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"

Need to check latest version for libnl2 support. Should not be to hard to change
it.

> 86	ANGSTROM_BLACKLIST_pn-lowpan-tools = "lowpan-tools depends on libnl we want libnl2"

When I'm going to poke ibrdtn I could take a look here as well. Maybe Dmitry .

> 87	ANGSTROM_BLACKLIST_pn-networkmanager = "networkmanager depends on libnl we want libnl2"
> 88	ANGSTROM_BLACKLIST_pn-networkmanager-openvpn = "networkmanager-openvpn depends on libnl we want libnl2"

Those two have a pretty big code base and I'm not going to dive into it (again).

> 89	ANGSTROM_BLACKLIST_pn-pstree = "pstree depends on libnl we want libnl2"
> 90	ANGSTROM_BLACKLIST_pn-rfkill = "rfkill depends on libnl we want libnl2"

Need to check latest version for libnl2 support. Should not be to hard to change
it.

> I didn't check if those can be built with libnl2 or if they work properly with libnl2.

As you can see I would be willing to work on some of these. Personally I would
aim for getting the recipes over to libnl2 and the kill libnl1 for good in OE.

> But be aware that we had enough issues in runtime (build passes ok) caused by libnl1 built in same sysroot 
> as libnl2 (that's why I had to blacklist libnl2 in SHR).

Thats one more reason to get rid of the old version. :)

regards
Stefan Schmidt

Re: libnl vs. libnl2 madness

[Stefan Schmidt]

Hello.

On Thu, 2011-02-03 at 13:42, Stefan Schmidt wrote:
> 
> On Thu, 2011-02-03 at 11:12, Martin Jansa wrote:
> > and:
> > http://gitorious.org/~jama/angstrom/jama-shr-experimental/commit/70b13d30b335b8fd67116d3bc0834382a77bc319
> > 81      ANGSTROM_BLACKLIST_pn-bmon = "bmon depends on libnl we want libnl2"
> > 82	ANGSTROM_BLACKLIST_pn-crda = "crda depends on libnl we want libnl2"
> 
> No idea about these two.

crda should be able to work with libnl2 but have issues during do_install()

> 
> > 85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"
> 
> Need to check latest version for libnl2 support. Should not be to hard to change
> it.

Koen spotted that it should be workig with libnl2 and a build test was positive.

> > I didn't check if those can be built with libnl2 or if they work properly with libnl2.
> 
> As you can see I would be willing to work on some of these. Personally I would
> aim for getting the recipes over to libnl2 and the kill libnl1 for good in OE.

I have an patch removing it but this may have just to many sideeffects on our
metadata. Koens pointed out that we maybe should switch the inc dir from libnl
to a "wrong" place and leave libnl2 header at the correct place. Obviously we
need to fix most of our recipes for libnl2 before such a step.

regards
Stefan Schmidt

Re: libnl vs. libnl2 madness

[Stefan Schmidt]

Hello.

On Thu, 2011-02-03 at 15:08, Stefan Schmidt wrote:
> 
> On Thu, 2011-02-03 at 13:42, Stefan Schmidt wrote:
> > 
> > > 85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"
> > 
> > Need to check latest version for libnl2 support. Should not be to hard to change
> > it.
> 
> Koen spotted that it should be workig with libnl2 and a build test was positive.
> 
> > > I didn't check if those can be built with libnl2 or if they work properly with libnl2.
> > 
> > As you can see I would be willing to work on some of these. Personally I would
> > aim for getting the recipes over to libnl2 and the kill libnl1 for good in OE.
> 
> I have an patch removing it but this may have just to many sideeffects on our
> metadata. Koens pointed out that we maybe should switch the inc dir from libnl
> to a "wrong" place and leave libnl2 header at the correct place. Obviously we
> need to fix most of our recipes for libnl2 before such a step.

To make this step less frustrating for current users I added the libnl2
includedir path for wpa-supplicant and iw in the recipe to get it working with
libnl2.

After the release I would like to take it a step further and move the libnl2
header files and adjust only the few recipes that have not been converted to
libnl2. Which is of course also something I try to look at.

regards
Stefan Schmidt

Re: libnl vs. libnl2 madness

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09-02-11 11:00, Stefan Schmidt wrote:
> Hello.
> 
> On Thu, 2011-02-03 at 15:08, Stefan Schmidt wrote:
>>
>> On Thu, 2011-02-03 at 13:42, Stefan Schmidt wrote:
>>>
>>>> 85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"
>>>
>>> Need to check latest version for libnl2 support. Should not be to hard to change
>>> it.
>>
>> Koen spotted that it should be workig with libnl2 and a build test was positive.
>>
>>>> I didn't check if those can be built with libnl2 or if they work properly with libnl2.
>>>
>>> As you can see I would be willing to work on some of these. Personally I would
>>> aim for getting the recipes over to libnl2 and the kill libnl1 for good in OE.
>>
>> I have an patch removing it but this may have just to many sideeffects on our
>> metadata. Koens pointed out that we maybe should switch the inc dir from libnl
>> to a "wrong" place and leave libnl2 header at the correct place. Obviously we
>> need to fix most of our recipes for libnl2 before such a step.
> 
> To make this step less frustrating for current users I added the libnl2
> includedir path for wpa-supplicant and iw in the recipe to get it working with
> libnl2.
> 
> After the release I would like to take it a step further and move the libnl2
> header files and adjust only the few recipes that have not been converted to
> libnl2. Which is of course also something I try to look at.

I'm looking at what is needed to get networkmanager up to nl2, but it
looks like we'd need to do the patches ourselves and get it into nm 0.9.

regards,

Koen

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFNUmpdMkyGM64RGpERAjtWAKCC8MZqxIXGljlw0BDNDsObpADX1gCfatiP
+KjqpcvK5IWww1r6OQugilM=
=OOcI
-----END PGP SIGNATURE-----

Re: libnl vs. libnl2 madness

[Stefan Schmidt]

Hello.

On Wed, 2011-02-09 at 11:20, Koen Kooi wrote:
> On 09-02-11 11:00, Stefan Schmidt wrote:
> > Hello.
> > 
> > On Thu, 2011-02-03 at 15:08, Stefan Schmidt wrote:
> >>
> >> On Thu, 2011-02-03 at 13:42, Stefan Schmidt wrote:
> >>>
> >>>> 85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"
> >>>
> >>> Need to check latest version for libnl2 support. Should not be to hard to change
> >>> it.
> >>
> >> Koen spotted that it should be workig with libnl2 and a build test was positive.
> >>
> >>>> I didn't check if those can be built with libnl2 or if they work properly with libnl2.
> >>>
> >>> As you can see I would be willing to work on some of these. Personally I would
> >>> aim for getting the recipes over to libnl2 and the kill libnl1 for good in OE.
> >>
> >> I have an patch removing it but this may have just to many sideeffects on our
> >> metadata. Koens pointed out that we maybe should switch the inc dir from libnl
> >> to a "wrong" place and leave libnl2 header at the correct place. Obviously we
> >> need to fix most of our recipes for libnl2 before such a step.
> > 
> > To make this step less frustrating for current users I added the libnl2
> > includedir path for wpa-supplicant and iw in the recipe to get it working with
> > libnl2.
> > 
> > After the release I would like to take it a step further and move the libnl2
> > header files and adjust only the few recipes that have not been converted to
> > libnl2. Which is of course also something I try to look at.
> 
> I'm looking at what is needed to get networkmanager up to nl2, but it
> looks like we'd need to do the patches ourselves and get it into nm 0.9.

NM is a pretty big customer. As far as I got it from the ml the maintainer would
like to get it up to libnl2 as well but has other work to do.

NM is one of the things I really don't want to try working out a patch on my
own. Smaller things might be easy to do and I will look into lowpan-tools and
ibrdtn after the release.

regards
Stefan Schmidt

Re: libnl vs. libnl2 madness

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03-02-11 11:12, Martin Jansa wrote:
> On Thu, Feb 03, 2011 at 10:54:08AM +0100, Thomas Zimmermann wrote:
>> On Thursday 03 February 2011 10:32:53 Stefan Schmidt wrote:
>>> Hello.
>>>
>>> The last hours I was trying to compile wpa_supplicant with nl80211 support.
>>> That needs netlink support through libnl or libnl2.
>>>
>>> Enabling the option to use libnl2, not that easy to find, still breaks with
>>> missing header files. Checking the libnl2 source shows that they are
>>> provided. After some head on the table banging I found:
>>>
>>> commit 880e00d3b7ccf66d9421a06bc28e553e07842b59
>>> Author: Michael 'Mickey' Lauer <mickey@vanille-media.de>
>>> Date:   Tue Nov 24 16:33:06 2009 +0000
>>>
>>>     libnl2: change includedir to not step over libnl1; also convert to new
>>> staging
>>
>> At that time there was just one recipe depending on libnl2 and that was FSO2.
>> I think one month ago JaMa has build the first SHR image without libnl(1), 
> 
> yes after couple of changes
> http://git.openembedded.org/cgit.cgi/openembedded/log/?h=org.openembedded.dev&qt=grep&q=libnl
> 
>> until then it was impossible because of some important parts. I'm not sure but 
>> i think the main blocker was bluez4 (Can't check now).
> 
> Yes older bluez4 with nl plugin (which was removed later).
> 
> and:
> http://gitorious.org/~jama/angstrom/jama-shr-experimental/commit/70b13d30b335b8fd67116d3bc0834382a77bc319

> 85	ANGSTROM_BLACKLIST_pn-iw = "iw depends on libnl we want libnl2"

That one was fixed

> 90	ANGSTROM_BLACKLIST_pn-rfkill = "rfkill depends on libnl we want libnl2"

That one was fixed as well

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFNVQxEMkyGM64RGpERAtyqAJ9RH01ASiC4/BebDfkW4Vj/BdQiHACePQBQ
chy6qKs4vwoz7joQk0u/x9E=
=H06U
-----END PGP SIGNATURE-----