interested in py80211?

interested in py80211?

[Arend van Spriel]

A common say in Linux arena is "when you have an itch, just scratch it". 
My itch is that tools like ifconfig and iw are great, but in an 
automated test environment it kind of sucks to parse output, which is 
confirmed by blurb from iw: "Do NOT screenscrape this tool, we don't 
consider its output stable.".

Ever since my first contact with Python I tend to favor it over other 
scripting alternatives so I decided to scratch my itch with that and 
another old acquaintance called SWIG. With those I went to create 
py80211. A first attempt was to have SWIG create a wrapper API directly 
exposing the libnl-3 API, but that did not feel comfortable in a 
scripting environment. So the level of abstraction is a bit higher. It 
is just in a kick-off state (eg. can only send u32 attributes), but I 
decided to push it to github anyway.

https://github.com/arend/py80211

Regards,
Arend

Re: interested in py80211?

[Johannes Berg]

Yep, I'd totally be interested. Not that I don't have enough things on
my plate already ;-)

On Wed, 2013-07-03 at 22:57 +0200, Arend van Spriel wrote:
> A common say in Linux arena is "when you have an itch, just scratch it".

:-)

> My itch is that tools like ifconfig and iw are great, but in an 
> automated test environment it kind of sucks to parse output, which is 
> confirmed by blurb from iw: "Do NOT screenscrape this tool, we don't 
> consider its output stable.".

Heh, yeah ...

> Ever since my first contact with Python I tend to favor it over other 
> scripting alternatives so I decided to scratch my itch with that and 
> another old acquaintance called SWIG. With those I went to create 
> py80211. A first attempt was to have SWIG create a wrapper API directly 
> exposing the libnl-3 API, but that did not feel comfortable in a 
> scripting environment. So the level of abstraction is a bit higher. It 
> is just in a kick-off state (eg. can only send u32 attributes), but I 
> decided to push it to github anyway.

Another approach might be exposing the libnl APIs and then build a
higher-level library in python. Have you considered that? That might
make it useful to other users of netlink as well, while keeping a 'nice'
nl80211 API?

johannes

Re: interested in py80211?

[Arend van Spriel]

On 07/12/13 10:02, Johannes Berg wrote:
> Yep, I'd totally be interested. Not that I don't have enough things on
> my plate already ;-)
>
> On Wed, 2013-07-03 at 22:57 +0200, Arend van Spriel wrote:
>> A common say in Linux arena is "when you have an itch, just scratch it".
>
> :-)
>
>> My itch is that tools like ifconfig and iw are great, but in an
>> automated test environment it kind of sucks to parse output, which is
>> confirmed by blurb from iw: "Do NOT screenscrape this tool, we don't
>> consider its output stable.".
>
> Heh, yeah ...
>
>> Ever since my first contact with Python I tend to favor it over other
>> scripting alternatives so I decided to scratch my itch with that and
>> another old acquaintance called SWIG. With those I went to create
>> py80211. A first attempt was to have SWIG create a wrapper API directly
>> exposing the libnl-3 API, but that did not feel comfortable in a
>> scripting environment. So the level of abstraction is a bit higher. It
>> is just in a kick-off state (eg. can only send u32 attributes), but I
>> decided to push it to github anyway.
>
> Another approach might be exposing the libnl APIs and then build a
> higher-level library in python. Have you considered that? That might
> make it useful to other users of netlink as well, while keeping a 'nice'
> nl80211 API?

I am still at that fork in the road and not sure about it. The libnl 
project itself already has libnl stuff exposed in a python lib being the 
core api and route. So I could go and add genl support to that and build 
the high-level python library from there.

Regards,
Arend

Re: interested in py80211?

[Will Hawkins]

Thanks for posting about this effort. I would be very interested in
helping out on development. I will stay up-to-date on things via your
github page. Otherwise, please continue to keep the list updated on your
progress.

Will

On 07/12/2013 09:32 AM, Arend van Spriel wrote:
> On 07/12/13 10:02, Johannes Berg wrote:
>> Yep, I'd totally be interested. Not that I don't have enough things on
>> my plate already ;-)
>>
>> On Wed, 2013-07-03 at 22:57 +0200, Arend van Spriel wrote:
>>> A common say in Linux arena is "when you have an itch, just scratch it".
>>
>> :-)
>>
>>> My itch is that tools like ifconfig and iw are great, but in an
>>> automated test environment it kind of sucks to parse output, which is
>>> confirmed by blurb from iw: "Do NOT screenscrape this tool, we don't
>>> consider its output stable.".
>>
>> Heh, yeah ...
>>
>>> Ever since my first contact with Python I tend to favor it over other
>>> scripting alternatives so I decided to scratch my itch with that and
>>> another old acquaintance called SWIG. With those I went to create
>>> py80211. A first attempt was to have SWIG create a wrapper API directly
>>> exposing the libnl-3 API, but that did not feel comfortable in a
>>> scripting environment. So the level of abstraction is a bit higher. It
>>> is just in a kick-off state (eg. can only send u32 attributes), but I
>>> decided to push it to github anyway.
>>
>> Another approach might be exposing the libnl APIs and then build a
>> higher-level library in python. Have you considered that? That might
>> make it useful to other users of netlink as well, while keeping a 'nice'
>> nl80211 API?
> 
> I am still at that fork in the road and not sure about it. The libnl
> project itself already has libnl stuff exposed in a python lib being the
> core api and route. So I could go and add genl support to that and build
> the high-level python library from there.
> 
> Regards,
> Arend
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe
> linux-wireless" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: interested in py80211?

[Arend van Spriel]

On 07/24/2013 05:55 PM, Will Hawkins wrote:
> Thanks for posting about this effort. I would be very interested in
> helping out on development. I will stay up-to-date on things via your
> github page. Otherwise, please continue to keep the list updated on your
> progress.


Thanks, Will

Based on feedback from Johannes I have taken a bit different approach. I 
submitted some patches to libnl adding the SWIG api for python there. 
Thomas took those patches recently. So the idea is to do the rest of the 
nl80211 specific stuff in Python. I am looking into pycparser as I would 
like to extract certain code fragments from nl80211.[ch] so keeping 
py80211 in sync with nl80211 can be automated.


Regards,
Arend

> On 07/12/2013 09:32 AM, Arend van Spriel wrote:
>> On 07/12/13 10:02, Johannes Berg wrote:
>>> Yep, I'd totally be interested. Not that I don't have enough things on
>>> my plate already ;-)
>>>
>>> On Wed, 2013-07-03 at 22:57 +0200, Arend van Spriel wrote:
>>>> A common say in Linux arena is "when you have an itch, just scratch it".
>>>
>>> :-)
>>>
>>>> My itch is that tools like ifconfig and iw are great, but in an
>>>> automated test environment it kind of sucks to parse output, which is
>>>> confirmed by blurb from iw: "Do NOT screenscrape this tool, we don't
>>>> consider its output stable.".
>>>
>>> Heh, yeah ...
>>>
>>>> Ever since my first contact with Python I tend to favor it over other
>>>> scripting alternatives so I decided to scratch my itch with that and
>>>> another old acquaintance called SWIG. With those I went to create
>>>> py80211. A first attempt was to have SWIG create a wrapper API directly
>>>> exposing the libnl-3 API, but that did not feel comfortable in a
>>>> scripting environment. So the level of abstraction is a bit higher. It
>>>> is just in a kick-off state (eg. can only send u32 attributes), but I
>>>> decided to push it to github anyway.
>>>
>>> Another approach might be exposing the libnl APIs and then build a
>>> higher-level library in python. Have you considered that? That might
>>> make it useful to other users of netlink as well, while keeping a 'nice'
>>> nl80211 API?
>>
>> I am still at that fork in the road and not sure about it. The libnl
>> project itself already has libnl stuff exposed in a python lib being the
>> core api and route. So I could go and add genl support to that and build
>> the high-level python library from there.
>>
>> Regards,
>> Arend
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-wireless" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>

Re: interested in py80211?

[Arend van Spriel]

On 07/12/2013 10:02 AM, Johannes Berg wrote:
>> Ever since my first contact with Python I tend to favor it over other
>> >scripting alternatives so I decided to scratch my itch with that and
>> >another old acquaintance called SWIG. With those I went to create
>> >py80211. A first attempt was to have SWIG create a wrapper API directly
>> >exposing the libnl-3 API, but that did not feel comfortable in a
>> >scripting environment. So the level of abstraction is a bit higher. It
>> >is just in a kick-off state (eg. can only send u32 attributes), but I
>> >decided to push it to github anyway.
> Another approach might be exposing the libnl APIs and then build a
> higher-level library in python. Have you considered that? That might
> make it useful to other users of netlink as well, while keeping a 'nice'
> nl80211 API?

Hi Johannes,

It has been a while that I touched this topic. It is a spare-time 
project and that is somewhat limited. Since then I added genl support to 
libnl python API. I want to extract part of py80211 from nl80211.[ch] 
files. The easy part were the attribute ids and enum values from 
nl80211.h. The hurdle I want to leap know is extracting message layout 
information from nl80211.c. Do the struct nla_policy definitions hold 
all the information or would it be better to get it from the .doit 
callbacks.

Regards,
Arend

Re: interested in py80211?

[Johannes Berg]

On Mon, 2013-11-11 at 09:42 +0100, Arend van Spriel wrote:

> It has been a while that I touched this topic. It is a spare-time 
> project and that is somewhat limited. Since then I added genl support to 
> libnl python API. I want to extract part of py80211 from nl80211.[ch] 
> files. The easy part were the attribute ids and enum values from 
> nl80211.h. The hurdle I want to leap know is extracting message layout 
> information from nl80211.c. Do the struct nla_policy definitions hold 
> all the information or would it be better to get it from the .doit 
> callbacks.

Well, you probably can't get it in a scripted fashion from the doit
callbacks. The policy certainly has the first level information, but as
soon as you have nested information it will no longer work. The policy
also doesn't always cover data fram the kernel (i.e. kernel->userspace
encoding) since the kernel doesn't have to verify that.

johannes

py80211 update on github

[Arend van Spriel]

On 07/03/2013 10:57 PM, Arend van Spriel wrote:
> A common say in Linux arena is "when you have an itch, just scratch it".
> My itch is that tools like ifconfig and iw are great, but in an
> automated test environment it kind of sucks to parse output, which is
> confirmed by blurb from iw: "Do NOT screenscrape this tool, we don't
> consider its output stable.".

In the christmas holiday I tinkered a bit more on py80211. It comes with
a utility to extract definitions and netlink policies from the nl80211
source files. What is currently on github is in prototype state. It can
retrieve all wiphy information including band, frequency and rate
information. It relies on policy information and it turned out that not
all attributes are properly initialized in nl80211_policy. One that I
added manually is NL80211_ATTR_SUPPORTED_IFTYPES, which should actually
be typed as NLA_NESTED. There are other attributes that are not covered yet.

> 
> https://github.com/arend/py80211
> 

The source files are found in the github repository. It has two
branches. The master branch is based on 3.12.6 and I intend to follow
the kernel releases on that. The wl-next branch is based on the
wireless-next repository.

Regards,
Arend

Re: py80211 update on github

[Johannes Berg]

On Mon, 2014-01-06 at 10:30 +0100, Arend van Spriel wrote:

> In the christmas holiday I tinkered a bit more on py80211. It comes with
> a utility to extract definitions and netlink policies from the nl80211
> source files. What is currently on github is in prototype state. It can
> retrieve all wiphy information including band, frequency and rate
> information. It relies on policy information and it turned out that not
> all attributes are properly initialized in nl80211_policy. One that I
> added manually is NL80211_ATTR_SUPPORTED_IFTYPES, which should actually
> be typed as NLA_NESTED. There are other attributes that are not covered yet.

This makes sense because the kernel policy only covers things that are
sent *to* the kernel. In some very rare cases, the stuff sent *to* the
kernel and *by* the kernel may even differ, so you may have to add some
special cases.

johannes

Re: py80211 update on github

[Arend van Spriel]

On 01/06/2014 11:36 AM, Johannes Berg wrote:
> On Mon, 2014-01-06 at 10:30 +0100, Arend van Spriel wrote:
> 
>> In the christmas holiday I tinkered a bit more on py80211. It comes with
>> a utility to extract definitions and netlink policies from the nl80211
>> source files. What is currently on github is in prototype state. It can
>> retrieve all wiphy information including band, frequency and rate
>> information. It relies on policy information and it turned out that not
>> all attributes are properly initialized in nl80211_policy. One that I
>> added manually is NL80211_ATTR_SUPPORTED_IFTYPES, which should actually
>> be typed as NLA_NESTED. There are other attributes that are not covered yet.
> 
> This makes sense because the kernel policy only covers things that are
> sent *to* the kernel. In some very rare cases, the stuff sent *to* the
> kernel and *by* the kernel may even differ, so you may have to add some
> special cases.

Yeah, I realized that indeed the policies in the kernel only cover
validation *to* the kernel. In iw the policies are used rarely and most
type information is covered in the code. py80211 relies on policies to
obtain the type information and I was hoping to extract it from the
kernel/iw code. I may end up maintaining it in py80211.

Gr. AvS

> --
> To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>