ipsets: examples?

ipsets: examples?

[Reuben Martin]

I'm trying to put together an extension to forward udp packets to an arbritary set of destinations using ipsets. Specifically I want the extension to be able to take either an ipmap or iphash as a set of destinations for the forwarded udp packets. (I'm starting with the rawdnat, stripping out the tcp code, and changing it so that it duplicates the packet as needed for destinations instead of changing the dest address in the origional packet.

But I'm having a hard time finding code examples of how to use ipsets. How do I take an ipset as an argument for an extension? How do I test the set type to make sure it's an ipmap or iphash? How do I iterate over the set? Is it an array, linked list, etc?

I've only been poking around for a couple hours with the xtables code, so I am by no means very familiar with the API.

-Reuben

Re: ipsets: examples?

[Reuben Martin]

On Sunday, December 19, 2010, Reuben Martin wrote:
> 
> I'm trying to put together an extension to forward udp packets to an arbritary set of destinations using ipsets. Specifically I want the extension to be able to take either an ipmap or iphash as a set of destinations for the forwarded udp packets. (I'm starting with the rawdnat, stripping out the tcp code, and changing it so that it duplicates the packet as needed for destinations instead of changing the dest address in the origional packet.
> 
> But I'm having a hard time finding code examples of how to use ipsets. How do I take an ipset as an argument for an extension? How do I test the set type to make sure it's an ipmap or iphash? How do I iterate over the set? Is it an array, linked list, etc?
> 
> I've only been poking around for a couple hours with the xtables code, so I am by no means very familiar with the API.
> 
> -Reuben
> 

Or, am I perhaps mistaken assuming that ipsets provides an api for other extensions to use sets as a dynamic target?

Re: ipsets: examples?

[Jozsef Kadlecsik]

Hi,

On Sun, 19 Dec 2010, Reuben Martin wrote:

> I'm trying to put together an extension to forward udp packets to an 
> arbritary set of destinations using ipsets. Specifically I want the 
> extension to be able to take either an ipmap or iphash as a set of 
> destinations for the forwarded udp packets. (I'm starting with the 
> rawdnat, stripping out the tcp code, and changing it so that it 
> duplicates the packet as needed for destinations instead of changing the 
> dest address in the origional packet.

I don't really see why you need to embed ipset. Why don't you call it as a 
normal match and use your extension as a target?

> But I'm having a hard time finding code examples of how to use ipsets. 

Have a look at kernel/xt_set.c. That contains the API by which application 
can use the sets.

> How do I take an ipset as an argument for an extension?

You have to use the index of the set, because that's its identifier, and 
must get a reference to it too (ip_set_nfnl_get or 
ip_set_nfnl_get_byindex).

> How do I test the set type to make sure it's an ipmap or iphash?

That is an internal data and not exported to the application. Unneeded 
as well, because the access to the sets is independet from the set type.

> How do I iterate over the set? Is it an array, linked list, etc?

There's no iterators but functions to test, add or delete elements in the 
sets, based on the data in the packet: ip_set_test[|add|del]. Therefore 
the type of the set is indifferent for the application.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipsets: examples?

[Reuben Martin]

On Monday, December 20, 2010, Jozsef Kadlecsik wrote:
> Hi,
> 
> On Sun, 19 Dec 2010, Reuben Martin wrote:
> 
> > I'm trying to put together an extension to forward udp packets to an 
> > arbritary set of destinations using ipsets. Specifically I want the 
> > extension to be able to take either an ipmap or iphash as a set of 
> > destinations for the forwarded udp packets. (I'm starting with the 
> > rawdnat, stripping out the tcp code, and changing it so that it 
> > duplicates the packet as needed for destinations instead of changing the 
> > dest address in the origional packet.
> 
> I don't really see why you need to embed ipset. Why don't you call it as a 
> normal match and use your extension as a target?
> 

I think I have misunderstood what can be done with ipsets. I knew you could use it as a match, but for some reason I though it had an API to allow other extensions to use sets as targets. (i.e. my intention is, as you suggested, to use the extension as a target)

Is there any way via the standard API to request the contents of a named set? That's really what I want to do. Basically for each packet I would request I want to be able to check what addresses are in the set named in the parameters of my extension, and then copy the packets to each of those addresses, and hence have a dynamic list of destination addresses.

If there is no such API would it be too hard to create one? It would be great to be able to access the sets as linked lists, while having ipset internally managing the locking so that something is not accessing the list while an element in the list is being added or removed.

-Reuben

Re: ipsets: examples?

[Jozsef Kadlecsik]

On Mon, 20 Dec 2010, Reuben Martin wrote:

> On Monday, December 20, 2010, Jozsef Kadlecsik wrote:
> > 
> > On Sun, 19 Dec 2010, Reuben Martin wrote:
> > 
> > > I'm trying to put together an extension to forward udp packets to an 
> > > arbritary set of destinations using ipsets. Specifically I want the 
> > > extension to be able to take either an ipmap or iphash as a set of 
> > > destinations for the forwarded udp packets. (I'm starting with the 
> > > rawdnat, stripping out the tcp code, and changing it so that it 
> > > duplicates the packet as needed for destinations instead of changing the 
> > > dest address in the origional packet.
> > 
> > I don't really see why you need to embed ipset. Why don't you call it as a 
> > normal match and use your extension as a target?
> 
> I think I have misunderstood what can be done with ipsets. I knew you 
> could use it as a match, but for some reason I though it had an API to 
> allow other extensions to use sets as targets. (i.e. my intention is, as 
> you suggested, to use the extension as a target)

But what is the difference between

... -m set --match-set foo src -j YOUR_TARGET

and

... -j YOUR_TARGET

where your target calls internally the same or hardcoded ipset match?
 
> Is there any way via the standard API to request the contents of a named 
> set? That's really what I want to do. Basically for each packet I would 
> request I want to be able to check what addresses are in the set named 
> in the parameters of my extension, and then copy the packets to each of 
> those addresses, and hence have a dynamic list of destination addresses.

In my previous mail I wrote about the current API and possibilities.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipsets: examples?

[Reuben Martin]

On Tue, Dec 21, 2010 at 3:36 AM, Jozsef Kadlecsik
<kadlec@blackhole.kfki.hu> wrote:
> On Mon, 20 Dec 2010, Reuben Martin wrote:
>
>> On Monday, December 20, 2010, Jozsef Kadlecsik wrote:
>> >
>> > On Sun, 19 Dec 2010, Reuben Martin wrote:
>> >
>> > > I'm trying to put together an extension to forward udp packets to an
>> > > arbritary set of destinations using ipsets. Specifically I want the
>> > > extension to be able to take either an ipmap or iphash as a set of
>> > > destinations for the forwarded udp packets. (I'm starting with the
>> > > rawdnat, stripping out the tcp code, and changing it so that it
>> > > duplicates the packet as needed for destinations instead of changing the
>> > > dest address in the origional packet.
>> >
>> > I don't really see why you need to embed ipset. Why don't you call it as a
>> > normal match and use your extension as a target?
>>
>> I think I have misunderstood what can be done with ipsets. I knew you
>> could use it as a match, but for some reason I though it had an API to
>> allow other extensions to use sets as targets. (i.e. my intention is, as
>> you suggested, to use the extension as a target)
>
> But what is the difference between
>
> ... -m set --match-set foo src -j YOUR_TARGET
>
> and
>
> ... -j YOUR_TARGET
>
> where your target calls internally the same or hardcoded ipset match?

The difference is that I'm not really concerned about the -m option at
all. Any of the existing match methods are sufficient.

I want to be able to do:

... -j MY_TARGET --ipset foo

where MY_TARGET would iterate over the members of foo, and for each
member of foo the original packet is duplicated, the duplicate's
destination is changed to the address value of the member, checksums
recalculated, and sent on it's way.

My intent is to be able to take RTP media stream packets, and
transparently forward them to a dynamically changing group of
destination addresses.in a network where multicast is not an option.
I'm sure there might be other uses for it, but that's all I'm
concerned about at the moment.

>
>> Is there any way via the standard API to request the contents of a named
>> set? That's really what I want to do. Basically for each packet I would
>> request I want to be able to check what addresses are in the set named
>> in the parameters of my extension, and then copy the packets to each of
>> those addresses, and hence have a dynamic list of destination addresses.
>
> In my previous mail I wrote about the current API and possibilities.
>

Yes, thank you so much. I finally got home last night and was able to
take a look at it. I think that in order to do what I want, ipset
would need a new exported function that would provide access to
members of a set.

-Reuben

Re: ipsets: examples?

[Jozsef Kadlecsik]

On Tue, 21 Dec 2010, Reuben Martin wrote:

> I want to be able to do:
> 
> ... -j MY_TARGET --ipset foo
> 
> where MY_TARGET would iterate over the members of foo, and for each
> member of foo the original packet is duplicated, the duplicate's
> destination is changed to the address value of the member, checksums
> recalculated, and sent on it's way.
> 
> My intent is to be able to take RTP media stream packets, and
> transparently forward them to a dynamically changing group of
> destination addresses.in a network where multicast is not an option.
> I'm sure there might be other uses for it, but that's all I'm
> concerned about at the moment.

I see. Sorry, it's not possible to use ipset for this purpose: there are 
no iterators which'd return the members of a set (listing is not similar 
or applicable). To design such an interface is not trivial. Maybe instead 
of iterating, passing your function to ipset is more feasible: ipset would 
call your function for every member of the given set.

(I Cc-ed netfilter-devel, because that's more appropriate list for such a 
discussion.)

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipsets: examples?

[Reuben Martin]

On Wed, Dec 22, 2010 at 4:45 AM, Jozsef Kadlecsik
<kadlec@blackhole.kfki.hu> wrote:
> On Tue, 21 Dec 2010, Reuben Martin wrote:
>
>> I want to be able to do:
>>
>> ... -j MY_TARGET --ipset foo
>>
>> where MY_TARGET would iterate over the members of foo, and for each
>> member of foo the original packet is duplicated, the duplicate's
>> destination is changed to the address value of the member, checksums
>> recalculated, and sent on it's way.
>>
>> My intent is to be able to take RTP media stream packets, and
>> transparently forward them to a dynamically changing group of
>> destination addresses.in a network where multicast is not an option.
>> I'm sure there might be other uses for it, but that's all I'm
>> concerned about at the moment.
>
> I see. Sorry, it's not possible to use ipset for this purpose: there are
> no iterators which'd return the members of a set (listing is not similar
> or applicable). To design such an interface is not trivial. Maybe instead
> of iterating, passing your function to ipset is more feasible: ipset would
> call your function for every member of the given set.

In that case, maybe all that ipset would have to do is duplicate the
packet, and then use RAWDNAT to change the destination addresses for
the duplicates.

-Reuben