>> Yeah, but the set is empty!
>>
>> It is flushed, so why is it that after the set is cleared (and there are no
>> elements in that set!), it still occupies 4 times as much memory it had
>> initially with the same number of elements, i.e. zero? If this isn't a memory
>> leak it is a very bad practice I would think.
>>     
>
> Hashes are never shrunk. The hash was initiated with the size 1024. Then 
> it was doubled, again and again. Even after deleting all the elements, the 
> base structures are there, emptied, ready to occupy new elements.
>   
Right! OK, I just realised that the has value has grown as well, thus 
increasing the memory footprint, which is fair enough i suppose. I also 
assumed that the hash value could double up indefinitely until the 
maxelem is reached, but that turned out not to be the case - the hash 
value doubled up just once:
[root@test1 ~]# ipset -N test hash:ip maxelem 262144
[root@test1 ~]# ipset -A test 10.4.0.0-10.7.255.255
ipset v6.4: Element cannot be added to the set: it's already added
[root@test1 ~]# ipset -F test
[root@test1 ~]# ipset -L test
Name: test
Type: hash:ip
Header: family inet hashsize 2048 maxelem 262144
Size in memory: 32896
References: 0
Members:
[root@test1 ~]#

Is this intentional?

Performing the same set of statements as in my initial post, but with 
maxelem 262144 (the number of ip addresses in that range) as well as 
hashsize set to the same number produces no errors and after "flush" the 
hash set uses the same memory footprint as when there were no elements 
stored there.

> If you don't need a large set anymore, swap it with a smaller one.
>   
What happens if this set is used in iptables statements, would this work?

> The problem is that at some point the conversion has to be done.
> It can be done before feeding the data to ipset too. 
>   
So you think it is a good idea for everybody out there using  ipset to 
get some sort of conversion utility so that hash:net is utilised in that 
manner then?

Wouldn't it be better if you could adapt ipset and make it behave the 
same way as it currently does with hash:ip type set in terms of 
specifying members? The same way as you did with specifying port ranges 
as well, not to mention that in versions 4+ that used to also be the 
case. The hash:net type set is the only exception in the whole range of 
sets on offer!

> hash:net and iptreemap are quite different. Let's look at 10.1.0.0/16: 
> with hash:net that is a single element, interpreted as a network and 
> matching all elements in it. In iptreemap, that's 65536 different, 
> individual IP addresses.
>   
ipset -L on a iptreemap type set tells different - it contains a single 
element - 10.1.0.0/16 - and that's it! It is even better because I could 
use both forms - cidr and ranges alike - and it shows me the ip address 
ranges when I list the actual set. In that respect iptreemap's 
implementation is much better than what currently exists with hash:net 
set in 6.4!

> Convert the ranges to networks and use a hash:net type of set. There are 
> countless tools to do the conversion.
There may be, but that isn't the point - do I have to arm myself with 
yet another tool, which would make my job that bit more complicated and 
difficult (not to mention the time I have to spend adapting to this as 
well as further maintaining it!), compared to if I had ipset implemented 
in such a way, that it is flexible enough to accept IP ranges as well as 
those listed with cidr notation - a flexibility which already existed in 
version 4 of the same tool?

> I see that automatic input 
> conversion could be a useful feature in ipset but at least for a few weeks 
> I'll not be able to deal with it.
>   
I am attaching something to this email (though I am not sure whether 
this would be accepted by the mail list daemon!) to get you started if 
you so desire.