LRU using DPDK 1.7

LRU using DPDK 1.7

[Saha, Avik (AWS)]

Hello
   I was wondering if there is way to use the rte_table_hash_lru without building a pipeline - Basically using the same hash table like functionality of add, delete and lookup without setting up a pipeline and connect it to ports etc.

Thanks
Avik

Re: LRU using DPDK 1.7

[Matthew Hall]

On Tue, Sep 23, 2014 at 01:08:21AM +0000, Saha, Avik (AWS) wrote:
> I was wondering if there is way to use the rte_table_hash_lru without 
> building a pipeline - Basically using the same hash table like functionality 
> of add, delete and lookup without setting up a pipeline and connect it to 
> ports etc.

I've been finding that rte_hash is designed only for some very specialized 
purposes. It doesn't work well if you use unexpected sizes of keys or want 
behavior that isn't precisely doing what the designers of the hash used it 
for... it's not very general-purpose.

I did try to point out one example of the issue but I didn't get much response 
yet to my questions about its limitations and whether a more general-purpose 
table was available, or at least some discussion what rte_hash is for and what 
it's not for.

Matthew.

Re: LRU using DPDK 1.7

[Venkatesan, Venky]

On 9/22/2014 6:33 PM, Matthew Hall wrote:
> On Tue, Sep 23, 2014 at 01:08:21AM +0000, Saha, Avik (AWS) wrote:
>> I was wondering if there is way to use the rte_table_hash_lru without
>> building a pipeline - Basically using the same hash table like functionality
>> of add, delete and lookup without setting up a pipeline and connect it to
>> ports etc.
> I've been finding that rte_hash is designed only for some very specialized
> purposes. It doesn't work well if you use unexpected sizes of keys or want
> behavior that isn't precisely doing what the designers of the hash used it
> for... it's not very general-purpose.
Yes, it isn't very general purpose as of now (arbitrary key sizes, 
failure path on add if bucket is full etc.). It was designed to be a 
tuple matcher, albeit for a much different performance level than what 
we have driven the DPDK to at present.

That said, the structure of this is something that I expect that we will 
change to make much more flexible, and have a few ideas I'm exploring 
along that line. What would be helpful is start adding to that list of 
limitations from your perspective so that we can address as much of that 
with one patch set.
>
> I did try to point out one example of the issue but I didn't get much response
> yet to my questions about its limitations and whether a more general-purpose
> table was available, or at least some discussion what rte_hash is for and what
> it's not for.
>
> Matthew.

I missed the original email you refer to - I'll dig that up as well from 
the archives.

Regards,
-Venky

Re: LRU using DPDK 1.7

[Saha, Avik (AWS)]

So with DPDK 1.7 there are 2 separate implementations - one is the rte_hash which does not support LRU (at least to my understanding - I could be wrong here) and then there is the librte_table library which has support for LRU in a hash table. I m a little confused as to which one you are referring to Matthew.

-----Original Message-----
From: Matthew Hall [mailto:mhall-Hv3ogNYU3JfZZajBQzqCxQ@public.gmane.org] 
Sent: Monday, September 22, 2014 6:34 PM
To: Saha, Avik (AWS)
Cc: dev-VfR2kkLFssw@public.gmane.org
Subject: Re: [dpdk-dev] LRU using DPDK 1.7

On Tue, Sep 23, 2014 at 01:08:21AM +0000, Saha, Avik (AWS) wrote:
> I was wondering if there is way to use the rte_table_hash_lru without 
> building a pipeline - Basically using the same hash table like 
> functionality of add, delete and lookup without setting up a pipeline 
> and connect it to ports etc.

I've been finding that rte_hash is designed only for some very specialized purposes. It doesn't work well if you use unexpected sizes of keys or want behavior that isn't precisely doing what the designers of the hash used it for... it's not very general-purpose.

I did try to point out one example of the issue but I didn't get much response yet to my questions about its limitations and whether a more general-purpose table was available, or at least some discussion what rte_hash is for and what it's not for.

Matthew.

Re: LRU using DPDK 1.7

[Matthew Hall]

On Tue, Sep 23, 2014 at 03:43:59AM +0000, Saha, Avik (AWS) wrote:

> So with DPDK 1.7 there are 2 separate implementations - one is the rte_hash 
> which does not support LRU (at least to my understanding - I could be wrong 
> here) and then there is the librte_table library which has support for LRU 
> in a hash table. I m a little confused as to which one you are referring to 
> Matthew.

I'm referring to the fact that rte_hash'es of all types are kind of 
special-purpose. So it's important to clarify what kind of data you're 
planning to match, using which of the hashes, and which strategy.

In my case I wanted to use them for variable-length data which will likely not 
fit onto a cacheline such as URL's, and they don't work for this application.

Matthew.