problem with rte_pktmbuf_prepend (possible bug?)

problem with rte_pktmbuf_prepend (possible bug?)

[Mario Gianni]

Hi all, I'm experimenting some code with DPDK v1.5.0 and I have the following problem:

I have a thread that receives packets from NIC, once I received a packet I want to prepend some data to it and I try to do so through the function rte_pktmbuf_prepend()
then the packet is enqueued in a ring buffer where it will be used by a client thread before being dropped through the function rte_pktmbuf_free() called by the client thread.

Now, if I try to send packets to this program I have the following behaviour:
In a first time it seems to work correctly, then after a certain number of received packets (approximately the same number as the number of mbufs present in the mempool) if I call the rte_pktmbuf_headroom it returns that the headroom is shrinking more than the expected, until after a certain number of packets the headroom goes to zero.

It seems like that when I call the rte_pktmbuf_free() function it doesn't reset the data position inside the mbuf, so when I call for a second time the mbuf the headroom continues to shrink until it finishes.


Do you have any idea of this strange behaviour?Could it be a bug in the prepend/free function?


Thank you,

Mario

Re: problem with rte_pktmbuf_prepend (possible bug?)

[Periklis Akritidis]

Hi Mario,

Are you passing rte_pktmbuf_init as the obj_init argument to rte_mempool_create?
It is called when the mbuf if freed and it will reset the fields. I vaguely remember I had the same issue at some point and resolved it somehow. This comes to mind.

Cheers,
Periklis

On 18 Feb, 2014, at 6:27 pm, Mario Gianni <m.gianni-Gbsd49wXnelWk0Htik3J/w@public.gmane.org> wrote:

> Hi all, I'm experimenting some code with DPDK v1.5.0 and I have the following problem:
> 
> I have a thread that receives packets from NIC, once I received a packet I want to prepend some data to it and I try to do so through the function rte_pktmbuf_prepend()
> then the packet is enqueued in a ring buffer where it will be used by a client thread before being dropped through the function rte_pktmbuf_free() called by the client thread.
> 
> Now, if I try to send packets to this program I have the following behaviour:
> In a first time it seems to work correctly, then after a certain number of received packets (approximately the same number as the number of mbufs present in the mempool) if I call the rte_pktmbuf_headroom it returns that the headroom is shrinking more than the expected, until after a certain number of packets the headroom goes to zero.
> 
> It seems like that when I call the rte_pktmbuf_free() function it doesn't reset the data position inside the mbuf, so when I call for a second time the mbuf the headroom continues to shrink until it finishes.
> 
> 
> Do you have any idea of this strange behaviour?Could it be a bug in the prepend/free function?
> 
> 
> Thank you,
> 
> Mario

Re: problem with rte_pktmbuf_prepend (possible bug?)

[Benson, Bryan]

Whoa, be careful, we used a custom init function and had a nasty bug because we assumed it was called on mbuf_free as well.  The rte_pktmbuf_init function pointer passed into mempool create is only used at pool initialization time, not when an mbuf is freed - A reference to the obj_initi function is not stored anywhere for future use during mbuf_free.

Some of the fields are reset when the NIC has completed the send of the mbufs, but it does not use a custom function ptr.

Thanks,
Bryan Benson

________________________________________
From: dev [dev-bounces-VfR2kkLFssw@public.gmane.org] on behalf of Periklis Akritidis [akritid@niometrics.com]
Sent: Tuesday, February 18, 2014 6:19 AM
To: Mario Gianni
Cc: dev-VfR2kkLFssw@public.gmane.org
Subject: Re: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?)

Hi Mario,

Are you passing rte_pktmbuf_init as the obj_init argument to rte_mempool_create?
It is called when the mbuf if freed and it will reset the fields. I vaguely remember I had the same issue at some point and resolved it somehow. This comes to mind.

Cheers,
Periklis

On 18 Feb, 2014, at 6:27 pm, Mario Gianni <m.gianni-Gbsd49wXnelWk0Htik3J/w@public.gmane.org> wrote:

> Hi all, I'm experimenting some code with DPDK v1.5.0 and I have the following problem:
>
> I have a thread that receives packets from NIC, once I received a packet I want to prepend some data to it and I try to do so through the function rte_pktmbuf_prepend()
> then the packet is enqueued in a ring buffer where it will be used by a client thread before being dropped through the function rte_pktmbuf_free() called by the client thread.
>
> Now, if I try to send packets to this program I have the following behaviour:
> In a first time it seems to work correctly, then after a certain number of received packets (approximately the same number as the number of mbufs present in the mempool) if I call the rte_pktmbuf_headroom it returns that the headroom is shrinking more than the expected, until after a certain number of packets the headroom goes to zero.
>
> It seems like that when I call the rte_pktmbuf_free() function it doesn't reset the data position inside the mbuf, so when I call for a second time the mbuf the headroom continues to shrink until it finishes.
>
>
> Do you have any idea of this strange behaviour?Could it be a bug in the prepend/free function?
>
>
> Thank you,
>
> Mario

Re: problem with rte_pktmbuf_prepend (possible bug?)

[Mario Gianni]

This is my init function:

pktmbuf_pool[socketid] = rte_mempool_create(
 name,
 APP_DEFAULT_MEMPOOL_BUFFERS,
 APP_DEFAULT_MBUF_SIZE,
 APP_DEFAULT_MEMPOOL_CACHE_SIZE,
 sizeof(struct rte_pktmbuf_pool_private),
 rte_pktmbuf_pool_init, NULL,
 rte_pktmbuf_init, NULL,
 socketid,
 0);
I copied it from a sample code.

Anyway it seems to me that mbuf_free does not reset the *data pointer, actually I have partially resolved by prepending manually the data without moving the * data pointer at all albeit it's not a very clean solution.

Could this be a bug in the DPDK mbuf_free() function?
----- Original Message -----
From: Benson, Bryan
Sent: 02/18/14 06:16 PM
To: Periklis Akritidis, Mario Gianni
Subject: RE: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?)

Whoa, be careful, we used a custom init function and had a nasty bug because we assumed it was called on mbuf_free as well. The rte_pktmbuf_init function pointer passed into mempool create is only used at pool initialization time, not when an mbuf is freed - A reference to the obj_initi function is not stored anywhere for future use during mbuf_free. Some of the fields are reset when the NIC has completed the send of the mbufs, but it does not use a custom function ptr. Thanks, Bryan Benson ________________________________________ From: dev [dev-bounces-VfR2kkLFssw@public.gmane.org] on behalf of Periklis Akritidis [akritid-wlH+TjtWvpzuufBYgWm87A@public.gmane.org] Sent: Tuesday, February 18, 2014 6:19 AM To: Mario Gianni Cc: dev-VfR2kkLFssw@public.gmane.org Subject: Re: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?) Hi Mario, Are you passing rte_pktmbuf_init as the obj_init argument to rte_mempool_create? It is called when the mbuf if freed and it will reset the fields. I vaguely remember I had the same issue at some point and resolved it somehow. This comes to mind. Cheers, Periklis On 18 Feb, 2014, at 6:27 pm, Mario Gianni <m.gianni-Gbsd49wXnelWk0Htik3J/w@public.gmane.org> wrote: > Hi all, I'm experimenting some code with DPDK v1.5.0 and I have the following problem: > > I have a thread that receives packets from NIC, once I received a packet I want to prepend some data to it and I try to do so through the function rte_pktmbuf_prepend() > then the packet is enqueued in a ring buffer where it will be used by a client thread before being dropped through the function rte_pktmbuf_free() called by the client thread. > > Now, if I try to send packets to this program I have the following behaviour: > In a first time it seems to work correctly, then after a certain number of received packets (approximately the same number as the number of mbufs present in the mempool) if I call the rte_pktmbuf_headroom it returns that the headroom is shrinking more than the expected, until after a certain number of packets the headroom goes to zero. > > It seems like that when I call the rte_pktmbuf_free() function it doesn't reset the data position inside the mbuf, so when I call for a second time the mbuf the headroom continues to shrink until it finishes. > > > Do you have any idea of this strange behaviour?Could it be a bug in the prepend/free function? > > > Thank you, > > Mario

Re: problem with rte_pktmbuf_prepend (possible bug?)

[Ananyev, Konstantin]

Hi Mario,
Yes rte_pktmbuf_free doesn't touch data/data_len fields.
In fact for 'normal' mbuf it would only update refcnt.
It is responsibility of alloc path to reset all mbuf fields to some initial values.
Usually it is done by: rte_pktmbuf_alloc()->rte_pktmbuf_reset().
If the code path doesn't use rte_pktmbuf_alloc() to allocate mbufs, then it its own responsibility to reset mbuf fields properly.
ixgbe and e1000 PMDs work that way - they use their own alloc routinies (for example to alloc a bulk of mbufs at once) and then reset mbuf fields manually.
Thanks
Konstantin    

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Mario Gianni
Sent: Wednesday, February 19, 2014 2:05 PM
To: Benson, Bryan; Periklis Akritidis
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?)

This is my init function:

pktmbuf_pool[socketid] = rte_mempool_create(  name,  APP_DEFAULT_MEMPOOL_BUFFERS,  APP_DEFAULT_MBUF_SIZE,  APP_DEFAULT_MEMPOOL_CACHE_SIZE,  sizeof(struct rte_pktmbuf_pool_private),  rte_pktmbuf_pool_init, NULL,  rte_pktmbuf_init, NULL,  socketid,  0); I copied it from a sample code.

Anyway it seems to me that mbuf_free does not reset the *data pointer, actually I have partially resolved by prepending manually the data without moving the * data pointer at all albeit it's not a very clean solution.

Could this be a bug in the DPDK mbuf_free() function?
----- Original Message -----
From: Benson, Bryan
Sent: 02/18/14 06:16 PM
To: Periklis Akritidis, Mario Gianni
Subject: RE: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?)

Whoa, be careful, we used a custom init function and had a nasty bug because we assumed it was called on mbuf_free as well. The rte_pktmbuf_init function pointer passed into mempool create is only used at pool initialization time, not when an mbuf is freed - A reference to the obj_initi function is not stored anywhere for future use during mbuf_free. Some of the fields are reset when the NIC has completed the send of the mbufs, but it does not use a custom function ptr. Thanks, Bryan Benson ________________________________________ From: dev [dev-bounces@dpdk.org] on behalf of Periklis Akritidis [akritid@niometrics.com] Sent: Tuesday, February 18, 2014 6:19 AM To: Mario Gianni Cc: dev@dpdk.org Subject: Re: [dpdk-dev] problem with rte_pktmbuf_prepend (possible bug?) Hi Mario, Are you passing rte_pktmbuf_init as the obj_init argument to rte_mempool_create? It is called when the mbuf if freed and it will reset the fields. I vaguely remember I had the same issue at some point and resolved
  it somehow. This comes to mind. Cheers, Periklis On 18 Feb, 2014, at 6:27 pm, Mario Gianni <m.gianni@engineer.com> wrote: > Hi all, I'm experimenting some code with DPDK v1.5.0 and I have the following problem: > > I have a thread that receives packets from NIC, once I received a packet I want to prepend some data to it and I try to do so through the function rte_pktmbuf_prepend() > then the packet is enqueued in a ring buffer where it will be used by a client thread before being dropped through the function rte_pktmbuf_free() called by the client thread. > > Now, if I try to send packets to this program I have the following behaviour: > In a first time it seems to work correctly, then after a certain number of received packets (approximately the same number as the number of mbufs present in the mempool) if I call the rte_pktmbuf_headroom it returns that the headroom is shrinking more than the expected, until after a certain number of packets the headroom goes to zero. > > It seems
  like that when I call the rte_pktmbuf_free() function it doesn't reset the data position inside the mbuf, so when I call for a second time the mbuf the headroom continues to shrink until it finishes. > > > Do you have any idea of this strange behaviour?Could it be a bug in the prepend/free function? > > > Thank you, > > Mario
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.