Re: [PATCH 2/2] ixgbe: don't override mbuf buffer length

[Bruce Richardson <bruce.richardson-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>]

On Thu, Dec 04, 2014 at 03:29:04PM +0000, Ananyev, Konstantin wrote:
> 
> 
> > -----Original Message-----
> > From: Richardson, Bruce
> > Sent: Thursday, December 04, 2014 3:15 PM
> > To: Ananyev, Konstantin
> > Cc: Jean-Mickael Guerin; dev-VfR2kkLFssw@public.gmane.org
> > Subject: Re: [dpdk-dev] [PATCH 2/2] ixgbe: don't override mbuf buffer length
> > 
> > On Thu, Dec 04, 2014 at 02:50:11PM +0000, Ananyev, Konstantin wrote:
> > > Hi,
> > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces-VfR2kkLFssw@public.gmane.org] On Behalf Of Jean-Mickael Guerin
> > > > Sent: Thursday, December 04, 2014 2:26 PM
> > > > To: dev-VfR2kkLFssw@public.gmane.org
> > > > Subject: [dpdk-dev] [PATCH 2/2] ixgbe: don't override mbuf buffer length
> > > >
> > > > The template mbuf_initializer is hard coded with a buflen which
> > > > might have been set differently by the application at the time of
> > > > mbuf pool creation.
> > > >
> > > > Switch to a mbuf allocation, to fetch the correct default values.
> > > > There is no performance impact because this is not a data-plane API.
> > > >
> > > > Signed-off-by: Jean-Mickael Guerin <jean-mickael.guerin-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
> > > > Acked-by: David Marchand <david.marchand-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>
> > > > Fixes: 0ff3324da2 ("ixgbe: rework vector pmd following mbuf changes")
> > > > ---
> > > >  lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c | 19 ++++++++++++-------
> > > >  1 file changed, 12 insertions(+), 7 deletions(-)
> > > >
> > > > diff --git a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
> > > > index c1b5a78..f7b02f5 100644
> > > > --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
> > > > +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx_vec.c
> > > > @@ -732,17 +732,22 @@ static struct ixgbe_txq_ops vec_txq_ops = {
> > > >  int
> > > >  ixgbe_rxq_vec_setup(struct igb_rx_queue *rxq)
> > > >  {
> > > > -	struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */
> > > > +	struct rte_mbuf *mb_def;
> > > >
> > > > -	mb_def.nb_segs = 1;
> > > > -	mb_def.data_off = RTE_PKTMBUF_HEADROOM;
> > > > -	mb_def.buf_len = rxq->mb_pool->elt_size - sizeof(struct rte_mbuf);
> > > > -	mb_def.port = rxq->port_id;
> > > > -	rte_mbuf_refcnt_set(&mb_def, 1);
> > > > +	mb_def = rte_pktmbuf_alloc(rxq->mb_pool);
> > >
> > > Could you explain to me, what is an advantage of using dynamic allocation vs local struct here?
> > > I don't see any.
> > 
> > It means that we get an mbuf that is initialized as done by the initialization
> > function passed to the mempool_create call. The static variable method assumes
> > that we configure the mbuf using default setting for buf_len etc.
> > 
> 
> I understand that, but  why it can't be done in some other way?
> Without allocating/freeing?
> Let say, at mempool_create() store obj_init() and then add ability to call it again?
> Anyway, it doesn't look to me like a critical problem, that requires an urgent patch for 1.8.
> 
> > > Plus if rte_pktmbuf_alloc() would fail, we'll leave our rx queue not configured properly.
> > > As I can see ixgbe_dev_rx_queue_setup() doesn't check the return value of >  ixgbe_rxq_vec_setup()
> > > (as it is just not supposed to fail).
> > > So ixgbe_dev_rx_queue_setup() will return OK for unconfigured RX queue.
> > 
> > Good catch, that's something that should perhaps be looked at in a V2 patch, I
> > think.
> > 
> > >
> > > > +	if (mb_def == NULL) {
> > > > +		PMD_INIT_LOG(ERR, "ixgbe_rxq_vec_setup: could not allocate one mbuf");
> > > > +		return -1;
> > > > +	}
> > > > +	/* nb_segs, refcnt, data_off and buf_len are already set */
> > > > +	mb_def->port = rxq->port_id;
> > > >
> > > >  	/* prevent compiler reordering: rearm_data covers previous fields */
> > > >  	rte_compiler_barrier();
> > >
> > > I don't think we need it here.
> > 
> > I think we might, as the compiler doesn't know that the rearm data overlaps
> > with the previously set fields, so may reorder the reads and writes thinking
> > they are independent.
> 
> Why it doesn't?
> I suppose compiler has all the knowledge of the mbuf structure layout at that point.
> Or there is a some sort of bug in some version of the compiler?
> 

No, we're just violating the layout here by dereferencing past the end of the array :-)

/Bruce

> > >
> > > > -	rxq->mbuf_initializer = *((uint64_t *)&mb_def.rearm_data);
> > > > +	rxq->mbuf_initializer = *((uint64_t *)&mb_def->rearm_data);
> > > > +
> > > > +	rte_pktmbuf_free(mb_def);
> > > > +
> > > >  	return 0;
> > > >  }
> > > >
> > > > --
> > > > 2.1.3
> > >
> > > Somy vote -  NACK for the whole series.
> > > Konstantin
> > >