From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zoltan Kiss Subject: Re: Free up completed TX buffers Date: Mon, 01 Jun 2015 18:51:04 +0100 Message-ID: <556C9B88.9080409@linaro.org> References: <55689B42.6060800@linaro.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit To: Andriy Berestovskyy , dev@dpdk.org Return-path: Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) by dpdk.org (Postfix) with ESMTP id 80C1D5A37 for ; Mon, 1 Jun 2015 19:51:06 +0200 (CEST) Received: by wizo1 with SMTP id o1so114813395wiz.1 for ; Mon, 01 Jun 2015 10:51:06 -0700 (PDT) In-Reply-To: List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 01/06/15 09:50, Andriy Berestovskyy wrote: > Hi Zoltan, > > On Fri, May 29, 2015 at 7:00 PM, Zoltan Kiss wrote: >> The easy way is just to increase your buffer pool's size to make >> sure that doesn't happen. > > Go for it! I went for it, my question is whether is it a good and worthwhile idea to give the applications a last resort option for rainy days? It's a problem which probably won't occur very often, but when it does, I think it can take painfully long until you figure out what's wrong. > >> But there is no bulletproof way to calculate such >> a number > > Yeah, there are many places for mbufs to stay :( I would try: > > Mempool size = sum(numbers of all TX descriptors) > + sum(rx_free_thresh) > + (mempool cache size * (number of lcores - 1)) > + (burst size * number of lcores) It heavily depends on what your application does, and I think it's easy to make a mistake in these calculations. > >> I'm thinking about a foolproof way, which is exposing functions like >> ixgbe_tx_free_bufs from the PMDs, so the application can call it as a last >> resort to avoid deadlock. > > Have a look at rte_eth_dev_tx_queue_stop()/start(). Some NICs (i.e. > ixgbe) do reset the queue and free all the mbufs. That's a bit drastic, I just want to flush the finished TX buffers, even if tx_free_thresh were not reached. An easy option would be to use rte_eth_tx_burst(..., nb_pkts=0), I'm already using this to enforce TX completion if it's really needed. It checks for tx_free_thresh, like this: /* Check if the descriptor ring needs to be cleaned. */ if ((txq->nb_tx_desc - txq->nb_tx_free) > txq->tx_free_thresh) i40e_xmit_cleanup(txq); My idea is to extend this condition and add " || nb_pkts == 0", so you can force cleanup. But there might be others who uses this same way to do manual TX completion, and they expect that it only happens when tx_free_thresh is reached. > > Regards, > Andriy >