All of lore.kernel.org
 help / color / mirror / Atom feed
From: Matthew Hall <mhall-Hv3ogNYU3JfZZajBQzqCxQ@public.gmane.org>
To: dev-VfR2kkLFssw@public.gmane.org
Subject: Possible bug in eal_pci pci_scan_one
Date: Mon, 6 Oct 2014 02:13:44 -0700	[thread overview]
Message-ID: <20141006091344.GA14759@mhcomputing.net> (raw)

Hi Guys,

I'm doing my development on kind of a cheap machine with no NUMA support... 
but several years ago I used DPDK to build a NUMA box that could do 40 gbits 
bidirectional L4-L7 stateful traffic replay.

So given the past experiences I had before, I wanted to clean the code up so 
it'd work well if some crazy guy tried my code on one of these huge boxes, 
too, but then I ran into some weird issues.

1) When I call rte_eth_dev_socket_id() I get back -1. But the call can return 
-1 if the port_id is bogus or if pci_scan_one didn't get a numa_node (because 
you're on a non-NUMA box for example).

int rte_eth_dev_socket_id(uint8_t port_id)
{
        if (port_id >= nb_ports)
                return -1;
        return rte_eth_devices[port_id].pci_dev->numa_node;
}

So you couldn't tell the different between non-NUMA or a bad port value, etc.

2) The code's behavior and comments disagree with one another. In the 
pci_scan_one function, there's this code:

/* get numa node */
snprintf(filename, sizeof(filename), "%s/numa_node",
         dirname);
if (access(filename, R_OK) != 0) {
        /* if no NUMA support just set node to 0 */
        dev->numa_node = -1;
} else {
        if (eal_parse_sysfs_value(filename, &tmp) < 0) {
                free(dev);
                return -1;
        }
        dev->numa_node = tmp;
}

It says, just use NUMA node 0 if there is no NUMA support. But then proceeds 
to set the value to -1 in disagreement with the comment, and also stomping on 
the other meaning for -1 in the higher function rte_eth_dev_socket_id.

3) In conclusion, it seems like some stuff is missing... first there needs to 
be a function that will tell you the number of NUMA nodes present on the box 
so you can create the right number of mbuf_pools, but I couldn't find that function.

Then if you have the function, you can do some magic and shuffle the NICs 
around to get them hooked to a core on the same NUMA, and the mbuf_pool on the 
same NUMA.

When NUMA is not present, can we return 0 instead of -1, or return a specific 
error code that the client can use to know he should just use Socket 0? Right 
now I can't tell apart any potential errors or weird values from correct 
values.

4) I'm willing to help make and test some patches... but first I want to 
understand what is happening with these funny functions before doing things 
blindly.

Thanks,
Matthew.

             reply	other threads:[~2014-10-06  9:13 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-10-06  9:13 Matthew Hall [this message]
     [not found] ` <20141006091344.GA14759-Hv3ogNYU3JfZZajBQzqCxQ@public.gmane.org>
2014-10-14  5:47   ` Possible bug in eal_pci pci_scan_one Matthew Hall
     [not found]     ` <20141014054711.GB16919-Hv3ogNYU3JfZZajBQzqCxQ@public.gmane.org>
2014-10-23  0:44       ` Matthew Hall
2014-10-24 13:06   ` Stephen Hemminger
     [not found]     ` <20141024183629.0740fd72-CA4OZQ/Yy2Lykuyl+CZolw@public.gmane.org>
2014-10-24 19:03       ` Matthew Hall

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141006091344.GA14759@mhcomputing.net \
    --to=mhall-hv3ognyu3jfzzajbqzqcxq@public.gmane.org \
    --cc=dev-VfR2kkLFssw@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.