From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Tan, Jianfeng" Subject: Re: [PATCH] eal: add option --avail-cores to detect lcores Date: Wed, 9 Mar 2016 22:55:51 +0800 Message-ID: <56E03977.7050103@intel.com> References: <1453661393-85704-1-git-send-email-jianfeng.tan@intel.com> <1457085957-115339-1-git-send-email-jianfeng.tan@intel.com> <56DE9359.1090705@redhat.com> <56DF0E0A.8000108@intel.com> <56E01F94.2060906@redhat.com> <56E02AC2.7010704@intel.com> <2601191342CEEE43887BDE71AB97725836B1A536@irsmsx105.ger.corp.intel.com> <56E03078.3000501@intel.com> <2601191342CEEE43887BDE71AB97725836B1A5A2@irsmsx105.ger.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit To: "Ananyev, Konstantin" , Panu Matilainen , "dev@dpdk.org" Return-path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 00ED42B96 for ; Wed, 9 Mar 2016 15:55:58 +0100 (CET) In-Reply-To: <2601191342CEEE43887BDE71AB97725836B1A5A2@irsmsx105.ger.corp.intel.com> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Konstantin, On 3/9/2016 10:44 PM, Ananyev, Konstantin wrote: > >> -----Original Message----- >> From: Tan, Jianfeng >> Sent: Wednesday, March 09, 2016 2:17 PM >> To: Ananyev, Konstantin; Panu Matilainen; dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH] eal: add option --avail-cores to detect lcores >> >> >> >> On 3/9/2016 10:01 PM, Ananyev, Konstantin wrote: >>>> -----Original Message----- >>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tan, Jianfeng >>>> Sent: Wednesday, March 09, 2016 1:53 PM >>>> To: Panu Matilainen; dev@dpdk.org >>>> Subject: Re: [dpdk-dev] [PATCH] eal: add option --avail-cores to detect lcores >>>> >>>> >>>> >>>> On 3/9/2016 9:05 PM, Panu Matilainen wrote: >>>>> On 03/08/2016 07:38 PM, Tan, Jianfeng wrote: >>>>>> Hi Panu, >>>>>> >>>>>> On 3/8/2016 4:54 PM, Panu Matilainen wrote: >>>>>>> On 03/04/2016 12:05 PM, Jianfeng Tan wrote: >>>>>>>> This patch adds option, --avail-cores, to use lcores which are >>>>>>>> available >>>>>>>> by calling pthread_getaffinity_np() to narrow down detected cores >>>>>>>> before >>>>>>>> parsing coremask (-c), corelist (-l), and coremap (--lcores). >>>>>>>> >>>>>>>> Test example: >>>>>>>> $ taskset 0xc0000 ./examples/helloworld/build/helloworld \ >>>>>>>> --avail-cores -m 1024 >>>>>>>> >>>>>>>> Signed-off-by: Jianfeng Tan >>>>>>>> Acked-by: Neil Horman >>>>>>> Hmm, to me this sounds like something that should be done always so >>>>>>> there's no need for an option. Or if there's a chance it might do the >>>>>>> wrong thing in some rare circumstance then perhaps there should be a >>>>>>> disabler option instead? >>>>>> Thanks for comments. >>>>>> >>>>>> Yes, there's a use case that we cannot handle. >>>>>> >>>>>> If we make it as default, DPDK applications may fail to start, when user >>>>>> specifies a core in isolcpus and its parent process (say bash) has a >>>>>> cpuset affinity that excludes isolcpus. Originally, DPDK applications >>>>>> just blindly do pthread_setaffinity_np() and it always succeeds because >>>>>> it always has root privilege to change any cpu affinity. >>>>>> >>>>>> Now, if we do the checking in rte_eal_cpu_init(), those lcores will be >>>>>> flagged as undetected (in my older implementation) and leads to failure. >>>>>> To make it correct, we would always add "taskset mask" (or other ways) >>>>>> before DPDK application cmd lines. >>>>>> >>>>>> How do you think? >>>>> I still think it sounds like something that should be done by default >>>>> and maybe be overridable with some flag, rather than the other way >>>>> around. Another alternative might be detecting the cores always but if >>>>> running as root, override but with a warning. >>>> For your second solution, only root can setaffinity to isolcpus? >>>> Your first solution seems like a promising way for me. >>>> >>>>> But I dont know, just wondering. To look at it from another angle: why >>>>> would somebody use this new --avail-cores option and in what >>>>> situation, if things "just work" otherwise anyway? >>>> For DPDK applications, the most common case to initialize DPDK is like >>>> this: "$dpdk-app [options for DPDK] -- [options for app]", so users need >>>> to specify which cores to run and how much hugepages are used. Suppose >>>> we need this dpdk-app to run in a container, users already give those >>>> information when they build up the cgroup for it to run inside, this >>>> option or this patch is to make DPDK more smart to discover how much >>>> resource will be used. Make sense? >>> But then, all we need might be just a script that would extract this information from the system >>> and form a proper cmdline parameter for DPDK? >> Yes, a script will work. Or to construct (argc, argv) to call >> rte_eal_init() in the application. But as Neil Horman once suggested, a >> simple pthread_getaffinity_np() will get all things done. So if it worth >> a patch here? > Don't know... > Personally I would prefer not to put extra logic inside EAL. > For me - there are too many different options already. Then how about make it default in rte_eal_cpu_init()? And it is already known it will bring trouble to those use isolcpus users, they need to add "taskset [mask]" before starting a DPDK app. > From other side looking at the patch itself: > You are updating lcore_count and lcore_config[],based on physical cpu availability, > but these days it is not always one-to-one mapping between EAL lcore and physical cpu. > Shouldn't that be taken into account? I have not see the problem so far, because this work is done before parsing coremask (-c), corelist (-l), and coremap (--lcores). If a core is disabled here, it's like it is not detected in rte_eal_cpu_init(). Or could you please give more hints? Thanks, Jianfeng > Konstantin > > >