From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============3083226943789390389==" MIME-Version: 1.0 From: Harris, James R Subject: [SPDK] Re: About "core_mask" for DPDK EAL Date: Thu, 27 Feb 2020 16:01:38 +0000 Message-ID: <74C855EC-AEB9-4089-9099-D9F5F5F2E774@intel.com> In-Reply-To: 4B5E6274-E569-4BBB-B12C-E85B73E42F2D@intel.com List-ID: To: spdk@lists.01.org --===============3083226943789390389== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Hi Niu, Do you have the ability to call spdk_env_init() before setting the CPU affi= nity? If not, you'll likely need to save the CPU affinity before calling spdk_env= _init(), and then set it back again after spdk_env_init() returns. -Jim = =EF=BB=BFOn 2/27/20, 8:54 AM, "Niu, Yawei" wrote: Hi, Jim = Our threading model looks like: On server start, the service process cr= eates all service threads one by one, each thread function will set CPU aff= inity before doing anything, and only the first created thread will call sp= dk_env_init(). So in my case, will the following created threads inherit the affinity = set by spdk_env_init() from the first thread? And I suppose calling spdk_un= affinitize_thread() after spdk_env_init() could break our CPU affinity as w= ell, so we need re-set affinity after the spdk_unaffinitize_thread() call? = Thanks -Niu = On 2020/2/27, 11:11 PM, "Harris, James R" = wrote: = Hi Niu, = The core mask is used to specify which cores will be used for polli= ng threads managed by the SPDK/DPDK application framework. There really is= n't a way to disable the core mask, but there are some ways to minimize its= effects. = I'm curious - in your environment, when do you create your DAOS thr= eads? And you create them from the same thread that is calling spdk_env_in= it()? = Note that the thread calling spdk_env_init() will be pinned to that= single core you've specified in your core mask. And threads created after= spdk_env_init() will inherit that affinity. You can call spdk_unaffinitiz= e_thread() immediately after spdk_env_init() returns to eliminate that affi= nity. Could you try that and see if it helps when specifying core_mask 0x1= in your environment? = -Jim = = = On 2/27/20, 2:02 AM, "Niu, Yawei" wrote: = Hi, = We (DAOS) used to call spdk_env_init() with default =E2=80=98co= re_mask=E2=80=99 =E2=80=9C0x1=E2=80=9D to initialize DPDK EAL, during recen= t tests, we realized that performance can be boosted a lot by changing the = =E2=80=98core_mask=E2=80=99 to some other value like =E2=80=980x8=E2=80=99,= so I=E2=80=99m wondering if it because some sort of DPDK service thread wa= s stealing CPU cycles from DAOS service thread, and when the =E2=80=9Ccore_= mask=E2=80=9D is changed, the DPDK thread is happen to be created on some u= nused core? But I didn=E2=80=99t observe any suspicious thread by =E2=80=98= ps=E2=80=99 command (with core_mask =E2=80=980x1=E2=80=99 or =E2=80=980x8= =E2=80=99). = So my question is what this =E2=80=98core_mask=E2=80=99 is used= for? I suppose it=E2=80=99s not useful for our server (we have our own pol= ling thread and user level scheduler), is there any way to disable it? What= =E2=80=99s the best practice of this =E2=80=98core_mask=E2=80=99 configurat= ion for SPDK apps? = Thanks -Niu _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org = = _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org = = _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org = --===============3083226943789390389==--