From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stan Hoeppner Subject: Re: [patch 2/2 v3]raid5: create multiple threads to handle stripes Date: Tue, 12 Mar 2013 19:44:19 -0500 Message-ID: <513FCBE3.90205@hardwarefreak.com> References: <20120809085808.GB30111@kernel.org> <20130307073123.GA5819@kernel.org> <20130312123927.16e49ea9@notabene.brown> Reply-To: stan@hardwarefreak.com Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20130312123927.16e49ea9@notabene.brown> Sender: linux-raid-owner@vger.kernel.org To: NeilBrown Cc: Shaohua Li , linux-raid@vger.kernel.org, dan.j.williams@gmail.com List-Id: linux-raid.ids On 3/11/2013 8:39 PM, NeilBrown wrote: > On Thu, 7 Mar 2013 15:31:23 +0800 Shaohua Li wrote: ... >>> #echo 1-3 > /sys/block/md0/md/auxth0/cpulist >>> This will bind auxiliary thread 0 to cpu 1-3, and this thread will only handle >>> stripes produced by cpu 1-3. User tool can further change the thread's >>> affinity, but the thread can only handle stripes produced by cpu 1-3 till the >>> sysfs entry is changed again. Would it not be better to use the existing cpusets infrastructure for this, instead of manually binding threads to specific cores or sets of cores? Also, I understand the hot cache issue driving the desire to have a raid thread only process stripes created by its CPU. But what of the scenario where an HPC user pins application threads to cores and needs all the L1/L2 cache? Say this user has a dual socket 24 core NUMA system with 2 NUMA nodes per socket, 4 nodes total. Each NUMA node has 6 cores and shared L3 cache. The user pins 5 processes to 5 cores in each node, and wants to pin a raid thread to the remaining core in each node to handle the write IO generated by the 5 user threads on the node. Does your patch series allow this? Using the above example, if the user creates 4 cpusets, can he assign a raid thread to that set and the thread will execute on any core in the set, and only that set, on any stripes created by any CPU in that set, and only that set? The infrastructure for this already exists, has since 2004. And it seems is more flexible than what you've implemented here. I suggest we make use of it, as it is the kernel standard for doing such things. See: http://man7.org/linux/man-pages/man7/cpuset.7.html > Hi Shaohua, > I still have this sitting in my queue, but I haven't had a chance to look at > is properly yet - I'm sorry about that. I'll try to get to it soon. -- Stan