Re: [PATCH V6 0/8] blk-mq: improvement CPU hotplug

[Daniel Wagner <dwagner@suse.de>]

On Wed, Apr 08, 2020 at 02:29:51PM +0100, John Garry wrote:
> On 08/04/2020 14:10, Daniel Wagner wrote:
> ok, but to really test this you need to ensure that all the cpus for a
> managed interrupt affinity mask are offlined together for some period of
> time greater than the IO timeout. Otherwise the hw queue's managed interrupt
> would not be shut down, and you're not verifying that the queues are fully
> drained.

Not sure if I understand you correctly: Are you saying that the IRQ
related resources are not freed/moved from the offlining CPU?

> > > Will the fio processes migrate back onto cpus which have been onlined again?
> > 
> > Hmm, good question. I've tried to assign them to a specific CPU via
> > --cpus_allowed_policy=split and --cpus_allowed.
> > 
> >    fio --rw=randwrite --name=test --size=50M --iodepth=32 --direct=1 \
> >        --bs=4k --numjobs=40 --time_based --runtime=1h --ioengine=libaio \
> >        --group_reporting --cpus_allowed_policy=split --cpus_allowed=0-40
> > 
> > Though I haven't verified what happens when the CPU get's back online.
> 
> Maybe this will work since you're offlining patterns of cpus and the fio
> processes have to migrate somewhere. But see above.

At least after the initial setup a fio thread will be migrated away
from the offlining CPU.

A quick test shows, that the affinity mask for a fio thread will be
cleared when the CPU goes offline. There seems to be a discussion
going on about the cpu hotplug and the affinity mask:

https://lore.kernel.org/lkml/1251528473.590671.1579196495905.JavaMail.zimbra@efficios.com

TL;DR: it can be scheduled back if affinity is tweaked via
e.g. taskset, it won't if it's via cpusets

Thanks,
Daniel