Re: QoS for iSCSI target?

[Chris Friesen <chris.friesen@windriver.com>]

On 04/04/2016 06:25 PM, Nicholas A. Bellinger wrote:
> On Mon, 2016-04-04 at 17:01 -0600, Chris Friesen wrote:

>> I'm not trying to globally throttle IO on a particular block device.  I'm trying
>> to control how much IO the iSCSI target in the kernel is allowed to drive on a
>> particular block device.
>>
>> The goal is to ensure that the iSCSI target does not consume all of the
>> available bandwidth for a particular block device.  I want to ensure that some
>> of the bandwidth for that device is available to other processes on the host
>> (for management purposes) rather than being consumed by a greedy iSCSI initiator.
>>
>
> AFAIK, different traffic classes for a single block device is not
> supported by block cgroups.
>
> Also given target_core_iblock claims a given block_device for exclusive
> access, you can't actually use the same block device for a mounted
> filesystem, LVM, MD, etc, once it's been registered for target usage.

The issue we're running into is in the context of OpenStack Cinder.  It layers 
LVM on top of the bare block device, then creates an LVM volume per cinder 
volume, and exports the logical volumes via iSCSI.

The problem we're seeing is that if one or more iSCSI initiators drive heavy 
traffic on a slow disk, it can become very slow to access anything at all on the 
bare block device (/dev/sdb or equivalent).  This can cause the server-side 
management code to time out on operations like making a new volume, checking 
free disk space, etc.

>> In an ideal world I would like a set of rules that say things like:
>> 1) if there is contention, ensure that the host is guaranteed X percent of the
>> available /dev/sdb IOPS
>> 2) if there is contention, do not allow the iSCSI target traffic to consume more
>> than Y percent of /dev/sdb's write traffic
>>
>
> Yeah, that doesn't exist in standlone block groups.
>
> You could probably use network cgroups to limit bandwidth at the socket
> level for iscsi_target_mod, but that's going to be across all LUNs in a
> session, and not at individual LUN level.

Are there a set of dedicated kernel threads being used for the iSCSI target?  Or 
is it handled by generic softirq mechanisms?

Chris