Re: RFC: Automatically making a PCI device assignable in the config file

[George Dunlap <george.dunlap@eu.citrix.com>]

On 12/07/13 10:55, David Vrabel wrote:
> On 12/07/13 10:36, George Dunlap wrote:
>> On Thu, Jul 11, 2013 at 12:35 PM, David Vrabel <david.vrabel@citrix.com> wrote:
>>> On 10/07/13 15:48, George Dunlap wrote:
>>>> On 10/07/13 14:53, Ian Jackson wrote:
>>>>> George Dunlap writes ("RFC: Automatically making a PCI device
>>>>> assignable in the config file"):
>>>>>> I've been doing some work to try to make driver domains easier to set
>>>>>> up and use.  At the moment, in order to pass a device through to a
>>>>>> guest, you first need to assign it to pciback.  This involves doing
>>>>>> one of three things:
>>>>>> * Running xl pci-assignable-add for the device
>>>>>> * Specifying the device to be grabbed on the dom0 Linux command-line
>>>>>> * Doing some hackery in /etc/modules.d
>>>>>>
>>>>>> None of these are very satisfying.  What I think would be better is if
>>>>>> there was a way to specify in the guest config file, "If device X is
>>>>>> not assignable, try to make it assignable".  That way you can have a
>>>>>> driver domain grab the appropriate device just by running "xl create
>>>>>> domnet"; and once we have the xendomains script up and running with
>>>>>> xl, you can simply configure your domnet appropriately, and then put
>>>>>> it in /etc/xen/auto, to be started automatically on boot.
>>>>>>
>>>>>> My initial idea was to add a parameter to the pci argument in the
>>>>>> config file; for example:
>>>>>>
>>>>>> pci = ['08:04.1,permissive=1,seize=1']
>>>>>>
>>>>>> The 'seize=1' would indicate that if bdf 08:04.1 is not already
>>>>>> assignable, that xl should try to make is assignable.
>>>>> I think it's a design error that this isn't done automatically by
>>>>> default.
>>>>>
>>>>> It would be nice if there was a safety check that the device isn't
>>>>> currently in use by dom0, but I don't think it's essential.  We could
>>>>> just put a note in the docs saying "if you specify your dom0 nic it
>>>>> will go away, duh" or something.
>>>> I think it's a really bad interface decision if a simple typo might
>>>> result in you yanking out your disk.
>>> I don't think this proposal really helps with avoiding this.  I think
>>> most people will end up always adding 'seize=1' because to avoid having
>>> to do so means altering config files elsewhere and rebooting.
>> I guess what I'm worried about is the fact that we would be changing
>> things that are now "safe" to things that are not safe.  At the
>> moment, "xl pci-assignable-add" might yank out a system device if you
>> make a typo; but it was introduced that way, so people always had to
>> be careful.  But currently, "pci=[]" and "xl pci-attach" do *not*
>> behave that way; you have to make the device assignable first.  So you
>> don't need to be particularly careful.  Adding "seize" at least should
>> flag up to people that they need to double-check.
> You could see if you can determine if a PCI device provides either a
> block device or a network device and then check whether the block device
> is mounted or the network device has a IP address configured.
>
> These checks should probably be handled by a script that xl calls.

I think we'd be opening a can of worms.  For example, if you do ifconfig 
on an eth device that's being used by a bridge, it comes up empty; so 
the script would have to be smart enough to find the associated bridge 
as well.  That seems to be the case both for Linux bridging and 
openvswitch.  This might become arbitrarily complicated for various 
devices: what about a USB root switch with an external hub that has a 
mounted disk?

I think the only maintainable option is to have it just done with no 
checks; the only question is if we should require an option to enable 
the auto-assigning or having it happen by default.

  -George