Re: Adding new custom devices to Xen via QEMU

[Jason Dickens <jdickens@grammatech.com>]

On 9/30/2016 3:47 PM, Konrad Rzeszutek Wilk wrote:
> On Fri, Sep 30, 2016 at 03:20:09PM -0400, Jason Dickens wrote:
>> Thanks Konrad,
> [CC-ing Xen-devel again.]
>> I think you and David have successfully answered my question and pointed me
>> to the key code. I have already verified that the device operates if I move
>> it into the space of the TPM, but see below for reasons why I don't really
>> want that.  The conclusion I'm drawing from your help is that to add a
>> device where I need it,  I have to modify xen at least for areas set up in
>> xen_ram_init. I've also made a few comments inline below. Its perhaps worth
>> the Xen team looking at why such modification is not necessary for KVM and
>> considering supporting something more automatic. I don't know but I suspect
>> that for KVM, RAM is anything not overridden by a hardware device.
> I don't know KVM enough to tell you.
>
> Keep in mind that under Xen you can launch guests without QEMU. That
> means the orchestration and layout of memory is not in the hands
> of QEMU (like it is with KVM). Hence xen_ram_init follows the suit of
> what the ABI expects (where the MMIO region is, etc).
It makes sense. I;m aware we are doing very non-standard things and it 
didn't surprise me that it would expect a device where I put it.
>
> This is all good if you have an emulated IO device (which are under 1MB)
> or an emulated PCI device as they all follow the norm an allocate
> themsevles in well understood locations where there are no RAM.
>> Jason
>>
>> On 9/30/2016 2:42 PM, Konrad Rzeszutek Wilk wrote:
>>> On Fri, Sep 30, 2016 at 10:29:20AM -0400, Jason Dickens wrote:
>>>> Thanks David,
>>>>
>>>> This could very well be the issue, but could you please elaborate?
>>>> The questions that come up are the following:
>>>> What is the physical address range given to RAM? What range of addresses
>>>> would work for my device?
>>> I am assuming that you implemented the emulation the same way
>>> as other devices - that is you picked an MMIO region for your
>>> device?
>> Yes its essentially the same way of choosing memory as the tpm-tis.c
>> implementation.
> Which AFAICT works with Xen.
Actually, I think it was originally developed for a custom Xen 
implementation. And yes, it works.
>>>> And, if this is the case, how would I unpopulate the RAM?
>>> See xen_ram_init. But I would just choose an region that is
>>> most definitly in MMIO (or IO) region for your emulation.
>> As I said in a previous post there are important reasons why I need this
>> device in a non-standard location. The nature the project has me searching
>> for a sanitized but satisfying explanation for this post.  Its not that I
>> couldn't move it, as I said above I tried the setting the address range in
>> the TPM space and it worked.  I think what I'll say is the following:
>> 1. Its for a proprietary, transparent, and invisible security feature.
>> 2. It has to collaborate with other transparent features which help define
>> its location (perhaps restrict the location is more correct).
>>
>>>> There are reasons for the address chosen, and it works on other hypervisors
>>>> (e.g. KVM) so although it might be easiest to change the address I really
>>> What qemu call do you use to carve out the ranges for your device?
>> The realization function uses:
>> memory_region_add_subregion(isa_address_space(ISA_DEVICE(dev)),PORT_ADDR_BASE,
>> &s->mmio);
> Which should have worked? It didn't?
Of course this does work for the normal device space. It all depends on 
what PORT_ADDR_BASE is set to whether it works or not. I think David's 
comment about overlapping with RAM was correct in our case, and the 
primary problem.
>
>>>> don't want to unless its the only way to keep from a Xen modification
>>>> entirely.
>>>> Jason
>>>>
>>>> On 9/30/2016 9:53 AM, David Vrabel wrote:
>>>>> On 30/09/16 14:35, Jason Dickens wrote:
>>>>>> Hi Wei,
>>>>>>
>>>>>> Thanks for the response. It make sense to me that if the device were on
>>>>>> the PCI bus (or other such bus, e.g. USB) that it could be discovered,
>>>>>> at least by an OS. Its something to consider. I should mention that our
>>>>>> guest VM doesn't actually use an OS.
>>>>>>
>>>>>> However, the device is not implemented that as PCI it is simply memory
>>>>>> mapped. Technically, in QEMU is has type ISA because it was derived as a
>>>>>> modification of the TPM device. Is it possible something is lacking in
>>>>>> the QEMU model that Xen needs but KVM doesn't?
>>>>>> If the answer is that Xen should not need modification for any new
>>>>>> devices then this gives me hope.  You've also inspired some things to
>>>>>> try, like whether or not smaller modifications to the TPM device work.
>>>>>> One change that is significant to mention is that the physical address
>>>>>> range use is anomalous, by which I mean it not in the normal device range.
>>>>> Does device MMIO overlap with guest RAM?  If so, you'll need to
>>>>> unpopulate the RAM first.
>>>>>
>>>>> David
>>>>
>>>> _______________________________________________
>>>> Xen-devel mailing list
>>>> Xen-devel@lists.xen.org
>>>> https://lists.xen.org/xen-devel
>>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel