Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

[Avi Kivity <avi@redhat.com>]

On 12/29/2011 06:49 PM, Anthony Liguori wrote:
> On 12/29/2011 10:36 AM, Avi Kivity wrote:
>> On 12/29/2011 06:12 PM, Anthony Liguori wrote:
>>>>> That's why I've also proposed qtest.  But having written quite a few
>>>>> qtest unit tests by now, you hit the limits of this type of testing
>>>>> pretty quickly.
>>>>
>>>> Can you describe those limits?
>>>
>>>
>>> I started writing a finger print test.  While it's easy to do PCI
>>> enumeration, it gets pretty nasty if you want to actually access BARs
>>> (to read virtio registers)
>>
>> Why is that? it should be pretty easy to poke values into the BAR
>> registers.
>
> You have to map them which means you need to figure out what the
> memory layout looks like.  Since there's no BIOS, you need to poke
> fw_cfg to figure all of this out and then program the BARs appropriately.
>
> It's definitely non-trivial.

Or just map sequentially at 0xe0000000 ascending.

>
>> Something else we can do is access the BARs directly.  We have APIs to
>> poke values into mmio, add an api to poke a value into a device's BAR.
>> Now that each BAR is represented by exactly one memory region, known to
>> the pci core even if it is not mapped, it should be easy.
>>
>>> and forget about trying to get SMBIOS information as that involves
>>> mucking around with ACPI.
>>
>> SMBIOS is built by seabios, yes?
>
> I think QEMU actually builds the blob of information and passes the
> full blob to SeaBIOS.  You could read and parse the blob out of fw_cfg
> I guess but that's still pretty sucky.  It's a lot nicer to just poke
> sysfs.

Seems like two lines of code, either way.

>
>> So fingerprinting it should be done
>> from seabios-test.git.  Let's divide it into two problems:
>
> That seems awkward especially since fingerprinting Just Works with
> qemu-test already (and it was only 50 lines of shell code!).
>
>> - seabios pokes qemu to get information.  We should fingerprint this
>> information to make sure different qemu version can interoperate with
>> different seabios versions (needed since we migrate the bios along with
>> the rest of the guest).  I'm guessing most of this info is from fwcfg?
>> We can easily fingerprint it like any other device.
>> - we need to make sure seabios generates the same tables when using -M.
>> Here, we're not testing a device, rather we're testing guest code, so it
>> makes sense to use a guest for this.
>>
>> However, I don't think it's even necessary.  From a quick read of the
>> manual, SMBIOS is just a set of static tables in memory which are picked
>> up using a signature.  So all we need to do is boot an empty guest, read
>> its memory, and look for the tables.
>
> Doesn't it sound a whole nicer use linux.git to parse this information
> for us in a friendly, easy to consume fashion?

Run 'dmidecode -d /path/to/memory/dump', if you must.

I don't think the qemu-test approach is bad, per se, it's just that
qtest is better.  It gives you full control over what to fingerprint and
is not reliant on Linux not changing.

>>
>> Yes; but using Linux limits you to what it exposes (of course Linux
>> exposes quite a lot, so that's mostly a non issue; but we'll have
>> counterexamples).
>
>
> Maybe.  And for those counter examples, we can drill down to qtest. 
> But just because we may need to do fancy things, it doesn't mean we
> shouldn't take the easy approach 95% of the time.

I guess we have to see how difficult it will be with qtest.  My feeling
is that it will be very easy.

>> I don't see why.  A python library to read pci config should be a couple
>> of dozens of lines long.  Then you iterate over all functions and print
>> selected registers.
>
>
> PCI enumeration is easy.  It's mapping the bars and then poking other
> sources of information that's more challenging.
>
> Everything is doable by writing a libOS but I'd rather just use Linux
> than invent a libOS just for testing.

We will have to write a libOS to test devices.  To test, say, virtio-net
with indirect buffers, you have to map BARs (or poke unmapped BARs
directly).

>>
>> You mention in the changelog replacing the APIC.  Why can't you do that?
>  
>
> I currently replace the I/O APIC which seems to be limited to 16
> IRQs.  I think MSI takes a side channel directly to the local APIC, no?

Yes, writes to the 1MB @ 0xfee00000 gets translated to MSIs.  So you
just translate them to events.

>> It looks great.  One thing I'd change is to use the qmp protocol
>> (perhaps not the monitor, just the schema/codegen).  Does something
>> prohibit this?
>
>
> Yeah, I thought about using QMP.  But events are posted in QMP which
> means that you never get an explicit ACK.  That may not be a problem
> but it's something I had in mind.
>
> It also seemed to be reasonably complicated without a clear
> advantage.  qtest isn't going to be a supported protocol so we can
> certainly change it down the road if we want to.

It's a pity not to reuse all the tooling?  Seems like self-NIH.

-- 
error compiling committee.c: too many arguments to function