Re: Powerdown problem on XEN | ACPI S5

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 14/08/13 18:00, Atom2 wrote:
> Hi Andrew,
> please see my inline comments further below.
>
> And many thanks to all of you for your support so far!
>
> Am 14.08.13 16:00, schrieb Andrew Cooper:
>> On 14/08/13 14:52, Atom2 wrote:
>>> Hi Jan,
>>> thanks for reply. You are obviously right that the first thing
>>> device_power_down does, is console_suspend(). I don't know why that
>>> escaped my eyes when I originally searched the file ...
>>>
>>> Anyways, I have now disabled console_suspend() and also added a few
>>> more lines to the code with printk statements indicating up to which
>>> point the system had gone (without errors). With hindsight I guess the
>>> new printk() statements might not have been required as now, with the
>>> console still active, a panic message pops up at the end, resulting in
>>> rebooting the system:
>>>
>>> (XEN) Disabling non-boot CPUs ...
>>> (XEN) Entering ACPI S5 state.
>>> (XEN) After local_irq_save
>>> (XEN) After spin_debug_disable
>>> (XEN) After time_suspend
>>> (XEN) After li8259_suspend
>>> (XEN) After ioapic_suspend
>>> (XEN) DMAR_IQA_REG = 80d87c002
>>> (XEN) DMAR_IQH_REG = 120
>>> (XEN) DMAR_IQT_REG = 140
>>> (XEN)
>>> (XEN) ****************************************
>>> (XEN) Panic on CPU 0:
>>> (XEN) queue invalidate wait descriptor was not executed
>>> (XEN) ****************************************
>>> (XEN)
>>> (XEN) Reboot in five seconds...
>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>>
>>> NOTE: All the messages starting with "(XEN) After" are from my changes
>>> to the code; the rest is as is - except me commenting out
>>> console_suspend() in power.c. I hope that helps in resolving the issue
>>> and the panic is not just the result of a knock-on effect from
>>> commenting out console_suspend() earlier.
>>
>> Huh - I thought I had fixed this issue already.
>>
>> Can you confirm exactly which version of Xen you are using (including
>> changeset), and perhaps compile in this patch:
> The version I am using is 4.2.2 from the gentoo ebuild. Interesting
> enough, the log in the consoles states
> (XEN) Latest ChangeSet: unavailable
> I don't know what to make out of this - or is there any other way to
> figure out, what you are after?

Ah - that good old gem.  The old logic to detect changesets was very bad
if the build wasn't happening in a mercurial tree.  It is better in 4.3.

>
> The alternative would be to apply the WARN() to the 4.3.0 version I
> have downloaded yesterday from xenbits at
> http://www.xenproject.org/downloads/xen-archives/supported-xen-43-series/xen-430/287-xen-430-2/file.html
> (FYI: the reboot also happened there). If that helps, I'll rerun it on
> the 4.3.0 version. So far I have used the gentoo version as this
> allows me to stay within the portage system.

That's fine - the bug is likely more damage from XSA-36

>>
>> diff --git a/xen/drivers/passthrough/vtd/qinval.c
>> b/xen/drivers/passthrough/vtd/qinval.c
>> index 6a410d8..d023b26 100644
>> --- a/xen/drivers/passthrough/vtd/qinval.c
>> +++ b/xen/drivers/passthrough/vtd/qinval.c
>> @@ -220,6 +220,7 @@ static int queue_invalidate_wait(struct iommu
>> *iommu,
>>               if ( NOW() > (start_time + DMAR_OPERATION_TIMEOUT) )
>>               {
>>                   print_qi_regs(iommu);
>> +                WARN();
>>                   panic("queue invalidate wait descriptor was not
>> executed\n");
>>               }
>>               cpu_relax();
> I have manually applied the patch - which was just an added
>     WARN();
> inbetween if I read the patch correctly; the rest was already there in
> 4.2.2 (and also 4.3.0 - I checked its source as well). I have attached
> the serial log from my 4.2.2 run to prevent line-wrap. I hope that helps.

Do you have the boot time dmesg output?

The problem here is that a queued_invalidate wait descriptor has been
issued, and has not been completed within 1 second.  In all previous
cases I have debugged, this is actually because we already turned off
the IOMMU, then tried to turn it off again.

Could you perhaps try with this patch as well?

diff --git a/xen/drivers/passthrough/vtd/iommu.c
b/xen/drivers/passthrough/vtd/iommu.c
index 071a91b..45fff48 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -791,6 +791,9 @@ static void iommu_disable_translation(struct iommu
*iommu)
     u32 sts;
     unsigned long flags;
 
+    printk("**Debug: Disabling translation for iommu %"PRId32"\n",
iommu->index);
+    WARN();
+
     /* apply platform specific errata workarounds */
     vtd_ops_preamble_quirk(iommu);
 

~Andrew