Re: long latency of domain shutdown

["Jan Beulich" <jbeulich@novell.com>]

>>> Keir Fraser <keir.fraser@eu.citrix.com> 08.05.08 12:12 >>>
>On 8/5/08 10:58, "Jan Beulich" <jbeulich@novell.com> wrote:
>
>> While looking at this I wondered whether there really is a way for
>> Xen heap pages to end up being guest page tables (or similarly
>> descriptor table ones)? I would think if that happened this would be
>> a bug (and perhaps a security issue). If it cannot happen, then the
>> RELMEM_* states could be simplified and
>> domain_relinquish_resources() shortened.
>
>You mean just force page-table type counts to zero, and drop main reference
>count by the same amount? Might work. Would need some thought.

No, here I mean having just RELMEM_xen and RELMEM_l{1,2,3,4}.
Then simply release Xen pages first, then l4...l1.

For the suggested workaround to reduce latency of relinquish_memory()
preemption, I simply mean utilizing the code to deal with circular
references also for releasing simple ones (that code path doesn't seem
to care to force the type count to zero, but as I understand that's no
problem since these pages end up being freed anyway, and that's
where the whole type_info field gets re-initialized - or was this
happening when the page gets allocated the next time).

>8500 cycles per pte is pretty fantastic. I suppose a few atomic ops are
>involved. Are you running on an old P4? :-)

Not too old, it's what they called Tulsa as codename (i.e. some of the
about two year old Xeons). But I suppose that generally the bigger
the box (in terms of number of threads/cores/sockets) the higher the
price for atomic ops.

In trying to get a picture, I e.g. measured both the cumulative full
free_domheap_pages()' and free_l1_table()'s contributions as well as
just the d != NULL sub-part of free_domheap_pages() - example
results are
0x2a4990400 clocks for full free_l1_table()
0x10f1a3749 clocks for full free_domheap_pages()
0x0ec6748eb clocks for the d != NULL body
Given how little it is that happens outside of that d != NULL body I'm
concluding that the atomic ops are by far not alone responsible for
the long execution time. These are dual-threaded CPUs, however,
so that while the system was doing nothing else I cannot exclude that
the CPUs do something dumb in switching between the threads. But
excluding this possible effect from the picture seems to have little
sense, since we need to be able to deal with the situation anyway.

Jan