Re: [PROPOSAL] Doing work in idle-vcpu context

[Dulloor <dulloor@gmail.com>]

On Mon, Apr 19, 2010 at 6:52 AM, Keir Fraser <keir.fraser@eu.citrix.com> wrote:
> So I've now implemented this at the tip of xen-unstable staging tree. Except
> that I retasked the concept of 'tasklets' to implement this, rather than
> introducing a whole new abstraction like Linux workqueues.
Yeah, this looks better.
>
> Thanks to Dulloor for initial changes to the credit scheduler. I should have
> acknowledged you in the changeset comment too: sorry about that. :-(
No problem :)

>
> George: let me know if the scheduler changes in c/s 21197 look okay.

George might be able to comment better, but two things :
1. Should we not set (ret.time) to some timeslice (rather than -1)
when we BOOST the idle_vcpu (for csched and csched2).
2. Is it fine to use a simple list_empty in checking if the
tasklet_queue is empty for a cpu, with other cpus possibly accessing
the list too.

>
>  Thanks,
>  Keir
>
> On 16/04/2010 19:05, "Keir Fraser" <keir.fraser@eu.citrix.com> wrote:
>
>> George, Yunhong, and others,
>>
>> So, it seems that runing stop_machine_run(), and now
>> continue_hypercall_on_cpu(), in softirq context is a bit of a problem.
>> Because the softirq can stop the currently-running vcpu from being
>> descheduled we can end up with subtle deadlocks. For example, with s_m_r()
>> we try to rendezvous all cpus in softirq context -- we can have CPU A enter
>> the softirq interrupting VCPU X, meanwhile VCPU Y on CPU B is spinning
>> trying to pause VCPU X. Hence CPU B doesn't get into softirq, and so CPU A
>> never leaves it, and we have deadlock.
>>
>> There are various possible solutions to this, but one of the architecturally
>> neatest would be to run the s_m_r() and c_h_o_c() work in a
>> 'Linux-workqueue' type of environment -- i.e., in a proper non-guest vcpu
>> context. Rather than introducing the whole kthread concept into Xen, one
>> possibility would be to schedule this work on the idle vcpus -- effectively
>> promoting idle vcpus to a more general kind of 'Xen worker vcpu' whose job
>> can include running the idle loop.
>>
>> One bit of mechanism this would require is the ability to bump the idle vcpu
>> priority up - preferably to 'max' priority forcing it to run next until we
>> return it to idle/lowest priority. George: how hard would such a mechanism
>> be to implement do you think?
>>
>> More generally: what do people think of this idea?
>>
>>  Thanks,
>>  Keir
>>
>
>
>