Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

[Vincent JARDIN]

Tim,

cc-ing Paolo and qemu-devel@ again in order to get their take on it.

>>> Did you make any progress in Qemu/KVM community?
>>> We need to be sync'ed up with them to be sure we share the same goal.
>>> I want also to avoid using a solution which doesn't fit with their plan.
>>> Remember that we already had this problem with ivshmem which was
>>> planned to be dropped.
>>>

>> Unfortunately, I have not yet received any feedback:
>> http://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg01103.html
>
> Just to add to what Alan said above, this capability does not exist in qemu at the moment, and based on there having been no feedback on the qemu mailing list so far, I think it's reasonable to assume that it will not be implemented in the immediate future. The VM Power Management feature has also been designed to allow easy migration to a qemu-based solution when this is supported in future. Therefore, I'd be in favour of accepting this feature into DPDK now.
>
> It's true that the implementation is a work-around, but there have been similar cases in DPDK in the past. One recent example that comes to mind is userspace vhost. The original implementation could also be considered a work-around, but it met the needs of many in the community. Now, with support for vhost-user in qemu 2.1, that implementation is being improved. I'd see VM Power Management following a similar path when this capability is supported in qemu.

Best regards,
   Vincent

Re: [Qemu-devel] [PATCH v4 00/10] VM Power Management

[Paolo Bonzini]

I had replied to this message, but my reply never got to the list.
Let's try again.

I wonder if this might be papering over a bug in the host cpufreq
driver.  If the guest is not doing much and leaving a lot of idle CPU
time, the host should scale down the frequency of that CPU.  In the case
of pinned VCPUs this should really "just work".  What is the problem
that is being solved?

Paolo

On 22/11/2014 18:17, Vincent JARDIN wrote:
> Tim,
> 
> cc-ing Paolo and qemu-devel@ again in order to get their take on it.
> 
>>>> Did you make any progress in Qemu/KVM community?
>>>> We need to be sync'ed up with them to be sure we share the same goal.
>>>> I want also to avoid using a solution which doesn't fit with their
>>>> plan.
>>>> Remember that we already had this problem with ivshmem which was
>>>> planned to be dropped.
>>>>
> 
>>> Unfortunately, I have not yet received any feedback:
>>> http://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg01103.html
>>
>> Just to add to what Alan said above, this capability does not exist in
>> qemu at the moment, and based on there having been no feedback on the
>> qemu mailing list so far, I think it's reasonable to assume that it
>> will not be implemented in the immediate future. The VM Power
>> Management feature has also been designed to allow easy migration to a
>> qemu-based solution when this is supported in future. Therefore, I'd
>> be in favour of accepting this feature into DPDK now.
>>
>> It's true that the implementation is a work-around, but there have
>> been similar cases in DPDK in the past. One recent example that comes
>> to mind is userspace vhost. The original implementation could also be
>> considered a work-around, but it met the needs of many in the
>> community. Now, with support for vhost-user in qemu 2.1, that
>> implementation is being improved. I'd see VM Power Management
>> following a similar path when this capability is supported in qemu.
> 
> Best regards,
>   Vincent
> 

Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

[Thomas Monjalon]

2014-12-09 18:35, Paolo Bonzini:
> >>>> Did you make any progress in Qemu/KVM community?
> >>>> We need to be sync'ed up with them to be sure we share the same goal.
> >>>> I want also to avoid using a solution which doesn't fit with their
> >>>> plan.
> >>>> Remember that we already had this problem with ivshmem which was
> >>>> planned to be dropped.
> >>> 
> >>> Unfortunately, I have not yet received any feedback:
> >>> http://lists.nongnu.org/archive/html/qemu-devel/2014-11/msg01103.html
> >>
> >> Just to add to what Alan said above, this capability does not exist in
> >> qemu at the moment, and based on there having been no feedback on th
> >> qemu mailing list so far, I think it's reasonable to assume that it
> >> will not be implemented in the immediate future. The VM Power
> >> Management feature has also been designed to allow easy migration to a
> >> qemu-based solution when this is supported in future. Therefore, I'd
> >> be in favour of accepting this feature into DPDK now.
> >>
> >> It's true that the implementation is a work-around, but there have
> >> been similar cases in DPDK in the past. One recent example that comes
> >> to mind is userspace vhost. The original implementation could also be
> >> considered a work-around, but it met the needs of many in the
> >> community. Now, with support for vhost-user in qemu 2.1, that
> >> implementation is being improved. I'd see VM Power Management
> >> following a similar path when this capability is supported in qemu.
> 
> I wonder if this might be papering over a bug in the host cpufreq
> driver.  If the guest is not doing much and leaving a lot of idle CPU
> time, the host should scale down the frequency of that CPU.  In the case
> of pinned VCPUs this should really "just work".  What is the problem
> that is being solved?
> 
> Paolo

Alan, Pablo, please could you explain your logic with VM power management?

-- 
Thomas

Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

[Carew, Alan]

Hi Paolo,

> 2014-12-09 18:35, Paolo Bonzini:
> > >>>> Did you make any progress in Qemu/KVM community?
> > >>>> We need to be sync'ed up with them to be sure we share the same
> goal.
> > >>>> I want also to avoid using a solution which doesn't fit with
> > >>>> their plan.
> > >>>> Remember that we already had this problem with ivshmem which
> was
> > >>>> planned to be dropped.
> > >>>
> > >>> Unfortunately, I have not yet received any feedback:
> > >>> http://lists.nongnu.org/archive/html/qemu-devel/2014-
> 11/msg01103.h
> > >>> tml
> > >>
> > >> Just to add to what Alan said above, this capability does not exist
> > >> in qemu at the moment, and based on there having been no feedback
> > >> on th qemu mailing list so far, I think it's reasonable to assume
> > >> that it will not be implemented in the immediate future. The VM
> > >> Power Management feature has also been designed to allow easy
> > >> migration to a qemu-based solution when this is supported in
> > >> future. Therefore, I'd be in favour of accepting this feature into DPDK
> now.
> > >>
> > >> It's true that the implementation is a work-around, but there have
> > >> been similar cases in DPDK in the past. One recent example that
> > >> comes to mind is userspace vhost. The original implementation could
> > >> also be considered a work-around, but it met the needs of many in
> > >> the community. Now, with support for vhost-user in qemu 2.1, that
> > >> implementation is being improved. I'd see VM Power Management
> > >> following a similar path when this capability is supported in qemu.
> >
> > I wonder if this might be papering over a bug in the host cpufreq
> > driver.  If the guest is not doing much and leaving a lot of idle CPU
> > time, the host should scale down the frequency of that CPU.  In the
> > case of pinned VCPUs this should really "just work".  What is the
> > problem that is being solved?
> >
> > Paolo
> 
> Alan, Pablo, please could you explain your logic with VM power
> management?
> 
> --
> Thomas

The problem is deterministic control of host CPU frequency and the DPDK usage
model.
A hands-off power governor will scale based on workload, whether this is a host
application or VM, so no problems or bug there.

Where this solution fits is where an application wants to control its own
power policy, for example l3fwd_power uses librte_power library to change
frequency via apci_cpufreq based on application heuristics rather than
relying on an inbuilt policy for example ondemand or performance.

This ability has existed in DPDK for host usage for some time and VM power
management allows this use case to be extended to cater for virtual machines
by re-using the librte_power interface to encapsulate the VM->Host
comms and provide an example means of managing such communications.

 I hope this clears it up a bit.

Thanks,
Alan.

Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

[Paolo Bonzini]

On 12/12/2014 14:00, Carew, Alan wrote:
> The problem is deterministic control of host CPU frequency and the DPDK usage
> model.
> A hands-off power governor will scale based on workload, whether this is a host
> application or VM, so no problems or bug there.
> 
> Where this solution fits is where an application wants to control its own
> power policy, for example l3fwd_power uses librte_power library to change
> frequency via apci_cpufreq based on application heuristics rather than
> relying on an inbuilt policy for example ondemand or performance.
> 
> This ability has existed in DPDK for host usage for some time and VM power
> management allows this use case to be extended to cater for virtual machines
> by re-using the librte_power interface to encapsulate the VM->Host
> comms and provide an example means of managing such communications.
> 
>  I hope this clears it up a bit.

Ok, this looks specific enough that an out-of-band solution within DPDK
sounds like the best approach.  It seems unnecessary to involve the
hypervisor (neither KVM nor QEMU).

Paolo

Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

[Thomas Monjalon]

2014-12-12 15:50, Paolo Bonzini:
> On 12/12/2014 14:00, Carew, Alan wrote:
> > The problem is deterministic control of host CPU frequency and the DPDK usage
> > model.
> > A hands-off power governor will scale based on workload, whether this is a host
> > application or VM, so no problems or bug there.
> > 
> > Where this solution fits is where an application wants to control its own
> > power policy, for example l3fwd_power uses librte_power library to change
> > frequency via apci_cpufreq based on application heuristics rather than
> > relying on an inbuilt policy for example ondemand or performance.
> > 
> > This ability has existed in DPDK for host usage for some time and VM power
> > management allows this use case to be extended to cater for virtual machines
> > by re-using the librte_power interface to encapsulate the VM->Host
> > comms and provide an example means of managing such communications.
> > 
> >  I hope this clears it up a bit.
> 
> Ok, this looks specific enough that an out-of-band solution within DPDK
> sounds like the best approach.  It seems unnecessary to involve the
> hypervisor (neither KVM nor QEMU).

Paolo, I don't understand why you don't imagine controlling frequency scaling
of a pinned vCPU transparently?
In my understanding, we currently cannot control frequency scaling without
knowing wether we are in a VM or not.

-- 
Thomas

Re: [Qemu-devel] [dpdk-dev] [PATCH v4 00/10] VM Power Management

[Paolo Bonzini]

On 12/12/2014 17:10, Thomas Monjalon wrote:
> > Ok, this looks specific enough that an out-of-band solution within DPDK
> > sounds like the best approach.  It seems unnecessary to involve the
> > hypervisor (neither KVM nor QEMU).
>
> Paolo, I don't understand why you don't imagine controlling frequency scaling
> of a pinned vCPU transparently?

Probably because I don't imagine controlling frequency scaling from the
application on bare metal, either. :)  It seems to me that this is just
working around limitations of the kernel.

Paolo

> In my understanding, we currently cannot control frequency scaling without
> knowing wether we are in a VM or not.