[Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Tomáš Golembiovský]

Hi,

We'd like to include information about reclaimable memory into the
statistics in VirtiO Balloon driver. Namely, we'd like to include
counters for bufferes and caches of Linux kernel. The patch itself is
pretty trivial -- no problem there. But before we do that I'd like to
get some input from the QEMU community.

1) Is there any reason not to have the stats there?

2) Considering the balloon device is multiplatform (Linux, BSD,
Windows), is there a problem with including buffers/caches? These seem
to be specific to the Linux virtual memory subsystem. Of course, other
OSes could just report zeros. Are there some internal stats on those
OSes that could be filled in? I don't now if such or similar statistic
are available on BSD. On Windows only SystemCache stat looks like
something relevant. Anyone familiar with those OSes has any suggestions?

    Tomas

-- 
Tomáš Golembiovský <tgolembi@redhat.com>

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Stefan Hajnoczi]

On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote:
> Hi,

I have CCed the relevant mailing lists and people most recently involved
in virtio-balloon discussions.  Hopefully this will help get the right
people to see your questions.

> We'd like to include information about reclaimable memory into the
> statistics in VirtiO Balloon driver. Namely, we'd like to include
> counters for bufferes and caches of Linux kernel. The patch itself is
> pretty trivial -- no problem there. But before we do that I'd like to
> get some input from the QEMU community.
> 
> 1) Is there any reason not to have the stats there?
> 
> 2) Considering the balloon device is multiplatform (Linux, BSD,
> Windows), is there a problem with including buffers/caches? These seem
> to be specific to the Linux virtual memory subsystem. Of course, other
> OSes could just report zeros. Are there some internal stats on those
> OSes that could be filled in? I don't now if such or similar statistic
> are available on BSD. On Windows only SystemCache stat looks like
> something relevant. Anyone familiar with those OSes has any suggestions?
> 
>     Tomas
> 
> -- 
> Tomáš Golembiovský <tgolembi@redhat.com>
> 

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Wei Wang]

On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
> On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote:
>> Hi,
> I have CCed the relevant mailing lists and people most recently involved
> in virtio-balloon discussions.  Hopefully this will help get the right
> people to see your questions.
>
>> We'd like to include information about reclaimable memory into the
>> statistics in VirtiO Balloon driver. Namely, we'd like to include
>> counters for bufferes and caches of Linux kernel. The patch itself is
>> pretty trivial -- no problem there. But before we do that I'd like to
>> get some input from the QEMU community.
>>
>> 1) Is there any reason not to have the stats there?

Could you please share the usages of reclaimable memory via the stats?


>>
>> 2) Considering the balloon device is multiplatform (Linux, BSD,
>> Windows), is there a problem with including buffers/caches? These seem
>> to be specific to the Linux virtual memory subsystem. Of course, other
>> OSes could just report zeros. Are there some internal stats on those
>> OSes that could be filled in? I don't now if such or similar statistic
>> are available on BSD. On Windows only SystemCache stat looks like
>> something relevant. Anyone familiar with those OSes has any suggestions?
>>
>>     

One of the solutions that I'm thinking about is to make virtio 
platform-ware.

That is, the device by default supports
VIRTIO_F_LINUX,
VIRTIO_F_WINDOWS,
VIRTIO_F_BSD.

For the Linux driver, only VIRTIO_F_LINUX is supported, then we can
have Linux specific driver implementations under that feature.


Best,
Wei

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Tomáš Golembiovský]

On Tue, 29 Aug 2017 20:01:53 +0800
Wei Wang <wei.w.wang@intel.com> wrote:

> On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
> > On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote:  
> >> Hi,  
> > I have CCed the relevant mailing lists and people most recently involved
> > in virtio-balloon discussions.  Hopefully this will help get the right
> > people to see your questions.
> >  
> >> We'd like to include information about reclaimable memory into the
> >> statistics in VirtiO Balloon driver. Namely, we'd like to include
> >> counters for bufferes and caches of Linux kernel. The patch itself is
> >> pretty trivial -- no problem there. But before we do that I'd like to
> >> get some input from the QEMU community.
> >>
> >> 1) Is there any reason not to have the stats there?  
> 
> Could you please share the usages of reclaimable memory via the stats?

I'll go ahead then and start sending patches. What would be the proper
course of action here? Send patch for the driver first, or send patch
for QEMU first or send both patches right away?


> >>
> >> 2) Considering the balloon device is multiplatform (Linux, BSD,
> >> Windows), is there a problem with including buffers/caches? These seem
> >> to be specific to the Linux virtual memory subsystem. Of course, other
> >> OSes could just report zeros. Are there some internal stats on those
> >> OSes that could be filled in? I don't now if such or similar statistic
> >> are available on BSD. On Windows only SystemCache stat looks like
> >> something relevant. Anyone familiar with those OSes has any suggestions?
> >>
> >>       
> 
> One of the solutions that I'm thinking about is to make virtio 
> platform-ware.

This is not necessary. IIUC the driver does not need to send all the
stats. We can simply treat those stats as specific to Linux driver and
other drivers will not send them. Then QEMU will treat them as if zero
was reported.

> 
> That is, the device by default supports
> VIRTIO_F_LINUX,
> VIRTIO_F_WINDOWS,
> VIRTIO_F_BSD.
> 
> For the Linux driver, only VIRTIO_F_LINUX is supported, then we can
> have Linux specific driver implementations under that feature.


Since there were no suggestions for similar stats on other OSes I'd say
we treat the stats for buffers and caches as Linux specific. If there is
any need to send similar stats for other OSes we will add new stat fields
(specific for that particular OS).


Thanks,

    Tomas

-- 
Tomáš Golembiovský <tgolembi@redhat.com>

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Tomáš Golembiovský]

Resending. It did not reach virtio-dev the first time.

On Tue, 12 Sep 2017 16:29:47 +0200
Tomáš Golembiovský <tgolembi@redhat.com> wrote:

> On Tue, 29 Aug 2017 20:01:53 +0800
> Wei Wang <wei.w.wang@intel.com> wrote:
> 
> > On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
> > > On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote:  
> > >> Hi,  
> > > I have CCed the relevant mailing lists and people most recently involved
> > > in virtio-balloon discussions.  Hopefully this will help get the right
> > > people to see your questions.
> > >  
> > >> We'd like to include information about reclaimable memory into the
> > >> statistics in VirtiO Balloon driver. Namely, we'd like to include
> > >> counters for bufferes and caches of Linux kernel. The patch itself is
> > >> pretty trivial -- no problem there. But before we do that I'd like to
> > >> get some input from the QEMU community.
> > >>
> > >> 1) Is there any reason not to have the stats there?  
> > 
> > Could you please share the usages of reclaimable memory via the stats?
> 
> I'll go ahead then and start sending patches. What would be the proper
> course of action here? Send patch for the driver first, or send patch
> for QEMU first or send both patches right away?
> 
> 
> > >>
> > >> 2) Considering the balloon device is multiplatform (Linux, BSD,
> > >> Windows), is there a problem with including buffers/caches? These seem
> > >> to be specific to the Linux virtual memory subsystem. Of course, other
> > >> OSes could just report zeros. Are there some internal stats on those
> > >> OSes that could be filled in? I don't now if such or similar statistic
> > >> are available on BSD. On Windows only SystemCache stat looks like
> > >> something relevant. Anyone familiar with those OSes has any suggestions?
> > >>
> > >>       
> > 
> > One of the solutions that I'm thinking about is to make virtio 
> > platform-ware.
> 
> This is not necessary. IIUC the driver does not need to send all the
> stats. We can simply treat those stats as specific to Linux driver and
> other drivers will not send them. Then QEMU will treat them as if zero
> was reported.
> 
> > 
> > That is, the device by default supports
> > VIRTIO_F_LINUX,
> > VIRTIO_F_WINDOWS,
> > VIRTIO_F_BSD.
> > 
> > For the Linux driver, only VIRTIO_F_LINUX is supported, then we can
> > have Linux specific driver implementations under that feature.
> 
> 
> Since there were no suggestions for similar stats on other OSes I'd say
> we treat the stats for buffers and caches as Linux specific. If there is
> any need to send similar stats for other OSes we will add new stat fields
> (specific for that particular OS).
> 
> 
> Thanks,
> 
>     Tomas
> 
> -- 
> Tomáš Golembiovský <tgolembi@redhat.com>


-- 
Tomáš Golembiovský <tgolembi@redhat.com>

Re: [Qemu-devel] [RFC] Buffers/caches in VirtIO Balloon driver stats

[Wei Wang]

On 09/12/2017 10:29 PM, Tomáš Golembiovský wrote:
> On Tue, 29 Aug 2017 20:01:53 +0800
> Wei Wang <wei.w.wang@intel.com> wrote:
>
>> On 08/29/2017 05:57 PM, Stefan Hajnoczi wrote:
>>> On Sun, Aug 27, 2017 at 11:30:33PM +0200, Tomáš Golembiovský wrote:
>>>> Hi,
>>> I have CCed the relevant mailing lists and people most recently involved
>>> in virtio-balloon discussions.  Hopefully this will help get the right
>>> people to see your questions.
>>>   
>>>> We'd like to include information about reclaimable memory into the
>>>> statistics in VirtiO Balloon driver. Namely, we'd like to include
>>>> counters for bufferes and caches of Linux kernel. The patch itself is
>>>> pretty trivial -- no problem there. But before we do that I'd like to
>>>> get some input from the QEMU community.
>>>>
>>>> 1) Is there any reason not to have the stats there?
>> Could you please share the usages of reclaimable memory via the stats?
> I'll go ahead then and start sending patches. What would be the proper
> course of action here? Send patch for the driver first, or send patch
> for QEMU first or send both patches right away?

If you have both ready, I think it would be fine to send them all.


>
>
>>>> 2) Considering the balloon device is multiplatform (Linux, BSD,
>>>> Windows), is there a problem with including buffers/caches? These seem
>>>> to be specific to the Linux virtual memory subsystem. Of course, other
>>>> OSes could just report zeros. Are there some internal stats on those
>>>> OSes that could be filled in? I don't now if such or similar statistic
>>>> are available on BSD. On Windows only SystemCache stat looks like
>>>> something relevant. Anyone familiar with those OSes has any suggestions?
>>>>
>>>>        
>> One of the solutions that I'm thinking about is to make virtio
>> platform-ware.
> This is not necessary. IIUC the driver does not need to send all the
> stats. We can simply treat those stats as specific to Linux driver and
> other drivers will not send them. Then QEMU will treat them as if zero
> was reported.

Just had a quick check, I think that should function well too.
QEMU will report -1 for the stats.

Best,
Wei