Re: [Qemu-devel] [PATCH v3] rcu: reduce more than 7MB heap memory by malloc_trim()

[Yang Zhong <yang.zhong@intel.com>]

On Wed, Dec 06, 2017 at 10:48:45AM +0100, Paolo Bonzini wrote:
> On 06/12/2017 10:26, Yang Zhong wrote:
> >   Hello Paolo,
> > 
> >   The best option is only trim one time after guest kernel bootup or VM bootup, and as for
> >   hotplug/unhotplug operations during the VM running, the trim still can do for each batch
> >   memory free because trim will not impact VM performance during VM running status.
> > 
> >   So, the key point is qemu is hard to know when guest ernel bootup is over. If you have some 
> >   suggestions, please let me know. thanks!
> 
> It shouldn't be hard.  Does QEMU's RCU thread actually get any
> significant activity after bootup?  Hence the suggestion of keeping
> malloc_trim in the RCU thread, but only do it if some time has passed
> since the last time.
> 
> Maybe something like this every time the RCU thread runs:
> 
>  static uint64_t next_trim_time, last_trim_time;
>  if (current time < next_trim_time) {
>      next_trim_time -= last_trim_time / 2    /* or higher */
>      last_trim_time -= last_trim_time / 2    /* same as previous line */
>  } else {
>      trim_start_time = current time
>      malloc_trim(...)
>      last_trim_time = current time - trim_start_time
>      next_trim_time = current time + last_trim_time
>  }
> 
> Where the "2" factor should be tuned so that both your and Shannon's
> scenario work fine.
> 
  Hello Paolo,

  Thanks for your help!
  I changed the patch per your advice, and new TEMP patch as below:

  static void *call_rcu_thread(void *opaque)
  {
     struct rcu_head *node;
 +    int num=1;

     rcu_register_thread();

  @@ -272,6 +273,21 @@ static void *call_rcu_thread(void *opaque)
             node->func(node);
         }
         qemu_mutex_unlock_iothread();
 +
 +        static uint64_t next_trim_time, last_trim_time;
 +        int delta=100;
 +
 +        if ( qemu_clock_get_ns(QEMU_CLOCK_HOST) < next_trim_time ) {
 +            next_trim_time -= last_trim_time / delta;   /* or higher */
 +            last_trim_time -= last_trim_time / delta;   /* same as previous line */
 +        } else {
 +            uint64_t trim_start_time = qemu_clock_get_ns(QEMU_CLOCK_HOST);
 +            malloc_trim(4 * 1024 *1024);
 +            last_trim_time = qemu_clock_get_ns(QEMU_CLOCK_HOST) - trim_start_time;
 +            next_trim_time = qemu_clock_get_ns(QEMU_CLOCK_HOST) + last_trim_time;
 +            printf("---------memory trim----------num=%d------last_trim_time=%ld next_trim_time=%ld --\n", num++,last_trim_time,next_trim_time);
 +       }
 +
 
 The print log for your reference
 ---------memory trim----------num=1------last_trim_time=165000 next_trim_time=1512658205270477000 --
 ---------memory trim----------num=2------last_trim_time=656000 next_trim_time=1512658205278032000 --
 ---------memory trim----------num=3------last_trim_time=620000 next_trim_time=1512658205298888000 --
 ---------memory trim----------num=4------last_trim_time=635000 next_trim_time=1512658205339967000 --
 ---------memory trim----------num=6------last_trim_time=526000 next_trim_time=1512658207659599000 --
 ---------memory trim----------num=7------last_trim_time=744000 next_trim_time=1512658208121249000 --
 ---------memory trim----------num=8------last_trim_time=872000 next_trim_time=1512658208132805000 --
 ---------memory trim----------num=9------last_trim_time=380000 next_trim_time=1512658208376950000 --
 ---------memory trim----------num=10------last_trim_time=521000 next_trim_time=1512658210648843000 --

 Which show trim cost time less than 1ms and call_rcu_thread() do 10 times batch free, the trim also 10 times.

 I also did below changes: 
    delta=1000,  and 
    next_trim_time = qemu_clock_get_ns(QEMU_CLOCK_HOST) + delta * last_trim_time

 The whole VM bootup will trim 3 times.

 Regards,

 Yang


> Thanks,
> 
> Paolo