Re: [NEW]: Introducing shrink_all_memory from user space

[Ying Han <yinghan@google.com>]

On Sun, Apr 15, 2012 at 2:47 AM, PINTU KUMAR <pintu_agarwal@yahoo.com> wrote:
> Dear All,
>
> This is regarding a small proposal for shrink_all_memory( ) function which is found in mm/vmscan.c.
> For those who are not aware, this function helps in reclaiming specified amount of physical memory and returns number of freed pages.
>
> Currently this function is under CONFIG_HIBERNATION flag, so cannot be used by others without enabling hibernation.
> Moreover this function is not exported to the outside world, so no driver can use it directly without including EXPORT_SYMBOL(shrink_all_memory) and recompiling the kernel.
> The purpose of using it under hibernation(kernel/power/snapshot.c) is to regain enough physical pages to create hibernation image.
>
> The same can be useful for some drivers who wants to allocate large contiguous memory (in MBs) but his/her system is in very bad state and could not do so without rebooting.
> Due to this bad state of the system the user space application will be badly hurt on performance, and there could be a need to quickly reclaim all physical memory from user space.
> This could be very helpful for small embedded products and smart phones where rebooting is never a preferred choice.
>
> With this support in kernel, a small utility can be developed in user space which user can run and reclaim as many physical pages and noticed that his system performance is increased without rebooting.
>
> To make this work, I have performed a sample experiment on my ubuntu(10.x) machine running with kernel-3.3.0.
>
> Also I performed the similar experiment of one of our Samsung smart phones as well.
>
> Following are the results from Ubuntu:
>
> 1) I downloaded kernel3.3.0 and made the respective changes in mm/vmscan.c. That is added EXPORT_SYMBOL(shrink_all_memory) under the function shrink_all_memory( ).
>     (Note: CONFIG_HIBERNATION was already enabled for my system.)
>
> 2) Then I build the kernel and installed it on my Ubuntu PC.
>
> 3) After that I have developed a small kernel module (miscdevice: /dev/shrinkmem) to call shrink_all_memory( ) under my driver write( ) function.
>
> 4) Then from user space I just need to do this:
>
>     # echo 100 > /dev/shrinkmem   (To reclaim 100MB of physical memory without reboot)
>
>
> The results that were obtained with this experiment is as follows:
>
> 1) At some point of time, the buddyinfo and free pages on my Ubuntu were as follows:
> root@pintu-ubuntu:~/PintuHomeTest/KERNEL_ORG# cat /proc/buddyinfo ; free -tmNode 0, zone      DMA    468     23      0      0      0      0      0      0      0      0      0
> Node 0, zone   Normal    898    161     38      8      0      0      0      0      0      0      0
>                  total       used       free     shared    buffers     cached
> Mem:           497        489          7          0         37        405
> -/+ buffers/cache:         47        449
> Swap:         1458        158       1300
> Total:        1956        648       1308
>
>
> 2) After doing "echo 100 > /dev/shrinkmem" :
> [19653.833916] [SHRINKMEM]: memsize(in MB) = 100
> [19653.863618] <SHRINKMEM>: Number of Pages Freed: 26756
> [19653.863664] [SHRINKMEM] : Device is Closed....
>
> Node 0, zone      DMA    411    166     51      5      0      0      0      0      0      0      0
> Node 0, zone   Normal   2915   3792   2475   1335    730     23      0      0      0      0      0
>                  total       used       free     shared    buffers     cached
> Mem:           497        323        173          0         37        238
> -/+ buffers/cache:         47        449
> Swap:         1458        158       1300
> Total:        1956        481       1474
>
>
> 3) After running again with : echo 512 > /dev/shrinkmem
> [21961.064534] [SHRINKMEM]: memsize(in MB) = 512
> [21961.109497] <SHRINKMEM>: Number of Pages Freed: 61078
> [21961.109562] [SHRINKMEM] : Device is Closed....
> Node 0, zone      DMA    116     99     87     58     16      6      1      0      0      0      0
> Node 0, zone   Normal   6697   6939   5529   3756   1442    203     17      0      0      0      0
>                  total       used       free     shared    buffers     cached
> Mem:           497         87        410          0         37          9
> -/+ buffers/cache:         40        456
> Swap:         1458        158       1300
> Total:        1956        245       1711
>
>
> 4) Thus in both the cases huge number of free pages were reclaimed.
>
> 5) After running this on my system, the performance was improved quickly.
>
> 6) I performed the same experiment on our Samsung Smart phones as well. And I have seen a drastic improve in performance after running this for 3/4 times.
>     In case of phones it is more helpful as there is no swap space.
>
> 7) Your feedback and suggestion is important. Based on the feedback, I can plan to submit the patches officially after performing basic cleanups.
>
>
> Future Work
> ==========
> Our final goal is to use it during lowmem notifier where shrink_all_memory will be called automatically in background if the memory pressure falls below certain limit defined by the system.
> For example we can measure the percentage memory fragmentation level of the system across each zone and if the fragmentation percentage goes above say 80-85% during lowmem notifier, we can invoke shrink_all_memory() in background.

Does it make sense to let kswapd reclaim pages at background w/ user
configured watermark?

--Ying

>
> This can be used by some application which requires large mmap or shared memory mapping.
>
> This can be even using inside the multimedia drivers that requires large contiguous memory to check if that many memory pages can be reclaimed or not.
>
>
> Please provide your valuable feedback and suggestion.
>
>
> Thank you very much !
>
>
> With Regards,
> Pintu Kumar
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a hrefmailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>