* Re: [PATCH v8 1/5] mm: introduce a common interface for balloon pages mobility
From: Michael S. Tsirkin @ 2012-08-24 8:03 UTC (permalink / raw)
To: Rik van Riel
Cc: Rafael Aquini, Konrad Rzeszutek Wilk, Peter Zijlstra,
linux-kernel, virtualization, linux-mm, Andi Kleen, Minchan Kim,
Andrew Morton, Paul E. McKenney
In-Reply-To: <5036F111.4040607@redhat.com>
On Thu, Aug 23, 2012 at 11:12:17PM -0400, Rik van Riel wrote:
> On 08/23/2012 07:36 PM, Michael S. Tsirkin wrote:
>
> >--->
> >
> >virtio-balloon: replace page->lru list with page->private.
> >
> >The point is to free up page->lru for use by compaction.
> >Warning: completely untested, will provide tested version
> >if we agree on this direction.
>
> A singly linked list is not going to work for page migration,
> which needs to get pages that might be in the middle of the
> balloon list.
For virtballoon_migratepage? Hmm I think you are right. I'll
need to think it over but if we can think of no other way
to avoid ther need to handle isolation in virtio,
we'll just have to use the original plan and add
balloon core to mm.
> --
> All rights reversed
^ permalink raw reply
* Re: [PATCH 1/2 v1] blkdrv: Add queue limits parameters for sg block drive
From: Stefan Hajnoczi @ 2012-08-24 9:05 UTC (permalink / raw)
To: Hannes Reinecke
Cc: zwanp, linuxram, qemu-devel, virtualization, Cong Meng,
Paolo Bonzini, Christoph Hellwig
In-Reply-To: <50375AD6.8060203@suse.de>
On Fri, Aug 24, 2012 at 12:43:34PM +0200, Hannes Reinecke wrote:
> On 08/24/2012 09:56 AM, Paolo Bonzini wrote:
> >Il 24/08/2012 02:45, Nicholas A. Bellinger ha scritto:
> >>So up until very recently, TCM would accept an I/O request for an DATA
> >>I/O type CDB with a max_sectors larger than the reported max_sectors for
> >>it's TCM backend (regardless of backend type), and silently generate N
> >>backend 'tasks' to complete the single initiator generated command.
> >
> >This is what QEMU does if you use scsi-block, except for MMC devices
> >(because of the insanity of the commands used for burning).
> >
> >>Also FYI for Paolo, for control type CDBs I've never actually seen an
> >>allocation length exceed max_sectors, so in practice AFAIK this only
> >>happens for DATA I/O type CDBs.
> >
> >Yes, that was my impression as well.
> >
> >>This was historically required by the pSCSI backend driver (using a
> >>number of old SCSI passthrough interfaces) in order to support this very
> >>type of case described above, but over the years the logic ended up
> >>creeping into various other non-passthrough backend drivers like IBLOCK
> >>+FILEIO. So for v3.6-rc1 code, hch ended up removing the 'task' logic
> >>thus allowing backends (and the layers below) to the I/O sectors >
> >>max_sectors handling work, allowing modern pSCSI using struct request to
> >>do the same. (hch assured me this works now for pSCSI)
> >
> >So now LIO and QEMU work the same. (Did he test tapes too?)
> >
> >>Anyways, I think having the guest limit virtio-scsi DATA I/O to
> >>max_sectors based upon the host accessible block limits is reasonable
> >>approach to consider. Reducing this value even further based upon the
> >>lowest max_sectors available amongst possible migration hosts would be a
> >>good idea here to avoid having to reject any I/O's exceeding a new
> >>host's device block queue limits.
> >
> >Yeah, it's reasonable _assuming it is needed at all_. For disks, it is
> >not needed. For CD-ROMs it is, but right now we have only one report
> >and it is using USB so we don't know if the problem is in the drive or
> >rather in the USB bridge (whose quality usually leaves much to be desired).
> >
> >So in the only observed case, the fix would really be a workaround; the
> >right thing to do with USB devices is to use USB passthrough.
> >
>
> Hehe. So finally someone else stumbled across this one.
>
> All is fine and dandy as long as you're able to use scsi-disk.
> As soon as you're forced to use scsi-generic we're in trouble.
>
> With scsi-generic we actually have two problems:
> 1) scsi-generic just acts as a pass-through and passes the commands
> as-is, including the scatter-gather information as formatted by
> the guest. So the guest could easily format an SG_IO comand
> which will not be compatible with the host.
> 2) The host is not able to differentiate between a malformed
> SG_IO command and a real I/O error; in both cases it'll return
> -EIO.
>
> So we can fix this by either
> a) ignore (as we do nowadays :-)
> b) Fixup scsi-generic to inspect and modify SG_IO information
> to ensure the host-limits are respected
> c) Fixup the host to differentiate between a malformed SG_IO
> and a real I/O error.
>
> c) would only be feasible for Linux et al. _personally_ I would
> prefer that approach, as I fail to see why we cannot return a proper
> error code here.
> But I already can hear the outraged cry 'POSIX! POSIX!', so I guess
> it's not going to happen anytime soon.
> So I would vote for b).
> Yes, it's painful. But in the long run we'll have to do an SG_IO
> inspection anyway, otherwise we'll always be susceptible to
> malicious SG_IO attacks.
Are you suggesting we do not expose host block queue limits to the
guest. Instead we should inspect and reformat SG_IO requests in QEMU?
Reformatting seems hard because there are many possible SCSI
commands/sub-commands and we would have to whitelist them on a
case-by-case basis.
That sounds like more work than exposing the block queue limits using
Cong Meng's patches.
On a side-note, are you thinking of blacklisting/whitelisting certain
commands that don't make sense or would have an unintended effect if
sent from a guest (e.g. reservations)? That would be an interesting
topic for another email thread, I'd love to learn what weird things we
need to protect against.
Stefan
^ permalink raw reply
* Re: [PATCH 1/2 v1] blkdrv: Add queue limits parameters for sg block drive
From: Paolo Bonzini @ 2012-08-24 9:14 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Stefan Hajnoczi, zwanp, linuxram, qemu-devel, virtualization,
Cong Meng, Christoph Hellwig
In-Reply-To: <50375AD6.8060203@suse.de>
Il 24/08/2012 12:43, Hannes Reinecke ha scritto:
> Hehe. So finally someone else stumbled across this one.
>
> All is fine and dandy as long as you're able to use scsi-disk.
> As soon as you're forced to use scsi-generic we're in trouble.
>
> With scsi-generic we actually have two problems:
> 1) scsi-generic just acts as a pass-through and passes the commands
> as-is, including the scatter-gather information as formatted by
> the guest. So the guest could easily format an SG_IO comand
> which will not be compatible with the host.
> 2) The host is not able to differentiate between a malformed
> SG_IO command and a real I/O error; in both cases it'll return
> -EIO.
>
> So we can fix this by either
> a) ignore (as we do nowadays :-)
> b) Fixup scsi-generic to inspect and modify SG_IO information
> to ensure the host-limits are respected
That's what scsi-block already does.
Perhaps sooner or later we will need a scsi-tape? That would be fine.
> Yes, it's painful. But in the long run we'll have to do an SG_IO
> inspection anyway, otherwise we'll always be susceptible to malicious
> SG_IO attacks.
I would like to do this in the kernel using BPF. I posted a possible
spec at
http://www.redhat.com/archives/libvir-list/2012-June/msg00505.html but
the response was, ehm, underwhelming.
Paolo
^ permalink raw reply
* Re: [PATCH 1/2 v1] blkdrv: Add queue limits parameters for sg block drive
From: Hannes Reinecke @ 2012-08-24 10:43 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Stefan Hajnoczi, zwanp, linuxram, qemu-devel, virtualization,
Cong Meng, Christoph Hellwig
In-Reply-To: <503733A2.1050300@redhat.com>
On 08/24/2012 09:56 AM, Paolo Bonzini wrote:
> Il 24/08/2012 02:45, Nicholas A. Bellinger ha scritto:
>> So up until very recently, TCM would accept an I/O request for an DATA
>> I/O type CDB with a max_sectors larger than the reported max_sectors for
>> it's TCM backend (regardless of backend type), and silently generate N
>> backend 'tasks' to complete the single initiator generated command.
>
> This is what QEMU does if you use scsi-block, except for MMC devices
> (because of the insanity of the commands used for burning).
>
>> Also FYI for Paolo, for control type CDBs I've never actually seen an
>> allocation length exceed max_sectors, so in practice AFAIK this only
>> happens for DATA I/O type CDBs.
>
> Yes, that was my impression as well.
>
>> This was historically required by the pSCSI backend driver (using a
>> number of old SCSI passthrough interfaces) in order to support this very
>> type of case described above, but over the years the logic ended up
>> creeping into various other non-passthrough backend drivers like IBLOCK
>> +FILEIO. So for v3.6-rc1 code, hch ended up removing the 'task' logic
>> thus allowing backends (and the layers below) to the I/O sectors >
>> max_sectors handling work, allowing modern pSCSI using struct request to
>> do the same. (hch assured me this works now for pSCSI)
>
> So now LIO and QEMU work the same. (Did he test tapes too?)
>
>> Anyways, I think having the guest limit virtio-scsi DATA I/O to
>> max_sectors based upon the host accessible block limits is reasonable
>> approach to consider. Reducing this value even further based upon the
>> lowest max_sectors available amongst possible migration hosts would be a
>> good idea here to avoid having to reject any I/O's exceeding a new
>> host's device block queue limits.
>
> Yeah, it's reasonable _assuming it is needed at all_. For disks, it is
> not needed. For CD-ROMs it is, but right now we have only one report
> and it is using USB so we don't know if the problem is in the drive or
> rather in the USB bridge (whose quality usually leaves much to be desired).
>
> So in the only observed case, the fix would really be a workaround; the
> right thing to do with USB devices is to use USB passthrough.
>
Hehe. So finally someone else stumbled across this one.
All is fine and dandy as long as you're able to use scsi-disk.
As soon as you're forced to use scsi-generic we're in trouble.
With scsi-generic we actually have two problems:
1) scsi-generic just acts as a pass-through and passes the commands
as-is, including the scatter-gather information as formatted by
the guest. So the guest could easily format an SG_IO comand
which will not be compatible with the host.
2) The host is not able to differentiate between a malformed
SG_IO command and a real I/O error; in both cases it'll return
-EIO.
So we can fix this by either
a) ignore (as we do nowadays :-)
b) Fixup scsi-generic to inspect and modify SG_IO information
to ensure the host-limits are respected
c) Fixup the host to differentiate between a malformed SG_IO
and a real I/O error.
c) would only be feasible for Linux et al. _personally_ I would prefer
that approach, as I fail to see why we cannot return a proper error code
here.
But I already can hear the outraged cry 'POSIX! POSIX!', so I guess it's
not going to happen anytime soon.
So I would vote for b).
Yes, it's painful. But in the long run we'll have to do an SG_IO
inspection anyway, otherwise we'll always be susceptible to malicious
SG_IO attacks.
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)
^ permalink raw reply
* Call for Participation: The 9th ACM International Conference on Autonomic Computing (ICAC 2012) - September 17-21, 2012 in San Jose, California, USA
From: Ioan Raicu @ 2012-08-25 3:09 UTC (permalink / raw)
To: iraicu@cs.iit.edu
**********************************************************************
CALL FOR PARTICIPATION
======================
The 9th ACM International Conference on Autonomic Computing (ICAC 2012)
San Jose, California, USA
September 17-21, 2012
http://icac2012.cs.fiu.edu
Sponsored by ACM
**********************************************************************
Online registration is open at
http://icac2012.cs.fiu.edu/registration.shtm
Reduced fees are available for those registering by September 2, 2012.
This year's technical program features:
- 3 distinguished keynote speakers:
* Dr. Amin Vahdat, Google/UCSD
* Dr. Subutai Ahmad (VP Engineering), Numenta
* Dr. Eitan Frachtenberg, Facebook
- 24 outstanding technical papers (15 full + 9 short):
* covering core and emerging topics such as clouds, virtualization,
control, monitoring and diagnosis, and energy
* Half of the papers involve authors from industry or government labs
- 4 co-located workshops covering hot topics in:
* Feedback Computing
* Self-Aware Internet of Things
* Management of Big Data Systems
* Federated Clouds
The Conference will be held at the Fairmont Hotel in downtown San Jose,
CA, USA.
**********************************************************************
IMPORTANT DATES
===============
Early registration deadline: September 2, 2012
Hotel special rate deadline: September 7, 2012
**********************************************************************
CORPORATE SPONSORS
==================
Gold Level Partner: IBM
Conference partners: VMware, HP, Neustar
PhD Student Sponsor: Google
Other Sponsor: NEC Labs
**********************************************************************
PRELIMINARY PROGRAM
===================
======================================================================
MONDAY, SEPTEMBER 17, 2012 - WORKSHOPS
Feedback Computing 2012
Self-Aware Internet of Things 2012
======================================================================
TUESDAY, SEPTEMBER 18, 2012 - MAIN CONFERENCE
8:00AM – 8:45AM
Registration
8:45AM – 9:00AM
Welcome Remarks
9:00AM – 10:00AM
Keynote Talk I
Symbiosis in Scale Out Networking and Data Management
Amin Vahdat, Google and University of California, San Diego
10:00AM – 10.30AM
Break
10:30AM – 12:00PM
Session : Virtualization
Net-Cohort: Detecting and Managing VM Ensembles in Virtualized Data Centers
Liting Hu, Karsten Schwan (Georgia Institute of Technology);
Ajay Gulati (VMware); Junjie Zhang, Chengwei Wang (Georgia Institute
of Technology)
Application-aware Cross-layer Virtual Machine Resource Management
Lixi Wang, Jing Xu, Ming Zhao (Florida International University)
Shifting GEARS to Enable Guest-context Virtual Services
Kyle Hale, Lei Xia, Peter Dinda (Northwestern University)
12:00PM-1:30PM
Lunch
----------------------------------------------------------------------
1:30PM - 3:30PM
Session: Performance and Resource Management
When Average is Not Average: Large Response Time Fluctuations in N-tier
Systems
Qingyang Wang (Georgia Institute of Technology); Yasuhiko Kanemasa,
Motoyuki Kawaba (Fujitsu Laboratories Ltd.); Calton Pu (Georgia Institute
of Technology)
Provisioning Multi-tier Cloud Applications Using Statistical Bounds on
Sojourn Time
Upendra Sharma, Prashant Shenoy, Don Towsley (University of
Massachusetts Amherst)
Automated Profiling and Resource Management of Pig Programs for Meeting
Service Level Objectives
Zhuoyao Zhang (University of Pennsylvania); Ludmila Cherkasova
(Hewlett-Packard Labs);
Abhishek Verma (University of Illinois at Urbana-Champaign);
Boon Thau Loo (University of Pennsylvania)
AROMA: Automated Resource Allocation and Configuration of MapReduce
Environment
in the Cloud
Palden Lama, Xiaobo Zhou (University of Colorado at Colorado Springs)
3:30PM-4:00PM
Break
4:00PM – 5:15PM
Short Papers I
Locomotion@Location: When the Rubber hits the Road
Gerold Hoelzl, Marc Kurz, Alois Ferscha (Johannes Kepler University
Linz, Austria)
An Autonomic Resource Provisioning Framework for Mobile Computing Grids
Hariharasudhan Viswanathan, Eun Kyung Lee, Ivan Rodero, Dario Pompili
(Rutgers University)
A Self-Tuning Self-Optimizing Approach for Automated Network Anomaly
Detection Systems
Dennis Ippoliti, Xiaobo Zhou (University of Colorado at Colorado Springs)
Offline and On-Demand Event Correlation for Operations Management of
Large Scale IT Systems
Chetan Gupta (Hewlett-Packard Labs)
PowerTracer: Tracing Requests in Multi-tier Services to Diagnose Energy
Inefficiency
Gang Lu, Jianfeng Zhan (Institute of Computing Technology, Chinese
Academy of Sciences);
Haining Wang (College of William and Mary); Lin Yuan (Institute of
Computing Technology,
Chinese Academy of Sciences); Chuliang Weng (Shanghai Jiao Tong
University, China)
6:00PM - 9:00PM
Conference Dinner
======================================================================
WEDNESDAY, SEPTEMBER 19 - MAIN CONFERENCE
8:00AM – 9:00AM
Registration
9:00AM – 10:00AM
Keynote Talk II
Automated Machine Learning For Autonomic Computing
Subutai Ahmad, VP Engineering, Numenta
10:00AM - 10:30AM
Break
10:30AM – 12:00PM
Session: Control-Based Approaches
Budget-based Control for Interactive Services with Adaptive Execution
Yuxiong He, Zihao Ye, Qiang Fu, Sameh Elnikety (Microsoft Research)
On the Design of Decentralized Control Architectures for Workload
Consolidation in
Large-Scale Server Clusters
Rui Wang, Nagarajan Kandasamy (Drexel University)
Transactional Auto Scaler: Elastic Scaling of In-Memory Transactional
Data Grids
Diego Didona, Paolo Romano (Instituto Superior Técnico/INESC-ID);
Sebastiano Peluso,
Francesco Quaglia (Sapienza, Università di Roma)
12:00PM - 1:30PM
Lunch
----------------------------------------------------------------------
1:30PM - 2:30PM
Session: Energy
Adaptive Green Hosting
Nan Deng, Christopher Stewart, Jaimie Kelley (The Ohio State
University); Daniel Gmach,
Martin Arlitt (Hewlett Packard Labs)
Dynamic Energy-Aware Capacity Provisioning for Cloud Computing Environments
Qi Zhang, Mohamed Faten Zhani (University of Waterloo); Shuo Zhang
(National University
of Defense Technology); Quanyan Zhu (University of Illinois at
Urbana-Champaign);
Raouf Boutaba (University of Waterloo); Joseph L. Hellerstein (Google,
Inc.)
2:30PM - 3:30PM
Short Papers II
VESPA: Multi-Layered Self-Protection for Cloud Resources
Aurélien Wailly, Marc Lacoste (Orange Labs); Hervé Debar (Télécom SudParis)
Usage Patterns in Multi-tenant Data Centers: a Temporal Perspective
Robert Birke, Lydia Y. Chen (IBM Research Zurich Lab); Evgenia Smirni
(College of
William and Mary)
Toward Fast Eventual Consistency with Performance Guarantees
Feng Yan (College of William and Mary); Alma Riska (EMC Corporation);
Evgenia Smirni (College
of William and Mary)
Optimal Autoscaling in the IaaS Cloud
Hamoun Ghanbari, Bradley Simmons, Marin Litoiu, Cornel Barna (York
University);
Gabriel Iszlai (IBM Toronto)
3:30PM – 4:00PM
Break
4:00PM - 6:00PM
Poster and Demo Session
6:00PM-9:00PM
Conference Outing (tentative)
======================================================================
THURSDAY, SEPTEMBER 20, 2012 - MAIN CONFERENCE
8:00AM – 9:00AM
Registration
9:00AM – 10:00AM
Keynote Talk III
High Efficiency at Web Scale
Eitan Frachtenberg, Facebook
10:00AM-10:30AM
Break
10:30AM - 12:00PM
Session: Diagnosis and Monitoring
Chair: TBD
3-Dimensional Root Cause Diagnosis via Co-analysi
Ziming Zheng, Li Yu, Zhiling Lan (Illinois Institute of Technology);
Terry Jones
(Oak Ridge National Laboratory)
UBL: Unsupervised Behavior Learning for Predicting Performance Anomalies
in Virtualized Cloud Systems
Daniel J. Dean, Hiep Nguyen, Xiaohui Gu (North Carolina State University)
Evaluating Compressive Sampling Strategies for Performance Monitoring of
Data Centers
Tingshan Huang, Nagarajan Kandasamy, Harish Sethu (Drexel University)
12:00PM
Adjourn
======================================================================
FRIDAY, SEPTEMBER 21, 2012 - WORKSHOPS
Management of Big Data Systems 2012
Federated Clouds 2012
**********************************************************************
ORGANIZERS
==========
GENERAL CHAIR
Dejan Milojicic, HP Labs
PROGRAM CHAIRS
Dongyan Xu, Purdue University
Vanish Talwar, HP Labs
INDUSTRY CHAIR
Xiaoyun Zhu, VMware
WORKSHOPS CHAIR
Fred Douglis, EMC
POSTERS/DEMO/EXHIBITS CHAIR
Eno Thereska, Microsoft Research
FINANCE CHAIR
Michael Kozuch, Intel
LOCAL ARRANGEMENT CHAIR
Jessica Blaine
PUBLICITY CHAIRS
Daniel Batista, University of São Paulo
Vartan Padaryan, ISP/Russian Academy of Sci.
Ioan Raicu, Illinois Inst. of Technology
Jianfeng Zhan, ICT/Chinese Academy of Sci.
Ming Zhao, Florida Intl. University
PROGRAM COMMITTEE
Tarek Abdelzaher, UIUC
Umesh Bellur, IIT, Bombay
Ken Birman, Cornell University
Rajkumar Buyya, Univ. of Melbourne
Rocky Chang, Hong Kong Polytechnic University
Yuan Chen, HP Labs
Alva Couch, Tufts University
Peter Dinda, Northwestern University
Fred Douglis, EMC
Renato Figueiredo, University of Florida
Mohamed Hefeeda, QCRI
Joe Hellerstein, Google
Geoff Jiang, NEC Labs
Jeff Kephart, IBM Research
Emre Kiciman, Microsoft Research
Fabio Kon, University of São Paulo
Mike Kozuch, Intel Labs
Dejan Milojicic, HP Labs
Klara Nahrstedt, UIUC
Priya Narasimhan, CMU
Manish Parashar, Rutgers University
Ioan Raicu, Illinois Inst. of Technology
Omer Rana, Cardiff University
Masoud Sadjadi, Florida Intl. University
Richard Schlichting, AT&T Labs
Hartmut Schmeck, KIT
Karsten Schwan, Georgia Tech
Onn Shehory, IBM Research
Eno Thereska, Microsoft Research
Xiaoyun Zhu, VMware
**********************************************************************
--
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel: 1-847-722-0876
Office: 1-312-567-5704
Email: iraicu@cs.iit.edu
Web: http://www.cs.iit.edu/~iraicu/
Web: http://datasys.cs.iit.edu/
=================================================================
=================================================================
^ permalink raw reply
* [PATCH v9 0/5] make balloon pages movable by compaction
From: Rafael Aquini @ 2012-08-25 5:24 UTC (permalink / raw)
To: linux-mm
Cc: Rik van Riel, Rafael Aquini, Konrad Rzeszutek Wilk,
Michael S. Tsirkin, linux-kernel, virtualization, Minchan Kim,
Peter Zijlstra, Andi Kleen, Andrew Morton, Paul E. McKenney
Memory fragmentation introduced by ballooning might reduce significantly
the number of 2MB contiguous memory blocks that can be used within a guest,
thus imposing performance penalties associated with the reduced number of
transparent huge pages that could be used by the guest workload.
This patch-set follows the main idea discussed at 2012 LSFMMS session:
"Ballooning for transparent huge pages" -- http://lwn.net/Articles/490114/
to introduce the required changes to the virtio_balloon driver, as well as
the changes to the core compaction & migration bits, in order to make those
subsystems aware of ballooned pages and allow memory balloon pages become
movable within a guest, thus avoiding the aforementioned fragmentation issue
Rafael Aquini (5):
mm: introduce a common interface for balloon pages mobility
mm: introduce compaction and migration for ballooned pages
virtio_balloon: introduce migration primitives to balloon pages
mm: introduce putback_movable_pages()
mm: add vm event counters for balloon pages compaction
drivers/virtio/virtio_balloon.c | 287 ++++++++++++++++++++++++++++++++++---
include/linux/balloon_compaction.h | 137 ++++++++++++++++++
include/linux/migrate.h | 2 +
include/linux/pagemap.h | 18 +++
include/linux/vm_event_item.h | 8 +-
mm/Kconfig | 15 ++
mm/Makefile | 2 +-
mm/balloon_compaction.c | 174 ++++++++++++++++++++++
mm/compaction.c | 51 ++++---
mm/migrate.c | 57 +++++++-
mm/page_alloc.c | 2 +-
mm/vmstat.c | 10 +-
12 files changed, 715 insertions(+), 48 deletions(-)
create mode 100644 include/linux/balloon_compaction.h
create mode 100644 mm/balloon_compaction.c
Change log:
v9:
* Adjust rcu_dereference usage to leverage page lock protection (Paul, Peter);
* Enhance doc on compaction interface introduced to balloon driver (Michael);
* Fix issue with isolated pages breaking leak_balloon() logics (Michael);
v8:
* introduce a common MM interface for balloon driver page compaction (Michael);
* remove the global state preventing multiple balloon device support (Michael);
* introduce RCU protection/syncrhonization to balloon page->mapping (Michael);
v7:
* fix a potential page leak case at 'putback_balloon_page' (Mel);
* adjust vm-events-counter patch and remove its drop-on-merge message (Rik);
* add 'putback_movable_pages' to avoid hacks on 'putback_lru_pages' (Minchan);
v6:
* rename 'is_balloon_page()' to 'movable_balloon_page()' (Rik);
v5:
* address Andrew Morton's review comments on the patch series;
* address a couple extra nitpick suggestions on PATCH 01 (Minchan);
v4:
* address Rusty Russel's review comments on PATCH 02;
* re-base virtio_balloon patch on 9c378abc5c0c6fc8e3acf5968924d274503819b3;
V3:
* address reviewers nitpick suggestions on PATCH 01 (Mel, Minchan);
V2:
* address Mel Gorman's review comments on PATCH 01;
Preliminary test results:
(2 VCPU 2048mB RAM KVM guest running 3.6.0_rc3+ -- after a reboot)
* 64mB balloon:
[root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
compact_blocks_moved 0
compact_pages_moved 0
compact_pagemigrate_failed 0
compact_stall 0
compact_fail 0
compact_success 0
compact_balloon_isolated 0
compact_balloon_migrated 0
compact_balloon_released 0
compact_balloon_returned 0
[root@localhost ~]#
[root@localhost ~]# for i in $(seq 1 6); do echo 1 > /proc/sys/vm/compact_memory & done &>/dev/null
[1] Done echo 1 > /proc/sys/vm/compact_memory
[2] Done echo 1 > /proc/sys/vm/compact_memory
[3] Done echo 1 > /proc/sys/vm/compact_memory
[4] Done echo 1 > /proc/sys/vm/compact_memory
[5]- Done echo 1 > /proc/sys/vm/compact_memory
[6]+ Done echo 1 > /proc/sys/vm/compact_memory
[root@localhost ~]#
[root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
compact_blocks_moved 3108
compact_pages_moved 43169
compact_pagemigrate_failed 95
compact_stall 0
compact_fail 0
compact_success 0
compact_balloon_isolated 16384
compact_balloon_migrated 16384
compact_balloon_released 16384
compact_balloon_returned 0
* 128 mB balloon:
[root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
compact_blocks_moved 0
compact_pages_moved 0
compact_pagemigrate_failed 0
compact_stall 0
compact_fail 0
compact_success 0
compact_balloon_isolated 0
compact_balloon_migrated 0
compact_balloon_released 0
compact_balloon_returned 0
[root@localhost ~]#
[root@localhost ~]# for i in $(seq 1 6); do echo 1 > /proc/sys/vm/compact_memory & done &>/dev/null
[1] Done echo 1 > /proc/sys/vm/compact_memory
[2] Done echo 1 > /proc/sys/vm/compact_memory
[3] Done echo 1 > /proc/sys/vm/compact_memory
[4] Done echo 1 > /proc/sys/vm/compact_memory
[5]- Done echo 1 > /proc/sys/vm/compact_memory
[6]+ Done echo 1 > /proc/sys/vm/compact_memory
[root@localhost ~]#
[root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
compact_blocks_moved 3062
compact_pages_moved 49774
compact_pagemigrate_failed 129
compact_stall 0
compact_fail 0
compact_success 0
compact_balloon_isolated 26076
compact_balloon_migrated 25957
compact_balloon_released 25957
compact_balloon_returned 119
--
1.7.11.4
^ permalink raw reply
* [PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility
From: Rafael Aquini @ 2012-08-25 5:24 UTC (permalink / raw)
To: linux-mm
Cc: Rik van Riel, Rafael Aquini, Konrad Rzeszutek Wilk,
Michael S. Tsirkin, linux-kernel, virtualization, Minchan Kim,
Peter Zijlstra, Andi Kleen, Andrew Morton, Paul E. McKenney
In-Reply-To: <cover.1345869378.git.aquini@redhat.com>
Memory fragmentation introduced by ballooning might reduce significantly
the number of 2MB contiguous memory blocks that can be used within a guest,
thus imposing performance penalties associated with the reduced number of
transparent huge pages that could be used by the guest workload.
This patch introduces a common interface to help a balloon driver on
making its page set movable to compaction, and thus allowing the system
to better leverage the compation efforts on memory defragmentation.
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
include/linux/balloon_compaction.h | 137 +++++++++++++++++++++++++++++
include/linux/pagemap.h | 18 ++++
mm/Kconfig | 15 ++++
mm/Makefile | 2 +-
mm/balloon_compaction.c | 172 +++++++++++++++++++++++++++++++++++++
5 files changed, 343 insertions(+), 1 deletion(-)
create mode 100644 include/linux/balloon_compaction.h
create mode 100644 mm/balloon_compaction.c
diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
new file mode 100644
index 0000000..7afb0ae
--- /dev/null
+++ b/include/linux/balloon_compaction.h
@@ -0,0 +1,137 @@
+/*
+ * include/linux/balloon_compaction.h
+ *
+ * Common interface definitions for making balloon pages movable to compaction.
+ *
+ * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <aquini@redhat.com>
+ */
+#ifndef _LINUX_BALLOON_COMPACTION_H
+#define _LINUX_BALLOON_COMPACTION_H
+#ifdef __KERNEL__
+
+#include <linux/rcupdate.h>
+#include <linux/pagemap.h>
+#include <linux/gfp.h>
+
+#ifdef CONFIG_BALLOON_COMPACTION
+#define count_balloon_event(e) count_vm_event(e)
+extern bool isolate_balloon_page(struct page *);
+extern void putback_balloon_page(struct page *);
+extern int migrate_balloon_page(struct page *newpage,
+ struct page *page, enum migrate_mode mode);
+
+static inline gfp_t balloon_mapping_gfp_mask(void)
+{
+ return GFP_HIGHUSER_MOVABLE;
+}
+
+/*
+ * movable_balloon_page - test page->mapping->flags to identify balloon pages
+ * that can be moved by compaction/migration.
+ *
+ * This function is used at core compaction's page isolation scheme and so it's
+ * exposed to several system pages which may, or may not, be part of a memory
+ * balloon, and thus we cannot afford to hold a page locked to perform tests.
+ *
+ * Therefore, as we might return false positives in the case a balloon page
+ * is just released under us, the page->mapping->flags need to be retested
+ * with the proper page lock held, on the functions that will cope with the
+ * balloon page later.
+ */
+static inline bool movable_balloon_page(struct page *page)
+{
+ /*
+ * Before dereferencing and testing mapping->flags, lets make sure
+ * this is not a page that uses ->mapping in a different way
+ */
+ if (!PageSlab(page) && !PageSwapCache(page) &&
+ !PageAnon(page) && !page_mapped(page)) {
+ /*
+ * While doing compaction core work, we cannot afford to hold
+ * page lock as it might cause very undesirable side effects.
+ */
+ struct address_space *mapping;
+ mapping = rcu_dereference_raw(page->mapping);
+ if (mapping)
+ return mapping_balloon(mapping);
+ }
+ return false;
+}
+
+/*
+ * __page_balloon_device - return the balloon device owing the page.
+ *
+ * This shall only be used at driver callbacks under proper page lock,
+ * to get access to the balloon device structure that owns @page.
+ */
+static inline void *__page_balloon_device(struct page *page)
+{
+ struct address_space *mapping;
+ mapping = rcu_dereference_protected(page->mapping, PageLocked(page));
+ if (mapping)
+ mapping = mapping->assoc_mapping;
+ return (void *)mapping;
+}
+
+/*
+ * DEFINE_BALLOON_MAPPING_AOPS - declare and instantiate a callback descriptor
+ * to be used as balloon page->mapping->a_ops.
+ *
+ * @label : declaration identifier (var name)
+ * @isolatepg : callback symbol name for performing the page isolation step
+ * @migratepg : callback symbol name for performing the page migration step
+ * @putbackpg : callback symbol name for performing the page putback step
+ *
+ * address_space_operations utilized methods for ballooned pages:
+ * .migratepage - used to perform balloon's page migration (as is)
+ * .invalidatepage - used to isolate a page from balloon's page list
+ * .freepage - used to reinsert an isolated page to balloon's page list
+ */
+#define DEFINE_BALLOON_MAPPING_AOPS(label, isolatepg, migratepg, putbackpg) \
+ const struct address_space_operations (label) = { \
+ .migratepage = (migratepg), \
+ .invalidatepage = (isolatepg), \
+ .freepage = (putbackpg), \
+ }
+
+#else
+#define count_balloon_event(e) do { } while (0)
+static inline bool movable_balloon_page(struct page *page) { return false; }
+static inline bool isolate_balloon_page(struct page *page) { return false; }
+static inline void putback_balloon_page(struct page *page) { return; }
+
+static inline int migrate_balloon_page(struct page *newpage,
+ struct page *page, enum migrate_mode mode)
+{
+ return 0;
+}
+
+static inline gfp_t balloon_mapping_gfp_mask(void)
+{
+ return GFP_HIGHUSER;
+}
+
+#define DEFINE_BALLOON_MAPPING_AOPS(label, migratepg, isolatepg, putbackpg) \
+ const struct address_space_operations *(label) = NULL
+
+#endif /* CONFIG_BALLOON_COMPACTION */
+
+/* return code to identify when a ballooned page has been migrated */
+#define BALLOON_MIGRATION_RETURN 0xba1100
+
+extern struct address_space *alloc_balloon_mapping(void *balloon_device,
+ const struct address_space_operations *a_ops);
+
+static inline void assign_balloon_mapping(struct page *page,
+ struct address_space *mapping)
+{
+ rcu_assign_pointer(page->mapping, mapping);
+}
+
+static inline void clear_balloon_mapping(struct page *page)
+{
+ rcu_assign_pointer(page->mapping, NULL);
+}
+
+#endif /* __KERNEL__ */
+#endif /* _LINUX_BALLOON_COMPACTION_H */
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index e42c762..6df0664 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -24,6 +24,7 @@ enum mapping_flags {
AS_ENOSPC = __GFP_BITS_SHIFT + 1, /* ENOSPC on async write */
AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
+ AS_BALLOON_MAP = __GFP_BITS_SHIFT + 4, /* balloon page special map */
};
static inline void mapping_set_error(struct address_space *mapping, int error)
@@ -53,6 +54,23 @@ static inline int mapping_unevictable(struct address_space *mapping)
return !!mapping;
}
+static inline void mapping_set_balloon(struct address_space *mapping)
+{
+ set_bit(AS_BALLOON_MAP, &mapping->flags);
+}
+
+static inline void mapping_clear_balloon(struct address_space *mapping)
+{
+ clear_bit(AS_BALLOON_MAP, &mapping->flags);
+}
+
+static inline int mapping_balloon(struct address_space *mapping)
+{
+ if (mapping)
+ return test_bit(AS_BALLOON_MAP, &mapping->flags);
+ return !!mapping;
+}
+
static inline gfp_t mapping_gfp_mask(struct address_space * mapping)
{
return (__force gfp_t)mapping->flags & __GFP_BITS_MASK;
diff --git a/mm/Kconfig b/mm/Kconfig
index d5c8019..0bd783b 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -188,6 +188,21 @@ config SPLIT_PTLOCK_CPUS
default "4"
#
+# support for memory balloon compaction
+config BALLOON_COMPACTION
+ bool "Allow for balloon memory compaction/migration"
+ select COMPACTION
+ depends on VIRTIO_BALLOON
+ help
+ Memory fragmentation introduced by ballooning might reduce
+ significantly the number of 2MB contiguous memory blocks that can be
+ used within a guest, thus imposing performance penalties associated
+ with the reduced number of transparent huge pages that could be used
+ by the guest workload. Allowing the compaction & migration for memory
+ pages enlisted as being part of memory balloon devices avoids the
+ scenario aforementioned and helps improving memory defragmentation.
+
+#
# support for memory compaction
config COMPACTION
bool "Allow for memory compaction"
diff --git a/mm/Makefile b/mm/Makefile
index 92753e2..78d8caa 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
readahead.o swap.o truncate.o vmscan.o shmem.o \
prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
mm_init.o mmu_context.o percpu.o slab_common.o \
- compaction.o $(mmu-y)
+ compaction.o balloon_compaction.o $(mmu-y)
obj-y += init-mm.o
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
new file mode 100644
index 0000000..86a3692
--- /dev/null
+++ b/mm/balloon_compaction.c
@@ -0,0 +1,172 @@
+/*
+ * mm/balloon_compaction.c
+ *
+ * Common interface for making balloon pages movable to compaction.
+ *
+ * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <aquini@redhat.com>
+ */
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/export.h>
+#include <linux/balloon_compaction.h>
+
+/*
+ * alloc_balloon_mapping - allocates a special ->mapping for ballooned pages.
+ * @balloon_device: pointer address that references the balloon device which
+ * owns pages bearing this ->mapping.
+ * @a_ops: balloon_mapping address_space_operations descriptor.
+ *
+ * Users must call it to properly allocate and initialize an instance of
+ * struct address_space which will be used as the special page->mapping for
+ * balloon devices enlisted page instances.
+ */
+struct address_space *alloc_balloon_mapping(void *balloon_device,
+ const struct address_space_operations *a_ops)
+{
+ struct address_space *mapping;
+
+ mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
+ if (!mapping)
+ return NULL;
+
+ /*
+ * Give a clean 'zeroed' status to all elements of this special
+ * balloon page->mapping struct address_space instance.
+ */
+ address_space_init_once(mapping);
+
+ /*
+ * Set mapping->flags appropriately, to allow balloon ->mapping
+ * identification, as well as give a proper hint to the balloon
+ * driver on what GFP allocation mask shall be used.
+ */
+ mapping_set_balloon(mapping);
+ mapping_set_gfp_mask(mapping, balloon_mapping_gfp_mask());
+
+ /* balloon's page->mapping->a_ops callback descriptor */
+ mapping->a_ops = a_ops;
+
+ /*
+ * balloon special page->mapping overloads ->assoc_mapping
+ * to held a reference back to the balloon device wich 'owns'
+ * a given page. This is the way we can cope with multiple
+ * balloon devices without losing reference of several
+ * ballooned pagesets.
+ */
+ mapping->assoc_mapping = balloon_device;
+
+ return mapping;
+}
+EXPORT_SYMBOL_GPL(alloc_balloon_mapping);
+
+#ifdef CONFIG_BALLOON_COMPACTION
+
+static inline bool __is_movable_balloon_page(struct page *page)
+{
+ struct address_space *mapping;
+
+ mapping = rcu_dereference_protected(page->mapping, PageLocked(page));
+ if (mapping)
+ return mapping_balloon(mapping);
+
+ return false;
+}
+
+static inline void __isolate_balloon_page(struct page *page)
+{
+ page->mapping->a_ops->invalidatepage(page, 0);
+}
+
+static inline void __putback_balloon_page(struct page *page)
+{
+ page->mapping->a_ops->freepage(page);
+}
+
+static inline int __migrate_balloon_page(struct address_space *mapping,
+ struct page *newpage, struct page *page, enum migrate_mode mode)
+{
+ return page->mapping->a_ops->migratepage(mapping, newpage, page, mode);
+}
+
+/* __isolate_lru_page() counterpart for a ballooned page */
+bool isolate_balloon_page(struct page *page)
+{
+ if (likely(get_page_unless_zero(page))) {
+ /*
+ * As balloon pages are not isolated from LRU lists, concurrent
+ * compaction threads can race against page migration functions
+ * move_to_new_page() & __unmap_and_move().
+ * In order to avoid having an already isolated balloon page
+ * being (wrongly) re-isolated while it is under migration,
+ * lets be sure we have the page lock before proceeding with
+ * the balloon page isolation steps.
+ */
+ if (likely(trylock_page(page))) {
+ /*
+ * A ballooned page, by default, has just one refcount.
+ * Prevent concurrent compaction threads from isolating
+ * an already isolated balloon page by refcount check.
+ */
+ if (__is_movable_balloon_page(page) &&
+ (page_count(page) == 2)) {
+ __isolate_balloon_page(page);
+ unlock_page(page);
+ return true;
+ } else if (unlikely(!__is_movable_balloon_page(page))) {
+ dump_page(page);
+ __WARN();
+ }
+ unlock_page(page);
+ }
+ /*
+ * The page is either under migration, or it's isolated already
+ * Drop the refcount taken for it.
+ */
+ put_page(page);
+ }
+ return false;
+}
+
+/* putback_lru_page() counterpart for a ballooned page */
+void putback_balloon_page(struct page *page)
+{
+ /*
+ * 'lock_page()' stabilizes the page and prevents races against
+ * concurrent isolation threads attempting to re-isolate it.
+ */
+ lock_page(page);
+
+ if (__is_movable_balloon_page(page)) {
+ __putback_balloon_page(page);
+ put_page(page);
+ } else {
+ dump_page(page);
+ __WARN();
+ }
+ unlock_page(page);
+}
+
+/* move_to_new_page() counterpart for a ballooned page */
+int migrate_balloon_page(struct page *newpage,
+ struct page *page, enum migrate_mode mode)
+{
+ struct address_space *mapping;
+ int rc = -EAGAIN;
+
+ BUG_ON(!trylock_page(newpage));
+
+ if (WARN_ON(!__is_movable_balloon_page(page))) {
+ dump_page(page);
+ unlock_page(newpage);
+ return rc;
+ }
+
+ mapping = rcu_dereference_protected(page->mapping, PageLocked(page));
+ if (mapping)
+ rc = __migrate_balloon_page(mapping, newpage, page, mode);
+
+ unlock_page(newpage);
+ return rc;
+}
+
+#endif /* CONFIG_BALLOON_COMPACTION */
--
1.7.11.4
^ permalink raw reply related
* [PATCH v9 2/5] mm: introduce compaction and migration for ballooned pages
From: Rafael Aquini @ 2012-08-25 5:24 UTC (permalink / raw)
To: linux-mm
Cc: Rik van Riel, Rafael Aquini, Konrad Rzeszutek Wilk,
Michael S. Tsirkin, linux-kernel, virtualization, Minchan Kim,
Peter Zijlstra, Andi Kleen, Andrew Morton, Paul E. McKenney
In-Reply-To: <cover.1345869378.git.aquini@redhat.com>
Memory fragmentation introduced by ballooning might reduce significantly
the number of 2MB contiguous memory blocks that can be used within a guest,
thus imposing performance penalties associated with the reduced number of
transparent huge pages that could be used by the guest workload.
This patch introduces the helper functions as well as the necessary changes
to teach compaction and migration bits how to cope with pages which are
part of a guest memory balloon, in order to make them movable by memory
compaction procedures.
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
mm/compaction.c | 47 ++++++++++++++++++++++++++++-------------------
mm/migrate.c | 36 ++++++++++++++++++++++++++++++++++--
2 files changed, 62 insertions(+), 21 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 7fcd3a5..e50836b 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -14,6 +14,7 @@
#include <linux/backing-dev.h>
#include <linux/sysctl.h>
#include <linux/sysfs.h>
+#include <linux/balloon_compaction.h>
#include "internal.h"
#if defined CONFIG_COMPACTION || defined CONFIG_CMA
@@ -358,32 +359,40 @@ isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
continue;
}
- if (!PageLRU(page))
- continue;
-
/*
- * PageLRU is set, and lru_lock excludes isolation,
- * splitting and collapsing (collapsing has already
- * happened if PageLRU is set).
+ * It is possible to migrate LRU pages and balloon pages.
+ * Skip any other type of page.
*/
- if (PageTransHuge(page)) {
- low_pfn += (1 << compound_order(page)) - 1;
- continue;
- }
+ if (PageLRU(page)) {
+ /*
+ * PageLRU is set, and lru_lock excludes isolation,
+ * splitting and collapsing (collapsing has already
+ * happened if PageLRU is set).
+ */
+ if (PageTransHuge(page)) {
+ low_pfn += (1 << compound_order(page)) - 1;
+ continue;
+ }
- if (!cc->sync)
- mode |= ISOLATE_ASYNC_MIGRATE;
+ if (!cc->sync)
+ mode |= ISOLATE_ASYNC_MIGRATE;
- lruvec = mem_cgroup_page_lruvec(page, zone);
+ lruvec = mem_cgroup_page_lruvec(page, zone);
- /* Try isolate the page */
- if (__isolate_lru_page(page, mode) != 0)
- continue;
+ /* Try isolate the page */
+ if (__isolate_lru_page(page, mode) != 0)
+ continue;
- VM_BUG_ON(PageTransCompound(page));
+ VM_BUG_ON(PageTransCompound(page));
+
+ /* Successfully isolated */
+ del_page_from_lru_list(page, lruvec, page_lru(page));
+ } else if (unlikely(movable_balloon_page(page))) {
+ if (!isolate_balloon_page(page))
+ continue;
+ } else
+ continue;
- /* Successfully isolated */
- del_page_from_lru_list(page, lruvec, page_lru(page));
list_add(&page->lru, migratelist);
cc->nr_migratepages++;
nr_isolated++;
diff --git a/mm/migrate.c b/mm/migrate.c
index 77ed2d7..ec439f8 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -35,6 +35,7 @@
#include <linux/hugetlb.h>
#include <linux/hugetlb_cgroup.h>
#include <linux/gfp.h>
+#include <linux/balloon_compaction.h>
#include <asm/tlbflush.h>
@@ -79,7 +80,10 @@ void putback_lru_pages(struct list_head *l)
list_del(&page->lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
- putback_lru_page(page);
+ if (unlikely(movable_balloon_page(page)))
+ putback_balloon_page(page);
+ else
+ putback_lru_page(page);
}
}
@@ -799,6 +803,18 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
goto skip_unmap;
}
+ if (unlikely(movable_balloon_page(page))) {
+ /*
+ * A ballooned page does not need any special attention from
+ * physical to virtual reverse mapping procedures.
+ * Skip any attempt to unmap PTEs or to remap swap cache,
+ * in order to avoid burning cycles at rmap level, and perform
+ * the page migration right away (proteced by page lock).
+ */
+ rc = migrate_balloon_page(newpage, page, mode);
+ goto uncharge;
+ }
+
/* Establish migration ptes or remove ptes */
try_to_unmap(page, TTU_MIGRATION|TTU_IGNORE_MLOCK|TTU_IGNORE_ACCESS);
@@ -814,7 +830,8 @@ skip_unmap:
put_anon_vma(anon_vma);
uncharge:
- mem_cgroup_end_migration(mem, page, newpage, rc == 0);
+ mem_cgroup_end_migration(mem, page, newpage,
+ (rc == 0 || rc == BALLOON_MIGRATION_RETURN));
unlock:
unlock_page(page);
out:
@@ -846,6 +863,21 @@ static int unmap_and_move(new_page_t get_new_page, unsigned long private,
goto out;
rc = __unmap_and_move(page, newpage, force, offlining, mode);
+
+ if (unlikely(rc == BALLOON_MIGRATION_RETURN)) {
+ /*
+ * A ballooned page has been migrated already.
+ * Now, it's the time to remove the old page from the isolated
+ * pageset list and handle it back to Buddy, wrap-up counters
+ * and return.
+ */
+ dec_zone_page_state(page, NR_ISOLATED_ANON +
+ page_is_file_cache(page));
+ list_del(&page->lru);
+ put_page(page);
+ __free_page(page);
+ return 0;
+ }
out:
if (rc != -EAGAIN) {
/*
--
1.7.11.4
^ permalink raw reply related
* [PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages
From: Rafael Aquini @ 2012-08-25 5:24 UTC (permalink / raw)
To: linux-mm
Cc: Rik van Riel, Rafael Aquini, Konrad Rzeszutek Wilk,
Michael S. Tsirkin, linux-kernel, virtualization, Minchan Kim,
Peter Zijlstra, Andi Kleen, Andrew Morton, Paul E. McKenney
In-Reply-To: <cover.1345869378.git.aquini@redhat.com>
Memory fragmentation introduced by ballooning might reduce significantly
the number of 2MB contiguous memory blocks that can be used within a guest,
thus imposing performance penalties associated with the reduced number of
transparent huge pages that could be used by the guest workload.
Besides making balloon pages movable at allocation time and introducing
the necessary primitives to perform balloon page migration/compaction,
the patch changes the balloon bookeeping pages counter into an atomic
counter, as well as it introduces the following locking scheme, in order to
enhance the syncronization methods for accessing elements of struct
virtio_balloon, thus providing protection against the concurrent accesses
introduced by parallel memory compaction threads.
- balloon_lock (mutex) : synchronizes the access demand to elements of
struct virtio_balloon and its queue operations;
- pages_lock (spinlock): special protection to balloon's pages bookmarking
elements (list and atomic counters) against the
potential memory compaction concurrency;
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
drivers/virtio/virtio_balloon.c | 286 +++++++++++++++++++++++++++++++++++++---
1 file changed, 265 insertions(+), 21 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 0908e60..9b0bc46 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -27,6 +27,8 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/module.h>
+#include <linux/balloon_compaction.h>
+#include <linux/atomic.h>
/*
* Balloon device works in 4K page units. So each page is pointed to by
@@ -34,6 +36,7 @@
* page units.
*/
#define VIRTIO_BALLOON_PAGES_PER_PAGE (PAGE_SIZE >> VIRTIO_BALLOON_PFN_SHIFT)
+#define VIRTIO_BALLOON_ARRAY_PFNS_MAX 256
struct virtio_balloon
{
@@ -46,11 +49,24 @@ struct virtio_balloon
/* The thread servicing the balloon. */
struct task_struct *thread;
+ /* balloon special page->mapping */
+ struct address_space *mapping;
+
+ /* Synchronize access/update to this struct virtio_balloon elements */
+ struct mutex balloon_lock;
+
/* Waiting for host to ack the pages we released. */
wait_queue_head_t acked;
+ /* Number of balloon pages isolated from 'pages' list for compaction */
+ atomic_t num_isolated_pages;
+
/* Number of balloon pages we've told the Host we're not using. */
- unsigned int num_pages;
+ atomic_t num_pages;
+
+ /* Protect pages list, and pages bookeeping counters */
+ spinlock_t pages_lock;
+
/*
* The pages we've told the Host we're not using.
* Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
@@ -60,7 +76,7 @@ struct virtio_balloon
/* The array of pfns we tell the Host about. */
unsigned int num_pfns;
- u32 pfns[256];
+ u32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
/* Memory statistics */
int need_stats_update;
@@ -122,13 +138,17 @@ static void set_page_pfns(u32 pfns[], struct page *page)
static void fill_balloon(struct virtio_balloon *vb, size_t num)
{
+ /* Get the proper GFP alloc mask from vb->mapping flags */
+ gfp_t vb_gfp_mask = mapping_gfp_mask(vb->mapping);
+
/* We can only do one array worth at a time. */
num = min(num, ARRAY_SIZE(vb->pfns));
+ mutex_lock(&vb->balloon_lock);
for (vb->num_pfns = 0; vb->num_pfns < num;
vb->num_pfns += VIRTIO_BALLOON_PAGES_PER_PAGE) {
- struct page *page = alloc_page(GFP_HIGHUSER | __GFP_NORETRY |
- __GFP_NOMEMALLOC | __GFP_NOWARN);
+ struct page *page = alloc_page(vb_gfp_mask | __GFP_NORETRY |
+ __GFP_NOWARN | __GFP_NOMEMALLOC);
if (!page) {
if (printk_ratelimit())
dev_printk(KERN_INFO, &vb->vdev->dev,
@@ -139,9 +159,15 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num)
break;
}
set_page_pfns(vb->pfns + vb->num_pfns, page);
- vb->num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE;
totalram_pages--;
+
+ BUG_ON(!trylock_page(page));
+ spin_lock(&vb->pages_lock);
list_add(&page->lru, &vb->pages);
+ assign_balloon_mapping(page, vb->mapping);
+ atomic_add(VIRTIO_BALLOON_PAGES_PER_PAGE, &vb->num_pages);
+ spin_unlock(&vb->pages_lock);
+ unlock_page(page);
}
/* Didn't get any? Oh well. */
@@ -149,6 +175,7 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num)
return;
tell_host(vb, vb->inflate_vq);
+ mutex_unlock(&vb->balloon_lock);
}
static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
@@ -162,19 +189,97 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
}
}
+#ifdef CONFIG_BALLOON_COMPACTION
+/* helper to __wait_on_isolated_pages() getting vb->pages list lenght */
+static inline int __pages_at_balloon_list(struct virtio_balloon *vb)
+{
+ return atomic_read(&vb->num_pages) -
+ atomic_read(&vb->num_isolated_pages);
+}
+
+/*
+ * __wait_on_isolated_pages - check if leak_balloon() must wait on isolated
+ * pages before proceeding with the page release.
+ * @vb : pointer to the struct virtio_balloon describing this device.
+ * @leak_target: how many pages we are attempting to release this round.
+ *
+ * Shall only be called by leak_balloon() and under spin_lock(&vb->pages_lock);
+ */
+static inline void __wait_on_isolated_pages(struct virtio_balloon *vb,
+ size_t leak_target)
+{
+ /*
+ * There are no isolated pages for this balloon device, or
+ * the leak target is smaller than # of pages on vb->pages list.
+ * No need to wait, then.
+ */
+ if (!atomic_read(&vb->num_isolated_pages) ||
+ leak_target < __pages_at_balloon_list(vb))
+ return;
+ else {
+ /*
+ * isolated pages are making our leak target bigger than the
+ * total pages that we can release this round. Let's wait for
+ * migration returning enough pages back to balloon's list.
+ */
+ spin_unlock(&vb->pages_lock);
+ wait_event(vb->config_change,
+ (!atomic_read(&vb->num_isolated_pages) ||
+ leak_target <= __pages_at_balloon_list(vb)));
+ spin_lock(&vb->pages_lock);
+ }
+}
+#else
+#define __wait_on_isolated_pages(a, b) do { } while (0)
+#endif /* CONFIG_BALLOON_COMPACTION */
+
static void leak_balloon(struct virtio_balloon *vb, size_t num)
{
- struct page *page;
+ int i;
+ /* The array of pfns we tell the Host about. */
+ u32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
+ unsigned int num_pfns = 0;
/* We can only do one array worth at a time. */
- num = min(num, ARRAY_SIZE(vb->pfns));
+ size_t leak_target = num = min(num, ARRAY_SIZE(pfns));
- for (vb->num_pfns = 0; vb->num_pfns < num;
- vb->num_pfns += VIRTIO_BALLOON_PAGES_PER_PAGE) {
- page = list_first_entry(&vb->pages, struct page, lru);
- list_del(&page->lru);
- set_page_pfns(vb->pfns + vb->num_pfns, page);
- vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
+ while (num_pfns < num) {
+ struct page *page = NULL;
+
+ spin_lock(&vb->pages_lock);
+ /*
+ * leak_balloon() works releasing balloon pages by groups
+ * of 'VIRTIO_BALLOON_ARRAY_PFNS_MAX' size at each round.
+ * When compaction isolates pages from balloon page list,
+ * we might end up finding less pages on balloon's list than
+ * what is our desired 'leak_target'. If such occurrence
+ * happens, we shall wait for enough pages being re-inserted
+ * into balloon's page list before we proceed releasing them.
+ */
+ __wait_on_isolated_pages(vb, leak_target);
+
+ if (!list_empty(&vb->pages))
+ page = list_first_entry(&vb->pages, struct page, lru);
+ /*
+ * Grab the page lock to avoid racing against threads isolating
+ * pages from, or migrating pages back to vb->pages list.
+ * (both tasks are done under page lock protection)
+ *
+ * Failing to grab the page lock here means this page is being
+ * isolated already, or its migration has not finished yet.
+ */
+ if (page && trylock_page(page)) {
+ clear_balloon_mapping(page);
+ list_del(&page->lru);
+ set_page_pfns(pfns + num_pfns, page);
+ atomic_sub(VIRTIO_BALLOON_PAGES_PER_PAGE,
+ &vb->num_pages);
+ num_pfns += VIRTIO_BALLOON_PAGES_PER_PAGE;
+ unlock_page(page);
+ /* compensate leak_target for this released page */
+ leak_target--;
+ }
+ spin_unlock(&vb->pages_lock);
}
/*
@@ -182,8 +287,15 @@ static void leak_balloon(struct virtio_balloon *vb, size_t num)
* virtio_has_feature(vdev, VIRTIO_BALLOON_F_MUST_TELL_HOST);
* is true, we *have* to do it in this order
*/
+ mutex_lock(&vb->balloon_lock);
+
+ for (i = 0; i < num; i++)
+ vb->pfns[i] = pfns[i];
+
+ vb->num_pfns = num_pfns;
tell_host(vb, vb->deflate_vq);
- release_pages_by_pfn(vb->pfns, vb->num_pfns);
+ release_pages_by_pfn(pfns, num_pfns);
+ mutex_unlock(&vb->balloon_lock);
}
static inline void update_stat(struct virtio_balloon *vb, int idx,
@@ -239,6 +351,7 @@ static void stats_handle_request(struct virtio_balloon *vb)
struct scatterlist sg;
unsigned int len;
+ mutex_lock(&vb->balloon_lock);
vb->need_stats_update = 0;
update_balloon_stats(vb);
@@ -249,6 +362,7 @@ static void stats_handle_request(struct virtio_balloon *vb)
if (virtqueue_add_buf(vq, &sg, 1, 0, vb, GFP_KERNEL) < 0)
BUG();
virtqueue_kick(vq);
+ mutex_unlock(&vb->balloon_lock);
}
static void virtballoon_changed(struct virtio_device *vdev)
@@ -267,12 +381,12 @@ static inline s64 towards_target(struct virtio_balloon *vb)
offsetof(struct virtio_balloon_config, num_pages),
&v, sizeof(v));
target = le32_to_cpu(v);
- return target - vb->num_pages;
+ return target - atomic_read(&vb->num_pages);
}
static void update_balloon_size(struct virtio_balloon *vb)
{
- __le32 actual = cpu_to_le32(vb->num_pages);
+ __le32 actual = cpu_to_le32(atomic_read(&vb->num_pages));
vb->vdev->config->set(vb->vdev,
offsetof(struct virtio_balloon_config, actual),
@@ -339,9 +453,124 @@ static int init_vqs(struct virtio_balloon *vb)
return 0;
}
+#ifdef CONFIG_BALLOON_COMPACTION
+/*
+ * virtballoon_isolatepage - perform the balloon page isolation on behalf of
+ * a compation thread.
+ * (must be called under page lock)
+ * @page: the page to isolated from balloon's page list.
+ * @mode: not used for balloon page isolation.
+ *
+ * A memory compaction thread works isolating pages from private lists,
+ * like LRUs or the balloon's page list (here), to a privative pageset that
+ * will be migrated subsequently. After the mentioned pageset gets isolated
+ * compaction relies on page migration procedures to do the heavy lifting.
+ *
+ * This function populates a balloon_mapping->a_ops callback method to help
+ * a compaction thread on isolating a page from the balloon page list, and
+ * thus allowing its posterior migration.
+ */
+void virtballoon_isolatepage(struct page *page, unsigned long mode)
+{
+ struct virtio_balloon *vb = __page_balloon_device(page);
+
+ BUG_ON(!vb);
+
+ spin_lock(&vb->pages_lock);
+ list_del(&page->lru);
+ atomic_inc(&vb->num_isolated_pages);
+ spin_unlock(&vb->pages_lock);
+}
+
+/*
+ * virtballoon_migratepage - perform the balloon page migration on behalf of
+ * a compation thread.
+ * (must be called under page lock)
+ * @mapping: the page->mapping which will be assigned to the new migrated page.
+ * @newpage: page that will replace the isolated page after migration finishes.
+ * @page : the isolated (old) page that is about to be migrated to newpage.
+ * @mode : compaction mode -- not used for balloon page migration.
+ *
+ * After a ballooned page gets isolated by compaction procedures, this is the
+ * function that performs the page migration on behalf of a compaction running
+ * thread. The page migration for virtio balloon is done in a simple swap
+ * fashion which follows these two macro steps:
+ * 1) insert newpage into vb->pages list and update the host about it;
+ * 2) update the host about the removed old page from vb->pages list;
+ *
+ * This function populates a balloon_mapping->a_ops callback method to allow
+ * a compaction thread to perform the balloon page migration task.
+ */
+int virtballoon_migratepage(struct address_space *mapping,
+ struct page *newpage, struct page *page, enum migrate_mode mode)
+{
+ struct virtio_balloon *vb = __page_balloon_device(page);
+
+ BUG_ON(!vb);
+
+ mutex_lock(&vb->balloon_lock);
+
+ /* balloon's page migration 1st step */
+ vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
+ spin_lock(&vb->pages_lock);
+ list_add(&newpage->lru, &vb->pages);
+ assign_balloon_mapping(newpage, mapping);
+ atomic_dec(&vb->num_isolated_pages);
+ spin_unlock(&vb->pages_lock);
+ set_page_pfns(vb->pfns, newpage);
+ tell_host(vb, vb->inflate_vq);
+
+ /* balloon's page migration 2nd step */
+ vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
+ clear_balloon_mapping(page);
+ set_page_pfns(vb->pfns, page);
+ tell_host(vb, vb->deflate_vq);
+
+ mutex_unlock(&vb->balloon_lock);
+ wake_up(&vb->config_change);
+
+ return BALLOON_MIGRATION_RETURN;
+}
+
+/*
+ * virtballoon_putbackpage - insert an isolated page back into the list it was
+ * once taken off by a compaction thread.
+ * (must be called under page lock)
+ * @page: page that will be re-inserted into balloon page list.
+ *
+ * If by any mean, a compaction thread cannot finish all its job on its round,
+ * and some isolated pages are still remaining at compaction's thread privative
+ * pageset (waiting for migration), then those pages will get re-inserted into
+ * their appropriate lists before the compaction thread exits.
+ *
+ * This function populates a balloon_mapping->a_ops callback method to help
+ * compaction on inserting back into the appropriate list an isolated but
+ * not migrated balloon page.
+ */
+void virtballoon_putbackpage(struct page *page)
+{
+ struct virtio_balloon *vb = __page_balloon_device(page);
+
+ BUG_ON(!vb);
+
+ spin_lock(&vb->pages_lock);
+ list_add(&page->lru, &vb->pages);
+ atomic_dec(&vb->num_isolated_pages);
+ spin_unlock(&vb->pages_lock);
+ wake_up(&vb->config_change);
+}
+#endif /* CONFIG_BALLOON_COMPACTION */
+
+/* define the balloon_mapping->a_ops callbacks to allow compaction/migration */
+static DEFINE_BALLOON_MAPPING_AOPS(virtio_balloon_aops,
+ virtballoon_isolatepage,
+ virtballoon_migratepage,
+ virtballoon_putbackpage);
+
static int virtballoon_probe(struct virtio_device *vdev)
{
struct virtio_balloon *vb;
+ struct address_space *vb_mapping;
int err;
vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
@@ -351,15 +580,26 @@ static int virtballoon_probe(struct virtio_device *vdev)
}
INIT_LIST_HEAD(&vb->pages);
- vb->num_pages = 0;
+ mutex_init(&vb->balloon_lock);
+ spin_lock_init(&vb->pages_lock);
+
+ atomic_set(&vb->num_pages, 0);
+ atomic_set(&vb->num_isolated_pages, 0);
init_waitqueue_head(&vb->config_change);
init_waitqueue_head(&vb->acked);
vb->vdev = vdev;
vb->need_stats_update = 0;
+ vb_mapping = alloc_balloon_mapping(vb, &virtio_balloon_aops);
+ if (!vb_mapping) {
+ err = -ENOMEM;
+ goto out_free_vb;
+ }
+ vb->mapping = vb_mapping;
+
err = init_vqs(vb);
if (err)
- goto out_free_vb;
+ goto out_free_vb_mapping;
vb->thread = kthread_run(balloon, vb, "vballoon");
if (IS_ERR(vb->thread)) {
@@ -371,6 +611,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
out_del_vqs:
vdev->config->del_vqs(vdev);
+out_free_vb_mapping:
+ kfree(vb_mapping);
out_free_vb:
kfree(vb);
out:
@@ -379,9 +621,11 @@ out:
static void remove_common(struct virtio_balloon *vb)
{
+ size_t num_pages;
/* There might be pages left in the balloon: free them. */
- while (vb->num_pages)
- leak_balloon(vb, vb->num_pages);
+ while ((num_pages = atomic_read(&vb->num_pages)) > 0)
+ leak_balloon(vb, num_pages);
+
update_balloon_size(vb);
/* Now we reset the device so we can clean up the queues. */
@@ -396,6 +640,7 @@ static void __devexit virtballoon_remove(struct virtio_device *vdev)
kthread_stop(vb->thread);
remove_common(vb);
+ kfree(vb->mapping);
kfree(vb);
}
@@ -408,7 +653,6 @@ static int virtballoon_freeze(struct virtio_device *vdev)
* The kthread is already frozen by the PM core before this
* function is called.
*/
-
remove_common(vb);
return 0;
}
--
1.7.11.4
^ permalink raw reply related
* [PATCH v9 4/5] mm: introduce putback_movable_pages()
From: Rafael Aquini @ 2012-08-25 5:24 UTC (permalink / raw)
To: linux-mm
Cc: Rik van Riel, Rafael Aquini, Konrad Rzeszutek Wilk,
Michael S. Tsirkin, linux-kernel, virtualization, Minchan Kim,
Peter Zijlstra, Andi Kleen, Andrew Morton, Paul E. McKenney
In-Reply-To: <cover.1345869378.git.aquini@redhat.com>
The PATCH "mm: introduce compaction and migration for virtio ballooned pages"
hacks around putback_lru_pages() in order to allow ballooned pages to be
re-inserted on balloon page list as if a ballooned page was like a LRU page.
As ballooned pages are not legitimate LRU pages, this patch introduces
putback_movable_pages() to properly cope with cases where the isolated
pageset contains ballooned pages and LRU pages, thus fixing the mentioned
inelegant hack around putback_lru_pages().
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
include/linux/migrate.h | 2 ++
mm/compaction.c | 4 ++--
mm/migrate.c | 20 ++++++++++++++++++++
mm/page_alloc.c | 2 +-
4 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/include/linux/migrate.h b/include/linux/migrate.h
index ce7e667..ff103a1 100644
--- a/include/linux/migrate.h
+++ b/include/linux/migrate.h
@@ -10,6 +10,7 @@ typedef struct page *new_page_t(struct page *, unsigned long private, int **);
#ifdef CONFIG_MIGRATION
extern void putback_lru_pages(struct list_head *l);
+extern void putback_movable_pages(struct list_head *l);
extern int migrate_page(struct address_space *,
struct page *, struct page *, enum migrate_mode);
extern int migrate_pages(struct list_head *l, new_page_t x,
@@ -33,6 +34,7 @@ extern int migrate_huge_page_move_mapping(struct address_space *mapping,
#else
static inline void putback_lru_pages(struct list_head *l) {}
+static inline void putback_movable_pages(struct list_head *l) {}
static inline int migrate_pages(struct list_head *l, new_page_t x,
unsigned long private, bool offlining,
enum migrate_mode mode) { return -ENOSYS; }
diff --git a/mm/compaction.c b/mm/compaction.c
index e50836b..409b2f5 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -817,9 +817,9 @@ static int compact_zone(struct zone *zone, struct compact_control *cc)
trace_mm_compaction_migratepages(nr_migrate - nr_remaining,
nr_remaining);
- /* Release LRU pages not migrated */
+ /* Release isolated pages not migrated */
if (err) {
- putback_lru_pages(&cc->migratepages);
+ putback_movable_pages(&cc->migratepages);
cc->nr_migratepages = 0;
if (err == -ENOMEM) {
ret = COMPACT_PARTIAL;
diff --git a/mm/migrate.c b/mm/migrate.c
index ec439f8..e47daf5 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -80,6 +80,26 @@ void putback_lru_pages(struct list_head *l)
list_del(&page->lru);
dec_zone_page_state(page, NR_ISOLATED_ANON +
page_is_file_cache(page));
+ putback_lru_page(page);
+ }
+}
+
+/*
+ * Put previously isolated pages back onto the appropriate lists
+ * from where they were once taken off for compaction/migration.
+ *
+ * This function shall be used instead of putback_lru_pages(),
+ * whenever the isolated pageset has been built by isolate_migratepages_range()
+ */
+void putback_movable_pages(struct list_head *l)
+{
+ struct page *page;
+ struct page *page2;
+
+ list_for_each_entry_safe(page, page2, l, lru) {
+ list_del(&page->lru);
+ dec_zone_page_state(page, NR_ISOLATED_ANON +
+ page_is_file_cache(page));
if (unlikely(movable_balloon_page(page)))
putback_balloon_page(page);
else
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index c66fb87..a0c2cc5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5675,7 +5675,7 @@ static int __alloc_contig_migrate_range(unsigned long start, unsigned long end)
0, false, MIGRATE_SYNC);
}
- putback_lru_pages(&cc.migratepages);
+ putback_movable_pages(&cc.migratepages);
return ret > 0 ? 0 : ret;
}
--
1.7.11.4
^ permalink raw reply related
* [PATCH v9 5/5] mm: add vm event counters for balloon pages compaction
From: Rafael Aquini @ 2012-08-25 5:25 UTC (permalink / raw)
To: linux-mm
Cc: Rik van Riel, Rafael Aquini, Konrad Rzeszutek Wilk,
Michael S. Tsirkin, linux-kernel, virtualization, Minchan Kim,
Peter Zijlstra, Andi Kleen, Andrew Morton, Paul E. McKenney
In-Reply-To: <cover.1345869378.git.aquini@redhat.com>
This patch introduces a new set of vm event counters to keep track of
ballooned pages compaction activity.
Signed-off-by: Rafael Aquini <aquini@redhat.com>
---
drivers/virtio/virtio_balloon.c | 1 +
include/linux/vm_event_item.h | 8 +++++++-
mm/balloon_compaction.c | 2 ++
mm/migrate.c | 1 +
mm/vmstat.c | 10 +++++++++-
5 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
index 9b0bc46..e1e8e30 100644
--- a/drivers/virtio/virtio_balloon.c
+++ b/drivers/virtio/virtio_balloon.c
@@ -528,6 +528,7 @@ int virtballoon_migratepage(struct address_space *mapping,
mutex_unlock(&vb->balloon_lock);
wake_up(&vb->config_change);
+ count_balloon_event(COMPACTBALLOONMIGRATED);
return BALLOON_MIGRATION_RETURN;
}
diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
index 57f7b10..13573fe 100644
--- a/include/linux/vm_event_item.h
+++ b/include/linux/vm_event_item.h
@@ -41,7 +41,13 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
#ifdef CONFIG_COMPACTION
COMPACTBLOCKS, COMPACTPAGES, COMPACTPAGEFAILED,
COMPACTSTALL, COMPACTFAIL, COMPACTSUCCESS,
-#endif
+#ifdef CONFIG_BALLOON_COMPACTION
+ COMPACTBALLOONISOLATED, /* isolated from balloon pagelist */
+ COMPACTBALLOONMIGRATED, /* balloon page sucessfully migrated */
+ COMPACTBALLOONRELEASED, /* old-page released after migration */
+ COMPACTBALLOONRETURNED, /* putback to pagelist, not-migrated */
+#endif /* CONFIG_BALLOON_COMPACTION */
+#endif /* CONFIG_COMPACTION */
#ifdef CONFIG_HUGETLB_PAGE
HTLB_BUDDY_PGALLOC, HTLB_BUDDY_PGALLOC_FAIL,
#endif
diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 86a3692..00e7ea9 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -110,6 +110,7 @@ bool isolate_balloon_page(struct page *page)
if (__is_movable_balloon_page(page) &&
(page_count(page) == 2)) {
__isolate_balloon_page(page);
+ count_balloon_event(COMPACTBALLOONISOLATED);
unlock_page(page);
return true;
} else if (unlikely(!__is_movable_balloon_page(page))) {
@@ -139,6 +140,7 @@ void putback_balloon_page(struct page *page)
if (__is_movable_balloon_page(page)) {
__putback_balloon_page(page);
put_page(page);
+ count_balloon_event(COMPACTBALLOONRETURNED);
} else {
dump_page(page);
__WARN();
diff --git a/mm/migrate.c b/mm/migrate.c
index e47daf5..124b16b 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -896,6 +896,7 @@ static int unmap_and_move(new_page_t get_new_page, unsigned long private,
list_del(&page->lru);
put_page(page);
__free_page(page);
+ count_balloon_event(COMPACTBALLOONRELEASED);
return 0;
}
out:
diff --git a/mm/vmstat.c b/mm/vmstat.c
index df7a674..5824ad2 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -768,7 +768,15 @@ const char * const vmstat_text[] = {
"compact_stall",
"compact_fail",
"compact_success",
-#endif
+
+#ifdef CONFIG_BALLOON_COMPACTION
+ "compact_balloon_isolated",
+ "compact_balloon_migrated",
+ "compact_balloon_released",
+ "compact_balloon_returned",
+#endif /* CONFIG_BALLOON_COMPACTION */
+
+#endif /* CONFIG_COMPACTION */
#ifdef CONFIG_HUGETLB_PAGE
"htlb_buddy_alloc_success",
--
1.7.11.4
^ permalink raw reply related
* [PATCH] vhost: remove duplicated include from tcm_vhost.c
From: Wei Yongjun @ 2012-08-26 1:42 UTC (permalink / raw)
To: mst; +Cc: netdev, yongjun_wei, kvm, virtualization
From: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
From: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Remove duplicated include.
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
---
drivers/vhost/tcm_vhost.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/vhost/tcm_vhost.c b/drivers/vhost/tcm_vhost.c
index fb36654..587ca7e 100644
--- a/drivers/vhost/tcm_vhost.c
+++ b/drivers/vhost/tcm_vhost.c
@@ -45,7 +45,6 @@
#include <target/target_core_fabric_configfs.h>
#include <target/target_core_configfs.h>
#include <target/configfs_macros.h>
-#include <linux/vhost.h>
#include <linux/virtio_net.h> /* TODO vhost.h currently depends on this */
#include <linux/virtio_scsi.h>
^ permalink raw reply related
* Re: [PATCH] vhost: remove duplicated include from tcm_vhost.c
From: Stefan Hajnoczi @ 2012-08-26 7:22 UTC (permalink / raw)
To: Wei Yongjun; +Cc: netdev, yongjun_wei, virtualization, kvm, mst
In-Reply-To: <CAPgLHd-GRu8Dc3bxLL4ogjsE6gY=OS7n0H-5JFyMi_dy2OXP9g@mail.gmail.com>
On Sun, Aug 26, 2012 at 2:42 AM, Wei Yongjun <weiyj.lk@gmail.com> wrote:
> From: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
>
> From: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
>
> Remove duplicated include.
>
> Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
> ---
> drivers/vhost/tcm_vhost.c | 1 -
> 1 file changed, 1 deletion(-)
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
^ permalink raw reply
* Re: [PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages
From: Michael S. Tsirkin @ 2012-08-26 7:42 UTC (permalink / raw)
To: Rafael Aquini
Cc: Rik van Riel, Konrad Rzeszutek Wilk, linux-kernel, virtualization,
linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim, Andrew Morton,
Paul E. McKenney
In-Reply-To: <a1ceca79d95bc7de2a6b62a2e565b95286dbdf75.1345869378.git.aquini@redhat.com>
On Sat, Aug 25, 2012 at 02:24:58AM -0300, Rafael Aquini wrote:
> Memory fragmentation introduced by ballooning might reduce significantly
> the number of 2MB contiguous memory blocks that can be used within a guest,
> thus imposing performance penalties associated with the reduced number of
> transparent huge pages that could be used by the guest workload.
>
> Besides making balloon pages movable at allocation time and introducing
> the necessary primitives to perform balloon page migration/compaction,
> the patch changes the balloon bookeeping pages counter into an atomic
> counter, as well as it introduces the following locking scheme, in order to
> enhance the syncronization methods for accessing elements of struct
> virtio_balloon, thus providing protection against the concurrent accesses
> introduced by parallel memory compaction threads.
>
> - balloon_lock (mutex) : synchronizes the access demand to elements of
> struct virtio_balloon and its queue operations;
> - pages_lock (spinlock): special protection to balloon's pages bookmarking
> elements (list and atomic counters) against the
> potential memory compaction concurrency;
>
> Signed-off-by: Rafael Aquini <aquini@redhat.com>
OK, this looks better.
Some comments below.
> ---
> drivers/virtio/virtio_balloon.c | 286 +++++++++++++++++++++++++++++++++++++---
> 1 file changed, 265 insertions(+), 21 deletions(-)
>
> diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
> index 0908e60..9b0bc46 100644
> --- a/drivers/virtio/virtio_balloon.c
> +++ b/drivers/virtio/virtio_balloon.c
> @@ -27,6 +27,8 @@
> #include <linux/delay.h>
> #include <linux/slab.h>
> #include <linux/module.h>
> +#include <linux/balloon_compaction.h>
> +#include <linux/atomic.h>
>
> /*
> * Balloon device works in 4K page units. So each page is pointed to by
> @@ -34,6 +36,7 @@
> * page units.
> */
> #define VIRTIO_BALLOON_PAGES_PER_PAGE (PAGE_SIZE >> VIRTIO_BALLOON_PFN_SHIFT)
> +#define VIRTIO_BALLOON_ARRAY_PFNS_MAX 256
>
> struct virtio_balloon
> {
> @@ -46,11 +49,24 @@ struct virtio_balloon
> /* The thread servicing the balloon. */
> struct task_struct *thread;
>
> + /* balloon special page->mapping */
> + struct address_space *mapping;
> +
> + /* Synchronize access/update to this struct virtio_balloon elements */
> + struct mutex balloon_lock;
> +
> /* Waiting for host to ack the pages we released. */
> wait_queue_head_t acked;
>
> + /* Number of balloon pages isolated from 'pages' list for compaction */
> + atomic_t num_isolated_pages;
> +
> /* Number of balloon pages we've told the Host we're not using. */
> - unsigned int num_pages;
> + atomic_t num_pages;
> +
> + /* Protect pages list, and pages bookeeping counters */
> + spinlock_t pages_lock;
> +
> /*
> * The pages we've told the Host we're not using.
> * Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE
> @@ -60,7 +76,7 @@ struct virtio_balloon
>
> /* The array of pfns we tell the Host about. */
> unsigned int num_pfns;
> - u32 pfns[256];
> + u32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
>
> /* Memory statistics */
> int need_stats_update;
> @@ -122,13 +138,17 @@ static void set_page_pfns(u32 pfns[], struct page *page)
>
> static void fill_balloon(struct virtio_balloon *vb, size_t num)
> {
> + /* Get the proper GFP alloc mask from vb->mapping flags */
> + gfp_t vb_gfp_mask = mapping_gfp_mask(vb->mapping);
> +
> /* We can only do one array worth at a time. */
> num = min(num, ARRAY_SIZE(vb->pfns));
>
> + mutex_lock(&vb->balloon_lock);
> for (vb->num_pfns = 0; vb->num_pfns < num;
> vb->num_pfns += VIRTIO_BALLOON_PAGES_PER_PAGE) {
> - struct page *page = alloc_page(GFP_HIGHUSER | __GFP_NORETRY |
> - __GFP_NOMEMALLOC | __GFP_NOWARN);
> + struct page *page = alloc_page(vb_gfp_mask | __GFP_NORETRY |
> + __GFP_NOWARN | __GFP_NOMEMALLOC);
> if (!page) {
> if (printk_ratelimit())
> dev_printk(KERN_INFO, &vb->vdev->dev,
> @@ -139,9 +159,15 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num)
> break;
> }
> set_page_pfns(vb->pfns + vb->num_pfns, page);
> - vb->num_pages += VIRTIO_BALLOON_PAGES_PER_PAGE;
> totalram_pages--;
> +
> + BUG_ON(!trylock_page(page));
> + spin_lock(&vb->pages_lock);
> list_add(&page->lru, &vb->pages);
> + assign_balloon_mapping(page, vb->mapping);
> + atomic_add(VIRTIO_BALLOON_PAGES_PER_PAGE, &vb->num_pages);
> + spin_unlock(&vb->pages_lock);
> + unlock_page(page);
> }
>
> /* Didn't get any? Oh well. */
> @@ -149,6 +175,7 @@ static void fill_balloon(struct virtio_balloon *vb, size_t num)
> return;
>
> tell_host(vb, vb->inflate_vq);
> + mutex_unlock(&vb->balloon_lock);
> }
>
> static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
> @@ -162,19 +189,97 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)
> }
> }
>
> +#ifdef CONFIG_BALLOON_COMPACTION
> +/* helper to __wait_on_isolated_pages() getting vb->pages list lenght */
> +static inline int __pages_at_balloon_list(struct virtio_balloon *vb)
> +{
> + return atomic_read(&vb->num_pages) -
> + atomic_read(&vb->num_isolated_pages);
> +}
> +
Reading two atomics and doing math? Result can even be negative.
I did not look at use closely but it looks suspicious.
It's already the case everywhere except __wait_on_isolated_pages,
so just fix that, and then we can keep using int instead of atomics.
> +/*
> + * __wait_on_isolated_pages - check if leak_balloon() must wait on isolated
> + * pages before proceeding with the page release.
> + * @vb : pointer to the struct virtio_balloon describing this device.
> + * @leak_target: how many pages we are attempting to release this round.
> + *
> + * Shall only be called by leak_balloon() and under spin_lock(&vb->pages_lock);
> + */
> +static inline void __wait_on_isolated_pages(struct virtio_balloon *vb,
> + size_t leak_target)
> +{
> + /*
> + * There are no isolated pages for this balloon device, or
> + * the leak target is smaller than # of pages on vb->pages list.
> + * No need to wait, then.
> + */
This just repeats what's below. So it does not help
at all, better drop it. But maybe you could explain
why does it make sense?
> + if (!atomic_read(&vb->num_isolated_pages) ||
> + leak_target < __pages_at_balloon_list(vb))
> + return;
> + else {
> + /*
> + * isolated pages are making our leak target bigger than the
> + * total pages that we can release this round. Let's wait for
> + * migration returning enough pages back to balloon's list.
> + */
> + spin_unlock(&vb->pages_lock);
> + wait_event(vb->config_change,
> + (!atomic_read(&vb->num_isolated_pages) ||
> + leak_target <= __pages_at_balloon_list(vb)));
Why did we repeat the logic above? optimization to skip lock/unlock?
> + spin_lock(&vb->pages_lock);
> + }
> +}
> +#else
> +#define __wait_on_isolated_pages(a, b) do { } while (0)
> +#endif /* CONFIG_BALLOON_COMPACTION */
> +
> static void leak_balloon(struct virtio_balloon *vb, size_t num)
> {
> - struct page *page;
> + int i;
> + /* The array of pfns we tell the Host about. */
> + u32 pfns[VIRTIO_BALLOON_ARRAY_PFNS_MAX];
That's 1K on stack - and can become more if we increase
VIRTIO_BALLOON_ARRAY_PFNS_MAX. Probably too much - this is the reason
we use vb->pfns.
> + unsigned int num_pfns = 0;
>
> /* We can only do one array worth at a time. */
> - num = min(num, ARRAY_SIZE(vb->pfns));
> + size_t leak_target = num = min(num, ARRAY_SIZE(pfns));
>
> - for (vb->num_pfns = 0; vb->num_pfns < num;
> - vb->num_pfns += VIRTIO_BALLOON_PAGES_PER_PAGE) {
> - page = list_first_entry(&vb->pages, struct page, lru);
> - list_del(&page->lru);
> - set_page_pfns(vb->pfns + vb->num_pfns, page);
> - vb->num_pages -= VIRTIO_BALLOON_PAGES_PER_PAGE;
> + while (num_pfns < num) {
> + struct page *page = NULL;
> +
> + spin_lock(&vb->pages_lock);
> + /*
> + * leak_balloon() works releasing balloon pages by groups
> + * of 'VIRTIO_BALLOON_ARRAY_PFNS_MAX' size at each round.
> + * When compaction isolates pages from balloon page list,
> + * we might end up finding less pages on balloon's list than
> + * what is our desired 'leak_target'. If such occurrence
> + * happens, we shall wait for enough pages being re-inserted
> + * into balloon's page list before we proceed releasing them.
> + */
> + __wait_on_isolated_pages(vb, leak_target);
> +
> + if (!list_empty(&vb->pages))
> + page = list_first_entry(&vb->pages, struct page, lru);
> + /*
> + * Grab the page lock to avoid racing against threads isolating
> + * pages from, or migrating pages back to vb->pages list.
> + * (both tasks are done under page lock protection)
> + *
> + * Failing to grab the page lock here means this page is being
> + * isolated already, or its migration has not finished yet.
> + */
> + if (page && trylock_page(page)) {
> + clear_balloon_mapping(page);
> + list_del(&page->lru);
> + set_page_pfns(pfns + num_pfns, page);
> + atomic_sub(VIRTIO_BALLOON_PAGES_PER_PAGE,
> + &vb->num_pages);
> + num_pfns += VIRTIO_BALLOON_PAGES_PER_PAGE;
> + unlock_page(page);
> + /* compensate leak_target for this released page */
> + leak_target--;
> + }
> + spin_unlock(&vb->pages_lock);
> }
>
> /*
> @@ -182,8 +287,15 @@ static void leak_balloon(struct virtio_balloon *vb, size_t num)
> * virtio_has_feature(vdev, VIRTIO_BALLOON_F_MUST_TELL_HOST);
> * is true, we *have* to do it in this order
> */
> + mutex_lock(&vb->balloon_lock);
> +
> + for (i = 0; i < num; i++)
> + vb->pfns[i] = pfns[i];
> +
> + vb->num_pfns = num_pfns;
> tell_host(vb, vb->deflate_vq);
> - release_pages_by_pfn(vb->pfns, vb->num_pfns);
> + release_pages_by_pfn(pfns, num_pfns);
> + mutex_unlock(&vb->balloon_lock);
> }
>
> static inline void update_stat(struct virtio_balloon *vb, int idx,
> @@ -239,6 +351,7 @@ static void stats_handle_request(struct virtio_balloon *vb)
> struct scatterlist sg;
> unsigned int len;
>
> + mutex_lock(&vb->balloon_lock);
> vb->need_stats_update = 0;
> update_balloon_stats(vb);
>
> @@ -249,6 +362,7 @@ static void stats_handle_request(struct virtio_balloon *vb)
> if (virtqueue_add_buf(vq, &sg, 1, 0, vb, GFP_KERNEL) < 0)
> BUG();
> virtqueue_kick(vq);
> + mutex_unlock(&vb->balloon_lock);
> }
>
> static void virtballoon_changed(struct virtio_device *vdev)
> @@ -267,12 +381,12 @@ static inline s64 towards_target(struct virtio_balloon *vb)
> offsetof(struct virtio_balloon_config, num_pages),
> &v, sizeof(v));
> target = le32_to_cpu(v);
> - return target - vb->num_pages;
> + return target - atomic_read(&vb->num_pages);
> }
>
> static void update_balloon_size(struct virtio_balloon *vb)
> {
> - __le32 actual = cpu_to_le32(vb->num_pages);
> + __le32 actual = cpu_to_le32(atomic_read(&vb->num_pages));
>
> vb->vdev->config->set(vb->vdev,
> offsetof(struct virtio_balloon_config, actual),
> @@ -339,9 +453,124 @@ static int init_vqs(struct virtio_balloon *vb)
> return 0;
> }
>
> +#ifdef CONFIG_BALLOON_COMPACTION
> +/*
> + * virtballoon_isolatepage - perform the balloon page isolation on behalf of
> + * a compation thread.
> + * (must be called under page lock)
Better 'called under page lock' - driver is not supposed
to call this.
> + * @page: the page to isolated from balloon's page list.
> + * @mode: not used for balloon page isolation.
> + *
> + * A memory compaction thread works isolating pages from private lists,
by isolating pages
> + * like LRUs or the balloon's page list (here), to a privative pageset that
> + * will be migrated subsequently. After the mentioned pageset gets isolated
> + * compaction relies on page migration procedures to do the heavy lifting.
> + *
> + * This function populates a balloon_mapping->a_ops callback method to help
> + * a compaction thread on isolating a page from the balloon page list, and
> + * thus allowing its posterior migration.
This function isolates a page from the balloon private page list.
Called through balloon_mapping->a_ops.
> + */
> +void virtballoon_isolatepage(struct page *page, unsigned long mode)
> +{
> + struct virtio_balloon *vb = __page_balloon_device(page);
> +
> + BUG_ON(!vb);
> +
> + spin_lock(&vb->pages_lock);
> + list_del(&page->lru);
> + atomic_inc(&vb->num_isolated_pages);
> + spin_unlock(&vb->pages_lock);
> +}
> +
> +/*
> + * virtballoon_migratepage - perform the balloon page migration on behalf of
> + * a compation thread.
> + * (must be called under page lock)
> + * @mapping: the page->mapping which will be assigned to the new migrated page.
> + * @newpage: page that will replace the isolated page after migration finishes.
> + * @page : the isolated (old) page that is about to be migrated to newpage.
> + * @mode : compaction mode -- not used for balloon page migration.
> + *
> + * After a ballooned page gets isolated by compaction procedures, this is the
> + * function that performs the page migration on behalf of a compaction running
> + * thread.
of a compaction thread
> The page migration for virtio balloon is done in a simple swap
> + * fashion which follows these two macro steps:
> + * 1) insert newpage into vb->pages list and update the host about it;
> + * 2) update the host about the removed old page from vb->pages list;
the old page removed from
> + *
> + * This function populates a balloon_mapping->a_ops callback method to allow
> + * a compaction thread to perform the balloon page migration task.
This function preforms the balloon page migration task.
Called through balloon_mapping->a_ops.
> + */
> +int virtballoon_migratepage(struct address_space *mapping,
> + struct page *newpage, struct page *page, enum migrate_mode mode)
> +{
> + struct virtio_balloon *vb = __page_balloon_device(page);
> +
> + BUG_ON(!vb);
> +
> + mutex_lock(&vb->balloon_lock);
> +
> + /* balloon's page migration 1st step */
> + vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> + spin_lock(&vb->pages_lock);
> + list_add(&newpage->lru, &vb->pages);
> + assign_balloon_mapping(newpage, mapping);
> + atomic_dec(&vb->num_isolated_pages);
> + spin_unlock(&vb->pages_lock);
> + set_page_pfns(vb->pfns, newpage);
> + tell_host(vb, vb->inflate_vq);
> +
> + /* balloon's page migration 2nd step */
> + vb->num_pfns = VIRTIO_BALLOON_PAGES_PER_PAGE;
> + clear_balloon_mapping(page);
> + set_page_pfns(vb->pfns, page);
> + tell_host(vb, vb->deflate_vq);
> +
> + mutex_unlock(&vb->balloon_lock);
> + wake_up(&vb->config_change);
> +
> + return BALLOON_MIGRATION_RETURN;
> +}
> +
> +/*
> + * virtballoon_putbackpage - insert an isolated page back into the list it was
> + * once taken off by a compaction thread.
> + * (must be called under page lock)
> + * @page: page that will be re-inserted into balloon page list.
> + *
> + * If by any mean,
If for some reason
> a compaction thread cannot finish all its job on its round,
in one round
> + * and some isolated pages are still remaining at compaction's thread privative
> + * pageset (waiting for migration), then those pages will get re-inserted into
> + * their appropriate lists
appropriate -> private balloon
> before the compaction thread exits.
will exit.
> + *
> + * This function populates a balloon_mapping->a_ops callback method to help
> + * compaction on inserting back into the appropriate list an isolated but
> + * not migrated balloon page.
This function inserts an isolated but not migrated balloon page
back into private balloon list.
Called through balloon_mapping->a_ops.
> + */
> +void virtballoon_putbackpage(struct page *page)
> +{
> + struct virtio_balloon *vb = __page_balloon_device(page);
> +
> + BUG_ON(!vb);
> +
> + spin_lock(&vb->pages_lock);
> + list_add(&page->lru, &vb->pages);
> + atomic_dec(&vb->num_isolated_pages);
> + spin_unlock(&vb->pages_lock);
> + wake_up(&vb->config_change);
> +}
> +#endif /* CONFIG_BALLOON_COMPACTION */
> +
> +/* define the balloon_mapping->a_ops callbacks to allow compaction/migration */
> +static DEFINE_BALLOON_MAPPING_AOPS(virtio_balloon_aops,
> + virtballoon_isolatepage,
> + virtballoon_migratepage,
> + virtballoon_putbackpage);
> +
> static int virtballoon_probe(struct virtio_device *vdev)
> {
> struct virtio_balloon *vb;
> + struct address_space *vb_mapping;
> int err;
>
> vdev->priv = vb = kmalloc(sizeof(*vb), GFP_KERNEL);
> @@ -351,15 +580,26 @@ static int virtballoon_probe(struct virtio_device *vdev)
> }
>
> INIT_LIST_HEAD(&vb->pages);
> - vb->num_pages = 0;
> + mutex_init(&vb->balloon_lock);
> + spin_lock_init(&vb->pages_lock);
> +
> + atomic_set(&vb->num_pages, 0);
> + atomic_set(&vb->num_isolated_pages, 0);
> init_waitqueue_head(&vb->config_change);
> init_waitqueue_head(&vb->acked);
> vb->vdev = vdev;
> vb->need_stats_update = 0;
>
> + vb_mapping = alloc_balloon_mapping(vb, &virtio_balloon_aops);
> + if (!vb_mapping) {
> + err = -ENOMEM;
> + goto out_free_vb;
> + }
> + vb->mapping = vb_mapping;
> +
> err = init_vqs(vb);
> if (err)
> - goto out_free_vb;
> + goto out_free_vb_mapping;
>
> vb->thread = kthread_run(balloon, vb, "vballoon");
> if (IS_ERR(vb->thread)) {
> @@ -371,6 +611,8 @@ static int virtballoon_probe(struct virtio_device *vdev)
>
> out_del_vqs:
> vdev->config->del_vqs(vdev);
> +out_free_vb_mapping:
> + kfree(vb_mapping);
We have alloc_balloon_mapping, it would be cleaner to have
free_balloon_mapping.
> out_free_vb:
> kfree(vb);
> out:
> @@ -379,9 +621,11 @@ out:
>
> static void remove_common(struct virtio_balloon *vb)
> {
> + size_t num_pages;
> /* There might be pages left in the balloon: free them. */
> - while (vb->num_pages)
> - leak_balloon(vb, vb->num_pages);
> + while ((num_pages = atomic_read(&vb->num_pages)) > 0)
> + leak_balloon(vb, num_pages);
> +
> update_balloon_size(vb);
>
> /* Now we reset the device so we can clean up the queues. */
> @@ -396,6 +640,7 @@ static void __devexit virtballoon_remove(struct virtio_device *vdev)
>
> kthread_stop(vb->thread);
> remove_common(vb);
> + kfree(vb->mapping);
> kfree(vb);
> }
>
> @@ -408,7 +653,6 @@ static int virtballoon_freeze(struct virtio_device *vdev)
> * The kthread is already frozen by the PM core before this
> * function is called.
> */
> -
> remove_common(vb);
> return 0;
> }
> --
> 1.7.11.4
^ permalink raw reply
* Re: [PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility
From: Michael S. Tsirkin @ 2012-08-26 7:55 UTC (permalink / raw)
To: Rafael Aquini
Cc: Rik van Riel, Konrad Rzeszutek Wilk, linux-kernel, virtualization,
linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim, Andrew Morton,
Paul E. McKenney
In-Reply-To: <aa4af6e819584cb05fc0dba44594ae23ab761d03.1345869378.git.aquini@redhat.com>
On Sat, Aug 25, 2012 at 02:24:56AM -0300, Rafael Aquini wrote:
> Memory fragmentation introduced by ballooning might reduce significantly
> the number of 2MB contiguous memory blocks that can be used within a guest,
> thus imposing performance penalties associated with the reduced number of
> transparent huge pages that could be used by the guest workload.
>
> This patch introduces a common interface to help a balloon driver on
> making its page set movable to compaction, and thus allowing the system
> to better leverage the compation efforts on memory defragmentation.
>
> Signed-off-by: Rafael Aquini <aquini@redhat.com>
Tons of rcu uses but not sync in sight. This looks suspicious.
> ---
> include/linux/balloon_compaction.h | 137 +++++++++++++++++++++++++++++
> include/linux/pagemap.h | 18 ++++
> mm/Kconfig | 15 ++++
> mm/Makefile | 2 +-
> mm/balloon_compaction.c | 172 +++++++++++++++++++++++++++++++++++++
> 5 files changed, 343 insertions(+), 1 deletion(-)
> create mode 100644 include/linux/balloon_compaction.h
> create mode 100644 mm/balloon_compaction.c
>
> diff --git a/include/linux/balloon_compaction.h b/include/linux/balloon_compaction.h
> new file mode 100644
> index 0000000..7afb0ae
> --- /dev/null
> +++ b/include/linux/balloon_compaction.h
> @@ -0,0 +1,137 @@
> +/*
> + * include/linux/balloon_compaction.h
> + *
> + * Common interface definitions for making balloon pages movable to compaction.
> + *
> + * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <aquini@redhat.com>
> + */
> +#ifndef _LINUX_BALLOON_COMPACTION_H
> +#define _LINUX_BALLOON_COMPACTION_H
> +#ifdef __KERNEL__
We don't need ifdef __KERNEL__ - it's only useful
for headers exported to userspace. This one isn't.
> +
> +#include <linux/rcupdate.h>
> +#include <linux/pagemap.h>
> +#include <linux/gfp.h>
> +
> +#ifdef CONFIG_BALLOON_COMPACTION
> +#define count_balloon_event(e) count_vm_event(e)
> +extern bool isolate_balloon_page(struct page *);
> +extern void putback_balloon_page(struct page *);
> +extern int migrate_balloon_page(struct page *newpage,
> + struct page *page, enum migrate_mode mode);
> +
> +static inline gfp_t balloon_mapping_gfp_mask(void)
> +{
> + return GFP_HIGHUSER_MOVABLE;
> +}
> +
> +/*
> + * movable_balloon_page - test page->mapping->flags to identify balloon pages
> + * that can be moved by compaction/migration.
> + *
> + * This function is used at core compaction's page isolation scheme and so it's
> + * exposed to several system pages which may, or may not, be part of a memory
> + * balloon, and thus we cannot afford to hold a page locked to perform tests.
> + *
> + * Therefore, as we might return false positives in the case a balloon page
> + * is just released under us, the page->mapping->flags need to be retested
> + * with the proper page lock held, on the functions that will cope with the
> + * balloon page later.
> + */
> +static inline bool movable_balloon_page(struct page *page)
> +{
> + /*
> + * Before dereferencing and testing mapping->flags, lets make sure
> + * this is not a page that uses ->mapping in a different way
> + */
> + if (!PageSlab(page) && !PageSwapCache(page) &&
> + !PageAnon(page) && !page_mapped(page)) {
> + /*
> + * While doing compaction core work, we cannot afford to hold
> + * page lock as it might cause very undesirable side effects.
> + */
> + struct address_space *mapping;
> + mapping = rcu_dereference_raw(page->mapping);
> + if (mapping)
> + return mapping_balloon(mapping);
> + }
> + return false;
> +}
> +
> +/*
> + * __page_balloon_device - return the balloon device owing the page.
owning?
> + *
> + * This shall only be used at driver callbacks under proper page lock,
> + * to get access to the balloon device structure that owns @page.
> + */
> +static inline void *__page_balloon_device(struct page *page)
> +{
> + struct address_space *mapping;
> + mapping = rcu_dereference_protected(page->mapping, PageLocked(page));
> + if (mapping)
> + mapping = mapping->assoc_mapping;
> + return (void *)mapping;
Never cast pointers to void *.
> +}
> +
> +/*
> + * DEFINE_BALLOON_MAPPING_AOPS - declare and instantiate a callback descriptor
> + * to be used as balloon page->mapping->a_ops.
> + *
> + * @label : declaration identifier (var name)
> + * @isolatepg : callback symbol name for performing the page isolation step
> + * @migratepg : callback symbol name for performing the page migration step
> + * @putbackpg : callback symbol name for performing the page putback step
> + *
> + * address_space_operations utilized methods for ballooned pages:
> + * .migratepage - used to perform balloon's page migration (as is)
> + * .invalidatepage - used to isolate a page from balloon's page list
> + * .freepage - used to reinsert an isolated page to balloon's page list
> + */
> +#define DEFINE_BALLOON_MAPPING_AOPS(label, isolatepg, migratepg, putbackpg) \
> + const struct address_space_operations (label) = { \
> + .migratepage = (migratepg), \
> + .invalidatepage = (isolatepg), \
> + .freepage = (putbackpg), \
> + }
> +
> +#else
> +#define count_balloon_event(e) do { } while (0)
> +static inline bool movable_balloon_page(struct page *page) { return false; }
> +static inline bool isolate_balloon_page(struct page *page) { return false; }
> +static inline void putback_balloon_page(struct page *page) { return; }
> +
> +static inline int migrate_balloon_page(struct page *newpage,
> + struct page *page, enum migrate_mode mode)
> +{
> + return 0;
> +}
> +
> +static inline gfp_t balloon_mapping_gfp_mask(void)
> +{
> + return GFP_HIGHUSER;
> +}
> +
> +#define DEFINE_BALLOON_MAPPING_AOPS(label, migratepg, isolatepg, putbackpg) \
> + const struct address_space_operations *(label) = NULL
Looks like the result can't be passed to alloc_balloon_mapping
anyway. Did you test-build with compaction off?
I would suggest adding free_balloon_mapping
and making both it and alloc_balloon_mapping
empty macros, or in case of alloc_balloon_mapping,
macro returning err_ptr(-EOPNOTSUPP).
This can then become
+ const struct {} (label) = {}
?
> +
> +#endif /* CONFIG_BALLOON_COMPACTION */
> +
> +/* return code to identify when a ballooned page has been migrated */
> +#define BALLOON_MIGRATION_RETURN 0xba1100
> +
> +extern struct address_space *alloc_balloon_mapping(void *balloon_device,
> + const struct address_space_operations *a_ops);
> +
> +static inline void assign_balloon_mapping(struct page *page,
> + struct address_space *mapping)
> +{
> + rcu_assign_pointer(page->mapping, mapping);
> +}
> +
> +static inline void clear_balloon_mapping(struct page *page)
> +{
> + rcu_assign_pointer(page->mapping, NULL);
> +}
> +
> +#endif /* __KERNEL__ */
> +#endif /* _LINUX_BALLOON_COMPACTION_H */
> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
> index e42c762..6df0664 100644
> --- a/include/linux/pagemap.h
> +++ b/include/linux/pagemap.h
> @@ -24,6 +24,7 @@ enum mapping_flags {
> AS_ENOSPC = __GFP_BITS_SHIFT + 1, /* ENOSPC on async write */
> AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT + 2, /* under mm_take_all_locks() */
> AS_UNEVICTABLE = __GFP_BITS_SHIFT + 3, /* e.g., ramdisk, SHM_LOCK */
> + AS_BALLOON_MAP = __GFP_BITS_SHIFT + 4, /* balloon page special map */
> };
>
> static inline void mapping_set_error(struct address_space *mapping, int error)
> @@ -53,6 +54,23 @@ static inline int mapping_unevictable(struct address_space *mapping)
> return !!mapping;
> }
>
> +static inline void mapping_set_balloon(struct address_space *mapping)
> +{
> + set_bit(AS_BALLOON_MAP, &mapping->flags);
> +}
> +
> +static inline void mapping_clear_balloon(struct address_space *mapping)
> +{
> + clear_bit(AS_BALLOON_MAP, &mapping->flags);
> +}
> +
> +static inline int mapping_balloon(struct address_space *mapping)
> +{
> + if (mapping)
> + return test_bit(AS_BALLOON_MAP, &mapping->flags);
> + return !!mapping;
> +}
> +
> static inline gfp_t mapping_gfp_mask(struct address_space * mapping)
> {
> return (__force gfp_t)mapping->flags & __GFP_BITS_MASK;
> diff --git a/mm/Kconfig b/mm/Kconfig
> index d5c8019..0bd783b 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -188,6 +188,21 @@ config SPLIT_PTLOCK_CPUS
> default "4"
>
> #
> +# support for memory balloon compaction
> +config BALLOON_COMPACTION
> + bool "Allow for balloon memory compaction/migration"
> + select COMPACTION
> + depends on VIRTIO_BALLOON
> + help
> + Memory fragmentation introduced by ballooning might reduce
> + significantly the number of 2MB contiguous memory blocks that can be
> + used within a guest, thus imposing performance penalties associated
> + with the reduced number of transparent huge pages that could be used
> + by the guest workload. Allowing the compaction & migration for memory
> + pages enlisted as being part of memory balloon devices avoids the
> + scenario aforementioned and helps improving memory defragmentation.
> +
> +#
> # support for memory compaction
> config COMPACTION
> bool "Allow for memory compaction"
> diff --git a/mm/Makefile b/mm/Makefile
> index 92753e2..78d8caa 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \
> readahead.o swap.o truncate.o vmscan.o shmem.o \
> prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
> mm_init.o mmu_context.o percpu.o slab_common.o \
> - compaction.o $(mmu-y)
> + compaction.o balloon_compaction.o $(mmu-y)
>
> obj-y += init-mm.o
>
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> new file mode 100644
> index 0000000..86a3692
> --- /dev/null
> +++ b/mm/balloon_compaction.c
> @@ -0,0 +1,172 @@
> +/*
> + * mm/balloon_compaction.c
> + *
> + * Common interface for making balloon pages movable to compaction.
> + *
> + * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <aquini@redhat.com>
> + */
> +#include <linux/mm.h>
> +#include <linux/slab.h>
> +#include <linux/export.h>
> +#include <linux/balloon_compaction.h>
> +
> +/*
> + * alloc_balloon_mapping - allocates a special ->mapping for ballooned pages.
> + * @balloon_device: pointer address that references the balloon device which
> + * owns pages bearing this ->mapping.
> + * @a_ops: balloon_mapping address_space_operations descriptor.
> + *
> + * Users must call it to properly allocate and initialize an instance of
> + * struct address_space which will be used as the special page->mapping for
> + * balloon devices enlisted page instances.
> + */
> +struct address_space *alloc_balloon_mapping(void *balloon_device,
> + const struct address_space_operations *a_ops)
> +{
> + struct address_space *mapping;
> +
> + mapping = kmalloc(sizeof(*mapping), GFP_KERNEL);
> + if (!mapping)
> + return NULL;
> +
> + /*
> + * Give a clean 'zeroed' status to all elements of this special
> + * balloon page->mapping struct address_space instance.
> + */
> + address_space_init_once(mapping);
> +
> + /*
> + * Set mapping->flags appropriately, to allow balloon ->mapping
> + * identification, as well as give a proper hint to the balloon
> + * driver on what GFP allocation mask shall be used.
> + */
> + mapping_set_balloon(mapping);
> + mapping_set_gfp_mask(mapping, balloon_mapping_gfp_mask());
> +
> + /* balloon's page->mapping->a_ops callback descriptor */
> + mapping->a_ops = a_ops;
> +
> + /*
> + * balloon special page->mapping overloads ->assoc_mapping
> + * to held a reference back to the balloon device wich 'owns'
> + * a given page. This is the way we can cope with multiple
> + * balloon devices without losing reference of several
> + * ballooned pagesets.
> + */
> + mapping->assoc_mapping = balloon_device;
> +
> + return mapping;
> +}
> +EXPORT_SYMBOL_GPL(alloc_balloon_mapping);
> +
> +#ifdef CONFIG_BALLOON_COMPACTION
> +
> +static inline bool __is_movable_balloon_page(struct page *page)
> +{
> + struct address_space *mapping;
> +
> + mapping = rcu_dereference_protected(page->mapping, PageLocked(page));
> + if (mapping)
> + return mapping_balloon(mapping);
> +
> + return false;
> +}
> +
> +static inline void __isolate_balloon_page(struct page *page)
> +{
> + page->mapping->a_ops->invalidatepage(page, 0);
> +}
> +
> +static inline void __putback_balloon_page(struct page *page)
> +{
> + page->mapping->a_ops->freepage(page);
> +}
> +
> +static inline int __migrate_balloon_page(struct address_space *mapping,
> + struct page *newpage, struct page *page, enum migrate_mode mode)
> +{
> + return page->mapping->a_ops->migratepage(mapping, newpage, page, mode);
> +}
> +
> +/* __isolate_lru_page() counterpart for a ballooned page */
> +bool isolate_balloon_page(struct page *page)
> +{
> + if (likely(get_page_unless_zero(page))) {
> + /*
> + * As balloon pages are not isolated from LRU lists, concurrent
> + * compaction threads can race against page migration functions
> + * move_to_new_page() & __unmap_and_move().
> + * In order to avoid having an already isolated balloon page
> + * being (wrongly) re-isolated while it is under migration,
> + * lets be sure we have the page lock before proceeding with
> + * the balloon page isolation steps.
> + */
> + if (likely(trylock_page(page))) {
> + /*
> + * A ballooned page, by default, has just one refcount.
> + * Prevent concurrent compaction threads from isolating
> + * an already isolated balloon page by refcount check.
> + */
> + if (__is_movable_balloon_page(page) &&
> + (page_count(page) == 2)) {
> + __isolate_balloon_page(page);
> + unlock_page(page);
> + return true;
> + } else if (unlikely(!__is_movable_balloon_page(page))) {
> + dump_page(page);
> + __WARN();
> + }
> + unlock_page(page);
> + }
> + /*
> + * The page is either under migration, or it's isolated already
> + * Drop the refcount taken for it.
> + */
> + put_page(page);
> + }
> + return false;
> +}
> +
> +/* putback_lru_page() counterpart for a ballooned page */
> +void putback_balloon_page(struct page *page)
> +{
> + /*
> + * 'lock_page()' stabilizes the page and prevents races against
> + * concurrent isolation threads attempting to re-isolate it.
> + */
> + lock_page(page);
> +
> + if (__is_movable_balloon_page(page)) {
> + __putback_balloon_page(page);
> + put_page(page);
> + } else {
> + dump_page(page);
> + __WARN();
> + }
> + unlock_page(page);
> +}
> +
> +/* move_to_new_page() counterpart for a ballooned page */
> +int migrate_balloon_page(struct page *newpage,
> + struct page *page, enum migrate_mode mode)
> +{
> + struct address_space *mapping;
> + int rc = -EAGAIN;
> +
> + BUG_ON(!trylock_page(newpage));
> +
> + if (WARN_ON(!__is_movable_balloon_page(page))) {
> + dump_page(page);
> + unlock_page(newpage);
> + return rc;
> + }
> +
> + mapping = rcu_dereference_protected(page->mapping, PageLocked(page));
> + if (mapping)
> + rc = __migrate_balloon_page(mapping, newpage, page, mode);
> +
> + unlock_page(newpage);
> + return rc;
> +}
> +
> +#endif /* CONFIG_BALLOON_COMPACTION */
> --
> 1.7.11.4
^ permalink raw reply
* Re: [PATCH v9 0/5] make balloon pages movable by compaction
From: Michael S. Tsirkin @ 2012-08-26 7:58 UTC (permalink / raw)
To: Rafael Aquini
Cc: Rik van Riel, Konrad Rzeszutek Wilk, linux-kernel, virtualization,
linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim, Andrew Morton,
Paul E. McKenney
In-Reply-To: <cover.1345869378.git.aquini@redhat.com>
On Sat, Aug 25, 2012 at 02:24:55AM -0300, Rafael Aquini wrote:
> Memory fragmentation introduced by ballooning might reduce significantly
> the number of 2MB contiguous memory blocks that can be used within a guest,
> thus imposing performance penalties associated with the reduced number of
> transparent huge pages that could be used by the guest workload.
>
> This patch-set follows the main idea discussed at 2012 LSFMMS session:
> "Ballooning for transparent huge pages" -- http://lwn.net/Articles/490114/
> to introduce the required changes to the virtio_balloon driver, as well as
> the changes to the core compaction & migration bits, in order to make those
> subsystems aware of ballooned pages and allow memory balloon pages become
> movable within a guest, thus avoiding the aforementioned fragmentation issue
Meta-question: are there any numbers showing gain from this patchset?
The reason I ask, on migration we notify host about each page
individually. If this is rare maybe the patchset does not help much.
If this is common we would be better off building up a list of multiple
pages and passing them in one go.
> Rafael Aquini (5):
> mm: introduce a common interface for balloon pages mobility
> mm: introduce compaction and migration for ballooned pages
> virtio_balloon: introduce migration primitives to balloon pages
> mm: introduce putback_movable_pages()
> mm: add vm event counters for balloon pages compaction
>
> drivers/virtio/virtio_balloon.c | 287 ++++++++++++++++++++++++++++++++++---
> include/linux/balloon_compaction.h | 137 ++++++++++++++++++
> include/linux/migrate.h | 2 +
> include/linux/pagemap.h | 18 +++
> include/linux/vm_event_item.h | 8 +-
> mm/Kconfig | 15 ++
> mm/Makefile | 2 +-
> mm/balloon_compaction.c | 174 ++++++++++++++++++++++
> mm/compaction.c | 51 ++++---
> mm/migrate.c | 57 +++++++-
> mm/page_alloc.c | 2 +-
> mm/vmstat.c | 10 +-
> 12 files changed, 715 insertions(+), 48 deletions(-)
> create mode 100644 include/linux/balloon_compaction.h
> create mode 100644 mm/balloon_compaction.c
>
>
> Change log:
> v9:
> * Adjust rcu_dereference usage to leverage page lock protection (Paul, Peter);
> * Enhance doc on compaction interface introduced to balloon driver (Michael);
> * Fix issue with isolated pages breaking leak_balloon() logics (Michael);
> v8:
> * introduce a common MM interface for balloon driver page compaction (Michael);
> * remove the global state preventing multiple balloon device support (Michael);
> * introduce RCU protection/syncrhonization to balloon page->mapping (Michael);
> v7:
> * fix a potential page leak case at 'putback_balloon_page' (Mel);
> * adjust vm-events-counter patch and remove its drop-on-merge message (Rik);
> * add 'putback_movable_pages' to avoid hacks on 'putback_lru_pages' (Minchan);
> v6:
> * rename 'is_balloon_page()' to 'movable_balloon_page()' (Rik);
> v5:
> * address Andrew Morton's review comments on the patch series;
> * address a couple extra nitpick suggestions on PATCH 01 (Minchan);
> v4:
> * address Rusty Russel's review comments on PATCH 02;
> * re-base virtio_balloon patch on 9c378abc5c0c6fc8e3acf5968924d274503819b3;
> V3:
> * address reviewers nitpick suggestions on PATCH 01 (Mel, Minchan);
> V2:
> * address Mel Gorman's review comments on PATCH 01;
>
>
> Preliminary test results:
> (2 VCPU 2048mB RAM KVM guest running 3.6.0_rc3+ -- after a reboot)
>
> * 64mB balloon:
> [root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
> compact_blocks_moved 0
> compact_pages_moved 0
> compact_pagemigrate_failed 0
> compact_stall 0
> compact_fail 0
> compact_success 0
> compact_balloon_isolated 0
> compact_balloon_migrated 0
> compact_balloon_released 0
> compact_balloon_returned 0
> [root@localhost ~]#
> [root@localhost ~]# for i in $(seq 1 6); do echo 1 > /proc/sys/vm/compact_memory & done &>/dev/null
> [1] Done echo 1 > /proc/sys/vm/compact_memory
> [2] Done echo 1 > /proc/sys/vm/compact_memory
> [3] Done echo 1 > /proc/sys/vm/compact_memory
> [4] Done echo 1 > /proc/sys/vm/compact_memory
> [5]- Done echo 1 > /proc/sys/vm/compact_memory
> [6]+ Done echo 1 > /proc/sys/vm/compact_memory
> [root@localhost ~]#
> [root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
> compact_blocks_moved 3108
> compact_pages_moved 43169
> compact_pagemigrate_failed 95
> compact_stall 0
> compact_fail 0
> compact_success 0
> compact_balloon_isolated 16384
> compact_balloon_migrated 16384
> compact_balloon_released 16384
> compact_balloon_returned 0
>
>
> * 128 mB balloon:
> [root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
> compact_blocks_moved 0
> compact_pages_moved 0
> compact_pagemigrate_failed 0
> compact_stall 0
> compact_fail 0
> compact_success 0
> compact_balloon_isolated 0
> compact_balloon_migrated 0
> compact_balloon_released 0
> compact_balloon_returned 0
> [root@localhost ~]#
> [root@localhost ~]# for i in $(seq 1 6); do echo 1 > /proc/sys/vm/compact_memory & done &>/dev/null
> [1] Done echo 1 > /proc/sys/vm/compact_memory
> [2] Done echo 1 > /proc/sys/vm/compact_memory
> [3] Done echo 1 > /proc/sys/vm/compact_memory
> [4] Done echo 1 > /proc/sys/vm/compact_memory
> [5]- Done echo 1 > /proc/sys/vm/compact_memory
> [6]+ Done echo 1 > /proc/sys/vm/compact_memory
> [root@localhost ~]#
> [root@localhost ~]# awk '/compact/ {print}' /proc/vmstat
> compact_blocks_moved 3062
> compact_pages_moved 49774
> compact_pagemigrate_failed 129
> compact_stall 0
> compact_fail 0
> compact_success 0
> compact_balloon_isolated 26076
> compact_balloon_migrated 25957
> compact_balloon_released 25957
> compact_balloon_returned 119
>
> --
> 1.7.11.4
^ permalink raw reply
* CFP: The 5th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) 2012 -- co-located with SC12
From: Ioan Raicu @ 2012-08-26 14:17 UTC (permalink / raw)
To: virtualization
Call for Papers
---------------------------------------------------------------------------------------
The 5th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) 2012
http://datasys.cs.iit.edu/events/MTAGS12/
---------------------------------------------------------------------------------------
November 12th, 2012
Salt Lake City, Utah, USA
Co-located with with IEEE/ACM International Conference for
High Performance Computing, Networking, Storage and Analysis (SC12)
=======================================================================================
The 5th workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) will
provide the scientific community a dedicated forum for presenting new research,
development, and deployment efforts of large-scale many-task computing (MTC)
applications on large scale clusters, Grids, Supercomputers, and Cloud Computing
infrastructure. MTC, the theme of the workshop encompasses loosely coupled applications,
which are generally composed of many tasks (both independent and dependent tasks) to
achieve some larger application goal. This workshop will cover challenges that can
hamper efficiency and utilization in running applications on large-scale systems, such
as local resource manager scalability and granularity, efficient utilization of raw
hardware, parallel file system contention and scalability, data management, I/O
management, reliability at scale, and application scalability. We welcome paper
submissions on all theoretical, simulations, and systems topics related to MTC, but we
give special consideration to papers addressing petascale to exascale challenges.
Papers will be peer-reviewed, and accepted papers will be published in the workshop
proceedings as part of the IEEE digital library (pending approval). The workshop will
be co-located with the IEEE/ACM Supercomputing 2012 Conference in Salt Lake City Utah
on November 12th, 2012.
For more information, please see http://datasys.cs.iit.edu/events/MTAGS12/. For more
information on past workshops, please see MTAGS11, MTAGS10, MTAGS09, and MTAGS08. We also
ran a Special Issue on Many-Task Computing in the IEEE Transactions on Parallel and
Distributed Systems (TPDS) which has appeared in June 2011; the proceedings can be found
online at http://www.computer.org/portal/web/csdl/abs/trans/td/2011/06/ttd201106toc.htm.
We, the workshop organizers, also published a highly relevant paper that defines
Many-Task Computing which was published in MTAGS08, titled "Many-Task Computing for
Grids and Supercomputers"
(http://www.cs.iit.edu/~iraicu/research/publications/2008_MTAGS08_MTC.pdf); we encourage
potential authors to read this paper.
Topics
---------------------------------------------------------------------------------------
We invite the submission of original work that is related to the topics below. The
papers can be either short (4 pages) position papers, or long (8 pages) research
papers. Topics of interest include (in the context of Many-Task Computing):
* Compute Resource Management
* Scheduling
* Job execution frameworks
* Local resource manager extensions
* Performance evaluation of resource managers in use on large scale systems
* Dynamic resource provisioning
* Techniques to manage many-core resources and/or GPUs
* Challenges and opportunities in running many-task workloads on HPC systems
* Challenges and opportunities in running many-task workloads on Cloud Computing
infrastructure
* Storage architectures and implementations
* Distributed file systems
* Parallel file systems
* Distributed meta-data management
* Content distribution systems for large data
* Data caching frameworks and techniques
* Data management within and across data centers
* Data-aware scheduling
* Data-intensive computing applications
* Eventual-consistency storage usage and management
* Programming models and tools
* Map-reduce and its generalizations
* Many-task computing middleware and applications
* Parallel programming frameworks
* Ensemble MPI techniques and frameworks
* Service-oriented science applications
* Large-Scale Workflow Systems
* Workflow system performance and scalability analysis
* Scalability of workflow systems
* Workflow infrastructure and e-Science middleware
* Programming Paradigms and Models
* Large-Scale Many-Task Applications
* High-throughput computing (HTC) applications
* Data-intensive applications
* Quasi-supercomputing applications, deployments, and experiences
* Performance Evaluation
* Performance evaluation
* Real systems
* Simulations
* Reliability of large systems
* How MTC Addresses Challenges of Petascale and Exascale Computing
* Concurrency & Programmability
* I/O & Memory
* Energy
* Resilience
* Heterogeneity
Paper Submission and Publication
---------------------------------------------------------------------------------------
Authors are invited to submit papers with unpublished, original work of not more than 8
pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages,
as per IEEE 8.5 x 11 manuscript guidelines; document templates can be found at
http://www.ieee.org/conferences_events/conferences/publishing/templates.html. We are
also seeking position papers of no more than 4 pages in length. The final 4/8 page
papers in PDF format must be submitted online at
https://cmt.research.microsoft.com/MTAGS2012/ before the deadline. Papers will be
peer-reviewed, and accepted papers will be published in the workshop proceedings as part
of the IEEE digital library (pending approval). Notifications of the paper decisions
will be sent out by October 12th, 2012. Selected excellent work may be eligible for
additional post-conference publication as journal articles or book chapters, such as the
previous Special Issue on Many-Task Computing in the IEEE Transactions on Parallel and
Distributed Systems (TPDS) which has appeared in June 2011. Submission implies the
willingness of at least one of the authors to register and present the paper. For more
information, please http://datasys.cs.iit.edu/events/MTAGS12/, or send email to
mtags12-chairs@datasys.cs.iit.edu.
Important Dates
---------------------------------------------------------------------------------------
* Abstract submission: September 10th, 2012 (11:59PM PST)
* Paper submission: September 17th, 2012 (11:59PM PST)
* Acceptance notification: October 12th, 2012
* Final papers due: November 7th, 2012
Committee Members
---------------------------------------------------------------------------------------
Workshop Chairs (mtags12-chairs@datasys.cs.iit.edu)
* Ioan Raicu, Illinois Institute of Technology & Argonne National Laboratory
* Ian Foster, University of Chicago & Argonne National Laboratory
* Yong Zhao, University of Electronic Science and Technology of China
Steering Committee
* David Abramson, Monash University, Australia
* Jack Dongara, University of Tennessee, USA
* Geoffrey Fox, Indiana University, USA
* Manish Parashar, Rutgers University, USA
* Marc Snir, Argonne National Laboratory & University of Illinois at Urbana Champaign, USA
* Xian-He Sun, Illinois Institute of Technology, USA
* Weimin Zheng, Tsinghua University, China
Publicity Chair (mtags12-publicity@datasys.cs.iit.edu)
* Zhao Zhang, University of Chicago, USA
Program Committee Chair (mtags12-pc-chair@datasys.cs.iit.edu)
* Justin Wozniak, Argonne National Laboratory, USA
Technical Committee
* Roger Barga, Microsoft Research, USA
* Mihai Budiu, Microsoft Research, USA
* Kyle Chard, University of Chicago, USA
* Yong Chen, Texas Tech University, USA
* Evangelinos Constantinos, Massachusetts Institute of Technology, USA
* John Dennis, National Center for Atmospheric Research, USA
* Catalin Dumitrescu, Fermi National Labs, USA
* Dennis Gannon, Microsoft Research, USA
* Indranil Gupta, University of Illinois at Urbana Champaign, USA
* Florin Isaila, Universidad Carlos III de Madrid, Spain
* Kamil Iskra, Argonne National Laboratory, USA
* Alexandru Iosup, Delft University of Technology, Netherlands
* Hui Jin, Oracle Corporation, USA
* Daniel S. Katz, University of Chicago & Argonne National Laboratory, USA
* Carl Kesselman, University of Southern California, USA
* Zhiling Lan, Illinois Institute of Technology, USA
* Mike Lang, Los Alamos National Laboratory, USA
* Gregor von Laszewski, Indiana University, USA
* Reagan Moore, University of North Carolina, Chappel Hill, USA
* Jose Moreira, IBM Research, USA
* Chris Moretti, Princeton University, USA
* David O'Hallaron, Carnegie Mellon University, Intel Labs, USA
* Marlon Pierce, Indiana University, USA
* Judy Qiu, Indiana University, USA
* Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory, USA
* Kui Ren, SUNY Buffalo, USA
* Matei Ripeanu, University of British Columbia, Canada
* Karen Schuchardt, Pacific Northwest National Laboratory, USA
* Wei Tang, Argonne National Laboratory, USA
* Valerie Taylor, Texas A&M, USA
* Douglas Thain University of Notre Dame, USA
* Edward Walker, Whitworth University, USA
* Matthew Woitaszek, Occipital, Inc., USA
* Ken Yocum, University of California, San Diego, USA
* Zhifeng Yun, Louisiana State University, USA
* Zhao Zhang, University of Chicago, USA
* Ziming Zheng, Illinois Institute of Technology, USA
--
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel: 1-847-722-0876
Office: 1-312-567-5704
Email: iraicu@cs.iit.edu
Web: http://www.cs.iit.edu/~iraicu/
Web: http://datasys.cs.iit.edu/
=================================================================
=================================================================
^ permalink raw reply
* Re: [PATCH v9 0/5] make balloon pages movable by compaction
From: Rik van Riel @ 2012-08-26 14:40 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Rafael Aquini, Konrad Rzeszutek Wilk, linux-kernel,
virtualization, linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim,
Andrew Morton, Paul E. McKenney
In-Reply-To: <20120826075840.GE19551@redhat.com>
On 08/26/2012 03:58 AM, Michael S. Tsirkin wrote:
> On Sat, Aug 25, 2012 at 02:24:55AM -0300, Rafael Aquini wrote:
>> Memory fragmentation introduced by ballooning might reduce significantly
>> the number of 2MB contiguous memory blocks that can be used within a guest,
>> thus imposing performance penalties associated with the reduced number of
>> transparent huge pages that could be used by the guest workload.
>>
>> This patch-set follows the main idea discussed at 2012 LSFMMS session:
>> "Ballooning for transparent huge pages" -- http://lwn.net/Articles/490114/
>> to introduce the required changes to the virtio_balloon driver, as well as
>> the changes to the core compaction & migration bits, in order to make those
>> subsystems aware of ballooned pages and allow memory balloon pages become
>> movable within a guest, thus avoiding the aforementioned fragmentation issue
>
> Meta-question: are there any numbers showing gain from this patchset?
>
> The reason I ask, on migration we notify host about each page
> individually. If this is rare maybe the patchset does not help much.
> If this is common we would be better off building up a list of multiple
> pages and passing them in one go.
The gain is in getting a better THP allocation rate inside the
guest, allowing applications to run faster.
The rarer it is for this code to run, the better - it means we
are getting the benefits without the overhead :)
--
All rights reversed
^ permalink raw reply
* Re: [PATCH v9 0/5] make balloon pages movable by compaction
From: Michael S. Tsirkin @ 2012-08-26 15:44 UTC (permalink / raw)
To: Rik van Riel
Cc: Rafael Aquini, Konrad Rzeszutek Wilk, linux-kernel,
virtualization, linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim,
Andrew Morton, Paul E. McKenney
In-Reply-To: <503A3565.2060004@redhat.com>
On Sun, Aug 26, 2012 at 10:40:37AM -0400, Rik van Riel wrote:
> On 08/26/2012 03:58 AM, Michael S. Tsirkin wrote:
> >On Sat, Aug 25, 2012 at 02:24:55AM -0300, Rafael Aquini wrote:
> >>Memory fragmentation introduced by ballooning might reduce significantly
> >>the number of 2MB contiguous memory blocks that can be used within a guest,
> >>thus imposing performance penalties associated with the reduced number of
> >>transparent huge pages that could be used by the guest workload.
> >>
> >>This patch-set follows the main idea discussed at 2012 LSFMMS session:
> >>"Ballooning for transparent huge pages" -- http://lwn.net/Articles/490114/
> >>to introduce the required changes to the virtio_balloon driver, as well as
> >>the changes to the core compaction & migration bits, in order to make those
> >>subsystems aware of ballooned pages and allow memory balloon pages become
> >>movable within a guest, thus avoiding the aforementioned fragmentation issue
> >
> >Meta-question: are there any numbers showing gain from this patchset?
> >
> >The reason I ask, on migration we notify host about each page
> >individually. If this is rare maybe the patchset does not help much.
> >If this is common we would be better off building up a list of multiple
> >pages and passing them in one go.
>
> The gain is in getting a better THP allocation rate inside the
> guest, allowing applications to run faster.
>
> The rarer it is for this code to run, the better - it means we
> are getting the benefits without the overhead :)
I am simply asking how was this patchset tested.
It would be nice to have this info in commit log.
Since this is an optimization patch it is strange
to see one with no numbers at all.
For example, you probably run some workload and
played with the balloon, and then saw less huge pages
without the patch and more with?
Please put this info in the cover letter.
> --
> All rights reversed
^ permalink raw reply
* ping latency using vhost_net, macvtap and virtio
From: Pozsár Balázs @ 2012-08-27 16:09 UTC (permalink / raw)
To: virtualization
Hi all,
I have been testing network throughput and latency and I was wondering
if my measurements are as expected.
For the test, I used Fedora 17 for both host and guest, using kernel
3.5.2-3.fc17.86_64.
Pinging an external server on the LAN from the host, using a gigabit
interface, the results are:
# ping -c 10 172.16.1.1
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_req=1 ttl=64 time=0.109 ms
64 bytes from 172.16.1.1: icmp_req=2 ttl=64 time=0.131 ms
64 bytes from 172.16.1.1: icmp_req=3 ttl=64 time=0.145 ms
64 bytes from 172.16.1.1: icmp_req=4 ttl=64 time=0.116 ms
64 bytes from 172.16.1.1: icmp_req=5 ttl=64 time=0.110 ms
64 bytes from 172.16.1.1: icmp_req=6 ttl=64 time=0.114 ms
64 bytes from 172.16.1.1: icmp_req=7 ttl=64 time=0.112 ms
64 bytes from 172.16.1.1: icmp_req=8 ttl=64 time=0.117 ms
64 bytes from 172.16.1.1: icmp_req=9 ttl=64 time=0.119 ms
64 bytes from 172.16.1.1: icmp_req=10 ttl=64 time=0.128 ms
--- 172.16.1.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 8999ms
rtt min/avg/max/mdev = 0.109/0.120/0.145/0.011 ms
Pinging the same external host on the LAN from the guest, the latency
seems to be much higher:
# ping -c 10 172.16.1.1
PING 172.16.1.1 (172.16.1.1) 56(84) bytes of data.
64 bytes from 172.16.1.1: icmp_req=1 ttl=64 time=0.206 ms
64 bytes from 172.16.1.1: icmp_req=2 ttl=64 time=0.352 ms
64 bytes from 172.16.1.1: icmp_req=3 ttl=64 time=0.518 ms
64 bytes from 172.16.1.1: icmp_req=4 ttl=64 time=0.351 ms
64 bytes from 172.16.1.1: icmp_req=5 ttl=64 time=0.543 ms
64 bytes from 172.16.1.1: icmp_req=6 ttl=64 time=0.387 ms
64 bytes from 172.16.1.1: icmp_req=7 ttl=64 time=0.348 ms
64 bytes from 172.16.1.1: icmp_req=8 ttl=64 time=0.364 ms
64 bytes from 172.16.1.1: icmp_req=9 ttl=64 time=0.345 ms
64 bytes from 172.16.1.1: icmp_req=10 ttl=64 time=0.334 ms
--- 172.16.1.1 ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 8999ms
rtt min/avg/max/mdev = 0.206/0.374/0.543/0.093 ms
(The LAN, the host and guest are idle otherwise during the tests.)
The vhost_net and macvtap modules are loaded on the host, and qemu was
started (by libvirtd) with the -netdev vhost=on option.
Is this expected and normal, or do others see better latencies? Can I
try anything to make it better?
Thanks,
Balazs Pozsar
^ permalink raw reply
* Re: [PATCH v9 3/5] virtio_balloon: introduce migration primitives to balloon pages
From: Rafael Aquini @ 2012-08-27 19:47 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Rik van Riel, Konrad Rzeszutek Wilk, linux-kernel, virtualization,
linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim, Andrew Morton,
Paul E. McKenney
In-Reply-To: <20120826074244.GC19551@redhat.com>
On Sun, Aug 26, 2012 at 10:42:44AM +0300, Michael S. Tsirkin wrote:
>
> Reading two atomics and doing math? Result can even be negative.
> I did not look at use closely but it looks suspicious.
Doc on atomic_read says:
"
The read is atomic in that the return value is guaranteed to be one of the
values initialized or modified with the interface operations if a proper
implicit or explicit memory barrier is used after possible runtime
initialization by any other thread and the value is modified only with the
interface operations.
"
There's no runtime init by other thread than balloon's itself at device register,
and the operations (inc, dec) are made by the proper interface operations
only when protected by the spinlock pages_lock. It does not look suspicious, IMHO.
I'm failing to see how it could become a negative on that case, since you cannot
isolate more pages than what was previoulsy inflated to balloon's list.
> It's already the case everywhere except __wait_on_isolated_pages,
> so just fix that, and then we can keep using int instead of atomics.
>
Sorry, I quite didn't get you here. fix what?
> That's 1K on stack - and can become more if we increase
> VIRTIO_BALLOON_ARRAY_PFNS_MAX. Probably too much - this is the reason
> we use vb->pfns.
>
If we want to use vb->pfns we'll have to make leak_balloon mutual exclusive with
page migration (as it was before), but that will inevictably bring us back to
the discussion on breaking the loop when isolated pages make leak_balloon find
less pages than it wants to release at each leak round.
^ permalink raw reply
* Re: [PATCH v9 0/5] make balloon pages movable by compaction
From: Rafael Aquini @ 2012-08-27 20:22 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Rik van Riel, Konrad Rzeszutek Wilk, linux-kernel, virtualization,
linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim, Andrew Morton,
Paul E. McKenney
In-Reply-To: <20120826154423.GA15478@redhat.com>
On Sun, Aug 26, 2012 at 06:44:23PM +0300, Michael S. Tsirkin wrote:
>
> I am simply asking how was this patchset tested.
> It would be nice to have this info in commit log.
> Since this is an optimization patch it is strange
> to see one with no numbers at all.
> For example, you probably run some workload and
> played with the balloon, and then saw less huge pages
> without the patch and more with?
> Please put this info in the cover letter.
Will do it, for sure. As soon as we get closer to an agreement on how the code
has to behave and looks like. I'll use Mel's mmtests bench suite for that.
Cheers!
^ permalink raw reply
* Re: [PATCH v9 1/5] mm: introduce a common interface for balloon pages mobility
From: Rafael Aquini @ 2012-08-27 20:28 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Rik van Riel, Konrad Rzeszutek Wilk, linux-kernel, virtualization,
linux-mm, Peter Zijlstra, Andi Kleen, Minchan Kim, Andrew Morton,
Paul E. McKenney
In-Reply-To: <20120826075557.GD19551@redhat.com>
On Sun, Aug 26, 2012 at 10:55:58AM +0300, Michael S. Tsirkin wrote:
> On Sat, Aug 25, 2012 at 02:24:56AM -0300, Rafael Aquini wrote:
> > Memory fragmentation introduced by ballooning might reduce significantly
> > the number of 2MB contiguous memory blocks that can be used within a guest,
> > thus imposing performance penalties associated with the reduced number of
> > transparent huge pages that could be used by the guest workload.
> >
> > This patch introduces a common interface to help a balloon driver on
> > making its page set movable to compaction, and thus allowing the system
> > to better leverage the compation efforts on memory defragmentation.
> >
> > Signed-off-by: Rafael Aquini <aquini@redhat.com>
>
> Tons of rcu uses but not sync in sight. This looks suspicious.
There's no critical section marked with rcu_read_lock/rcu_read_unlock. that's
why there's no call for sync anywhere. As we are behaving mostly as updaters,
the hole rcu usage is awkward and it's placed basically to enforce the proper
order. To avoid hurting the RCU API usage with this awk approach I'll drop it
for the next series submission (it will use barriers instead).
^ permalink raw reply
* Reminder: KVM Forum 2012 Call For Participation
From: KVM Forum 2012 Program Committee @ 2012-08-27 23:15 UTC (permalink / raw)
To: kvm, libvir-list, qemu-devel, virtualization; +Cc: kvm-forum-2012-pc
Just a reminder, the CFP ends this Friday.
--
=================================================================
KVM Forum 2012: Call For Participation
November 7-9, 2012 - Hotel Fira Palace - Barcelona, Spain
(All submissions must be received before midnight Aug 31st, 2012)
=================================================================
KVM is an industry leading open source hypervisor that provides
an ideal platform for datacenter virtualization, virtual desktop
infrastructure, and cloud computing. Once again, it's time to bring
together the community of developers and users that define the KVM
ecosystem for our annual technical conference. We will discuss the
current state of affairs and plan for the future of KVM, its surrounding
infrastructure, and management tools. We are also excited to announce
the oVirt Workshop will run in parallel with the KVM Forum, bringing in
a community focused on enterprise datacenter virtualization management
built on KVM. For topics which overlap we will have shared sessions.
So mark your calendar and join us in advancing KVM.
http://events.linuxfoundation.org/events/kvm-forum/
Once again we are colocated with The Linux Foundation's LinuxCon,
Based on feedback from last year, this time it's LinuxCon Europe!
KVM Forum attendees will be able to attend oVirt Workshop sessions and
are eligible to attend LinuxCon Europe for a discounted rate.
http://events.linuxfoundation.org/events/kvm-forum/register
We invite you to lead part of the discussion by submitting a speaking
proposal for KVM Forum 2012.
http://events.linuxfoundation.org/cfp
Suggested topics:
KVM
- Scaling and performance
- Nested virtualization
- I/O improvements
- PCI device assignment
- Driver domains
- Time keeping
- Resource management (cpu, memory, i/o)
- Memory management (page sharing, swapping, huge pages, etc)
- VEPA, VN-Link, vswitch
- Security
- Architecture ports
QEMU
- Device model improvements
- New devices and chipsets
- Scaling and performance
- Desktop virtualization
- Spice
- Increasing robustness and hardening
- Security model
- Management interfaces
- QMP protocol and implementation
- Image formats
- Firmware (SeaBIOS, OVMF, UEFI, etc)
- Live migration
- Live snapshots and merging
- Fault tolerance, high availability, continuous backup
- Real-time guest support
Virtio
- Speeding up existing devices
- Alternatives
- Virtio on non-Linux or non-virtualized
Management infrastructure
- oVirt (shared track w/ oVirt Workshop)
- Libvirt
- KVM autotest
- OpenStack
- Network virtualization management
- Enterprise storage management
Cloud computing
- Scalable storage
- Virtual networking
- Security
- Provisioning
SUBMISSION REQUIREMENTS
Abstracts due: Aug 31st, 2012
Notification: Sep 14th, 2012
Please submit a short abstract (~150 words) describing your presentation
proposal. In your submission please note how long your talk will take.
Slots vary in length up to 45 minutes. Also include in your proposal
the proposal type -- one of:
- technical talk
- end-user talk
- birds of a feather (BOF) session
Submit your proposal here:
http://events.linuxfoundation.org/cfp
You will receive a notification whether or not your presentation proposal
was accepted by Sep 14th.
END-USER COLLABORATION
One of the big challenges as developers is to know what, where and how
people actually use our software. We will reserve a few slots for end
users talking about their deployment challenges and achievements.
If you are using KVM in production you are encouraged submit a speaking
proposal. Simply mark it as an end-user collaboration proposal. As an
end user, this is a unique opportunity to get your input to developers.
BOF SESSION
We will reserve some slots in the evening after the main conference
tracks, for birds of a feather (BOF) sessions. These sessions will be
less formal than presentation tracks and targetted for people who would
like to discuss specific issues with other developers and/or users.
If you are interested in getting developers and/or uses together to
discuss a specific problem, please submit a BOF proposal.
LIGHTNING TALKS
In addition to submitted talks we will also have some room for lightning
talks. These are short (5 minute) discussions to highlight new work or
ideas that aren't complete enough to warrant a full presentation slot.
Lightning talk submissions and scheduling will be handled on-site at
KVM Forum.
HOTEL / TRAVEL
The KVM Forum 2012 will be held in Barcelona, Spain at the Hotel Fira Palace.
http://events.linuxfoundation.org/events/kvm-forum/hotel
Thank you for your interest in KVM. We're looking forward to your
submissions and seeing you at the KVM Forum 2012 in November!
Thanks,
your KVM Forum 2012 Program Commitee
Please contact us with any questions or comments.
KVM-Forum-2012-PC@redhat.com
^ permalink raw reply
* [PATCH 0/5] Multiqueue virtio-scsi
From: Paolo Bonzini @ 2012-08-28 11:54 UTC (permalink / raw)
To: linux-kernel; +Cc: linux-scsi, kvm, mst, virtualization
Hi all,
this series adds multiqueue support to the virtio-scsi driver, based
on Jason Wang's work on virtio-net. It uses a simple queue steering
algorithm that expects one queue per CPU. LUNs in the same target always
use the same queue (so that commands are not reordered); queue switching
occurs when the request being queued is the only one for the target.
Also based on Jason's patches, the virtqueue affinity is set so that
each CPU is associated to one virtqueue.
I tested the patches with fio, using up to 32 virtio-scsi disks backed
by tmpfs on the host, and 1 LUN per target.
FIO configuration
-----------------
[global]
rw=read
bsrange=4k-64k
ioengine=libaio
direct=1
iodepth=4
loops=20
overall bandwidth (MB/s)
-----------------
# of targets single-queue multi-queue, 4 VCPUs multi-queue, 8 VCPUs
1 540 626 599
2 795 965 925
4 997 1376 1500
8 1136 2130 2060
16 1440 2269 2474
24 1408 2179 2436
32 1515 1978 2319
(These numbers for single-queue are with 4 VCPUs, but the impact of adding
more VCPUs is very limited).
avg bandwidth per LUN (MB/s)
---------------------
# of targets single-queue multi-queue, 4 VCPUs multi-queue, 8 VCPUs
1 540 626 599
2 397 482 462
4 249 344 375
8 142 266 257
16 90 141 154
24 58 90 101
32 47 61 72
Testing this may require an irqbalance daemon that is built from git,
due to http://code.google.com/p/irqbalance/issues/detail?id=37.
Alternatively you can just set the affinity manually in /proc.
Rusty, can you please give your Acked-by to the first two patches?
Jason Wang (2):
virtio-ring: move queue_index to vring_virtqueue
virtio: introduce an API to set affinity for a virtqueue
Paolo Bonzini (3):
virtio-scsi: allocate target pointers in a separate memory block
virtio-scsi: pass struct virtio_scsi to virtqueue completion function
virtio-scsi: introduce multiqueue support
drivers/lguest/lguest_device.c | 1 +
drivers/remoteproc/remoteproc_virtio.c | 1 +
drivers/s390/kvm/kvm_virtio.c | 1 +
drivers/scsi/virtio_scsi.c | 200 ++++++++++++++++++++++++--------
drivers/virtio/virtio_mmio.c | 11 +-
drivers/virtio/virtio_pci.c | 58 ++++++++-
drivers/virtio/virtio_ring.c | 17 +++
include/linux/virtio.h | 4 +
include/linux/virtio_config.h | 21 ++++
9 files changed, 253 insertions(+), 61 deletions(-)
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox