From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32886) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VOe5H-0008FB-Ja for qemu-devel@nongnu.org; Tue, 24 Sep 2013 21:40:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VOe5B-000484-IU for qemu-devel@nongnu.org; Tue, 24 Sep 2013 21:40:07 -0400 Received: from [222.73.24.84] (port=46350 helo=song.cn.fujitsu.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VOe5A-00047F-Ph for qemu-devel@nongnu.org; Tue, 24 Sep 2013 21:40:01 -0400 Message-ID: <52423C30.7030807@cn.fujitsu.com> Date: Wed, 25 Sep 2013 09:28:16 +0800 From: Wanlong Gao MIME-Version: 1.0 References: <1379387785-14554-1-git-send-email-gaowanlong@cn.fujitsu.com> In-Reply-To: <1379387785-14554-1-git-send-email-gaowanlong@cn.fujitsu.com> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=UTF-8 Subject: Re: [Qemu-devel] [PATCH V13 00/13] Add support for binding guest numa nodes to host numa nodes Reply-To: gaowanlong@cn.fujitsu.com List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: aliguori@us.ibm.com, ehabkost@redhat.com, lersek@redhat.com, peter.huangpeng@huawei.com, lcapitulino@redhat.com, drjones@redhat.com, bsd@redhat.com, hutao@cn.fujitsu.com, y-goto@jp.fujitsu.com, pbonzini@redhat.com, afaerber@suse.de, Wanlong Gao Hi folks, Any comments on this version? Thanks, Wanlong Gao > As you know, QEMU can't direct it's memory allocation now, this may cause > guest cross node access performance regression. > And, the worse thing is that if PCI-passthrough is used, > direct-attached-device uses DMA transfer between device and qemu process. > All pages of the guest will be pinned by get_user_pages(). > > KVM_ASSIGN_PCI_DEVICE ioctl > kvm_vm_ioctl_assign_device() > =>kvm_assign_device() > => kvm_iommu_map_memslots() > => kvm_iommu_map_pages() > => kvm_pin_pages() > > So, with direct-attached-device, all guest page's page count will be +1 and > any page migration will not work. AutoNUMA won't too. > > So, we should set the guest nodes memory allocation policy before > the pages are really mapped. > > According to this patch set, we are able to set guest nodes memory policy > like following: > > -numa node,nodeid=0,cpus=0, \ > -numa mem,size=1024M,policy=membind,host-nodes=0-1 \ > -numa node,nodeid=1,cpus=1 \ > -numa mem,size=1024M,policy=interleave,host-nodes=1 > > This supports "policy={default|membind|interleave|preferred},relative=true,host-nodes=N-N" like format. > > > Also add "set-mem-policy" QMP and hmp command to set memory policy. > > And add a QMP command "query-numa" to show numa info through > this API. > > And convert the "info numa" monitor command to use this > QMP command "query-numa". > > > V1->V2: > change to use QemuOpts in numa options (Paolo) > handle Error in mpol parser (Paolo) > change qmp command format to mem-policy=membind,mem-hostnode=0-1 like (Paolo) > V2->V3: > also handle Error in cpus parser (5/10) > split out common parser from cpus and hostnode parser (Bandan 6/10) > V3-V4: > rebase to request for comments > V4->V5: > use OptVisitor and split -numa option (Paolo) > - s/set-mpol/set-mem-policy (Andreas) > - s/mem-policy/policy > - s/mem-hostnode/host-nodes > fix hmp command process after error (Luiz) > add qmp command query-numa and convert info numa to it (Luiz) > V5->V6: > remove tabs in json file (Laszlo, Paolo) > add back "-numa node,mem=xxx" as legacy (Paolo) > change cpus and host-nodes to array (Laszlo, Eric) > change "nodeid" to "uint16" > add NumaMemPolicy enum type (Eric) > rebased on Laszlo's "OptsVisitor: support / flatten integer ranges for repeating options" patch set, thanks for Laszlo's help > V6-V7: > change UInt16 to uint16 (Laszlo) > fix a typo in adding qmp command set-mem-policy > V7-V8: > rebase to current master with Laszlo's V2 of OptsVisitor patch set > fix an adding white space line error > V8->V9: > rebase to current master > check if total numa memory size is equal to ram_size (Paolo) > add comments to the OptsVisitor stuff in qapi-schema.json (Eric, Laszlo) > replace the use of numa_num_configured_nodes() (Andrew) > avoid abusing the fact i==nodeid (Andrew) > V9->V10: > rebase to current master > remove libnuma (Andrew) > MAX_NODES=64 -> MAX_NODES=128 since libnuma selected 128 (Andrew) > use MAX_NODES instead of MAX_CPUMASK_BITS for host_mem bitmap (Andrew) > remove a useless clear_bit() operation (Andrew) > V10->V11: > rebase to current master > fix "maxnode" argument of mbind(2) > V11->V12: > rebase to current master > split patch 02/11 of V11 (Eduardo) > add some max value check (Eduardo) > split MAX_NODES change patch (Eduardo) > V12->V13: > rebase to current master > thanks for Luiz's review (Luiz) > doc hmp command set-mem-policy (Luiz) > rename: NUMAInfo -> NUMANode (Luiz) > > > *I hope this can catch up the train of 1.7.* > > Thanks, > Wanlong Gao > > Wanlong Gao (13): > NUMA: move numa related code to new file numa.c > NUMA: check if the total numa memory size is equal to ram_size > NUMA: Add numa_info structure to contain numa nodes info > NUMA: convert -numa option to use OptsVisitor > NUMA: introduce NumaMemOptions > NUMA: add "-numa mem," options > NUMA: expand MAX_NODES from 64 to 128 > NUMA: parse guest numa nodes memory policy > NUMA: set guest numa nodes memory policy > NUMA: add qmp command set-mem-policy to set memory policy for NUMA > node > NUMA: add hmp command set-mem-policy > NUMA: add qmp command query-numa > NUMA: convert hmp command info_numa to use qmp command query_numa > > Makefile.target | 2 +- > cpus.c | 14 -- > hmp-commands.hx | 28 +++ > hmp.c | 119 +++++++++++++ > hmp.h | 2 + > hw/i386/pc.c | 4 +- > include/sysemu/cpus.h | 1 - > include/sysemu/sysemu.h | 18 +- > monitor.c | 21 +-- > numa.c | 460 ++++++++++++++++++++++++++++++++++++++++++++++++ > qapi-schema.json | 133 ++++++++++++++ > qemu-options.hx | 6 +- > qmp-commands.hx | 90 ++++++++++ > vl.c | 160 ++--------------- > 14 files changed, 873 insertions(+), 185 deletions(-) > create mode 100644 numa.c >