[Qemu-devel] [RFC] scripts: qmp: Introduce vcpu pinning helper script

[Qemu-devel] [RFC] scripts: qmp: Introduce vcpu pinning helper script

[Maxime Coquelin]

This python script calls 'query-cpus' QMP command to retrieve
vCPUs thread IDs.
Thread IDs are then used by taskset to pin vCPUs to physical
CPUs passed in command line.

In case more vCPUs are present than the number of CPUs assigned
in command line, multiple vCPUs get pinned to physical CPUs.

Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
---

This script is not for production, where libvirt should be used.
However, I find it useful during development, so I propose to share
it as it might interest other people.

Note: it seems that tcp socket support is broken with QMP class.
The bug has already been reported and patch proposed[0].
So it has only been tested with UNIX socket.

[0]: <b9d0e4270773564788dc15a2db8e407119648956.1471108930.git.carl@resonantcircuitlabs.com>

 scripts/qmp/qmp-vcpu-pin | 39 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
 create mode 100755 scripts/qmp/qmp-vcpu-pin

diff --git a/scripts/qmp/qmp-vcpu-pin b/scripts/qmp/qmp-vcpu-pin
new file mode 100755
index 0000000..7bf96f7
--- /dev/null
+++ b/scripts/qmp/qmp-vcpu-pin
@@ -0,0 +1,39 @@
+#!/usr/bin/python
+# QEMU vCPU pinning tool
+#
+# Copyright (C) 2016 Red Hat Inc.
+#
+# Authors:
+#  Maxime Coquelin <maxime.coquelin@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2.  See
+# the COPYING file in the top-level directory
+import argparse
+import json
+import os
+
+from subprocess import call
+from qmp import QEMUMonitorProtocol
+
+parser = argparse.ArgumentParser(description='Pin QEMU vCPUs to physical CPUs')
+parser.add_argument('-s', '--server', type=str, required=True,
+                    help='QMP server path or address:port')
+parser.add_argument('cpu', type=int, nargs='+',
+                    help='Physical CPUs IDs')
+args = parser.parse_args()
+
+devnull = open(os.devnull, 'w')
+
+srv = QEMUMonitorProtocol(args.server)
+srv.connect()
+
+for vcpu in srv.command('query-cpus'):
+    vcpuid = vcpu['CPU']
+    tid = vcpu['thread_id']
+    cpuid = args.cpu[vcpuid % len(args.cpu)]
+    print 'Pin vCPU {} (tid {}) to physical CPU {}'.format(vcpuid, tid, cpuid)
+    try:
+        call(['taskset', '-pc', str(cpuid), str(tid)], stdout=devnull)
+    except OSError:
+        print 'Failed to pin vCPU{} to CPU{}'.format(vcpuid, cpuid)
+
-- 
2.7.4

Re: [Qemu-devel] [RFC] scripts: qmp: Introduce vcpu pinning helper script

[Peter Xu]

On Tue, Sep 20, 2016 at 09:04:56AM +0200, Maxime Coquelin wrote:
> This python script calls 'query-cpus' QMP command to retrieve
> vCPUs thread IDs.
> Thread IDs are then used by taskset to pin vCPUs to physical
> CPUs passed in command line.
> 
> In case more vCPUs are present than the number of CPUs assigned
> in command line, multiple vCPUs get pinned to physical CPUs.
> 
> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> ---
> 
> This script is not for production, where libvirt should be used.
> However, I find it useful during development, so I propose to share
> it as it might interest other people.
> 
> Note: it seems that tcp socket support is broken with QMP class.
> The bug has already been reported and patch proposed[0].
> So it has only been tested with UNIX socket.
> 
> [0]: <b9d0e4270773564788dc15a2db8e407119648956.1471108930.git.carl@resonantcircuitlabs.com>
> 
>  scripts/qmp/qmp-vcpu-pin | 39 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 39 insertions(+)
>  create mode 100755 scripts/qmp/qmp-vcpu-pin
> 
> diff --git a/scripts/qmp/qmp-vcpu-pin b/scripts/qmp/qmp-vcpu-pin
> new file mode 100755
> index 0000000..7bf96f7
> --- /dev/null
> +++ b/scripts/qmp/qmp-vcpu-pin
> @@ -0,0 +1,39 @@
> +#!/usr/bin/python
> +# QEMU vCPU pinning tool
> +#
> +# Copyright (C) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Maxime Coquelin <maxime.coquelin@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2.  See
> +# the COPYING file in the top-level directory
> +import argparse
> +import json
> +import os
> +
> +from subprocess import call
> +from qmp import QEMUMonitorProtocol
> +
> +parser = argparse.ArgumentParser(description='Pin QEMU vCPUs to physical CPUs')
> +parser.add_argument('-s', '--server', type=str, required=True,
> +                    help='QMP server path or address:port')
> +parser.add_argument('cpu', type=int, nargs='+',
> +                    help='Physical CPUs IDs')
> +args = parser.parse_args()
> +
> +devnull = open(os.devnull, 'w')
> +
> +srv = QEMUMonitorProtocol(args.server)
> +srv.connect()
> +
> +for vcpu in srv.command('query-cpus'):
> +    vcpuid = vcpu['CPU']
> +    tid = vcpu['thread_id']
> +    cpuid = args.cpu[vcpuid % len(args.cpu)]
> +    print 'Pin vCPU {} (tid {}) to physical CPU {}'.format(vcpuid, tid, cpuid)
> +    try:
> +        call(['taskset', '-pc', str(cpuid), str(tid)], stdout=devnull)
> +    except OSError:
> +        print 'Failed to pin vCPU{} to CPU{}'.format(vcpuid, cpuid)
> +
> -- 
> 2.7.4

Reviewed-by: Peter Xu <peterx@redhat.com>
Tested-by: Peter Xu <peterx@redhat.com>

Nit: for TCG, looks like we are using a single thread for all vCPUs.
In that case, I'd suggest we make sure KVM is enabled before running
this script, otherwise raise error (It won't make sense we pin the
same TID for lots of times). I believe that's a prerequisite for this
one.

Thanks,

-- peterx

Re: [Qemu-devel] [RFC] scripts: qmp: Introduce vcpu pinning helper script

[Maxime Coquelin]

On 09/29/2016 04:40 AM, Peter Xu wrote:
> On Tue, Sep 20, 2016 at 09:04:56AM +0200, Maxime Coquelin wrote:
>> This python script calls 'query-cpus' QMP command to retrieve
>> vCPUs thread IDs.
>> Thread IDs are then used by taskset to pin vCPUs to physical
>> CPUs passed in command line.
>>
>> In case more vCPUs are present than the number of CPUs assigned
>> in command line, multiple vCPUs get pinned to physical CPUs.
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
>> ---
>>
>> This script is not for production, where libvirt should be used.
>> However, I find it useful during development, so I propose to share
>> it as it might interest other people.
>>
>> Note: it seems that tcp socket support is broken with QMP class.
>> The bug has already been reported and patch proposed[0].
>> So it has only been tested with UNIX socket.
>>
>> [0]: <b9d0e4270773564788dc15a2db8e407119648956.1471108930.git.carl@resonantcircuitlabs.com>
>>
>>  scripts/qmp/qmp-vcpu-pin | 39 +++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 39 insertions(+)
>>  create mode 100755 scripts/qmp/qmp-vcpu-pin
>>
>> diff --git a/scripts/qmp/qmp-vcpu-pin b/scripts/qmp/qmp-vcpu-pin
>> new file mode 100755
>> index 0000000..7bf96f7
>> --- /dev/null
>> +++ b/scripts/qmp/qmp-vcpu-pin
>> @@ -0,0 +1,39 @@
>> +#!/usr/bin/python
>> +# QEMU vCPU pinning tool
>> +#
>> +# Copyright (C) 2016 Red Hat Inc.
>> +#
>> +# Authors:
>> +#  Maxime Coquelin <maxime.coquelin@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2.  See
>> +# the COPYING file in the top-level directory
>> +import argparse
>> +import json
>> +import os
>> +
>> +from subprocess import call
>> +from qmp import QEMUMonitorProtocol
>> +
>> +parser = argparse.ArgumentParser(description='Pin QEMU vCPUs to physical CPUs')
>> +parser.add_argument('-s', '--server', type=str, required=True,
>> +                    help='QMP server path or address:port')
>> +parser.add_argument('cpu', type=int, nargs='+',
>> +                    help='Physical CPUs IDs')
>> +args = parser.parse_args()
>> +
>> +devnull = open(os.devnull, 'w')
>> +
>> +srv = QEMUMonitorProtocol(args.server)
>> +srv.connect()
>> +
>> +for vcpu in srv.command('query-cpus'):
>> +    vcpuid = vcpu['CPU']
>> +    tid = vcpu['thread_id']
>> +    cpuid = args.cpu[vcpuid % len(args.cpu)]
>> +    print 'Pin vCPU {} (tid {}) to physical CPU {}'.format(vcpuid, tid, cpuid)
>> +    try:
>> +        call(['taskset', '-pc', str(cpuid), str(tid)], stdout=devnull)
>> +    except OSError:
>> +        print 'Failed to pin vCPU{} to CPU{}'.format(vcpuid, cpuid)
>> +
>> --
>> 2.7.4
>
> Reviewed-by: Peter Xu <peterx@redhat.com>
> Tested-by: Peter Xu <peterx@redhat.com>
Thanks Peter.

> Nit: for TCG, looks like we are using a single thread for all vCPUs.
> In that case, I'd suggest we make sure KVM is enabled before running
> this script, otherwise raise error (It won't make sense we pin the
> same TID for lots of times). I believe that's a prerequisite for this
> one.

In the script, we could check that vCPUs TIDs are different.
Checking for TCG or KVM is not a good solution, as IIUC, multi-threaded
TCG support is in the pipe?

Regards,
Maxime

> Thanks,
>
> -- peterx
>

Re: [Qemu-devel] [RFC] scripts: qmp: Introduce vcpu pinning helper script

[Peter Xu]

On Thu, Sep 29, 2016 at 08:50:51AM +0200, Maxime Coquelin wrote:

[...]

> >Nit: for TCG, looks like we are using a single thread for all vCPUs.
> >In that case, I'd suggest we make sure KVM is enabled before running
> >this script, otherwise raise error (It won't make sense we pin the
> >same TID for lots of times). I believe that's a prerequisite for this
> >one.
> 
> In the script, we could check that vCPUs TIDs are different.
> Checking for TCG or KVM is not a good solution, as IIUC, multi-threaded
> TCG support is in the pipe?

Sounds good. :)

-- peterx