[Qemu-devel] [PATCH 0/1] qtest: hex nib buffering

[Qemu-devel] [PATCH 0/1] qtest: hex nib buffering

[John Snow]

This patch requires (as context only) patches 1,2
(but excluding patch 4) from:
Message-id: 1430864578-22072-1-git-send-email-jsnow@redhat.com
[PATCH v3 0/5] qtest: base64 r/w and faster memset

Or, alternatively, use the github mirrors below.

==
For convenience, this branch is available at:
https://github.com/jnsnow/qemu.git branch qtest_batching
https://github.com/jnsnow/qemu/tree/qtest_batching

This version is tagged qtest_batching-v1:
https://github.com/jnsnow/qemu/releases/tag/qtest_batching-v1
==

John Snow (1):
  qtest: pre-buffer hex nibs

 qtest.c          | 11 +++++++----
 tests/libqtest.c |  8 +++++---
 2 files changed, 12 insertions(+), 7 deletions(-)

-- 
2.1.0

[Qemu-devel] [PATCH 1/1] qtest: pre-buffer hex nibs

[John Snow]

Instead of converting each byte one-at-a-time and then sending each byte
over the wire, use sprintf() to pre-compute all of the hex nibs into a
single buffer, then send the entire buffer all at once.

This gives a moderate speed boost to memread() and memwrite() functions.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 qtest.c          | 11 +++++++----
 tests/libqtest.c |  8 +++++---
 2 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/qtest.c b/qtest.c
index c4999c3..d4e931f 100644
--- a/qtest.c
+++ b/qtest.c
@@ -414,6 +414,7 @@ static void qtest_process_command(CharDriverState *chr, gchar **words)
     } else if (strcmp(words[0], "read") == 0) {
         uint64_t addr, len, i;
         uint8_t *data;
+        char *enc;
 
         g_assert(words[1] && words[2]);
         addr = strtoull(words[1], NULL, 0);
@@ -422,14 +423,16 @@ static void qtest_process_command(CharDriverState *chr, gchar **words)
         data = g_malloc(len);
         cpu_physical_memory_read(addr, data, len);
 
-        qtest_send_prefix(chr);
-        qtest_send(chr, "OK 0x");
+        enc = g_malloc(2 * len + 1);
         for (i = 0; i < len; i++) {
-            qtest_sendf(chr, "%02x", data[i]);
+            sprintf(&enc[i * 2], "%02x", data[i]);
         }
-        qtest_send(chr, "\n");
+
+        qtest_send_prefix(chr);
+        qtest_sendf(chr, "OK 0x%s\n", enc);
 
         g_free(data);
+        g_free(enc);
     } else if (strcmp(words[0], "b64read") == 0) {
         uint64_t addr, len;
         uint8_t *data;
diff --git a/tests/libqtest.c b/tests/libqtest.c
index 055aad6..e5188e0 100644
--- a/tests/libqtest.c
+++ b/tests/libqtest.c
@@ -730,13 +730,15 @@ void qtest_memwrite(QTestState *s, uint64_t addr, const void *data, size_t size)
 {
     const uint8_t *ptr = data;
     size_t i;
+    char *enc = g_malloc(2 * size + 1);
 
-    qtest_sendf(s, "write 0x%" PRIx64 " 0x%zx 0x", addr, size);
     for (i = 0; i < size; i++) {
-        qtest_sendf(s, "%02x", ptr[i]);
+        sprintf(&enc[i * 2], "%02x", ptr[i]);
     }
-    qtest_sendf(s, "\n");
+
+    qtest_sendf(s, "write 0x%" PRIx64 " 0x%zx 0x%s\n", addr, size, enc);
     qtest_rsp(s, 0);
+    g_free(enc);
 }
 
 void qtest_memset(QTestState *s, uint64_t addr, uint8_t pattern, size_t size)
-- 
2.1.0

Re: [Qemu-devel] [PATCH 1/1] qtest: pre-buffer hex nibs

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

On 05/07/2015 11:51 AM, John Snow wrote:
> Instead of converting each byte one-at-a-time and then sending each byte
> over the wire, use sprintf() to pre-compute all of the hex nibs into a
> single buffer, then send the entire buffer all at once.
> 
> This gives a moderate speed boost to memread() and memwrite() functions.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---

> -        qtest_send_prefix(chr);
> -        qtest_send(chr, "OK 0x");
> +        enc = g_malloc(2 * len + 1);
>          for (i = 0; i < len; i++) {
> -            qtest_sendf(chr, "%02x", data[i]);
> +            sprintf(&enc[i * 2], "%02x", data[i]);

Making a function call to sprintf() has a lot of overhead.  Isn't it
even faster to open-code the conversion, something like:

for (i = 0; i < len; i++) {
    const char digits[] = "0123456789abcdef";
    enc[i * 2] = digits[data[i] >> 4];
    enc[i * 2 + 1] = digits[data[i] & 0xf];
}
enc[len * 2] = '\0';

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]

Re: [Qemu-devel] [PATCH 1/1] qtest: pre-buffer hex nibs

[John Snow]

On 05/07/2015 04:21 PM, Eric Blake wrote:
> On 05/07/2015 11:51 AM, John Snow wrote:
>> Instead of converting each byte one-at-a-time and then sending
>> each byte over the wire, use sprintf() to pre-compute all of the
>> hex nibs into a single buffer, then send the entire buffer all at
>> once.
>> 
>> This gives a moderate speed boost to memread() and memwrite()
>> functions.
>> 
>> Signed-off-by: John Snow <jsnow@redhat.com> ---
> 
>> -        qtest_send_prefix(chr); -        qtest_send(chr, "OK
>> 0x"); +        enc = g_malloc(2 * len + 1); for (i = 0; i < len;
>> i++) { -            qtest_sendf(chr, "%02x", data[i]); +
>> sprintf(&enc[i * 2], "%02x", data[i]);
> 
> Making a function call to sprintf() has a lot of overhead.  Isn't
> it even faster to open-code the conversion, something like:
> 

"Maybe."

> for (i = 0; i < len; i++) { const char digits[] =
> "0123456789abcdef"; enc[i * 2] = digits[data[i] >> 4]; enc[i * 2 +
> 1] = digits[data[i] & 0xf]; } enc[len * 2] = '\0';
> 

In practice I didn't see a statistical difference between my previous
nib-encoding function on my machine and this sprintf version, so I
opted not to add any new bit twiddling.

The bulk of the savings comes from not engaging the full socket send
for two bytes over and over and over again.

--js

Re: [Qemu-devel] [PATCH 1/1] qtest: pre-buffer hex nibs

[Markus Armbruster]

John Snow <jsnow@redhat.com> writes:

> Instead of converting each byte one-at-a-time and then sending each byte
> over the wire, use sprintf() to pre-compute all of the hex nibs into a
> single buffer, then send the entire buffer all at once.
>
> This gives a moderate speed boost to memread() and memwrite() functions.
>
> Signed-off-by: John Snow <jsnow@redhat.com>

Reviewed-by: Markus Armbruster <armbru@redhat.com>