From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([2001:4830:134:3::10]:35892)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sanidhya.iiith@gmail.com>) id 1XD3I3-0008Ju-3p
	for qemu-devel@nongnu.org; Thu, 31 Jul 2014 23:14:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <sanidhya.iiith@gmail.com>) id 1XD3Hw-0000e2-5x
	for qemu-devel@nongnu.org; Thu, 31 Jul 2014 23:13:55 -0400
Received: from mail-pd0-x22c.google.com ([2607:f8b0:400e:c02::22c]:33355)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <sanidhya.iiith@gmail.com>) id 1XD3Hv-0000dU-QG
	for qemu-devel@nongnu.org; Thu, 31 Jul 2014 23:13:48 -0400
Received: by mail-pd0-f172.google.com with SMTP id ft15so4666269pdb.17
	for <qemu-devel@nongnu.org>; Thu, 31 Jul 2014 20:13:47 -0700 (PDT)
From: Sanidhya Kashyap <sanidhya.iiith@gmail.com>
Date: Fri,  1 Aug 2014 08:42:31 +0530
Message-Id: <1406862751-24008-7-git-send-email-sanidhya.iiith@gmail.com>
In-Reply-To: <1406862751-24008-1-git-send-email-sanidhya.iiith@gmail.com>
References: <1406862751-24008-1-git-send-email-sanidhya.iiith@gmail.com>
Subject: [Qemu-devel] [PATCH v5 6/6] BitmapLog: python script for extracting
	bitmap from a binary file
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: qemu list <qemu-devel@nongnu.org>
Cc: Sanidhya Kashyap <sanidhya.iiith@gmail.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, Juan Quintela <quintela@redhat.com>

The file not only extracts the bitmap from the file but also draws the figure
if required. Currently, figure is drawn for all the bitmaps. Later, I'll make
the change to draw for different blocks.

The picture is drawn by generating a matrix of 0s and 1s from the bitmap. The
dimensions are calculated on the basis of total bitmap pages which is represented
as sqrt(total pages) X (sqrt(total pages) + 1). The white parts indicate non dirtied
region while the black - dirtied region.

The python code requires some libraries such as numpy, pylab and math to generate
the images.

Signed-off-by: Sanidhya Kashyap <sanidhya.iiith@gmail.com>
---
 scripts/extract-bitmap.py | 144 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)
 create mode 100755 scripts/extract-bitmap.py

diff --git a/scripts/extract-bitmap.py b/scripts/extract-bitmap.py
new file mode 100755
index 0000000..942deca
--- /dev/null
+++ b/scripts/extract-bitmap.py
@@ -0,0 +1,144 @@
+#!/usr/bin/python
+# This python script helps in extracting the dirty bitmap present
+# in the file after executing the log-dirty-bitmap command either
+# from the qmp or hmp interface. This file only processes binary
+# file obtained via command.
+#
+# Copyright (C) 2014 Sanidhya Kashyap <sanidhya.iiith@gmail.com>
+#
+# Authors:
+#       Sanidhya Kashyap
+#
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+
+import struct
+import argparse
+from functools import partial
+from math import sqrt
+from numpy import array
+from pylab import figure,imshow,show,gray
+
+long_bytes = 8
+byte_size = 8
+int_bytes = 4
+complete_bitmap_list = []
+block_list = []
+
+def get_unsigned_long_integer(value):
+	return struct.unpack('<Q', value)[0]
+
+def get_long_integer(value):
+	return struct.unpack('<q', value)[0]
+
+def get_integer(value):
+	return struct.unpack('<i', value)[0]
+
+def get_char(value):
+	return struct.unpack('<c', value)[0]
+
+def get_string(value, length):
+	name = struct.unpack('<'+str(length)+'s', value)[0]
+	for i in range(len(name)):
+		if name[i] == '\x00':
+			return name[:i]
+
+def dec2bin(decimal):
+    bin_value = bin(decimal)[2:]
+    if len(bin_value) < long_bytes * byte_size:
+        add_zeroes = long_bytes * byte_size - len(bin_value)
+        for i in range(add_zeroes):
+            bin_value += "0"
+    return str(bin_value)
+
+def get_bitmap_length(ram_bitmap_pages):
+    bitmap_length = ram_bitmap_pages / (long_bytes * byte_size)
+    if ram_bitmap_pages % (long_bytes * byte_size) != 0:
+        bitmap_length += 1
+    return bitmap_length
+
+def dump_ram_block_info(infile):
+    total_blocks = get_integer(infile.read(int_bytes))
+    for i in range(total_blocks):
+        block_name_length = get_integer(infile.read(int_bytes))
+        block_name = get_string(infile.read(block_name_length), block_name_length)
+        block_offset = get_unsigned_long_integer(infile.read(long_bytes))
+        block_length = get_unsigned_long_integer(infile.read(long_bytes))
+        block_list.append(dict(name=block_name, offset=block_offset, length=block_length))
+
+def generate_images():
+    r = 0
+    for list in complete_bitmap_list:
+        all_digits = ""
+        for element in list:
+            all_digits += dec2bin(element)
+        l = len(all_digits)
+        sqrtvalue = int(sqrt(l))
+        for i in range(sqrtvalue * (sqrtvalue+1)-l):
+            all_digits+="0"
+
+        v = []
+        l = len(all_digits)
+        for i in range(sqrtvalue+1):
+            v1 = []
+            for j in range(sqrtvalue):
+                v1.append(int(all_digits[i*sqrtvalue+j]))
+            v.append(v1)
+
+        im_array = array(v)
+        figure(r)
+        imshow(im_array, cmap=gray())
+        r += 1
+    show()
+
+def dump_bitmap(infile, draw):
+    marker = 'M'
+    count = 0
+    value = ' '
+    current_ram_bitmap_pages = 0
+    prev_ram_bitmap_pages = 0
+    while True:
+        if len(value) == 0  or marker != 'M':
+            print "issue with the dump"
+            return
+        bitmap_page_raw_value = infile.read(long_bytes)
+        if not bitmap_page_raw_value:
+            break
+        current_ram_bitmap_pages = get_long_integer(bitmap_page_raw_value)
+        if current_ram_bitmap_pages != prev_ram_bitmap_pages:
+            prev_ram_bitmap_pages = current_ram_bitmap_pages
+            dump_ram_block_info(infile)
+
+        bitmap_length = get_bitmap_length(current_ram_bitmap_pages)
+        bitmap_list = []
+        bitmap_raw_value = infile.read(long_bytes * bitmap_length)
+        if not bitmap_raw_value:
+            break
+        count+=1
+        for i in range(bitmap_length):
+            mark = i * long_bytes
+            bitmap_list.append((get_unsigned_long_integer(bitmap_raw_value[mark:mark+long_bytes])))
+        complete_bitmap_list.append(bitmap_list)
+        value = infile.read(1)
+        marker = get_char(value)
+    if draw is True:
+        generate_images()
+    else:
+        print complete_bitmap_list
+
+def main():
+    extracter = argparse.ArgumentParser(description='Extract dirty bitmap from binary file.')
+    extracter.add_argument('infile', help='Input file to extract the bitmap')
+    extracter.add_argument('-d', action='store_true', dest='draw', default=False,
+            help='Draw a black and white image of the processed dirty bitmap')
+    args = extracter.parse_args()
+    print 'The filename is {}'.format(args.infile)
+
+    infile = open(format(args.infile), 'rb')
+
+    dump_bitmap(infile, args.draw);
+
+    infile.close()
+
+if __name__ == '__main__':
+    main()
-- 
1.9.3