qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata
@ 2013-04-18 15:17 Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 1/5] qcow2.py: rename class to Qcow Paolo Bonzini
                   ` (5 more replies)
  0 siblings, 6 replies; 7+ messages in thread
From: Paolo Bonzini @ 2013-04-18 15:17 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, stefanha

This series includes several improvements to qcow2.py, mostly
the ability to dump the L1 and L2 tables for a full backing
chain.

Doing this completely would require support for all image formats,
not just qcow2.  To cover the common cases, I included support
for raw backing files.

The right thing to do perhaps would be to add the read-metadata
patches from the in-place QED conversion summer of code project,
but I wanted to throw out these anyway if anyone has comments.

Paolo

Paolo Bonzini (5):
  qcow2.py: rename class to Qcow
  qcow2.py: add ImageFile superclass
  qcow2.py: add load_image() method
  qcow2.py: add method to load backing image
  qcow2.py: add dump-map command

 tests/qemu-iotests/qcow2.py | 258 +++++++++++++++++++++++++++++++++++++-------
 1 file changed, 217 insertions(+), 41 deletions(-)

-- 
1.8.2

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [Qemu-devel] [RFC PATCH 1/5] qcow2.py: rename class to Qcow
  2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
@ 2013-04-18 15:17 ` Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 2/5] qcow2.py: add ImageFile superclass Paolo Bonzini
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Paolo Bonzini @ 2013-04-18 15:17 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, stefanha

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qemu-iotests/qcow2.py | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py
index fecf5b9..40241b1 100755
--- a/tests/qemu-iotests/qcow2.py
+++ b/tests/qemu-iotests/qcow2.py
@@ -15,7 +15,7 @@ class QcowHeaderExtension:
     def create(cls, magic, data):
         return QcowHeaderExtension(magic, len(data), data)
 
-class QcowHeader:
+class Qcow:
 
     uint32_t = 'I'
     uint64_t = 'Q'
@@ -44,18 +44,18 @@ class QcowHeader:
         [ uint32_t, '%d',   'header_length' ],
     ];
 
-    fmt = '>' + ''.join(field[0] for field in fields)
+    HEADER_FMT = '>' + ''.join(field[0] for field in fields)
 
     def __init__(self, fd):
 
-        buf_size = struct.calcsize(QcowHeader.fmt)
+        buf_size = struct.calcsize(Qcow.HEADER_FMT)
 
         fd.seek(0)
         buf = fd.read(buf_size)
 
-        header = struct.unpack(QcowHeader.fmt, buf)
+        header = struct.unpack(Qcow.HEADER_FMT, buf)
         self.__dict__ = dict((field[2], header[i])
-            for i, field in enumerate(QcowHeader.fields))
+            for i, field in enumerate(Qcow.fields))
 
         self.set_defaults()
         self.cluster_size = 1 << self.cluster_bits
@@ -118,13 +118,13 @@ class QcowHeader:
         self.update_extensions(fd)
 
         fd.seek(0)
-        header = tuple(self.__dict__[f] for t, p, f in QcowHeader.fields)
-        buf = struct.pack(QcowHeader.fmt, *header)
+        header = tuple(self.__dict__[f] for t, p, f in Qcow.fields)
+        buf = struct.pack(Qcow.HEADER_FMT, *header)
         buf = buf[0:header_bytes-1]
         fd.write(buf)
 
     def dump(self):
-        for f in QcowHeader.fields:
+        for f in Qcow.fields:
             print "%-25s" % f[2], f[1] % self.__dict__[f[2]]
         print ""
 
@@ -145,7 +145,7 @@ class QcowHeader:
 
 
 def cmd_dump_header(fd):
-    h = QcowHeader(fd)
+    h = Qcow(fd)
     h.dump()
     h.dump_extensions()
 
@@ -156,7 +156,7 @@ def cmd_add_header_ext(fd, magic, data):
         print "'%s' is not a valid magic number" % magic
         sys.exit(1)
 
-    h = QcowHeader(fd)
+    h = Qcow(fd)
     h.extensions.append(QcowHeaderExtension.create(magic, data))
     h.update(fd)
 
@@ -167,7 +167,7 @@ def cmd_del_header_ext(fd, magic):
         print "'%s' is not a valid magic number" % magic
         sys.exit(1)
 
-    h = QcowHeader(fd)
+    h = Qcow(fd)
     found = False
 
     for ex in h.extensions:
@@ -190,7 +190,7 @@ def cmd_set_feature_bit(fd, group, bit):
         print "'%s' is not a valid bit number in range [0, 64)" % bit
         sys.exit(1)
 
-    h = QcowHeader(fd)
+    h = Qcow(fd)
     if group == 'incompatible':
         h.incompatible_features |= 1 << bit
     elif group == 'compatible':
-- 
1.8.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [Qemu-devel] [RFC PATCH 2/5] qcow2.py: add ImageFile superclass
  2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 1/5] qcow2.py: rename class to Qcow Paolo Bonzini
@ 2013-04-18 15:17 ` Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 3/5] qcow2.py: add load_image() method Paolo Bonzini
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Paolo Bonzini @ 2013-04-18 15:17 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, stefanha

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qemu-iotests/qcow2.py | 60 +++++++++++++++++++++++++++------------------
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py
index 40241b1..773c947 100755
--- a/tests/qemu-iotests/qcow2.py
+++ b/tests/qemu-iotests/qcow2.py
@@ -15,7 +15,20 @@ class QcowHeaderExtension:
     def create(cls, magic, data):
         return QcowHeaderExtension(magic, len(data), data)
 
-class Qcow:
+class ImageFile:
+    def __init__(self, fd, fields):
+        self.__dict__ = fields
+        self.fd = fd
+
+    def raw_pread(self, offset, size):
+        self.fd.seek(offset)
+        return self.fd.read(size)
+
+    def raw_pwrite(self, offset, data):
+        self.fd.seek(offset)
+        return self.fd.write(data)
+
+class Qcow(ImageFile):
 
     uint32_t = 'I'
     uint64_t = 'Q'
@@ -54,18 +67,19 @@ class Qcow:
         buf = fd.read(buf_size)
 
         header = struct.unpack(Qcow.HEADER_FMT, buf)
-        self.__dict__ = dict((field[2], header[i])
+        fields = dict((field[2], header[i])
             for i, field in enumerate(Qcow.fields))
 
+        ImageFile.__init__(self, fd, fields)
         self.set_defaults()
         self.cluster_size = 1 << self.cluster_bits
 
         fd.seek(self.header_length)
-        self.load_extensions(fd)
+        self.load_extensions()
 
         if self.backing_file_offset:
-            fd.seek(self.backing_file_offset)
-            self.backing_file = fd.read(self.backing_file_size)
+            self.backing_file = self.raw_pread(self.backing_file_offset,
+                                               self.backing_file_size)
         else:
             self.backing_file = None
 
@@ -77,7 +91,7 @@ class Qcow:
             self.refcount_order = 4
             self.header_length = 72
 
-    def load_extensions(self, fd):
+    def load_extensions(self):
         self.extensions = []
 
         if self.backing_file_offset != 0:
@@ -85,43 +99,41 @@ class Qcow:
         else:
             end = self.cluster_size
 
-        while fd.tell() < end:
-            (magic, length) = struct.unpack('>II', fd.read(8))
+        while self.fd.tell() < end:
+            (magic, length) = struct.unpack('>II', self.fd.read(8))
             if magic == 0:
                 break
             else:
                 padded = (length + 7) & ~7
-                data = fd.read(padded)
+                data = self.fd.read(padded)
                 self.extensions.append(QcowHeaderExtension(magic, length, data))
 
-    def update_extensions(self, fd):
+    def update_extensions(self):
 
-        fd.seek(self.header_length)
+        self.fd.seek(self.header_length)
         extensions = self.extensions
         extensions.append(QcowHeaderExtension(0, 0, ""))
         for ex in extensions:
             buf = struct.pack('>II', ex.magic, ex.length)
-            fd.write(buf)
-            fd.write(ex.data)
+            self.fd.write(buf)
+            self.fd.write(ex.data)
 
         if self.backing_file != None:
-            self.backing_file_offset = fd.tell()
-            fd.write(self.backing_file)
+            self.backing_file_offset = self.fd.tell()
+            self.fd.write(self.backing_file)
 
-        if fd.tell() > self.cluster_size:
+        if self.fd.tell() > self.cluster_size:
             raise Exception("I think I just broke the image...")
 
-
-    def update(self, fd):
+    def update(self):
         header_bytes = self.header_length
 
-        self.update_extensions(fd)
+        self.update_extensions()
 
-        fd.seek(0)
         header = tuple(self.__dict__[f] for t, p, f in Qcow.fields)
         buf = struct.pack(Qcow.HEADER_FMT, *header)
         buf = buf[0:header_bytes-1]
-        fd.write(buf)
+        self.raw_pwrite(0, buf)
 
     def dump(self):
         for f in Qcow.fields:
@@ -158,7 +170,7 @@ def cmd_add_header_ext(fd, magic, data):
 
     h = Qcow(fd)
     h.extensions.append(QcowHeaderExtension.create(magic, data))
-    h.update(fd)
+    h.update()
 
 def cmd_del_header_ext(fd, magic):
     try:
@@ -179,7 +191,7 @@ def cmd_del_header_ext(fd, magic):
         print "No such header extension"
         return
 
-    h.update(fd)
+    h.update()
 
 def cmd_set_feature_bit(fd, group, bit):
     try:
@@ -201,7 +213,7 @@ def cmd_set_feature_bit(fd, group, bit):
         print "'%s' is not a valid group, try 'incompatible', 'compatible', or 'autoclear'" % group
         sys.exit(1)
 
-    h.update(fd)
+    h.update()
 
 cmds = [
     [ 'dump-header',    cmd_dump_header,    0, 'Dump image header and header extensions' ],
-- 
1.8.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [Qemu-devel] [RFC PATCH 3/5] qcow2.py: add load_image() method
  2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 1/5] qcow2.py: rename class to Qcow Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 2/5] qcow2.py: add ImageFile superclass Paolo Bonzini
@ 2013-04-18 15:17 ` Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 4/5] qcow2.py: add method to load backing image Paolo Bonzini
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 7+ messages in thread
From: Paolo Bonzini @ 2013-04-18 15:17 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, stefanha

The load_image() method optionally probes for the image format
and returns an instance of a subclass of ImageFile.  So far
only qcow2 is supported.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qemu-iotests/qcow2.py | 44 +++++++++++++++++++++++++++++++++-----------
 1 file changed, 33 insertions(+), 11 deletions(-)

diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py
index 773c947..77e03cb 100755
--- a/tests/qemu-iotests/qcow2.py
+++ b/tests/qemu-iotests/qcow2.py
@@ -1,6 +1,7 @@
 #!/usr/bin/env python
 
 import sys
+import os
 import struct
 import string
 
@@ -20,6 +21,10 @@ class ImageFile:
         self.__dict__ = fields
         self.fd = fd
 
+    def close(self):
+        self.fd.close()
+        self.fd = None
+
     def raw_pread(self, offset, size):
         self.fd.seek(offset)
         return self.fd.read(size)
@@ -28,6 +33,7 @@ class ImageFile:
         self.fd.seek(offset)
         return self.fd.write(data)
 
+
 class Qcow(ImageFile):
 
     uint32_t = 'I'
@@ -58,6 +64,7 @@ class Qcow(ImageFile):
     ];
 
     HEADER_FMT = '>' + ''.join(field[0] for field in fields)
+    QCOW_MAGIC = 0x514649fb
 
     def __init__(self, fd):
 
@@ -71,6 +78,9 @@ class Qcow(ImageFile):
             for i, field in enumerate(Qcow.fields))
 
         ImageFile.__init__(self, fd, fields)
+        if self.magic != Qcow.QCOW_MAGIC:
+            raise Error('not a QCOW2 image')
+
         self.set_defaults()
         self.cluster_size = 1 << self.cluster_bits
 
@@ -155,31 +165,44 @@ class Qcow(ImageFile):
             print "%-25s %s" % ("data", data)
             print ""
 
+def load_image(filename, mode='rb', format=None):
+    fd = open(filename, mode)
+    if format is None:
+        fd.seek(0)
+        buf = fd.read(4)
+        magic = struct.unpack('>I', buf)
+        if magic == Qcow.QCOW_MAGIC:
+            format = 'qcow2'
+        else:
+            format = 'raw'
+
+    if format == 'qcow2':
+        return Qcow(fd)
+
+    raise Error('unknown format %s' % format)
+
 
-def cmd_dump_header(fd):
-    h = Qcow(fd)
+def cmd_dump_header(h):
     h.dump()
     h.dump_extensions()
 
-def cmd_add_header_ext(fd, magic, data):
+def cmd_add_header_ext(h, magic, data):
     try:
         magic = int(magic, 0)
     except:
         print "'%s' is not a valid magic number" % magic
         sys.exit(1)
 
-    h = Qcow(fd)
     h.extensions.append(QcowHeaderExtension.create(magic, data))
     h.update()
 
-def cmd_del_header_ext(fd, magic):
+def cmd_del_header_ext(h, magic):
     try:
         magic = int(magic, 0)
     except:
         print "'%s' is not a valid magic number" % magic
         sys.exit(1)
 
-    h = Qcow(fd)
     found = False
 
     for ex in h.extensions:
@@ -193,7 +216,7 @@ def cmd_del_header_ext(fd, magic):
 
     h.update()
 
-def cmd_set_feature_bit(fd, group, bit):
+def cmd_set_feature_bit(h, group, bit):
     try:
         bit = int(bit, 0)
         if bit < 0 or bit >= 64:
@@ -202,7 +225,6 @@ def cmd_set_feature_bit(fd, group, bit):
         print "'%s' is not a valid bit number in range [0, 64)" % bit
         sys.exit(1)
 
-    h = Qcow(fd)
     if group == 'incompatible':
         h.incompatible_features |= 1 << bit
     elif group == 'compatible':
@@ -223,7 +245,7 @@ cmds = [
 ]
 
 def main(filename, cmd, args):
-    fd = open(filename, "r+b")
+    h = load_image(filename, 'r+b', 'qcow2')
     try:
         for name, handler, num_args, desc in cmds:
             if name != cmd:
@@ -232,11 +254,11 @@ def main(filename, cmd, args):
                 usage()
                 return
             else:
-                handler(fd, *args)
+                handler(h, *args)
                 return
         print "Unknown command '%s'" % cmd
     finally:
-        fd.close()
+        h.close()
 
 def usage():
     print "Usage: %s <file> <cmd> [<arg>, ...]" % sys.argv[0]
-- 
1.8.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [Qemu-devel] [RFC PATCH 4/5] qcow2.py: add method to load backing image
  2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
                   ` (2 preceding siblings ...)
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 3/5] qcow2.py: add load_image() method Paolo Bonzini
@ 2013-04-18 15:17 ` Paolo Bonzini
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 5/5] qcow2.py: add dump-map command Paolo Bonzini
  2013-04-22 12:38 ` [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Stefan Hajnoczi
  5 siblings, 0 replies; 7+ messages in thread
From: Paolo Bonzini @ 2013-04-18 15:17 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, stefanha

This also adds support for raw ImageFiles.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qemu-iotests/qcow2.py | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py
index 77e03cb..77a9d52 100755
--- a/tests/qemu-iotests/qcow2.py
+++ b/tests/qemu-iotests/qcow2.py
@@ -19,12 +19,22 @@ class QcowHeaderExtension:
 class ImageFile:
     def __init__(self, fd, fields):
         self.__dict__ = fields
+        if 'backing_file' not in fields:
+            self.backing_file = None
+        self.backing_image = None
         self.fd = fd
 
     def close(self):
         self.fd.close()
         self.fd = None
 
+    def open_backing_image(self, mode='rb'):
+        if not self.backing_image:
+            if self.backing_file:
+                self.backing_image = load_image(self.backing_file, mode,
+                                                self.backing_format)
+        return None
+
     def raw_pread(self, offset, size):
         self.fd.seek(offset)
         return self.fd.read(size)
@@ -33,6 +43,10 @@ class ImageFile:
         self.fd.seek(offset)
         return self.fd.write(data)
 
+class Raw(ImageFile):
+    def __init__(self, fd):
+        size = os.fstat(fd.fileno()).st_size
+        ImageFile.__init__(self, fd, dict(size=size))
 
 class Qcow(ImageFile):
 
@@ -65,6 +79,8 @@ class Qcow(ImageFile):
 
     HEADER_FMT = '>' + ''.join(field[0] for field in fields)
     QCOW_MAGIC = 0x514649fb
+    EXT_BACKING_FORMAT = 0xE2792ACA
+    EXT_FEATURE_NAMES = 0x6803f857
 
     def __init__(self, fd):
 
@@ -93,6 +109,8 @@ class Qcow(ImageFile):
         else:
             self.backing_file = None
 
+        self.backing_format = self.get_extension(Qcow.EXT_BACKING_FORMAT)
+
     def set_defaults(self):
         if self.version == 2:
             self.incompatible_features = 0
@@ -135,6 +153,12 @@ class Qcow(ImageFile):
         if self.fd.tell() > self.cluster_size:
             raise Exception("I think I just broke the image...")
 
+    def get_extension(self, magic):
+        for ex in self.extensions:
+            if ex.magic == magic:
+                return ex.data
+        return None
+
     def update(self):
         header_bytes = self.header_length
 
@@ -178,6 +202,8 @@ def load_image(filename, mode='rb', format=None):
 
     if format == 'qcow2':
         return Qcow(fd)
+    elif format == 'raw':
+        return Raw(fd)
 
     raise Error('unknown format %s' % format)
 
-- 
1.8.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [Qemu-devel] [RFC PATCH 5/5] qcow2.py: add dump-map command
  2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
                   ` (3 preceding siblings ...)
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 4/5] qcow2.py: add method to load backing image Paolo Bonzini
@ 2013-04-18 15:17 ` Paolo Bonzini
  2013-04-22 12:38 ` [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Stefan Hajnoczi
  5 siblings, 0 replies; 7+ messages in thread
From: Paolo Bonzini @ 2013-04-18 15:17 UTC (permalink / raw)
  To: qemu-devel; +Cc: kwolf, stefanha

The dump-map command visits the L1 and L2 tables, and uses them
to dump the guest->host cluster allocation for an entire chain
of qcow2 files.  Currently, it assumes that all files in the
chain are qcow2.  The output presents lines in two formats:

    <start> <length>                 for unallocated (zero) areas
    <start> <length> <depth> <pos>   for allocated areas

Depth identifies the file that holds the data (the top file in
the chain is 0, its backing file is 1, and so on), and pos is
the offset inside that file in bytes.  Start and length are in
bytes too.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 tests/qemu-iotests/qcow2.py | 116 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 116 insertions(+)

diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py
index 77a9d52..1609389 100755
--- a/tests/qemu-iotests/qcow2.py
+++ b/tests/qemu-iotests/qcow2.py
@@ -43,11 +43,47 @@ class ImageFile:
         self.fd.seek(offset)
         return self.fd.write(data)
 
+    def get_map(self, first, last):
+        if first > self.size:
+           yield (first, last - first)
+           return
+
+        for result in self.get_map_impl(first, min(last, self.size)):
+            yield result
+        if last > self.size:
+            yield (self.size, last - self.size)
+
+    def get_map_coalesced(self, first, last):
+        # Modify the list "cur" and return True if "next" represents an
+        # immediately following part of the same file.  Otherwise return
+        # False.
+        def coalesce(cur, next):
+            if len(cur) == len(next) and \
+                    cur[0] + cur[1] == next[0] and \
+                    (len(cur) == 2 or
+                     (cur[2] == next[2] and cur[3] + cur[1] == next[3])):
+                cur[1] = cur[1] + next[1]
+                return True
+            else:
+                return False
+
+        cur = None
+        for result in self.get_map(first, last):
+            if cur is None:
+                cur = list(result)
+            elif not coalesce(cur, result):
+                yield tuple(cur)
+                cur = list(result)
+        yield tuple(cur)
+
 class Raw(ImageFile):
     def __init__(self, fd):
         size = os.fstat(fd.fileno()).st_size
         ImageFile.__init__(self, fd, dict(size=size))
 
+    def get_map_impl(self, first, last):
+        yield (first, last - first, self, first)
+
 class Qcow(ImageFile):
 
     uint32_t = 'I'
@@ -81,6 +117,7 @@ class Qcow(ImageFile):
     QCOW_MAGIC = 0x514649fb
     EXT_BACKING_FORMAT = 0xE2792ACA
     EXT_FEATURE_NAMES = 0x6803f857
+    L2_BIT_ZERO = 1
 
     def __init__(self, fd):
 
@@ -102,6 +139,7 @@ class Qcow(ImageFile):
 
         fd.seek(self.header_length)
         self.load_extensions()
+        self.load_l1_table()
 
         if self.backing_file_offset:
             self.backing_file = self.raw_pread(self.backing_file_offset,
@@ -189,6 +227,69 @@ class Qcow(ImageFile):
             print "%-25s %s" % ("data", data)
             print ""
 
+    def read_table(self, host_offset, size):
+        s = self.raw_pread(host_offset, size)
+        table = [struct.unpack('>Q', s[i:i + 8])[0] for i in xrange(0, size, 8)]
+        return table
+
+    def load_l1_table(self):
+        self.l1_table = self.read_table(self.l1_table_offset,
+                                        self.l1_size * 8)
+
+    def read_l2_table(self, l1_entry):
+        if l1_entry == 0:
+            return None
+        return self.read_table(l1_entry & 0x7FFFFFFFFFFFFFFF,
+                               self.cluster_size)
+
+    def get_map_impl(self, first, last):
+        def get_host_offset(l2_entry):
+            if (l2_entry & 0x4000000000000000):
+                raise Error('compressed clusters not supported yet')
+            if (l2_entry == 0):
+                return None
+            if (l2_entry & Qcow.L2_BIT_ZERO) != 0:
+                return 0
+            return l2_entry & 0x00FFFFFFFFFFFE00
+
+        l1_entries = len(self.l1_table)
+        l2_entries = self.cluster_size / 8
+        last_l1_ofs = None
+        last_l2_table = None
+        offset = first % self.cluster_size
+        for pos in xrange(first - offset, last, self.cluster_size):
+            if pos <= self.size:
+                cluster = pos / self.cluster_size
+                left = min(last, pos + self.cluster_size) - pos
+                l2_ofs = cluster % l2_entries
+                l1_ofs = cluster / l2_entries
+
+                if l1_ofs != last_l1_ofs:
+                    last_l1_ofs = l1_ofs
+                    last_l2_table = self.read_l2_table(self.l1_table[l1_ofs])
+
+                if last_l2_table is None:
+                    l2_entry = 0
+                else:
+                    l2_entry = last_l2_table[l2_ofs]
+            else:
+                l2_entry = Qcow.L2_BIT_ZERO
+
+            host_offset = get_host_offset(l2_entry)
+            if host_offset is None:
+                self.open_backing_image()
+                if self.backing_image is not None:
+                    for result in self.backing_image.get_map(pos + offset, pos + left):
+                        yield result
+                    continue
+                else:
+                    host_offset = 0
+            if host_offset != 0:
+                yield (pos, left, self, host_offset + offset)
+            else:
+                yield (pos, left)
+            offset = 0
+
 def load_image(filename, mode='rb', format=None):
     fd = open(filename, mode)
     if format is None:
@@ -207,6 +308,20 @@ def load_image(filename, mode='rb', format=None):
 
     raise Error('unknown format %s' % format)
 
+def cmd_dump_map(q):
+    def do_load_chain(q):
+        q.open_backing_image()
+        if q.backing_image:
+            q.backing_image.depth = q.depth + 1
+            do_load_chain(q.backing_image)
+
+    q.depth = 0
+    do_load_chain(q)
+    for result in q.get_map_coalesced(0, q.size):
+        if len(result) == 2:
+            print "%d %d" % (result[0], result[1])
+        else:
+            print "%d %d %d %d" % (result[0], result[1], result[2].depth, result[3])
 
 def cmd_dump_header(h):
     h.dump()
@@ -265,6 +380,7 @@ def cmd_set_feature_bit(h, group, bit):
 
 cmds = [
     [ 'dump-header',    cmd_dump_header,    0, 'Dump image header and header extensions' ],
+    [ 'dump-map',       cmd_dump_map,       0, 'Dump cluster map' ],
     [ 'add-header-ext', cmd_add_header_ext, 2, 'Add a header extension' ],
     [ 'del-header-ext', cmd_del_header_ext, 1, 'Delete a header extension' ],
     [ 'set-feature-bit', cmd_set_feature_bit, 2, 'Set a feature bit'],
-- 
1.8.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata
  2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
                   ` (4 preceding siblings ...)
  2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 5/5] qcow2.py: add dump-map command Paolo Bonzini
@ 2013-04-22 12:38 ` Stefan Hajnoczi
  5 siblings, 0 replies; 7+ messages in thread
From: Stefan Hajnoczi @ 2013-04-22 12:38 UTC (permalink / raw)
  To: Paolo Bonzini; +Cc: kwolf, qemu-devel, stefanha

On Thu, Apr 18, 2013 at 05:17:24PM +0200, Paolo Bonzini wrote:
> This series includes several improvements to qcow2.py, mostly
> the ability to dump the L1 and L2 tables for a full backing
> chain.
> 
> Doing this completely would require support for all image formats,
> not just qcow2.  To cover the common cases, I included support
> for raw backing files.
> 
> The right thing to do perhaps would be to add the read-metadata
> patches from the in-place QED conversion summer of code project,
> but I wanted to throw out these anyway if anyone has comments.

The patches look sane by themselves but it would be nicer to integrate
with qemu-img by extending the block layer C code rather than
reimplementing part of the block layer in Python for this single command.

Stefan

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2013-04-22 12:38 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-04-18 15:17 [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Paolo Bonzini
2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 1/5] qcow2.py: rename class to Qcow Paolo Bonzini
2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 2/5] qcow2.py: add ImageFile superclass Paolo Bonzini
2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 3/5] qcow2.py: add load_image() method Paolo Bonzini
2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 4/5] qcow2.py: add method to load backing image Paolo Bonzini
2013-04-18 15:17 ` [Qemu-devel] [RFC PATCH 5/5] qcow2.py: add dump-map command Paolo Bonzini
2013-04-22 12:38 ` [Qemu-devel] [RFC PATCH 0/5] qcow2.py: dump metadata Stefan Hajnoczi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).