[Qemu-devel] [PATCH 0/2] image-fuzzer: Extend test coverage

[Qemu-devel] [PATCH 0/2] image-fuzzer: Extend test coverage

[Maria Kustova]

This patch series contains changes improving test coverage.

Maria Kustova (2):
  runner: Expand the list of default test commands
  fuzz: Make fuzzing functions and values relevant to the qemu
    implementation

 tests/image-fuzzer/qcow2/fuzz.py | 71 +++++++++++++++++++------------------
 tests/image-fuzzer/runner.py     | 75 +++++++++++++++++++++++++++++-----------
 2 files changed, 92 insertions(+), 54 deletions(-)

-- 
1.9.3

[Qemu-devel] [PATCH 1/2] runner: Expand the list of default test commands

[Maria Kustova]

Additional commands were added to the default runner list to cover all qcow2
related code. This qcow2 specificity is selected to reduce number of
non-relevant tests. After implementation of a fuzzer for a new format the
default list should be updated.

Signed-off-by: Maria Kustova <maria.k@catit.be>
---
 tests/image-fuzzer/runner.py | 75 ++++++++++++++++++++++++++++++++------------
 1 file changed, 55 insertions(+), 20 deletions(-)

diff --git a/tests/image-fuzzer/runner.py b/tests/image-fuzzer/runner.py
index 2e1bd51..fd97c40 100755
--- a/tests/image-fuzzer/runner.py
+++ b/tests/image-fuzzer/runner.py
@@ -30,6 +30,9 @@ import getopt
 import StringIO
 import resource
 
+# All formats supported by the 'qemu-img create' command.
+WRITABLE_FORMATS = ['raw', 'vmdk', 'vdi', 'cow', 'qcow2', 'file', 'qed', 'vpc']
+
 try:
     import json
 except ImportError:
@@ -137,24 +140,57 @@ class TestEnv(object):
         self.qemu_img = os.environ.get('QEMU_IMG', 'qemu-img')\
                                   .strip().split(' ')
         self.qemu_io = os.environ.get('QEMU_IO', 'qemu-io').strip().split(' ')
-        self.commands = [['qemu-img', 'check', '-f', 'qcow2', '$test_img'],
-                         ['qemu-img', 'info', '-f', 'qcow2', '$test_img'],
-                         ['qemu-io', '$test_img', '-c', 'read $off $len'],
-                         ['qemu-io', '$test_img', '-c', 'write $off $len'],
-                         ['qemu-io', '$test_img', '-c',
-                          'aio_read $off $len'],
-                         ['qemu-io', '$test_img', '-c',
-                          'aio_write $off $len'],
-                         ['qemu-io', '$test_img', '-c', 'flush'],
-                         ['qemu-io', '$test_img', '-c',
-                          'discard $off $len'],
-                         ['qemu-io', '$test_img', '-c',
-                          'truncate $off']]
-        for fmt in ['raw', 'vmdk', 'vdi', 'cow', 'qcow2', 'file',
-                    'qed', 'vpc']:
+        strings = ['%s%p%x%d', '.1024d', '%.2049d', '%p%p%p%p', '%x%x%x%x',
+                   '%d%d%d%d', '%s%s%s%s', '%99999999999s', '%08x', '%%20d',
+                   '%%20n', '%%20x', '%%20s', '%s%s%s%s%s%s%s%s%s%s',
+                   '%p%p%p%p%p%p%p%p%p%p',
+                   '%#0123456x%08x%x%s%p%d%n%o%u%c%h%l%q%j%z%Z%t%i%e%g%f%a%C' +
+                   '%S%08x%%', '%s x 129', '%x x 257']
+        self.commands = [
+            ['qemu-img', 'check', '-f', 'qcow2', '$test_img'],
+            ['qemu-img', 'check', '-f', 'qcow2', '-r', 'leaks', '$test_img'],
+            ['qemu-img', 'check', '-f', 'qcow2', '-r', 'all', '$test_img'],
+            ['qemu-img', 'snapshot', '-c', 'new', '$test_img'],
+            ['qemu-img', 'info', '-f', 'qcow2', '$test_img'],
+            ['qemu-img', 'convert', '-c', '-f', 'qcow2', '-O', 'qcow2',
+             '$test_img', 'converted_image.qcow2'],
+            ['qemu-img', 'amend', '-o', 'compat=0.10', '-f', 'qcow2',
+             '$test_img'],
+            ['qemu-img', 'amend', '-o', 'lazy_refcounts=on', '-f', 'qcow2',
+             '$test_img'],
+            ['qemu-img', 'amend', '-o', 'lazy_refcounts=off', '-f', 'qcow2',
+             '$test_img'],
+            ['qemu-img', 'amend', '-o',
+             'backing_file=' + random.choice(strings), '-f', 'qcow2',
+             '$test_img'],
+            ['qemu-img', 'amend', '-o', 'backing_fmt=' + random.choice(strings),
+             '-f', 'qcow2', '$test_img'],
+            ['qemu-io', '$test_img', '-c', 'read $off $len'],
+            ['qemu-io', '$test_img', '-c', 'read -p $off $len'],
+            ['qemu-io', '$test_img', '-c', 'write $off $len'],
+            ['qemu-io', '$test_img', '-c', 'write -c $off $len'],
+            ['qemu-io', '$test_img', '-c', 'write -p $off $len'],
+            ['qemu-io', '$test_img', '-c', 'write -z $off $len'],
+            ['qemu-io', '$test_img', '-c', 'aio_read $off $len'],
+            ['qemu-io', '$test_img', '-c', 'aio_write $off $len'],
+            ['qemu-io', '$test_img', '-c', 'flush'],
+            ['qemu-io', '$test_img', '-c', 'discard $off $len'],
+            ['qemu-io', '$test_img', '-c', 'truncate $off'],
+            ['qemu-io', '$test_img', '-c', 'info'],
+            ['qemu-io', '$test_img', '-c', 'map']
+        ]
+
+        for fmt in WRITABLE_FORMATS:
+            cache_opt = random.choice([
+                [], ['-t', 'unsafe'],
+                ['-t', 'writethrough'],
+                ['-t', 'writeback'],
+                ['-t', 'none']
+            ])
+
             self.commands.append(
-                ['qemu-img', 'convert', '-f', 'qcow2', '-O', fmt,
-                 '$test_img', 'converted_image.' + fmt])
+                ['qemu-img', 'convert', '-f', 'qcow2', '-O', fmt] + cache_opt +
+                ['$test_img', 'converted_image.' + fmt])
 
         try:
             os.makedirs(self.current_dir)
@@ -177,9 +213,8 @@ class TestEnv(object):
         Format of a backing file is randomly chosen from all formats supported
         by 'qemu-img create'.
         """
-        # All formats supported by the 'qemu-img create' command.
-        backing_file_fmt = random.choice(['raw', 'vmdk', 'vdi', 'cow', 'qcow2',
-                                          'file', 'qed', 'vpc'])
+
+        backing_file_fmt = random.choice(WRITABLE_FORMATS)
         backing_file_name = 'backing_img.' + backing_file_fmt
         backing_file_size = random.randint(MIN_BACKING_FILE_SIZE,
                                            MAX_BACKING_FILE_SIZE) * (1 << 20)
-- 
1.9.3

[Qemu-devel] [PATCH 2/2] fuzz: Make fuzzing functions and values relevant to the qemu implementation

[Maria Kustova]

Heuristic values were added to fuzzing constraints and vectors.

Signed-off-by: Maria Kustova <maria.k@catit.be>
---
 tests/image-fuzzer/qcow2/fuzz.py | 71 +++++++++++++++++++++-------------------
 1 file changed, 37 insertions(+), 34 deletions(-)

diff --git a/tests/image-fuzzer/qcow2/fuzz.py b/tests/image-fuzzer/qcow2/fuzz.py
index 5852b4d..6e272c6 100644
--- a/tests/image-fuzzer/qcow2/fuzz.py
+++ b/tests/image-fuzzer/qcow2/fuzz.py
@@ -17,6 +17,7 @@
 #
 
 import random
+from sys import maxint as INT_MAX
 
 UINT8 = 0xff
 UINT16 = 0xffff
@@ -25,16 +26,21 @@ UINT64 = 0xffffffffffffffff
 # Most significant bit orders
 UINT32_M = 31
 UINT64_M = 63
+# Sizes
+UINT64_S = 8
 # Fuzz vectors
-UINT8_V = [0, 0x10, UINT8/4, UINT8/2 - 1, UINT8/2, UINT8/2 + 1, UINT8 - 1,
+UINT8_V = [0, 1, 0x10, UINT8/4, UINT8/2 - 1, UINT8/2, UINT8/2 + 1, UINT8 - 1,
            UINT8]
-UINT16_V = [0, 0x100, 0x1000, UINT16/4, UINT16/2 - 1, UINT16/2, UINT16/2 + 1,
-            UINT16 - 1, UINT16]
-UINT32_V = [0, 0x100, 0x1000, 0x10000, 0x100000, UINT32/4, UINT32/2 - 1,
-            UINT32/2, UINT32/2 + 1, UINT32 - 1, UINT32]
-UINT64_V = UINT32_V + [0x1000000, 0x10000000, 0x100000000, UINT64/4,
-                       UINT64/2 - 1, UINT64/2, UINT64/2 + 1, UINT64 - 1,
-                       UINT64]
+UINT16_V = [0, 1, 0x100, 0x1000, UINT16/4, UINT16/2 - 1, UINT16/2,
+            UINT16/2 + 1, UINT16 - 1, UINT16]
+UINT32_V = UINT16_V + [UINT16 + 1, UINT16 + 2, 0x10000, 0x100000,
+                       0x1000000, 0x10000000, UINT32/4, UINT32/2 - 1,
+                       UINT32/2, UINT32/2 + 1, UINT32 - 1, UINT32]
+# Exclude the vector of 16 bit values
+UINT64_V = UINT32_V[len(UINT16_V):] + \
+           [0, 1, UINT32 + 1, UINT32 + 2, 0x100000000, INT_MAX/UINT64_S - 1,
+            INT_MAX / UINT64_S, INT_MAX/UINT64_S + 1, UINT64/4,
+            UINT64/2 - 1, UINT64/2, UINT64/2 + 1, UINT64 - 1, UINT64]
 STRING_V = ['%s%p%x%d', '.1024d', '%.2049d', '%p%p%p%p', '%x%x%x%x',
             '%d%d%d%d', '%s%s%s%s', '%99999999999s', '%08x', '%%20d', '%%20n',
             '%%20x', '%%20s', '%s%s%s%s%s%s%s%s%s%s', '%p%p%p%p%p%p%p%p%p%p',
@@ -124,6 +130,7 @@ def selector(current, constraints, validate=int_validator):
     randomly selects one value satisfying at least one constraint (depending on
     constraints overlaps).
     """
+
     def iter_validate(c):
         """Apply validate() only to constraints represented as lists.
 
@@ -135,33 +142,22 @@ def selector(current, constraints, validate=int_validator):
         else:
             return c
 
-    fuzz_values = [iter_validate(c) for c in constraints]
-    # Remove current for cases it's implicitly specified in constraints
-    # Duplicate validator functionality to prevent decreasing of probability
-    # to get one of allowable values
-    # TODO: remove validators after implementation of intelligent selection
-    # of fields will be fuzzed
-    try:
-        fuzz_values.remove(current)
-    except ValueError:
-        pass
+    v_constraints = [x for x in constraints if x != current]
+    fuzz_values = [iter_validate(c) for c in v_constraints]
     return random.choice(fuzz_values)
 
 
 def magic(current):
-    """Fuzz magic header field.
-
-    The function just returns the current magic value and provides uniformity
-    of calls for all fuzzing functions.
-    """
-    return current
+    """Fuzz magic header field."""
+    constraints = ['VMDK', 'QED', '', 'OOOM'] + \
+                  [truncate_string(STRING_V, len(current))]
+    return selector(current, constraints, string_validator)
 
 
 def version(current):
     """Fuzz version header field."""
     constraints = UINT32_V + [
-        [(2, 3)],  # correct values
-        [(0, 1), (4, UINT32)]
+        [(0, 4)]  # includes valid values
     ]
     return selector(current, constraints)
 
@@ -195,16 +191,18 @@ def size(current):
 
 def crypt_method(current):
     """Fuzz crypt method header field."""
-    constraints = UINT32_V + [
-        1,
-        [(2, UINT32)]
-    ]
+    # UINT32_V includes valid values [0, 1]
+    constraints = UINT32_V
     return selector(current, constraints)
 
 
 def l1_size(current):
     """Fuzz L1 table size header field."""
-    constraints = UINT32_V
+    # QCOW_MAX_L1_SIZE = 0x2000000
+    max_size = 0x2000000 / UINT64_S
+    constraints = UINT32_V + \
+                  [max_size - 1, max_size, max_size + 1] + \
+                  [[(0, current + 1)]]
     return selector(current, constraints)
 
 
@@ -222,12 +220,18 @@ def refcount_table_offset(current):
 
 def refcount_table_clusters(current):
     """Fuzz refcount table clusters header field."""
-    constraints = UINT32_V
+    # QCOW_MAX_REFTABLE_SIZE = 0x800000, MIN_CLUSTER_BITS = 9 =>
+    # max size of reftable in clusters = 1 << 14
+    max_size = 1 << 14
+    constraints = UINT32_V + \
+                  [max_size - 1, max_size, max_size + 1] + \
+                  [[(0, current + 1)]]
     return selector(current, constraints)
 
 
 def nb_snapshots(current):
     """Fuzz number of snapshots header field."""
+    # QCOW_MAX_SNAPSHOTS = 1 << 16, included in UINT32_V
     constraints = UINT32_V
     return selector(current, constraints)
 
@@ -273,8 +277,7 @@ def header_length(current):
     """Fuzz number of refcount order header field."""
     constraints = UINT32_V + [
         72,
-        104,
-        [(0, UINT32)]
+        104
     ]
     return selector(current, constraints)
 
-- 
1.9.3

Re: [Qemu-devel] [PATCH 0/2] image-fuzzer: Extend test coverage

[M.Kustova]

ping

http://patchwork.ozlabs.org/patch/381137
http://patchwork.ozlabs.org/patch/381138

On Tue, Aug 19, 2014 at 1:38 AM, Maria Kustova <maxa@catit.be> wrote:
> This patch series contains changes improving test coverage.
>
> Maria Kustova (2):
>   runner: Expand the list of default test commands
>   fuzz: Make fuzzing functions and values relevant to the qemu
>     implementation
>
>  tests/image-fuzzer/qcow2/fuzz.py | 71 +++++++++++++++++++------------------
>  tests/image-fuzzer/runner.py     | 75 +++++++++++++++++++++++++++++-----------
>  2 files changed, 92 insertions(+), 54 deletions(-)
>
> --
> 1.9.3
>

Re: [Qemu-devel] [PATCH 0/2] image-fuzzer: Extend test coverage

[M.Kustova]

On Thu, Aug 28, 2014 at 9:25 PM, M.Kustova <maria.k@catit.be> wrote:
> ping
>
ping

> http://patchwork.ozlabs.org/patch/381137
> http://patchwork.ozlabs.org/patch/381138
>
> On Tue, Aug 19, 2014 at 1:38 AM, Maria Kustova <maxa@catit.be> wrote:
>> This patch series contains changes improving test coverage.
>>
>> Maria Kustova (2):
>>   runner: Expand the list of default test commands
>>   fuzz: Make fuzzing functions and values relevant to the qemu
>>     implementation
>>
>>  tests/image-fuzzer/qcow2/fuzz.py | 71 +++++++++++++++++++------------------
>>  tests/image-fuzzer/runner.py     | 75 +++++++++++++++++++++++++++++-----------
>>  2 files changed, 92 insertions(+), 54 deletions(-)
>>
>> --
>> 1.9.3
>>