From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40270) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X1hQ7-000120-5r for qemu-devel@nongnu.org; Mon, 30 Jun 2014 15:39:24 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1X1hQ0-0006UW-Uy for qemu-devel@nongnu.org; Mon, 30 Jun 2014 15:39:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:3679) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1X1hQ0-0006UP-ML for qemu-devel@nongnu.org; Mon, 30 Jun 2014 15:39:12 -0400 Message-ID: <53B1BCDD.1080701@redhat.com> Date: Mon, 30 Jun 2014 13:39:09 -0600 From: Eric Blake MIME-Version: 1.0 References: <39109fb5b97b3b1cab21ea0c6f69e9a4955a45d7.1404128388.git.maria.k@catit.be> In-Reply-To: <39109fb5b97b3b1cab21ea0c6f69e9a4955a45d7.1404128388.git.maria.k@catit.be> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="mUOWOGX93hKtnprdKSABedhvhR7ROpcXw" Subject: Re: [Qemu-devel] [PATCH 1/5] docs: Specification for the image fuzzer List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Maria Kustova , qemu-devel@nongnu.org Cc: kwolf@redhat.com, famz@redhat.com, Maria Kustova , stefanha@redhat.com This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --mUOWOGX93hKtnprdKSABedhvhR7ROpcXw Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 06/30/2014 05:48 AM, Maria Kustova wrote: > 'Overall fuzzer requirements' chapter contains the current product visi= on and > features done and to be done. This chapter is still in progress. >=20 > Signed-off-by: Maria Kustova > --- > tests/image-fuzzer/docs/image-fuzzer.txt | 176 +++++++++++++++++++++++= ++++++++ > 1 file changed, 176 insertions(+) > create mode 100644 tests/image-fuzzer/docs/image-fuzzer.txt >=20 > diff --git a/tests/image-fuzzer/docs/image-fuzzer.txt b/tests/image-fuz= zer/docs/image-fuzzer.txt > new file mode 100644 > index 0000000..a9ed4b6 > --- /dev/null > +++ b/tests/image-fuzzer/docs/image-fuzzer.txt > @@ -0,0 +1,176 @@ > +Image fuzzer > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D You're no worse than the bulk of the files in this directory for omitting copyright and license information, but it would be nice to buck the trend and provide it explicitly. > + > +Description > +----------- > + > +The goal of the image fuzzer is to catch crashes of qemu-io/qemu-img p= roviding > +to them randomly corrupted images. s/providing to/by providing/ > +Test images are generated from scratch and have valid inner structure = with some > +elements, e.g. L1/L2 tables, having random invalid values. > + > + > +Test runner > +----------- > + > +The test runner generates test images, executes tests utilizing genera= ted > +images, indicates their results and collect all test related artifacts= (logs, s/collect/collects/ > +core dumps, test images). > +The test means one start of a system under test (SUT), e.g. qemu-io, w= ith > +specified arguments and one test image. > +By default, the test runner generates new tests and executes them till= s/till/until/ > +keyboard interruption. But if a test seed is specified via '-s' runner= s/via/via the/ > +parameter, then only one test with this seed will be executed, after i= ts finish > +the runner will exit. > + > +The runner uses an external image fuzzer to generate test images. An i= mage > +generator should be specified as a mandatory parameter of the test run= ner. > +Details about interactions between the runner and fuzzers see "Module > +interfaces". > + > +The runner activates generation of core dumps during test executions, = but it > +assumes that core dumps will be generated in the current working direc= tory. > +For comprehensive test results, please, set up your test environment > +properly. > + > +Path to a SUT can be specified via environment variables (for now only= > +qemu-img) or via '--binary' parameter of the test runner. For details = about > +environment variables see qemu-iotests/check. If both are specified, which wins? (Command line should always trump environment) > + > + > +Qcow2 image generator > +--------------------- > + > +The 'qcow2' generator is a Python package providing 'create_image' met= hod as > +a single public API. See details in 'Test runner/image fuzzer' in 'Mod= ule > +interfaces'. > + > +Qcow2 contains two submodules: fuzz.py and layout.py. > + > +'fuzz.py' contains all fuzzing functions one per image field. It's ass= umed that s/functions/functions,/ > +after code analysis every field will have own constraints for its valu= e. For > +now only universal potentially dangerous values are used, e.g. type li= mits for > +integers or unsafe symbols as '%s' for strings. For bitmasks random am= ount of > +bits are set to ones. All fuzzed values are checked on non-equality to= the > +current valid value of the field. In case of equality the value will b= e > +regenerated. > + > +'layout.py' creates a random valid image, fuzzes a random subset of th= e image > +fields by 'fuzz.py' module and writes a fuzzed image to the file speci= fied. > + > +For now only header fields and header extensions are generated, the re= maining > +file is filled with zeros. > + > + > +Module interfaces > +----------------- > + > +* Test runner/image fuzzer > + > +The runner calls an image generator specifying path to a test image fi= le. s/path/the path/ > +An image generator is expected to provide 'create_image(test_img_path)= ' method s/provide/provide a/ > +that creates a test image and writes it to the specified file. The fil= e should > +be created if it doesn't exist or overwritten otherwise. > +Random seed is set by the runner at every test execution for the regre= ssion > +purpose, so an image generator is not recommended to modify it interna= lly. > + > +* Test runner/SUT > + > +A full test command is composed from the SUT, its arguments specified > +via '-c' runner parameter and the name of generated image. To indicate= where > +a test image name should be placed TEST_IMG placeholder can be used. > +For example, for the next test command > + > + qemu-img convert -O vmdk fuzzed_image test.vmdk > + > +a call of the image fuzzer will be following: > + > + ./runner.py -b qemu-img -c 'convert -O vmdk TEST_IMG test.vmdk' work_= dir qcow2 > + > + > +Overall fuzzer requirements > +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > + > +Input data: > +---------- > + > + - image template (generator) > + - work directory > + - test run duration (optional) > + - action vector (optional) > + - seed (optional) > + - SUT and its arguments (optional) > + > + > +Fuzzer requirements: > +------------------- > + > +1. Should be able to inject random data > +2. Should be able to select a random value from the manually pregener= ated > + vector (boundary values, e.g. max/min cluster size) > +3. Image template should describe a general structure invariant for a= ll > + test images (image format description) > +4. Image template should be autonomous and other fuzzer parts should = not > + relate on it s/relate/rely/ > +5. Image template should contain reference rules (not only block+size= > + description) > +6. Should generate the test image with the correct structure based on= an image > + template > +7. Should accept a seed as an argument (for regression purpose) > +8. Should generate a seed if it is not specified as an input paramete= r. > +9. The same seed should generate the same image, if no or the same ac= tion > + vector is specified s/no or the same action vector/the same action vector (including the case of no action)/ > +10. Should accept a vector of actions as an argument (for test reprodu= cing and > + for test case specification, e.g. group of tests for header struct= ure, > + group of test for snapshots, etc) > +11. Action vector should be randomly generated from the pool of availa= ble > + actions, if it is not specified as an input parameter > +12. Pool of actions should be defined automatically based on an image = template > +13. Should accept a SUT and its call parameters as an argument or sele= ct them > + randomly otherwise. As far as it's expected to be rarely changed, = the list > + of all possible test commands can be available in the test runner > + internally. > +14. Should accept a test run duration as an argument. Tests should be = executed > + during a minimum period from a test run duration and time while fu= zzer can > + generate different test images > +15. Should support an external cancellation of a test run > +16. Seed and action vector should be logged (for regression purpose) > +17. All files related to test result should be collected: a test image= , > + SUT logs, fuzzer logs and crash dumps > +18. Should be compatible with python version 2.4-2.7 > +19. Usage of external libraries should be limited as much as possible.= > + > + > +Image formats: > +------------- > + > +Main target image format is qcow2, but support of image templates shou= ld > +provide an ability to add any other image format. > + > + > +Effectiveness: > +------------- > + > +Fuzzer can be controlled via template, seed and action vector; > +it makes the fuzzer itself invariant to an image format and test logic= =2E > +It should be able to perform rather complex and precise tests, that ca= n be > +specified via action vector. Otherwise, knowledge about an image struc= ture > +allows the fuzzer to generate the pool of all available areas can be f= uzzed > +and randomly select some of them and so compose its own action vector.= > +Also complexity of a template defines complexity of the fuzzer, so its= > +functionality can be varied from simple model-independent fuzzing to s= mart > +model-based one. > + > + > +Glossary: > +-------- > + > +Action vector is a sequence of structure elements retrieved from an im= age > +format, each of them will be fuzzed for the test image. It's a subset = of > +elements of the action pool. Example: header, refcount table, etc. > +Action pool is all available elements of an image structure that gener= ated > +automatically from an image template. > +Image template is a formal description of an image structure and relat= ions > +between image blocks. > +Test image is an output image of the fuzzer defined by the current see= d and > +action vector. >=20 --=20 Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org --mUOWOGX93hKtnprdKSABedhvhR7ROpcXw Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJTsbzdAAoJEKeha0olJ0NqJ7kH/R6G/X+iYcm40/UHAlc9U7EH pEiQDSebYz3CaZrVzCq5pdNGb/m+4yHfJOHMu4m3x8HiAdUQOCYWaRKs4H9icXIJ ho/6ICcS/ED/qG2D//XlBkYFK2qHliUP6EuHHjJAzBSDQLT/GfzVYqu1xcwFaYne DY2TThKWzHifbZXYa8wODgQVC5FyvIg+u0fD1OgxMDq3Tw+qb7GxwBZxdSCoZvhA bC8tecpwrCWySt3RZVIcVINRrBQr9JDq1UbabvVI/DwwR5GgK3nuU8GUenTg7TnV WrzazG37iAIjsB7hqRcr1IeC+bZRvmObSGt3DMKKXCFRhO0bRTNs78wfUy8SCpA= =8GX/ -----END PGP SIGNATURE----- --mUOWOGX93hKtnprdKSABedhvhR7ROpcXw--