From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1KGEYu-0004mv-Nf for qemu-devel@nongnu.org; Tue, 08 Jul 2008 10:53:00 -0400 Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1KGEYt-0004mO-7F for qemu-devel@nongnu.org; Tue, 08 Jul 2008 10:53:00 -0400 Received: from [199.232.76.173] (port=53400 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1KGEYt-0004mL-4E for qemu-devel@nongnu.org; Tue, 08 Jul 2008 10:52:59 -0400 Received: from rv-out-0708.google.com ([209.85.198.243]:20765) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1KGEYs-0000hp-Kl for qemu-devel@nongnu.org; Tue, 08 Jul 2008 10:52:58 -0400 Received: by rv-out-0708.google.com with SMTP id f25so2301813rvb.22 for ; Tue, 08 Jul 2008 07:52:57 -0700 (PDT) Message-ID: <761ea48b0807080752m729b9892sd997347f6f4462b9@mail.gmail.com> Date: Tue, 8 Jul 2008 16:52:57 +0200 From: "Laurent Desnogues" Subject: Re: [Qemu-devel] [RFC] ARM test generator In-Reply-To: <200807081426.43774.paul@codesourcery.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <761ea48b0807080535h4dadd1eev8c076bfb86f72e92@mail.gmail.com> <200807081426.43774.paul@codesourcery.com> Reply-To: qemu-devel@nongnu.org List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paul Brook Cc: qemu-devel@nongnu.org On Tue, Jul 8, 2008 at 3:26 PM, Paul Brook wrote: > I'm unconvinced how much value this actually adds unless you can get fairly > extensive coverage. i.e. the interesting bit is how you generate > test-arm.txt and test-ref.txt. Of course the problem is the data coverage (as opposed to code coverage) for which no tool will tell us how close we are to 100%. For test-ref.txt generation, the solution is trivial: use real hardware. As long as you are not testing unpredictable or implementation defined behaviours, that can be considered as a reference that qemu should match. > You should also cover different combinations > of overlapping source/destination operands. That's a good point. Anyway, the aim of my generator is not to fully validate qemu ARM implementation. I just wanted some simple stuff to provide demonstrable evidence of a problem when words are not enough :) Laurent