From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ryan Harper Subject: Re: kvm-autotest -- introducing kvm_runtest_2 Date: Thu, 12 Mar 2009 07:54:33 -0500 Message-ID: <20090312125433.GY11777@us.ibm.com> References: <316573781.1616221236842323850.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> <1503567742.1616341236842702283.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Ryan Harper , KVM List , Uri Lublin To: Michael Goldish Return-path: Received: from e7.ny.us.ibm.com ([32.97.182.137]:51946 "EHLO e7.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752530AbZCLMyi (ORCPT ); Thu, 12 Mar 2009 08:54:38 -0400 Received: from d01relay02.pok.ibm.com (d01relay02.pok.ibm.com [9.56.227.234]) by e7.ny.us.ibm.com (8.13.1/8.13.1) with ESMTP id n2CCjj9m010170 for ; Thu, 12 Mar 2009 08:45:45 -0400 Received: from d01av02.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by d01relay02.pok.ibm.com (8.13.8/8.13.8/NCO v9.2) with ESMTP id n2CCsaUk188884 for ; Thu, 12 Mar 2009 08:54:36 -0400 Received: from d01av02.pok.ibm.com (loopback [127.0.0.1]) by d01av02.pok.ibm.com (8.12.11.20060308/8.13.3) with ESMTP id n2CCrDoc006335 for ; Thu, 12 Mar 2009 08:53:14 -0400 Content-Disposition: inline In-Reply-To: <1503567742.1616341236842702283.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: * Michael Goldish [2009-03-12 02:26]: >=20 > > > Regarding the stepfiles you created for Linux -- I can't help muc= h > > > with those since I don't have the data. I do believe that if I ha= d > > the > > > data and the stepfiles I could quickly identify the problem, so i= f > > you > > > think those can be sent to us, I'd like to have them. > >=20 > > I created a stepfile for RHEL5 and what I'm seeing is that one of t= he > > screens I captured in stepmaker ended up having a focus ring around > > something and on replay the focus isn't there. =A0This situation is= n't > > something that a new algo will fix as you pointed out. =A0I'm wonde= ring > > if > > this is something you've seen. =A0I don't quite understand how it w= ould > > happen since stepmaker and the replace send the same keystrokes. =A0= I > > also > > don't see how in general this can be avoided. >=20 > The problem sounds familiar. Does the ring appear around one of the > GNOME menubars, i.e. around "Applications" or "System"? GNOME seems t= o > be somewhat indeterministic with those rings. If you run the stepfile > several times, you'll notice that in most cases you'll see a focus > ring (or no focus ring, I don't quite remember) and the rest of the > time you'll get the other case. Ding Ding Ding! =3D) >=20 > This can be avoided either with experience, or a good wiki entry on > picking the right barriers (which we plan to create). But you don't > have to avoid making mistakes with stepmaker -- most types of mistake= s > are fixed very quickly and easily with stepeditor. yep, used stepeditor to fix; defintely worth documenting where one should be invoking stepeditor -- from the steps dir; if you don't run i= t from there, it won't find the steps_data dir =3D( >=20 > The fix depends on exactly what you were trying to do: >=20 > - If you sent "alt-f1" to open the menu, and in the following step > picked the open menu (including the "Applications" caption itself) to > make sure it was open -- use stepeditor to modify the barrier so that > it doesn't include the "Applications" caption or anything that might > have a ring around it. That worked for me. >=20 > The following text was copied from your previous e-mail: >=20 > > I do have the debug dir data from these runs. =A0Looking at the cro= pped > > ppm and screendump ppm is how I determined that there must be > > something > > wrong with how the image is rendered since the cropped ppm matches > > the > > screendump output, but with whatever subtle difference that generat= es > > a > > different md5sum. >=20 > I'm not sure my previous e-mail was clear enough, so just in case it > wasn't, let me rephrase: The cropped ppm is generated from the > screendump ppm every time the stepfile running module receives a > screendump from the guest in order to see if it matches a barrier. > This is done for debugging purposes. If you somehow check, you'll see > there is no subtle difference between those two files. It wouldn't > make sense to find a subtle difference between them, and if you did > find one, it certainly wouldn't indicate a stepfile problem, but > rather a very strange bug in the framework. You should be looking fo= r > subtle differences between the screendump ppm and the reference > screendump ppm, as well as between the cropped screendump ppm and the > reference cropped screendump ppm. By "reference" I mean coming from > the stepmaker data. If you don't have the stepmaker data, you have no > way of knowing what caused the difference in the md5sums. Right -- the real win was comparing the full screendump to the referenc= e screendump - basically, without the reference dumps, the debug output isn't useful. I'll have to go back and re-read your email on where to put the reference ppm files so one gets the refrence comparision. >=20 >=20 > There are two other things I forgot to mention in my previous e-mail: >=20 > The Windows failures you're seeing might be caused by KVM bugs other > than the one I mentioned. KVM-84 has a very strong tendency to crash > during Windows installations. You can use the logs to find out if tha= t > happened in your case. If you have the latest git HEAD the exception > info will look something like "Barrier timed out at step ... (VM is > dead)", and if you have a slightly older version, you'll probably see > "(guest is stuck)" at the end of the info string. You should also see > the system consistently complaining that it can't fetch any > screendumps from the guest (this will appear in stdout). I've seen those on kvm-84. > The other thing has to do with the ISO files. kvm_runtest has a very > important feature that we innocently forgot to implement in > kvm_runtest_2 -- md5sum verification of the ISO files. This means tha= t > the framework currently makes no use of the "md5sum" and "md5sum_1m" > parameters in the config file. This means you might be using differen= t > ISOs than the ones we made our stepfiles with. In that case I wouldn'= t > expect any stepfile to succeed. However, if you used these same ISOs > with kvm_runtest then they should be fine. In any case, I'll add the > feature ASAP to the git repository. Right - I suppose it might be better if the names of the windows iso disks matched how MS names them in MSDN, for example, kvm_runtest refer= s to Windows2008-x64.iso which doesn't match any name from MSDN, what w= e have is: en_windows_server_2008_datacenter_enterprise_standard_x64_dvd_X14-26714= =2Eiso --=20 Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh@us.ibm.com