From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Campbell Subject: Re: [PATCH v2 0/4] raisin: introduce tests Date: Wed, 6 May 2015 15:46:33 +0100 Message-ID: <1430923593.2660.304.camel@citrix.com> References: <1430827167.2660.64.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Stefano Stabellini Cc: George Dunlap , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org On Wed, 2015-05-06 at 15:39 +0100, Stefano Stabellini wrote: > On Wed, 6 May 2015, George Dunlap wrote: > > On Tue, May 5, 2015 at 2:55 PM, Stefano Stabellini > > wrote: > > > On Tue, 5 May 2015, Ian Campbell wrote: > > >> On Fri, 2015-05-01 at 16:48 +0100, Stefano Stabellini wrote: > > >> > Hi all, > > >> > > > >> > this patch series introduces a framework to execute simple unit and > > >> > functional tests in raisin. It can be used by developers to validate > > >> > their changes before submitting a patch series to xen-devel. It can also > > >> > be used by OSSTest to test for regressions on one particular > > >> > functionality. This patch series only introduces two tests: a PV guest > > >> > creation test and an HVM guest creation test. They are both based on > > >> > busybox. More tests will follow. > > >> > > >> What is the intended scope of these tests? e.g. when should a test be > > >> added here rather than to osstest? > > > > > > Small functional tests that can be easily run on a single host, without > > > requiring a specific host or guest operating system. Tests that every > > > developer should run on their test machine before submitting a patch > > > series. > > > > So a sort of BVT (build verification test), such that that we could be > > reasonably annoyed at if an experienced developer submitted a patch > > that broke said functionality? > > That's right. > > In general any functionality tests, that can be added without > introducing too much complexity, should be in raisin. osstest will be > able to run these tests via raisin. I think if it is intended as a BVT which I'll get shouted out for not having run then it needs to be time bound as much as functionality bound. Ian. > More complex tests, involving more than one host, or specific guest/host > OS combinations, should only be in osstest.