From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by yocto-www.yoctoproject.org (Postfix, from userid 118) id B0D4DE0045F; Thu, 3 Jul 2014 09:07:52 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on yocto-www.yoctoproject.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-HAM-Report: * -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low * trust * [209.85.223.180 listed in list.dnswl.org] * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 80F32E00404 for ; Thu, 3 Jul 2014 09:07:39 -0700 (PDT) Received: by mail-ie0-f180.google.com with SMTP id rp18so362634iec.25 for ; Thu, 03 Jul 2014 09:07:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=rhhAhOQxLLhr4Jo+JSeiqkti7na2L/JidSye2AMCfOw=; b=JB/FEVXIeYQjIr7XiBFsWIcZoRe3sSPLaq1Mow08h3+jKCRDoUeK4/q60E2mVsjkxQ Yo+mk1R+lR7Bn1DJaEPM4Bf0TvGUUp5/67SiRZmKSDkscwy8sNk/qofYeyBqERUxuFgc EnJnt91NU62YmiVULpkGsk2vzqDnYfU22aafVKgDektUmLs2TzzEvunpGb9xiDlHXQjm ulyhegJsR1/pF/YoZfSU8TBBVLIKGXl3kYVJMvTIUEcRJUSjumXblTO24kEb9u1nMMel TP6NyQ9gTWo1h7a944//1cnWR3+tOXJ8s6aBNOFXhraR7Kmhw3XX9xabw6q8yuFcsIjG B80A== X-Gm-Message-State: ALoCoQl0pIKBPKmf5OVJQ95MpzF1o/jM8Un2B+NfnaGDK+GlnQQzQuIY6tiK34YwgRgXObjP4K4w X-Received: by 10.50.27.4 with SMTP id p4mr12362872igg.22.1404403659107; Thu, 03 Jul 2014 09:07:39 -0700 (PDT) Received: from [192.168.141.83] (69-165-220-158.dsl.teksavvy.com. [69.165.220.158]) by mx.google.com with ESMTPSA id n10sm53942091igv.21.2014.07.03.09.07.38 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 03 Jul 2014 09:07:38 -0700 (PDT) Message-ID: <53B57FC9.3050807@linaro.org> Date: Thu, 03 Jul 2014 12:07:37 -0400 From: Trevor Woerner User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Stephano Cetola , meta-freescale@yoctoproject.org References: In-Reply-To: Subject: Re: fsl-community-bsp: signing up for testing X-BeenThere: meta-freescale@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Usage and development list for the meta-fsl-* layers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Jul 2014 16:07:52 -0000 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 07/03/14 11:52, Stephano Cetola wrote: ... All excellent points! In my opinion, the testing needs to be split into two phases: 1) help/guidelines/docs on testing 2) reporting I think we can agree that Daiane has done an excellent job on the reporting side: to create forms to collect and aggregate the test result information. I think where we could use more work (or maybe this already exists, perhaps I'm just not aware of it?) is on the "how to test" side. If we had a set of procedures that were so "dumbed down" that a person with little OE experience could follow them, perhaps it would be easier to get more members of the community involved in testing? Or, even better than documentation, what if we had build targets that were functional and ready to run "out of the box" for various tests? Why isn't it good enough to simply run "bitbake core-image-weston" or "fsl-image-multimedia"? Why are tweaks always required to local.conf (i.e. DISTRO_FEATURES, EXTRA_IMAGES, PREFERRED_PROVIDER)? If the multimedia target _always_ requires the gstreamer packages to be added, shouldn't that happen automatically when someone builds that target? We should probably also look into those ptests to see if they can't help with some of this testing/reporting.