From mboxrd@z Thu Jan 1 00:00:00 1970 From: Heiko Schocher Date: Tue, 19 Jan 2016 12:42:14 +0100 Subject: [U-Boot] U-Boot: using tbot for U-Boot tests was: [PATCH V2 1/7] test/py: Implement pytest infrastructure In-Reply-To: References: <1449094708-14784-1-git-send-email-swarren@wwwdotorg.org> <56685784.9040907@wwwdotorg.org> <56717F09.8020407@monstr.eu> <567190E2.5010308@wwwdotorg.org> <56724BFB.5070502@denx.de> <5699E345.7000002@denx.de> Message-ID: <569E2116.3050302@denx.de> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de Hello Simon, Am 19.01.2016 um 04:42 schrieb Simon Glass: > Hi Heiko, > > On 15 January 2016 at 23:29, Heiko Schocher wrote: >> Hello Simon, >> >> Am 15.01.2016 um 00:12 schrieb Simon Glass: >>> >>> Hi Heiko, >>> >>> On 16 December 2015 at 22:45, Heiko Schocher wrote: >>>> >>>> Hello Stephen, >>>> >>>> >>>> Am 16.12.2015 um 17:27 schrieb Stephen Warren: >>>>> >>>>> >>>>> On 12/16/2015 08:11 AM, Michal Simek wrote: >>>>>> >>>>>> >>>>>> On 9.12.2015 17:32, Stephen Warren wrote: >>>>>>> >>>>>>> >>>>>>> On 12/02/2015 03:18 PM, Stephen Warren wrote: >> >> [...] >>>>> >>>>> In my current setup I leave the board on all the time (or rather, >>>>> manually >>>>> turn on the power when >>>>> I'm about to run the tests). Automating control of the power source is a >>>>> step I'll take later. >>>> >>>> >>>> >>>> Maybe you give tbot (I mentioned it in this thread) a chance? >>>> >>>> There, this things are automated, also you can do linux (and other >>>> console >>>> based) >>>> tests ... Currently added a testcase [1] which adds patches from >>>> patchwork, >>>> which are in a users ToDo list, to a git tree! In this case it is a >>>> u-boot >>>> git >>>> tree ... checks them with checkpatch, compiles it, and tries the new >>>> image >>>> on >>>> the board and calls testcases ... fully automated in a now weekly build >>>> [2] >>>> ... >>>> (only weekly, but thats a setup parameter in buildbot, as I have tbot and >>>> buildbot >>>> running on a raspberry pi) >>>> and don;t forget, I have the board not where I run tbot, the boards are >>>> ~1000km >>>> from me .. So, it is possible to add a U-Boot Testsetup on a server, >>>> and test boards all over the world ... >>> >>> >>> This sounds like a great development. >> >> >> Thanks! >> >>> How can we get this so that it can be used by U-Boot people? Do you >>> think you could add a README to the mainline, or some scripts to aid >>> setting it up? I would be interested in setting up a few boards that >>> run continuous testing, and I suspect others would also if it were >>> easier. >> >> >> Yes, good idea! >> >> I think about preparing ASAP a patch for U-Boot, creating: >> u-boot:/tools/tbot/README -> common infos about tbot >> u-boot:/tools/tbot/README.install -> HowTo install / using it >> u-boot:/tools/tbot/README.create_a_testcase >> u-boot:/tools/tbot/README-ToDo -> my current list of ToDo >> >> Is this Ok? > > Sounds great. Ok, sorry, I had no time yet, to making it ... >> As a motivation for using tbot ;-) I just created a video from tbot >> running the smartweb testcase: >> https://github.com/hsdenx/tbot/blob/master/src/tc/tc_board_smartweb.py >> >> https://www.youtube.com/watch?v=ZwUA0QNDnP4 > > OK that helps explain it. It looks like it uses DFU to write new > images to the boards - is that right? No, the board gets the new images through tftp ... I only test U-Boot DFU functionality on this board. >> Keep in mind, that I run tbot for the video here on my laptop at my home >> in hungary, the smartweb board is in munich/germany. >> >> I use this testcase also in my weekly buildbot setup on my raspberry pi [2]. >> Tbot logs (not only U-Boot also linux tests) can be found there for >> interested people >> (Remark: Wolfgang said, the logs are unreadable, because filled with a >> lot of unnecessary developer output, Yes, he is correct!! I have on my >> ToDo list, to add a new logging level, where only board input/output is >> printed ... the loglevel tbot uses is definded in the board config file >> https://github.com/hsdenx/tbot/blob/master/tbot_smartweb.cfg#L13 ) > > Yes that's my main comment. Yes, on my todo list ... BTW: is the console output in the video helpful? >> What is done in the smartweb testcase: >> - rm old u-boot code, checkout current U-Boot master >> @ 00:14 >> - set a toolchain >> @ 00:18 >> - get all patchwork patches from my patchwork ToDo list >> calling testcase: >> >> https://github.com/hsdenx/tbot/blob/master/src/tc/tc_workfd_get_patchwork_number_list.py >> @ 00:18 >> - adding some patchwork patches, I have in a python list >> (This list is setup in the board config file: >> https://github.com/hsdenx/tbot/blob/master/tbot_smartweb.cfg#L34 >> (Heiko speculating: it would be nice if tbot removes from >> this list a patchworkpatchnumber, if it detects, that this patch >> is already now in mainline ... ) >> - apply local patches (If there are) >> - apply the patchwork patches >> >> https://github.com/hsdenx/tbot/blob/master/src/tc/tc_workfd_apply_patchwork_patches.py >> currently not failing, when checkpatch detects errors/warnings >> @ 00:59 >> - compile U-Boot >> @ 02:35 >> - update SPL on the board >> @ 03:26 >> - update U-Boot on the board >> @ 03:55 >> - start DFU testcase on the board >> @ 04:13 >> https://github.com/hsdenx/tbot/blob/master/src/tc/tc_ub_dfu.py >> >> This testcase starts the "dfu" U-Boot command, which waits until Ctrl-C >> is pressed >> Then I start on the lab PC the Userspace tool "dfu-util" which >> communicates >> over USB with the smartweb board ... and do some dfu up and downloads. >> >> - at the end save the SPL and U-Boot bins, so I always have the lastest >> working bins [1] >> >> - power off the board >> >> bye, >> Heiko > > Well I'll await your patches to the list and then see if I can get it > working. What sort of hardware do I need for the power / reset > control? Ok. You can use what you want for powering on/off the board! If you can power on/off the board through a linux shell command, you are ready for tbot ... Also you need a shell command for getting the current power state (on or off) And you need to tell tbot, how you get access to the boards console ... I seperated this tasks in testcases, so you need to write for this tasks a tbot testcase. If you want to access the serial console with kermit, this should work already ... But let me try to find time for writing a patch ... bye, Heiko -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany