From mboxrd@z Thu Jan 1 00:00:00 1970 From: Cyril Hrubis Date: Wed, 12 Apr 2017 11:07:06 +0200 Subject: [LTP] q-rig: qemu ltprun on armhf and arm64 In-Reply-To: <20170411150820.GA31990@localhost> References: <20170411150820.GA31990@localhost> Message-ID: <20170412090705.GC12732@rei> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ltp@lists.linux.it Hi! > I've been toying with Travis CI, qemu and LTP recently. My > effort is here: https://github.com/q-rig. It runs LTP on top > of the latest linux -rc on Debian sid, inside qemu on armhf and > arm64. Binaries are built when needed. You can find them here: > https://github.com/q-rig/qemu-ltprun-results. > > Activities are run in Travis CI [0]. LTP tests are split so to finish > in less than 49 minutes (Travis CI constraint). The LPT results are > put again in https://github.com/q-rig/qemu-ltprun-results, which is > probably the wrong place. Interesting work, although I'm not sure that travis is right tool for the job. On the other hand I've been thinking of similar misuse as well... > Now I'd like to tackle the aggregation of the data generated by LTP. The > best I could do is just generating also the .html page and upload it > to github [1] but I think more is actually desirable here. I would like to be able to do more with the LTP results as well, for instance compare two testruns for the purpose of automatic pre-release testing which would allow us to do much shorter release cycles. Unfortunately the best the legacy LTP runltp script + ltp-pan can do is to write a list of failed and skipped tests, it also can't run the open posix testcases, cannot kill tests on timeout, etc. This are the reasons I've started to experiment with writing a new test runner that would allow for writing different output formats among others some kind of machine parseable output (I was thinking of json but anything structured would do) which would allow us to produce much more interesting output than the current html page. Proof of concet is done in my private GitHub repo[1], but it's not complete enough to be usable and it can produce only html[2] page as an output so far, but hacking in json formatter would be fairly easy. [1] https://github.com/metan-ucw/ltp https://github.com/metan-ucw/ltp/tree/master/tools/runltp-ng [2] http://metan.ucw.cz/outgoing/ltp-openposix.html http://metan.ucw.cz/outgoing/ltp-syscalls.html The latest version also includes hardware and sofware version but apart from that it's more or less it. > Is there any documentation I'm expected to read before asking for > support here? :) There is probably none or close to that hence asking here is the best course of action. -- Cyril Hrubis chrubis@suse.cz