From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mason Date: Mon, 25 Apr 2016 14:46:04 +0200 Subject: [LTP] Issue with the reporting of failed tests In-Reply-To: <571E0226.3020104@free.fr> References: <5718D4CA.5010607@free.fr> <5718E81B.6060101@free.fr> <20160421153041.GA25307@rei> <5718F4EE.2060009@free.fr> <571907A2.10208@free.fr> <571A274A.60707@free.fr> <571DCF8D.30306@free.fr> <571E0226.3020104@free.fr> Message-ID: <571E118C.1080707@free.fr> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ltp@lists.linux.it On 25/04/2016 13:40, Mason wrote: > On 25/04/2016 10:04, Mason wrote: >> On 22/04/2016 15:29, Mason wrote: >>> On 21/04/2016 19:02, Mason wrote: >>>> On 21/04/2016 17:42, Mason wrote: >>>>> On 21/04/2016 17:30, Cyril Hrubis wrote: >>>>>> Hi! >>>>>>> Calling fflush does appear to solve the problem: >>>>>>> >>>>>>> # grep -c FAIL result-log.5160 >>>>>>> 38 >>>>>>> >>>>>>> # wc -l result-failed.5160 >>>>>>> 38 result-failed.5160 >>>>>>> >>>>>>> Does this mean no one ever reads FAILCMDFILE?! >>>>>> >>>>>> Just to have complete image here. I've discussed this a bit more at #ltp >>>>>> irc channel and the reason why we do not see this bug often is that the >>>>>> exec() that is done by the child after pan forks must fail (for example >>>>>> since bash is missing) then the code proceeds to exit(errno) line which >>>>>> flushes the buffers on open files... >>>>> >>>>> Note: in my latest LTP run, I disabled memory overcommit, and pan >>>>> failed to launch several tests: >>>>> >>>>> pan(5163): fork failed (tag proc01). errno:12 Cannot allocate memory >>>>> pan(5163): fork failed (tag fs_racer). errno:12 Cannot allocate memory >>>>> pan(5163): fork failed (tag quota_remount_test01). errno:12 Cannot allocate memory >>>>> pan(5163): fork failed (tag isofs). errno:12 Cannot allocate memory >>>>> pan(5163): fork failed (tag fs_perms01). errno:12 Cannot allocate memory >>>>> [747 similar lines] >>>>> >>>>> /tmp is a ramfs. Maybe something fills it up... >>>> >>>> With overcommit enabled, all tests complete, but some problems remain: >>>> >>>> $ grep -c FAIL result-log.24583 >>>> 85 >>>> >>>> $ wc -l result-failed.24583 >>>> 157 result-failed.24583 >>>> >>>> $ sort result-failed.24583 | uniq | wc -l >>>> 83 >>>> >>>> I'm confused as to what else is causing this... >>> >>> I am now testing yet another LTP run with the following patch applied. >> >> There still is an inconsistency between result-log and result-failed. >> >> $ grep -c FAIL result-log.30032 >> 119 >> >> $ wc -l result-failed.30032 >> 319 result-failed.30032 >> >> $ sort result-failed.30032 | uniq | wc -l >> 117 >> >> Either I didn't use the modified ltp-pan binary, or the change has no effect. >> (I will print something to stderr to make sure.) > > I do see an improvement now: no more duplicates. However, it seems > 2 tests are missing from result-failed: aio01, aio02. These two tests are not being built. FILTER_OUT_DIRS := $(if $(AIO_LIBS),,aio ltp-aiodio) so pan fails to launch them, obviously. pan(1130): execvp of 'aio01' (tag aio01) failed. errno:2 No such file or directory pan(1130): execvp of 'aio_tio' (tag aio02) failed. errno:2 No such file or directory I don't think these should be considered failures? Regards.