From mboxrd@z Thu Jan 1 00:00:00 1970 From: fixed-term.Oleksij.Rempel Date: Mon, 18 Jul 2016 14:13:49 +0200 Subject: [LTP] [PATCH] consolidate 4 TCs message_queue_test_02_* into message_queue_test_02.sh to avoid fail in case of random execution In-Reply-To: <20160718115932.GB18221@rei.suse.cz> References: <1468576023-32754-1-git-send-email-nga.hoangthithuy@vn.bosch.com> <20160715095531.GA3268@rei.scz.novell.com> <4ab8990043b24c1786a12b9dc8dfff3c@FE-MBX1024.de.bosch.com> <20160718115932.GB18221@rei.suse.cz> Message-ID: <578CC7FD.4030305@de.bosch.com> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ltp@lists.linux.it On 18.07.2016 13:59, Cyril Hrubis wrote: > Hi! >>>> sem02 sem02 >>>> >>>> message_queue_test_01 message_queue_test_01 >>>> -message_queue_test_02_get message_queue_test_02_get >>>> -message_queue_test_02_snd message_queue_test_02_snd >>>> -message_queue_test_02_rcv message_queue_test_02_rcv >>>> -message_queue_test_02_ctl message_queue_test_02_ctl -r >>>> +message_queue_test_02 message_queue_test_02.sh >>>> message_queue_test_04 message_queue_test_04 >>>> message_queue_test_05 message_queue_test_05 >>>> pipe_test_01 pipe_test_01 >>> >>> This is exactly against our guidelines, see: >>> >>> https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#211-runtest- >>> files >> >> What exactly do you mean? > > What I mean and is written in the guidelines is: > > The runtest files should have one entry per a test. Creating a wrapper > that runs all your tests and adding it as a single test into runtest > file is strongly discouraged. > >> To avoid confusions I need to explain background of this patch: We >> working on remote execution of LTPs on the target. It means, some >> script is taking test list and execute each of entry over ssh. In >> this case we are able to track resets and continue testing after it. >> >> So, now we found that some resets are not actially triggered by some >> specific test. So we decided to randomize test order. This allowed us >> to find some more bugs on the system, but introduced issues with tests >> which depend on each other. For example message_queue_test_02_* can be >> executed only in some specific order. > > Ah, so they depend on each other. That is a valid reason for executing > them in a defined order. > > You should have written better patch description since this is not > exactly clear. Yea, we are learning to work with upstream directly :) It will take some time until the patches will be good from first try. >> What would be the proper way to solve this issue? > > Looking at the code we should probably avoid running these testcasese in > ipc runtest file, since they were designed to be executed in a loop to > stress the target system. They does not seem very useful when they are > executed just once as from the runtest file. Hmm.. so, they can be removed for now? Or there is some other place for this kind of tests? How about cpuhotplug* tests? Some times they fail, but since they are executed only once, i would expect bad reproducibility.