public inbox for ltp@lists.linux.it
 help / color / mirror / Atom feed
* Re: [LTP] Reg: Script to extract description of test case from LTPROOT/doc/testcases/*.txt files
@ 2009-06-06 13:09 CAI Qian
  0 siblings, 0 replies; 10+ messages in thread
From: CAI Qian @ 2009-06-06 13:09 UTC (permalink / raw)
  To: ltp-list




--- On Tue, 5/26/09, Subrata Modak <subrata@linux.vnet.ibm.com> wrote:

> From: Subrata Modak <subrata@linux.vnet.ibm.com>
> Subject: Re: Reg: Script to extract description of test case from LTPROOT/doc/testcases/*.txt files
> To: "rohit verma" <rohit.170309@gmail.com>
> Cc: "CAI Qian" <caiqian@cclom.cn>, "ltp-list" <ltp-list@lists.sf.net>, "Mike Frysinger" <vapier@gentoo.org>, "Masatake YAMATO" <yamato@redhat.com>
> Date: Tuesday, May 26, 2009, 8:22 PM
> On Mon, 2009-05-25 at 21:40 -0700,
> rohit verma wrote: 
> > Hi Subrata,
> > 
> > I request you to finalize the format of the
> documentation ASAP.
> 
> I would like to see something like:
> 
> <testname=abort(3)>
>   <test_description="Basic test for abort(3)">
>   <\test_description>
>  
> <test_location="ltp/testcases/kernel/syscalls/abort/abort01.c">
>   <\test_location>
>   <test_cmdfile_name="ltp/runtest/syscalls">
>   <\test_cmdfile_name>
>   <test_cmdfile_entry="abort01 ulimit -c
> 1024;abort01">
>   <\test_cmdfile_entry>
>   <Execute_through_commandline="ulimit -c
> 1024; ./ltp/testcases/bin/abort01">
>   <\Execute_through_commandline>
>   <Execute_through_runltp="./runltp -s
> abort01">
>   <\Execute_through_runltp>
> <\testname>
> 

I feel this is a little bit hard to maintain unless we can generate this
automatically from the code and relevent runtest files.

CAI Qian


------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

^ permalink raw reply	[flat|nested] 10+ messages in thread
* Re: [LTP] Reg: Script to extract description of test case from LTPROOT/doc/testcases/*.txt files
@ 2009-06-06 13:44 CAI Qian
  0 siblings, 0 replies; 10+ messages in thread
From: CAI Qian @ 2009-06-06 13:44 UTC (permalink / raw)
  To: ltp-list




--- On Mon, 5/18/09, rohit verma <rohit.170309@gmail.com> wrote:

> From: rohit verma <rohit.170309@gmail.com>
> Subject: Re: Reg: Script to extract description of test case from  LTPROOT/doc/testcases/*.txt files
> To: subrata@linux.vnet.ibm.com
> Cc: "CAI Qian" <caiqian@cclom.cn>
> Date: Monday, May 18, 2009, 7:55 PM
> Dear Subrata/Qian,
> 
> I am trying to develop script which would extracts
> description information about user required test case from
> LTPROOT/doc/testcases/ folder. 
> 
> So to make the description text file  as standard file,
> can we represent the description as mentioned below.
>  
> For example in syscalls category:
>  
> Currently available format of description in LTP as
> follows.
>  
>   abort01
>    Basic test for abort(3).
> 
>   accept01  
>    Verify that accept() returns the proper errno for
> various failure cases
>  
> 
> To make it in consistent way, description should be
> enclosed by executables as follows.
> 
>  <abort01>
>    Basic test for abort(3).
>  <\abort01>
>  
>  <accept01>  
>    Verify that accept() returns the proper errno for
> various failure cases
>  <\accept01>
>  
> Please confirm if this format is O.K or suggest any
> other format by which we can easily parse the test cases and
> extracts the description.
> 
> If this format is O.K , I would send the patch for those
> text files
>  

Another thing it that, your approach is not really document the 
description of test cases but test code (the build blocks). For example, 
in runtest/fs has the following test cases,

gf01 growfiles -W gf01 -b -e 1 -u -i 0 -L 20 -w -C 1 -l -I r -T 10 glseek20 glseek20.2
gf02 growfiles -W gf02 -b -e 1 -L 10 -i 100 -I p -S 2 -u -f gf03_
gf03 growfiles -W gf03 -b -e 1 -g 1 -i 1 -S 150 -u -f gf05_
gf04 growfiles -W gf04 -b -e 1 -g 4090 -i 500 -t 39000 -u -f gf06_
gf05 growfiles -W gf05 -b -e 1 -g 5000 -i 500 -t 49900 -T10 -c9 -I p -u -f gf07_
gf06 growfiles -W gf06 -b -e 1 -u -r 1-5000 -R 0--1 -i 0 -L 30 -C 1 g_rand10 g_rand10.2
gf07 growfiles -W gf07 -b -e 1 -u -r 1-5000 -R 0--2 -i 0 -L 30 -C 1 -I p g_rand13 g_rand13.2
gf08 growfiles -W gf08 -b -e 1 -u -r 1-5000 -R 0--2 -i 0 -L 30 -C 1 g_rand11 g_rand11.2
gf09 growfiles -W gf09 -b -e 1 -u -r 1-5000 -R 0--1 -i 0 -L 30 -C 1 -I p g_rand12 g_rand12.2
gf10 growfiles -W gf10 -b -e 1 -u -r 1-5000 -i 0 -L 30 -C 1 -I l g_lio14 g_lio14.2
gf11 growfiles -W gf11 -b -e 1 -u -r 1-5000 -i 0 -L 30 -C 1 -I L g_lio15 g_lio15.2
gf12 mkfifo gffifo17; growfiles -b -W gf12 -e 1 -u -i 0 -L 30 gffifo17

They all utilizes growfiles.c to perform different test scenarios. How can
we make sure we do document what those tests are actually trying to test?
If you really want to document what the test cases claiming to test, the 
more proper places to look at would be based on LTPROOT/runtest/* files.
For example, we will have something in LTPROOT/doc/testcases/fs like,

<gf11>...</gf11>

The description of a test case might not reside in test code itself,
because the test code has no view of how users are going to test different
scenarios. For example, mount.c might be used to test ext3, ext4 or
loopback filesystems. Therefore, The existing test cases descriptions might
be found from the mailing list when they were introduced.

If we have such documents about real test cases in place, it would be much
easier to maintain the tests, since we have the requirements. In addition,
We can fight back the complains about LTP that users have no visible view
what each test cases is claiming to test, and its coverage.

CAI Qian


------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2009-06-23 15:12 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <d51f89b0905130216h36e32efbwf48f241f2d961a0a@mail.gmail.com>
     [not found] ` <20090513.225510.104053150.caiqian@cclom.cn>
     [not found]   ` <d51f89b0905180455w514b014boedb4670610987533@mail.gmail.com>
     [not found]     ` <20090518.192133.233689723.caiqian@cclom.cn>
2009-05-19 10:31       ` [LTP] Reg: Script to extract description of test case from LTPROOT/doc/testcases/*.txt files Subrata Modak
2009-05-26  4:40         ` rohit verma
2009-05-26 12:22           ` Subrata Modak
2009-05-27  7:18             ` rohit verma
2009-05-27 14:23               ` Subrata Modak
2009-06-04  9:27                 ` Subrata Modak
     [not found]                   ` <d51f89b0906040312y69332585m1566cd8f922b62cf@mail.gmail.com>
     [not found]                     ` <1244112148.4879.19.camel@subratamodak.linux.ibm.com>
     [not found]                       ` <d51f89b0906150141j21908ba5s377289d0db51fa98@mail.gmail.com>
     [not found]                         ` <1245346284.5041.31.camel@subratamodak.linux.ibm.com>
2009-06-19  6:30                           ` rohit verma
2009-06-23 14:50                             ` Subrata Modak
2009-06-06 13:09 CAI Qian
  -- strict thread matches above, loose matches on Subject: below --
2009-06-06 13:44 CAI Qian

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox