* [Ocfs2-devel] ocfs2-test for v4.3 done
@ 2015-12-21 3:00 Junxiao Bi
2015-12-21 13:50 ` Eric Ren
0 siblings, 1 reply; 9+ messages in thread
From: Junxiao Bi @ 2015-12-21 3:00 UTC (permalink / raw)
To: ocfs2-devel
Hi,
I have run a full ocfs2-test(single/multiple/discontig) to v4.3 mainline
kernel. The following three issues are found. The first two are
regression bugs.
1. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1357
Fix merged. Upstream commit 087ffd4eae99("jbd2: fix null committed
data return in undo_access")
2. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1358
A recursive locking issue, discussing fix in ocfs2-devel.
3. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1359
o2hb-increase-unsteady-iterations.patch to fix this issue, merged by
-mm tree
I have setup a test env to build and auto do ocfs2 test. With it, Ocfs2
for mainline and linux-next will be test regularly, the test status and
bugs will be reported to ocfs2-devel. Feel free to take any bug if you
are interested, it will be a good start point with ocfs2. Hope this can
catch regression bugs earlier before merged by mainline.
For the test cases, now only ocfs2-test is supported. Ltp is going to be
merged in the future. If any other test cases or benchmark useful to
ocfs2 test, please let me know.
Thanks,
Junxiao.
^ permalink raw reply [flat|nested] 9+ messages in thread* [Ocfs2-devel] ocfs2-test for v4.3 done 2015-12-21 3:00 [Ocfs2-devel] ocfs2-test for v4.3 done Junxiao Bi @ 2015-12-21 13:50 ` Eric Ren 2015-12-22 2:34 ` Junxiao Bi 0 siblings, 1 reply; 9+ messages in thread From: Eric Ren @ 2015-12-21 13:50 UTC (permalink / raw) To: ocfs2-devel Hi Junxiao, Thanks for your sharing, and very appreciated your efforts. On Mon, Dec 21, 2015 at 11:00:59AM +0800, Junxiao Bi wrote: > Hi, > > I have run a full ocfs2-test(single/multiple/discontig) to v4.3 mainline > kernel. The following three issues are found. The first two are > regression bugs. > > 1. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1357 > Fix merged. Upstream commit 087ffd4eae99("jbd2: fix null committed > data return in undo_access") > > 2. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1358 > A recursive locking issue, discussing fix in ocfs2-devel. > > 3. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1359 > o2hb-increase-unsteady-iterations.patch to fix this issue, merged by > -mm tree > > I have setup a test env to build and auto do ocfs2 test. With it, Ocfs2 > for mainline and linux-next will be test regularly, the test status and > bugs will be reported to ocfs2-devel. Feel free to take any bug if you > are interested, it will be a good start point with ocfs2. Hope this can > catch regression bugs earlier before merged by mainline. I'm very interested in how your testing env works, could you briefly introduce it? > > For the test cases, now only ocfs2-test is supported. Ltp is going to be > merged in the future. If any other test cases or benchmark useful to > ocfs2 test, please let me know. Btw, recently, I did a performance testing on both ocfs2 and gfs2 using iozone. The testing has 3 different cases: 1. local mode: ocfs2 better than gfs2 2. two files mutiple nodes 3. same file mutiple nodes, for 2 and 3 gfs2 better than ocfs2 generally Just provide a reference here. I tried to make it as simple as possible, leaving all formating and mount options as default, and with fixed-size file(2GB) and buffer(4kb). Did you or anyone ever test performance for ocfs2? maybe we can discuss further on this topic in future;-) Thanks, Eric > > Thanks, > Junxiao. > > _______________________________________________ > Ocfs2-devel mailing list > Ocfs2-devel at oss.oracle.com > https://oss.oracle.com/mailman/listinfo/ocfs2-devel > ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2015-12-21 13:50 ` Eric Ren @ 2015-12-22 2:34 ` Junxiao Bi 2015-12-22 7:39 ` Eric Ren 2016-02-16 9:54 ` Eric Ren 0 siblings, 2 replies; 9+ messages in thread From: Junxiao Bi @ 2015-12-22 2:34 UTC (permalink / raw) To: ocfs2-devel On 12/21/2015 09:50 PM, Eric Ren wrote: > Hi Junxiao, > > Thanks for your sharing, and very appreciated your efforts. > > On Mon, Dec 21, 2015 at 11:00:59AM +0800, Junxiao Bi wrote: >> Hi, >> >> I have run a full ocfs2-test(single/multiple/discontig) to v4.3 mainline >> kernel. The following three issues are found. The first two are >> regression bugs. >> >> 1. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1357 >> Fix merged. Upstream commit 087ffd4eae99("jbd2: fix null committed >> data return in undo_access") >> >> 2. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1358 >> A recursive locking issue, discussing fix in ocfs2-devel. >> >> 3. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1359 >> o2hb-increase-unsteady-iterations.patch to fix this issue, merged by >> -mm tree >> >> I have setup a test env to build and auto do ocfs2 test. With it, Ocfs2 >> for mainline and linux-next will be test regularly, the test status and >> bugs will be reported to ocfs2-devel. Feel free to take any bug if you >> are interested, it will be a good start point with ocfs2. Hope this can >> catch regression bugs earlier before merged by mainline. > I'm very interested in how your testing env works, could you briefly introduce > it? Four vm are used, one for git server, and the other three to build kernel and run ocfs2 test. Tag a branch and push it to the git server, the test will be started. The test cases to run can be set in tag message. >> >> For the test cases, now only ocfs2-test is supported. Ltp is going to be >> merged in the future. If any other test cases or benchmark useful to >> ocfs2 test, please let me know. > Btw, recently, I did a performance testing on both ocfs2 and gfs2 using iozone. The > testing has 3 different cases: > 1. local mode: ocfs2 better than gfs2 > 2. two files mutiple nodes > 3. same file mutiple nodes, for 2 and 3 gfs2 better than ocfs2 generally Can you share the perf data, i am curious about the performance diff? > Just provide a reference here. I tried to make it as simple as possible, > leaving all formating and mount options as default, and with fixed-size file(2GB) > and buffer(4kb). > > Did you or anyone ever test performance for ocfs2? maybe we can discuss further on > this topic in future;-) Yes, a lot of works need to do for ocfs2 performance, to make local ocfs2 performance more close to ext4, and try best to make cluster ocfs2 performance close to local. Thanks, Junxiao. > > Thanks, > Eric >> >> Thanks, >> Junxiao. >> >> _______________________________________________ >> Ocfs2-devel mailing list >> Ocfs2-devel at oss.oracle.com >> https://oss.oracle.com/mailman/listinfo/ocfs2-devel >> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2015-12-22 2:34 ` Junxiao Bi @ 2015-12-22 7:39 ` Eric Ren 2016-02-16 9:54 ` Eric Ren 1 sibling, 0 replies; 9+ messages in thread From: Eric Ren @ 2015-12-22 7:39 UTC (permalink / raw) To: ocfs2-devel Hi, On Tue, Dec 22, 2015 at 10:34:20AM +0800, Junxiao Bi wrote: > On 12/21/2015 09:50 PM, Eric Ren wrote: > > Hi Junxiao, > > > > Thanks for your sharing, and very appreciated your efforts. > > > > On Mon, Dec 21, 2015 at 11:00:59AM +0800, Junxiao Bi wrote: > >> Hi, > >> > >> I have run a full ocfs2-test(single/multiple/discontig) to v4.3 mainline > >> kernel. The following three issues are found. The first two are > >> regression bugs. > >> > >> 1. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1357 > >> Fix merged. Upstream commit 087ffd4eae99("jbd2: fix null committed > >> data return in undo_access") > >> > >> 2. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1358 > >> A recursive locking issue, discussing fix in ocfs2-devel. > >> > >> 3. https://oss.oracle.com/bugzilla/show_bug.cgi?id=1359 > >> o2hb-increase-unsteady-iterations.patch to fix this issue, merged by > >> -mm tree > >> > >> I have setup a test env to build and auto do ocfs2 test. With it, Ocfs2 > >> for mainline and linux-next will be test regularly, the test status and > >> bugs will be reported to ocfs2-devel. Feel free to take any bug if you > >> are interested, it will be a good start point with ocfs2. Hope this can > >> catch regression bugs earlier before merged by mainline. > > I'm very interested in how your testing env works, could you briefly introduce > > it? > Four vm are used, one for git server, and the other three to build > kernel and run ocfs2 test. Tag a branch and push it to the git server, > the test will be started. The test cases to run can be set in tag message. > > >> > >> For the test cases, now only ocfs2-test is supported. Ltp is going to be > >> merged in the future. If any other test cases or benchmark useful to > >> ocfs2 test, please let me know. > > Btw, recently, I did a performance testing on both ocfs2 and gfs2 using iozone. The > > testing has 3 different cases: > > 1. local mode: ocfs2 better than gfs2 > > 2. two files mutiple nodes > > 3. same file mutiple nodes, for 2 and 3 gfs2 better than ocfs2 generally > Can you share the perf data, i am curious about the performance diff? Sure, but I cannot guarantee it's quality;-) I'd like to put some handled data here, and then attach the raw data. My env has two sle12 nodes(vm) with same configuration: 4GB RAM, 4 core, 50G iscsi shared disk. Poor equipment, isn't? hah 1. local mode #iozone -R -s 10G -r 4 -i 0 -i 1 -i 2 -f /mnt/shared/test -b output Write Re-write Read Re-read Random Read Random Write gfs2 62458 109542 104830 113967 8860 71114 ocfs2 79411 117789 123872 135462 9988 73653 2. two files mutiple nodes - use "pdsh" tools from working machine to start iozone on both nodes concurrently. node1 iozone on file1, node2 iozone on file2. Then add up the IO data and divide by 2, to get the average value. #iozone -R -s 2G -r 4 -i 0 -i 1 -i 2 -f /mnt/shared/file1 -b multi_two-files_${FS}-2G.wks Write Re-write Read Re-read Random Read Random Write GFS2 50347 51479 6079973.5 6215918.5 3885683.5 4619.5 OCFS2 38921.5 67632.5 4104709.5 3891825.5 2964093 5863.5 3. same file mutiple nodes - similar with 2th case, but both node1 and node2 iozone on the same file. Write Re-write Read Re-read Random Read Random Write GFS2 45337 26388 48032.5 3192007 1682927.5 45014 OCFS2 19428 404.5 86296 1888067.5 1399657.5 3226.5 Again, raw data attatched. > > Just provide a reference here. I tried to make it as simple as possible, > > leaving all formating and mount options as default, and with fixed-size file(2GB) > > and buffer(4kb). > > > > Did you or anyone ever test performance for ocfs2? maybe we can discuss further on > > this topic in future;-) > Yes, a lot of works need to do for ocfs2 performance, to make local > ocfs2 performance more close to ext4, and try best to make cluster ocfs2 > performance close to local. Let's do it one by one;-) Thanks, Eric > > Thanks, > Junxiao. > > > > Thanks, > > Eric > >> > >> Thanks, > >> Junxiao. > >> > >> _______________________________________________ > >> Ocfs2-devel mailing list > >> Ocfs2-devel at oss.oracle.com > >> https://oss.oracle.com/mailman/listinfo/ocfs2-devel > >> > > > _______________________________________________ > Ocfs2-devel mailing list > Ocfs2-devel at oss.oracle.com > https://oss.oracle.com/mailman/listinfo/ocfs2-devel > -------------- next part -------------- A non-text attachment was scrubbed... Name: ocfs2-vs-gfs2-performace-testing-data.gz Type: application/x-gzip Size: 57925 bytes Desc: not available Url : http://oss.oracle.com/pipermail/ocfs2-devel/attachments/20151222/5e926fd8/attachment-0001.gz ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2015-12-22 2:34 ` Junxiao Bi 2015-12-22 7:39 ` Eric Ren @ 2016-02-16 9:54 ` Eric Ren 2016-02-17 2:15 ` Junxiao Bi 1 sibling, 1 reply; 9+ messages in thread From: Eric Ren @ 2016-02-16 9:54 UTC (permalink / raw) To: ocfs2-devel Hi Junxiao, > >> I have setup a test env to build and auto do ocfs2 test. With it, Ocfs2 > >> for mainline and linux-next will be test regularly, the test status and > >> bugs will be reported to ocfs2-devel. Feel free to take any bug if you > >> are interested, it will be a good start point with ocfs2. Hope this can > >> catch regression bugs earlier before merged by mainline. > > I'm very interested in how your testing env works, could you briefly introduce > > it? > Four vm are used, one for git server, and the other three to build > kernel and run ocfs2 test. Tag a branch and push it to the git server, > the test will be started. The test cases to run can be set in tag message. Recently, I get free days and also want to setup automatic testing env for ocfs2. I'll use pcmk as cluster stack while you probably use o2cb. I think we can complement each other. May I bother you to describe the work flow more specifically, or share experiences and references? That way it would save my life;-) Thanks! Eric ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2016-02-16 9:54 ` Eric Ren @ 2016-02-17 2:15 ` Junxiao Bi 2016-02-19 3:01 ` Eric Ren 0 siblings, 1 reply; 9+ messages in thread From: Junxiao Bi @ 2016-02-17 2:15 UTC (permalink / raw) To: ocfs2-devel Hi Eric, I remember i described it before, please search it on ocfs2-devel. For ocfs2 env setup, please refer to README in ocfs2-test. Thanks, Junxiao. On 02/16/2016 05:54 PM, Eric Ren wrote: > Hi Junxiao, > >>>> I have setup a test env to build and auto do ocfs2 test. With it, Ocfs2 >>>> for mainline and linux-next will be test regularly, the test status and >>>> bugs will be reported to ocfs2-devel. Feel free to take any bug if you >>>> are interested, it will be a good start point with ocfs2. Hope this can >>>> catch regression bugs earlier before merged by mainline. >>> I'm very interested in how your testing env works, could you briefly introduce >>> it? >> Four vm are used, one for git server, and the other three to build >> kernel and run ocfs2 test. Tag a branch and push it to the git server, >> the test will be started. The test cases to run can be set in tag message. > > Recently, I get free days and also want to setup automatic testing env for ocfs2. > I'll use pcmk as cluster stack while you probably use o2cb. I think we can complement > each other. May I bother you to describe the work flow more specifically, or share > experiences and references? That way it would save my life;-) > > Thanks! > Eric > ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2016-02-17 2:15 ` Junxiao Bi @ 2016-02-19 3:01 ` Eric Ren 2016-02-24 1:48 ` Junxiao Bi 0 siblings, 1 reply; 9+ messages in thread From: Eric Ren @ 2016-02-19 3:01 UTC (permalink / raw) To: ocfs2-devel Hi Junxiao, On Wed, Feb 17, 2016 at 10:15:56AM +0800, Junxiao Bi wrote: > Hi Eric, > > I remember i described it before, please search it on ocfs2-devel. For > ocfs2 env setup, please refer to README in ocfs2-test. Yes, you did. Actually, it's the quoted paragraph below;-) Thanks, but more things what I really want to learn about are, such as: 1. git hook/auto control scripts, if it's OK to share; 2. pain point and solution, for example, a latest tagged release kernel may not compile successfully by `make defconfig` or `cp /boot/config-`uname -r``; Or cannot boot up even if we've built kernel RPM and installed it. Did you have this problem? Any suggestion;-) What I can think of is to try opensuse tumbleweed distribution(a rolling release). > > On 02/16/2016 05:54 PM, Eric Ren wrote: > > Hi Junxiao, > > > >> Four vm are used, one for git server, and the other three to build > >> kernel and run ocfs2 test. Tag a branch and push it to the git server, > >> the test will be started. The test cases to run can be set in tag message. > > > > Recently, I get free days and also want to setup automatic testing env for ocfs2. > > I'll use pcmk as cluster stack while you probably use o2cb. I think we can complement > > each other. May I bother you to describe the work flow more specifically, or share > > experiences and references? That way it would save my life;-) > > Thanks! Eric ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2016-02-19 3:01 ` Eric Ren @ 2016-02-24 1:48 ` Junxiao Bi 2016-02-24 3:30 ` Eric Ren 0 siblings, 1 reply; 9+ messages in thread From: Junxiao Bi @ 2016-02-24 1:48 UTC (permalink / raw) To: ocfs2-devel Hi Eric, On 02/19/2016 11:01 AM, Eric Ren wrote: > Hi Junxiao, > > On Wed, Feb 17, 2016 at 10:15:56AM +0800, Junxiao Bi wrote: >> Hi Eric, >> >> I remember i described it before, please search it on ocfs2-devel. For >> ocfs2 env setup, please refer to README in ocfs2-test. > > Yes, you did. Actually, it's the quoted paragraph below;-) > Thanks, but more things what I really want to learn about are, such as: > 1. git hook/auto control scripts, if it's OK to share; Sure. Attached. > 2. pain point and solution, for example, a latest tagged release kernel may > not compile successfully by `make defconfig` or `cp /boot/config-`uname -r``; > Or cannot boot up even if we've built kernel RPM and installed it. That maybe because some config not enabled for your platform. I used xen vm as test nodes, and use make xenconfig to generate .config and it works well. There are two pain points for this test framework: 1. auto bisect to spot regression issue. 2. improve ocfs2-test speed. Now it needs several days to done the test. Better split the test as function test and stress test. Please think about it when build your test framework. Thanks, Junxiao. > > Did you have this problem? Any suggestion;-) What I can think of is to try opensuse > tumbleweed distribution(a rolling release). > >> >> On 02/16/2016 05:54 PM, Eric Ren wrote: >>> Hi Junxiao, >>> >>>> Four vm are used, one for git server, and the other three to build >>>> kernel and run ocfs2 test. Tag a branch and push it to the git server, >>>> the test will be started. The test cases to run can be set in tag message. >>> >>> Recently, I get free days and also want to setup automatic testing env for ocfs2. >>> I'll use pcmk as cluster stack while you probably use o2cb. I think we can complement >>> each other. May I bother you to describe the work flow more specifically, or share >>> experiences and references? That way it would save my life;-) >>> > Thanks! > Eric > -------------- next part -------------- #!/bin/sh # # An example hook script to prepare a packed repository for use over # dumb transports. # # To enable this hook, rename this file to "post-update". #exec git update-server-info function parse_tag() { tag=`git tag -n1 -l $tag_name | awk '{print $2}'` echo $tag | grep -sq $BUILD_TAG if [ $? -eq 0 ]; then command=1 fi echo $tag | grep -sq $TEST_TAG if [ $? -eq 0 ]; then echo $tag | grep -sq $TESTONLY_TAG if [ $? -eq 0 ]; then command=3 cases=`echo $tag | sed "s:^${TESTONLY_TAG}-::"` else command=2 cases=`echo $tag | sed "s:^${TEST_TAG}-::"` fi if [ X"$cases" = X"$tag" ];then cases="all" fi for cas in `echo $cases | sed "s:,: :g"`; do echo $USERDEF_TESTCASES " " $SINGLE_TESTCASES " " $MULTIPLE_TESTCASES | grep -sqw $cas if [ $? -ne 0 ]; then echo "error, testcase [${cas}] not supported." echo "The following testcases are supported:" echo "user defined testcases: $USERDEF_TESTCASES" echo "single testcases: $SINGLE_TESTCASES" echo "multiple testcases: $MULTIPLE_TESTCASES" exit 1 else testcase=${testcase}" "${cas} fi done if [ -z "$testcase" ]; then exit 1 fi fi if [ $command -eq 0 ]; then exit 0 elif [ $command -eq 1 ];then echo "command: build $version" elif [ $command -eq 2 ]; then echo "command: test $version" echo "testcase: $testcase" else echo "command: testonly" echo "testcase: $testcase" fi } function build_kernel() { echo "archive tag: $tag_name to ${version}.tar" git archive --prefix=${version}/ $tag_name > /tmp/${version}.tar echo "copy ${version}.tar to $BUILD_SERVER:$BUILD_PATH" scp /tmp/${version}.tar root@${BUILD_SERVER}:${BUILD_PATH} && rm -rf /tmp/${version}.tar echo "untar ${version}.tar on build server" $REMOTE_BUILD "tar xf ${BUILD_PATH}/${version}.tar -C ${BUILD_PATH} && rm -rf ${BUILD_PATH}/${version}.tar" echo "building, please wait" scp /repo/mainline.git/hooks/ocfs2.config root@${BUILD_SERVER}:/tmp $REMOTE_BUILD "cd $build_dir; make mrproper > /dev/null; \ make defconfig >> ${build_dir}/build.log; \ cat /tmp/ocfs2.config >> ${build_dir}/.config; \ make xenconfig >> ${build_dir}/build.log; \ make binrpm-pkg >> ${build_dir}/build.log;" } function install_kernel() { echo "copy kernel to test server $1:/tmp" scp /tmp/${version}.rpm root@$1:/tmp if [ $? -ne 0 ]; then echo "copy to server $1 failed." exit 1 fi echo "install kernel to test server $1." ret=`ssh root@$1 "rpm -ivh --force /tmp/${version}.rpm &> /dev/null; echo $?"` if [ $ret -ne 0 ];then echo "install kernel to $1 failed $ret." exit 1; fi # update grub to boot the new install kernel by default ssh root@$1 "rm -f /boot/grub/grub.conf.orig; \ mv /boot/grub/grub.conf /boot/grub/grub.conf.orig; \ sed 's/^default=.*$/default=0/' /boot/grub/grub.conf.orig > /boot/grub/grub.conf; \ rm -rf /tmp/${version}.rpm" } function exit_error() { if [ $1 -ne 0 ];then echo "ocfs2 $2 test case [ $3 ] failed." exit $1 fi } function run_single_testcase() { umount_testdev $test_dev $REMOTE_TEST "cd ${exe_dir}; $single_test -m $mntdir -l $logdir \ -d $test_dev -k $kernelsrc -t $1" exit_error $? "single" "$1" } function run_multiple_testcase() { umount_testdev $test_dev $REMOTE_TEST "cd ${exe_dir}; $multi_test -n ${TEST_SERVER_LIST} \ -d $test_dev -o $logdir -k $kernelsrc -t $1 $mntdir" exit_error $? "multiple" "$1" } function run_discontig_testcase() { umount_testdev $test_dev echo "discontig multi node test start." $REMOTE_TEST "cd ${exe_dir}; $discontig_test -m ${TEST_SERVER_LIST} \ -d $test_dev -o $logdir $mntdir" exit_error $? "discontig" "multiple" echo "discontig multi node test done." echo "discontig single node test start." $REMOTE_TEST "cd ${exe_dir}; $discontig_test -d $test_dev -o $logdir $mntdir" exit_error $? "discontig" "single" echo "discontig single node test done." } function umount_testdev() { ssh root@${ML_TEST_SERVER1} "mount | grep -sq $1 && umount $1 &> /dev/null" ssh root@${ML_TEST_SERVER2} "mount | grep -sq $1 && umount $1 &> /dev/null" ssh root@${ML_TEST_SERVER3} "mount | grep -sq $1 && umount $1 &> /dev/null" } BUILD_SERVER=mainline-test1.cn.oracle.com ML_TEST_SERVER1=mainline-test1.cn.oracle.com ML_TEST_SERVER2=mainline-test2.cn.oracle.com ML_TEST_SERVER3=mainline-test3.cn.oracle.com REMOTE_BUILD="ssh root@${BUILD_SERVER}" RPMBUILD_PATH=/root/rpmbuild BUILD_PATH=${RPMBUILD_PATH}/build BIN_PATH=${RPMBUILD_PATH}/rpms/x86_64 PROD=ML BUILD_TAG=${PROD}build TEST_TAG=${PROD}test TESTONLY_TAG=${TEST_TAG}only command=0 USERDEF_TESTCASES="all single multiple discontig" SINGLE_TESTCASES="all create_and_open directaio fillverifyholes renamewriterace aiostress\ filesizelimits mmaptruncate buildkernel splice sendfile mmap reserve_space inline xattr reflink mkfs tunefs backup_super" MULTIPLE_TESTCASES="all xattr inline reflink write_append_truncate multi_mmap create_racer flock_unit cross_delete open_delete lvb_torture" testcase="" # skip any non-tag push refname="$1" echo $refname | grep -sq 'tags' if [ $? -eq 0 ];then tag_name=${refname##refs/tags/} else exit 0 fi # this update is to remove tag, ignore tag=`git tag -l $tag_name` if [ -z "$tag" ]; then exit 0 fi echo pushing tag: $tag. version=`git tag -n1 -l $tag_name | awk '{print $3}'` # parse tag to see what to do, build, test or testonly? "command" and "testcase" will be set parse_tag # build and install kernel if command is not "testonly" if [ $command -ne 3 ]; then # build kernel build_dir=${BUILD_PATH}/${version} build_kernel # get kernel rpm package path from build log. bin=$($REMOTE_BUILD "tail -n 10 ${build_dir}/build.log | grep kernel-.*.rpm | grep -v kernel-headers | awk '{print \$2}'") if [ -z "$bin" ]; then echo "build fail" exit 1 fi echo "build success. get kernel at root@${BUILD_SERVER}:${bin}." if [ $command -eq 1 ];then exit 0 fi # install kernel scp root@${BUILD_SERVER}:$bin /tmp/${version}.rpm if [ $? -ne 0 ]; then echo "copy from server $BUILD_SERVER failed." exit 1 fi install_kernel ${ML_TEST_SERVER1} install_kernel ${ML_TEST_SERVER2} install_kernel ${ML_TEST_SERVER3} rm -rf /tmp/${version}.rpm echo "reboot test server to new kernel" ssh root@${ML_TEST_SERVER1} 'reboot' ssh root@${ML_TEST_SERVER2} 'reboot' ssh root@${ML_TEST_SERVER3} 'reboot' echo "wait 200s to reboot" sleep 200 fi exe_dir=/home/test/bin/ocfs2/bin/ single_test=./single_run-WIP.sh multi_test=./multiple_run.sh discontig_test=./discontig_runner.sh baselogdir=/home/test/ocfs2_test/log kernelsrc=/home/test/ocfs2_test/linux-kernel.tar.gz mntdir=/ocfs2_vol test_dev=/dev/sda REMOTE_TEST="ssh test@${ML_TEST_SERVER1}" TEST_SERVER_LIST="mainline-test1,mainline-test2,mainline-test3" logdir=${baselogdir}/${tag_name}-`date +%y-%m-%d-%H-%M` $REMOTE_TEST "if [ ! -d "$logdir" ]; then mkdir $logdir; fi" single_run=0 multiple_run=0 for cas in $testcase; do if [ "$cas"X = "multiple"X -o "$cas"X = "all"X ]; then echo "ocfs2 multi node test start." run_multiple_testcase "all" echo "ocfs2 multi node test done." multiple_run=1 fi if [ "$cas"X = "single"X -o "$cas"X = "all"X ]; then echo "ocfs2 single node test start." run_single_testcase "all" echo "ocfs2 single node test done." single_run=1 fi #if [ "$cas"X = "discontig"X -o "$cas"X = "all"X ]; then if [ "$cas"X = "discontig"X ]; then echo "ocfs2 discontig test start." run_discontig_testcase echo "ocfs2 discontig test done." fi if [ $multiple_run -eq 0 ]; then echo $MULTIPLE_TESTCASES | grep -sqw $cas if [ $? -eq 0 ]; then echo "ocfs2 multiple [${cas}] test start." run_multiple_testcase "$cas" echo "ocfs2 multiple [${cas}] test done." fi fi if [ $single_run -eq 0 ]; then echo $SINGLE_TESTCASES | grep -sqw $cas if [ $? -eq 0 ]; then echo "ocfs2 single [${cas}] test start." run_single_testcase "$cas" echo "ocfs2 single [${cas}] test done." fi fi done echo "test done, checking log at test@${ML_TEST_SERVER1}:${logdir}" ^ permalink raw reply [flat|nested] 9+ messages in thread
* [Ocfs2-devel] ocfs2-test for v4.3 done 2016-02-24 1:48 ` Junxiao Bi @ 2016-02-24 3:30 ` Eric Ren 0 siblings, 0 replies; 9+ messages in thread From: Eric Ren @ 2016-02-24 3:30 UTC (permalink / raw) To: ocfs2-devel Hi Junxiao, On 02/24/2016 09:48 AM, Junxiao Bi wrote: > Hi Eric, > > On 02/19/2016 11:01 AM, Eric Ren wrote: >> Hi Junxiao, >> >> On Wed, Feb 17, 2016 at 10:15:56AM +0800, Junxiao Bi wrote: >>> Hi Eric, >>> >>> I remember i described it before, please search it on ocfs2-devel. For >>> ocfs2 env setup, please refer to README in ocfs2-test. >> Yes, you did. Actually, it's the quoted paragraph below;-) >> Thanks, but more things what I really want to learn about are, such as: >> 1. git hook/auto control scripts, if it's OK to share; > Sure. Attached. >> 2. pain point and solution, for example, a latest tagged release kernel may >> not compile successfully by `make defconfig` or `cp /boot/config-`uname -r``; >> Or cannot boot up even if we've built kernel RPM and installed it. > That maybe because some config not enabled for your platform. I used xen > vm as test nodes, and use make xenconfig to generate .config and it > works well. > > There are two pain points for this test framework: > 1. auto bisect to spot regression issue. > 2. improve ocfs2-test speed. > Now it needs several days to done the test. Better split the test as > function test and stress test. > > Please think about it when build your test framework. Thanks very much! I really appreciate your help;-) Eric > > Thanks, > Junxiao. >> Did you have this problem? Any suggestion;-) What I can think of is to try opensuse >> tumbleweed distribution(a rolling release). >> >>> On 02/16/2016 05:54 PM, Eric Ren wrote: >>>> Hi Junxiao, >>>> >>>>> Four vm are used, one for git server, and the other three to build >>>>> kernel and run ocfs2 test. Tag a branch and push it to the git server, >>>>> the test will be started. The test cases to run can be set in tag message. >>>> Recently, I get free days and also want to setup automatic testing env for ocfs2. >>>> I'll use pcmk as cluster stack while you probably use o2cb. I think we can complement >>>> each other. May I bother you to describe the work flow more specifically, or share >>>> experiences and references? That way it would save my life;-) >>>> >> Thanks! >> Eric >> ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-02-24 3:30 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-12-21 3:00 [Ocfs2-devel] ocfs2-test for v4.3 done Junxiao Bi 2015-12-21 13:50 ` Eric Ren 2015-12-22 2:34 ` Junxiao Bi 2015-12-22 7:39 ` Eric Ren 2016-02-16 9:54 ` Eric Ren 2016-02-17 2:15 ` Junxiao Bi 2016-02-19 3:01 ` Eric Ren 2016-02-24 1:48 ` Junxiao Bi 2016-02-24 3:30 ` Eric Ren
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).