[PATCH 1/2] mdadm/clustermd_tests: add some APIs in func.sh to support running the tests without errors

[PATCH 1/2] mdadm/clustermd_tests: add some APIs in func.sh to support running the tests without errors

[Heming Zhao]

clustermd_tests/func.sh lacks some APIs to run, this patch makes
clustermd_tests runnable from the test suite.

Signed-off-by: Heming Zhao <heming.zhao@suse.com>
---
 clustermd_tests/func.sh | 60 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/clustermd_tests/func.sh b/clustermd_tests/func.sh
index 801d6043de61..e659c0ba4c74 100644
--- a/clustermd_tests/func.sh
+++ b/clustermd_tests/func.sh
@@ -1,5 +1,22 @@
 #!/bin/bash
 
+COLOR_FAIL='\033[0;31m' #RED
+COLOR_WARN='\033[1;33m' #YELLOW
+COLOR_SUCCESS='\033[0;32m' #GREEN
+COLOR_NONE='\033[0m'
+
+fail() {
+	printf "${COLOR_FAIL}$1${COLOR_NONE}"
+}
+
+warn() {
+	printf "${COLOR_WARN}$1${COLOR_NONE}"
+}
+
+succeed() {
+	printf "${COLOR_SUCCESS}$1${COLOR_NONE}"
+}
+
 check_ssh()
 {
 	NODE1="$(grep '^NODE1' $CLUSTER_CONF | cut -d'=' -f2)"
@@ -151,6 +168,33 @@ stop_md()
 	fi
 }
 
+record_system_speed_limit() {
+	system_speed_limit_max=`cat /proc/sys/dev/raid/speed_limit_max`
+	system_speed_limit_min=`cat /proc/sys/dev/raid/speed_limit_min`
+}
+
+# To avoid sync action finishes before checking it, it needs to limit
+# the sync speed
+control_system_speed_limit() {
+	echo $test_speed_limit_min > /proc/sys/dev/raid/speed_limit_min
+	echo $test_speed_limit_max > /proc/sys/dev/raid/speed_limit_max
+}
+
+restore_system_speed_limit() {
+	echo $system_speed_limit_min > /proc/sys/dev/raid/speed_limit_max
+	echo $system_speed_limit_max > /proc/sys/dev/raid/speed_limit_max
+}
+
+record_selinux() {
+	# empty
+	return 0
+}
+
+restore_selinux() {
+	# empty
+	return 0
+}
+
 # $1/optional, it shows why to save log
 save_log()
 {
@@ -240,6 +284,22 @@ check()
 					die "$ip: check '$2' failed."
 			done
 		;;
+		recovery-remote )
+			cnt=5
+			for ip in ${NODES[@]}
+			do
+				while ! ssh $ip "grep -sqE 'recovery|REMOTE' /proc/mdstat"
+				do
+					if [ "$cnt" -gt '0' ]
+					then
+						sleep 0.2
+						cnt=$[cnt-1]
+					else
+						die "$ip: no '$2' happening!"
+					fi
+				done
+			done
+		;;
 		PENDING | recovery | resync | reshape )
 			cnt=5
 			for ip in ${NODES[@]}
-- 
2.35.3

[PATCH 2/2] mdadm/clustermd_tests: adjust test cases to support md module changes

[Heming Zhao]

Since kernel commit db5e653d7c9f ("md: delay choosing sync action to
md_start_sync()") delays the start of the sync action, clustermd
array sync/resync jobs can happen on any leg of the array. This
commit adjusts the test cases to follow the new kernel layer behavior.

Signed-off-by: Heming Zhao <heming.zhao@suse.com>
---
 clustermd_tests/02r10_Manage_re-add   | 3 ++-
 clustermd_tests/02r1_Manage_re-add    | 1 +
 clustermd_tests/03r10_switch-recovery | 4 ++--
 clustermd_tests/03r1_switch-recovery  | 4 ++--
 4 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/clustermd_tests/02r10_Manage_re-add b/clustermd_tests/02r10_Manage_re-add
index 2288a00866bd..d8764667ca83 100644
--- a/clustermd_tests/02r10_Manage_re-add
+++ b/clustermd_tests/02r10_Manage_re-add
@@ -9,7 +9,8 @@ check all state UU
 check all dmesg
 mdadm --manage $md0 --fail $dev0 --remove $dev0
 mdadm --manage $md0 --re-add $dev0
-check $NODE1 recovery
+#non-clustered array also doesn't do sync job
+#check $NODE1 recovery
 check all wait
 check all state UU
 check all dmesg
diff --git a/clustermd_tests/02r1_Manage_re-add b/clustermd_tests/02r1_Manage_re-add
index d0d13e53fc0c..811df87be81e 100644
--- a/clustermd_tests/02r1_Manage_re-add
+++ b/clustermd_tests/02r1_Manage_re-add
@@ -9,6 +9,7 @@ check all state UU
 check all dmesg
 mdadm --manage $md0 --fail $dev0 --remove $dev0
 mdadm --manage $md0 --re-add $dev0
+check all wait
 check all state UU
 check all dmesg
 stop_md all $md0
diff --git a/clustermd_tests/03r10_switch-recovery b/clustermd_tests/03r10_switch-recovery
index 867388d04ad8..7d0b88125f46 100644
--- a/clustermd_tests/03r10_switch-recovery
+++ b/clustermd_tests/03r10_switch-recovery
@@ -10,9 +10,9 @@ check all state UU
 check all dmesg
 mdadm --manage $md0 --fail $dev0
 sleep 0.2
-check $NODE1 recovery
+check $NODE1 recovery-remote
 stop_md $NODE1 $md0
-check $NODE2 recovery
+check $NODE2 recovery-remote
 check $NODE2 wait
 check $NODE2 state UU
 check all dmesg
diff --git a/clustermd_tests/03r1_switch-recovery b/clustermd_tests/03r1_switch-recovery
index a1a7cbe71b66..d8483c458297 100644
--- a/clustermd_tests/03r1_switch-recovery
+++ b/clustermd_tests/03r1_switch-recovery
@@ -10,9 +10,9 @@ check all state UU
 check all dmesg
 mdadm --manage $md0 --fail $dev0
 sleep 0.3
-check $NODE1 recovery
+check $NODE1 recovery-remote
 stop_md $NODE1 $md0
-check $NODE2 recovery
+check $NODE2 recovery-remote
 check $NODE2 wait
 check $NODE2 state UU
 check all dmesg
-- 
2.35.3

Re: [PATCH 1/2] mdadm/clustermd_tests: add some APIs in func.sh to support running the tests without errors

[Mariusz Tkaczyk]

On Tue,  9 Jul 2024 20:04:51 +0800
Heming Zhao <heming.zhao@suse.com> wrote:

> clustermd_tests/func.sh lacks some APIs to run, this patch makes
> clustermd_tests runnable from the test suite.
> 
> Signed-off-by: Heming Zhao <heming.zhao@suse.com>
> ---
Applied both but please use cover-letter next time.

Thanks,
Mariusz

Re: [PATCH 1/2] mdadm/clustermd_tests: add some APIs in func.sh to support running the tests without errors

[Heming Zhao]

On 7/10/24 22:10, Mariusz Tkaczyk wrote:
> On Tue,  9 Jul 2024 20:04:51 +0800
> Heming Zhao <heming.zhao@suse.com> wrote:
> 
>> clustermd_tests/func.sh lacks some APIs to run, this patch makes
>> clustermd_tests runnable from the test suite.
>>
>> Signed-off-by: Heming Zhao <heming.zhao@suse.com>
>> ---
> Applied both but please use cover-letter next time.
> 
> Thanks,
> Mariusz

Got it.

Thanks,
Heming