Commit e5160883 authored by Michal Berger's avatar Michal Berger Committed by Tomasz Zawadzki
Browse files

test/common: Indicate failure when run_test()'s cmd returns es != 0 



This is relevant for a bit more obscure cases (though still valid
ones). Consider the following scenario:

  $ foo() { false | false; }
  $ NOT run_test "foo" foo
  $ echo test succeeded

The goal here is to perform a negative test against foo(). Based on
the example the test should succeed since the pipeline clearly fails
on both ends. However, this test actually fails, i.e. NOT() sees
run_test() returning 0.

The reason is that NOT() executes the cmd as part of the || list to
preserve the es ("$@" || es=$?). In such a case, errexit won't be
triggered inside the run_test() (where foo() is executed) allowing
run_test() to successfully complete.

For similar scenarios, we need to force errexit into action by
explicitly returning from the run_test() when the command it executes
fails. The extra benefit here is that time will have a chance to
complete as well, meaning that even in case of a failure, its report
will be still returned.

In general, autotest should also consider use of pipefail to narrow
down potential issues with missing failures in between the pipes.
Such change however needs to be carefuly tested across the entire
autotest suite first.

Change-Id: Ic7842f886460a9c1638e5b14d8494001b3ec2958
Signed-off-by: default avatarMichal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23795


Reviewed-by: default avatarTomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarKonrad Sztyber <konrad.sztyber@intel.com>
parent b18e53d4

test/common/autotest_common.sh

+4 −2
Original line number Diff line number Diff line
@@ -1103,7 +1103,7 @@ function run_test() { 
	fi



	xtrace_disable

	local test_name="$1" pid

	local test_name="$1" pid es=0

	shift



	if [ -n "${test_domain:-}" ]; then
@@ -1120,7 +1120,7 @@ function run_test() { 
	echo "START TEST $test_name"

	echo "************************************"

	xtrace_restore

	time "$@"

	time "$@" || es=$?

	xtrace_disable

	echo "************************************"

	echo "END TEST $test_name"
@@ -1138,6 +1138,8 @@ function run_test() { 
		echo "$test_domain $test_name" >> $output_dir/test_completions.txt

	fi

	xtrace_restore



	return "$es"

}



function skip_run_test_with_warning() {