Commit 7671073b authored by Seth Howell's avatar Seth Howell Committed by Jim Harris
Browse files

config: add define for rdma send with inval 



On newer versions of the kernel, we see a significant performance impact
if send with invalidate is not enabled. However, depending on the user's
version of IB verbs, it may not be supported. We need to warn them of
the performance issues.

Change-Id: I93a5f5375af90ca3102a6bd084df430b1153419c
Signed-off-by: default avatarSeth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/422460


Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: default avatarBen Walker <benjamin.walker@intel.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
parent 5abf2ecf

configure

+37 −0
Original line number Diff line number Diff line
@@ -307,6 +307,40 @@ if [[ "$OSTYPE" == "freebsd"* ]]; then 
	fi

fi



if [ "$CONFIG_RDMA" = "y" ]; then

	if [ "$OSTYPE" != "FreeBSD"* ]; then

		ibv_lib_file="$(readlink -f /usr/lib64/libibverbs.so)" || true

		if [ -z $ibv_lib_file ]; then

			ibv_lib_file="libibverbs.so.0.0.0"

		fi

		ibv_ver_str="$(basename $ibv_lib_file)"

		ibv_maj_ver=`echo $ibv_ver_str | cut -d. -f3`

		ibv_min_ver=`echo $ibv_ver_str | cut -d. -f4`

		if [[ "$ibv_maj_var" > 1 || ("$ibv_maj_ver" -eq 1 && "$ibv_min_ver" -ge 1) ]]; then

			CONFIG_RDMA_SEND_WITH_INVAL="y"

		else

			CONFIG_RDMA_SEND_WITH_INVAL="n"

			cat <<EOF

*******************************************************************************

WARNING: The Infiniband Verbs opcode Send With Invalidate is either not

supported or is not functional with the current version of libibverbs installed

on this system. Please upgrade to at least version 1.1.



Beginning with Linux kernel 4.14, the kernel NVMe-oF initiator leverages Send

With Invalidate RDMA operations to improve performance. Failing to use the

Send With Invalidate operation on the NVMe-oF target side results in full

functionality, but greatly reduced performance. The SPDK NVMe-oF target will

be unable to leverage that operation using the currently installed version

of libibverbs, so Linux kernel NVMe-oF initiators based on kernels greater

than or equal to 4.14 will see significantly reduced performance.

*******************************************************************************

EOF

		fi

	fi

fi







echo -n "Creating CONFIG.local..."



# Write the configuration file
@@ -377,6 +411,9 @@ fi 
if [ -n "$CONFIG_RDMA" ]; then

	echo "CONFIG_RDMA?=$CONFIG_RDMA" >> CONFIG.local

fi

if [ -n "$CONFIG_RDMA_SEND_WITH_INVAL" ]; then

	echo "CONFIG_RDMA_SEND_WITH_INVAL?=$CONFIG_RDMA_SEND_WITH_INVAL" >> CONFIG.local

fi

if [ -n "$CONFIG_SHARED" ]; then

	echo "CONFIG_SHARED?=$CONFIG_SHARED" >> CONFIG.local

fi