Commit 47c2ee64 authored by Michal Berger's avatar Michal Berger Committed by Jim Harris
Browse files

test/vhost: Remove cpuset limit enforced upon vhost cpus 



This limit didn't do much in a first place. It was creating separate
cgroup with mem nodes already set to 0-1 nodes - in practice these are
all the NUMA nodes available for mem allocation by the very default.
Regarding cpus, vhost is already limited by its own cpumask's affinity
hence there's no need to enforce this limit via a dedicated cgroup.
Lastly, this was not taking into consideration the fact that other
processes may still be scheduled on the vhost cpus as the cgroups
they belong to were not modified in any way (like in case of the
scheduler tests for instance). That said, the amount of jitter coming
from these 3rd party processes would not have much bearing on vhost
anyway - the only processes that could be more noisy are QEMU's but
each VM instance is already put into a separate cgroup (see
test/vhost/common.sh).

Signed-off-by: default avatarMichal Berger <michal.berger@intel.com>
Change-Id: I1de24bfc9e24f8f6391207e579cc599ea5c82094
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16890


Reviewed-by: default avatarKarol Latecki <karol.latecki@intel.com>
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
parent 98d98ecb

test/vhost/perf_bench/vhost_perf.sh

+0 −21
Original line number Diff line number Diff line
@@ -19,7 +19,6 @@ vm_throttle="" 
bpf_traces=()

ctrl_type="spdk_vhost_scsi"

use_split=false

kernel_cpus=""

run_precondition=false

lvol_stores=()

lvol_bdevs=()
@@ -78,7 +77,6 @@ function usage() { 
	echo "                            Default: spdk_vhost_scsi"

	echo "    --packed-ring           Use packed ring support. Requires Qemu 4.2.0 or greater. Default: disabled."

	echo "    --use-split             Use split vbdevs instead of Logical Volumes"

	echo "    --limit-kernel-vhost=INT  Limit kernel vhost to run only on a number of CPU cores."

	echo "    --run-precondition      Precondition lvols after creating. Default: true."

	echo "    --precond-fio-bin       FIO binary used for SPDK fio plugin precondition. Default: $CONFIG_FIO_SOURCE_DIR/fio."

	echo "    --custom-cpu-cfg=PATH   Custom CPU config for test."
@@ -203,7 +201,6 @@ while getopts 'xhip-:' optchar; do 
				use-split) use_split=true ;;

				run-precondition) run_precondition=true ;;

				precond-fio-bin=*) precond_fio_bin="${OPTARG#*=}" ;;

				limit-kernel-vhost=*) kernel_cpus="${OPTARG#*=}" ;;

				custom-cpu-cfg=*) custom_cpu_cfg="${OPTARG#*=}" ;;

				disk-map=*) disk_map="${OPTARG#*=}" ;;

				iobuf-small-pool-count=*) iobuf_small_count="${OPTARG#*=}" ;;
@@ -431,24 +428,6 @@ fi 
vm_run $used_vms

vm_wait_for_boot 300 $used_vms



if [[ -n "$kernel_cpus" ]]; then

	echo "+cpuset" > /sys/fs/cgroup/cgroup.subtree_control

	mkdir -p /sys/fs/cgroup/spdk

	kernel_mask=$vhost_0_reactor_mask

	kernel_mask=${kernel_mask#"["}

	kernel_mask=${kernel_mask%"]"}



	echo "threaded" > /sys/fs/cgroup/spdk/cgroup.type

	echo "$kernel_mask" > /sys/fs/cgroup/spdk/cpuset.cpus

	echo "0-1" > /sys/fs/cgroup/spdk/cpuset.mems



	kernel_vhost_pids=$(pgrep "vhost" -U root)

	for kpid in $kernel_vhost_pids; do

		echo "Limiting kernel vhost pid ${kpid}"

		echo "${kpid}" > /sys/fs/cgroup/spdk/cgroup.threads

	done

fi



# Run FIO

fio_disks=""