Commit 3ff9c136 authored by Changpeng's avatar Changpeng Committed by Jim Harris
Browse files

NVMe/vfio-user: add initial version vfio-user transport to NVMe driver 



When NVMf target linked with vfio-user library, we can use
vfio-user client library to connect to the target.

Here is the three examples that can work with target:

identify -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g
perf -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g -q 1 -o 4096 -w read -t 10
reconnect -r 'trtype:VFIOUSER traddr:/var/run/muser/domain/muser0/8' -g -q 32 -o 4096 -w randrw \
-M 50 -t 10 -c 0xE

You can run the following test script test/nvmf/target/nvmf_vfio_user.sh to have a quick test,
currently enabled with NVMe Identify,Perf,Reconnect tools.

Change-Id: Ieb9842b2f372184fffbf7f23e4aad26feb47c350
Signed-off-by: default avatarChangpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3839


Community-CI: Broadcom CI
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
parent 1b0ec0b7

CHANGELOG.md

+6 −0
Original line number Diff line number Diff line
@@ -54,6 +54,9 @@ dwords. 
Added a new custom transport `SPDK_NVME_TRANSPORT_VFIOUSER` to enable NVMe

driver running with NVMe over vfio-user target.



Added the vfio-user custom transport implementation in NVMe driver which can connect

to NVMe over vfio-user target via vfio-user transport.



### event



The pci_whitelist and pci_blacklist members of struct spdk_app_opts have been
@@ -102,6 +105,9 @@ An `opts_size`element was added in the `spdk_nvmf_transport_opts` structure 
to solve the ABI compatiblity issue between different SPDK version. And also add

`opts_size` parameter in spdk_nvmf_transport_opts_init function.



Added a new custom vfio-user transport implementation in NVMf which can provide

emulated NVMe devices to QEMU and SPDK NVMe driver.



### json



A new API `spdk_jsonrpc_send_bool_response` was added to allow sending response for

CONFIG

+1 −1
Original line number Diff line number Diff line
@@ -117,7 +117,7 @@ CONFIG_VHOST=y 
# Build vhost initiator (Virtio) driver.

CONFIG_VIRTIO=y



# Build NVMf custom vfio-user target.

# Build custom vfio-user transport for NVMf target and NVMe initiator.

CONFIG_VFIO_USER=y



# Build with PMDK backends

configure

+1 −1
Original line number Diff line number Diff line
@@ -62,7 +62,7 @@ function usage() 
	echo "                           No path required."

	echo " virtio                    Build vhost initiator and virtio-pci bdev modules."

	echo "                           No path required."

	echo " vfio-user                 Build NVMf custom vfio-user target."

	echo " vfio-user                 Build custom vfio-user transport for NVMf target and NVMe initiator."

	echo "                           No path required."

	echo " pmdk                      Build persistent memory bdev."

	echo "                           example: /usr/share/pmdk"

examples/nvme/perf/perf.c

+3 −0
Original line number Diff line number Diff line
@@ -1011,6 +1011,9 @@ build_nvme_name(char *name, size_t length, struct spdk_nvme_ctrlr *ctrlr) 
	case SPDK_NVME_TRANSPORT_TCP:

		res = snprintf(name, length, "TCP (addr:%s subnqn:%s)", trid->traddr, trid->subnqn);

		break;

	case SPDK_NVME_TRANSPORT_VFIOUSER:

		res = snprintf(name, length, "VFIOUSER (%s)", trid->traddr);

		break;

	case SPDK_NVME_TRANSPORT_CUSTOM:

		res = snprintf(name, length, "CUSTOM (%s)", trid->traddr);

		break;

examples/nvme/reconnect/reconnect.c

+3 −0
Original line number Diff line number Diff line
@@ -266,6 +266,9 @@ build_nvme_name(char *name, size_t length, struct spdk_nvme_ctrlr *ctrlr) 
	case SPDK_NVME_TRANSPORT_TCP:

		snprintf(name, length, "TCP (addr:%s subnqn:%s)", trid->traddr, trid->subnqn);

		break;

	case SPDK_NVME_TRANSPORT_VFIOUSER:

		snprintf(name, length, "VFIOUSER (%s)", trid->traddr);

		break;

	case SPDK_NVME_TRANSPORT_CUSTOM:

		snprintf(name, length, "CUSTOM (%s)", trid->traddr);

		break;