Commit cdf61c2f authored by Shuhei Matsumoto's avatar Shuhei Matsumoto Committed by Tomasz Zawadzki
Browse files

nvme: Polls only the qpair if ctrlr is not fabrics when connecting synchronously 



For non-fabric controllers, the corresponding I/O qpairs are simply
re-enabled at controller reset.

This had a issue when I/O qpairs span multiple threads and poll group
is used.

spdk_nvme_ctrlr_reconnect_poll_async() calls
nvme_transport_ctrlr_connect_qpair() with qpair->async being false.
Then nvme_transport_ctrlr_connect_qpair() calls
spdk_nvme_poll_group_process_completions() until the qpair is connected.
spdk_nvme_poll_group_process_completions() may poll other qpairs.
This may cause I/O to complete on a wrong thread.

For PCIe controller, spdk_nvme_poll_group_process_completions() calls
spdk_nvme_qpair_process_completions() simply for each qpair.

Hence change nvme_transport_ctrlr_connect_qpair() to call
spdk_nvme_qpair_process_completions() if the controller is non-fabrics.

Signed-off-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ieb270c2fb154124021ef6d25577b817d05e5ca9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14295


Community-CI: Mellanox Build Bot
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarBen Walker <benjamin.walker@intel.com>
Reviewed-by: default avatarMichael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
Reviewed-by: default avatarDong Yi <dongx.yi@intel.com>
Reviewed-by: default avatarAleksey Marchuk <alexeymar@nvidia.com>
parent 2e7a7fe5

lib/nvme/nvme_transport.c

+1 −1
Original line number Diff line number Diff line
@@ -484,7 +484,7 @@ nvme_transport_ctrlr_connect_qpair(struct spdk_nvme_ctrlr *ctrlr, struct spdk_nv 
	if (!qpair->async) {

		/* Busy wait until the qpair exits the connecting state */

		while (nvme_qpair_get_state(qpair) == NVME_QPAIR_CONNECTING) {

			if (qpair->poll_group) {

			if (qpair->poll_group && spdk_nvme_ctrlr_is_fabrics(ctrlr)) {

				rc = spdk_nvme_poll_group_process_completions(

					     qpair->poll_group->group, 0,

					     nvme_transport_connect_qpair_fail);

test/unit/lib/nvme/nvme_transport.c/nvme_transport_ut.c

+1 −0
Original line number Diff line number Diff line
@@ -24,6 +24,7 @@ DEFINE_STUB(spdk_nvme_poll_group_process_completions, int64_t, (struct spdk_nvme 
		spdk_nvme_disconnected_qpair_cb disconnected_qpair_cb), 0);

DEFINE_STUB(nvme_ctrlr_get_current_process, struct spdk_nvme_ctrlr_process *,

	    (struct spdk_nvme_ctrlr *ctrlr), NULL);

DEFINE_STUB(spdk_nvme_ctrlr_is_fabrics, bool, (struct spdk_nvme_ctrlr *ctrlr), false);



static void

ut_construct_transport(struct spdk_nvme_transport *transport, const char name[])