Commit 3455bfad authored by Ziye Yang's avatar Ziye Yang Committed by Tomasz Zawadzki
Browse files

nvme/rdma: fix the reaped number caculation issue. 



To address the error message:

SPDK_ERRLOG("Unable to resubmit as many requests as we completed.\n");

Reason: The "reaped" variable is used to caculate the free slots
of rdma_reqs after calling the nvme_transport_qpair_process_completions.
And we should correctly caculate the free slots when the rdma_req is
really put.

If we caculate the slots more than we will have, we will trigger
the error print described above.

Signed-off-by: default avatarZiye Yang <ziye.yang@intel.com>
Change-Id: I269bdb63646eee6444d340b904882736c4cbca36
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477913


Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarSeth Howell <seth.howell@intel.com>
Reviewed-by: default avatarqun wan <qun.wan@intel.com>
Reviewed-by: default avatarChangpeng Liu <changpeng.liu@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
parent f198b8ca

lib/nvme/nvme_rdma.c

+4 −4
Original line number Diff line number Diff line
@@ -699,7 +699,7 @@ fail: 
}



static int

nvme_rdma_recv(struct nvme_rdma_qpair *rqpair, uint64_t rsp_idx)

nvme_rdma_recv(struct nvme_rdma_qpair *rqpair, uint64_t rsp_idx, int *reaped)

{

	struct spdk_nvme_rdma_req *rdma_req;

	struct spdk_nvme_cpl *rsp;
@@ -713,6 +713,7 @@ nvme_rdma_recv(struct nvme_rdma_qpair *rqpair, uint64_t rsp_idx) 
	nvme_rdma_req_complete(req, rsp);



	if (rdma_req->request_ready_to_put) {

		(*reaped)++;

		nvme_rdma_req_put(rqpair, rdma_req);

	} else {

		rdma_req->request_ready_to_put = true;
@@ -1936,14 +1937,12 @@ nvme_rdma_qpair_process_completions(struct spdk_nvme_qpair *qpair, 
			case IBV_WC_RECV:

				SPDK_DEBUGLOG(SPDK_LOG_NVME, "CQ recv completion\n");



				reaped++;



				if (wc[i].byte_len < sizeof(struct spdk_nvme_cpl)) {

					SPDK_ERRLOG("recv length %u less than expected response size\n", wc[i].byte_len);

					goto fail;

				}



				if (nvme_rdma_recv(rqpair, wc[i].wr_id)) {

				if (nvme_rdma_recv(rqpair, wc[i].wr_id, &reaped)) {

					SPDK_ERRLOG("nvme_rdma_recv processing failure\n");

					goto fail;

				}
@@ -1953,6 +1952,7 @@ nvme_rdma_qpair_process_completions(struct spdk_nvme_qpair *qpair, 
				rdma_req = (struct spdk_nvme_rdma_req *)wc[i].wr_id;



				if (rdma_req->request_ready_to_put) {

					reaped++;

					nvme_rdma_req_put(rqpair, rdma_req);

				} else {

					rdma_req->request_ready_to_put = true;