Commit f36c033c authored by Richael Zhuang's avatar Richael Zhuang Committed by Jim Harris
Browse files

uring: fix bug when inserting sock into pending_recv list



There is io error when running NVMe over TCP fio test with uring
socket. It's easy to reproduce the bug with the following
configuration:
target 1 core, 16NVMe SSD, 2 initiators each connects to 8 NVMe
namespaces, each runs fio with numjobs=3.

For if in each round, we inset the sock to the head of the
pending_recv list, and then get max_events socks from head of the
list to process, there is possibility that some socks are always
not processed.
Although there was a strategy to cycle the pending_recv list to make
sure we poll things not in the same order. Such as a list: A B C D E F,
if max_events is 3, then this strategy makes the list is rearranged to
D E F A B C. But it will make this strategy not effective if using
TAILQ_INSERT_HEAD(&group->pending_recv, sock...).

Using TAILQ_INSERT_TAIL(&group->pending_recv, sock...) can fix it.

Signed-off-by: default avatarRichael Zhuang <richael.zhuang@arm.com>
Change-Id: I8429b8eee29a9f9f820ad291d1b65ce2c2be22ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11154


Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarAleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
Reviewed-by: default avatarBen Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
parent 5eb363cf
Loading
Loading
Loading
Loading
+1 −1
Original line number Diff line number Diff line
@@ -1024,7 +1024,7 @@ sock_uring_group_reap(struct spdk_uring_sock_group_impl *group, int max, int max
				if (sock->base.cb_fn != NULL &&
				    sock->pending_recv == false) {
					sock->pending_recv = true;
					TAILQ_INSERT_HEAD(&group->pending_recv, sock, link);
					TAILQ_INSERT_TAIL(&group->pending_recv, sock, link);
				}
			}
			break;