Commit e45450d2 authored Jul 13, 2021 by Jiewei Ke Committed by Tomasz Zawadzki Jul 16, 2021

thread: spdk_get_io_channel() use i->dev as the search key



In current implementation, io_channel list will be accessed by
spdk_for_each_channel() and spdk_get_io_channel(). We will try to
accelerate spdk_get_io_channel() in the following change "thread: speed
up io_channel lookup by using rbtree" by changing io_channel from list
into RB tree.

To make it cleaner, we prefer to use ch->dev as the key for the
io_channel RB tree instead of ch->dev->io_device. This patch makes
spdk_for_each_channel() use the i->dev to find the expected io_channel.
And the io_device in structure spdk_io_channel_iter is not needed in
spdk_for_each_channel_continue() but we keep it for the compatibility of
spdk_io_channel_iter_get_io_device().

After this patch, spdk_for_each_channel() has to access both io_device
list and io_channel list, and spdk_for_each_channel_continue() still has
to access only io_channel list.

Both io_device list and io_channel list will become RB tree. Hence
performance degradation will be negligible. spdk_for_each_channel() is
not so performance critical than spdk_get_io_channel().

Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: Idd486b0aa1b63b57ede90527dcd1631cbb008a1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8749


Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

parent 281a7c2a

lib/thread/thread.c

+11 −5

Original line number	Diff line number	Diff line
		@@ -2284,7 +2284,7 @@ _call_channel(void *ctx)
		*/
		pthread_mutex_lock(&g_devlist_mutex);
		TAILQ_FOREACH(ch, &i->cur_thread->io_channels, tailq) {
		if (ch->dev->io_device == i->io_device) {
		if (ch->dev == i->dev) {
		break;
		}
		}
		@@ -2316,15 +2316,20 @@ spdk_for_each_channel(void io_device, spdk_channel_msg fn, void ctx,
		i->fn = fn;
		i->ctx = ctx;
		i->cpl = cpl;
		i->orig_thread = _get_thread();

		pthread_mutex_lock(&g_devlist_mutex);
		i->orig_thread = _get_thread();
		i->dev = io_device_get(io_device);
		if (i->dev == NULL) {
		SPDK_ERRLOG("could not find io_device %p\n", io_device);
		assert(false);
		goto end;
		}

		TAILQ_FOREACH(thread, &g_threads, tailq) {
		TAILQ_FOREACH(ch, &thread->io_channels, tailq) {
		if (ch->dev->io_device == io_device) {
		if (ch->dev == i->dev) {
		ch->dev->for_each_count++;
		i->dev = ch->dev;
		i->cur_thread = thread;
		i->ch = ch;
		pthread_mutex_unlock(&g_devlist_mutex);
		@@ -2335,6 +2340,7 @@ spdk_for_each_channel(void io_device, spdk_channel_msg fn, void ctx,
		}
		}

		end:
		pthread_mutex_unlock(&g_devlist_mutex);

		rc = spdk_thread_send_msg(i->orig_thread, _call_completion, i);
		@@ -2359,7 +2365,7 @@ spdk_for_each_channel_continue(struct spdk_io_channel_iter *i, int status)
		thread = TAILQ_NEXT(i->cur_thread, tailq);
		while (thread) {
		TAILQ_FOREACH(ch, &thread->io_channels, tailq) {
		if (ch->dev->io_device == i->io_device) {
		if (ch->dev == i->dev) {
		i->cur_thread = thread;
		i->ch = ch;
		pthread_mutex_unlock(&g_devlist_mutex);