Commit ab0a454d authored by Jim Harris's avatar Jim Harris
Browse files

bdev: insert freed spdk_bdev_io to the head of the cache 



This keeps the hottest structures at the head of the
cache and helps improve performance.

Improves microbenchmark (8 null bdevs on 8 lcores,
bdevperf seq read with qd=1) from 67M to 78M on my
Xeon E5-v3 system.  Core 0 performance remains about
the same (10.7-10.8M) but others cores improve from
around 8.0M each to 9.4M.

Signed-off-by: default avatarJim Harris <james.r.harris@intel.com>
Change-Id: Ia3ccf94ab39b6f911127f0bd1016e352027b11fc

Reviewed-on: https://review.gerrithub.io/c/443277


Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarBen Walker <benjamin.walker@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
parent 2446c5c6

lib/bdev/bdev.c

+2 −2
Original line number Diff line number Diff line
@@ -726,7 +726,7 @@ spdk_bdev_mgmt_channel_create(void *io_device, void *ctx_buf) 
		bdev_io = spdk_mempool_get(g_bdev_mgr.bdev_io_pool);

		assert(bdev_io != NULL);

		ch->per_thread_cache_count++;

		STAILQ_INSERT_TAIL(&ch->per_thread_cache, bdev_io, internal.buf_link);

		STAILQ_INSERT_HEAD(&ch->per_thread_cache, bdev_io, internal.buf_link);

	}



	TAILQ_INIT(&ch->shared_resources);
@@ -1192,7 +1192,7 @@ spdk_bdev_free_io(struct spdk_bdev_io *bdev_io) 


	if (ch->per_thread_cache_count < ch->bdev_io_cache_size) {

		ch->per_thread_cache_count++;

		STAILQ_INSERT_TAIL(&ch->per_thread_cache, bdev_io, internal.buf_link);

		STAILQ_INSERT_HEAD(&ch->per_thread_cache, bdev_io, internal.buf_link);

		while (ch->per_thread_cache_count > 0 && !TAILQ_EMPTY(&ch->io_wait_queue)) {

			struct spdk_bdev_io_wait_entry *entry;