Commit 6895e9d9 authored by Jim Harris's avatar Jim Harris Committed by Konrad Sztyber
Browse files

event: shrink size of event message pool 



These spdk_event structures are used for passing messages between
reactors. When it was first created, we did not have the concept
of spdk_threads, so this pool needed to be very large since all
"per-thread" messaging was done using this pool. But now spdk_threads
are the primary mechanism for per-thread messaging. This event
pool in the reactor code is only for messages sent between cores which
is basically for the scheduler and interrupt mode.

256K pool size was way too big, and ends up consuming over 50MB of
memory. Shrink it to 16K, which is probably still an order of
magnitude more than needed, but still significantly cuts the amount
of memory wasted by this mempool.

Signed-off-by: default avatarJim Harris <jim.harris@samsung.com>
Change-Id: Id46aeae7fd6a495d90ab40ed90f2dcb89b6ef885
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24998


Reviewed-by: default avatarPawel Piatek <pawelx.piatek@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: default avatarKonrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
parent 92108e0a

lib/event/reactor.c

+5 −1
Original line number Diff line number Diff line
@@ -268,6 +268,10 @@ spdk_reactor_get(uint32_t lcore) 
static int reactor_thread_op(struct spdk_thread *thread, enum spdk_thread_op op);

static bool reactor_thread_op_supported(enum spdk_thread_op op);



/* Power of 2 minus 1 is optimal for memory consumption */

#define EVENT_MSG_MEMPOOL_SHIFT 14 /* 2^14 = 16384 */

#define EVENT_MSG_MEMPOOL_SIZE ((1 << EVENT_MSG_MEMPOOL_SHIFT) - 1)



int

spdk_reactors_init(size_t msg_mempool_size)

{
@@ -278,7 +282,7 @@ spdk_reactors_init(size_t msg_mempool_size) 


	snprintf(mempool_name, sizeof(mempool_name), "evtpool_%d", getpid());

	g_spdk_event_mempool = spdk_mempool_create(mempool_name,

			       262144 - 1, /* Power of 2 minus 1 is optimal for memory consumption */

			       EVENT_MSG_MEMPOOL_SIZE,

			       sizeof(struct spdk_event),

			       SPDK_MEMPOOL_DEFAULT_CACHE_SIZE,

			       SPDK_ENV_NUMA_ID_ANY);