Commit a71cd521 authored by Alexis Lescouet's avatar Alexis Lescouet Committed by Tomasz Zawadzki
Browse files

event: Add a user option to change the size of spdk_msg_mempool 



The spdk_msg_mempool structure has a fixed size, which is not flexible
enough. The size of the memory allocation can now be changed in the
options given to the spdk_app_start function.

Signed-off-by: default avatarAlexis Lescouet <alexis.lescouet@nutanix.com>
Change-Id: I1d6524ab8cf23f69f553aedb0f5b0cdc9dde374b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11635


Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: default avatarJim Harris <james.r.harris@intel.com>
Reviewed-by: default avatarAleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: default avatarBen Walker <benjamin.walker@intel.com>
parent c58d5161

CHANGELOG.md

+6 −0
Original line number Diff line number Diff line
@@ -31,6 +31,12 @@ A new flag `ACCEL_FLAG_PERSISTENT` was added to indicate the target memory is PM 
Added `bdev_nvme_add_error_injection` and `bdev_nvme_remove_error_injection` RPCs to add and

remove NVMe error injections.



### event



Added `msg_mempool_size` parameter to `spdk_reactors_init` and `spdk_thread_lib_init_ext`.

The size of `g_spdk_msg_mempool` can now be controlled through the same-named

user option of `spdk_app_opts` structure.



### nvmf



Removed deprecated max_qpairs_per_ctrlr parameter from nvmf_create_transport RPC. Use

examples/nvmf/nvmf/nvmf.c

+1 −1
Original line number Diff line number Diff line
@@ -334,7 +334,7 @@ nvmf_init_threads(void) 
	 * framework. The size of the extra memory allocated is the second parameter.

	 */

	spdk_thread_lib_init_ext(nvmf_reactor_thread_op, nvmf_reactor_thread_op_supported,

				 sizeof(struct nvmf_lw_thread));

				 sizeof(struct nvmf_lw_thread), SPDK_DEFAULT_MSG_MEMPOOL_SIZE);



	/* Spawn one system thread per CPU core. The system thread is called a reactor.

	 * SPDK will spawn lightweight threads that must be mapped to reactors in

include/spdk/event.h

+7 −0
Original line number Diff line number Diff line
@@ -147,6 +147,13 @@ struct spdk_app_opts { 
	 * Default is `false`.

	 */

	bool disable_signal_handlers;



	/**

	 * The allocated size for the message pool used by the threading library.

	 *

	 * Default is `SPDK_DEFAULT_MSG_MEMPOOL_SIZE`.

	 */

	size_t msg_mempool_size;

};



/**

include/spdk/thread.h

+5 −1
Original line number Diff line number Diff line
@@ -213,6 +213,9 @@ typedef void (*spdk_channel_for_each_cpl)(struct spdk_io_channel_iter *i, int st 


#define SPDK_IO_CHANNEL_STRUCT_SIZE	96



/* Power of 2 minus 1 is optimal for memory consumption */

#define SPDK_DEFAULT_MSG_MEMPOOL_SIZE (262144 - 1)



/**

 * Initialize the threading library. Must be called once prior to allocating any threads.

 *
@@ -236,12 +239,13 @@ int spdk_thread_lib_init(spdk_new_thread_fn new_thread_fn, size_t ctx_sz); 
 * \param thread_op_supported_fn Called to check whether the SPDK thread operation is supported.

 * \param ctx_sz For each thread allocated, for use by the thread scheduler. A pointer

 * to this region may be obtained by calling spdk_thread_get_ctx().

 * \param msg_mempool_size Size of the allocated spdk_msg_mempool.

 *

 * \return 0 on success. Negated errno on failure.

 */

int spdk_thread_lib_init_ext(spdk_thread_op_fn thread_op_fn,

			     spdk_thread_op_supported_fn thread_op_supported_fn,

			     size_t ctx_sz);

			     size_t ctx_sz, size_t msg_mempool_size);



/**

 * Release all resources associated with this library.

include/spdk_internal/event.h

+1 −1
Original line number Diff line number Diff line
@@ -116,7 +116,7 @@ struct spdk_reactor { 
	int						resched_fd;

} __attribute__((aligned(SPDK_CACHE_LINE_SIZE)));



int spdk_reactors_init(void);

int spdk_reactors_init(size_t msg_mempool_size);

void spdk_reactors_fini(void);



void spdk_reactors_start(void);