Commit d37555b4 authored by Jonas Pfefferle's avatar Jonas Pfefferle Committed by Jim Harris
Browse files

nvmf: per-host namespace masking 



This patch introduces namespace masking for NVMe-over-fabrics
targets by allowing to (dynamically) add or remove hosts for a given
namespace on a given subsystem.  A host can only
perform operations to a namespace it has been given access.

To preserve current behavior all namespaces are automatically visible.
To control visibility on a per-host basis, the nvmf_subsystem_add_ns RPC
shall be called with "--no-auto-visible". We introduce two new
RPC calls:
- nvmf_ns_add_host <subsysNQN> <NSID> <hostNQN>
- nvmf_ns_remove_host <subsysNQN> <NSID> <hostNQN>
These RPCs apply both to existing and future controllers for the given
subsystem/hostnqn pair.

The list in spdk_nvmf_ns is used to keep track of hostNQNs
which controllers should be attached on connect.

The visible_ns array in spdk_nvmf_ctrlr is used for fast lookup
to check whether a NSID is visible on command execution.

Change-Id: I1c3a3c2bf3615fa76e7bfe3aaf08e2b82d6b4ec0
Signed-off-by: default avatarJonas Pfefferle <pepperjo@japf.ch>
Signed-off-by: default avatarJim Harris <jim.haris@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7821


Reviewed-by: default avatarBen Walker <ben@nvidia.com>
Reviewed-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: default avatarSPDK CI Jenkins <sys_sgci@intel.com>
parent 0517b0c7

CHANGELOG.md

+7 −0
Original line number Diff line number Diff line
@@ -29,6 +29,13 @@ Merged `struct spdk_trace_flags` and `struct spdk_trace_histories` into 
new `struct spdk_trace_file`. Also renamed `spdk_get_trace_histories_size()`

to `spdk_get_trace_file_size()`.



### nvmf



Added support for namespace masking using new C APIs `spdk_nvmf_ns_add_host` and

`spdk_nvmf_ns_remove_host` and RPCs `nvmf_ns_add_host` and `nvmf_ns_remove_host`.

Users must pass new `--no-auto-visible` parameter to `nvmf_subsystem_add_ns` RPC to allow

namespace masking to be controlled by these new RPCs.



## v24.01



### accel

doc/jsonrpc.md

+87 −0
Original line number Diff line number Diff line
@@ -8330,6 +8330,7 @@ Name | Optional | Type | Description 
nqn                     | Required | string      | Subsystem NQN

namespace               | Required | object      | @ref rpc_nvmf_namespace object

tgt_name                | Optional | string      | Parent NVMe-oF target name.

no_auto_visible         | Optional | bool        | Namespace is not automatically visible to controllers (default: false)



#### namespace {#rpc_nvmf_namespace}
@@ -8668,6 +8669,92 @@ Example response: 
}

~~~



### nvmf_ns_add_host {#rpc_nvmf_ns_add_host}



Make the specified namespace of the specified subnqn visible to any existing

or future controllers with the specified hostnqn.



Note: the namespace must have been added with no_auto_visible = false

(see @ref rpc_nvmf_subsystem_add_ns).



#### Parameters



Name                    | Optional | Type        | Description

----------------------- | -------- | ----------- | -----------

nqn                     | Required | string      | Subsystem NQN

nsid                    | Required | number      | Namespace ID

host                    | Required | string      | Host NQN



#### Example



Example request:



~~~json

{

  "jsonrpc": "2.0",

  "id": 1,

  "method": "nvmf_ns_add_host",

  "params": {

    "nqn": "nqn.2016-06.io.spdk:cnode1",

    "nsid": 1,

    "host": "nqn.2024-01.io.spdk:host0"

  }

}

~~~



Example response:



~~~json

{

  "jsonrpc": "2.0",

  "id": 1,

  "result": true

}

~~~



### nvmf_ns_remove_host {#rpc_nvmf_ns_remove_host}



Make the specified namespace of the specified subnqn not visible to any existing

or future controllers with the specified hostnqn.



Note: the namespace must have been added to the subsystem with

no_auto_visible = false (see @ref rpc_nvmf_subsystem_add_ns).



#### Parameters



Name                    | Optional | Type        | Description

----------------------- | -------- | ----------- | -----------

nqn                     | Required | string      | Subsystem NQN

nsid                    | Required | number      | Namespace ID

host                    | Required | string      | Host NQN



#### Example



Example request:



~~~json

{

  "jsonrpc": "2.0",

  "id": 1,

  "method": "nvmf_ns_remove_host",

  "params": {

    "nqn": "nqn.2016-06.io.spdk:cnode1",

    "nsid": 1,

    "host": "nqn.2024-01.io.spdk:host0"

  }

}

~~~



Example response:



~~~json

{

  "jsonrpc": "2.0",

  "id": 1,

  "result": true

}

~~~



### nvmf_set_max_subsystems {#rpc_nvmf_set_max_subsystems}



Set the maximum allowed subsystems for the NVMe-oF target.  This RPC may only be called

include/spdk/nvmf.h

+43 −2
Original line number Diff line number Diff line
@@ -565,6 +565,40 @@ struct spdk_nvmf_subsystem *spdk_nvmf_subsystem_get_first(struct spdk_nvmf_tgt * 
 */

struct spdk_nvmf_subsystem *spdk_nvmf_subsystem_get_next(struct spdk_nvmf_subsystem *subsystem);



/**

 * Make the specified namespace visible to the specified host.

 *

 * May only be performed on subsystems in the PAUSED or INACTIVE states.

 *

 * \param subsystem Subsystem the namespace belong to.

 * \param nsid Namespace ID to be made visible.

 * \param hostnqn The NQN for the host.

 * \param flags Must be zero (reserved for future use).

 *

 * \return 0 on success, or negated errno value on failure.

 */

int spdk_nvmf_ns_add_host(struct spdk_nvmf_subsystem *subsystem,

			  uint32_t nsid,

			  const char *hostnqn,

			  uint32_t flags);



/**

 * Make the specified namespace not visible to the specified host.

 *

 * May only be performed on subsystems in the PAUSED or INACTIVE states.

 *

 * \param subsystem Subsystem the namespace belong to.

 * \param nsid Namespace ID to be made not visible.

 * \param hostnqn The NQN for the host.

 * \param flags Must be zero (reserved for future use).

 *

 * \return 0 on success, or negated errno value on failure.

 */

int spdk_nvmf_ns_remove_host(struct spdk_nvmf_subsystem *subsystem,

			     uint32_t nsid,

			     const char *hostnqn,

			     uint32_t flags);



/**

 * Allow the given host NQN to connect to the given subsystem.

 *
@@ -927,8 +961,15 @@ struct spdk_nvmf_ns_opts { 
	 */

	uint32_t anagrpid;



	/* Hole at bytes 60-63. */

	uint8_t reserved60[4];

	/**

	 * Do not automatically make namespace visible to controllers

	 *

	 * False if not specified

	 */

	bool no_auto_visible;



	/* Hole at bytes 61-63. */

	uint8_t reserved61[3];



	/* Transport specific json values.

	 *

lib/nvmf/ctrlr.c

+26 −2
Original line number Diff line number Diff line
@@ -362,6 +362,20 @@ nvmf_subsys_has_multi_iocs(struct spdk_nvmf_subsystem *subsystem) 
	return false;

}



static void

nvmf_ctrlr_init_visible_ns(struct spdk_nvmf_ctrlr *ctrlr)

{

	struct spdk_nvmf_subsystem *subsystem = ctrlr->subsys;

	struct spdk_nvmf_ns *ns;



	for (ns = spdk_nvmf_subsystem_get_first_ns(subsystem); ns != NULL;

	     ns = spdk_nvmf_subsystem_get_next_ns(subsystem, ns)) {

		if (ns->always_visible || nvmf_ns_find_host(ns, ctrlr->hostnqn) != NULL) {

			spdk_bit_array_set(ctrlr->visible_ns, ns->nsid - 1);

		}

	}

}



static struct spdk_nvmf_ctrlr *

nvmf_ctrlr_create(struct spdk_nvmf_subsystem *subsystem,

		  struct spdk_nvmf_request *req,
@@ -461,6 +475,13 @@ nvmf_ctrlr_create(struct spdk_nvmf_subsystem *subsystem, 
	spdk_uuid_copy(&ctrlr->hostid, (struct spdk_uuid *)connect_data->hostid);

	memcpy(ctrlr->hostnqn, connect_data->hostnqn, sizeof(ctrlr->hostnqn));



	ctrlr->visible_ns = spdk_bit_array_create(subsystem->max_nsid);

	if (!ctrlr->visible_ns) {

		SPDK_ERRLOG("Failed to allocate visible namespace array\n");

		goto err_visible_ns;

	}

	nvmf_ctrlr_init_visible_ns(ctrlr);



	ctrlr->vcprop.cap.raw = 0;

	ctrlr->vcprop.cap.bits.cqr = 1; /* NVMe-oF specification required */

	ctrlr->vcprop.cap.bits.mqes = transport->opts.max_queue_depth -
@@ -520,6 +541,8 @@ nvmf_ctrlr_create(struct spdk_nvmf_subsystem *subsystem, 


	return ctrlr;

err_listener:

	spdk_bit_array_free(&ctrlr->visible_ns);

err_visible_ns:

	spdk_bit_array_free(&ctrlr->qpair_mask);

err_qpair_mask:

	free(ctrlr);
@@ -558,6 +581,7 @@ _nvmf_ctrlr_destruct(void *ctx) 
		STAILQ_REMOVE(&ctrlr->async_events, event, spdk_nvmf_async_event_completion, link);

		free(event);

	}

	spdk_bit_array_free(&ctrlr->visible_ns);

	free(ctrlr);

}
@@ -1988,7 +2012,7 @@ nvmf_ctrlr_set_features_number_of_queues(struct spdk_nvmf_request *req) 
	return SPDK_NVMF_REQUEST_EXEC_STATUS_COMPLETE;

}



SPDK_STATIC_ASSERT(sizeof(struct spdk_nvmf_ctrlr) == 4920,

SPDK_STATIC_ASSERT(sizeof(struct spdk_nvmf_ctrlr) == 4928,

		   "Please check migration fields that need to be added or not");



static void
@@ -2994,7 +3018,7 @@ nvmf_ctrlr_identify_active_ns_list(struct spdk_nvmf_ctrlr *ctrlr, 


	for (ns = spdk_nvmf_subsystem_get_first_ns(subsystem); ns != NULL;

	     ns = spdk_nvmf_subsystem_get_next_ns(subsystem, ns)) {

		if (ns->opts.nsid <= cmd->nsid) {

		if (ns->opts.nsid <= cmd->nsid || !nvmf_ctrlr_ns_is_visible(ctrlr, ns->opts.nsid)) {

			continue;

		}

lib/nvmf/nvmf.c

+2 −0
Original line number Diff line number Diff line
@@ -652,6 +652,8 @@ nvmf_write_nvme_subsystem_config(struct spdk_json_write_ctx *w, 
			spdk_json_write_named_uint32(w, "anagrpid", ns_opts.anagrpid);

		}



		spdk_json_write_named_bool(w, "no_auto_visible", !ns->always_visible);



		/*     "namespace" */

		spdk_json_write_object_end(w);