bdev/crypto: Use accel framework

A new API `spkd_bdev_part_submit_request_ext` was added to specify a custom completion callback.

vbdev_crypto is updated to use accel framework instead of DPDK PMDs.

### scheduler

Changing scheduler from dynamic back to static is no longer possible,

	if [ $SPDK_TEST_CRYPTO -eq 1 ]; then

		run_test "blockdev_crypto_aesni" ./test/bdev/blockdev.sh "crypto_aesni"

		run_test "blockdev_crypto_sw" ./test/bdev/blockdev.sh "crypto_sw"

		# Proceed with the test only if QAT devices are in place

		if [[ $(lspci -d:37c8) ]]; then

			run_test "blockdev_crypto_qat" ./test/bdev/blockdev.sh "crypto_qat"

## Crypto Virtual Bdev Module {#bdev_config_crypto}

The crypto virtual bdev module can be configured to provide at rest data encryption

for any underlying bdev. The module relies on the DPDK CryptoDev Framework to provide

all cryptographic functionality. The framework provides support for many different software

only cryptographic modules as well hardware assisted support for the Intel QAT board and

NVIDIA crypto enabled NICs.

The framework also provides support for cipher, hash, authentication and AEAD functions.

At this time the SPDK virtual bdev module supports cipher only as follows:

for any underlying bdev. The module relies on the SPDK Accel Framework to provide

all cryptographic functionality.

One of the accel modules, dpdk_cryptodev is implemented with the DPDK CryptoDev API,

it provides support for many different software only cryptographic modules as well hardware

assisted support for the Intel QAT board and NVIDIA crypto enabled NICs.

For reads, the buffer provided to the crypto block device will be used as the destination buffer

for unencrypted data.  For writes, however, a temporary scratch buffer is used as the

destination buffer for encryption which is then passed on to the underlying bdev as the

write buffer.  This is done to avoid encrypting the data in the original source buffer which

may cause problems in some use cases.

Below is information about accel modules which support crypto operations:

### dpdk_cryptodev accel module

Supports the following ciphers:

- AESN-NI Multi Buffer Crypto Poll Mode Driver: RTE_CRYPTO_CIPHER_AES128_CBC

- Intel(R) QuickAssist (QAT) Crypto Poll Mode Driver: RTE_CRYPTO_CIPHER_AES128_CBC,

size of the underlying bdev.  For example, a 4K I/O to a bdev with a 512B block size,

would result in 8 cryptographic operations.

For reads, the buffer provided to the crypto module will be used as the destination buffer

for unencrypted data.  For writes, however, a temporary scratch buffer is used as the

destination buffer for encryption which is then passed on to the underlying bdev as the

write buffer.  This is done to avoid encrypting the data in the original source buffer which

may cause problems in some use cases.

### SW accel module

Example command

Supports the following ciphers:

- AES_XTS cipher with 128 or 256 bit keys implemented with ISA-L_crypto

### General workflow

- Set desired accel module to perform crypto operations, that can be done with `accel_assign_opc` RPC command

- Create a named crypto key using `accel_crypto_key_create` RPC command. The key will use the assigned accel

  module. Set of parameters and supported ciphers may be different in each accel module.

- Create virtual crypto block device providing the base block device name and the crypto key name

  using `bdev_crypto_create` RPC command

`rpc.py bdev_crypto_create NVMe1n1 CryNvmeA crypto_aesni_mb 01234567891234560123456789123456`

#### Example

This command will create a crypto vbdev called 'CryNvmeA' on top of the NVMe bdev

'NVMe1n1' and will use the DPDK software driver 'crypto_aesni_mb' and the key

'01234567891234560123456789123456'.

Example command which uses dpdk_cryptodev accel module

```

# start SPDK application with `--wait-for-rpc` parameter

rpc.py dpdk_cryptodev_scan_accel_module

rpc.py dpdk_cryptodev_set_driver crypto_aesni_mb

rpc.py accel_assign_opc -o encrypt -m dpdk_cryptodev

rpc.py accel_assign_opc -o decrypt -m dpdk_cryptodev

rpc.py framework_start_init

rpc.py accel_crypto_key_create -c AES_CBC -k 01234567891234560123456789123456 -n key_aesni_cbc_1

rpc.py bdev_crypto_create NVMe1n1 CryNvmeA -n key_aesni_cbc_1

```

These commands will create a crypto vbdev called 'CryNvmeA' on top of the NVMe bdev

'NVMe1n1' and will use a key named `key_aesni_cbc_1`. The key will work with the accel module which

has been assigned for encrypt operations, in this example it will be the dpdk_cryptodev.

### Crypto key format

Please make sure the keys are provided in hexlified format. This means string passed to

rpc.py must be twice as long than the key length in binary form.

Example command

#### Example command

`rpc.py bdev_crypto_create -c AES_XTS -k2 7859243a027411e581e0c40a35c8228f NVMe1n1 CryNvmeA mlx5_pci d16a2f3a9e9f5b32daefacd7f5984f4578add84425be4a0baa489b9de8884b09`

`rpc.py accel_crypto_key_create -c AES_XTS -k2 7859243a027411e581e0c40a35c8228f -k d16a2f3a9e9f5b32daefacd7f5984f4578add84425be4a0baa489b9de8884b09 -n sample_key`

This command will create a crypto vbdev called 'CryNvmeA' on top of the NVMe bdev

'NVMe1n1' and will use the DPDK software driver 'mlx5_pci', the AES key

This command will create a key called `sample_key`, the AES key

'd16a2f3a9e9f5b32daefacd7f5984f4578add84425be4a0baa489b9de8884b09' and the XTS key

'7859243a027411e581e0c40a35c8228f'. In other words, the compound AES_XTS key to be used is

'd16a2f3a9e9f5b32daefacd7f5984f4578add84425be4a0baa489b9de8884b097859243a027411e581e0c40a35c8228f'

### Delete the virtual crypto block device

To remove the vbdev use the bdev_crypto_delete command.

`rpc.py bdev_crypto_delete CryNvmeA`

The MLX5 driver works with crypto enabled Nvidia NICs and requires special configuration of

### dpdk_cryptodev mlx5_pci driver configuration

The mlx5_pci driver works with crypto enabled Nvidia NICs and requires special configuration of

DPDK environment to enable crypto function. It can be done via SPDK event library by configuring

`env_context` member of `spdk_app_opts` structure or by passing corresponding CLI arguments in

the following form: `--allow=BDF,class=crypto,wcs_file=/full/path/to/wrapped/credentials`, e.g.

    "spdk_kill_instance",

    "accel_get_opc_assignments",

    "accel_crypto_key_create",

    "accel_crypto_key_destroy",

    "accel_crypto_keys_get",

    "ioat_scan_accel_module",

    "dsa_scan_accel_module",

### accel_crypto_key_create {#rpc_accel_crypto_key_create}

Create a crypt key which will be used in accel framework

Create a crypto key which will be used in accel framework

#### Parameters

}

~~~

### accel_crypto_key_destroy {#rpc_accel_crypto_key_destroy}

Destroy a crypto key. The user is responsible for ensuring that the deleted key is not used by acceleration modules.

#### Parameters

Name       | Optional | Type        | Description

-----------|----------| ----------- | -----------------

name       | Required | string      | The key name

#### Example

Example request:

~~~json

{

  "jsonrpc": "2.0",

  "method": "accel_crypto_key_destroy",

  "id": 1,

  "params": {

    "name": "super_key"

  }

}

~~~

Example response:

~~~json

{

  "jsonrpc": "2.0",

  "id": 1,

  "result": true

}

~~~

### accel_crypto_keys_get {#rpc_accel_crypto_keys_get}

Get information about existing crypto keys

Name                    | Optional | Type   | Description

----------------------- |----------|--------| -----------

driver_name             | Required | string | The driver, can be one of crypto_aesni_mb, crypto_qat or mlx5_pci

crypto_pmd              | Required | string | The driver, can be one of crypto_aesni_mb, crypto_qat or mlx5_pci

#### Example

  "method": "dpdk_cryptodev_set_driver",

  "id": 1,

  "params": {

    "driver_name": "crypto_aesni_mb"

    "crypto_pmd": "crypto_aesni_mb"

  }

}

~~~

#### Parameters

Name                    | Optional | Type        | Description

----------------------- | -------- | ----------- | -----------

----------------------- |----------| ----------- | -----------

base_bdev_name          | Required | string      | Name of the base bdev

name                    | Required | string      | Name of the crypto vbdev to create

crypto_pmd              | Required | string      | Name of the crypto device driver

key                     | Required | string      | Key in hex form

cipher                  | Required | string      | Cipher to use, AES_CBC or AES_XTS (QAT and MLX5)

key2                    | Required | string      | 2nd key in hex form only required for cipher AES_XTS

crypto_pmd              | Optional | string      | Name of the crypto device driver. Obsolete, see accel_crypto_key_create

key                     | Optional | string      | Key in hex form. Obsolete, see accel_crypto_key_create

cipher                  | Optional | string      | Cipher to use, AES_CBC or AES_XTS (QAT and MLX5). Obsolete, see accel_crypto_key_create

key2                    | Optional | string      | 2nd key in hex form only required for cipher AET_XTS. Obsolete, see accel_crypto_key_create

key_name                | Optional | string      | Name of the key created with accel_crypto_key_create

module                  | Optional | string      | Name of the accel module which is used to create a key (if no key_name specified)

Both key and key2 must be passed in the hexlified form. For example, 256bit AES key may look like this:

afd9477abf50254219ccb75965fbe39f23ebead5676e292582a0a67f66b88215

};

static const struct spdk_json_object_decoder rpc_accel_crypto_keys_get_decoders[] = {

	{"key_name", offsetof(struct rpc_accel_crypto_keys_get_ctx, key_name), spdk_json_decode_string, true},

	{"key_name", offsetof(struct rpc_accel_crypto_keys_get_ctx, key_name), spdk_json_decode_string},

};

static void

	spdk_jsonrpc_end_result(request, w);

}

SPDK_RPC_REGISTER("accel_crypto_keys_get", rpc_accel_crypto_keys_get, SPDK_RPC_RUNTIME)

static const struct spdk_json_object_decoder rpc_accel_crypto_key_destroy_decoders[] = {

	{"key_name", offsetof(struct rpc_accel_crypto_keys_get_ctx, key_name), spdk_json_decode_string, true},

};

static void

rpc_accel_crypto_key_destroy(struct spdk_jsonrpc_request *request,

			     const struct spdk_json_val *params)

{

	struct rpc_accel_crypto_keys_get_ctx req = {};

	struct spdk_accel_crypto_key *key = NULL;

	int rc;

	if (params && spdk_json_decode_object(params, rpc_accel_crypto_key_destroy_decoders,

					      SPDK_COUNTOF(rpc_accel_crypto_key_destroy_decoders),

					      &req)) {

		spdk_jsonrpc_send_error_response(request, SPDK_JSONRPC_ERROR_PARSE_ERROR,

						 "spdk_json_decode_object failed");

		free(req.key_name);

		return;

	}

	key = spdk_accel_crypto_key_get(req.key_name);

	if (!key) {

		spdk_jsonrpc_send_error_response(request, SPDK_JSONRPC_ERROR_INVALID_PARAMS,

						 "No key object found");

		free(req.key_name);

		return;

	}

	rc = spdk_accel_crypto_key_destroy(key);

	if (rc) {

		spdk_jsonrpc_send_error_response_fmt(request, SPDK_JSONRPC_ERROR_INVALID_PARAMS,

						     "Failed to destroy key, rc %d\n", rc);

	} else {

		spdk_jsonrpc_send_bool_response(request, true);

	}

	free(req.key_name);

}

SPDK_RPC_REGISTER("accel_crypto_key_destroy", rpc_accel_crypto_key_destroy, SPDK_RPC_RUNTIME)