scripts/perf: SPDK NVMeOF benchmark scripts

## Running NVMe-OF Performace Testcases

In order to reproduce test cases described in [SPDK NVMe-OF Performance Test Cases](https://ci.spdk.io/download/performance-reports/SPDK_nvmeof_perf_report_18.04.pdf) follow the following instructions.

Currently RDMA NIC IP address assignment must be done manually before running the tests.

# Prepare the configuration file

Configure the target, initiators, and FIO workload in the json configuration file.

## General

Options which apply to both target and all initiator servers such as "password" and "username" fields.

All servers are required to have the same user credentials for running the test.

Test results can be found in /tmp/results directory.

## Target

Configure the target server information.

### rdma_ips

List of IP addresses othat will be used in this test..

NVMe namespaces will be split between provided IP addresses.

So for example providing 2 IP's with 16 NVMe drives present will result in each IP managing

8 NVMe subystems.

### mode

"spdk" or "kernel" values allowed.

### use_null_block

Use null block device instead of present NVMe drives. Used for latency measurements as described

in Test Case 3 of performance report.

### num_cores

List of CPU cores to assign for running SPDK NVMe-OF Target process. Can specify exact core numbers or ranges, eg:

[0, 1, 10-15].

### nvmet_dir

Path to directory with nvmetcli application. If not provided then system-wide package will be used

by default. Not used if "mode" is set to "spdk".

### num_shared_buffers

Number of shared buffers to use when creating transport layer.

## Initiator

Describes initiator arguments. There can be more than one initiator section in the configuration file.

For the sake of easier results parsing from multiple initiators please use only digits and letters

in initiator section name.

### ip

Management IP address used for SSH communication with initiator server.

### rdma_ips

List of target IP addresses to which the initiator should try to connect.

### mode

"spdk" or "kernel" values allowed.

### num_cores

Applies only to SPDK initiator. Number of CPUs core to use for running FIO job.

If not specified then by default each connected subsystem gets its own CPU core.

## fio

Fio job parameters.

- bs: block size

- qd: io depth

- rw: workload mode

- rwmixread: percentage of reads in readwrite workloads

- run_time: time (in seconds) to run workload

- ramp_time: time (in seconds) to run workload before statistics are gathered

- run_num: how many times to run given workload in loop

import os

import re

import json

from itertools import product, chain

from subprocess import check_output, Popen

def get_used_numa_nodes():

    used_numa_nodes = set()

    for bdf in get_nvme_devices_bdf():

        with open("/sys/bus/pci/devices/%s/numa_node" % bdf, "r") as numa_file:

            output = numa_file.read()

        used_numa_nodes.add(int(output))

    return used_numa_nodes

def get_nvme_devices_count():

    output = get_nvme_devices_bdf()

    return len(output)

def get_nvme_devices_bdf():

    print("Getting BDFs for NVMe section")

    output = check_output("source scripts/common.sh; iter_pci_class_code 01 08 02",

                          executable="/bin/bash", shell=True)

    output = [str(x, encoding="utf-8") for x in output.split()]

    print("Done getting BDFs")

    return output

def get_nvme_devices():

    print("Getting kernel NVMe names")

    output = check_output("lsblk -o NAME -nlp", shell=True).decode(encoding="utf-8")

    output = [x for x in output.split("\n") if "nvme" in x]

    print("Done getting kernel NVMe names")

    return output

def nvmet_command(nvmet_bin, command):

    return check_output("%s %s" % (nvmet_bin, command), shell=True).decode(encoding="utf-8")

{

  "general": {

    "username": "uname",

    "password": "pass"

  },

  "target": {

    "rdma_ips": ["192.0.1.1", "192.0.2.1"],

    "mode": "spdk",

    "use_null_block": false,

    "nvmet_dir": "/path/to/nvmetcli",

    "num_cores": "1",

    "num_shared_buffers": 4096

  },

  "initiator1": {

    "ip": "10.0.0.1",

    "rdma_ips": ["192.0.1.1"],

    "mode": "spdk"

  },

  "initiator2": {

    "ip": "10.0.0.2",

    "rdma_ips": ["192.0.2.1"],

    "mode": "spdk"

  },

  "fio": {

    "bs": ["4k"],

    "qd": [128],

    "rw": ["read"],

    "rwmixread": 100,

    "run_time": 5,

    "ramp_time": 1,

    "run_num": 3

  }

}