Commit 68184a50 authored May 03, 2022 by Jim Harris

env_dpdk: remove assert from pci_device_fini()



When using the SPDK nvme driver in multi-process mode,
and multiple processes detach from their devices
(i.e. at application exit) very close to each other,
there are cases where a process can have started its
own detach, and then get two remove notifications - one
from its own detach, and another from another process'
detach.

So we need to remove the assertion in pci_device_fini(),
if the removed flag has already been set.

Fixes #2456.

Tested using a modified version of the
nvme_multi_secondary() function in test/nvme/nvme.sh
that starts several additional perf applications.
The test consistently failed without this patch,
and passes every time with this patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I91e16985cdc4a463aaac2c45096bb967aab85560

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12454


Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

parent 8ccb3c10

lib/env_dpdk/pci.c

+6 −1

Original line number	Diff line number	Diff line
		@@ -536,7 +536,12 @@ pci_device_fini(struct rte_pci_device *_dev)
		set_allowed_at(_dev->device.devargs, 0);
		}

		assert(!dev->internal.removed);
		/* It is possible that removed flag was already set when there is a race
		* between the remove notification for this process, and another process
		* that is also detaching from this same device (for example, when using
		* nvme driver in multi-process mode. So do not assert here. See
		* #2456 for additional details.
		*/
		dev->internal.removed = true;
		pthread_mutex_unlock(&g_pci_mutex);
		return 0;