Commit 29949a93 authored by Nathan Claudel's avatar Nathan Claudel Committed by Jim Harris
Browse files

lib/bdev: submit child splitted I/Os with public memory domain



If a bdev doesn't support memory domain, a bounce-buffer is allocated
and data is either pulled to the bounce-buffer (WRITEs) or a push from
the bounce-buffer is scheduled after completion (READs). If this
fallback is used, `bdev_io->u.bdev.memory_domain` is reset to NULL.

The fallback is executed before splitting. Thus, the splitted I/Os
should not use the memory domain of the parent I/O if the fallback has
already been executed (data has been pulled or a push was scheduled
after the completion of the parent I/Os).

The current implementation causes a wrong pull/push request between the
parent I/O's bounce-buffer and the child I/O's bounce-buffer, which is
nonsensical because the parent I/O's bouce-buffer doesn't belong to the
memory domain region.

I assume that:
- `bdev_io->internal.memory_domain` is supposed to be the memory domain
  at submission, to be used for push/pull on bounce-buffers
- `bdev_io->u.bdev.memory_domain` is supposed to be the memory domain
  used (e.g. by modules) to submit child I/Os
The Bdev layer breaks this assuption when it submits splitted I/Os. I
suggest to use `bdev_io->u.bdev.memory_domain` when submitted splitted
I/Os.

Reproducer:
- create a bdev with an optimal I/O boundary
- submit an I/O that spans multiple I/O boundaries with a memory domain
- check the requested transfers

Change-Id: I03cc76ade16500e74d547b9f07f3a74f5e5e391b
Signed-off-by: default avatarNathan Claudel <nclaudel@kalrayinc.com>
Reviewed-on: https://review.spdk.io/c/spdk/spdk/+/25739


Tested-by: default avatarSPDK Automated Test System <spdkbot@gmail.com>
Reviewed-by: default avatarDaniil Ushkov <daniil.ushkov@yandex.ru>
Reviewed-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: default avatarAleksey Marchuk <alexeymar@nvidia.com>
parent db9536e6
Loading
Loading
Loading
Loading
+4 −4
Original line number Diff line number Diff line
@@ -3216,8 +3216,8 @@ bdev_io_split_submit(struct spdk_bdev_io *bdev_io, struct iovec *iov, int iovcnt
					       spdk_io_channel_from_ctx(bdev_io->internal.ch),
					       iov, iovcnt, md_buf, current_offset,
					       num_blocks,
					       bdev_io_use_memory_domain(bdev_io) ? bdev_io->internal.memory_domain : NULL,
					       bdev_io_use_memory_domain(bdev_io) ? bdev_io->internal.memory_domain_ctx : NULL,
					       bdev_io_use_memory_domain(bdev_io) ? bdev_io->u.bdev.memory_domain : NULL,
					       bdev_io_use_memory_domain(bdev_io) ? bdev_io->u.bdev.memory_domain_ctx : NULL,
					       NULL,
					       bdev_io->u.bdev.dif_check_flags,
					       bdev_io_split_done, bdev_io);
@@ -3228,8 +3228,8 @@ bdev_io_split_submit(struct spdk_bdev_io *bdev_io, struct iovec *iov, int iovcnt
						spdk_io_channel_from_ctx(bdev_io->internal.ch),
						iov, iovcnt, md_buf, current_offset,
						num_blocks,
						bdev_io_use_memory_domain(bdev_io) ? bdev_io->internal.memory_domain : NULL,
						bdev_io_use_memory_domain(bdev_io) ? bdev_io->internal.memory_domain_ctx : NULL,
						bdev_io_use_memory_domain(bdev_io) ? bdev_io->u.bdev.memory_domain : NULL,
						bdev_io_use_memory_domain(bdev_io) ? bdev_io->u.bdev.memory_domain_ctx : NULL,
						NULL,
						bdev_io->u.bdev.dif_check_flags,
						bdev_io->u.bdev.nvme_cdw12.raw,