Commit 607f6702 authored by Jinhong Kim's avatar Jinhong Kim Committed by Tomasz Zawadzki
Browse files

bdev/raid: reset base bdev after quiesce to avoid UAF on removal 



When an NVMe controller faults (fast-io-fail + reconnect), NVMe retry I/Os
may complete FAILED (propagated as ENXIO). RAID then schedules base-bdev
removal. During removal we quiesce the RAID bdev and the bdev layer starts
tearing down core channels. Later, while destroying module channels, the
NVMe module finishes aborting remaining retry I/Os; those completions
re-enter the RAID path and spdk_bdev_free_io() dereferences a core channel
that was already freed, causing a use-after-free.

Fix: #3703
after the RAID bdev is quiesced, issue spdk_bdev_reset() to the base
bdev so outstanding I/Os are drained/aborted before core-channel resources
are released. Continue the existing removal flow only after reset
completion.

Change-Id: Ifa715c06ceecc8eed709e3049484eaf50663d9a0
Signed-off-by: default avatarjinhong.kim0 <jinhong.kim0@navercorp.com>
Reviewed-on: https://review.spdk.io/c/spdk/spdk/+/26443


Community-CI: Mellanox Build Bot
Reviewed-by: default avatarShuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: default avatarArtur Paszkiewicz <artur.paszkiewicz@solidigm.com>
Tested-by: default avatarSPDK Automated Test System <spdkbot@gmail.com>
Reviewed-by: default avatarJim Harris <jim.harris@nvidia.com>
parent cea92448

module/bdev/raid/bdev_raid.c

+26 −1
Original line number Diff line number Diff line
@@ -2000,7 +2000,7 @@ raid_bdev_channels_remove_base_bdev_done(struct spdk_io_channel_iter *i, int sta 
}



static void

raid_bdev_remove_base_bdev_cont(struct raid_base_bdev_info *base_info)

raid_bdev_remove_base_bdev_do_remove(struct raid_base_bdev_info *base_info)

{

	raid_bdev_deconfigure_base_bdev(base_info);
@@ -2008,6 +2008,31 @@ raid_bdev_remove_base_bdev_cont(struct raid_base_bdev_info *base_info) 
			      raid_bdev_channels_remove_base_bdev_done);

}



static void

raid_bdev_remove_base_bdev_reset_done(struct spdk_bdev_io *bdev_io, bool success, void *cb_arg)

{

	struct raid_base_bdev_info *base_info = cb_arg;



	spdk_bdev_free_io(bdev_io);



	raid_bdev_remove_base_bdev_do_remove(base_info);

}



static void

raid_bdev_remove_base_bdev_cont(struct raid_base_bdev_info *base_info)

{

	int rc;



	rc = spdk_bdev_reset(base_info->desc, base_info->app_thread_ch,

			     raid_bdev_remove_base_bdev_reset_done, base_info);

	if (rc != 0) {

		SPDK_WARNLOG("Reset base bdev '%s' before removal failed: %s\n",

			     base_info->name, spdk_strerror(-rc));

		/* Proceed with removal even if reset submission failed */

		raid_bdev_remove_base_bdev_do_remove(base_info);

	}

}



static void

raid_bdev_remove_base_bdev_write_sb_cb(int status, struct raid_bdev *raid_bdev, void *ctx)

{