thread: don't move to EXITED if there are pending messages

At end of spdk_thread_poll(), if thread is in EXITING state,
we call thread_exit() to see if the thread can move to
EXITED state.  If there are any pollers, io_channels
or pending device unregistrations in progress, thread_exit()
will keep the thread in EXITING mode for this iteration.

But a thread may post messages to itself during this cleanup
process, so thread_exit() should also check if there are
any messages on its queue.

Found during testing of spdk_thread lifetime patch set.
rbd bdev module will send messages to itself like this
during cleanup.  Without this change, rbd module testing
with bdevperf would cause an spdk_thread to move to
EXITED state prematurely.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie611026a67b7fa48640ae83be03e29a9c64883a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15533


Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>