dm: improve performance by moving dm_io structure to per-bio-data

Eliminates need for a separate mempool to allocate 'struct dm_io'
objects from.  As such, it saves an extra mempool allocation for each
original bio that DM core is issued.

This complicates the per-bio-data accessor functions by needing to
conditonally add extra padding to get to a target's per-bio-data.  But
in the end this provides a decent performance improvement for all
bio-based DM devices.

On an NVMe-loop based testbed to a ramdisk (~3100 MB/s): bio-based
DM linear performance improved by 2% (went from 2665 to 2777 MB/s).

Signed-off-by: Mike Snitzer <snitzer@redhat.com>

1 files changed, 1 insertions, 0 deletions

diff --git a/drivers/md/dm-core.h b/drivers/md/dm-core.h
index 6a14f945783c..8a7dc8f9e40f 100644
--- a/drivers/md/dm-core.h
+++ b/drivers/md/dm-core.h
@@ -91,6 +91,7 @@ struct mapped_device {
 	/*
 	 * io objects are allocated from here.
 	 */
+	struct bio_set *io_bs;
 	mempool_t *io_pool;
 
 	struct bio_set *bs;