megaraid_sas : Use Block layer tag support for internal command indexing

megaraid_sas driver will use block layer provided tag for indexing internal
MPT frames to get any unique MPT frame tied with tag.  Each IO request
submitted from SCSI mid layer will get associated MPT frame from MPT framepool
(retrieved and return back using spinlock inside megaraid_sas driver's
submission/completion call back). Getting MPT frame from MPT Frame pool is
very expensive operation because of associated spin lock operation (spinlock
overhead increase on multi NUMA node). This type of locking in driver is very
expensive call considering each IO request need - Acquire and Release of the
same lock.

With this support, in IO path driver will directly provide the unique command
index(which is based on block layer tag) and will get the MPT frame tied to
the tag and this way driver can get rid off lock, which synchronizes the
access to MPT frame pool while fetching and returning MPT frame from the pool.

This support in driver provides siginificant performance improvement(on multi
NUMA node system)on latest upstream with SCSI.MQ as well as on existing linux
distributions.

Here is the data for test executed at Avago-
- IO Tool- FIO
- 4 Socket SMC server. (4 NUMA node server)
- 12 SSDs in JBOD mode .
- 4K Rand READ, QD=32
- SCSI MQ x86_64 (Latest Upstream kernel)
- upto 300% Performance Improvement.

If IOs are running on single Node, perfromance gain is less, but as soon as
increase number of nodes, performance improvement is significant. IOs running
on all 4 NUMA nodes, with this patch applied IOPs observed was 1170K vs 344K
IOPs seen without this patch.

Logically, there are two parts of this patch- 1) Block layer tag support 2)
changes in calling convention of return_cmd.  part 2 will revert the changes
done by patch- 90dc9d9 megaraid_sas : MFI MPT linked list corruption fix
because changes done in part 1 has fixed the problem of MFI MPT linked list
corruption. part 2 is very much dependent on part 1, so we decided to have
single patch for these two logical changes.

[jejb: remove chatty printk pointed out by hch]
Signed-off-by: Kashyap Desai <kashyap.desai@avagotech.com>
Signed-off-by: Sumit Saxena <sumit.saxena@avagotech.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: James Bottomley <JBottomley@Odin.com>

1 files changed, 0 insertions, 5 deletions

diff --git a/drivers/scsi/megaraid/megaraid_sas_fusion.h b/drivers/scsi/megaraid/megaraid_sas_fusion.h
index 56e6db2d5874..c66194ebd118 100644
--- a/drivers/scsi/megaraid/megaraid_sas_fusion.h
+++ b/drivers/scsi/megaraid/megaraid_sas_fusion.h
@@ -679,7 +679,6 @@ struct megasas_cmd_fusion {
 	 */
 	u32 sync_cmd_idx;
 	u32 index;
-	u8 flags;
 	u8 pd_r1_lb;
 };
 
@@ -792,10 +791,6 @@ struct MR_FW_RAID_MAP_EXT {
 
 struct fusion_context {
 	struct megasas_cmd_fusion **cmd_list;
-	struct list_head cmd_pool;
-
-	spinlock_t mpt_pool_lock;
-
 	dma_addr_t req_frames_desc_phys;
 	u8 *req_frames_desc;