summaryrefslogtreecommitdiffstats
path: root/include/target
diff options
context:
space:
mode:
authorMike Christie <michael.christie@oracle.com>2021-09-29 21:04:19 -0500
committerMartin K. Petersen <martin.petersen@oracle.com>2021-10-18 22:38:35 -0400
commited1227e080990ffec5bf39006ec8a57358e6689a (patch)
tree2a5d669021c695bc740b2b6ee75c98779e8413a0 /include/target
parent945a160794a90442329fdd6ff30f02a5c5b39481 (diff)
downloadlinux-ed1227e080990ffec5bf39006ec8a57358e6689a.tar.gz
linux-ed1227e080990ffec5bf39006ec8a57358e6689a.tar.bz2
linux-ed1227e080990ffec5bf39006ec8a57358e6689a.zip
scsi: target: Fix ordered tag handling
This patch fixes the following bugs: 1. If there are multiple ordered cmds queued and multiple simple cmds completing, target_restart_delayed_cmds() could be called on different CPUs and each instance could start a ordered cmd. They could then run in different orders than they were queued. 2. target_restart_delayed_cmds() and target_handle_task_attr() can race where: 1. target_handle_task_attr() has passed the simple_cmds == 0 check. 2. transport_complete_task_attr() then decrements simple_cmds to 0. 3. transport_complete_task_attr() runs target_restart_delayed_cmds() and it does not see any cmds on the delayed_cmd_list. 4. target_handle_task_attr() adds the cmd to the delayed_cmd_list. The cmd will then end up timing out. 3. If we are sent > 1 ordered cmds and simple_cmds == 0, we can execute them out of order, because target_handle_task_attr() will hit that simple_cmds check first and return false for all ordered cmds sent. 4. We run target_restart_delayed_cmds() after every cmd completion, so if there is more than 1 simple cmd running, we start executing ordered cmds after that first cmd instead of waiting for all of them to complete. 5. Ordered cmds are not supposed to start until HEAD OF QUEUE and all older cmds have completed, and not just simple. 6. It's not a bug but it doesn't make sense to take the delayed_cmd_lock for every cmd completion when ordered cmds are almost never used. Just replacing that lock with an atomic increases IOPs by up to 10% when completions are spread over multiple CPUs and there are multiple sessions/ mqs/thread accessing the same device. This patch moves the queued delayed handling to a per device work to serialze the cmd executions for each device and adds a new counter to track HEAD_OF_QUEUE and SIMPLE cmds. We can then check the new counter to determine when to run the work on the completion path. Link: https://lore.kernel.org/r/20210930020422.92578-3-michael.christie@oracle.com Signed-off-by: Mike Christie <michael.christie@oracle.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Diffstat (limited to 'include/target')
-rw-r--r--include/target/target_core_base.h6
1 files changed, 4 insertions, 2 deletions
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 2130f102798d..a7600b311b84 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -812,8 +812,9 @@ struct se_device {
atomic_long_t read_bytes;
atomic_long_t write_bytes;
/* Active commands on this virtual SE device */
- atomic_t simple_cmds;
- atomic_t dev_ordered_sync;
+ atomic_t non_ordered;
+ bool ordered_sync_in_progress;
+ atomic_t delayed_cmd_count;
atomic_t dev_qf_count;
u32 export_count;
spinlock_t delayed_cmd_lock;
@@ -834,6 +835,7 @@ struct se_device {
struct list_head dev_sep_list;
struct list_head dev_tmr_list;
struct work_struct qf_work_queue;
+ struct work_struct delayed_cmd_work;
struct list_head delayed_cmd_list;
struct list_head qf_cmd_list;
/* Pointer to associated SE HBA */