diff options
| author | Huacai Chen <chenhuacai@loongson.cn> | 2023-06-29 20:58:44 +0800 |
|---|---|---|
| committer | Huacai Chen <chenhuacai@loongson.cn> | 2023-06-29 20:58:44 +0800 |
| commit | e031a5f3f1eddb961a6ded8a21ab8189d8760860 (patch) | |
| tree | cf7a0032a2c7d7a8c9593987b5ec0715634937e7 /arch/loongarch/mm | |
| parent | f6f0c9a74a48448583c3cb0f3f067bc3fe0f13c6 (diff) | |
| download | linux-stable-e031a5f3f1eddb961a6ded8a21ab8189d8760860.tar.gz linux-stable-e031a5f3f1eddb961a6ded8a21ab8189d8760860.tar.bz2 linux-stable-e031a5f3f1eddb961a6ded8a21ab8189d8760860.zip | |
LoongArch: Support dbar with different hints
Traditionally, LoongArch uses "dbar 0" (full completion barrier) for
everything. But the full completion barrier is a performance killer, so
Loongson-3A6000 and newer processors have made finer granularity hints
available:
Bit4: ordering or completion (0: completion, 1: ordering)
Bit3: barrier for previous read (0: true, 1: false)
Bit2: barrier for previous write (0: true, 1: false)
Bit1: barrier for succeeding read (0: true, 1: false)
Bit0: barrier for succeeding write (0: true, 1: false)
Hint 0x700: barrier for "read after read" from the same address, which
is needed by LL-SC loops on old models (dbar 0x700 behaves the same as
nop if such reordering is disabled on new models).
This patch makes use of the various new hints for different kinds of
memory barriers. It brings performance improvements on Loongson-3A6000
series, while not affecting the existing models because all variants are
treated as 'dbar 0' there.
Why override queued_spin_unlock()?
After commit 01e3b958efe85a26d9b ("drivers: Remove explicit invocations
of mmiowb()") we need a completion barrier in queued_spin_unlock(), but
the generic implementation use smp_store_release() which only provide an
ordering barrier.
Signed-off-by: Jun Yi <yijun@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Diffstat (limited to 'arch/loongarch/mm')
| -rw-r--r-- | arch/loongarch/mm/tlbex.S | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/arch/loongarch/mm/tlbex.S b/arch/loongarch/mm/tlbex.S index 244e2f5aeee5..240ced55586e 100644 --- a/arch/loongarch/mm/tlbex.S +++ b/arch/loongarch/mm/tlbex.S @@ -184,7 +184,7 @@ tlb_huge_update_load: ertn nopage_tlb_load: - dbar 0 + dbar 0x700 csrrd ra, EXCEPTION_KS2 la_abs t0, tlb_do_page_fault_0 jr t0 @@ -333,7 +333,7 @@ tlb_huge_update_store: ertn nopage_tlb_store: - dbar 0 + dbar 0x700 csrrd ra, EXCEPTION_KS2 la_abs t0, tlb_do_page_fault_1 jr t0 @@ -480,7 +480,7 @@ tlb_huge_update_modify: ertn nopage_tlb_modify: - dbar 0 + dbar 0x700 csrrd ra, EXCEPTION_KS2 la_abs t0, tlb_do_page_fault_1 jr t0 |