diff options
| author | Laurent Dufour <ldufour@linux.ibm.com> | 2023-02-27 09:36:30 -0800 |
|---|---|---|
| committer | Andrew Morton <akpm@linux-foundation.org> | 2023-04-05 20:03:02 -0700 |
| commit | 70d4cbc80c88251de0a5b3e8df3275901f1fa99a (patch) | |
| tree | 9e28b305b0865834fbef6ea57f8efce5f13db10e /arch/powerpc/mm/fault.c | |
| parent | cd7f176aea5f5929a09a91c661a26912cc995d1b (diff) | |
| download | linux-70d4cbc80c88251de0a5b3e8df3275901f1fa99a.tar.gz linux-70d4cbc80c88251de0a5b3e8df3275901f1fa99a.tar.bz2 linux-70d4cbc80c88251de0a5b3e8df3275901f1fa99a.zip | |
powerc/mm: try VMA lock-based page fault handling first
Attempt VMA lock-based page fault handling first, and fall back to the
existing mmap_lock-based handling if that fails. Copied from "x86/mm: try
VMA lock-based page fault handling first"
[ldufour@linux.ibm.com: powerpc/mm: fix mmap_lock bad unlock]
Link: https://lkml.kernel.org/r/20230306154244.17560-1-ldufour@linux.ibm.com
Link: https://lore.kernel.org/linux-mm/842502FB-F99C-417C-9648-A37D0ECDC9CE@linux.ibm.com
Link: https://lkml.kernel.org/r/20230227173632.3292573-32-surenb@google.com
Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: Sachin Sant <sachinp@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Diffstat (limited to 'arch/powerpc/mm/fault.c')
| -rw-r--r-- | arch/powerpc/mm/fault.c | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index af46aa88422b..531177a4ee08 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -474,6 +474,40 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, if (is_exec) flags |= FAULT_FLAG_INSTRUCTION; +#ifdef CONFIG_PER_VMA_LOCK + if (!(flags & FAULT_FLAG_USER)) + goto lock_mmap; + + vma = lock_vma_under_rcu(mm, address); + if (!vma) + goto lock_mmap; + + if (unlikely(access_pkey_error(is_write, is_exec, + (error_code & DSISR_KEYFAULT), vma))) { + vma_end_read(vma); + goto lock_mmap; + } + + if (unlikely(access_error(is_write, is_exec, vma))) { + vma_end_read(vma); + goto lock_mmap; + } + + fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); + vma_end_read(vma); + + if (!(fault & VM_FAULT_RETRY)) { + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + goto done; + } + count_vm_vma_lock_event(VMA_LOCK_RETRY); + + if (fault_signal_pending(fault, regs)) + return user_mode(regs) ? 0 : SIGBUS; + +lock_mmap: +#endif /* CONFIG_PER_VMA_LOCK */ + /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -550,6 +584,9 @@ retry: mmap_read_unlock(current->mm); +#ifdef CONFIG_PER_VMA_LOCK +done: +#endif if (unlikely(fault & VM_FAULT_ERROR)) return mm_fault_error(regs, address, fault); |