summaryrefslogtreecommitdiffstats
path: root/arch/arm64/include/asm/brk-imm.h
Commit message (Collapse)AuthorAgeFilesLines
* arm64: kretprobes: acquire the regs via a BRK exceptionMark Rutland2024-02-201-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On arm64, kprobes always take an exception and so create a struct pt_regs through the usual exception entry logic. Similarly kretprobes taskes and exception for function entry, but for function returns it uses a trampoline which attempts to create a struct pt_regs without taking an exception. This is problematic for a few reasons, including: 1) The kretprobes trampoline neither saves nor restores all of the portions of PSTATE. Before invoking the handler it saves a number of portions of PSTATE, and after returning from the handler it restores NZCV before returning to the original return address provided by the handler. 2) The kretprobe trampoline constructs the PSTATE value piecemeal from special purpose registers as it cannot read all of PSTATE atomically without taking an exception. This is somewhat fragile, and it's not possible to reliably recover PSTATE information which only exists on some physical CPUs (e.g. when SSBS support is mismatched). Today the kretprobes trampoline does not record: - BTYPE - SSBS - ALLINT - SS - PAN - UAO - DIT - TCO ... and this will only get worse with future architecture extensions which add more PSTATE bits. 3) The kretprobes trampoline doesn't store portions of struct pt_regs (e.g. the PMR value when using pseudo-NMIs). Due to this, helpers which operate on a struct pt_regs, such as interrupts_enabled(), may not work correctly. 4) The function entry and function exit handlers run in different contexts. The entry handler will always be run in a debug exception context (which is currently treated as an NMI), but the return will be treated as whatever context the instrumented function was executed in. The differences between these contexts are liable to cause problems (e.g. as the two can be differently interruptible or preemptible, adversely affecting synchronization between the handlers). 5) As the kretprobes trampoline runs in the same context as the code being probed, it is subject to the same single-stepping context, which may not be desirable if this is being driven by the kprobes handlers. Overall, this is fragile, painful to maintain, and gets in the way of supporting other things (e.g. RELIABLE_STACKTRACE, FEAT_NMI). This patch addresses these issues by replacing the kretprobes trampoline with a `BRK` instruction, and using an exception boundary to acquire and restore the regs, in the same way as the regular kprobes trampoline. Ive tested this atop v6.8-rc3: | KTAP version 1 | 1..1 | KTAP version 1 | # Subtest: kprobes_test | # module: test_kprobes | 1..7 | ok 1 test_kprobe | ok 2 test_kprobes | ok 3 test_kprobe_missed | ok 4 test_kretprobe | ok 5 test_kretprobes | ok 6 test_stacktrace_on_kretprobe | ok 7 test_stacktrace_on_nested_kretprobe | # kprobes_test: pass:7 fail:0 skip:0 total:7 | # Totals: pass:7 fail:0 skip:0 total:7 | ok 1 kprobes_test Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Florent Revest <revest@chromium.org> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Steven Rostedt <rostedt@goodmis.org> Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Link: https://lore.kernel.org/r/20240208145916.2004154-1-mark.rutland@arm.com Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
* arm64: Support Clang UBSAN trap codes for better reportingKees Cook2023-02-081-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When building with CONFIG_UBSAN_TRAP=y on arm64, Clang encodes the UBSAN check (handler) type in the esr. Extract this and actually report these traps as coming from the specific UBSAN check that tripped. Before: Internal error: BRK handler: 00000000f20003e8 [#1] PREEMPT SMP After: Internal error: UBSAN: shift out of bounds: 00000000f2005514 [#1] PREEMPT SMP Acked-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Ard Biesheuvel <ardb@kernel.org> Acked-by: Mukesh Ojha <quic_mojha@quicinc.com> Reviewed-by: Fangrui Song <maskray@google.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: John Stultz <jstultz@google.com> Cc: Yongqin Liu <yongqin.liu@linaro.org> Cc: Sami Tolvanen <samitolvanen@google.com> Cc: Yury Norov <yury.norov@gmail.com> Cc: Andrey Konovalov <andreyknvl@gmail.com> Cc: Marco Elver <elver@google.com> Cc: linux-arm-kernel@lists.infradead.org Cc: llvm@lists.linux.dev Signed-off-by: Kees Cook <keescook@chromium.org>
* arm64: Add CFI error handlingSami Tolvanen2022-09-261-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With -fsanitize=kcfi, CFI always traps. Add arm64 support for handling CFI failures. The registers containing the target address and the expected type are encoded in the first ten bits of the ESR as follows: - 0-4: n, where the register Xn contains the target address - 5-9: m, where the register Wm contains the type hash This produces the following oops on CFI failure (generated using lkdtm): [ 21.885179] CFI failure at lkdtm_indirect_call+0x2c/0x44 [lkdtm] (target: lkdtm_increment_int+0x0/0x1c [lkdtm]; expected type: 0x7e0c52a) [ 21.886593] Internal error: Oops - CFI: 0 [#1] PREEMPT SMP [ 21.891060] Modules linked in: lkdtm [ 21.893363] CPU: 0 PID: 151 Comm: sh Not tainted 5.19.0-rc1-00021-g852f4e48dbab #1 [ 21.895560] Hardware name: linux,dummy-virt (DT) [ 21.896543] pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 21.897583] pc : lkdtm_indirect_call+0x2c/0x44 [lkdtm] [ 21.898551] lr : lkdtm_CFI_FORWARD_PROTO+0x3c/0x6c [lkdtm] [ 21.899520] sp : ffff8000083a3c50 [ 21.900191] x29: ffff8000083a3c50 x28: ffff0000027e0ec0 x27: 0000000000000000 [ 21.902453] x26: 0000000000000000 x25: ffffc2aa3d07e7b0 x24: 0000000000000002 [ 21.903736] x23: ffffc2aa3d079088 x22: ffffc2aa3d07e7b0 x21: ffff000003379000 [ 21.905062] x20: ffff8000083a3dc0 x19: 0000000000000012 x18: 0000000000000000 [ 21.906371] x17: 000000007e0c52a5 x16: 000000003ad55aca x15: ffffc2aa60d92138 [ 21.907662] x14: ffffffffffffffff x13: 2e2e2e2065707974 x12: 0000000000000018 [ 21.909775] x11: ffffc2aa62322b88 x10: ffffc2aa62322aa0 x9 : c7e305fb5195d200 [ 21.911898] x8 : ffffc2aa3d077e20 x7 : 6d20676e696c6c61 x6 : 43203a6d74646b6c [ 21.913108] x5 : ffffc2aa6266c9df x4 : ffffc2aa6266c9e1 x3 : ffff8000083a3968 [ 21.914358] x2 : 80000000fffff122 x1 : 00000000fffff122 x0 : ffffc2aa3d07e8f8 [ 21.915827] Call trace: [ 21.916375] lkdtm_indirect_call+0x2c/0x44 [lkdtm] [ 21.918060] lkdtm_CFI_FORWARD_PROTO+0x3c/0x6c [lkdtm] [ 21.919030] lkdtm_do_action+0x34/0x4c [lkdtm] [ 21.919920] direct_entry+0x170/0x1ac [lkdtm] [ 21.920772] full_proxy_write+0x84/0x104 [ 21.921759] vfs_write+0x188/0x3d8 [ 21.922387] ksys_write+0x78/0xe8 [ 21.922986] __arm64_sys_write+0x1c/0x2c [ 21.923696] invoke_syscall+0x58/0x134 [ 21.924554] el0_svc_common+0xb4/0xf4 [ 21.925603] do_el0_svc+0x2c/0xb4 [ 21.926563] el0_svc+0x2c/0x7c [ 21.927147] el0t_64_sync_handler+0x84/0xf0 [ 21.927985] el0t_64_sync+0x18c/0x190 [ 21.929133] Code: 728a54b1 72afc191 6b11021f 54000040 (d4304500) [ 21.930690] ---[ end trace 0000000000000000 ]--- [ 21.930971] Kernel panic - not syncing: Oops - CFI: Fatal exception Suggested-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Tested-by: Kees Cook <keescook@chromium.org> Tested-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Peter Zijlstra (Intel) <peterz@infradead.org> Signed-off-by: Kees Cook <keescook@chromium.org> Link: https://lore.kernel.org/r/20220908215504.3686827-11-samitolvanen@google.com
* arm64: kprobes: Use BRK instead of single-step when executing instructions ↵Jean-Philippe Brucker2020-11-031-0/+2
| | | | | | | | | | | | | | | | | | | | out-of-line Commit 36dadef23fcc ("kprobes: Init kprobes in early_initcall") enabled using kprobes from early_initcall. Unfortunately at this point the hardware debug infrastructure is not operational. The OS lock may still be locked, and the hardware watchpoints may have unknown values when kprobe enables debug monitors to single-step instructions. Rather than using hardware single-step, append a BRK instruction after the instruction to be executed out-of-line. Fixes: 36dadef23fcc ("kprobes: Init kprobes in early_initcall") Suggested-by: Will Deacon <will@kernel.org> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Link: https://lore.kernel.org/r/20201103134900.337243-1-jean-philippe@linaro.org Signed-off-by: Will Deacon <will@kernel.org>
* treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500Thomas Gleixner2019-06-191-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Based on 2 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation # extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 4122 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Enrico Weigelt <info@metux.net> Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Allison Randal <allison@lohutok.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* arm64: probes: Move magic BRK values into brk-imm.hWill Deacon2019-04-091-0/+4
| | | | | | | | | | kprobes and uprobes reserve some BRK immediates for installing their probes. Define these along with the other reservations in brk-imm.h and rename the ESR definitions to be consistent with the others that we already have. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
* arm64: debug: Separate debug hooks based on target exception levelWill Deacon2019-04-091-0/+1
| | | | | | | | | | | | | | Mixing kernel and user debug hooks together is highly error-prone as it relies on all of the hooks to figure out whether the exception came from kernel or user, and then to act accordingly. Make our debug hook code a little more robust by maintaining separate hook lists for user and kernel, with separate registration functions to force callers to be explicit about the exception levels that they care about. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
* kasan, arm64: add brk handler for inline instrumentationAndrey Konovalov2018-12-281-0/+2
| | | | | | | | | | | | | | | | | | | | | Tag-based KASAN inline instrumentation mode (which embeds checks of shadow memory into the generated code, instead of inserting a callback) generates a brk instruction when a tag mismatch is detected. This commit adds a tag-based KASAN specific brk handler, that decodes the immediate value passed to the brk instructions (to extract information about the memory access that triggered the mismatch), reads the register values (x0 contains the guilty address) and reports the bug. Link: http://lkml.kernel.org/r/c91fe7684070e34dc34b419e6b69498f4dcacc2d.1544099024.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov <andreyknvl@google.com> Reviewed-by: Andrey Ryabinin <aryabinin@virtuozzo.com> Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Acked-by: Will Deacon <will.deacon@arm.com> Cc: Christoph Lameter <cl@linux.com> Cc: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* arm64: move brk immediate argument definitions to separate headerArd Biesheuvel2016-02-231-0/+25
Instead of reversing the header dependency between asm/bug.h and asm/debug-monitors.h, split off the brk instruction immediate value defines into a new header asm/brk-imm.h, and include it from both. This solves the circular dependency issue that prevents BUG() from being used in some header files, and keeps the definitions together. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-28 18:02:15 +0000