summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds2020-06-0124-170/+122
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull ARM updates from Russell King: - remove a now unnecessary usage of the KERNEL_DS for sys_oabi_epoll_ctl() - update my email address in a number of drivers - decompressor EFI updates from Ard Biesheuvel - module unwind section handling updates - sparsemem Kconfig cleanups - make act_mm macro respect THREAD_SIZE * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: ARM: 8980/1: Allow either FLATMEM or SPARSEMEM on the multiplatform build ARM: 8979/1: Remove redundant ARCH_SPARSEMEM_DEFAULT setting ARM: 8978/1: mm: make act_mm() respect THREAD_SIZE ARM: decompressor: run decompressor in place if loaded via UEFI ARM: decompressor: move GOT into .data for EFI enabled builds ARM: decompressor: defer loading of the contents of the LC0 structure ARM: decompressor: split off _edata and stack base into separate object ARM: decompressor: move headroom variable out of LC0 ARM: 8976/1: module: allow arch overrides for .init section names ARM: 8975/1: module: fix handling of unwind init sections ARM: 8974/1: use SPARSMEM_STATIC when SPARSEMEM is enabled ARM: 8971/1: replace the sole use of a symbol with its definition ARM: 8969/1: decompressor: simplify libfdt builds Update rmk's email address in various drivers ARM: compat: remove KERNEL_DS usage in sys_oabi_epoll_ctl()
| * ARM: 8980/1: Allow either FLATMEM or SPARSEMEM on the multiplatform buildGregory Fong2020-05-261-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ARMv7 chips with LPAE can often benefit from SPARSEMEM, as portions of system memory can be located deep in the 36-bit address space. Allow FLATMEM or SPARSEMEM to be selectable at compile time; FLATMEM remains the default. This is based on Kevin's "[PATCH 3/3] ARM: Allow either FLATMEM or SPARSEMEM on the multi-v7 build" from [1] and shamelessly rips off his commit message text above. As Arnd pointed out at [2] there doesn't seem to be any reason to tie this specifically to ARMv7, so this has been changed to apply to all multiplatform kernels. The addition of this option does not change the defaults and a build with any defconfig will behave the same way as previously. The only effect this change has is to enable user to change "Memory model" selection in interactive kernel configuration (menuconfig, xconfig etc). [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-September/286837.html [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2014-October/298950.html [ rppt: added ARCH_SELECT_MEMORY_MODEL and updated the changelog ] Cc: Kevin Cernekee <cernekee@gmail.com> Tested-by: Stephen Boyd <sboyd@codeaurora.org> Signed-off-by: Gregory Fong <gregory.0xf0@gmail.com> Signed-off-by: Doug Berger <opendmb@gmail.com> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Mike Rapoport <mike.rapoport@gmail.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * ARM: 8979/1: Remove redundant ARCH_SPARSEMEM_DEFAULT settingKevin Cernekee2020-05-261-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If ARCH_SPARSEMEM_ENABLE=y and ARCH_{FLATMEM,DISCONTIGMEM}_ENABLE=n, then the logic in mm/Kconfig already makes CONFIG_SPARSEMEM the only choice. This is true for all of the existing ARM users of ARCH_SPARSEMEM_ENABLE. Forcing ARCH_SPARSEMEM_DEFAULT=y if ARCH_SPARSEMEM_ENABLE=y prevents us from ever defaulting to FLATMEM, so we should remove this setting. Link: https://lkml.org/lkml/2015/6/4/757 Signed-off-by: Kevin Cernekee <cernekee@gmail.com> Tested-by: Stephen Boyd <sboyd@codeaurora.org> Acked-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Gregory Fong <gregory.0xf0@gmail.com> Signed-off-by: Doug Berger <opendmb@gmail.com> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Mike Rapoport <mike.rapoport@gmail.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * ARM: 8978/1: mm: make act_mm() respect THREAD_SIZELinus Walleij2020-05-261-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recent work with KASan exposed the folling hard-coded bitmask in arch/arm/mm/proc-macros.S: bic rd, sp, #8128 bic rd, rd, #63 This forms the bitmask 0x1FFF that is coinciding with (PAGE_SIZE << THREAD_SIZE_ORDER) - 1, this code was assuming that THREAD_SIZE is always 8K (8192). As KASan was increasing THREAD_SIZE_ORDER to 2, I ran into this bug. Fix it by this little oneline suggested by Ard: bic rd, sp, #(THREAD_SIZE - 1) & ~63 Where THREAD_SIZE is defined using THREAD_SIZE_ORDER. We have to also include <linux/const.h> since the THREAD_SIZE expands to use the _AC() macro. Cc: Ard Biesheuvel <ardb@kernel.org> Cc: Florian Fainelli <f.fainelli@gmail.com> Suggested-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * Merge tag 'efi-arm-no-relocate-for-rmk' of ↵Russell King2020-05-263-86/+43
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux into misc Simplify EFI handover to decompressor The EFI stub in the ARM kernel runs in the context of the firmware, which means it usually runs with the caches and MMU on. Currently, we relocate the zImage so it appears in the first 128 MiB, disable the MMU and caches and invoke the decompressor via its ordinary entry point. However, since we can pass the base of DRAM directly, there is no need to relocate the zImage, which also means there is no need to disable and re-enable the caches and create new page tables etc. This also allows systems whose DRAM start address is not a round multiple of 128 MB to decompress the kernel proper to the base of memory, ensuring that all memory is usable at runtime.
| | * ARM: decompressor: run decompressor in place if loaded via UEFIArd Biesheuvel2020-05-192-64/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The decompressor can load from anywhere in memory, and the only reason the EFI stub code relocates it is to ensure it appears within the first 128 MiB of memory, so that the uncompressed kernel ends up at the right offset in memory. We can short circuit this, and simply jump into the decompressor startup code at the point where it knows where the base of memory lives. This also means there is no need to disable the MMU and caches, create new page tables and re-enable them. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Nicolas Pitre <nico@fluxnic.net>
| | * ARM: decompressor: move GOT into .data for EFI enabled buildsArd Biesheuvel2020-05-191-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We will be running the decompressor in place after a future patch, instead of copying it around first. This means we no longer have to disable and re-enable the MMU and caches either. However, this means we will be loaded with the restricted permissions set by the UEFI firmware, which means that we have to move the GOT table into the data section in order for the contents to be writable by the code itself. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Nicolas Pitre <nico@fluxnic.net>
| | * ARM: decompressor: defer loading of the contents of the LC0 structureArd Biesheuvel2020-05-191-11/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The remaining contents of LC0 are only used after the point in the decompressor startup code where we enter via 'wont_overwrite'. So move the loading of the LC0 structure after it. This will allow us to jump to wont_overwrite directly from the EFI stub, and execute the decompressor in place at the offset it was loaded by the UEFI firmware. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Nicolas Pitre <nico@fluxnic.net>
| | * ARM: decompressor: split off _edata and stack base into separate objectArd Biesheuvel2020-05-191-11/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In preparation of moving the handling of the LC0 object to a later stage in the decompressor startup code, move out _edata and the initial value of the stack pointer, which are needed earlier than the remaining contents of LC0. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Nicolas Pitre <nico@fluxnic.net>
| | * ARM: decompressor: move headroom variable out of LC0Ard Biesheuvel2020-05-191-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before breaking up LC0 into different pieces, move out the variable that is already place-relative (given that it subtracts 'restart' in the expression) and so its value does not need to be added to the runtime address of the LC0 symbol itself. Signed-off-by: Ard Biesheuvel <ardb@kernel.org> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be> Reviewed-by: Nicolas Pitre <nico@fluxnic.net>
| * | ARM: 8976/1: module: allow arch overrides for .init section namesVincent Whitchurch2020-05-192-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ARM stores unwind information for .init.text in sections named .ARM.extab.init.text and .ARM.exidx.init.text. Since those aren't currently recognized as init sections, they're allocated along with the core section, and relocation fails if the core and the init section are allocated from different regions and can't reach other. final section addresses: ... 0x7f800000 .init.text .. 0xcbb54078 .ARM.exidx.init.text .. section 16 reloc 0 sym '': relocation 42 out of range (0xcbb54078 -> 0x7f800000) Allow architectures to override the section name so that ARM can fix this. Acked-by: Jessica Yu <jeyu@kernel.org> Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * | ARM: 8975/1: module: fix handling of unwind init sectionsVincent Whitchurch2020-05-191-3/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Unwind information for init sections is placed in .ARM.exidx.init.text and .ARM.extab.init.text. The module core doesn't know that these are init sections so they are allocated along with the core sections, and if the core and init sections get allocated in different memory regions (which is possible with CONFIG_ARM_MODULE_PLTS=y) and they can't reach each other, relocation fails: final section addresses: ... 0x7f800000 .init.text .. 0xcbb54078 .ARM.exidx.init.text .. section 16 reloc 0 sym '': relocation 42 out of range (0xcbb54078 -> 0x7f800000) Fix this by informing the module core that these sections are init sections, and by removing the init unwind tables before the module core frees the init sections. Signed-off-by: Vincent Whitchurch <vincent.whitchurch@axis.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * | ARM: 8974/1: use SPARSMEM_STATIC when SPARSEMEM is enabledMike Rapoport2020-05-191-0/+1
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The commit 3e347261a80b5 ("[PATCH] sparsemem extreme implementation") made SPARSMEM_EXTREME the default option for configurations that enable SPARSEMEM. For ARM systems with handful of memory banks SPARSEMEM_EXTREME is an overkill. Ensure that SPARSMEM_STATIC is enabled in the configurations that use SPARSEMEM. Fixes: 3e347261a80b5 ("[PATCH] sparsemem extreme implementation") Acked-by: Florian Fainelli <f.fainelli@gmail.com> Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * ARM: 8971/1: replace the sole use of a symbol with its definitionJian Cai2020-04-291-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ALT_UP_B macro sets symbol up_b_offset via .equ to an expression involving another symbol. The macro gets expanded twice when arch/arm/kernel/sleep.S is assembled, creating a scenario where up_b_offset is set to another expression involving symbols while its current value is based on symbols. LLVM integrated assembler does not allow such cases, and based on the documentation of binutils, "Values that are based on expressions involving other symbols are allowed, but some targets may restrict this to only being done once per assembly", so it may be better to avoid such cases as it is not clearly stated which targets should support or disallow them. The fix in this case is simple, as up_b_offset has only one use, so we can replace the use with the definition and get rid of up_b_offset. Link:https://github.com/ClangBuiltLinux/linux/issues/920 Reviewed-by: Stefan Agner <stefan@agner.ch> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Signed-off-by: Jian Cai <caij2003@gmail.com> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * ARM: 8969/1: decompressor: simplify libfdt buildsMasahiro Yamada2020-04-298-56/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Copying source files during the build time may not end up with as clean code as expected. lib/fdt*.c simply wrap scripts/dtc/libfdt/fdt*.c, and it works nicely. Let's follow this approach for the arm decompressor, too. Add four wrappers, arch/arm/boot/compressed/fdt*.c and remove the Makefile messes. Another nice thing is we no longer need to maintain the own libfdt_env.h because the decompressor can include <linux/libfdt_env.h>. There is a subtle problem when generated files are turned into check-in files. When you are doing a rebuild of an existing object tree with O= option, there exists stale "shipped" copies that the old Makefile implementation created. The build system ends up with compiling the stale generated files because Make searches for prerequisites in the current directory, i.e. $(objtree) first, and then the directory listed in VPATH, i.e. $(srctree). To mend this issue, I added the following code: ifdef building_out_of_srctree $(shell rm -f $(addprefix $(obj)/, fdt_rw.c fdt_ro.c fdt_wip.c fdt.c)) endif This will need to stay for a while because "git bisect" crossing this commit, otherwise, would result in a build error. Signed-off-by: Masahiro Yamada <masahiroy@kernel.org> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * Update rmk's email address in various driversRussell King2020-04-216-6/+6
| | | | | | | | | | | | | | | | | | Globally update my email address in six files scattered through the tree. Acked-by: Sam Ravnborg <sam@ravnborg.org> Acked-by: Hans Verkuil <hverkuil-cisco@xs4all.nl> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
| * ARM: compat: remove KERNEL_DS usage in sys_oabi_epoll_ctl()Russell King2020-04-211-10/+5
| | | | | | | | | | | | | | | | We no longer need to switch to KERNEL_DS mode in sys_oabi_epoll_ctl() as we can use do_epoll_ctl() to avoid the additional copy. Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
* | Merge tag 'arm64-upstream' of ↵Linus Torvalds2020-06-01159-976/+2559
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux Pull arm64 updates from Will Deacon: "A sizeable pile of arm64 updates for 5.8. Summary below, but the big two features are support for Branch Target Identification and Clang's Shadow Call stack. The latter is currently arm64-only, but the high-level parts are all in core code so it could easily be adopted by other architectures pending toolchain support Branch Target Identification (BTI): - Support for ARMv8.5-BTI in both user- and kernel-space. This allows branch targets to limit the types of branch from which they can be called and additionally prevents branching to arbitrary code, although kernel support requires a very recent toolchain. - Function annotation via SYM_FUNC_START() so that assembly functions are wrapped with the relevant "landing pad" instructions. - BPF and vDSO updates to use the new instructions. - Addition of a new HWCAP and exposure of BTI capability to userspace via ID register emulation, along with ELF loader support for the BTI feature in .note.gnu.property. - Non-critical fixes to CFI unwind annotations in the sigreturn trampoline. Shadow Call Stack (SCS): - Support for Clang's Shadow Call Stack feature, which reserves platform register x18 to point at a separate stack for each task that holds only return addresses. This protects function return control flow from buffer overruns on the main stack. - Save/restore of x18 across problematic boundaries (user-mode, hypervisor, EFI, suspend, etc). - Core support for SCS, should other architectures want to use it too. - SCS overflow checking on context-switch as part of the existing stack limit check if CONFIG_SCHED_STACK_END_CHECK=y. CPU feature detection: - Removed numerous "SANITY CHECK" errors when running on a system with mismatched AArch32 support at EL1. This is primarily a concern for KVM, which disabled support for 32-bit guests on such a system. - Addition of new ID registers and fields as the architecture has been extended. Perf and PMU drivers: - Minor fixes and cleanups to system PMU drivers. Hardware errata: - Unify KVM workarounds for VHE and nVHE configurations. - Sort vendor errata entries in Kconfig. Secure Monitor Call Calling Convention (SMCCC): - Update to the latest specification from Arm (v1.2). - Allow PSCI code to query the SMCCC version. Software Delegated Exception Interface (SDEI): - Unexport a bunch of unused symbols. - Minor fixes to handling of firmware data. Pointer authentication: - Add support for dumping the kernel PAC mask in vmcoreinfo so that the stack can be unwound by tools such as kdump. - Simplification of key initialisation during CPU bringup. BPF backend: - Improve immediate generation for logical and add/sub instructions. vDSO: - Minor fixes to the linker flags for consistency with other architectures and support for LLVM's unwinder. - Clean up logic to initialise and map the vDSO into userspace. ACPI: - Work around for an ambiguity in the IORT specification relating to the "num_ids" field. - Support _DMA method for all named components rather than only PCIe root complexes. - Minor other IORT-related fixes. Miscellaneous: - Initialise debug traps early for KGDB and fix KDB cacheflushing deadlock. - Minor tweaks to early boot state (documentation update, set TEXT_OFFSET to 0x0, increase alignment of PE/COFF sections). - Refactoring and cleanup" * tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (148 commits) KVM: arm64: Move __load_guest_stage2 to kvm_mmu.h KVM: arm64: Check advertised Stage-2 page size capability arm64/cpufeature: Add get_arm64_ftr_reg_nowarn() ACPI/IORT: Remove the unused __get_pci_rid() arm64/cpuinfo: Add ID_MMFR4_EL1 into the cpuinfo_arm64 context arm64/cpufeature: Add remaining feature bits in ID_AA64PFR1 register arm64/cpufeature: Add remaining feature bits in ID_AA64PFR0 register arm64/cpufeature: Add remaining feature bits in ID_AA64ISAR0 register arm64/cpufeature: Add remaining feature bits in ID_MMFR4 register arm64/cpufeature: Add remaining feature bits in ID_PFR0 register arm64/cpufeature: Introduce ID_MMFR5 CPU register arm64/cpufeature: Introduce ID_DFR1 CPU register arm64/cpufeature: Introduce ID_PFR2 CPU register arm64/cpufeature: Make doublelock a signed feature in ID_AA64DFR0 arm64/cpufeature: Drop TraceFilt feature exposure from ID_DFR0 register arm64/cpufeature: Add explicit ftr_id_isar0[] for ID_ISAR0 register arm64: mm: Add asid_gen_match() helper firmware: smccc: Fix missing prototype warning for arm_smccc_version_init arm64: vdso: Fix CFI directives in sigreturn trampoline arm64: vdso: Don't prefix sigreturn trampoline with a BTI C instruction ...
| * \ Merge branch 'for-next/scs' into for-next/coreWill Deacon2020-05-2831-9/+406
| |\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Support for Clang's Shadow Call Stack in the kernel (Sami Tolvanen and Will Deacon) * for-next/scs: arm64: entry-ftrace.S: Update comment to indicate that x18 is live scs: Move DEFINE_SCS macro into core code scs: Remove references to asm/scs.h from core code scs: Move scs_overflow_check() out of architecture code arm64: scs: Use 'scs_sp' register alias for x18 scs: Move accounting into alloc/free functions arm64: scs: Store absolute SCS stack pointer value in thread_info efi/libstub: Disable Shadow Call Stack arm64: scs: Add shadow stacks for SDEI arm64: Implement Shadow Call Stack arm64: Disable SCS for hypervisor code arm64: vdso: Disable Shadow Call Stack arm64: efi: Restore register x18 if it was corrupted arm64: Preserve register x18 when CPU is suspended arm64: Reserve register x18 from general allocation with SCS scs: Disable when function graph tracing is enabled scs: Add support for stack usage debugging scs: Add page accounting for shadow call stack allocations scs: Add support for Clang's Shadow Call Stack (SCS)
| | * | arm64: entry-ftrace.S: Update comment to indicate that x18 is liveWill Deacon2020-05-181-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Shadow Call Stack pointer is held in x18, so update the ftrace entry comment to indicate that it cannot be safely clobbered. Reported-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Move DEFINE_SCS macro into core codeWill Deacon2020-05-182-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Defining static shadow call stacks is not architecture-specific, so move the DEFINE_SCS() macro into the core header file. Tested-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Remove references to asm/scs.h from core codeWill Deacon2020-05-182-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | asm/scs.h is no longer needed by the core code, so remove a redundant header inclusion and update the stale Kconfig text. Tested-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Move scs_overflow_check() out of architecture codeWill Deacon2020-05-186-24/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is nothing architecture-specific about scs_overflow_check() as it's just a trivial wrapper around scs_corrupted(). For parity with task_stack_end_corrupted(), rename scs_corrupted() to task_scs_end_corrupted() and call it from schedule_debug() when CONFIG_SCHED_STACK_END_CHECK_is enabled, which better reflects its purpose as a debug feature to catch inadvertent overflow of the SCS. Finally, remove the unused scs_overflow_check() function entirely. This has absolutely no impact on architectures that do not support SCS (currently arm64 only). Tested-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: scs: Use 'scs_sp' register alias for x18Will Deacon2020-05-183-8/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | x18 holds the SCS stack pointer value, so introduce a register alias to make this easier to read in assembly code. Tested-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Move accounting into alloc/free functionsWill Deacon2020-05-181-24/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's no need to perform the shadow stack page accounting independently of the lifetime of the underlying allocation, so call the accounting code from the {alloc,free}() functions and simplify the code in the process. Tested-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: scs: Store absolute SCS stack pointer value in thread_infoWill Deacon2020-05-185-14/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Storing the SCS information in thread_info as a {base,offset} pair introduces an additional load instruction on the ret-to-user path, since the SCS stack pointer in x18 has to be converted back to an offset by subtracting the base. Replace the offset with the absolute SCS stack pointer value instead and avoid the redundant load. Tested-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | efi/libstub: Disable Shadow Call StackSami Tolvanen2020-05-151-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Shadow stacks are not available in the EFI stub, filter out SCS flags. Suggested-by: James Morse <james.morse@arm.com> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Ard Biesheuvel <ardb@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: scs: Add shadow stacks for SDEISami Tolvanen2020-05-152-1/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds per-CPU shadow call stacks for the SDEI handler. Similarly to how the kernel stacks are handled, we add separate shadow stacks for normal and critical events. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: James Morse <james.morse@arm.com> Tested-by: James Morse <james.morse@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: Implement Shadow Call StackSami Tolvanen2020-05-159-2/+114
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change implements shadow stack switching, initial SCS set-up, and interrupt shadow stacks for arm64. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: Disable SCS for hypervisor codeSami Tolvanen2020-05-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Disable SCS for code that runs at a different exception level by adding __noscs to __hyp_text. Suggested-by: James Morse <james.morse@arm.com> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: vdso: Disable Shadow Call StackSami Tolvanen2020-05-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Shadow stacks are only available in the kernel, so disable SCS instrumentation for the vDSO. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: efi: Restore register x18 if it was corruptedSami Tolvanen2020-05-151-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we detect a corrupted x18, restore the register before jumping back to potentially SCS instrumented code. This is safe, because the wrapper is called with preemption disabled and a separate shadow stack is used for interrupt handling. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: Preserve register x18 when CPU is suspendedSami Tolvanen2020-05-152-1/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Don't lose the current task's shadow stack when the CPU is suspended. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | arm64: Reserve register x18 from general allocation with SCSSami Tolvanen2020-05-151-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reserve the x18 register from general allocation when SCS is enabled, because the compiler uses the register to store the current task's shadow stack pointer. Note that all external kernel modules must also be compiled with -ffixed-x18 if the kernel has SCS enabled. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Nick Desaulniers <ndesaulniers@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Disable when function graph tracing is enabledSami Tolvanen2020-05-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The graph tracer hooks returns by modifying frame records on the (regular) stack, but with SCS the return address is taken from the shadow stack, and the value in the frame record has no effect. As we don't currently have a mechanism to determine the corresponding slot on the shadow stack (and to pass this through the ftrace infrastructure), for now let's disable SCS when the graph tracer is enabled. With SCS the return address is taken from the shadow stack and the value in the frame record has no effect. The mcount based graph tracer hooks returns by modifying frame records on the (regular) stack, and thus is not compatible. The patchable-function-entry graph tracer used for DYNAMIC_FTRACE_WITH_REGS modifies the LR before it is saved to the shadow stack, and is compatible. Modifying the mcount based graph tracer to work with SCS would require a mechanism to determine the corresponding slot on the shadow stack (and to pass this through the ftrace infrastructure), and we expect that everyone will eventually move to the patchable-function-entry based graph tracer anyway, so for now let's disable SCS when the mcount-based graph tracer is enabled. SCS and patchable-function-entry are both supported from LLVM 10.x. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Add support for stack usage debuggingSami Tolvanen2020-05-151-1/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implements CONFIG_DEBUG_STACK_USAGE for shadow stacks. When enabled, also prints out the highest shadow stack usage per process. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Will Deacon <will@kernel.org> [will: rewrote most of scs_check_usage()] Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Add page accounting for shadow call stack allocationsSami Tolvanen2020-05-156-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds accounting for the memory allocated for shadow stacks. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Acked-by: Will Deacon <will@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | scs: Add support for Clang's Shadow Call Stack (SCS)Sami Tolvanen2020-05-1510-0/+191
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds generic support for Clang's Shadow Call Stack, which uses a shadow stack to protect return addresses from being overwritten by an attacker. Details are available here: https://clang.llvm.org/docs/ShadowCallStack.html Note that security guarantees in the kernel differ from the ones documented for user space. The kernel must store addresses of shadow stacks in memory, which means an attacker capable reading and writing arbitrary memory may be able to locate them and hijack control flow by modifying the stacks. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com> [will: Numerous cosmetic changes] Signed-off-by: Will Deacon <will@kernel.org>
| * | | Merge branch 'for-next/kvm/errata' into for-next/coreWill Deacon2020-05-289-77/+68
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KVM CPU errata rework (Andrew Scull and Marc Zyngier) * for-next/kvm/errata: KVM: arm64: Move __load_guest_stage2 to kvm_mmu.h arm64: Unify WORKAROUND_SPECULATIVE_AT_{NVHE,VHE}
| | * | | KVM: arm64: Move __load_guest_stage2 to kvm_mmu.hMarc Zyngier2020-05-282-18/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Having __load_guest_stage2 in kvm_hyp.h is quickly going to trigger a circular include problem. In order to avoid this, let's move it to kvm_mmu.h, where it will be a better fit anyway. In the process, drop the __hyp_text annotation, which doesn't help as the function is marked as __always_inline. Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: Unify WORKAROUND_SPECULATIVE_AT_{NVHE,VHE}Andrew Scull2020-05-048-58/+50
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Errata 1165522, 1319367 and 1530923 each allow TLB entries to be allocated as a result of a speculative AT instruction. In order to avoid mandating VHE on certain affected CPUs, apply the workaround to both the nVHE and the VHE case for all affected CPUs. Signed-off-by: Andrew Scull <ascull@google.com> Acked-by: Will Deacon <will@kernel.org> CC: Marc Zyngier <maz@kernel.org> CC: James Morse <james.morse@arm.com> CC: Suzuki K Poulose <suzuki.poulose@arm.com> CC: Will Deacon <will@kernel.org> CC: Steven Price <steven.price@arm.com> Link: https://lore.kernel.org/r/20200504094858.108917-1-ascull@google.com Signed-off-by: Will Deacon <will@kernel.org>
| * | | Merge branch 'for-next/bti' into for-next/coreWill Deacon2020-05-2875-219/+963
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Support for Branch Target Identification (BTI) in user and kernel (Mark Brown and others) * for-next/bti: (39 commits) arm64: vdso: Fix CFI directives in sigreturn trampoline arm64: vdso: Don't prefix sigreturn trampoline with a BTI C instruction arm64: bti: Fix support for userspace only BTI arm64: kconfig: Update and comment GCC version check for kernel BTI arm64: vdso: Map the vDSO text with guarded pages when built for BTI arm64: vdso: Force the vDSO to be linked as BTI when built for BTI arm64: vdso: Annotate for BTI arm64: asm: Provide a mechanism for generating ELF note for BTI arm64: bti: Provide Kconfig for kernel mode BTI arm64: mm: Mark executable text as guarded pages arm64: bpf: Annotate JITed code for BTI arm64: Set GP bit in kernel page tables to enable BTI for the kernel arm64: asm: Override SYM_FUNC_START when building the kernel with BTI arm64: bti: Support building kernel C code using BTI arm64: Document why we enable PAC support for leaf functions arm64: insn: Report PAC and BTI instructions as skippable arm64: insn: Don't assume unrecognized HINTs are skippable arm64: insn: Provide a better name for aarch64_insn_is_nop() arm64: insn: Add constants for new HINT instruction decode arm64: Disable old style assembly annotations ...
| | * | | arm64: vdso: Fix CFI directives in sigreturn trampolineWill Deacon2020-05-212-7/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Daniel reports that the .cfi_startproc is misplaced for the sigreturn trampoline, which causes LLVM's unwinder to misbehave: | I run into this with LLVM’s unwinder. | This combination was always broken. This prompted Dave to question our use of CFI directives more generally, and I ended up going down a rabbit hole trying to figure out how this very poorly documented stuff gets used. Move the CFI directives so that the "mysterious NOP" is included in the .cfi_{start,end}proc block and add a bunch of comments so that I can save myself another headache in future. Cc: Tamas Zsoldos <tamas.zsoldos@arm.com> Reported-by: Dave Martin <dave.martin@arm.com> Reported-by: Daniel Kiss <daniel.kiss@arm.com> Tested-by: Daniel Kiss <daniel.kiss@arm.com> Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: vdso: Don't prefix sigreturn trampoline with a BTI C instructionWill Deacon2020-05-213-13/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For better or worse, GDB relies on the exact instruction sequence in the VDSO sigreturn trampoline in order to unwind from signals correctly. Commit c91db232da48 ("arm64: vdso: Convert to modern assembler annotations") unfortunately added a BTI C instruction to the start of __kernel_rt_sigreturn, which breaks this check. Thankfully, it's also not required, since the trampoline is called from a RET instruction when returning from the signal handler Remove the unnecessary BTI C instruction from __kernel_rt_sigreturn, and do the same for the 32-bit VDSO as well for good measure. Cc: Daniel Kiss <daniel.kiss@arm.com> Cc: Tamas Zsoldos <tamas.zsoldos@arm.com> Reviewed-by: Dave Martin <dave.martin@arm.com> Reviewed-by: Mark Brown <broonie@kernel.org> Fixes: c91db232da48 ("arm64: vdso: Convert to modern assembler annotations") Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: bti: Fix support for userspace only BTIMark Brown2020-05-121-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When setting PTE_MAYBE_GP we check system_supports_bti() but this is true for systems where only CONFIG_BTI is set causing us to enable BTI on some kernel text. Add an extra check for the kernel mode option, using an ifdef due to line length. Fixes: c8027285e366 ("arm64: Set GP bit in kernel page tables to enable BTI for the kernel") Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20200512113950.29996-1-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: kconfig: Update and comment GCC version check for kernel BTIWill Deacon2020-05-121-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some versions of GCC are known to suffer from a BTI code generation bug, meaning that CONFIG_CC_HAS_BRANCH_PROT_PAC_RET_BTI cannot be solely used to determine whether or not we can compile with kernel with BTI enabled. Update the BTI Kconfig entry to refer to the relevant GCC bugzilla entry (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697) and update the check now that the fix has been merged into GCC release 10.1. Acked-by: Mark Brown <broonie@kernel.org> Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: vdso: Map the vDSO text with guarded pages when built for BTIMark Brown2020-05-071-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The kernel is responsible for mapping the vDSO into userspace processes, including mapping the text section as executable. Handle the mapping of the vDSO for BTI similarly, mapping the text section as guarded pages so the BTI annotations in the vDSO become effective when they are present. This will mean that we can have BTI active for the vDSO in processes that do not otherwise support BTI. This should not be an issue for any expected use of the vDSO. Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/20200506195138.22086-12-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: vdso: Force the vDSO to be linked as BTI when built for BTIMark Brown2020-05-071-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the kernel and hence vDSO are built with BTI enabled force the linker to link the vDSO as BTI. This will cause the linker to warn if any of the input files do not have the BTI annotation, ensuring that we don't silently fail to provide a vDSO that is built and annotated for BTI when the kernel is being built with BTI. Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/20200506195138.22086-11-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: vdso: Annotate for BTIMark Brown2020-05-073-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Generate BTI annotations for all assembly files included in the 64 bit vDSO. Signed-off-by: Mark Brown <broonie@kernel.org> Reviewed-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/20200506195138.22086-10-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
| | * | | arm64: asm: Provide a mechanism for generating ELF note for BTIMark Brown2020-05-071-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ELF files built for BTI should have a program property note section which identifies them as such. The linker expects to find this note in all object files it is linking into a BTI annotated output, the compiler will ensure that this happens for C files but for assembler files we need to do this in the source so provide a macro which can be used for this purpose. To support likely future requirements for additional notes we split the defininition of the flags to set for BTI code from the macro that creates the note itself. This is mainly for use in the vDSO which should be a normal ELF shared library and should therefore include BTI annotations when built for BTI. Signed-off-by: Mark Brown <broonie@kernel.org> Link: https://lore.kernel.org/r/20200506195138.22086-9-broonie@kernel.org Signed-off-by: Will Deacon <will@kernel.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-10-22 05:44:39 +0000