diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2022-10-09 08:56:54 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2022-10-09 08:56:54 -0700 |
| commit | 0e470763d84dcad27284067647dfb4b1a94dfce0 (patch) | |
| tree | 34b7ab8b490ce5bb74255c5810357fc6fc819eba | |
| parent | a6afa4199d3d038fbfdff5511f7523b0e30cb774 (diff) | |
| parent | d3549a938b73f203ef522562ae9f2d38aa43d234 (diff) | |
| download | linux-stable-0e470763d84dcad27284067647dfb4b1a94dfce0.tar.gz linux-stable-0e470763d84dcad27284067647dfb4b1a94dfce0.tar.bz2 linux-stable-0e470763d84dcad27284067647dfb4b1a94dfce0.zip | |
Merge tag 'efi-next-for-v6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/efi/efi
Pull EFI updates from Ard Biesheuvel:
"A bit more going on than usual in the EFI subsystem. The main driver
for this has been the introduction of the LoonArch architecture last
cycle, which inspired some cleanup and refactoring of the EFI code.
Another driver for EFI changes this cycle and in the future is
confidential compute.
The LoongArch architecture does not use either struct bootparams or DT
natively [yet], and so passing information between the EFI stub and
the core kernel using either of those is undesirable. And in general,
overloading DT has been a source of issues on arm64, so using DT for
this on new architectures is a to avoid for the time being (even if we
might converge on something DT based for non-x86 architectures in the
future). For this reason, in addition to the patch that enables EFI
boot for LoongArch, there are a number of refactoring patches applied
on top of which separate the DT bits from the generic EFI stub bits.
These changes are on a separate topich branch that has been shared
with the LoongArch maintainers, who will include it in their pull
request as well. This is not ideal, but the best way to manage the
conflicts without stalling LoongArch for another cycle.
Another development inspired by LoongArch is the newly added support
for EFI based decompressors. Instead of adding yet another
arch-specific incarnation of this pattern for LoongArch, we are
introducing an EFI app based on the existing EFI libstub
infrastructure that encapulates the decompression code we use on other
architectures, but in a way that is fully generic. This has been
developed and tested in collaboration with distro and systemd folks,
who are eager to start using this for systemd-boot and also for arm64
secure boot on Fedora. Note that the EFI zimage files this introduces
can also be decompressed by non-EFI bootloaders if needed, as the
image header describes the location of the payload inside the image,
and the type of compression that was used. (Note that Fedora's arm64
GRUB is buggy [0] so you'll need a recent version or switch to
systemd-boot in order to use this.)
Finally, we are adding TPM measurement of the kernel command line
provided by EFI. There is an oversight in the TCG spec which results
in a blind spot for command line arguments passed to loaded images,
which means that either the loader or the stub needs to take the
measurement. Given the combinatorial explosion I am anticipating when
it comes to firmware/bootloader stacks and firmware based attestation
protocols (SEV-SNP, TDX, DICE, DRTM), it is good to set a baseline now
when it comes to EFI measured boot, which is that the kernel measures
the initrd and command line. Intermediate loaders can measure
additional assets if needed, but with the baseline in place, we can
deploy measured boot in a meaningful way even if you boot into Linux
straight from the EFI firmware.
Summary:
- implement EFI boot support for LoongArch
- implement generic EFI compressed boot support for arm64, RISC-V and
LoongArch, none of which implement a decompressor today
- measure the kernel command line into the TPM if measured boot is in
effect
- refactor the EFI stub code in order to isolate DT dependencies for
architectures other than x86
- avoid calling SetVirtualAddressMap() on arm64 if the configured
size of the VA space guarantees that doing so is unnecessary
- move some ARM specific code out of the generic EFI source files
- unmap kernel code from the x86 mixed mode 1:1 page tables"
* tag 'efi-next-for-v6.1' of git://git.kernel.org/pub/scm/linux/kernel/git/efi/efi: (24 commits)
efi/arm64: libstub: avoid SetVirtualAddressMap() when possible
efi: zboot: create MemoryMapped() device path for the parent if needed
efi: libstub: fix up the last remaining open coded boot service call
efi/arm: libstub: move ARM specific code out of generic routines
efi/libstub: measure EFI LoadOptions
efi/libstub: refactor the initrd measuring functions
efi/loongarch: libstub: remove dependency on flattened DT
efi: libstub: install boot-time memory map as config table
efi: libstub: remove DT dependency from generic stub
efi: libstub: unify initrd loading between architectures
efi: libstub: remove pointless goto kludge
efi: libstub: simplify efi_get_memory_map() and struct efi_boot_memmap
efi: libstub: avoid efi_get_memory_map() for allocating the virt map
efi: libstub: drop pointless get_memory_map() call
efi: libstub: fix type confusion for load_options_size
arm64: efi: enable generic EFI compressed boot
loongarch: efi: enable generic EFI compressed boot
riscv: efi: enable generic EFI compressed boot
efi/libstub: implement generic EFI zboot
efi/libstub: move efi_system_table global var into separate object
...
50 files changed, 1638 insertions, 563 deletions
diff --git a/Documentation/arm/uefi.rst b/Documentation/arm/uefi.rst index 9b0b5e458a1e..baebe688a006 100644 --- a/Documentation/arm/uefi.rst +++ b/Documentation/arm/uefi.rst @@ -65,10 +65,6 @@ linux,uefi-mmap-desc-size 32-bit Size in bytes of each entry in the UEFI linux,uefi-mmap-desc-ver 32-bit Version of the mmap descriptor format. -linux,initrd-start 64-bit Physical start address of an initrd - -linux,initrd-end 64-bit Physical end address of an initrd - kaslr-seed 64-bit Entropy used to randomize the kernel image base address location. ========================== ====== =========================================== diff --git a/arch/arm/include/asm/efi.h b/arch/arm/include/asm/efi.h index 3088ef72704e..4bdd930167c0 100644 --- a/arch/arm/include/asm/efi.h +++ b/arch/arm/include/asm/efi.h @@ -17,6 +17,7 @@ #ifdef CONFIG_EFI void efi_init(void); +void arm_efi_init(void); int efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md); int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md); @@ -37,7 +38,7 @@ void efi_virtmap_load(void); void efi_virtmap_unload(void); #else -#define efi_init() +#define arm_efi_init() #endif /* CONFIG_EFI */ /* arch specific definitions used by the stub code */ diff --git a/arch/arm/kernel/efi.c b/arch/arm/kernel/efi.c index e57dbcc89123..e50ad7eefc02 100644 --- a/arch/arm/kernel/efi.c +++ b/arch/arm/kernel/efi.c @@ -4,6 +4,7 @@ */ #include <linux/efi.h> +#include <linux/memblock.h> #include <asm/efi.h> #include <asm/mach/map.h> #include <asm/mmu_context.h> @@ -73,3 +74,81 @@ int __init efi_create_mapping(struct mm_struct *mm, efi_memory_desc_t *md) return efi_set_mapping_permissions(mm, md); return 0; } + +static unsigned long __initdata screen_info_table = EFI_INVALID_TABLE_ADDR; +static unsigned long __initdata cpu_state_table = EFI_INVALID_TABLE_ADDR; + +const efi_config_table_type_t efi_arch_tables[] __initconst = { + {LINUX_EFI_ARM_SCREEN_INFO_TABLE_GUID, &screen_info_table}, + {LINUX_EFI_ARM_CPU_STATE_TABLE_GUID, &cpu_state_table}, + {} +}; + +static void __init load_screen_info_table(void) +{ + struct screen_info *si; + + if (screen_info_table != EFI_INVALID_TABLE_ADDR) { + si = early_memremap_ro(screen_info_table, sizeof(*si)); + if (!si) { + pr_err("Could not map screen_info config table\n"); + return; + } + screen_info = *si; + early_memunmap(si, sizeof(*si)); + + /* dummycon on ARM needs non-zero values for columns/lines */ + screen_info.orig_video_cols = 80; + screen_info.orig_video_lines = 25; + + if (memblock_is_map_memory(screen_info.lfb_base)) + memblock_mark_nomap(screen_info.lfb_base, + screen_info.lfb_size); + } +} + +static void __init load_cpu_state_table(void) +{ + if (cpu_state_table != EFI_INVALID_TABLE_ADDR) { + struct efi_arm_entry_state *state; + bool dump_state = true; + + state = early_memremap_ro(cpu_state_table, + sizeof(struct efi_arm_entry_state)); + if (state == NULL) { + pr_warn("Unable to map CPU entry state table.\n"); + return; + } + + if ((state->sctlr_before_ebs & 1) == 0) + pr_warn(FW_BUG "EFI stub was entered with MMU and Dcache disabled, please fix your firmware!\n"); + else if ((state->sctlr_after_ebs & 1) == 0) + pr_warn(FW_BUG "ExitBootServices() returned with MMU and Dcache disabled, please fix your firmware!\n"); + else + dump_state = false; + + if (dump_state || efi_enabled(EFI_DBG)) { + pr_info("CPSR at EFI stub entry : 0x%08x\n", + state->cpsr_before_ebs); + pr_info("SCTLR at EFI stub entry : 0x%08x\n", + state->sctlr_before_ebs); + pr_info("CPSR after ExitBootServices() : 0x%08x\n", + state->cpsr_after_ebs); + pr_info("SCTLR after ExitBootServices(): 0x%08x\n", + state->sctlr_after_ebs); + } + early_memunmap(state, sizeof(struct efi_arm_entry_state)); + } +} + +void __init arm_efi_init(void) +{ + efi_init(); + + load_screen_info_table(); + + /* ARM does not permit early mappings to persist across paging_init() */ + efi_memmap_unmap(); + + load_cpu_state_table(); +} diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c index 1e8a50a97edf..cb88c6e69377 100644 --- a/arch/arm/kernel/setup.c +++ b/arch/arm/kernel/setup.c @@ -1141,7 +1141,7 @@ void __init setup_arch(char **cmdline_p) #endif setup_dma_zone(mdesc); xen_early_init(); - efi_init(); + arm_efi_init(); /* * Make sure the calculation for lowmem/highmem is set appropriately * before reserving/allocating any memory diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 6d9d4a58b898..a82bb3599094 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -151,12 +151,17 @@ libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a # Default target when executing plain make boot := arch/arm64/boot + +ifeq ($(CONFIG_EFI_ZBOOT),) KBUILD_IMAGE := $(boot)/Image.gz +else +KBUILD_IMAGE := $(boot)/vmlinuz.efi +endif -all: Image.gz +all: $(notdir $(KBUILD_IMAGE)) -Image: vmlinux +Image vmlinuz.efi: vmlinux $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ Image.%: Image diff --git a/arch/arm64/boot/.gitignore b/arch/arm64/boot/.gitignore index 9a7a9009d43a..af5dc61f8b43 100644 --- a/arch/arm64/boot/.gitignore +++ b/arch/arm64/boot/.gitignore @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only Image Image.gz +vmlinuz* diff --git a/arch/arm64/boot/Makefile b/arch/arm64/boot/Makefile index a0e3dedd2883..c65aee088410 100644 --- a/arch/arm64/boot/Makefile +++ b/arch/arm64/boot/Makefile @@ -38,3 +38,9 @@ $(obj)/Image.lzo: $(obj)/Image FORCE $(obj)/Image.zst: $(obj)/Image FORCE $(call if_changed,zstd) + +EFI_ZBOOT_PAYLOAD := Image +EFI_ZBOOT_BFD_TARGET := elf64-littleaarch64 +EFI_ZBOOT_MACH_TYPE := ARM64 + +include $(srctree)/drivers/firmware/efi/libstub/Makefile.zboot diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 4aaa5f3d1f65..8151412653de 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -24,9 +24,6 @@ PROVIDE(__efistub_primary_entry_offset = primary_entry - _text); */ PROVIDE(__efistub_memcmp = __pi_memcmp); PROVIDE(__efistub_memchr = __pi_memchr); -PROVIDE(__efistub_memcpy = __pi_memcpy); -PROVIDE(__efistub_memmove = __pi_memmove); -PROVIDE(__efistub_memset = __pi_memset); PROVIDE(__efistub_strlen = __pi_strlen); PROVIDE(__efistub_strnlen = __pi_strnlen); PROVIDE(__efistub_strcmp = __pi_strcmp); @@ -40,16 +37,6 @@ PROVIDE(__efistub__edata = _edata); PROVIDE(__efistub_screen_info = screen_info); PROVIDE(__efistub__ctype = _ctype); -/* - * The __ prefixed memcpy/memset/memmove symbols are provided by KASAN, which - * instruments the conventional ones. Therefore, any references from the EFI - * stub or other position independent, low level C code should be redirected to - * the non-instrumented versions as well. - */ -PROVIDE(__efistub___memcpy = __pi_memcpy); -PROVIDE(__efistub___memmove = __pi_memmove); -PROVIDE(__efistub___memset = __pi_memset); - PROVIDE(__pi___memcpy = __pi_memcpy); PROVIDE(__pi___memmove = __pi_memmove); PROVIDE(__pi___memset = __pi_memset); diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 551dd99e98b8..cfd976065a0d 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -105,8 +105,6 @@ config LOONGARCH select MODULES_USE_ELF_RELA if MODULES select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK - select OF - select OF_EARLY_FLATTREE select PCI select PCI_DOMAINS_GENERIC select PCI_ECAM if ACPI @@ -313,12 +311,20 @@ config DMI config EFI bool "EFI runtime service support" select UCS2_STRING - select EFI_PARAMS_FROM_FDT select EFI_RUNTIME_WRAPPERS help This enables the kernel to use EFI runtime services that are available (such as the EFI variable services). +config EFI_STUB + bool "EFI boot stub support" + default y + depends on EFI + select EFI_GENERIC_STUB + help + This kernel feature allows the kernel to be loaded directly by + EFI firmware without the use of a bootloader. + config SMP bool "Multi-Processing support" help diff --git a/arch/loongarch/Makefile b/arch/loongarch/Makefile index ec3de6191276..84689c3ee3af 100644 --- a/arch/loongarch/Makefile +++ b/arch/loongarch/Makefile @@ -7,7 +7,14 @@ boot := arch/loongarch/boot KBUILD_DEFCONFIG := loongson3_defconfig -KBUILD_IMAGE = $(boot)/vmlinux +image-name-y := vmlinux +image-name-$(CONFIG_EFI_ZBOOT) := vmlinuz + +ifndef CONFIG_EFI_STUB +KBUILD_IMAGE := $(boot)/vmlinux.elf +else +KBUILD_IMAGE := $(boot)/$(image-name-y).efi +endif # # Select the object file format to substitute into the linker script. @@ -75,6 +82,7 @@ endif head-y := arch/loongarch/kernel/head.o libs-y += arch/loongarch/lib/ +libs-$(CONFIG_EFI_STUB) += $(objtree)/drivers/firmware/efi/libstub/lib.a ifeq ($(KBUILD_EXTMOD),) prepare: vdso_prepare @@ -86,13 +94,13 @@ PHONY += vdso_install vdso_install: $(Q)$(MAKE) $(build)=arch/loongarch/vdso $@ -all: $(KBUILD_IMAGE) +all: $(notdir $(KBUILD_IMAGE)) -$(KBUILD_IMAGE): vmlinux - $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $@ +vmlinux.elf vmlinux.efi vmlinuz.efi: vmlinux + $(Q)$(MAKE) $(build)=$(boot) $(bootvars-y) $(boot)/$@ install: - $(Q)install -D -m 755 $(KBUILD_IMAGE) $(INSTALL_PATH)/vmlinux-$(KERNELRELEASE) + $(Q)install -D -m 755 $(KBUILD_IMAGE) $(INSTALL_PATH)/$(image-name-y)-$(KERNELRELEASE) $(Q)install -D -m 644 .config $(INSTALL_PATH)/config-$(KERNELRELEASE) $(Q)install -D -m 644 System.map $(INSTALL_PATH)/System.map-$(KERNELRELEASE) diff --git a/arch/loongarch/boot/.gitignore b/arch/loongarch/boot/.gitignore index 49423ee96ef3..e5dc594dc4b6 100644 --- a/arch/loongarch/boot/.gitignore +++ b/arch/loongarch/boot/.gitignore @@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only vmlinux* +vmlinuz* diff --git a/arch/loongarch/boot/Makefile b/arch/loongarch/boot/Makefile index 0125b17edc98..4e1c374c5782 100644 --- a/arch/loongarch/boot/Makefile +++ b/arch/loongarch/boot/Makefile @@ -8,9 +8,19 @@ drop-sections := .comment .note .options .note.gnu.build-id strip-flags := $(addprefix --remove-section=,$(drop-sections)) -S OBJCOPYFLAGS_vmlinux.efi := -O binary $(strip-flags) -targets := vmlinux quiet_cmd_strip = STRIP $@ cmd_strip = $(STRIP) -s -o $@ $< -$(obj)/vmlinux: vmlinux FORCE +targets := vmlinux.elf +$(obj)/vmlinux.elf: vmlinux FORCE $(call if_changed,strip) + +targets += vmlinux.efi +$(obj)/vmlinux.efi: vmlinux FORCE + $(call if_changed,objcopy) + +EFI_ZBOOT_PAYLOAD := vmlinux.efi +EFI_ZBOOT_BFD_TARGET := elf64-loongarch +EFI_ZBOOT_MACH_TYPE := LOONGARCH64 + +include $(srctree)/drivers/firmware/efi/libstub/Makefile.zboot diff --git a/arch/loongarch/include/asm/bootinfo.h b/arch/loongarch/include/asm/bootinfo.h index e02ac4af7f6e..8e5881bc5ad1 100644 --- a/arch/loongarch/include/asm/bootinfo.h +++ b/arch/loongarch/include/asm/bootinfo.h @@ -36,7 +36,7 @@ struct loongson_system_configuration { }; extern u64 efi_system_table; -extern unsigned long fw_arg0, fw_arg1; +extern unsigned long fw_arg0, fw_arg1, fw_arg2; extern struct loongson_board_info b_info; extern struct loongson_system_configuration loongson_sysconf; diff --git a/arch/loongarch/include/asm/efi.h b/arch/loongarch/include/asm/efi.h index 9d44c6948be1..174567b00ddb 100644 --- a/arch/loongarch/include/asm/efi.h +++ b/arch/loongarch/include/asm/efi.h @@ -17,9 +17,16 @@ void efifb_setup_from_dmi(struct screen_info *si, const char *opt); #define arch_efi_call_virt_teardown() #define EFI_ALLOC_ALIGN SZ_64K +#define EFI_RT_VIRTUAL_OFFSET CSR_DMW0_BASE -struct screen_info *alloc_screen_info(void); -void free_screen_info(struct screen_info *si); +static inline struct screen_info *alloc_screen_info(void) +{ + return &screen_info; +} + +static inline void free_screen_info(struct screen_info *si) +{ +} static inline unsigned long efi_get_max_initrd_addr(unsigned long image_addr) { diff --git a/arch/loongarch/kernel/efi-header.S b/arch/loongarch/kernel/efi-header.S new file mode 100644 index 000000000000..8c1d229a2afa --- /dev/null +++ b/arch/loongarch/kernel/efi-header.S @@ -0,0 +1,99 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited + */ + +#include <linux/pe.h> +#include <linux/sizes.h> + + .macro __EFI_PE_HEADER + .long PE_MAGIC +.Lcoff_header: + .short IMAGE_FILE_MACHINE_LOONGARCH64 /* Machine */ + .short .Lsection_count /* NumberOfSections */ + .long 0 /* TimeDateStamp */ + .long 0 /* PointerToSymbolTable */ + .long 0 /* NumberOfSymbols */ + .short .Lsection_table - .Loptional_header /* SizeOfOptionalHeader */ + .short IMAGE_FILE_DEBUG_STRIPPED | \ + IMAGE_FILE_EXECUTABLE_IMAGE | \ + IMAGE_FILE_LINE_NUMS_STRIPPED /* Characteristics */ + +.Loptional_header: + .short PE_OPT_MAGIC_PE32PLUS /* PE32+ format */ + .byte 0x02 /* MajorLinkerVersion */ + .byte 0x14 /* MinorLinkerVersion */ + .long __inittext_end - .Lefi_header_end /* SizeOfCode */ + .long _end - __initdata_begin /* SizeOfInitializedData */ + .long 0 /* SizeOfUninitializedData */ + .long __efistub_efi_pe_entry - _head /* AddressOfEntryPoint */ + .long .Lefi_header_end - _head /* BaseOfCode */ + +.Lextra_header_fields: + .quad 0 /* ImageBase */ + .long PECOFF_SEGMENT_ALIGN /* SectionAlignment */ + .long PECOFF_FILE_ALIGN /* FileAlignment */ + .short 0 /* MajorOperatingSystemVersion */ + .short 0 /* MinorOperatingSystemVersion */ + .short LINUX_EFISTUB_MAJOR_VERSION /* MajorImageVersion */ + .short LINUX_EFISTUB_MINOR_VERSION /* MinorImageVersion */ + .short 0 /* MajorSubsystemVersion */ + .short 0 /* MinorSubsystemVersion */ + .long 0 /* Win32VersionValue */ + + .long _end - _head /* SizeOfImage */ + + /* Everything before the kernel image is considered part of the header */ + .long .Lefi_header_end - _head /* SizeOfHeaders */ + .long 0 /* CheckSum */ + .short IMAGE_SUBSYSTEM_EFI_APPLICATION /* Subsystem */ + .short 0 /* DllCharacteristics */ + .quad 0 /* SizeOfStackReserve */ + .quad 0 /* SizeOfStackCommit */ + .quad 0 /* SizeOfHeapReserve */ + .quad 0 /* SizeOfHeapCommit */ + .long 0 /* LoaderFlags */ + .long (.Lsection_table - .) / 8 /* NumberOfRvaAndSizes */ + + .quad 0 /* ExportTable */ + .quad 0 /* ImportTable */ + .quad 0 /* ResourceTable */ + .quad 0 /* ExceptionTable */ + .quad 0 /* CertificationTable */ + .quad 0 /* BaseRelocationTable */ + + /* Section table */ +.Lsection_table: + .ascii ".text\0\0\0" + .long __inittext_end - .Lefi_header_end /* VirtualSize */ + .long .Lefi_header_end - _head /* VirtualAddress */ + .long __inittext_end - .Lefi_header_end /* SizeOfRawData */ + .long .Lefi_header_end - _head /* PointerToRawData */ + + .long 0 /* PointerToRelocations */ + .long 0 /* PointerToLineNumbers */ + .short 0 /* NumberOfRelocations */ + .short 0 /* NumberOfLineNumbers */ + .long IMAGE_SCN_CNT_CODE | \ + IMAGE_SCN_MEM_READ | \ + IMAGE_SCN_MEM_EXECUTE /* Characteristics */ + + .ascii ".data\0\0\0" + .long _end - __initdata_begin /* VirtualSize */ + .long __initdata_begin - _head /* VirtualAddress */ + .long _edata - __initdata_begin /* SizeOfRawData */ + .long __initdata_begin - _head /* PointerToRawData */ + + .long 0 /* PointerToRelocations */ + .long 0 /* PointerToLineNumbers */ + .short 0 /* NumberOfRelocations */ + .short 0 /* NumberOfLineNumbers */ + .long IMAGE_SCN_CNT_INITIALIZED_DATA | \ + IMAGE_SCN_MEM_READ | \ + IMAGE_SCN_MEM_WRITE /* Characteristics */ + + .set .Lsection_count, (. - .Lsection_table) / 40 + + .balign 0x10000 /* PECOFF_SEGMENT_ALIGN */ +.Lefi_header_end: + .endm diff --git a/arch/loongarch/kernel/efi.c b/arch/loongarch/kernel/efi.c index a50b60c587fa..a31329971133 100644 --- a/arch/loongarch/kernel/efi.c +++ b/arch/loongarch/kernel/efi.c @@ -27,8 +27,13 @@ static unsigned long efi_nr_tables; static unsigned long efi_config_table; +static unsigned long __initdata boot_memmap = EFI_INVALID_TABLE_ADDR; + static efi_system_table_t *efi_systab; -static efi_config_table_type_t arch_tables[] __initdata = {{},}; +static efi_config_table_type_t arch_tables[] __initdata = { + {LINUX_EFI_BOOT_MEMMAP_GUID, &boot_memmap, "MEMMAP" }, + {}, +}; void __init efi_runtime_init(void) { @@ -51,6 +56,7 @@ void __init efi_init(void) { int size; void *config_tables; + struct efi_boot_memmap *tbl; if (!efi_system_table) return; @@ -61,6 +67,8 @@ void __init efi_init(void) return; } + efi_systab_report_header(&efi_systab->hdr, efi_systab->fw_vendor); + set_bit(EFI_64BIT, &efi.flags); efi_nr_tables = efi_systab->nr_tables; efi_config_table = (unsigned long)efi_systab->tables; @@ -69,4 +77,27 @@ void __init efi_init(void) config_tables = early_memremap(efi_config_table, efi_nr_tables * size); efi_config_parse_tables(config_tables, efi_systab->nr_tables, arch_tables); early_memunmap(config_tables, efi_nr_tables * size); + + set_bit(EFI_CONFIG_TABLES, &efi.flags); + + if (screen_info.orig_video_isVGA == VIDEO_TYPE_EFI) + memblock_reserve(screen_info.lfb_base, screen_info.lfb_size); + + if (boot_memmap == EFI_INVALID_TABLE_ADDR) + return; + + tbl = early_memremap_ro(boot_memmap, sizeof(*tbl)); + if (tbl) { + struct efi_memory_map_data data; + + data.phys_map = boot_memmap + sizeof(*tbl); + data.size = tbl->map_size; + data.desc_size = tbl->desc_size; + data.desc_version = tbl->desc_ver; + + if (efi_memmap_init_early(&data) < 0) + panic("Unable to map EFI memory map.\n"); + + early_memunmap(tbl, sizeof(*tbl)); + } } diff --git a/arch/loongarch/kernel/env.c b/arch/loongarch/kernel/env.c index 82b478a5c665..6d56a463b091 100644 --- a/arch/loongarch/kernel/env.c +++ b/arch/loongarch/kernel/env.c @@ -8,7 +8,6 @@ #include <linux/efi.h> #include <linux/export.h> #include <linux/memblock.h> -#include <linux/of_fdt.h> #include <asm/early_ioremap.h> #include <asm/bootinfo.h> #include <asm/loongson.h> @@ -20,21 +19,17 @@ EXPORT_SYMBOL(loongson_sysconf); void __init init_environ(void) { int efi_boot = fw_arg0; - struct efi_memory_map_data data; - void *fdt_ptr = early_memremap_ro(fw_arg1, SZ_64K); + char *cmdline = early_memremap_ro(fw_arg1, COMMAND_LINE_SIZE); if (efi_boot) set_bit(EFI_BOOT, &efi.flags); else clear_bit(EFI_BOOT, &efi.flags); - early_init_dt_scan(fdt_ptr); - early_init_fdt_reserve_self(); - efi_system_table = efi_get_fdt_params(&data); + strscpy(boot_command_line, cmdline, COMMAND_LINE_SIZE); + early_memunmap(cmdline, COMMAND_LINE_SIZE); - efi_memmap_init_early(&data); - memblock_reserve(data.phys_map & PAGE_MASK, - PAGE_ALIGN(data.size + (data.phys_map & ~PAGE_MASK))); + efi_system_table = fw_arg2; } static int __init init_cpu_fullname(void) diff --git a/arch/loongarch/kernel/head.S b/arch/loongarch/kernel/head.S index 331864369e49..7e57ae8741b1 100644 --- a/arch/loongarch/kernel/head.S +++ b/arch/loongarch/kernel/head.S @@ -12,6 +12,26 @@ #include <asm/loongarch.h> #include <asm/stackframe.h> +#ifdef CONFIG_EFI_STUB + +#include "efi-header.S" + + __HEAD + +_head: + .word MZ_MAGIC /* "MZ", MS-DOS header */ + .org 0x3c /* 0x04 ~ 0x3b reserved */ + .long pe_header - _head /* Offset to the PE header */ + +pe_header: + __EFI_PE_HEADER + +SYM_DATA(kernel_asize, .long _end - _text); +SYM_DATA(kernel_fsize, .long _edata - _text); +SYM_DATA(kernel_offset, .long kernel_offset - _text); + +#endif + __REF .align 12 @@ -49,6 +69,8 @@ SYM_CODE_START(kernel_entry) # kernel entry point st.d a0, t0, 0 # firmware arguments la t0, fw_arg1 st.d a1, t0, 0 + la t0, fw_arg2 + st.d a2, t0, 0 /* KSave3 used for percpu base, initialized as 0 */ csrwr zero, PERCPU_BASE_KS diff --git a/arch/loongarch/kernel/image-vars.h b/arch/loongarch/kernel/image-vars.h new file mode 100644 index 000000000000..88f5d81702df --- /dev/null +++ b/arch/loongarch/kernel/image-vars.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited + */ +#ifndef __LOONGARCH_KERNEL_IMAGE_VARS_H +#define __LOONGARCH_KERNEL_IMAGE_VARS_H + +#ifdef CONFIG_EFI_STUB + +__efistub_memcmp = memcmp; +__efistub_memchr = memchr; +__efistub_strcat = strcat; +__efistub_strcmp = strcmp; +__efistub_strlen = strlen; +__efistub_strncat = strncat; +__efistub_strnstr = strnstr; +__efistub_strnlen = strnlen; +__efistub_strrchr = strrchr; +__efistub_kernel_entry = kernel_entry; +__efistub_kernel_asize = kernel_asize; +__efistub_kernel_fsize = kernel_fsize; +__efistub_kernel_offset = kernel_offset; +__efistub_screen_info = screen_info; + +#endif + +#endif /* __LOONGARCH_KERNEL_IMAGE_VARS_H */ diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c index d97c69dbe553..6ca5a0dcbb1c 100644 --- a/arch/loongarch/kernel/setup.c +++ b/arch/loongarch/kernel/setup.c @@ -49,11 +49,9 @@ #define SMBIOS_CORE_PACKAGE_OFFSET 0x23 #define LOONGSON_EFI_ENABLE (1 << 3) -#ifdef CONFIG_VT -struct screen_info screen_info; -#endif +struct screen_info screen_info __section(".data"); -unsigned long fw_arg0, fw_arg1; +unsigned long fw_arg0, fw_arg1, fw_arg2; DEFINE_PER_CPU(unsigned long, kernelsp); struct cpuinfo_loongarch cpu_data[NR_CPUS] __read_mostly; @@ -122,16 +120,9 @@ static void __init parse_cpu_table(const struct dmi_header *dm) static void __init parse_bios_table(const struct dmi_header *dm) { - int bios_extern; char *dmi_data = (char *)dm; - bios_extern = *(dmi_data + SMBIOS_BIOSEXTERN_OFFSET); b_info.bios_size = (*(dmi_data + SMBIOS_BIOSSIZE_OFFSET) + 1) << 6; - - if (bios_extern & LOONGSON_EFI_ENABLE) - set_bit(EFI_BOOT, &efi.flags); - else - clear_bit(EFI_BOOT, &efi.flags); } static void __init find_tokens(const struct dmi_header *dm, void *dummy) @@ -196,7 +187,6 @@ early_param("mem", early_parse_mem); void __init platform_init(void) { - efi_init(); #ifdef CONFIG_ACPI_TABLE_UPGRADE acpi_table_upgrade(); #endif @@ -356,6 +346,7 @@ void __init setup_arch(char **cmdline_p) *cmdline_p = boot_command_line; init_environ(); + efi_init(); memblock_init(); parse_early_param(); diff --git a/arch/loongarch/kernel/vmlinux.lds.S b/arch/loongarch/kernel/vmlinux.lds.S index f4831df5d2f9..e5890bec2bf6 100644 --- a/arch/loongarch/kernel/vmlinux.lds.S +++ b/arch/loongarch/kernel/vmlinux.lds.S @@ -12,6 +12,7 @@ #define BSS_FIRST_SECTIONS *(.bss..swapper_pg_dir) #include <asm-generic/vmlinux.lds.h> +#include "image-vars.h" /* * Max avaliable Page Size is 64K, so we set SectionAlignment diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 3fa8ef336822..d63295e21373 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -136,10 +136,14 @@ ifneq ($(CONFIG_XIP_KERNEL),y) ifeq ($(CONFIG_RISCV_M_MODE)$(CONFIG_SOC_CANAAN),yy) KBUILD_IMAGE := $(boot)/loader.bin else +ifeq ($(CONFIG_EFI_ZBOOT),) KBUILD_IMAGE := $(boot)/Image.gz +else +KBUILD_IMAGE := $(boot)/vmlinuz.efi +endif endif endif -BOOT_TARGETS := Image Image.gz loader loader.bin xipImage +BOOT_TARGETS := Image Image.gz loader loader.bin xipImage vmlinuz.efi all: $(notdir $(KBUILD_IMAGE)) diff --git a/arch/riscv/boot/.gitignore b/arch/riscv/boot/.gitignore index 0cea9f7fa9d5..e1bc507e8cb2 100644 --- a/arch/riscv/boot/.gitignore +++ b/arch/riscv/boot/.gitignore @@ -4,4 +4,5 @@ Image.* loader loader.lds loader.bin +vmlinuz* xipImage diff --git a/arch/riscv/boot/Makefile b/arch/riscv/boot/Makefile index becd0621071c..d1a49adcb1d7 100644 --- a/arch/riscv/boot/Makefile +++ b/arch/riscv/boot/Makefile @@ -58,3 +58,9 @@ $(obj)/Image.lzo: $(obj)/Image FORCE $(obj)/loader.bin: $(obj)/loader FORCE $(call if_changed,objcopy) + +EFI_ZBOOT_PAYLOAD := Image +EFI_ZBOOT_BFD_TARGET := elf$(BITS)-littleriscv +EFI_ZBOOT_MACH_TYPE := RISCV$(BITS) + +include $(srctree)/drivers/firmware/efi/libstub/Makefile.zboot diff --git a/arch/riscv/kernel/image-vars.h b/arch/riscv/kernel/image-vars.h index 71a76a623257..d6e5f739905e 100644 --- a/arch/riscv/kernel/image-vars.h +++ b/arch/riscv/kernel/image-vars.h @@ -25,21 +25,12 @@ */ __efistub_memcmp = memcmp; __efistub_memchr = memchr; -__efistub_memcpy = memcpy; -__efistub_memmove = memmove; -__efistub_memset = memset; __efistub_strlen = strlen; __efistub_strnlen = strnlen; __efistub_strcmp = strcmp; __efistub_strncmp = strncmp; __efistub_strrchr = strrchr; -#ifdef CONFIG_KASAN -__efistub___memcpy = memcpy; -__efistub___memmove = memmove; -__efistub___memset = memset; -#endif - __efistub__start = _start; __efistub__start_kernel = _start_kernel; __efistub__end = _end; diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index 1f3675453a57..b36596bf0fc3 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -176,7 +176,8 @@ virt_to_phys_or_null_size(void *va, unsigned long size) int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) { - unsigned long pfn, text, pf, rodata; + extern const u8 __efi64_thunk_ret_tramp[]; + unsigned long pfn, text, pf, rodata, tramp; struct page *page; unsigned npages; pgd_t *pgd = efi_mm.pgd; @@ -238,11 +239,9 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) npages = (_etext - _text) >> PAGE_SHIFT; text = __pa(_text); - pfn = text >> PAGE_SHIFT; - pf = _PAGE_ENC; - if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, pf)) { - pr_err("Failed to map kernel text 1:1\n"); + if (kernel_unmap_pages_in_pgd(pgd, text, npages)) { + pr_err("Failed to unmap kernel text 1:1 mapping\n"); return 1; } @@ -256,6 +255,15 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) return 1; } + tramp = __pa(__efi64_thunk_ret_tramp); + pfn = tramp >> PAGE_SHIFT; + + pf = _PAGE_ENC; + if (kernel_map_pages_in_pgd(pgd, pfn, tramp, 1, pf)) { + pr_err("Failed to map mixed mode return trampoline\n"); + return 1; + } + return 0; } diff --git a/arch/x86/platform/efi/efi_thunk_64.S b/arch/x86/platform/efi/efi_thunk_64.S index 4e5257a4811b..c4b1144f99f6 100644 --- a/arch/x86/platform/efi/efi_thunk_64.S +++ b/arch/x86/platform/efi/efi_thunk_64.S @@ -23,7 +23,6 @@ #include <linux/objtool.h> #include <asm/page_types.h> #include <asm/segment.h> -#include <asm/nospec-branch.h> .text .code64 @@ -73,10 +72,18 @@ STACK_FRAME_NON_STANDARD __efi64_thunk pushq %rdi /* EFI runtime service address */ lretq + // This return instruction is not needed for correctness, as it will + // never be reached. It only exists to make objtool happy, which will + // otherwise complain about unreachable instructions in the callers. + RET +SYM_FUNC_END(__efi64_thunk) + + .section ".rodata", "a", @progbits + .balign 16 +SYM_DATA_START(__efi64_thunk_ret_tramp) 1: movq 0x20(%rsp), %rsp pop %rbx pop %rbp - ANNOTATE_UNRET_SAFE ret int3 @@ -84,7 +91,7 @@ STACK_FRAME_NON_STANDARD __efi64_thunk 2: pushl $__KERNEL_CS pushl %ebp lret -SYM_FUNC_END(__efi64_thunk) +SYM_DATA_END(__efi64_thunk_ret_tramp) .bss .balign 8 diff --git a/drivers/firmware/efi/Kconfig b/drivers/firmware/efi/Kconfig index 6cb7384ad2ac..5b79a4a4a88d 100644 --- a/drivers/firmware/efi/Kconfig +++ b/drivers/firmware/efi/Kconfig @@ -105,9 +105,50 @@ config EFI_RUNTIME_WRAPPERS config EFI_GENERIC_STUB bool +config EFI_ZBOOT + bool "Enable the generic EFI decompressor" + depends on EFI_GENERIC_STUB && !ARM + select HAVE_KERNEL_GZIP + select HAVE_KERNEL_LZ4 + select HAVE_KERNEL_LZMA + select HAVE_KERNEL_LZO + select HAVE_KERNEL_XZ + select HAVE_KERNEL_ZSTD + help + Create the bootable image as an EFI application that carries the + actual kernel image in compressed form, and decompresses it into + memory before executing it via LoadImage/StartImage EFI boot service + calls. For compatibility with non-EFI loaders, the payload can be + decompressed and executed by the loader as well, provided that the + loader implements the decompression algorithm and that non-EFI boot + is supported by the encapsulated image. (The compression algorithm + used is described in the zboot image header) + +config EFI_ZBOOT_SIGNED + def_bool y + depends on EFI_ZBOOT_SIGNING_CERT != "" + depends on EFI_ZBOOT_SIGNING_KEY != "" + +config EFI_ZBOOT_SIGNING + bool "Sign the EFI decompressor for UEFI secure boot" + depends on EFI_ZBOOT + help + Use the 'sbsign' command line tool (which must exist on the host + path) to sign both the EFI decompressor PE/COFF image, as well as the + encapsulated PE/COFF image, which is subsequently compressed and + wrapped by the former image. + +config EFI_ZBOOT_SIGNING_CERT + string "Certificate to use for signing the compressed EFI boot image" + depends on EFI_ZBOOT_SIGNING + +config EFI_ZBOOT_SIGNING_KEY + string "Private key to use for signing the compressed EFI boot image" + depends on EFI_ZBOOT_SIGNING + config EFI_ARMSTUB_DTB_LOADER bool "Enable the DTB loader" - depends on EFI_GENERIC_STUB && !RISCV + depends on EFI_GENERIC_STUB && !RISCV && !LOONGARCH default y help Select this config option to add support for the dtb= command @@ -124,7 +165,7 @@ config EFI_GENERIC_STUB_INITRD_CMDLINE_LOADER bool "Enable the command line initrd loader" if !X86 depends on EFI_STUB && (EFI_GENERIC_STUB || X86) default y if X86 - depends on !RISCV + depends on !RISCV && !LOONGARCH help Select this config option to add support for the initrd= command line parameter, allowing an initrd that resides on the same volume diff --git a/drivers/firmware/efi/efi-init.c b/drivers/firmware/efi/efi-init.c index 3928dbff76d0..2fd770b499a3 100644 --- a/drivers/firmware/efi/efi-init.c +++ b/drivers/firmware/efi/efi-init.c @@ -51,34 +51,10 @@ static phys_addr_t __init efi_to_phys(unsigned long addr) return addr; } -static __initdata unsigned long screen_info_table = EFI_INVALID_TABLE_ADDR; -static __initdata unsigned long cpu_state_table = EFI_INVALID_TABLE_ADDR; - -static const efi_config_table_type_t arch_tables[] __initconst = { - {LINUX_EFI_ARM_SCREEN_INFO_TABLE_GUID, &screen_info_table}, - {LINUX_EFI_ARM_CPU_STATE_TABLE_GUID, &cpu_state_table}, - {} -}; +extern __weak const efi_config_table_type_t efi_arch_tables[]; static void __init init_screen_info(void) { - struct screen_info *si; - - if (IS_ENABLED(CONFIG_ARM) && - screen_info_table != EFI_INVALID_TABLE_ADDR) { - si = early_memremap_ro(screen_info_table, sizeof(*si)); - if (!si) { - pr_err("Could not map screen_info config table\n"); - return; - } - screen_info = *si; - early_memunmap(si, sizeof(*si)); - - /* dummycon on ARM needs non-zero values for columns/lines */ - screen_info.orig_video_cols = 80; - screen_info.orig_video_lines = 25; - } - if (screen_info.orig_video_isVGA == VIDEO_TYPE_EFI && memblock_is_map_memory(screen_info.lfb_base)) memblock_mark_nomap(screen_info.lfb_base, screen_info.lfb_size); @@ -119,8 +95,7 @@ static int __init uefi_init(u64 efi_system_table) goto out; } retval = efi_config_parse_tables(config_tables, systab->nr_tables, - IS_ENABLED(CONFIG_ARM) ? arch_tables - : NULL); + efi_arch_tables); early_memunmap(config_tables, table_size); out: @@ -248,36 +223,4 @@ void __init efi_init(void) PAGE_ALIGN(data.size + (data.phys_map & ~PAGE_MASK))); init_screen_info(); - -#ifdef CONFIG_ARM - /* ARM does not permit early mappings to persist across paging_init() */ - efi_memmap_unmap(); - - if (cpu_state_table != EFI_INVALID_TABLE_ADDR) { - struct efi_arm_entry_state *state; - bool dump_state = true; - - state = early_memremap_ro(cpu_state_table, - sizeof(struct efi_arm_entry_state)); - if (state == NULL) { - pr_warn("Unable to map CPU entry state table.\n"); - return; - } - - if ((state->sctlr_before_ebs & 1) == 0) - pr_warn(FW_BUG "EFI stub was entered with MMU and Dcache disabled, please fix your firmware!\n"); - else if ((state->sctlr_after_ebs & 1) == 0) - pr_warn(FW_BUG "ExitBootServices() returned with MMU and Dcache disabled, please fix your firmware!\n"); - else - dump_state = false; - - if (dump_state || efi_enabled(EFI_DBG)) { - pr_info("CPSR at EFI stub entry : 0x%08x\n", state->cpsr_before_ebs); - pr_info("SCTLR at EFI stub entry : 0x%08x\n", state->sctlr_before_ebs); - pr_info("CPSR after ExitBootServices() : 0x%08x\n", state->cpsr_after_ebs); - pr_info("SCTLR after ExitBootServices(): 0x%08x\n", state->sctlr_after_ebs); - } - early_memunmap(state, sizeof(struct efi_arm_entry_state)); - } -#endif } diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c index e4080ad96089..11857af72859 100644 --- a/drivers/firmware/efi/efi.c +++ b/drivers/firmware/efi/efi.c @@ -21,6 +21,7 @@ #include <linux/device.h> #include <linux/efi.h> #include <linux/of.h> +#include <linux/initrd.h> #include <linux/io.h> #include <linux/kexec.h> #include <linux/platform_device.h> @@ -55,6 +56,7 @@ EXPORT_SYMBOL(efi); unsigned long __ro_after_init efi_rng_seed = EFI_INVALID_TABLE_ADDR; static unsigned long __initdata mem_reserve = EFI_INVALID_TABLE_ADDR; static unsigned long __initdata rt_prop = EFI_INVALID_TABLE_ADDR; +static unsigned long __initdata initrd = EFI_INVALID_TABLE_ADDR; struct mm_struct efi_mm = { .mm_rb = RB_ROOT, @@ -532,6 +534,7 @@ static const efi_config_table_type_t common_tables[] __initconst = { {LINUX_EFI_TPM_EVENT_LOG_GUID, &efi.tpm_log, "TPMEventLog" }, {LINUX_EFI_TPM_FINAL_LOG_GUID, &efi.tpm_final_log, "TPMFinalLog" }, {LINUX_EFI_MEMRESERVE_TABLE_GUID, &mem_reserve, "MEMRESERVE" }, + {LINUX_EFI_INITRD_MEDIA_GUID, &initrd, "INITRD" }, {EFI_RT_PROPERTIES_TABLE_GUID, &rt_prop, "RTPROP" }, #ifdef CONFIG_EFI_RCI2_TABLE {DELLEMC_EFI_RCI2_TABLE_GUID, &rci2_table_phys }, @@ -674,6 +677,18 @@ int __init efi_config_parse_tables(const efi_config_table_t *config_tables, } } + if (IS_ENABLED(CONFIG_BLK_DEV_INITRD) && + initrd != EFI_INVALID_TABLE_ADDR && phys_initrd_size == 0) { + struct linux_efi_initrd *tbl; + + tbl = early_memremap(initrd, sizeof(*tbl)); + if (tbl) { + phys_initrd_start = tbl->base; + phys_initrd_size = tbl->size; + early_memunmap(tbl, sizeof(*tbl)); + } + } + return 0; } diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index b43fdb319fd4..01a01be4a2a0 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -26,8 +26,10 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \ $(call cc-option,-mno-single-pic-base) cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \ -fpic +cflags-$(CONFIG_LOONGARCH) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \ + -fpie -cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt +cflags-$(CONFIG_EFI_PARAMS_FROM_FDT) += -I$(srctree)/scripts/dtc/libfdt KBUILD_CFLAGS := $(cflags-y) -Os -DDISABLE_BRANCH_PROFILING \ -include $(srctree)/include/linux/hidden.h \ @@ -66,21 +68,32 @@ lib-y := efi-stub-helper.o gop.o secureboot.o tpm.o \ skip_spaces.o lib-cmdline.o lib-ctype.o \ alignedmem.o relocate.o vsprintf.o -# include the stub's generic dependencies from lib/ when building for ARM/arm64 -efi-deps-y := fdt_rw.c fdt_ro.c fdt_wip.c fdt.c fdt_empty_tree.c fdt_sw.c +# include the stub's libfdt dependencies from lib/ when needed +libfdt-deps := fdt_rw.c fdt_ro.c fdt_wip.c fdt.c \ + fdt_empty_tree.c fdt_sw.c + +lib-$(CONFIG_EFI_PARAMS_FROM_FDT) += fdt.o \ + $(patsubst %.c,lib-%.o,$(libfdt-deps)) $(obj)/lib-%.o: $(srctree)/lib/%.c FORCE $(call if_changed_rule,cc_o_c) -lib-$(CONFIG_EFI_GENERIC_STUB) += efi-stub.o fdt.o string.o \ - $(patsubst %.c,lib-%.o,$(efi-deps-y)) +lib-$(CONFIG_EFI_GENERIC_STUB) += efi-stub.o string.o intrinsics.o systable.o lib-$(CONFIG_ARM) += arm32-stub.o lib-$(CONFIG_ARM64) += arm64-stub.o lib-$(CONFIG_X86) += x86-stub.o lib-$(CONFIG_RISCV) += riscv-stub.o +lib-$(CONFIG_LOONGARCH) += loongarch-stub.o + CFLAGS_arm32-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET) +zboot-obj-$(CONFIG_RISCV) := lib-clz_ctz.o lib-ashldi3.o +lib-$(CONFIG_EFI_ZBOOT) += zboot.o $(zboot-obj-y) + +extra-y := $(lib-y) +lib-y := $(patsubst %.o,%.stub.o,$(lib-y)) + # Even when -mbranch-protection=none is set, Clang will generate a # .note.gnu.property for code-less object files (like lib/ctype.c), # so work around this by explicitly removing the unwanted section. @@ -120,9 +133,6 @@ STUBCOPY_RELOC-$(CONFIG_ARM) := R_ARM_ABS # a verification pass to see if any absolute relocations exist in any of the # object files. # -extra-y := $(lib-y) -lib-y := $(patsubst %.o,%.stub.o,$(lib-y)) - STUBCOPY_FLAGS-$(CONFIG_ARM64) += --prefix-alloc-sections=.init \ --prefix-symbols=__efistub_ STUBCOPY_RELOC-$(CONFIG_ARM64) := R_AARCH64_ABS @@ -134,6 +144,12 @@ STUBCOPY_FLAGS-$(CONFIG_RISCV) += --prefix-alloc-sections=.init \ --prefix-symbols=__efistub_ STUBCOPY_RELOC-$(CONFIG_RISCV) := R_RISCV_HI20 +# For LoongArch, keep all the symbols in .init section and make sure that no +# absolute symbols references exist. +STUBCOPY_FLAGS-$(CONFIG_LOONGARCH) += --prefix-alloc-sections=.init \ + --prefix-symbols=__efistub_ +STUBCOPY_RELOC-$(CONFIG_LOONGARCH) := R_LARCH_MARK_LA + $(obj)/%.stub.o: $(obj)/%.o FORCE $(call if_changed,stubcopy) diff --git a/drivers/firmware/efi/libstub/Makefile.zboot b/drivers/firmware/efi/libstub/Makefile.zboot new file mode 100644 index 000000000000..35f234ad8738 --- /dev/null +++ b/drivers/firmware/efi/libstub/Makefile.zboot @@ -0,0 +1,70 @@ +# SPDX-License-Identifier: GPL-2.0 + +# to be include'd by arch/$(ARCH)/boot/Makefile after setting +# EFI_ZBOOT_PAYLOAD, EFI_ZBOOT_BFD_TARGET and EFI_ZBOOT_MACH_TYPE + +comp-type-$(CONFIG_KERNEL_GZIP) := gzip +comp-type-$(CONFIG_KERNEL_LZ4) := lz4 +comp-type-$(CONFIG_KERNEL_LZMA) := lzma +comp-type-$(CONFIG_KERNEL_LZO) := lzo +comp-type-$(CONFIG_KERNEL_XZ) := xzkern +comp-type-$(CONFIG_KERNEL_ZSTD) := zstd22 + +# in GZIP, the appended le32 carrying the uncompressed size is part of the +# format, but in other cases, we just append it at the end for convenience, +# causing the original tools to complain when checking image integrity. +# So disregard it when calculating the payload size in the zimage header. +zboot-method-y := $(comp-type-y)_with_size +zboot-size-len-y := 4 + +zboot-method-$(CONFIG_KERNEL_GZIP) := gzip +zboot-size-len-$(CONFIG_KERNEL_GZIP) := 0 + +quiet_cmd_sbsign = SBSIGN $@ + cmd_sbsign = sbsign --out $@ $< \ + --key $(CONFIG_EFI_ZBOOT_SIGNING_KEY) \ + --cert $(CONFIG_EFI_ZBOOT_SIGNING_CERT) + +$(obj)/$(EFI_ZBOOT_PAYLOAD).signed: $(obj)/$(EFI_ZBOOT_PAYLOAD) FORCE + $(call if_changed,sbsign) + +ZBOOT_PAYLOAD-y := $(EFI_ZBOOT_PAYLOAD) +ZBOOT_PAYLOAD-$(CONFIG_EFI_ZBOOT_SIGNED) := $(EFI_ZBOOT_PAYLOAD).signed + +$(obj)/vmlinuz: $(obj)/$(ZBOOT_PAYLOAD-y) FORCE + $(call if_changed,$(zboot-method-y)) + +OBJCOPYFLAGS_vmlinuz.o := -I binary -O $(EFI_ZBOOT_BFD_TARGET) \ + --rename-section .data=.gzdata,load,alloc,readonly,contents +$(obj)/vmlinuz.o: $(obj)/vmlinuz FORCE + $(call if_changed,objcopy) + +AFLAGS_zboot-header.o += -DMACHINE_TYPE=IMAGE_FILE_MACHINE_$(EFI_ZBOOT_MACH_TYPE) \ + -DZBOOT_EFI_PATH="\"$(realpath $(obj)/vmlinuz.efi.elf)\"" \ + -DZBOOT_SIZE_LEN=$(zboot-size-len-y) \ + -DCOMP_TYPE="\"$(comp-type-y)\"" + +$(obj)/zboot-header.o: $(srctree)/drivers/firmware/efi/libstub/zboot-header.S FORCE + $(call if_changed_rule,as_o_S) + +ZBOOT_DEPS := $(obj)/zboot-header.o $(objtree)/drivers/firmware/efi/libstub/lib.a + +LDFLAGS_vmlinuz.efi.elf := -T $(srctree)/drivers/firmware/efi/libstub/zboot.lds +$(obj)/vmlinuz.efi.elf: $(obj)/vmlinuz.o $(ZBOOT_DEPS) FORCE + $(call if_changed,ld) + +ZBOOT_EFI-y := vmlinuz.efi +ZBOOT_EFI-$(CONFIG_EFI_ZBOOT_SIGNED) := vmlinuz.efi.unsigned + +OBJCOPYFLAGS_$(ZBOOT_EFI-y) := -O binary +$(obj)/$(ZBOOT_EFI-y): $(obj)/vmlinuz.efi.elf FORCE + $(call if_changed,objcopy) + +targets += zboot-header.o vmlinuz vmlinuz.o vmlinuz.efi.elf vmlinuz.efi + +ifneq ($(CONFIG_EFI_ZBOOT_SIGNED),) +$(obj)/vmlinuz.efi: $(obj)/vmlinuz.efi.unsigned FORCE + $(call if_changed,sbsign) +endif + +targets += $(EFI_ZBOOT_PAYLOAD).signed vmlinuz.efi.unsigned diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c index 60973e84d7ab..259e4b852d63 100644 --- a/drivers/firmware/efi/libstub/arm64-stub.c +++ b/drivers/firmware/efi/libstub/arm64-stub.c @@ -19,6 +19,14 @@ efi_status_t check_platform_features(void) { u64 tg; + /* + * If we have 48 bits of VA space for TTBR0 mappings, we can map the + * UEFI runtime regions 1:1 and so calling SetVirtualAddressMap() is + * unnecessary. + */ + if (VA_BITS_MIN >= 48) + efi_novamap = true; + /* UEFI mandates support for 4 KB granularity, no need to check */ if (IS_ENABLED(CONFIG_ARM64_4K_PAGES)) return EFI_SUCCESS; @@ -42,26 +50,17 @@ efi_status_t check_platform_features(void) */ static bool check_image_region(u64 base, u64 size) { - unsigned long map_size, desc_size, buff_size; - efi_memory_desc_t *memory_map; - struct efi_boot_memmap map; + struct efi_boot_memmap *map; efi_status_t status; bool ret = false; int map_offset; - map.map = &memory_map; - map.map_size = &map_size; - map.desc_size = &desc_size; - map.desc_ver = NULL; - map.key_ptr = NULL; - map.buff_size = &buff_size; - - status = efi_get_memory_map(&map); + status = efi_get_memory_map(&map, false); if (status != EFI_SUCCESS) return false; - for (map_offset = 0; map_offset < map_size; map_offset += desc_size) { - efi_memory_desc_t *md = (void *)memory_map + map_offset; + for (map_offset = 0; map_offset < map->map_size; map_offset += map->desc_size) { + efi_memory_desc_t *md = (void *)map->map + map_offset; u64 end = md->phys_addr + md->num_pages * EFI_PAGE_SIZE; /* @@ -74,7 +73,7 @@ static bool check_image_region(u64 base, u64 size) } } - efi_bs_call(free_pool, memory_map); + efi_bs_call(free_pool, map); return ret; } diff --git a/drivers/firmware/efi/libstub/efi-stub-helper.c b/drivers/firmware/efi/libstub/efi-stub-helper.c index 3d972061c1b0..0c493521b25b 100644 --- a/drivers/firmware/efi/libstub/efi-stub-helper.c +++ b/drivers/firmware/efi/libstub/efi-stub-helper.c @@ -218,7 +218,7 @@ efi_status_t efi_parse_options(char const *cmdline) efi_noinitrd = true; } else if (!strcmp(param, "efi") && val) { efi_nochunk = parse_option_str(val, "nochunk"); - efi_novamap = parse_option_str(val, "novamap"); + efi_novamap |= parse_option_str(val, "novamap"); efi_nosoftreserve = IS_ENABLED(CONFIG_EFI_SOFT_RESERVE) && parse_option_str(val, "nosoftreserve"); @@ -310,7 +310,7 @@ bool efi_load_option_unpack(efi_load_option_unpacked_t *dest, * * Detect this case and extract OptionalData. */ -void efi_apply_loadoptions_quirk(const void **load_options, int *load_options_size) +void efi_apply_loadoptions_quirk(const void **load_options, u32 *load_options_size) { const efi_load_option_t *load_option = *load_options; efi_load_option_unpacked_t load_option_unpacked; @@ -334,6 +334,85 @@ void efi_apply_loadoptions_quirk(const void **load_options, int *load_options_si *load_options_size = load_option_unpacked.optional_data_size; } +enum efistub_event { + EFISTUB_EVT_INITRD, + EFISTUB_EVT_LOAD_OPTIONS, + EFISTUB_EVT_COUNT, +}; + +#define STR_WITH_SIZE(s) sizeof(s), s + +static const struct { + u32 pcr_index; + u32 event_id; + u32 event_data_len; + u8 event_data[52]; +} events[] = { + [EFISTUB_EVT_INITRD] = { + 9, + INITRD_EVENT_TAG_ID, + STR_WITH_SIZE("Linux initrd") + }, + [EFISTUB_EVT_LOAD_OPTIONS] = { + 9, + LOAD_OPTIONS_EVENT_TAG_ID, + STR_WITH_SIZE("LOADED_IMAGE::LoadOptions") + }, +}; + +static efi_status_t efi_measure_tagged_event(unsigned long load_addr, + unsigned long load_size, + enum efistub_event event) +{ + efi_guid_t tcg2_guid = EFI_TCG2_PROTOCOL_GUID; + efi_tcg2_protocol_t *tcg2 = NULL; + efi_status_t status; + + efi_bs_call(locate_protocol, &tcg2_guid, NULL, (void **)&tcg2); + if (tcg2) { + struct efi_measured_event { + efi_tcg2_event_t event_data; + efi_tcg2_tagged_event_t tagged_event; + u8 tagged_event_data[]; + } *evt; + int size = sizeof(*evt) + events[event].event_data_len; + + status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, size, + (void **)&evt); + if (status != EFI_SUCCESS) + goto fail; + + evt->event_data = (struct efi_tcg2_event){ + .event_size = size, + .event_header.header_size = sizeof(evt->event_data.event_header), + .event_header.header_version = EFI_TCG2_EVENT_HEADER_VERSION, + .event_header.pcr_index = events[event].pcr_index, + .event_header.event_type = EV_EVENT_TAG, + }; + + evt->tagged_event = (struct efi_tcg2_tagged_event){ + .tagged_event_id = events[event].event_id, + .tagged_event_data_size = events[event].event_data_len, + }; + + memcpy(evt->tagged_event_data, events[event].event_data, + events[event].event_data_len); + + status = efi_call_proto(tcg2, hash_log_extend_event, 0, + load_addr, load_size, &evt->event_data); + efi_bs_call(free_pool, evt); + + if (status != EFI_SUCCESS) + goto fail; + return EFI_SUCCESS; + } + + return EFI_UNSUPPORTED; +fail: + efi_warn("Failed to measure data for event %d: 0x%lx\n", event, status); + return status; +} + /* * Convert the unicode UEFI command line to ASCII to pass to kernel. * Size of memory allocated return in *cmd_line_len. @@ -341,21 +420,26 @@ void efi_apply_loadoptions_quirk(const void **load_options, int *load_options_si */ char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len) { - const u16 *s2; - unsigned long cmdline_addr = 0; - int options_chars = efi_table_attr(image, load_options_size); - const u16 *options = efi_table_attr(image, load_options); + const efi_char16_t *options = efi_table_attr(image, load_options); + u32 options_size = efi_table_attr(image, load_options_size); int options_bytes = 0, safe_options_bytes = 0; /* UTF-8 bytes */ + unsigned long cmdline_addr = 0; + const efi_char16_t *s2; bool in_quote = false; efi_status_t status; + u32 options_chars; + + if (options_size > 0) + efi_measure_tagged_event((unsigned long)options, options_size, + EFISTUB_EVT_LOAD_OPTIONS); - efi_apply_loadoptions_quirk((const void **)&options, &options_chars); - options_chars /= sizeof(*options); + efi_apply_loadoptions_quirk((const void **)&options, &options_size); + options_chars = options_size / sizeof(efi_char16_t); if (options) { s2 = options; while (options_bytes < COMMAND_LINE_SIZE && options_chars--) { - u16 c = *s2++; + efi_char16_t c = *s2++; if (c < 0x80) { if (c == L'\0' || c == L'\n') @@ -419,7 +503,6 @@ char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len) /** * efi_exit_boot_services() - Exit boot services * @handle: handle of the exiting image - * @map: pointer to receive the memory map * @priv: argument to be passed to @priv_func * @priv_func: function to process the memory map before exiting boot services * @@ -432,26 +515,26 @@ char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len) * * Return: status code */ -efi_status_t efi_exit_boot_services(void *handle, - struct efi_boot_memmap *map, - void *priv, +efi_status_t efi_exit_boot_services(void *handle, void *priv, efi_exit_boot_map_processing priv_func) { + struct efi_boot_memmap *map; efi_status_t status; - status = efi_get_memory_map(map); - + status = efi_get_memory_map(&map, true); if (status != EFI_SUCCESS) - goto fail; + return status; status = priv_func(map, priv); - if (status != EFI_SUCCESS) - goto free_map; + if (status != EFI_SUCCESS) { + efi_bs_call(free_pool, map); + return status; + } if (efi_disable_pci_dma) efi_pci_disable_bridge_busmaster(); - status = efi_bs_call(exit_boot_services, handle, *map->key_ptr); + status = efi_bs_call(exit_boot_services, handle, map->map_key); if (status == EFI_INVALID_PARAMETER) { /* @@ -467,35 +550,26 @@ efi_status_t efi_exit_boot_services(void *handle, * buffer should account for any changes in the map so the call * to get_memory_map() is expected to succeed here. */ - *map->map_size = *map->buff_size; + map->map_size = map->buff_size; status = efi_bs_call(get_memory_map, - map->map_size, - *map->map, - map->key_ptr, - map->desc_size, - map->desc_ver); + &map->map_size, + &map->map, + &map->map_key, + &map->desc_size, + &map->desc_ver); /* exit_boot_services() was called, thus cannot free */ if (status != EFI_SUCCESS) - goto fail; + return status; status = priv_func(map, priv); /* exit_boot_services() was called, thus cannot free */ if (status != EFI_SUCCESS) - goto fail; + return status; - status = efi_bs_call(exit_boot_services, handle, *map->key_ptr); + status = efi_bs_call(exit_boot_services, handle, map->map_key); } - /* exit_boot_services() was called, thus cannot free */ - if (status != EFI_SUCCESS) - goto fail; - - return EFI_SUCCESS; - -free_map: - efi_bs_call(free_pool, *map->map); -fail: return status; } @@ -560,20 +634,16 @@ static const struct { * * %EFI_SUCCESS if the initrd was loaded successfully, in which * case @load_addr and @load_size are assigned accordingly * * %EFI_NOT_FOUND if no LoadFile2 protocol exists on the initrd device path - * * %EFI_INVALID_PARAMETER if load_addr == NULL or load_size == NULL * * %EFI_OUT_OF_RESOURCES if memory allocation failed * * %EFI_LOAD_ERROR in all other cases */ static -efi_status_t efi_load_initrd_dev_path(unsigned long *load_addr, - unsigned long *load_size, +efi_status_t efi_load_initrd_dev_path(struct linux_efi_initrd *initrd, unsigned long max) { efi_guid_t lf2_proto_guid = EFI_LOAD_FILE2_PROTOCOL_GUID; efi_device_path_protocol_t *dp; efi_load_file2_protocol_t *lf2; - unsigned long initrd_addr; - unsigned long initrd_size; efi_handle_t handle; efi_status_t status; @@ -587,124 +657,98 @@ efi_status_t efi_load_initrd_dev_path(unsigned long *load_addr, if (status != EFI_SUCCESS) return status; - status = efi_call_proto(lf2, load_file, dp, false, &initrd_size, NULL); + initrd->size = 0; + status = efi_call_proto(lf2, load_file, dp, false, &initrd->size, NULL); if (status != EFI_BUFFER_TOO_SMALL) return EFI_LOAD_ERROR; - status = efi_allocate_pages(initrd_size, &initrd_addr, max); + status = efi_allocate_pages(initrd->size, &initrd->base, max); if (status != EFI_SUCCESS) return status; - status = efi_call_proto(lf2, load_file, dp, false, &initrd_size, - (void *)initrd_addr); + status = efi_call_proto(lf2, load_file, dp, false, &initrd->size, + (void *)initrd->base); if (status != EFI_SUCCESS) { - efi_free(initrd_size, initrd_addr); + efi_free(initrd->size, initrd->base); return EFI_LOAD_ERROR; } - - *load_addr = initrd_addr; - *load_size = initrd_size; return EFI_SUCCESS; } static efi_status_t efi_load_initrd_cmdline(efi_loaded_image_t *image, - unsigned long *load_addr, - unsigned long *load_size, + struct linux_efi_initrd *initrd, unsigned long soft_limit, unsigned long hard_limit) { if (!IS_ENABLED(CONFIG_EFI_GENERIC_STUB_INITRD_CMDLINE_LOADER) || - (IS_ENABLED(CONFIG_X86) && (!efi_is_native() || image == NULL))) { - *load_addr = *load_size = 0; - return EFI_SUCCESS; - } + (IS_ENABLED(CONFIG_X86) && (!efi_is_native() || image == NULL))) + return EFI_UNSUPPORTED; return handle_cmdline_files(image, L"initrd=", sizeof(L"initrd=") - 2, soft_limit, hard_limit, - load_addr, load_size); -} - -static const struct { - efi_tcg2_event_t event_data; - efi_tcg2_tagged_event_t tagged_event; - u8 tagged_event_data[]; -} initrd_tcg2_event = { - { - sizeof(initrd_tcg2_event) + sizeof("Linux initrd"), - { - sizeof(initrd_tcg2_event.event_data.event_header), - EFI_TCG2_EVENT_HEADER_VERSION, - 9, - EV_EVENT_TAG, - }, - }, - { - INITRD_EVENT_TAG_ID, - sizeof("Linux initrd"), - }, - { "Linux initrd" }, -}; - -static void efi_measure_initrd(unsigned long load_addr, unsigned long load_size) -{ - efi_guid_t tcg2_guid = EFI_TCG2_PROTOCOL_GUID; - efi_tcg2_protocol_t *tcg2 = NULL; - efi_status_t status; - - efi_bs_call(locate_protocol, &tcg2_guid, NULL, (void **)&tcg2); - if (tcg2) { - status = efi_call_proto(tcg2, hash_log_extend_event, - 0, load_addr, load_size, - &initrd_tcg2_event.event_data); - if (status != EFI_SUCCESS) - efi_warn("Failed to measure initrd data: 0x%lx\n", - status); - else - efi_info("Measured initrd data into PCR %d\n", - initrd_tcg2_event.event_data.event_header.pcr_index); - } + &initrd->base, &initrd->size); } /** * efi_load_initrd() - Load initial RAM disk * @image: EFI loaded image protocol - * @load_addr: pointer to loaded initrd - * @load_size: size of loaded initrd * @soft_limit: preferred address for loading the initrd * @hard_limit: upper limit address for loading the initrd * * Return: status code */ efi_status_t efi_load_initrd(efi_loaded_image_t *image, - unsigned long *load_addr, - unsigned long *load_size, unsigned long soft_limit, - unsigned long hard_limit) + unsigned long hard_limit, + const struct linux_efi_initrd **out) { - efi_status_t status; + efi_guid_t tbl_guid = LINUX_EFI_INITRD_MEDIA_GUID; + efi_status_t status = EFI_SUCCESS; + struct linux_efi_initrd initrd, *tbl; - if (efi_noinitrd) { - *load_addr = *load_size = 0; - status = EFI_SUCCESS; - } else { - status = efi_load_initrd_dev_path(load_addr, load_size, hard_limit); - if (status == EFI_SUCCESS) { - efi_info("Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path\n"); - if (*load_size > 0) - efi_measure_initrd(*load_addr, *load_size); - } else if (status == EFI_NOT_FOUND) { - status = efi_load_initrd_cmdline(image, load_addr, load_size, - soft_limit, hard_limit); - if (status == EFI_SUCCESS && *load_size > 0) - efi_info("Loaded initrd from command line option\n"); - } - if (status != EFI_SUCCESS) { - efi_err("Failed to load initrd: 0x%lx\n", status); - *load_addr = *load_size = 0; - } + if (!IS_ENABLED(CONFIG_BLK_DEV_INITRD) || efi_noinitrd) + return EFI_SUCCESS; + + status = efi_load_initrd_dev_path(&initrd, hard_limit); + if (status == EFI_SUCCESS) { + efi_info("Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path\n"); + if (initrd.size > 0 && + efi_measure_tagged_event(initrd.base, initrd.size, + EFISTUB_EVT_INITRD) == EFI_SUCCESS) + efi_info("Measured initrd data into PCR 9\n"); + } else if (status == EFI_NOT_FOUND) { + status = efi_load_initrd_cmdline(image, &initrd, soft_limit, + hard_limit); + /* command line loader disabled or no initrd= passed? */ + if (status == EFI_UNSUPPORTED || status == EFI_NOT_READY) + return EFI_SUCCESS; + if (status == EFI_SUCCESS) + efi_info("Loaded initrd from command line option\n"); } + if (status != EFI_SUCCESS) + goto failed; + + status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, sizeof(initrd), + (void **)&tbl); + if (status != EFI_SUCCESS) + goto free_initrd; + + *tbl = initrd; + status = efi_bs_call(install_configuration_table, &tbl_guid, tbl); + if (status != EFI_SUCCESS) + goto free_tbl; + + if (out) + *out = tbl; + return EFI_SUCCESS; +free_tbl: + efi_bs_call(free_pool, tbl); +free_initrd: + efi_free(initrd.size, initrd.base); +failed: + efi_err("Failed to load initrd: 0x%lx\n", status); return status; } diff --git a/drivers/firmware/efi/libstub/efi-stub.c b/drivers/firmware/efi/libstub/efi-stub.c index f515394cce6e..cf474f0dd261 100644 --- a/drivers/firmware/efi/libstub/efi-stub.c +++ b/drivers/firmware/efi/libstub/efi-stub.c @@ -10,7 +10,6 @@ */ #include <linux/efi.h> -#include <linux/libfdt.h> #include <asm/efi.h> #include "efistub.h" @@ -40,16 +39,22 @@ #ifdef CONFIG_ARM64 # define EFI_RT_VIRTUAL_LIMIT DEFAULT_MAP_WINDOW_64 -#elif defined(CONFIG_RISCV) +#elif defined(CONFIG_RISCV) || defined(CONFIG_LOONGARCH) # define EFI_RT_VIRTUAL_LIMIT TASK_SIZE_MIN -#else +#else /* Only if TASK_SIZE is a constant */ # define EFI_RT_VIRTUAL_LIMIT TASK_SIZE #endif -static u64 virtmap_base = EFI_RT_VIRTUAL_BASE; -static bool flat_va_mapping; +/* + * Some architectures map the EFI regions into the kernel's linear map using a + * fixed offset. + */ +#ifndef EFI_RT_VIRTUAL_OFFSET +#define EFI_RT_VIRTUAL_OFFSET 0 +#endif -const efi_system_table_t *efi_system_table; +static u64 virtmap_base = EFI_RT_VIRTUAL_BASE; +static bool flat_va_mapping = (EFI_RT_VIRTUAL_OFFSET != 0); static struct screen_info *setup_graphics(void) { @@ -124,16 +129,11 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, unsigned long image_addr; unsigned long image_size = 0; /* addr/point and size pairs for memory management*/ - unsigned long initrd_addr = 0; - unsigned long initrd_size = 0; - unsigned long fdt_addr = 0; /* Original DTB */ - unsigned long fdt_size = 0; char *cmdline_ptr = NULL; int cmdline_size = 0; efi_guid_t loaded_image_proto = LOADED_IMAGE_PROTOCOL_GUID; unsigned long reserve_addr = 0; unsigned long reserve_size = 0; - enum efi_secureboot_mode secure_boot; struct screen_info *si; efi_properties_table_t *prop_tbl; @@ -154,8 +154,8 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, * information about the running image, such as size and the command * line. */ - status = efi_system_table->boottime->handle_protocol(handle, - &loaded_image_proto, (void *)&image); + status = efi_bs_call(handle_protocol, handle, &loaded_image_proto, + (void *)&image); if (status != EFI_SUCCESS) { efi_err("Failed to get loaded image protocol\n"); goto fail; @@ -209,40 +209,8 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, /* Ask the firmware to clear memory on unclean shutdown */ efi_enable_reset_attack_mitigation(); - secure_boot = efi_get_secureboot(); - - /* - * Unauthenticated device tree data is a security hazard, so ignore - * 'dtb=' unless UEFI Secure Boot is disabled. We assume that secure - * boot is enabled if we can't determine its state. - */ - if (!IS_ENABLED(CONFIG_EFI_ARMSTUB_DTB_LOADER) || - secure_boot != efi_secureboot_mode_disabled) { - if (strstr(cmdline_ptr, "dtb=")) - efi_err("Ignoring DTB from command line.\n"); - } else { - status = efi_load_dtb(image, &fdt_addr, &fdt_size); - - if (status != EFI_SUCCESS) { - efi_err("Failed to load device tree!\n"); - goto fail_free_image; - } - } - - if (fdt_addr) { - efi_info("Using DTB from command line\n"); - } else { - /* Look for a device tree configuration table entry. */ - fdt_addr = (uintptr_t)get_fdt(&fdt_size); - if (fdt_addr) - efi_info("Using DTB from configuration table\n"); - } - - if (!fdt_addr) - efi_info("Generating empty DTB\n"); - - efi_load_initrd(image, &initrd_addr, &initrd_size, ULONG_MAX, - efi_get_max_initrd_addr(image_addr)); + efi_load_initrd(image, ULONG_MAX, efi_get_max_initrd_addr(image_addr), + NULL); efi_random_get_seed(); @@ -254,8 +222,8 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, * The easiest way to achieve that is to simply use a 1:1 mapping. */ prop_tbl = get_efi_config_table(EFI_PROPERTIES_TABLE_GUID); - flat_va_mapping = prop_tbl && - (prop_tbl->memory_protection_attribute & + flat_va_mapping |= prop_tbl && + (prop_tbl->memory_protection_attribute & EFI_PROPERTIES_RUNTIME_MEMORY_PROTECTION_NON_EXECUTABLE_PE_DATA); /* force efi_novamap if SetVirtualAddressMap() is unsupported */ @@ -284,25 +252,8 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle, install_memreserve_table(); - status = allocate_new_fdt_and_exit_boot(handle, &fdt_addr, - initrd_addr, initrd_size, - cmdline_ptr, fdt_addr, fdt_size); - if (status != EFI_SUCCESS) - goto fail_free_initrd; - - if (IS_ENABLED(CONFIG_ARM)) - efi_handle_post_ebs_state(); - - efi_enter_kernel(image_addr, fdt_addr, fdt_totalsize((void *)fdt_addr)); - /* not reached */ - -fail_free_initrd: - efi_err("Failed to update FDT and exit boot services\n"); + status = efi_boot_kernel(handle, image, image_addr, cmdline_ptr); - efi_free(initrd_size, initrd_addr); - efi_free(fdt_size, fdt_addr); - -fail_free_image: efi_free(image_size, image_addr); efi_free(reserve_size, reserve_addr); fail_free_screeninfo: @@ -314,6 +265,35 @@ fail: } /* + * efi_allocate_virtmap() - create a pool allocation for the virtmap + * + * Create an allocation that is of sufficient size to hold all the memory + * descriptors that will be passed to SetVirtualAddressMap() to inform the + * firmware about the virtual mapping that will be used under the OS to call + * into the firmware. + */ +efi_status_t efi_alloc_virtmap(efi_memory_desc_t **virtmap, + unsigned long *desc_size, u32 *desc_ver) +{ + unsigned long size, mmap_key; + efi_status_t status; + + /* + * Use the size of the current memory map as an upper bound for the + * size of the buffer we need to pass to SetVirtualAddressMap() to + * cover all EFI_MEMORY_RUNTIME regions. + */ + size = 0; + status = efi_bs_call(get_memory_map, &size, NULL, &mmap_key, desc_size, + desc_ver); + if (status != EFI_BUFFER_TOO_SMALL) + return EFI_LOAD_ERROR; + + return efi_bs_call(allocate_pool, EFI_LOADER_DATA, size, + (void **)virtmap); +} + +/* * efi_get_virtmap() - create a virtual mapping for the EFI memory map * * This function populates the virt_addr fields of all memory region descriptors @@ -328,6 +308,8 @@ void efi_get_virtmap(efi_memory_desc_t *memory_map, unsigned long map_size, efi_memory_desc_t *in, *out = runtime_map; int l; + *count = 0; + for (l = 0; l < map_size; l += desc_size) { u64 paddr, size; @@ -338,7 +320,7 @@ void efi_get_virtmap(efi_memory_desc_t *memory_map, unsigned long map_size, paddr = in->phys_addr; size = in->num_pages * EFI_PAGE_SIZE; - in->virt_addr = in->phys_addr; + in->virt_addr = in->phys_addr + EFI_RT_VIRTUAL_OFFSET; if (efi_novamap) { continue; } diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h index b0ae0a454404..a30fb5d8ef05 100644 --- a/drivers/firmware/efi/libstub/efistub.h +++ b/drivers/firmware/efi/libstub/efistub.h @@ -160,16 +160,24 @@ void efi_set_u64_split(u64 data, u32 *lo, u32 *hi) */ #define EFI_MMAP_NR_SLACK_SLOTS 8 -struct efi_boot_memmap { - efi_memory_desc_t **map; - unsigned long *map_size; - unsigned long *desc_size; - u32 *desc_ver; - unsigned long *key_ptr; - unsigned long *buff_size; +typedef struct efi_generic_dev_path efi_device_path_protocol_t; + +union efi_device_path_to_text_protocol { + struct { + efi_char16_t *(__efiapi *convert_device_node_to_text)( + const efi_device_path_protocol_t *, + bool, bool); + efi_char16_t *(__efiapi *convert_device_path_to_text)( + const efi_device_path_protocol_t *, + bool, bool); + }; + struct { + u32 convert_device_node_to_text; + u32 convert_device_path_to_text; + } mixed_mode; }; -typedef struct efi_generic_dev_path efi_device_path_protocol_t; +typedef union efi_device_path_to_text_protocol efi_device_path_to_text_protocol_t; typedef void *efi_event_t; /* Note that notifications won't work in mixed mode */ @@ -254,13 +262,17 @@ union efi_boot_services { efi_handle_t *); efi_status_t (__efiapi *install_configuration_table)(efi_guid_t *, void *); - void *load_image; - void *start_image; + efi_status_t (__efiapi *load_image)(bool, efi_handle_t, + efi_device_path_protocol_t *, + void *, unsigned long, + efi_handle_t *); + efi_status_t (__efiapi *start_image)(efi_handle_t, unsigned long *, + efi_char16_t **); efi_status_t __noreturn (__efiapi *exit)(efi_handle_t, efi_status_t, unsigned long, efi_char16_t *); - void *unload_image; + efi_status_t (__efiapi *unload_image)(efi_handle_t); efi_status_t (__efiapi *exit_boot_services)(efi_handle_t, unsigned long); void *get_next_monotonic_count; @@ -277,11 +289,11 @@ union efi_boot_services { void *locate_handle_buffer; efi_status_t (__efiapi *locate_protocol)(efi_guid_t *, void *, void **); - void *install_multiple_protocol_interfaces; - void *uninstall_multiple_protocol_interfaces; + efi_status_t (__efiapi *install_multiple_protocol_interfaces)(efi_handle_t *, ...); + efi_status_t (__efiapi *uninstall_multiple_protocol_interfaces)(efi_handle_t, ...); void *calculate_crc32; - void *copy_mem; - void *set_mem; + void (__efiapi *copy_mem)(void *, const void *, unsigned long); + void (__efiapi *set_mem)(void *, unsigned long, unsigned char); void *create_event_ex; }; struct { @@ -741,6 +753,7 @@ union apple_properties_protocol { typedef u32 efi_tcg2_event_log_format; #define INITRD_EVENT_TAG_ID 0x8F3B22ECU +#define LOAD_OPTIONS_EVENT_TAG_ID 0x8F3B22EDU #define EV_EVENT_TAG 0x00000006U #define EFI_TCG2_EVENT_HEADER_VERSION 0x1 @@ -840,7 +853,7 @@ typedef struct { u16 file_path_list_length; const efi_char16_t *description; const efi_device_path_protocol_t *file_path_list; - size_t optional_data_size; + u32 optional_data_size; const void *optional_data; } efi_load_option_unpacked_t; @@ -850,20 +863,16 @@ typedef efi_status_t (*efi_exit_boot_map_processing)( struct efi_boot_memmap *map, void *priv); -efi_status_t efi_exit_boot_services(void *handle, - struct efi_boot_memmap *map, - void *priv, +efi_status_t efi_exit_boot_services(void *handle, void *priv, efi_exit_boot_map_processing priv_func); -efi_status_t allocate_new_fdt_and_exit_boot(void *handle, - unsigned long *new_fdt_addr, - u64 initrd_addr, u64 initrd_size, - char *cmdline_ptr, - unsigned long fdt_addr, - unsigned long fdt_size); +efi_status_t efi_boot_kernel(void *handle, efi_loaded_image_t *image, + unsigned long kernel_addr, char *cmdline_ptr); void *get_fdt(unsigned long *fdt_size); +efi_status_t efi_alloc_virtmap(efi_memory_desc_t **virtmap, + unsigned long *desc_size, u32 *desc_ver); void efi_get_virtmap(efi_memory_desc_t *memory_map, unsigned long map_size, unsigned long desc_size, efi_memory_desc_t *runtime_map, int *count); @@ -885,11 +894,12 @@ __printf(1, 2) int efi_printk(char const *fmt, ...); void efi_free(unsigned long size, unsigned long addr); -void efi_apply_loadoptions_quirk(const void **load_options, int *load_options_size); +void efi_apply_loadoptions_quirk(const void **load_options, u32 *load_options_size); char *efi_convert_cmdline(efi_loaded_image_t *image, int *cmd_line_len); -efi_status_t efi_get_memory_map(struct efi_boot_memmap *map); +efi_status_t efi_get_memory_map(struct efi_boot_memmap **map, + bool install_cfg_tbl); efi_status_t efi_allocate_pages(unsigned long size, unsigned long *addr, unsigned long max); @@ -932,10 +942,9 @@ static inline efi_status_t efi_load_dtb(efi_loaded_image_t *image, } efi_status_t efi_load_initrd(efi_loaded_image_t *image, - unsigned long *load_addr, - unsigned long *load_size, unsigned long soft_limit, - unsigned long hard_limit); + unsigned long hard_limit, + const struct linux_efi_initrd **out); /* * This function handles the architcture specific differences between arm and * arm64 regarding where the kernel image must be loaded and any memory that diff --git a/drivers/firmware/efi/libstub/fdt.c b/drivers/firmware/efi/libstub/fdt.c index fe567be0f118..4f4d98e51fbf 100644 --- a/drivers/firmware/efi/libstub/fdt.c +++ b/drivers/firmware/efi/libstub/fdt.c @@ -28,8 +28,7 @@ static void fdt_update_cell_size(void *fdt) } static efi_status_t update_fdt(void *orig_fdt, unsigned long orig_fdt_size, - void *fdt, int new_fdt_size, char *cmdline_ptr, - u64 initrd_addr, u64 initrd_size) + void *fdt, int new_fdt_size, char *cmdline_ptr) { int node, num_rsv; int status; @@ -93,21 +92,6 @@ static efi_status_t update_fdt(void *orig_fdt, unsigned long orig_fdt_size, goto fdt_set_fail; } - /* Set initrd address/end in device tree, if present */ - if (initrd_size != 0) { - u64 initrd_image_end; - u64 initrd_image_start = cpu_to_fdt64(initrd_addr); - - status = fdt_setprop_var(fdt, node, "linux,initrd-start", initrd_image_start); - if (status) - goto fdt_set_fail; - - initrd_image_end = cpu_to_fdt64(initrd_addr + initrd_size); - status = fdt_setprop_var(fdt, node, "linux,initrd-end", initrd_image_end); - if (status) - goto fdt_set_fail; - } - /* Add FDT entries for EFI runtime services in chosen node. */ node = fdt_subnode_offset(fdt, 0, "chosen"); fdt_val64 = cpu_to_fdt64((u64)(unsigned long)efi_system_table); @@ -170,25 +154,25 @@ static efi_status_t update_fdt_memmap(void *fdt, struct efi_boot_memmap *map) if (node < 0) return EFI_LOAD_ERROR; - fdt_val64 = cpu_to_fdt64((unsigned long)*map->map); + fdt_val64 = cpu_to_fdt64((unsigned long)map->map); err = fdt_setprop_inplace_var(fdt, node, "linux,uefi-mmap-start", fdt_val64); if (err) return EFI_LOAD_ERROR; - fdt_val32 = cpu_to_fdt32(*map->map_size); + fdt_val32 = cpu_to_fdt32(map->map_size); err = fdt_setprop_inplace_var(fdt, node, "linux,uefi-mmap-size", fdt_val32); if (err) return EFI_LOAD_ERROR; - fdt_val32 = cpu_to_fdt32(*map->desc_size); + fdt_val32 = cpu_to_fdt32(map->desc_size); err = fdt_setprop_inplace_var(fdt, node, "linux,uefi-mmap-desc-size", fdt_val32); if (err) return EFI_LOAD_ERROR; - fdt_val32 = cpu_to_fdt32(*map->desc_ver); + fdt_val32 = cpu_to_fdt32(map->desc_ver); err = fdt_setprop_inplace_var(fdt, node, "linux,uefi-mmap-desc-ver", fdt_val32); if (err) @@ -198,22 +182,25 @@ static efi_status_t update_fdt_memmap(void *fdt, struct efi_boot_memmap *map) } struct exit_boot_struct { + struct efi_boot_memmap *boot_memmap; efi_memory_desc_t *runtime_map; - int *runtime_entry_count; + int runtime_entry_count; void *new_fdt_addr; }; -static efi_status_t exit_boot_func(struct efi_boot_memmap *map, - void *priv) +static efi_status_t exit_boot_func(struct efi_boot_memmap *map, void *priv) { struct exit_boot_struct *p = priv; + + p->boot_memmap = map; + /* * Update the memory map with virtual addresses. The function will also * populate @runtime_map with copies of just the EFI_MEMORY_RUNTIME * entries so that we can pass it straight to SetVirtualAddressMap() */ - efi_get_virtmap(*map->map, *map->map_size, *map->desc_size, - p->runtime_map, p->runtime_entry_count); + efi_get_virtmap(map->map, map->map_size, map->desc_size, + p->runtime_map, &p->runtime_entry_count); return update_fdt_memmap(p->new_fdt_addr, map); } @@ -223,86 +210,86 @@ static efi_status_t exit_boot_func(struct efi_boot_memmap *map, #endif /* - * Allocate memory for a new FDT, then add EFI, commandline, and - * initrd related fields to the FDT. This routine increases the - * FDT allocation size until the allocated memory is large - * enough. EFI allocations are in EFI_PAGE_SIZE granules, - * which are fixed at 4K bytes, so in most cases the first - * allocation should succeed. - * EFI boot services are exited at the end of this function. - * There must be no allocations between the get_memory_map() - * call and the exit_boot_services() call, so the exiting of - * boot services is very tightly tied to the creation of the FDT - * with the final memory map in it. + * Allocate memory for a new FDT, then add EFI and commandline related fields + * to the FDT. This routine increases the FDT allocation size until the + * allocated memory is large enough. EFI allocations are in EFI_PAGE_SIZE + * granules, which are fixed at 4K bytes, so in most cases the first allocation + * should succeed. EFI boot services are exited at the end of this function. + * There must be no allocations between the get_memory_map() call and the + * exit_boot_services() call, so the exiting of boot services is very tightly + * tied to the creation of the FDT with the final memory map in it. */ - +static efi_status_t allocate_new_fdt_and_exit_boot(void *handle, + efi_loaded_image_t *image, unsigned long *new_fdt_addr, - u64 initrd_addr, u64 initrd_size, - char *cmdline_ptr, - unsigned long fdt_addr, - unsigned long fdt_size) + char *cmdline_ptr) { - unsigned long map_size, desc_size, buff_size; + unsigned long desc_size; u32 desc_ver; - unsigned long mmap_key; - efi_memory_desc_t *memory_map, *runtime_map; efi_status_t status; - int runtime_entry_count; - struct efi_boot_memmap map; struct exit_boot_struct priv; + unsigned long fdt_addr = 0; + unsigned long fdt_size = 0; - map.map = &runtime_map; - map.map_size = &map_size; - map.desc_size = &desc_size; - map.desc_ver = &desc_ver; - map.key_ptr = &mmap_key; - map.buff_size = &buff_size; + if (!efi_novamap) { + status = efi_alloc_virtmap(&priv.runtime_map, &desc_size, + &desc_ver); + if (status != EFI_SUCCESS) { + efi_err("Unable to retrieve UEFI memory map.\n"); + return status; + } + } /* - * Get a copy of the current memory map that we will use to prepare - * the input for SetVirtualAddressMap(). We don't have to worry about - * subsequent allocations adding entries, since they could not affect - * the number of EFI_MEMORY_RUNTIME regions. + * Unauthenticated device tree data is a security hazard, so ignore + * 'dtb=' unless UEFI Secure Boot is disabled. We assume that secure + * boot is enabled if we can't determine its state. */ - status = efi_get_memory_map(&map); - if (status != EFI_SUCCESS) { - efi_err("Unable to retrieve UEFI memory map.\n"); - return status; + if (!IS_ENABLED(CONFIG_EFI_ARMSTUB_DTB_LOADER) || + efi_get_secureboot() != efi_secureboot_mode_disabled) { + if (strstr(cmdline_ptr, "dtb=")) + efi_err("Ignoring DTB from command line.\n"); + } else { + status = efi_load_dtb(image, &fdt_addr, &fdt_size); + + if (status != EFI_SUCCESS && status != EFI_NOT_READY) { + efi_err("Failed to load device tree!\n"); + goto fail; + } } + if (fdt_addr) { + efi_info("Using DTB from command line\n"); + } else { + /* Look for a device tree configuration table entry. */ + fdt_addr = (uintptr_t)get_fdt(&fdt_size); + if (fdt_addr) + efi_info("Using DTB from configuration table\n"); + } + + if (!fdt_addr) + efi_info("Generating empty DTB\n"); + efi_info("Exiting boot services...\n"); - map.map = &memory_map; status = efi_allocate_pages(MAX_FDT_SIZE, new_fdt_addr, ULONG_MAX); if (status != EFI_SUCCESS) { efi_err("Unable to allocate memory for new device tree.\n"); goto fail; } - /* - * Now that we have done our final memory allocation (and free) - * we can get the memory map key needed for exit_boot_services(). - */ - status = efi_get_memory_map(&map); - if (status != EFI_SUCCESS) - goto fail_free_new_fdt; - status = update_fdt((void *)fdt_addr, fdt_size, - (void *)*new_fdt_addr, MAX_FDT_SIZE, cmdline_ptr, - initrd_addr, initrd_size); + (void *)*new_fdt_addr, MAX_FDT_SIZE, cmdline_ptr); if (status != EFI_SUCCESS) { efi_err("Unable to construct new device tree.\n"); goto fail_free_new_fdt; } - runtime_entry_count = 0; - priv.runtime_map = runtime_map; - priv.runtime_entry_count = &runtime_entry_count; - priv.new_fdt_addr = (void *)*new_fdt_addr; + priv.new_fdt_addr = (void *)*new_fdt_addr; - status = efi_exit_boot_services(handle, &map, &priv, exit_boot_func); + status = efi_exit_boot_services(handle, &priv, exit_boot_func); if (status == EFI_SUCCESS) { efi_set_virtual_address_map_t *svam; @@ -312,8 +299,8 @@ efi_status_t allocate_new_fdt_and_exit_boot(void *handle, /* Install the new virtual address map */ svam = efi_system_table->runtime->set_virtual_address_map; - status = svam(runtime_entry_count * desc_size, desc_size, - desc_ver, runtime_map); + status = svam(priv.runtime_entry_count * desc_size, desc_size, + desc_ver, priv.runtime_map); /* * We are beyond the point of no return here, so if the call to @@ -321,6 +308,7 @@ efi_status_t allocate_new_fdt_and_exit_boot(void *handle, * incoming kernel but proceed normally otherwise. */ if (status != EFI_SUCCESS) { + efi_memory_desc_t *p; int l; /* @@ -329,8 +317,9 @@ efi_status_t allocate_new_fdt_and_exit_boot(void *handle, * the incoming kernel that no virtual translation has * been installed. */ - for (l = 0; l < map_size; l += desc_size) { - efi_memory_desc_t *p = (void *)memory_map + l; + for (l = 0; l < priv.boot_memmap->map_size; + l += priv.boot_memmap->desc_size) { + p = (void *)priv.boot_memmap->map + l; if (p->attribute & EFI_MEMORY_RUNTIME) p->virt_addr = 0; @@ -345,11 +334,33 @@ fail_free_new_fdt: efi_free(MAX_FDT_SIZE, *new_fdt_addr); fail: - efi_system_table->boottime->free_pool(runtime_map); + efi_free(fdt_size, fdt_addr); + + efi_bs_call(free_pool, priv.runtime_map); return EFI_LOAD_ERROR; } +efi_status_t efi_boot_kernel(void *handle, efi_loaded_image_t *image, + unsigned long kernel_addr, char *cmdline_ptr) +{ + unsigned long fdt_addr; + efi_status_t status; + + status = allocate_new_fdt_and_exit_boot(handle, image, &fdt_addr, + cmdline_ptr); + if (status != EFI_SUCCESS) { + efi_err("Failed to update FDT and exit boot services\n"); + return status; + } + + if (IS_ENABLED(CONFIG_ARM)) + efi_handle_post_ebs_state(); + + efi_enter_kernel(kernel_addr, fdt_addr, fdt_totalsize((void *)fdt_addr)); + /* not reached */ +} + void *get_fdt(unsigned long *fdt_size) { void *fdt; diff --git a/drivers/firmware/efi/libstub/file.c b/drivers/firmware/efi/libstub/file.c index dd95f330fe6e..f756c61396e9 100644 --- a/drivers/firmware/efi/libstub/file.c +++ b/drivers/firmware/efi/libstub/file.c @@ -66,10 +66,28 @@ static efi_status_t efi_open_file(efi_file_protocol_t *volume, static efi_status_t efi_open_volume(efi_loaded_image_t *image, efi_file_protocol_t **fh) { + struct efi_vendor_dev_path *dp = image->file_path; + efi_guid_t li_proto = LOADED_IMAGE_PROTOCOL_GUID; efi_guid_t fs_proto = EFI_FILE_SYSTEM_GUID; efi_simple_file_system_protocol_t *io; efi_status_t status; + // If we are using EFI zboot, we should look for the file system + // protocol on the parent image's handle instead + if (IS_ENABLED(CONFIG_EFI_ZBOOT) && + image->parent_handle != NULL && + dp != NULL && + dp->header.type == EFI_DEV_MEDIA && + dp->header.sub_type == EFI_DEV_MEDIA_VENDOR && + !efi_guidcmp(dp->vendorguid, LINUX_EFI_ZBOOT_MEDIA_GUID)) { + status = efi_bs_call(handle_protocol, image->parent_handle, + &li_proto, (void *)&image); + if (status != EFI_SUCCESS) { + efi_err("Failed to locate parent image handle\n"); + return status; + } + } + status = efi_bs_call(handle_protocol, image->device_handle, &fs_proto, (void **)&io); if (status != EFI_SUCCESS) { @@ -136,7 +154,7 @@ efi_status_t handle_cmdline_files(efi_loaded_image_t *image, unsigned long *load_size) { const efi_char16_t *cmdline = image->load_options; - int cmdline_len = image->load_options_size; + u32 cmdline_len = image->load_options_size; unsigned long efi_chunk_size = ULONG_MAX; efi_file_protocol_t *volume = NULL; efi_file_protocol_t *file; @@ -238,6 +256,9 @@ efi_status_t handle_cmdline_files(efi_loaded_image_t *image, if (volume) volume->close(volume); + + if (*load_size == 0) + return EFI_NOT_READY; return EFI_SUCCESS; err_close_file: diff --git a/drivers/firmware/efi/libstub/intrinsics.c b/drivers/firmware/efi/libstub/intrinsics.c new file mode 100644 index 000000000000..a04ab39292b6 --- /dev/null +++ b/drivers/firmware/efi/libstub/intrinsics.c @@ -0,0 +1,30 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/efi.h> +#include <asm/efi.h> +#include <asm/string.h> + +#include "efistub.h" + +#ifdef CONFIG_KASAN +#undef memcpy +#undef memmove +#undef memset +void *__memcpy(void *__dest, const void *__src, size_t __n) __alias(memcpy); +void *__memmove(void *__dest, const void *__src, size_t count) __alias(memmove); +void *__memset(void *s, int c, size_t count) __alias(memset); +#endif + +void *memcpy(void *dst, const void *src, size_t len) +{ + efi_bs_call(copy_mem, dst, src, len); + return dst; +} + +extern void *memmove(void *dst, const void *src, size_t len) __alias(memcpy); + +void *memset(void *dst, int c, size_t len) +{ + efi_bs_call(set_mem, dst, len, c & U8_MAX); + return dst; +} diff --git a/drivers/firmware/efi/libstub/loongarch-stub.c b/drivers/firmware/efi/libstub/loongarch-stub.c new file mode 100644 index 000000000000..32329f2a92f9 --- /dev/null +++ b/drivers/firmware/efi/libstub/loongarch-stub.c @@ -0,0 +1,102 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Author: Yun Liu <liuyun@loongson.cn> + * Huacai Chen <chenhuacai@loongson.cn> + * Copyright (C) 2020-2022 Loongson Technology Corporation Limited + */ + +#include <asm/efi.h> +#include <asm/addrspace.h> +#include "efistub.h" + +typedef void __noreturn (*kernel_entry_t)(bool efi, unsigned long cmdline, + unsigned long systab); + +extern int kernel_asize; +extern int kernel_fsize; +extern int kernel_offset; +extern kernel_entry_t kernel_entry; + +efi_status_t check_platform_features(void) +{ + return EFI_SUCCESS; +} + +efi_status_t handle_kernel_image(unsigned long *image_addr, + unsigned long *image_size, + unsigned long *reserve_addr, + unsigned long *reserve_size, + efi_loaded_image_t *image, + efi_handle_t image_handle) +{ + efi_status_t status; + unsigned long kernel_addr = 0; + + kernel_addr = (unsigned long)&kernel_offset - kernel_offset; + + status = efi_relocate_kernel(&kernel_addr, kernel_fsize, kernel_asize, + PHYSADDR(VMLINUX_LOAD_ADDRESS), SZ_2M, 0x0); + + *image_addr = kernel_addr; + *image_size = kernel_asize; + + return status; +} + +struct exit_boot_struct { + efi_memory_desc_t *runtime_map; + int runtime_entry_count; +}; + +static efi_status_t exit_boot_func(struct efi_boot_memmap *map, void *priv) +{ + struct exit_boot_struct *p = priv; + + /* + * Update the memory map with virtual addresses. The function will also + * populate @runtime_map with copies of just the EFI_MEMORY_RUNTIME + * entries so that we can pass it straight to SetVirtualAddressMap() + */ + efi_get_virtmap(map->map, map->map_size, map->desc_size, + p->runtime_map, &p->runtime_entry_count); + + return EFI_SUCCESS; +} + +efi_status_t efi_boot_kernel(void *handle, efi_loaded_image_t *image, + unsigned long kernel_addr, char *cmdline_ptr) +{ + kernel_entry_t real_kernel_entry; + struct exit_boot_struct priv; + unsigned long desc_size; + efi_status_t status; + u32 desc_ver; + + status = efi_alloc_virtmap(&priv.runtime_map, &desc_size, &desc_ver); + if (status != EFI_SUCCESS) { + efi_err("Unable to retrieve UEFI memory map.\n"); + return status; + } + + efi_info("Exiting boot services\n"); + + efi_novamap = false; + status = efi_exit_boot_services(handle, &priv, exit_boot_func); + if (status != EFI_SUCCESS) + return status; + + /* Install the new virtual address map */ + efi_rt_call(set_virtual_address_map, + priv.runtime_entry_count * desc_size, desc_size, + desc_ver, priv.runtime_map); + + /* Config Direct Mapping */ + csr_write64(CSR_DMW0_INIT, LOONGARCH_CSR_DMWIN0); + csr_write64(CSR_DMW1_INIT, LOONGARCH_CSR_DMWIN1); + + real_kernel_entry = (kernel_entry_t) + ((unsigned long)&kernel_entry - kernel_addr + VMLINUX_LOAD_ADDRESS); + + real_kernel_entry(true, (unsigned long)cmdline_ptr, + (unsigned long)efi_system_table); +} diff --git a/drivers/firmware/efi/libstub/mem.c b/drivers/firmware/efi/libstub/mem.c index feef8d4be113..45841ef55a9f 100644 --- a/drivers/firmware/efi/libstub/mem.c +++ b/drivers/firmware/efi/libstub/mem.c @@ -5,71 +5,66 @@ #include "efistub.h" -static inline bool mmap_has_headroom(unsigned long buff_size, - unsigned long map_size, - unsigned long desc_size) -{ - unsigned long slack = buff_size - map_size; - - return slack / desc_size >= EFI_MMAP_NR_SLACK_SLOTS; -} - /** * efi_get_memory_map() - get memory map - * @map: on return pointer to memory map + * @map: pointer to memory map pointer to which to assign the + * newly allocated memory map + * @install_cfg_tbl: whether or not to install the boot memory map as a + * configuration table * * Retrieve the UEFI memory map. The allocated memory leaves room for * up to EFI_MMAP_NR_SLACK_SLOTS additional memory map entries. * * Return: status code */ -efi_status_t efi_get_memory_map(struct efi_boot_memmap *map) +efi_status_t efi_get_memory_map(struct efi_boot_memmap **map, + bool install_cfg_tbl) { - efi_memory_desc_t *m = NULL; + int memtype = install_cfg_tbl ? EFI_ACPI_RECLAIM_MEMORY + : EFI_LOADER_DATA; + efi_guid_t tbl_guid = LINUX_EFI_BOOT_MEMMAP_GUID; + struct efi_boot_memmap *m, tmp; efi_status_t status; - unsigned long key; - u32 desc_version; - - *map->desc_size = sizeof(*m); - *map->map_size = *map->desc_size * 32; - *map->buff_size = *map->map_size; -again: - status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, - *map->map_size, (void **)&m); + unsigned long size; + + tmp.map_size = 0; + status = efi_bs_call(get_memory_map, &tmp.map_size, NULL, &tmp.map_key, + &tmp.desc_size, &tmp.desc_ver); + if (status != EFI_BUFFER_TOO_SMALL) + return EFI_LOAD_ERROR; + + size = tmp.map_size + tmp.desc_size * EFI_MMAP_NR_SLACK_SLOTS; + status = efi_bs_call(allocate_pool, memtype, sizeof(*m) + size, + (void **)&m); if (status != EFI_SUCCESS) - goto fail; - - *map->desc_size = 0; - key = 0; - status = efi_bs_call(get_memory_map, map->map_size, m, - &key, map->desc_size, &desc_version); - if (status == EFI_BUFFER_TOO_SMALL || - !mmap_has_headroom(*map->buff_size, *map->map_size, - *map->desc_size)) { - efi_bs_call(free_pool, m); + return status; + + if (install_cfg_tbl) { /* - * Make sure there is some entries of headroom so that the - * buffer can be reused for a new map after allocations are - * no longer permitted. Its unlikely that the map will grow to - * exceed this headroom once we are ready to trigger - * ExitBootServices() + * Installing a configuration table might allocate memory, and + * this may modify the memory map. This means we should install + * the configuration table first, and re-install or delete it + * as needed. */ - *map->map_size += *map->desc_size * EFI_MMAP_NR_SLACK_SLOTS; - *map->buff_size = *map->map_size; - goto again; + status = efi_bs_call(install_configuration_table, &tbl_guid, m); + if (status != EFI_SUCCESS) + goto free_map; } - if (status == EFI_SUCCESS) { - if (map->key_ptr) - *map->key_ptr = key; - if (map->desc_ver) - *map->desc_ver = desc_version; - } else { - efi_bs_call(free_pool, m); - } + m->buff_size = m->map_size = size; + status = efi_bs_call(get_memory_map, &m->map_size, m->map, &m->map_key, + &m->desc_size, &m->desc_ver); + if (status != EFI_SUCCESS) + goto uninstall_table; + + *map = m; + return EFI_SUCCESS; -fail: - *map->map = m; +uninstall_table: + if (install_cfg_tbl) + efi_bs_call(install_configuration_table, &tbl_guid, NULL); +free_map: + efi_bs_call(free_pool, m); return status; } diff --git a/drivers/firmware/efi/libstub/randomalloc.c b/drivers/firmware/efi/libstub/randomalloc.c index 715f37479154..9fb5869896be 100644 --- a/drivers/firmware/efi/libstub/randomalloc.c +++ b/drivers/firmware/efi/libstub/randomalloc.c @@ -55,22 +55,13 @@ efi_status_t efi_random_alloc(unsigned long size, unsigned long *addr, unsigned long random_seed) { - unsigned long map_size, desc_size, total_slots = 0, target_slot; + unsigned long total_slots = 0, target_slot; unsigned long total_mirrored_slots = 0; - unsigned long buff_size; + struct efi_boot_memmap *map; efi_status_t status; - efi_memory_desc_t *memory_map; int map_offset; - struct efi_boot_memmap map; - map.map = &memory_map; - map.map_size = &map_size; - map.desc_size = &desc_size; - map.desc_ver = NULL; - map.key_ptr = NULL; - map.buff_size = &buff_size; - - status = efi_get_memory_map(&map); + status = efi_get_memory_map(&map, false); if (status != EFI_SUCCESS) return status; @@ -80,8 +71,8 @@ efi_status_t efi_random_alloc(unsigned long size, size = round_up(size, EFI_ALLOC_ALIGN); /* count the suitable slots in each memory map entry */ - for (map_offset = 0; map_offset < map_size; map_offset += desc_size) { - efi_memory_desc_t *md = (void *)memory_map + map_offset; + for (map_offset = 0; map_offset < map->map_size; map_offset += map->desc_size) { + efi_memory_desc_t *md = (void *)map->map + map_offset; unsigned long slots; slots = get_entry_num_slots(md, size, ilog2(align)); @@ -109,8 +100,8 @@ efi_status_t efi_random_alloc(unsigned long size, * to calculate the randomly chosen address, and allocate it directly * using EFI_ALLOCATE_ADDRESS. */ - for (map_offset = 0; map_offset < map_size; map_offset += desc_size) { - efi_memory_desc_t *md = (void *)memory_map + map_offset; + for (map_offset = 0; map_offset < map->map_size; map_offset += map->desc_size) { + efi_memory_desc_t *md = (void *)map->map + map_offset; efi_physical_addr_t target; unsigned long pages; @@ -133,7 +124,7 @@ efi_status_t efi_random_alloc(unsigned long size, break; } - efi_bs_call(free_pool, memory_map); + efi_bs_call(free_pool, map); return status; } diff --git a/drivers/firmware/efi/libstub/relocate.c b/drivers/firmware/efi/libstub/relocate.c index 8ee9eb2b9039..bf6fbd5d22a1 100644 --- a/drivers/firmware/efi/libstub/relocate.c +++ b/drivers/firmware/efi/libstub/relocate.c @@ -23,21 +23,12 @@ efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, unsigned long *addr, unsigned long min) { - unsigned long map_size, desc_size, buff_size; - efi_memory_desc_t *map; + struct efi_boot_memmap *map; efi_status_t status; unsigned long nr_pages; int i; - struct efi_boot_memmap boot_map; - boot_map.map = ↦ - boot_map.map_size = &map_size; - boot_map.desc_size = &desc_size; - boot_map.desc_ver = NULL; - boot_map.key_ptr = NULL; - boot_map.buff_size = &buff_size; - - status = efi_get_memory_map(&boot_map); + status = efi_get_memory_map(&map, false); if (status != EFI_SUCCESS) goto fail; @@ -52,12 +43,12 @@ efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, size = round_up(size, EFI_ALLOC_ALIGN); nr_pages = size / EFI_PAGE_SIZE; - for (i = 0; i < map_size / desc_size; i++) { + for (i = 0; i < map->map_size / map->desc_size; i++) { efi_memory_desc_t *desc; - unsigned long m = (unsigned long)map; + unsigned long m = (unsigned long)map->map; u64 start, end; - desc = efi_early_memdesc_ptr(m, desc_size, i); + desc = efi_early_memdesc_ptr(m, map->desc_size, i); if (desc->type != EFI_CONVENTIONAL_MEMORY) continue; @@ -87,7 +78,7 @@ efi_status_t efi_low_alloc_above(unsigned long size, unsigned long align, } } - if (i == map_size / desc_size) + if (i == map->map_size / map->desc_size) status = EFI_NOT_FOUND; efi_bs_call(free_pool, map); diff --git a/drivers/firmware/efi/libstub/systable.c b/drivers/firmware/efi/libstub/systable.c new file mode 100644 index 000000000000..91d016b02f8c --- /dev/null +++ b/drivers/firmware/efi/libstub/systable.c @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/efi.h> +#include <asm/efi.h> + +#include "efistub.h" + +const efi_system_table_t *efi_system_table; diff --git a/drivers/firmware/efi/libstub/x86-stub.c b/drivers/firmware/efi/libstub/x86-stub.c index 7a7abc8959d2..b9ce6393e353 100644 --- a/drivers/firmware/efi/libstub/x86-stub.c +++ b/drivers/firmware/efi/libstub/x86-stub.c @@ -722,32 +722,22 @@ static efi_status_t exit_boot_func(struct efi_boot_memmap *map, efi_set_u64_split((unsigned long)efi_system_table, &p->efi->efi_systab, &p->efi->efi_systab_hi); - p->efi->efi_memdesc_size = *map->desc_size; - p->efi->efi_memdesc_version = *map->desc_ver; - efi_set_u64_split((unsigned long)*map->map, + p->efi->efi_memdesc_size = map->desc_size; + p->efi->efi_memdesc_version = map->desc_ver; + efi_set_u64_split((unsigned long)map->map, &p->efi->efi_memmap, &p->efi->efi_memmap_hi); - p->efi->efi_memmap_size = *map->map_size; + p->efi->efi_memmap_size = map->map_size; return EFI_SUCCESS; } static efi_status_t exit_boot(struct boot_params *boot_params, void *handle) { - unsigned long map_sz, key, desc_size, buff_size; - efi_memory_desc_t *mem_map; struct setup_data *e820ext = NULL; __u32 e820ext_size = 0; efi_status_t status; - __u32 desc_version; - struct efi_boot_memmap map; struct exit_boot_struct priv; - map.map = &mem_map; - map.map_size = &map_sz; - map.desc_size = &desc_size; - map.desc_ver = &desc_version; - map.key_ptr = &key; - map.buff_size = &buff_size; priv.boot_params = boot_params; priv.efi = &boot_params->efi_info; @@ -756,7 +746,7 @@ static efi_status_t exit_boot(struct boot_params *boot_params, void *handle) return status; /* Might as well exit boot services now */ - status = efi_exit_boot_services(handle, &map, &priv, exit_boot_func); + status = efi_exit_boot_services(handle, &priv, exit_boot_func); if (status != EFI_SUCCESS) return status; @@ -782,7 +772,7 @@ unsigned long efi_main(efi_handle_t handle, unsigned long bzimage_addr = (unsigned long)startup_32; unsigned long buffer_start, buffer_end; struct setup_header *hdr = &boot_params->hdr; - unsigned long addr, size; + const struct linux_efi_initrd *initrd = NULL; efi_status_t status; efi_system_table = sys_table_arg; @@ -877,17 +867,18 @@ unsigned long efi_main(efi_handle_t handle, * arguments will be processed only if image is not NULL, which will be * the case only if we were loaded via the PE entry point. */ - status = efi_load_initrd(image, &addr, &size, hdr->initrd_addr_max, - ULONG_MAX); + status = efi_load_initrd(image, hdr->initrd_addr_max, ULONG_MAX, + &initrd); if (status != EFI_SUCCESS) goto fail; - if (size > 0) { - efi_set_u64_split(addr, &hdr->ramdisk_image, + if (initrd && initrd->size > 0) { + efi_set_u64_split(initrd->base, &hdr->ramdisk_image, &boot_params->ext_ramdisk_image); - efi_set_u64_split(size, &hdr->ramdisk_size, + efi_set_u64_split(initrd->size, &hdr->ramdisk_size, &boot_params->ext_ramdisk_size); } + /* * If the boot loader gave us a value for secure_boot then we use that, * otherwise we ask the BIOS. diff --git a/drivers/firmware/efi/libstub/zboot-header.S b/drivers/firmware/efi/libstub/zboot-header.S new file mode 100644 index 000000000000..9e6fe061ab07 --- /dev/null +++ b/drivers/firmware/efi/libstub/zboot-header.S @@ -0,0 +1,143 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <linux/pe.h> + +#ifdef CONFIG_64BIT + .set .Lextra_characteristics, 0x0 + .set .Lpe_opt_magic, PE_OPT_MAGIC_PE32PLUS +#else + .set .Lextra_characteristics, IMAGE_FILE_32BIT_MACHINE + .set .Lpe_opt_magic, PE_OPT_MAGIC_PE32 +#endif + + .section ".head", "a" + .globl __efistub_efi_zboot_header +__efistub_efi_zboot_header: +.Ldoshdr: + .long MZ_MAGIC + .ascii "zimg" // image type + .long __efistub__gzdata_start - .Ldoshdr // payload offset + .long __efistub__gzdata_size - ZBOOT_SIZE_LEN // payload size + .long 0, 0 // reserved + .asciz COMP_TYPE // compression type + .org .Ldoshdr + 0x3c + .long .Lpehdr - .Ldoshdr // PE header offset + +.Lpehdr: + .long PE_MAGIC + .short MACHINE_TYPE + .short .Lsection_count + .long 0 + .long 0 + .long 0 + .short .Lsection_table - .Loptional_header + .short IMAGE_FILE_DEBUG_STRIPPED | \ + IMAGE_FILE_EXECUTABLE_IMAGE | \ + IMAGE_FILE_LINE_NUMS_STRIPPED |\ + .Lextra_characteristics + +.Loptional_header: + .short .Lpe_opt_magic + .byte 0, 0 + .long _etext - .Lefi_header_end + .long __data_size + .long 0 + .long __efistub_efi_zboot_entry - .Ldoshdr + .long .Lefi_header_end - .Ldoshdr + +#ifdef CONFIG_64BIT + .quad 0 +#else + .long _etext - .Ldoshdr, 0x0 +#endif + .long 4096 + .long 512 + .short 0, 0 + .short LINUX_EFISTUB_MAJOR_VERSION // MajorImageVersion + .short LINUX_EFISTUB_MINOR_VERSION // MinorImageVersion + .short 0, 0 + .long 0 + .long _end - .Ldoshdr + + .long .Lefi_header_end - .Ldoshdr + .long 0 + .short IMAGE_SUBSYSTEM_EFI_APPLICATION + .short 0 +#ifdef CONFIG_64BIT + .quad 0, 0, 0, 0 +#else + .long 0, 0, 0, 0 +#endif + .long 0 + .long (.Lsection_table - .) / 8 + + .quad 0 // ExportTable + .quad 0 // ImportTable + .quad 0 // ResourceTable + .quad 0 // ExceptionTable + .quad 0 // CertificationTable + .quad 0 // BaseRelocationTable +#ifdef CONFIG_DEBUG_EFI + .long .Lefi_debug_table - .Ldoshdr // DebugTable + .long .Lefi_debug_table_size +#endif + +.Lsection_table: + .ascii ".text\0\0\0" + .long _etext - .Lefi_header_end + .long .Lefi_header_end - .Ldoshdr + .long _etext - .Lefi_header_end + .long .Lefi_header_end - .Ldoshdr + + .long 0, 0 + .short 0, 0 + .long IMAGE_SCN_CNT_CODE | \ + IMAGE_SCN_MEM_READ | \ + IMAGE_SCN_MEM_EXECUTE + + .ascii ".data\0\0\0" + .long __data_size + .long _etext - .Ldoshdr + .long __data_rawsize + .long _etext - .Ldoshdr + + .long 0, 0 + .short 0, 0 + .long IMAGE_SCN_CNT_INITIALIZED_DATA | \ + IMAGE_SCN_MEM_READ | \ + IMAGE_SCN_MEM_WRITE + + .set .Lsection_count, (. - .Lsection_table) / 40 + +#ifdef CONFIG_DEBUG_EFI + .section ".rodata", "a" + .align 2 +.Lefi_debug_table: + // EFI_IMAGE_DEBUG_DIRECTORY_ENTRY + .long 0 // Characteristics + .long 0 // TimeDateStamp + .short 0 // MajorVersion + .short 0 // MinorVersion + .long IMAGE_DEBUG_TYPE_CODEVIEW // Type + .long .Lefi_debug_entry_size // SizeOfData + .long 0 // RVA + .long .Lefi_debug_entry - .Ldoshdr // FileOffset + + .set .Lefi_debug_table_size, . - .Lefi_debug_table + .previous + +.Lefi_debug_entry: + // EFI_IMAGE_DEBUG_CODEVIEW_NB10_ENTRY + .ascii "NB10" // Signature + .long 0 // Unknown + .long 0 // Unknown2 + .long 0 // Unknown3 + + .asciz ZBOOT_EFI_PATH + + .set .Lefi_debug_entry_size, . - .Lefi_debug_entry +#endif + + .p2align 12 +.Lefi_header_end: + diff --git a/drivers/firmware/efi/libstub/zboot.c b/drivers/firmware/efi/libstub/zboot.c new file mode 100644 index 000000000000..ea72c8f27da6 --- /dev/null +++ b/drivers/firmware/efi/libstub/zboot.c @@ -0,0 +1,302 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/efi.h> +#include <linux/pe.h> +#include <asm/efi.h> +#include <asm/unaligned.h> + +#include "efistub.h" + +static unsigned char zboot_heap[SZ_256K] __aligned(64); +static unsigned long free_mem_ptr, free_mem_end_ptr; + +#define STATIC static +#if defined(CONFIG_KERNEL_GZIP) +#include "../../../../lib/decompress_inflate.c" +#elif defined(CONFIG_KERNEL_LZ4) +#include "../../../../lib/decompress_unlz4.c" +#elif defined(CONFIG_KERNEL_LZMA) +#include "../../../../lib/decompress_unlzma.c" +#elif defined(CONFIG_KERNEL_LZO) +#include "../../../../lib/decompress_unlzo.c" +#elif defined(CONFIG_KERNEL_XZ) +#undef memcpy +#define memcpy memcpy +#undef memmove +#define memmove memmove +#include "../../../../lib/decompress_unxz.c" +#elif defined(CONFIG_KERNEL_ZSTD) +#include "../../../../lib/decompress_unzstd.c" +#endif + +extern char efi_zboot_header[]; +extern char _gzdata_start[], _gzdata_end[]; + +static void log(efi_char16_t str[]) +{ + efi_call_proto(efi_table_attr(efi_system_table, con_out), + output_string, L"EFI decompressor: "); + efi_call_proto(efi_table_attr(efi_system_table, con_out), + output_string, str); + efi_call_proto(efi_table_attr(efi_system_table, con_out), + output_string, L"\n"); +} + +static void error(char *x) +{ + log(L"error() called from decompressor library\n"); +} + +// Local version to avoid pulling in memcmp() +static bool guids_eq(const efi_guid_t *a, const efi_guid_t *b) +{ + const u32 *l = (u32 *)a; + const u32 *r = (u32 *)b; + + return l[0] == r[0] && l[1] == r[1] && l[2] == r[2] && l[3] == r[3]; +} + +static efi_status_t __efiapi +load_file(efi_load_file_protocol_t *this, efi_device_path_protocol_t *rem, + bool boot_policy, unsigned long *bufsize, void *buffer) +{ + unsigned long compressed_size = _gzdata_end - _gzdata_start; + struct efi_vendor_dev_path *vendor_dp; + bool decompress = false; + unsigned long size; + int ret; + + if (rem == NULL || bufsize == NULL) + return EFI_INVALID_PARAMETER; + + if (boot_policy) + return EFI_UNSUPPORTED; + + // Look for our vendor media device node in the remaining file path + if (rem->type == EFI_DEV_MEDIA && + rem->sub_type == EFI_DEV_MEDIA_VENDOR) { + vendor_dp = container_of(rem, struct efi_vendor_dev_path, header); + if (!guids_eq(&vendor_dp->vendorguid, &LINUX_EFI_ZBOOT_MEDIA_GUID)) + return EFI_NOT_FOUND; + + decompress = true; + rem = (void *)(vendor_dp + 1); + } + + if (rem->type != EFI_DEV_END_PATH || + rem->sub_type != EFI_DEV_END_ENTIRE) + return EFI_NOT_FOUND; + + // The uncompressed size of the payload is appended to the raw bit + // stream, and may therefore appear misaligned in memory + size = decompress ? get_unaligned_le32(_gzdata_end - 4) + : compressed_size; + if (buffer == NULL || *bufsize < size) { + *bufsize = size; + return EFI_BUFFER_TOO_SMALL; + } + + if (decompress) { + ret = __decompress(_gzdata_start, compressed_size, NULL, NULL, + buffer, size, NULL, error); + if (ret < 0) { + log(L"Decompression failed"); + return EFI_DEVICE_ERROR; + } + } else { + memcpy(buffer, _gzdata_start, compressed_size); + } + + return EFI_SUCCESS; +} + +// Return the length in bytes of the device path up to the first end node. +static int device_path_length(const efi_device_path_protocol_t *dp) +{ + int len = 0; + + while (dp->type != EFI_DEV_END_PATH) { + len += dp->length; + dp = (void *)((u8 *)dp + dp->length); + } + return len; +} + +static void append_rel_offset_node(efi_device_path_protocol_t **dp, + unsigned long start, unsigned long end) +{ + struct efi_rel_offset_dev_path *rodp = (void *)*dp; + + rodp->header.type = EFI_DEV_MEDIA; + rodp->header.sub_type = EFI_DEV_MEDIA_REL_OFFSET; + rodp->header.length = sizeof(struct efi_rel_offset_dev_path); + rodp->reserved = 0; + rodp->starting_offset = start; + rodp->ending_offset = end; + + *dp = (void *)(rodp + 1); +} + +static void append_ven_media_node(efi_device_path_protocol_t **dp, + efi_guid_t *guid) +{ + struct efi_vendor_dev_path *vmdp = (void *)*dp; + + vmdp->header.type = EFI_DEV_MEDIA; + vmdp->header.sub_type = EFI_DEV_MEDIA_VENDOR; + vmdp->header.length = sizeof(struct efi_vendor_dev_path); + vmdp->vendorguid = *guid; + + *dp = (void *)(vmdp + 1); +} + +static void append_end_node(efi_device_path_protocol_t **dp) +{ + (*dp)->type = EFI_DEV_END_PATH; + (*dp)->sub_type = EFI_DEV_END_ENTIRE; + (*dp)->length = sizeof(struct efi_generic_dev_path); + + ++*dp; +} + +asmlinkage efi_status_t __efiapi +efi_zboot_entry(efi_handle_t handle, efi_system_table_t *systab) +{ + struct efi_mem_mapped_dev_path mmdp = { + .header.type = EFI_DEV_HW, + .header.sub_type = EFI_DEV_MEM_MAPPED, + .header.length = sizeof(struct efi_mem_mapped_dev_path) + }; + efi_device_path_protocol_t *parent_dp, *dpp, *lf2_dp, *li_dp; + efi_load_file2_protocol_t zboot_load_file2; + efi_loaded_image_t *parent, *child; + unsigned long exit_data_size; + efi_handle_t child_handle; + efi_handle_t zboot_handle; + efi_char16_t *exit_data; + efi_status_t status; + void *dp_alloc; + int dp_len; + + WRITE_ONCE(efi_system_table, systab); + + free_mem_ptr = (unsigned long)&zboot_heap; + free_mem_end_ptr = free_mem_ptr + sizeof(zboot_heap); + + exit_data = NULL; + exit_data_size = 0; + + status = efi_bs_call(handle_protocol, handle, + &LOADED_IMAGE_PROTOCOL_GUID, (void **)&parent); + if (status != EFI_SUCCESS) { + log(L"Failed to locate parent's loaded image protocol"); + return status; + } + + status = efi_bs_call(handle_protocol, handle, + &LOADED_IMAGE_DEVICE_PATH_PROTOCOL_GUID, + (void **)&parent_dp); + if (status != EFI_SUCCESS || parent_dp == NULL) { + // Create a MemoryMapped() device path node to describe + // the parent image if no device path was provided. + mmdp.memory_type = parent->image_code_type; + mmdp.starting_addr = (unsigned long)parent->image_base; + mmdp.ending_addr = (unsigned long)parent->image_base + + parent->image_size - 1; + parent_dp = &mmdp.header; + dp_len = sizeof(mmdp); + } else { + dp_len = device_path_length(parent_dp); + } + + // Allocate some pool memory for device path protocol data + status = efi_bs_call(allocate_pool, EFI_LOADER_DATA, + 2 * (dp_len + sizeof(struct efi_rel_offset_dev_path) + + sizeof(struct efi_generic_dev_path)) + + sizeof(struct efi_vendor_dev_path), + (void **)&dp_alloc); + if (status != EFI_SUCCESS) { + log(L"Failed to allocate device path pool memory"); + return status; + } + + // Create a device path describing the compressed payload in this image + // <...parent_dp...>/Offset(<start>, <end>) + lf2_dp = memcpy(dp_alloc, parent_dp, dp_len); + dpp = (void *)((u8 *)lf2_dp + dp_len); + append_rel_offset_node(&dpp, + (unsigned long)(_gzdata_start - efi_zboot_header), + (unsigned long)(_gzdata_end - efi_zboot_header - 1)); + append_end_node(&dpp); + + // Create a device path describing the decompressed payload in this image + // <...parent_dp...>/Offset(<start>, <end>)/VenMedia(ZBOOT_MEDIA_GUID) + dp_len += sizeof(struct efi_rel_offset_dev_path); + li_dp = memcpy(dpp, lf2_dp, dp_len); + dpp = (void *)((u8 *)li_dp + dp_len); + append_ven_media_node(&dpp, &LINUX_EFI_ZBOOT_MEDIA_GUID); + append_end_node(&dpp); + + zboot_handle = NULL; + zboot_load_file2.load_file = load_file; + status = efi_bs_call(install_multiple_protocol_interfaces, + &zboot_handle, + &EFI_DEVICE_PATH_PROTOCOL_GUID, lf2_dp, + &EFI_LOAD_FILE2_PROTOCOL_GUID, &zboot_load_file2, + NULL); + if (status != EFI_SUCCESS) { + log(L"Failed to install LoadFile2 protocol and device path"); + goto free_dpalloc; + } + + status = efi_bs_call(load_image, false, handle, li_dp, NULL, 0, + &child_handle); + if (status != EFI_SUCCESS) { + log(L"Failed to load image"); + goto uninstall_lf2; + } + + status = efi_bs_call(handle_protocol, child_handle, + &LOADED_IMAGE_PROTOCOL_GUID, (void **)&child); + if (status != EFI_SUCCESS) { + log(L"Failed to locate child's loaded image protocol"); + goto unload_image; + } + + // Copy the kernel command line + child->load_options = parent->load_options; + child->load_options_size = parent->load_options_size; + + status = efi_bs_call(start_image, child_handle, &exit_data_size, + &exit_data); + if (status != EFI_SUCCESS) { + log(L"StartImage() returned with error"); + if (exit_data_size > 0) + log(exit_data); + + // If StartImage() returns EFI_SECURITY_VIOLATION, the image is + // not unloaded so we need to do it by hand. + if (status == EFI_SECURITY_VIOLATION) +unload_image: + efi_bs_call(unload_image, child_handle); + } + +uninstall_lf2: + efi_bs_call(uninstall_multiple_protocol_interfaces, + zboot_handle, + &EFI_DEVICE_PATH_PROTOCOL_GUID, lf2_dp, + &EFI_LOAD_FILE2_PROTOCOL_GUID, &zboot_load_file2, + NULL); + +free_dpalloc: + efi_bs_call(free_pool, dp_alloc); + + efi_bs_call(exit, handle, status, exit_data_size, exit_data); + + // Free ExitData in case Exit() returned with a failure code, + // but return the original status code. + log(L"Exit() returned with failure code"); + if (exit_data != NULL) + efi_bs_call(free_pool, exit_data); + return status; +} diff --git a/drivers/firmware/efi/libstub/zboot.lds b/drivers/firmware/efi/libstub/zboot.lds new file mode 100644 index 000000000000..87a62765bafd --- /dev/null +++ b/drivers/firmware/efi/libstub/zboot.lds @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +ENTRY(__efistub_efi_zboot_header); + +SECTIONS +{ + .head : ALIGN(4096) { + *(.head) + } + + .text : { + *(.text* .init.text*) + } + + .rodata : ALIGN(8) { + __efistub__gzdata_start = .; + *(.gzdata) + __efistub__gzdata_end = .; + *(.rodata* .init.rodata* .srodata*) + _etext = ALIGN(4096); + . = _etext; + } + + .data : ALIGN(4096) { + *(.data* .init.data*) + _edata = ALIGN(512); + . = _edata; + } + + .bss : { + *(.bss* .init.bss*) + _end = ALIGN(512); + . = _end; + } + + /DISCARD/ : { + *(.modinfo .init.modinfo) + } +} + +PROVIDE(__efistub__gzdata_size = ABSOLUTE(. - __efistub__gzdata_start)); + +PROVIDE(__data_rawsize = ABSOLUTE(_edata - _etext)); +PROVIDE(__data_size = ABSOLUTE(_end - _etext)); diff --git a/include/linux/efi.h b/include/linux/efi.h index d2b84c2fec39..da3974bf05d3 100644 --- a/include/linux/efi.h +++ b/include/linux/efi.h @@ -368,6 +368,9 @@ void efi_native_runtime_setup(void); #define UV_SYSTEM_TABLE_GUID EFI_GUID(0x3b13a7d4, 0x633e, 0x11dd, 0x93, 0xec, 0xda, 0x25, 0x56, 0xd8, 0x95, 0x93) #define LINUX_EFI_CRASH_GUID EFI_GUID(0xcfc8fc79, 0xbe2e, 0x4ddc, 0x97, 0xf0, 0x9f, 0x98, 0xbf, 0xe2, 0x98, 0xa0) #define LOADED_IMAGE_PROTOCOL_GUID EFI_GUID(0x5b1b31a1, 0x9562, 0x11d2, 0x8e, 0x3f, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b) +#define LOADED_IMAGE_DEVICE_PATH_PROTOCOL_GUID EFI_GUID(0xbc62157e, 0x3e33, 0x4fec, 0x99, 0x20, 0x2d, 0x3b, 0x36, 0xd7, 0x50, 0xdf) +#define EFI_DEVICE_PATH_PROTOCOL_GUID EFI_GUID(0x09576e91, 0x6d3f, 0x11d2, 0x8e, 0x39, 0x00, 0xa0, 0xc9, 0x69, 0x72, 0x3b) +#define EFI_DEVICE_PATH_TO_TEXT_PROTOCOL_GUID EFI_GUID(0x8b843e20, 0x8132, 0x4852, 0x90, 0xcc, 0x55, 0x1a, 0x4e, 0x4a, 0x7f, 0x1c) #define EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID EFI_GUID(0x9042a9de, 0x23dc, 0x4a38, 0x96, 0xfb, 0x7a, 0xde, 0xd0, 0x80, 0x51, 0x6a) #define EFI_UGA_PROTOCOL_GUID EFI_GUID(0x982c298b, 0xf4fa, 0x41cb, 0xb8, 0x38, 0x77, 0xaa, 0x68, 0x8f, 0xb8, 0x39) #define EFI_PCI_IO_PROTOCOL_GUID EFI_GUID(0x4cf5b200, 0x68b8, 0x4ca5, 0x9e, 0xec, 0xb2, 0x3e, 0x3f, 0x50, 0x02, 0x9a) @@ -408,8 +411,10 @@ void efi_native_runtime_setup(void); #define LINUX_EFI_TPM_FINAL_LOG_GUID EFI_GUID(0x1e2ed096, 0x30e2, 0x4254, 0xbd, 0x89, 0x86, 0x3b, 0xbe, 0xf8, 0x23, 0x25) #define LINUX_EFI_MEMRESERVE_TABLE_GUID EFI_GUID(0x888eb0c6, 0x8ede, 0x4ff5, 0xa8, 0xf0, 0x9a, 0xee, 0x5c, 0xb9, 0x77, 0xc2) #define LINUX_EFI_INITRD_MEDIA_GUID EFI_GUID(0x5568e427, 0x68fc, 0x4f3d, 0xac, 0x74, 0xca, 0x55, 0x52, 0x31, 0xcc, 0x68) +#define LINUX_EFI_ZBOOT_MEDIA_GUID EFI_GUID(0xe565a30d, 0x47da, 0x4dbd, 0xb3, 0x54, 0x9b, 0xb5, 0xc8, 0x4f, 0x8b, 0xe2) #define LINUX_EFI_MOK_VARIABLE_TABLE_GUID EFI_GUID(0xc451ed2b, 0x9694, 0x45d3, 0xba, 0xba, 0xed, 0x9f, 0x89, 0x88, 0xa3, 0x89) #define LINUX_EFI_COCO_SECRET_AREA_GUID EFI_GUID(0xadf956ad, 0xe98c, 0x484c, 0xae, 0x11, 0xb5, 0x1c, 0x7d, 0x33, 0x64, 0x47) +#define LINUX_EFI_BOOT_MEMMAP_GUID EFI_GUID(0x800f683f, 0xd08b, 0x423a, 0xa2, 0x93, 0x96, 0x5c, 0x3c, 0x6f, 0xe2, 0xb4) #define RISCV_EFI_BOOT_PROTOCOL_GUID EFI_GUID(0xccd15fec, 0x6f73, 0x4eec, 0x83, 0x95, 0x3e, 0x69, 0xe4, 0xb9, 0x40, 0xbf) @@ -518,6 +523,15 @@ typedef union { efi_system_table_32_t mixed_mode; } efi_system_table_t; +struct efi_boot_memmap { + unsigned long map_size; + unsigned long desc_size; + u32 desc_ver; + unsigned long map_key; + unsigned long buff_size; + efi_memory_desc_t map[]; +}; + /* * Architecture independent structure for describing a memory map for the * benefit of efi_memmap_init_early(), and for passing context between @@ -952,6 +966,7 @@ extern int efi_status_to_err(efi_status_t status); #define EFI_DEV_MEDIA_VENDOR 3 #define EFI_DEV_MEDIA_FILE 4 #define EFI_DEV_MEDIA_PROTOCOL 5 +#define EFI_DEV_MEDIA_REL_OFFSET 8 #define EFI_DEV_BIOS_BOOT 0x05 #define EFI_DEV_END_PATH 0x7F #define EFI_DEV_END_PATH2 0xFF @@ -982,12 +997,27 @@ struct efi_vendor_dev_path { u8 vendordata[]; } __packed; +struct efi_rel_offset_dev_path { + struct efi_generic_dev_path header; + u32 reserved; + u64 starting_offset; + u64 ending_offset; +} __packed; + +struct efi_mem_mapped_dev_path { + struct efi_generic_dev_path header; + u32 memory_type; + u64 starting_addr; + u64 ending_addr; +} __packed; + struct efi_dev_path { union { struct efi_generic_dev_path header; struct efi_acpi_dev_path acpi; struct efi_pci_dev_path pci; struct efi_vendor_dev_path vendor; + struct efi_rel_offset_dev_path rel_offset; }; } __packed; @@ -1321,6 +1351,11 @@ struct linux_efi_coco_secret_area { u64 size; }; +struct linux_efi_initrd { + unsigned long base; + unsigned long size; +}; + /* Header of a populated EFI secret area */ #define EFI_SECRET_TABLE_HEADER_GUID EFI_GUID(0x1e74f542, 0x71dd, 0x4d66, 0x96, 0x3e, 0xef, 0x42, 0x87, 0xff, 0x17, 0x3b) diff --git a/include/linux/pe.h b/include/linux/pe.h index daf09ffffe38..1d3836ef9d92 100644 --- a/include/linux/pe.h +++ b/include/linux/pe.h @@ -65,6 +65,8 @@ #define IMAGE_FILE_MACHINE_SH5 0x01a8 #define IMAGE_FILE_MACHINE_THUMB 0x01c2 #define IMAGE_FILE_MACHINE_WCEMIPSV2 0x0169 +#define IMAGE_FILE_MACHINE_LOONGARCH32 0x6232 +#define IMAGE_FILE_MACHINE_LOONGARCH64 0x6264 /* flags */ #define IMAGE_FILE_RELOCS_STRIPPED 0x0001 |