UefiCpuPkg: Allow AP booting under SEV-ES

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2198

Typically, an AP is booted using the INIT-SIPI-SIPI sequence. This
sequence is intercepted by the hypervisor, which sets the AP's registers
to the values requested by the sequence. At that point, the hypervisor can
start the AP, which will then begin execution at the appropriate location.

Under SEV-ES, AP booting presents some challenges since the hypervisor is
not allowed to alter the AP's register state. In this situation, we have
to distinguish between the AP's first boot and AP's subsequent boots.

First boot:
 Once the AP's register state has been defined (which is before the guest
 is first booted) it cannot be altered. Should the hypervisor attempt to
 alter the register state, the change would be detected by the hardware
 and the VMRUN instruction would fail. Given this, the first boot for the
 AP is required to begin execution with this initial register state, which
 is typically the reset vector. This prevents the BSP from directing the
 AP startup location through the INIT-SIPI-SIPI sequence.

 To work around this, the firmware will provide a build time reserved area
 that can be used as the initial IP value. The hypervisor can extract this
 location value by checking for the SEV-ES reset block GUID that must be
 located 48-bytes from the end of the firmware. The format of the SEV-ES
 reset block area is:

   0x00 - 0x01 - SEV-ES Reset IP
   0x02 - 0x03 - SEV-ES Reset CS Segment Base[31:16]
   0x04 - 0x05 - Size of the SEV-ES reset block
   0x06 - 0x15 - SEV-ES Reset Block GUID
                   (00f771de-1a7e-4fcb-890e-68c77e2fb44e)

   The total size is 22 bytes. Any expansion to this block must be done
   by adding new values before existing values.

 The hypervisor will use the IP and CS values obtained from the SEV-ES
 reset block to set as the AP's initial values. The CS Segment Base
 represents the upper 16 bits of the CS segment base and must be left
 shifted by 16 bits to form the complete CS segment base value.

 Before booting the AP for the first time, the BSP must initialize the
 SEV-ES reset area. This consists of programming a FAR JMP instruction
 to the contents of a memory location that is also located in the SEV-ES
 reset area. The BSP must program the IP and CS values for the FAR JMP
 based on values drived from the INIT-SIPI-SIPI sequence.

Subsequent boots:
 Again, the hypervisor cannot alter the AP register state, so a method is
 required to take the AP out of halt state and redirect it to the desired
 IP location. If it is determined that the AP is running in an SEV-ES
 guest, then instead of calling CpuSleep(), a VMGEXIT is issued with the
 AP Reset Hold exit code (0x80000004). The hypervisor will put the AP in
 a halt state, waiting for an INIT-SIPI-SIPI sequence. Once the sequence
 is recognized, the hypervisor will resume the AP. At this point the AP
 must transition from the current 64-bit long mode down to 16-bit real
 mode and begin executing at the derived location from the INIT-SIPI-SIPI
 sequence.

 Another change is around the area of obtaining the (x2)APIC ID during AP
 startup. During AP startup, the AP can't take a #VC exception before the
 AP has established a stack. However, the AP stack is set by using the
 (x2)APIC ID, which is obtained through CPUID instructions. A CPUID
 instruction will cause a #VC, so a different method must be used. The
 GHCB protocol supports a method to obtain CPUID information from the
 hypervisor through the GHCB MSR. This method does not require a stack,
 so it is used to obtain the necessary CPUID information to determine the
 (x2)APIC ID.

The new 16-bit protected mode GDT entry is used in order to transition
from 64-bit long mode down to 16-bit real mode.

A new assembler routine is created that takes the AP from 64-bit long mode
to 16-bit real mode.  This is located under 1MB in memory and transitions
from 64-bit long mode to 32-bit compatibility mode to 16-bit protected
mode and finally 16-bit real mode.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Regression-tested-by: Laszlo Ersek <lersek@redhat.com>

2 files changed, 242 insertions, 1 deletions

diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
index 58ef369342..c92daaaffd 100644
--- a/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
+++ b/UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc
@@ -19,7 +19,7 @@ CPU_SWITCH_STATE_IDLE         equ        0
 CPU_SWITCH_STATE_STORED       equ        1

 CPU_SWITCH_STATE_LOADED       equ        2

 

-LockLocation                  equ        (RendezvousFunnelProcEnd - RendezvousFunnelProcStart)

+LockLocation                  equ        (SwitchToRealProcEnd - RendezvousFunnelProcStart)

 StackStartAddressLocation     equ        LockLocation + 08h

 StackSizeLocation             equ        LockLocation + 10h

 ApProcedureLocation           equ        LockLocation + 18h

@@ -41,3 +41,5 @@ ModeTransitionSegmentLocation       equ  LockLocation + 98h
 ModeHighMemoryLocation              equ  LockLocation + 9Ah

 ModeHighSegmentLocation             equ  LockLocation + 9Eh

 Enable5LevelPagingLocation          equ  LockLocation + 0A0h

+SevEsIsEnabledLocation              equ  LockLocation + 0A1h

+GhcbBaseLocation                    equ  LockLocation + 0A2h

diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
index 87f2523e85..6956b408d0 100644
--- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
+++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
@@ -184,9 +184,97 @@ Releaselock:
     add        edi, StackStartAddressLocation

     add        rax, qword [edi]

     mov        rsp, rax

+

+    lea        edi, [esi + SevEsIsEnabledLocation]

+    cmp        byte [edi], 1        ; SevEsIsEnabled

+    jne        CProcedureInvoke

+

+    ;

+    ; program GHCB

+    ;   Each page after the GHCB is a per-CPU page, so the calculation programs

+    ;   a GHCB to be every 8KB.

+    ;

+    mov        eax, SIZE_4KB

+    shl        eax, 1                            ; EAX = SIZE_4K * 2

+    mov        ecx, ebx

+    mul        ecx                               ; EAX = SIZE_4K * 2 * CpuNumber

+    mov        edi, esi

+    add        edi, GhcbBaseLocation

+    add        rax, qword [edi]

+    mov        rdx, rax

+    shr        rdx, 32

+    mov        rcx, 0xc0010130

+    wrmsr

     jmp        CProcedureInvoke

 

 GetApicId:

+    lea        edi, [esi + SevEsIsEnabledLocation]

+    cmp        byte [edi], 1        ; SevEsIsEnabled

+    jne        DoCpuid

+

+    ;

+    ; Since we don't have a stack yet, we can't take a #VC

+    ; exception. Use the GHCB protocol to perform the CPUID

+    ; calls.

+    ;

+    mov        rcx, 0xc0010130

+    rdmsr

+    shl        rdx, 32

+    or         rax, rdx

+    mov        rdi, rax             ; RDI now holds the original GHCB GPA

+

+    mov        rdx, 0               ; CPUID function 0

+    mov        rax, 0               ; RAX register requested

+    or         rax, 4

+    wrmsr

+    rep vmmcall

+    rdmsr

+    cmp        edx, 0bh

+    jb         NoX2ApicSevEs        ; CPUID level below CPUID_EXTENDED_TOPOLOGY

+

+    mov        rdx, 0bh             ; CPUID function 0x0b

+    mov        rax, 040000000h      ; RBX register requested

+    or         rax, 4

+    wrmsr

+    rep vmmcall

+    rdmsr

+    test       edx, 0ffffh

+    jz         NoX2ApicSevEs        ; CPUID.0BH:EBX[15:0] is zero

+

+    mov        rdx, 0bh             ; CPUID function 0x0b

+    mov        rax, 0c0000000h      ; RDX register requested

+    or         rax, 4

+    wrmsr

+    rep vmmcall

+    rdmsr

+

+    ; Processor is x2APIC capable; 32-bit x2APIC ID is now in EDX

+    jmp        RestoreGhcb

+

+NoX2ApicSevEs:

+    ; Processor is not x2APIC capable, so get 8-bit APIC ID

+    mov        rdx, 1               ; CPUID function 1

+    mov        rax, 040000000h      ; RBX register requested

+    or         rax, 4

+    wrmsr

+    rep vmmcall

+    rdmsr

+    shr        edx, 24

+

+RestoreGhcb:

+    mov        rbx, rdx             ; Save x2APIC/APIC ID

+

+    mov        rdx, rdi             ; RDI holds the saved GHCB GPA

+    shr        rdx, 32

+    mov        eax, edi

+    wrmsr

+

+    mov        rdx, rbx

+

+    ; x2APIC ID or APIC ID is in EDX

+    jmp        GetProcessorNumber

+

+DoCpuid:

     mov        eax, 0

     cpuid

     cmp        eax, 0bh

@@ -254,11 +342,157 @@ CProcedureInvoke:
 RendezvousFunnelProcEnd:

 

 ;-------------------------------------------------------------------------------------

+;SwitchToRealProc procedure follows.

+;ALSO THIS PROCEDURE IS EXECUTED BY APs TRANSITIONING TO 16 BIT MODE. HENCE THIS PROC

+;IS IN MACHINE CODE.

+;  SwitchToRealProc (UINTN BufferStart, UINT16 Code16, UINT16 Code32, UINTN StackStart)

+;  rcx - Buffer Start

+;  rdx - Code16 Selector Offset

+;  r8  - Code32 Selector Offset

+;  r9  - Stack Start

+;-------------------------------------------------------------------------------------

+global ASM_PFX(SwitchToRealProc)

+ASM_PFX(SwitchToRealProc):

+SwitchToRealProcStart:

+BITS 64

+    cli

+

+    ;

+    ; Get RDX reset value before changing stacks since the

+    ; new stack won't be able to accomodate a #VC exception.

+    ;

+    push       rax

+    push       rbx

+    push       rcx

+    push       rdx

+

+    mov        rax, 1

+    cpuid

+    mov        rsi, rax                    ; Save off the reset value for RDX

+

+    pop        rdx

+    pop        rcx

+    pop        rbx

+    pop        rax

+

+    ;

+    ; Establish stack below 1MB

+    ;

+    mov        rsp, r9

+

+    ;

+    ; Push ultimate Reset Vector onto the stack

+    ;

+    mov        rax, rcx

+    shr        rax, 4

+    push       word 0x0002                 ; RFLAGS

+    push       ax                          ; CS

+    push       word 0x0000                 ; RIP

+    push       word 0x0000                 ; For alignment, will be discarded

+

+    ;

+    ; Get address of "16-bit operand size" label

+    ;

+    lea        rbx, [PM16Mode]

+

+    ;

+    ; Push addresses used to change to compatibility mode

+    ;

+    lea        rax, [CompatMode]

+    push       r8

+    push       rax

+

+    ;

+    ; Clear R8 - R15, for reset, before going into 32-bit mode

+    ;

+    xor        r8, r8

+    xor        r9, r9

+    xor        r10, r10

+    xor        r11, r11

+    xor        r12, r12

+    xor        r13, r13

+    xor        r14, r14

+    xor        r15, r15

+

+    ;

+    ; Far return into 32-bit mode

+    ;

+o64 retf

+

+BITS 32

+CompatMode:

+    ;

+    ; Set up stack to prepare for exiting protected mode

+    ;

+    push       edx                         ; Code16 CS

+    push       ebx                         ; PM16Mode label address

+

+    ;

+    ; Disable paging

+    ;

+    mov        eax, cr0                    ; Read CR0

+    btr        eax, 31                     ; Set PG=0

+    mov        cr0, eax                    ; Write CR0

+

+    ;

+    ; Disable long mode

+    ;

+    mov        ecx, 0c0000080h             ; EFER MSR number

+    rdmsr                                  ; Read EFER

+    btr        eax, 8                      ; Set LME=0

+    wrmsr                                  ; Write EFER

+

+    ;

+    ; Disable PAE

+    ;

+    mov        eax, cr4                    ; Read CR4

+    btr        eax, 5                      ; Set PAE=0

+    mov        cr4, eax                    ; Write CR4

+

+    mov        edx, esi                    ; Restore RDX reset value

+

+    ;

+    ; Switch to 16-bit operand size

+    ;

+    retf

+

+BITS 16

+    ;

+    ; At entry to this label

+    ;   - RDX will have its reset value

+    ;   - On the top of the stack

+    ;     - Alignment data (two bytes) to be discarded

+    ;     - IP for Real Mode (two bytes)

+    ;     - CS for Real Mode (two bytes)

+    ;

+PM16Mode:

+    mov        eax, cr0                    ; Read CR0

+    btr        eax, 0                      ; Set PE=0

+    mov        cr0, eax                    ; Write CR0

+

+    pop        ax                          ; Discard alignment data

+

+    ;

+    ; Clear registers (except RDX and RSP) before going into 16-bit mode

+    ;

+    xor        eax, eax

+    xor        ebx, ebx

+    xor        ecx, ecx

+    xor        esi, esi

+    xor        edi, edi

+    xor        ebp, ebp

+

+    iret

+

+SwitchToRealProcEnd:

+

+;-------------------------------------------------------------------------------------

 ;  AsmRelocateApLoop (MwaitSupport, ApTargetCState, PmCodeSegment, TopOfApStack, CountTofinish);

 ;-------------------------------------------------------------------------------------

 global ASM_PFX(AsmRelocateApLoop)

 ASM_PFX(AsmRelocateApLoop):

 AsmRelocateApLoopStart:

+BITS 64

     cli                          ; Disable interrupt before switching to 32-bit mode

     mov        rax, [rsp + 40]   ; CountTofinish

     lock dec   dword [rax]       ; (*CountTofinish)--

@@ -324,6 +558,11 @@ ASM_PFX(AsmGetAddressMap):
     mov        qword [rcx + 18h], rax

     mov        qword [rcx + 20h], AsmRelocateApLoopEnd - AsmRelocateApLoopStart

     mov        qword [rcx + 28h], Flat32Start - RendezvousFunnelProcStart

+    mov        qword [rcx + 30h], SwitchToRealProcEnd - SwitchToRealProcStart          ; SwitchToRealSize

+    mov        qword [rcx + 38h], SwitchToRealProcStart - RendezvousFunnelProcStart    ; SwitchToRealOffset

+    mov        qword [rcx + 40h], SwitchToRealProcStart - Flat32Start                  ; SwitchToRealNoNxOffset

+    mov        qword [rcx + 48h], PM16Mode - RendezvousFunnelProcStart                 ; SwitchToRealPM16ModeOffset

+    mov        qword [rcx + 50h], SwitchToRealProcEnd - PM16Mode                       ; SwitchToRealPM16ModeSize

     ret

 

 ;-------------------------------------------------------------------------------------