UefiCpuPkg: Allow AP booting under SEV-ES

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2198

Typically, an AP is booted using the INIT-SIPI-SIPI sequence. This
sequence is intercepted by the hypervisor, which sets the AP's registers
to the values requested by the sequence. At that point, the hypervisor can
start the AP, which will then begin execution at the appropriate location.

Under SEV-ES, AP booting presents some challenges since the hypervisor is
not allowed to alter the AP's register state. In this situation, we have
to distinguish between the AP's first boot and AP's subsequent boots.

First boot:
 Once the AP's register state has been defined (which is before the guest
 is first booted) it cannot be altered. Should the hypervisor attempt to
 alter the register state, the change would be detected by the hardware
 and the VMRUN instruction would fail. Given this, the first boot for the
 AP is required to begin execution with this initial register state, which
 is typically the reset vector. This prevents the BSP from directing the
 AP startup location through the INIT-SIPI-SIPI sequence.

 To work around this, the firmware will provide a build time reserved area
 that can be used as the initial IP value. The hypervisor can extract this
 location value by checking for the SEV-ES reset block GUID that must be
 located 48-bytes from the end of the firmware. The format of the SEV-ES
 reset block area is:

   0x00 - 0x01 - SEV-ES Reset IP
   0x02 - 0x03 - SEV-ES Reset CS Segment Base[31:16]
   0x04 - 0x05 - Size of the SEV-ES reset block
   0x06 - 0x15 - SEV-ES Reset Block GUID
                   (00f771de-1a7e-4fcb-890e-68c77e2fb44e)

   The total size is 22 bytes. Any expansion to this block must be done
   by adding new values before existing values.

 The hypervisor will use the IP and CS values obtained from the SEV-ES
 reset block to set as the AP's initial values. The CS Segment Base
 represents the upper 16 bits of the CS segment base and must be left
 shifted by 16 bits to form the complete CS segment base value.

 Before booting the AP for the first time, the BSP must initialize the
 SEV-ES reset area. This consists of programming a FAR JMP instruction
 to the contents of a memory location that is also located in the SEV-ES
 reset area. The BSP must program the IP and CS values for the FAR JMP
 based on values drived from the INIT-SIPI-SIPI sequence.

Subsequent boots:
 Again, the hypervisor cannot alter the AP register state, so a method is
 required to take the AP out of halt state and redirect it to the desired
 IP location. If it is determined that the AP is running in an SEV-ES
 guest, then instead of calling CpuSleep(), a VMGEXIT is issued with the
 AP Reset Hold exit code (0x80000004). The hypervisor will put the AP in
 a halt state, waiting for an INIT-SIPI-SIPI sequence. Once the sequence
 is recognized, the hypervisor will resume the AP. At this point the AP
 must transition from the current 64-bit long mode down to 16-bit real
 mode and begin executing at the derived location from the INIT-SIPI-SIPI
 sequence.

 Another change is around the area of obtaining the (x2)APIC ID during AP
 startup. During AP startup, the AP can't take a #VC exception before the
 AP has established a stack. However, the AP stack is set by using the
 (x2)APIC ID, which is obtained through CPUID instructions. A CPUID
 instruction will cause a #VC, so a different method must be used. The
 GHCB protocol supports a method to obtain CPUID information from the
 hypervisor through the GHCB MSR. This method does not require a stack,
 so it is used to obtain the necessary CPUID information to determine the
 (x2)APIC ID.

The new 16-bit protected mode GDT entry is used in order to transition
from 64-bit long mode down to 16-bit real mode.

A new assembler routine is created that takes the AP from 64-bit long mode
to 16-bit real mode.  This is located under 1MB in memory and transitions
from 64-bit long mode to 32-bit compatibility mode to 16-bit protected
mode and finally 16-bit real mode.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Eric Dong <eric.dong@intel.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Regression-tested-by: Laszlo Ersek <lersek@redhat.com>

1 files changed, 325 insertions, 11 deletions

diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c
index 2a3fbeef35..90416c81b6 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
@@ -9,6 +9,9 @@
 **/

 

 #include "MpLib.h"

+#include <Library/VmgExitLib.h>

+#include <Register/Amd/Fam17Msr.h>

+#include <Register/Amd/Ghcb.h>

 

 EFI_GUID mCpuInitMpLibHobGuid = CPU_INIT_MP_LIB_HOB_GUID;

 

@@ -291,6 +294,14 @@ GetApLoopMode (
       //

       ApLoopMode = ApInHltLoop;

     }

+

+    if (PcdGetBool (PcdSevEsIsEnabled)) {

+      //

+      // For SEV-ES, force AP in Hlt-loop mode in order to use the GHCB

+      // protocol for starting APs

+      //

+      ApLoopMode = ApInHltLoop;

+    }

   }

 

   if (ApLoopMode != ApInMwaitLoop) {

@@ -588,6 +599,112 @@ InitializeApData (
 }

 

 /**

+  Get Protected mode code segment with 16-bit default addressing

+  from current GDT table.

+

+  @return  Protected mode 16-bit code segment value.

+**/

+STATIC

+UINT16

+GetProtectedMode16CS (

+  VOID

+  )

+{

+  IA32_DESCRIPTOR          GdtrDesc;

+  IA32_SEGMENT_DESCRIPTOR  *GdtEntry;

+  UINTN                    GdtEntryCount;

+  UINT16                   Index;

+

+  Index = (UINT16) -1;

+  AsmReadGdtr (&GdtrDesc);

+  GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR);

+  GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;

+  for (Index = 0; Index < GdtEntryCount; Index++) {

+    if (GdtEntry->Bits.L == 0 &&

+        GdtEntry->Bits.DB == 0 &&

+        GdtEntry->Bits.Type > 8) {

+      break;

+    }

+    GdtEntry++;

+  }

+  ASSERT (Index != GdtEntryCount);

+  return Index * 8;

+}

+

+/**

+  Get Protected mode code segment with 32-bit default addressing

+  from current GDT table.

+

+  @return  Protected mode 32-bit code segment value.

+**/

+STATIC

+UINT16

+GetProtectedMode32CS (

+  VOID

+  )

+{

+  IA32_DESCRIPTOR          GdtrDesc;

+  IA32_SEGMENT_DESCRIPTOR  *GdtEntry;

+  UINTN                    GdtEntryCount;

+  UINT16                   Index;

+

+  Index = (UINT16) -1;

+  AsmReadGdtr (&GdtrDesc);

+  GdtEntryCount = (GdtrDesc.Limit + 1) / sizeof (IA32_SEGMENT_DESCRIPTOR);

+  GdtEntry = (IA32_SEGMENT_DESCRIPTOR *) GdtrDesc.Base;

+  for (Index = 0; Index < GdtEntryCount; Index++) {

+    if (GdtEntry->Bits.L == 0 &&

+        GdtEntry->Bits.DB == 1 &&

+        GdtEntry->Bits.Type > 8) {

+      break;

+    }

+    GdtEntry++;

+  }

+  ASSERT (Index != GdtEntryCount);

+  return Index * 8;

+}

+

+/**

+  Reset an AP when in SEV-ES mode.

+

+  If successful, this function never returns.

+

+  @param[in] Ghcb                 Pointer to the GHCB

+  @param[in] CpuMpData            Pointer to CPU MP Data

+

+**/

+STATIC

+VOID

+MpInitLibSevEsAPReset (

+  IN GHCB                         *Ghcb,

+  IN CPU_MP_DATA                  *CpuMpData

+  )

+{

+  UINT16           Code16, Code32;

+  AP_RESET         *APResetFn;

+  UINTN            BufferStart;

+  UINTN            StackStart;

+

+  Code16 = GetProtectedMode16CS ();

+  Code32 = GetProtectedMode32CS ();

+

+  if (CpuMpData->WakeupBufferHigh != 0) {

+    APResetFn = (AP_RESET *) (CpuMpData->WakeupBufferHigh + CpuMpData->AddressMap.SwitchToRealNoNxOffset);

+  } else {

+    APResetFn = (AP_RESET *) (CpuMpData->MpCpuExchangeInfo->BufferStart + CpuMpData->AddressMap.SwitchToRealOffset);

+  }

+

+  BufferStart = CpuMpData->MpCpuExchangeInfo->BufferStart;

+  StackStart = CpuMpData->SevEsAPResetStackStart -

+                 (AP_RESET_STACK_SIZE * GetApicId ());

+

+  //

+  // This call never returns.

+  //

+  APResetFn (BufferStart, Code16, Code32, StackStart);

+}

+

+/**

   This function will be called from AP reset code if BSP uses WakeUpAP.

 

   @param[in] ExchangeInfo     Pointer to the MP exchange info buffer

@@ -648,7 +765,14 @@ ApWakeupFunction (
       InitializeApData (CpuMpData, ProcessorNumber, BistData, ApTopOfStack);

       ApStartupSignalBuffer = CpuMpData->CpuData[ProcessorNumber].StartupApSignal;

 

-      InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->NumApsExecuting);

+      //

+      // Delay decrementing the APs executing count when SEV-ES is enabled

+      // to allow the APs to issue an AP_RESET_HOLD before the BSP possibly

+      // performs another INIT-SIPI-SIPI sequence.

+      //

+      if (!CpuMpData->SevEsIsEnabled) {

+        InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->NumApsExecuting);

+      }

     } else {

       //

       // Execute AP function if AP is ready

@@ -755,7 +879,52 @@ ApWakeupFunction (
       //

       while (TRUE) {

         DisableInterrupts ();

-        CpuSleep ();

+        if (CpuMpData->SevEsIsEnabled) {

+          MSR_SEV_ES_GHCB_REGISTER  Msr;

+          GHCB                      *Ghcb;

+          UINT64                    Status;

+          BOOLEAN                   DoDecrement;

+

+          if (CpuMpData->InitFlag == ApInitConfig) {

+            DoDecrement = TRUE;

+          }

+

+          while (TRUE) {

+            Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);

+            Ghcb = Msr.Ghcb;

+

+            VmgInit (Ghcb);

+

+            if (DoDecrement) {

+              DoDecrement = FALSE;

+

+              //

+              // Perform the delayed decrement just before issuing the first

+              // VMGEXIT with AP_RESET_HOLD.

+              //

+              InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->NumApsExecuting);

+            }

+

+            Status = VmgExit (Ghcb, SVM_EXIT_AP_RESET_HOLD, 0, 0);

+            if ((Status == 0) && (Ghcb->SaveArea.SwExitInfo2 != 0)) {

+              VmgDone (Ghcb);

+              break;

+            }

+

+            VmgDone (Ghcb);

+          }

+

+          //

+          // Awakened in a new phase? Use the new CpuMpData

+          //

+          if (CpuMpData->NewCpuMpData != NULL) {

+            CpuMpData = CpuMpData->NewCpuMpData;

+          }

+

+          MpInitLibSevEsAPReset (Ghcb, CpuMpData);

+        } else {

+          CpuSleep ();

+        }

         CpuPause ();

       }

     }

@@ -868,6 +1037,9 @@ FillExchangeInfoData (
   ExchangeInfo->Enable5LevelPaging = (BOOLEAN) (Cr4.Bits.LA57 == 1);

   DEBUG ((DEBUG_INFO, "%a: 5-Level Paging = %d\n", gEfiCallerBaseName, ExchangeInfo->Enable5LevelPaging));

 

+  ExchangeInfo->SevEsIsEnabled  = CpuMpData->SevEsIsEnabled;

+  ExchangeInfo->GhcbBase        = (UINTN) CpuMpData->GhcbBase;

+

   //

   // Get the BSP's data of GDT and IDT

   //

@@ -894,8 +1066,9 @@ FillExchangeInfoData (
   // EfiBootServicesCode to avoid page fault if NX memory protection is enabled.

   //

   if (CpuMpData->WakeupBufferHigh != 0) {

-    Size = CpuMpData->AddressMap.RendezvousFunnelSize -

-           CpuMpData->AddressMap.ModeTransitionOffset;

+    Size = CpuMpData->AddressMap.RendezvousFunnelSize +

+             CpuMpData->AddressMap.SwitchToRealSize -

+             CpuMpData->AddressMap.ModeTransitionOffset;

     CopyMem (

       (VOID *)CpuMpData->WakeupBufferHigh,

       CpuMpData->AddressMap.RendezvousFunnelAddress +

@@ -948,7 +1121,8 @@ BackupAndPrepareWakeupBuffer(
   CopyMem (

     (VOID *) CpuMpData->WakeupBuffer,

     (VOID *) CpuMpData->AddressMap.RendezvousFunnelAddress,

-    CpuMpData->AddressMap.RendezvousFunnelSize

+    CpuMpData->AddressMap.RendezvousFunnelSize +

+      CpuMpData->AddressMap.SwitchToRealSize

     );

 }

 

@@ -970,6 +1144,44 @@ RestoreWakeupBuffer(
 }

 

 /**

+  Calculate the size of the reset stack.

+

+  @return                 Total amount of memory required for stacks

+**/

+STATIC

+UINTN

+GetApResetStackSize (

+  VOID

+  )

+{

+  return AP_RESET_STACK_SIZE * PcdGet32(PcdCpuMaxLogicalProcessorNumber);

+}

+

+/**

+  Calculate the size of the reset vector.

+

+  @param[in]  AddressMap  The pointer to Address Map structure.

+

+  @return                 Total amount of memory required for the AP reset area

+**/

+STATIC

+UINTN

+GetApResetVectorSize (

+  IN MP_ASSEMBLY_ADDRESS_MAP  *AddressMap

+  )

+{

+  UINTN  Size;

+

+  Size = ALIGN_VALUE (AddressMap->RendezvousFunnelSize +

+                        AddressMap->SwitchToRealSize +

+                        sizeof (MP_CPU_EXCHANGE_INFO),

+                      CPU_STACK_ALIGNMENT);

+  Size += GetApResetStackSize ();

+

+  return Size;

+}

+

+/**

   Allocate reset vector buffer.

 

   @param[in, out]  CpuMpData  The pointer to CPU MP Data structure.

@@ -982,16 +1194,22 @@ AllocateResetVector (
   UINTN           ApResetVectorSize;

 

   if (CpuMpData->WakeupBuffer == (UINTN) -1) {

-    ApResetVectorSize = CpuMpData->AddressMap.RendezvousFunnelSize +

-                          sizeof (MP_CPU_EXCHANGE_INFO);

+    ApResetVectorSize = GetApResetVectorSize (&CpuMpData->AddressMap);

 

     CpuMpData->WakeupBuffer      = GetWakeupBuffer (ApResetVectorSize);

     CpuMpData->MpCpuExchangeInfo = (MP_CPU_EXCHANGE_INFO *) (UINTN)

-                    (CpuMpData->WakeupBuffer + CpuMpData->AddressMap.RendezvousFunnelSize);

+                    (CpuMpData->WakeupBuffer +

+                       CpuMpData->AddressMap.RendezvousFunnelSize +

+                       CpuMpData->AddressMap.SwitchToRealSize);

     CpuMpData->WakeupBufferHigh  = GetModeTransitionBuffer (

-                                    CpuMpData->AddressMap.RendezvousFunnelSize -

+                                    CpuMpData->AddressMap.RendezvousFunnelSize +

+                                    CpuMpData->AddressMap.SwitchToRealSize -

                                     CpuMpData->AddressMap.ModeTransitionOffset

                                     );

+    //

+    // The reset stack starts at the end of the buffer.

+    //

+    CpuMpData->SevEsAPResetStackStart = CpuMpData->WakeupBuffer + ApResetVectorSize;

   }

   BackupAndPrepareWakeupBuffer (CpuMpData);

 }

@@ -1006,7 +1224,80 @@ FreeResetVector (
   IN CPU_MP_DATA              *CpuMpData

   )

 {

-  RestoreWakeupBuffer (CpuMpData);

+  //

+  // If SEV-ES is enabled, the reset area is needed for AP parking and

+  // and AP startup in the OS, so the reset area is reserved. Do not

+  // perform the restore as this will overwrite memory which has data

+  // needed by SEV-ES.

+  //

+  if (!CpuMpData->SevEsIsEnabled) {

+    RestoreWakeupBuffer (CpuMpData);

+  }

+}

+

+/**

+  Allocate the SEV-ES AP jump table buffer.

+

+  @param[in, out]  CpuMpData  The pointer to CPU MP Data structure.

+**/

+VOID

+AllocateSevEsAPMemory (

+  IN OUT CPU_MP_DATA          *CpuMpData

+  )

+{

+  if (CpuMpData->SevEsAPBuffer == (UINTN) -1) {

+    CpuMpData->SevEsAPBuffer =

+      CpuMpData->SevEsIsEnabled ? GetSevEsAPMemory () : 0;

+  }

+}

+

+/**

+  Program the SEV-ES AP jump table buffer.

+

+  @param[in]  SipiVector  The SIPI vector used for the AP Reset

+**/

+VOID

+SetSevEsJumpTable (

+  IN UINTN  SipiVector

+  )

+{

+  SEV_ES_AP_JMP_FAR *JmpFar;

+  UINT32            Offset, InsnByte;

+  UINT8             LoNib, HiNib;

+

+  JmpFar = (SEV_ES_AP_JMP_FAR *) FixedPcdGet32 (PcdSevEsWorkAreaBase);

+  ASSERT (JmpFar != NULL);

+

+  //

+  // Obtain the address of the Segment/Rip location in the workarea.

+  // This will be set to a value derived from the SIPI vector and will

+  // be the memory address used for the far jump below.

+  //

+  Offset = FixedPcdGet32 (PcdSevEsWorkAreaBase);

+  Offset += sizeof (JmpFar->InsnBuffer);

+  LoNib = (UINT8) Offset;

+  HiNib = (UINT8) (Offset >> 8);

+

+  //

+  // Program the workarea (which is the initial AP boot address) with

+  // far jump to the SIPI vector (where XX and YY represent the

+  // address of where the SIPI vector is stored.

+  //

+  //   JMP FAR [CS:XXYY] => 2E FF 2E YY XX

+  //

+  InsnByte = 0;

+  JmpFar->InsnBuffer[InsnByte++] = 0x2E;  // CS override prefix

+  JmpFar->InsnBuffer[InsnByte++] = 0xFF;  // JMP (FAR)

+  JmpFar->InsnBuffer[InsnByte++] = 0x2E;  // ModRM (JMP memory location)

+  JmpFar->InsnBuffer[InsnByte++] = LoNib; // YY offset ...

+  JmpFar->InsnBuffer[InsnByte++] = HiNib; // XX offset ...

+

+  //

+  // Program the Segment/Rip based on the SIPI vector (always at least

+  // 16-byte aligned, so Rip is set to 0).

+  //

+  JmpFar->Rip = 0;

+  JmpFar->Segment = (UINT16) (SipiVector >> 4);

 }

 

 /**

@@ -1043,6 +1334,7 @@ WakeUpAP (
       CpuMpData->InitFlag   != ApInitDone) {

     ResetVectorRequired = TRUE;

     AllocateResetVector (CpuMpData);

+    AllocateSevEsAPMemory (CpuMpData);

     FillExchangeInfoData (CpuMpData);

     SaveLocalApicTimerSetting (CpuMpData);

   }

@@ -1080,6 +1372,15 @@ WakeUpAP (
     }

     if (ResetVectorRequired) {

       //

+      // For SEV-ES, the initial AP boot address will be defined by

+      // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address

+      // from the original INIT-SIPI-SIPI.

+      //

+      if (CpuMpData->SevEsIsEnabled) {

+        SetSevEsJumpTable (ExchangeInfo->BufferStart);

+      }

+

+      //

       // Wakeup all APs

       //

       SendInitSipiSipiAllExcludingSelf ((UINT32) ExchangeInfo->BufferStart);

@@ -1170,6 +1471,16 @@ WakeUpAP (
     *(UINT32 *) CpuData->StartupApSignal = WAKEUP_AP_SIGNAL;

     if (ResetVectorRequired) {

       CpuInfoInHob = (CPU_INFO_IN_HOB *) (UINTN) CpuMpData->CpuInfoInHob;

+

+      //

+      // For SEV-ES, the initial AP boot address will be defined by

+      // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address

+      // from the original INIT-SIPI-SIPI.

+      //

+      if (CpuMpData->SevEsIsEnabled) {

+        SetSevEsJumpTable (ExchangeInfo->BufferStart);

+      }

+

       SendInitSipiSipi (

         CpuInfoInHob[ProcessorNumber].ApicId,

         (UINT32) ExchangeInfo->BufferStart

@@ -1646,7 +1957,7 @@ MpInitLibInitialize (
   ASSERT (MaxLogicalProcessorNumber != 0);

 

   AsmGetAddressMap (&AddressMap);

-  ApResetVectorSize = AddressMap.RendezvousFunnelSize + sizeof (MP_CPU_EXCHANGE_INFO);

+  ApResetVectorSize = GetApResetVectorSize (&AddressMap);

   ApStackSize = PcdGet32(PcdCpuApStackSize);

   ApLoopMode  = GetApLoopMode (&MonitorFilterSize);

 

@@ -1705,6 +2016,8 @@ MpInitLibInitialize (
   CpuMpData->CpuInfoInHob     = (UINT64) (UINTN) (CpuMpData->CpuData + MaxLogicalProcessorNumber);

   InitializeSpinLock(&CpuMpData->MpLock);

   CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled);

+  CpuMpData->SevEsAPBuffer  = (UINTN) -1;

+  CpuMpData->GhcbBase       = PcdGet64 (PcdGhcbBase);

 

   //

   // Make sure no memory usage outside of the allocated buffer.

@@ -1763,6 +2076,7 @@ MpInitLibInitialize (
     // APs have been wakeup before, just get the CPU Information

     // from HOB

     //

+    OldCpuMpData->NewCpuMpData = CpuMpData;

     CpuMpData->CpuCount  = OldCpuMpData->CpuCount;

     CpuMpData->BspNumber = OldCpuMpData->BspNumber;

     CpuMpData->CpuInfoInHob = OldCpuMpData->CpuInfoInHob;