UefiCpuPkg/RegisterCpuFeaturesLib: Add logic to support semaphore type.

V4 changes include:
1. Serial debug message for different threads when program the register table.

V3 changes include:
1. Use global variable instead of internal function to return string for register type
   and dependence type.
2. Add comments for some complicated logic.

V2 changes include:
1. Add more description for the code part which need easy to understand.
2. Refine some code base on feedback for V1 changes.

V1 changes include:
In a system which has multiple cores, current set register value task costs huge times.
After investigation, current set MSR task costs most of the times. Current logic uses
SpinLock to let set MSR task as an single thread task for all cores. Because MSR has
scope attribute which may cause GP fault if multiple APs set MSR at the same time,
current logic use an easiest solution (use SpinLock) to avoid this issue, but it will
cost huge times.

In order to fix this performance issue, new solution will set MSRs base on their scope
attribute. After this, the SpinLock will not needed. Without SpinLock, new issue raised
which is caused by MSR dependence. For example, MSR A depends on MSR B which means MSR A
must been set after MSR B has been set. Also MSR B is package scope level and MSR A is
thread scope level. If system has multiple threads, Thread 1 needs to set the thread level
MSRs and thread 2 needs to set thread and package level MSRs. Set MSRs task for thread 1
and thread 2 like below:

            Thread 1                 Thread 2
MSR B          N                        Y
MSR A          Y                        Y

If driver don't control execute MSR order, for thread 1, it will execute MSR A first, but
at this time, MSR B not been executed yet by thread 2. system may trig exception at this
time.

In order to fix the above issue, driver introduces semaphore logic to control the MSR
execute sequence. For the above case, a semaphore will be add between MSR A and B for
all threads. Semaphore has scope info for it. The possible scope value is core or package.
For each thread, when it meets a semaphore during it set registers, it will 1) release
semaphore (+1) for each threads in this core or package(based on the scope info for this
semaphore) 2) acquire semaphore (-1) for all the threads in this core or package(based
on the scope info for this semaphore). With these two steps, driver can control MSR
sequence. Sample code logic like below:

  //
  // First increase semaphore count by 1 for processors in this package.
  //
  for (ProcessorIndex = 0; ProcessorIndex < PackageThreadsCount ; ProcessorIndex ++) {
    LibReleaseSemaphore ((UINT32 *) &SemaphorePtr[PackageOffset + ProcessorIndex]);
  }
  //
  // Second, check whether the count has reach the check number.
  //
  for (ProcessorIndex = 0; ProcessorIndex < ValidApCount; ProcessorIndex ++) {
    LibWaitForSemaphore (&SemaphorePtr[ApOffset]);
  }

Platform Requirement:
1. This change requires register MSR setting base on MSR scope info. If still register MSR
   for all threads, exception may raised.

Known limitation:
1. Current CpuFeatures driver supports DXE instance and PEI instance. But semaphore logic
   requires Aps execute in async mode which is not supported by PEI driver. So CpuFeature
   PEI instance not works after this change. We plan to support async mode for PEI in phase
   2 for this task.

Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Eric Dong <eric.dong@intel.com>
Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>

1 files changed, 59 insertions, 3 deletions

diff --git a/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h b/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
index edd266934f..42a3f91fbf 100644
--- a/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
+++ b/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
@@ -23,6 +23,7 @@
 #include <Library/MemoryAllocationLib.h>

 #include <Library/SynchronizationLib.h>

 #include <Library/IoLib.h>

+#include <Library/LocalApicLib.h>

 

 #include <AcpiCpuData.h>

 

@@ -46,16 +47,27 @@ typedef struct {
   CPU_FEATURE_INITIALIZE       InitializeFunc;

   UINT8                        *BeforeFeatureBitMask;

   UINT8                        *AfterFeatureBitMask;

+  UINT8                        *CoreBeforeFeatureBitMask;

+  UINT8                        *CoreAfterFeatureBitMask;

+  UINT8                        *PackageBeforeFeatureBitMask;

+  UINT8                        *PackageAfterFeatureBitMask;

   VOID                         *ConfigData;

   BOOLEAN                      BeforeAll;

   BOOLEAN                      AfterAll;

 } CPU_FEATURES_ENTRY;

 

+//

+// Flags used when program the register.

+//

+typedef struct {

+  volatile UINTN           ConsoleLogLock;       // Spinlock used to control console.

+  volatile UINTN           MemoryMappedLock;     // Spinlock used to program mmio

+  volatile UINT32          *SemaphoreCount;      // Semaphore used to program semaphore.

+} PROGRAM_CPU_REGISTER_FLAGS;

+

 typedef struct {

   UINTN                    FeaturesCount;

   UINT32                   BitMaskSize;

-  SPIN_LOCK                MsrLock;

-  SPIN_LOCK                MemoryMappedLock;

   LIST_ENTRY               FeatureList;

 

   CPU_FEATURES_INIT_ORDER  *InitOrder;

@@ -64,9 +76,14 @@ typedef struct {
   UINT8                    *ConfigurationPcd;

   UINT8                    *SettingPcd;

 

+  UINT32                   NumberOfCpus;

+  ACPI_CPU_DATA            *AcpiCpuData;

+

   CPU_REGISTER_TABLE       *RegisterTable;

   CPU_REGISTER_TABLE       *PreSmmRegisterTable;

   UINTN                    BspNumber;

+

+  PROGRAM_CPU_REGISTER_FLAGS  CpuFlags;

 } CPU_FEATURES_DATA;

 

 #define CPU_FEATURE_ENTRY_FROM_LINK(a) \

@@ -118,10 +135,13 @@ GetProcessorInformation (
 

   @param[in]  Procedure               A pointer to the function to be run on

                                       enabled APs of the system.

+  @param[in]  MpEvent                 A pointer to the event to be used later

+                                      to check whether procedure has done.

 **/

 VOID

 StartupAPsWorker (

-  IN  EFI_AP_PROCEDURE                 Procedure

+  IN  EFI_AP_PROCEDURE                 Procedure,

+  IN  EFI_EVENT                        MpEvent

   );

 

 /**

@@ -170,4 +190,40 @@ DumpCpuFeature (
   IN CPU_FEATURES_ENTRY  *CpuFeature

   );

 

+/**

+  Return feature dependence result.

+

+  @param[in]  CpuFeature        Pointer to CPU feature.

+  @param[in]  Before            Check before dependence or after.

+

+  @retval     return the dependence result.

+**/

+CPU_FEATURE_DEPENDENCE_TYPE

+DetectFeatureScope (

+  IN CPU_FEATURES_ENTRY         *CpuFeature,

+  IN BOOLEAN                    Before

+  );

+

+/**

+  Programs registers for the calling processor.

+

+  @param[in,out] Buffer  The pointer to private data buffer.

+

+**/

+VOID

+EFIAPI

+SetProcessorRegister (

+  IN OUT VOID            *Buffer

+  );

+

+/**

+  Return ACPI_CPU_DATA data.

+

+  @return  Pointer to ACPI_CPU_DATA data.

+**/

+ACPI_CPU_DATA *

+GetAcpiCpuData (

+  VOID

+  );

+

 #endif