MdeModulePkg/Variable: Add RT GetVariable() cache support

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2220

This change reduces SMIs for GetVariable () by maintaining a
UEFI variable cache in Runtime DXE in addition to the pre-
existing cache in SMRAM. When the Runtime Service GetVariable()
is invoked, a Runtime DXE cache is used instead of triggering an
SMI to VariableSmm. This can improve overall system performance
by servicing variable read requests without rendezvousing all
cores into SMM.

The runtime cache  can be disabled with by setting the FeaturePCD
gEfiMdeModulePkgTokenSpaceGuid.PcdEnableVariableRuntimeCache
to FALSE. If the PCD is set to FALSE, the runtime cache will not be
used and an SMI will be triggered for Runtime Service
GetVariable () and GetNextVariableName () invocations.

The following are important points regarding the behavior of the
variable drivers when the variable runtime cache is enabled.

1. All of the non-volatile storage contents are loaded into the
   cache upon driver load. This one time load operation from storage
   is preferred as opposed to building the cache on demand. An on-
   demand cache would require a fallback SMI to load data into the
   cache as variables are requested.

2. SetVariable () requests will continue to always trigger an SMI.
   This occurs regardless of whether the variable is volatile or
   non-volatile.

3. Both volatile and non-volatile variables are cached in a runtime
   buffer. As is the case in the current EDK II variable driver, they
   continue to be cached in separate buffers.

4. The cache in Runtime DXE and SMM are intended to be exact copies
   of one another. All SMM variable accesses only return data from the
   SMM cache. The runtime caches are only updated after the variable I/O
   operation is successful in SMM. The runtime caches are only updated
   from SMM.

5. Synchronization mechanisms are in place to ensure the runtime cache
   content integrity with the SMM cache. These may result in updates to
   runtime cache that are the same in content but different in offset and
   size from updates to the SMM cache.

When using SMM variables with runtime cache enabled, two caches will now
be present.
1. "Runtime Cache" - Maintained in VariableSmmRuntimeDxe. Used to service
   Runtime Services GetVariable () and GetNextVariableName () callers.
2. "SMM Cache" - Maintained in VariableSmm to service SMM GetVariable ()
   and GetNextVariableName () callers.
   a. This cache is retained so SMM modules do not operate on data outside
      SMRAM.

Because a race condition can occur if an SMI occurs during the execution
of runtime code reading from the runtime cache, a runtime cache read lock
is introduced that explicitly moves pending updates from SMM to the runtime
cache if an SMM update occurs while the runtime cache is locked. Note that
it is not expected a Runtime services call will interrupt SMM processing
since all CPU cores rendezvous in SMM.

It is possible to view UEFI variable read and write statistics by setting
the gEfiMdeModulePkgTokenSpaceGuid.PcdVariableCollectStatistics FeaturePcd
to TRUE and using the VariableInfo UEFI application in MdeModulePkg to dump
variable statistics to the console. By doing so, a user can view the number
of GetVariable () hits from the Runtime DXE variable driver (Runtime Cache
hits) and the SMM variable driver (SMM Cache hits). SMM Cache hits for
GetVariable () will occur when SMM modules invoke GetVariable ().

Cc: Dandan Bi <dandan.bi@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Signed-off-by: Michael Kubacki <michael.a.kubacki@intel.com>
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>

1 files changed, 172 insertions, 17 deletions

diff --git a/MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.c b/MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.c
index 5e24bc4a62..caca5c3241 100644
--- a/MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.c
+++ b/MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmm.c
@@ -31,6 +31,9 @@ SPDX-License-Identifier: BSD-2-Clause-Patent
 #include <Guid/SmmVariableCommon.h>

 #include "Variable.h"

 #include "VariableParsing.h"

+#include "VariableRuntimeCache.h"

+

+extern VARIABLE_STORE_HEADER                         *mNvVariableCache;

 

 BOOLEAN                                              mAtRuntime              = FALSE;

 UINT8                                                *mVariableBufferPayload = NULL;

@@ -451,25 +454,29 @@ SmmVariableGetStatistics (
 EFI_STATUS

 EFIAPI

 SmmVariableHandler (

-  IN     EFI_HANDLE                                DispatchHandle,

-  IN     CONST VOID                                *RegisterContext,

-  IN OUT VOID                                      *CommBuffer,

-  IN OUT UINTN                                     *CommBufferSize

+  IN     EFI_HANDLE                                       DispatchHandle,

+  IN     CONST VOID                                       *RegisterContext,

+  IN OUT VOID                                             *CommBuffer,

+  IN OUT UINTN                                            *CommBufferSize

   )

 {

-  EFI_STATUS                                       Status;

-  SMM_VARIABLE_COMMUNICATE_HEADER                  *SmmVariableFunctionHeader;

-  SMM_VARIABLE_COMMUNICATE_ACCESS_VARIABLE         *SmmVariableHeader;

-  SMM_VARIABLE_COMMUNICATE_GET_NEXT_VARIABLE_NAME  *GetNextVariableName;

-  SMM_VARIABLE_COMMUNICATE_QUERY_VARIABLE_INFO     *QueryVariableInfo;

-  SMM_VARIABLE_COMMUNICATE_GET_PAYLOAD_SIZE        *GetPayloadSize;

-  VARIABLE_INFO_ENTRY                              *VariableInfo;

-  SMM_VARIABLE_COMMUNICATE_LOCK_VARIABLE           *VariableToLock;

-  SMM_VARIABLE_COMMUNICATE_VAR_CHECK_VARIABLE_PROPERTY *CommVariableProperty;

-  UINTN                                            InfoSize;

-  UINTN                                            NameBufferSize;

-  UINTN                                            CommBufferPayloadSize;

-  UINTN                                            TempCommBufferSize;

+  EFI_STATUS                                              Status;

+  SMM_VARIABLE_COMMUNICATE_HEADER                         *SmmVariableFunctionHeader;

+  SMM_VARIABLE_COMMUNICATE_ACCESS_VARIABLE                *SmmVariableHeader;

+  SMM_VARIABLE_COMMUNICATE_GET_NEXT_VARIABLE_NAME         *GetNextVariableName;

+  SMM_VARIABLE_COMMUNICATE_QUERY_VARIABLE_INFO            *QueryVariableInfo;

+  SMM_VARIABLE_COMMUNICATE_GET_PAYLOAD_SIZE               *GetPayloadSize;

+  SMM_VARIABLE_COMMUNICATE_RUNTIME_VARIABLE_CACHE_CONTEXT *RuntimeVariableCacheContext;

+  SMM_VARIABLE_COMMUNICATE_GET_RUNTIME_CACHE_INFO         *GetRuntimeCacheInfo;

+  SMM_VARIABLE_COMMUNICATE_LOCK_VARIABLE                  *VariableToLock;

+  SMM_VARIABLE_COMMUNICATE_VAR_CHECK_VARIABLE_PROPERTY    *CommVariableProperty;

+  VARIABLE_INFO_ENTRY                                     *VariableInfo;

+  VARIABLE_RUNTIME_CACHE_CONTEXT                          *VariableCacheContext;

+  VARIABLE_STORE_HEADER                                   *VariableCache;

+  UINTN                                                   InfoSize;

+  UINTN                                                   NameBufferSize;

+  UINTN                                                   CommBufferPayloadSize;

+  UINTN                                                   TempCommBufferSize;

 

   //

   // If input is invalid, stop processing this SMI

@@ -789,6 +796,154 @@ SmmVariableHandler (
                  );

       CopyMem (SmmVariableFunctionHeader->Data, mVariableBufferPayload, CommBufferPayloadSize);

       break;

+    case SMM_VARIABLE_FUNCTION_INIT_RUNTIME_VARIABLE_CACHE_CONTEXT:

+      if (CommBufferPayloadSize < sizeof (SMM_VARIABLE_COMMUNICATE_RUNTIME_VARIABLE_CACHE_CONTEXT)) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: SMM communication buffer size invalid!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+      if (mEndOfDxe) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Cannot init context after end of DXE!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+

+      //

+      // Copy the input communicate buffer payload to the pre-allocated SMM variable payload buffer.

+      //

+      CopyMem (mVariableBufferPayload, SmmVariableFunctionHeader->Data, CommBufferPayloadSize);

+      RuntimeVariableCacheContext = (SMM_VARIABLE_COMMUNICATE_RUNTIME_VARIABLE_CACHE_CONTEXT *) mVariableBufferPayload;

+

+      //

+      // Verify required runtime cache buffers are provided.

+      //

+      if (RuntimeVariableCacheContext->RuntimeVolatileCache == NULL ||

+          RuntimeVariableCacheContext->RuntimeNvCache == NULL ||

+          RuntimeVariableCacheContext->PendingUpdate == NULL ||

+          RuntimeVariableCacheContext->ReadLock == NULL ||

+          RuntimeVariableCacheContext->HobFlushComplete == NULL) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Required runtime cache buffer is NULL!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+

+      //

+      // Verify minimum size requirements for the runtime variable store buffers.

+      //

+      if ((RuntimeVariableCacheContext->RuntimeHobCache != NULL &&

+          RuntimeVariableCacheContext->RuntimeHobCache->Size < sizeof (VARIABLE_STORE_HEADER)) ||

+          RuntimeVariableCacheContext->RuntimeVolatileCache->Size < sizeof (VARIABLE_STORE_HEADER) ||

+          RuntimeVariableCacheContext->RuntimeNvCache->Size < sizeof (VARIABLE_STORE_HEADER)) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: A runtime cache buffer size is invalid!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+

+      //

+      // Verify runtime buffers do not overlap with SMRAM ranges.

+      //

+      if (RuntimeVariableCacheContext->RuntimeHobCache != NULL &&

+          !VariableSmmIsBufferOutsideSmmValid (

+            (UINTN) RuntimeVariableCacheContext->RuntimeHobCache,

+            (UINTN) RuntimeVariableCacheContext->RuntimeHobCache->Size)) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Runtime HOB cache buffer in SMRAM or overflow!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+      if (!VariableSmmIsBufferOutsideSmmValid (

+            (UINTN) RuntimeVariableCacheContext->RuntimeVolatileCache,

+            (UINTN) RuntimeVariableCacheContext->RuntimeVolatileCache->Size)) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Runtime volatile cache buffer in SMRAM or overflow!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+      if (!VariableSmmIsBufferOutsideSmmValid (

+            (UINTN) RuntimeVariableCacheContext->RuntimeNvCache,

+            (UINTN) RuntimeVariableCacheContext->RuntimeNvCache->Size)) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Runtime non-volatile cache buffer in SMRAM or overflow!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+      if (!VariableSmmIsBufferOutsideSmmValid (

+            (UINTN) RuntimeVariableCacheContext->PendingUpdate,

+            sizeof (*(RuntimeVariableCacheContext->PendingUpdate)))) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Runtime cache pending update buffer in SMRAM or overflow!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+      if (!VariableSmmIsBufferOutsideSmmValid (

+            (UINTN) RuntimeVariableCacheContext->ReadLock,

+            sizeof (*(RuntimeVariableCacheContext->ReadLock)))) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Runtime cache read lock buffer in SMRAM or overflow!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+      if (!VariableSmmIsBufferOutsideSmmValid (

+            (UINTN) RuntimeVariableCacheContext->HobFlushComplete,

+            sizeof (*(RuntimeVariableCacheContext->HobFlushComplete)))) {

+        DEBUG ((DEBUG_ERROR, "InitRuntimeVariableCacheContext: Runtime cache HOB flush complete buffer in SMRAM or overflow!\n"));

+        Status = EFI_ACCESS_DENIED;

+        goto EXIT;

+      }

+

+      VariableCacheContext = &mVariableModuleGlobal->VariableGlobal.VariableRuntimeCacheContext;

+      VariableCacheContext->VariableRuntimeHobCache.Store      = RuntimeVariableCacheContext->RuntimeHobCache;

+      VariableCacheContext->VariableRuntimeVolatileCache.Store = RuntimeVariableCacheContext->RuntimeVolatileCache;

+      VariableCacheContext->VariableRuntimeNvCache.Store       = RuntimeVariableCacheContext->RuntimeNvCache;

+      VariableCacheContext->PendingUpdate                      = RuntimeVariableCacheContext->PendingUpdate;

+      VariableCacheContext->ReadLock                           = RuntimeVariableCacheContext->ReadLock;

+      VariableCacheContext->HobFlushComplete                   = RuntimeVariableCacheContext->HobFlushComplete;

+

+      // Set up the intial pending request since the RT cache needs to be in sync with SMM cache

+      VariableCacheContext->VariableRuntimeHobCache.PendingUpdateOffset = 0;

+      VariableCacheContext->VariableRuntimeHobCache.PendingUpdateLength = 0;

+      if (mVariableModuleGlobal->VariableGlobal.HobVariableBase > 0 &&

+          VariableCacheContext->VariableRuntimeHobCache.Store != NULL) {

+        VariableCache = (VARIABLE_STORE_HEADER *) (UINTN) mVariableModuleGlobal->VariableGlobal.HobVariableBase;

+        VariableCacheContext->VariableRuntimeHobCache.PendingUpdateLength = (UINT32) ((UINTN) GetEndPointer (VariableCache) - (UINTN) VariableCache);

+        CopyGuid (&(VariableCacheContext->VariableRuntimeHobCache.Store->Signature), &(VariableCache->Signature));

+      }

+      VariableCache = (VARIABLE_STORE_HEADER  *) (UINTN) mVariableModuleGlobal->VariableGlobal.VolatileVariableBase;

+      VariableCacheContext->VariableRuntimeVolatileCache.PendingUpdateOffset   = 0;

+      VariableCacheContext->VariableRuntimeVolatileCache.PendingUpdateLength   = (UINT32) ((UINTN) GetEndPointer (VariableCache) - (UINTN) VariableCache);

+      CopyGuid (&(VariableCacheContext->VariableRuntimeVolatileCache.Store->Signature), &(VariableCache->Signature));

+

+      VariableCache = (VARIABLE_STORE_HEADER  *) (UINTN) mNvVariableCache;

+      VariableCacheContext->VariableRuntimeNvCache.PendingUpdateOffset = 0;

+      VariableCacheContext->VariableRuntimeNvCache.PendingUpdateLength = (UINT32) ((UINTN) GetEndPointer (VariableCache) - (UINTN) VariableCache);

+      CopyGuid (&(VariableCacheContext->VariableRuntimeNvCache.Store->Signature), &(VariableCache->Signature));

+

+      *(VariableCacheContext->PendingUpdate) = TRUE;

+      *(VariableCacheContext->ReadLock) = FALSE;

+      *(VariableCacheContext->HobFlushComplete) = FALSE;

+

+      Status = EFI_SUCCESS;

+      break;

+    case SMM_VARIABLE_FUNCTION_SYNC_RUNTIME_CACHE:

+      Status = FlushPendingRuntimeVariableCacheUpdates ();

+      break;

+    case SMM_VARIABLE_FUNCTION_GET_RUNTIME_CACHE_INFO:

+      if (CommBufferPayloadSize < sizeof (SMM_VARIABLE_COMMUNICATE_GET_RUNTIME_CACHE_INFO)) {

+        DEBUG ((DEBUG_ERROR, "GetRuntimeCacheInfo: SMM communication buffer size invalid!\n"));

+        return EFI_SUCCESS;

+      }

+      GetRuntimeCacheInfo = (SMM_VARIABLE_COMMUNICATE_GET_RUNTIME_CACHE_INFO *) SmmVariableFunctionHeader->Data;

+

+      if (mVariableModuleGlobal->VariableGlobal.HobVariableBase > 0) {

+        VariableCache = (VARIABLE_STORE_HEADER *) (UINTN) mVariableModuleGlobal->VariableGlobal.HobVariableBase;

+        GetRuntimeCacheInfo->TotalHobStorageSize = VariableCache->Size;

+      } else {

+        GetRuntimeCacheInfo->TotalHobStorageSize = 0;

+      }

+

+      VariableCache = (VARIABLE_STORE_HEADER  *) (UINTN) mVariableModuleGlobal->VariableGlobal.VolatileVariableBase;

+      GetRuntimeCacheInfo->TotalVolatileStorageSize = VariableCache->Size;

+      VariableCache = (VARIABLE_STORE_HEADER  *) (UINTN) mNvVariableCache;

+      GetRuntimeCacheInfo->TotalNvStorageSize = (UINTN) VariableCache->Size;

+      GetRuntimeCacheInfo->AuthenticatedVariableUsage = mVariableModuleGlobal->VariableGlobal.AuthFormat;

+

+      Status = EFI_SUCCESS;

+      break;

 

     default:

       Status = EFI_UNSUPPORTED;