ShellPkg: UefiShellDebug1CommandsLib: fix hex string parsing in SETVAR

The ShellCommandRunSetVar() function calls the ShellIsHexOrDecimalNumber()
UefiShellLib function to determine if the data string in the SETVAR
command is a string of hexadecimal bytes.

However, ShellIsHexOrDecimalNumber() calls ShellConvertStringToUint64()
for validation. Therefore SETVAR rejects hex strings that cannot be
interpreted as UINT64 values due to range errors, despite the fact that
the hexadecimal data string of the SETVAR command is supposed to describe
an arbitrary array of bytes, rather than UINT64 values.

The internal library function InternalShellIsHexOrDecimalNumber() comes
close, as the first idea for the fix, however it is not quite right
either; it removes a leading minus sign, plus it allows 0x / 0X prefixes.
This would not be correct for the SETVAR command.

Instead, add a trivial utility function that is specific to the SETVAR
implementation. IsStringOfHexNibbles() accepts empty strings for
simplicity, but where we call it we have already ensured that the data
string is not empty (because that is handled earlier by the "delete
variable" branch of SETVAR). An even number of nibbles is also enforced
near the call site.

Cc: Jaben Carsey <jaben.carsey@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17128 6f19259b-4bc3-4df7-8a09-765794883524

0 files changed, 0 insertions, 0 deletions