summaryrefslogtreecommitdiffstats
path: root/BaseTools/Source/Python/AutoGen/UniClassObject.py
Commit message (Collapse)AuthorAgeFilesLines
* BaseTools: fix ucs-2 lookup on python 3.9Cole Robinson2020-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | python3.9 changed/fixed codec.register behavior to always replace hyphen with underscore for passed in codec names: https://bugs.python.org/issue37751 So the custom Ucs2Search needs to be adapted to handle 'ucs_2' in addition to existing 'ucs-2' for back compat. This fixes test failures on python3.9, example: ====================================================================== FAIL: testUtf16InUniFile (CheckUnicodeSourceFiles.Tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 375, in PreProcess FileIn = UniFileClassObject.OpenUniFile(LongFilePath(File.Path)) File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 303, in OpenUniFile UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding) File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Python/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data Ucs2Info = codecs.lookup('ucs-2') LookupError: unknown encoding: ucs-2 Signed-off-by: Cole Robinson <crobinso@redhat.com> Reviewed-by: Yuwei Chen <yuwei.chen@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: remove unnecessary calls of os.existFeng, Bob C2019-09-171-3/+2
| | | | | | | | | | | REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2101 This patch is going to remove unnecessary calls of os.exist() Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Replace BSD License with BSD+Patent LicenseMichael D Kinney2019-04-091-7/+1
| | | | | | | | | | | | | | | | | | | | https://bugzilla.tianocore.org/show_bug.cgi?id=1373 Replace BSD 2-Clause License with BSD+Patent License. This change is based on the following emails: https://lists.01.org/pipermail/edk2-devel/2019-February/036260.html https://lists.01.org/pipermail/edk2-devel/2018-October/030385.html RFCs with detailed process for the license change: V3: https://lists.01.org/pipermail/edk2-devel/2019-March/038116.html V2: https://lists.01.org/pipermail/edk2-devel/2019-March/037669.html V1: https://lists.01.org/pipermail/edk2-devel/2019-March/037500.html Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: Various typoAntoine Coeur2019-02-141-2/+2
| | | | | | | | Various typo in BaseTools. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Coeur <coeur@gmx.fr> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Handle the bytes and str differenceFeng, Bob C2019-02-011-2/+2
| | | | | | | | | | | | | | | | Deal with bytes and str is different, remove the unicode(), correct open file parameter. Using utcfromtimestamp instead of fromtimestamp. Cc: Bob Feng <bob.c.feng@intel.com> Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Reviewed-by: Liming Gao <liming.gao@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: cleanup LongFilePathSupport usageJaben Carsey2018-12-061-13/+1
| | | | | | | | | | | 1) remove an identical function and import it from Common.LongFilePathSupport 2) remove an import that is not needed/used. Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by : Bob Feng <bob.c.feng@intel.com>
* Revert BaseTools: PYTHON3 migrationLiming Gao2018-10-151-1/+14
| | | | | | | | | | | | | | This reverts commit 6693f359b3c213513c5096a06c6f67244a44dc52.. 678f85131238622e576705117e299d81cff755c9. Python3 migration is the fundamental change. It requires every developer to install Python3. Before this migration, the well communication and wide verification must be done. But now, most people is not aware of this change, and not try it. So, Python3 migration is reverted and be moved to edk2-staging Python3 branch for the edk2 user evaluation. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Handle the bytes and str differenceYunhua Feng2018-10-131-13/+1
| | | | | | | | | | | Deal with bytes and str is different, remove the unicode() Using utcfromtimestamp instead of fromtimestamp. Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Remove the "from __future__ import" itemsYunhua Feng2018-10-131-1/+0
| | | | | | | | Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Yunhua Feng <yunhuax.feng@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Clean up source filesLiming Gao2018-07-091-9/+9
| | | | | | | | | | | 1. Do not use tab characters 2. No trailing white space in one line 3. All files must end with CRLF Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Replace StringIO.StringIO with io.BytesIOGary Lin2018-06-271-3/+3
| | | | | | | | | | | | Replace StringIO.StringIO with io.BytesIO to be compatible with python3. This commit also removes "import StringIO" from those python scripts that don't really use it. Contributed-under: TianoCore Contribution Agreement 1.1 Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Gary Lin <glin@suse.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Refactor python print statementsGary Lin2018-06-271-3/+4
| | | | | | | | | | | Refactor print statements to be compatible with python 3. Based on "futurize -f libfuturize.fixes.fix_print_with_import" Contributed-under: TianoCore Contribution Agreement 1.1 Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Gary Lin <glin@suse.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Refactor python except statementsGary Lin2018-06-271-2/+2
| | | | | | | | | | | Convert "except ... ," to "except ... as" to be compatible with python3. Based on "futurize -f lib2to3.fixes.fix_except" Contributed-under: TianoCore Contribution Agreement 1.1 Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Gary Lin <glin@suse.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Rename String to StringUtils.Marvin.Haeuser@outlook.com2018-05-281-1/+1
| | | | | | | | | | | | For case-insensitive file systems, edk2 String.py collides with the Python string.py, which results in build errors. This,for example, applies to building via the Windows Subsystem for Linux from a DriveFS file system. This patch renames String to StringUtils to prevent conflicts for case-insensitive file systems. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Marvin Haeuser <Marvin.Haeuser@outlook.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: AutoGen - UniClassObject refactor static methodsCarsey, Jaben2018-05-041-5/+7
| | | | | | | | | | | change methods which do not use self to @staticmethod change their calls to use class name instead of instance Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: AutoGen - remove global lineCarsey, Jaben2018-05-041-2/+0
| | | | | | | | this serves no purpose since we dont change the global or assign to it. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: AutoGen - share StripComments APICarsey, Jaben2018-05-041-15/+14
| | | | | | | | | | | | add the API root in one class file. delete the static API out of both classes. share it in the single location. Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: remove redundant checkCarsey, Jaben2018-04-081-2/+2
| | | | | | | | | | The RegEx matches begining and end of the string, dont then check length. Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: AutoGen - use the new shared RegExCarsey, Jaben2018-04-081-2/+2
| | | | | | | | Cc: Liming Gao <liming.gao@intel.com> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Remove equality operator with NoneCarsey, Jaben2018-03-301-11/+11
| | | | | | | | | | replace "== None" with "is None" and "!= None" with "is not None" Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Use precompiled RegExpCarsey, Jaben2018-03-291-3/+4
| | | | | | | | | | avoid recompiling the regular expression for each use in a while loop Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
* BaseTools: Allow string token identifier to use lower case lettersYonghong Zhu2016-08-081-6/+6
| | | | | | | | | | | This patch is to align the code behavior with UNI spec that string token identifier can use upper case and lower case letters. Cc: Liming Gao <liming.gao@intel.com> Cc: Felix <Felixp@ami.com> Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: fix a bug for uni file \x####\ format handlingYonghong Zhu2016-05-111-1/+1
| | | | | | | | | | It should start from the last '\x' position + 1 to find next '\x' character. Cc: Liming Gao <liming.gao@intel.com> Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Support \x####\ in UNI files to specify non-ascii charactersYonghong Zhu2016-05-061-4/+14
| | | | | | | | | | UNI spec updated to allow using \x####\ to specify non-ascii characters, # is a hex digit. Cc: Liming Gao <liming.gao@intel.com> Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Clean some coding style issuesYonghong Zhu2015-12-011-4/+4
| | | | | | | | | | This patch clean some coding style issues, majorly for space character. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@19080 6f19259b-4bc3-4df7-8a09-765794883524
* BaseTools/UniClassObject: Support UTF-8 string data in .uni filesJordan Justen2015-06-231-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows .uni input files to be encoded with UTF-8. Today, we only support UTF-16 encoding. The strings are still converted to UCS-2 data for use in EDK II modules. (This is the only unicode character format supported by UEFI and EDK II.) Although UTF-8 would allow any UCS-4 character to be present in the source file, we restrict the entire file to the UCS-2 range. (Including comments.) This allows the files to be converted to UTF-16 if needed. v2: * Drop .utf8 extension. Use .uni file for UTF-8 data (mdkinney) * Merge in 'BaseTools/UniClassObject: Verify string data is 16-bit' commit v3: * Restrict the entire file's characters (including comments) to the UCS-2 range in addition to string data. (mdkinney) Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17696 6f19259b-4bc3-4df7-8a09-765794883524
* BaseTools/UniClassObject: Verify valid UCS-2 chars in UTF-16 .uni filesJordan Justen2015-06-231-3/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Supplementary Plane characters can exist in UTF-16 files, but they are not valid UCS-2 characters. For example, refer to this python interpreter code: >>> import codecs >>> codecs.encode(u'\U00010300', 'utf-16') '\xff\xfe\x00\xd8\x00\xdf' Therefore the UCS-4 0x00010300 character is encoded as two 16-bit numbers (0xd800 0xdf00) in a little endian UTF-16 file. For more information, see: http://en.wikipedia.org/wiki/UTF-16#U.2B10000_to_U.2B10FFFF This means that our current BaseTools code could be allowing unsupported UTF-16 characters be used. To fix this, we decode the file using python's utf-16 decode support. Then we verify that each character's code point is 0xffff or less. v3: * Based on Mike Kinney's feedback, we now read the whole file and verify up-front that it contains valid UCS-2 characters. Thanks also to Laszlo Ersek for pointing out the Supplementary Plane characters. v4: * Reject code points in 0xd800-0xdfff range since they are reserved for UTF-16 surrogate pairs. (lersek) Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Laszlo Ersek <lersek@redhat.com> Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17694 6f19259b-4bc3-4df7-8a09-765794883524
* BaseTools: Fix a bug that UNI file can't have comment after #include "file.uni"Yingke Liu2015-06-231-2/+2
| | | | | | | | | | The 'include' regular expression cannot match spaces before or after this statement. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yingke Liu <yingke.d.liu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@17680 6f19259b-4bc3-4df7-8a09-765794883524
* Fix a regression bug to uni parser.Yingke Liu2014-12-031-0/+1
| | | | | | | | Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yingke Liu <yingke.d.liu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@16469 6f19259b-4bc3-4df7-8a09-765794883524
* Corrected slash and quote handling in the strings of UNI files.Cecil Sheng2014-12-011-38/+30
| | | | | | | | Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Cecil Sheng <cecil.sheng@hp.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@16456 6f19259b-4bc3-4df7-8a09-765794883524
* License header updated to match correct format.Yingke Liu2014-08-281-4/+3
| | | | | | | | Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Yingke Liu <yingke.d.liu@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@15971 6f19259b-4bc3-4df7-8a09-765794883524
* There is a limitation on WINDOWS OS for the length of entire file path ↵Hess Chen2014-08-151-5/+5
| | | | | | | | | | can’t be larger than 255. There is an OS API provided by Microsoft to add “\\?\” before the path header to support the long file path. Enable this feature on basetools. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Hess Chen <hesheng.chen@intel.com> Reviewed-by: Yingke Liu <yingke.d.liu@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@15809 6f19259b-4bc3-4df7-8a09-765794883524
* Sync BaseTool trunk (version r2599) into EDKII BaseTools.Liming Gao2013-08-231-3/+9
| | | | | | | | Signed-off-by: Liming Gao <liming.gao@intel.com> Reviewed-by: Heshen Chen <chen.heshen@intel.com> git-svn-id: https://svn.code.sf.net/p/edk2/code/trunk/edk2@14591 6f19259b-4bc3-4df7-8a09-765794883524
* Sync BaseTool trunk (version r2423) into EDKII BaseTools. The change mainly ↵lgao42011-11-251-1/+1
| | | | | | | | | | | | | | | includes: 1. Fix !include issues 2. Fix Trim to skip the postfix 'U' for hexadecimal and decimal numbers 3. Fix building error C2733 when building C++ code. 4. Add GCC46 tool chain definition 5. Add new RVCT and RVCTLINUX tool chains Signed-off-by: lgao4 git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@12782 6f19259b-4bc3-4df7-8a09-765794883524
* Sync BaseTools Branch (version r2362) to EDKII main trunk. lgao42011-10-111-16/+28
| | | | | | | | Signed-off-by: lgao4 Reviewed-by: jsu1 Reviewed-by: ydliu git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@12525 6f19259b-4bc3-4df7-8a09-765794883524
* Sync BaseTools Branch (version r2149) to EDKII main trunk. lgao42011-05-111-2/+25
| | | | | | | | | | | BaseTool Branch: https://edk2-buildtools.svn.sourceforge.net/svnroot/edk2-buildtools/branches/Releases/BaseTools_r2100 git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@11640 6f19259b-4bc3-4df7-8a09-765794883524
* Sync EDKII BaseTools to BaseTools project r2065.lgao42010-10-111-0/+7
| | | | git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@10915 6f19259b-4bc3-4df7-8a09-765794883524
* Sync EDKII BaseTools to BaseTools project r2006.lgao42010-08-031-60/+80
| | | | git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@10764 6f19259b-4bc3-4df7-8a09-765794883524
* Sync EDKII BaseTools to BaseTools project r1971lgao42010-05-181-6/+9
| | | | git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@10502 6f19259b-4bc3-4df7-8a09-765794883524
* Sync EDKII BaseTools to BaseTools project r1937.lgao42010-03-191-2/+2
| | | | git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@10287 6f19259b-4bc3-4df7-8a09-765794883524
* Sync EDKII BaseTools to BaseTools project r1903.lgao42010-02-281-12/+13
| | | | git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@10123 6f19259b-4bc3-4df7-8a09-765794883524
* Sync tool code to BuildTools project r1739.lgao42009-11-091-0/+10
| | | | git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@9397 6f19259b-4bc3-4df7-8a09-765794883524
* Check In tool source code based on Build tool project revision r1655.lgao42009-07-171-0/+530
git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@8964 6f19259b-4bc3-4df7-8a09-765794883524
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-15 03:22:53 +0000