summaryrefslogtreecommitdiffstats
path: root/BaseTools/Source/Python/AutoGen/AutoGenWorker.py
Commit message (Collapse)AuthorAgeFilesLines
* BaseTools: Move CreateAsBuiltInf into AutoGenWorker for parallelBob Feng2020-08-141-1/+1
| | | | | | | | | | | AsBuiltInf can be created during AutoGen phase. Move CreateAsBuiltInf into AutoGenWorker to make this step run in parallel. Signed-off-by: Bob Feng <bob.c.feng@intel.com> Cc: Liming Gao <liming.gao@intel.com> Cc: Yuwei Chen <yuwei.chen@intel.com> Reviewed-by: Jaben Carsey <jaben.carsey@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: Improve the method of checking queue emptyFeng, Bob C2020-08-071-6/+20
| | | | | | | | | | | | | | | | | | Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2807 The Queue.empty() method is not reliable in the multiple process runtime environment. This patch uses a new method to check if all modules are processed and workers need to be stopped. That is to add a None item at the bottom of the queue. Worker check if it gets that None item to know if all the module is processed. Signed-off-by: Bob Feng <bob.c.feng@intel.com> Cc: Liming Gao <liming.gao@intel.com> Cc: Yuwei Chen <yuwei.chen@intel.com> Cc: Lucy Yan <lucyyan@google.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools:Fix build tools print traceback info issueFan, ZhijuX2020-03-241-3/+0
| | | | | | | | | | | | | | | | REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2610 We meet a case that the DEC file declaring the PCD isn't included in the INF.it cause build tools report Traceback error. Remove raise statements that generate Tracebacks that were only intended for development/debug. With the raise statements removed proper error messages are shown. Cc: Bob Feng <bob.c.feng@intel.com> Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: Leverage compiler output to optimize binary cacheSteven Shi2019-12-101-22/+55
| | | | | | | | | | | | | | | | | Redesign the binary cache and bases on the compiler to output the dependency header files info for every module. The binary cache will directly consume the dependency header files info and doesn't parse the C source code by iteself. Also redesign the dependency files list format for module and try to share the common lib hash result as more as possible in local process. Remove the unnecessary share data access across multiprocessing. Signed-off-by: Steven Shi <steven.shi@intel.com> Cc: Liming Gao <liming.gao@intel.com> Cc: Bob Feng <bob.c.feng@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: remove unnecessary calls of os.existFeng, Bob C2019-09-171-3/+4
| | | | | | | | | | | REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2101 This patch is going to remove unnecessary calls of os.exist() Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Fixed the bug of multi-thread genffs for override infFeng, Bob C2019-09-111-1/+1
| | | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1302 If there is a inf override, and multi-thread genffs is enabled, build will fail. This patch is going to fix this bug. Cc: Liming Gao <liming.gao@intel.com> Cc: Steven Shi <steven.shi@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Reviewed-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Fixed issue for IgnoreAutoGenFeng, Bob C2019-09-101-1/+1
| | | | | | | | | | | | | https://bugzilla.tianocore.org/show_bug.cgi?id=2080 This patch is to improve build -u option to re-use GlobalVar_<platformguid>_<arch>.bin file which is introduced by multiple-process-autogen feature. Cc: Liming Gao <liming.gao@intel.com> Cc: Steven Shi <steven.shi@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Acked-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Improve the file saving and copying reliabilityShi, Steven2019-08-201-2/+4
| | | | | | | | | | | | | | | | | BZ:https://bugzilla.tianocore.org/show_bug.cgi?id=2079 The Basetool CopyFileOnChange() and SaveFileOnChange() functions might raise the IOError occasionally when build in Windows with multi-process and build cache enabled. The CopyFileOnChange() and SaveFileOnChange() might be invoked in multiple sub-processes simultaneously, and this patch adds global locks to sync these functions invoking which can harden their reliability. Cc: Liming Gao <liming.gao@intel.com> Cc: Bob Feng <bob.c.feng@intel.com> Signed-off-by: Steven Shi <steven.shi@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: Print first cache missing file for build cachleShi, Steven2019-08-201-0/+2
| | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1925 When a module build cache miss, add support to print the first cache missing file path and name. Cc: Liming Gao <liming.gao@intel.com> Cc: Bob Feng <bob.c.feng@intel.com> Signed-off-by: Steven Shi <steven.shi@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: Improve the cache hit in the edk2 build cacheShi, Steven2019-08-201-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1927 Current cache hash algorithm does not parse and generate the makefile to get the accurate dependency files for a module. It instead use the platform and package meta files to get the module depenedency in a quick but over approximate way. These meta files are monolithic and involve many redundant dependency for the module, which cause the module build cache miss easily. This patch introduces one more cache checkpoint and a new hash algorithm besides the current quick one. The new hash algorithm leverages the module makefile to achieve more accurate and precise dependency info for a module. When the build cache miss with the first quick hash, the Basetool will caculate new one after makefile is generated and then check again. Cc: Liming Gao <liming.gao@intel.com> Cc: Bob Feng <bob.c.feng@intel.com> Signed-off-by: Steven Shi <steven.shi@intel.com> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
* BaseTools: Enable block queue log agent.Feng, Bob C2019-08-091-2/+4
| | | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875 To support Ctrl+S and Ctrl+Q, we enable block queue for log. Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Acked-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Acked-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Add the support for python 2Feng, Bob C2019-08-091-1/+4
| | | | | | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875 python3 change the module name of Queue to queue. python3 add a new log handler of QueueHandler. This patch is to make Multiple process AutoGen feature work for python2 Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Acked-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Acked-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Add LogAgent to support multiple process AutogenFeng, Bob C2019-08-091-6/+68
| | | | | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875 AutoGen processes race the logfile. To resolve this issue, this patch create a LogAgent thread in main process to write the log content to console or file, Other process will send the log content to the LogAgent. Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Acked-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Acked-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Add shared data for processesFeng, Bob C2019-08-091-1/+2
| | | | | | | | | | | | BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875 Add shared data for autogen processes. Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Acked-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Acked-by: Liming Gao <liming.gao@intel.com>
* BaseTools: Enable Multiple Process AutoGenFeng, Bob C2019-08-091-0/+191
BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=1875 Assign the Module AutoGen tasks into multiple sub process. Cc: Liming Gao <liming.gao@intel.com> Signed-off-by: Bob Feng <bob.c.feng@intel.com> Acked-by: Laszlo Ersek <lersek@redhat.com> Tested-by: Laszlo Ersek <lersek@redhat.com> Acked-by: Liming Gao <liming.gao@intel.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-14 03:38:02 +0000