summaryrefslogtreecommitdiffstats
path: root/Documentation/technotes/2021-05-code-coverage.md
blob: d33fe5d700af9668b5962056aecf01be8bd16144 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# Unit Test Code Coverage

Code coverage for the coreboot unit tests allows us to see what lines of
code in the coreboot library are covered by unit tests, and allows a test
author to see where they need to add test cases for additional coverage.

Code coverage requires `lcov`; install the tool if necessary by
`sudo apt install lcov` or the equivalent for your system.

Enable code coverage in your unit test build by setting the environment
variable `COV` to 1; either `export COV=1` in your shell, or add it to your
`make` command, e.g. `COV=1 make unit-tests`.

The build output directory is either `build/tests` or `build/coverage`,
depending on whether `COV=1` is set in the environment.

All of the unit test targets are available with and without `COV=1`
* `clean-unit-tests`
* `build-unit-tests`
* `run-unit-tests`
* `unit-tests` (which is just `build-unit-tests` followed by `run-unit-tests`)

There are two new `make` targets:
* `coverage-report` generates a code coverage report from all of the
GCOV data (`*.gcda` and `*.gcno` files) in the build directory. To view the
coverage report, open `build/coverage/coverage_reports/index.html` in your web
browser.
* `clean-coverage-report` deletes just the coverage report.

The `coverage-report` and `clean-coverage-report` targets automatically set
`COV=1` if it is not already set in the environment.


## Examples

`COV=1 make unit-tests coverage-report` builds all of the unit tests with code
coverage, runs the unit tests, and generates the code coverage report.

`COV=1 make build-unit-tests` builds all of the unit tests with code coverage.

`COV=1 make run-unit-tests` runs the unit tests, building them with code
coverage if they are out-of-date.

`COV=1 make coverage-report` creates the code coverage report. This
target does not explicitly depend on the tests being built and run; it gathers
the code coverage data from the output directory, which it assumes already
exists.

`COV=1 make tests/lib/uuid-test coverage-report` builds the uuid test
with code coverage, runs it, and generates a code coverage report just for
that test.

As a demonstration that building with and without coverage uses different
output directories:
1. `make build-unit-tests` builds unit tests without code coverage into
`build/tests`.
2. `COV=1 make clean-unit-tests` cleans `build/coverage`
3. `make build-unit-tests` doesn't need to build anything in `build/tests`,
because those files weren't affected by the previous `clean-unit-tests`.