summaryrefslogtreecommitdiffstats
path: root/tools/testing/selftests/bpf/test_progs.c
Commit message (Collapse)AuthorAgeFilesLines
* selftests/bpf: Utility functions to capture libbpf log in test_progsEduard Zingerman2024-03-061-0/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several test_progs tests already capture libbpf log in order to check for some expected output, e.g bpf_tcp_ca.c, kfunc_dynptr_param.c, log_buf.c and a few others. This commit provides a, hopefully, simple API to capture libbpf log w/o necessity to define new print callback in each test: /* Creates a global memstream capturing INFO and WARN level output * passed to libbpf_print_fn. * Returns 0 on success, negative value on failure. * On failure the description is printed using PRINT_FAIL and * current test case is marked as fail. */ int start_libbpf_log_capture(void) /* Destroys global memstream created by start_libbpf_log_capture(). * Returns a pointer to captured data which has to be freed. * Returned buffer is null terminated. */ char *stop_libbpf_log_capture(void) The intended usage is as follows: if (start_libbpf_log_capture()) return; use_libbpf(); char *log = stop_libbpf_log_capture(); ASSERT_HAS_SUBSTR(log, "... expected ...", "expected some message"); free(log); As a safety measure, free(start_libbpf_log_capture()) is invoked in the epilogue of the test_progs.c:run_one_test(). Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20240306104529.6453-6-eddyz87@gmail.com
* selftests/bpf: Move is_jit_enabled() into testing_helpersTiezhu Yang2024-01-231-18/+0
| | | | | | | | | | | | Currently, is_jit_enabled() is only used in test_progs, move it into testing_helpers so that it can be used in test_verifier. While at it, remove the second argument "0" of open() as Hou Tao suggested. Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Hou Tao <houtao1@huawei.com> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/bpf/20240123090351.2207-2-yangtiezhu@loongson.cn
* selftests/bpf: Don't truncate #test/subtest fieldAndrii Nakryiko2023-10-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | We currently expect up to a three-digit number of tests and subtests, so: #999/999: some_test/some_subtest: ... Is the largest test/subtest we can see. If we happen to cross into 1000s, current logic will just truncate everything after 7th character. This patch fixes this truncate and allows to go way higher (up to 31 characters in total). We still nicely align test numbers: #60/66 core_reloc_btfgen/type_based___incompat:OK #60/67 core_reloc_btfgen/type_based___fn_wrong_args:OK #60/68 core_reloc_btfgen/type_id:OK #60/69 core_reloc_btfgen/type_id___missing_targets:OK #60/70 core_reloc_btfgen/enumval:OK Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20231006175744.3136675-3-andrii@kernel.org
* selftests/bpf: Do not unload bpf_testmod in load_bpf_testmodJiri Olsa2023-05-161-3/+8
| | | | | | | | | | | | | Do not unload bpf_testmod in load_bpf_testmod, instead call unload_bpf_testmod separatelly. This way we will be able use un/load_bpf_testmod functions in other tests that un/load bpf_testmod module. Acked-by: David Vernet <void@manifault.com> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230515133756.1658301-6-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftests/bpf: Move test_progs helpers to testing_helpers objectJiri Olsa2023-05-161-65/+2
| | | | | | | | | | | | | | | | | Moving test_progs helpers to testing_helpers object so they can be used from test_verifier in following changes. Also adding missing ifndef header guard to testing_helpers.h header. Using stderr instead of env.stderr because un/load_bpf_testmod helpers will be used outside test_progs. Also at the point of calling them in test_progs the std files are not hijacked yet and stderr is the same as env.stderr. Acked-by: David Vernet <void@manifault.com> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230515133756.1658301-4-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftests/bpf: Test_progs can read test lists from fileStephen Veiss2023-05-011-10/+27
| | | | | | | | | | | | | | | | Improve test selection logic when using -a/-b/-d/-t options. The list of tests to include or exclude can now be read from a file, specified as @<filename>. The file contains one name (or wildcard pattern) per line, and comments beginning with # are ignored. These options can be passed multiple times to read more than one file. Signed-off-by: Stephen Veiss <sveiss@meta.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20230427225333.3506052-3-sveiss@meta.com
* selftests/bpf: Replace extract_build_id with read_build_idJiri Olsa2023-03-311-25/+0
| | | | | | | | | | Replacing extract_build_id with read_build_id that parses out build id directly from elf without using readelf tool. Acked-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20230331093157.1749137-4-jolsa@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftests/bpf: Add --json-summary option to test_progsManu Bretelle2023-03-171-5/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, test_progs outputs all stdout/stderr as it runs, and when it is done, prints a summary. It is non-trivial for tooling to parse that output and extract meaningful information from it. This change adds a new option, `--json-summary`/`-J` that let the caller specify a file where `test_progs{,-no_alu32}` can write a summary of the run in a json format that can later be parsed by tooling. Currently, it creates a summary section with successes/skipped/failures followed by a list of failed tests and subtests. A test contains the following fields: - name: the name of the test - number: the number of the test - message: the log message that was printed by the test. - failed: A boolean indicating whether the test failed or not. Currently we only output failed tests, but in the future, successful tests could be added. - subtests: A list of subtests associated with this test. A subtest contains the following fields: - name: same as above - number: sanme as above - message: the log message that was printed by the subtest. - failed: same as above but for the subtest An example run and json content below: ``` $ sudo ./test_progs -a $(grep -v '^#' ./DENYLIST.aarch64 | awk '{print $1","}' | tr -d '\n') -j -J /tmp/test_progs.json $ jq < /tmp/test_progs.json | head -n 30 { "success": 29, "success_subtest": 23, "skipped": 3, "failed": 28, "results": [ { "name": "bpf_cookie", "number": 10, "message": "test_bpf_cookie:PASS:skel_open 0 nsec\n", "failed": true, "subtests": [ { "name": "multi_kprobe_link_api", "number": 2, "message": "kprobe_multi_link_api_subtest:PASS:load_kallsyms 0 nsec\nlibbpf: extern 'bpf_testmod_fentry_test1' (strong): not resolved\nlibbpf: failed to load object 'kprobe_multi'\nlibbpf: failed to load BPF skeleton 'kprobe_multi': -3\nkprobe_multi_link_api_subtest:FAIL:fentry_raw_skel_load unexpected error: -3\n", "failed": true }, { "name": "multi_kprobe_attach_api", "number": 3, "message": "libbpf: extern 'bpf_testmod_fentry_test1' (strong): not resolved\nlibbpf: failed to load object 'kprobe_multi'\nlibbpf: failed to load BPF skeleton 'kprobe_multi': -3\nkprobe_multi_attach_api_subtest:FAIL:fentry_raw_skel_load unexpected error: -3\n", "failed": true }, { "name": "lsm", "number": 8, "message": "lsm_subtest:PASS:lsm.link_create 0 nsec\nlsm_subtest:FAIL:stack_mprotect unexpected stack_mprotect: actual 0 != expected -1\n", "failed": true } ``` The file can then be used to print a summary of the test run and list of failing tests/subtests: ``` $ jq -r < /tmp/test_progs.json '"Success: \(.success)/\(.success_subtest), Skipped: \(.skipped), Failed: \(.failed)"' Success: 29/23, Skipped: 3, Failed: 28 $ jq -r < /tmp/test_progs.json '.results | map([ if .failed then "#\(.number) \(.name)" else empty end, ( . as {name: $tname, number: $tnum} | .subtests | map( if .failed then "#\($tnum)/\(.number) \($tname)/\(.name)" else empty end ) ) ]) | flatten | .[]' | head -n 20 #10 bpf_cookie #10/2 bpf_cookie/multi_kprobe_link_api #10/3 bpf_cookie/multi_kprobe_attach_api #10/8 bpf_cookie/lsm #15 bpf_mod_race #15/1 bpf_mod_race/ksym (used_btfs UAF) #15/2 bpf_mod_race/kfunc (kfunc_btf_tab UAF) #36 cgroup_hierarchical_stats #61 deny_namespace #61/1 deny_namespace/unpriv_userns_create_no_bpf #73 fexit_stress #83 get_func_ip_test #99 kfunc_dynptr_param #99/1 kfunc_dynptr_param/dynptr_data_null #99/4 kfunc_dynptr_param/dynptr_data_null #100 kprobe_multi_bench_attach #100/1 kprobe_multi_bench_attach/kernel #100/2 kprobe_multi_bench_attach/modules #101 kprobe_multi_test #101/1 kprobe_multi_test/skel_api ``` Signed-off-by: Manu Bretelle <chantr4@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20230317163256.3809328-1-chantr4@gmail.com
* selftests/bpf: Query BPF_MAX_TRAMP_LINKS using BTFIlya Leoshkevich2023-01-281-0/+38
| | | | | | | | | Do not hard-code the value, since for s390x it will be smaller than for x86. Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com> Link: https://lore.kernel.org/r/20230128000650.1516334-4-iii@linux.ibm.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
* selftest/bpf: Make crashes more debuggable in test_progsStanislav Fomichev2023-01-271-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reset stdio before printing verbose log of the SIGSEGV'ed test. Otherwise, it's hard to understand what's going on in the cases like [0]. With the following patch applied: --- a/tools/testing/selftests/bpf/prog_tests/xdp_metadata.c +++ b/tools/testing/selftests/bpf/prog_tests/xdp_metadata.c @@ -392,6 +392,11 @@ void test_xdp_metadata(void) "generate freplace packet")) goto out; + + ASSERT_EQ(1, 2, "oops"); + int *x = 0; + *x = 1; /* die */ + while (!retries--) { if (bpf_obj2->bss->called) break; Before: #281 xdp_metadata:FAIL Caught signal #11! Stack trace: ./test_progs(crash_handler+0x1f)[0x55c919d98bcf] /lib/x86_64-linux-gnu/libc.so.6(+0x3bf90)[0x7f36aea5df90] ./test_progs(test_xdp_metadata+0x1db0)[0x55c919d8c6d0] ./test_progs(+0x23b438)[0x55c919d9a438] ./test_progs(main+0x534)[0x55c919d99454] /lib/x86_64-linux-gnu/libc.so.6(+0x2718a)[0x7f36aea4918a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f36aea49245] ./test_progs(_start+0x21)[0x55c919b82ef1] After: test_xdp_metadata:PASS:ip netns add xdp_metadata 0 nsec open_netns:PASS:malloc token 0 nsec open_netns:PASS:open /proc/self/ns/net 0 nsec open_netns:PASS:open netns fd 0 nsec open_netns:PASS:setns 0 nsec .. test_xdp_metadata:FAIL:oops unexpected oops: actual 1 != expected 2 #281 xdp_metadata:FAIL Caught signal #11! Stack trace: ./test_progs(crash_handler+0x1f)[0x562714a76bcf] /lib/x86_64-linux-gnu/libc.so.6(+0x3bf90)[0x7fa663f9cf90] ./test_progs(test_xdp_metadata+0x1db0)[0x562714a6a6d0] ./test_progs(+0x23b438)[0x562714a78438] ./test_progs(main+0x534)[0x562714a77454] /lib/x86_64-linux-gnu/libc.so.6(+0x2718a)[0x7fa663f8818a] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7fa663f88245] ./test_progs(_start+0x21)[0x562714860ef1] 0: https://github.com/kernel-patches/bpf/actions/runs/4019879316/jobs/6907358876 Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20230127215705.1254316-1-sdf@google.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/netJakub Kicinski2022-11-171-1/+1
|\ | | | | | | | | | | | | | | | | | | include/linux/bpf.h 1f6e04a1c7b8 ("bpf: Fix offset calculation error in __copy_map_value and zero_map_value") aa3496accc41 ("bpf: Refactor kptr_off_tab into btf_record") f71b2f64177a ("bpf: Refactor map->off_arr handling") https://lore.kernel.org/all/20221114095000.67a73239@canb.auug.org.au/ Signed-off-by: Jakub Kicinski <kuba@kernel.org>
| * selftests/bpf: Fix test_progs compilation failure in 32-bit archYang Jihong2022-11-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | test_progs fails to be compiled in the 32-bit arch, log is as follows: test_progs.c:1013:52: error: format '%ld' expects argument of type 'long int', but argument 3 has type 'size_t' {aka 'unsigned int'} [-Werror=format=] 1013 | sprintf(buf, "MSG_TEST_LOG (cnt: %ld, last: %d)", | ~~^ | | | long int | %d 1014 | strlen(msg->test_log.log_buf), | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | | | size_t {aka unsigned int} Fix it. Fixes: 91b2c0afd00c ("selftests/bpf: Add parallelism to test_progs") Signed-off-by: Yang Jihong <yangjihong1@huawei.com> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/r/20221108015857.132457-1-yangjihong1@huawei.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* | selftests: Fix test group SKIPPED resultDomenico Cerasuolo2022-11-091-16/+22
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When showing the result of a test group, if one of the subtests was skipped, while still having passing subtests, the group result was marked as SKIP. E.g.: 223/1 usdt/basic:SKIP 223/2 usdt/multispec:OK 223/3 usdt/urand_auto_attach:OK 223/4 usdt/urand_pid_attach:OK 223 usdt:SKIP The test result of usdt in the example above should be OK instead of SKIP, because the test group did have passing tests and it would be considered in "normal" state. With this change, only if all of the subtests were skipped, the group test is marked as SKIP. When only some of the subtests are skipped, a more detailed result is given, stating how many of the subtests were skipped. E.g: 223/1 usdt/basic:SKIP 223/2 usdt/multispec:OK 223/3 usdt/urand_auto_attach:OK 223/4 usdt/urand_pid_attach:OK 223 usdt:OK (SKIP: 1/4) Signed-off-by: Domenico Cerasuolo <dceras@meta.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20221109184039.3514033-1-cerasuolodomenico@gmail.com
* selftests/bpf: Deduplicate write_sysctl() to test_progs.cYiFei Zhu2022-09-091-0/+17
| | | | | | | | | | | | | | This helper is needed in multiple tests. Instead of copying it over and over, better to deduplicate this helper to test_progs.c. test_progs.c is chosen over testing_helpers.c because of this helper's use of CHECK / ASSERT_*, and the CHECK was modified to use ASSERT_* so it does not rely on a duration variable. Suggested-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: YiFei Zhu <zhuyifei@google.com> Link: https://lore.kernel.org/r/9b4fc9a27bd52f771b657b4c4090fc8d61f3a6b5.1662682323.git.zhuyifei@google.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
* selftests/bpf: Bump internal send_signal/send_signal_tracepoint timeoutDaniel Müller2022-07-291-5/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The send_signal/send_signal_tracepoint is pretty flaky, with at least one failure in every ten runs on a few attempts I've tried it: > test_send_signal_common:PASS:pipe_c2p 0 nsec > test_send_signal_common:PASS:pipe_p2c 0 nsec > test_send_signal_common:PASS:fork 0 nsec > test_send_signal_common:PASS:skel_open_and_load 0 nsec > test_send_signal_common:PASS:skel_attach 0 nsec > test_send_signal_common:PASS:pipe_read 0 nsec > test_send_signal_common:PASS:pipe_write 0 nsec > test_send_signal_common:PASS:reading pipe 0 nsec > test_send_signal_common:PASS:reading pipe error: size 0 0 nsec > test_send_signal_common:FAIL:incorrect result unexpected incorrect result: actual 48 != expected 50 > test_send_signal_common:PASS:pipe_write 0 nsec > #139/1 send_signal/send_signal_tracepoint:FAIL The reason does not appear to be a correctness issue in the strict sense. Rather, we merely do not receive the signal we are waiting for within the provided timeout. Let's bump the timeout by a factor of ten. With that change I have not been able to reproduce the failure in 150+ iterations. I am also sneaking in a small simplification to the test_progs test selection logic. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20220727182955.4044988-1-deso@posteo.net
* selftests/bpf: Remove filtered subtests from outputMykola Lysenko2022-05-201-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Currently filtered subtests show up in the output as skipped. Before: $ sudo ./test_progs -t log_fixup/missing_map #94 /1 log_fixup/bad_core_relo_trunc_none:SKIP #94 /2 log_fixup/bad_core_relo_trunc_partial:SKIP #94 /3 log_fixup/bad_core_relo_trunc_full:SKIP #94 /4 log_fixup/bad_core_relo_subprog:SKIP #94 /5 log_fixup/missing_map:OK #94 log_fixup:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED After: $ sudo ./test_progs -t log_fixup/missing_map #94 /5 log_fixup/missing_map:OK #94 log_fixup:OK Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Mykola Lysenko <mykolal@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20220520061303.4004808-1-mykolal@fb.com
* selftests/bpf: Fix subtest number formatting in test_progsMykola Lysenko2022-05-201-3/+9
| | | | | | | | | | Remove weird spaces around / while preserving proper indentation Signed-off-by: Mykola Lysenko <mykolal@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Daniel Müller <deso@posteo.net> Link: https://lore.kernel.org/bpf/20220520070144.10312-1-mykolal@fb.com
* selftests/bpf: Fix two memory leaks in prog_testsMykola Lysenko2022-04-281-6/+11
| | | | | | | | | | | | Fix log_fp memory leak in dispatch_thread_read_log. Remove obsolete log_fp clean-up code in dispatch_thread. Also, release memory of subtest_selector. This can be reproduced with -n 2/1 parameters. Signed-off-by: Mykola Lysenko <mykolal@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220428225744.1961643-1-mykolal@fb.com
* bpf/selftests: Add granular subtest output for prog_testMykola Lysenko2022-04-271-168/+458
| | | | | | | | | | | | | | | | | | | | | | | | | Implement per subtest log collection for both parallel and sequential test execution. This allows granular per-subtest error output in the 'All error logs' section. Add subtest log transfer into the protocol during the parallel test execution. Move all test log printing logic into dump_test_log function. One exception is the output of test names when verbose printing is enabled. Move test name/result printing into separate functions to avoid repetition. Print all successful subtest results in the log. Print only failed test logs when test does not have subtests. Or only failed subtests' logs when test has subtests. Disable 'All error logs' output when verbose mode is enabled. This functionality was already broken and is causing confusion. Signed-off-by: Mykola Lysenko <mykolal@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220427041353.246007-1-mykolal@fb.com
* selftests/bpf: Refactor prog_tests logging and test executionMykola Lysenko2022-04-181-211/+117
| | | | | | | | | | | | | | | | | | | | | | | | This is a pre-req to add separate logging for each subtest in test_progs. Move all the mutable test data to the test_result struct. Move per-test init/de-init into the run_one_test function. Consolidate data aggregation and final log output in calculate_and_print_summary function. As a side effect, this patch fixes double counting of errors for subtests and possible duplicate output of subtest log on failures. Also, add prog_tests_framework.c test to verify some of the counting logic. As part of verification, confirmed that number of reported tests is the same before and after the change for both parallel and sequential test execution. Signed-off-by: Mykola Lysenko <mykolal@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220418222507.1726259-1-mykolal@fb.com
* selftests/bpf: Improve by-name subtest selection logic in prog_testsMykola Lysenko2022-04-101-83/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improve subtest selection logic when using -t/-a/-d parameters. In particular, more than one subtest can be specified or a combination of tests / subtests. -a send_signal -d send_signal/send_signal_nmi* - runs send_signal test without nmi tests -a send_signal/send_signal_nmi*,find_vma - runs two send_signal subtests and find_vma test -a 'send_signal*' -a find_vma -d send_signal/send_signal_nmi* - runs 2 send_signal test and find_vma test. Disables two send_signal nmi subtests -t send_signal -t find_vma - runs two *send_signal* tests and one *find_vma* test This will allow us to have granular control over which subtests to disable in the CI system instead of disabling whole tests. Also, add new selftest to avoid possible regression when changing prog_test test name selection logic. Signed-off-by: Mykola Lysenko <mykolal@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220409001750.529930-1-mykolal@fb.com
* selftests/bpf: Fix cd_flavor_subdir() of test_progsYuntao Wang2022-04-031-2/+4
| | | | | | | | | | | | | | | Currently, when we run test_progs with just executable file name, for example 'PATH=. test_progs-no_alu32', cd_flavor_subdir() will not check if test_progs is running as a flavored test runner and switch into corresponding sub-directory. This will cause test_progs-no_alu32 executed by the 'PATH=. test_progs-no_alu32' command to run in the wrong directory and load the wrong BPF objects. Signed-off-by: Yuntao Wang <ytcoode@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220403135245.1713283-1-ytcoode@gmail.com
* selftests/bpf: Remove explicit setrlimit(RLIMIT_MEMLOCK) in main selftestsAndrii Nakryiko2021-12-141-2/+0
| | | | | | | | | | | As libbpf now is able to automatically take care of RLIMIT_MEMLOCK increase (or skip it altogether on recent enough kernels), remove explicit setrlimit() invocations in bench, test_maps, test_verifier, and test_progs. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20211214195904.1785155-3-andrii@kernel.org
* selftests/bpf: Mark variable as staticYucong Sun2021-11-161-2/+2
| | | | | | | | Fix warnings from checkstyle.pl Signed-off-by: Yucong Sun <sunyucong@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211112192535.898352-4-fallentree@fb.com
* selftests/bpf: Variable naming fixYucong Sun2021-11-161-8/+8
| | | | | | | | Change log_fd to log_fp to reflect its type correctly. Signed-off-by: Yucong Sun <sunyucong@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211112192535.898352-3-fallentree@fb.com
* selftests/bpf: Move summary line after the error logsYucong Sun2021-11-161-4/+4
| | | | | | | | | Makes it easier to find the summary line when there is a lot of logs to scroll back. Signed-off-by: Yucong Sun <sunyucong@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211112192535.898352-2-fallentree@fb.com
* selftests/bpf: Fix fclose/pclose mismatch in test_progsAndrea Righi2021-10-291-2/+2
| | | | | | | | | | | Make sure to use pclose() to properly close the pipe opened by popen(). Fixes: 81f77fd0deeb ("bpf: add selftest for stackmap with BPF_F_STACK_BUILD_ID") Signed-off-by: Andrea Righi <andrea.righi@canonical.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Shuah Khan <skhan@linuxfoundation.org> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20211026143409.42666-1-andrea.righi@canonical.com
* selftests/bpf: Allow some tests to be executed in sequenceYucong Sun2021-10-081-6/+54
| | | | | | | | | | This patch allows tests to define serial_test_name() instead of test_name(), and this will make test_progs execute those in sequence after all other tests finished executing concurrently. Signed-off-by: Yucong Sun <sunyucong@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211006185619.364369-3-fallentree@fb.com
* selftests/bpf: Add parallelism to test_progsYucong Sun2021-10-081-36/+575
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds "-j" mode to test_progs, executing tests in multiple process. "-j" mode is optional, and works with all existing test selection mechanism, as well as "-v", "-l" etc. In "-j" mode, main process use UDS/SEQPACKET to communicate to each forked worker, commanding it to run tests and collect logs. After all tests are finished, a summary is printed. main process use multiple competing threads to dispatch work to worker, trying to keep them all busy. The test status will be printed as soon as it is finished, if there are error logs, it will be printed after the final summary line. By specifying "--debug", additional debug information on server/worker communication will be printed. Example output: > ./test_progs -n 15-20 -j [ 12.801730] bpf_testmod: loading out-of-tree module taints kernel. Launching 8 workers. #20 btf_split:OK #16 btf_endian:OK #18 btf_module:OK #17 btf_map_in_map:OK #19 btf_skc_cls_ingress:OK #15 btf_dump:OK Summary: 6/20 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Yucong Sun <sunyucong@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211006185619.364369-2-fallentree@fb.com
* bpf/selftests: Add test for writable bare tracepointHou Tao2021-10-081-2/+2
| | | | | | | | | | | | | Add a writable bare tracepoint in bpf_testmod module, and trigger its calling when reading /sys/kernel/bpf_testmod with a specific buffer length. The reading will return the value in writable context if the early return flag is enabled in writable context. Signed-off-by: Hou Tao <houtao1@huawei.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20211004094857.30868-4-hotforest@gmail.com
* selftests/bpf: Add test for bpf_get_branch_snapshotSong Liu2021-09-131-0/+39
| | | | | | | | | | | | This test uses bpf_get_branch_snapshot from a fexit program. The test uses a target function (bpf_testmod_loop_test) and compares the record against kallsyms. If there isn't enough record matching kallsyms, the test fails. Signed-off-by: Song Liu <songliubraving@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: John Fastabend <john.fastabend@gmail.com> Link: https://lore.kernel.org/bpf/20210910183352.3151445-4-songliubraving@fb.com
* selftests/bpf: Support glob matching for test selector.Yucong Sun2021-08-171-16/+62
| | | | | | | | | | | | | | | | This patch adds '-a' and '-d' arguments supporting both exact string match as well as using '*' wildcard in test/subtests selection. '-a' and '-t' can co-exists, same as '-d' and '-b', in which case they just add to the list of allowed or denied test selectors. Caveat: Same as the current substring matching mechanism, test and subtest selector applies independently, 'a*/b*' will execute all tests matching "a*", and with subtest name matching "b*", but tests matching "a*" that has no subtests will also be executed. Signed-off-by: Yucong Sun <fallentree@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210817044732.3263066-5-fallentree@fb.com
* selftests/bpf: Also print test name in subtest status messageYucong Sun2021-08-171-2/+2
| | | | | | | | | | | | | | | | | | This patch add test name in subtest status message line, making it possible to grep ':OK' in the output to generate a list of passed test+subtest names, which can be processed to generate argument list to be used with "-a", "-d" exact string matching. Example: #1/1 align/mov:OK .. #1/12 align/pointer variable subtraction:OK #1 align:OK Signed-off-by: Yucong Sun <fallentree@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210817044732.3263066-4-fallentree@fb.com
* selftests/bpf: Correctly display subtest skip statusYucong Sun2021-08-171-12/+13
| | | | | | | | | | | | In skip_account(), test->skip_cnt is set to 0 at the end, this makes next print statement never display SKIP status for the subtest. This patch moves the accounting logic after the print statement, fixing the issue. This patch also added SKIP status display for normal tests. Signed-off-by: Yucong Sun <fallentree@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210817044732.3263066-3-fallentree@fb.com
* selftests/bpf: Skip loading bpf_testmod when using -l to list tests.Yucong Sun2021-08-171-2/+2
| | | | | | | | | | When using "-l", test_progs often is executed as non-root user, load_bpf_testmod() will fail and output errors. This patch skips loading bpf testmod when "-l" is specified, making output cleaner. Signed-off-by: Yucong Sun <fallentree@fb.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20210817044732.3263066-2-fallentree@fb.com
* selftests/bpf: Turn on libbpf 1.0 mode and fix all IS_ERR checksAndrii Nakryiko2021-05-251-0/+3
| | | | | | | | | | | | | | | | | | | | | Turn ony libbpf 1.0 mode. Fix all the explicit IS_ERR checks that now will be broken because libbpf returns NULL on error (and sets errno). Fix ASSERT_OK_PTR and ASSERT_ERR_PTR to work for both old mode and new modes and use them throughout selftests. This is trivial to do by using libbpf_get_error() API that all libbpf users are supposed to use, instead of IS_ERR checks. A bunch of checks also did explicit -1 comparison for various fd-returning APIs. Such checks are replaced with >= 0 or < 0 cases. There were also few misuses of bpf_object__find_map_by_name() in test_maps. Those are fixed in this patch as well. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: John Fastabend <john.fastabend@gmail.com> Acked-by: Toke Høiland-Jørgensen <toke@redhat.com> Link: https://lore.kernel.org/bpf/20210525035935.1461796-3-andrii@kernel.org
* selftests/bpf: Don't exit on failed bpf_testmod unloadAndrii Nakryiko2021-01-261-1/+1
| | | | | | | | | | | | | Fix bug in handling bpf_testmod unloading that will cause test_progs exiting prematurely if bpf_testmod unloading failed. This is especially problematic when running a subset of test_progs that doesn't require root permissions and doesn't rely on bpf_testmod, yet will fail immediately due to exit(1) in unload_bpf_testmod(). Fixes: 9f7fa225894c ("selftests/bpf: Add bpf_testmod kernel module for testing") Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20210126065019.1268027-1-andrii@kernel.org
* selftests/bpf: Sync RCU before unloading bpf_testmodAndrii Nakryiko2021-01-121-0/+11
| | | | | | | | | | | | | | | | | | | | If some of the subtests use module BTFs through ksyms, they will cause bpf_prog to take a refcount on bpf_testmod module, which will prevent it from successfully unloading. Module's refcnt is decremented when bpf_prog is freed, which generally happens in RCU callback. So we need to trigger syncronize_rcu() in the kernel, which can be achieved nicely with membarrier(MEMBARRIER_CMD_SHARED) or membarrier(MEMBARRIER_CMD_GLOBAL) syscall. So do that in kernel_sync_rcu() and make it available to other test inside the test_progs. This synchronize_rcu() is called before attempting to unload bpf_testmod. Fixes: 9f7fa225894c ("selftests/bpf: Add bpf_testmod kernel module for testing") Suggested-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Acked-by: Hao Luo <haoluo@google.com> Link: https://lore.kernel.org/bpf/20210112075520.4103414-5-andrii@kernel.org
* selftests/bpf: Silence ima_setup.sh when not running in verbose mode.KP Singh2020-12-101-0/+10
| | | | | | | | | | | | | | | | | | | | | Currently, ima_setup.sh spews outputs from commands like mkfs and dd on the terminal without taking into account the verbosity level of the test framework. Update test_progs to set the environment variable SELFTESTS_VERBOSE=1 when a verbose output is requested. This environment variable is then used by ima_setup.sh (and can be used by other similar scripts) to obey the verbosity level of the test harness without needing to re-implement command line options for verbosity. In "silent" mode, the script saves the output to a temporary file, the contents of which are echoed back to stderr when the script encounters an error. Fixes: 34b82d3ac105 ("bpf: Add a selftest for bpf_ima_inode_hash") Reported-by: Andrii Nakryiko <andrii@kernel.org> Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: KP Singh <kpsingh@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20201211010711.3716917-1-kpsingh@kernel.org
* selftests/bpf: Add support for marking sub-tests as skippedAndrii Nakryiko2020-12-031-3/+3
| | | | | | | | | Previously skipped sub-tests would be counted as passing with ":OK" appened in the log. Change that to be accounted as ":SKIP". Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20201203204634.1325171-8-andrii@kernel.org
* selftests/bpf: Add bpf_testmod kernel module for testingAndrii Nakryiko2020-12-031-0/+59
| | | | | | | | | | | | | | | | Add bpf_testmod module, which is conceptually out-of-tree module and provides ways for selftests/bpf to test various kernel module-related functionality: raw tracepoint, fentry/fexit/fmod_ret, etc. This module will be auto-loaded by test_progs test runner and expected by some of selftests to be present and loaded. Pahole currently isn't able to generate BTF for static functions in kernel modules, so make sure traced function is global. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20201203204634.1325171-7-andrii@kernel.org
* selftests/bpf: Fix test_progs-flavor run getting number of testsJesper Dangaard Brouer2020-08-241-1/+3
| | | | | | | | | | | | | | | | | | | | | | Commit 643e7233aa94 ("selftests/bpf: Test_progs option for getting number of tests") introduced ability to getting number of tests, which is targeted towards scripting. As demonstrate in the commit the number can be use as a shell variable for further scripting. The test_progs program support "flavor", which is detected by the binary have a "-flavor" in the executable name. One example is test_progs-no_alu32, which load bpf-progs compiled with disabled alu32, located in dir 'no_alu32/'. The problem is that invoking a "flavor" binary prints to stdout e.g.: "Switching to flavor 'no_alu32' subdirectory..." Thus, intermixing with the number of tests, making it unusable for scripting. Fix the issue by only printing "flavor" info when verbose -v option is used. Fixes: 643e7233aa94 ("selftests/bpf: Test_progs option for getting number of tests") Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/159827024012.923543.7104106594870150597.stgit@firesoul
* selftests/bpf: test_progs avoid minus shell exit codesJesper Dangaard Brouer2020-07-091-4/+5
| | | | | | | | | | | | | | | | | There are a number of places in test_progs that use minus-1 as the argument to exit(). This is confusing as a process exit status is masked to be a number between 0 and 255 as defined in man exit(3). Thus, users will see status 255 instead of minus-1. This patch use positive exit code 3 instead of minus-1. These cases are put in the same group of infrastructure setup errors. Fixes: fd27b1835e70 ("selftests/bpf: Reset process and thread affinity after each test/sub-test") Fixes: 811d7e375d08 ("bpf: selftests: Restore netns after each test") Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/159410594499.1093222.11080787853132708654.stgit@firesoul
* selftests/bpf: test_progs use another shell exit on non-actionsJesper Dangaard Brouer2020-07-091-1/+3
| | | | | | | | | | | | | | | | | | This is a follow up adjustment to commit 6c92bd5cd465 ("selftests/bpf: Test_progs indicate to shell on non-actions"), that returns shell exit indication EXIT_FAILURE (value 1) when user selects a non-existing test. The problem with using EXIT_FAILURE is that a shell script cannot tell the difference between a non-existing test and the test failing. This patch uses value 2 as shell exit indication. (Aside note unrecognized option parameters use value 64). Fixes: 6c92bd5cd465 ("selftests/bpf: Test_progs indicate to shell on non-actions") Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/159410593992.1093222.90072558386094370.stgit@firesoul
* bpf: selftests: Restore netns after each testMartin KaFai Lau2020-07-021-2/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is common for networking tests creating its netns and making its own setting under this new netns (e.g. changing tcp sysctl). If the test forgot to restore to the original netns, it would affect the result of other tests. This patch saves the original netns at the beginning and then restores it after every test. Since the restore "setns()" is not expensive, it does it on all tests without tracking if a test has created a new netns or not. The new restore_netns() could also be done in test__end_subtest() such that each subtest will get an automatic netns reset. However, the individual test would lose flexibility to have total control on netns for its own subtests. In some cases, forcing a test to do unnecessary netns re-configure for each subtest is time consuming. e.g. In my vm, forcing netns re-configure on each subtest in sk_assign.c increased the runtime from 1s to 8s. On top of that, test_progs.c is also doing per-test (instead of per-subtest) cleanup for cgroup. Thus, this patch also does per-test restore_netns(). The only existing per-subtest cleanup is reset_affinity() and no test is depending on this. Thus, it is removed from test__end_subtest() to give a consistent expectation to the individual tests. test_progs.c only ensures any affinity/netns/cgroup change made by an earlier test does not affect the following tests. Signed-off-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/20200702004858.2103728-1-kafai@fb.com
* selftests/bpf: Test_progs option for listing test namesJesper Dangaard Brouer2020-07-011-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The program test_progs have some very useful ability to specify a list of test name substrings for selecting which tests to run. This patch add the ability to list the selected test names without running them. This is practical for seeing which tests gets selected with given select arguments (which can also contain a exclude list via --name-blacklist). This output can also be used by shell-scripts in a for-loop: for N in $(./test_progs --list -t xdp); do \ ./test_progs -t $N 2>&1 > result_test_${N}.log & \ done ; wait This features can also be used for looking up a test number and returning a testname. If the selection was empty then a shell EXIT_FAILURE is returned. This is useful for scripting. e.g. like this: n=1; while [ $(./test_progs --list -n $n) ] ; do \ ./test_progs -n $n ; n=$(( n+1 )); \ done Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/159363985751.930467.9610992940793316982.stgit@firesoul
* selftests/bpf: Test_progs option for getting number of testsJesper Dangaard Brouer2020-07-011-0/+18
| | | | | | | | | | | | | | | | | | | | It can be practial to get the number of tests that test_progs contain. This could for example be used to create a shell for-loop construct that runs the individual tests. Like: for N in $(seq 1 $(./test_progs -c)); do ./test_progs -n $N 2>&1 > result_test_${N}.log & done ; wait V2: Add the ability to return the count for the selected tests. This is useful for getting a count e.g. after excluding some tests with option -b. The current beakers test script like to report the max test count upfront. Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/159363985244.930467.12617117873058936829.stgit@firesoul
* selftests/bpf: Test_progs indicate to shell on non-actionsJesper Dangaard Brouer2020-07-011-0/+3
| | | | | | | | | | | | When a user selects a non-existing test the summary is printed with indication 0 for all info types, and shell "success" (EXIT_SUCCESS) is indicated. This can be understood by a human end-user, but for shell scripting is it useful to indicate a shell failure (EXIT_FAILURE). Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Andrii Nakryiko <andriin@fb.com> Link: https://lore.kernel.org/bpf/159363984736.930467.17956007131403952343.stgit@firesoul
* selftests/bpf: Extract parse_num_list into generic testing_helpers.cAndrii Nakryiko2020-05-131-63/+4
| | | | | | | | | | | Add testing_helpers.c, which will contain generic helpers for test runners and tests needing some common generic functionality, like parsing a set of numbers. Signed-off-by: Andrii Nakryiko <andriin@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20200512192445.2351848-2-andriin@fb.com
* selftests/bpf: Move existing common networking parts into network_helpersStanislav Fomichev2020-05-091-30/+0
| | | | | | | | | | | 1. Move pkt_v4 and pkt_v6 into network_helpers and adjust the users. 2. Copy-paste spin_lock_thread into two tests that use it. Signed-off-by: Stanislav Fomichev <sdf@google.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Martin KaFai Lau <kafai@fb.com> Acked-by: Andrey Ignatov <rdna@fb.com> Link: https://lore.kernel.org/bpf/20200508174611.228805-3-sdf@google.com
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-15 20:11:03 +0000