summaryrefslogtreecommitdiffstats
path: root/lib/kunit/test.c
Commit message (Collapse)AuthorAgeFilesLines
* kunit: bail out early in __kunit_test_suites_init() if there are no suites ↵Scott Mayhew2024-05-061-0/+3
| | | | | | | | | | | | | | | | | | | | to test Commit c72a870926c2 added a mutex to prevent kunit tests from running concurrently. Unfortunately that mutex gets locked during module load regardless of whether the module actually has any kunit tests. This causes a problem for kunit tests that might need to load other kernel modules (e.g. gss_krb5_test loading the camellia module). So check to see if there are actually any tests to run before locking the kunit_run_lock mutex. Fixes: c72a870926c2 ("kunit: add ability to run tests after boot using debugfs") Reported-by: Nico Pache <npache@redhat.com> Signed-off-by: Scott Mayhew <smayhew@redhat.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: device: Unregister the kunit_bus on shutdownDavid Gow2024-02-061-0/+3
| | | | | | | | | | | | | | | | | | | If KUnit is built as a module, and it's unloaded, the kunit_bus is not unregistered. This causes an error if it's then re-loaded later, as we try to re-register the bus. Unregister the bus and root_device on shutdown, if it looks valid. In addition, be more specific about the value of kunit_bus_device. It is: - a valid struct device* if the kunit_bus initialised correctly. - an ERR_PTR if it failed to initialise. - NULL before initialisation and after shutdown. Fixes: d03c720e03bd ("kunit: Add APIs for managing devices") Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: run test suites only after module initialization completesMarco Pagani2024-01-221-3/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()") fixed a wild-memory-access bug that could have happened during the loading phase of test suites built and executed as loadable modules. However, it also introduced a problematic side effect that causes test suites modules to crash when they attempt to register fake devices. When a module is loaded, it traverses the MODULE_STATE_UNFORMED and MODULE_STATE_COMING states before reaching the normal operating state MODULE_STATE_LIVE. Finally, when the module is removed, it moves to MODULE_STATE_GOING before being released. However, if the loading function load_module() fails between complete_formation() and do_init_module(), the module goes directly from MODULE_STATE_COMING to MODULE_STATE_GOING without passing through MODULE_STATE_LIVE. This behavior was causing kunit_module_exit() to be called without having first executed kunit_module_init(). Since kunit_module_exit() is responsible for freeing the memory allocated by kunit_module_init() through kunit_filter_suites(), this behavior was resulting in a wild-memory-access bug. Commit 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()") fixed this issue by running the tests when the module is still in MODULE_STATE_COMING. However, modules in that state are not fully initialized, lacking sysfs kobjects. Therefore, if a test module attempts to register a fake device, it will inevitably crash. This patch proposes a different approach to fix the original wild-memory-access bug while restoring the normal module execution flow by making kunit_module_exit() able to detect if kunit_module_init() has previously initialized the tests suite set. In this way, test modules can once again register fake devices without crashing. This behavior is achieved by checking whether mod->kunit_suites is a virtual or direct mapping address. If it is a virtual address, then kunit_module_init() has allocated the suite_set in kunit_filter_suites() using kmalloc_array(). On the contrary, if mod->kunit_suites is still pointing to the original address that was set when looking up the .kunit_test_suites section of the module, then the loading phase has failed and there's no memory to be freed. v4: - rebased on 6.8 - noted that kunit_filter_suites() must return a virtual address v3: - add a comment to clarify why the start address is checked v2: - add include <linux/mm.h> Fixes: 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()") Reviewed-by: David Gow <davidgow@google.com> Tested-by: Rae Moar <rmoar@google.com> Tested-by: Richard Fitzgerald <rf@opensource.cirrus.com> Reviewed-by: Javier Martinez Canillas <javierm@redhat.com> Signed-off-by: Marco Pagani <marpagan@redhat.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Fix NULL-dereference in kunit_init_suite() if suite->log is NULLRichard Fitzgerald2024-01-031-1/+3
| | | | | | | | | | | | | suite->log must be checked for NULL before passing it to string_stream_clear(). This was done in kunit_init_test() but was missing from kunit_init_suite(). Signed-off-by: Richard Fitzgerald <rf@opensource.cirrus.com> Fixes: 6d696c4695c5 ("kunit: add ability to run tests after boot using debugfs") Reviewed-by: Rae Moar <rmoar@google.com> Acked-by: David Gow <davidgow@google.com> Reviewed-by: Muhammad Usama Anjum <usama.anjum@collabora.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Reset test->priv after each param iterationMichal Wajdeczko2023-12-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we run parameterized test that uses test->priv to prepare some custom data, then value of test->priv will leak to the next param iteration and may be unexpected. This could be easily seen if we promote example_priv_test to parameterized test as then only first test iteration will be successful: $ ./tools/testing/kunit/kunit.py run \ --kunitconfig ./lib/kunit/.kunitconfig *.example_priv* [ ] Starting KUnit Kernel (1/1)... [ ] ============================================================ [ ] =================== example (1 subtest) ==================== [ ] ==================== example_priv_test ==================== [ ] [PASSED] example value 3 [ ] # example_priv_test: initializing [ ] # example_priv_test: ASSERTION FAILED at lib/kunit/kunit-example-test.c:230 [ ] Expected test->priv == ((void *)0), but [ ] test->priv == 0000000060dfe290 [ ] ((void *)0) == 0000000000000000 [ ] # example_priv_test: cleaning up [ ] [FAILED] example value 2 [ ] # example_priv_test: initializing [ ] # example_priv_test: ASSERTION FAILED at lib/kunit/kunit-example-test.c:230 [ ] Expected test->priv == ((void *)0), but [ ] test->priv == 0000000060dfe290 [ ] ((void *)0) == 0000000000000000 [ ] # example_priv_test: cleaning up [ ] [FAILED] example value 1 [ ] # example_priv_test: initializing [ ] # example_priv_test: ASSERTION FAILED at lib/kunit/kunit-example-test.c:230 [ ] Expected test->priv == ((void *)0), but [ ] test->priv == 0000000060dfe290 [ ] ((void *)0) == 0000000000000000 [ ] # example_priv_test: cleaning up [ ] [FAILED] example value 0 [ ] # example_priv_test: initializing [ ] # example_priv_test: cleaning up [ ] # example_priv_test: pass:1 fail:3 skip:0 total:4 [ ] ================ [FAILED] example_priv_test ================ [ ] # example: initializing suite [ ] # module: kunit_example_test [ ] # example: exiting suite [ ] # Totals: pass:1 fail:3 skip:0 total:4 [ ] ===================== [FAILED] example ===================== Fix that by resetting test->priv after each param iteration, in similar way what we did for the test->status. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Add APIs for managing devicesdavidgow@google.com2023-12-181-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Tests for drivers often require a struct device to pass to other functions. While it's possible to create these with root_device_register(), or to use something like a platform device, this is both a misuse of those APIs, and can be difficult to clean up after, for example, a failed assertion. Add some KUnit-specific functions for registering and unregistering a struct device: - kunit_device_register() - kunit_device_register_with_driver() - kunit_device_unregister() These helpers allocate a on a 'kunit' bus which will either probe the driver passed in (kunit_device_register_with_driver), or will create a stub driver (kunit_device_register) which is cleaned up on test shutdown. Devices are automatically unregistered on test shutdown, but can be manually unregistered earlier with kunit_device_unregister() in order to, for example, test device release code. Reviewed-by: Matti Vaittinen <mazziesaccount@gmail.com> Reviewed-by: Maxime Ripard <mripard@kernel.org> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: add ability to run tests after boot using debugfsRae Moar2023-12-181-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add functionality to run built-in tests after boot by writing to a debugfs file. Add a new debugfs file labeled "run" for each test suite to use for this purpose. As an example, write to the file using the following: echo "any string" > /sys/kernel/debugfs/kunit/<testsuite>/run This will trigger the test suite to run and will print results to the kernel log. To guard against running tests concurrently with this feature, add a mutex lock around running kunit. This supports the current practice of not allowing tests to be run concurrently on the same kernel. This new functionality could be used to design a parameter injection feature in the future. Fixed up merge conflict duing rebase to Linux 6.7-rc6 Signed-off-by: Shuah Khan <skhan@linuxfoundation.org> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: add KUNIT_INIT_TABLE to init linker sectionRae Moar2023-12-181-7/+19
| | | | | | | | | | | | | | Add KUNIT_INIT_TABLE to the INIT_DATA linker section. Alter the KUnit macros to create init tests: kunit_test_init_section_suites Update lib/kunit/executor.c to run both the suites in KUNIT_TABLE and KUNIT_INIT_TABLE. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Add a macro to wrap a deferred action functionDavid Gow2023-12-181-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KUnit's deferred action API accepts a void(*)(void *) function pointer which is called when the test is exited. However, we very frequently want to use existing functions which accept a single pointer, but which may not be of type void*. While this is probably dodgy enough to be on the wrong side of the C standard, it's been often used for similar callbacks, and gcc's -Wcast-function-type seems to ignore cases where the only difference is the type of the argument, assuming it's compatible (i.e., they're both pointers to data). However, clang 16 has introduced -Wcast-function-type-strict, which no longer permits any deviation in function pointer type. This seems to be because it'd break CFI, which validates the type of function calls. This rather ruins our attempts to cast functions to defer them, and leaves us with a few options. The one we've chosen is to implement a macro which will generate a wrapper function which accepts a void*, and casts the argument to the appropriate type. For example, if you were trying to wrap: void foo_close(struct foo *handle); you could use: KUNIT_DEFINE_ACTION_WRAPPER(kunit_action_foo_close, foo_close, struct foo *); This would create a new kunit_action_foo_close() function, of type kunit_action_t, which could be passed into kunit_add_action() and similar functions. In addition to defining this macro, update KUnit and its tests to use it. Link: https://github.com/ClangBuiltLinux/linux/issues/1750 Reviewed-by: Nathan Chancellor <nathan@kernel.org> Tested-by: Nathan Chancellor <nathan@kernel.org> Acked-by: Daniel Vetter <daniel@ffwll.ch> Reviewed-by: Maxime Ripard <mripard@kernel.org> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Reset suite counter right before running testsMichal Wajdeczko2023-11-141-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Today we reset the suite counter as part of the suite cleanup, called from the module exit callback, but it might not work that well as one can try to collect results without unloading a previous test (either unintentionally or due to dependencies). For easy reproduction try to load the kunit-test.ko and then collect and parse results from the kunit-example-test.ko load. Parser will complain about mismatch of expected test number: [ ] KTAP version 1 [ ] 1..1 [ ] # example: initializing suite [ ] KTAP version 1 [ ] # Subtest: example .. [ ] # example: pass:5 fail:0 skip:4 total:9 [ ] # Totals: pass:6 fail:0 skip:6 total:12 [ ] ok 7 example [ ] [ERROR] Test: example: Expected test number 1 but found 7 [ ] ===================== [PASSED] example ===================== [ ] ============================================================ [ ] Testing complete. Ran 12 tests: passed: 6, skipped: 6, errors: 1 Since we are now printing suite test plan on every module load, right before running suite tests, we should make sure that suite counter will also start from 1. Easiest solution seems to be move counter reset to the __kunit_test_suites_init() function. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Warn if tests are slowMaxime Ripard2023-11-141-0/+38
| | | | | | | | | | | | | | | | Kunit recently gained support to setup attributes, the first one being the speed of a given test, then allowing to filter out slow tests. A slow test is defined in the documentation as taking more than one second. There's an another speed attribute called "super slow" but whose definition is less clear. Add support to the test runner to check the test execution time, and report tests that should be marked as slow but aren't. Signed-off-by: Maxime Ripard <mripard@kernel.org> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Reset test status on each param iterationMichal Wajdeczko2023-09-181-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we skip one parametrized test case then test status remains SKIP for all subsequent test params leading to wrong reports: $ ./tools/testing/kunit/kunit.py run \ --kunitconfig ./lib/kunit/.kunitconfig *.example_params* --raw_output \ [ ] Starting KUnit Kernel (1/1)... KTAP version 1 1..1 # example: initializing suite KTAP version 1 # Subtest: example # module: kunit_example_test 1..1 KTAP version 1 # Subtest: example_params_test # example_params_test: initializing # example_params_test: cleaning up ok 1 example value 3 # SKIP unsupported param value 3 # example_params_test: initializing # example_params_test: cleaning up ok 2 example value 2 # SKIP unsupported param value 3 # example_params_test: initializing # example_params_test: cleaning up ok 3 example value 1 # SKIP unsupported param value 3 # example_params_test: initializing # example_params_test: cleaning up ok 4 example value 0 # SKIP unsupported param value 0 # example_params_test: pass:0 fail:0 skip:4 total:4 ok 1 example_params_test # SKIP unsupported param value 0 # example: exiting suite ok 1 example # SKIP Reset test status and status comment after each param iteration to avoid using stale results. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Use string_stream for test logRichard Fitzgerald2023-09-181-39/+5
| | | | | | | | | | | | | | | | | | | Replace the fixed-size log buffer with a string_stream so that the log can grow as lines are added. The existing kunit log tests have been updated for using a string_stream as the log. No new test have been added because there are already tests for the underlying string_stream. As the log tests now depend on string_stream functions they cannot build when kunit-test is a module. They have been surrounded by a #if to replace them with skipping version when the test is build as a module. Though this isn't pretty, it avoids moving code to another file while that code is also being changed. Signed-off-by: Richard Fitzgerald <rf@opensource.cirrus.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: string-stream: Decouple string_stream from kunitRichard Fitzgerald2023-09-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Re-work string_stream so that it is not tied to a struct kunit. This is to allow using it for the log of struct kunit_suite. Instead of resource-managing individual allocations the whole string_stream can be resource-managed, if required. alloc_string_stream() now allocates a string stream that is not resource-managed. string_stream_destroy() now works on an unmanaged string_stream allocated by alloc_string_stream() and frees the entire string_stream (previously it only freed the fragments). string_stream_clear() has been made public for callers that want to free the fragments without destroying the string_stream. For resource-managed allocations use kunit_alloc_string_stream() and kunit_free_string_stream(). In addition to this, string_stream_get_string() now returns an unmanaged buffer that the caller must kfree(). Signed-off-by: Richard Fitzgerald <rf@opensource.cirrus.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: string-stream: Add kunit_alloc_string_stream()Richard Fitzgerald2023-09-181-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Add function kunit_alloc_string_stream() to do a resource-managed allocation of a string stream, and corresponding kunit_free_string_stream() to free the resource-managed stream. This is preparing for decoupling the string_stream implementation from struct kunit, to reduce the amount of code churn when that happens. Currently: - kunit_alloc_string_stream() only calls alloc_string_stream(). - kunit_free_string_stream() takes a struct kunit* which isn't used yet. Callers of the old alloc_string_stream() and string_stream_destroy() are all requesting a managed allocation so have been changed to use the new functions. alloc_string_stream() has been temporarily made static because its current behavior has been replaced with kunit_alloc_string_stream(). Signed-off-by: Richard Fitzgerald <rf@opensource.cirrus.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Fix wild-memory-access bug in kunit_free_suite_set()Jinjie Ruan2023-09-051-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Inject fault while probing kunit-example-test.ko, if kstrdup() fails in mod_sysfs_setup() in load_module(), the mod->state will switch from MODULE_STATE_COMING to MODULE_STATE_GOING instead of from MODULE_STATE_LIVE to MODULE_STATE_GOING, so only kunit_module_exit() will be called without kunit_module_init(), and the mod->kunit_suites is no set correctly and the free in kunit_free_suite_set() will cause below wild-memory-access bug. The mod->state state machine when load_module() succeeds: MODULE_STATE_UNFORMED ---> MODULE_STATE_COMING ---> MODULE_STATE_LIVE ^ | | | delete_module +---------------- MODULE_STATE_GOING <---------+ The mod->state state machine when load_module() fails at mod_sysfs_setup(): MODULE_STATE_UNFORMED ---> MODULE_STATE_COMING ---> MODULE_STATE_GOING ^ | | | +-----------------------------------------------+ Call kunit_module_init() at MODULE_STATE_COMING state to fix the issue because MODULE_STATE_LIVE is transformed from it. Unable to handle kernel paging request at virtual address ffffff341e942a88 KASAN: maybe wild-memory-access in range [0x0003f9a0f4a15440-0x0003f9a0f4a15447] Mem abort info: ESR = 0x0000000096000004 EC = 0x25: DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 FSC = 0x04: level 0 translation fault Data abort info: ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000 CM = 0, WnR = 0, TnD = 0, TagAccess = 0 GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0 swapper pgtable: 4k pages, 48-bit VAs, pgdp=00000000441ea000 [ffffff341e942a88] pgd=0000000000000000, p4d=0000000000000000 Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP Modules linked in: kunit_example_test(-) cfg80211 rfkill 8021q garp mrp stp llc ipv6 [last unloaded: kunit_example_test] CPU: 3 PID: 2035 Comm: modprobe Tainted: G W N 6.5.0-next-20230828+ #136 Hardware name: linux,dummy-virt (DT) pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : kfree+0x2c/0x70 lr : kunit_free_suite_set+0xcc/0x13c sp : ffff8000829b75b0 x29: ffff8000829b75b0 x28: ffff8000829b7b90 x27: 0000000000000000 x26: dfff800000000000 x25: ffffcd07c82a7280 x24: ffffcd07a50ab300 x23: ffffcd07a50ab2e8 x22: 1ffff00010536ec0 x21: dfff800000000000 x20: ffffcd07a50ab2f0 x19: ffffcd07a50ab2f0 x18: 0000000000000000 x17: 0000000000000000 x16: 0000000000000000 x15: ffffcd07c24b6764 x14: ffffcd07c24b63c0 x13: ffffcd07c4cebb94 x12: ffff700010536ec7 x11: 1ffff00010536ec6 x10: ffff700010536ec6 x9 : dfff800000000000 x8 : 00008fffefac913a x7 : 0000000041b58ab3 x6 : 0000000000000000 x5 : 1ffff00010536ec5 x4 : ffff8000829b7628 x3 : dfff800000000000 x2 : ffffff341e942a80 x1 : ffffcd07a50aa000 x0 : fffffc0000000000 Call trace: kfree+0x2c/0x70 kunit_free_suite_set+0xcc/0x13c kunit_module_notify+0xd8/0x360 blocking_notifier_call_chain+0xc4/0x128 load_module+0x382c/0x44a4 init_module_from_file+0xd4/0x128 idempotent_init_module+0x2c8/0x524 __arm64_sys_finit_module+0xac/0x100 invoke_syscall+0x6c/0x258 el0_svc_common.constprop.0+0x160/0x22c do_el0_svc+0x44/0x5c el0_svc+0x38/0x78 el0t_64_sync_handler+0x13c/0x158 el0t_64_sync+0x190/0x194 Code: aa0003e1 b25657e0 d34cfc42 8b021802 (f9400440) ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Oops: Fatal exception SMP: stopping secondary CPUs Kernel Offset: 0x4d0742200000 from 0xffff800080000000 PHYS_OFFSET: 0xffffee43c0000000 CPU features: 0x88000203,3c020000,1000421b Memory Limit: none Rebooting in 1 seconds.. Fixes: 3d6e44623841 ("kunit: unify module and builtin suite definitions") Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Allow kunit test modules to use test filteringJanusz Krzysztofik2023-08-081-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | External tools, e.g., Intel GPU tools (IGT), support execution of individual selftests provided by kernel modules. That could be also applicable to kunit test modules if they provided test filtering. But test filtering is now possible only when kunit code is built into the kernel. Moreover, a filter can be specified only at boot time, then reboot is required each time a different filter is needed. Build the test filtering code also when kunit is configured as a module, expose test filtering functions to other kunit source files, and use them in kunit module notifier callback functions. Userspace can then reload the kunit module with a value of the filter_glob parameter tuned to a specific kunit test module every time it wants to limit the scope of tests executed on that module load. Make the kunit.filter* parameters visible in sysfs for user convenience. v5: Refresh on tpp of attributes filtering fix v4: Refresh on top of newly applied attributes patches and changes introdced by new versions of other patches submitted in series with this one. v3: Fix CONFIG_GLOB, required by filtering functions, not selected when building as a module (lkp@intel.com). v2: Fix new name of a structure moved to kunit namespace not updated across all uses (lkp@intel.com). Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Make 'list' action available to kunit test modulesJanusz Krzysztofik2023-08-081-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Results from kunit tests reported via dmesg may be interleaved with other kernel messages. When parsing dmesg for modular kunit results in real time, external tools, e.g., Intel GPU tools (IGT), may want to insert their own test name markers into dmesg at the start of each test, before any kernel message related to that test appears there, so existing upper level test result parsers have no doubt which test to blame for a specific kernel message. Unfortunately, kunit reports names of tests only at their completion (with the exeption of a not standarized "# Subtest: <name>" header above a test plan of each test suite or parametrized test). External tools could be able to insert their own "start of the test" markers with test names included if they new those names in advance. Test names could be learned from a list if provided by a kunit test module. There exists a feature of listing kunit tests without actually executing them, but it is now limited to configurations with the kunit module built in and covers only built-in tests, already available at boot time. Moreover, switching from list to normal mode requires reboot. If that feature was also available when kunit is built as a module, userspace could load the module with action=list parameter, load some kunit test modules they are interested in and learn about the list of tests provided by those modules, then unload them, reload the kunit module in normal mode and execute the tests with their lists already known. Extend kunit module notifier initialization callback with a processing path for only listing the tests provided by a module if the kunit action parameter is set to "list" or "list_attr". For user convenience, make the kunit.action parameter visible in sysfs. v2: Don't use a different format, use kunit_exec_list_tests() (Rae), - refresh on top of new attributes patches, handle newly introduced kunit.action=list_attr case (Rae). Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Report the count of test suites in a moduleJanusz Krzysztofik2023-08-081-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to KTAP specification[1], results should always start from a header that provides a TAP protocol version, followed by a test plan with a count of items to be executed. That pattern should be followed at each nesting level. In the current implementation of the top-most, i.e., test suite level, those rules apply only for test suites built into the kernel, executed and reported on boot. Results submitted to dmesg from kunit test modules loaded later are missing those top-level headers. As a consequence, if a kunit test module provides more than one test suite then, without the top level test plan, external tools that are parsing dmesg for kunit test output are not able to tell how many test suites should be expected and whether to continue parsing after complete output from the first test suite is collected. Submit the top-level headers also from the kunit test module notifier initialization callback. v3: Fix new name of a structure moved to kunit namespace not updated in executor_test functions (lkp@intel.com). v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from emitting the headers when called on load of non-test modules. [1] https://docs.kernel.org/dev-tools/ktap.html# Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com> Cc: Mauro Carvalho Chehab <mchehab@kernel.org> Cc: Rae Moar <rmoar@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Add ability to filter attributesRae Moar2023-07-261-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add filtering of test attributes. Users can filter tests using the module_param called "filter". Filters are imputed in the format: <attribute_name><operation><value> Example: kunit.filter="speed>slow" Operations include: >, <, >=, <=, !=, and =. These operations will act the same for attributes of the same type but may not between types. Note multiple filters can be inputted by separating them with a comma. Example: kunit.filter="speed=slow, module!=example" Since both suites and test cases can have attributes, there may be conflicts. The process of filtering follows these rules: - Filtering always operates at a per-test level. - If a test has an attribute set, then the test's value is filtered on. - Otherwise, the value falls back to the suite's value. - If neither are set, the attribute has a global "default" value, which is used. Filtered tests will not be run or show in output. The tests can instead be skipped using the configurable option "kunit.filter_action=skip". Note the default settings for running tests remains unfiltered. Finally, add "filter" methods for the speed and module attributes to parse and compare attribute values. Note this filtering functionality will be added to kunit.py in the next patch. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Add test attributes API structureRae Moar2023-07-261-7/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the basic structure of the test attribute API to KUnit, which can be used to save and access test associated data. Add attributes.c and attributes.h to hold associated structs and functions for the API. Create a struct that holds a variety of associated helper functions for each test attribute. These helper functions will be used to get the attribute value, convert the value to a string, and filter based on the value. This struct is flexible by design to allow for attributes of numerous types and contexts. Add a method to print test attributes in the format of "# [<test_name if not suite>.]<attribute_name>: <attribute_value>". Example for a suite: "# speed: slow" Example for a test case: "# test_case.speed: very_slow" Use this method to report attributes in the KTAP output (KTAP spec: https://docs.kernel.org/dev-tools/ktap.html) and _list_tests output when kernel's new kunit.action=list_attr option is used. Note this is derivative of the kunit.action=list option. In test.h, add fields and associated helper functions to test cases and suites to hold user-inputted test attributes. Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Move kunit_abort() call out of kunit_do_failed_assertion()David Gow2023-06-011-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | | KUnit aborts the current thread when an assertion fails. Currently, this is done conditionally as part of the kunit_do_failed_assertion() function, but this hides the kunit_abort() call from the compiler (particularly if it's in another module). This, in turn, can lead to both suboptimal code generation (the compiler can't know if kunit_do_failed_assertion() will return), and to static analysis tools like smatch giving false positives. Moving the kunit_abort() call into the macro should give the compiler and tools a better chance at understanding what's going on. Doing so requires exporting kunit_abort(), though it's recommended to continue to use assertions in lieu of aborting directly. In addition, kunit_abort() and kunit_do_failed_assertion() are renamed to make it clear they they're intended for internal KUnit use, to: __kunit_do_failed_assertion() and __kunit_abort() Suggested-by: Dan Carpenter <dan.carpenter@linaro.org> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Update kunit_print_ok_not_ok functionMichal Wajdeczko2023-05-261-17/+28
| | | | | | | | | | | | | | | | | | | There is no need use opaque test_or_suite pointer and is_test flag as we don't use anything from the suite struct. Always expect test pointer and use NULL as indication that provided results are from the suite so we can treat them differently. Since results could be from nested tests, like parameterized tests, add explicit level parameter to properly indent output messages and thus allow to reuse this function from other places. While around, remove small code duplication near skip directive. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Cc: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Fix reporting of the skipped parameterized testsMichal Wajdeczko2023-05-261-2/+4
| | | | | | | | | | | Logs from the parameterized tests that were skipped don't include SKIP directive thus they are displayed as PASSED. Fix that. Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com> Cc: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: kmalloc_array: Use kunit_add_action()David Gow2023-05-251-39/+9
| | | | | | | | | | | | | | | | | | | | The kunit_add_action() function is much simpler and cleaner to use that the full KUnit resource API for simple things like the kunit_kmalloc_array() functionality. Replacing it allows us to get rid of a number of helper functions, and leaves us with no uses of kunit_alloc_resource(), which has some usability problems and is going to have its behaviour modified in an upcoming patch. Note that we need to use kunit_defer_trigger_all() to implement kunit_kfree(). Reviewed-by: Benjamin Berg <benjamin.berg@intel.com> Reviewed-by: Maxime Ripard <maxime@cerno.tech> Tested-by: Maxime Ripard <maxime@cerno.tech> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Always run cleanup from a test kthreadDavid Gow2023-05-111-8/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KUnit tests run in a kthread, with the current->kunit_test pointer set to the test's context. This allows the kunit_get_current_test() and kunit_fail_current_test() macros to work. Normally, this pointer is still valid during test shutdown (i.e., the suite->exit function, and any resource cleanup). However, if the test has exited early (e.g., due to a failed assertion), the cleanup is done in the parent KUnit thread, which does not have an active context. Instead, in the event test terminates early, run the test exit and cleanup from a new 'cleanup' kthread, which sets current->kunit_test, and better isolates the rest of KUnit from issues which arise in test cleanup. If a test cleanup function itself aborts (e.g., due to an assertion failing), there will be no further attempts to clean up: an error will be logged and the test failed. For example: # example_simple_test: test aborted during cleanup. continuing without cleaning up This should also make it easier to get access to the KUnit context, particularly from within resource cleanup functions, which may, for example, need access to data in test->priv. Reviewed-by: Benjamin Berg <benjamin.berg@intel.com> Reviewed-by: Maxime Ripard <maxime@cerno.tech> Tested-by: Maxime Ripard <maxime@cerno.tech> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: fix bug of extra newline characters in debugfs logsRae Moar2023-03-101-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix bug of the extra newline characters in debugfs logs. When a line is added to debugfs with a newline character at the end, an extra line appears in the debugfs log. This is due to a discrepancy between how the lines are printed and how they are added to the logs. Remove this discrepancy by checking if a newline character is present before adding a newline character. This should closely match the printk behavior. Add kunit_log_newline_test to provide test coverage for this issue. (Also, move kunit_log_test above suite definition to remove the unnecessary declaration prior to the suite definition) As an example, say we add these two lines to the log: kunit_log(..., "KTAP version 1\n"); kunit_log(..., "1..1"); The debugfs log before this fix: KTAP version 1 1..1 The debugfs log after this fix: KTAP version 1 1..1 Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: fix bug in the order of lines in debugfs logsRae Moar2023-03-101-7/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix bug in debugfs logs that causes an incorrect order of lines in the debugfs log. Currently, the test counts lines that show the number of tests passed, failed, and skipped, as well as any suite diagnostic lines, appear prior to the individual results, which is a bug. Ensure the order of printing for the debugfs log is correct. Additionally, add a KTAP header to so the debugfs logs can be valid KTAP. This is an example of a log prior to these fixes: KTAP version 1 # Subtest: kunit_status 1..2 # kunit_status: pass:2 fail:0 skip:0 total:2 # Totals: pass:2 fail:0 skip:0 total:2 ok 1 kunit_status_set_failure_test ok 2 kunit_status_mark_skipped_test ok 1 kunit_status Note the two lines with stats are out of order. This is the same debugfs log after the fixes (in combination with the third patch to remove the extra line): KTAP version 1 1..1 KTAP version 1 # Subtest: kunit_status 1..2 ok 1 kunit_status_set_failure_test ok 2 kunit_status_mark_skipped_test # kunit_status: pass:2 fail:0 skip:0 total:2 # Totals: pass:2 fail:0 skip:0 total:2 ok 1 kunit_status Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: fix bug in debugfs logs of parameterized testsRae Moar2023-03-101-6/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix bug in debugfs logs that causes individual parameterized results to not appear because the log is reinitialized (cleared) when each parameter is run. Ensure these results appear in the debugfs logs, increase log size to allow for the size of parameterized results. As a result, append lines to the log directly rather than using an intermediate variable that can cause stack size warnings due to the increased log size. Here is the debugfs log of ext4_inode_test which uses parameterized tests before the fix: KTAP version 1 # Subtest: ext4_inode_test 1..1 # Totals: pass:16 fail:0 skip:0 total:16 ok 1 ext4_inode_test As you can see, this log does not include any of the individual parametrized results. After (in combination with the next two fixes to remove extra empty line and ensure KTAP valid format): KTAP version 1 1..1 KTAP version 1 # Subtest: ext4_inode_test 1..1 KTAP version 1 # Subtest: inode_test_xtimestamp_decoding ok 1 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits ... (the rest of the individual parameterized tests) ok 16 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16 ok 1 inode_test_xtimestamp_decoding # Totals: pass:16 fail:0 skip:0 total:16 ok 1 ext4_inode_test Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Add printf attribute to fail_current_test_implDavid Gow2023-02-081-1/+1
| | | | | | | | | | | | | | | Add the gnu_printf (__printf()) attribute to the kunit_fail_current_test() implementation in __kunit_fail_current_test_impl(). While it's not actually useful here, as this function is never called directly, it nevertheless was triggering -Wsuggest-attribute=format warnings, so we should add it to reduce the noise. Fixes: cc3ed2fe5c93 ("kunit: Add "hooks" to call into KUnit when it's built as a module") Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Add "hooks" to call into KUnit when it's built as a moduleDavid Gow2023-02-081-8/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KUnit has several macros and functions intended for use from non-test code. These hooks, currently the kunit_get_current_test() and kunit_fail_current_test() macros, didn't work when CONFIG_KUNIT=m. In order to support this case, the required functions and static data need to be available unconditionally, even when KUnit itself is not built-in. The new 'hooks.c' file is therefore always included, and has both the static key required for kunit_get_current_test(), and a table of function pointers in struct kunit_hooks_table. This is filled in with the real implementations by kunit_install_hooks(), which is kept in hooks-impl.h and called when the kunit module is loaded. This can be extended for future features which require similar "hook" behaviour, such as static stubs, by simply adding new entries to the struct, and the appropriate code to set them. Fixed white-space errors during commit: Shuah Khan <skhan@linuxfoundation.org> Resolved merge conflicts with: db105c37a4d6 ("kunit: Export kunit_running()") This patch supersedes the above. Shuah Khan <skhan@linuxfoundation.org> Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Rae Moar <rmoar@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Export kunit_running()Arnd Bergmann2023-01-201-0/+1
| | | | | | | | | | | | | | Using kunit_fail_current_test() in a loadable module causes a link error like: ERROR: modpost: "kunit_running" [drivers/gpu/drm/vc4/vc4.ko] undefined! Export the symbol to allow using it from modules. Fixes: da43ff045c3f ("drm/vc4: tests: Fail the current test if we access a register") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* Merge tag 'linux-kselftest-kunit-next-6.2-rc1' of ↵Linus Torvalds2022-12-121-3/+12
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest Pull KUnit updates from Shuah Khan: "Several enhancements, fixes, clean-ups, documentation updates, improvements to logging and KTAP compliance of KUnit test output: - log numbers in decimal and hex - parse KTAP compliant test output - allow conditionally exposing static symbols to tests when KUNIT is enabled - make static symbols visible during kunit testing - clean-ups to remove unused structure definition" * tag 'linux-kselftest-kunit-next-6.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: (29 commits) Documentation: dev-tools: Clarify requirements for result description apparmor: test: make static symbols visible during kunit testing kunit: add macro to allow conditionally exposing static symbols to tests kunit: tool: make parser preserve whitespace when printing test log Documentation: kunit: Fix "How Do I Use This" / "Next Steps" sections kunit: tool: don't include KTAP headers and the like in the test log kunit: improve KTAP compliance of KUnit test output kunit: tool: parse KTAP compliant test output mm: slub: test: Use the kunit_get_current_test() function kunit: Use the static key when retrieving the current test kunit: Provide a static key to check if KUnit is actively running tests kunit: tool: make --json do nothing if --raw_ouput is set kunit: tool: tweak error message when no KTAP found kunit: remove KUNIT_INIT_MEM_ASSERTION macro Documentation: kunit: Remove redundant 'tips.rst' page Documentation: KUnit: reword description of assertions Documentation: KUnit: make usage.rst a superset of tips.rst, remove duplication kunit: eliminate KUNIT_INIT_*_ASSERT_STRUCT macros kunit: tool: remove redundant file.close() call in unit test kunit: tool: unit tests all check parser errors, standardize formatting a bit ...
| * kunit: improve KTAP compliance of KUnit test outputRae Moar2022-12-121-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change KUnit test output to better comply with KTAP v1 specifications found here: https://kernel.org/doc/html/latest/dev-tools/ktap.html. 1) Use "KTAP version 1" instead of "TAP version 14" as test output header 2) Remove '-' between test number and test name on test result lines 2) Add KTAP version lines to each subtest header as well Note that the new KUnit output still includes the “# Subtest” line now located after the KTAP version line. This does not completely match the KTAP v1 spec but since it is classified as a diagnostic line, it is not expected to be disruptive or break any existing parsers. This “# Subtest” line comes from the TAP 14 spec (https://testanything.org/tap-version-14-specification.html) and it is used to define the test name before the results. Original output: TAP version 14 1..1 # Subtest: kunit-test-suite 1..3 ok 1 - kunit_test_1 ok 2 - kunit_test_2 ok 3 - kunit_test_3 # kunit-test-suite: pass:3 fail:0 skip:0 total:3 # Totals: pass:3 fail:0 skip:0 total:3 ok 1 - kunit-test-suite New output: KTAP version 1 1..1 KTAP version 1 # Subtest: kunit-test-suite 1..3 ok 1 kunit_test_1 ok 2 kunit_test_2 ok 3 kunit_test_3 # kunit-test-suite: pass:3 fail:0 skip:0 total:3 # Totals: pass:3 fail:0 skip:0 total:3 ok 1 kunit-test-suite Signed-off-by: Rae Moar <rmoar@google.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Tested-by: Anders Roxell <anders.roxell@linaro.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
| * kunit: Provide a static key to check if KUnit is actively running testsDavid Gow2022-12-121-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KUnit does a few expensive things when enabled. This hasn't been a problem because KUnit was only enabled on test kernels, but with a few people enabling (but not _using_) KUnit on production systems, we need a runtime way of handling this. Provide a 'kunit_running' static key (defaulting to false), which allows us to hide any KUnit code behind a static branch. This should reduce the performance impact (on other code) of having KUnit enabled to a single NOP when no tests are running. Note that, while it looks unintuitive, tests always run entirely within __kunit_test_suites_init(), so it's safe to decrement the static key at the end of this function, rather than in __kunit_test_suites_exit(), which is only there to clean up results in debugfs. Signed-off-by: David Gow <davidgow@google.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* | kunit: update NULL vs IS_ERR() testsDan Carpenter2022-10-181-1/+1
|/ | | | | | | | | | | | The alloc_string_stream() functions were changed from returning NULL on error to returning error pointers so these caller needs to be updated as well. Fixes: 78b1c6584fce ("kunit: string-stream: Simplify resource use") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: remove format func from struct kunit_assert, get it to 0 bytesDaniel Latypov2022-10-071-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | Each calll to a KUNIT_EXPECT_*() macro creates a local variable which contains a struct kunit_assert. Normally, we'd hope the compiler would be able to optimize this away, but we've seen cases where it hasn't, see https://groups.google.com/g/kunit-dev/c/i3fZXgvBrfA/m/GbrMNej2BAAJ. In changes like commit 21957f90b28f ("kunit: split out part of kunit_assert into a static const"), we've moved more and more parts out of struct kunit_assert and its children types (kunit_binary_assert). This patch removes the final field and gets us to: sizeof(struct kunit_assert) == 0 sizeof(struct kunit_binary_assert) == 24 (on UML x86_64). This also reduces the amount of macro plumbing going on at the cost of passing in one more arg to the base KUNIT_ASSERTION macro and kunit_do_failed_assertion(). Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: make kunit_kfree(NULL) a no-op to match kfree()Daniel Latypov2022-10-071-0/+3
| | | | | | | | | | | | | | | | | | | | The real kfree() function will silently return when given a NULL. So a user might reasonably think they can write the following code: char *buffer = NULL; if (param->use_buffer) buffer = kunit_kzalloc(test, 10, GFP_KERNEL); ... kunit_kfree(test, buffer); As-is, kunit_kfree() will mark the test as FAILED when buffer is NULL. (And in earlier times, it would segfault). Let's match the semantics of kfree(). Suggested-by: David Gow <davidgow@google.com> Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: make kunit_kfree() not segfault on invalid inputsDaniel Latypov2022-10-071-12/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | kunit_kfree() can only work on data ("resources") allocated by KUnit. Currently for code like this, > void *ptr = kmalloc(4, GFP_KERNEL); > kunit_kfree(test, ptr); kunit_kfree() will segfault. It'll try and look up the kunit_resource associated with `ptr` and get a NULL back, but it won't check for this. This means we also segfault if you double-free. Change kunit_kfree() so it'll notice these invalid pointers and respond by failing the test. Implementation: kunit_destroy_resource() does what kunit_kfree() does, but is more generic and returns -ENOENT when it can't find the resource. Sadly, unlike just letting it crash, this means we don't get a stack trace. But kunit_kfree() is so infrequently used it shouldn't be hard to track down the bad callsite anyways. After this change, the above code gives: > # example_simple_test: EXPECTATION FAILED at lib/kunit/test.c:702 > kunit_kfree: 00000000626ec200 already freed or not allocated by kunit Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: make kunit_kfree() only work on pointers from kunit_malloc() and friendsDaniel Latypov2022-10-071-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | kunit_kfree() exists to clean up allocations from kunit_kmalloc() and friends early instead of waiting for this to happen automatically at the end of the test. But it can be used on *anything* registered with the kunit resource API. E.g. the last 2 statements are equivalent: struct kunit_resource *res = something(); kfree(res->data); kunit_put_resource(res); The problem is that there could be multiple resources that point to the same `data`. E.g. you can have a named resource acting as a pseudo-global variable in a test. If you point it to data allocated with kunit_kmalloc(), then calling `kunit_kfree(ptr)` has the chance to delete either the named resource or to kfree `ptr`. Which one it does depends on the order the resources are registered as kunit_kfree() will delete resources in LIFO order. So this patch restricts kunit_kfree() to only working on resources created by kunit_kmalloc(). Calling it is therefore guaranteed to free the memory, not do anything else. Note: kunit_resource_instance_match() wasn't used outside of KUnit, so it should be safe to remove from the public interface. It's also generally dangerous, as shown above, and shouldn't be used. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: string-stream: Simplify resource useDavid Gow2022-10-071-1/+1
| | | | | | | | | | | | | | | | | | | | Currently, KUnit's string streams are themselves "KUnit resources". This is redundant since the stream itself is already allocated with kunit_kzalloc() and will thus be freed automatically at the end of the test. string-stream is only used internally within KUnit, and isn't using the extra features that resources provide like reference counting, being able to locate them dynamically as "test-local variables", etc. Indeed, the resource's refcount is never incremented when the pointer is returned. The fact that it's always manually destroyed is more evidence that the reference counting is unused. Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: add kunit.enable to enable/disable KUnit testJoe Fradley2022-09-301-0/+24
| | | | | | | | | | | | | This patch adds the kunit.enable module parameter that will need to be set to true in addition to KUNIT being enabled for KUnit tests to run. The default value is true giving backwards compatibility. However, for the production+testing use case the new config option KUNIT_DEFAULT_ENABLED can be set to N requiring the tester to opt-in by passing kunit.enable=1 to the kernel. Signed-off-by: Joe Fradley <joefradley@google.com> Reviewed-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: flatten kunit_suite*** to kunit_suite** in .kunit_test_suitesDaniel Latypov2022-07-111-12/+6
| | | | | | | | | | | | | | | | | | | | | We currently store kunit suites in the .kunit_test_suites ELF section as a `struct kunit_suite***` (modulo some `const`s). For every test file, we store a struct kunit_suite** NULL-terminated array. This adds quite a bit of complexity to the test filtering code in the executor. Instead, let's just make the .kunit_test_suites section contain a single giant array of struct kunit_suite pointers, which can then be directly manipulated. This array is not NULL-terminated, and so none of the test filtering code needs to NULL-terminate anything. Tested-by: Maíra Canal <maira.canal@usp.br> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Daniel Latypov <dlatypov@google.com> Co-developed-by: David Gow <davidgow@google.com> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: unify module and builtin suite definitionsJeremy Kerr2022-07-111-1/+51
| | | | | | | | | | | | | | | | | | | | | | | | | Currently, KUnit runs built-in tests and tests loaded from modules differently. For built-in tests, the kunit_test_suite{,s}() macro adds a list of suites in the .kunit_test_suites linker section. However, for kernel modules, a module_init() function is used to run the test suites. This causes problems if tests are included in a module which already defines module_init/exit_module functions, as they'll conflict with the kunit-provided ones. This change removes the kunit-defined module inits, and instead parses the kunit tests from their own section in the module. After module init, we call __kunit_test_suites_init() on the contents of that section, which prepares and runs the suite. This essentially unifies the module- and non-module kunit init formats. Tested-by: Maíra Canal <maira.canal@usp.br> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Jeremy Kerr <jk@codeconstruct.com.au> Signed-off-by: Daniel Latypov <dlatypov@google.com> Signed-off-by: David Gow <davidgow@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: Taint the kernel when KUnit tests are runDavid Gow2022-07-011-0/+4
| | | | | | | | | | | | | | | | | | | Make KUnit trigger the new TAINT_TEST taint when any KUnit test is run. Due to KUnit tests not being intended to run on production systems, and potentially causing problems (or security issues like leaking kernel addresses), the kernel's state should not be considered safe for production use after KUnit tests are run. This both marks KUnit modules as test modules using MODULE_INFO() and manually taints the kernel when tests are run (which catches builtin tests). Acked-by: Luis Chamberlain <mcgrof@kernel.org> Tested-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: David Gow <davidgow@google.com> Tested-by: Maíra Canal <mairacanal@riseup.net> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: take `kunit_assert` as `const`Miguel Ojeda2022-05-161-2/+2
| | | | | | | | | | | | | | The `kunit_do_failed_assertion` function passes its `struct kunit_assert` argument to `kunit_fail`. This one, in turn, calls its `format` field passing the assert again as a `const` pointer. Therefore, the whole chain may be made `const`. Signed-off-by: Miguel Ojeda <ojeda@kernel.org> Reviewed-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: add ability to specify suite-level init and exit functionsDaniel Latypov2022-05-021-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KUnit has support for setup/cleanup logic for each test case in a suite. But it lacks the ability to specify setup/cleanup for the entire suite itself. This can be used to do setup that is too expensive or cumbersome to do for each test. Or it can be used to do simpler things like log debug information after the suite completes. It's a fairly common feature, so the lack of it is noticeable. Some examples in other frameworks and languages: * https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass * https://google.github.io/googletest/reference/testing.html#Test::SetUpTestSuite Meta: This is very similar to this patch here: https://lore.kernel.org/linux-kselftest/20210805043503.20252-3-bvanassche@acm.org/ The changes from that patch: * pass in `struct kunit *` so users can do stuff like `kunit_info(suite, "debug message")` * makes sure the init failure is bubbled up as a failure * updates kunit-example-test.c to use a suite init * Updates kunit/usage.rst to mention the new support * some minor cosmetic things * use `suite_{init,exit}` instead of `{init/exit}_suite` * make suite init error message more consistent w/ test init * etc. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: rename print_subtest_{start,end} for clarity (s/subtest/suite)Daniel Latypov2022-05-021-4/+4
| | | | | | | | | | | | | | | | These names sound more general than they are. The _end() function increments a `static int kunit_suite_counter`, so it can only safely be called on suites, aka top-level subtests. It would need to have a separate counter for each level of subtest to be generic enough. So rename it to make it clear it's only appropriate for suites. Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: split resource API impl from test.c into new resource.cDaniel Latypov2022-04-041-115/+1
| | | | | | | | | | | | | | | We've split out the declarations from include/kunit/test.h into resource.h. This patch splits out the definitions as well for consistency. A side effect of this is git blame won't properly track history by default, users need to run $ git blame -L ,1 -C13 lib/kunit/resource.c Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
* kunit: remove va_format from kunit_assertDaniel Latypov2022-01-311-5/+7
| | | | | | | | | | | | | | | | | | | | | | The concern is that having a lot of redundant fields in kunit_assert can blow up stack usage if the compiler doesn't optimize them away [1]. The comment on this field implies that it was meant to be initialized when the expect/assert was declared, but this only happens when we run kunit_do_failed_assertion(). We don't need to access it outside of that function, so move it out of the struct and make it a local variable there. This change also takes the chance to reduce the number of macros by inlining the now simplified KUNIT_INIT_ASSERT_STRUCT() macro. [1] https://groups.google.com/g/kunit-dev/c/i3fZXgvBrfA/m/VULQg1z6BAAJ Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: David Gow <davidgow@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-09 14:57:55 +0000