summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
| * libbpf: Disable SEC pragma macro on GCCJames Hilliard2022-07-081-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It seems the gcc preprocessor breaks with pragmas when surrounding __attribute__. Disable these pragmas on GCC due to upstream bugs see: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55578 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90400 Fixes errors like: error: expected identifier or '(' before '#pragma' 106 | SEC("cgroup/bind6") | ^~~ error: expected '=', ',', ';', 'asm' or '__attribute__' before '#pragma' 114 | char _license[] SEC("license") = "GPL"; | ^~~ Signed-off-by: James Hilliard <james.hilliard1@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220706111839.1247911-1-james.hilliard1@gmail.com
| * bpf: Check attach_func_proto more carefully in check_return_codeStanislav Fomichev2022-07-084-11/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Syzkaller reports the following crash: RIP: 0010:check_return_code kernel/bpf/verifier.c:10575 [inline] RIP: 0010:do_check kernel/bpf/verifier.c:12346 [inline] RIP: 0010:do_check_common+0xb3d2/0xd250 kernel/bpf/verifier.c:14610 With the following reproducer: bpf$PROG_LOAD_XDP(0x5, &(0x7f00000004c0)={0xd, 0x3, &(0x7f0000000000)=ANY=[@ANYBLOB="1800000000000019000000000000000095"], &(0x7f0000000300)='GPL\x00', 0x0, 0x0, 0x0, 0x0, 0x0, '\x00', 0x0, 0x2b, 0xffffffffffffffff, 0x8, 0x0, 0x0, 0x10, 0x0}, 0x80) Because we don't enforce expected_attach_type for XDP programs, we end up in hitting 'if (prog->expected_attach_type == BPF_LSM_CGROUP' part in check_return_code and follow up with testing `prog->aux->attach_func_proto->type`, but `prog->aux->attach_func_proto` is NULL. Add explicit prog_type check for the "Note, BPF_LSM_CGROUP that attach ..." condition. Also, don't skip return code check for LSM/STRUCT_OPS. The above actually brings an issue with existing selftest which tries to return EPERM from void inet_csk_clone. Fix the test (and move called_socket_clone to make sure it's not incremented in case of an error) and add a new one to explicitly verify this condition. Fixes: 69fd337a975c ("bpf: per-cgroup lsm flavor") Reported-by: syzbot+5cc0730bd4b4d2c5f152@syzkaller.appspotmail.com Signed-off-by: Stanislav Fomichev <sdf@google.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20220708175000.2603078-1-sdf@google.com
| * selftests/bpf: Add test involving restrict type qualifierDaniel Müller2022-07-083-2/+13
| | | | | | | | | | | | | | | | | | | | | | This change adds a type based test involving the restrict type qualifier to the BPF selftests. On the btfgen path, this will verify that bpftool correctly handles the corresponding RESTRICT BTF kind. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/20220706212855.1700615-3-deso@posteo.net
| * bpftool: Add support for KIND_RESTRICT to gen min_core_btf commandDaniel Müller2022-07-081-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | This change adjusts bpftool's type marking logic, as used in conjunction with TYPE_EXISTS relocations, to correctly recognize and handle the RESTRICT BTF kind. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/20220623212205.2805002-1-deso@posteo.net/T/#m4c75205145701762a4b398e0cdb911d5b5305ffc Link: https://lore.kernel.org/bpf/20220706212855.1700615-2-deso@posteo.net
| * MAINTAINERS: Add entry for AF_XDP selftests filesMaciej Fijalkowski2022-07-081-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lukas reported that after commit f36600634282 ("libbpf: move xsk.{c,h} into selftests/bpf") MAINTAINERS file needed an update. In the meantime, Magnus removed AF_XDP samples in commit cfb5a2dbf141 ("bpf, samples: Remove AF_XDP samples"), but selftests part still misses its entry in MAINTAINERS. Now that xdpxceiver became xskxceiver, tools/testing/selftests/bpf/*xsk* will match all of the files related to AF_XDP testing (test_xsk.sh, xskxceiver, xsk_prereqs.sh, xsk.{c,h}). Reported-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20220707111613.49031-3-maciej.fijalkowski@intel.com
| * selftests, xsk: Rename AF_XDP testing appMaciej Fijalkowski2022-07-086-13/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recently, xsk part of libbpf was moved to selftests/bpf directory and lives on its own because there is an AF_XDP testing application that needs it called xdpxceiver. That name makes it a bit hard to indicate who maintains it as there are other XDP samples in there, whereas this one is strictly about AF_XDP. Do s/xdpxceiver/xskxceiver so that it will be easier to figure out who maintains it. A follow-up patch will correct MAINTAINERS file. Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20220707111613.49031-2-maciej.fijalkowski@intel.com
| * bpf, docs: Remove deprecated xsk libbpf APIs descriptionPu Lehui2022-07-081-11/+2
| | | | | | | | | | | | | | | | | | | | Since xsk APIs has been removed from libbpf, let's clean up the BPF docs simutaneously. Signed-off-by: Pu Lehui <pulehui@huawei.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/bpf/20220708042736.669132-1-pulehui@huawei.com
| * selftests/bpf: Add benchmark for local_storage RCU Tasks Trace usageDave Marchevsky2022-07-076-1/+416
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This benchmark measures grace period latency and kthread cpu usage of RCU Tasks Trace when many processes are creating/deleting BPF local_storage. Intent here is to quantify improvement on these metrics after Paul's recent RCU Tasks patches [0]. Specifically, fork 15k tasks which call a bpf prog that creates/destroys task local_storage and sleep in a loop, resulting in many call_rcu_tasks_trace calls. To determine grace period latency, trace time elapsed between rcu_tasks_trace_pregp_step and rcu_tasks_trace_postgp; for cpu usage look at rcu_task_trace_kthread's stime in /proc/PID/stat. On my virtualized test environment (Skylake, 8 cpus) benchmark results demonstrate significant improvement: BEFORE Paul's patches: SUMMARY tasks_trace grace period latency avg 22298.551 us stddev 1302.165 us SUMMARY ticks per tasks_trace grace period avg 2.291 stddev 0.324 AFTER Paul's patches: SUMMARY tasks_trace grace period latency avg 16969.197 us stddev 2525.053 us SUMMARY ticks per tasks_trace grace period avg 1.146 stddev 0.178 Note that since these patches are not in bpf-next benchmarking was done by cherry-picking this patch onto rcu tree. [0] https://lore.kernel.org/rcu/20220620225402.GA3842369@paulmck-ThinkPad-P17-Gen-1/ Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Paul E. McKenney <paulmck@kernel.org> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/bpf/20220705190018.3239050-1-davemarchevsky@fb.com
| * libbpf, riscv: Use a0 for RC registerYixun Lan2022-07-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to the RISC-V calling convention register usage here [0], a0 is used as return value register, so rename it to make it consistent with the spec. [0] section 18.2, table 18.2 https://riscv.org/wp-content/uploads/2015/01/riscv-calling.pdf Fixes: 589fed479ba1 ("riscv, libbpf: Add RISC-V (RV64) support to bpf_tracing.h") Signed-off-by: Yixun Lan <dlan@gentoo.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Björn Töpel <bjorn@kernel.org> Acked-by: Amjad OULED-AMEUR <ouledameur.amjad@gmail.com> Link: https://lore.kernel.org/bpf/20220706140204.47926-1-dlan@gentoo.org
| * libbpf: Remove unnecessary usdt_rel_ip assignmentsAndrii Nakryiko2022-07-061-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | Coverity detected that usdt_rel_ip is unconditionally overwritten anyways, so there is no need to unnecessarily initialize it with unused value. Clean this up. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Yonghong Song <yhs@fb.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20220705224818.4026623-4-andrii@kernel.org
| * selftests/bpf: Fix few more compiler warningsAndrii Nakryiko2022-07-063-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When compiling with -O2, GCC detects few problems with selftests/bpf, so fix all of them. Two are real issues (uninitialized err and nums out-of-bounds access), but two other uninitialized variables warnings are due to GCC not being able to prove that variables are indeed initialized under conditions under which they are used. Fix all 4 cases, though. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Yonghong Song <yhs@fb.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20220705224818.4026623-3-andrii@kernel.org
| * selftests/bpf: Fix bogus uninitialized variable warningAndrii Nakryiko2022-07-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When compiling selftests/bpf in optimized mode (-O2), GCC erroneously complains about uninitialized token variable: In file included from network_helpers.c:22: network_helpers.c: In function ‘open_netns’: test_progs.h:355:22: error: ‘token’ may be used uninitialized [-Werror=maybe-uninitialized] 355 | int ___err = libbpf_get_error(___res); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ network_helpers.c:440:14: note: in expansion of macro ‘ASSERT_OK_PTR’ 440 | if (!ASSERT_OK_PTR(token, "malloc token")) | ^~~~~~~~~~~~~ In file included from /data/users/andriin/linux/tools/testing/selftests/bpf/tools/include/bpf/libbpf.h:21, from bpf_util.h:9, from network_helpers.c:20: /data/users/andriin/linux/tools/testing/selftests/bpf/tools/include/bpf/libbpf_legacy.h:113:17: note: by argument 1 of type ‘const void *’ to ‘libbpf_get_error’ declared here 113 | LIBBPF_API long libbpf_get_error(const void *ptr); | ^~~~~~~~~~~~~~~~ cc1: all warnings being treated as errors make: *** [Makefile:522: /data/users/andriin/linux/tools/testing/selftests/bpf/network_helpers.o] Error 1 This is completely bogus becuase libbpf_get_error() doesn't dereference pointer, but the only easy way to silence this is to allocate initialized memory with calloc(). Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Yonghong Song <yhs@fb.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20220705224818.4026623-2-andrii@kernel.org
| * bpftool: Remove zlib feature test from MakefileQuentin Monnet2022-07-051-9/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The feature test to detect the availability of zlib in bpftool's Makefile does not bring much. The library is not optional: it may or may not be required along libbfd for disassembling instructions, but in any case it is necessary to build feature.o or even libbpf, on which bpftool depends. If we remove the feature test, we lose the nicely formatted error message, but we get a compiler error about "zlib.h: No such file or directory", which is equally informative. Let's get rid of the test. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220705200456.285943-1-quentin@isovalent.com
| * Merge branch 'cleanup the legacy probe_event on failed scenario'Andrii Nakryiko2022-07-051-8/+27
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Chuang Wang says: ==================== A potential scenario, when an error is returned after add_uprobe_event_legacy() in perf_event_uprobe_open_legacy(), or bpf_program__attach_perf_event_opts() in bpf_program__attach_uprobe_opts() returns an error, the uprobe_event that was previously created is not cleaned. At the same time, the legacy kprobe_event also have similar problems. With these patches, whenever an error is returned, it ensures that the created kprobe_event/uprobe_event is cleaned. V1 -> v3: - add detail commits - call remove_kprobe_event_legacy() on failed bpf_program__attach_perf_event_opts() v3 -> v4: - cleanup the legacy kprobe_event on failed add/attach_event ==================== Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
| | * libbpf: Cleanup the legacy uprobe_event on failed add/attach_event()Chuang Wang2022-07-051-5/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A potential scenario, when an error is returned after add_uprobe_event_legacy() in perf_event_uprobe_open_legacy(), or bpf_program__attach_perf_event_opts() in bpf_program__attach_uprobe_opts() returns an error, the uprobe_event that was previously created is not cleaned. So, with this patch, when an error is returned, fix this by adding remove_uprobe_event_legacy() Signed-off-by: Chuang Wang <nashuiliang@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220629151848.65587-4-nashuiliang@gmail.com
| | * libbpf: Fix wrong variable used in perf_event_uprobe_open_legacy()Chuang Wang2022-07-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Use "type" as opposed to "err" in pr_warn() after determine_uprobe_perf_type_legacy() returns an error. Signed-off-by: Chuang Wang <nashuiliang@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220629151848.65587-3-nashuiliang@gmail.com
| | * libbpf: Cleanup the legacy kprobe_event on failed add/attach_event()Chuang Wang2022-07-051-4/+14
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before the 0bc11ed5ab60 commit ("kprobes: Allow kprobes coexist with livepatch"), in a scenario where livepatch and kprobe coexist on the same function entry, the creation of kprobe_event using add_kprobe_event_legacy() will be successful, at the same time as a trace event (e.g. /debugfs/tracing/events/kprobe/XXX) will exist, but perf_event_open() will return an error because both livepatch and kprobe use FTRACE_OPS_FL_IPMODIFY. As follows: 1) add a livepatch $ insmod livepatch-XXX.ko 2) add a kprobe using tracefs API (i.e. add_kprobe_event_legacy) $ echo 'p:mykprobe XXX' > /sys/kernel/debug/tracing/kprobe_events 3) enable this kprobe (i.e. sys_perf_event_open) This will return an error, -EBUSY. On Andrii Nakryiko's comment, few error paths in bpf_program__attach_kprobe_opts() that should need to call remove_kprobe_event_legacy(). With this patch, whenever an error is returned after add_kprobe_event_legacy() or bpf_program__attach_perf_event_opts(), this ensures that the created kprobe_event is cleaned. Signed-off-by: Chuang Wang <nashuiliang@gmail.com> Signed-off-by: Jingren Zhou <zhoujingren@didiglobal.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220629151848.65587-2-nashuiliang@gmail.com
| * Merge branch 'Introduce type match support'Andrii Nakryiko2022-07-0513-18/+648
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Daniel Müller says: ==================== This patch set proposes the addition of a new way for performing type queries to BPF. It introduces the "type matches" relation, similar to what is already present with "type exists" (in the form of bpf_core_type_exists). "type exists" performs fairly superficial checking, mostly concerned with whether a type exists in the kernel and is of the same kind (enum/struct/...). Notably, compatibility checks for members of composite types is lacking. The newly introduced "type matches" (bpf_core_type_matches) fills this gap in that it performs stricter checks: compatibility of members and existence of similarly named enum variants is checked as well. E.g., given these definitions: struct task_struct___og { int pid; int tgid; }; struct task_struct___foo { int foo; } 'task_struct___og' would "match" the kernel type 'task_struct', because the members match up, while 'task_struct___foo' would not match, because the kernel's 'task_struct' has no member named 'foo'. More precisely, the "type match" relation is defined as follows (copied from source): - modifiers and typedefs are stripped (and, hence, effectively ignored) - generally speaking types need to be of same kind (struct vs. struct, union vs. union, etc.) - exceptions are struct/union behind a pointer which could also match a forward declaration of a struct or union, respectively, and enum vs. enum64 (see below) Then, depending on type: - integers: - match if size and signedness match - arrays & pointers: - target types are recursively matched - structs & unions: - local members need to exist in target with the same name - for each member we recursively check match unless it is already behind a pointer, in which case we only check matching names and compatible kind - enums: - local variants have to have a match in target by symbolic name (but not numeric value) - size has to match (but enum may match enum64 and vice versa) - function pointers: - number and position of arguments in local type has to match target - for each argument and the return value we recursively check match Enabling this feature requires a new relocation to be made known to the compiler. This is being taken care of for LLVM as part of https://reviews.llvm.org/D126838. If applied, among other things, usage of this functionality could have helped flag issues such as the one discussed here https://lore.kernel.org/all/93a20759600c05b6d9e4359a1517c88e06b44834.camel@fb.com/ earlier. Suggested-by: Andrii Nakryiko <andrii@kernel.org> --- Changelog: v2 -> v3: - renamed btfgen_mark_types_match - covered BTF_KIND_RESTRICT in type match marking logic - used bpf_core_names_match in more places - reworked "behind pointer" logic - added test using live task_struct v1 -> v2: - deduplicated and moved core algorithm into relo_core.c - adjusted bpf_core_names_match to get btf_type passed in - removed some length equality checks before strncmp usage - correctly use kflag from targ_t instead of local_t - added comment for meaning of kflag w/ FWD kind - __u32 -> u32 - handle BTF_KIND_FWD properly in bpftool marking logic - rebased ==================== Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
| | * selftests/bpf: Add type match test against kernel's task_structDaniel Müller2022-07-053-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change extends the existing core_reloc/kernel test to include a type match check of a local task_struct against the kernel's definition -- which we assume to succeed. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220628160127.607834-11-deso@posteo.net
| | * selftests/bpf: Add nested type to type based testsDaniel Müller2022-07-053-20/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change extends the type based tests with another struct type (in addition to a_struct) to check relocations against: a_complex_struct. This type is nested more deeply to provide additional coverage of certain paths in the type match logic. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220628160127.607834-10-deso@posteo.net
| | * selftests/bpf: Add test checking more characteristicsDaniel Müller2022-07-053-0/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds another type-based self-test that specifically aims to test some more characteristics of the TYPE_MATCH logic. Specifically, it covers a few more potential differences between types, such as different orders, enum variant values, and integer signedness. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220628160127.607834-9-deso@posteo.net
| | * selftests/bpf: Add type-match checks to type-based testsDaniel Müller2022-07-053-4/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that we have type-match logic in both libbpf and the kernel, this change adjusts the existing BPF self tests to check this functionality. Specifically, we extend the existing type-based tests to check the previously introduced bpf_core_type_matches macro. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220628160127.607834-8-deso@posteo.net
| | * libbpf: add bpf_core_type_matches() helper macroAndrii Nakryiko2022-07-051-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch finalizes support for the proposed type match relation in libbpf by adding bpf_core_type_matches() macro which emits TYPE_MATCH relocation. Clang support for this relocation was added in [0]. [0] https://reviews.llvm.org/D126838 Signed-off-by: Daniel Müller <deso@posteo.net>¬ Signed-off-by: Andrii Nakryiko <andrii@kernel.org>¬ Link: https://lore.kernel.org/bpf/20220628160127.607834-7-deso@posteo.net¬
| | * bpf, libbpf: Add type match supportDaniel Müller2022-07-054-4/+294
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds support for the proposed type match relation to relo_core where it is shared between userspace and kernel. It plumbs through both kernel-side and libbpf-side support. The matching relation is defined as follows (copy from source): - modifiers and typedefs are stripped (and, hence, effectively ignored) - generally speaking types need to be of same kind (struct vs. struct, union vs. union, etc.) - exceptions are struct/union behind a pointer which could also match a forward declaration of a struct or union, respectively, and enum vs. enum64 (see below) Then, depending on type: - integers: - match if size and signedness match - arrays & pointers: - target types are recursively matched - structs & unions: - local members need to exist in target with the same name - for each member we recursively check match unless it is already behind a pointer, in which case we only check matching names and compatible kind - enums: - local variants have to have a match in target by symbolic name (but not numeric value) - size has to match (but enum may match enum64 and vice versa) - function pointers: - number and position of arguments in local type has to match target - for each argument and the return value we recursively check match Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220628160127.607834-5-deso@posteo.net
| | * bpftool: Honor BPF_CORE_TYPE_MATCHES relocationDaniel Müller2022-07-051-0/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bpftool needs to know about the newly introduced BPF_CORE_TYPE_MATCHES relocation for its 'gen min_core_btf' command to work properly in the present of this relocation. Specifically, we need to make sure to mark types and fields so that they are present in the minimized BTF for "type match" checks to work out. However, contrary to the existing btfgen_record_field_relo, we need to rely on the BTF -- and not the spec -- to find fields. With this change we handle this new variant correctly. The functionality will be tested with follow on changes to BPF selftests, which already run against a minimized BTF created with bpftool. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/20220628160127.607834-3-deso@posteo.net
| | * bpf: Introduce TYPE_MATCH related constants/macrosDaniel Müller2022-07-053-0/+3
| |/ | | | | | | | | | | | | | | | | | | | | | | | | In order to provide type match support we require a new type of relocation which, in turn, requires toolchain support. Recent LLVM/Clang versions support a new value for the last argument to the __builtin_preserve_type_info builtin, for example. With this change we introduce the necessary constants into relevant header files, mirroring what the compiler may support. Signed-off-by: Daniel Müller <deso@posteo.net> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20220628160127.607834-2-deso@posteo.net
| * bpf, samples: Remove AF_XDP samplesMagnus Karlsson2022-07-057-3348/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the AF_XDP samples from samples/bpf/ as they are dependent on the AF_XDP support in libbpf. This support has now been removed in the 1.0 release, so these samples cannot be compiled anymore. Please start to use libxdp instead. It is backwards compatible with the AF_XDP support that was offered in libbpf. New samples can be found in the various xdp-project repositories connected to libxdp and by googling. Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Toke Høiland-Jørgensen <toke@redhat.com> Acked-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Acked-by: Jesper Dangaard Brouer <brouer@redhat.com> Link: https://lore.kernel.org/bpf/20220630093717.8664-1-magnus.karlsson@gmail.com
| * bpftool: Rename "bpftool feature list" into "... feature list_builtins"Quentin Monnet2022-07-053-11/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make it more explicit that the features listed with "bpftool feature list" are known to bpftool, but not necessary available on the system (as opposed to the probed features), rename the "feature list" command into "feature list_builtins". Note that "bpftool feature list" still works as before given that we recognise arguments from their prefixes; but the real name of the subcommand, in particular as displayed in the man page or the interactive help, will now include "_builtins". Since we update the bash completion accordingly, let's also take this chance to redirect error output to /dev/null in the completion script, to avoid displaying unexpected error messages when users attempt to tab-complete. Suggested-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Yonghong Song <yhs@fb.com> Link: https://lore.kernel.org/bpf/20220701093805.16920-1-quentin@isovalent.com
| * bpf: Omit superfluous address family check in __bpf_skc_lookupTobias Klauser2022-07-051-3/+2
| | | | | | | | | | | | | | | | | | | | family is only set to either AF_INET or AF_INET6 based on len. In all other cases we return early. Thus the check against AF_UNSPEC can be omitted. Signed-off-by: Tobias Klauser <tklauser@distanz.ch> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20220630082618.15649-1-tklauser@distanz.ch
| * selftests/bpf: Skip lsm_cgroup when we don't have trampolinesStanislav Fomichev2022-07-011-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With arch_prepare_bpf_trampoline removed on x86: [...] #98/1 lsm_cgroup/functional:SKIP #98 lsm_cgroup:SKIP Summary: 1/0 PASSED, 1 SKIPPED, 0 FAILED Fixes: dca85aac8895 ("selftests/bpf: lsm_cgroup functional test") Signed-off-by: Stanislav Fomichev <sdf@google.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Hao Luo <haoluo@google.com> Link: https://lore.kernel.org/bpf/20220630224203.512815-1-sdf@google.com
| * bpftool: Show also the name of type BPF_OBJ_LINKYafang Shao2022-06-301-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For example, /sys/fs/bpf/maps.debug is a BPF link. When you run `bpftool map show` to show it: Before: $ bpftool map show pinned /sys/fs/bpf/maps.debug Error: incorrect object type: unknown After: $ bpftool map show pinned /sys/fs/bpf/maps.debug Error: incorrect object type: link Signed-off-by: Yafang Shao <laoar.shao@gmail.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/20220629154832.56986-5-laoar.shao@gmail.com
| * selftests/xsk: Destroy BPF resources only when ctx refcount drops to 0Maciej Fijalkowski2022-06-301-5/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, xsk_socket__delete frees BPF resources regardless of ctx refcount. Xdpxceiver has a test to verify whether underlying BPF resources would not be wiped out after closing XSK socket that was bound to interface with other active sockets. From library's xsk part perspective it also means that the internal xsk context is shared and its refcount is bumped accordingly. After a switch to loading XDP prog based on previously opened XSK socket, mentioned xdpxceiver test fails with: not ok 16 [xdpxceiver.c:swap_xsk_resources:1334]: ERROR: 9/"Bad file descriptor which means that in swap_xsk_resources(), xsk_socket__delete() released xskmap which in turn caused a failure of xsk_socket__update_xskmap(). To fix this, when deleting socket, decrement ctx refcount before releasing BPF resources and do so only when refcount dropped to 0 which means there are no more active sockets for this ctx so BPF resources can be freed safely. Fixes: 2f6324a3937f ("libbpf: Support shared umems between queues and devices") Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Magnus Karlsson <magnus.karlsson@intel.com> Link: https://lore.kernel.org/bpf/20220629143458.934337-5-maciej.fijalkowski@intel.com
| * selftests/xsk: Verify correctness of XDP prog attach pointMaciej Fijalkowski2022-06-301-0/+17
| | | | | | | | | | | | | | | | | | | | | | To prevent the case we had previously where for TEST_MODE_SKB, XDP prog was attached in native mode, call bpf_xdp_query() after loading prog and make sure that attach_mode is as expected. Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Magnus Karlsson <magnus.karlsson@intel.com> Link: https://lore.kernel.org/bpf/20220629143458.934337-4-maciej.fijalkowski@intel.com
| * selftests/xsk: Introduce XDP prog load based on existing AF_XDP socketMaciej Fijalkowski2022-06-303-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, xsk_setup_xdp_prog() uses anonymous xsk_socket struct which means that during xsk_create_bpf_link() call, xsk->config.xdp_flags is always 0. This in turn means that from xdpxceiver it is impossible to use xdpgeneric attachment, so since commit 3b22523bca02 ("selftests, xsk: Fix bpf_res cleanup test") we were not testing SKB mode at all. To fix this, introduce a function, called xsk_setup_xdp_prog_xsk(), that will load XDP prog based on the existing xsk_socket, so that xsk context's refcount is correctly bumped and flags from application side are respected. Use this from xdpxceiver side so we get coverage of generic and native XDP program attach points. Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Magnus Karlsson <magnus.karlsson@intel.com> Link: https://lore.kernel.org/bpf/20220629143458.934337-3-maciej.fijalkowski@intel.com
| * selftests/xsk: Avoid bpf_link probe for existing xskMaciej Fijalkowski2022-06-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently bpf_link probe is done for each call of xsk_socket__create(). For cases where xsk context was previously created and current socket creation uses it, has_bpf_link will be overwritten, where it has already been initialized. Optimize this by moving the query to the xsk_create_ctx() so that when xsk_get_ctx() finds a ctx then no further bpf_link probes are needed. Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Magnus Karlsson <magnus.karlsson@intel.com> Link: https://lore.kernel.org/bpf/20220629143458.934337-2-maciej.fijalkowski@intel.com
| * bpftool: Use feature list in bash completionQuentin Monnet2022-06-302-34/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that bpftool is able to produce a list of known program, map, attach types, let's use as much of this as we can in the bash completion file, so that we don't have to expand the list each time a new type is added to the kernel. Also update the relevant test script to remove some checks that are no longer needed. Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Daniel Müller <deso@posteo.net> Link: https://lore.kernel.org/bpf/20220629203637.138944-3-quentin@isovalent.com
| * bpftool: Add feature list (prog/map/link/attach types, helpers)Quentin Monnet2022-06-303-1/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a "bpftool feature list" subcommand to list BPF "features". Contrarily to "bpftool feature probe", this is not about the features available on the system. Instead, it lists all features known to bpftool from compilation time; in other words, all program, map, attach, link types known to the libbpf version in use, and all helpers found in the UAPI BPF header. The first use case for this feature is bash completion: running the command provides a list of types that can be used to produce the list of candidate map types, for example. Now that bpftool uses "standard" names provided by libbpf for the program, map, link, and attach types, having the ability to list these types and helpers could also be useful in scripts to loop over existing items. Sample output: # bpftool feature list prog_types | grep -vw unspec | head -n 6 socket_filter kprobe sched_cls sched_act tracepoint xdp # bpftool -p feature list map_types | jq '.[1]' "hash" # bpftool feature list attach_types | grep '^cgroup_' cgroup_inet_ingress cgroup_inet_egress [...] cgroup_inet_sock_release # bpftool feature list helpers | grep -vw bpf_unspec | wc -l 207 The "unspec" types and helpers are not filtered out by bpftool, so as to remain closer to the enums, and to preserve the indices in the JSON arrays (e.g. "hash" at index 1 == BPF_MAP_TYPE_HASH in map types list). Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Daniel Müller <deso@posteo.net> Link: https://lore.kernel.org/bpf/20220629203637.138944-2-quentin@isovalent.com
| * bpftool: Remove attach_type_name forward declarationTobias Klauser2022-06-301-2/+0
| | | | | | | | | | | | | | | | | | | | | | The attach_type_name definition was removed in commit 1ba5ad36e00f ("bpftool: Use libbpf_bpf_attach_type_str"). Remove its forward declaration in main.h as well. Signed-off-by: Tobias Klauser <tklauser@distanz.ch> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Quentin Monnet <quentin@isovalent.com> Link: https://lore.kernel.org/bpf/20220630093638.25916-1-tklauser@distanz.ch
| * bpftool: Probe for memcg-based accounting before bumping rlimitQuentin Monnet2022-06-291-3/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bpftool used to bump the memlock rlimit to make sure to be able to load BPF objects. After the kernel has switched to memcg-based memory accounting [0] in 5.11, bpftool has relied on libbpf to probe the system for memcg-based accounting support and for raising the rlimit if necessary [1]. But this was later reverted, because the probe would sometimes fail, resulting in bpftool not being able to load all required objects [2]. Here we add a more efficient probe, in bpftool itself. We first lower the rlimit to 0, then we attempt to load a BPF object (and finally reset the rlimit): if the load succeeds, then memcg-based memory accounting is supported. This approach was earlier proposed for the probe in libbpf itself [3], but given that the library may be used in multithreaded applications, the probe could have undesirable consequences if one thread attempts to lock kernel memory while memlock rlimit is at 0. Since bpftool is single-threaded and the rlimit is process-based, this is fine to do in bpftool itself. This probe was inspired by the similar one from the cilium/ebpf Go library [4]. [0] commit 97306be45fbe ("Merge branch 'switch to memcg-based memory accounting'") [1] commit a777e18f1bcd ("bpftool: Use libbpf 1.0 API mode instead of RLIMIT_MEMLOCK") [2] commit 6b4384ff1088 ("Revert "bpftool: Use libbpf 1.0 API mode instead of RLIMIT_MEMLOCK"") [3] https://lore.kernel.org/bpf/20220609143614.97837-1-quentin@isovalent.com/t/#u [4] https://github.com/cilium/ebpf/blob/v0.9.0/rlimit/rlimit.go#L39 Suggested-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Quentin Monnet <quentin@isovalent.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Stanislav Fomichev <sdf@google.com> Acked-by: Yafang Shao <laoar.shao@gmail.com> Link: https://lore.kernel.org/bpf/20220629111351.47699-1-quentin@isovalent.com
| * Merge branch 'bpf: cgroup_sock lsm flavor'Alexei Starovoitov2022-06-2926-166/+1432
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Stanislav Fomichev says: ==================== This series implements new lsm flavor for attaching per-cgroup programs to existing lsm hooks. The cgroup is taken out of 'current', unless the first argument of the hook is 'struct socket'. In this case, the cgroup association is taken out of socket. The attachment looks like a regular per-cgroup attachment: we add new BPF_LSM_CGROUP attach type which, together with attach_btf_id, signals per-cgroup lsm. Behind the scenes, we allocate trampoline shim program and attach to lsm. This program looks up cgroup from current/socket and runs cgroup's effective prog array. The rest of the per-cgroup BPF stays the same: hierarchy, local storage, retval conventions (return 1 == success). Current limitations: * haven't considered sleepable bpf; can be extended later on * not sure the verifier does the right thing with null checks; see latest selftest for details * total of 10 (global) per-cgroup LSM attach points v11: - Martin: address selftest memory & fd leaks - Martin: address moving into root (instead have another temp leaf cgroup) - Martin: move tools/include/uapi/linux/bpf.h change from libbpf patch into 'sync tools' patch v10: - Martin: reword commit message, drop outdated items - Martin: remove rcu_real_lock from __cgroup_bpf_run_lsm_current - Martin: remove CONFIG_BPF_LSM from cgroup_bpf_release - Martin: fix leaking shim reference in bpf_cgroup_link_release - Martin: WARN_ON_ONCE for bpf_trampoline_lookup in bpf_trampoline_unlink_cgroup_shim - Martin: sync tools/include/linux/btf_ids.h - Martin: move progs/flags closer to the places where they are used in __cgroup_bpf_query - Martin: remove sk_clone_security & sctp_bind_connect from bpf_lsm_locked_sockopt_hooks - Martin: try to determine vmlinux btf_id in bpftool - Martin: update tools header in a separate commit - Quentin: do libbpf_find_kernel_btf from the ops that need it - lkp@intel.com: another build failure v9: Major change since last version is the switch to bpf_setsockopt to change the socket state instead of letting the progs poke socket directly. This, in turn, highlights the challenge that we need to care about whether the socket is locked or not when we call bpf_setsockopt. (with my original example selftest, the hooks are running early in the init phase for this not to matter). For now, I've added two btf id lists: * hooks where we know the socket is locked and it's safe to call bpf_setsockopt * hooks where we know the socket is _not_ locked, but the hook works on the socket that's not yet exposed to userspace so it should be safe (for this mode, special new set of bpf_{s,g}etsockopt helpers is added; they don't have sock_owned_by_me check) Going forward, for the rest of the hooks, this might be a good motivation to expand lsm cgroup to support sleeping bpf and allow the callers to lock/unlock sockets or have a new bpf_setsockopt variant that does the locking. - ifdef around cleanup in cgroup_bpf_release - Andrii: a few nits in libbpf patches - Martin: remove unused btf_id_set_index - Martin: bring back refcnt for cgroup_atype - Martin: make __cgroup_bpf_query a bit more readable - Martin: expose dst_prog->aux->attach_btf as attach_btf_obj_id as well - Martin: reorg check_return_code path for BPF_LSM_CGROUP - Martin: return directly from check_helper_call (instead of goto err) - Martin: add note to new warning in check_return_code, print only for void hooks - Martin: remove confusing shim reuse - Martin: use bpf_{s,g}etsockopt instead of poking into socket data - Martin: use CONFIG_CGROUP_BPF in bpf_prog_alloc_no_stats/bpf_prog_free_deferred v8: - CI: fix compile issue - CI: fix broken bpf_cookie - Yonghong: remove __bpf_trampoline_unlink_prog comment - Yonghong: move cgroup_atype around to fill the gap - Yonghong: make bpf_lsm_find_cgroup_shim void - Yonghong: rename regs to args - Yonghong: remove if(current) check - Martin: move refcnt into bpf_link - Martin: move shim management to bpf_link ops - Martin: use cgroup_atype for shim only - Martin: go back to arrays for managing cgroup_atype(s) - Martin: export bpf_obj_id(aux->attach_btf) - Andrii: reorder SEC_DEF("lsm_cgroup+") - Andrii: OPTS_SET instead of OPTS_HAS - Andrii: rename attach_btf_func_id - Andrii: move into 1.0 map v7: - there were a lot of comments last time, hope I didn't forget anything, some of the bigger ones: - Martin: use/extend BTF_SOCK_TYPE_SOCKET - Martin: expose bpf_set_retval - Martin: reject 'return 0' at the verifier for 'void' hooks - Martin: prog_query returns all BPF_LSM_CGROUP, prog_info returns attach_btf_func_id - Andrii: split libbpf changes - Andrii: add field access test to test_progs, not test_verifier (still using asm though) - things that I haven't addressed, stating them here explicitly, let me know if some of these are still problematic: 1. Andrii: exposing only link-based api: seems like the changes to support non-link-based ones are minimal, couple of lines, so seems like it worth having it? 2. Alexei: applying cgroup_atype for all cgroup hooks, not only cgroup lsm: looks a bit harder to apply everywhere that I originally thought; with lsm cgroup, we have a shim_prog pointer where we store cgroup_atype; for non-lsm programs, we don't have a trace program where to store it, so we still need some kind of global table to map from "static" hook to "dynamic" slot. So I'm dropping this "can be easily extended" clause from the description for now. I have converted this whole machinery to an RCU-managed list to remove synchronize_rcu(). - also note that I had to introduce new bpf_shim_tramp_link and moved refcnt there; we need something to manage new bpf_tramp_link v6: - remove active count & stats for shim program (Martin KaFai Lau) - remove NULL/error check for btf_vmlinux (Martin) - don't check cgroup_atype in bpf_cgroup_lsm_shim_release (Martin) - use old_prog (instead of passed one) in __cgroup_bpf_detach (Martin) - make sure attach_btf_id is the same in __cgroup_bpf_replace (Martin) - enable cgroup local storage and test it (Martin) - properly implement prog query and add bpftool & tests (Martin) - prohibit non-shared cgroup storage mode for BPF_LSM_CGROUP (Martin) v5: - __cgroup_bpf_run_lsm_socket remove NULL sock/sk checks (Martin KaFai Lau) - __cgroup_bpf_run_lsm_{socket,current} s/prog/shim_prog/ (Martin) - make sure bpf_lsm_find_cgroup_shim works for hooks without args (Martin) - __cgroup_bpf_attach make sure attach_btf_id is the same when replacing (Martin) - call bpf_cgroup_lsm_shim_release only for LSM_CGROUP (Martin) - drop BPF_LSM_CGROUP from bpf_attach_type_to_tramp (Martin) - drop jited check from cgroup_shim_find (Martin) - new patch to convert cgroup_bpf to hlist_node (Jakub Sitnicki) - new shim flavor for 'struct sock' + list of exceptions (Martin) v4: - fix build when jit is on but syscall is off v3: - add BPF_LSM_CGROUP to bpftool - use simple int instead of refcnt_t (to avoid use-after-free false positive) v2: - addressed build bot failures ==================== Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * selftests/bpf: lsm_cgroup functional testStanislav Fomichev2022-06-293-0/+474
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Functional test that exercises the following: 1. apply default sk_priority policy 2. permit TX-only AF_PACKET socket 3. cgroup attach/detach/replace 4. reusing trampoline shim Signed-off-by: Stanislav Fomichev <sdf@google.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Link: https://lore.kernel.org/r/20220628174314.1216643-12-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * bpftool: implement cgroup tree for BPF_LSM_CGROUPStanislav Fomichev2022-06-291-22/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | $ bpftool --nomount prog loadall $KDIR/tools/testing/selftests/bpf/lsm_cgroup.o /sys/fs/bpf/x $ bpftool cgroup attach /sys/fs/cgroup lsm_cgroup pinned /sys/fs/bpf/x/socket_alloc $ bpftool cgroup attach /sys/fs/cgroup lsm_cgroup pinned /sys/fs/bpf/x/socket_bind $ bpftool cgroup attach /sys/fs/cgroup lsm_cgroup pinned /sys/fs/bpf/x/socket_clone $ bpftool cgroup attach /sys/fs/cgroup lsm_cgroup pinned /sys/fs/bpf/x/socket_post_create $ bpftool cgroup tree CgroupPath ID AttachType AttachFlags Name /sys/fs/cgroup 6 lsm_cgroup socket_post_create bpf_lsm_socket_post_create 8 lsm_cgroup socket_bind bpf_lsm_socket_bind 10 lsm_cgroup socket_alloc bpf_lsm_sk_alloc_security 11 lsm_cgroup socket_clone bpf_lsm_inet_csk_clone $ bpftool cgroup detach /sys/fs/cgroup lsm_cgroup pinned /sys/fs/bpf/x/socket_post_create $ bpftool cgroup tree CgroupPath ID AttachType AttachFlags Name /sys/fs/cgroup 8 lsm_cgroup socket_bind bpf_lsm_socket_bind 10 lsm_cgroup socket_alloc bpf_lsm_sk_alloc_security 11 lsm_cgroup socket_clone bpf_lsm_inet_csk_clone Reviewed-by: Quentin Monnet <quentin@isovalent.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-11-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * libbpf: implement bpf_prog_query_optsStanislav Fomichev2022-06-293-7/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement bpf_prog_query_opts as a more expendable version of bpf_prog_query. Expose new prog_attach_flags and attach_btf_func_id as well: * prog_attach_flags is a per-program attach_type; relevant only for lsm cgroup program which might have different attach_flags per attach_btf_id * attach_btf_func_id is a new field expose for prog_query which specifies real btf function id for lsm cgroup attachments Acked-by: Martin KaFai Lau <kafai@fb.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-10-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * libbpf: add lsm_cgoup_sock typeStanislav Fomichev2022-06-291-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lsm_cgroup/ is the prefix for BPF_LSM_CGROUP. Acked-by: Martin KaFai Lau <kafai@fb.com> Acked-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-9-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * tools/bpf: Sync btf_ids.h to toolsStanislav Fomichev2022-06-293-8/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Has been slowly getting out of sync, let's update it. resolve_btfids usage has been updated to match the header changes. Also bring new parts of tools/include/uapi/linux/bpf.h. Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-8-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * bpf: expose bpf_{g,s}etsockopt to lsm cgroupStanislav Fomichev2022-06-293-7/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I don't see how to make it nice without introducing btf id lists for the hooks where these helpers are allowed. Some LSM hooks work on the locked sockets, some are triggering early and don't grab any locks, so have two lists for now: 1. LSM hooks which trigger under socket lock - minority of the hooks, but ideal case for us, we can expose existing BTF-based helpers 2. LSM hooks which trigger without socket lock, but they trigger early in the socket creation path where it should be safe to do setsockopt without any locks 3. The rest are prohibited. I'm thinking that this use-case might be a good gateway to sleeping lsm cgroup hooks in the future. We can either expose lock/unlock operations (and add tracking to the verifier) or have another set of bpf_setsockopt wrapper that grab the locks and might sleep. Reviewed-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-7-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * bpf: implement BPF_PROG_QUERY for BPF_LSM_CGROUPStanislav Fomichev2022-06-293-32/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have two options: 1. Treat all BPF_LSM_CGROUP the same, regardless of attach_btf_id 2. Treat BPF_LSM_CGROUP+attach_btf_id as a separate hook point I was doing (2) in the original patch, but switching to (1) here: * bpf_prog_query returns all attached BPF_LSM_CGROUP programs regardless of attach_btf_id * attach_btf_id is exported via bpf_prog_info Reviewed-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-6-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * bpf: minimize number of allocated lsm slots per programStanislav Fomichev2022-06-298-24/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previous patch adds 1:1 mapping between all 211 LSM hooks and bpf_cgroup program array. Instead of reserving a slot per possible hook, reserve 10 slots per cgroup for lsm programs. Those slots are dynamically allocated on demand and reclaimed. struct cgroup_bpf { struct bpf_prog_array * effective[33]; /* 0 264 */ /* --- cacheline 4 boundary (256 bytes) was 8 bytes ago --- */ struct hlist_head progs[33]; /* 264 264 */ /* --- cacheline 8 boundary (512 bytes) was 16 bytes ago --- */ u8 flags[33]; /* 528 33 */ /* XXX 7 bytes hole, try to pack */ struct list_head storages; /* 568 16 */ /* --- cacheline 9 boundary (576 bytes) was 8 bytes ago --- */ struct bpf_prog_array * inactive; /* 584 8 */ struct percpu_ref refcnt; /* 592 16 */ struct work_struct release_work; /* 608 72 */ /* size: 680, cachelines: 11, members: 7 */ /* sum members: 673, holes: 1, sum holes: 7 */ /* last cacheline: 40 bytes */ }; Reviewed-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-5-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * bpf: per-cgroup lsm flavorStanislav Fomichev2022-06-2915-20/+498
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allow attaching to lsm hooks in the cgroup context. Attaching to per-cgroup LSM works exactly like attaching to other per-cgroup hooks. New BPF_LSM_CGROUP is added to trigger new mode; the actual lsm hook we attach to is signaled via existing attach_btf_id. For the hooks that have 'struct socket' or 'struct sock' as its first argument, we use the cgroup associated with that socket. For the rest, we use 'current' cgroup (this is all on default hierarchy == v2 only). Note that for some hooks that work on 'struct sock' we still take the cgroup from 'current' because some of them work on the socket that hasn't been properly initialized yet. Behind the scenes, we allocate a shim program that is attached to the trampoline and runs cgroup effective BPF programs array. This shim has some rudimentary ref counting and can be shared between several programs attaching to the same lsm hook from different cgroups. Note that this patch bloats cgroup size because we add 211 cgroup_bpf_attach_type(s) for simplicity sake. This will be addressed in the subsequent patch. Also note that we only add non-sleepable flavor for now. To enable sleepable use-cases, bpf_prog_run_array_cg has to grab trace rcu, shim programs have to be freed via trace rcu, cgroup_bpf.effective should be also trace-rcu-managed + maybe some other changes that I'm not aware of. Reviewed-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-4-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
| | * bpf: convert cgroup_bpf.progs to hlistStanislav Fomichev2022-06-293-35/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This lets us reclaim some space to be used by new cgroup lsm slots. Before: struct cgroup_bpf { struct bpf_prog_array * effective[23]; /* 0 184 */ /* --- cacheline 2 boundary (128 bytes) was 56 bytes ago --- */ struct list_head progs[23]; /* 184 368 */ /* --- cacheline 8 boundary (512 bytes) was 40 bytes ago --- */ u32 flags[23]; /* 552 92 */ /* XXX 4 bytes hole, try to pack */ /* --- cacheline 10 boundary (640 bytes) was 8 bytes ago --- */ struct list_head storages; /* 648 16 */ struct bpf_prog_array * inactive; /* 664 8 */ struct percpu_ref refcnt; /* 672 16 */ struct work_struct release_work; /* 688 32 */ /* size: 720, cachelines: 12, members: 7 */ /* sum members: 716, holes: 1, sum holes: 4 */ /* last cacheline: 16 bytes */ }; After: struct cgroup_bpf { struct bpf_prog_array * effective[23]; /* 0 184 */ /* --- cacheline 2 boundary (128 bytes) was 56 bytes ago --- */ struct hlist_head progs[23]; /* 184 184 */ /* --- cacheline 5 boundary (320 bytes) was 48 bytes ago --- */ u8 flags[23]; /* 368 23 */ /* XXX 1 byte hole, try to pack */ /* --- cacheline 6 boundary (384 bytes) was 8 bytes ago --- */ struct list_head storages; /* 392 16 */ struct bpf_prog_array * inactive; /* 408 8 */ struct percpu_ref refcnt; /* 416 16 */ struct work_struct release_work; /* 432 72 */ /* size: 504, cachelines: 8, members: 7 */ /* sum members: 503, holes: 1, sum holes: 1 */ /* last cacheline: 56 bytes */ }; Suggested-by: Jakub Sitnicki <jakub@cloudflare.com> Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com> Reviewed-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Stanislav Fomichev <sdf@google.com> Link: https://lore.kernel.org/r/20220628174314.1216643-3-sdf@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-27 01:41:31 +0000