diff options
| author | Ilya Leoshkevich <iii@linux.ibm.com> | 2020-07-17 18:53:25 +0200 |
|---|---|---|
| committer | Alexei Starovoitov <ast@kernel.org> | 2020-07-21 13:26:25 -0700 |
| commit | 1491b73311a15bb5beeab5d30e03bff761ef6c18 (patch) | |
| tree | fcef3f57edcb12923456221bdbdb6532a589368d | |
| parent | 5fa6974471c5518a50bdd814067508dbcb477251 (diff) | |
| download | linux-1491b73311a15bb5beeab5d30e03bff761ef6c18.tar.gz linux-1491b73311a15bb5beeab5d30e03bff761ef6c18.tar.bz2 linux-1491b73311a15bb5beeab5d30e03bff761ef6c18.zip | |
s390/bpf: Tolerate not converging code shrinking
"BPF_MAXINSNS: Maximum possible literals" unnecessarily falls back to
the interpreter because of failing sanity check in bpf_set_addr. The
problem is that there are a lot of branches that can be shrunk, and
doing so opens up the possibility to shrink even more. This process
does not converge after 3 passes, causing code offsets to change during
the codegen pass, which must never happen.
Fix by inserting nops during codegen pass in order to preserve code
offets.
Fixes: 4e9b4a6883dd ("s390/bpf: Use relative long branches")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200717165326.6786-5-iii@linux.ibm.com
| -rw-r--r-- | arch/s390/net/bpf_jit_comp.c | 27 |
1 files changed, 26 insertions, 1 deletions
diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c index 6b8968f6e207..a78c5b59e1ab 100644 --- a/arch/s390/net/bpf_jit_comp.c +++ b/arch/s390/net/bpf_jit_comp.c @@ -489,6 +489,24 @@ static void save_restore_regs(struct bpf_jit *jit, int op, u32 stack_depth) } while (re <= last); } +static void bpf_skip(struct bpf_jit *jit, int size) +{ + if (size >= 6 && !is_valid_rel(size)) { + /* brcl 0xf,size */ + EMIT6_PCREL_RIL(0xc0f4000000, size); + size -= 6; + } else if (size >= 4 && is_valid_rel(size)) { + /* brc 0xf,size */ + EMIT4_PCREL(0xa7f40000, size); + size -= 4; + } + while (size >= 2) { + /* bcr 0,%0 */ + _EMIT2(0x0700); + size -= 2; + } +} + /* * Emit function prologue * @@ -1503,7 +1521,14 @@ static bool bpf_is_new_addr_sane(struct bpf_jit *jit, int i) */ static int bpf_set_addr(struct bpf_jit *jit, int i) { - if (!bpf_is_new_addr_sane(jit, i)) + int delta; + + if (is_codegen_pass(jit)) { + delta = jit->prg - jit->addrs[i]; + if (delta < 0) + bpf_skip(jit, -delta); + } + if (WARN_ON_ONCE(!bpf_is_new_addr_sane(jit, i))) return -1; jit->addrs[i] = jit->prg; return 0; |