Exceptions in BPF
Kumar Kartikeya Dwivedi posted the BPF exceptions patch set on July 13. The API presented to BPF programs is simple, taking the form of two kfuncs. To raise an exception, a BPF program can call:
void bpf_throw(u64 cookie);
A call to bpf_throw() will cause the program's entire call stack to be unwound, and the program will return to its caller with (by default) a return status of zero; the cookie value is ignored. There is no way for a program to catch an exception called further down the call stack. It is, however, possible to define a function to be called after the call stack has been unwound, but before control is returned to the caller:
void bpf_set_exception_callback(int (*callback)(u64));
The given callback() will be called once unwinding is complete, and will be passed the cookie value given to bpf_throw(); its return value will then be returned to the original caller of the BPF program. There can be only one bpf_set_exception_callback() call in a program; once the callback is set, it cannot be changed.
One might thus be forgiven for thinking that this exception mechanism does not look like it does in other languages supporting the feature, and that bpf_throw() might better be spelled exit(). It clearly is not meant to allow BPF programs to catch and respond to unusual situations. The use case for exceptions, as it turns out, is different and unique to BPF.
BPF programs must, famously, convince the kernel's verifier that they are safe to run before they can be successfully loaded. Doing so requires handling every possible case — even cases that the programmer knows can never happen, but which the verifier is less certain about. So, for example, a BPF function far down the call stack might have to check that an integer value is within a given range, even though the developer knows that it must be, because the verifier does not know that. The check must do something reasonable in response to an out-of-bounds value and, perhaps, return a failure status all the way back up the call chain, all for a case that can never happen.
And, as we all know, developers are never wrong about cases that can never happen.
As Dwivedi described, exceptions are intended to address this problem:
The primary requirement was for implementing assertions within a program, which when untrue still ensure that the program terminates safely. Typically this would require the user to handle the other case, freeing any resources, and returning from a possibly deep callchain back to the kernel. Testing a condition can be used to update the verifier's knowledge about a particular register.
So, in other words, the real reason for exceptions is to provide a mechanism by which the verifier can be informed of invariants that the developer knows about while having an emergency exit mechanism for those times when the developer is wrong. There is a set of assertion macros provided to make this feature easily available in BPF programs. So, for example, a developer will be able to write:
bpf_assert_lt(foo, 256);
This assertion will perform the indicated test and, should it fail, make a call to bpf_throw(). Meanwhile, the verifier will be able to use the knowledge that foo is, indeed, less than 256 as it evaluates the subsequent code.
There is one notable problem still, as described in the changelog to this patch in the series:
For now, bpf_throw invocation fails when lingering resources or locks exist in that path of the program. In a future followup, bpf_throw will be extended to perform frame-by-frame unwinding to release lingering resources for each stack frame, removing this limitation.
Given that the verifier is now counting on bpf_throw() to prevent execution from proceeding past a failed assertion, this seems like a significant limitation indeed. It could probably be used by a sufficiently malicious developer to convince the verifier to accept a program that does something unpleasant. That suggests that implementing the frame-by-frame unwinding will be a prerequisite to getting this work merged.
Both BPF and Rust are intended to make kernel programming safer, but they
take a different approach to the problem. A Rust program will, by default,
panic if any of a large number of things goes wrong. BPF programs,
instead, are intended to be verified as simply lacking that sort of wrong
behavior before they are ever allowed to execute. BPF exceptions can be
seen as an admission that the "prove correctness before loading" approach
has its limits, and that sometimes it is necessary to just throw up your
hands and bail out.
| Index entries for this article | |
|---|---|
| Kernel | BPF |
| Kernel | Releases/6.7 |
Posted Jul 21, 2023 16:25 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (11 responses)
This is not just "feature creep", it's a "feature runaway train at 100mph".
Posted Jul 21, 2023 16:55 UTC (Fri)
by adobriyan (subscriber, #30858)
[Link]
struct kunit_try_catch {
Posted Jul 21, 2023 20:00 UTC (Fri)
by Smon (guest, #104795)
[Link] (2 responses)
Posted Jul 22, 2023 16:00 UTC (Sat)
by NYKevin (subscriber, #129325)
[Link] (1 responses)
The problem is that you need to return all the way up the stack, and therefore you need to return some kind of "we're bailing out" status code to indicate the problem to the caller. But you probably already have return values at many of those call sites, so now you need to transform those return values in some way, probably into an option type or tagged union (or some equivalent). In principle that should be possible, but I don't know if BPF makes it straightforward or performant.
Posted Jul 24, 2023 4:13 UTC (Mon)
by Cyberax (✭ supporter ✭, #52523)
[Link]
With the addition of exceptions, this guarantee is lost.
Not that it mattered either way in practice, but still. BPF is now just adding features without even considering their impact on the overall BPF model.
Posted Jul 22, 2023 8:42 UTC (Sat)
by jezuch (subscriber, #52988)
[Link] (6 responses)
Posted Jul 22, 2023 9:15 UTC (Sat)
by softball (subscriber, #160655)
[Link] (3 responses)
Posted Jul 23, 2023 7:35 UTC (Sun)
by xi0n (guest, #138144)
[Link] (2 responses)
This saying, the mechanism proposed here is so close to Rust panics (and Go panics) that NOT calling it such will only lead to confusion, esp. when “exception” is such an overloaded term already.
Posted Jul 25, 2023 6:59 UTC (Tue)
by taladar (subscriber, #68407)
[Link]
Posted Nov 5, 2023 13:03 UTC (Sun)
by ibukanov (subscriber, #3942)
[Link]
Posted Jul 22, 2023 12:07 UTC (Sat)
by dezgeg (subscriber, #92243)
[Link] (1 responses)
Posted Jul 23, 2023 18:48 UTC (Sun)
by rqosa (subscriber, #24136)
[Link]
Maybe naming it "error" (or something like that) would be a reasonable choice that isn't too similar to those 2 other terms? (Java uses the word "error" in a similar sense in java.lang.Error's name — i.e. for "serious problems that a reasonable application should not try to catch".)
Posted Jul 21, 2023 23:07 UTC (Fri)
by randomguy3 (subscriber, #71063)
[Link] (1 responses)
The difference is primarily scope: as a general purpose language, rust's borrow checker is weaker than bpf's verification (with a few escape hatches provided as well), and its use of asserts/panics more extensive.
Posted Jul 22, 2023 9:07 UTC (Sat)
by softball (subscriber, #160655)
[Link]
Posted Jul 22, 2023 1:30 UTC (Sat)
by walters (subscriber, #7396)
[Link] (2 responses)
I don’t think so. If you’re talking about Rust code using std certainty there can be a lot of implicit OOM panics. But kernel Rust doesn’t use std.
Idiomatic Rust avoids gratuitous unwrap invocations and array accesses, etc. Iterators can often remove implicit bounds checks too. Now, writing probably panic-free code is an active topic.
But I don’t think “if any of a large number of things” is really accurate, on balance.
Posted Jul 22, 2023 4:24 UTC (Sat)
by wahern (subscriber, #37304)
[Link] (1 responses)
True, kernel Rust has its own standard library, but most of it seems to be littered with '#[cfg(not(no_global_oom_handling))]' just like the userspace std library.
Posted Jul 22, 2023 18:28 UTC (Sat)
by walters (subscriber, #7396)
[Link]
(We're now a bit past my relatively superficial knowledge of Linux kernel Rust but...)
I'm pretty sure that's because they don't want a long term fork of alloc.rs and vec.rs etc. The upstream Rust project has this config option I'm pretty sure *precisely* to help enable usage of the upstream battle-tested collections while disabling the APIs that will implicitly panic on OOM.
And the Linux build system does pass "--cfg no_global_oom_handling". And the example Rust code does use try_push() not push(), etc.
Or to say this simply: kernel Rust does not have implicit panics on OOM.
What would certainly be interesting to try to evaluate is how many possible panics there are in any nontrivial kernel Rust code. The classic example is somevec[offset]. I'd expect the number is greater than zero. But are there "a lot"? Are there enough where it *actually* feels like "panic if any of a large number of things goes wrong" is true? I'm doubtful.
Posted Jul 22, 2023 1:32 UTC (Sat)
by geofft (subscriber, #59789)
[Link] (1 responses)
Well... kind of. There are indeed a large number of things that cause a Rust panic (which, for kernelspace Rust code, turns into an oops, not a kernel panic), such as indexing an array out of bounds. But there are also a large number of things that are verified by the Rust compiler, preventing wrong behavior before it can ever execute, and I think that's sort of the selling point of Rust!
A good example is the venerable null pointer. It's not actually a pointer - it's someone using a pointer type to convey there is nothing to point to. In C, you can attempt to "dereference" a null pointer, which is a fundamentally meaningless operation that will lead to incorrect behavior. In Rust, pointer types are defined as non-null, and the standard Optional data type when wrapping a pointer ends up in memory just like a nullable C pointer, but the language prevents you from using it if the value is equal to zero. An Optional type cannot be used directly; you have to use an if or match statement that breaks down the two possibilities, Some actual value or None. (Or you can call a function that does so - such as the standard .unwrap() function which will generate a Rust panic if it's None.)
In other words, at compile time, a Rust program can be verified as never dereferencing a null reference.
This can be generalized in a few ways. Rust also ensures that all references are to valid data, which is what the feared "borrow checker" does: if the compiler can't be convinced that the pointed-to data is still around when you're using the pointer, it will fail to compile. You can also imagine data types that have more than just null as special cases, such as the kernel's ERR_PTR scheme, where small negative values are actually errnos. A function that returns a char * might actually return (char *)-ENOMEM, aka ERR_PTR(-ENOMEM), and expect callers to check IS_ERR on the pointer before using it, with bad consequences if they forget. In Rust this is better defined as a data type that can be either a pointer or an error code (and indeed rust/kernel/error.rs defines it this way): you can't misinterpret an error code as a pointer or vice versa. Currently this isn't stored all in one pointer-sized data type the way it is in C, but it will be soon.
The big difference between Rust and BPF in this context is that Rust treats all of these tools as aids to the programmer, which can be bypassed if needed if you're doing something complex, and BPF treats these as hard requirements and simply refuses to let you do complex things. Rust is trying to eliminate common types of mistakes, but it's not intended to be used in a way where the output is more highly privileged than the input. So it is possible (and quite common for interop with C) to "unsafely" produce a reference from somewhere, effectively telling Rust, hey trust me on this one, this is a valid pointer even though I can't prove it to you. BPF, on the other hand, is all about allowing userspace to load programs into the kernel without the security risk of loading a real kernel module. So it can't have any bypass mechanisms. (And so it needs something like the mechanism in this article to say, I can't prove this assumption to you, so you can just evaluate whether it's true at runtime and bail out of executing the program if it isn't.)
Posted Jul 22, 2023 16:19 UTC (Sat)
by randomguy3 (subscriber, #71063)
[Link]
Posted Jul 23, 2023 2:44 UTC (Sun)
by iteratedlateralus (guest, #102183)
[Link]
Exceptions in BPF
Exceptions in BPF
/* private: internal use only. */
struct kunit *test;
struct completion *try_completion;
int try_result;
kunit_try_catch_func_t try;
kunit_try_catch_func_t catch;
void *context;
};
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Exceptions in BPF
Rust and static safety
Rust and static safety
Exceptions in BPF