Re: [PATCH v13 net-next 07/11] bpf: verifier (add ability to receive verification log)

[Alexei Starovoitov <ast@plumgrid.com>]

On Wed, Sep 17, 2014 at 12:37 PM, Alexei Starovoitov <ast@plumgrid.com> wrote:
>
>> Hm, thinking out loudly ... perhaps this could be made a library problem.
>> Such that the library which wraps the syscall needs to be aware of a
>> marker where the initial version ends, and if the application doesn't
>> make use of any of the new features, it would just pass in the length up
>> to the marker as size attribute into the syscall. Similarly, if new
>> features are always added to the end of a structure and the library
>> truncates the overall-length after the last used member we might have
>> a chance to load something on older kernels, haven't tried that though.
>
> that's a 3rd option. I think it's cleaner than 2nd, since it solves it
> completely from user space.
> It can even be smarter than that. If this syscall wrapper library
> sees that newer features are used and it can workaround them:
> it can chop size and pass older fields into the older kernel
> and when it returns, do a workaround based on newer fields.

the more I think about 'new user space + old kernel' problem,
the more certain I am that kernel should not try to help
user space, since most of the time it's not going to be enough,
but additional code in kernel would need to be maintained.

syscall commands and size of bpf_attr is the least of problems.
New map_type and prog_type will be added, new helper
functions will be available to programs.
One would think that md5 of uapi/linux/bpf.h would be
enough to say that user app is compatible... In reality,
it's not. The 'state pruning' verifier optimization I've talked
about will not change a single bit in bpf.h, but it will be
able to recognize more programs as safe.
A program developed on a new kernel with more
advanced verifier will load just fine on new kernel, but
this valid program will not load on old kernel, only because
verifier is not smart enough. Now we would need a version
of verifier exposed all the way to user space?!
imo that's too much. I think for eBPF infra kernel
should only guarantee backwards compatibility
(old user space must work with new kernel) and that's it.
That's what this patch is trying to do.
Thoughts?