Re: [GIT PULL] kdbus for 4.1-rc1

[Greg Kroah-Hartman <gregkh@linuxfoundation.org>]

On Mon, Apr 20, 2015 at 11:16:49PM +0200, Richard Weinberger wrote:
> Greg,
> 
> Am 20.04.2015 um 22:56 schrieb Greg Kroah-Hartman:
> >> In which situation on a common Linux system is the current dbus too slow today?
> >> I've never seen a issue like "Oh my system is slow because dbus is
> >> eating too much CPU cycles".
> > 
> > See the original email which explained all of the things we can not do
> > with D-Bus, some of which are due to speed, that can now be done with the
> > kdbus code.
> 
> okay, let's do it together.
> 
> 1. Performance
> You write:
> "DBus is not used for performance sensitive applications because DBus is slow.
> We want to make it fast so we can finally use it for low-latency,
> high-throughput applications."
> 
> Which applications exactly?
> This reads to me like a solution for a non-existing problem.

Anything that uses UDS for large buffers today can switch to using kdbus
for it's data stream as it is faster.  I know the Pulse Audio people
have discussed this, and there are other people as well (Enlightenment
library developers, glib, wayland, etc.)  Without the code being in the
kernel, no project is going to spend the time to convert their codebase
to a feature that isn't accepted.

> 2. Security
> I don't think that you need a 13k piece of code in the kernel to solve
> that issue.

Wait, what?  How can you blow by that requirement by just saying that
this proposal isn't acceptable?  You can't do that, sorry.  Please show
how what we have proposed does not provide the security requirements as
is documented.

> 3. Semantics for apps with heavy data payloads
> 
> Again, sounds like a solution for a non-existing problem.

No, media apps need to share their data somehow, and kdbus provides a
way to do that.  GNOME portals are one such proposed codebase that is
looking to use kdbus for this, and again, so is Pulse Audio and the
other groups listed above.

> 4. "Being in the kernel closes a lot of races which can't be fixed with
>    the current userspace solutions."
> 
> You really need a in-kernel dbus with 13k to solve that?

Do you know of a smaller amount of code to solve this problem?  If so,
wonderful, please show us, but we aren't playing code golf here.  We are
proposing something that is well documented and easy to maintain, while
still being fast and correct.  If it you think this can be done in a
smaller amount of code, please show us where we are doing needless
things in the patches.

> 5. Eavesdropping on the kernel level
> 
> Same here.

same here for what?  Again, please point out the wasted code we have,
and we will be glad to remove it.  Or change it to be smaller.  Odds are
we have done something wrong and it can be reduced, but I don't see it.
Hints are greatly appreciated.

> 6. dbus-daemon is not available during early-boot or shutdown.
> 
> Why do you need dbus in that stage?

You need a way to transfer system state from the initrd to the root
disk, and using D-Bus is a great way to do that.  systemd goes through a
lot of special gyrations to achieve this that can be greatly simplified
by using kdbus.  If the kernel provides the service, you can also ensure
that you have a working D-Bus early on and very late, so that
applications / daemons can properly rely on it for startup/shutdown,
which can not be done today.

> Can you now please start answering questions instead of pointing to random mails?

I've done nothing _but_ answer questions in this email-thread-of-doom.

> >> dbus my have issues which are worth to fix. But moving dbus more or
> >> less in the kernel
> >> seems overkill.
> > 
> > Why do you think so?  How is this code "overkill"?
> 
> Did you try to review it? ;-)
> No really, it is by means no IPC primitive it is a super high level
> in-kernel IPC and not trivial.

Yes, it's not "trivial" as it solves a "non-trivial" problem.  I agree.
But to claim that somehow a much smaller solution can be found without
pointing out what we did wrong with our proposal is just rude.

> >> So, what exactly are these issues and why can't we add new IPC
> >> primitives to Linux which
> >> allow a decent userland dbus?
> > 
> > That's exactly what this patchset does.
> 
> No, it moves dbus into the kernel.

No, it moves part of the dbus model into the kernel.  There are still
other things that are outside of the kernel, as they do not belong
in the kernel.  You need a library in userspace to properly implement
the D-Bus protocol and interaction that userspace programs are needing.
If we had implemented the "whole thing" in the kernel, then that library
would not be needed.

> >> To me kdbus seems much like an ad-hoc solution which is very dbus centric.
> > 
> > Yes it is, but the "dbus centric" thing is a valid model that is quite
> > useful and in use by a lot of programs as it solves a real problem.
> 
> What programs?

Really?  Come on, look at all of the user stack that relies on D-Bus
today (GNOME, KDE), and all of the programs that take advantage of the
libraries that provide D-Bus bindings (Qt, glib, Go, python, perl, etc.)

Yes, it's possible to run a Linux without using D-Bus, but almost no
distro these days does that anymore.  Just look at what is on your
desktop and server today that all major Linux companies support.

As was pointed out, even the tiny IoT devices running Linux are now
using D-Bus, it's everywhere :)

> >> IIRC Alan asked the same question.
> > 
> > Yes, you can build everything off of tiny socket calls, but when you do
> > that, you end up with the D-Bus userspace implementation we have today,
> > with the issues that it has.  By moving portions of that model into the
> > kernel, as is done here, it solves a number of these issues, and allows
> > for a lot more flexibility and things to be done that are impossible
> > with the current model of trying to build on top of tiny ipc functions.
> > 
> > The existing code is much stripped down from what you think of as a
> > D-Bus daemon today, only the exact needed pieces are implemented here.
> > 
> > Do you see anything wrong with the code as is submitted (aside from the
> > issues that Al has pointed out that are being resolved already?)
> 
> The code is fine, the concepts are fishy as pointed out many times in
> this thread.  My point is that we should try hard to fix dbus instead
> of moving it into the kernel.

What needs to be "fixed" in D-Bus that this patchset provide a solution
for?

And no, the concepts are not "fishy" at all.  They solve a real problem,
and need, that programs today rely on.  The concepts have been used for
many decades, and this specific implementation has lasted for over a
decade as it seems to be the best model that has evolved over time.  And
there is no known proposed model out there to succeed it that I know of,
do you know of one?

Because of that, and the thread where the proposed security problems
were agreed not to be a security problem, I don't see a reason anymore
why this code should not be merged.

With the exception of Al's code review, which is being addressed.  But
that's a minor thing, not a major design flaw at all.

thanks,

greg k-h