Re: Upstream Homa?

[Jiri Pirko <jiri@resnulli.us>]

Sun, Nov 13, 2022 at 07:09:48AM CET, ouster@cs.stanford.edu wrote:
>On Fri, Nov 11, 2022 at 11:25 AM Andrew Lunn <andrew@lunn.ch> wrote:
>>
>> On Fri, Nov 11, 2022 at 10:59:58AM -0800, John Ousterhout wrote:
>> > The netlink and 32-bit kernel issues are new for me; I've done some digging to
>> > learn more, but still have some questions.
>> >
>>
>> > * Is the intent that netlink replaces *all* uses of /proc and ioctl? Homa
>> > currently uses ioctls on sockets for I/O (its APIs aren't sockets-compatible).
>> > It looks like switching to netlink would double the number of system calls that
>> > have to be invoked, which would be unfortunate given Homa's goal of getting the
>> > lowest possible latency. It also looks like netlink might be awkward for
>> > dumping large volumes of kernel data to user space (potential for buffer
>> > overflow?).
>>
>> I've not looked at the actually code, i'm making general comments.
>>
>> netlink, like ioctl, is meant for the control plain, not the data
>> plain. Your statistics should be reported via netlink, for
>> example. netlink is used to configure routes, setup bonding, bridges
>> etc. netlink can also dump large volumes of data, it has no problems
>> dumping the full Internet routing table for example.
>>
>> How you get real packet data between the userspace and kernel space is
>> a different question. You say it is not BSD socket compatible. But
>> maybe there is another existing kernel API which will work? Maybe post
>> what your ideal API looks like and why sockets don't work. Eric
>> Dumazet could give you some ideas about what the kernel has which
>> might do what you need. This is the uAPI point that Stephen raised.
>
>OK, will do. I'm in the middle of a major API refactor, so I'll wait
>until that is
>resolved before pursing this issue more.
>
>> > * By "32 bit kernel problems" are you referring to the lack of atomic 64-bit
>> > operations and using the facilities of u64_stats_sync.h, or is there a more
>> > general issue with 64-bit operations?
>>
>> Those helpers do the real work, and should optimise to pretty much
>> nothing on an 64 bit kernel, but do the right thing on 32 bit kernels.
>>
>> But you are right, the general point is that they are not atomic, so
>> you need to be careful with threads, and any access to a 64 bit values
>> needs to be protected somehow, hopefully in a way that is optimised
>> out on 64bit systems.
>
>Is it acceptable to have features that are only supported on 64-bit kernels?

I don't think so. There are plenty 32bit platforms supported, all should
work there.


>This would be my first choice, since I don't think there will be much interest
>in Homa on 32-bit platforms.
>
>If that's not OK, are there any mechanisms available for helping people
>test on 32-bit platforms? For example, is it possible to configure Linux to
>compile in 32-bit mode so I could test that even on a 64-bit machine
>(I don't have access to a 32-bit machine)?

You can do it easily in emulated environment, like qemu.


>
>-John-