netlink_deliver_tap is broken

netlink_deliver_tap is broken

[Marcel Holtmann]

Hi Daniel,

the netlink tap functionality is not really usable. At least not from a nlmon perspective. It has three fundamental problems.

a) Multicast netlink messages are not delivered to a registered tap when you do not have any member subscribed to the multicast group

b) Multicast netlink messages are delivered multiple times when you have multiple clients subscribed to that multicast group.

c) Unicast netlink messages are filtered out by the client socket filter meaning they never get to the tap

I think all 3 need to be actually fixed to have nlmon being useful for netlink tracing. I did a simple approach by moving netlink_deliver_tap into the actual right places before the filters. However that does not seems to work since it causes sleeping in wrong context.

You might want to have a look at it to make nlmon actually functional complete and not depending on some external multicast membership or socket filters.

Regards

Marcel

Re: netlink_deliver_tap is broken

[Daniel Borkmann]

On 08/11/2014 11:38 PM, Marcel Holtmann wrote:
...
> the netlink tap functionality is not really usable. At least not from a nlmon perspective. It has three fundamental problems.
>
> a) Multicast netlink messages are not delivered to a registered tap when you do not have any member subscribed to the multicast group
>
> b) Multicast netlink messages are delivered multiple times when you have multiple clients subscribed to that multicast group.

The rationale so far I had in mind was that the tap only gets messages
that actually reach another socket/endpoint through netlink. Perhaps
analogous to non-promisc mode ... I think otherwise it's quite hard to
tell if a client actually got a message or not. E.g. it would for some
reason screw up the subscribe, and you could not tell if the netlink
skb actually landed validly in the receive queue. Perhaps we could make
a difference in that behaviour when nlmon is put into promisc mode?

> c) Unicast netlink messages are filtered out by the client socket filter meaning they never get to the tap

Do you mean BPF filter on the packet socket? What filter program do you
have attached and in what scenario?

Re: netlink_deliver_tap is broken

[Marcel Holtmann]

Hi Daniel,

>> the netlink tap functionality is not really usable. At least not from a nlmon perspective. It has three fundamental problems.
>> 
>> a) Multicast netlink messages are not delivered to a registered tap when you do not have any member subscribed to the multicast group
>> 
>> b) Multicast netlink messages are delivered multiple times when you have multiple clients subscribed to that multicast group.
> 
> The rationale so far I had in mind was that the tap only gets messages
> that actually reach another socket/endpoint through netlink. Perhaps
> analogous to non-promisc mode ... I think otherwise it's quite hard to
> tell if a client actually got a message or not. E.g. it would for some
> reason screw up the subscribe, and you could not tell if the netlink
> skb actually landed validly in the receive queue. Perhaps we could make
> a difference in that behaviour when nlmon is put into promisc mode?

this makes no sense to me. If the client screws up, then nobody knows if the message was sent. The kernel will just filter it out. That is pretty much useless.

What I want is that you can use the monitoring tool to check that the multicast message actually was send. That is the only sane way to do this. And since this is a TAP device, that is what it should do. I have no idea why you would put promiscuous mode in the mix here.

The multiple deliveries of the multicast messages is also useless. It makes no sense to see that it reached 10 clients, because I have 10 clients subscribed to the RTNL multicast messages. I can not distinguish these 10 messages since they are all identical once they reach nlmon. It will look like the kernel is misbehaving and sending messages multiple times.

While it does send one to each socket, but why would you care how many times the kernel delivered the exactly same message. I care that the kernel has a multicast message to deliver. If it delivered it 0 or 100 times in the end is of no use.

>> c) Unicast netlink messages are filtered out by the client socket filter meaning they never get to the tap
> 
> Do you mean BPF filter on the packet socket? What filter program do you
> have attached and in what scenario?

Yes, if you open a netlink socket and use SO_FILTER. If the messages never reaches the client socket, you also never see it via nlmon. Again, this is pretty much useless. If I want to use nlmon to see what is actually going on, then I need it to give me all netlink messages going in and out of the kernel.

The current netlink TAP functionality is way too limited to be useful when you try to debug your system and see what clients and kernel is exchanging over netlink.

Regards

Marcel

Re: netlink_deliver_tap is broken

[Daniel Borkmann]

Ok, fair enough, I'll try to address all items/suggestions and get
back to you, most probably including the warning as Stephen suggested
as well if noone else does it earlier.

Thanks,

Daniel