Re: Port mirroring (RFC)

[Alex Elder <elder@linaro.org>]

On 12/15/21 11:48 AM, Florian Fainelli wrote:
> On 12/14/21 6:47 AM, Alex Elder wrote:
>> I am implementing what amounts to port mirroring functionality
>> for the IPA driver.
>>
>> The IPA hardware isn't exactly a network switch (it's sort of
>> more than that), but it has the ability to supply replicas of
>> packets transferred within it to a special (read only) interface.
>>
>> My plan is to implement this using a new "ipa_mirror" network
>> device, so it could be used with a raw socket to capture the
>> arriving packets.  There currently exists one other netdev,
>> which represents access through a modem to a WWAN network.
>>
>> I would like some advice on how to proceed with this.  I want
>> the result to match "best practice" upstream, and would like
>> this to be as well integrated possible with existing network
>> tools.
>>
>> A few details about the stream of packets that arrive on
>> this hardware interface:
>> - Packet data is truncated if it's larger than a certain size
>> - Each packet is preceded by a fixed-size header describing it
>> - Packets (and their headers) are aggregated into a buffer; i.e.
>>    a single receive might carry a dozen (truncated) packets
> 
> Andrew has already responded, but there are currently sort of two ways
> that mirroring is implemented in upstream supported drivers:
> 
> - for "classic" Ethernet switches mirroring is typically from one port,
> or a set of ports, to another with a choice in whether you want to
> capture ingress, egress, or both towards that port. Because each port is
> a netdev already you just run tcpdump/pcap the way you normally do on
> your Ethernet NIC and get the captured packets. Configuration is via
> offloading the tc_mired action.

I read about switch port mirroring, and it sounded along the
lines of what I'm trying to implement, which is why I suggested
this might be a case of "port mirroring."  That said, it is
not configurable in the same way.  Here, we really have a
dedicated port through which replicated packets arrive.  It is
not possible to just use one of the existing switch ports.  So
it isn't really the same.

> - mlxsw implements devlink traps which is not exaclty designed for
> capturing mirrored packets but more like management events such as why
> the packet was trapped etc. You could however imagine using devlink
> traps for that purpose of capturing mirrored packets in the absence of a
> suitable network device

Just your description tells me this approach is probably not right.
But I'll look into it some more.

> Not sure if this helps, but that is the situation.

Right now I find it useful to hear about anything, even things
I might not ultimately use.

>> Here are a few specific questions, but I would love to get
>> *any* feedback about what I'm doing.
>> - Is representing this as a separate netdev a reasonable
>>    thing to do?
> 
> I would think so, given this allows standard tools to work.

It's reassuring knowing there isn't disagreement.

>> - Is there anything wrong with making a netdev read-only?
>>    (Any packets supplied for transmit would be dropped)
> 
> This looks reasonable.
> 
>> - Are there things I should do so it's clear this interface
>>    does not carry IP traffic (or even UDP, etc.)?
> 
> There were various patches in the past to prevent a network device from
> getting any IP stack to be configured:
> 
> https://yhbt.net/lore/all/20150825232021.GA8482@Alexeis-MacBook-Pro-2.local/t/
> 
> I forgot whether as a driver you can just refuse to have an IP address
> assigned to your network device, AFAIR it requires changes to the
> network stack as proposed in the patch set.

Thanks, I'll follow up with that as well.

>> - Should the driver de-aggregate the received packets, i.e.
>>    separating each into a separate SKB for reading?
> 
> If this is truly a mirroring device, you would expect it to "render" the
> mirrored packets exactly the way they are, maybe add an ethtool private
> option to de-aggregate packets if this helps debugging?

I have some details to work out before I can implement such a
thing, but this would be helpful.

Thanks a lot for your input.

					-Alex

>> I might have *many* more questions, but I'd just like to make
>> sure I'm on the right track, and would like both specific and
>> general suggestions about how to do this the right way.
>>
>> Thanks.
>>                      -Alex
> 
>