L2 switching in igb

L2 switching in igb

[Or Gerlitz]

Hi Alex, Jeff

I see that igb_vmm_control enables L2 switch functionality & replication of
multicast/broadcast packets across multiple pools (it calls both
igb_vmdq_set_loopback/replication_pf with true).

I wanted to check few related things with you:

First, what you think would be the correct method for letting the user control
if she wants the NIC to operate in VEPA mode (e.g forward VF/VF traffic to an
outside bridge and let the later do a Uturn) or to handle VF/VF traffic
internally? maybe it can be part or extension of the kernel DCB netlink API?

second, does igb_vmdq_set_replication_pf cause multicast packets to be
replicated to all VFs? I see that after invoking igb_vmm_control, there's
a call to igb_set_vmolr which sets the "Accept packets matched in MTA"
bit, so I wasn't sure what's is the final result, will the PF flood all
multicast packets and the VF accept only those that have a match in the MTA?

Or.

Re: L2 switching in igb

[Alexander Duyck]

In regards to changing the VEPA mode I haven't given it much thought.
I don't think it would work out as an extension to the DCB netlink API
since the two technologies are mostly unrelated.  The suggestion I
received from Dave and Stephen was to consider an rtnl_link_ops for
configuring the VFs, but I still have issues trying to visualize how
that would work since I don't want the VFs spawning in the
host/hypervisor OS as network devices.

The replication enable allows multicast and broadcast packets to be
replicated across multiple VFs and the PF.  With it disabled the
packet will be forwarded to the first matching VF/PF pool and will not
be delivered to any others.  The Multicast Table Arrary (MTA) is
shared between all of the enabled VFs and the PF.  By setting the
accept packets matched in the MTA bit it essentially enables multicast
addresses for the VF/PF pool being enabled.  Without that bit being
set no multicast packets would be accepted.

Thanks,

Alex

On Thu, Sep 3, 2009 at 2:16 AM, Or Gerlitz<ogerlitz@voltaire.com> wrote:
> Hi Alex, Jeff
>
> I see that igb_vmm_control enables L2 switch functionality & replication of
> multicast/broadcast packets across multiple pools (it calls both
> igb_vmdq_set_loopback/replication_pf with true).
>
> I wanted to check few related things with you:
>
> First, what you think would be the correct method for letting the user control
> if she wants the NIC to operate in VEPA mode (e.g forward VF/VF traffic to an
> outside bridge and let the later do a Uturn) or to handle VF/VF traffic
> internally? maybe it can be part or extension of the kernel DCB netlink API?
>
> second, does igb_vmdq_set_replication_pf cause multicast packets to be
> replicated to all VFs? I see that after invoking igb_vmm_control, there's
> a call to igb_set_vmolr which sets the "Accept packets matched in MTA"
> bit, so I wasn't sure what's is the final result, will the PF flood all
> multicast packets and the VF accept only those that have a match in the MTA?
>
> Or.
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: L2 switching in igb

[Or Gerlitz]

Alexander Duyck wrote:
> The suggestion I received from Dave and Stephen was to consider an rtnl_link_ops for
> configuring the VFs, but I still have issues trying to visualize how that would work since I don't want the VFs spawning in the host/hypervisor OS as network devices.
Note that VEPA mode is a characteristic of the PF, correct? and the PF 
resides in the host kernel. Also, as I wrote you earlier, I do see many 
schemes where a VF spawned in the host kernel IS very useful, and as 
such I'd be happy to continue the discussion on the approach suggested 
by Dave and Stephen, can you provide a pointer? (thanks).

Or.

Re: L2 switching in igb

[Alexander Duyck]

Or Gerlitz wrote:
> Note that VEPA mode is a characteristic of the PF, correct? and the PF 
> resides in the host kernel. Also, as I wrote you earlier, I do see many 
> schemes where a VF spawned in the host kernel IS very useful, and as 
> such I'd be happy to continue the discussion on the approach suggested 
> by Dave and Stephen, can you provide a pointer? (thanks).
> 
> Or.

You are correct, the vSwitch can basically do VEPA by disabling local 
loopback enable bit in the DTXSWC register.  This would force all 
traffic from the PF/VFs out the lan physical port and from the lan 
physical port to the appropriate PF/VFs without doing any switching in 
between PF/VFs.

I am currently thinking what probably needs to be done is to add an 
rtnl_link_ops interface to handle vSwitch configuration that could then 
be applied to the igb netdevs that support VEPA/vSwitch technologies.  A 
subset of that interface could then be dedicated to VF configuration to 
handle things such as spawning VFs, setting the default mac addresses, 
security controls, etc.

Thanks,

Alex

Re: L2 switching in igb

[Or Gerlitz]

Alexander Duyck wrote:
> You are correct, the vSwitch can basically do VEPA by disabling local 
> loopback enable bit in the DTXSWC register.  This would force all 
> traffic from the PF/VFs out the lan physical port and from the lan 
> physical port to the appropriate PF/VFs without doing any switching in 
> between PF/VFs.
To have VEPA support another bit has to be programmed... its the one 
that doesn't let the PF to forward a packet to a VF whose source mac 
matches the one in the packet (e.g multicast sender).

> add an rtnl_link_ops interface to handle vSwitch configuration that 
> could then be applied to the igb netdevs that support VEPA/vSwitch 
> technologies.  A subset of that interface could then be dedicated to 
> VF configuration to handle things such as spawning VFs, setting the 
> default mac addresses, security controls, etc.
Yes, lets do that. I'd like to suggest that a "VF programmable from user 
space" context  will contain a <mac, vlan-id, priority-bits, rate> 
tuple, such that in the absence of vlan tag, the VF driver will "sign" 
the packet (skb) with vlan-id and priority-bits assigned by the admin 
and the PF NIC will mandate that the VF originated traffic will not 
exceed the rate.

Or.


Or.

Re: L2 switching in igb

[Alexander Duyck]

On Mon, Sep 14, 2009 at 3:02 AM, Or Gerlitz <ogerlitz@voltaire.com> wrote:
> To have VEPA support another bit has to be programmed... its the one that
> doesn't let the PF to forward a packet to a VF whose source mac matches the
> one in the packet (e.g multicast sender).

The bit I was referring to not setting would handle that.  By
disabling the DTXSWC local loopback bit the PF will not send anything
to the VFs or visa versa.

> Yes, lets do that. I'd like to suggest that a "VF programmable from user
> space" context  will contain a <mac, vlan-id, priority-bits, rate> tuple,
> such that in the absence of vlan tag, the VF driver will "sign" the packet
> (skb) with vlan-id and priority-bits assigned by the admin and the PF NIC
> will mandate that the VF originated traffic will not exceed the rate.

Well whenever I can get to it I will try to add that support.  In the
meantime I believe there is a BOF session covering "Virtual Ethernet
switch enhancements and configuration" at the Linux Plumbers Conf that
will cover some of this so hopefully we can come up with a solid plan
on how to address this.

Thanks,

Alex

Re: L2 switching in igb

[Or Gerlitz]

Alexander Duyck wrote:
> Well whenever I can get to it I will try to add that support.  In the meantime I believe there is a BOF session covering "Virtual Ethernet switch enhancements and configuration" at the Linux Plumbers Conf that will cover some of this so hopefully we can come up with a solid plan on how to address this.
>   
thanks for the BOF pointer, I see that this takes place next week, so it 
would be nice if someone will drop a note to the list on what is more or 
less the proposed design.

Or.