* extending ndo_add_rx_vxlan_port
@ 2013-10-27 15:21 Or Gerlitz
2013-10-27 17:25 ` Joseph Gasparakis
0 siblings, 1 reply; 10+ messages in thread
From: Or Gerlitz @ 2013-10-27 15:21 UTC (permalink / raw)
To: Joseph Gasparakis, John Fastabend; +Cc: Yan Burman, netdev, Stephen Hemminger
Hi,
So with commit 53cf527513eed6e7170e9dceacd198f9267171b0 "vxlan: Notify
drivers for listening UDP port changes" drivers that have HW offloads
for vxlan can be notified on which UDP port to listen. Taking this
further, some HW may need to know the multicast address and/or the VNID
used by the vxlan instance/s set above them. In that respect, do we
prefer to extend ndo_add_rx_vxlan_port() or introduce new ndo?
Or.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-27 15:21 extending ndo_add_rx_vxlan_port Or Gerlitz
@ 2013-10-27 17:25 ` Joseph Gasparakis
2013-10-27 20:34 ` Or Gerlitz
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Gasparakis @ 2013-10-27 17:25 UTC (permalink / raw)
To: Or Gerlitz
Cc: Joseph Gasparakis, John Fastabend, Yan Burman, netdev,
Stephen Hemminger
On Sun, 27 Oct 2013, Or Gerlitz wrote:
> Hi,
>
> So with commit 53cf527513eed6e7170e9dceacd198f9267171b0 "vxlan: Notify drivers
> for listening UDP port changes" drivers that have HW offloads for vxlan can be
> notified on which UDP port to listen. Taking this further, some HW may need to
> know the multicast address and/or the VNID used by the vxlan instance/s set
> above them. In that respect, do we prefer to extend ndo_add_rx_vxlan_port() or
> introduce new ndo?
>
> Or.
>
The way this patch works is to notify the drivers when a VXLAN UDP port
comes up or down. This way drivers do not need to do any sort of accounting. As
long as this remains, it sounds fine to me to extend the existing ndo. If
by extedning it, drivers now have to keep track of the udp ports so they
can determine if a notification is for a new port or not, I would much
rather go for introducing a new ndo.
Joseph
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-27 17:25 ` Joseph Gasparakis
@ 2013-10-27 20:34 ` Or Gerlitz
2013-10-27 23:41 ` John Fastabend
2013-10-27 23:44 ` Joseph Gasparakis
0 siblings, 2 replies; 10+ messages in thread
From: Or Gerlitz @ 2013-10-27 20:34 UTC (permalink / raw)
To: Joseph Gasparakis
Cc: Or Gerlitz, John Fastabend, Yan Burman, netdev, Stephen Hemminger
On Sun, Oct 27, 2013 at 7:25 PM, Joseph Gasparakis
<joseph.gasparakis@intel.com> wrote:
>
>
> On Sun, 27 Oct 2013, Or Gerlitz wrote:
>
>> Hi,
>>
>> So with commit 53cf527513eed6e7170e9dceacd198f9267171b0 "vxlan: Notify drivers
>> for listening UDP port changes" drivers that have HW offloads for vxlan can be
>> notified on which UDP port to listen. Taking this further, some HW may need to
>> know the multicast address and/or the VNID used by the vxlan instance/s set
>> above them. In that respect, do we prefer to extend ndo_add_rx_vxlan_port() or
>> introduce new ndo?
>>
>> Or.
>>
>
> The way this patch works is to notify the drivers when a VXLAN UDP port
> comes up or down. This way drivers do not need to do any sort of accounting. As
Could you elaborate why do we want to notify all the netdev instances
in the system (on a certain name-space)
that vxlan instance/s are set to listen on certain UDP port and not
only the device over which the
vxlan device is being set? say the HW can listen limited amount of UDP
ports and few vxlan instances are created
one of top of each "real" netdev in the system and each on different
port. Each netdevice will get all callbacks on port addition
and at some point will start to fail adding them into the HW when the
HW limit is met.
Or.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-27 20:34 ` Or Gerlitz
@ 2013-10-27 23:41 ` John Fastabend
2013-10-27 23:44 ` Joseph Gasparakis
1 sibling, 0 replies; 10+ messages in thread
From: John Fastabend @ 2013-10-27 23:41 UTC (permalink / raw)
To: Or Gerlitz
Cc: Joseph Gasparakis, Or Gerlitz, John Fastabend, Yan Burman, netdev,
Stephen Hemminger
On 10/27/2013 01:34 PM, Or Gerlitz wrote:
> On Sun, Oct 27, 2013 at 7:25 PM, Joseph Gasparakis
> <joseph.gasparakis@intel.com> wrote:
>>
>>
>> On Sun, 27 Oct 2013, Or Gerlitz wrote:
>>
>>> Hi,
>>>
>>> So with commit 53cf527513eed6e7170e9dceacd198f9267171b0 "vxlan: Notify drivers
>>> for listening UDP port changes" drivers that have HW offloads for vxlan can be
>>> notified on which UDP port to listen. Taking this further, some HW may need to
>>> know the multicast address and/or the VNID used by the vxlan instance/s set
>>> above them. In that respect, do we prefer to extend ndo_add_rx_vxlan_port() or
>>> introduce new ndo?
>>>
>>> Or.
>>>
>>
>> The way this patch works is to notify the drivers when a VXLAN UDP port
>> comes up or down. This way drivers do not need to do any sort of accounting. As
>
> Could you elaborate why do we want to notify all the netdev instances
> in the system (on a certain name-space)
> that vxlan instance/s are set to listen on certain UDP port and not
> only the device over which the
> vxlan device is being set? say the HW can listen limited amount of UDP
> ports and few vxlan instances are created
> one of top of each "real" netdev in the system and each on different
> port. Each netdevice will get all callbacks on port addition
> and at some point will start to fail adding them into the HW when the
> HW limit is met.
>
> Or.
> --
The issue is how to determine "the device over which the vlan device is
being set". Because the vxlan device is a layer three device we do not
have a 1:1 binding between vxlan port and l2 net_device. In the
mentioned patches the work around for this was to add the UDP port to
every l2 net_device. I agree it is a bit of a big hammer.
It might be better to only add the UDP port to the default route if it
is specified then add the ndo call to the snooping logic (vxlan_snoop)
and the explicit added netlink fdb entries. This would be a bit more
precise than what we have today.
On the other hand the HW table will eventually be full and even with
this optimization may fail. At this point the user has no way to set
which ports are added and which are failed. Joseph, wanted this to
be automatic so users did not have to configure it so we end up with
this case. The other alternative is to provide a hook into the driver
via rtnetlink I'm not sure its worth the effort though. With this
and a feature flag you could let users manage the device manually.
But is this a problem in practice? My feeling is typically the
number of UDP ports in use is smaller than the number the HW supports.
Thanks,
.John
--
John Fastabend Intel Corporation
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-27 20:34 ` Or Gerlitz
2013-10-27 23:41 ` John Fastabend
@ 2013-10-27 23:44 ` Joseph Gasparakis
2013-10-28 7:35 ` Or Gerlitz
1 sibling, 1 reply; 10+ messages in thread
From: Joseph Gasparakis @ 2013-10-27 23:44 UTC (permalink / raw)
To: Or Gerlitz
Cc: Joseph Gasparakis, Or Gerlitz, John Fastabend, Yan Burman, netdev,
Stephen Hemminger
On Sun, 27 Oct 2013, Or Gerlitz wrote:
> On Sun, Oct 27, 2013 at 7:25 PM, Joseph Gasparakis
> <joseph.gasparakis@intel.com> wrote:
> >
> >
> > On Sun, 27 Oct 2013, Or Gerlitz wrote:
> >
> >> Hi,
> >>
> >> So with commit 53cf527513eed6e7170e9dceacd198f9267171b0 "vxlan: Notify drivers
> >> for listening UDP port changes" drivers that have HW offloads for vxlan can be
> >> notified on which UDP port to listen. Taking this further, some HW may need to
> >> know the multicast address and/or the VNID used by the vxlan instance/s set
> >> above them. In that respect, do we prefer to extend ndo_add_rx_vxlan_port() or
> >> introduce new ndo?
> >>
> >> Or.
> >>
> >
> > The way this patch works is to notify the drivers when a VXLAN UDP port
> > comes up or down. This way drivers do not need to do any sort of accounting. As
>
> Could you elaborate why do we want to notify all the netdev instances
> in the system (on a certain name-space)
> that vxlan instance/s are set to listen on certain UDP port and not
> only the device over which the
> vxlan device is being set? say the HW can listen limited amount of UDP
> ports and few vxlan instances are created
> one of top of each "real" netdev in the system and each on different
> port. Each netdevice will get all callbacks on port addition
> and at some point will start to fail adding them into the HW when the
> HW limit is met.
>
> Or.
>
VXLAN implementation is not done like VLAN. VLANs are stacked on top of
real interfaces and what you are saying makes sense. VXLAN however is
using ip[6]_tunnel_xmit, and this is why we need to notify all the
netdevs in the system that implement the add/del vxlan port ndo (i.e. are
capable of offloading inner csums). In effect all physical "real" netdevs
are candidates to receive/transmit VXLAN traffic, subject to routing
tables changing or not.
Joseph
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-27 23:44 ` Joseph Gasparakis
@ 2013-10-28 7:35 ` Or Gerlitz
2013-10-28 17:59 ` Joseph Gasparakis
0 siblings, 1 reply; 10+ messages in thread
From: Or Gerlitz @ 2013-10-28 7:35 UTC (permalink / raw)
To: Joseph Gasparakis
Cc: Or Gerlitz, John Fastabend, Yan Burman, netdev, Stephen Hemminger
On Mon, Oct 28, 2013 at 1:44 AM, Joseph Gasparakis wrote:
> VXLAN implementation is not done like VLAN. VLANs are stacked on top of
> real interfaces and what you are saying makes sense. VXLAN however is
> using ip[6]_tunnel_xmit, and this is why we need to notify all the
As the name of the ndo you added ndo_add_rx_vxlan_port suggests -- HW
needs that for RX offloads. So basically, what I am saying is: in a
similar manner that we already program the NIC "over which" the vxlan
instance is set to listen on the multicast address which is associated
with that vxlan segement, lets give them a hint that packets arriving
on this group are vxlan ones, so they can use it for programming
steering rules.
> netdevs in the system that implement the add/del vxlan port ndo (i.e. are
> capable of offloading inner csums). In effect all physical "real" netdevs
> are candidates to receive/transmit VXLAN traffic, subject to routing
> tables changing or not.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-28 7:35 ` Or Gerlitz
@ 2013-10-28 17:59 ` Joseph Gasparakis
2013-10-28 18:19 ` Or Gerlitz
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Gasparakis @ 2013-10-28 17:59 UTC (permalink / raw)
To: Or Gerlitz
Cc: Joseph Gasparakis, Or Gerlitz, John Fastabend, Yan Burman, netdev,
Stephen Hemminger
On Mon, 28 Oct 2013, Or Gerlitz wrote:
> On Mon, Oct 28, 2013 at 1:44 AM, Joseph Gasparakis wrote:
>
> > VXLAN implementation is not done like VLAN. VLANs are stacked on top of
> > real interfaces and what you are saying makes sense. VXLAN however is
> > using ip[6]_tunnel_xmit, and this is why we need to notify all the
>
> As the name of the ndo you added ndo_add_rx_vxlan_port suggests -- HW
> needs that for RX offloads. So basically, what I am saying is: in a
> similar manner that we already program the NIC "over which" the vxlan
> instance is set to listen on the multicast address which is associated
> with that vxlan segement, lets give them a hint that packets arriving
> on this group are vxlan ones, so they can use it for programming
> steering rules.
Why don't you write up some code so everyone has something to look at?
Then we can see what makes sense to do in terms of the existing or new
ndos.
>
>
> > netdevs in the system that implement the add/del vxlan port ndo (i.e. are
> > capable of offloading inner csums). In effect all physical "real" netdevs
> > are candidates to receive/transmit VXLAN traffic, subject to routing
> > tables changing or not.
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-28 17:59 ` Joseph Gasparakis
@ 2013-10-28 18:19 ` Or Gerlitz
2013-10-28 20:24 ` Joseph Gasparakis
0 siblings, 1 reply; 10+ messages in thread
From: Or Gerlitz @ 2013-10-28 18:19 UTC (permalink / raw)
To: Joseph Gasparakis
Cc: Or Gerlitz, John Fastabend, Yan Burman, netdev, Stephen Hemminger
On Mon, Oct 28, 2013 at 7:59 PM, Joseph Gasparakis
> On Mon, 28 Oct 2013, Or Gerlitz wrote:
>> On Mon, Oct 28, 2013 at 1:44 AM, Joseph Gasparakis wrote:
>> > VXLAN implementation is not done like VLAN. VLANs are stacked on top of
>> > real interfaces and what you are saying makes sense. VXLAN however is
>> > using ip[6]_tunnel_xmit, and this is why we need to notify all the
>>
>> As the name of the ndo you added ndo_add_rx_vxlan_port suggests -- HW
>> needs that for RX offloads. So basically, what I am saying is: in a
>> similar manner that we already program the NIC "over which" the vxlan
>> instance is set to listen on the multicast address which is associated
>> with that vxlan segement, lets give them a hint that packets arriving
>> on this group are vxlan ones, so they can use it for programming
>> steering rules.
>
> Why don't you write up some code so everyone has something to look at?
> Then we can see what makes sense to do in terms of the existing or new ndos.
sure, code talks, indeed, still, looking on net-next, for the current
ndo there's no in tree consumer unless I miss anything, did I?
Or.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-28 18:19 ` Or Gerlitz
@ 2013-10-28 20:24 ` Joseph Gasparakis
2013-10-28 21:13 ` Joseph Gasparakis
0 siblings, 1 reply; 10+ messages in thread
From: Joseph Gasparakis @ 2013-10-28 20:24 UTC (permalink / raw)
To: Or Gerlitz
Cc: Joseph Gasparakis, Or Gerlitz, John Fastabend, Yan Burman, netdev,
Stephen Hemminger
On Mon, 28 Oct 2013, Or Gerlitz wrote:
> On Mon, Oct 28, 2013 at 7:59 PM, Joseph Gasparakis
> > On Mon, 28 Oct 2013, Or Gerlitz wrote:
> >> On Mon, Oct 28, 2013 at 1:44 AM, Joseph Gasparakis wrote:
>
> >> > VXLAN implementation is not done like VLAN. VLANs are stacked on top of
> >> > real interfaces and what you are saying makes sense. VXLAN however is
> >> > using ip[6]_tunnel_xmit, and this is why we need to notify all the
> >>
> >> As the name of the ndo you added ndo_add_rx_vxlan_port suggests -- HW
> >> needs that for RX offloads. So basically, what I am saying is: in a
> >> similar manner that we already program the NIC "over which" the vxlan
> >> instance is set to listen on the multicast address which is associated
> >> with that vxlan segement, lets give them a hint that packets arriving
> >> on this group are vxlan ones, so they can use it for programming
> >> steering rules.
> >
> > Why don't you write up some code so everyone has something to look at?
> > Then we can see what makes sense to do in terms of the existing or new ndos.
>
> sure, code talks, indeed, still, looking on net-next, for the current
> ndo there's no in tree consumer unless I miss anything, did I?
>
> Or.
>
I don't think you missed anything. My patches for the i40e are currently
in our trees and making their way to netdev. It will be very soon.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: extending ndo_add_rx_vxlan_port
2013-10-28 20:24 ` Joseph Gasparakis
@ 2013-10-28 21:13 ` Joseph Gasparakis
0 siblings, 0 replies; 10+ messages in thread
From: Joseph Gasparakis @ 2013-10-28 21:13 UTC (permalink / raw)
To: Joseph Gasparakis
Cc: Or Gerlitz, Or Gerlitz, John Fastabend, Yan Burman, netdev,
Stephen Hemminger
On Mon, 28 Oct 2013, Joseph Gasparakis wrote:
>
>
> On Mon, 28 Oct 2013, Or Gerlitz wrote:
>
> > On Mon, Oct 28, 2013 at 7:59 PM, Joseph Gasparakis
> > > On Mon, 28 Oct 2013, Or Gerlitz wrote:
> > >> On Mon, Oct 28, 2013 at 1:44 AM, Joseph Gasparakis wrote:
> >
> > >> > VXLAN implementation is not done like VLAN. VLANs are stacked on top of
> > >> > real interfaces and what you are saying makes sense. VXLAN however is
> > >> > using ip[6]_tunnel_xmit, and this is why we need to notify all the
> > >>
> > >> As the name of the ndo you added ndo_add_rx_vxlan_port suggests -- HW
> > >> needs that for RX offloads. So basically, what I am saying is: in a
> > >> similar manner that we already program the NIC "over which" the vxlan
> > >> instance is set to listen on the multicast address which is associated
> > >> with that vxlan segement, lets give them a hint that packets arriving
> > >> on this group are vxlan ones, so they can use it for programming
> > >> steering rules.
> > >
> > > Why don't you write up some code so everyone has something to look at?
> > > Then we can see what makes sense to do in terms of the existing or new ndos.
> >
> > sure, code talks, indeed, still, looking on net-next, for the current
> > ndo there's no in tree consumer unless I miss anything, did I?
> >
> > Or.
> >
>
> I don't think you missed anything. My patches for the i40e are currently
> in our trees and making their way to netdev. It will be very soon.
>
>
Oh! And I also forgot to mention that if you want to see how it
will be used, together with the patch mentioned above that introduces
the ndos, I had added an RFC patch against ixgbe that showcases how the
driver could use them. The RFC patch was just loging the info, but the
actual talking to the HW would be vendor specific anyway.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-10-28 20:56 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-10-27 15:21 extending ndo_add_rx_vxlan_port Or Gerlitz
2013-10-27 17:25 ` Joseph Gasparakis
2013-10-27 20:34 ` Or Gerlitz
2013-10-27 23:41 ` John Fastabend
2013-10-27 23:44 ` Joseph Gasparakis
2013-10-28 7:35 ` Or Gerlitz
2013-10-28 17:59 ` Joseph Gasparakis
2013-10-28 18:19 ` Or Gerlitz
2013-10-28 20:24 ` Joseph Gasparakis
2013-10-28 21:13 ` Joseph Gasparakis
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).