* Re: [RFC] dev->refcnt long term holder
From: Stephen Hemminger @ 2009-11-16 19:02 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David S. Miller, Herbert Xu, Linux Netdev List
In-Reply-To: <4B019381.2010509@gmail.com>
On Mon, 16 Nov 2009 19:01:37 +0100
Eric Dumazet <eric.dumazet@gmail.com> wrote:
> time ip link del eth3.103 ; time ip link del eth3.104 ; time ip link del eth3.105
>
> real 0m0.266s
> user 0m0.000s
> sys 0m0.001s
>
> real 0m0.770s
> user 0m0.000s
> sys 0m0.000s
>
> real 0m1.022s
> user 0m0.000s
> sys 0m0.000s
>
>
> One problem of current schem in vlan dismantle phase is the
> holding of device done by following chain :
>
> vlan_dev_stop() ->
> netif_carrier_off(dev) ->
> linkwatch_fire_event(dev) ->
> dev_hold() ...
>
> And __linkwatch_run_queue() run up to one second later...
>
> Is following patch one way to avoid the problem, or should
> we add a new linkwatch_forgetpro_device(dev) method to immediately
> release the device reference (and unlink device from the list) ?
> (This would probably need a doubly linked list instead of single link list)
Or both..
--
^ permalink raw reply
* Re: [PATCH] ixgbe: Fixing EEH handler to handle more than one error
From: Waskiewicz Jr, Peter P @ 2009-11-16 19:01 UTC (permalink / raw)
To: David Miller
Cc: leitao@linux.vnet.ibm.com, netdev@vger.kernel.org,
Waskiewicz Jr, Peter P, Kirsher, Jeffrey T
In-Reply-To: <20091115.221628.243358818.davem@davemloft.net>
On Sun, 15 Nov 2009, David Miller wrote:
> From: leitao@linux.vnet.ibm.com
> Date: Tue, 10 Nov 2009 13:37:47 -0500
>
> > After commmit 4b77b0a2ba27d64f58f16d8d4d48d8319dda36ff EEH breaks
> > after the second error, since it calls pci_restore_state()
> > but it returns 0, since pci->state_saved is false.
> >
> > So, this patch just call pci_save_state() after pci_restore_state().
> >
> > Signed-off-by: Breno Leitao <leitao@linux.vnet.ibm.com>
>
> Intel folks, are you integrating or looking at this patch?
This patch looks correct and makes sense to me. I'm going to ACK it now,
but we'll still pull it in and test it. If we see any issues, we'll
report them with possible fixes. But at this point, I think this patch is
fine.
Acked-by: Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@intel.com>
^ permalink raw reply
* Re: [PATCH] net/can: add driver for mscan family & mpc52xx_mscan
From: Wolfram Sang @ 2009-11-16 18:44 UTC (permalink / raw)
To: Grant Likely
Cc: socketcan-core-0fE9KPoRgkgATYTw5x5z8w,
netdev-u79uwXL29TY76Z2rM5mHXA, David Miller, Wolfgang Grandegger,
linuxppc-dev-mnsaURCQ41sdnm+yROfE0A
In-Reply-To: <fa686aa40911130939x54ac53f9x173a875a5a4435d3-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
[-- Attachment #1.1: Type: text/plain, Size: 506 bytes --]
Hi Grant,
Wolfgang commented on some points already, I will pick up the other remarks,
just one question:
> > + clk_src = of_get_property(np, "fsl,mscan-clk-src", NULL);
> > + if (clk_src && strcmp(clk_src, "ip") == 0)
>
> Should protect against non-null. strncmp() maybe?
"ip" is null-terminated, or what do you mean?
--
Pengutronix e.K. | Wolfram Sang |
Industrial Linux Solutions | http://www.pengutronix.de/ |
[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]
[-- Attachment #2: Type: text/plain, Size: 188 bytes --]
_______________________________________________
Socketcan-core mailing list
Socketcan-core-0fE9KPoRgkgATYTw5x5z8w@public.gmane.org
https://lists.berlios.de/mailman/listinfo/socketcan-core
^ permalink raw reply
* Re: Those gpio-mdio patches...
From: Lennart Sorensen @ 2009-11-16 18:21 UTC (permalink / raw)
To: David Miller; +Cc: lsorense, netdev
In-Reply-To: <20091113.135908.249908732.davem@davemloft.net>
On Fri, Nov 13, 2009 at 01:59:08PM -0800, David Miller wrote:
> Please don't fix the issue that way. If the routine is returning a
> true or false value, make it return a 'bool' instead of an 'int'.
> That way we don't have to "accept something other than 0 and 1"
Hmm. Well I just tried to fix what was already there. gpio-lib does
NOT return 0 or 1, it returns 0 or 1 shifted by the gpio position in a
gpio register, but soemthing has to convert that to a boolean value.
Making it return a bool doesn't change the fact it needs the '!!' applied
to make it a bool.
I can look at changing the mdio-gpio to using bool instead of integer
where necesary.
--
Len Sorensen
^ permalink raw reply
* [RFC] dev->refcnt long term holder
From: Eric Dumazet @ 2009-11-16 18:01 UTC (permalink / raw)
To: David S. Miller, Herbert Xu, Stephen Hemminger; +Cc: Linux Netdev List
time ip link del eth3.103 ; time ip link del eth3.104 ; time ip link del eth3.105
real 0m0.266s
user 0m0.000s
sys 0m0.001s
real 0m0.770s
user 0m0.000s
sys 0m0.000s
real 0m1.022s
user 0m0.000s
sys 0m0.000s
One problem of current schem in vlan dismantle phase is the
holding of device done by following chain :
vlan_dev_stop() ->
netif_carrier_off(dev) ->
linkwatch_fire_event(dev) ->
dev_hold() ...
And __linkwatch_run_queue() run up to one second later...
Is following patch one way to avoid the problem, or should
we add a new linkwatch_forgetpro_device(dev) method to immediately
release the device reference (and unlink device from the list) ?
(This would probably need a doubly linked list instead of single link list)
Thanks
diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c
index 4198ec5..17216f9 100644
--- a/net/8021q/vlan_dev.c
+++ b/net/8021q/vlan_dev.c
@@ -512,7 +512,6 @@ static int vlan_dev_stop(struct net_device *dev)
if (compare_ether_addr(dev->dev_addr, real_dev->dev_addr))
dev_unicast_delete(real_dev, dev->dev_addr);
- netif_carrier_off(dev);
return 0;
}
^ permalink raw reply related
* RE: Sharing VF device among Xen VMs
From: Rose, Gregory V @ 2009-11-16 17:39 UTC (permalink / raw)
To: Satish Chowdhury, netdev@vger.kernel.org
In-Reply-To: <be5d34890911141149v45ba176ax5705d681def3470e@mail.gmail.com>
>-----Original Message-----
>From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org]
>On Behalf Of Satish Chowdhury
>Sent: Saturday, November 14, 2009 11:50 AM
>To: netdev@vger.kernel.org
>Subject: Sharing VF device among Xen VMs
>
>Hi,
>
>I am trying to verify a situation where VMs share a VF device of Intel
>82576 dual port for data traffic.
>
>Setup:
>Case -1: On a Vt-d machine Xen(xen-1) is installed. On dom0 multiple
>VFs are created for 82576ET dual port card. One of the VF is
>pass-through to a VM. Now, the VM again has Xen (xen-2) installed.
>So, the VMs of Xen-2 have to share the passthrough VF device for data
>traffic.
>
>Will I be able to send data from Xen-2 VMs to external world?
>
>I had issues while creating VMs for Xen2. So, could not do the
>experiment.
>
>Case-2: On Xen-1 itself I loaded igbvf driver. Changed xen
>configuration to make VF as default interface on dom0. Now VMs of
>Xen-1 should share the VF device.
>
>Ping from VF interface on VM and PF ip address works.
>Ping between VMs goes through.
>But, ping from domU to another machine on same network on switch doesn't
>work.
>
>The arp broadcast request goes out through VF interface. But the arp
>reply doesn't reach VF interface, they get routed to PF interface. If
>PF interface to the bridge on dom0 then ping from VMs to external
>machine work.
>
>In my experiment, the arp reply that reaches the NIC, has mac address
>of interface on VM(domU). 82576 performs L2 filtering based on VF MAC
>address. So, packet is not queued to VF interface.
>
>Is it possible to add VMs mac address to L2 filtering pool of the NIC?
>
>Regards,
>-Satish
>--
[Rose, Gregory V]
No, there is no tool for this right now. Extensions to the iproute2 package are under review internally and either a set of proposed patches or an RFC should be forthcoming soon.
You are correct that you need to be able to add the MAC addresses of VMs using the bridge interface to the L2 filter table of the NIC before this setup will work correctly.
For now the 82576 in SR-IOV modes of operation only supports communication among the VFs. You should not try to use the physical function as a bridge interface until the tool I mentioned above becomes available.
- Greg
^ permalink raw reply
* Re: [PATCH net-next 1/2] Phonet: put protocols array under RCU
From: Rémi Denis-Courmont @ 2009-11-16 17:34 UTC (permalink / raw)
To: paulmck; +Cc: netdev, Rémi Denis-Courmont
In-Reply-To: <20091116172617.GC6746@linux.vnet.ibm.com>
Le lundi 16 novembre 2009 19:26:17 Paul E. McKenney, vous avez écrit :
> On Fri, Nov 13, 2009 at 05:01:18PM +0200, Rémi Denis-Courmont wrote:
> > From: Rémi Denis-Courmont <remi.denis-courmont@nokia.com>
> >
> > Signed-off-by: Rémi Denis-Courmont <remi.denis-courmont@nokia.com>
> > ---
> > net/phonet/af_phonet.c | 20 +++++++++++---------
> > 1 files changed, 11 insertions(+), 9 deletions(-)
> >
> > diff --git a/net/phonet/af_phonet.c b/net/phonet/af_phonet.c
> > index 8d3a55b..ed65da2 100644
> > --- a/net/phonet/af_phonet.c
> > +++ b/net/phonet/af_phonet.c
> > @@ -35,7 +35,6 @@
> >
> > /* Transport protocol registration */
> > static struct phonet_protocol *proto_tab[PHONET_NPROTO] __read_mostly;
> > -static DEFINE_SPINLOCK(proto_tab_lock);
> >
> > static struct phonet_protocol *phonet_proto_get(int protocol)
> > {
> > @@ -44,11 +43,11 @@ static struct phonet_protocol *phonet_proto_get(int
> > protocol) if (protocol >= PHONET_NPROTO)
> > return NULL;
> >
> > - spin_lock(&proto_tab_lock);
> > + rcu_read_lock();
> > pp = proto_tab[protocol];
>
> Don't we need an rcu_dereference() in here somewhere?
>
> Perhaps something like the following?
Err yes, we do.
--
Rémi Denis-Courmont
http://www.remlab.net/
^ permalink raw reply
* Re: [PATCH net-next 1/2] Phonet: put protocols array under RCU
From: Paul E. McKenney @ 2009-11-16 17:26 UTC (permalink / raw)
To: Rémi Denis-Courmont; +Cc: netdev, Rémi Denis-Courmont
In-Reply-To: <1258124479-1167-1-git-send-email-remi@remlab.net>
On Fri, Nov 13, 2009 at 05:01:18PM +0200, Rémi Denis-Courmont wrote:
> From: Rémi Denis-Courmont <remi.denis-courmont@nokia.com>
>
> Signed-off-by: Rémi Denis-Courmont <remi.denis-courmont@nokia.com>
> ---
> net/phonet/af_phonet.c | 20 +++++++++++---------
> 1 files changed, 11 insertions(+), 9 deletions(-)
>
> diff --git a/net/phonet/af_phonet.c b/net/phonet/af_phonet.c
> index 8d3a55b..ed65da2 100644
> --- a/net/phonet/af_phonet.c
> +++ b/net/phonet/af_phonet.c
> @@ -35,7 +35,6 @@
>
> /* Transport protocol registration */
> static struct phonet_protocol *proto_tab[PHONET_NPROTO] __read_mostly;
> -static DEFINE_SPINLOCK(proto_tab_lock);
>
> static struct phonet_protocol *phonet_proto_get(int protocol)
> {
> @@ -44,11 +43,11 @@ static struct phonet_protocol *phonet_proto_get(int protocol)
> if (protocol >= PHONET_NPROTO)
> return NULL;
>
> - spin_lock(&proto_tab_lock);
> + rcu_read_lock();
> pp = proto_tab[protocol];
Don't we need an rcu_dereference() in here somewhere?
Perhaps something like the following?
pp = rcu_dereference(proto_tab[protocol]);
Thanx, Paul
> if (pp && !try_module_get(pp->prot->owner))
> pp = NULL;
> - spin_unlock(&proto_tab_lock);
> + rcu_read_unlock();
>
> return pp;
> }
> @@ -439,6 +438,8 @@ static struct packet_type phonet_packet_type __read_mostly = {
> .func = phonet_rcv,
> };
>
> +static DEFINE_MUTEX(proto_tab_lock);
> +
> int __init_or_module phonet_proto_register(int protocol,
> struct phonet_protocol *pp)
> {
> @@ -451,12 +452,12 @@ int __init_or_module phonet_proto_register(int protocol,
> if (err)
> return err;
>
> - spin_lock(&proto_tab_lock);
> + mutex_lock(&proto_tab_lock);
> if (proto_tab[protocol])
> err = -EBUSY;
> else
> - proto_tab[protocol] = pp;
> - spin_unlock(&proto_tab_lock);
> + rcu_assign_pointer(proto_tab[protocol], pp);
> + mutex_unlock(&proto_tab_lock);
>
> return err;
> }
> @@ -464,10 +465,11 @@ EXPORT_SYMBOL(phonet_proto_register);
>
> void phonet_proto_unregister(int protocol, struct phonet_protocol *pp)
> {
> - spin_lock(&proto_tab_lock);
> + mutex_lock(&proto_tab_lock);
> BUG_ON(proto_tab[protocol] != pp);
> - proto_tab[protocol] = NULL;
> - spin_unlock(&proto_tab_lock);
> + rcu_assign_pointer(proto_tab[protocol], NULL);
> + mutex_unlock(&proto_tab_lock);
> + synchronize_rcu();
> proto_unregister(pp->prot);
> }
> EXPORT_SYMBOL(phonet_proto_unregister);
> --
> 1.6.3.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [RFC PATCH] net: add dataref destructor to sk_buff
From: Avi Kivity @ 2009-11-16 17:08 UTC (permalink / raw)
To: David Miller
Cc: gregory.haskins, herbert, ghaskins, mst, alacrityvm-devel,
linux-kernel, netdev
In-Reply-To: <20091113.190438.78469912.davem@davemloft.net>
On 11/14/2009 05:04 AM, David Miller wrote:
> From: Gregory Haskins<gregory.haskins@gmail.com>
> Date: Fri, 13 Nov 2009 20:33:35 -0500
>
>
>> Well, not with respect to the overall protocol, of course not. But with
>> respect to the buffer in question, it _has_ to be. Or am I missing
>> something?
>>
> sendfile() absolutely, and positively, is not.
>
> Any entity can write to the pages being send via sendfile(), at will,
> and those writes will show up in the packet stream if they occur
> before the NIC DMA's the memory backed by those pages into it's
> buffer.
>
> There is zero data synchronization whatsoever, we don't lock the
> pages, we don't block their usage while they are queued up in the
> socket send queue, nothing like that.
>
>
But it must maintain a reference count on the page being dmaed and drop
it only after dma is complete. Otherwise we risk the page being
recycled and arbitrary memory sent out on the wire; and an application
can trivially cause this by truncate()ing a sendfile.
> The user returns long before it every hits the wire and there is zero
> "notification" to the user that the pages in question for the
> sendfile() request are no longer in use.
>
The put_page() is a notification except it doesn't reach the caller.
Gregory's patch (and previous shared info destructor patches) is an
attempt to make it reach the caller, IIUC.
--
error compiling committee.c: too many arguments to function
^ permalink raw reply
* Re: [PATCH 1/2] rps: core implementation
From: Tom Herbert @ 2009-11-16 17:02 UTC (permalink / raw)
To: Andi Kleen; +Cc: David Miller, netdev
In-Reply-To: <87hbt0kaae.fsf@basil.nowhere.org>
>> + case __constant_htons(ETH_P_IPV6):
>> + if (!pskb_may_pull(skb, sizeof(*ip6)))
>> + return -1;
>> +
>> + ip6 = (struct ipv6hdr *) skb->data;
>> + ip_proto = ip6->nexthdr;
>> + addr1 = ip6->saddr.s6_addr32[3];
>> + addr2 = ip6->daddr.s6_addr32[3];
>
> Why only [3] ? Is this future proof?
>
No. But it's same as inet6_ehashfn :-)
>> + for_each_cpu_mask_nr(cpu, __get_cpu_var(rps_remote_softirq_cpus)) {
>> + struct softnet_data *queue = &per_cpu(softnet_data, cpu);
>> + __smp_call_function_single(cpu, &queue->csd, 0);
>
> How do you get around the standard deadlocks with IPI called from
> irq disabled section?
>
What are the standard deadlocks? Looks like __send_remote_softirq
will call __smp_call_function with irq's disabled...
> And why are the interrupts are disabled here anyways?
>
Protects rps_remote_softirq_cpus.
> It's a standard pet peeve of me, but it's quite unlikely you'll
> get any useful entropy at this time of kernel startup.
>
> Normally it's always the same.
>
Would it make sense to just use skb_tx_hashrnd for the receive hash
key also (renaming it to be more general)?
>> + if (err)
>> + return err;
>> +
>> + rtnl_lock();
>
> It seems weird to do user parsing while holding that lock.
> Better first set up and allocate and then finally initialize global state.
Yes. We could build a new map each time and then insert it into the
device structure using an rcu lock (which I hope would be sufficient
locking)
Thanks,
Tom
^ permalink raw reply
* Re: [net-next-2.6 PATCH v2] allow access to sysfs_groups member
From: Stephen Hemminger @ 2009-11-16 16:46 UTC (permalink / raw)
To: Kurt Van Dijck; +Cc: Oliver Hartkopp, Wolfgang Grandegger, netdev
In-Reply-To: <20091114165408.GA283@e-circ.dyndns.org>
On Sat, 14 Nov 2009 17:54:09 +0100
Kurt Van Dijck <kurt.van.dijck@eia.be> wrote:
> On Fri, Nov 13, 2009 at 02:27:38PM -0800, Stephen Hemminger wrote:
> > On Fri, 13 Nov 2009 11:51:57 +0100
> > Kurt Van Dijck <kurt.van.dijck@eia.be> wrote:
> [...]
> > EXPORT_SYMBOL_GPL() for all device/sysfs related stuff.
> Ok, no problem
> >
> > Also, need some way to BUG() if this is done after device has
> > been registered.
> Ok. I gave it a try. I _think_ I did it write, but a second look would
> not harm :-). I was not able to run a real test yet.
> >
> > Another way to add sub-directories which is what bridge, bonding,
> > and others do is to use another kobject. I think this is what you
> > want for the case of two CAN objects under one netdevice.
> In fact, I wanted to add this for cards that have multiple seperate CAN
> network device, combined on 1 PCI or PCMCIA device.
> In the 'add network' uevent, a udev rule could find properties of the
> card (device/ symlink). Right now, there is no way to tell if a network
> device is bus 1 or 2 on the card. in CAN, there's no such thing as a MAC
> address.
> I encountered this issue with a softing CAN card (not yet in mainline),
> and there are other drivers in the socketCAN queue that have the same
> problem.
>
> The ethernet cards with multiple busses (that I've seen yet :-) )
> combine multiple PCI devices on 1 card, and identification of the
> 'instance on the card' can happen with the sysfs properties delivered by
> the PCI bus.
> CAN devices with multiple busses typically are combined all together on
> 1 single device on the PCI or PCMCIA bus.
>
> So, I wanted to add a 'channel' property in /sys/class/net/canX, which
> could indicate the instance on the device. Such property must be
> installed by the driver, not the bus the device is on.
> This patch allows me to have this channel property present at the moment
> of the uevent.
>
> I can imagine other subsystems may benefit from this too.
Okay by me.
Acked-by: Stephen Hemminger <shemminger@vyatta.com>
--
^ permalink raw reply
* Re: [PATCH 1/2] rps: core implementation
From: Tom Herbert @ 2009-11-16 16:43 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20091116.031914.65020185.davem@davemloft.net>
On Mon, Nov 16, 2009 at 3:19 AM, David Miller <davem@davemloft.net> wrote:
> From: Tom Herbert <therbert@google.com>
> Date: Tue, 10 Nov 2009 22:53:17 -0800
>
>> + /* Schedule NAPI for backlog device */
>> + if (napi_schedule_prep(&queue->backlog)) {
>> + if (cpu != smp_processor_id()) {
>> + cpu_set(cpu,
>> + get_cpu_var(rps_remote_softirq_cpus));
>> + __raise_softirq_irqoff(NET_RPS_SOFTIRQ);
>> + } else
>> + __napi_schedule(&queue->backlog);
>> + }
>> + goto enqueue;
>
> {,__}send_remote_softirq() doesn't work? :-)
>
NET_RPS_SOFTIRQ is intended to provide coalescing of IPIs.
send_remote_softirq could be used, but we would also need to get the
napi structure on the remote cpu poll list so that would probably need
to be protected by locks (something like __napi_schedule_oncpu could
be defined). Would this be better to do?
^ permalink raw reply
* Re: [PATCH] ifb: add multi-queue support
From: Stephen Hemminger @ 2009-11-16 16:39 UTC (permalink / raw)
To: xiaosuo; +Cc: David S. Miller, Patrick McHardy, Eric Dumazet, Tom Herbert,
netdev
In-Reply-To: <4AFCE273.7010901@gmail.com>
On Fri, 13 Nov 2009 12:37:07 +0800
Changli Gao <xiaosuo@gmail.com> wrote:
> ifb: add multi-queue support
>
> Add multi-queue support, and one kernel thread is created for per queue.
> It can used to emulate multi-queue NIC in software, and distribute work
> among CPUs.
My $.02 is that receive packet steering RPS should be done generically at
receive layer. Then all the CPU, mapping and configuration issues can be
done once, not just for IFB, Bridge, VLAN, ... The number of users of IFB
is small, and setup is complex. Steering packets in IFB is optimizing only
a rarely used corner.
Layered link services like IFB need to be multi-threaded lockless to maintain
the advantages of multi-queue and RPS.
--
^ permalink raw reply
* Re: pull request: wireless-2.6 2009-11-16
From: John W. Linville @ 2009-11-16 16:32 UTC (permalink / raw)
To: davem; +Cc: linux-wireless, netdev, linux-kernel
In-Reply-To: <20091116162655.GD14410@tuxdriver.com>
On Mon, Nov 16, 2009 at 11:26:55AM -0500, John W. Linville wrote:
> activity in ipw2200, so it the revert should be set. Plus, I've
> tested the revert locally.
s/so it the revert should be set/so the revert should be safe/
John
--
John W. Linville Someday the world will need a hero, and you
linville@tuxdriver.com might be all we have. Be ready.
^ permalink raw reply
* pull request: wireless-2.6 2009-11-16
From: John W. Linville @ 2009-11-16 16:26 UTC (permalink / raw)
To: davem-fT/PcQaiUtIeIZ0/mPfg9Q
Cc: linux-wireless-u79uwXL29TY76Z2rM5mHXA,
netdev-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Dave,
One last(?) set of fixes intended for 2.6.32...
The biggest is a revert of the patch that added cfg80211 support
to ipw2200. It was ill conceived and inadvertantly caused problems
for ipw2100. There are fixes available now, but they are just as
big as the revert -- I'll restage them for 2.6.33. There is little
activity in ipw2200, so it the revert should be set. Plus, I've
tested the revert locally.
There is also a warning fix for where p54usb was using stack memory for
DMA. And finally, two small MAINTAINERS fixes.
Please let me know if there are problems!
Thanks,
John
---
Individual patches are available here:
http://www.kernel.org/pub/linux/kernel/people/linville/wireless-2.6/
---
The following changes since commit d01032e4fd33110f9f3a085a36cb819c1dfc5827:
Wey-Yi Guy (1):
iwlwifi: Fix issue on file transfer stalled in HT mode
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/linville/wireless-2.6.git master
Gertjan van Wingerde (1):
rt2x00: Update MAINTAINERS
Joe Perches (1):
MAINTAINERS: RFKILL - Fix pattern entry missing colon
John W. Linville (1):
Revert "libipw: initiate cfg80211 API conversion"
Larry Finger (1):
p54usb: Remove DMA buffer from stack
MAINTAINERS | 3 +-
drivers/net/wireless/ipw2x00/ipw2100.c | 6 +-
drivers/net/wireless/ipw2x00/ipw2200.c | 141 +++++---------------------
drivers/net/wireless/ipw2x00/libipw.h | 8 +-
drivers/net/wireless/ipw2x00/libipw_module.c | 42 +-------
drivers/net/wireless/p54/p54usb.c | 10 ++-
6 files changed, 45 insertions(+), 165 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index b37e0cd..887b50b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4281,6 +4281,7 @@ F: drivers/video/aty/aty128fb.c
RALINK RT2X00 WIRELESS LAN DRIVER
P: rt2x00 project
M: Ivo van Doorn <IvDoorn-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
+M: Gertjan van Wingerde <gwingerde-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
L: linux-wireless-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
L: users-poMEt7QlJxcwIE2E9O76wjtx2kNaKg5H@public.gmane.org (moderated for non-subscribers)
W: http://rt2x00.serialmonkey.com/
@@ -4368,7 +4369,7 @@ RFKILL
M: Johannes Berg <johannes-cdvu00un1VgdHxzADdlk8Q@public.gmane.org>
L: linux-wireless-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
S: Maintained
-F Documentation/rfkill.txt
+F: Documentation/rfkill.txt
F: net/rfkill/
RISCOM8 DRIVER
diff --git a/drivers/net/wireless/ipw2x00/ipw2100.c b/drivers/net/wireless/ipw2x00/ipw2100.c
index 240cff1..6e2fc0c 100644
--- a/drivers/net/wireless/ipw2x00/ipw2100.c
+++ b/drivers/net/wireless/ipw2x00/ipw2100.c
@@ -6029,7 +6029,7 @@ static struct net_device *ipw2100_alloc_device(struct pci_dev *pci_dev,
struct ipw2100_priv *priv;
struct net_device *dev;
- dev = alloc_ieee80211(sizeof(struct ipw2100_priv), 0);
+ dev = alloc_ieee80211(sizeof(struct ipw2100_priv));
if (!dev)
return NULL;
priv = libipw_priv(dev);
@@ -6342,7 +6342,7 @@ static int ipw2100_pci_init_one(struct pci_dev *pci_dev,
sysfs_remove_group(&pci_dev->dev.kobj,
&ipw2100_attribute_group);
- free_ieee80211(dev, 0);
+ free_ieee80211(dev);
pci_set_drvdata(pci_dev, NULL);
}
@@ -6400,7 +6400,7 @@ static void __devexit ipw2100_pci_remove_one(struct pci_dev *pci_dev)
if (dev->base_addr)
iounmap((void __iomem *)dev->base_addr);
- free_ieee80211(dev, 0);
+ free_ieee80211(dev);
}
pci_release_regions(pci_dev);
diff --git a/drivers/net/wireless/ipw2x00/ipw2200.c b/drivers/net/wireless/ipw2x00/ipw2200.c
index 8d58e6e..0d30a9e 100644
--- a/drivers/net/wireless/ipw2x00/ipw2200.c
+++ b/drivers/net/wireless/ipw2x00/ipw2200.c
@@ -103,25 +103,6 @@ static int antenna = CFG_SYS_ANTENNA_BOTH;
static int rtap_iface = 0; /* def: 0 -- do not create rtap interface */
#endif
-static struct ieee80211_rate ipw2200_rates[] = {
- { .bitrate = 10 },
- { .bitrate = 20, .flags = IEEE80211_RATE_SHORT_PREAMBLE },
- { .bitrate = 55, .flags = IEEE80211_RATE_SHORT_PREAMBLE },
- { .bitrate = 110, .flags = IEEE80211_RATE_SHORT_PREAMBLE },
- { .bitrate = 60 },
- { .bitrate = 90 },
- { .bitrate = 120 },
- { .bitrate = 180 },
- { .bitrate = 240 },
- { .bitrate = 360 },
- { .bitrate = 480 },
- { .bitrate = 540 }
-};
-
-#define ipw2200_a_rates (ipw2200_rates + 4)
-#define ipw2200_num_a_rates 8
-#define ipw2200_bg_rates (ipw2200_rates + 0)
-#define ipw2200_num_bg_rates 12
#ifdef CONFIG_IPW2200_QOS
static int qos_enable = 0;
@@ -8673,6 +8654,24 @@ static int ipw_sw_reset(struct ipw_priv *priv, int option)
*
*/
+static int ipw_wx_get_name(struct net_device *dev,
+ struct iw_request_info *info,
+ union iwreq_data *wrqu, char *extra)
+{
+ struct ipw_priv *priv = libipw_priv(dev);
+ mutex_lock(&priv->mutex);
+ if (priv->status & STATUS_RF_KILL_MASK)
+ strcpy(wrqu->name, "radio off");
+ else if (!(priv->status & STATUS_ASSOCIATED))
+ strcpy(wrqu->name, "unassociated");
+ else
+ snprintf(wrqu->name, IFNAMSIZ, "IEEE 802.11%c",
+ ipw_modes[priv->assoc_request.ieee_mode]);
+ IPW_DEBUG_WX("Name: %s\n", wrqu->name);
+ mutex_unlock(&priv->mutex);
+ return 0;
+}
+
static int ipw_set_channel(struct ipw_priv *priv, u8 channel)
{
if (channel == 0) {
@@ -9972,7 +9971,7 @@ static int ipw_wx_sw_reset(struct net_device *dev,
/* Rebase the WE IOCTLs to zero for the handler array */
#define IW_IOCTL(x) [(x)-SIOCSIWCOMMIT]
static iw_handler ipw_wx_handlers[] = {
- IW_IOCTL(SIOCGIWNAME) = (iw_handler) cfg80211_wext_giwname,
+ IW_IOCTL(SIOCGIWNAME) = ipw_wx_get_name,
IW_IOCTL(SIOCSIWFREQ) = ipw_wx_set_freq,
IW_IOCTL(SIOCGIWFREQ) = ipw_wx_get_freq,
IW_IOCTL(SIOCSIWMODE) = ipw_wx_set_mode,
@@ -11416,100 +11415,16 @@ static void ipw_bg_down(struct work_struct *work)
/* Called by register_netdev() */
static int ipw_net_init(struct net_device *dev)
{
- int i, rc = 0;
struct ipw_priv *priv = libipw_priv(dev);
- const struct libipw_geo *geo = libipw_get_geo(priv->ieee);
- struct wireless_dev *wdev = &priv->ieee->wdev;
mutex_lock(&priv->mutex);
if (ipw_up(priv)) {
- rc = -EIO;
- goto out;
- }
-
- memcpy(wdev->wiphy->perm_addr, priv->mac_addr, ETH_ALEN);
-
- /* fill-out priv->ieee->bg_band */
- if (geo->bg_channels) {
- struct ieee80211_supported_band *bg_band = &priv->ieee->bg_band;
-
- bg_band->band = IEEE80211_BAND_2GHZ;
- bg_band->n_channels = geo->bg_channels;
- bg_band->channels =
- kzalloc(geo->bg_channels *
- sizeof(struct ieee80211_channel), GFP_KERNEL);
- /* translate geo->bg to bg_band.channels */
- for (i = 0; i < geo->bg_channels; i++) {
- bg_band->channels[i].band = IEEE80211_BAND_2GHZ;
- bg_band->channels[i].center_freq = geo->bg[i].freq;
- bg_band->channels[i].hw_value = geo->bg[i].channel;
- bg_band->channels[i].max_power = geo->bg[i].max_power;
- if (geo->bg[i].flags & LIBIPW_CH_PASSIVE_ONLY)
- bg_band->channels[i].flags |=
- IEEE80211_CHAN_PASSIVE_SCAN;
- if (geo->bg[i].flags & LIBIPW_CH_NO_IBSS)
- bg_band->channels[i].flags |=
- IEEE80211_CHAN_NO_IBSS;
- if (geo->bg[i].flags & LIBIPW_CH_RADAR_DETECT)
- bg_band->channels[i].flags |=
- IEEE80211_CHAN_RADAR;
- /* No equivalent for LIBIPW_CH_80211H_RULES,
- LIBIPW_CH_UNIFORM_SPREADING, or
- LIBIPW_CH_B_ONLY... */
- }
- /* point at bitrate info */
- bg_band->bitrates = ipw2200_bg_rates;
- bg_band->n_bitrates = ipw2200_num_bg_rates;
-
- wdev->wiphy->bands[IEEE80211_BAND_2GHZ] = bg_band;
- }
-
- /* fill-out priv->ieee->a_band */
- if (geo->a_channels) {
- struct ieee80211_supported_band *a_band = &priv->ieee->a_band;
-
- a_band->band = IEEE80211_BAND_5GHZ;
- a_band->n_channels = geo->a_channels;
- a_band->channels =
- kzalloc(geo->a_channels *
- sizeof(struct ieee80211_channel), GFP_KERNEL);
- /* translate geo->bg to a_band.channels */
- for (i = 0; i < geo->a_channels; i++) {
- a_band->channels[i].band = IEEE80211_BAND_2GHZ;
- a_band->channels[i].center_freq = geo->a[i].freq;
- a_band->channels[i].hw_value = geo->a[i].channel;
- a_band->channels[i].max_power = geo->a[i].max_power;
- if (geo->a[i].flags & LIBIPW_CH_PASSIVE_ONLY)
- a_band->channels[i].flags |=
- IEEE80211_CHAN_PASSIVE_SCAN;
- if (geo->a[i].flags & LIBIPW_CH_NO_IBSS)
- a_band->channels[i].flags |=
- IEEE80211_CHAN_NO_IBSS;
- if (geo->a[i].flags & LIBIPW_CH_RADAR_DETECT)
- a_band->channels[i].flags |=
- IEEE80211_CHAN_RADAR;
- /* No equivalent for LIBIPW_CH_80211H_RULES,
- LIBIPW_CH_UNIFORM_SPREADING, or
- LIBIPW_CH_B_ONLY... */
- }
- /* point at bitrate info */
- a_band->bitrates = ipw2200_a_rates;
- a_band->n_bitrates = ipw2200_num_a_rates;
-
- wdev->wiphy->bands[IEEE80211_BAND_5GHZ] = a_band;
- }
-
- set_wiphy_dev(wdev->wiphy, &priv->pci_dev->dev);
-
- /* With that information in place, we can now register the wiphy... */
- if (wiphy_register(wdev->wiphy)) {
- rc = -EIO;
- goto out;
+ mutex_unlock(&priv->mutex);
+ return -EIO;
}
-out:
mutex_unlock(&priv->mutex);
- return rc;
+ return 0;
}
/* PCI driver stuff */
@@ -11640,7 +11555,7 @@ static int ipw_prom_alloc(struct ipw_priv *priv)
if (priv->prom_net_dev)
return -EPERM;
- priv->prom_net_dev = alloc_ieee80211(sizeof(struct ipw_prom_priv), 1);
+ priv->prom_net_dev = alloc_ieee80211(sizeof(struct ipw_prom_priv));
if (priv->prom_net_dev == NULL)
return -ENOMEM;
@@ -11659,7 +11574,7 @@ static int ipw_prom_alloc(struct ipw_priv *priv)
rc = register_netdev(priv->prom_net_dev);
if (rc) {
- free_ieee80211(priv->prom_net_dev, 1);
+ free_ieee80211(priv->prom_net_dev);
priv->prom_net_dev = NULL;
return rc;
}
@@ -11673,7 +11588,7 @@ static void ipw_prom_free(struct ipw_priv *priv)
return;
unregister_netdev(priv->prom_net_dev);
- free_ieee80211(priv->prom_net_dev, 1);
+ free_ieee80211(priv->prom_net_dev);
priv->prom_net_dev = NULL;
}
@@ -11701,7 +11616,7 @@ static int __devinit ipw_pci_probe(struct pci_dev *pdev,
struct ipw_priv *priv;
int i;
- net_dev = alloc_ieee80211(sizeof(struct ipw_priv), 0);
+ net_dev = alloc_ieee80211(sizeof(struct ipw_priv));
if (net_dev == NULL) {
err = -ENOMEM;
goto out;
@@ -11849,7 +11764,7 @@ static int __devinit ipw_pci_probe(struct pci_dev *pdev,
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
out_free_ieee80211:
- free_ieee80211(priv->net_dev, 0);
+ free_ieee80211(priv->net_dev);
out:
return err;
}
@@ -11916,7 +11831,7 @@ static void __devexit ipw_pci_remove(struct pci_dev *pdev)
pci_release_regions(pdev);
pci_disable_device(pdev);
pci_set_drvdata(pdev, NULL);
- free_ieee80211(priv->net_dev, 0);
+ free_ieee80211(priv->net_dev);
free_firmware();
}
diff --git a/drivers/net/wireless/ipw2x00/libipw.h b/drivers/net/wireless/ipw2x00/libipw.h
index bf45391..1e334ff 100644
--- a/drivers/net/wireless/ipw2x00/libipw.h
+++ b/drivers/net/wireless/ipw2x00/libipw.h
@@ -31,7 +31,6 @@
#include <linux/ieee80211.h>
#include <net/lib80211.h>
-#include <net/cfg80211.h>
#define LIBIPW_VERSION "git-1.1.13"
@@ -784,15 +783,12 @@ struct libipw_geo {
struct libipw_device {
struct net_device *dev;
- struct wireless_dev wdev;
struct libipw_security sec;
/* Bookkeeping structures */
struct libipw_stats ieee_stats;
struct libipw_geo geo;
- struct ieee80211_supported_band bg_band;
- struct ieee80211_supported_band a_band;
/* Probe / Beacon management */
struct list_head network_free_list;
@@ -1018,8 +1014,8 @@ static inline int libipw_is_cck_rate(u8 rate)
}
/* ieee80211.c */
-extern void free_ieee80211(struct net_device *dev, int monitor);
-extern struct net_device *alloc_ieee80211(int sizeof_priv, int monitor);
+extern void free_ieee80211(struct net_device *dev);
+extern struct net_device *alloc_ieee80211(int sizeof_priv);
extern int libipw_change_mtu(struct net_device *dev, int new_mtu);
extern void libipw_networks_age(struct libipw_device *ieee,
diff --git a/drivers/net/wireless/ipw2x00/libipw_module.c b/drivers/net/wireless/ipw2x00/libipw_module.c
index a0e9f6a..eb2b608 100644
--- a/drivers/net/wireless/ipw2x00/libipw_module.c
+++ b/drivers/net/wireless/ipw2x00/libipw_module.c
@@ -62,9 +62,6 @@ MODULE_DESCRIPTION(DRV_DESCRIPTION);
MODULE_AUTHOR(DRV_COPYRIGHT);
MODULE_LICENSE("GPL");
-struct cfg80211_ops libipw_config_ops = { };
-void *libipw_wiphy_privid = &libipw_wiphy_privid;
-
static int libipw_networks_allocate(struct libipw_device *ieee)
{
if (ieee->networks)
@@ -143,7 +140,7 @@ int libipw_change_mtu(struct net_device *dev, int new_mtu)
}
EXPORT_SYMBOL(libipw_change_mtu);
-struct net_device *alloc_ieee80211(int sizeof_priv, int monitor)
+struct net_device *alloc_ieee80211(int sizeof_priv)
{
struct libipw_device *ieee;
struct net_device *dev;
@@ -160,31 +157,10 @@ struct net_device *alloc_ieee80211(int sizeof_priv, int monitor)
ieee->dev = dev;
- if (!monitor) {
- ieee->wdev.wiphy = wiphy_new(&libipw_config_ops, 0);
- if (!ieee->wdev.wiphy) {
- LIBIPW_ERROR("Unable to allocate wiphy.\n");
- goto failed_free_netdev;
- }
-
- ieee->dev->ieee80211_ptr = &ieee->wdev;
- ieee->wdev.iftype = NL80211_IFTYPE_STATION;
-
- /* Fill-out wiphy structure bits we know... Not enough info
- here to call set_wiphy_dev or set MAC address or channel info
- -- have to do that in ->ndo_init... */
- ieee->wdev.wiphy->privid = libipw_wiphy_privid;
-
- ieee->wdev.wiphy->max_scan_ssids = 1;
- ieee->wdev.wiphy->max_scan_ie_len = 0;
- ieee->wdev.wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION)
- | BIT(NL80211_IFTYPE_ADHOC);
- }
-
err = libipw_networks_allocate(ieee);
if (err) {
LIBIPW_ERROR("Unable to allocate beacon storage: %d\n", err);
- goto failed_free_wiphy;
+ goto failed_free_netdev;
}
libipw_networks_initialize(ieee);
@@ -217,31 +193,19 @@ struct net_device *alloc_ieee80211(int sizeof_priv, int monitor)
return dev;
-failed_free_wiphy:
- if (!monitor)
- wiphy_free(ieee->wdev.wiphy);
failed_free_netdev:
free_netdev(dev);
failed:
return NULL;
}
-void free_ieee80211(struct net_device *dev, int monitor)
+void free_ieee80211(struct net_device *dev)
{
struct libipw_device *ieee = netdev_priv(dev);
lib80211_crypt_info_free(&ieee->crypt_info);
libipw_networks_free(ieee);
-
- /* free cfg80211 resources */
- if (!monitor) {
- wiphy_unregister(ieee->wdev.wiphy);
- kfree(ieee->a_band.channels);
- kfree(ieee->bg_band.channels);
- wiphy_free(ieee->wdev.wiphy);
- }
-
free_netdev(dev);
}
diff --git a/drivers/net/wireless/p54/p54usb.c b/drivers/net/wireless/p54/p54usb.c
index 17e1995..92af9b9 100644
--- a/drivers/net/wireless/p54/p54usb.c
+++ b/drivers/net/wireless/p54/p54usb.c
@@ -426,12 +426,16 @@ static const char p54u_romboot_3887[] = "~~~~";
static int p54u_firmware_reset_3887(struct ieee80211_hw *dev)
{
struct p54u_priv *priv = dev->priv;
- u8 buf[4];
+ u8 *buf;
int ret;
- memcpy(&buf, p54u_romboot_3887, sizeof(buf));
+ buf = kmalloc(4, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+ memcpy(buf, p54u_romboot_3887, 4);
ret = p54u_bulk_msg(priv, P54U_PIPE_DATA,
- buf, sizeof(buf));
+ buf, 4);
+ kfree(buf);
if (ret)
dev_err(&priv->udev->dev, "(p54usb) unable to jump to "
"boot ROM (%d)!\n", ret);
--
John W. Linville Someday the world will need a hero, and you
linville-2XuSBdqkA4R54TAoqtyWWQ@public.gmane.org might be all we have. Be ready.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [net-next] bnx2x: Handle Rx and Tx together in NAPI
From: Vladislav Zolotarov @ 2009-11-16 16:05 UTC (permalink / raw)
To: davem; +Cc: netdev, eilong
Put Tx and Rx DPC to be handled in the NAPI:
- Saves status blocks.
- Moves the Tx work from hardIRQ to NAPI.
Signed-off-by: Vladislav Zolotarov <vladz@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
---
drivers/net/bnx2x.h | 21 +--
drivers/net/bnx2x_main.c | 367 +++++++++++++++++++---------------------------
2 files changed, 157 insertions(+), 231 deletions(-)
diff --git a/drivers/net/bnx2x.h b/drivers/net/bnx2x.h
index 928942b..602ab86 100644
--- a/drivers/net/bnx2x.h
+++ b/drivers/net/bnx2x.h
@@ -259,9 +259,6 @@ struct bnx2x_eth_q_stats {
struct bnx2x_fastpath {
struct napi_struct napi;
-
- u8 is_rx_queue;
-
struct host_status_block *status_blk;
dma_addr_t status_blk_mapping;
@@ -970,8 +967,7 @@ struct bnx2x {
#define BNX2X_STATE_ERROR 0xf000
int multi_mode;
- int num_rx_queues;
- int num_tx_queues;
+ int num_queues;
u32 rx_mode;
#define BNX2X_RX_MODE_NONE 0
@@ -1074,20 +1070,15 @@ struct bnx2x {
};
-#define BNX2X_MAX_QUEUES(bp) (IS_E1HMF(bp) ? (MAX_CONTEXT/(2 * E1HVN_MAX)) \
- : (MAX_CONTEXT/2))
-#define BNX2X_NUM_QUEUES(bp) (bp->num_rx_queues + bp->num_tx_queues)
-#define is_multi(bp) (BNX2X_NUM_QUEUES(bp) > 2)
+#define BNX2X_MAX_QUEUES(bp) (IS_E1HMF(bp) ? (MAX_CONTEXT/E1HVN_MAX) \
+ : MAX_CONTEXT)
+#define BNX2X_NUM_QUEUES(bp) (bp->num_queues)
+#define is_multi(bp) (BNX2X_NUM_QUEUES(bp) > 1)
-#define for_each_rx_queue(bp, var) \
- for (var = 0; var < bp->num_rx_queues; var++)
-#define for_each_tx_queue(bp, var) \
- for (var = bp->num_rx_queues; \
- var < BNX2X_NUM_QUEUES(bp); var++)
#define for_each_queue(bp, var) \
for (var = 0; var < BNX2X_NUM_QUEUES(bp); var++)
#define for_each_nondefault_queue(bp, var) \
- for (var = 1; var < bp->num_rx_queues; var++)
+ for (var = 1; var < BNX2X_NUM_QUEUES(bp); var++)
void bnx2x_read_dmae(struct bnx2x *bp, u32 src_addr, u32 len32);
diff --git a/drivers/net/bnx2x_main.c b/drivers/net/bnx2x_main.c
index e2cf686..bdecd42 100644
--- a/drivers/net/bnx2x_main.c
+++ b/drivers/net/bnx2x_main.c
@@ -57,7 +57,7 @@
#include "bnx2x_init_ops.h"
#include "bnx2x_dump.h"
-#define DRV_MODULE_VERSION "1.52.1-4"
+#define DRV_MODULE_VERSION "1.52.1-5"
#define DRV_MODULE_RELDATE "2009/11/09"
#define BNX2X_BC_VER 0x040200
@@ -91,15 +91,10 @@ module_param(multi_mode, int, 0);
MODULE_PARM_DESC(multi_mode, " Multi queue mode "
"(0 Disable; 1 Enable (default))");
-static int num_rx_queues;
-module_param(num_rx_queues, int, 0);
-MODULE_PARM_DESC(num_rx_queues, " Number of Rx queues for multi_mode=1"
- " (default is half number of CPUs)");
-
-static int num_tx_queues;
-module_param(num_tx_queues, int, 0);
-MODULE_PARM_DESC(num_tx_queues, " Number of Tx queues for multi_mode=1"
- " (default is half number of CPUs)");
+static int num_queues;
+module_param(num_queues, int, 0);
+MODULE_PARM_DESC(num_queues, " Number of queues for multi_mode=1"
+ " (default is as a number of CPUs)");
static int disable_tpa;
module_param(disable_tpa, int, 0);
@@ -558,7 +553,7 @@ static void bnx2x_panic_dump(struct bnx2x *bp)
bp->def_att_idx, bp->attn_state, bp->spq_prod_idx);
/* Rx */
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
BNX2X_ERR("fp%d: rx_bd_prod(%x) rx_bd_cons(%x)"
@@ -575,7 +570,7 @@ static void bnx2x_panic_dump(struct bnx2x *bp)
}
/* Tx */
- for_each_tx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
BNX2X_ERR("fp%d: tx_pkt_prod(%x) tx_pkt_cons(%x)"
@@ -590,7 +585,7 @@ static void bnx2x_panic_dump(struct bnx2x *bp)
/* Rings */
/* Rx */
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
start = RX_BD(le16_to_cpu(*fp->rx_cons_sb) - 10);
@@ -624,7 +619,7 @@ static void bnx2x_panic_dump(struct bnx2x *bp)
}
/* Tx */
- for_each_tx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
start = TX_BD(le16_to_cpu(*fp->tx_cons_sb) - 10);
@@ -792,21 +787,13 @@ static inline void bnx2x_ack_sb(struct bnx2x *bp, u8 sb_id,
barrier();
}
-static inline u16 bnx2x_update_fpsb_idx(struct bnx2x_fastpath *fp)
+static inline void bnx2x_update_fpsb_idx(struct bnx2x_fastpath *fp)
{
struct host_status_block *fpsb = fp->status_blk;
- u16 rc = 0;
barrier(); /* status block is written to by the chip */
- if (fp->fp_c_idx != fpsb->c_status_block.status_block_index) {
- fp->fp_c_idx = fpsb->c_status_block.status_block_index;
- rc |= 1;
- }
- if (fp->fp_u_idx != fpsb->u_status_block.status_block_index) {
- fp->fp_u_idx = fpsb->u_status_block.status_block_index;
- rc |= 2;
- }
- return rc;
+ fp->fp_c_idx = fpsb->c_status_block.status_block_index;
+ fp->fp_u_idx = fpsb->u_status_block.status_block_index;
}
static u16 bnx2x_ack_int(struct bnx2x *bp)
@@ -846,6 +833,9 @@ static u16 bnx2x_free_tx_pkt(struct bnx2x *bp, struct bnx2x_fastpath *fp,
u16 bd_idx = TX_BD(tx_buf->first_bd), new_cons;
int nbd;
+ /* prefetch skb end pointer to speedup dev_kfree_skb() */
+ prefetch(&skb->end);
+
DP(BNX2X_MSG_OFF, "pkt_idx %d buff @(%p)->skb %p\n",
idx, tx_buf, skb);
@@ -890,7 +880,7 @@ static u16 bnx2x_free_tx_pkt(struct bnx2x *bp, struct bnx2x_fastpath *fp,
/* release skb */
WARN_ON(!skb);
- dev_kfree_skb_any(skb);
+ dev_kfree_skb(skb);
tx_buf->first_bd = 0;
tx_buf->skb = NULL;
@@ -920,19 +910,28 @@ static inline u16 bnx2x_tx_avail(struct bnx2x_fastpath *fp)
return (s16)(fp->bp->tx_ring_size) - used;
}
-static void bnx2x_tx_int(struct bnx2x_fastpath *fp)
+static inline int bnx2x_has_tx_work(struct bnx2x_fastpath *fp)
+{
+ u16 hw_cons;
+
+ /* Tell compiler that status block fields can change */
+ barrier();
+ hw_cons = le16_to_cpu(*fp->tx_cons_sb);
+ return hw_cons != fp->tx_pkt_cons;
+}
+
+static int bnx2x_tx_int(struct bnx2x_fastpath *fp)
{
struct bnx2x *bp = fp->bp;
struct netdev_queue *txq;
u16 hw_cons, sw_cons, bd_cons = fp->tx_bd_cons;
- int done = 0;
#ifdef BNX2X_STOP_ON_ERROR
if (unlikely(bp->panic))
- return;
+ return -1;
#endif
- txq = netdev_get_tx_queue(bp->dev, fp->index - bp->num_rx_queues);
+ txq = netdev_get_tx_queue(bp->dev, fp->index);
hw_cons = le16_to_cpu(*fp->tx_cons_sb);
sw_cons = fp->tx_pkt_cons;
@@ -953,7 +952,6 @@ static void bnx2x_tx_int(struct bnx2x_fastpath *fp)
*/
bd_cons = bnx2x_free_tx_pkt(bp, fp, pkt_cons);
sw_cons++;
- done++;
}
fp->tx_pkt_cons = sw_cons;
@@ -975,6 +973,7 @@ static void bnx2x_tx_int(struct bnx2x_fastpath *fp)
(bnx2x_tx_avail(fp) >= MAX_SKB_FRAGS + 3))
netif_tx_wake_queue(txq);
}
+ return 0;
}
#ifdef BCM_CNIC
@@ -1561,6 +1560,8 @@ static int bnx2x_rx_int(struct bnx2x_fastpath *fp, int budget)
} else {
rx_buf = &fp->rx_buf_ring[bd_cons];
skb = rx_buf->skb;
+ prefetch(skb);
+ prefetch((u8 *)skb + 256);
len = le16_to_cpu(cqe->fast_path_cqe.pkt_len);
pad = cqe->fast_path_cqe.placement_offset;
@@ -1742,27 +1743,13 @@ static irqreturn_t bnx2x_msix_fp_int(int irq, void *fp_cookie)
if (unlikely(bp->panic))
return IRQ_HANDLED;
#endif
- /* Handle Rx or Tx according to MSI-X vector */
- if (fp->is_rx_queue) {
- prefetch(fp->rx_cons_sb);
- prefetch(&fp->status_blk->u_status_block.status_block_index);
-
- napi_schedule(&bnx2x_fp(bp, fp->index, napi));
-
- } else {
- prefetch(fp->tx_cons_sb);
- prefetch(&fp->status_blk->c_status_block.status_block_index);
-
- bnx2x_update_fpsb_idx(fp);
- rmb();
- bnx2x_tx_int(fp);
- /* Re-enable interrupts */
- bnx2x_ack_sb(bp, fp->sb_id, USTORM_ID,
- le16_to_cpu(fp->fp_u_idx), IGU_INT_NOP, 1);
- bnx2x_ack_sb(bp, fp->sb_id, CSTORM_ID,
- le16_to_cpu(fp->fp_c_idx), IGU_INT_ENABLE, 1);
- }
+ /* Handle Rx and Tx according to MSI-X vector */
+ prefetch(fp->rx_cons_sb);
+ prefetch(fp->tx_cons_sb);
+ prefetch(&fp->status_blk->u_status_block.status_block_index);
+ prefetch(&fp->status_blk->c_status_block.status_block_index);
+ napi_schedule(&bnx2x_fp(bp, fp->index, napi));
return IRQ_HANDLED;
}
@@ -1797,31 +1784,14 @@ static irqreturn_t bnx2x_interrupt(int irq, void *dev_instance)
mask = 0x2 << fp->sb_id;
if (status & mask) {
- /* Handle Rx or Tx according to SB id */
- if (fp->is_rx_queue) {
- prefetch(fp->rx_cons_sb);
- prefetch(&fp->status_blk->u_status_block.
- status_block_index);
-
- napi_schedule(&bnx2x_fp(bp, fp->index, napi));
-
- } else {
- prefetch(fp->tx_cons_sb);
- prefetch(&fp->status_blk->c_status_block.
- status_block_index);
-
- bnx2x_update_fpsb_idx(fp);
- rmb();
- bnx2x_tx_int(fp);
-
- /* Re-enable interrupts */
- bnx2x_ack_sb(bp, fp->sb_id, USTORM_ID,
- le16_to_cpu(fp->fp_u_idx),
- IGU_INT_NOP, 1);
- bnx2x_ack_sb(bp, fp->sb_id, CSTORM_ID,
- le16_to_cpu(fp->fp_c_idx),
- IGU_INT_ENABLE, 1);
- }
+ /* Handle Rx and Tx according to SB id */
+ prefetch(fp->rx_cons_sb);
+ prefetch(&fp->status_blk->u_status_block.
+ status_block_index);
+ prefetch(fp->tx_cons_sb);
+ prefetch(&fp->status_blk->c_status_block.
+ status_block_index);
+ napi_schedule(&bnx2x_fp(bp, fp->index, napi));
status &= ~mask;
}
}
@@ -4027,7 +3997,7 @@ static int bnx2x_storm_stats_update(struct bnx2x *bp)
estats->no_buff_discard_hi = 0;
estats->no_buff_discard_lo = 0;
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
int cl_id = fp->cl_id;
struct tstorm_per_client_stats *tclient =
@@ -4244,7 +4214,7 @@ static void bnx2x_net_stats_update(struct bnx2x *bp)
nstats->tx_bytes = bnx2x_hilo(&estats->total_bytes_transmitted_hi);
nstats->rx_dropped = estats->mac_discard;
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
nstats->rx_dropped +=
le32_to_cpu(bp->fp[i].old_tclient.checksum_discard);
@@ -4298,7 +4268,7 @@ static void bnx2x_drv_stats_update(struct bnx2x *bp)
estats->rx_err_discard_pkt = 0;
estats->rx_skb_alloc_failed = 0;
estats->hw_csum_err = 0;
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_eth_q_stats *qstats = &bp->fp[i].eth_q_stats;
estats->driver_xoff += qstats->driver_xoff;
@@ -4329,7 +4299,7 @@ static void bnx2x_stats_update(struct bnx2x *bp)
if (bp->msglevel & NETIF_MSG_TIMER) {
struct bnx2x_fastpath *fp0_rx = bp->fp;
- struct bnx2x_fastpath *fp0_tx = &(bp->fp[bp->num_rx_queues]);
+ struct bnx2x_fastpath *fp0_tx = bp->fp;
struct tstorm_per_client_stats *old_tclient =
&bp->fp->old_tclient;
struct bnx2x_eth_q_stats *qstats = &bp->fp->eth_q_stats;
@@ -4984,7 +4954,7 @@ static void bnx2x_init_rx_rings(struct bnx2x *bp)
if (bp->flags & TPA_ENABLE_FLAG) {
- for_each_rx_queue(bp, j) {
+ for_each_queue(bp, j) {
struct bnx2x_fastpath *fp = &bp->fp[j];
for (i = 0; i < max_agg_queues; i++) {
@@ -5007,16 +4977,13 @@ static void bnx2x_init_rx_rings(struct bnx2x *bp)
}
}
- for_each_rx_queue(bp, j) {
+ for_each_queue(bp, j) {
struct bnx2x_fastpath *fp = &bp->fp[j];
fp->rx_bd_cons = 0;
fp->rx_cons_sb = BNX2X_RX_SB_INDEX;
fp->rx_bd_cons_sb = BNX2X_RX_SB_BD_INDEX;
- /* Mark queue as Rx */
- fp->is_rx_queue = 1;
-
/* "next page" elements initialization */
/* SGE ring */
for (i = 1; i <= NUM_RX_SGE_PAGES; i++) {
@@ -5122,7 +5089,7 @@ static void bnx2x_init_tx_ring(struct bnx2x *bp)
{
int i, j;
- for_each_tx_queue(bp, j) {
+ for_each_queue(bp, j) {
struct bnx2x_fastpath *fp = &bp->fp[j];
for (i = 1; i <= NUM_TX_RINGS; i++) {
@@ -5148,10 +5115,6 @@ static void bnx2x_init_tx_ring(struct bnx2x *bp)
fp->tx_cons_sb = BNX2X_TX_SB_INDEX;
fp->tx_pkt = 0;
}
-
- /* clean tx statistics */
- for_each_rx_queue(bp, i)
- bnx2x_fp(bp, i, tx_pkt) = 0;
}
static void bnx2x_init_sp_ring(struct bnx2x *bp)
@@ -5180,7 +5143,8 @@ static void bnx2x_init_context(struct bnx2x *bp)
{
int i;
- for_each_rx_queue(bp, i) {
+ /* Rx */
+ for_each_queue(bp, i) {
struct eth_context *context = bnx2x_sp(bp, context[i].eth);
struct bnx2x_fastpath *fp = &bp->fp[i];
u8 cl_id = fp->cl_id;
@@ -5232,10 +5196,11 @@ static void bnx2x_init_context(struct bnx2x *bp)
ETH_CONNECTION_TYPE);
}
- for_each_tx_queue(bp, i) {
+ /* Tx */
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
struct eth_context *context =
- bnx2x_sp(bp, context[i - bp->num_rx_queues].eth);
+ bnx2x_sp(bp, context[i].eth);
context->cstorm_st_context.sb_index_number =
C_SB_ETH_TX_CQ_INDEX;
@@ -5263,7 +5228,7 @@ static void bnx2x_init_ind_table(struct bnx2x *bp)
for (i = 0; i < TSTORM_INDIRECTION_TABLE_SIZE; i++)
REG_WR8(bp, BAR_TSTRORM_INTMEM +
TSTORM_INDIRECTION_TABLE_OFFSET(func) + i,
- bp->fp->cl_id + (i % bp->num_rx_queues));
+ bp->fp->cl_id + (i % bp->num_queues));
}
static void bnx2x_set_client_config(struct bnx2x *bp)
@@ -5507,7 +5472,7 @@ static void bnx2x_init_internal_func(struct bnx2x *bp)
min((u32)(min((u32)8, (u32)MAX_SKB_FRAGS) *
SGE_PAGE_SIZE * PAGES_PER_SGE),
(u32)0xffff);
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
REG_WR(bp, BAR_USTRORM_INTMEM +
@@ -5542,7 +5507,7 @@ static void bnx2x_init_internal_func(struct bnx2x *bp)
rx_pause.cqe_thr_high = 350;
rx_pause.sge_thr_high = 0;
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
if (!fp->disable_tpa) {
@@ -5637,9 +5602,6 @@ static void bnx2x_nic_init(struct bnx2x *bp, u32 load_code)
#else
fp->sb_id = fp->cl_id;
#endif
- /* Suitable Rx and Tx SBs are served by the same client */
- if (i >= bp->num_rx_queues)
- fp->cl_id -= bp->num_rx_queues;
DP(NETIF_MSG_IFUP,
"queue[%d]: bnx2x_init_sb(%p,%p) cl_id %d sb %d\n",
i, bp, fp->status_blk, fp->cl_id, fp->sb_id);
@@ -6749,7 +6711,7 @@ static void bnx2x_free_mem(struct bnx2x *bp)
sizeof(struct host_status_block));
}
/* Rx */
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
/* fastpath rx rings: rx_buf rx_desc rx_comp */
BNX2X_FREE(bnx2x_fp(bp, i, rx_buf_ring));
@@ -6769,7 +6731,7 @@ static void bnx2x_free_mem(struct bnx2x *bp)
BCM_PAGE_SIZE * NUM_RX_SGE_PAGES);
}
/* Tx */
- for_each_tx_queue(bp, i) {
+ for_each_queue(bp, i) {
/* fastpath tx rings: tx_buf tx_desc */
BNX2X_FREE(bnx2x_fp(bp, i, tx_buf_ring));
@@ -6831,7 +6793,7 @@ static int bnx2x_alloc_mem(struct bnx2x *bp)
sizeof(struct host_status_block));
}
/* Rx */
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
/* fastpath rx rings: rx_buf rx_desc rx_comp */
BNX2X_ALLOC(bnx2x_fp(bp, i, rx_buf_ring),
@@ -6853,7 +6815,7 @@ static int bnx2x_alloc_mem(struct bnx2x *bp)
BCM_PAGE_SIZE * NUM_RX_SGE_PAGES);
}
/* Tx */
- for_each_tx_queue(bp, i) {
+ for_each_queue(bp, i) {
/* fastpath tx rings: tx_buf tx_desc */
BNX2X_ALLOC(bnx2x_fp(bp, i, tx_buf_ring),
@@ -6909,7 +6871,7 @@ static void bnx2x_free_tx_skbs(struct bnx2x *bp)
{
int i;
- for_each_tx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
u16 bd_cons = fp->tx_bd_cons;
@@ -6927,7 +6889,7 @@ static void bnx2x_free_rx_skbs(struct bnx2x *bp)
{
int i, j;
- for_each_rx_queue(bp, j) {
+ for_each_queue(bp, j) {
struct bnx2x_fastpath *fp = &bp->fp[j];
for (i = 0; i < NUM_RX_BD; i++) {
@@ -7042,12 +7004,8 @@ static int bnx2x_req_msix_irqs(struct bnx2x *bp)
#endif
for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
-
- if (i < bp->num_rx_queues)
- sprintf(fp->name, "%s-rx-%d", bp->dev->name, i);
- else
- sprintf(fp->name, "%s-tx-%d",
- bp->dev->name, i - bp->num_rx_queues);
+ snprintf(fp->name, sizeof(fp->name), "%s-fp-%d",
+ bp->dev->name, i);
rc = request_irq(bp->msix_table[i + offset].vector,
bnx2x_msix_fp_int, 0, fp->name, fp);
@@ -7106,7 +7064,7 @@ static void bnx2x_napi_enable(struct bnx2x *bp)
{
int i;
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
napi_enable(&bnx2x_fp(bp, i, napi));
}
@@ -7114,7 +7072,7 @@ static void bnx2x_napi_disable(struct bnx2x *bp)
{
int i;
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
napi_disable(&bnx2x_fp(bp, i, napi));
}
@@ -7410,88 +7368,60 @@ static int bnx2x_setup_multi(struct bnx2x *bp, int index)
static int bnx2x_poll(struct napi_struct *napi, int budget);
-static void bnx2x_set_int_mode_msix(struct bnx2x *bp, int *num_rx_queues_out,
- int *num_tx_queues_out)
+static void bnx2x_set_num_queues_msix(struct bnx2x *bp)
{
- int _num_rx_queues = 0, _num_tx_queues = 0;
switch (bp->multi_mode) {
case ETH_RSS_MODE_DISABLED:
- _num_rx_queues = 1;
- _num_tx_queues = 1;
+ bp->num_queues = 1;
break;
case ETH_RSS_MODE_REGULAR:
- if (num_rx_queues)
- _num_rx_queues = min_t(u32, num_rx_queues,
- BNX2X_MAX_QUEUES(bp));
- else
- _num_rx_queues = min_t(u32, num_online_cpus(),
- BNX2X_MAX_QUEUES(bp));
-
- if (num_tx_queues)
- _num_tx_queues = min_t(u32, num_tx_queues,
- BNX2X_MAX_QUEUES(bp));
+ if (num_queues)
+ bp->num_queues = min_t(u32, num_queues,
+ BNX2X_MAX_QUEUES(bp));
else
- _num_tx_queues = min_t(u32, num_online_cpus(),
- BNX2X_MAX_QUEUES(bp));
-
- /* There must be not more Tx queues than Rx queues */
- if (_num_tx_queues > _num_rx_queues) {
- BNX2X_ERR("number of tx queues (%d) > "
- "number of rx queues (%d)"
- " defaulting to %d\n",
- _num_tx_queues, _num_rx_queues,
- _num_rx_queues);
- _num_tx_queues = _num_rx_queues;
- }
+ bp->num_queues = min_t(u32, num_online_cpus(),
+ BNX2X_MAX_QUEUES(bp));
break;
default:
- _num_rx_queues = 1;
- _num_tx_queues = 1;
+ bp->num_queues = 1;
break;
}
-
- *num_rx_queues_out = _num_rx_queues;
- *num_tx_queues_out = _num_tx_queues;
}
-static int bnx2x_set_int_mode(struct bnx2x *bp)
+static int bnx2x_set_num_queues(struct bnx2x *bp)
{
int rc = 0;
switch (int_mode) {
case INT_MODE_INTx:
case INT_MODE_MSI:
- bp->num_rx_queues = 1;
- bp->num_tx_queues = 1;
+ bp->num_queues = 1;
DP(NETIF_MSG_IFUP, "set number of queues to 1\n");
break;
case INT_MODE_MSIX:
default:
- /* Set interrupt mode according to bp->multi_mode value */
- bnx2x_set_int_mode_msix(bp, &bp->num_rx_queues,
- &bp->num_tx_queues);
+ /* Set number of queues according to bp->multi_mode value */
+ bnx2x_set_num_queues_msix(bp);
- DP(NETIF_MSG_IFUP, "set number of queues to: rx %d tx %d\n",
- bp->num_rx_queues, bp->num_tx_queues);
+ DP(NETIF_MSG_IFUP, "set number of queues to %d\n",
+ bp->num_queues);
/* if we can't use MSI-X we only need one fp,
* so try to enable MSI-X with the requested number of fp's
* and fallback to MSI or legacy INTx with one fp
*/
rc = bnx2x_enable_msix(bp);
- if (rc) {
+ if (rc)
/* failed to enable MSI-X */
- bp->num_rx_queues = 1;
- bp->num_tx_queues = 1;
- }
+ bp->num_queues = 1;
break;
}
- bp->dev->real_num_tx_queues = bp->num_tx_queues;
+ bp->dev->real_num_tx_queues = bp->num_queues;
return rc;
}
@@ -7513,16 +7443,16 @@ static int bnx2x_nic_load(struct bnx2x *bp, int load_mode)
bp->state = BNX2X_STATE_OPENING_WAIT4_LOAD;
- rc = bnx2x_set_int_mode(bp);
+ rc = bnx2x_set_num_queues(bp);
if (bnx2x_alloc_mem(bp))
return -ENOMEM;
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
bnx2x_fp(bp, i, disable_tpa) =
((bp->flags & TPA_ENABLE_FLAG) == 0);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
netif_napi_add(bp->dev, &bnx2x_fp(bp, i, napi),
bnx2x_poll, 128);
@@ -7536,7 +7466,7 @@ static int bnx2x_nic_load(struct bnx2x *bp, int load_mode)
}
} else {
/* Fall to INTx if failed to enable MSI-X due to lack of
- memory (in bnx2x_set_int_mode()) */
+ memory (in bnx2x_set_num_queues()) */
if ((rc != -ENOMEM) && (int_mode != INT_MODE_INTx))
bnx2x_enable_msi(bp);
bnx2x_ack_int(bp);
@@ -7730,14 +7660,14 @@ load_error3:
bp->port.pmf = 0;
/* Free SKBs, SGEs, TPA pool and driver internals */
bnx2x_free_skbs(bp);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
bnx2x_free_rx_sge_range(bp, bp->fp + i, NUM_RX_SGE);
load_error2:
/* Release IRQs */
bnx2x_free_irq(bp);
load_error1:
bnx2x_napi_disable(bp);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
netif_napi_del(&bnx2x_fp(bp, i, napi));
bnx2x_free_mem(bp);
@@ -7928,7 +7858,7 @@ static int bnx2x_nic_unload(struct bnx2x *bp, int unload_mode)
bnx2x_free_irq(bp);
/* Wait until tx fastpath tasks complete */
- for_each_tx_queue(bp, i) {
+ for_each_queue(bp, i) {
struct bnx2x_fastpath *fp = &bp->fp[i];
cnt = 1000;
@@ -8071,9 +8001,9 @@ unload_error:
/* Free SKBs, SGEs, TPA pool and driver internals */
bnx2x_free_skbs(bp);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
bnx2x_free_rx_sge_range(bp, bp->fp + i, NUM_RX_SGE);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
netif_napi_del(&bnx2x_fp(bp, i, napi));
bnx2x_free_mem(bp);
@@ -10269,7 +10199,7 @@ static int bnx2x_run_loopback(struct bnx2x *bp, int loopback_mode, u8 link_up)
struct sk_buff *skb;
unsigned char *packet;
struct bnx2x_fastpath *fp_rx = &bp->fp[0];
- struct bnx2x_fastpath *fp_tx = &bp->fp[bp->num_rx_queues];
+ struct bnx2x_fastpath *fp_tx = &bp->fp[0];
u16 tx_start_idx, tx_idx;
u16 rx_start_idx, rx_idx;
u16 pkt_prod, bd_prod;
@@ -10346,7 +10276,7 @@ static int bnx2x_run_loopback(struct bnx2x *bp, int loopback_mode, u8 link_up)
fp_tx->tx_db.data.prod += 2;
barrier();
- DOORBELL(bp, fp_tx->index - bp->num_rx_queues, fp_tx->tx_db.raw);
+ DOORBELL(bp, fp_tx->index, fp_tx->tx_db.raw);
mmiowb();
@@ -10725,7 +10655,7 @@ static int bnx2x_get_sset_count(struct net_device *dev, int stringset)
switch(stringset) {
case ETH_SS_STATS:
if (is_multi(bp)) {
- num_stats = BNX2X_NUM_Q_STATS * bp->num_rx_queues;
+ num_stats = BNX2X_NUM_Q_STATS * bp->num_queues;
if (!IS_E1HMF_MODE_STAT(bp))
num_stats += BNX2X_NUM_STATS;
} else {
@@ -10756,7 +10686,7 @@ static void bnx2x_get_strings(struct net_device *dev, u32 stringset, u8 *buf)
case ETH_SS_STATS:
if (is_multi(bp)) {
k = 0;
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
for (j = 0; j < BNX2X_NUM_Q_STATS; j++)
sprintf(buf + (k + j)*ETH_GSTRING_LEN,
bnx2x_q_stats_arr[j].string, i);
@@ -10793,7 +10723,7 @@ static void bnx2x_get_ethtool_stats(struct net_device *dev,
if (is_multi(bp)) {
k = 0;
- for_each_rx_queue(bp, i) {
+ for_each_queue(bp, i) {
hw_stats = (u32 *)&bp->fp[i].eth_q_stats;
for (j = 0; j < BNX2X_NUM_Q_STATS; j++) {
if (bnx2x_q_stats_arr[j].size == 0) {
@@ -10989,54 +10919,60 @@ static inline int bnx2x_has_rx_work(struct bnx2x_fastpath *fp)
static int bnx2x_poll(struct napi_struct *napi, int budget)
{
+ int work_done = 0;
struct bnx2x_fastpath *fp = container_of(napi, struct bnx2x_fastpath,
napi);
struct bnx2x *bp = fp->bp;
- int work_done = 0;
+ while (1) {
#ifdef BNX2X_STOP_ON_ERROR
- if (unlikely(bp->panic))
- goto poll_panic;
+ if (unlikely(bp->panic)) {
+ napi_complete(napi);
+ return 0;
+ }
#endif
- prefetch(fp->rx_buf_ring[RX_BD(fp->rx_bd_cons)].skb);
- prefetch((char *)(fp->rx_buf_ring[RX_BD(fp->rx_bd_cons)].skb) + 256);
-
- bnx2x_update_fpsb_idx(fp);
-
- if (bnx2x_has_rx_work(fp)) {
- work_done = bnx2x_rx_int(fp, budget);
+ if (bnx2x_has_tx_work(fp))
+ bnx2x_tx_int(fp);
- /* must not complete if we consumed full budget */
- if (work_done >= budget)
- goto poll_again;
- }
+ if (bnx2x_has_rx_work(fp)) {
+ work_done += bnx2x_rx_int(fp, budget - work_done);
- /* bnx2x_has_rx_work() reads the status block, thus we need to
- * ensure that status block indices have been actually read
- * (bnx2x_update_fpsb_idx) prior to this check (bnx2x_has_rx_work)
- * so that we won't write the "newer" value of the status block to IGU
- * (if there was a DMA right after bnx2x_has_rx_work and
- * if there is no rmb, the memory reading (bnx2x_update_fpsb_idx)
- * may be postponed to right before bnx2x_ack_sb). In this case
- * there will never be another interrupt until there is another update
- * of the status block, while there is still unhandled work.
- */
- rmb();
+ /* must not complete if we consumed full budget */
+ if (work_done >= budget)
+ break;
+ }
- if (!bnx2x_has_rx_work(fp)) {
-#ifdef BNX2X_STOP_ON_ERROR
-poll_panic:
-#endif
- napi_complete(napi);
+ /* Fall out from the NAPI loop if needed */
+ if (!(bnx2x_has_rx_work(fp) || bnx2x_has_tx_work(fp))) {
+ bnx2x_update_fpsb_idx(fp);
+ /* bnx2x_has_rx_work() reads the status block, thus we need
+ * to ensure that status block indices have been actually read
+ * (bnx2x_update_fpsb_idx) prior to this check
+ * (bnx2x_has_rx_work) so that we won't write the "newer"
+ * value of the status block to IGU (if there was a DMA right
+ * after bnx2x_has_rx_work and if there is no rmb, the memory
+ * reading (bnx2x_update_fpsb_idx) may be postponed to right
+ * before bnx2x_ack_sb). In this case there will never be
+ * another interrupt until there is another update of the
+ * status block, while there is still unhandled work.
+ */
+ rmb();
- bnx2x_ack_sb(bp, fp->sb_id, USTORM_ID,
- le16_to_cpu(fp->fp_u_idx), IGU_INT_NOP, 1);
- bnx2x_ack_sb(bp, fp->sb_id, CSTORM_ID,
- le16_to_cpu(fp->fp_c_idx), IGU_INT_ENABLE, 1);
+ if (!(bnx2x_has_rx_work(fp) || bnx2x_has_tx_work(fp))) {
+ napi_complete(napi);
+ /* Re-enable interrupts */
+ bnx2x_ack_sb(bp, fp->sb_id, CSTORM_ID,
+ le16_to_cpu(fp->fp_c_idx),
+ IGU_INT_NOP, 1);
+ bnx2x_ack_sb(bp, fp->sb_id, USTORM_ID,
+ le16_to_cpu(fp->fp_u_idx),
+ IGU_INT_ENABLE, 1);
+ break;
+ }
+ }
}
-poll_again:
return work_done;
}
@@ -11221,7 +11157,7 @@ exit_lbl:
static netdev_tx_t bnx2x_start_xmit(struct sk_buff *skb, struct net_device *dev)
{
struct bnx2x *bp = netdev_priv(dev);
- struct bnx2x_fastpath *fp, *fp_stat;
+ struct bnx2x_fastpath *fp;
struct netdev_queue *txq;
struct sw_tx_bd *tx_buf;
struct eth_tx_start_bd *tx_start_bd;
@@ -11243,11 +11179,10 @@ static netdev_tx_t bnx2x_start_xmit(struct sk_buff *skb, struct net_device *dev)
fp_index = skb_get_queue_mapping(skb);
txq = netdev_get_tx_queue(dev, fp_index);
- fp = &bp->fp[fp_index + bp->num_rx_queues];
- fp_stat = &bp->fp[fp_index];
+ fp = &bp->fp[fp_index];
if (unlikely(bnx2x_tx_avail(fp) < (skb_shinfo(skb)->nr_frags + 3))) {
- fp_stat->eth_q_stats.driver_xoff++;
+ fp->eth_q_stats.driver_xoff++;
netif_tx_stop_queue(txq);
BNX2X_ERR("BUG! Tx ring full when queue awake!\n");
return NETDEV_TX_BUSY;
@@ -11473,7 +11408,7 @@ static netdev_tx_t bnx2x_start_xmit(struct sk_buff *skb, struct net_device *dev)
fp->tx_db.data.prod += nbd;
barrier();
- DOORBELL(bp, fp->index - bp->num_rx_queues, fp->tx_db.raw);
+ DOORBELL(bp, fp->index, fp->tx_db.raw);
mmiowb();
@@ -11484,11 +11419,11 @@ static netdev_tx_t bnx2x_start_xmit(struct sk_buff *skb, struct net_device *dev)
/* We want bnx2x_tx_int to "see" the updated tx_bd_prod
if we put Tx into XOFF state. */
smp_mb();
- fp_stat->eth_q_stats.driver_xoff++;
+ fp->eth_q_stats.driver_xoff++;
if (bnx2x_tx_avail(fp) >= MAX_SKB_FRAGS + 3)
netif_tx_wake_queue(txq);
}
- fp_stat->tx_pkt++;
+ fp->tx_pkt++;
return NETDEV_TX_OK;
}
@@ -12376,9 +12311,9 @@ static int bnx2x_eeh_nic_unload(struct bnx2x *bp)
/* Free SKBs, SGEs, TPA pool and driver internals */
bnx2x_free_skbs(bp);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
bnx2x_free_rx_sge_range(bp, bp->fp + i, NUM_RX_SGE);
- for_each_rx_queue(bp, i)
+ for_each_queue(bp, i)
netif_napi_del(&bnx2x_fp(bp, i, napi));
bnx2x_free_mem(bp);
--
1.6.3.3
^ permalink raw reply related
* Re: [PATCH 2/2 v2] KS8695: fix ks8695_rx() unreasonable action.
From: Figo.zhang @ 2009-11-12 17:16 UTC (permalink / raw)
To: zeal; +Cc: netdev, ben, davem
In-Reply-To: <4b016886.16115e0a.355d.ffffca52@mx.google.com>
On Mon, 2009-11-16 at 22:58 +0800, zeal wrote:
> From: zeal <zealcook@gmail.com>
>
> ks8695_rx() will call refill_buffers() for every incoming packet.
> Its not necessary. We just need do it after finishing receiving thing.
> And the 'RX dma engine' is in the same situation.
> This blocks our user space application. The following patch may fix it.
>
yes, it is right.
Best,
Figo.zhang
> Signed-off-by: zeal <zealcook@gmail.com>
> ---
> drivers/net/arm/ks8695net.c | 13 ++++++-------
> 1 files changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/net/arm/ks8695net.c b/drivers/net/arm/ks8695net.c
> index e15451a..be256b3 100644
> --- a/drivers/net/arm/ks8695net.c
> +++ b/drivers/net/arm/ks8695net.c
> @@ -544,14 +544,13 @@ rx_finished:
> ksp->next_rx_desc_read =
> (last_rx_processed + 1) &
> MAX_RX_DESC_MASK;
> -
> - /* And refill the buffers */
> - ks8695_refill_rxbuffers(ksp);
> -
> - /* Kick the RX DMA engine, in case it became
> - * suspended */
> - ks8695_writereg(ksp, KS8695_DRSC, 0);
> }
> + /* And refill the buffers */
> + ks8695_refill_rxbuffers(ksp);
> +
> + /* Kick the RX DMA engine, in case it became
> + * suspended */
> + ks8695_writereg(ksp, KS8695_DRSC, 0);
> return received;
> }
>
^ permalink raw reply
* Re: [PATCH 1/2 v2] KS8695: fix ks8695_rx_irq() bug.
From: Figo.zhang @ 2009-11-12 17:11 UTC (permalink / raw)
To: zeal; +Cc: netdev, ben, davem
In-Reply-To: <efc9214f0911160718o4c7cd0ecv2e86b8d5ad7691ae@mail.gmail.com>
On Mon, 2009-11-16 at 23:18 +0800, zeal wrote:
> On Fri, Nov 13, 2009 at 12:24 AM, Figo.zhang <figo1802@gmail.com> wrote:
> > On Mon, 2009-11-16 at 22:58 +0800, zeal wrote:
> >> From: zeal <zealcook@gmail.com>
> >>
> >> Sorry for the previously patches. THEY'RE NOT RIGHT. It's my mistake.
> >> Please forgive my noise.
> >> Please review the following patches and ignore the last (v1).
> >>
> >> ks8695 rx irq is edge-level. Before arriving at irq handler, the
> >> corresponding status bit has been clear(irq's ack).
> >> So we should not check it after that.
> >
> > see <KS8695X Integrated Multi-Port Gateway Solution Register
> > Description> Version 1.00
> >
> > Interrupt Status Register(INTST Offset 0xE208)
> >
> > it has said: This edge-triggered interrupt status is cleared by writting
> > 1 , so we should write 1 to clear status bit manually.
> >
> Yeah, but irq_chip's ack has done that.
> Please check arch/arm/mach-ks8695/irq.c ks8695_irq_edge_chip->ack()
yes, you are right, but you have better add this description at commit
log.
>
> >>
> >> Signed-off-by: zeal <zealcook@gmail.com>
> >> ---
> >> drivers/net/arm/ks8695net.c | 22 +++++++---------------
> >> 1 files changed, 7 insertions(+), 15 deletions(-)
> >>
> >> diff --git a/drivers/net/arm/ks8695net.c b/drivers/net/arm/ks8695net.c
> >> index 0073d19..e15451a 100644
> >> --- a/drivers/net/arm/ks8695net.c
> >> +++ b/drivers/net/arm/ks8695net.c
> >> @@ -433,24 +433,16 @@ ks8695_rx_irq(int irq, void *dev_id)
> >> {
> >> struct net_device *ndev = (struct net_device *)dev_id;
> >> struct ks8695_priv *ksp = netdev_priv(ndev);
> >> - unsigned long status;
> >> -
> >> - unsigned long mask_bit = 1 << ks8695_get_rx_enable_bit(ksp);
> >>
> >> spin_lock(&ksp->rx_lock);
> >>
> >> - status = readl(KS8695_IRQ_VA + KS8695_INTST);
> >> -
> >> - /*clean rx status bit*/
> >> - writel(status | mask_bit , KS8695_IRQ_VA + KS8695_INTST);
> >> -
> >> - if (status & mask_bit) {
> >> - if (napi_schedule_prep(&ksp->napi)) {
> >> - /*disable rx interrupt*/
> >> - status &= ~mask_bit;
> >> - writel(status , KS8695_IRQ_VA + KS8695_INTEN);
> >> - __napi_schedule(&ksp->napi);
> >> - }
> >> + if (napi_schedule_prep(&ksp->napi)) {
> >> + unsigned long status = readl(KS8695_IRQ_VA + KS8695_INTEN);
> >> + unsigned long mask_bit = 1 << ks8695_get_rx_enable_bit(ksp);
> >> + /*disable rx interrupt*/
> >> + status &= ~mask_bit;
> >> + writel(status , KS8695_IRQ_VA + KS8695_INTEN);
> >> + __napi_schedule(&ksp->napi);
> >> }
> >>
> >> spin_unlock(&ksp->rx_lock);
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe netdev" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
>
>
>
^ permalink raw reply
* [PATCHv6 10/10] mlx4: Add RDMAoE support - allow interfaces to correspond to each other
From: Eli Cohen @ 2009-11-16 15:54 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
This patch adds RDMAoE support for mlx4. Since mlx4_ib now needs to reference
mlx4_en netdevices, a new mechanism was added. Two new fields were added to
struct mlx4_interface to define a protocol and a get_prot_dev method to
retrieve the corresponding protocol's net device. An implementation of the new
verb ib_get_port_link_type() - mlx4_ib_get_port_link_type - was added.
mlx4_ib_query_port() has been modified to support eth link types. An interface
is considered to be active if its corresponding eth interface is active. Code
for setting the GID table of a port has been added. Currently, each IB port has
a single GID entry in its table and that GID entery equals the link local IPv6
address.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/hw/mlx4/main.c | 436 +++++++++++++++++++++++++++++++---
drivers/infiniband/hw/mlx4/mlx4_ib.h | 11 +
drivers/infiniband/hw/mlx4/qp.c | 27 ++-
drivers/net/mlx4/en_main.c | 15 +-
drivers/net/mlx4/en_port.c | 4 +-
drivers/net/mlx4/en_port.h | 3 +-
drivers/net/mlx4/intf.c | 20 ++
drivers/net/mlx4/main.c | 6 +
drivers/net/mlx4/mlx4.h | 1 +
include/linux/mlx4/cmd.h | 1 +
include/linux/mlx4/driver.h | 16 +-
11 files changed, 493 insertions(+), 47 deletions(-)
diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c
index e596537..df52e57 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -34,9 +34,13 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/errno.h>
+#include <linux/netdevice.h>
+#include <linux/inetdevice.h>
+#include <linux/rtnetlink.h>
#include <rdma/ib_smi.h>
#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_addr.h>
#include <linux/mlx4/driver.h>
#include <linux/mlx4/cmd.h>
@@ -57,6 +61,15 @@ static const char mlx4_ib_version[] =
DRV_NAME ": Mellanox ConnectX InfiniBand driver v"
DRV_VERSION " (" DRV_RELDATE ")\n";
+struct update_gid_work {
+ struct work_struct work;
+ union ib_gid gids[128];
+ int port;
+ struct mlx4_ib_dev *dev;
+};
+
+static struct workqueue_struct *wq;
+
static void init_query_mad(struct ib_smp *mad)
{
mad->base_version = 1;
@@ -152,28 +165,19 @@ out:
return err;
}
-static int mlx4_ib_query_port(struct ib_device *ibdev, u8 port,
- struct ib_port_attr *props)
+static enum rdma_transport_type
+mlx4_ib_port_get_transport(struct ib_device *device, u8 port_num)
{
- struct ib_smp *in_mad = NULL;
- struct ib_smp *out_mad = NULL;
- int err = -ENOMEM;
-
- in_mad = kzalloc(sizeof *in_mad, GFP_KERNEL);
- out_mad = kmalloc(sizeof *out_mad, GFP_KERNEL);
- if (!in_mad || !out_mad)
- goto out;
-
- memset(props, 0, sizeof *props);
+ struct mlx4_dev *dev = to_mdev(device)->dev;
- init_query_mad(in_mad);
- in_mad->attr_id = IB_SMP_ATTR_PORT_INFO;
- in_mad->attr_mod = cpu_to_be32(port);
-
- err = mlx4_MAD_IFC(to_mdev(ibdev), 1, 1, port, NULL, NULL, in_mad, out_mad);
- if (err)
- goto out;
+ return dev->caps.port_mask & (1 << (port_num - 1)) ?
+ RDMA_TRANSPORT_IB : RDMA_TRANSPORT_RDMAOE;
+}
+static void ib_link_query_port(struct ib_device *ibdev, u8 port,
+ struct ib_port_attr *props,
+ struct ib_smp *out_mad)
+{
props->lid = be16_to_cpup((__be16 *) (out_mad->data + 16));
props->lmc = out_mad->data[34] & 0x7;
props->sm_lid = be16_to_cpup((__be16 *) (out_mad->data + 18));
@@ -193,6 +197,67 @@ static int mlx4_ib_query_port(struct ib_device *ibdev, u8 port,
props->subnet_timeout = out_mad->data[51] & 0x1f;
props->max_vl_num = out_mad->data[37] >> 4;
props->init_type_reply = out_mad->data[41] >> 4;
+ props->transport = RDMA_TRANSPORT_IB;
+}
+
+static void eth_link_query_port(struct ib_device *ibdev, u8 port,
+ struct ib_port_attr *props,
+ struct ib_smp *out_mad)
+{
+ struct mlx4_ib_rdmaoe *rdmaoe = &to_mdev(ibdev)->rdmaoe;
+ struct net_device *ndev;
+
+ props->port_cap_flags = IB_PORT_CM_SUP;
+ props->gid_tbl_len = to_mdev(ibdev)->dev->caps.gid_table_len[port];
+ props->max_msg_sz = to_mdev(ibdev)->dev->caps.max_msg_sz;
+ props->pkey_tbl_len = 1;
+ props->bad_pkey_cntr = be16_to_cpup((__be16 *) (out_mad->data + 46));
+ props->qkey_viol_cntr = be16_to_cpup((__be16 *) (out_mad->data + 48));
+ props->active_width = 0;
+ props->active_speed = 0;
+ props->max_mtu = out_mad->data[41] & 0xf;
+ props->subnet_timeout = 0;
+ props->max_vl_num = out_mad->data[37] >> 4;
+ props->init_type_reply = 0;
+ props->transport = RDMA_TRANSPORT_RDMAOE;
+ spin_lock(&rdmaoe->lock);
+ ndev = rdmaoe->netdevs[port - 1];
+ if (!ndev)
+ goto out;
+
+ props->active_mtu = rdmaoe_get_mtu(ndev->mtu);
+ props->state = netif_running(ndev) && netif_oper_up(ndev) ?
+ IB_PORT_ACTIVE : IB_PORT_DOWN;
+ props->phys_state = props->state;
+out:
+ spin_unlock(&rdmaoe->lock);
+}
+
+static int mlx4_ib_query_port(struct ib_device *ibdev, u8 port,
+ struct ib_port_attr *props)
+{
+ struct ib_smp *in_mad = NULL;
+ struct ib_smp *out_mad = NULL;
+ int err = -ENOMEM;
+
+ in_mad = kzalloc(sizeof *in_mad, GFP_KERNEL);
+ out_mad = kmalloc(sizeof *out_mad, GFP_KERNEL);
+ if (!in_mad || !out_mad)
+ goto out;
+
+ memset(props, 0, sizeof *props);
+
+ init_query_mad(in_mad);
+ in_mad->attr_id = IB_SMP_ATTR_PORT_INFO;
+ in_mad->attr_mod = cpu_to_be32(port);
+
+ err = mlx4_MAD_IFC(to_mdev(ibdev), 1, 1, port, NULL, NULL, in_mad, out_mad);
+ if (err)
+ goto out;
+
+ mlx4_ib_port_get_transport(ibdev, port) == RDMA_TRANSPORT_IB ?
+ ib_link_query_port(ibdev, port, props, out_mad) :
+ eth_link_query_port(ibdev, port, props, out_mad);
out:
kfree(in_mad);
@@ -201,8 +266,8 @@ out:
return err;
}
-static int mlx4_ib_query_gid(struct ib_device *ibdev, u8 port, int index,
- union ib_gid *gid)
+static int __mlx4_ib_query_gid(struct ib_device *ibdev, u8 port, int index,
+ union ib_gid *gid)
{
struct ib_smp *in_mad = NULL;
struct ib_smp *out_mad = NULL;
@@ -239,6 +304,25 @@ out:
return err;
}
+static int rdmaoe_query_gid(struct ib_device *ibdev, u8 port, int index,
+ union ib_gid *gid)
+{
+ struct mlx4_ib_dev *dev = to_mdev(ibdev);
+
+ *gid = dev->rdmaoe.gid_table[port - 1][index];
+
+ return 0;
+}
+
+static int mlx4_ib_query_gid(struct ib_device *ibdev, u8 port, int index,
+ union ib_gid *gid)
+{
+ if (rdma_port_get_transport(ibdev, port) == RDMA_TRANSPORT_IB)
+ return __mlx4_ib_query_gid(ibdev, port, index, gid);
+ else
+ return rdmaoe_query_gid(ibdev, port, index, gid);
+}
+
static int mlx4_ib_query_pkey(struct ib_device *ibdev, u8 port, u16 index,
u16 *pkey)
{
@@ -287,6 +371,7 @@ static int mlx4_SET_PORT(struct mlx4_ib_dev *dev, u8 port, int reset_qkey_viols,
{
struct mlx4_cmd_mailbox *mailbox;
int err;
+ u8 is_eth = dev->dev->caps.port_type[port] == MLX4_PORT_TYPE_ETH;
mailbox = mlx4_alloc_cmd_mailbox(dev->dev);
if (IS_ERR(mailbox))
@@ -302,7 +387,7 @@ static int mlx4_SET_PORT(struct mlx4_ib_dev *dev, u8 port, int reset_qkey_viols,
((__be32 *) mailbox->buf)[1] = cpu_to_be32(cap_mask);
}
- err = mlx4_cmd(dev->dev, mailbox->dma, port, 0, MLX4_CMD_SET_PORT,
+ err = mlx4_cmd(dev->dev, mailbox->dma, port, is_eth, MLX4_CMD_SET_PORT,
MLX4_CMD_TIME_CLASS_B);
mlx4_free_cmd_mailbox(dev->dev, mailbox);
@@ -445,18 +530,131 @@ static int mlx4_ib_dealloc_pd(struct ib_pd *pd)
return 0;
}
+static int add_gid_entry(struct ib_qp *ibqp, union ib_gid *gid)
+{
+ struct mlx4_ib_qp *mqp = to_mqp(ibqp);
+ struct mlx4_ib_dev *mdev = to_mdev(ibqp->device);
+ struct gid_entry *ge;
+
+ ge = kzalloc(sizeof *ge, GFP_KERNEL);
+ if (!ge)
+ return -ENOMEM;
+
+ ge->gid = *gid;
+ if (mlx4_ib_add_mc(mdev, mqp, gid)) {
+ ge->port = mqp->port;
+ ge->added = 1;
+ }
+
+ mutex_lock(&mqp->mutex);
+ list_add_tail(&ge->list, &mqp->gid_list);
+ mutex_unlock(&mqp->mutex);
+
+ return 0;
+}
+
+int mlx4_ib_add_mc(struct mlx4_ib_dev *mdev, struct mlx4_ib_qp *mqp,
+ union ib_gid *gid)
+{
+ u8 mac[6];
+ struct net_device *ndev;
+ int ret = 0;
+
+ if (!mqp->port)
+ return 0;
+
+ spin_lock(&mdev->rdmaoe.lock);
+ ndev = mdev->rdmaoe.netdevs[mqp->port - 1];
+ if (ndev)
+ dev_hold(ndev);
+ spin_unlock(&mdev->rdmaoe.lock);
+ if (ndev) {
+ rdma_get_mcast_mac((struct in6_addr *)gid, mac);
+ rtnl_lock();
+ dev_mc_add(mdev->rdmaoe.netdevs[mqp->port - 1], mac, 6, 0);
+ ret = 1;
+ rtnl_unlock();
+ dev_put(ndev);
+ }
+
+ return ret;
+}
+
static int mlx4_ib_mcg_attach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
{
- return mlx4_multicast_attach(to_mdev(ibqp->device)->dev,
- &to_mqp(ibqp)->mqp, gid->raw,
- !!(to_mqp(ibqp)->flags &
- MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK));
+ int err;
+ struct mlx4_ib_dev *mdev = to_mdev(ibqp->device);
+ struct mlx4_ib_qp *mqp = to_mqp(ibqp);
+
+ err = mlx4_multicast_attach(mdev->dev, &mqp->mqp, gid->raw, !!mqp->flags &
+ MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK);
+ if (err)
+ return err;
+
+ err = add_gid_entry(ibqp, gid);
+ if (err)
+ goto err_add;
+
+ return 0;
+
+err_add:
+ mlx4_multicast_detach(mdev->dev, &mqp->mqp, gid->raw);
+ return err;
+}
+
+struct gid_entry *find_gid_entry(struct mlx4_ib_qp *qp, u8 *raw)
+{
+ struct gid_entry *ge;
+ struct gid_entry *tmp;
+ struct gid_entry *ret = NULL;
+
+ list_for_each_entry_safe(ge, tmp, &qp->gid_list, list) {
+ if (!memcmp(raw, ge->gid.raw, 16)) {
+ ret = ge;
+ break;
+ }
+ }
+
+ return ret;
}
static int mlx4_ib_mcg_detach(struct ib_qp *ibqp, union ib_gid *gid, u16 lid)
{
- return mlx4_multicast_detach(to_mdev(ibqp->device)->dev,
- &to_mqp(ibqp)->mqp, gid->raw);
+ int err;
+ struct mlx4_ib_dev *mdev = to_mdev(ibqp->device);
+ struct mlx4_ib_qp *mqp = to_mqp(ibqp);
+ u8 mac[6];
+ struct net_device *ndev;
+ struct gid_entry *ge;
+
+ err = mlx4_multicast_detach(mdev->dev,
+ &mqp->mqp, gid->raw);
+ if (err)
+ return err;
+
+ mutex_lock(&mqp->mutex);
+ ge = find_gid_entry(mqp, gid->raw);
+ if (ge) {
+ spin_lock(&mdev->rdmaoe.lock);
+ ndev = ge->added ? mdev->rdmaoe.netdevs[ge->port - 1] : NULL;
+ if (ndev)
+ dev_hold(ndev);
+ spin_unlock(&mdev->rdmaoe.lock);
+ rdma_get_mcast_mac((struct in6_addr *)gid, mac);
+ if (ndev) {
+ rtnl_lock();
+ dev_mc_delete(mdev->rdmaoe.netdevs[ge->port - 1], mac, 6, 0);
+ rtnl_unlock();
+ dev_put(ndev);
+ }
+ list_del(&ge->list);
+ kfree(ge);
+ } else
+ printk(KERN_WARNING "could not find mgid entry\n");
+
+ mutex_unlock(&mqp->mutex);
+
+ return 0;
}
static int init_node_data(struct mlx4_ib_dev *dev)
@@ -541,15 +739,143 @@ static struct device_attribute *mlx4_class_attributes[] = {
&dev_attr_board_id
};
+static void mlx4_addrconf_ifid_eui48(u8 *eui, struct net_device *dev)
+{
+ memcpy(eui, dev->dev_addr, 3);
+ memcpy(eui + 5, dev->dev_addr + 3, 3);
+ eui[3] = 0xFF;
+ eui[4] = 0xFE;
+ eui[0] ^= 2;
+}
+
+static void update_gids_task(struct work_struct *work)
+{
+ struct update_gid_work *gw = container_of(work, struct update_gid_work, work);
+ struct mlx4_cmd_mailbox *mailbox;
+ union ib_gid *gids;
+ int err;
+ struct mlx4_dev *dev = gw->dev->dev;
+ struct ib_event event;
+
+ mailbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(mailbox)) {
+ printk(KERN_WARNING "update gid table failed %ld\n", PTR_ERR(mailbox));
+ return;
+ }
+
+ gids = mailbox->buf;
+ memcpy(gids, gw->gids, sizeof gw->gids);
+
+ err = mlx4_cmd(dev, mailbox->dma, MLX4_SET_PORT_GID_TABLE << 8 | gw->port,
+ 1, MLX4_CMD_SET_PORT, MLX4_CMD_TIME_CLASS_B);
+ if (err)
+ printk(KERN_WARNING "set port command failed\n");
+ else {
+ memcpy(gw->dev->rdmaoe.gid_table[gw->port - 1], gw->gids, sizeof gw->gids);
+ event.device = &gw->dev->ib_dev;
+ event.element.port_num = gw->port;
+ event.event = IB_EVENT_LID_CHANGE;
+ ib_dispatch_event(&event);
+ }
+
+ mlx4_free_cmd_mailbox(dev, mailbox);
+ kfree(gw);
+}
+
+static int update_ipv6_gids(struct mlx4_ib_dev *dev, int port, int clear)
+{
+ struct net_device *ndev = dev->rdmaoe.netdevs[port - 1];
+ struct update_gid_work *work;
+
+ work = kzalloc(sizeof *work, GFP_ATOMIC);
+ if (!work)
+ return -ENOMEM;
+
+ if (!clear) {
+ mlx4_addrconf_ifid_eui48(&work->gids[0].raw[8], ndev);
+ work->gids[0].global.subnet_prefix = cpu_to_be64(0xfe80000000000000LL);
+ }
+
+ INIT_WORK(&work->work, update_gids_task);
+ work->port = port;
+ work->dev = dev;
+ queue_work(wq, &work->work);
+
+ return 0;
+}
+
+static void handle_en_event(struct mlx4_ib_dev *dev, int port, unsigned long event)
+{
+ switch (event) {
+ case NETDEV_UP:
+ update_ipv6_gids(dev, port, 0);
+ break;
+
+ case NETDEV_DOWN:
+ update_ipv6_gids(dev, port, 1);
+ dev->rdmaoe.netdevs[port - 1] = NULL;
+ }
+}
+
+static void netdev_added(struct mlx4_ib_dev *dev, int port)
+{
+ update_ipv6_gids(dev, port, 0);
+}
+
+static void netdev_removed(struct mlx4_ib_dev *dev, int port)
+{
+ update_ipv6_gids(dev, port, 1);
+}
+
+static int mlx4_ib_netdev_event(struct notifier_block *this, unsigned long event,
+ void *ptr)
+{
+ struct net_device *dev = ptr;
+ struct mlx4_ib_dev *ibdev;
+ struct net_device *oldnd;
+ struct mlx4_ib_rdmaoe *rdmaoe;
+ int port;
+
+ if (!net_eq(dev_net(dev), &init_net))
+ return NOTIFY_DONE;
+
+ ibdev = container_of(this, struct mlx4_ib_dev, rdmaoe.nb);
+ rdmaoe = &ibdev->rdmaoe;
+
+ spin_lock(&rdmaoe->lock);
+ mlx4_foreach_ib_transport_port(port, ibdev->dev) {
+ oldnd = rdmaoe->netdevs[port - 1];
+ rdmaoe->netdevs[port - 1] = mlx4_get_prot_dev(ibdev->dev, MLX4_PROT_EN, port);
+ if (oldnd != rdmaoe->netdevs[port - 1]) {
+ if (rdmaoe->netdevs[port - 1])
+ netdev_added(ibdev, port);
+ else
+ netdev_removed(ibdev, port);
+ }
+ }
+
+ if (dev == rdmaoe->netdevs[0])
+ handle_en_event(ibdev, 1, event);
+ else if (dev == rdmaoe->netdevs[1])
+ handle_en_event(ibdev, 2, event);
+
+ spin_unlock(&rdmaoe->lock);
+
+ return NOTIFY_DONE;
+}
+
static void *mlx4_ib_add(struct mlx4_dev *dev)
{
struct mlx4_ib_dev *ibdev;
int num_ports = 0;
int i;
+ int err;
+ int port;
+ struct mlx4_ib_rdmaoe *rdmaoe;
printk_once(KERN_INFO "%s", mlx4_ib_version);
- mlx4_foreach_port(i, dev, MLX4_PORT_TYPE_IB)
+ mlx4_foreach_ib_transport_port(i, dev)
num_ports++;
/* No point in registering a device with no ports... */
@@ -562,6 +888,8 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
return NULL;
}
+ rdmaoe = &ibdev->rdmaoe;
+
if (mlx4_pd_alloc(dev, &ibdev->priv_pdn))
goto err_dealloc;
@@ -606,10 +934,12 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
(1ull << IB_USER_VERBS_CMD_CREATE_SRQ) |
(1ull << IB_USER_VERBS_CMD_MODIFY_SRQ) |
(1ull << IB_USER_VERBS_CMD_QUERY_SRQ) |
- (1ull << IB_USER_VERBS_CMD_DESTROY_SRQ);
+ (1ull << IB_USER_VERBS_CMD_DESTROY_SRQ) |
+ (1ull << IB_USER_VERBS_CMD_GET_MAC);
ibdev->ib_dev.query_device = mlx4_ib_query_device;
ibdev->ib_dev.query_port = mlx4_ib_query_port;
+ ibdev->ib_dev.get_port_transport = mlx4_ib_port_get_transport;
ibdev->ib_dev.query_gid = mlx4_ib_query_gid;
ibdev->ib_dev.query_pkey = mlx4_ib_query_pkey;
ibdev->ib_dev.modify_device = mlx4_ib_modify_device;
@@ -653,15 +983,26 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
ibdev->ib_dev.map_phys_fmr = mlx4_ib_map_phys_fmr;
ibdev->ib_dev.unmap_fmr = mlx4_ib_unmap_fmr;
ibdev->ib_dev.dealloc_fmr = mlx4_ib_fmr_dealloc;
+ ibdev->ib_dev.get_mac = mlx4_ib_get_mac;
+
+ mlx4_foreach_ib_transport_port(port, dev)
+ rdmaoe->netdevs[port - 1] = mlx4_get_prot_dev(dev, MLX4_PROT_EN, port);
+ spin_lock_init(&rdmaoe->lock);
+ if (dev->caps.flags & MLX4_DEV_CAP_FLAG_RDMAOE && !rdmaoe->nb.notifier_call) {
+ rdmaoe->nb.notifier_call = mlx4_ib_netdev_event;
+ err = register_netdevice_notifier(&rdmaoe->nb);
+ if (err)
+ goto err_map;
+ }
if (init_node_data(ibdev))
- goto err_map;
+ goto err_notif;
spin_lock_init(&ibdev->sm_lock);
mutex_init(&ibdev->cap_mask_mutex);
if (ib_register_device(&ibdev->ib_dev))
- goto err_map;
+ goto err_notif;
if (mlx4_ib_mad_init(ibdev))
goto err_reg;
@@ -679,6 +1020,10 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
err_reg:
ib_unregister_device(&ibdev->ib_dev);
+err_notif:
+ unregister_netdevice_notifier(&rdmaoe->nb);
+ flush_workqueue(wq);
+
err_map:
iounmap(ibdev->uar_map);
@@ -701,11 +1046,16 @@ static void mlx4_ib_remove(struct mlx4_dev *dev, void *ibdev_ptr)
mlx4_ib_mad_cleanup(ibdev);
ib_unregister_device(&ibdev->ib_dev);
+ if (ibdev->rdmaoe.nb.notifier_call) {
+ unregister_netdevice_notifier(&ibdev->rdmaoe.nb);
+ flush_workqueue(wq);
+ ibdev->rdmaoe.nb.notifier_call = NULL;
+ }
+ iounmap(ibdev->uar_map);
- for (p = 1; p <= ibdev->num_ports; ++p)
+ mlx4_foreach_port(p, dev, MLX4_PORT_TYPE_IB)
mlx4_CLOSE_PORT(dev, p);
- iounmap(ibdev->uar_map);
mlx4_uar_free(dev, &ibdev->priv_uar);
mlx4_pd_free(dev, ibdev->priv_pdn);
ib_dealloc_device(&ibdev->ib_dev);
@@ -747,17 +1097,31 @@ static void mlx4_ib_event(struct mlx4_dev *dev, void *ibdev_ptr,
static struct mlx4_interface mlx4_ib_interface = {
.add = mlx4_ib_add,
.remove = mlx4_ib_remove,
- .event = mlx4_ib_event
+ .event = mlx4_ib_event,
+ .protocol = MLX4_PROT_IB
};
static int __init mlx4_ib_init(void)
{
- return mlx4_register_interface(&mlx4_ib_interface);
+ int err;
+
+ wq = create_singlethread_workqueue("mlx4_ib");
+ if (!wq)
+ return -ENOMEM;
+
+ err = mlx4_register_interface(&mlx4_ib_interface);
+ if (err) {
+ destroy_workqueue(wq);
+ return err;
+ }
+
+ return 0;
}
static void __exit mlx4_ib_cleanup(void)
{
mlx4_unregister_interface(&mlx4_ib_interface);
+ destroy_workqueue(wq);
}
module_init(mlx4_ib_init);
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index a0435cd..7f3590b 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -112,6 +112,13 @@ enum mlx4_ib_qp_flags {
MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 << 1,
};
+struct gid_entry {
+ struct list_head list;
+ union ib_gid gid;
+ int added;
+ u8 port;
+};
+
struct mlx4_ib_qp {
struct ib_qp ibqp;
struct mlx4_qp mqp;
@@ -139,6 +146,7 @@ struct mlx4_ib_qp {
u8 sq_no_prefetch;
u8 state;
int mlx_type;
+ struct list_head gid_list;
};
struct mlx4_ib_srq {
@@ -333,4 +341,7 @@ static inline int mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah)
return !!(ah->av.ib.g_slid & 0x80);
}
+int mlx4_ib_add_mc(struct mlx4_ib_dev *mdev, struct mlx4_ib_qp *mqp,
+ union ib_gid *gid);
+
#endif /* MLX4_IB_H */
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index ce2a47e..7e7a165 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -470,6 +470,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd,
mutex_init(&qp->mutex);
spin_lock_init(&qp->sq.lock);
spin_lock_init(&qp->rq.lock);
+ INIT_LIST_HEAD(&qp->gid_list);
qp->state = IB_QPS_RESET;
if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
@@ -657,6 +658,16 @@ static void mlx4_ib_unlock_cqs(struct mlx4_ib_cq *send_cq, struct mlx4_ib_cq *re
}
}
+static void del_gid_entries(struct mlx4_ib_qp *qp)
+{
+ struct gid_entry *ge, *tmp;
+
+ list_for_each_entry_safe(ge, tmp, &qp->gid_list, list) {
+ list_del(&ge->list);
+ kfree(ge);
+ }
+}
+
static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp,
int is_user)
{
@@ -703,6 +714,8 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp,
if (!qp->ibqp.srq)
mlx4_db_free(dev->dev, &qp->db);
}
+
+ del_gid_entries(qp);
}
struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd,
@@ -914,6 +927,16 @@ static int mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah,
return 0;
}
+static void update_mcg_macs(struct mlx4_ib_dev *dev, struct mlx4_ib_qp *qp)
+{
+ struct gid_entry *ge, *tmp;
+
+ list_for_each_entry_safe(ge, tmp, &qp->gid_list, list) {
+ if (!ge->added && mlx4_ib_add_mc(dev, qp, &ge->gid))
+ ge->added = 1;
+ }
+}
+
static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,
const struct ib_qp_attr *attr, int attr_mask,
enum ib_qp_state cur_state, enum ib_qp_state new_state)
@@ -1147,8 +1170,10 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,
qp->atomic_rd_en = attr->qp_access_flags;
if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
qp->resp_depth = attr->max_dest_rd_atomic;
- if (attr_mask & IB_QP_PORT)
+ if (attr_mask & IB_QP_PORT) {
qp->port = attr->port_num;
+ update_mcg_macs(dev, qp);
+ }
if (attr_mask & IB_QP_ALT_PATH)
qp->alt_port = attr->alt_port_num;
diff --git a/drivers/net/mlx4/en_main.c b/drivers/net/mlx4/en_main.c
index 507e11f..00e72b3 100644
--- a/drivers/net/mlx4/en_main.c
+++ b/drivers/net/mlx4/en_main.c
@@ -100,6 +100,13 @@ static int mlx4_en_get_profile(struct mlx4_en_dev *mdev)
return 0;
}
+static void *get_netdev(struct mlx4_dev *dev, void *ctx, u8 port)
+{
+ struct mlx4_en_dev *endev = ctx;
+
+ return endev->pndev[port];
+}
+
static void mlx4_en_event(struct mlx4_dev *dev, void *endev_ptr,
enum mlx4_dev_event event, int port)
{
@@ -262,9 +269,11 @@ err_free_res:
}
static struct mlx4_interface mlx4_en_interface = {
- .add = mlx4_en_add,
- .remove = mlx4_en_remove,
- .event = mlx4_en_event,
+ .add = mlx4_en_add,
+ .remove = mlx4_en_remove,
+ .event = mlx4_en_event,
+ .get_prot_dev = get_netdev,
+ .protocol = MLX4_PROT_EN,
};
static int __init mlx4_en_init(void)
diff --git a/drivers/net/mlx4/en_port.c b/drivers/net/mlx4/en_port.c
index a29abe8..a249887 100644
--- a/drivers/net/mlx4/en_port.c
+++ b/drivers/net/mlx4/en_port.c
@@ -127,8 +127,8 @@ int mlx4_SET_PORT_qpn_calc(struct mlx4_dev *dev, u8 port, u32 base_qpn,
memset(context, 0, sizeof *context);
context->base_qpn = cpu_to_be32(base_qpn);
- context->promisc = cpu_to_be32(promisc << SET_PORT_PROMISC_SHIFT | base_qpn);
- context->mcast = cpu_to_be32(1 << SET_PORT_PROMISC_SHIFT | base_qpn);
+ context->promisc = cpu_to_be32(promisc << SET_PORT_PROMISC_EN_SHIFT | base_qpn);
+ context->mcast = cpu_to_be32(1 << SET_PORT_PROMISC_MODE_SHIFT | base_qpn);
context->intra_no_vlan = 0;
context->no_vlan = MLX4_NO_VLAN_IDX;
context->intra_vlan_miss = 0;
diff --git a/drivers/net/mlx4/en_port.h b/drivers/net/mlx4/en_port.h
index e6477f1..9354891 100644
--- a/drivers/net/mlx4/en_port.h
+++ b/drivers/net/mlx4/en_port.h
@@ -36,7 +36,8 @@
#define SET_PORT_GEN_ALL_VALID 0x7
-#define SET_PORT_PROMISC_SHIFT 31
+#define SET_PORT_PROMISC_EN_SHIFT 31
+#define SET_PORT_PROMISC_MODE_SHIFT 30
enum {
MLX4_CMD_SET_VLAN_FLTR = 0x47,
diff --git a/drivers/net/mlx4/intf.c b/drivers/net/mlx4/intf.c
index 0e7eb10..d64530e 100644
--- a/drivers/net/mlx4/intf.c
+++ b/drivers/net/mlx4/intf.c
@@ -159,3 +159,23 @@ void mlx4_unregister_device(struct mlx4_dev *dev)
mutex_unlock(&intf_mutex);
}
+
+void *mlx4_find_get_prot_dev(struct mlx4_dev *dev, enum mlx4_prot proto, int port)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_device_context *dev_ctx;
+ unsigned long flags;
+ void *result = NULL;
+
+ spin_lock_irqsave(&priv->ctx_lock, flags);
+
+ list_for_each_entry(dev_ctx, &priv->ctx_list, list)
+ if (dev_ctx->intf->protocol == proto && dev_ctx->intf->get_prot_dev) {
+ result = dev_ctx->intf->get_prot_dev(dev, dev_ctx->context, port);
+ break;
+ }
+
+ spin_unlock_irqrestore(&priv->ctx_lock, flags);
+
+ return result;
+}
diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c
index 291a505..2407396 100644
--- a/drivers/net/mlx4/main.c
+++ b/drivers/net/mlx4/main.c
@@ -104,6 +104,12 @@ static int log_mtts_per_seg = ilog2(MLX4_MTT_ENTRY_PER_SEG);
module_param_named(log_mtts_per_seg, log_mtts_per_seg, int, 0444);
MODULE_PARM_DESC(log_mtts_per_seg, "Log2 number of MTT entries per segment (1-5)");
+void *mlx4_get_prot_dev(struct mlx4_dev *dev, enum mlx4_prot proto, int port)
+{
+ return mlx4_find_get_prot_dev(dev, proto, port);
+}
+EXPORT_SYMBOL(mlx4_get_prot_dev);
+
int mlx4_check_port_params(struct mlx4_dev *dev,
enum mlx4_port_type *port_type)
{
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index bc72d6e..ab63445 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -362,6 +362,7 @@ int mlx4_restart_one(struct pci_dev *pdev);
int mlx4_register_device(struct mlx4_dev *dev);
void mlx4_unregister_device(struct mlx4_dev *dev);
void mlx4_dispatch_event(struct mlx4_dev *dev, enum mlx4_dev_event type, int port);
+void *mlx4_find_get_prot_dev(struct mlx4_dev *dev, enum mlx4_prot proto, int port);
struct mlx4_dev_cap;
struct mlx4_init_hca_param;
diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h
index 0f82293..22bd8d3 100644
--- a/include/linux/mlx4/cmd.h
+++ b/include/linux/mlx4/cmd.h
@@ -140,6 +140,7 @@ enum {
MLX4_SET_PORT_MAC_TABLE = 0x2,
MLX4_SET_PORT_VLAN_TABLE = 0x3,
MLX4_SET_PORT_PRIO_MAP = 0x4,
+ MLX4_SET_PORT_GID_TABLE = 0x5,
};
struct mlx4_dev;
diff --git a/include/linux/mlx4/driver.h b/include/linux/mlx4/driver.h
index 53c5fdb..0083256 100644
--- a/include/linux/mlx4/driver.h
+++ b/include/linux/mlx4/driver.h
@@ -44,15 +44,23 @@ enum mlx4_dev_event {
MLX4_DEV_EVENT_PORT_REINIT,
};
+enum mlx4_prot {
+ MLX4_PROT_IB,
+ MLX4_PROT_EN,
+};
+
struct mlx4_interface {
- void * (*add) (struct mlx4_dev *dev);
- void (*remove)(struct mlx4_dev *dev, void *context);
- void (*event) (struct mlx4_dev *dev, void *context,
- enum mlx4_dev_event event, int port);
+ void * (*add) (struct mlx4_dev *dev);
+ void (*remove)(struct mlx4_dev *dev, void *context);
+ void (*event) (struct mlx4_dev *dev, void *context,
+ enum mlx4_dev_event event, int port);
+ void * (*get_prot_dev) (struct mlx4_dev *dev, void *context, u8 port);
+ enum mlx4_prot protocol;
struct list_head list;
};
int mlx4_register_interface(struct mlx4_interface *intf);
void mlx4_unregister_interface(struct mlx4_interface *intf);
+void *mlx4_get_prot_dev(struct mlx4_dev *dev, enum mlx4_prot proto, int port);
#endif /* MLX4_DRIVER_H */
--
1.6.5.2
^ permalink raw reply related
* [PATCHv6 09/10] mlx4: Add support for RDMAoE - address resolution
From: Eli Cohen @ 2009-11-16 15:54 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
The following patch handles address vectors creation for RDMAoE ports. mlx4
needs the MAC address of the remote node to include it in the WQE of a UD QP or
in the QP context of connected QPs. Address resolution is done atomically in
the case of a link local address or a multicast GID and otherwise -EINVAL is
returned. mlx4 transport packets were changed too to accomodate for RDMAoE.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/hw/mlx4/ah.c | 181 +++++++++++++++++++++++++++------
drivers/infiniband/hw/mlx4/mad.c | 32 ++++---
drivers/infiniband/hw/mlx4/mlx4_ib.h | 18 +++-
drivers/infiniband/hw/mlx4/qp.c | 172 ++++++++++++++++++++++----------
drivers/net/mlx4/fw.c | 3 +-
include/linux/mlx4/device.h | 31 ++++++-
include/linux/mlx4/qp.h | 8 +-
7 files changed, 340 insertions(+), 105 deletions(-)
diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c
index c75ac94..3451929 100644
--- a/drivers/infiniband/hw/mlx4/ah.c
+++ b/drivers/infiniband/hw/mlx4/ah.c
@@ -31,63 +31,160 @@
*/
#include "mlx4_ib.h"
+#include <rdma/ib_addr.h>
+#include <linux/inet.h>
+#include <linux/string.h>
-struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+int mlx4_ib_resolve_grh(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah_attr,
+ u8 *mac, int *is_mcast, u8 port)
{
- struct mlx4_dev *dev = to_mdev(pd->device)->dev;
- struct mlx4_ib_ah *ah;
+ struct mlx4_ib_rdmaoe *rdmaoe = &dev->rdmaoe;
+ struct in6_addr in6;
- ah = kmalloc(sizeof *ah, GFP_ATOMIC);
- if (!ah)
- return ERR_PTR(-ENOMEM);
+ *is_mcast = 0;
+ spin_lock(&rdmaoe->lock);
+ if (!rdmaoe->netdevs[port - 1]) {
+ spin_unlock(&rdmaoe->lock);
+ return -EINVAL;
+ }
+ spin_unlock(&rdmaoe->lock);
- memset(&ah->av, 0, sizeof ah->av);
+ memcpy(&in6, ah_attr->grh.dgid.raw, sizeof in6);
+ if (rdma_link_local_addr(&in6))
+ rdma_get_ll_mac(&in6, mac);
+ else if (rdma_is_multicast_addr(&in6)) {
+ rdma_get_mcast_mac(&in6, mac);
+ *is_mcast = 1;
+ } else
+ return -EINVAL;
- ah->av.port_pd = cpu_to_be32(to_mpd(pd)->pdn | (ah_attr->port_num << 24));
- ah->av.g_slid = ah_attr->src_path_bits;
- ah->av.dlid = cpu_to_be16(ah_attr->dlid);
- if (ah_attr->static_rate) {
- ah->av.stat_rate = ah_attr->static_rate + MLX4_STAT_RATE_OFFSET;
- while (ah->av.stat_rate > IB_RATE_2_5_GBPS + MLX4_STAT_RATE_OFFSET &&
- !(1 << ah->av.stat_rate & dev->caps.stat_rate_support))
- --ah->av.stat_rate;
- }
- ah->av.sl_tclass_flowlabel = cpu_to_be32(ah_attr->sl << 28);
+ return 0;
+}
+
+static struct ib_ah *create_ib_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct mlx4_ib_ah *ah)
+{
+ struct mlx4_dev *dev = to_mdev(pd->device)->dev;
+
+ ah->av.ib.port_pd = cpu_to_be32(to_mpd(pd)->pdn | (ah_attr->port_num << 24));
+ ah->av.ib.g_slid = ah_attr->src_path_bits;
if (ah_attr->ah_flags & IB_AH_GRH) {
- ah->av.g_slid |= 0x80;
- ah->av.gid_index = ah_attr->grh.sgid_index;
- ah->av.hop_limit = ah_attr->grh.hop_limit;
- ah->av.sl_tclass_flowlabel |=
+ ah->av.ib.g_slid |= 0x80;
+ ah->av.ib.gid_index = ah_attr->grh.sgid_index;
+ ah->av.ib.hop_limit = ah_attr->grh.hop_limit;
+ ah->av.ib.sl_tclass_flowlabel |=
cpu_to_be32((ah_attr->grh.traffic_class << 20) |
ah_attr->grh.flow_label);
- memcpy(ah->av.dgid, ah_attr->grh.dgid.raw, 16);
+ memcpy(ah->av.ib.dgid, ah_attr->grh.dgid.raw, 16);
+ }
+
+ ah->av.ib.dlid = cpu_to_be16(ah_attr->dlid);
+ if (ah_attr->static_rate) {
+ ah->av.ib.stat_rate = ah_attr->static_rate + MLX4_STAT_RATE_OFFSET;
+ while (ah->av.ib.stat_rate > IB_RATE_2_5_GBPS + MLX4_STAT_RATE_OFFSET &&
+ !(1 << ah->av.ib.stat_rate & dev->caps.stat_rate_support))
+ --ah->av.ib.stat_rate;
}
+ ah->av.ib.sl_tclass_flowlabel = cpu_to_be32(ah_attr->sl << 28);
return &ah->ibah;
}
+static struct ib_ah *create_rdmaoe_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
+ struct mlx4_ib_ah *ah)
+{
+ struct mlx4_ib_dev *ibdev = to_mdev(pd->device);
+ struct mlx4_dev *dev = ibdev->dev;
+ u8 mac[6];
+ int err;
+ int is_mcast;
+
+ err = mlx4_ib_resolve_grh(ibdev, ah_attr, mac, &is_mcast, ah_attr->port_num);
+ if (err)
+ return ERR_PTR(err);
+
+ memcpy(ah->av.eth.mac_0_1, mac, 2);
+ memcpy(ah->av.eth.mac_2_5, mac + 2, 4);
+ ah->av.ib.port_pd = cpu_to_be32(to_mpd(pd)->pdn | (ah_attr->port_num << 24));
+ ah->av.ib.g_slid = 0x80;
+ if (ah_attr->static_rate) {
+ ah->av.ib.stat_rate = ah_attr->static_rate + MLX4_STAT_RATE_OFFSET;
+ while (ah->av.ib.stat_rate > IB_RATE_2_5_GBPS + MLX4_STAT_RATE_OFFSET &&
+ !(1 << ah->av.ib.stat_rate & dev->caps.stat_rate_support))
+ --ah->av.ib.stat_rate;
+ }
+
+ /*
+ * HW requires multicast LID so we just choose one.
+ */
+ if (is_mcast)
+ ah->av.ib.dlid = cpu_to_be16(0xc000);
+
+ memcpy(ah->av.ib.dgid, ah_attr->grh.dgid.raw, 16);
+ ah->av.ib.sl_tclass_flowlabel = cpu_to_be32(ah_attr->sl << 28);
+
+ return &ah->ibah;
+}
+
+struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
+{
+ struct mlx4_ib_ah *ah;
+ enum rdma_transport_type transport;
+ struct ib_ah *ret;
+
+ ah = kzalloc(sizeof *ah, GFP_ATOMIC);
+ if (!ah)
+ return ERR_PTR(-ENOMEM);
+
+ transport = rdma_port_get_transport(pd->device, ah_attr->port_num);
+ if (transport == RDMA_TRANSPORT_RDMAOE) {
+ if (!(ah_attr->ah_flags & IB_AH_GRH)) {
+ ret = ERR_PTR(-EINVAL);
+ goto out;
+ } else {
+ /* TBD: need to handle the case when we get called
+ in an atomic context and there we might sleep. We
+ don't expect this currently since we're working with
+ link local addresses which we can translate without
+ going to sleep */
+ ret = create_rdmaoe_ah(pd, ah_attr, ah);
+ if (IS_ERR(ret))
+ goto out;
+ else
+ return ret;
+ }
+ } else
+ return create_ib_ah(pd, ah_attr, ah); /* never fails */
+
+out:
+ kfree(ah);
+ return ret;
+}
+
int mlx4_ib_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
{
struct mlx4_ib_ah *ah = to_mah(ibah);
+ enum rdma_transport_type transport;
+ transport = rdma_port_get_transport(ibah->device, ah_attr->port_num);
memset(ah_attr, 0, sizeof *ah_attr);
- ah_attr->dlid = be16_to_cpu(ah->av.dlid);
- ah_attr->sl = be32_to_cpu(ah->av.sl_tclass_flowlabel) >> 28;
- ah_attr->port_num = be32_to_cpu(ah->av.port_pd) >> 24;
- if (ah->av.stat_rate)
- ah_attr->static_rate = ah->av.stat_rate - MLX4_STAT_RATE_OFFSET;
- ah_attr->src_path_bits = ah->av.g_slid & 0x7F;
+ ah_attr->dlid = transport == RDMA_TRANSPORT_IB ? be16_to_cpu(ah->av.ib.dlid) : 0;
+ ah_attr->sl = be32_to_cpu(ah->av.ib.sl_tclass_flowlabel) >> 28;
+ ah_attr->port_num = be32_to_cpu(ah->av.ib.port_pd) >> 24;
+ if (ah->av.ib.stat_rate)
+ ah_attr->static_rate = ah->av.ib.stat_rate - MLX4_STAT_RATE_OFFSET;
+ ah_attr->src_path_bits = ah->av.ib.g_slid & 0x7F;
if (mlx4_ib_ah_grh_present(ah)) {
ah_attr->ah_flags = IB_AH_GRH;
ah_attr->grh.traffic_class =
- be32_to_cpu(ah->av.sl_tclass_flowlabel) >> 20;
+ be32_to_cpu(ah->av.ib.sl_tclass_flowlabel) >> 20;
ah_attr->grh.flow_label =
- be32_to_cpu(ah->av.sl_tclass_flowlabel) & 0xfffff;
- ah_attr->grh.hop_limit = ah->av.hop_limit;
- ah_attr->grh.sgid_index = ah->av.gid_index;
- memcpy(ah_attr->grh.dgid.raw, ah->av.dgid, 16);
+ be32_to_cpu(ah->av.ib.sl_tclass_flowlabel) & 0xfffff;
+ ah_attr->grh.hop_limit = ah->av.ib.hop_limit;
+ ah_attr->grh.sgid_index = ah->av.ib.gid_index;
+ memcpy(ah_attr->grh.dgid.raw, ah->av.ib.dgid, 16);
}
return 0;
@@ -98,3 +195,21 @@ int mlx4_ib_destroy_ah(struct ib_ah *ah)
kfree(to_mah(ah));
return 0;
}
+
+int mlx4_ib_get_mac(struct ib_device *device, u8 port, u8 *gid, u8 *mac)
+{
+ int err;
+ struct mlx4_ib_dev *ibdev = to_mdev(device);
+ struct ib_ah_attr ah_attr = {
+ .port_num = port,
+ };
+ int is_mcast;
+
+ memcpy(ah_attr.grh.dgid.raw, gid, 16);
+ err = mlx4_ib_resolve_grh(ibdev, &ah_attr, mac, &is_mcast, port);
+ if (err)
+ ERR_PTR(err);
+
+ return 0;
+}
+
diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index 19e68ab..3df4f64 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -310,19 +310,25 @@ int mlx4_ib_mad_init(struct mlx4_ib_dev *dev)
struct ib_mad_agent *agent;
int p, q;
int ret;
+ enum rdma_transport_type tt;
- for (p = 0; p < dev->num_ports; ++p)
+ for (p = 0; p < dev->num_ports; ++p) {
+ tt = rdma_port_get_transport(&dev->ib_dev, p + 1);
for (q = 0; q <= 1; ++q) {
- agent = ib_register_mad_agent(&dev->ib_dev, p + 1,
- q ? IB_QPT_GSI : IB_QPT_SMI,
- NULL, 0, send_handler,
- NULL, NULL);
- if (IS_ERR(agent)) {
- ret = PTR_ERR(agent);
- goto err;
- }
- dev->send_agent[p][q] = agent;
+ if (tt == RDMA_TRANSPORT_IB) {
+ agent = ib_register_mad_agent(&dev->ib_dev, p + 1,
+ q ? IB_QPT_GSI : IB_QPT_SMI,
+ NULL, 0, send_handler,
+ NULL, NULL);
+ if (IS_ERR(agent)) {
+ ret = PTR_ERR(agent);
+ goto err;
+ }
+ dev->send_agent[p][q] = agent;
+ } else
+ dev->send_agent[p][q] = NULL;
}
+ }
return 0;
@@ -343,8 +349,10 @@ void mlx4_ib_mad_cleanup(struct mlx4_ib_dev *dev)
for (p = 0; p < dev->num_ports; ++p) {
for (q = 0; q <= 1; ++q) {
agent = dev->send_agent[p][q];
- dev->send_agent[p][q] = NULL;
- ib_unregister_mad_agent(agent);
+ if (agent) {
+ dev->send_agent[p][q] = NULL;
+ ib_unregister_mad_agent(agent);
+ }
}
if (dev->sm_ah[p])
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index 3486d76..a0435cd 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -138,6 +138,7 @@ struct mlx4_ib_qp {
u8 resp_depth;
u8 sq_no_prefetch;
u8 state;
+ int mlx_type;
};
struct mlx4_ib_srq {
@@ -157,7 +158,14 @@ struct mlx4_ib_srq {
struct mlx4_ib_ah {
struct ib_ah ibah;
- struct mlx4_av av;
+ union mlx4_ext_av av;
+};
+
+struct mlx4_ib_rdmaoe {
+ spinlock_t lock;
+ struct net_device *netdevs[MLX4_MAX_PORTS];
+ struct notifier_block nb;
+ union ib_gid gid_table[MLX4_MAX_PORTS][128];
};
struct mlx4_ib_dev {
@@ -176,6 +184,7 @@ struct mlx4_ib_dev {
struct mutex cap_mask_mutex;
bool ib_active;
+ struct mlx4_ib_rdmaoe rdmaoe;
};
static inline struct mlx4_ib_dev *to_mdev(struct ib_device *ibdev)
@@ -314,9 +323,14 @@ int mlx4_ib_map_phys_fmr(struct ib_fmr *ibfmr, u64 *page_list, int npages,
int mlx4_ib_unmap_fmr(struct list_head *fmr_list);
int mlx4_ib_fmr_dealloc(struct ib_fmr *fmr);
+int mlx4_ib_resolve_grh(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah_attr,
+ u8 *mac, int *is_mcast, u8 port);
+
+int mlx4_ib_get_mac(struct ib_device *device, u8 port, u8 *gid, u8 *mac);
+
static inline int mlx4_ib_ah_grh_present(struct mlx4_ib_ah *ah)
{
- return !!(ah->av.g_slid & 0x80);
+ return !!(ah->av.ib.g_slid & 0x80);
}
#endif /* MLX4_IB_H */
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 847030c..ce2a47e 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -32,6 +32,7 @@
*/
#include <linux/log2.h>
+#include <linux/netdevice.h>
#include <rdma/ib_cache.h>
#include <rdma/ib_pack.h>
@@ -47,23 +48,33 @@ enum {
enum {
MLX4_IB_DEFAULT_SCHED_QUEUE = 0x83,
- MLX4_IB_DEFAULT_QP0_SCHED_QUEUE = 0x3f
+ MLX4_IB_DEFAULT_QP0_SCHED_QUEUE = 0x3f,
+ MLX4_IB_LINK_TYPE_IB = 0,
+ MLX4_IB_LINK_TYPE_ETH = 1
};
enum {
/*
* Largest possible UD header: send with GRH and immediate data.
+ * 4 bytes added to accommodate for eth header instead of lrh
*/
- MLX4_IB_UD_HEADER_SIZE = 72,
+ MLX4_IB_UD_HEADER_SIZE = 76,
MLX4_IB_LSO_HEADER_SPARE = 128,
};
+enum {
+ MLX4_RDMAOE_ETHERTYPE = 0x8915
+};
+
struct mlx4_ib_sqp {
struct mlx4_ib_qp qp;
int pkey_index;
u32 qkey;
u32 send_psn;
- struct ib_ud_header ud_header;
+ union {
+ struct ib_ud_header ib;
+ struct eth_ud_header eth;
+ } hdr;
u8 header_buf[MLX4_IB_UD_HEADER_SIZE];
};
@@ -548,9 +559,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev, struct ib_pd *pd,
}
}
- if (sqpn) {
+ if (sqpn)
qpn = sqpn;
- } else {
+ else {
err = mlx4_qp_reserve_range(dev->dev, 1, 1, &qpn);
if (err)
goto err_wrid;
@@ -849,6 +860,12 @@ static void mlx4_set_sched(struct mlx4_qp_path *path, u8 port)
static int mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah,
struct mlx4_qp_path *path, u8 port)
{
+ int err;
+ int is_eth = rdma_port_get_transport(&dev->ib_dev, port) ==
+ RDMA_TRANSPORT_RDMAOE ? 1 : 0;
+ u8 mac[6];
+ int is_mcast;
+
path->grh_mylmc = ah->src_path_bits & 0x7f;
path->rlid = cpu_to_be16(ah->dlid);
if (ah->static_rate) {
@@ -879,6 +896,21 @@ static int mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah,
path->sched_queue = MLX4_IB_DEFAULT_SCHED_QUEUE |
((port - 1) << 6) | ((ah->sl & 0xf) << 2);
+ if (is_eth) {
+ if (!(ah->ah_flags & IB_AH_GRH))
+ return -1;
+
+ err = mlx4_ib_resolve_grh(dev, ah, mac, &is_mcast, port);
+ if (err)
+ return err;
+
+ memcpy(path->dmac_h, mac, 2);
+ memcpy(path->dmac_l, mac + 2, 4);
+ path->ackto = MLX4_IB_LINK_TYPE_ETH;
+ /* use index 0 into MAC table for RDMAoE */
+ path->grh_mylmc &= 0x80;
+ }
+
return 0;
}
@@ -977,7 +1009,7 @@ static int __mlx4_ib_modify_qp(struct ib_qp *ibqp,
}
if (attr_mask & IB_QP_TIMEOUT) {
- context->pri_path.ackto = attr->timeout << 3;
+ context->pri_path.ackto |= (attr->timeout << 3);
optpar |= MLX4_QP_OPTPAR_ACK_TIMEOUT;
}
@@ -1223,79 +1255,109 @@ static int build_mlx_header(struct mlx4_ib_sqp *sqp, struct ib_send_wr *wr,
int header_size;
int spc;
int i;
+ void *tmp;
+ struct ib_ud_header *ib = NULL;
+ struct eth_ud_header *eth = NULL;
+ struct ib_unpacked_grh *grh;
+ struct ib_unpacked_bth *bth;
+ struct ib_unpacked_deth *deth;
send_size = 0;
for (i = 0; i < wr->num_sge; ++i)
send_size += wr->sg_list[i].length;
- ib_ud_header_init(send_size, mlx4_ib_ah_grh_present(ah), &sqp->ud_header);
+ if (rdma_port_get_transport(sqp->qp.ibqp.device, sqp->qp.port) == RDMA_TRANSPORT_IB) {
+ ib = &sqp->hdr.ib;
+ grh = &ib->grh;
+ bth = &ib->bth;
+ deth = &ib->deth;
+ ib_ud_header_init(send_size, mlx4_ib_ah_grh_present(ah), ib);
+ ib->lrh.service_level =
+ be32_to_cpu(ah->av.ib.sl_tclass_flowlabel) >> 28;
+ ib->lrh.destination_lid = ah->av.ib.dlid;
+ ib->lrh.source_lid = cpu_to_be16(ah->av.ib.g_slid & 0x7f);
+ } else {
+ eth = &sqp->hdr.eth;
+ grh = ð->grh;
+ bth = ð->bth;
+ deth = ð->deth;
+ ib_rdmaoe_ud_header_init(send_size, mlx4_ib_ah_grh_present(ah), eth);
+ }
- sqp->ud_header.lrh.service_level =
- be32_to_cpu(ah->av.sl_tclass_flowlabel) >> 28;
- sqp->ud_header.lrh.destination_lid = ah->av.dlid;
- sqp->ud_header.lrh.source_lid = cpu_to_be16(ah->av.g_slid & 0x7f);
if (mlx4_ib_ah_grh_present(ah)) {
- sqp->ud_header.grh.traffic_class =
- (be32_to_cpu(ah->av.sl_tclass_flowlabel) >> 20) & 0xff;
- sqp->ud_header.grh.flow_label =
- ah->av.sl_tclass_flowlabel & cpu_to_be32(0xfffff);
- sqp->ud_header.grh.hop_limit = ah->av.hop_limit;
- ib_get_cached_gid(ib_dev, be32_to_cpu(ah->av.port_pd) >> 24,
- ah->av.gid_index, &sqp->ud_header.grh.source_gid);
- memcpy(sqp->ud_header.grh.destination_gid.raw,
- ah->av.dgid, 16);
+ grh->traffic_class =
+ (be32_to_cpu(ah->av.ib.sl_tclass_flowlabel) >> 20) & 0xff;
+ grh->flow_label =
+ ah->av.ib.sl_tclass_flowlabel & cpu_to_be32(0xfffff);
+ grh->hop_limit = ah->av.ib.hop_limit;
+ ib_get_cached_gid(ib_dev, be32_to_cpu(ah->av.ib.port_pd) >> 24,
+ ah->av.ib.gid_index, &grh->source_gid);
+ memcpy(grh->destination_gid.raw,
+ ah->av.ib.dgid, 16);
}
mlx->flags &= cpu_to_be32(MLX4_WQE_CTRL_CQ_UPDATE);
- mlx->flags |= cpu_to_be32((!sqp->qp.ibqp.qp_num ? MLX4_WQE_MLX_VL15 : 0) |
- (sqp->ud_header.lrh.destination_lid ==
- IB_LID_PERMISSIVE ? MLX4_WQE_MLX_SLR : 0) |
- (sqp->ud_header.lrh.service_level << 8));
- mlx->rlid = sqp->ud_header.lrh.destination_lid;
+
+ if (ib) {
+ mlx->flags |= cpu_to_be32((!sqp->qp.ibqp.qp_num ? MLX4_WQE_MLX_VL15 : 0) |
+ (ib->lrh.destination_lid ==
+ IB_LID_PERMISSIVE ? MLX4_WQE_MLX_SLR : 0) |
+ (ib->lrh.service_level << 8));
+ mlx->rlid = ib->lrh.destination_lid;
+ }
switch (wr->opcode) {
case IB_WR_SEND:
- sqp->ud_header.bth.opcode = IB_OPCODE_UD_SEND_ONLY;
- sqp->ud_header.immediate_present = 0;
+ bth->opcode = IB_OPCODE_UD_SEND_ONLY;
+ if (ib)
+ ib->immediate_present = 0;
+ else
+ eth->immediate_present = 0;
break;
case IB_WR_SEND_WITH_IMM:
- sqp->ud_header.bth.opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;
- sqp->ud_header.immediate_present = 1;
- sqp->ud_header.immediate_data = wr->ex.imm_data;
+ bth->opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;
+ if (ib) {
+ ib->immediate_present = 1;
+ ib->immediate_data = wr->ex.imm_data;
+ } else {
+ eth->immediate_present = 1;
+ eth->immediate_data = wr->ex.imm_data;
+ }
break;
default:
return -EINVAL;
}
- sqp->ud_header.lrh.virtual_lane = !sqp->qp.ibqp.qp_num ? 15 : 0;
- if (sqp->ud_header.lrh.destination_lid == IB_LID_PERMISSIVE)
- sqp->ud_header.lrh.source_lid = IB_LID_PERMISSIVE;
- sqp->ud_header.bth.solicited_event = !!(wr->send_flags & IB_SEND_SOLICITED);
+ if (ib) {
+ ib->lrh.virtual_lane = !sqp->qp.ibqp.qp_num ? 15 : 0;
+ if (ib->lrh.destination_lid == IB_LID_PERMISSIVE)
+ ib->lrh.source_lid = IB_LID_PERMISSIVE;
+ } else {
+ memcpy(eth->eth.dmac_h, ah->av.eth.mac_0_1, 2);
+ memcpy(eth->eth.dmac_h + 2, ah->av.eth.mac_2_5, 2);
+ memcpy(eth->eth.dmac_l, ah->av.eth.mac_2_5 + 2, 2);
+ tmp = to_mdev(sqp->qp.ibqp.device)->rdmaoe.netdevs[sqp->qp.port - 1]->dev_addr;
+ memcpy(eth->eth.smac_h, tmp, 2);
+ memcpy(eth->eth.smac_l, tmp + 2, 4);
+ eth->eth.type = cpu_to_be16(MLX4_RDMAOE_ETHERTYPE);
+ }
+ bth->solicited_event = !!(wr->send_flags & IB_SEND_SOLICITED);
+
if (!sqp->qp.ibqp.qp_num)
ib_get_cached_pkey(ib_dev, sqp->qp.port, sqp->pkey_index, &pkey);
else
ib_get_cached_pkey(ib_dev, sqp->qp.port, wr->wr.ud.pkey_index, &pkey);
- sqp->ud_header.bth.pkey = cpu_to_be16(pkey);
- sqp->ud_header.bth.destination_qpn = cpu_to_be32(wr->wr.ud.remote_qpn);
- sqp->ud_header.bth.psn = cpu_to_be32((sqp->send_psn++) & ((1 << 24) - 1));
- sqp->ud_header.deth.qkey = cpu_to_be32(wr->wr.ud.remote_qkey & 0x80000000 ?
+ bth->pkey = cpu_to_be16(pkey);
+ bth->destination_qpn = cpu_to_be32(wr->wr.ud.remote_qpn);
+ bth->psn = cpu_to_be32((sqp->send_psn++) & ((1 << 24) - 1));
+ deth->qkey = cpu_to_be32(wr->wr.ud.remote_qkey & 0x80000000 ?
sqp->qkey : wr->wr.ud.remote_qkey);
- sqp->ud_header.deth.source_qpn = cpu_to_be32(sqp->qp.ibqp.qp_num);
-
- header_size = ib_ud_header_pack(&sqp->ud_header, sqp->header_buf);
-
- if (0) {
- printk(KERN_ERR "built UD header of size %d:\n", header_size);
- for (i = 0; i < header_size / 4; ++i) {
- if (i % 8 == 0)
- printk(" [%02x] ", i * 4);
- printk(" %08x",
- be32_to_cpu(((__be32 *) sqp->header_buf)[i]));
- if ((i + 1) % 8 == 0)
- printk("\n");
- }
- printk("\n");
- }
+ deth->source_qpn = cpu_to_be32(sqp->qp.ibqp.qp_num);
+
+ if (ib)
+ header_size = ib_ud_header_pack(ib, sqp->header_buf);
+ else
+ header_size = rdmaoe_ud_header_pack(eth, sqp->header_buf);
/*
* Inline data segments may not cross a 64 byte boundary. If
@@ -1419,6 +1481,8 @@ static void set_datagram_seg(struct mlx4_wqe_datagram_seg *dseg,
memcpy(dseg->av, &to_mah(wr->wr.ud.ah)->av, sizeof (struct mlx4_av));
dseg->dqpn = cpu_to_be32(wr->wr.ud.remote_qpn);
dseg->qkey = cpu_to_be32(wr->wr.ud.remote_qkey);
+ dseg->vlan = to_mah(wr->wr.ud.ah)->av.eth.vlan;
+ memcpy(dseg->mac_0_1, to_mah(wr->wr.ud.ah)->av.eth.mac_0_1, 6);
}
static void set_mlx_icrc_seg(void *dseg)
diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c
index 7194be3..1a8d375 100644
--- a/drivers/net/mlx4/fw.c
+++ b/drivers/net/mlx4/fw.c
@@ -98,7 +98,8 @@ static void dump_dev_cap_flags(struct mlx4_dev *dev, u32 flags)
[20] = "Address vector port checking support",
[21] = "UD multicast support",
[24] = "Demand paging support",
- [25] = "Router support"
+ [25] = "Router support",
+ [30] = "RDMAoE support"
};
int i;
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index e92d1bf..5412e94 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -67,7 +67,8 @@ enum {
MLX4_DEV_CAP_FLAG_ATOMIC = 1 << 18,
MLX4_DEV_CAP_FLAG_RAW_MCAST = 1 << 19,
MLX4_DEV_CAP_FLAG_UD_AV_PORT = 1 << 20,
- MLX4_DEV_CAP_FLAG_UD_MCAST = 1 << 21
+ MLX4_DEV_CAP_FLAG_UD_MCAST = 1 << 21,
+ MLX4_DEV_CAP_FLAG_RDMAOE = 1 << 30
};
enum {
@@ -373,6 +374,28 @@ struct mlx4_av {
u8 dgid[16];
};
+struct mlx4_eth_av {
+ __be32 port_pd;
+ u8 reserved1;
+ u8 smac_idx;
+ u16 reserved2;
+ u8 reserved3;
+ u8 gid_index;
+ u8 stat_rate;
+ u8 hop_limit;
+ __be32 sl_tclass_flowlabel;
+ u8 dgid[16];
+ u32 reserved4[2];
+ __be16 vlan;
+ u8 mac_0_1[2];
+ u8 mac_2_5[4];
+};
+
+union mlx4_ext_av {
+ struct mlx4_av ib;
+ struct mlx4_eth_av eth;
+};
+
struct mlx4_dev {
struct pci_dev *pdev;
unsigned long flags;
@@ -401,6 +424,12 @@ struct mlx4_init_port_param {
if (((type) == MLX4_PORT_TYPE_IB ? (dev)->caps.port_mask : \
~(dev)->caps.port_mask) & 1 << ((port) - 1))
+#define mlx4_foreach_ib_transport_port(port, dev) \
+ for ((port) = 1; (port) <= (dev)->caps.num_ports; (port)++) \
+ if (((dev)->caps.port_mask & 1 << ((port) - 1)) || \
+ ((dev)->caps.flags & MLX4_DEV_CAP_FLAG_RDMAOE))
+
+
int mlx4_buf_alloc(struct mlx4_dev *dev, int size, int max_direct,
struct mlx4_buf *buf);
void mlx4_buf_free(struct mlx4_dev *dev, int size, struct mlx4_buf *buf);
diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h
index 9f29d86..43bfef2 100644
--- a/include/linux/mlx4/qp.h
+++ b/include/linux/mlx4/qp.h
@@ -112,7 +112,9 @@ struct mlx4_qp_path {
u8 snooper_flags;
u8 reserved3[2];
u8 counter_index;
- u8 reserved4[7];
+ u8 reserved4;
+ u8 dmac_h[2];
+ u8 dmac_l[4];
};
struct mlx4_qp_context {
@@ -219,7 +221,9 @@ struct mlx4_wqe_datagram_seg {
__be32 av[8];
__be32 dqpn;
__be32 qkey;
- __be32 reservd[2];
+ __be16 vlan;
+ u8 mac_0_1[2];
+ u8 mac_2_5[4];
};
struct mlx4_wqe_lso_seg {
--
1.6.5.2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCHv6 08/10] ib_core: Add API to support RDMAoE from userspace
From: Eli Cohen @ 2009-11-16 15:54 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
Add ib_uverbs_get_mac() to be used by ibv_create_ah() to retirieve the remore
port's MAC address. Port transport is also returned by ibv_query_port().
ABI version is incremented from 6 to 7.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/core/uverbs.h | 1 +
drivers/infiniband/core/uverbs_cmd.c | 32 ++++++++++++++++++++++++++++++++
drivers/infiniband/core/uverbs_main.c | 1 +
drivers/infiniband/core/verbs.c | 10 ++++++++++
include/rdma/ib_user_verbs.h | 21 ++++++++++++++++++---
include/rdma/ib_verbs.h | 12 ++++++++++++
6 files changed, 74 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/uverbs.h b/drivers/infiniband/core/uverbs.h
index b3ea958..e69b04c 100644
--- a/drivers/infiniband/core/uverbs.h
+++ b/drivers/infiniband/core/uverbs.h
@@ -194,5 +194,6 @@ IB_UVERBS_DECLARE_CMD(create_srq);
IB_UVERBS_DECLARE_CMD(modify_srq);
IB_UVERBS_DECLARE_CMD(query_srq);
IB_UVERBS_DECLARE_CMD(destroy_srq);
+IB_UVERBS_DECLARE_CMD(get_mac);
#endif /* UVERBS_H */
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 56feab6..012aadf 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -452,6 +452,7 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file *file,
resp.active_width = attr.active_width;
resp.active_speed = attr.active_speed;
resp.phys_state = attr.phys_state;
+ resp.transport = attr.transport;
if (copy_to_user((void __user *) (unsigned long) cmd.response,
&resp, sizeof resp))
@@ -1824,6 +1825,37 @@ err:
return ret;
}
+ssize_t ib_uverbs_get_mac(struct ib_uverbs_file *file, const char __user *buf,
+ int in_len, int out_len)
+{
+ struct ib_uverbs_get_mac cmd;
+ struct ib_uverbs_get_mac_resp resp;
+ int ret;
+ struct ib_pd *pd;
+
+ if (out_len < sizeof resp)
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, buf, sizeof cmd))
+ return -EFAULT;
+
+ pd = idr_read_pd(cmd.pd_handle, file->ucontext);
+ if (!pd)
+ return -EINVAL;
+
+ ret = ib_get_mac(pd->device, cmd.port, cmd.gid, resp.mac);
+ put_pd_read(pd);
+ if (!ret) {
+ if (copy_to_user((void __user *) (unsigned long) cmd.response,
+ &resp, sizeof resp))
+ return -EFAULT;
+
+ return in_len;
+ }
+
+ return ret;
+}
+
ssize_t ib_uverbs_destroy_ah(struct ib_uverbs_file *file,
const char __user *buf, int in_len, int out_len)
{
diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c
index aec0fbd..d8f98f3 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -109,6 +109,7 @@ static ssize_t (*uverbs_cmd_table[])(struct ib_uverbs_file *file,
[IB_USER_VERBS_CMD_MODIFY_SRQ] = ib_uverbs_modify_srq,
[IB_USER_VERBS_CMD_QUERY_SRQ] = ib_uverbs_query_srq,
[IB_USER_VERBS_CMD_DESTROY_SRQ] = ib_uverbs_destroy_srq,
+ [IB_USER_VERBS_CMD_GET_MAC] = ib_uverbs_get_mac,
};
static struct vfsmount *uverbs_event_mnt;
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index d81e217..aaa8778 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -925,3 +925,13 @@ int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid)
return qp->device->detach_mcast(qp, gid, lid);
}
EXPORT_SYMBOL(ib_detach_mcast);
+
+int ib_get_mac(struct ib_device *device, u8 port, u8 *gid, u8 *mac)
+{
+ if (!device->get_mac)
+ return -ENOSYS;
+
+ return device->get_mac(device, port, gid, mac);
+}
+EXPORT_SYMBOL(ib_get_mac);
+
diff --git a/include/rdma/ib_user_verbs.h b/include/rdma/ib_user_verbs.h
index a17f771..49eee8a 100644
--- a/include/rdma/ib_user_verbs.h
+++ b/include/rdma/ib_user_verbs.h
@@ -42,7 +42,7 @@
* Increment this value if any changes that break userspace ABI
* compatibility are made.
*/
-#define IB_USER_VERBS_ABI_VERSION 6
+#define IB_USER_VERBS_ABI_VERSION 7
enum {
IB_USER_VERBS_CMD_GET_CONTEXT,
@@ -81,7 +81,8 @@ enum {
IB_USER_VERBS_CMD_MODIFY_SRQ,
IB_USER_VERBS_CMD_QUERY_SRQ,
IB_USER_VERBS_CMD_DESTROY_SRQ,
- IB_USER_VERBS_CMD_POST_SRQ_RECV
+ IB_USER_VERBS_CMD_POST_SRQ_RECV,
+ IB_USER_VERBS_CMD_GET_MAC
};
/*
@@ -205,7 +206,8 @@ struct ib_uverbs_query_port_resp {
__u8 active_width;
__u8 active_speed;
__u8 phys_state;
- __u8 reserved[3];
+ __u8 transport;
+ __u8 reserved[2];
};
struct ib_uverbs_alloc_pd {
@@ -621,6 +623,19 @@ struct ib_uverbs_destroy_ah {
__u32 ah_handle;
};
+struct ib_uverbs_get_mac {
+ __u64 response;
+ __u32 pd_handle;
+ __u8 port;
+ __u8 reserved[3];
+ __u8 gid[16];
+};
+
+struct ib_uverbs_get_mac_resp {
+ __u8 mac[6];
+ __u16 reserved;
+};
+
struct ib_uverbs_attach_mcast {
__u8 gid[16];
__u32 qp_handle;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index d9146c4..bf6e860 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1131,6 +1131,9 @@ struct ib_device {
struct ib_grh *in_grh,
struct ib_mad *in_mad,
struct ib_mad *out_mad);
+ int (*get_mac)(struct ib_device *device, u8 port,
+ u8 *gid, u8 *mac);
+
struct ib_dma_mapping_ops *dma_ops;
@@ -2037,4 +2040,13 @@ int ib_attach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid);
*/
int ib_detach_mcast(struct ib_qp *qp, union ib_gid *gid, u16 lid);
+/**
+ * ib_get_mac - get the mac address for the specified gid
+ * @device: IB device used for traffic
+ * @port: port number used.
+ * @gid: gid to be resolved into mac
+ * @mac: mac of the port bearing this gid
+ */
+int ib_get_mac(struct ib_device *device, u8 port, u8 *gid, u8 *mac);
+
#endif /* IB_VERBS_H */
--
1.6.5.2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCHv6 07/10] ib_core: RDMAoE UD packet packing support
From: Eli Cohen @ 2009-11-16 15:53 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
Add support functions to aid in packing RDMAoE packets.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/core/ud_header.c | 111 +++++++++++++++++++++++++++++++++++
include/rdma/ib_pack.h | 26 ++++++++
2 files changed, 137 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/core/ud_header.c b/drivers/infiniband/core/ud_header.c
index 8ec7876..d04b6f2 100644
--- a/drivers/infiniband/core/ud_header.c
+++ b/drivers/infiniband/core/ud_header.c
@@ -80,6 +80,29 @@ static const struct ib_field lrh_table[] = {
.size_bits = 16 }
};
+static const struct ib_field eth_table[] = {
+ { STRUCT_FIELD(eth, dmac_h),
+ .offset_words = 0,
+ .offset_bits = 0,
+ .size_bits = 32 },
+ { STRUCT_FIELD(eth, dmac_l),
+ .offset_words = 1,
+ .offset_bits = 0,
+ .size_bits = 16 },
+ { STRUCT_FIELD(eth, smac_h),
+ .offset_words = 1,
+ .offset_bits = 16,
+ .size_bits = 16 },
+ { STRUCT_FIELD(eth, smac_l),
+ .offset_words = 2,
+ .offset_bits = 0,
+ .size_bits = 32 },
+ { STRUCT_FIELD(eth, type),
+ .offset_words = 3,
+ .offset_bits = 0,
+ .size_bits = 16 }
+};
+
static const struct ib_field grh_table[] = {
{ STRUCT_FIELD(grh, ip_version),
.offset_words = 0,
@@ -241,6 +264,53 @@ void ib_ud_header_init(int payload_bytes,
EXPORT_SYMBOL(ib_ud_header_init);
/**
+ * ib_rdmaoe_ud_header_init - Initialize UD header structure
+ * @payload_bytes:Length of packet payload
+ * @grh_present:GRH flag (if non-zero, GRH will be included)
+ * @header:Structure to initialize
+ *
+ * ib_rdmaoe_ud_header_init() initializes the grh.ip_version, grh.payload_length,
+ * grh.next_header, bth.opcode, bth.pad_count and
+ * bth.transport_header_version fields of a &struct eth_ud_header given
+ * the payload length and whether a GRH will be included.
+ */
+void ib_rdmaoe_ud_header_init(int payload_bytes,
+ int grh_present,
+ struct eth_ud_header *header)
+{
+ int header_len;
+
+ memset(header, 0, sizeof *header);
+
+ header_len =
+ sizeof header->eth +
+ IB_BTH_BYTES +
+ IB_DETH_BYTES;
+ if (grh_present)
+ header_len += IB_GRH_BYTES;
+
+ header->grh_present = grh_present;
+ if (grh_present) {
+ header->grh.ip_version = 6;
+ header->grh.payload_length =
+ cpu_to_be16((IB_BTH_BYTES +
+ IB_DETH_BYTES +
+ payload_bytes +
+ 4 + /* ICRC */
+ 3) & ~3); /* round up */
+ header->grh.next_header = 0x1b;
+ }
+
+ if (header->immediate_present)
+ header->bth.opcode = IB_OPCODE_UD_SEND_ONLY_WITH_IMMEDIATE;
+ else
+ header->bth.opcode = IB_OPCODE_UD_SEND_ONLY;
+ header->bth.pad_count = (4 - payload_bytes) & 3;
+ header->bth.transport_header_version = 0;
+}
+EXPORT_SYMBOL(ib_rdmaoe_ud_header_init);
+
+/**
* ib_ud_header_pack - Pack UD header struct into wire format
* @header:UD header struct
* @buf:Buffer to pack into
@@ -281,6 +351,47 @@ int ib_ud_header_pack(struct ib_ud_header *header,
EXPORT_SYMBOL(ib_ud_header_pack);
/**
+ * rdmaoe_ud_header_pack - Pack UD header struct into eth wire format
+ * @header:UD header struct
+ * @buf:Buffer to pack into
+ *
+ * ib_ud_header_pack() packs the UD header structure @header into wire
+ * format in the buffer @buf.
+ */
+int rdmaoe_ud_header_pack(struct eth_ud_header *header,
+ void *buf)
+{
+ int len = 0;
+
+ ib_pack(eth_table, ARRAY_SIZE(eth_table),
+ &header->eth, buf);
+ len += IB_ETH_BYTES;
+
+ if (header->grh_present) {
+ ib_pack(grh_table, ARRAY_SIZE(grh_table),
+ &header->grh, buf + len);
+ len += IB_GRH_BYTES;
+ }
+
+ ib_pack(bth_table, ARRAY_SIZE(bth_table),
+ &header->bth, buf + len);
+ len += IB_BTH_BYTES;
+
+ ib_pack(deth_table, ARRAY_SIZE(deth_table),
+ &header->deth, buf + len);
+ len += IB_DETH_BYTES;
+
+ if (header->immediate_present) {
+ memcpy(buf + len, &header->immediate_data,
+ sizeof header->immediate_data);
+ len += sizeof header->immediate_data;
+ }
+
+ return len;
+}
+EXPORT_SYMBOL(rdmaoe_ud_header_pack);
+
+/**
* ib_ud_header_unpack - Unpack UD header struct from wire format
* @header:UD header struct
* @buf:Buffer to pack into
diff --git a/include/rdma/ib_pack.h b/include/rdma/ib_pack.h
index d7fc45c..bf199eb 100644
--- a/include/rdma/ib_pack.h
+++ b/include/rdma/ib_pack.h
@@ -37,6 +37,7 @@
enum {
IB_LRH_BYTES = 8,
+ IB_ETH_BYTES = 14,
IB_GRH_BYTES = 40,
IB_BTH_BYTES = 12,
IB_DETH_BYTES = 8
@@ -210,6 +211,14 @@ struct ib_unpacked_deth {
__be32 source_qpn;
};
+struct ib_unpacked_eth {
+ u8 dmac_h[4];
+ u8 dmac_l[2];
+ u8 smac_h[2];
+ u8 smac_l[4];
+ __be16 type;
+};
+
struct ib_ud_header {
struct ib_unpacked_lrh lrh;
int grh_present;
@@ -220,6 +229,16 @@ struct ib_ud_header {
__be32 immediate_data;
};
+struct eth_ud_header {
+ struct ib_unpacked_eth eth;
+ int grh_present;
+ struct ib_unpacked_grh grh;
+ struct ib_unpacked_bth bth;
+ struct ib_unpacked_deth deth;
+ int immediate_present;
+ __be32 immediate_data;
+};
+
void ib_pack(const struct ib_field *desc,
int desc_len,
void *structure,
@@ -234,10 +253,17 @@ void ib_ud_header_init(int payload_bytes,
int grh_present,
struct ib_ud_header *header);
+void ib_rdmaoe_ud_header_init(int payload_bytes,
+ int grh_present,
+ struct eth_ud_header *header);
+
int ib_ud_header_pack(struct ib_ud_header *header,
void *buf);
int ib_ud_header_unpack(void *buf,
struct ib_ud_header *header);
+int rdmaoe_ud_header_pack(struct eth_ud_header *header,
+ void *buf);
+
#endif /* IB_PACK_H */
--
1.6.5.2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCHv6 06/10] ib_core: CMA device binding
From: Eli Cohen @ 2009-11-16 15:53 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
Add support for RDMAoE device binding and IP --> GID resolution. Path resolving
and multicast joining are implemented within cma.c by filling the responses and
pushing the callbacks to the cma work queue. IP->GID resolution always yields
IPv6 link local addresses - remote GIDs are derived from the destination MAC
address of the remote port. Multicast GIDs are always mapped to multicast MACs
as is done in IPv6. Some helper functions are added to ib_addr.h. IPv4
multicast is enabled by translating IPv4 multicast addresses to IPv6 multicast
as described in
http://www.mail-archive.com/ipng-Va8kgSFw1KyzQ7sTrsxvMkEOCMrvLtNR@public.gmane.org/msg02134.html.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/core/cma.c | 218 ++++++++++++++++++++++++++++++++++++++-
drivers/infiniband/core/ucma.c | 31 ++++++
include/rdma/ib_addr.h | 92 +++++++++++++++++
3 files changed, 335 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 8dc3472..ad1cd75 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -58,6 +58,7 @@ MODULE_LICENSE("Dual BSD/GPL");
#define CMA_CM_RESPONSE_TIMEOUT 20
#define CMA_MAX_CM_RETRIES 15
#define CMA_CM_MRA_SETTING (IB_CM_MRA_FLAG_DELAY | 24)
+#define RDMAOE_PACKET_LIFETIME 18
static void cma_add_one(struct ib_device *device);
static void cma_remove_one(struct ib_device *device);
@@ -157,6 +158,7 @@ struct cma_multicast {
struct list_head list;
void *context;
struct sockaddr_storage addr;
+ struct kref mcref;
};
struct cma_work {
@@ -173,6 +175,12 @@ struct cma_ndev_work {
struct rdma_cm_event event;
};
+struct rdmaoe_mcast_work {
+ struct work_struct work;
+ struct rdma_id_private *id;
+ struct cma_multicast *mc;
+};
+
union cma_ip_addr {
struct in6_addr ip6;
struct {
@@ -290,6 +298,14 @@ static inline void cma_deref_dev(struct cma_device *cma_dev)
complete(&cma_dev->comp);
}
+static inline void release_mc(struct kref *kref)
+{
+ struct cma_multicast *mc = container_of(kref, struct cma_multicast, mcref);
+
+ kfree(mc->multicast.ib);
+ kfree(mc);
+}
+
static void cma_detach_from_dev(struct rdma_id_private *id_priv)
{
list_del(&id_priv->list);
@@ -340,6 +356,9 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv)
case RDMA_TRANSPORT_IWARP:
iw_addr_get_sgid(dev_addr, &gid);
break;
+ case RDMA_TRANSPORT_RDMAOE:
+ rdmaoe_addr_get_sgid(dev_addr, &gid);
+ break;
default:
return -ENODEV;
}
@@ -568,10 +587,16 @@ static int cma_ib_init_qp_attr(struct rdma_id_private *id_priv,
{
struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
int ret;
+ u16 pkey;
+
+ if (rdma_port_get_transport(id_priv->id.device, id_priv->id.port_num) ==
+ RDMA_TRANSPORT_IB)
+ pkey = ib_addr_get_pkey(dev_addr);
+ else
+ pkey = 0xffff;
ret = ib_find_cached_pkey(id_priv->id.device, id_priv->id.port_num,
- ib_addr_get_pkey(dev_addr),
- &qp_attr->pkey_index);
+ pkey, &qp_attr->pkey_index);
if (ret)
return ret;
@@ -601,6 +626,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
id_priv = container_of(id, struct rdma_id_private, id);
switch (rdma_port_get_transport(id_priv->id.device, id_priv->id.port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
if (!id_priv->cm_id.ib || cma_is_ud_ps(id_priv->id.ps))
ret = cma_ib_init_qp_attr(id_priv, qp_attr, qp_attr_mask);
else
@@ -828,8 +854,17 @@ static void cma_leave_mc_groups(struct rdma_id_private *id_priv)
mc = container_of(id_priv->mc_list.next,
struct cma_multicast, list);
list_del(&mc->list);
- ib_sa_free_multicast(mc->multicast.ib);
- kfree(mc);
+ switch (rdma_port_get_transport(id_priv->cma_dev->device, id_priv->id.port_num)) {
+ case RDMA_TRANSPORT_IB:
+ ib_sa_free_multicast(mc->multicast.ib);
+ kfree(mc);
+ break;
+ case RDMA_TRANSPORT_RDMAOE:
+ kref_put(&mc->mcref, release_mc);
+ break;
+ default:
+ break;
+ }
}
}
@@ -847,6 +882,7 @@ void rdma_destroy_id(struct rdma_cm_id *id)
mutex_unlock(&lock);
switch (rdma_port_get_transport(id_priv->id.device, id_priv->id.port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
if (id_priv->cm_id.ib && !IS_ERR(id_priv->cm_id.ib))
ib_destroy_cm_id(id_priv->cm_id.ib);
break;
@@ -1504,6 +1540,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
if (id->device) {
switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
ret = cma_ib_listen(id_priv);
if (ret)
goto err;
@@ -1719,6 +1756,66 @@ static int cma_resolve_iw_route(struct rdma_id_private *id_priv, int timeout_ms)
return 0;
}
+static int cma_resolve_rdmaoe_route(struct rdma_id_private *id_priv)
+{
+ struct rdma_route *route = &id_priv->id.route;
+ struct rdma_addr *addr = &route->addr;
+ struct cma_work *work;
+ int ret;
+ struct sockaddr_in *src_addr = (struct sockaddr_in *)&route->addr.src_addr;
+ struct sockaddr_in *dst_addr = (struct sockaddr_in *)&route->addr.dst_addr;
+
+ if (src_addr->sin_family != dst_addr->sin_family)
+ return -EINVAL;
+
+ work = kzalloc(sizeof *work, GFP_KERNEL);
+ if (!work)
+ return -ENOMEM;
+
+ work->id = id_priv;
+ INIT_WORK(&work->work, cma_work_handler);
+
+ route->path_rec = kzalloc(sizeof *route->path_rec, GFP_KERNEL);
+ if (!route->path_rec) {
+ ret = -ENOMEM;
+ goto err1;
+ }
+
+ route->num_paths = 1;
+
+ rdmaoe_mac_to_ll(&route->path_rec->sgid, addr->dev_addr.src_dev_addr);
+ rdmaoe_mac_to_ll(&route->path_rec->dgid, addr->dev_addr.dst_dev_addr);
+
+ route->path_rec->hop_limit = 2;
+ route->path_rec->reversible = 1;
+ route->path_rec->pkey = cpu_to_be16(0xffff);
+ route->path_rec->mtu_selector = 2;
+ route->path_rec->mtu = rdmaoe_get_mtu(addr->dev_addr.src_dev->mtu);
+ route->path_rec->rate_selector = 2;
+ route->path_rec->rate = rdmaoe_get_rate(addr->dev_addr.src_dev);
+ route->path_rec->packet_life_time_selector = 2;
+ route->path_rec->packet_life_time = RDMAOE_PACKET_LIFETIME;
+ if (!route->path_rec->mtu) {
+ ret = -EINVAL;
+ goto err2;
+ }
+
+ work->old_state = CMA_ROUTE_QUERY;
+ work->new_state = CMA_ROUTE_RESOLVED;
+ work->event.event = RDMA_CM_EVENT_ROUTE_RESOLVED;
+ work->event.status = 0;
+
+ queue_work(cma_wq, &work->work);
+
+ return 0;
+
+err2:
+ kfree(route->path_rec);
+err1:
+ kfree(work);
+ return ret;
+}
+
int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
{
struct rdma_id_private *id_priv;
@@ -1736,6 +1833,9 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
case RDMA_TRANSPORT_IWARP:
ret = cma_resolve_iw_route(id_priv, timeout_ms);
break;
+ case RDMA_TRANSPORT_RDMAOE:
+ ret = cma_resolve_rdmaoe_route(id_priv);
+ break;
default:
ret = -ENOSYS;
break;
@@ -2411,6 +2511,7 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
if (cma_is_ud_ps(id->ps))
ret = cma_resolve_ib_udp(id_priv, conn_param);
else
@@ -2524,6 +2625,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
if (cma_is_ud_ps(id->ps))
ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
conn_param->private_data,
@@ -2585,6 +2687,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
if (cma_is_ud_ps(id->ps))
ret = cma_send_sidr_rep(id_priv, IB_SIDR_REJECT,
private_data, private_data_len);
@@ -2616,6 +2719,7 @@ int rdma_disconnect(struct rdma_cm_id *id)
switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
+ case RDMA_TRANSPORT_RDMAOE:
ret = cma_modify_qp_err(id_priv);
if (ret)
goto out;
@@ -2742,6 +2846,94 @@ static int cma_join_ib_multicast(struct rdma_id_private *id_priv,
return 0;
}
+
+static void rdmaoe_mcast_work_handler(struct work_struct *work)
+{
+ struct rdmaoe_mcast_work *mw = container_of(work, struct rdmaoe_mcast_work, work);
+ struct cma_multicast *mc = mw->mc;
+ struct ib_sa_multicast *m = mc->multicast.ib;
+
+ mc->multicast.ib->context = mc;
+ cma_ib_mc_handler(0, m);
+ kref_put(&mc->mcref, release_mc);
+ kfree(mw);
+}
+
+static void cma_rdmaoe_set_mgid(struct sockaddr *addr, union ib_gid *mgid)
+{
+ struct sockaddr_in *sin = (struct sockaddr_in *)addr;
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)addr;
+
+ if (cma_any_addr(addr)) {
+ memset(mgid, 0, sizeof *mgid);
+ } else if (addr->sa_family == AF_INET6)
+ memcpy(mgid, &sin6->sin6_addr, sizeof *mgid);
+ else {
+ mgid->raw[0] = 0xff;
+ mgid->raw[1] = 0x0e;
+ mgid->raw[2] = 0xff;
+ mgid->raw[3] = 0xff;
+ mgid->raw[4] = 0;
+ mgid->raw[5] = 0;
+ mgid->raw[6] = 0;
+ mgid->raw[7] = 0;
+ mgid->raw[8] = 0;
+ mgid->raw[9] = 0;
+ mgid->raw[10] = 0xff;
+ mgid->raw[11] = 0xff;
+ *(__be32 *)(&mgid->raw[12]) = sin->sin_addr.s_addr;
+ }
+}
+
+static int cma_rdmaoe_join_multicast(struct rdma_id_private *id_priv,
+ struct cma_multicast *mc)
+{
+ struct rdmaoe_mcast_work *work;
+ struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
+ int err;
+ struct sockaddr *addr = (struct sockaddr *)&mc->addr;
+
+ if (cma_zero_addr((struct sockaddr *)&mc->addr))
+ return -EINVAL;
+
+ work = kzalloc(sizeof *work, GFP_KERNEL);
+ if (!work)
+ return -ENOMEM;
+
+ mc->multicast.ib = kzalloc(sizeof(struct ib_sa_multicast), GFP_KERNEL);
+ if (!mc->multicast.ib) {
+ err = -ENOMEM;
+ goto out1;
+ }
+
+ cma_rdmaoe_set_mgid(addr, &mc->multicast.ib->rec.mgid);
+
+ mc->multicast.ib->rec.pkey = cpu_to_be16(0xffff);
+ if (id_priv->id.ps == RDMA_PS_UDP)
+ mc->multicast.ib->rec.qkey = cpu_to_be32(RDMA_UDP_QKEY);
+ mc->multicast.ib->rec.rate = rdmaoe_get_rate(dev_addr->src_dev);
+ mc->multicast.ib->rec.hop_limit = 1;
+ mc->multicast.ib->rec.mtu = rdmaoe_get_mtu(dev_addr->src_dev->mtu);
+ if (!mc->multicast.ib->rec.mtu) {
+ err = -EINVAL;
+ goto out2;
+ }
+ rdmaoe_addr_get_sgid(dev_addr, &mc->multicast.ib->rec.port_gid);
+ work->id = id_priv;
+ work->mc = mc;
+ INIT_WORK(&work->work, rdmaoe_mcast_work_handler);
+ kref_get(&mc->mcref);
+ queue_work(cma_wq, &work->work);
+
+ return 0;
+
+out2:
+ kfree(mc->multicast.ib);
+out1:
+ kfree(work);
+ return err;
+}
+
int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
void *context)
{
@@ -2770,6 +2962,10 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
case RDMA_TRANSPORT_IB:
ret = cma_join_ib_multicast(id_priv, mc);
break;
+ case RDMA_TRANSPORT_RDMAOE:
+ kref_init(&mc->mcref);
+ ret = cma_rdmaoe_join_multicast(id_priv, mc);
+ break;
default:
ret = -ENOSYS;
break;
@@ -2781,6 +2977,7 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
spin_unlock_irq(&id_priv->lock);
kfree(mc);
}
+
return ret;
}
EXPORT_SYMBOL(rdma_join_multicast);
@@ -2801,8 +2998,17 @@ void rdma_leave_multicast(struct rdma_cm_id *id, struct sockaddr *addr)
ib_detach_mcast(id->qp,
&mc->multicast.ib->rec.mgid,
mc->multicast.ib->rec.mlid);
- ib_sa_free_multicast(mc->multicast.ib);
- kfree(mc);
+ switch (rdma_port_get_transport(id_priv->cma_dev->device, id_priv->id.port_num)) {
+ case RDMA_TRANSPORT_IB:
+ ib_sa_free_multicast(mc->multicast.ib);
+ kfree(mc);
+ break;
+ case RDMA_TRANSPORT_RDMAOE:
+ kref_put(&mc->mcref, release_mc);
+ break;
+ default:
+ break;
+ }
return;
}
}
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index fd831fd..71d6d66 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -582,6 +582,34 @@ static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp,
}
}
+static void ucma_copy_rdmaoe_route(struct rdma_ucm_query_route_resp *resp,
+ struct rdma_route *route)
+{
+ struct rdma_dev_addr *dev_addr;
+
+ resp->num_paths = route->num_paths;
+ switch (route->num_paths) {
+ case 0:
+ dev_addr = &route->addr.dev_addr;
+ rdmaoe_mac_to_ll((union ib_gid *) &resp->ib_route[0].dgid,
+ dev_addr->dst_dev_addr);
+ rdmaoe_addr_get_sgid(dev_addr,
+ (union ib_gid *) &resp->ib_route[0].sgid);
+ resp->ib_route[0].pkey = cpu_to_be16(0xffff);
+ break;
+ case 2:
+ ib_copy_path_rec_to_user(&resp->ib_route[1],
+ &route->path_rec[1]);
+ /* fall through */
+ case 1:
+ ib_copy_path_rec_to_user(&resp->ib_route[0],
+ &route->path_rec[0]);
+ break;
+ default:
+ break;
+ }
+}
+
static ssize_t ucma_query_route(struct ucma_file *file,
const char __user *inbuf,
int in_len, int out_len)
@@ -620,6 +648,9 @@ static ssize_t ucma_query_route(struct ucma_file *file,
case RDMA_TRANSPORT_IB:
ucma_copy_ib_route(&resp, &ctx->cm_id->route);
break;
+ case RDMA_TRANSPORT_RDMAOE:
+ ucma_copy_rdmaoe_route(&resp, &ctx->cm_id->route);
+ break;
default:
break;
}
diff --git a/include/rdma/ib_addr.h b/include/rdma/ib_addr.h
index 483057b..ab06fe9 100644
--- a/include/rdma/ib_addr.h
+++ b/include/rdma/ib_addr.h
@@ -39,6 +39,8 @@
#include <linux/netdevice.h>
#include <linux/socket.h>
#include <rdma/ib_verbs.h>
+#include <linux/ethtool.h>
+#include <rdma/ib_pack.h>
struct rdma_addr_client {
atomic_t refcount;
@@ -157,4 +159,94 @@ static inline void iw_addr_get_dgid(struct rdma_dev_addr *dev_addr,
memcpy(gid, dev_addr->dst_dev_addr, sizeof *gid);
}
+static inline void rdmaoe_mac_to_ll(union ib_gid *gid, u8 *mac)
+{
+ memset(gid->raw, 0, 16);
+ *((u32 *)gid->raw) = cpu_to_be32(0xfe800000);
+ gid->raw[12] = 0xfe;
+ gid->raw[11] = 0xff;
+ memcpy(gid->raw + 13, mac + 3, 3);
+ memcpy(gid->raw + 8, mac, 3);
+ gid->raw[8] ^= 2;
+}
+
+static inline void rdmaoe_addr_get_sgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ rdmaoe_mac_to_ll(gid, dev_addr->src_dev_addr);
+}
+
+static inline enum ib_mtu rdmaoe_get_mtu(int mtu)
+{
+ /*
+ * reduce IB headers from effective RDMAoE MTU. 28 stands for
+ * atomic header which is the biggest possible header after BTH
+ */
+ mtu = mtu - IB_GRH_BYTES - IB_BTH_BYTES - 28;
+
+ if (mtu >= ib_mtu_enum_to_int(IB_MTU_4096))
+ return IB_MTU_4096;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_2048))
+ return IB_MTU_2048;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_1024))
+ return IB_MTU_1024;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_512))
+ return IB_MTU_512;
+ else if (mtu >= ib_mtu_enum_to_int(IB_MTU_256))
+ return IB_MTU_256;
+ else
+ return 0;
+}
+
+static inline int rdmaoe_get_rate(struct net_device *dev)
+{
+ struct ethtool_cmd cmd;
+
+ if (!dev->ethtool_ops || !dev->ethtool_ops->get_settings ||
+ dev->ethtool_ops->get_settings(dev, &cmd))
+ return IB_RATE_PORT_CURRENT;
+
+ if (cmd.speed >= 40000)
+ return IB_RATE_40_GBPS;
+ else if (cmd.speed >= 30000)
+ return IB_RATE_30_GBPS;
+ else if (cmd.speed >= 20000)
+ return IB_RATE_20_GBPS;
+ else if (cmd.speed >= 10000)
+ return IB_RATE_10_GBPS;
+ else
+ return IB_RATE_PORT_CURRENT;
+}
+
+static inline int rdma_link_local_addr(struct in6_addr *addr)
+{
+ if (addr->s6_addr32[0] == cpu_to_be32(0xfe800000) &&
+ addr->s6_addr32[1] == 0)
+ return 1;
+
+ return 0;
+}
+
+static inline void rdma_get_ll_mac(struct in6_addr *addr, u8 *mac)
+{
+ memcpy(mac, &addr->s6_addr[8], 3);
+ memcpy(mac + 3, &addr->s6_addr[13], 3);
+ mac[0] ^= 2;
+}
+
+static inline int rdma_is_multicast_addr(struct in6_addr *addr)
+{
+ return addr->s6_addr[0] == 0xff ? 1 : 0;
+}
+
+static inline void rdma_get_mcast_mac(struct in6_addr *addr, u8 *mac)
+{
+ int i;
+
+ mac[0] = 0x33;
+ mac[1] = 0x33;
+ for (i = 2; i < 6; ++i)
+ mac[i] = addr->s6_addr[i + 10];
+}
+
#endif /* IB_ADDR_H */
--
1.6.5.2
^ permalink raw reply related
* [PATCHv6 03/10] ib_core: RDMAoE support only QP1
From: Eli Cohen @ 2009-11-16 15:53 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
Since RDMAoE is using Ethernet as its link layer, there is no need for QP0. QP1
is still needed since it handles communications between CM agents. This patch
will create only QP1 for RDMAoE ports.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/core/agent.c | 39 +++++++++++++++++++++++++--------------
drivers/infiniband/core/mad.c | 37 +++++++++++++++++++++++++++++--------
2 files changed, 54 insertions(+), 22 deletions(-)
diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
index ae7c288..45c6d20 100644
--- a/drivers/infiniband/core/agent.c
+++ b/drivers/infiniband/core/agent.c
@@ -48,6 +48,8 @@
struct ib_agent_port_private {
struct list_head port_list;
struct ib_mad_agent *agent[2];
+ struct ib_device *device;
+ u8 port_num;
};
static DEFINE_SPINLOCK(ib_agent_port_list_lock);
@@ -58,11 +60,10 @@ __ib_get_agent_port(struct ib_device *device, int port_num)
{
struct ib_agent_port_private *entry;
- list_for_each_entry(entry, &ib_agent_port_list, port_list) {
- if (entry->agent[0]->device == device &&
- entry->agent[0]->port_num == port_num)
+ list_for_each_entry(entry, &ib_agent_port_list, port_list)
+ if (entry->device == device && entry->port_num == port_num)
return entry;
- }
+
return NULL;
}
@@ -146,6 +147,7 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
struct ib_agent_port_private *port_priv;
unsigned long flags;
int ret;
+ enum rdma_transport_type tt;
/* Create new device info */
port_priv = kzalloc(sizeof *port_priv, GFP_KERNEL);
@@ -155,14 +157,17 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
goto error1;
}
- /* Obtain send only MAD agent for SMI QP */
- port_priv->agent[0] = ib_register_mad_agent(device, port_num,
- IB_QPT_SMI, NULL, 0,
- &agent_send_handler,
- NULL, NULL);
- if (IS_ERR(port_priv->agent[0])) {
- ret = PTR_ERR(port_priv->agent[0]);
- goto error2;
+ tt = rdma_port_get_transport(device, port_num);
+ if (tt == RDMA_TRANSPORT_IB) {
+ /* Obtain send only MAD agent for SMI QP */
+ port_priv->agent[0] = ib_register_mad_agent(device, port_num,
+ IB_QPT_SMI, NULL, 0,
+ &agent_send_handler,
+ NULL, NULL);
+ if (IS_ERR(port_priv->agent[0])) {
+ ret = PTR_ERR(port_priv->agent[0]);
+ goto error2;
+ }
}
/* Obtain send only MAD agent for GSI QP */
@@ -175,6 +180,9 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
goto error3;
}
+ port_priv->device = device;
+ port_priv->port_num = port_num;
+
spin_lock_irqsave(&ib_agent_port_list_lock, flags);
list_add_tail(&port_priv->port_list, &ib_agent_port_list);
spin_unlock_irqrestore(&ib_agent_port_list_lock, flags);
@@ -182,7 +190,8 @@ int ib_agent_port_open(struct ib_device *device, int port_num)
return 0;
error3:
- ib_unregister_mad_agent(port_priv->agent[0]);
+ if (tt == RDMA_TRANSPORT_IB)
+ ib_unregister_mad_agent(port_priv->agent[0]);
error2:
kfree(port_priv);
error1:
@@ -205,7 +214,9 @@ int ib_agent_port_close(struct ib_device *device, int port_num)
spin_unlock_irqrestore(&ib_agent_port_list_lock, flags);
ib_unregister_mad_agent(port_priv->agent[1]);
- ib_unregister_mad_agent(port_priv->agent[0]);
+ if (rdma_port_get_transport(device, port_num) == RDMA_TRANSPORT_IB)
+ ib_unregister_mad_agent(port_priv->agent[0]);
+
kfree(port_priv);
return 0;
}
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 28a47a8..857049c 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -2610,6 +2610,9 @@ static void cleanup_recv_queue(struct ib_mad_qp_info *qp_info)
struct ib_mad_private *recv;
struct ib_mad_list_head *mad_list;
+ if (!qp_info->qp)
+ return;
+
while (!list_empty(&qp_info->recv_queue.list)) {
mad_list = list_entry(qp_info->recv_queue.list.next,
@@ -2651,6 +2654,9 @@ static int ib_mad_port_start(struct ib_mad_port_private *port_priv)
for (i = 0; i < IB_MAD_QPS_CORE; i++) {
qp = port_priv->qp_info[i].qp;
+ if (!qp)
+ continue;
+
/*
* PKey index for QP1 is irrelevant but
* one is needed for the Reset to Init transition
@@ -2692,6 +2698,9 @@ static int ib_mad_port_start(struct ib_mad_port_private *port_priv)
}
for (i = 0; i < IB_MAD_QPS_CORE; i++) {
+ if (!port_priv->qp_info[i].qp)
+ continue;
+
ret = ib_mad_post_receive_mads(&port_priv->qp_info[i], NULL);
if (ret) {
printk(KERN_ERR PFX "Couldn't post receive WRs\n");
@@ -2770,6 +2779,9 @@ error:
static void destroy_mad_qp(struct ib_mad_qp_info *qp_info)
{
+ if (!qp_info->qp)
+ return;
+
ib_destroy_qp(qp_info->qp);
kfree(qp_info->snoop_table);
}
@@ -2785,6 +2797,7 @@ static int ib_mad_port_open(struct ib_device *device,
struct ib_mad_port_private *port_priv;
unsigned long flags;
char name[sizeof "ib_mad123"];
+ int has_smi;
/* Create new device info */
port_priv = kzalloc(sizeof *port_priv, GFP_KERNEL);
@@ -2800,7 +2813,11 @@ static int ib_mad_port_open(struct ib_device *device,
init_mad_qp(port_priv, &port_priv->qp_info[0]);
init_mad_qp(port_priv, &port_priv->qp_info[1]);
- cq_size = (mad_sendq_size + mad_recvq_size) * 2;
+ cq_size = mad_sendq_size + mad_recvq_size;
+ has_smi = rdma_port_get_transport(device, port_num) == RDMA_TRANSPORT_IB;
+ if (has_smi)
+ cq_size *= 2;
+
port_priv->cq = ib_create_cq(port_priv->device,
ib_mad_thread_completion_handler,
NULL, port_priv, cq_size, 0);
@@ -2824,9 +2841,11 @@ static int ib_mad_port_open(struct ib_device *device,
goto error5;
}
- ret = create_mad_qp(&port_priv->qp_info[0], IB_QPT_SMI);
- if (ret)
- goto error6;
+ if (has_smi) {
+ ret = create_mad_qp(&port_priv->qp_info[0], IB_QPT_SMI);
+ if (ret)
+ goto error6;
+ }
ret = create_mad_qp(&port_priv->qp_info[1], IB_QPT_GSI);
if (ret)
goto error7;
@@ -2915,7 +2934,8 @@ static void ib_mad_init_device(struct ib_device *device)
int start, end, i;
enum rdma_transport_type tt;
- if (!rdma_is_transport_supported(device, RDMA_TRANSPORT_IB))
+ if (!rdma_is_transport_supported(device, RDMA_TRANSPORT_IB) &&
+ !rdma_is_transport_supported(device, RDMA_TRANSPORT_RDMAOE))
return;
if (device->node_type == RDMA_NODE_IB_SWITCH) {
@@ -2928,7 +2948,7 @@ static void ib_mad_init_device(struct ib_device *device)
for (i = start; i <= end; i++) {
tt = rdma_port_get_transport(device, i);
- if (tt != RDMA_TRANSPORT_IB)
+ if (tt != RDMA_TRANSPORT_IB && tt != RDMA_TRANSPORT_RDMAOE)
continue;
if (ib_mad_port_open(device, i)) {
@@ -2956,7 +2976,8 @@ error:
i--;
while (i >= start) {
- if (rdma_port_get_transport(device, i) == RDMA_TRANSPORT_IB) {
+ tt = rdma_port_get_transport(device, i);
+ if (tt == RDMA_TRANSPORT_IB || tt == RDMA_TRANSPORT_RDMAOE) {
if (ib_agent_port_close(device, i))
printk(KERN_ERR PFX "Couldn't close %s port %d "
"for agents\n",
@@ -2983,7 +3004,7 @@ static void ib_mad_remove_device(struct ib_device *device)
}
for (i = 0; i < num_ports; i++, cur_port++) {
tt = rdma_port_get_transport(device, i);
- if (tt == RDMA_TRANSPORT_IB) {
+ if (tt == RDMA_TRANSPORT_IB || tt == RDMA_TRANSPORT_RDMAOE) {
if (ib_agent_port_close(device, cur_port))
printk(KERN_ERR PFX "Couldn't close %s port %d "
"for agents\n",
--
1.6.5.2
^ permalink raw reply related
* [PATCHv6 01/10] ib_core: Refine device personality from node type to port type
From: Eli Cohen @ 2009-11-16 15:53 UTC (permalink / raw)
To: Roland Dreier; +Cc: Linux RDMA list, netdev, ewg
As a preparation to devices that, in general, support a different transport
protocol for each port, specifically RDMAoE, this patch defines a transport
type for each of a device's ports. As a result, rdma_node_get_transport() has
been unexported and is used internally by the implementation of the new API,
rdma_port_get_transport(), which gives the transport protocol of the queried
port. rdma_is_transport_supported() is also added to be used for verifying if a
given device supports a given protocol on any of its ports. All references to
rdma_node_get_transport() are changed to use the new APIs. Also, ib_port_attr
is extended to contain enum rdma_transport_type.
Signed-off-by: Eli Cohen <eli-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
drivers/infiniband/core/cm.c | 25 +++++++++----
drivers/infiniband/core/cma.c | 54 +++++++++++++++--------------
drivers/infiniband/core/mad.c | 49 ++++++++++++++++---------
drivers/infiniband/core/multicast.c | 4 +-
drivers/infiniband/core/sa_query.c | 39 +++++++++++++--------
drivers/infiniband/core/ucm.c | 8 +++-
drivers/infiniband/core/ucma.c | 2 +-
drivers/infiniband/core/user_mad.c | 6 +++-
drivers/infiniband/core/verbs.c | 25 ++++++++++++-
drivers/infiniband/ulp/ipoib/ipoib_main.c | 12 +++---
include/rdma/ib_verbs.h | 11 ++++--
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 3 +-
net/sunrpc/xprtrdma/svc_rdma_transport.c | 2 +-
13 files changed, 153 insertions(+), 87 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index 5130fc5..d082f59 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -3678,8 +3678,9 @@ static void cm_add_one(struct ib_device *ib_device)
unsigned long flags;
int ret;
u8 i;
+ enum rdma_transport_type tt;
- if (rdma_node_get_transport(ib_device->node_type) != RDMA_TRANSPORT_IB)
+ if (!rdma_is_transport_supported(ib_device, RDMA_TRANSPORT_IB))
return;
cm_dev = kzalloc(sizeof(*cm_dev) + sizeof(*port) *
@@ -3700,6 +3701,10 @@ static void cm_add_one(struct ib_device *ib_device)
set_bit(IB_MGMT_METHOD_SEND, reg_req.method_mask);
for (i = 1; i <= ib_device->phys_port_cnt; i++) {
+ tt = rdma_port_get_transport(ib_device, i);
+ if (tt != RDMA_TRANSPORT_IB)
+ continue;
+
port = kzalloc(sizeof *port, GFP_KERNEL);
if (!port)
goto error1;
@@ -3742,9 +3747,11 @@ error1:
port_modify.clr_port_cap_mask = IB_PORT_CM_SUP;
while (--i) {
port = cm_dev->port[i-1];
- ib_modify_port(ib_device, port->port_num, 0, &port_modify);
- ib_unregister_mad_agent(port->mad_agent);
- cm_remove_port_fs(port);
+ if (port) {
+ ib_modify_port(ib_device, port->port_num, 0, &port_modify);
+ ib_unregister_mad_agent(port->mad_agent);
+ cm_remove_port_fs(port);
+ }
}
device_unregister(cm_dev->device);
kfree(cm_dev);
@@ -3770,10 +3777,12 @@ static void cm_remove_one(struct ib_device *ib_device)
for (i = 1; i <= ib_device->phys_port_cnt; i++) {
port = cm_dev->port[i-1];
- ib_modify_port(ib_device, port->port_num, 0, &port_modify);
- ib_unregister_mad_agent(port->mad_agent);
- flush_workqueue(cm.wq);
- cm_remove_port_fs(port);
+ if (port) {
+ ib_modify_port(ib_device, port->port_num, 0, &port_modify);
+ ib_unregister_mad_agent(port->mad_agent);
+ flush_workqueue(cm.wq);
+ cm_remove_port_fs(port);
+ }
}
device_unregister(cm_dev->device);
kfree(cm_dev);
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 0753178..8dc3472 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -329,24 +329,26 @@ static int cma_acquire_dev(struct rdma_id_private *id_priv)
struct cma_device *cma_dev;
union ib_gid gid;
int ret = -ENODEV;
-
- switch (rdma_node_get_transport(dev_addr->dev_type)) {
- case RDMA_TRANSPORT_IB:
- ib_addr_get_sgid(dev_addr, &gid);
- break;
- case RDMA_TRANSPORT_IWARP:
- iw_addr_get_sgid(dev_addr, &gid);
- break;
- default:
- return -ENODEV;
- }
+ int port;
list_for_each_entry(cma_dev, &dev_list, list) {
- ret = ib_find_cached_gid(cma_dev->device, &gid,
- &id_priv->id.port_num, NULL);
- if (!ret) {
- cma_attach_to_dev(id_priv, cma_dev);
- break;
+ for (port = 1; port <= cma_dev->device->phys_port_cnt; ++port) {
+ switch (rdma_port_get_transport(cma_dev->device, port)) {
+ case RDMA_TRANSPORT_IB:
+ ib_addr_get_sgid(dev_addr, &gid);
+ break;
+ case RDMA_TRANSPORT_IWARP:
+ iw_addr_get_sgid(dev_addr, &gid);
+ break;
+ default:
+ return -ENODEV;
+ }
+ ret = ib_find_cached_gid(cma_dev->device, &gid,
+ &id_priv->id.port_num, NULL);
+ if (!ret) {
+ cma_attach_to_dev(id_priv, cma_dev);
+ return ret;
+ }
}
}
return ret;
@@ -597,7 +599,7 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
int ret = 0;
id_priv = container_of(id, struct rdma_id_private, id);
- switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
+ switch (rdma_port_get_transport(id_priv->id.device, id_priv->id.port_num)) {
case RDMA_TRANSPORT_IB:
if (!id_priv->cm_id.ib || cma_is_ud_ps(id_priv->id.ps))
ret = cma_ib_init_qp_attr(id_priv, qp_attr, qp_attr_mask);
@@ -747,7 +749,7 @@ static inline int cma_user_data_offset(enum rdma_port_space ps)
static void cma_cancel_route(struct rdma_id_private *id_priv)
{
- switch (rdma_node_get_transport(id_priv->id.device->node_type)) {
+ switch (rdma_port_get_transport(id_priv->id.device, id_priv->id.port_num)) {
case RDMA_TRANSPORT_IB:
if (id_priv->query)
ib_sa_cancel_query(id_priv->query_id, id_priv->query);
@@ -843,7 +845,7 @@ void rdma_destroy_id(struct rdma_cm_id *id)
mutex_lock(&lock);
if (id_priv->cma_dev) {
mutex_unlock(&lock);
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id_priv->id.device, id_priv->id.port_num)) {
case RDMA_TRANSPORT_IB:
if (id_priv->cm_id.ib && !IS_ERR(id_priv->cm_id.ib))
ib_destroy_cm_id(id_priv->cm_id.ib);
@@ -1500,7 +1502,7 @@ int rdma_listen(struct rdma_cm_id *id, int backlog)
id_priv->backlog = backlog;
if (id->device) {
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
ret = cma_ib_listen(id_priv);
if (ret)
@@ -1727,7 +1729,7 @@ int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
return -EINVAL;
atomic_inc(&id_priv->refcount);
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
ret = cma_resolve_ib_route(id_priv, timeout_ms);
break;
@@ -2407,7 +2409,7 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
id_priv->srq = conn_param->srq;
}
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
if (cma_is_ud_ps(id->ps))
ret = cma_resolve_ib_udp(id_priv, conn_param);
@@ -2520,7 +2522,7 @@ int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
id_priv->srq = conn_param->srq;
}
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
if (cma_is_ud_ps(id->ps))
ret = cma_send_sidr_rep(id_priv, IB_SIDR_SUCCESS,
@@ -2581,7 +2583,7 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
if (!cma_has_cm_dev(id_priv))
return -EINVAL;
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
if (cma_is_ud_ps(id->ps))
ret = cma_send_sidr_rep(id_priv, IB_SIDR_REJECT,
@@ -2612,7 +2614,7 @@ int rdma_disconnect(struct rdma_cm_id *id)
if (!cma_has_cm_dev(id_priv))
return -EINVAL;
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
ret = cma_modify_qp_err(id_priv);
if (ret)
@@ -2764,7 +2766,7 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
list_add(&mc->list, &id_priv->mc_list);
spin_unlock(&id_priv->lock);
- switch (rdma_node_get_transport(id->device->node_type)) {
+ switch (rdma_port_get_transport(id->device, id->port_num)) {
case RDMA_TRANSPORT_IB:
ret = cma_join_ib_multicast(id_priv, mc);
break;
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 7522008..28a47a8 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -2913,8 +2913,9 @@ static int ib_mad_port_close(struct ib_device *device, int port_num)
static void ib_mad_init_device(struct ib_device *device)
{
int start, end, i;
+ enum rdma_transport_type tt;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
+ if (!rdma_is_transport_supported(device, RDMA_TRANSPORT_IB))
return;
if (device->node_type == RDMA_NODE_IB_SWITCH) {
@@ -2926,6 +2927,10 @@ static void ib_mad_init_device(struct ib_device *device)
}
for (i = start; i <= end; i++) {
+ tt = rdma_port_get_transport(device, i);
+ if (tt != RDMA_TRANSPORT_IB)
+ continue;
+
if (ib_mad_port_open(device, i)) {
printk(KERN_ERR PFX "Couldn't open %s port %d\n",
device->name, i);
@@ -2941,21 +2946,25 @@ static void ib_mad_init_device(struct ib_device *device)
return;
error_agent:
- if (ib_mad_port_close(device, i))
- printk(KERN_ERR PFX "Couldn't close %s port %d\n",
- device->name, i);
+ tt = rdma_port_get_transport(device, i);
+ if (tt == RDMA_TRANSPORT_IB || tt == RDMA_TRANSPORT_RDMAOE)
+ if (ib_mad_port_close(device, i))
+ printk(KERN_ERR PFX "Couldn't close %s port %d\n",
+ device->name, i);
error:
i--;
while (i >= start) {
- if (ib_agent_port_close(device, i))
- printk(KERN_ERR PFX "Couldn't close %s port %d "
- "for agents\n",
- device->name, i);
- if (ib_mad_port_close(device, i))
- printk(KERN_ERR PFX "Couldn't close %s port %d\n",
- device->name, i);
+ if (rdma_port_get_transport(device, i) == RDMA_TRANSPORT_IB) {
+ if (ib_agent_port_close(device, i))
+ printk(KERN_ERR PFX "Couldn't close %s port %d "
+ "for agents\n",
+ device->name, i);
+ if (ib_mad_port_close(device, i))
+ printk(KERN_ERR PFX "Couldn't close %s port %d\n",
+ device->name, i);
+ }
i--;
}
}
@@ -2963,6 +2972,7 @@ error:
static void ib_mad_remove_device(struct ib_device *device)
{
int i, num_ports, cur_port;
+ enum rdma_transport_type tt;
if (device->node_type == RDMA_NODE_IB_SWITCH) {
num_ports = 1;
@@ -2972,13 +2982,16 @@ static void ib_mad_remove_device(struct ib_device *device)
cur_port = 1;
}
for (i = 0; i < num_ports; i++, cur_port++) {
- if (ib_agent_port_close(device, cur_port))
- printk(KERN_ERR PFX "Couldn't close %s port %d "
- "for agents\n",
- device->name, cur_port);
- if (ib_mad_port_close(device, cur_port))
- printk(KERN_ERR PFX "Couldn't close %s port %d\n",
- device->name, cur_port);
+ tt = rdma_port_get_transport(device, i);
+ if (tt == RDMA_TRANSPORT_IB) {
+ if (ib_agent_port_close(device, cur_port))
+ printk(KERN_ERR PFX "Couldn't close %s port %d "
+ "for agents\n",
+ device->name, cur_port);
+ if (ib_mad_port_close(device, cur_port))
+ printk(KERN_ERR PFX "Couldn't close %s port %d\n",
+ device->name, cur_port);
+ }
}
}
diff --git a/drivers/infiniband/core/multicast.c b/drivers/infiniband/core/multicast.c
index 8d82ba1..7d9887d 100644
--- a/drivers/infiniband/core/multicast.c
+++ b/drivers/infiniband/core/multicast.c
@@ -796,10 +796,10 @@ static void mcast_add_one(struct ib_device *device)
struct mcast_port *port;
int i;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
+ if (!rdma_is_transport_supported(device, RDMA_TRANSPORT_IB))
return;
- dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,
+ dev = kzalloc(sizeof *dev + device->phys_port_cnt * sizeof *port,
GFP_KERNEL);
if (!dev)
return;
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 7e1ffd8..18b09c4 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -416,14 +416,16 @@ static void ib_sa_event(struct ib_event_handler *handler, struct ib_event *event
struct ib_sa_port *port =
&sa_dev->port[event->element.port_num - sa_dev->start_port];
- spin_lock_irqsave(&port->ah_lock, flags);
- if (port->sm_ah)
- kref_put(&port->sm_ah->ref, free_sm_ah);
- port->sm_ah = NULL;
- spin_unlock_irqrestore(&port->ah_lock, flags);
-
- schedule_work(&sa_dev->port[event->element.port_num -
- sa_dev->start_port].update_task);
+ if (rdma_port_get_transport(handler->device, port->port_num) == RDMA_TRANSPORT_IB) {
+ spin_lock_irqsave(&port->ah_lock, flags);
+ if (port->sm_ah)
+ kref_put(&port->sm_ah->ref, free_sm_ah);
+ port->sm_ah = NULL;
+ spin_unlock_irqrestore(&port->ah_lock, flags);
+
+ schedule_work(&sa_dev->port[event->element.port_num -
+ sa_dev->start_port].update_task);
+ }
}
}
@@ -997,7 +999,7 @@ static void ib_sa_add_one(struct ib_device *device)
struct ib_sa_device *sa_dev;
int s, e, i;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
+ if (!rdma_is_transport_supported(device, RDMA_TRANSPORT_IB))
return;
if (device->node_type == RDMA_NODE_IB_SWITCH)
@@ -1007,7 +1009,7 @@ static void ib_sa_add_one(struct ib_device *device)
e = device->phys_port_cnt;
}
- sa_dev = kmalloc(sizeof *sa_dev +
+ sa_dev = kzalloc(sizeof *sa_dev +
(e - s + 1) * sizeof (struct ib_sa_port),
GFP_KERNEL);
if (!sa_dev)
@@ -1017,6 +1019,9 @@ static void ib_sa_add_one(struct ib_device *device)
sa_dev->end_port = e;
for (i = 0; i <= e - s; ++i) {
+ if (rdma_port_get_transport(device, i + 1) != RDMA_TRANSPORT_IB)
+ continue;
+
sa_dev->port[i].sm_ah = NULL;
sa_dev->port[i].port_num = i + s;
spin_lock_init(&sa_dev->port[i].ah_lock);
@@ -1045,13 +1050,15 @@ static void ib_sa_add_one(struct ib_device *device)
goto err;
for (i = 0; i <= e - s; ++i)
- update_sm_ah(&sa_dev->port[i].update_task);
+ if (rdma_port_get_transport(device, i + 1) == RDMA_TRANSPORT_IB)
+ update_sm_ah(&sa_dev->port[i].update_task);
return;
err:
while (--i >= 0)
- ib_unregister_mad_agent(sa_dev->port[i].agent);
+ if (rdma_port_get_transport(device, i + 1) == RDMA_TRANSPORT_IB)
+ ib_unregister_mad_agent(sa_dev->port[i].agent);
kfree(sa_dev);
@@ -1071,9 +1078,11 @@ static void ib_sa_remove_one(struct ib_device *device)
flush_scheduled_work();
for (i = 0; i <= sa_dev->end_port - sa_dev->start_port; ++i) {
- ib_unregister_mad_agent(sa_dev->port[i].agent);
- if (sa_dev->port[i].sm_ah)
- kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah);
+ if (rdma_port_get_transport(device, i + 1) == RDMA_TRANSPORT_IB) {
+ ib_unregister_mad_agent(sa_dev->port[i].agent);
+ if (sa_dev->port[i].sm_ah)
+ kref_put(&sa_dev->port[i].sm_ah->ref, free_sm_ah);
+ }
}
kfree(sa_dev);
diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
index f504c9b..43700b4 100644
--- a/drivers/infiniband/core/ucm.c
+++ b/drivers/infiniband/core/ucm.c
@@ -1240,11 +1240,15 @@ static DEVICE_ATTR(ibdev, S_IRUGO, show_ibdev, NULL);
static void ib_ucm_add_one(struct ib_device *device)
{
struct ib_ucm_device *ucm_dev;
+ int i;
- if (!device->alloc_ucontext ||
- rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
+ if (!device->alloc_ucontext)
return;
+ for (i = 1; i <= device->phys_port_cnt; ++i)
+ if (rdma_port_get_transport(device, i) != RDMA_TRANSPORT_IB)
+ return;
+
ucm_dev = kzalloc(sizeof *ucm_dev, GFP_KERNEL);
if (!ucm_dev)
return;
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index f1cbd26..fd831fd 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -616,7 +616,7 @@ static ssize_t ucma_query_route(struct ucma_file *file,
resp.node_guid = (__force __u64) ctx->cm_id->device->node_guid;
resp.port_num = ctx->cm_id->port_num;
- switch (rdma_node_get_transport(ctx->cm_id->device->node_type)) {
+ switch (rdma_port_get_transport(ctx->cm_id->device, ctx->cm_id->port_num)) {
case RDMA_TRANSPORT_IB:
ucma_copy_ib_route(&resp, &ctx->cm_id->route);
break;
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index 7de0296..6d6795d 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -1114,7 +1114,7 @@ static void ib_umad_add_one(struct ib_device *device)
struct ib_umad_device *umad_dev;
int s, e, i;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
+ if (!rdma_is_transport_supported(device, RDMA_TRANSPORT_IB))
return;
if (device->node_type == RDMA_NODE_IB_SWITCH)
@@ -1124,6 +1124,10 @@ static void ib_umad_add_one(struct ib_device *device)
e = device->phys_port_cnt;
}
+ for (i = s; i <= e; ++i)
+ if (rdma_port_get_transport(device, i) != RDMA_TRANSPORT_IB)
+ return;
+
umad_dev = kzalloc(sizeof *umad_dev +
(e - s + 1) * sizeof (struct ib_umad_port),
GFP_KERNEL);
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index a7da9be..d81e217 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -77,7 +77,7 @@ enum ib_rate mult_to_ib_rate(int mult)
}
EXPORT_SYMBOL(mult_to_ib_rate);
-enum rdma_transport_type
+static enum rdma_transport_type
rdma_node_get_transport(enum rdma_node_type node_type)
{
switch (node_type) {
@@ -92,7 +92,28 @@ rdma_node_get_transport(enum rdma_node_type node_type)
return 0;
}
}
-EXPORT_SYMBOL(rdma_node_get_transport);
+
+enum rdma_transport_type rdma_port_get_transport(struct ib_device *device,
+ u8 port_num)
+{
+ return device->get_port_transport ?
+ device->get_port_transport(device, port_num) :
+ rdma_node_get_transport(device->node_type);
+}
+EXPORT_SYMBOL(rdma_port_get_transport);
+
+int rdma_is_transport_supported(struct ib_device *device,
+ enum rdma_transport_type transport)
+{
+ int i;
+
+ for (i = 1; i <= device->phys_port_cnt; ++i)
+ if (rdma_port_get_transport(device, i) == transport)
+ return 1;
+
+ return 0;
+}
+EXPORT_SYMBOL(rdma_is_transport_supported);
/* Protection domains */
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 2bf5116..de89e80 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -1343,9 +1343,6 @@ static void ipoib_add_one(struct ib_device *device)
struct ipoib_dev_priv *priv;
int s, e, p;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
-
dev_list = kmalloc(sizeof *dev_list, GFP_KERNEL);
if (!dev_list)
return;
@@ -1361,6 +1358,9 @@ static void ipoib_add_one(struct ib_device *device)
}
for (p = s; p <= e; ++p) {
+ if (rdma_port_get_transport(device, p) != RDMA_TRANSPORT_IB)
+ continue;
+
dev = ipoib_add_port("ib%d", device, p);
if (!IS_ERR(dev)) {
priv = netdev_priv(dev);
@@ -1376,12 +1376,12 @@ static void ipoib_remove_one(struct ib_device *device)
struct ipoib_dev_priv *priv, *tmp;
struct list_head *dev_list;
- if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB)
- return;
-
dev_list = ib_get_client_data(device, &ipoib_client);
list_for_each_entry_safe(priv, tmp, dev_list, list) {
+ if (rdma_port_get_transport(device, priv->port) != RDMA_TRANSPORT_IB)
+ continue;
+
ib_unregister_event_handler(&priv->event_handler);
rtnl_lock();
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index c179318..4cf42f3 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -72,9 +72,6 @@ enum rdma_transport_type {
RDMA_TRANSPORT_IWARP
};
-enum rdma_transport_type
-rdma_node_get_transport(enum rdma_node_type node_type) __attribute_const__;
-
enum ib_device_cap_flags {
IB_DEVICE_RESIZE_MAX_WR = 1,
IB_DEVICE_BAD_PKEY_CNTR = (1<<1),
@@ -298,6 +295,7 @@ struct ib_port_attr {
u8 active_width;
u8 active_speed;
u8 phys_state;
+ enum rdma_transport_type transport;
};
enum ib_device_modify_flags {
@@ -1003,6 +1001,8 @@ struct ib_device {
int (*query_port)(struct ib_device *device,
u8 port_num,
struct ib_port_attr *port_attr);
+ enum rdma_transport_type (*get_port_transport)(struct ib_device *device,
+ u8 port_num);
int (*query_gid)(struct ib_device *device,
u8 port_num, int index,
union ib_gid *gid);
@@ -1213,6 +1213,11 @@ int ib_query_device(struct ib_device *device,
int ib_query_port(struct ib_device *device,
u8 port_num, struct ib_port_attr *port_attr);
+enum rdma_transport_type rdma_port_get_transport(struct ib_device *device,
+ u8 port_num);
+int rdma_is_transport_supported(struct ib_device *device,
+ enum rdma_transport_type transport);
+
int ib_query_gid(struct ib_device *device,
u8 port_num, int index, union ib_gid *gid);
diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
index 9e88438..1131c74 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
@@ -338,8 +338,7 @@ static int rdma_set_ctxt_sge(struct svcxprt_rdma *xprt,
static int rdma_read_max_sge(struct svcxprt_rdma *xprt, int sge_count)
{
if ((RDMA_TRANSPORT_IWARP ==
- rdma_node_get_transport(xprt->sc_cm_id->
- device->node_type))
+ rdma_port_get_transport(xprt->sc_cm_id->device, xprt->sc_cm_id->port_num))
&& sge_count > 1)
return 1;
else
diff --git a/net/sunrpc/xprtrdma/svc_rdma_transport.c b/net/sunrpc/xprtrdma/svc_rdma_transport.c
index 3fa5751..4e286f7 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_transport.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_transport.c
@@ -977,7 +977,7 @@ static struct svc_xprt *svc_rdma_accept(struct svc_xprt *xprt)
/*
* Determine if a DMA MR is required and if so, what privs are required
*/
- switch (rdma_node_get_transport(newxprt->sc_cm_id->device->node_type)) {
+ switch (rdma_port_get_transport(newxprt->sc_cm_id->device, newxprt->sc_cm_id->port_num)) {
case RDMA_TRANSPORT_IWARP:
newxprt->sc_dev_caps |= SVCRDMA_DEVCAP_READ_W_INV;
if (!(newxprt->sc_dev_caps & SVCRDMA_DEVCAP_FAST_REG)) {
--
1.6.5.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox