* Re: [PATCH 08/25] mlx4_core: fix buggy parsing of reserved eq cap
From: Roland Dreier @ 2009-11-05 20:10 UTC (permalink / raw)
To: Yevgeny Petrilin
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
Liran Liss, Tziporet Koren
In-Reply-To: <2ED289D4E09FBD4D92D911E869B97FDD0166CEF4-ia22CT07NJfiMCgWhms8HQC/G2K4zDHf@public.gmane.org>
> I am currently working on fixing the patches according to comments
> received so far,
> And thought to resubmit the entire patch set.
That's a good idea, but you can just drop this one and I'll take it
independent of SRIOV support (since it really is independent).
- R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: [PATCH 08/25] mlx4_core: fix buggy parsing of reserved eq cap
From: Yevgeny Petrilin @ 2009-11-05 20:14 UTC (permalink / raw)
To: Roland Dreier
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
Liran Liss, Tziporet Koren
In-Reply-To: <adaskcs68bx.fsf-BjVyx320WGW9gfZ95n9DRSW4+XlvGpQz@public.gmane.org>
>That's a good idea, but you can just drop this one and I'll take it
independent of SRIOV support (since it really is independent).
OK
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] netdev: Fix compile error in Octeon MGMT driver.
From: Ralf Baechle @ 2009-11-05 20:45 UTC (permalink / raw)
To: David Daney; +Cc: linux-mips, netdev
In-Reply-To: <1257449912-27978-1-git-send-email-ddaney@caviumnetworks.com>
On Thu, Nov 05, 2009 at 11:38:32AM -0800, David Daney wrote:
> Explicitly include linux/capability.h. Under some configurations it
> wasn't being indirectly included.
>
> Signed-off-by: David Daney <ddaney@caviumnetworks.com>
> ---
> This fixes a minor problem for the (already approved) patches that
> Ralf has queued for 2.6.33. It should probably be added to Ralf's
> queue.
>
> drivers/net/octeon/octeon_mgmt.c | 1 +
> 1 files changed, 1 insertions(+), 0 deletions(-)
Folded into the existing patch. Thanks!
Ralf
^ permalink raw reply
* Re: [rfc 4/4] igb: expose 82576 bandiwidth allocation
From: Alexander Duyck @ 2009-11-05 23:00 UTC (permalink / raw)
To: Simon Horman
Cc: e1000-devel@lists.sourceforge.net, netdev@vger.kernel.org,
Kirsher, Jeffrey T, Arnd Bergmann
In-Reply-To: <20091105010628.148945886@vergenet.net>
Simon Horman wrote:
> The 82576 has support for bandwidth allocation to VFs.
>
> Contrary to the documentation in the 82576 datasheet v2.41 this
> appears to work as follows:
>
> * The ratio supplied is always proportional to 1Gbit/s,
> regardless of if the link speed.
> * The ratio supplied is an upper-bound on bandwidth available
> to the VF, not a minimun guarantee
>
> This patch exposes bandwidth control to userspace through a simple
> per-device (PF) sysfs file, bandwidth_allocation.
>
> * The file contains a whitespace delimited list of values, one per VF.
> * The first value corresponds to the first VF and so on.
> * Valid values are integers from 0 to 1000
> * A value of 0 indicates that bandwidth_allocation is disabled.
> * Other values indicate the allocated bandwidth, in 1/1000ths of a gigabit/s
>
> e.g. The following for a PF with 4 VFs allocates ~20Mbits/ to VF 1,
> ~100Mbit/s to VF 2, and leave the other 2 VFs with no allocation.
>
> echo "20 100 0 0" > /sys/class/net/eth3/device/bandwidth_allocation
>
> This interface is intended to allow testing of the hardware feature.
> There are ongoing discussions about how to expose this feature
> to user-space in a more generic way.
>
> Signed-off-by: Simon Horman <horms@verge.net.au>
>
Of the patches it looks like the only one that really has any issues is
this one and it is mostly due to the sysfs implementation. The others I
would say can be applied and pushed up into the net-next-2.6 tree.
We're currently working on an iproute2 based solution for configuring
VFs and can incorporate this functionality into it at some point in the
future.
Thanks,
Alex
^ permalink raw reply
* Re: [rfc 4/4] igb: expose 82576 bandiwidth allocation
From: Simon Horman @ 2009-11-05 23:30 UTC (permalink / raw)
To: Alexander Duyck
Cc: e1000-devel@lists.sourceforge.net, netdev@vger.kernel.org,
Arnd Bergmann, Kirsher, Jeffrey T
In-Reply-To: <4AF358F2.9000109@intel.com>
On Thu, Nov 05, 2009 at 03:00:02PM -0800, Alexander Duyck wrote:
> Simon Horman wrote:
> >The 82576 has support for bandwidth allocation to VFs.
> >
> >Contrary to the documentation in the 82576 datasheet v2.41 this
> >appears to work as follows:
> >
> >* The ratio supplied is always proportional to 1Gbit/s,
> > regardless of if the link speed.
> >* The ratio supplied is an upper-bound on bandwidth available
> > to the VF, not a minimun guarantee
> >
> >This patch exposes bandwidth control to userspace through a simple
> >per-device (PF) sysfs file, bandwidth_allocation.
> >
> >* The file contains a whitespace delimited list of values, one per VF.
> >* The first value corresponds to the first VF and so on.
> >* Valid values are integers from 0 to 1000
> >* A value of 0 indicates that bandwidth_allocation is disabled.
> >* Other values indicate the allocated bandwidth, in 1/1000ths of a gigabit/s
> >
> >e.g. The following for a PF with 4 VFs allocates ~20Mbits/ to VF 1,
> > ~100Mbit/s to VF 2, and leave the other 2 VFs with no allocation.
> >
> > echo "20 100 0 0" > /sys/class/net/eth3/device/bandwidth_allocation
> >
> >This interface is intended to allow testing of the hardware feature.
> >There are ongoing discussions about how to expose this feature
> >to user-space in a more generic way.
> >
> >Signed-off-by: Simon Horman <horms@verge.net.au>
> >
>
> Of the patches it looks like the only one that really has any issues
> is this one and it is mostly due to the sysfs implementation. The
> others I would say can be applied and pushed up into the
> net-next-2.6 tree.
Thanks, I suspected as much.
> We're currently working on an iproute2 based solution for
> configuring VFs and can incorporate this functionality into it at
> some point in the future.
Do you have any pointers to discussions relating to that interface.
Do you think it would be worth putting in the sysfs interface in the
mean-time, or would you rather wait?
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
^ permalink raw reply
* Re: [rfc 4/4] igb: expose 82576 bandiwidth allocation
From: Alexander Duyck @ 2009-11-05 23:42 UTC (permalink / raw)
To: Simon Horman
Cc: e1000-devel@lists.sourceforge.net, netdev@vger.kernel.org,
Kirsher, Jeffrey T, Arnd Bergmann
In-Reply-To: <20091105233034.GA9588@verge.net.au>
Simon Horman wrote:
> On Thu, Nov 05, 2009 at 03:00:02PM -0800, Alexander Duyck wrote:
>> Simon Horman wrote:
>>> The 82576 has support for bandwidth allocation to VFs.
>>>
>>> Contrary to the documentation in the 82576 datasheet v2.41 this
>>> appears to work as follows:
>>>
>>> * The ratio supplied is always proportional to 1Gbit/s,
>>> regardless of if the link speed.
>>> * The ratio supplied is an upper-bound on bandwidth available
>>> to the VF, not a minimun guarantee
>>>
>>> This patch exposes bandwidth control to userspace through a simple
>>> per-device (PF) sysfs file, bandwidth_allocation.
>>>
>>> * The file contains a whitespace delimited list of values, one per VF.
>>> * The first value corresponds to the first VF and so on.
>>> * Valid values are integers from 0 to 1000
>>> * A value of 0 indicates that bandwidth_allocation is disabled.
>>> * Other values indicate the allocated bandwidth, in 1/1000ths of a gigabit/s
>>>
>>> e.g. The following for a PF with 4 VFs allocates ~20Mbits/ to VF 1,
>>> ~100Mbit/s to VF 2, and leave the other 2 VFs with no allocation.
>>>
>>> echo "20 100 0 0" > /sys/class/net/eth3/device/bandwidth_allocation
>>>
>>> This interface is intended to allow testing of the hardware feature.
>>> There are ongoing discussions about how to expose this feature
>>> to user-space in a more generic way.
>>>
>>> Signed-off-by: Simon Horman <horms@verge.net.au>
>>>
>> Of the patches it looks like the only one that really has any issues
>> is this one and it is mostly due to the sysfs implementation. The
>> others I would say can be applied and pushed up into the
>> net-next-2.6 tree.
>
> Thanks, I suspected as much.
>
>> We're currently working on an iproute2 based solution for
>> configuring VFs and can incorporate this functionality into it at
>> some point in the future.
>
> Do you have any pointers to discussions relating to that interface.
> Do you think it would be worth putting in the sysfs interface in the
> mean-time, or would you rather wait?
I'm not the one working on the interface so I don't know much about it
other than the fact it is being worked on. Hopefully we should see
something in regards to that soon though.
If anything it might be of some use to split this up into 2 patches.
One that contains the sysfs bits, and another for enabling the bandwidth
control registers. We won't be able to get the sysfs interface accepted
upstream so there isn't much point in us keeping it around for any other
purpose than testing to verify the registers work as you have described.
Thanks,
Alex
^ permalink raw reply
* Re: Doubt in implementations of mean loss interval at sender side
From: Ivo Calado @ 2009-11-06 0:05 UTC (permalink / raw)
To: Gerrit Renker, dccp, netdev, Ivo Calado
In-Reply-To: <cb00fa210911051603w6fb8de32qd7ebf37ce78408f7@mail.gmail.com>
On Wed, Oct 28, 2009 at 12:33 PM, Gerrit Renker <gerrit@erg.abdn.ac.uk> wrote:
> | > This is a good point. Personally, I can not really see an advantage in
> | > storing old data at the sender, as it seems to increase the complexity,
> | > without at the same time introducing a benefit.
> | >
> | > Adding the 'two RTTs old' worth of information at the sender re-introduces
> | > things that were removed already. The old CCID-3 sender used to store
> | > a lot of information about old packets, now it is much leaner and keeps
> | > only the minimum required information.
> |
> | So, how we can solve this? How can we determine if a loss interval is
> | (or not) 2 RTT long in the sender?
> |
> Yes, I also think that this is the core problem.
>
> To be honest, the reply had been receiver-based TFRC in mind but did not
> state the reasons. These are below, which contains also a sketch.
>
> In particular, the 'a lot of information about old packets' mentioned above
> could only be taken out (and with improved performance) since it relied on
> using a receiver-based implementation (in fact the code has always been,
> since the original Lulea code).
>
>
> I) (Minimum) set of data required to be stored at the sender
> ------------------------------------------------------------
> RFC 4342, 6 requires a feedback packet to contain
> (a) Elapsed Time or Timestamp Echo;
> (b) Receive Rate option;
> (c) Loss Intervals Option.
>
> Out of these only (b) is currently supported. (a) used to be supported,
> but it turned out that the elapsed time was in the order of circa 50
> microseconds. Timestamp Echo can only be sent if the sender has sent
> a DCCP timestamp option (RFC 4340, 13.3), so it can not be used for the
> general case.
>
> The sender must be able to handle three scenarios:
>
> (a) receiver sends Loss Event Rate option only
> (b) receiver sends Loss Intervals option only
> (c) receiver sends both Loss Event Rate and Loss Intervals option
>
> The implementation currently does (a) and enforces this by using a
> Mandatory Loss Event Rate option (ccid3_dependencies in net/dccp/feat.c),
> resetting the connection if the peer sender only implements (b).
>
> Case (b) is a pre-stage to case (c), otherwise it can only talk to
> DCCP receivers that implement the Loss Intervals option.
>
> In case (c) (and I think this is in part in your implementation), the
> question is what to trust if the options are mutually inconsistent.
> This is the subject of RFC 4342, 9.2, which suggests to store the sending
> times of (dropped) packets.
>
> Window counter timestamps are problematic here, due to the 'increment by 5'
> rule from RFC 4342, 8.1. Using timestamps raises again the timer-resolution
> question. If using 10usec from RFC 4342, 13.2 as baseline, the sequence
> number will probably also need to be stored since in 10usec multiple
> packets can be transmitted (also when using a lower resolution).
>
> Until here we have got the requirement to store, for each sent packet,
> * its sending time (min. 4 bytes to match RFC 4342, 13.2)
> * its sequence number (u48 or u64)
> Relating to your question at the top of the email, the next item is
> * the RTT estimate at the time the packet was sent, used for
> - verifying the length of the Lossy Part (RFC 4342, 6.1);
> - reducing the sending rate when a Data Dropped option is received, 5.2;
> - determining whether the loss interval was less than or more than 2 RTTs
> (your question, RFC 4828, 4.4).
>
> To sum up, here is whay I think is minimally required to satisfy the union
> of RFC 4340, 4342, 4828, 5348, and 5622:
>
> struct tfrc_tx_packet_info {
> u64 seqno:48,
> is_ect0:1,
> is_data_packet:1,
> is_in_loss_interval:1;
> u32 send_time;
> u32 rtt_estimate;
> struct tfrc_tx_packet_info *next; /* FIFO */
> };
>
> That would be a per-packet storage cost of about 16 bytes, plus the pointer
> (8 bytes on 64-bit architectures). One could avoid the pointer by defining a
> u64 base_seqno;
> and then
> struct tfrc_tx_packet_info[some constant here];
> and then index the array relative to the base_seqno.
>
Yes, I believe that struct is enough too. But how long would be necessary
the struct array to be?
>
> IIb) Further remarks
> --------------------
> At first sight it would seem that storing the RTT also solves the problem
> of inaccurate RTTs used at the receiver. Unfortunately, this is not the
> case. X_recv is sampled over intervals of varying length which may or may
> not equal the RTT. To factor out the effect of window counters, the sender
> would need to store the packet size as well and would need to use rather
> complicated computations - an ugly workaround.
I didn't understand how the packet size would help and what
computations are needed.
>
> One thing I stumbled across while reading your code was the fact that RFC 4342
> leaves it open as to how many Loss Intervals to send: on the one hand it follows
> the suggestion of RFC 5348 to use 1+NINTERVAL=9, but on the other hand it does
> not restrict the number of loss intervals. Also RFC 5622 does not limit the
> number of Loss Intervals / Data Dropped options.
>
> If receiving n > 9 Loss Intervals, what does the sender do with the n-9 older
> intervals? There must be some mechanism to stop these options from growing
> beyond bounds, so it needs to store also which loss intervals have been
> acknowledged, introducing the "Acknowledgment of Acknowledgments"
> problem.
>
In RFC 4342 section 8.6 it says that the limit of loss interval data
to send is 28, and RFC 5622 8.7 says 84 for dropped packets option.
But I don't see why to send so many data in these options.
Yes, the most recent 9 loss intervals are required to be reported,
except if the sender acknowledged previous sent loss intervals, so in
that case only one is required, the open interval.
And we can avoid the "Acknowledgment of Acknowledgments" if we always send
the required 9 loss intervals, I think.
> A second point is how to compute the loss event rate when n > 9. It seems
> that this would mean grinding through all loss intervals using a window
> of 9. If that is the case, the per-packet-computation costs become very
> expensive.
>
RFC 4342 section 8.6 suggests that only 9 loss intervals are required
anyway. And I believe that's enough for the computation of current
mean loss interval. What do you think?
>
> II) Computational part of the implementation
> --------------------------------------------
> If only Loss Intervals alone are used, only these need to be verified
> before being used to alter the sender behaviour.
>
> But when one or more other DCCP options also appear, the verification is
> * intra: make sure each received option is in itself consistent,
> * inter: make sure options are mutually consistent.
>
> The second has a combinatorial effect, i.e. n! verifications for n options.
>
> For n=2 we have Loss Intervals and Dropped Packets: the consistency must
> be in both directions, so we need two stages of verifications.
>
> If Ack Vectors are used in addition to Loss Intervals, then their data
> must also be verified. Here we have up to 6 = 3! testing stages.
>
> It gets more complicated (4! = 24 checks) by also adding Data Dropped
> options, where RFC 4340, 11.7 requires to check them against the Ack
> Vector, and thus ultimately also against the Loss Intervals option.
>
Yes, there's a combinatorial problem in checking the options for
consistence. But, what if we find out that some option doesn't match
against others? What action would be taken?
First, what can cause the receiver to send inconsistent options? A bad
implementation only?
Accordingly to ecn nonce echo sum algorithm, if a receiver is found to
be lying about loss or to be bad implemented, the sender adjusts the
send rate as if loss were perceived. Can we do the same in this
situation? If so, can we skip
checking options between them and only check ecn nonce sum?
If some option is wrong it show more loss (or any worse situation for
the receiver)
or conceals loss. In the first case, I don't believe we need to care,
and in the second, the ecn nonce sum can reveal the bad acting of the
receiver.
>
> III) Closing remarks in favour of receiver-based implementation
> ---------------------------------------------------------------
> Finally, both RFC 4342 and RFC 5622 do not explicitly discard the
> possibility of using a receiver-based implementation. Quoting
> RFC 4342, 3.2: "If it prefers, the sender can also use a loss event
> rate calculated and reported by the receiver."
> Furthermore, the revised TFRC specification points out in section 7
> the advantages that a receiver-based implementation has:
> * it does not mandate reliable delivery of packet loss data;
> * it is robust against the loss of feedback packets;
> * better suited for scalable server design.
>
> Quite likely, if the server does not have to store and validate a mass
> of data, it is also less prone to be toppled by DoS attacks.
>
You're right. But what the RFC's says about it is almost exactly the
opposite, isn't? What can we do about it? I like the receiver-based design,
but I believe that loss intervals are interesting, mostly because of
receiver behavior verification.
> | > As a second point, I still think that a receiver-based CCID-4 implementation
> | > would be the simplest possible starting point. In this light, do you see an
> | > advantage in supplying an RTT estimate from sender to receiver?
> |
> | Yes, better precision. But, at the cost of adding an option
> | undocumented by any RFC's?
> |
> No I wasn't suggesting that. As you rightly point out, the draft has
> expired. It would need to be overhauled (all the references have
> changed, but the problem has not), and I was asking whether returning
> to this has any benefit.
>
> The text is the equivalent of a bug report. RFCs are like software - if no
> one submits bug reports, they become features, until someone has enough of
> such 'features' and writes a new specification.
--
Ivo Augusto Andrade Rocha Calado
MSc. Candidate
Embedded Systems and Pervasive Computing Lab - http://embedded.ufcg.edu.br
Systems and Computing Department - http://www.dsc.ufcg.edu.br
Electrical Engineering and Informatics Center - http://www.ceei.ufcg.edu.br
Federal University of Campina Grande - http://www.ufcg.edu.br
PGP: 0x03422935
Putt's Law:
Technology is dominated by two types of people:
Those who understand what they do not manage.
Those who manage what they do not understand.
^ permalink raw reply
* [PATCH 02/25 v2] mlx4_core: add support for arbitrary bitmap sizes
From: Yevgeny Petrilin @ 2009-11-06 3:08 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
From: Liran Liss <liranl@mellanox.co.il>
Signed-off-by: Liran Liss <liranl@mellanox.co.il>
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/alloc.c | 10 ++++++++++
drivers/net/mlx4/mlx4.h | 2 ++
2 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/drivers/net/mlx4/alloc.c b/drivers/net/mlx4/alloc.c
index ad95d5f..bc68cc3 100644
--- a/drivers/net/mlx4/alloc.c
+++ b/drivers/net/mlx4/alloc.c
@@ -177,6 +177,16 @@ int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask,
return 0;
}
+/* Like bitmap_init, but doesn't require 'num' to be a power of 2 or
+ * a non-trivial mask */
+int mlx4_bitmap_init_no_mask(struct mlx4_bitmap *bitmap, u32 num,
+ u32 reserved_bot, u32 reserved_top)
+{
+ u32 num_rounded = roundup_pow_of_two(num);
+ return mlx4_bitmap_init(bitmap, num_rounded, num_rounded - 1,
+ reserved_bot, num_rounded - num + reserved_top);
+}
+
void mlx4_bitmap_cleanup(struct mlx4_bitmap *bitmap)
{
kfree(bitmap->table);
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index bc72d6e..5836c94 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -330,6 +330,8 @@ u32 mlx4_bitmap_alloc_range(struct mlx4_bitmap *bitmap, int cnt, int align);
void mlx4_bitmap_free_range(struct mlx4_bitmap *bitmap, u32 obj, int cnt);
int mlx4_bitmap_init(struct mlx4_bitmap *bitmap, u32 num, u32 mask,
u32 reserved_bot, u32 resetrved_top);
+int mlx4_bitmap_init_no_mask(struct mlx4_bitmap *bitmap, u32 num,
+ u32 reserved_bot, u32 reserved_top);
void mlx4_bitmap_cleanup(struct mlx4_bitmap *bitmap);
int mlx4_reset(struct mlx4_dev *dev);
--
1.5.3.7
^ permalink raw reply related
* [PATCH 03/25 v2] mlx4_core: add multi-function communication channel
From: Yevgeny Petrilin @ 2009-11-06 3:08 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
From: Liran Liss <liranl@mellanox.co.il>
The communication channel consists of 2 registers per vf (a slave function)
that are shared with the pf (the master function), as well as a new command for
inter-function memory copying (only exposed to the master).
The communication channel is used to establish a Virtual HCA Command Register
(vHCR) in each slave function, which allows it to pass FW commands to the master
function for execution.
The slave also uses the vHCR to pull slave-specific events from the master.
Signed-off-by: Liran Liss <liranl@mellanox.co.il>
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/cmd.c | 750 ++++++++++++++++++++++++++++++++++++++++++-
drivers/net/mlx4/en_port.h | 5 -
drivers/net/mlx4/eq.c | 67 ++++
drivers/net/mlx4/fw.c | 8 +
drivers/net/mlx4/mlx4.h | 72 ++++-
include/linux/mlx4/cmd.h | 12 +-
include/linux/mlx4/device.h | 3 +-
7 files changed, 899 insertions(+), 18 deletions(-)
diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c
index 65ec77d..19653e7 100644
--- a/drivers/net/mlx4/cmd.c
+++ b/drivers/net/mlx4/cmd.c
@@ -140,6 +140,46 @@ static int mlx4_status_to_errno(u8 status)
return trans_table[status];
}
+static int comm_pending(struct mlx4_dev *dev)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ u32 status = readl(&priv->mfunc.comm->slave_read);
+
+ return (swab32(status) >> 30) != priv->cmd.comm_toggle;
+}
+
+int mlx4_comm_cmd(struct mlx4_dev *dev, u8 cmd, u16 param, unsigned long timeout)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ unsigned long end;
+ u32 val;
+
+ /* First, verify that the master reports correct status */
+ if (comm_pending(dev)) {
+ mlx4_warn(dev, "Communication channel is not idle\n");
+ return -EAGAIN;
+ }
+
+ /* Write command */
+ if (cmd == MLX4_COMM_CMD_RESET)
+ priv->cmd.comm_toggle = 0;
+ else if (++priv->cmd.comm_toggle > 2)
+ priv->cmd.comm_toggle = 1;
+ val = param | (cmd << 16) | (priv->cmd.comm_toggle << 30);
+ __raw_writel((__force u32) cpu_to_be32(val), &priv->mfunc.comm->slave_write);
+ wmb();
+
+ end = msecs_to_jiffies(timeout) + jiffies;
+ while (comm_pending(dev) && time_before(jiffies, end))
+ cond_resched();
+
+ if (comm_pending(dev)) {
+ mlx4_warn(dev, "Communication channel timed out\n");
+ return -ETIMEDOUT;
+ }
+ return 0;
+}
+
static int cmd_pending(struct mlx4_dev *dev)
{
u32 status = readl(mlx4_priv(dev)->cmd.hcr + HCR_STATUS_OFFSET);
@@ -207,6 +247,33 @@ out:
return ret;
}
+static int mlx4_slave_cmd_poll(struct mlx4_dev *dev, u64 in_param, u64 *out_param,
+ int out_is_imm, u32 in_modifier, u8 op_modifier,
+ u16 op, unsigned long timeout)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_vhcr *vhcr = priv->mfunc.vhcr;
+ int ret;
+
+ down(&priv->cmd.poll_sem);
+ vhcr->in_param = in_param;
+ vhcr->out_param = out_param ? *out_param : 0;
+ vhcr->in_modifier = in_modifier;
+ vhcr->timeout = timeout;
+ vhcr->op = op;
+ vhcr->token = CMD_POLL_TOKEN;
+ vhcr->op_modifier = op_modifier;
+ vhcr->errno = 0;
+ ret = mlx4_comm_cmd(dev, MLX4_COMM_CMD_VHCR_POST, 0, MLX4_COMM_TIME + timeout);
+ if (!ret) {
+ if (out_is_imm)
+ *out_param = vhcr->out_param;
+ ret = vhcr->errno;
+ }
+ up(&priv->cmd.poll_sem);
+ return ret;
+}
+
static int mlx4_cmd_poll(struct mlx4_dev *dev, u64 in_param, u64 *out_param,
int out_is_imm, u32 in_modifier, u8 op_modifier,
u16 op, unsigned long timeout)
@@ -314,12 +381,647 @@ int __mlx4_cmd(struct mlx4_dev *dev, u64 in_param, u64 *out_param,
if (mlx4_priv(dev)->cmd.use_events)
return mlx4_cmd_wait(dev, in_param, out_param, out_is_imm,
in_modifier, op_modifier, op, timeout);
+
+ if (mlx4_is_slave(dev))
+ return mlx4_slave_cmd_poll(dev, in_param, out_param, out_is_imm,
+ in_modifier, op_modifier, op, timeout);
else
return mlx4_cmd_poll(dev, in_param, out_param, out_is_imm,
in_modifier, op_modifier, op, timeout);
}
EXPORT_SYMBOL_GPL(__mlx4_cmd);
+static int mlx4_ACCESS_MEM(struct mlx4_dev *dev, u64 master_addr,
+ int slave, u64 slave_addr,
+ int size, int is_read)
+{
+ u64 in_param;
+ u64 out_param;
+ int slave_id = slave + 1; /* index 0 is reserved for the master */
+
+ if ((slave_addr & 0xfff) | (master_addr & 0xfff) |
+ (slave_id & ~0x7f) | (size & 0xff)) {
+ mlx4_err(dev, "Bad access mem params - slave_addr:0x%llx "
+ "master_addr:0x%llx slave_id:%d size:%d\n",
+ slave_addr, master_addr, slave_id, size);
+ return -EINVAL;
+ }
+
+ if (is_read) {
+ in_param = (u64) slave_id | slave_addr;
+ out_param = master_addr;
+ } else {
+ in_param = master_addr;
+ out_param = (u64) slave_id | slave_addr;
+ }
+
+ return mlx4_cmd_imm(dev, in_param, &out_param, size, 0,
+ MLX4_CMD_ACCESS_MEM,
+ MLX4_CMD_TIME_CLASS_A);
+}
+
+static struct mlx4_cmd_info {
+ u8 opcode;
+ bool has_inbox;
+ bool has_outbox;
+ bool out_is_imm;
+ int (*verify)(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
+ struct mlx4_cmd_mailbox *inbox);
+ int (*wrapper)(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
+ struct mlx4_cmd_mailbox *inbox,
+ struct mlx4_cmd_mailbox *outbox);
+} cmd_info[] = {
+ {
+ .opcode = MLX4_CMD_QUERY_FW,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_QUERY_ADAPTER,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL
+ },
+
+ {
+ .opcode = MLX4_CMD_SW2HW_EQ,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /*need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_NOP,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL
+ },
+
+ {
+ .opcode = MLX4_CMD_SW2HW_MPT,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_QUERY_MPT,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_HW2SW_MPT,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_READ_MTT,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SYNC_TPT,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+
+ {
+ .opcode = MLX4_CMD_HW2SW_EQ,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_QUERY_EQ,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SW2HW_CQ,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_HW2SW_CQ,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_QUERY_CQ,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_MODIFY_CQ,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = true,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SW2HW_SRQ,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_HW2SW_SRQ,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_QUERY_SRQ,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_ARM_SRQ,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_RST2INIT_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_INIT2RTR_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_RTR2RTS_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_RTS2RTS_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SQERR2RTS_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_2ERR_QP,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_RTS2SQD_QP,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SQD2SQD_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SQD2RTS_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_2RST_QP,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_QUERY_QP,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_INIT2INIT_QP,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_SUSPEND_QP,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_UNSUSPEND_QP,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+ {
+ .opcode = MLX4_CMD_MAD_IFC,
+ .has_inbox = true,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+
+ /* Native multicast commands are not available for guests */
+ {
+ .opcode = MLX4_CMD_DIAG_RPRT,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = NULL
+ },
+
+ /* Ethernet specific commands */
+ {
+ .opcode = MLX4_CMD_SET_VLAN_FLTR,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL /* need wrapper*/
+ },
+ {
+ .opcode = MLX4_CMD_SET_MCAST_FLTR,
+ .has_inbox = false,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL /* need wrapper*/
+ },
+ {
+ .opcode = MLX4_CMD_DUMP_ETH_STATS,
+ .has_inbox = false,
+ .has_outbox = true,
+ .out_is_imm = false,
+ .verify = NULL,
+ .wrapper = NULL /* need wrapper*/
+ },
+};
+
+static int mlx4_master_process_vhcr(struct mlx4_dev *dev, int slave)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_cmd_info *cmd = NULL;
+ struct mlx4_vhcr *vhcr = priv->mfunc.vhcr;
+ struct mlx4_cmd_mailbox *inbox = NULL;
+ struct mlx4_cmd_mailbox *outbox = NULL;
+ u64 in_param;
+ u64 out_param;
+ int ret;
+ int i;
+
+ /* DMA in the vHCR */
+ ret = mlx4_ACCESS_MEM(dev, priv->mfunc.vhcr_dma, slave,
+ priv->mfunc.master.slave_state[slave].vhcr_dma,
+ ALIGN(sizeof(struct mlx4_vhcr),
+ MLX4_ACCESS_MEM_ALIGN), 1);
+ if (ret) {
+ mlx4_err(dev, "Failed reading vhcr\n");
+ return ret;
+ }
+
+ /* Lookup command */
+ for (i = 0; i < ARRAY_SIZE(cmd_info); ++i) {
+ if (vhcr->op == cmd_info[i].opcode) {
+ cmd = &cmd_info[i];
+ break;
+ }
+ }
+ if (!cmd) {
+ mlx4_err(dev, "Unknown command:0x%x accepted from slave:%d\n",
+ vhcr->op, slave);
+ vhcr->errno = -EINVAL;
+ goto out_status;
+ }
+
+ /* Read inbox */
+ if (cmd->has_inbox) {
+ inbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(inbox)) {
+ ret = PTR_ERR(inbox);
+ inbox = NULL;
+ goto out;
+ }
+
+ /* FIXME: add mailbox size per-command */
+ ret = mlx4_ACCESS_MEM(dev, inbox->dma, slave,
+ vhcr->in_param,
+ MLX4_MAILBOX_SIZE, 1);
+ if (ret) {
+ mlx4_err(dev, "Failed reading inbox\n");
+ goto out;
+ }
+ }
+
+ /* Apply permission and bound checks if applicable */
+ if (cmd->verify && cmd->verify(dev, slave, vhcr, inbox)) {
+ mlx4_warn(dev, "Command:0x%x failed protection checks\n", vhcr->op);
+ vhcr->errno = -EPERM;
+ goto out_status;
+ }
+
+ /* Allocate outbox */
+ if (cmd->has_outbox) {
+ outbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(outbox)) {
+ ret = PTR_ERR(outbox);
+ outbox = NULL;
+ goto out;
+ }
+ }
+
+ /* Execute the command! */
+ if (cmd->wrapper)
+ vhcr->errno = cmd->wrapper(dev, slave, vhcr, inbox, outbox);
+ else {
+ in_param = cmd->has_inbox ? (u64) inbox->dma : vhcr->in_param;
+ out_param = cmd->has_outbox ? (u64) outbox->dma : vhcr->out_param;
+ vhcr->errno = __mlx4_cmd(dev, in_param, &out_param,
+ cmd->out_is_imm,
+ vhcr->in_modifier,
+ vhcr->op_modifier,
+ vhcr->op,
+ vhcr->timeout);
+ if (cmd->out_is_imm)
+ vhcr->out_param = out_param;
+ }
+
+ /* Write outbox if command completed successfully */
+ if (cmd->has_outbox && !vhcr->errno) {
+ ret = mlx4_ACCESS_MEM(dev, outbox->dma, slave,
+ vhcr->out_param,
+ MLX4_MAILBOX_SIZE, 0);
+ if (ret) {
+ mlx4_err(dev, "Failed writing outbox\n");
+ goto out;
+ }
+ }
+
+out_status:
+ /* DMA back vhcr result */
+ ret = mlx4_ACCESS_MEM(dev, priv->mfunc.vhcr_dma, slave,
+ priv->mfunc.master.slave_state[slave].vhcr_dma,
+ ALIGN(sizeof(struct mlx4_vhcr),
+ MLX4_ACCESS_MEM_ALIGN), 0);
+ if (ret)
+ mlx4_err(dev, "Failed writing vhcr result\n");
+
+ if (vhcr->errno)
+ mlx4_warn(dev, "vhcr command:0x%x slave:%d failed with error:%d\n",
+ vhcr->op, slave, vhcr->errno);
+ /* Fall through... */
+
+out:
+ mlx4_free_cmd_mailbox(dev, inbox);
+ mlx4_free_cmd_mailbox(dev, outbox);
+ return ret;
+}
+
+static void mlx4_master_do_cmd(struct mlx4_dev *dev, int slave, u8 cmd, u16 param, u8 toggle)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_slave_state *slave_state = priv->mfunc.master.slave_state;
+ u8 toggle_next;
+ u32 reply;
+
+ if (cmd == MLX4_COMM_CMD_RESET) {
+ mlx4_warn(dev, "Received reset from slave:%d\n", slave);
+ goto reset_slave;
+ }
+
+ /* Increment next toggle token */
+ toggle_next = slave_state[slave].comm_toggle + 1;
+ if (toggle_next > 2)
+ toggle_next = 1;
+ if (toggle != toggle_next) {
+ mlx4_warn(dev, "Incorrect token:%d from slave:%d expected:%d\n",
+ toggle, toggle_next, slave);
+ goto reset_slave;
+ }
+
+ switch (cmd) {
+ case MLX4_COMM_CMD_VHCR0:
+ if (slave_state[slave].last_cmd != MLX4_COMM_CMD_RESET)
+ goto reset_slave;
+ slave_state[slave].vhcr_dma = ((u64) param) << 48;
+ break;
+ case MLX4_COMM_CMD_VHCR1:
+ if (slave_state[slave].last_cmd != MLX4_COMM_CMD_VHCR0)
+ goto reset_slave;
+ slave_state[slave].vhcr_dma |= ((u64) param) << 32;
+ break;
+ case MLX4_COMM_CMD_VHCR2:
+ if (slave_state[slave].last_cmd != MLX4_COMM_CMD_VHCR1)
+ goto reset_slave;
+ slave_state[slave].vhcr_dma |= ((u64) param) << 16;
+ break;
+ case MLX4_COMM_CMD_VHCR_EN:
+ if (slave_state[slave].last_cmd != MLX4_COMM_CMD_VHCR2)
+ goto reset_slave;
+ slave_state[slave].vhcr_dma |= param;
+ break;
+ case MLX4_COMM_CMD_VHCR_POST:
+ if ((slave_state[slave].last_cmd != MLX4_COMM_CMD_VHCR_EN) &&
+ (slave_state[slave].last_cmd != MLX4_COMM_CMD_VHCR_POST))
+ goto reset_slave;
+ if (mlx4_master_process_vhcr(dev, slave)) {
+ mlx4_err(dev, "Failed processing vhcr for slave:%d, reseting slave.\n", slave);
+ goto reset_slave;
+ }
+ break;
+ default:
+ mlx4_warn(dev, "Bad comm cmd:%d from slave:%d\n", cmd, slave);
+ goto reset_slave;
+ }
+
+ slave_state[slave].last_cmd = cmd;
+ slave_state[slave].comm_toggle = toggle_next;
+ reply = (u32) toggle_next << 30;
+ __raw_writel((__force u32) cpu_to_be32(reply),
+ &priv->mfunc.comm[slave].slave_read);
+ wmb();
+ return;
+
+reset_slave:
+ /* FIXME: cleanup any slave resources */
+ slave_state[slave].last_cmd = MLX4_COMM_CMD_RESET;
+ slave_state[slave].comm_toggle = 0;
+ __raw_writel((__force u32) 0, &priv->mfunc.comm[slave].slave_write);
+ __raw_writel((__force u32) 0, &priv->mfunc.comm[slave].slave_read);
+ wmb();
+}
+
+/* master command processing */
+static void mlx4_master_poll_comm(struct work_struct *work)
+{
+ struct delayed_work *delay = container_of(work, struct delayed_work, work);
+ struct mlx4_mfunc *mfunc = container_of(delay, struct mlx4_mfunc, comm_work);
+ struct mlx4_priv *priv = container_of(mfunc, struct mlx4_priv, mfunc);
+ struct mlx4_dev *dev = &priv->dev;
+ u32 comm_cmd;
+ int polled = 0;
+ int i;
+
+ /* Give each slave a chance for one command */
+ for (i = 0; i < dev->num_slaves; i++) {
+ comm_cmd = swab32(readl(&priv->mfunc.comm[i].slave_write));
+ if (comm_cmd >> 30 != priv->mfunc.master.slave_state[i].comm_toggle) {
+ mlx4_master_do_cmd(dev, i, comm_cmd >> 16, comm_cmd, comm_cmd >> 30);
+ polled = 1;
+ }
+ }
+ queue_delayed_work(priv->mfunc.comm_wq, &priv->mfunc.comm_work,
+ polled ? 0 : HZ / 10);
+}
+
+int mlx4_multi_func_init(struct mlx4_dev *dev)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ int i;
+
+ priv->mfunc.vhcr = dma_alloc_coherent(&(dev->pdev->dev), PAGE_SIZE,
+ &priv->mfunc.vhcr_dma,
+ GFP_KERNEL);
+ if (!priv->mfunc.vhcr) {
+ mlx4_err(dev, "Couldn't allocate vhcr.\n");
+ return -ENOMEM;
+ }
+
+ if (mlx4_is_master(dev))
+ priv->mfunc.comm = ioremap(pci_resource_start(dev->pdev,
+ priv->fw.comm_bar) +
+ priv->fw.comm_base,
+ MLX4_COMM_PAGESIZE);
+ else
+ priv->mfunc.comm = ioremap(pci_resource_start(dev->pdev, 0) +
+ MLX4_SLAVE_COMM_BASE,
+ MLX4_COMM_PAGESIZE);
+ if (!priv->mfunc.comm) {
+ mlx4_err(dev, "Couldn't map communication vector.");
+ goto err_vhcr;
+ }
+
+ if (mlx4_is_master(dev)) {
+ priv->mfunc.master.slave_state = kzalloc(dev->num_slaves *
+ sizeof(struct mlx4_slave_state),
+ GFP_KERNEL);
+ if (!priv->mfunc.master.slave_state)
+ goto err_comm;
+
+ for (i = 0; i < dev->num_slaves; ++i)
+ priv->mfunc.master.slave_state[i].last_cmd = MLX4_COMM_CMD_RESET;
+
+ INIT_DELAYED_WORK(&priv->mfunc.comm_work, mlx4_master_poll_comm);
+ priv->mfunc.comm_wq = create_singlethread_workqueue("mlx4_comm");
+ if (!priv->mfunc.comm_wq) {
+ kfree(priv->mfunc.master.slave_state);
+ goto err_comm;
+ }
+ } else {
+ priv->cmd.comm_toggle = 0;
+ INIT_DELAYED_WORK(&priv->mfunc.comm_work, mlx4_slave_async_eq_poll);
+ priv->mfunc.comm_wq = create_singlethread_workqueue("mlx4_event");
+ if (!priv->mfunc.comm_wq)
+ goto err_comm;
+ }
+ return 0;
+
+err_comm:
+ iounmap(priv->mfunc.comm);
+err_vhcr:
+ dma_free_coherent(&(dev->pdev->dev), PAGE_SIZE,
+ priv->mfunc.vhcr,
+ priv->mfunc.vhcr_dma);
+ priv->mfunc.vhcr = NULL;
+ return -ENOMEM;
+}
+
int mlx4_cmd_init(struct mlx4_dev *dev)
{
struct mlx4_priv *priv = mlx4_priv(dev);
@@ -329,22 +1031,48 @@ int mlx4_cmd_init(struct mlx4_dev *dev)
priv->cmd.use_events = 0;
priv->cmd.toggle = 1;
- priv->cmd.hcr = ioremap(pci_resource_start(dev->pdev, 0) + MLX4_HCR_BASE,
- MLX4_HCR_SIZE);
- if (!priv->cmd.hcr) {
- mlx4_err(dev, "Couldn't map command register.");
- return -ENOMEM;
+ priv->cmd.hcr = NULL;
+ priv->mfunc.vhcr = NULL;
+
+ if (!mlx4_is_slave(dev)) {
+ priv->cmd.hcr = ioremap(pci_resource_start(dev->pdev, 0) +
+ ((dev->flags & MLX4_FLAG_SRIOV) ?
+ MLX4_HCR_SRIOV_BASE :
+ MLX4_HCR_BASE),
+ MLX4_HCR_SIZE);
+ if (!priv->cmd.hcr) {
+ mlx4_err(dev, "Couldn't map command register.");
+ return -ENOMEM;
+ }
}
priv->cmd.pool = pci_pool_create("mlx4_cmd", dev->pdev,
MLX4_MAILBOX_SIZE,
MLX4_MAILBOX_SIZE, 0);
- if (!priv->cmd.pool) {
- iounmap(priv->cmd.hcr);
- return -ENOMEM;
- }
+ if (!priv->cmd.pool)
+ goto err_hcr;
return 0;
+
+err_hcr:
+ if (!mlx4_is_slave(dev))
+ iounmap(priv->cmd.hcr);
+ return -ENOMEM;
+}
+
+void mlx4_multi_func_cleanup(struct mlx4_dev *dev)
+{
+ struct mlx4_priv *priv = mlx4_priv(dev);
+
+ if (priv->mfunc.vhcr) {
+ destroy_workqueue(priv->mfunc.comm_wq);
+ kfree(priv->mfunc.master.slave_state);
+ iounmap(priv->mfunc.comm);
+ dma_free_coherent(&(dev->pdev->dev), PAGE_SIZE,
+ priv->mfunc.vhcr,
+ priv->mfunc.vhcr_dma);
+ priv->mfunc.vhcr = NULL;
+ }
}
void mlx4_cmd_cleanup(struct mlx4_dev *dev)
@@ -352,7 +1080,9 @@ void mlx4_cmd_cleanup(struct mlx4_dev *dev)
struct mlx4_priv *priv = mlx4_priv(dev);
pci_pool_destroy(priv->cmd.pool);
- iounmap(priv->cmd.hcr);
+
+ if (!mlx4_is_slave(dev))
+ iounmap(priv->cmd.hcr);
}
/*
diff --git a/drivers/net/mlx4/en_port.h b/drivers/net/mlx4/en_port.h
index e6477f1..3892896 100644
--- a/drivers/net/mlx4/en_port.h
+++ b/drivers/net/mlx4/en_port.h
@@ -38,11 +38,6 @@
#define SET_PORT_GEN_ALL_VALID 0x7
#define SET_PORT_PROMISC_SHIFT 31
-enum {
- MLX4_CMD_SET_VLAN_FLTR = 0x47,
- MLX4_CMD_SET_MCAST_FLTR = 0x48,
- MLX4_CMD_DUMP_ETH_STATS = 0x49,
-};
struct mlx4_set_port_general_context {
u8 reserved[3];
diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c
index bffb799..70c16d4 100644
--- a/drivers/net/mlx4/eq.c
+++ b/drivers/net/mlx4/eq.c
@@ -160,6 +160,22 @@ static struct mlx4_eqe *next_eqe_sw(struct mlx4_eq *eq)
return !!(eqe->owner & 0x80) ^ !!(eq->cons_index & eq->nent) ? NULL : eqe;
}
+static int mlx4_GET_EVENT(struct mlx4_dev *dev, struct mlx4_slave_eqe *eqe)
+{
+ int ret;
+ u64 out_param;
+
+ ret = mlx4_cmd_imm(dev, 0, &out_param, 0, 0, MLX4_CMD_GET_EVENT,
+ MLX4_CMD_TIME_CLASS_A);
+ if (!ret) {
+ eqe->type = out_param & 0xff;
+ eqe->port = (out_param >> 8) & 0xff;
+ eqe->param = out_param >> 32;
+ } else
+ mlx4_err(dev, "Failed retrieving event\n");
+ return ret;
+}
+
static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq)
{
struct mlx4_eqe *eqe;
@@ -262,6 +278,57 @@ static int mlx4_eq_int(struct mlx4_dev *dev, struct mlx4_eq *eq)
return eqes_found;
}
+void mlx4_slave_async_eq_poll(struct work_struct *work)
+{
+ struct delayed_work *delay = container_of(work, struct delayed_work, work);
+ struct mlx4_mfunc *mfunc = container_of(delay, struct mlx4_mfunc, comm_work);
+ struct mlx4_priv *priv = container_of(mfunc, struct mlx4_priv, mfunc);
+ struct mlx4_dev *dev = &priv->dev;
+ struct mlx4_slave_eqe eqe;
+ int ret;
+ int i;
+
+ for (i = 0; i < MLX4_MFUNC_MAX_EQES; i++) {
+ ret = mlx4_GET_EVENT(dev, &eqe);
+ if (ret || eqe.type == MLX4_EVENT_TYPE_NONE)
+ break;
+
+ switch (eqe.type) {
+ case MLX4_EVENT_TYPE_PATH_MIG:
+ case MLX4_EVENT_TYPE_COMM_EST:
+ case MLX4_EVENT_TYPE_SQ_DRAINED:
+ case MLX4_EVENT_TYPE_SRQ_QP_LAST_WQE:
+ case MLX4_EVENT_TYPE_WQ_CATAS_ERROR:
+ case MLX4_EVENT_TYPE_PATH_MIG_FAILED:
+ case MLX4_EVENT_TYPE_WQ_INVAL_REQ_ERROR:
+ case MLX4_EVENT_TYPE_WQ_ACCESS_ERROR:
+ mlx4_qp_event(dev, eqe.param, eqe.type);
+ break;
+
+ case MLX4_EVENT_TYPE_SRQ_LIMIT:
+ case MLX4_EVENT_TYPE_SRQ_CATAS_ERROR:
+ mlx4_srq_event(dev, eqe.param, eqe.type);
+ break;
+
+ case MLX4_EVENT_TYPE_PORT_CHANGE:
+ mlx4_dispatch_event(dev, eqe.param, eqe.port);
+ break;
+
+ case MLX4_EVENT_TYPE_CQ_ERROR:
+ mlx4_cq_event(dev, eqe.param, eqe.type);
+ break;
+
+ case MLX4_EVENT_TYPE_EQ_OVERFLOW:
+ mlx4_warn(dev, "slave async EQ overrun\n");
+ break;
+
+ default:
+ mlx4_warn(dev, "Unhandled event:%02x\n", eqe.type);
+ }
+ }
+ queue_delayed_work(priv->mfunc.comm_wq, &priv->mfunc.comm_work, HZ);
+}
+
static irqreturn_t mlx4_interrupt(int irq, void *dev_ptr)
{
struct mlx4_dev *dev = dev_ptr;
diff --git a/drivers/net/mlx4/fw.c b/drivers/net/mlx4/fw.c
index 3c16602..4ca8060 100644
--- a/drivers/net/mlx4/fw.c
+++ b/drivers/net/mlx4/fw.c
@@ -554,6 +554,9 @@ int mlx4_QUERY_FW(struct mlx4_dev *dev)
#define QUERY_FW_CLR_INT_BASE_OFFSET 0x20
#define QUERY_FW_CLR_INT_BAR_OFFSET 0x28
+#define QUERY_FW_COMM_BASE_OFFSET 0x40
+#define QUERY_FW_COMM_BAR_OFFSET 0x48
+
mailbox = mlx4_alloc_cmd_mailbox(dev);
if (IS_ERR(mailbox))
return PTR_ERR(mailbox);
@@ -614,6 +617,11 @@ int mlx4_QUERY_FW(struct mlx4_dev *dev)
MLX4_GET(fw->clr_int_bar, outbox, QUERY_FW_CLR_INT_BAR_OFFSET);
fw->clr_int_bar = (fw->clr_int_bar >> 6) * 2;
+ MLX4_GET(fw->comm_base, outbox, QUERY_FW_COMM_BASE_OFFSET);
+ MLX4_GET(fw->comm_bar, outbox, QUERY_FW_COMM_BAR_OFFSET);
+ fw->comm_bar = (fw->comm_bar >> 6) * 2;
+ mlx4_dbg(dev, "Communication vector bar:%d offset:0x%llx\n", fw->comm_bar,
+ fw->comm_base);
mlx4_dbg(dev, "FW size %d KB\n", fw->fw_pages >> 2);
/*
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index 5836c94..8713be2 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -53,8 +53,11 @@
enum {
MLX4_HCR_BASE = 0x80680,
+ MLX4_HCR_SRIOV_BASE = 0x4080680, /* good for SRIOV FW ony */
MLX4_HCR_SIZE = 0x0001c,
- MLX4_CLR_INT_SIZE = 0x00008
+ MLX4_CLR_INT_SIZE = 0x00008,
+ MLX4_SLAVE_COMM_BASE = 0x0,
+ MLX4_COMM_PAGESIZE = 0x1000
};
enum {
@@ -80,6 +83,21 @@ enum {
MLX4_NUM_CMPTS = MLX4_CMPT_NUM_TYPE << MLX4_CMPT_SHIFT
};
+#define MLX4_COMM_TIME 10000
+enum {
+ MLX4_COMM_CMD_RESET,
+ MLX4_COMM_CMD_VHCR0,
+ MLX4_COMM_CMD_VHCR1,
+ MLX4_COMM_CMD_VHCR2,
+ MLX4_COMM_CMD_VHCR_EN,
+ MLX4_COMM_CMD_VHCR_POST
+};
+
+enum {
+ MLX4_MFUNC_MAX_EQES = 8,
+ MLX4_MFUNC_EQE_MASK = (MLX4_MFUNC_MAX_EQES - 1)
+};
+
#ifdef CONFIG_MLX4_DEBUG
extern int mlx4_debug_level;
#else /* CONFIG_MLX4_DEBUG */
@@ -154,12 +172,56 @@ struct mlx4_profile {
struct mlx4_fw {
u64 clr_int_base;
u64 catas_offset;
+ u64 comm_base;
struct mlx4_icm *fw_icm;
struct mlx4_icm *aux_icm;
u32 catas_size;
u16 fw_pages;
u8 clr_int_bar;
u8 catas_bar;
+ u8 comm_bar;
+};
+
+struct mlx4_comm {
+ u32 slave_write;
+ u32 slave_read;
+};
+
+struct mlx4_slave_eqe {
+ u8 type;
+ u8 port;
+ u32 param;
+};
+
+struct mlx4_slave_state {
+ u8 comm_toggle;
+ u8 last_cmd;
+ dma_addr_t vhcr_dma;
+};
+
+struct mlx4_mfunc_master_ctx {
+ struct mlx4_slave_state *slave_state;
+};
+
+struct mlx4_vhcr {
+ u64 in_param;
+ u64 out_param;
+ u32 in_modifier;
+ u32 timeout;
+ u16 op;
+ u16 token;
+ u8 op_modifier;
+ int errno;
+};
+
+struct mlx4_mfunc {
+ struct mlx4_comm __iomem *comm;
+ struct workqueue_struct *comm_wq;
+ struct delayed_work comm_work;
+ struct mlx4_vhcr *vhcr;
+ dma_addr_t vhcr_dma;
+
+ struct mlx4_mfunc_master_ctx master;
};
struct mlx4_cmd {
@@ -175,6 +237,7 @@ struct mlx4_cmd {
u16 token_mask;
u8 use_events;
u8 toggle;
+ u8 comm_toggle;
};
struct mlx4_uar_table {
@@ -294,6 +357,7 @@ struct mlx4_priv {
struct mlx4_fw fw;
struct mlx4_cmd cmd;
+ struct mlx4_mfunc mfunc;
struct mlx4_bitmap pd_bitmap;
struct mlx4_uar_table uar_table;
@@ -372,13 +436,19 @@ u64 mlx4_make_profile(struct mlx4_dev *dev,
struct mlx4_profile *request,
struct mlx4_dev_cap *dev_cap,
struct mlx4_init_hca_param *init_hca);
+void mlx4_slave_async_eq_poll(struct work_struct *work);
int mlx4_cmd_init(struct mlx4_dev *dev);
void mlx4_cmd_cleanup(struct mlx4_dev *dev);
+int mlx4_multi_func_init(struct mlx4_dev *dev);
+void mlx4_multi_func_cleanup(struct mlx4_dev *dev);
void mlx4_cmd_event(struct mlx4_dev *dev, u16 token, u8 status, u64 out_param);
int mlx4_cmd_use_events(struct mlx4_dev *dev);
void mlx4_cmd_use_polling(struct mlx4_dev *dev);
+int mlx4_comm_cmd(struct mlx4_dev *dev, u8 cmd, u16 param, unsigned long timeout);
+
+
void mlx4_cq_completion(struct mlx4_dev *dev, u32 cqn);
void mlx4_cq_event(struct mlx4_dev *dev, u32 cqn, int event_type);
diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h
index 0f82293..b84ff08 100644
--- a/include/linux/mlx4/cmd.h
+++ b/include/linux/mlx4/cmd.h
@@ -117,6 +117,15 @@ enum {
/* miscellaneous commands */
MLX4_CMD_DIAG_RPRT = 0x30,
MLX4_CMD_NOP = 0x31,
+ MLX4_CMD_ACCESS_MEM = 0x2e,
+
+ /* Ethernet specific commands */
+ MLX4_CMD_SET_VLAN_FLTR = 0x47,
+ MLX4_CMD_SET_MCAST_FLTR = 0x48,
+ MLX4_CMD_DUMP_ETH_STATS = 0x49,
+
+ /* virtual commands */
+ MLX4_CMD_GET_EVENT = 0x52,
/* debug commands */
MLX4_CMD_QUERY_DEBUG_MSG = 0x2a,
@@ -130,7 +139,8 @@ enum {
};
enum {
- MLX4_MAILBOX_SIZE = 4096
+ MLX4_MAILBOX_SIZE = 4096,
+ MLX4_ACCESS_MEM_ALIGN = 256,
};
enum {
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index f35703d..9735f40 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -98,7 +98,8 @@ enum mlx4_event {
MLX4_EVENT_TYPE_PORT_CHANGE = 0x09,
MLX4_EVENT_TYPE_EQ_OVERFLOW = 0x0f,
MLX4_EVENT_TYPE_ECC_DETECT = 0x0e,
- MLX4_EVENT_TYPE_CMD = 0x0a
+ MLX4_EVENT_TYPE_CMD = 0x0a,
+ MLX4_EVENT_TYPE_NONE = 0xff,
};
enum {
--
1.5.3.7
^ permalink raw reply related
* [PATCH 04/25 v2] mlx4_core: add WRITE_MTT support
From: Yevgeny Petrilin @ 2009-11-06 3:08 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
From: Liran Liss <liranl@mellanox.co.il>
Used by vfs to modify mtts, since they cannot access in-memory mtts directly.
Signed-off-by: Liran Liss <liranl@mellanox.co.il>
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/cmd.c | 10 ++++++-
drivers/net/mlx4/mlx4.h | 5 ++++
drivers/net/mlx4/mr.c | 62 ++++++++++++++++++++++++++++++++++++++++++----
3 files changed, 69 insertions(+), 8 deletions(-)
diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c
index 19653e7..717dd50 100644
--- a/drivers/net/mlx4/cmd.c
+++ b/drivers/net/mlx4/cmd.c
@@ -36,8 +36,6 @@
#include <linux/pci.h>
#include <linux/errno.h>
-#include <linux/mlx4/cmd.h>
-
#include <asm/io.h>
#include "mlx4.h"
@@ -498,6 +496,14 @@ static struct mlx4_cmd_info {
.wrapper = NULL
},
{
+ .opcode = MLX4_CMD_WRITE_MTT,
+ .has_inbox = true,
+ .has_outbox = false,
+ .out_is_imm = false,
+ .verify = NULL, /* need verifier */
+ .wrapper = mlx4_WRITE_MTT_wrapper
+ },
+ {
.opcode = MLX4_CMD_SYNC_TPT,
.has_inbox = true,
.has_outbox = false,
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index 8713be2..7516730 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -45,6 +45,7 @@
#include <linux/mlx4/device.h>
#include <linux/mlx4/driver.h>
#include <linux/mlx4/doorbell.h>
+#include <linux/mlx4/cmd.h>
#define DRV_NAME "mlx4_core"
#define PFX DRV_NAME ": "
@@ -421,6 +422,10 @@ void mlx4_cleanup_qp_table(struct mlx4_dev *dev);
void mlx4_cleanup_srq_table(struct mlx4_dev *dev);
void mlx4_cleanup_mcg_table(struct mlx4_dev *dev);
+int mlx4_WRITE_MTT_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
+ struct mlx4_cmd_mailbox *inbox,
+ struct mlx4_cmd_mailbox *outbox);
+
void mlx4_start_catas_poll(struct mlx4_dev *dev);
void mlx4_stop_catas_poll(struct mlx4_dev *dev);
void mlx4_catas_init(void);
diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c
index ca7ab8e..11a3d26 100644
--- a/drivers/net/mlx4/mr.c
+++ b/drivers/net/mlx4/mr.c
@@ -262,6 +262,35 @@ static int mlx4_HW2SW_MPT(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox
!mailbox, MLX4_CMD_HW2SW_MPT, MLX4_CMD_TIME_CLASS_B);
}
+int mlx4_WRITE_MTT_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
+ struct mlx4_cmd_mailbox *inbox,
+ struct mlx4_cmd_mailbox *outbox)
+{
+ struct mlx4_mtt mtt;
+ u64 *page_list = inbox->buf;
+ int i;
+
+ /* Call the SW implementation of write_mtt:
+ * - Prepare a dummy mtt struct
+ * - Translate inbox contents to simple addresses in host endianess */
+ mtt.first_seg = 0;
+ mtt.order = 0;
+ mtt.page_shift = 0;
+ for (i = 0; i < vhcr->in_modifier; ++i)
+ page_list[i + 2] = be64_to_cpu(page_list[i + 2]) & ~1ULL;
+ vhcr->errno = mlx4_write_mtt(dev, &mtt, be64_to_cpu(page_list[0]),
+ vhcr->in_modifier,
+ page_list + 2);
+ return 0;
+}
+
+static int mlx4_WRITE_MTT(struct mlx4_dev *dev, struct mlx4_cmd_mailbox *mailbox,
+ int num_entries)
+{
+ return mlx4_cmd(dev, mailbox->dma, num_entries, 0, MLX4_CMD_WRITE_MTT,
+ MLX4_CMD_TIME_CLASS_A);
+}
+
int mlx4_mr_alloc(struct mlx4_dev *dev, u32 pd, u64 iova, u64 size, u32 access,
int npages, int page_shift, struct mlx4_mr *mr)
{
@@ -413,24 +442,45 @@ static int mlx4_write_mtt_chunk(struct mlx4_dev *dev, struct mlx4_mtt *mtt,
int mlx4_write_mtt(struct mlx4_dev *dev, struct mlx4_mtt *mtt,
int start_index, int npages, u64 *page_list)
{
+ struct mlx4_cmd_mailbox *mailbox = NULL;
int chunk;
- int err;
+ int err = 0;
+ __be64 *inbox = NULL;
+ int i;
if (mtt->order < 0)
return -EINVAL;
+ if (mlx4_is_slave(dev)) {
+ mailbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(mailbox))
+ return PTR_ERR(mailbox);
+ inbox = mailbox->buf;
+ }
+
while (npages > 0) {
- chunk = min_t(int, PAGE_SIZE / sizeof(u64), npages);
- err = mlx4_write_mtt_chunk(dev, mtt, start_index, chunk, page_list);
+ if (mlx4_is_slave(dev)) {
+ chunk = min_t(int, MLX4_MAILBOX_SIZE / sizeof(u64) - MLX4_MTT_ENTRY_PER_SEG, npages);
+ inbox[0] = cpu_to_be64(mtt->first_seg * MLX4_MTT_ENTRY_PER_SEG + start_index);
+ inbox[1] = 0;
+ for (i = 0; i < chunk; ++i)
+ inbox[i + 2] = cpu_to_be64(page_list[i] | MLX4_MTT_FLAG_PRESENT);
+ err = mlx4_WRITE_MTT(dev, mailbox, chunk);
+ } else {
+ chunk = min_t(int, PAGE_SIZE / sizeof(u64), npages);
+ err = mlx4_write_mtt_chunk(dev, mtt, start_index, chunk, page_list);
+ }
if (err)
- return err;
+ goto out;
npages -= chunk;
start_index += chunk;
page_list += chunk;
}
-
- return 0;
+out:
+ if (mlx4_is_slave(dev))
+ mlx4_free_cmd_mailbox(dev, mailbox);
+ return err;
}
EXPORT_SYMBOL_GPL(mlx4_write_mtt);
--
1.5.3.7
^ permalink raw reply related
* [PATCH 14/25 v2] mlx4_core: multi-function resource setup
From: Yevgeny Petrilin @ 2009-11-06 3:09 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
From: Liran Liss <liranl@mellanox.co.il>
Only master function needs to configure eq asynch events, and initialize resource
allocators.
Signed-off-by: Liran Liss <liranl@mellanox.co.il>
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/cq.c | 4 +++
drivers/net/mlx4/eq.c | 61 ++++++++++++++++++++++++------------------
drivers/net/mlx4/main.c | 45 +++++++++++++++++--------------
drivers/net/mlx4/mcg.c | 6 ++++
drivers/net/mlx4/mr.c | 6 ++++
drivers/net/mlx4/pd.c | 12 ++++++--
drivers/net/mlx4/qp.c | 33 ++++++++++++++++++++++-
drivers/net/mlx4/srq.c | 4 +++
include/linux/mlx4/device.h | 1 +
9 files changed, 121 insertions(+), 51 deletions(-)
diff --git a/drivers/net/mlx4/cq.c b/drivers/net/mlx4/cq.c
index 9f9f246..b605041 100644
--- a/drivers/net/mlx4/cq.c
+++ b/drivers/net/mlx4/cq.c
@@ -353,6 +353,8 @@ int mlx4_init_cq_table(struct mlx4_dev *dev)
spin_lock_init(&cq_table->lock);
INIT_RADIX_TREE(&cq_table->tree, GFP_ATOMIC);
+ if (mlx4_is_slave(dev))
+ return 0;
err = mlx4_bitmap_init(&cq_table->bitmap, dev->caps.num_cqs,
dev->caps.num_cqs - 1, dev->caps.reserved_cqs, 0);
@@ -364,6 +366,8 @@ int mlx4_init_cq_table(struct mlx4_dev *dev)
void mlx4_cleanup_cq_table(struct mlx4_dev *dev)
{
+ if (mlx4_is_slave(dev))
+ return;
/* Nothing to do to clean up radix_tree */
mlx4_bitmap_cleanup(&mlx4_priv(dev)->cq_table.bitmap);
}
diff --git a/drivers/net/mlx4/eq.c b/drivers/net/mlx4/eq.c
index 46064cc..b0c8022 100644
--- a/drivers/net/mlx4/eq.c
+++ b/drivers/net/mlx4/eq.c
@@ -732,39 +732,42 @@ int mlx4_init_eq_table(struct mlx4_dev *dev)
goto err_out_free;
}
- err = mlx4_bitmap_init(&priv->eq_table.bitmap, dev->caps.num_eqs,
- dev->caps.num_eqs - 1, dev->caps.reserved_eqs, 0);
+ err = mlx4_bitmap_init_no_mask(&priv->eq_table.bitmap, dev->caps.num_eqs,
+ dev->caps.reserved_eqs, 0);
if (err)
goto err_out_free;
for (i = 0; i < mlx4_num_eq_uar(dev); ++i)
priv->eq_table.uar_map[i] = NULL;
- err = mlx4_map_clr_int(dev);
- if (err)
- goto err_out_bitmap;
+ if (!mlx4_is_slave(dev)) {
+ err = mlx4_map_clr_int(dev);
+ if (err)
+ goto err_out_bitmap;
- priv->eq_table.clr_mask =
- swab32(1 << (priv->eq_table.inta_pin & 31));
- priv->eq_table.clr_int = priv->clr_base +
- (priv->eq_table.inta_pin < 32 ? 4 : 0);
+ priv->eq_table.clr_mask =
+ swab32(1 << (priv->eq_table.inta_pin & 31));
+ priv->eq_table.clr_int = priv->clr_base +
+ (priv->eq_table.inta_pin < 32 ? 4 : 0);
+ }
priv->eq_table.irq_names =
kmalloc(MLX4_IRQNAME_SIZE * (dev->caps.num_comp_vectors + 1),
GFP_KERNEL);
if (!priv->eq_table.irq_names) {
err = -ENOMEM;
- goto err_out_bitmap;
+ i = 0;
+ goto err_out_unmap;
}
for (i = 0; i < dev->caps.num_comp_vectors; ++i) {
- err = mlx4_create_eq(dev, dev->caps.num_cqs + MLX4_NUM_SPARE_EQE,
+ err = mlx4_create_eq(dev, dev->caps.num_cqs -
+ dev->caps.reserved_cqs +
+ MLX4_NUM_SPARE_EQE,
(dev->flags & MLX4_FLAG_MSI_X) ? i : 0,
&priv->eq_table.eq[i]);
- if (err) {
- --i;
+ if (err)
goto err_out_unmap;
- }
}
err = mlx4_create_eq(dev, MLX4_NUM_ASYNC_EQE + MLX4_NUM_SPARE_EQE,
@@ -814,11 +817,13 @@ int mlx4_init_eq_table(struct mlx4_dev *dev)
priv->eq_table.have_irq = 1;
}
- err = mlx4_MAP_EQ(dev, MLX4_ASYNC_EVENT_MASK, 0,
- priv->eq_table.eq[dev->caps.num_comp_vectors].eqn);
- if (err)
- mlx4_warn(dev, "MAP_EQ for async EQ %d failed (%d)\n",
- priv->eq_table.eq[dev->caps.num_comp_vectors].eqn, err);
+ if (!mlx4_is_slave(dev)) { /* hw async events cannot be shared */
+ err = mlx4_MAP_EQ(dev, MLX4_ASYNC_EVENT_MASK, 0,
+ priv->eq_table.eq[dev->caps.num_comp_vectors].eqn);
+ if (err)
+ mlx4_warn(dev, "MAP_EQ for async EQ %d failed (%d)\n",
+ priv->eq_table.eq[dev->caps.num_comp_vectors].eqn, err);
+ }
for (i = 0; i < dev->caps.num_comp_vectors + 1; ++i)
eq_set_ci(&priv->eq_table.eq[i], 1);
@@ -829,14 +834,15 @@ err_out_async:
mlx4_free_eq(dev, &priv->eq_table.eq[dev->caps.num_comp_vectors]);
err_out_comp:
- i = dev->caps.num_comp_vectors - 1;
+ i = dev->caps.num_comp_vectors;
err_out_unmap:
- while (i >= 0) {
- mlx4_free_eq(dev, &priv->eq_table.eq[i]);
+ while (i > 0) {
--i;
+ mlx4_free_eq(dev, &priv->eq_table.eq[i]);
}
- mlx4_unmap_clr_int(dev);
+ if (!mlx4_is_slave(dev))
+ mlx4_unmap_clr_int(dev);
mlx4_free_irqs(dev);
err_out_bitmap:
@@ -853,15 +859,18 @@ void mlx4_cleanup_eq_table(struct mlx4_dev *dev)
struct mlx4_priv *priv = mlx4_priv(dev);
int i;
- mlx4_MAP_EQ(dev, MLX4_ASYNC_EVENT_MASK, 1,
- priv->eq_table.eq[dev->caps.num_comp_vectors].eqn);
+ if (!mlx4_is_slave(dev)) {
+ mlx4_MAP_EQ(dev, MLX4_ASYNC_EVENT_MASK, 1,
+ priv->eq_table.eq[dev->caps.num_comp_vectors].eqn);
+ }
mlx4_free_irqs(dev);
for (i = 0; i < dev->caps.num_comp_vectors + 1; ++i)
mlx4_free_eq(dev, &priv->eq_table.eq[i]);
- mlx4_unmap_clr_int(dev);
+ if (!mlx4_is_slave(dev))
+ mlx4_unmap_clr_int(dev);
for (i = 0; i < mlx4_num_eq_uar(dev); ++i)
if (priv->eq_table.uar_map[i])
diff --git a/drivers/net/mlx4/main.c b/drivers/net/mlx4/main.c
index d8143c6..2852445 100644
--- a/drivers/net/mlx4/main.c
+++ b/drivers/net/mlx4/main.c
@@ -904,11 +904,14 @@ static int mlx4_setup_hca(struct mlx4_dev *dev)
goto err_mr_table_free;
}
- err = mlx4_cmd_use_events(dev);
- if (err) {
- mlx4_err(dev, "Failed to switch to event-driven "
- "firmware commands, aborting.\n");
- goto err_eq_table_free;
+ /* CX1: no comm channel events */
+ if (!mlx4_is_master(dev) && !mlx4_is_slave(dev)) {
+ err = mlx4_cmd_use_events(dev);
+ if (err) {
+ mlx4_err(dev, "Failed to switch to event-driven "
+ "firmware commands, aborting.\n");
+ goto err_eq_table_free;
+ }
}
err = mlx4_NOP(dev);
@@ -958,22 +961,23 @@ static int mlx4_setup_hca(struct mlx4_dev *dev)
goto err_qp_table_free;
}
- for (port = 1; port <= dev->caps.num_ports; port++) {
- ib_port_default_caps = 0;
- err = mlx4_get_port_ib_caps(dev, port, &ib_port_default_caps);
- if (err)
- mlx4_warn(dev, "failed to get port %d default "
- "ib capabilities (%d). Continuing with "
- "caps = 0\n", port, err);
- dev->caps.ib_port_def_cap[port] = ib_port_default_caps;
- err = mlx4_SET_PORT(dev, port);
- if (err) {
- mlx4_err(dev, "Failed to set port %d, aborting\n",
- port);
- goto err_mcg_table_free;
+ if (!mlx4_is_slave(dev)) {
+ for (port = 1; port <= dev->caps.num_ports; port++) {
+ ib_port_default_caps = 0;
+ err = mlx4_get_port_ib_caps(dev, port, &ib_port_default_caps);
+ if (err)
+ mlx4_warn(dev, "failed to get port %d default "
+ "ib capabilities (%d). Continuing with "
+ "caps = 0\n", port, err);
+ dev->caps.ib_port_def_cap[port] = ib_port_default_caps;
+ err = mlx4_SET_PORT(dev, port);
+ if (err) {
+ mlx4_err(dev, "Failed to set port %d, aborting\n",
+ port);
+ goto err_mcg_table_free;
+ }
}
}
-
return 0;
err_mcg_table_free:
@@ -989,7 +993,8 @@ err_cq_table_free:
mlx4_cleanup_cq_table(dev);
err_cmd_poll:
- mlx4_cmd_use_polling(dev);
+ if (!mlx4_is_master(dev) && !mlx4_is_slave(dev))
+ mlx4_cmd_use_polling(dev);
err_eq_table_free:
mlx4_cleanup_eq_table(dev);
diff --git a/drivers/net/mlx4/mcg.c b/drivers/net/mlx4/mcg.c
index 96f09ed..daa08f1 100644
--- a/drivers/net/mlx4/mcg.c
+++ b/drivers/net/mlx4/mcg.c
@@ -393,6 +393,10 @@ int mlx4_init_mcg_table(struct mlx4_dev *dev)
struct mlx4_priv *priv = mlx4_priv(dev);
int err;
+ /* Nothing to do for slaves - mcg handling is para-virtualized */
+ if (mlx4_is_slave(dev))
+ return 0;
+
err = mlx4_bitmap_init(&priv->mcg_table.bitmap, dev->caps.num_amgms,
dev->caps.num_amgms - 1, 0, 0);
if (err)
@@ -405,5 +409,7 @@ int mlx4_init_mcg_table(struct mlx4_dev *dev)
void mlx4_cleanup_mcg_table(struct mlx4_dev *dev)
{
+ if (mlx4_is_slave(dev))
+ return;
mlx4_bitmap_cleanup(&mlx4_priv(dev)->mcg_table.bitmap);
}
diff --git a/drivers/net/mlx4/mr.c b/drivers/net/mlx4/mr.c
index aa1eba6..522d29a 100644
--- a/drivers/net/mlx4/mr.c
+++ b/drivers/net/mlx4/mr.c
@@ -612,6 +612,10 @@ int mlx4_init_mr_table(struct mlx4_dev *dev)
struct mlx4_mr_table *mr_table = &mlx4_priv(dev)->mr_table;
int err;
+ /* Nothing to do for slaves - all MR handling is forwarded to the master */
+ if (mlx4_is_slave(dev))
+ return 0;
+
err = mlx4_bitmap_init(&mr_table->mpt_bitmap, dev->caps.num_mpts,
~0, dev->caps.reserved_mrws, 0);
if (err)
@@ -646,6 +650,8 @@ void mlx4_cleanup_mr_table(struct mlx4_dev *dev)
{
struct mlx4_mr_table *mr_table = &mlx4_priv(dev)->mr_table;
+ if (mlx4_is_slave(dev))
+ return;
mlx4_buddy_cleanup(&mr_table->mtt_buddy);
mlx4_bitmap_cleanup(&mr_table->mpt_bitmap);
}
diff --git a/drivers/net/mlx4/pd.c b/drivers/net/mlx4/pd.c
index 08eaf08..db7e2f4 100644
--- a/drivers/net/mlx4/pd.c
+++ b/drivers/net/mlx4/pd.c
@@ -96,6 +96,10 @@ EXPORT_SYMBOL_GPL(mlx4_uar_free);
int mlx4_init_uar_table(struct mlx4_dev *dev)
{
+ /* CX1: master doesn't have UARs */
+ if (mlx4_is_master(dev))
+ return 0;
+
if (dev->caps.num_uars <= 128) {
mlx4_err(dev, "Only %d UAR pages (need more than 128)\n",
dev->caps.num_uars);
@@ -103,12 +107,14 @@ int mlx4_init_uar_table(struct mlx4_dev *dev)
return -ENODEV;
}
- return mlx4_bitmap_init(&mlx4_priv(dev)->uar_table.bitmap,
- dev->caps.num_uars, dev->caps.num_uars - 1,
- max(128, dev->caps.reserved_uars), 0);
+ return mlx4_bitmap_init_no_mask(&mlx4_priv(dev)->uar_table.bitmap,
+ dev->caps.num_uars,
+ dev->caps.reserved_uars, 0);
}
void mlx4_cleanup_uar_table(struct mlx4_dev *dev)
{
+ if (mlx4_is_master(dev))
+ return;
mlx4_bitmap_cleanup(&mlx4_priv(dev)->uar_table.bitmap);
}
diff --git a/drivers/net/mlx4/qp.c b/drivers/net/mlx4/qp.c
index 99b9ded..60c1d7f 100644
--- a/drivers/net/mlx4/qp.c
+++ b/drivers/net/mlx4/qp.c
@@ -406,6 +406,23 @@ int mlx4_init_qp_table(struct mlx4_dev *dev)
spin_lock_init(&qp_table->lock);
INIT_RADIX_TREE(&dev->qp_table_tree, GFP_ATOMIC);
+ if (mlx4_is_slave(dev)) {
+ /* For each slave, just allocate a normal 8-byte alligned special-QP
+ * range intead of mlx4_init_qp_table() reservation */
+ err = mlx4_qp_reserve_range(dev, 8, 8, &dev->caps.sqp_start);
+ if (err) {
+ mlx4_err(dev, "Failed to allocate special QP range\n");
+ return err;
+ }
+
+ err = mlx4_CONF_SPECIAL_QP(dev, dev->caps.sqp_start);
+ if (err) {
+ mlx4_err(dev, "Failed to configure special QP range\n");
+ mlx4_qp_release_range(dev, dev->caps.sqp_start, 8);
+ return err;
+ }
+ return 0;
+ }
/*
* We reserve 2 extra QPs per port for the special QPs. The
@@ -415,6 +432,10 @@ int mlx4_init_qp_table(struct mlx4_dev *dev)
dev->caps.sqp_start =
ALIGN(dev->caps.reserved_qps_cnt[MLX4_QP_REGION_FW], 8);
+ /* If multi-function is enabled, we reserve an additional QP for qp0/1 tunneling.
+ * CX1: slave0 manages tunnel QP */
+ dev->caps.tunnel_qpn = mlx4_is_master(dev) ? dev->caps.sqp_start + 8 : 0;
+
{
int sort[MLX4_NUM_QP_REGION];
int i, j, tmp;
@@ -444,17 +465,25 @@ int mlx4_init_qp_table(struct mlx4_dev *dev)
}
err = mlx4_bitmap_init(&qp_table->bitmap, dev->caps.num_qps,
- (1 << 23) - 1, dev->caps.sqp_start + 8,
- reserved_from_top);
+ (1 << 23) - 1, dev->caps.sqp_start + 8 +
+ 2 * !!dev->caps.tunnel_qpn, reserved_from_top);
if (err)
return err;
+ /* CX1: master has no QPs */
+ if (mlx4_is_master(dev))
+ return 0;
+
return mlx4_CONF_SPECIAL_QP(dev, dev->caps.sqp_start);
}
void mlx4_cleanup_qp_table(struct mlx4_dev *dev)
{
mlx4_CONF_SPECIAL_QP(dev, 0);
+ if (mlx4_is_slave(dev)) {
+ mlx4_qp_release_range(dev, dev->caps.sqp_start, 8);
+ return;
+ }
mlx4_bitmap_cleanup(&mlx4_priv(dev)->qp_table.bitmap);
}
diff --git a/drivers/net/mlx4/srq.c b/drivers/net/mlx4/srq.c
index ed11f18..3050318 100644
--- a/drivers/net/mlx4/srq.c
+++ b/drivers/net/mlx4/srq.c
@@ -287,6 +287,8 @@ int mlx4_init_srq_table(struct mlx4_dev *dev)
spin_lock_init(&srq_table->lock);
INIT_RADIX_TREE(&srq_table->tree, GFP_ATOMIC);
+ if (mlx4_is_slave(dev))
+ return 0;
err = mlx4_bitmap_init(&srq_table->bitmap, dev->caps.num_srqs,
dev->caps.num_srqs - 1, dev->caps.reserved_srqs, 0);
@@ -298,5 +300,7 @@ int mlx4_init_srq_table(struct mlx4_dev *dev)
void mlx4_cleanup_srq_table(struct mlx4_dev *dev)
{
+ if (mlx4_is_slave(dev))
+ return;
mlx4_bitmap_cleanup(&mlx4_priv(dev)->srq_table.bitmap);
}
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index 4b9091a..bbd398b 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -204,6 +204,7 @@ struct mlx4_caps {
int max_qp_init_rdma;
int max_qp_dest_rdma;
int sqp_start;
+ int tunnel_qpn;
int num_srqs;
int max_srq_wqes;
int max_srq_sge;
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* [PATCH 15/25 v2] mlx4_core: multi-function interface setup
From: Yevgeny Petrilin @ 2009-11-06 3:09 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
From: Liran Liss <liranl@mellanox.co.il>
Master verifies hw healthiness.
Interface drivers are only supported on slaves.
Signed-off-by: Liran Liss <liranl@mellanox.co.il>
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/intf.c | 21 ++++++++++++++-------
1 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/drivers/net/mlx4/intf.c b/drivers/net/mlx4/intf.c
index 0e7eb10..f5f4523 100644
--- a/drivers/net/mlx4/intf.c
+++ b/drivers/net/mlx4/intf.c
@@ -88,8 +88,11 @@ int mlx4_register_interface(struct mlx4_interface *intf)
mutex_lock(&intf_mutex);
list_add_tail(&intf->list, &intf_list);
- list_for_each_entry(priv, &dev_list, dev_list)
- mlx4_add_device(intf, priv);
+ list_for_each_entry(priv, &dev_list, dev_list) {
+ /* CX1: master cannot run interfaces */
+ if (!mlx4_is_master(&priv->dev))
+ mlx4_add_device(intf, priv);
+ }
mutex_unlock(&intf_mutex);
@@ -135,11 +138,14 @@ int mlx4_register_device(struct mlx4_dev *dev)
mutex_lock(&intf_mutex);
list_add_tail(&priv->dev_list, &dev_list);
- list_for_each_entry(intf, &intf_list, list)
- mlx4_add_device(intf, priv);
-
+ /* CX1: master cannot run interfaces */
+ if (!mlx4_is_master(dev)) {
+ list_for_each_entry(intf, &intf_list, list)
+ mlx4_add_device(intf, priv);
+ }
mutex_unlock(&intf_mutex);
- mlx4_start_catas_poll(dev);
+ if (!mlx4_is_slave(dev))
+ mlx4_start_catas_poll(dev);
return 0;
}
@@ -149,7 +155,8 @@ void mlx4_unregister_device(struct mlx4_dev *dev)
struct mlx4_priv *priv = mlx4_priv(dev);
struct mlx4_interface *intf;
- mlx4_stop_catas_poll(dev);
+ if (!mlx4_is_slave(dev))
+ mlx4_stop_catas_poll(dev);
mutex_lock(&intf_mutex);
list_for_each_entry(intf, &intf_list, list)
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* [PATCH 18/25 v2] mlx4_en: Use reasonable resources for slaves.
From: Yevgeny Petrilin @ 2009-11-06 3:09 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
Each slave gets a single MSI-X vector when in multi function mode.
In this case each slave would allocate less resources (which
normally depend on number of completion vectors).
We set a minimum number of RX and TX queues to achieve an
acceptable performance in multi function mode.
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/en_main.c | 12 +++++++-----
drivers/net/mlx4/en_netdev.c | 3 ++-
drivers/net/mlx4/en_tx.c | 2 +-
drivers/net/mlx4/mlx4_en.h | 4 +++-
4 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/net/mlx4/en_main.c b/drivers/net/mlx4/en_main.c
index 507e11f..3883071 100644
--- a/drivers/net/mlx4/en_main.c
+++ b/drivers/net/mlx4/en_main.c
@@ -93,8 +93,6 @@ static int mlx4_en_get_profile(struct mlx4_en_dev *mdev)
params->prof[i].tx_ppp = pfctx;
params->prof[i].tx_ring_size = MLX4_EN_DEF_TX_RING_SIZE;
params->prof[i].rx_ring_size = MLX4_EN_DEF_RX_RING_SIZE;
- params->prof[i].tx_ring_num = MLX4_EN_NUM_TX_RINGS +
- (!!pfcrx) * MLX4_EN_NUM_PPP_RINGS;
}
return 0;
@@ -216,11 +214,15 @@ static void *mlx4_en_add(struct mlx4_dev *dev)
/* If we did not receive an explicit number of Rx rings, default to
* the number of completion vectors populated by the mlx4_core */
mlx4_foreach_port(i, dev, MLX4_PORT_TYPE_ETH) {
+ mdev->profile.prof[i].tx_ring_num = max_t(int,
+ min_t(int, roundup_pow_of_two(dev->caps.num_comp_vectors),
+ NUM_TX_RINGS), MIN_TX_RINGS) +
+ (!!pfcrx) * MLX4_EN_NUM_PPP_RINGS;
mlx4_info(mdev, "Using %d tx rings for port:%d\n",
mdev->profile.prof[i].tx_ring_num, i);
- mdev->profile.prof[i].rx_ring_num = min_t(int,
- roundup_pow_of_two(dev->caps.num_comp_vectors),
- MAX_RX_RINGS);
+ mdev->profile.prof[i].rx_ring_num = max_t(int,
+ min_t(int, roundup_pow_of_two(dev->caps.num_comp_vectors),
+ MAX_RX_RINGS), MIN_RX_RINGS);
mlx4_info(mdev, "Defaulting to %d rx rings for port:%d\n",
mdev->profile.prof[i].rx_ring_num, i);
}
diff --git a/drivers/net/mlx4/en_netdev.c b/drivers/net/mlx4/en_netdev.c
index fd96078..488f967 100644
--- a/drivers/net/mlx4/en_netdev.c
+++ b/drivers/net/mlx4/en_netdev.c
@@ -1024,7 +1024,8 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
*/
dev->netdev_ops = &mlx4_netdev_ops;
dev->watchdog_timeo = MLX4_EN_WATCHDOG_TIMEOUT;
- dev->real_num_tx_queues = MLX4_EN_NUM_TX_RINGS;
+ dev->real_num_tx_queues = priv->tx_ring_num -
+ !!(priv->prof->rx_ppp) * MLX4_EN_NUM_PPP_RINGS;
SET_ETHTOOL_OPS(dev, &mlx4_en_ethtool_ops);
diff --git a/drivers/net/mlx4/en_tx.c b/drivers/net/mlx4/en_tx.c
index 8c72799..a893733 100644
--- a/drivers/net/mlx4/en_tx.c
+++ b/drivers/net/mlx4/en_tx.c
@@ -583,7 +583,7 @@ u16 mlx4_en_select_queue(struct net_device *dev, struct sk_buff *skb)
*/
if (priv->prof->rx_ppp && priv->vlgrp && vlan_tx_tag_present(skb)) {
vlan_tag = vlan_tx_tag_get(skb);
- return MLX4_EN_NUM_TX_RINGS + (vlan_tag >> 13);
+ return priv->tx_ring_num - MLX4_EN_NUM_PPP_RINGS + (vlan_tag >> 13);
}
return skb_tx_hash(dev, skb);
diff --git a/drivers/net/mlx4/mlx4_en.h b/drivers/net/mlx4/mlx4_en.h
index 4376147..6e0c9ed 100644
--- a/drivers/net/mlx4/mlx4_en.h
+++ b/drivers/net/mlx4/mlx4_en.h
@@ -95,6 +95,9 @@
#define MLX4_EN_PAGE_SIZE (1 << MLX4_EN_PAGE_SHIFT)
#define MAX_TX_RINGS 16
#define MAX_RX_RINGS 16
+#define MIN_RX_RINGS 4
+#define NUM_TX_RINGS 8
+#define MIN_TX_RINGS 4
#define TXBB_SIZE 64
#define HEADROOM (2048 / TXBB_SIZE + 1)
#define STAMP_STRIDE 64
@@ -137,7 +140,6 @@ enum {
#define MLX4_EN_MIN_TX_SIZE (4096 / TXBB_SIZE)
#define MLX4_EN_SMALL_PKT_SIZE 64
-#define MLX4_EN_NUM_TX_RINGS 8
#define MLX4_EN_NUM_PPP_RINGS 8
#define MLX4_EN_DEF_TX_RING_SIZE 512
#define MLX4_EN_DEF_RX_RING_SIZE 1024
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* [PATCH 20/25 v2] mlx4: Managing SET_PORT for En port for multi-func
From: Yevgeny Petrilin @ 2009-11-06 3:10 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
MTU is set as max among all active functions.
The base QP number is the base QP number of the port.
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/mlx4.h | 2 +
drivers/net/mlx4/port.c | 56 ++++++++++++++++++++++++++++++++++++++++++---
include/linux/mlx4/cmd.h | 1 +
3 files changed, 55 insertions(+), 4 deletions(-)
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index 19c87e7..2c87cc5 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -225,6 +225,7 @@ struct mlx4_slave_state {
u8 last_cmd;
u8 init_port_mask;
dma_addr_t vhcr_dma;
+ u16 mtu[MLX4_MAX_PORTS + 1];
__be32 ib_cap_mask[MLX4_MAX_PORTS + 1];
struct mlx4_slave_eqe eq[MLX4_MFUNC_MAX_EQES];
u16 eq_pi;
@@ -236,6 +237,7 @@ struct mlx4_slave_state {
struct mlx4_mfunc_master_ctx {
struct mlx4_slave_state *slave_state;
int init_port_ref[MLX4_MAX_PORTS + 1];
+ u16 max_mtu[MLX4_MAX_PORTS + 1];
};
struct mlx4_vhcr {
diff --git a/drivers/net/mlx4/port.c b/drivers/net/mlx4/port.c
index 6e729dd..02b56ba 100644
--- a/drivers/net/mlx4/port.c
+++ b/drivers/net/mlx4/port.c
@@ -36,6 +36,7 @@
#include <linux/mlx4/cmd.h>
#include "mlx4.h"
+#include "en_port.h"
#define MLX4_MAC_VALID (1ull << 63)
#define MLX4_MAC_MASK 0xffffffffffffULL
@@ -351,9 +352,17 @@ int mlx4_SET_PORT_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhc
struct mlx4_cmd_mailbox *outbox)
{
struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_port_info *port_info;
+ struct mlx4_mfunc_master_ctx *master = &priv->mfunc.master;
+ struct mlx4_slave_state *slave_st = &master->slave_state[slave];
+ struct mlx4_set_port_rqp_calc_context *qpn_context;
+ struct mlx4_set_port_general_context *gen_context;
int reset_qkey_viols;
int port;
int is_eth;
+ u32 in_modifier;
+ u32 promisc;
+ u16 mtu, prev_mtu;
int err;
int i;
__be32 agg_cap_mask;
@@ -361,13 +370,52 @@ int mlx4_SET_PORT_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhc
__be32 new_cap_mask;
port = vhcr->in_modifier & 0xff;
+ in_modifier = vhcr->in_modifier >> 8;
is_eth = vhcr->op_modifier;
+ port_info = &priv->port[port];
- /* For Ethernet, we currently support only slave0.
- * TODO: add multi-vf support */
+ /* All slaves can perform SET_PORT operations, just need to verify
+ * we keep the mutual resources unchanged */
if (is_eth) {
- if (slave)
- return -EINVAL;
+ switch (in_modifier) {
+ case MLX4_SET_PORT_RQP_CALC:
+ qpn_context = inbox->buf;
+ qpn_context->base_qpn = cpu_to_be32(port_info->base_qpn);
+ qpn_context->n_mac = 0x7;
+ promisc = be32_to_cpu(qpn_context->promisc) >>
+ SET_PORT_PROMISC_SHIFT;
+ qpn_context->promisc = cpu_to_be32(
+ promisc << SET_PORT_PROMISC_SHIFT |
+ port_info->base_qpn);
+ promisc = be32_to_cpu(qpn_context->mcast) >>
+ SET_PORT_PROMISC_SHIFT;
+ qpn_context->mcast = cpu_to_be32(
+ promisc << SET_PORT_PROMISC_SHIFT |
+ port_info->base_qpn);
+ break;
+ case MLX4_SET_PORT_GENERAL:
+ gen_context = inbox->buf;
+ /* Mtu is configured as the max MTU among all the
+ * the functions on the port. */
+ mtu = be16_to_cpu(gen_context->mtu);
+ mtu = max_t(int, mtu, dev->caps.eth_mtu_cap[port]);
+ prev_mtu = slave_st->mtu[port];
+ slave_st->mtu[port] = mtu;
+ if (mtu > master->max_mtu[port])
+ master->max_mtu[port] = mtu;
+ if (mtu < prev_mtu && prev_mtu == master->max_mtu[port]) {
+ slave_st->mtu[port] = mtu;
+ master->max_mtu[port] = mtu;
+ for (i = 0; i < dev->num_slaves; i++) {
+ master->max_mtu[port] =
+ max(master->max_mtu[port],
+ master->slave_state[i].mtu[port]);
+ }
+ }
+
+ gen_context->mtu = cpu_to_be16(master->max_mtu[port]);
+ break;
+ }
return mlx4_cmd(dev, inbox->dma, vhcr->in_modifier,
vhcr->op_modifier,
MLX4_CMD_SET_PORT,
diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h
index c163d5e..e680ff1 100644
--- a/include/linux/mlx4/cmd.h
+++ b/include/linux/mlx4/cmd.h
@@ -156,6 +156,7 @@ enum {
MLX4_SET_PORT_MAC_TABLE = 0x2,
MLX4_SET_PORT_VLAN_TABLE = 0x3,
MLX4_SET_PORT_PRIO_MAP = 0x4,
+ MLX4_SET_PORT_MODIFIERS
};
struct mlx4_dev;
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* [PATCH 22/25 v2] mlx4_en: Attaching Multicast addresses
From: Yevgeny Petrilin @ 2009-11-06 3:10 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
The device attaches all the multicast addresses to it indirection QP
The multicast steering is now done through the multicast table.
If there is no match, the packet will arrive to the indirection QP
of the first function.
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/en_netdev.c | 33 +++++++++++++++++++++++++++++++--
drivers/net/mlx4/en_port.c | 6 ++++--
drivers/net/mlx4/en_port.h | 11 +++++++++--
drivers/net/mlx4/port.c | 4 ++--
4 files changed, 46 insertions(+), 8 deletions(-)
diff --git a/drivers/net/mlx4/en_netdev.c b/drivers/net/mlx4/en_netdev.c
index 488f967..8406fbb 100644
--- a/drivers/net/mlx4/en_netdev.c
+++ b/drivers/net/mlx4/en_netdev.c
@@ -177,6 +177,7 @@ static void mlx4_en_cache_mclist(struct net_device *dev)
struct dev_mc_list *tmp;
struct dev_mc_list *plist = NULL;
+ mlx4_en_clear_list(dev);
for (mclist = dev->mc_list; mclist; mclist = mclist->next) {
tmp = kmalloc(sizeof(struct dev_mc_list), GFP_ATOMIC);
if (!tmp) {
@@ -213,6 +214,7 @@ static void mlx4_en_do_set_multicast(struct work_struct *work)
struct net_device *dev = priv->dev;
struct dev_mc_list *mclist;
u64 mcast_addr = 0;
+ u8 mc_list[16] = {0};
int err;
mutex_lock(&mdev->state_lock);
@@ -292,6 +294,12 @@ static void mlx4_en_do_set_multicast(struct work_struct *work)
if (err)
en_err(priv, "Failed disabling multicast filter\n");
+ /* Detach our qp from all the multicast addresses */
+ for (mclist = priv->mc_list; mclist; mclist = mclist->next) {
+ memcpy(&mc_list[10], mclist->dmi_addr, ETH_ALEN);
+ mlx4_multicast_detach(mdev->dev, &priv->rss_map.indir_qp,
+ mc_list, MLX4_PROT_ETH);
+ }
/* Flush mcast filter and init it with broadcast address */
mlx4_SET_MCAST_FLTR(mdev->dev, priv->port, ETH_BCAST,
1, MLX4_MCAST_CONFIG);
@@ -302,6 +310,9 @@ static void mlx4_en_do_set_multicast(struct work_struct *work)
mlx4_en_cache_mclist(dev);
netif_tx_unlock_bh(dev);
for (mclist = priv->mc_list; mclist; mclist = mclist->next) {
+ memcpy(&mc_list[10], mclist->dmi_addr, ETH_ALEN);
+ mlx4_multicast_attach(mdev->dev, &priv->rss_map.indir_qp,
+ mc_list, 0, MLX4_PROT_ETH);
mcast_addr = mlx4_en_mac_to_u64(mclist->dmi_addr);
mlx4_SET_MCAST_FLTR(mdev->dev, priv->port,
mcast_addr, 0, MLX4_MCAST_CONFIG);
@@ -310,8 +321,6 @@ static void mlx4_en_do_set_multicast(struct work_struct *work)
0, MLX4_MCAST_ENABLE);
if (err)
en_err(priv, "Failed enabling multicast filter\n");
-
- mlx4_en_clear_list(dev);
}
out:
mutex_unlock(&mdev->state_lock);
@@ -557,6 +566,7 @@ int mlx4_en_start_port(struct net_device *dev)
int err = 0;
int i;
int j;
+ u8 mc_list[16] = {0};
if (priv->port_up) {
en_dbg(DRV, priv, "start port called while port already up\n");
@@ -669,6 +679,12 @@ int mlx4_en_start_port(struct net_device *dev)
goto tx_err;
}
+ /* Attach rx QP to bradcast address */
+ memset(&mc_list[10], 0xff, ETH_ALEN);
+ if (mlx4_multicast_attach(mdev->dev, &priv->rss_map.indir_qp, mc_list,
+ 0, MLX4_PROT_ETH))
+ mlx4_warn(mdev, "Failed Attaching Broadcast\n");
+
/* Schedule multicast task to populate multicast list */
queue_work(mdev->workqueue, &priv->mcast_task);
@@ -699,7 +715,9 @@ void mlx4_en_stop_port(struct net_device *dev)
{
struct mlx4_en_priv *priv = netdev_priv(dev);
struct mlx4_en_dev *mdev = priv->mdev;
+ struct dev_mc_list *mclist;
int i;
+ u8 mc_list[16] = {0};
if (!priv->port_up) {
en_dbg(DRV, priv, "stop port called while port already down\n");
@@ -715,6 +733,17 @@ void mlx4_en_stop_port(struct net_device *dev)
priv->port_up = false;
mlx4_CLOSE_PORT(mdev->dev, priv->port);
+ /* Detach All multicasts */
+ memset(&mc_list[10], 0xff, ETH_ALEN);
+ mlx4_multicast_detach(mdev->dev, &priv->rss_map.indir_qp, mc_list,
+ MLX4_PROT_ETH);
+ for (mclist = priv->mc_list; mclist; mclist = mclist->next) {
+ memcpy(&mc_list[10], mclist->dmi_addr, ETH_ALEN);
+ mlx4_multicast_detach(mdev->dev, &priv->rss_map.indir_qp,
+ mc_list, MLX4_PROT_ETH);
+ }
+ mlx4_en_clear_list(dev);
+
/* Unregister Mac address for the port */
mlx4_unregister_mac(mdev->dev, priv->port, priv->base_qpn);
diff --git a/drivers/net/mlx4/en_port.c b/drivers/net/mlx4/en_port.c
index c099cb4..c7aa86e 100644
--- a/drivers/net/mlx4/en_port.c
+++ b/drivers/net/mlx4/en_port.c
@@ -128,8 +128,10 @@ int mlx4_SET_PORT_qpn_calc(struct mlx4_dev *dev, u8 port, u32 base_qpn,
context->base_qpn = cpu_to_be32(base_qpn);
context->n_mac = 0x7;
- context->promisc = cpu_to_be32(promisc << SET_PORT_PROMISC_SHIFT | base_qpn);
- context->mcast = cpu_to_be32(1 << SET_PORT_PROMISC_SHIFT | base_qpn);
+ context->promisc = cpu_to_be32(promisc << SET_PORT_PROMISC_SHIFT |
+ base_qpn);
+ context->mcast = cpu_to_be32(MCAST_DIRECT << SET_PORT_MC_PROMISC_SHIFT |
+ base_qpn);
context->intra_no_vlan = 0;
context->no_vlan = MLX4_NO_VLAN_IDX;
context->intra_vlan_miss = 0;
diff --git a/drivers/net/mlx4/en_port.h b/drivers/net/mlx4/en_port.h
index 62c9135..1e65749 100644
--- a/drivers/net/mlx4/en_port.h
+++ b/drivers/net/mlx4/en_port.h
@@ -35,8 +35,15 @@
#define _MLX4_EN_PORT_H_
-#define SET_PORT_GEN_ALL_VALID 0x7
-#define SET_PORT_PROMISC_SHIFT 31
+#define SET_PORT_GEN_ALL_VALID 0x7
+#define SET_PORT_PROMISC_SHIFT 31
+#define SET_PORT_MC_PROMISC_SHIFT 30
+
+enum {
+ MCAST_DIRECT_ONLY = 0,
+ MCAST_DIRECT = 1,
+ MCAST_DEFAULT = 2
+};
struct mlx4_set_port_general_context {
diff --git a/drivers/net/mlx4/port.c b/drivers/net/mlx4/port.c
index 02b56ba..7317d0f 100644
--- a/drivers/net/mlx4/port.c
+++ b/drivers/net/mlx4/port.c
@@ -388,9 +388,9 @@ int mlx4_SET_PORT_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhc
promisc << SET_PORT_PROMISC_SHIFT |
port_info->base_qpn);
promisc = be32_to_cpu(qpn_context->mcast) >>
- SET_PORT_PROMISC_SHIFT;
+ SET_PORT_MC_PROMISC_SHIFT;
qpn_context->mcast = cpu_to_be32(
- promisc << SET_PORT_PROMISC_SHIFT |
+ promisc << SET_PORT_MC_PROMISC_SHIFT |
port_info->base_qpn);
break;
case MLX4_SET_PORT_GENERAL:
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* [PATCH 24/25 v2] mlx4: VLAN filter management by master function.
From: Yevgeny Petrilin @ 2009-11-06 3:10 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
The VLAN filter is a bitmap of 4096 bits.
The mster function sets the VLAN filter as a bitwise OR of the filters
of each one of the slaves.
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/cmd.c | 2 +-
drivers/net/mlx4/en_port.c | 35 ---------------------
drivers/net/mlx4/en_port.h | 5 ---
drivers/net/mlx4/mlx4.h | 9 +++++
drivers/net/mlx4/port.c | 72 ++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 82 insertions(+), 41 deletions(-)
diff --git a/drivers/net/mlx4/cmd.c b/drivers/net/mlx4/cmd.c
index f3d3a7b..4c472f3 100644
--- a/drivers/net/mlx4/cmd.c
+++ b/drivers/net/mlx4/cmd.c
@@ -939,7 +939,7 @@ static struct mlx4_cmd_info {
.has_outbox = false,
.out_is_imm = false,
.verify = NULL,
- .wrapper = NULL /* need wrapper*/
+ .wrapper = mlx4_SET_VLAN_FLTR_wrapper
},
{
.opcode = MLX4_CMD_SET_MCAST_FLTR,
diff --git a/drivers/net/mlx4/en_port.c b/drivers/net/mlx4/en_port.c
index 6dc07c9..a0e8a97 100644
--- a/drivers/net/mlx4/en_port.c
+++ b/drivers/net/mlx4/en_port.c
@@ -41,41 +41,6 @@
#include "mlx4_en.h"
-int mlx4_SET_VLAN_FLTR(struct mlx4_dev *dev, u8 port, struct vlan_group *grp)
-{
- struct mlx4_cmd_mailbox *mailbox;
- struct mlx4_set_vlan_fltr_mbox *filter;
- int i;
- int j;
- int index = 0;
- u32 entry;
- int err = 0;
-
- mailbox = mlx4_alloc_cmd_mailbox(dev);
- if (IS_ERR(mailbox))
- return PTR_ERR(mailbox);
-
- filter = mailbox->buf;
- if (grp) {
- memset(filter, 0, sizeof *filter);
- for (i = VLAN_FLTR_SIZE - 1; i >= 0; i--) {
- entry = 0;
- for (j = 0; j < 32; j++)
- if (vlan_group_get_device(grp, index++))
- entry |= 1 << j;
- filter->entry[i] = cpu_to_be32(entry);
- }
- } else {
- /* When no vlans are configured we block all vlans */
- memset(filter, 0, sizeof(*filter));
- }
- err = mlx4_cmd(dev, mailbox->dma, port, 0, MLX4_CMD_SET_VLAN_FLTR,
- MLX4_CMD_TIME_CLASS_B);
- mlx4_free_cmd_mailbox(dev, mailbox);
- return err;
-}
-
-
int mlx4_SET_PORT_general(struct mlx4_dev *dev, u8 port, int mtu,
u8 pptx, u8 pfctx, u8 pprx, u8 pfcrx)
{
diff --git a/drivers/net/mlx4/en_port.h b/drivers/net/mlx4/en_port.h
index 1e65749..8ddffaf 100644
--- a/drivers/net/mlx4/en_port.h
+++ b/drivers/net/mlx4/en_port.h
@@ -78,11 +78,6 @@ struct mlx4_set_port_rqp_calc_context {
__be32 mcast;
};
-#define VLAN_FLTR_SIZE 128
-struct mlx4_set_vlan_fltr_mbox {
- __be32 entry[VLAN_FLTR_SIZE];
-};
-
enum {
MLX4_MCAST_CONFIG = 0,
diff --git a/drivers/net/mlx4/mlx4.h b/drivers/net/mlx4/mlx4.h
index 2476fb5..ec01ab3 100644
--- a/drivers/net/mlx4/mlx4.h
+++ b/drivers/net/mlx4/mlx4.h
@@ -225,6 +225,11 @@ struct mlx4_mcast_entry {
u64 addr;
};
+#define VLAN_FLTR_SIZE 128
+struct mlx4_vlan_fltr {
+ __be32 entry[VLAN_FLTR_SIZE];
+};
+
struct mlx4_slave_state {
u8 comm_toggle;
u8 last_cmd;
@@ -234,6 +239,7 @@ struct mlx4_slave_state {
__be32 ib_cap_mask[MLX4_MAX_PORTS + 1];
struct mlx4_slave_eqe eq[MLX4_MFUNC_MAX_EQES];
struct list_head mcast_filters[MLX4_MAX_PORTS + 1];
+ struct mlx4_vlan_fltr vlan_filter[MLX4_MAX_PORTS + 1];
u16 eq_pi;
u16 eq_ci;
int sqp_start;
@@ -564,5 +570,8 @@ int mlx4_MCAST_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
int mlx4_SET_MCAST_FLTR_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
struct mlx4_cmd_mailbox *inbox,
struct mlx4_cmd_mailbox *outbox);
+int mlx4_SET_VLAN_FLTR_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
+ struct mlx4_cmd_mailbox *inbox,
+ struct mlx4_cmd_mailbox *outbox);
#endif /* MLX4_H */
diff --git a/drivers/net/mlx4/port.c b/drivers/net/mlx4/port.c
index 1bc527c..1a790ef 100644
--- a/drivers/net/mlx4/port.c
+++ b/drivers/net/mlx4/port.c
@@ -32,6 +32,7 @@
#include <linux/errno.h>
#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
#include <linux/mlx4/cmd.h>
@@ -616,3 +617,74 @@ int mlx4_SET_MCAST_FLTR(struct mlx4_dev *dev, u8 port,
MLX4_CMD_SET_MCAST_FLTR, MLX4_CMD_TIME_CLASS_B);
}
EXPORT_SYMBOL(mlx4_SET_MCAST_FLTR);
+
+
+int mlx4_SET_VLAN_FLTR_wrapper(struct mlx4_dev *dev, int slave, struct mlx4_vhcr *vhcr,
+ struct mlx4_cmd_mailbox *inbox,
+ struct mlx4_cmd_mailbox *outbox)
+{
+ struct mlx4_cmd_mailbox *mailbox;
+ struct mlx4_priv *priv = mlx4_priv(dev);
+ struct mlx4_vlan_fltr *filter;
+ struct mlx4_slave_state *s_state = &priv->mfunc.master.slave_state[slave];
+ int port = vhcr->in_modifier;
+ int i, j, err;
+
+ mailbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(mailbox))
+ return PTR_ERR(mailbox);
+
+ /* Update slave's Vlan filter */
+ memcpy(s_state->vlan_filter[port].entry, inbox->buf,
+ sizeof(struct mlx4_vlan_fltr));
+
+ /* We configure the Vlan filter to allow the vlans of
+ * all slaves */
+ filter = mailbox->buf;
+ memset(filter, 0, sizeof(*filter));
+ for (i = VLAN_FLTR_SIZE - 1; i >= 0; i--) {
+ for (j = 0; j < dev->num_slaves; j++) {
+ s_state = &priv->mfunc.master.slave_state[j];
+ filter->entry[i] |= s_state->vlan_filter[port].entry[i];
+ }
+ }
+ err = mlx4_cmd(dev, mailbox->dma, port, 0, MLX4_CMD_SET_VLAN_FLTR,
+ MLX4_CMD_TIME_CLASS_B);
+ mlx4_free_cmd_mailbox(dev, mailbox);
+ return err;
+}
+
+int mlx4_SET_VLAN_FLTR(struct mlx4_dev *dev, u8 port, struct vlan_group *grp)
+{
+ struct mlx4_cmd_mailbox *mailbox;
+ struct mlx4_vlan_fltr *filter;
+ int i;
+ int j;
+ int index = 0;
+ u32 entry;
+ int err = 0;
+
+ mailbox = mlx4_alloc_cmd_mailbox(dev);
+ if (IS_ERR(mailbox))
+ return PTR_ERR(mailbox);
+
+ filter = mailbox->buf;
+ if (grp) {
+ memset(filter, 0, sizeof *filter);
+ for (i = VLAN_FLTR_SIZE - 1; i >= 0; i--) {
+ entry = 0;
+ for (j = 0; j < 32; j++)
+ if (vlan_group_get_device(grp, index++))
+ entry |= 1 << j;
+ filter->entry[i] = cpu_to_be32(entry);
+ }
+ } else {
+ /* When no vlans are configured we block all vlans */
+ memset(filter, 0, sizeof(*filter));
+ }
+ err = mlx4_cmd(dev, mailbox->dma, port, 0, MLX4_CMD_SET_VLAN_FLTR,
+ MLX4_CMD_TIME_CLASS_B);
+ mlx4_free_cmd_mailbox(dev, mailbox);
+ return err;
+}
+EXPORT_SYMBOL(mlx4_SET_VLAN_FLTR);
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* [PATCH 25/25 v2] mlx4_en: Loopback enable
From: Yevgeny Petrilin @ 2009-11-06 3:10 UTC (permalink / raw)
To: rdreier; +Cc: linux-rdma, netdev, liranl, tziporet, yevgenyp
All packets are subject to loopback.
To enable loopback we write the dest mac to the wqe.
Signed-off-by: Yevgeny Petrilin <yevgenyp@mellanox.co.il>
---
drivers/net/mlx4/en_netdev.c | 2 +-
drivers/net/mlx4/en_tx.c | 14 ++++++++++++++
drivers/net/mlx4/mlx4_en.h | 1 +
3 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/drivers/net/mlx4/en_netdev.c b/drivers/net/mlx4/en_netdev.c
index 8406fbb..947fb2f 100644
--- a/drivers/net/mlx4/en_netdev.c
+++ b/drivers/net/mlx4/en_netdev.c
@@ -108,7 +108,7 @@ static void mlx4_en_vlan_rx_kill_vid(struct net_device *dev, unsigned short vid)
mutex_unlock(&mdev->state_lock);
}
-static u64 mlx4_en_mac_to_u64(u8 *addr)
+u64 mlx4_en_mac_to_u64(u8 *addr)
{
u64 mac = 0;
int i;
diff --git a/drivers/net/mlx4/en_tx.c b/drivers/net/mlx4/en_tx.c
index a893733..cfbe361 100644
--- a/drivers/net/mlx4/en_tx.c
+++ b/drivers/net/mlx4/en_tx.c
@@ -599,6 +599,7 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
struct mlx4_wqe_data_seg *data;
struct skb_frag_struct *frag;
struct mlx4_en_tx_info *tx_info;
+ struct ethhdr *ethh;
int tx_ind = 0;
int nr_txbb;
int desc_size;
@@ -610,6 +611,8 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
int i;
int lso_header_size;
void *fragptr;
+ u64 mac;
+ u32 mac_l, mac_h;
real_size = get_real_size(skb, dev, &lso_header_size);
if (unlikely(!real_size))
@@ -675,6 +678,17 @@ netdev_tx_t mlx4_en_xmit(struct sk_buff *skb, struct net_device *dev)
priv->port_stats.tx_chksum_offload++;
}
+ /* Copy dst mac address to wqe */
+ skb_reset_mac_header(skb);
+ ethh = eth_hdr(skb);
+ if (ethh && ethh->h_dest) {
+ mac = mlx4_en_mac_to_u64(ethh->h_dest);
+ mac_h = (u32) ((mac & 0xffff00000000) >> 16);
+ mac_l = (u32) (mac & 0xffffffff);
+ tx_desc->ctrl.srcrb_flags |= cpu_to_be32(mac_h);
+ tx_desc->ctrl.imm = cpu_to_be32(mac_l);
+ }
+
/* Handle LSO (TSO) packets */
if (lso_header_size) {
/* Mark opcode as LSO */
diff --git a/drivers/net/mlx4/mlx4_en.h b/drivers/net/mlx4/mlx4_en.h
index b4731d5..02d07e9 100644
--- a/drivers/net/mlx4/mlx4_en.h
+++ b/drivers/net/mlx4/mlx4_en.h
@@ -564,6 +564,7 @@ int mlx4_SET_PORT_qpn_calc(struct mlx4_dev *dev, u8 port, u32 base_qpn,
int mlx4_en_DUMP_ETH_STATS(struct mlx4_en_dev *mdev, u8 port, u8 reset);
+u64 mlx4_en_mac_to_u64(u8 *addr);
/*
* Globals
*/
--
1.5.3.7
From: Liran Liss <liranl@mellanox.co.il>
^ permalink raw reply related
* Re: dccp-test-tree [PATCH 1/1]: Count lost data packets in a burst loss
From: Ivo Calado @ 2009-11-06 0:16 UTC (permalink / raw)
To: Gerrit Renker, dccp, Ivo Calado, netdev
In-Reply-To: <20091019051642.GA3366@gerrit.erg.abdn.ac.uk>
On Mon, Oct 19, 2009 at 02:16, Gerrit Renker <gerrit@erg.abdn.ac.uk> wrote:
> dccp: Generalise data-loss condition
>
> This patch is thanks to Ivo Calado who had integrated this function into one
> of the TFRC-SP patches.
>
> It generalises the task of determining data loss from RFC 43430, 7.7.1.
>
> Let S_A, S_B be sequence numbers such that S_B is "after" S_A, and let
> N_B be the NDP count of packet S_B. Then, using module-2^48 arithmetic,
> D = S_B - S_A - 1 is an upper bound of the number of lost data packets,
> D - N_B is an approximation of the number of lost data packets
> (there are cases where this is not exact).
>
> The patch implements this as
> dccp_loss_count(S_A, S_B, N_B) := max(S_B - S_A - 1 - N_B, 0)
>
> Signed-off-by: Gerrit Renker <gerrit@erg.abdn.ac.uk>
> ---
> net/dccp/dccp.h | 21 +++++++++++++++------
> 1 file changed, 15 insertions(+), 6 deletions(-)
>
> --- a/net/dccp/dccp.h
> +++ b/net/dccp/dccp.h
> @@ -153,18 +153,27 @@ static inline u64 max48(const u64 seq1,
> }
>
> /**
> - * dccp_loss_free - Evaluates condition for data loss from RFC 4340, 7.7.1
> - * @s1: start sequence number
> - * @s2: end sequence number
> + * dccp_loss_count - Approximate the number of lost data packets in a burst loss
> + * @s1: last known sequence number before the loss ('hole')
> + * @s2: first sequence number seen after the 'hole'
> * @ndp: NDP count on packet with sequence number @s2
> - * Returns true if the sequence range s1...s2 has no data loss.
> */
> -static inline bool dccp_loss_free(const u64 s1, const u64 s2, const u64 ndp)
> +static inline u64 dccp_loss_count(const u64 s1, const u64 s2, const u64 ndp)
> {
> s64 delta = dccp_delta_seqno(s1, s2);
>
> WARN_ON(delta < 0);
> - return (u64)delta <= ndp + 1;
> + delta -= ndp + 1;
> +
> + return delta > 0 ? delta : 0;
> +}
> +
> +/**
> + * dccp_loss_free - Evaluate condition for data loss from RFC 4340, 7.7.1
> + */
> +static inline bool dccp_loss_free(const u64 s1, const u64 s2, const u64 ndp)
> +{
> + return dccp_loss_count(s1, s2, ndp) == 0;
> }
>
> enum {
>
Agree
--
Ivo Augusto Andrade Rocha Calado
MSc. Candidate
Embedded Systems and Pervasive Computing Lab - http://embedded.ufcg.edu.br
Systems and Computing Department - http://www.dsc.ufcg.edu.br
Electrical Engineering and Informatics Center - http://www.ceei.ufcg.edu.br
Federal University of Campina Grande - http://www.ufcg.edu.br
PGP: 0x03422935
Putt's Law:
Technology is dominated by two types of people:
Those who understand what they do not manage.
Those who manage what they do not understand.
^ permalink raw reply
* Re: [PATCH 1/4] Adds random ect generation to tfrc-sp sender side
From: Ivo Calado @ 2009-11-06 0:52 UTC (permalink / raw)
To: Gerrit Renker, dccp, netdev, Ivo Calado
In-Reply-To: <425e6efa0911051553x31fa44b1p207ecb5a378b89c4@mail.gmail.com>
On Mon, Oct 26, 2009 at 4:55 AM, Gerrit Renker <gerrit@erg.abdn.ac.uk> wrote:
> | > That is, at the moment both the sender and receiver side of the ECN Nonce
> | > sum verification are placeholders which currently have no effect.
> | >
> |
> | Okay, then the implementation would be useless now.
> I was not suggesting to throw away the patches, we can keep them for
> later use.
>
> They are a good starting basis once it makes sense to work with ECN.
> Or how can we test ECN if we are not sure that the other parts work
> as they are supposed to?
>
Okay, we'll keep the patches.
> | > 3) Starting an implementation throws up further questions that need to
> | > be addressed, both the basis and the extension need to be verified.
> | >
> | > I would like to suggest to implement the basis, that is CCID-4 with ECN
> | > (using plain ECT(0)), test with that until it works satisfactorily, and
> | > then continue adding measures such as the ECN Nonce verification.
> | >
> |
> | Okay. But, when would be good to at least include random ECT
> | generation? When DCCP ECN code will get fixed? Is there any work on
> | this?
> |
> I asked this on netdev earlier, there was not much enthusiasm.
>
> The issue is that ECN belongs both to the network and the transport layer.
> This network layer is in inet_ecn.h, outside of DCCP.
>
> I believe that the changes would not be too hard to do, by changing the
> macros. But it requires working with the people on netdev, in particular
> to ensure it does not break something in the TCP/SCTP subsystems (both
> also use ECN, and then there are also raw sockets).
>
> I also have an interest in resolving this, due to the ugliness at the
> moment for enabling ECN on IPv6.
>
> RFC 2884, written by one of the Linux ECN developers, describes early
> IPv4 ECN evaluation. It seems that initially it was planned to only
> support ECN over Ipv4, parts of the code may be still from that time.
>
> We can pursue this in parallel to the other issues. Ideally this would be
> resolved at the time the other parts of CCID-4 are ready for testing.
>
Ok.
> | > In summary, I would like to suggest to remove the ECN verification for
> | > the moment and focus on the "basic" issues first.
> | >
> | > Would you be ok with that?
> | >
> |
> | Yes, we'll keep the ECN verification code here at our git until the
> | scenario is ready.
> |
> I was going to suggest to put them onto a webpage, such as yours, or on
> www.erg.abdn.ac.uk/users/gerrit/dccp there is also still some space.
>
> Can you please elaborate how to keep your git tree and the one on
> eden-feed synchronized? At the moment I have not made any changes other
> than the ones I emailed you about. Is there a way of keeping both trees
> in synch without running into versioning difficulties?
>
> (The simplest way I can think of is to keep the patches in a separate
> set, or to spawn a subtree which contains the ECN patches on top of the
> CCID-4 tree.)
>
Ok, we'll put the patches at our webpage. And about the git, I can't
think a better way too.
>
> | > (Also for later) I wonder how to do the sums, with RFC 3168
> | > ECT(0) = 0x2 => !0x2 = 0
> | > ECT(1) = 0x1 => !0x1 = 0
> | >
> |
> | I don't understand. Can you try to explain it? Or cite RFC section
> | that address it?
>
> The values are from figure 1, page 7, the expressions evaluate both as 0:
>
> void main(void)
> {
> printf("!0x2 = %d, !0x1 = %d\n", !0x2, !0x1);
> }
Thanks.
--
Ivo Augusto Andrade Rocha Calado
MSc. Candidate
Embedded Systems and Pervasive Computing Lab - http://embedded.ufcg.edu.br
Systems and Computing Department - http://www.dsc.ufcg.edu.br
Electrical Engineering and Informatics Center - http://www.ceei.ufcg.edu.br
Federal University of Campina Grande - http://www.ufcg.edu.br
PGP: 0x03422935
Putt's Law:
Technology is dominated by two types of people:
Those who understand what they do not manage.
Those who manage what they do not understand.
^ permalink raw reply
* Using netconsole and getting double prints
From: Luis R. Rodriguez @ 2009-11-06 0:54 UTC (permalink / raw)
To: linux-kernel; +Cc: netdev
I'm getting double prints when using netconsole. This used to happen
to me and then I just enable debugging log level manually (dmesg -n 8)
but now no matter what I try I always get double prints.
For example:
[ 23.425448] console [netcon0] enabled
[ 23.425567] netconsole: network logging started
[ 23.425448] console [netcon0] enabled
[ 23.425567] netconsole: network logging started
[ 32.856073] eth0: no IPv6 routers present
[ 32.856073] eth0: no IPv6 routers present
[ 66.307342] kmemleak: 6 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)
[ 66.307342] kmemleak: 6 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)
Instead of dmesg -8 I'm now using ignore_loglevel as a kernel
parameter but I still get double prints. This also happens if I use
"debug" as a kernel parameter instead.
The netconsole is set up on the dev box as follows through an /etc/rc.local
dhclient eth0
IP=192.168.2
# sudo dmesg -n 8
modprobe netconsole netconsole=@/eth0,6666@${IP}/
exit 0
I capture my data with netcat script (I call it netlog):
#!/bin/bash
echo "You should now run in another window: tail -f $HOME/log"
while true; do
/bin/nc -u -l -p 6666 >> $HOME/log
done
This also just doesn't work if I run the nc manually without a redirect:
while true; do /bin/nc -u -l -p 6666; done
Any ideas?
Luis
^ permalink raw reply
* [RFC] move dma_head/dma_maps out of skb_shared_info and into sk_buff
From: Alexander Duyck @ 2009-11-06 0:54 UTC (permalink / raw)
To: netdev@vger.kernel.org, davem@davemloft.net
During testing we found issues with the use of skb_dma_map/unmap on
systems that had iommu enabled and were configured to use a bridge. The
issue is that if two ports are members of the same bridge, and a
broadcast packet is sent out on the bridge skb_clone will be used to
send a copy to all ports, but the clones run into issues because the
dma mappings for the cloned skbs all share the shared_info structure
where the dma mappings are stored.
To resolve that this patch moves those dma mappings out of the
shared_info structure and into the sk_buff itself. This allows cloned
skbs to be mapped separately without causing dma unmapping errors.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
diff --git a/drivers/net/bnx2.c b/drivers/net/bnx2.c
index 539d23b..fb80e6b 100644
--- a/drivers/net/bnx2.c
+++ b/drivers/net/bnx2.c
@@ -5688,7 +5688,7 @@ bnx2_run_loopback(struct bnx2 *bp, int loopback_mode)
dev_kfree_skb(skb);
return -EIO;
}
- map = skb_shinfo(skb)->dma_head;
+ map = skb->dma_head;
REG_WR(bp, BNX2_HC_COMMAND,
bp->hc_cmd | BNX2_HC_COMMAND_COAL_NOW_WO_INT);
@@ -6373,7 +6373,7 @@ bnx2_start_xmit(struct sk_buff *skb, struct net_device *dev)
}
sp = skb_shinfo(skb);
- mapping = sp->dma_head;
+ mapping = skb->dma_head;
tx_buf = &txr->tx_buf_ring[ring_prod];
tx_buf->skb = skb;
@@ -6397,7 +6397,7 @@ bnx2_start_xmit(struct sk_buff *skb, struct net_device *dev)
txbd = &txr->tx_desc_ring[ring_prod];
len = frag->size;
- mapping = sp->dma_maps[i];
+ mapping = skb->dma_maps[i];
txbd->tx_bd_haddr_hi = (u64) mapping >> 32;
txbd->tx_bd_haddr_lo = (u64) mapping & 0xffffffff;
diff --git a/drivers/net/e1000/e1000_main.c b/drivers/net/e1000/e1000_main.c
index c938114..86fb526 100644
--- a/drivers/net/e1000/e1000_main.c
+++ b/drivers/net/e1000/e1000_main.c
@@ -2696,7 +2696,7 @@ static int e1000_tx_map(struct e1000_adapter *adapter,
return 0;
}
- map = skb_shinfo(skb)->dma_maps;
+ map = skb->dma_maps;
offset = 0;
while (len) {
@@ -2735,7 +2735,7 @@ static int e1000_tx_map(struct e1000_adapter *adapter,
buffer_info->length = size;
/* set time_stamp *before* dma to help avoid a possible race */
buffer_info->time_stamp = jiffies;
- buffer_info->dma = skb_shinfo(skb)->dma_head + offset;
+ buffer_info->dma = skb->dma_head + offset;
buffer_info->next_to_watch = i;
len -= size;
diff --git a/drivers/net/e1000e/netdev.c b/drivers/net/e1000e/netdev.c
index 3769248..90266c2 100644
--- a/drivers/net/e1000e/netdev.c
+++ b/drivers/net/e1000e/netdev.c
@@ -3884,7 +3884,7 @@ static int e1000_tx_map(struct e1000_adapter *adapter,
return 0;
}
- map = skb_shinfo(skb)->dma_maps;
+ map = skb->dma_maps;
offset = 0;
while (len) {
@@ -3894,7 +3894,7 @@ static int e1000_tx_map(struct e1000_adapter *adapter,
buffer_info->length = size;
buffer_info->time_stamp = jiffies;
buffer_info->next_to_watch = i;
- buffer_info->dma = skb_shinfo(skb)->dma_head + offset;
+ buffer_info->dma = skb->dma_head + offset;
count++;
len -= size;
diff --git a/drivers/net/igb/igb_main.c b/drivers/net/igb/igb_main.c
index f01f5af..cf28f11 100644
--- a/drivers/net/igb/igb_main.c
+++ b/drivers/net/igb/igb_main.c
@@ -3570,7 +3570,7 @@ static inline int igb_tx_map_adv(struct igb_ring *tx_ring, struct sk_buff *skb,
return 0;
}
- map = skb_shinfo(skb)->dma_maps;
+ map = skb->dma_maps;
buffer_info = &tx_ring->buffer_info[i];
BUG_ON(len >= IGB_MAX_DATA_PER_TXD);
@@ -3578,7 +3578,7 @@ static inline int igb_tx_map_adv(struct igb_ring *tx_ring, struct sk_buff *skb,
/* set time_stamp *before* dma to help avoid a possible race */
buffer_info->time_stamp = jiffies;
buffer_info->next_to_watch = i;
- buffer_info->dma = skb_shinfo(skb)->dma_head;
+ buffer_info->dma = skb->dma_head;
for (f = 0; f < skb_shinfo(skb)->nr_frags; f++) {
struct skb_frag_struct *frag;
diff --git a/drivers/net/igbvf/netdev.c b/drivers/net/igbvf/netdev.c
index fad7f34..d7bb712 100644
--- a/drivers/net/igbvf/netdev.c
+++ b/drivers/net/igbvf/netdev.c
@@ -2100,7 +2100,7 @@ static inline int igbvf_tx_map_adv(struct igbvf_adapter *adapter,
return 0;
}
- map = skb_shinfo(skb)->dma_maps;
+ map = skb->dma_maps;
buffer_info = &tx_ring->buffer_info[i];
BUG_ON(len >= IGBVF_MAX_DATA_PER_TXD);
@@ -2108,7 +2108,7 @@ static inline int igbvf_tx_map_adv(struct igbvf_adapter *adapter,
/* set time_stamp *before* dma to help avoid a possible race */
buffer_info->time_stamp = jiffies;
buffer_info->next_to_watch = i;
- buffer_info->dma = skb_shinfo(skb)->dma_head;
+ buffer_info->dma = skb->dma_head;
for (f = 0; f < skb_shinfo(skb)->nr_frags; f++) {
struct skb_frag_struct *frag;
diff --git a/drivers/net/ixgb/ixgb_main.c b/drivers/net/ixgb/ixgb_main.c
index 1bd0ca1..bc401cf 100644
--- a/drivers/net/ixgb/ixgb_main.c
+++ b/drivers/net/ixgb/ixgb_main.c
@@ -1288,7 +1288,7 @@ ixgb_tx_map(struct ixgb_adapter *adapter, struct sk_buff *skb,
return 0;
}
- map = skb_shinfo(skb)->dma_maps;
+ map = skb->dma_maps;
while (len) {
buffer_info = &tx_ring->buffer_info[i];
@@ -1301,7 +1301,7 @@ ixgb_tx_map(struct ixgb_adapter *adapter, struct sk_buff *skb,
buffer_info->length = size;
WARN_ON(buffer_info->dma != 0);
buffer_info->time_stamp = jiffies;
- buffer_info->dma = skb_shinfo(skb)->dma_head + offset;
+ buffer_info->dma = skb->dma_head + offset;
pci_map_single(adapter->pdev,
skb->data + offset,
size,
diff --git a/drivers/net/ixgbe/ixgbe_main.c b/drivers/net/ixgbe/ixgbe_main.c
index 45c5faf..d857030 100644
--- a/drivers/net/ixgbe/ixgbe_main.c
+++ b/drivers/net/ixgbe/ixgbe_main.c
@@ -4953,7 +4953,7 @@ static int ixgbe_tx_map(struct ixgbe_adapter *adapter,
return 0;
}
- map = skb_shinfo(skb)->dma_maps;
+ map = skb->dma_maps;
if (tx_flags & IXGBE_TX_FLAGS_FCOE)
/* excluding fcoe_crc_eof for FCoE */
@@ -4965,7 +4965,7 @@ static int ixgbe_tx_map(struct ixgbe_adapter *adapter,
size = min(len, (uint)IXGBE_MAX_DATA_PER_TXD);
tx_buffer_info->length = size;
- tx_buffer_info->dma = skb_shinfo(skb)->dma_head + offset;
+ tx_buffer_info->dma = skb->dma_head + offset;
tx_buffer_info->time_stamp = jiffies;
tx_buffer_info->next_to_watch = i;
diff --git a/drivers/net/tg3.c b/drivers/net/tg3.c
index 47a4f09..559898b 100644
--- a/drivers/net/tg3.c
+++ b/drivers/net/tg3.c
@@ -5145,7 +5145,7 @@ static int tigon3_dma_hwbug_workaround(struct tg3 *tp, struct sk_buff *skb,
/* New SKB is guaranteed to be linear. */
entry = *start;
ret = skb_dma_map(&tp->pdev->dev, new_skb, DMA_TO_DEVICE);
- new_addr = skb_shinfo(new_skb)->dma_head;
+ new_addr = new_skb->dma_head;
/* Make sure new skb does not cross any 4G boundaries.
* Drop the packet if it does.
@@ -5294,7 +5294,7 @@ static netdev_tx_t tg3_start_xmit(struct sk_buff *skb,
sp = skb_shinfo(skb);
- mapping = sp->dma_head;
+ mapping = skb->dma_head;
tnapi->tx_buffers[entry].skb = skb;
@@ -5318,7 +5318,7 @@ static netdev_tx_t tg3_start_xmit(struct sk_buff *skb,
skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
len = frag->size;
- mapping = sp->dma_maps[i];
+ mapping = skb->dma_maps[i];
tnapi->tx_buffers[entry].skb = NULL;
tg3_set_txd(tnapi, entry, mapping, len,
@@ -5482,7 +5482,7 @@ static netdev_tx_t tg3_start_xmit_dma_bug(struct sk_buff *skb,
sp = skb_shinfo(skb);
- mapping = sp->dma_head;
+ mapping = skb->dma_head;
tnapi->tx_buffers[entry].skb = skb;
@@ -5516,7 +5516,7 @@ static netdev_tx_t tg3_start_xmit_dma_bug(struct sk_buff *skb,
skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
len = frag->size;
- mapping = sp->dma_maps[i];
+ mapping = skb->dma_maps[i];
tnapi->tx_buffers[entry].skb = NULL;
@@ -10422,7 +10422,7 @@ static int tg3_run_loopback(struct tg3 *tp, int loopback_mode)
num_pkts = 0;
tg3_set_txd(tnapi, tnapi->tx_prod,
- skb_shinfo(skb)->dma_head, tx_len, 0, 1);
+ skb->dma_head, tx_len, 0, 1);
tnapi->tx_prod++;
num_pkts++;
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index d0448c5..57c7e81 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -190,9 +190,6 @@ struct skb_shared_info {
atomic_t dataref;
unsigned short nr_frags;
unsigned short gso_size;
-#ifdef CONFIG_HAS_DMA
- dma_addr_t dma_head;
-#endif
/* Warning: this field is not always filled in (UFO)! */
unsigned short gso_segs;
unsigned short gso_type;
@@ -201,9 +198,6 @@ struct skb_shared_info {
struct sk_buff *frag_list;
struct skb_shared_hwtstamps hwtstamps;
skb_frag_t frags[MAX_SKB_FRAGS];
-#ifdef CONFIG_HAS_DMA
- dma_addr_t dma_maps[MAX_SKB_FRAGS];
-#endif
/* Intermediate layers must ensure that destructor_arg
* remains valid until skb destructor */
void * destructor_arg;
@@ -399,6 +393,10 @@ struct sk_buff {
sk_buff_data_t transport_header;
sk_buff_data_t network_header;
sk_buff_data_t mac_header;
+#ifdef CONFIG_HAS_DMA
+ dma_addr_t dma_head;
+ dma_addr_t dma_maps[MAX_SKB_FRAGS];
+#endif
/* These elements must be at the end, see alloc_skb() for details. */
sk_buff_data_t tail;
sk_buff_data_t end;
diff --git a/net/core/skb_dma_map.c b/net/core/skb_dma_map.c
index 79687df..64d3639 100644
--- a/net/core/skb_dma_map.c
+++ b/net/core/skb_dma_map.c
@@ -20,7 +20,7 @@ int skb_dma_map(struct device *dev, struct sk_buff *skb,
if (dma_mapping_error(dev, map))
goto out_err;
- sp->dma_head = map;
+ skb->dma_head = map;
for (i = 0; i < sp->nr_frags; i++) {
skb_frag_t *fp = &sp->frags[i];
@@ -28,7 +28,7 @@ int skb_dma_map(struct device *dev, struct sk_buff *skb,
fp->size, dir);
if (dma_mapping_error(dev, map))
goto unwind;
- sp->dma_maps[i] = map;
+ skb->dma_maps[i] = map;
}
return 0;
@@ -37,10 +37,10 @@ unwind:
while (--i >= 0) {
skb_frag_t *fp = &sp->frags[i];
- dma_unmap_page(dev, sp->dma_maps[i],
+ dma_unmap_page(dev, skb->dma_maps[i],
fp->size, dir);
}
- dma_unmap_single(dev, sp->dma_head,
+ dma_unmap_single(dev, skb->dma_head,
skb_headlen(skb), dir);
out_err:
return -ENOMEM;
@@ -53,12 +53,12 @@ void skb_dma_unmap(struct device *dev, struct sk_buff *skb,
struct skb_shared_info *sp = skb_shinfo(skb);
int i;
- dma_unmap_single(dev, sp->dma_head,
+ dma_unmap_single(dev, skb->dma_head,
skb_headlen(skb), dir);
for (i = 0; i < sp->nr_frags; i++) {
skb_frag_t *fp = &sp->frags[i];
- dma_unmap_page(dev, sp->dma_maps[i],
+ dma_unmap_page(dev, skb->dma_maps[i],
fp->size, dir);
}
}
^ permalink raw reply related
* linux-next: manual merge of the net tree with the net-current tree
From: Stephen Rothwell @ 2009-11-06 0:55 UTC (permalink / raw)
To: David Miller, netdev
Cc: linux-next, linux-kernel, Marcel Holtmann, Torgny Johansson
Hi all,
Today's linux-next merge of the net tree got a conflict in
drivers/net/usb/cdc_ether.c between commit
3a19d56c71bd3a08412d609d219ac8eec0819166 ("cdc_ether: additional Ericsson
MBM PID's to the whitelist") from the net-current tree and commit
e1e499eef2200c2a7120c9ebf297d48b195cf887 ("usbnet: Use wwan%d interface
name for mobile broadband devices") from the net tree.
Context changes. I fixed it up (see below) and can carry it as
necessary. This fix may not be completely correct ...
--
Cheers,
Stephen Rothwell sfr@canb.auug.org.au
diff --cc drivers/net/usb/cdc_ether.c
index 21e1ba1,71e65fc..0000000
--- a/drivers/net/usb/cdc_ether.c
+++ b/drivers/net/usb/cdc_ether.c
@@@ -542,62 -550,22 +550,62 @@@ static const struct usb_device_id produ
/* Ericsson F3607gw */
USB_DEVICE_AND_INTERFACE_INFO(0x0bdb, 0x1904, USB_CLASS_COMM,
USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
- .driver_info = (unsigned long) &cdc_info,
+ .driver_info = (unsigned long) &mbm_info,
}, {
- /* Ericsson F3307 */
+ /* Ericsson F3607gw ver 2 */
+ USB_DEVICE_AND_INTERFACE_INFO(0x0bdb, 0x1905, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
+ /* Ericsson F3607gw ver 3 */
USB_DEVICE_AND_INTERFACE_INFO(0x0bdb, 0x1906, USB_CLASS_COMM,
USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
- .driver_info = (unsigned long) &cdc_info,
+ .driver_info = (unsigned long) &mbm_info,
}, {
+ /* Ericsson F3307 */
+ USB_DEVICE_AND_INTERFACE_INFO(0x0bdb, 0x190a, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
+ /* Ericsson F3307 ver 2 */
+ USB_DEVICE_AND_INTERFACE_INFO(0x0bdb, 0x1909, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
+ /* Ericsson C3607w */
+ USB_DEVICE_AND_INTERFACE_INFO(0x0bdb, 0x1049, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
/* Toshiba F3507g */
USB_DEVICE_AND_INTERFACE_INFO(0x0930, 0x130b, USB_CLASS_COMM,
USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
- .driver_info = (unsigned long) &cdc_info,
+ .driver_info = (unsigned long) &mbm_info,
}, {
+ /* Toshiba F3607gw */
+ USB_DEVICE_AND_INTERFACE_INFO(0x0930, 0x130c, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
+ /* Toshiba F3607gw ver 2 */
+ USB_DEVICE_AND_INTERFACE_INFO(0x0930, 0x1311, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
/* Dell F3507g */
USB_DEVICE_AND_INTERFACE_INFO(0x413c, 0x8147, USB_CLASS_COMM,
USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
- .driver_info = (unsigned long) &cdc_info,
+ .driver_info = (unsigned long) &mbm_info,
+}, {
+ /* Dell F3607gw */
+ USB_DEVICE_AND_INTERFACE_INFO(0x413c, 0x8183, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
+}, {
+ /* Dell F3607gw ver 2 */
+ USB_DEVICE_AND_INTERFACE_INFO(0x413c, 0x8184, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_MDLM, USB_CDC_PROTO_NONE),
+ .driver_info = (unsigned long) &cdc_info,
},
{ }, // END
};
^ permalink raw reply
* Re: [RFC] move dma_head/dma_maps out of skb_shared_info and into sk_buff
From: David Miller @ 2009-11-06 1:40 UTC (permalink / raw)
To: alexander.h.duyck; +Cc: netdev
In-Reply-To: <4AF373C9.7040700@intel.com>
From: Alexander Duyck <alexander.h.duyck@intel.com>
Date: Thu, 05 Nov 2009 16:54:33 -0800
> During testing we found issues with the use of skb_dma_map/unmap on
> systems that had iommu enabled and were configured to use a bridge.
> The
> issue is that if two ports are members of the same bridge, and a
> broadcast packet is sent out on the bridge skb_clone will be used to
> send a copy to all ports, but the clones run into issues because the
> dma mappings for the cloned skbs all share the shared_info structure
> where the dma mappings are stored.
>
> To resolve that this patch moves those dma mappings out of the
> shared_info structure and into the sk_buff itself. This allows cloned
> skbs to be mapped separately without causing dma unmapping errors.
>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
DMA mappings are a property of the data not the SKB.
And the DMA mappings are in the shared area exactly so that
we don't eat the space for every clone of the data.
We need another way around this problem and I'll just as
soon remove the DMA mapping bits entirely from SKBs before
I'll put them into struct sk_buff
^ permalink raw reply
* Re: linux-next: manual merge of the net tree with the net-current tree
From: David Miller @ 2009-11-06 1:40 UTC (permalink / raw)
To: sfr; +Cc: netdev, linux-next, linux-kernel, marcel, torgny.johansson
In-Reply-To: <20091106115554.44622e53.sfr@canb.auug.org.au>
From: Stephen Rothwell <sfr@canb.auug.org.au>
Date: Fri, 6 Nov 2009 11:55:54 +1100
> Hi all,
>
> Today's linux-next merge of the net tree got a conflict in
> drivers/net/usb/cdc_ether.c between commit
> 3a19d56c71bd3a08412d609d219ac8eec0819166 ("cdc_ether: additional Ericsson
> MBM PID's to the whitelist") from the net-current tree and commit
> e1e499eef2200c2a7120c9ebf297d48b195cf887 ("usbnet: Use wwan%d interface
> name for mobile broadband devices") from the net tree.
>
> Context changes. I fixed it up (see below) and can carry it as
> necessary. This fix may not be completely correct ...
Thanks I'll take a look at this when merging net-2.6 into
net-next-2.6 later tonight.
^ permalink raw reply
* Re: [RFC] move dma_head/dma_maps out of skb_shared_info and into sk_buff
From: Eric Dumazet @ 2009-11-06 2:14 UTC (permalink / raw)
To: Alexander Duyck; +Cc: netdev@vger.kernel.org, davem@davemloft.net
In-Reply-To: <4AF373C9.7040700@intel.com>
Alexander Duyck a écrit :
> During testing we found issues with the use of skb_dma_map/unmap on
> systems that had iommu enabled and were configured to use a bridge. The
> issue is that if two ports are members of the same bridge, and a
> broadcast packet is sent out on the bridge skb_clone will be used to
> send a copy to all ports, but the clones run into issues because the
> dma mappings for the cloned skbs all share the shared_info structure
> where the dma mappings are stored.
>
> To resolve that this patch moves those dma mappings out of the
> shared_info structure and into the sk_buff itself. This allows cloned
> skbs to be mapped separately without causing dma unmapping errors.
>
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
Hello Alexander
You probably know such a change is a major one ;)
1) a diffstat -p1 -w70 for this kind of patch would be nice.
2) Your patch is garbled (tabulations were replaced by spaces
by your mailer)
3) Are you sure we need to clear dma_maps[] array and dma_head
in __alloc_skb() ? I guess not.
MAX_SKB_FRAGS = 18 on x86 -> 152 bytes on x86_64.
Previous implementation was not clearing them.
Thats would be a major slow down.
4) 152 bytes more in skb -> 304 bytes more in skbuff_fclone_cache
Do we really want two copies of dma_maps[] when skb are allocated
from fclone cache ?
5) It seems to me this stuff is needed for xmit only and few drivers,
could we find a way to not have it for RX path and drivers that dont
need it ? Maybe drivers themselves should allocate storage for this
stuff so we can remove it both from shared_info *and* skb
Thanks
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox